General system intercommunication high-level design
Alfeiks K. <madtirra <at> jarios.org>
2011-10-17 13:13:24 GMT
I decide to brake a silence about development process again.
Today I want to introduce intercommunication design within the
Firstly, I want to tell - IDL is a good idea in theory and is a bad idea
The main reason - it's implementation. IDL usually describe interfaces and,
in addition, creates some code, that packs the request, get it, call it,
reply. I.e. receive, call and reply to the message, nothing more.
But, in our system we have something more complex, let's enlist it:
- IPC message forwarding
- Postponed calls (blocking operations)
- IPC message modification operations
Well, on that point you need something more featured than receive - call
cycle, you need to determine when you need to forward a message, on which
point you need to do it, what is immutable message parts and what parts
should be modified and/or cut off.
The other "funny" thing is postponed messages, in this case IDL
must contain many stuff like quick allocation of memory and other stuff.
Finally, generated code is coming to be huge, and in addition you need
it in some different cases.
Another problem is a problem with interfaces, i will try to describe it
For each instance you have an interface with a predefined set of functions,
many functions are similar, or identical: for example, device has a
and file has a read() function, but you have a one set of interfaces for
and other set of interfaces for devices. That mean that you need to try
generic interface, and extend it every time with your own functions
again and again.
On practice you will have a big set of libraries with interfaces and it
will be a real headache.
From other point of view, you can resolve the problem with it on the
libc client side,
just determine the resource and call specific function i.e. POSIX read()
in our case
will call read_file() or read_device() functions depends on the opened
I think - the last idea is very very ugly, I don't think that it's a
good idea to change libc
every time when you adding something new to the system, also, we should
in some cases I can turn off support for pipes of sockets - and I don't
want to recompile
libc, all the other system parts for it.
That's why IDL sucks, I spent many time to solve the problems with
nontrivial things like
message advanced control and I don't see the sense. But, the old one -
uni_rpc_t is sucks too,
on practice uni_rpc_t creates more problems that it's should solve.
And, I faced that pretty solution going from other way to solve this
Each time while you design RPC/IDL/something_else_... you should think
to the system and it's objects, my error was in the way to solve this
problem, I think
about RPC/IDL/etc ... only, but you need to think about system also.
What I mean: system might be presented like a set of objects hosted on
different servers and
interacting each one with other, and in this case there are no
file and task, i.e. file, task, device, ipc object - it's node, with
predefined set of functions.
Well, yes, task and file has a different operations, but all possible
be represented via one set.
Anyway you will have a very similar operations, in example, changing
file owner uid and changing
task effective uid is very similar, on RPC level there are no difference
What I decide: I decide to represent all objects (or it's high-level
representation) as a node,
each node has a 2 groups of operations, first one for control the node
and manage it attributes,
second one for data i/o (yep, task will doesn't have data i/o interface,
but we can do it, if needed).
In that case RPC is going to be simple, you have RPC signature in every
this signature points to the following things:
- Function group
- Function within group
- IDs to select right node for operation
Other, after RPC signature, is going to the implementation.
How are ipcbox and sbuf abstractions used ?
It's a good question with a brief and simple answer, ipcbox is used by the
rpc library routines to operate with IPC, sbuf is used for data, because
gets the sbuf like a data, rpc code don't allocating anything for
Why is it pretty ?
Because it's simple and this solution doesn't require to implement a
huge set of
API for each task, i.e. good old getvfspid() getsomeothercrap() bla bla bla.
Each task, has the special system reserved iolink, depends on task role
regular task, translator, resource carrier, etc ...) its node has a set
For example, to make a fork() you just need to make a control request
iolink to your node with fork() related data, to change your effective
uid you just
send a stat request to your node representation, but if you are a
regular task (i.e. doesn't have a rights
to link file system onto namespace tree) your node don't have the
operation of such kind,
otherwise you will able to make a special call to your system iolink to
Is it simple ? I guess it's very simple solution.
Questions ? Comments ? you are welcome.