headers
Neal H. Walfield | 1 Sep 2008 12:12

virtualizing IPC primitives

At Thu, 28 Aug 2008 11:48:56 -0400,
Jonathan S. Shapiro wrote:
> Now the problem that you face in managing mailboxes is not quite
> analogous. Ultimately, the problem you are really dealing with is that
> you cannot use the communication substrate primitives to simulate
> themselves. There is a reductio problem.
> 
> It appears to me that there are (qualitatively) only two solutions to
> this reductio:
> 
>   1. Define the messaging architecture in such a way that the transient
>      message body can be elided in some cases, and ensure that the
>      traversal reductio can be implemented entirely within these cases.
> 
>      In particular, kernel-implemented objects such as cappages are
>      invariably very simple, and you may be able to exploit the fact
>      that all of the required operations for this object are both
>      unit-time and involve very small messages.
> 
>      OR
> 
>   2. Define the messaging queues as a *cache* backed by the respective
>      applications, and design the traversal solution in such a way that
>      the caches involved in the traversal are likely to converge.

When virtualizing a mailbox, how do you know when to deliver the
message vs. when the kernel should manipulate the mailbox?

Consider invoking the receive operation on a mailbox.  If mailbox is a
virtualized one, the proxy needs to get the message so that it can
(Continue reading)

Permalink | Reply | 1 comment |
headers
Jonathan S. Shapiro | 1 Sep 2008 19:57

Re: APIs and compatibility

On Sun, 2008-08-31 at 11:55 +0200, Neal H. Walfield wrote:
> At Sat, 30 Aug 2008 14:02:40 -0400,
> Jonathan S. Shapiro wrote:
> > It strikes me that rebuilding a very large number of drivers as a
> > precondition to success is probably not a good recipe.
> > 
> > Is there any reason why the linux driver framework cannot be adopted
> > directly by l4-ng?  I do understand that user-supplied drivers are
> > desirable, and I think that remains possible in all of the practical
> > use-cases that have come up here.
> 
> Although custom drivers offer the best potenial quality, writing new
> drivers for all hardware is impractical.  So there must be some reuse.
> However, some reuse does not imply that all drivers must be reused.
> That is, very common hardware or essential hardware can be provided by
> native drivers and the rest by way of reuse.

I agree with all of these points. But this does imply that certain
*interfaces* from Linux become mandatory: those that in SVR4 would have
been called the "Driver Kernel Interface". This is ad hoc in Linux, but
it exists.

> This hair can be avoided by reusing Linux in its entirety.  In this
> case, Linux is run on a VMM and the new operating system makes calls
> to the Linux driver instance.  Two examples of this are Afterburner or
> Xen.  Both use a paravirtualized Linux instance.

But if this is done, what is the advantage of having a microkernel at
all?
(Continue reading)

Permalink | Reply | 2 comments |
headers
Jonathan S. Shapiro | 1 Sep 2008 20:00

Re: virtualizing IPC primitives

On Mon, 2008-09-01 at 12:12 +0200, Neal H. Walfield wrote:
> When virtualizing a mailbox, how do you know when to deliver the
> message vs. when the kernel should manipulate the mailbox?

In abstract: separate the control channel from the data channel. The
primordial mailbox cache cannot be virtualized, but it's *behavior* can
be simulted..

shap
Permalink | Reply | 0 comments |
headers
Neal H. Walfield | 1 Sep 2008 20:20

Re: APIs and compatibility

At Mon, 01 Sep 2008 13:57:50 -0400,
Jonathan S. Shapiro wrote:
> 
> On Sun, 2008-08-31 at 11:55 +0200, Neal H. Walfield wrote:
> > At Sat, 30 Aug 2008 14:02:40 -0400,
> > Jonathan S. Shapiro wrote:
> > > It strikes me that rebuilding a very large number of drivers as a
> > > precondition to success is probably not a good recipe.
> > > 
> > > Is there any reason why the linux driver framework cannot be adopted
> > > directly by l4-ng?  I do understand that user-supplied drivers are
> > > desirable, and I think that remains possible in all of the practical
> > > use-cases that have come up here.
> > 
> > Although custom drivers offer the best potenial quality, writing new
> > drivers for all hardware is impractical.  So there must be some reuse.
> > However, some reuse does not imply that all drivers must be reused.
> > That is, very common hardware or essential hardware can be provided by
> > native drivers and the rest by way of reuse.
> 
> I agree with all of these points. But this does imply that certain
> *interfaces* from Linux become mandatory: those that in SVR4 would have
> been called the "Driver Kernel Interface". This is ad hoc in Linux, but
> it exists.

I don't see why you think this.

Assuming that the OS has its own interface for accessing devices,
reusing Linux device drivers does not mean that this interface must be
abandoned or that the Linux interface must be exposed to applications.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Jonathan S. Shapiro | 1 Sep 2008 20:39

Re: APIs and compatibility

On Mon, 2008-09-01 at 20:20 +0200, Neal H. Walfield wrote:
> Assuming that the OS has its own interface for accessing devices,
> reusing Linux device drivers does not mean that this interface must be
> abandoned or that the Linux interface must be exposed to applications.
> A server that implements the normal interface can be used to translate
> from one to the other.

You got what I was saying backwards. What I was saying was that re-using
Linux drivers requires that -- somewhere -- there must be an entity that
replicates the Linux DDK.

> First, I am trying to develop a way to first of all safely empower
> applications.  From a pure research perspective, I am not interested
> in creating a product but verifying my ideas.

That sounds perfectly reasonable. I must admit that I am biased by
trying to solve product problems these days.

shap
Permalink | Reply | 0 comments |
headers
Marcus Brinkmann | 2 Sep 2008 18:24
Picon
Favicon

Re: capability address space and virtualizing objects

At Thu, 28 Aug 2008 18:00:18 +0200,
Neal H. Walfield wrote:
> The problem isn't extending the address translation mechanism, it's
> the ability to fully virtualize an object.  On a system like Mach,
> this is not a problem because address space construction is not
> exported to user space so there is no need to virtualize this
> functionality.
> 
> The ability to virtualize objects is useful, for instance, for
> implementing rpctrace.  If we could only virtualize non-kernel
> objects, then a program could invoke a capability in a cappage that it
> got in a message.  If rpctrace does not proxy cappages, it would not
> see any invocations on them.  The solution here might be to simply
> proxy the capabilities aggressively (i.e., proxy all reachable
> capabilities in any message).  But that can mean a lot of memory!
> Also, it would require detecting capability cycles, which is hard.

There are other options for full virtualization, as is demonstrated by
many recent virtualization platforms.  These other techniques may be a
more appropriate virtualization method in some of the use cases that
concern you than capability based virtualization.  For example, they
may offer much improved performance and may utilize hardware support
better.  In fact, those techniques are mandatory if you want full
transparent virtualization (including processor time etc).  Memory
accesses are not the only resources in a computer that need
virtualization at times (see the blue pill papers for an extreme
case).

Therefore, I find it hard to follow your reasoning that 100%
virtualizability is a worthy goal.  It is certainly an interesting
(Continue reading)

Permalink | Reply | 0 comments |
headers
olafBuddenhagen | 9 Sep 2008 10:09
Picon

Re: APIs and compatibility

Hi,

On Sun, Aug 31, 2008 at 11:55:27AM +0200, Neal H. Walfield wrote:

> Although custom drivers offer the best potenial quality, writing new
> drivers for all hardware is impractical.  So there must be some reuse.
> However, some reuse does not imply that all drivers must be reused.
> That is, very common hardware or essential hardware can be provided by
> native drivers and the rest by way of reuse.
[...]

> Assuming that the system is so far that the limiting factor is
> drivers, I think the best approach is to reuse Linux as a whole:
> porting drivers is just too hairy.  Between the afterburner approach
> and the Xen approach, I would go with the latter.  From a practical
> perspective, Xen has the major advantage that it is commercially
> maintained and provides a relatively stable API.

Xen has a major disadvantage, though: It needs a hypervisor running
*below* the operating system. This makes system startup and setup quite
complicated... I'm not sure whether it would be possible to do all the
setup automatically so the user is not bothered by it.

As for mixed native/foreign drivers, I don't see how this would work
with the approach of running Linux as a whole. If we want any native
drivers, this means we need native bus drivers. But Linux will use its
own bus drivers. How would you prevent conflicts here?

-antrik-
(Continue reading)

Permalink | Reply | 0 comments |

Gmane