Re: A comment about changing kernels

On Mon, Oct 31, 2005 at 10:05:33AM -0500, Jonathan S. Shapiro wrote:
> In order to implement a sessionless RPC protocol, every CALL (the first
> IPC) must transmit an endpoint capability. This endpoint capability will
> later be used by the server to perform a RETURN (the 2nd IPC).
...
> First question: is this set of statements consistent with your
> assumptions about how sessionless RPC would work?

Yes, I assume that a return endpoint has to be send with every CALL.

> we may safely conclude that these CALL/RETURN patterns
> describe the overwhelming majority of IPCs -- all of the other patterns
> taken together accounted in EROS for less than 1% of all dynamic IPCs.

Is the CALL/RETURN pattern used for capability delegation (giving an cap
to another party) or only for usage of these capabilities?

> As we considered the session/sessionless issue in Coyotos, I became
> convinced that using the protected payload as a session ID would not
> scale well, and also presents certain challenges for bootstrapping and
> isolation.

Interesting. What are the scalability and isolation problems with badges?

> We have therefore concluded that Coyotos needs to support capability
> registers in addition to C-spaces, and that in the normal case the send
> and receive endpoints will live in these registers.

Ok, thats a valid optimization.

Re: L4-HURD , POSIX, UNIX

_______________________________________________
L4-hurd mailing list
L4-hurd <at> gnu.org
http://lists.gnu.org/mailman/listinfo/l4-hurd

RE: Changing from L4 to something else...

>This talk is very interesting about have a system administrator that
doesn't administer much, but I have some questions that I think are
important:

>1) How does an administrator help a user fix a misbehaving session
(i.e. if a malicious program finds some way to take over a user's
session by doing something like take focus any time the user moves the
mouse) >if they can't interact with the user's session?

Same way you do it on a Windows system: reboot. 

Re: L4-HURD , POSIX, UNIX

On Mon, 31 Oct 2005 19:12:10 -0300 (ART)
Fortes Marcelo <marcelosoftware <at> yahoo.com.br> wrote:

> > Message: 4
> > Date: Sun, 30 Oct 2005 20:15:38 +0100
> > From: Bas Wijnen <shevek <at> fmf.nl>
> > Subject: Re: L4-HURD , POSIX, UNIX
> > To: l4-hurd <at> gnu.org
> > Message-ID:
> > <20051030191538.GE25359 <at> pcbcn10.phys.rug.nl>
> > Content-Type: text/plain; charset="us-ascii"
> > 
> > On Fri, Oct 28, 2005 at 03:29:16PM -0300, Fortes
> > Marcelo wrote:
> > > Well some of participant are fighting about the
> > design etc and be or not
> > > POSIX compliant. Etc.
> > 
> > I think we all agree that POSIX applications _must_
> > run on the Hurd.  There is
> > some discussion about it if it should be the native
> > interface, or it can be
> > something on top of it.  Personally, I think they
> > can be implemented in a
> > shared library, to which native Hurd applications
> > don't link.  "Transitional"
> > applications may link to it to use some parts, and
> > use parts from the native
> > system as well.  The library might become less
> > important when more programs
> > refrain from using it.
> 
> In Towards of New OS Design, The original HURD
> designer wrote that Unix look and feel and
> compatibility would be inplemented using the glib
> layer that i think is not too much diferent.

All that is said about POSIX on Towards of New OS Design is that POSIX calls will be handled by glibc, there is
nothing more than that.

I will get a part of the original announcement of the GNU Project to explain what we want to do:
"GNU will be able to run Unix programs, but will not be identical to Unix.  We will make all improvements that
are convenient, based on our experience with other operating systems."

We want an OS that runs Unix programs, but we do not want a OS with the same problems of Unix.

> Native Hurd applications that are not linked to POSIX
> layer are not a good idea becouse youre creting a
> problem that means native Hurd aplications will be
> hard to port to other *UX/*NIX-Like platforms.

There are two points of view here.
 1- Our goal is create GNU apps that run on a GNU system or create GNU apps that run on every system?
 2- How hard would be create an library to create a layer to run native GNU apps on POSIX systems?

> > 
> > > That is the obvious basis! an enviroinment that
> > GCC can compile with a
> > > minimum of changes Unix/GNU programs(bash, emacs,
> > X-Windows, pico, VI ).
> > > Off course extensions are necessary and obviously
> > the system should provide
> > > new features and enhancements that are not found
> > in other Unix Flavours but
> > > it is a secoundary step that a system with
> > multi-servers running in a top of
> > > a microkernel can do easely and a monolithic not.
> > So i reinforce the idea
> > > that you are working in a GNU Kernel. A Unix-Like
> > Kernel that can run Unix
> > > like softwares and have a Unix confortable
> > enviroinment that is the basis!
> > 
> > The improvements we are hoping to make are not
> > possible if POSIX is the base
> > of the system.  However, a POSIX library should be
> > no problem at all on the
> > system we currently have in mind.  Therefore it
> > seems like a good idea to me
> > to make POSIX _not_ the *base* of the system, but
> > more an extension.
> > 
> > > So Friends maybe my email can help to clarify a
> > bit and you can trust my
> > > intention is to be constructive, in a reflection
> > basis.
> > 
> > I appreciate your opinion, and hope that you agree
> > with me that POSIX doesn't
> > need to be the _basis_ of the system, but it must be
> > supported.
> > 
> > Thanks,
> > Bas
> > 
> 
> I do appreciate your opinion too and really thanks.
> But If the whole OS will be build on this basis its
> time to stop and say; "That is not more about HURD or
> the GNU OS, its about a entire new system that just
> provide suport to Unix-like applications".
It _IS_ about the _GNU OS_ and _HURD_, it is not about a entire new system that just provide suport to
Unix-like applications.

We need to decide if we want a system that only run Unix-like apps, or want a system very powerful that also
runs Unix-like apps, but will delay a GNU release in some more years.


> I suggest to people try to keep in mind Simple ideas,
> the features that a multiserver system can bring to an
> traditional system are obvious. The idea must be the
> most simply as possible. A Unix Clone with Unix/Posix
> compatibles multiservers running on top of a
> microkernel that dont necessarely knows what kind of
> OS flavours are runing on its top.

Keeping in mind simple ideas we will have a simple system wish simple concepts that noone will wanna use it,
because others kernels like BSD and Linux will ever be ever a step ahead. I know that are some advantages in
the use of a multiserver system, but they have more things working than us, and when we have a kernel as
powerful as linux is today, they will have more new things that we will not have.

> There is the advantages of device drivers dont running
> inside the kernels and each server running protected
> each other and another great features that that
> architecture can offer.
Today, Mach's device driver is inside the kernel.

> I understand that a lot of Arcane principles of Unix
> that causes troubles of security failures must be
> avoided and thats right and are obvious! each system
> also must have your all extensions and facilitys.

> But if it is a compromisse with the GNU OS runing with a
> non Linux kernel the developers needs to remamber the
> basis, that is a UNIX Replacement.
Linux _IS NOT_ the kernel of GNU OS, so, we have no compromisse to run GNU on Linux.

More than an simple Unix Replacement project, GNU is a project to create a Free Operating System. Many GNU
apps are not compatible with their respectives apps on Unix, why the kernel need to keep this compatibility?

> And Unix and your
> standards have proof for years your reability and
> robustness despite your architectural problems.
> For exemple MINIX 3.0 its a step towards this way 
> most of the system was rewrited and what Andrew
> Tannembaum did ? Put it most Unix/posix compliant than
> previous versions. But that is my opinion.And im not
> the project ruller or mantener but as user i think my
> opinion are important just emulate Unix with external
> librarys is not enough it must be in the basis most
> Unix compliant as possibly.

But the unix compatibility on GNU Hurd based only in a library.

> 
> Thanks Marcelo Fortes.
> 
> 
> 
> 	
> 
> 
> 
> 	
> 		
> _______________________________________________________ 
> Promoção Yahoo! Acesso Grátis: a cada hora navegada você
> acumula cupons e concorre a mais de 500 prêmios! Participe!
> http://yahoo.fbiz.com.br/

> 
> 
> _______________________________________________
> L4-hurd mailing list
> L4-hurd <at> gnu.org
> http://lists.gnu.org/mailman/listinfo/l4-hurd



-- 
leonardolopespereira at gmail.com

GNU Privacy Guard (GPG)
ID da chave: 83E8AFBF | servidor: keys.indymedia.org
gpg --keyserver keys.indymedia.org --recv-keys 83E8AFBF

_______________________________________________
L4-hurd mailing list
L4-hurd <at> gnu.org
http://lists.gnu.org/mailman/listinfo/l4-hurd

Re: A comment about changing kernels

On Tue, 2005-11-01 at 00:53 +0100, Bernhard Kauer wrote:
> > It really was a bug, and it has nothing to do with the IPC path,
> 
> That's correct, it was a bug in the interrupt handler.
> 
> 
> > The correct behavior was to save the segment selectors, and this adds
> > about 20 cycles to the path.
> 
> I do not see a problem, if the IPC set the segment registers to default
> values. See it as output values of this syscall.
> 
> Shortly spoken: a syscall can change every user register, an interrupt
> has to change none of them.

Yes. The problem is that this behavior was occurring on *interrupts*
too.

shap

RE: Changing from L4 to something else...

On Mon, 2005-10-31 at 19:41 -0700, Christopher Nelson wrote:
> >This talk is very interesting about have a system administrator that
> doesn't administer much, but I have some questions that I think are
> important:
> 
> >1) How does an administrator help a user fix a misbehaving session
> (i.e. if a malicious program finds some way to take over a user's
> session by doing something like take focus any time the user moves the
> mouse) >if they can't interact with the user's session?
> 
> 
> Same way you do it on a Windows system: reboot. 

Doesn't work in a persistent system.

shap

Subject: Re: design goals vs mechanisms (was: Re: Let's do some coding

On Tue, 2005-11-01 at 00:26 +0100, Michal Suchanek wrote:
> On 10/28/05, Jonathan S. Shapiro <shap <at> eros-os.org> wrote:
> > On Fri, 2005-10-28 at 15:08 +0200, Michal Suchanek wrote:
> > > On 10/26/05, Marcus Brinkmann <marcus.brinkmann <at> ruhr-uni-bochum.de> wrote:
> > > > At Wed, 26 Oct 2005 22:43:06 +0200,
> > > > Bas Wijnen <shevek <at> fmf.nl> wrote:
> > >
> > > > If you want stability, you probably want to do some of the following:
> > > > * allocate a fixed amount of resources statically up front,
> > > >   instead dynamically at run time.
> > >
> > > I kind of dislike this. That is one of the things that is nice in
> > > Hurd/Mach: you have no limit on the size of strings like filenames. I
> > > do not think I would like a filename that has 1M characters, but I do
> > > not know what is the  "reasonable limit" for filename length.
> >
> > I don't know anybody who *does* like this, but go read your sentence
> > again. What you are saying is: "I want the system to run robustly, but I
> > am unable to specify the conditions under which it must do so."
> >
> > This just won't work. It isn't even a kernel vs. application issue.
> >
> hehe, I now understand why AMS likes to throw so much dirt in your direction :)
> 
> Some of your replies are so much constructive...
> 
> Like you publish a paper explaining ways of transferring unbound data
> and then reply "it just won't work" when I complain about size limit
> on strings.

I understand how the confusion came about, but this is not quite what I
said.

Robustness is a matter of degree. In most general purpose applications
we are prepared to tolerate rare application failure and recover. In
these applications, dynamically sized strings are perfectly okay **at
the application level**.

When extreme robustness is required, static preallocation really does
become necessary.

The trusted buffer object described in the synchronous IPC
vulnerabilities paper does not provide any *kernel* mechanism for
transferring an unbounded string. What it does is find a clever way to
avoid having the kernel do so -- this preserves kernel robustness, and
leaves application robustness for the application to decide.

Now: back to your file system example. One of the properties we would
like to have in a file system is knowledge that every operation will
complete (either successfully or by failure) in bounded time. If the
open() call must process an unbounded string, then this cannot be
achieved. Yes, EROS has a mechanism that would allow an unbounded string
to be transferred, but there is no way for the file system to *process*
that string in bounded time.

If the file system does not provide bounded time operations, then NO
application using that file system can be robust, because no application
using that file system has any contract that its open() operations will
*ever* complete in the face of unspecified but permitted usage by other
(unrelated) applications that happen to share that file system.

In fact, the issue is even more direct than this. In principle, strings
are bounded by the address space size. Therefore, this is not really a
debate about bounded vs. unbounded. It is a debate about what the bound
should be to ensure the degree of liveness and robustness that we want
in the system.

Speaking only for myself, I suspect that a PATH_MAX of 4096 is enough.
This is because file systems whose paths are longer than this cannot, in
practice, be managed successfully by real human beings, so the limit is
not hit in practice. The only case that I have ever seen where this
limit got hit was in the context of a backup operation on a file system
that was already near the limit. In Coyotos/EROS, this would not have
occurred, because the backup directory would not have been stuck "under"
some other directory.

Finally, let me emphasize a subtle distinction: I am not saying that
there needs to be a bounded number of *components* in the file name. I
am saying that the length of any *one* component (the part betweeen two
adjacent '/' characters) needs to be bounded, and also the number of
components that will be processed by a directory lookup operation in a
single unit of operation must also be bounded. This would not preclude
iterative traversal in order to have a longer effective path name.

shap

Re: A comment about changing kernels

On Tue, 2005-11-01 at 00:42 +0100, Bernhard Kauer wrote:
> > You are using your assumptions to reason in a circle. If your protocol
> > did not presume that sessions were desirable, and that an update to the
> > badge is therefore necessary, you would conclude that a single COPY from
> > A to B is sufficient.
> 
> Yes, it simply states that session-based protocols are independet of copy/map.
> They need only one of these operations.

How can you say this, when a correct session-based protocol must not use
MAP in many circumstances and therefore must pay a penalty of multiple
IPCs?

shap

Re: A comment about changing kernels

On Tue, 2005-11-01 at 01:22 +0100, Bernhard Kauer wrote:
> > we may safely conclude that these CALL/RETURN patterns
> > describe the overwhelming majority of IPCs -- all of the other patterns
> > taken together accounted in EROS for less than 1% of all dynamic IPCs.
> 
> Is the CALL/RETURN pattern used for capability delegation (giving an cap
> to another party) or only for usage of these capabilities?

For delegation, we tend to see a CALL/RETURN pair in which one of the
arguments to the CALL is the capability being delegated.

However, these delegations are dynamically very rare in comparison to
*uses* of capabilities. As an imperfect intuition: delegation tends to
occur during program setup, but rarely during program steady-state
execution. A delegation:use ratio of 1:100 or even 1:1000 would not be
an unreasonable expectation.

> > As we considered the session/sessionless issue in Coyotos, I became
> > convinced that using the protected payload as a session ID would not
> > scale well, and also presents certain challenges for bootstrapping and
> > isolation.
> 
> Interesting. What are the scalability and isolation problems with badges?

Remember that my opinion was based on the old, extensible badge idea. At
the time, there were several bits of protocol being considered whose
efficiency relied on being able to extend the badge, but with a 32-bit
(or even 64-bit) badge, there really wasn't any reason to feel confident
that you had "enough" badge bits to work with, and I could never find a
clear story for what to do when the badge bits ran out.

Since you now say that badges are no longer extensible, I would guess
that other protocols have been developed that avoid this issue and/or
solve it in other ways.

> > It involves a cache-aligned four-word move from one process to the other,
> > using a scaled index addressing mode using the respective process structure
> > pointers as base pointers.
> 
> This is a cache miss and perhaps a TLB miss since you have to touch the process
> structure (I assume a thread/task split like in L4).

There is no task/thread split in EROS or Coyotos, so there is no
additional TLB miss. We have only threads, and no tasks. Address spaces
are first-class. Each thread holds a capability to its address space;
two threads can hold capabilities to the same address space.

> But don't you allow the revocation of this capability later?

Not in general. In EROS, this is by far the unusual case (much less than
1/100,000 of all delegations).

> > I am *guessing* that this is significantly lower than the cost that you
> > anticipate for the MAP operation.
> >
> > Second question: do you agree that this difference in designs would
> > largely explain our differing views about the costs of a sessionless
> > protocol?
> 
> Yes, but up to now I am not convinced that you can build a revocable copy
> with just a single cache miss.

That is probably correct, but that is not the goal. In fact, quite the
opposite is desired. We consider revocable copy a necessary evil that
should be avoided whenever it is possible to do so -- there has been a
long discussion on this list over the last few weeks about why we have
this view.

> > If the kernel does not support the sessionless protocol, then the server
> > must execute the lookups necessary to map from session identifiers to the
> > appropriate endpoint capabilities. The cache and TLB misses associated
> > with this activity must be accounted as part of the IPC costs in order
> > to make an accurate comparison.
> 
> The session-id -> return-endpoint lookup in user-level could be the identity
> function, since the server is free to manage the badges. The following
> capability-space lookup could be more expensive, but this is optimizable
> with capability registers...

There exists in the world an infinite set of hypothetically feasible
optimizations. This is not interesting. The interesting questions are:

  1. What patterns are going to get used in realistic practice, and
  2. What is their cost.

I agree that the badges can be managed successfully in some servers, but
the optimization you suggest does not look like a general design pattern
(because badges have *other* uses that are important).

I personally tend to believe that hand-customizing servers is something
that should be a last resort, because these customizations become an
ongoing engineering burden.

But even if the lookup cost is zero, the C-space traversal cost is
likely to be significant. If the sessions have long lives, locality and
caching of the C-space entries would seem to be tricky.

> > And in fact, if you look at the EROS window system, you will discover
> > that it is sessionless. So is our current ethernet driver (though this
> > one needs to change).
> 
> Both components are session based in DROPS.

Strictly speaking, there is a small amount of session state in EWS as
well. We consider this a bug that we would like to repair. I was typing
too quickly. What I should have said is that EWS does not store any
per-session capabilities.

shap

Re: Persistence

On Mon, 2005-10-31 at 21:26 +0100, Emmanuel Colbus wrote:
> Marcus Brinkmann wrote :
> 
> > Capabilities to resources outside of the persistent core (device
> > drivers, external filesystems, network) have to be invalidated on
> > recover.
> > 
> > This will make the applications that rely on them get a fault, which
> > they can handle by reconnecting (and then verifying their consistency
> > requirements!) or by terminating.
> > 
> 
> Yes, this may work as soon as the application tries to perform an action 
> on the given capability, but what if it was just waiting for data 
> to be available?

Coyotos is an atomic action kernel, so the only way the process can be
waiting this way is if the read() has not made any progress. When the
system restarts, the read() will be restarted, and the caller will
discover that the descriptor is now invalid.

> Is it possible for it to know 
> which of its capabilities refer to something outside the persistent 
> core?

This is an OS-level design issue. There are unambiguous rules at the
microkernel level, but the real issue here is "remoted" objects, which
are an application-level issue.

> And what if the system crashes again, but has taken his last snapshot 
> during application reconnection? (Well, such a reconnection may take a 
> long time...) Will an application who is in its recovery fault handler 
> receive the fault another time?

There is no need to deliver any further fault. The application is
already recovering, and will simply continue recovering after the second
restart.

> And what if one finds a way to deterministically crash the system, and starts
> a task which will crash it just after a snapshot? Or, worse, after twenty 
> days, just after a snapshot, and everytime it receives the system recovery
> fault, and everytime the date is set over task start time + 20 days?

Then you get to fix the kernel.

In 35+ years we haven't found a way to do this, except at certain times
when unreleased code was under active development.

shap