headers
David Given | 1 Jun 2010 19:25

Re: Porting recommendations


On 28/05/10 00:00, Andrew Dennison wrote:
[...]
> I can describe it in general terms.
[...snip...]

That's very interesting; it's nice to hear how this sort of thing works
in the real world. ARM, I presume? If I may ask, what made you decide to
use Prex and not a more established OS like ECOS or FreeRTOS?

[...]
> I would like to add features to Prex like tickless operation, Futexes
> and a better model for task / server memory operations as a few
> examples but it would be nice to have a proper collaborative
> development model so I'm not effectively maintaining a fork that has
> to be rebased onto the moving target of the "official" tarballs that
> come out occasionally.

Yes, it's a pity. I suspect Kohsuke Ohtani simply doesn't have
time/inclination to be a source herder. As you know, it's a thankless
and surprisingly complicated job.

Incidentally, I'm making decent progress with the Cybiko port. I managed
to persuade the build system to produce a linked-together image
consisting of kernel and bootup tasks, which lets me run them all out of
flash. I've got it booting up into 'user' mode and trying to load
/boot/init. Currently that's failing because I haven't done an H8300S
elf relocator yet. But it's running multiple processes and threads,
interrupts are working, the timer is running --- no preemption or
exceptions yet, but that's not bad for under a week's work.
(Continue reading)

Permalink | Reply | 4 comments |
headers
Ivo Vachkov | 1 Jun 2010 21:54
Picon
Gravatar

Re: Porting recommendations

Hello,

On Tue, Jun 1, 2010 at 8:25 PM, David Given <dg <at> cowlark.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 28/05/10 00:00, Andrew Dennison wrote:
> [...]
>> I can describe it in general terms.
> [...snip...]
>
> That's very interesting; it's nice to hear how this sort of thing works
> in the real world. ARM, I presume? If I may ask, what made you decide to
> use Prex and not a more established OS like ECOS or FreeRTOS?
>
> [...]
>> I would like to add features to Prex like tickless operation, Futexes
>> and a better model for task / server memory operations as a few
>> examples but it would be nice to have a proper collaborative
>> development model so I'm not effectively maintaining a fork that has
>> to be rebased onto the moving target of the "official" tarballs that
>> come out occasionally.
>
> Yes, it's a pity. I suspect Kohsuke Ohtani simply doesn't have
> time/inclination to be a source herder. As you know, it's a thankless
> and surprisingly complicated job.
>
> Incidentally, I'm making decent progress with the Cybiko port. I managed
> to persuade the build system to produce a linked-together image
> consisting of kernel and bootup tasks, which lets me run them all out of
(Continue reading)

Permalink | Reply | 1 comment |
headers
David Given | 2 Jun 2010 02:07

Re: Porting recommendations


On 01/06/10 20:54, Ivo Vachkov wrote:
[...]
> Please, keep the make/gmake build system. It may be a bit clumsy but
> at least it's portable and already present on all major OSs. I really
> don't want to install yet another make alternative for every project i
> [try to] participate into :)

Don't worry, I have neither the authority nor the ability to change
anything with the project unless asked!

But I'm spending about half my time now rebuilding everything from
scratch whenever I make a minor change, because there's no dependency
management. (If you make a change to, say, systraps.h, you need to
rebuild all the user-space programs, recreate the boot disk, and then
regenerate the boot image. None of this happens automatically.)

I wrote Prime Mover when trying to deal with a big compiler project that
used lots of dynamically generated source files, lots of rebuilding the
same source file with slightly different options, lots of
cross-compiling big chunks of the tree multiple times for different
architectures --- all stuff that make does *really badly*.

Also, there are no dependencies! The whole Prime Mover executable is a
single shell script that deploys with the project, so no installation
needed.

Basic summary and example here:

http://primemover.sourceforge.net/about.html
(Continue reading)

Permalink | Reply | 0 comments |
headers
Andrew Dennison | 2 Jun 2010 03:41
Picon

Prex public tree

Hi Kohsuke,

Is there anything I can do to help setup a public tree that you can
push your changes into? I would really like to help work out a
mechanism that you are happy with so Prex can gain some momentum.

Do you have any particular tools and procedures in mind?

Regards,

Andrew

------------------------------------------------------------------------------
Permalink | Reply | 0 comments |
headers
Andrew Dennison | 2 Jun 2010 03:31
Picon

Re: Porting recommendations

On Wed, Jun 2, 2010 at 3:25 AM, David Given <dg <at> cowlark.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 28/05/10 00:00, Andrew Dennison wrote:
> [...]
>> I can describe it in general terms.
> [...snip...]
>
> That's very interesting; it's nice to hear how this sort of thing works
> in the real world. ARM, I presume? If I may ask, what made you decide to
> use Prex and not a more established OS like ECOS or FreeRTOS?

BookE PowerPC. First think I did with Prex was port it to a new
architecture. We wanted to use the MMU for security reasons as we run
"untrusted" generated code. Once we collated our requirements the list
of RTOS options got rather narrow. We seriously consider some
expensive commercial options too but in the end Prex was the best fit.

[...]

> I suspect Kohsuke Ohtani simply doesn't have
> time/inclination to be a source herder. As you know, it's a thankless
> and surprisingly complicated job.

In our early discussions he was always keen to get more contributors
to Prex. Maybe real life has got in the way over the last few years?

I started up my public tree because I wanted to help in this area but
all the changes in 0.9 have stalled that effort.
(Continue reading)

Permalink | Reply | 1 comment |
headers
David Given | 3 Jun 2010 01:56

vfork on non-MMU devices


I am confused about how the implementation of vfork for non-MMU devices
works. It's late, but I'm pretty sure it *can't* work, and would like to
know if anyone has any non-MMU devices where init starts properly.

What it does is to create a child task and then go to sleep. The child
longjmps back into the parent task's stack, vfork returns, and the child
runs code until it calls exec() or exit(). At this point the parent
wakes up again and continues executing.

Except that because both the child and the parent are running on the
same stack, and the child is *returning* from vfork, won't the child
trash the stack that the parent is going to need to use when it wakes up
again?

What's worse, when the new task is created, no new user-mode stack is
specified. This means that the new task runs on the same stack as the
old task --- but starting from the top again, so overwriting the old
task's stack, resulting in horrible stack corruption.

Has anyone seen this work?

--

-- 
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─────
http://www.cowlark.com ─────
│
│ life←{ ↑1 ⍵∨.^3 4=+/,¯1 0 1∘.⊖¯1 0 1∘.⌽⊂⍵ }
│ --- Conway's Game Of Life, in one line of APL
Permalink | Reply | 4 comments |
headers
Lwazi | 3 Jun 2010 03:58
Picon

Re: vfork on non-MMU devices

On Wed, Jun 2, 2010 at 7:56 PM, David Given <dg <at> cowlark.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I am confused about how the implementation of vfork for non-MMU devices
> works. It's late, but I'm pretty sure it *can't* work, and would like to
> know if anyone has any non-MMU devices where init starts properly.
>
I don't use the MMU even though my hardware has one. And I am new to prex.

Initializing RAM disk
RAM disk at 0x000737e4 (75K bytes)
Initializing TTY device
Driver initialized
Loading task:'boot'
Loading task:'proc'
Loading task:'fs'
Loading task:'exec'
Starting Bootstrap Server
Starting Process Server
Starting File System Server
Starting Exec Server
boot: Mounting file systems
VFS: Mounting ramfs dev= dir=/
VFS: Mounting devfs dev= dir=/dev
VFS: Mounting arfs dev=/dev/ram0 dir=/boot
VFS: Mounting fifofs dev= dir=/fifo
boot: Run init process
Prex version 0.8.1 (arm-orion)
[prex:/]#
(Continue reading)

Permalink | Reply | 0 comments |
headers
Andrew Dennison | 3 Jun 2010 13:48
Picon

Re: vfork on non-MMU devices

On Thu, Jun 3, 2010 at 9:56 AM, David Given <dg <at> cowlark.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I am confused about how the implementation of vfork for non-MMU devices
> works. It's late, but I'm pretty sure it *can't* work, and would like to
> know if anyone has any non-MMU devices where init starts properly.

Hi David,

It does work, we ran without the MMU for a while without issue. I've
also booted the arm and i386 ports both with and without the mmu and
everything started up ok.

At one stage I could have discussed this with you in great detail but
my memory of non-MMU operation is fading. I can refresh my memory on
this if you would like to discuss details, but fundamentally the
structure guarantees only the child or parent is running at any one
time and the setjump / longjump restores the context. The calls into
the proc and exec servers also do a lot of the heavy lifting behind
the scenes. The proc server may actually copy and restore the stack if
my memory serves me correctly.

Andrew

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate 
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the 
lucky parental unit.  See the prize list and enter to win: 
http://p.sf.net/sfu/thinkgeek-promo
(Continue reading)

Permalink | Reply | 2 comments |
headers
David Given | 3 Jun 2010 18:24

Re: vfork on non-MMU devices


On 03/06/10 12:48, Andrew Dennison wrote:
[...]
> It does work, we ran without the MMU for a while without issue. I've
> also booted the arm and i386 ports both with and without the mmu and
> everything started up ok.

Okay, in which case it's obviously me.

> ...fundamentally the
> structure guarantees only the child or parent is running at any one
> time and the setjump / longjump restores the context. The calls into
> the proc and exec servers also do a lot of the heavy lifting behind
> the scenes. The proc server may actually copy and restore the stack if
> my memory serves me correctly.

What I'm seeing is:

 - vfork is called with stack (SP-n)
 - vfork launches new thread, which gets stack (SP) (i.e. the same as
the parent's original stack pointer)
 - new thread runs a stub routine which longjmps to (SP-n)
 - vfork resumes, in the child thread
 - vfork crashes on return

It's crashing --- I think --- because the stub routine's own stack usage
is overwriting vfork()'s caller's stack, since n is quite small. If my
analysis is correct it ought to be easily fixable by telling the stub
not to overwrite the stack. This may not affect ARM because ARM's got
more registers and doesn't use the stack as much as the H8300S, but I
(Continue reading)

Permalink | Reply | 1 comment |
headers
David Given | 4 Jun 2010 13:17

Re: vfork on non-MMU devices


On 03/06/10 17:24, David Given wrote:
[...]
> It's crashing --- I think --- because the stub routine's own stack usage
> is overwriting vfork()'s caller's stack, since n is quite small. If my
> analysis is correct it ought to be easily fixable by telling the stub
> not to overwrite the stack.

The attached trivial patch does exactly this, and everything now works fine.

(Although init's calling execl() now fails later, but that's due to an
honest emulation bug and therefore Another Story.)

Any comments welcome --- if this *is* a Prex bug, I don't know why other
platforms aren't running into it, but if it's a bug in my BSP code, it's
one based on my understanding of how the stack handling works and
therefore not subject to analysis by code inspection.

diff --git a/usr/lib/posix/process/fork_nommu.c
b/usr/lib/posix/process/fork_nommu.c
index d69800d..c48eb6d 100755
--- a/usr/lib/posix/process/fork_nommu.c
+++ b/usr/lib/posix/process/fork_nommu.c
 <at>  <at>  -88,6 +88,7  <at>  <at>  vfork(void)
 	task_t tsk;
 	thread_t t;
 	int error, sts, pri;
+	char stubstack[256];

 	/* Save current stack pointer */
(Continue reading)

Permalink | Reply | 0 comments |

Gmane