Running real time task on one of the cores of a Dual Core CPU

Hi all,

I have some results and some questions on running Xenomai on a Dual Core CPU.
At my PC I have a Pentium D CPU (3.2 GHz), kernel 2.6.19.2, Xenomai 2.3 + some COW patches.
In the kernel I have enabled SMP.

The first interesting thing I noticed is that this kernel seems to run all standard processes
on core 0. The kernel does not automatically distribute processes on two cores.
I think I saw a different behaviour with an earlier kernel version (2.6.17 or even 2.4.33).
I do not have to use the isolcpus kernel parameter to force running all user space tasks on core 0.
This is exactly the thing I wanted to have. All "standard" linux tasks are running
on core 0.
Does anybody know if there was a change in the kernel?

Now I placed all real time tasks from Xenomai on core 1. This works fine.
My application calculates periodically a fairly complex simulation model. The interesting thing
is now, that the computation time needed for this model is much more constant in opposite to the
UP mode. On an UP system I had computation times between 111 and 170 microseconds.
With the SMP system I have now computation times between 111 and 115 microseconds.
Great! The worst case execution time is very close to the best case execution time!
I think this is caused by a more deterministic usage of the cache for the real time system.
In the UP system the cache cannot be reserved for the real time application, i.e. the real time
application has to re-read more often directly from memory.

Now my questions:
In a setup that uses core 0 for standard linux stuff and core 1 for Xenomai stuff the
Xenomai scheduler is running in SMP mode. However, I am not sure if this required in this setup.
As all real time code is bound to one core there does not have to be any multi processing
save synchronisation for the real time code.
My proposal is to support three different scheduling modes:

Re: Running real time task on one of the cores of a Dual Core CPU

M. Koehrer wrote:
> Hi all,
> 
> I have some results and some questions on running Xenomai on a Dual Core CPU.
> At my PC I have a Pentium D CPU (3.2 GHz), kernel 2.6.19.2, Xenomai 2.3 + some COW patches.
> In the kernel I have enabled SMP.
> 
> The first interesting thing I noticed is that this kernel seems to run all standard processes
> on core 0. The kernel does not automatically distribute processes on two cores.
> I think I saw a different behaviour with an earlier kernel version (2.6.17 or even 2.4.33).
> I do not have to use the isolcpus kernel parameter to force running all user space tasks on core 0.
> This is exactly the thing I wanted to have. All "standard" linux tasks are running
> on core 0.
> Does anybody know if there was a change in the kernel?
> 
> Now I placed all real time tasks from Xenomai on core 1. This works fine.
> My application calculates periodically a fairly complex simulation model. The interesting thing
> is now, that the computation time needed for this model is much more constant in opposite to the
> UP mode. On an UP system I had computation times between 111 and 170 microseconds.
> With the SMP system I have now computation times between 111 and 115 microseconds.
> Great! The worst case execution time is very close to the best case execution time!
> I think this is caused by a more deterministic usage of the cache for the real time system.
> In the UP system the cache cannot be reserved for the real time application, i.e. the real time
> application has to re-read more often directly from memory.

Yep, that makes sense.

> 
> Now my questions:
> In a setup that uses core 0 for standard linux stuff and core 1 for Xenomai stuff the

Re: Re:: Running real time task on one of the cores of a Dual

Hi Jan,

>> I also use rtnet. Unfortunately I am not able to force rtnet (RTDM based) to run on CPU core 1
>> without patching the RTDM code.
>> A couple of months ago, I already asked for this issue.
>> Jan, can you tell me the latest news regarding this issue?
>
>The global affinity mask is now part of Xenomai 2.3 (though I haven't
>tested it again after it was merged). 

Ah, I didn't know that! How can I set the affinity mask?
Perhaps this is the problem I have...

>What part of RTnet precisely is
>resisting to move to CPU 1, or what do you currently have to patch?

Currently I see the rtnet tasks "rtnet-stack" and "rtnet-rtpc" to be assigned to CPU 0.
I have patched xenomai/ksrc/skins/rtdm/drvlib.c in the call
to xnpod_start_thread(). I replace XNPOD_ALL_CPUS by cpumask_of_cpu(1).

Regards

Mathias

--

-- 
Mathias Koehrer
mathias_koehrer <at> arcor.de

Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig

Re: : Running real time task on one of the cores of a Dual

M. Koehrer wrote:
> Hi Jan,
> 
>>> I also use rtnet. Unfortunately I am not able to force rtnet (RTDM based) to run on CPU core 1
>>> without patching the RTDM code.
>>> A couple of months ago, I already asked for this issue.
>>> Jan, can you tell me the latest news regarding this issue?
>> The global affinity mask is now part of Xenomai 2.3 (though I haven't
>> tested it again after it was merged). 
> 
> Ah, I didn't know that! How can I set the affinity mask?

/proc/xenomai/affinity

> Perhaps this is the problem I have...
> 
>> What part of RTnet precisely is
>> resisting to move to CPU 1, or what do you currently have to patch?
> 
> Currently I see the rtnet tasks "rtnet-stack" and "rtnet-rtpc" to be assigned to CPU 0.
> I have patched xenomai/ksrc/skins/rtdm/drvlib.c in the call
> to xnpod_start_thread(). I replace XNPOD_ALL_CPUS by cpumask_of_cpu(1).

That should be obsolete when issuing

echo 2 > /proc/xenomai/affinity

before you load RTnet.

Jan

Re: Re: : Running real time task on one of the cores of

Hi Jan,

I found it in the latest SVN version but not in the 2.3 Xenomai... 
As my system is working currently very stable, I do not want to change the Xenomai version...

Regards

Mathias
> M. Koehrer wrote:
> > Hi Jan,
> > 
> >>> I also use rtnet. Unfortunately I am not able to force rtnet (RTDM
> based) to run on CPU core 1
> >>> without patching the RTDM code.
> >>> A couple of months ago, I already asked for this issue.
> >>> Jan, can you tell me the latest news regarding this issue?
> >> The global affinity mask is now part of Xenomai 2.3 (though I haven't
> >> tested it again after it was merged). 
> > 
> > Ah, I didn't know that! How can I set the affinity mask?
> 
> /proc/xenomai/affinity
> 
> > Perhaps this is the problem I have...
> > 
> >> What part of RTnet precisely is
> >> resisting to move to CPU 1, or what do you currently have to patch?
> > 
> > Currently I see the rtnet tasks "rtnet-stack" and "rtnet-rtpc" to be
> assigned to CPU 0.

Re: : Running real time task on one of the cores of

M. Koehrer wrote:
> Hi Jan,
> 
> I found it in the latest SVN version but not in the 2.3 Xenomai... 
> As my system is working currently very stable, I do not want to change the Xenomai version...

Oh, indeed, that got merged after the release. Obviously I'm too
bleeding-edge focused. :)

Try to extract the diff of #2011 and apply it on your tree. This should
fit seamlessly.

Jan

_______________________________________________
Xenomai-help mailing list
Xenomai-help <at> gna.org
https://mail.gna.org/listinfo/xenomai-help

Re: Re: : Running real time task on one of the cores of

Hi Jan, 

I tried to apply the patch of #2011 to the 2.3 Xenomai and this works fine.
I can now determine the CPU to run all real time stuff by writing into
/proc/xenomai/affinity.

Thanks a lot.

Mathias
> > I found it in the latest SVN version but not in the 2.3 Xenomai... 
> > As my system is working currently very stable, I do not want to change the
> Xenomai version...
> 
> Oh, indeed, that got merged after the release. Obviously I'm too
> bleeding-edge focused. :)
> 
> Try to extract the diff of #2011 and apply it on your tree. This should
> fit seamlessly.
> 

--

-- 
Mathias Koehrer
mathias_koehrer <at> arcor.de

Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Error: cat: /proc/xenomai/stat: Cannot allocate memory

Hi everybody,

I have a strange issue frequently when reading /proc/xenomai/stat.
I do a cat /proc/xenomai/stat and I get an "Cannot allocate memory" error.
This happens typcially after I have started and terminated by application that uses a number of tasks.
I am able to reproduce it deterministically...
While the application is running, I can see the stat values perfectly.
Here is the head output of all /proc/xenomai/* files:

==> affinity <==
00000002

==> apc <==
APC          CPU0         CPU1
  0:            0           0    (pipe_wakeup)
  1:           49          54    (lostage_handler)
  2:           12          23    (registry_export)

==> faults <==
TRAP         CPU0        CPU1
  0:            0           0    (Divide error)
  1:            0           0    (Debug)
  3:            0           0    (Int3)
  4:            0           0    (Overflow)
  5:            0           0    (Bounds)
  6:            0           0    (Invalid opcode)
  7:            0           0    (FPU not available)
  8:            0           0    (Double fault)
  9:            0           0    (FPU segment overrun)
 10:            0           0    (Invalid TSS)

Re: Error: cat: /proc/xenomai/stat: Cannot allocate memory

On Fri, 2007-02-02 at 09:43 +0100, M. Koehrer wrote:
> Hi everybody,
> 
> I have a strange issue frequently when reading /proc/xenomai/stat.
> I do a cat /proc/xenomai/stat and I get an "Cannot allocate memory" error.
> This happens typcially after I have started and terminated by application that uses a number of tasks.
> I am able to reproduce it deterministically...
> While the application is running, I can see the stat values perfectly.
> Here is the head output of all /proc/xenomai/* files:

At first sight looking at the code, there seems to be a memory leak
caused by an unfree kmalloc() block in the sched & stat sequence
routines. Will confirm and fix. Thanks,

--

-- 
Philippe.

Re: Error: cat: /proc/xenomai/stat: Cannot allocate memory

On Fri, 2007-02-02 at 10:33 +0100, Philippe Gerum wrote:
> On Fri, 2007-02-02 at 09:43 +0100, M. Koehrer wrote:
> > Hi everybody,
> > 
> > I have a strange issue frequently when reading /proc/xenomai/stat.
> > I do a cat /proc/xenomai/stat and I get an "Cannot allocate memory" error.
> > This happens typcially after I have started and terminated by application that uses a number of tasks.
> > I am able to reproduce it deterministically...
> > While the application is running, I can see the stat values perfectly.
> > Here is the head output of all /proc/xenomai/* files:
> 
> At first sight looking at the code, there seems to be a memory leak
> caused by an unfree kmalloc() block in the sched & stat sequence
> routines. Will confirm and fix. Thanks,
> 

Not confirmed for the leakage part regarding kmalloc(). We do have a
seq_release_private() freeing the allocated block, so back to square #1.
This said, some kernel memory is not released somewhere, as running
vmstat tells me here.

Btw, is the rest of your system functional after the error occurs, or is
a reboot needed afterwise?

--

-- 
Philippe.