Steven Rostedt | 1 Feb 02:35
Gravatar

Re: Unreliable Wireless network with 3.0.9-rt26

On Mon, 2012-01-30 at 18:45 +0100, Christian Kapeller wrote:
> Hi,
> 
> I'm having trouble running wifi in combination with a rt enabled kernel
> 3.0.9-rt26 on an imx51 arm platform. The distro is openwrt-trunk.
> 
> The device has 2 usb wifi interfaces (AR9271 chipset; driver: based on
> compat-wireless-testing 2011-12-01). I am running an iperf load on both
> interfaces, one interface is in AP mode, the other in STA mode.

You added your own driver?

> 
> When I configure the kernel without RT preemption, the device runs
> stable for >10 hours.
> 
> With RT preemption enabled, after a couple of minutes some interface
> will stop transmitting data. Sometimes ifconfig wlan[0,1] down will
> hang, sometimes ifconfig wlan[0,1] down will work, but then ping will
> 
> return the following:
> 
> # ping 192.168.2.170
> PING 192.168.2.170 (192.168.2.170): 56 data bytes
> ping: sendto: Operation not permitted
> 
> Right after loading the wireless drivers I get the following kernel
> warning:
> 
> [   36.917065] ------------[ cut here ]------------
(Continue reading)

Steven Rostedt | 1 Feb 02:36
Gravatar

Re: Unreliable Wireless network with 3.0.9-rt26

On Tue, 2012-01-31 at 17:08 +0100, Tim Sander wrote:
> Hi
> > I'm having trouble running wifi in combination with a rt enabled kernel
> > 3.0.9-rt26 on an imx51 arm platform. The distro is openwrt-trunk.
> > 
> > The device has 2 usb wifi interfaces (AR9271 chipset; driver: based on
> > compat-wireless-testing 2011-12-01). I am running an iperf load on both
> > interfaces, one interface is in AP mode, the other in STA mode.
> > 
> > When I configure the kernel without RT preemption, the device runs
> > stable for >10 hours.
> > 
> > With RT preemption enabled, after a couple of minutes some interface
> > will stop transmitting data. Sometimes ifconfig wlan[0,1] down will
> > hang, sometimes ifconfig wlan[0,1] down will work, but then ping will
> 
> Just my 5 cents: I am working on the imx 35 platform and i am also having 
> problems with the network setup. I have sent a bugsplat to this list which 
> occurs in the network setup with 3.2-rt. I suspect that these two errors are 
> related. Steven Rosted wanted to look after it, but it seems he hasn't until 
> now.
> 

Yeah, I've been hunting down other nasty bugs in the mean time. I could
start looking at this. I suspect it may be due to another try_lock()
thingy.

-- Steve

--
(Continue reading)

Hector Palacios | 1 Feb 13:28
Favicon

infinite spin in RT when booting with DHCP on

Hello,

I'm working on a 2.6.31.14 kernel on ARM where I applied the RT PREEMPT patch 
2.6.31.12-rt21.

When booting my platform with DHCP on, the DHCP request is sent by the network driver 
before the PHY has even started the autonegotiation.
Since the PHY is not ready, the TX interrupt returns with NETDEV_TX_BUSY but the 
softirq [sirq-net-tx] seems to have entered an infinite spin, as my system is 
practically hung and 'top' reveals [sirq-net-tx/0] is consuming 95% of CPU. This is 
preventing the PHY autonegotiation (which is scheduled as a delayed work) to start, so 
the PHY is never ready and the packet never reaches the network.

I was wondering if this situation resembles what the patch by Ingo Molnar "tasklet/rt: 
Prevent tasklets from going into infinite spin in RT" describes.

This patch is already in 2.6.31.12-rt21 patch which I'm using so either it is a 
different problem or a corner case of the same issue.

Could anyone tell whether it is the same or a different problem?
Thank you
--

-- 
Héctor Palacios

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

Christian Kapeller | 1 Feb 16:22
Picon

Re: Unreliable Wireless network with 3.0.9-rt26

Hi,

thank you for investigating this. I could reproduce the issue with with 
v3.0.9-rt26 from rt stable git patches, and the 
compat-wireless-2011-12-01 sources. The arch specific stuff came to me
as one big 1 MiB patch, which I hesitate to attach here. Here is a link:
http://www.cmotion.eu/wp/wp-content/uploads/2012/02/linux-3.0.9-rt26-karo-tx51-cworld.patch_.gz

On 02/01/2012 02:35 AM, Steven Rostedt wrote:
> On Mon, 2012-01-30 at 18:45 +0100, Christian Kapeller wrote:
>> Hi,
>>
>> I'm having trouble running wifi in combination with a rt enabled kernel
>> 3.0.9-rt26 on an imx51 arm platform. The distro is openwrt-trunk.
>>
>> The device has 2 usb wifi interfaces (AR9271 chipset; driver: based on
>> compat-wireless-testing 2011-12-01). I am running an iperf load on both
>> interfaces, one interface is in AP mode, the other in STA mode.
> 
> You added your own driver?

That's openwrt related patches. They regularly take compat-wireless snapshots,
and apply a number of patches against that. Openwrt comes as buildroot

> 
>>
>> When I configure the kernel without RT preemption, the device runs
>> stable for >10 hours.
>>
>> With RT preemption enabled, after a couple of minutes some interface
(Continue reading)

Steven Rostedt | 1 Feb 21:21
Gravatar

Re: Unreliable Wireless network with 3.0.9-rt26

Please do not remove Cc's! I may only read this mailing list a couple of
times a week. If you want me to see your reply, Cc me. When I'm on a
thread, I will usually ignore replies in the mailing list, because I
expected to have read it in my inbox. Thus I may not even see your
replies.

I know some other mailing lists ask to just reply to the list and not to
Cc. I never listen to them :-)  That's just a list maintainers arrogance
thinking that everyone reads the list every day. I don't and seldom do.
I have enough trouble keeping up with my inbox to read mailing lists.

On Wed, 2012-02-01 at 16:22 +0100, Christian Kapeller wrote:
> Hi,
> 
> thank you for investigating this. I could reproduce the issue with with 
> v3.0.9-rt26 from rt stable git patches, and the 
> compat-wireless-2011-12-01 sources. The arch specific stuff came to me
> as one big 1 MiB patch, which I hesitate to attach here. Here is a link:
> http://www.cmotion.eu/wp/wp-content/uploads/2012/02/linux-3.0.9-rt26-karo-tx51-cworld.patch_.gz

I applied it, and I still don't see the line number to below in rx.c.

> 
> On 02/01/2012 02:35 AM, Steven Rostedt wrote:
> > On Mon, 2012-01-30 at 18:45 +0100, Christian Kapeller wrote:
> >> Hi,
> >>
> >> I'm having trouble running wifi in combination with a rt enabled kernel
> >> 3.0.9-rt26 on an imx51 arm platform. The distro is openwrt-trunk.
> >>
(Continue reading)

Steven Rostedt | 1 Feb 21:27
Gravatar

Re: [ANNOUNCE] 3.0.14-rt31 - ksoftirq running wild - FEC ethernet driver to blame? Yep

On Wed, 2012-01-25 at 10:31 +0100, Tim Sander wrote:

> Bad news, the ksoftirq(sirq-net-tx) error seems to happen not so often any 
> more but it still happens :-(.

Tim,

Is the system still usable when this happens? If so, can you configure
in ftrace, and run a trace on what ksoftirq is doing:

mkdir /debug
mount -t debugfs nodev /debug
cd /debug/tracing
echo <pid-of-ksoftirq> > set_ftrace_pid
echo function > current_tracer
cat trace

Thanks!

-- Steve

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Raz | 2 Feb 11:52
Picon

preempt-rt: Real Time hogging task

hey

I am trying to understand why a user space MT process behaves in an
unexpected manner.
I have real time process, which executes most of its threads in RT priority,
and from time to time a task ( medium priority ) is executing without
trying to stop.

There are times, that this task is hogging the entire process and no
other **higher*** rt priority
task is getting any cpu time.

Linux is **not** getting hogged. when setting the serial console (
irq/serial ) and its shell ( /bin/sh )
to a higher priority linux is responsive.

Observing the task list  through ps command i can see that this task's
threads when
to un-interruptible sleep.

Any idea why is it happening ?
kernel is 3.2.0-rc5-rt8.
raz
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Tim Sander | 2 Feb 13:38
Favicon

Re: infinite spin in RT when booting with DHCP on

Hi
> When booting my platform with DHCP on, the DHCP request is sent by the
> network driver before the PHY has even started the autonegotiation.
> Since the PHY is not ready, the TX interrupt returns with NETDEV_TX_BUSY
> but the softirq [sirq-net-tx] seems to have entered an infinite spin, as
> my system is practically hung and 'top' reveals [sirq-net-tx/0] is
> consuming 95% of CPU. This is preventing the PHY autonegotiation (which is
> scheduled as a delayed work) to start, so the PHY is never ready and the
> packet never reaches the network.
Mh, i think i am hit by the same problem. I have a i.mx 35 and when dhcp is
enabled the ksoftirq is also running wild with 3.0-rt. This also happens when
static ip is configured and the probably during network transfer the network
interface is reconfigured. Basically it seems the sirq-net-tx thread tries to
send a packet on a non configured interface.

But one thing that makes me thinking, is that this thing is only happening on
arm and not on x86. So Hector what platform are you working on. Are you by
chance using the same fec.c ethernet driver?

I have verified that in my case the driver takes always the return statement in
line fec.c:247: return NETXDEV_TX_BUSY;  
It never stops on a breakpoint set on line 250 which shows that the interface
gets never configured.

I have taken some screenshots of my hw debugger:

trace:http://private.vlsi.informatik.tu-darmstadt.de/tstone/linux/fec_enet_start_xmit.png
stack:http://private.vlsi.informatik.tu-darmstadt.de/tstone/linux/fec_enet_start_xmit_stacktrace.png
locals:http://private.vlsi.informatik.tu-darmstadt.de/tstone/linux/fec_enet_start_xmit_stack+locals.png

(Continue reading)

Hector Palacios | 2 Feb 13:57
Favicon

Re: infinite spin in RT when booting with DHCP on

Hello,

On 02/02/2012 01:38 PM, Tim Sander wrote:
 > Mh, i think i am hit by the same problem. I have a i.mx 35 and when dhcp is
 > enabled the ksoftirq is also running wild with 3.0-rt. This also happens when
 > static ip is configured and the probably during network transfer the network
 > interface is reconfigured. Basically it seems the sirq-net-tx thread tries to
 > send a packet on a non configured interface.
 >
 > But one thing that makes me thinking, is that this thing is only happening on
 > arm and not on x86. So Hector what platform are you working on. Are you by
 > chance using the same fec.c ethernet driver?

I'm working on an i.MX51 and using the fec.c driver as well. Interesting that it is 
not happening on x86.

 > I have verified that in my case the driver takes always the return statement in
 > line fec.c:247: return NETXDEV_TX_BUSY;
 > It never stops on a breakpoint set on line 250 which shows that the interface
 > gets never configured.

Autonegotiation is triggered by phy_state_machine() at phy.c which is scheduled as a 
delayed work by phy_device.c upon PHY device creation.
This is not even started when the fec_enet_start_xmit() function is called.

 > I have taken some screenshots of my hw debugger:
 >
 > 
trace:http://private.vlsi.informatik.tu-darmstadt.de/tstone/linux/fec_enet_start_xmit.png
 > 
(Continue reading)

Steven Rostedt | 2 Feb 15:12
Gravatar

Re: preempt-rt: Real Time hogging task

On Thu, 2012-02-02 at 12:52 +0200, Raz wrote:
> hey
> 
> I am trying to understand why a user space MT process behaves in an
> unexpected manner.
> I have real time process, which executes most of its threads in RT priority,
> and from time to time a task ( medium priority ) is executing without
> trying to stop.
> 
> There are times, that this task is hogging the entire process and no
> other **higher*** rt priority
> task is getting any cpu time.
> 
> Linux is **not** getting hogged. when setting the serial console (
> irq/serial ) and its shell ( /bin/sh )
> to a higher priority linux is responsive.
> 
> Observing the task list  through ps command i can see that this task's
> threads when
> to un-interruptible sleep.
> 
> Any idea why is it happening ?

Some ideas, but I need a lot more information.

> kernel is 3.2.0-rc5-rt8.

Is this a x86 box?

Can you run a trace on the task that is going berserk?
(Continue reading)


Gmane