Carsten Emde | 4 Mar 22:27 2015

[OSADL QA 3.18.7-rt2 #2] BUG: scheduling while atomic: nfsd and irq/30-eth0

An NFS server occasionally runs into "BUG: scheduling while atomic". The 
message appears about once every 2 to 4 minutes at heavy NFS traffic.

There are only two different patterns, one through nfsd and the other 
through the network interrupt thread. I applied Steven's patch 
https://lkml.org/lkml/2014/1/16/366 to get as much information as possible.

Thanks,
Carsten.

irq/30-eth0:
[ 9346.799757] BUG: scheduling while atomic: irq/30-eth0/1123/0x00000002
[ 9346.799791] Modules linked in: eeprom nfsd lockd grace nfs_acl 
exportfs auth_rpcgss oid_registry sunrpc ip6t_REJECT nf_reject_ipv6 
bluetooth nf_conntrack_ipv6 rfkill nf_defrag_ipv6 ip6table_filter 
cpufreq_ondemand ip6_tables it87 hwmon_vid coretemp snd_hda_codec_hdmi 
snd_hda_codec_via snd_hda_codec_generic snd_hda_intel snd_hda_controller 
snd_hda_codec snd_hwdep r8169 mii iTCO_wdt iTCO_vendor_support snd_seq 
snd_seq_device snd_pcm snd_timer xhci_pci xhci_hcd wmi i2c_i801 
serio_raw lpc_ich mfd_core snd microcode pcspkr soundcore acpi_cpufreq 
ipv6 i915 drm_kms_helper drm video i2c_algo_bit i2c_core
[ 9346.799804] Preemption disabled at:[<ffffffffa03fc582>] 
svc_xprt_enqueue+0x16/0x18 [sunrpc]
[ 9346.799804]
[ 9346.799807] CPU: 2 PID: 1123 Comm: irq/30-eth0 Tainted: G        W 
    3.18.7-rt2-nodebug #32
[ 9346.799808] Hardware name: System manufacturer System Product 
Name/P8H61-I, BIOS 0403 02/11/2011
[ 9346.799810]  ffff8800adc49cb0 ffff880137d5b6e8 ffffffff814c6c96 
0000000000000000
(Continue reading)

Carsten Emde | 4 Mar 22:24 2015

[OSADL QA 3.18.7-rt2 #1] WARNING at kernel/time/tick-sched.c:167 can_stop_full_tick+0x2d/0xcd()

Some x86-based systems with

CONFIG_NO_HZ_FULL=y

run into this WARN_ON_ONCE during boot

static bool can_stop_full_tick(void)
{
         WARN_ON_ONCE(!irqs_disabled());

Thanks,
Carsten.

[   22.706429] WARNING: CPU: 2 PID: 33 at kernel/time/tick-sched.c:167 
can_stop_full_tick+0x2d/0xcd()
[   22.706700] Modules linked in: ppdev parport_pc parport cpufreq_stats 
eeprom lockd 8021q garp w83627ehf stp grace llc hwmon_vid coretemp 
ip6table_filter ip6_tables snd_hda_codec_hdmi snd_hda_codec_realtek 
snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec 
snd_hwdep microcode snd_seq iTCO_wdt iTCO_vendor_support snd_seq_device 
snd_pcm serio_raw pcspkr snd_timer snd r8169 lpc_ich soundcore mfd_core 
mii i2c_i801 acpi_cpufreq uinput sunrpc ipv6 i915 drm_kms_helper drm 
video i2c_algo_bit i2c_core
[   22.708082] CPU: 2 PID: 33 Comm: ksoftirqd/2 Not tainted 3.18.7-rt2 #7
[   22.708083] Hardware name: System manufacturer System Product 
Name/P8H67-I DELUXE, BIOS 0502 02/11/2011
[   22.708085]  00000000000000a7 ffff8802353b7c78 ffffffff814f36db 
0000000000000001
[   22.708086]  0000000000000000 ffff8802353b7cb8 ffffffff8103b011 
ffff8802353b7cb8
(Continue reading)

mikhail.avkhimenia | 4 Mar 15:21 2015

3.18 RT freezes on Raspberry Pi 2

Hello,

Recently, Raspberry Pi Foundation presented and started selling new Raspberry Pi 2 SBC. It’s based on
BCM2709 processor, which is an upgraded version of BCM2708. The main difference is that old BCM2708 has
one core and its architecture is ARMv6, while the new BCM2709 has 4 cores and ARMv7 architecture. Most of
the peripheral blocks are the same - DMA, I2C, UART, etc. 

RT patch has been working very well (after fixing the sd driver a couple of years ago) with Raspberry Pi 1 and
kernel 3.12. 
With the release of the new board Raspberry Pi Foundation also switched to the new kernel version - 3.18,
support for Raspberry Pi 2 has been introduced in that version.
I applied patch-3.18.7-rt2 on RPi’s kernel fork and built two kernels - one for Raspberry Pi 1 and another
one for Raspberry Pi 2.
On older Raspberry Pi 1 3.18.7-rt2 seems to work without any problems, I ran cyclictest and stress and it
performed well. 
But on new Raspberry Pi 2 I’m experiencing frequent freezes. 

Steps to reproduce: 
1. Clone https://github.com/raspberrypi/linux , checkout 3.18.7 - 0be82f722c097340632a59b879fbfee9c6148f53
2. Apply patch-3.18.7-rt2
3. (optionally) apply “[RT PATCH] mmc: sdhci: don't provide hard irq handler”
4. Make menuconfig bcm2709_defconfig
5 Configure: full rt preemption, periodic ticks, performance governor as default and optionally
preemption debugging
6. Compile kernel & modules
7. Copy Image to /boot/kernel7.img and arch/arm/boot/dts/bcm2709-rpi-2-b.dtb to
/boot/bcm2709-rpi-2-b.dtb, copy modules.
Boot and run: sudo cyclictest --smp -p51 -m

The bad thing is that it freezes without any message, even with preemption debug information and lockup
(Continue reading)

John Kacur | 3 Mar 19:58 2015
Picon

Re: [PATCH] Makefile: cleanup linking to librttest.a


On Tue, 3 Mar 2015, Michael Olbrich wrote:

> Only add '-lrttest -L.' where it's actually needed.
> Use '$<' instead of '$^'. Otherwise librttest is added twice:
> As 'librttest.a' and as '-lrttest'.
> 
> Signed-off-by: Michael Olbrich <m.olbrich <at> pengutronix.de>
> ---
> 
> Better like this?
> 
> Michael
> 
>  Makefile | 25 +++++++++++++------------
>  1 file changed, 13 insertions(+), 12 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index da7b419c4bcd..5d7f449cc026 100644
> --- a/Makefile
> +++ b/Makefile
>  <at>  <at>  -11,7 +11,8  <at>  <at>  sources += signaltest.c ptsematest.c sigwaittest.c svsematest.c sendme.c \
>  
>  TARGETS = $(sources:.c=)
>  
> -LIBS 	= -lrt -lpthread -lrttest -L.
> +LIBS	= -lrt -lpthread
> +RTTESTLIB = -lrttest -L.
>  EXTRA_LIBS ?= -ldl	# for get_cpu
>  DESTDIR	?=
(Continue reading)

John Kacur | 3 Mar 14:48 2015
Picon

[PATCH]Fix minor grammar mistake in the help output

From ddc409343e19cb2bbc75869a18cbf9b8a5343f31 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur <at> redhat.com>
Date: Tue, 3 Mar 2015 14:17:57 +0100
Subject: [PATCH] Fix minor grammar mistake in the help output

Change "quick" to quicker, so the help message reads
-c    --check               Stop if lower prio task is quicker than
higher (off)

Signed-off-by: John Kacur <jkacur <at> redhat.com>
---
 src/rt-migrate-test/rt-migrate-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/rt-migrate-test/rt-migrate-test.c b/src/rt-migrate-test/rt-migrate-test.c
index 67e1e6c82fe4..fc6fd81ba9ca 100644
--- a/src/rt-migrate-test/rt-migrate-test.c
+++ b/src/rt-migrate-test/rt-migrate-test.c
 <at>  <at>  -186,7 +186,7  <at>  <at>  static void usage(char **argv)
 	       "-s time --sleep-time time   Sleep time (ms) between intervals (100)\n"
 	       "-m time --maxerr time       Max allowed error (microsecs)\n"
 	       "-l loops --loops loops      Number of iterations to run (50)\n"
-	       "-c    --check               Stop if lower prio task is quick than higher (off)\n"
+	       "-c    --check               Stop if lower prio task is quicker than higher (off)\n"
 	       "  () above are defaults \n",
 		p);
 	exit(0);
--

-- 
1.8.3.1

(Continue reading)

Luis Claudio R. Goncalves | 2 Mar 21:15 2015

[PATCH] rcutorture: fix missing function at build time

Hello,

This is a minor issue, but as I seem to have woken up Paul at 4AM with the
many beeps from IRC I would feel bad if I did not send this patch to the
list. Actually, as that patch has been sitting on my computer for a few
days now, i already feel bad :)

rcutorture: fix missing function at build time

Trying to build the rcutorture module on 3.18.7-rt2 ends up on:

   ERROR: "rcu_force_quiescent_state" [kernel/rcu/rcutorture.ko] undefined!

Commit "71d22b5 Subject: rcu: Merge RCU-bh into RCU-preempt" encloses three
functions (rcu_batches_completed_bh, rcu_force_quiescent_state and
rcu_bh_force_quiescent_state) on an #ifndef CONFIG_PREEMPT_RT_FULL / #endif
pair, rendering them non-existent on RT. As the commit points out, that was
a wise and necessary move.

But in order to get rcutorture build working again we need to define an
empty rcu_force_quiescent_state function.

Signed-off-by: Luis Claudio R. Goncalves <lclaudio <at> uudg.org>

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index a4e9e25..28a54f6 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
 <at>  <at>  -378,6 +378,11  <at>  <at>  void rcu_bh_force_quiescent_state(void)
 	force_quiescent_state(&rcu_bh_state);
(Continue reading)

Daniel Wagner | 27 Feb 15:20 2015
Picon

[PATCH v2 0/2] mce patches rebased on simple work queue

Hi,

Here is rebased version of mce patches using the simple work
'framework'. The patch from Steven was straight forward to port. I am
not really sure if my version on Paul's patch is not too
crazy. Feedback is more than welcome.

I tested this with mce-inject and some additional explicit calls to
mce_work_irq() at various early boot phases. It didn't explode for
me. Unfortunalty, qemu's monitor mce command didn't do anything except
crashing the machine. I guess I just use the wrong paremeters.

cheers,
daniel

Paul Gortmaker (1):
  mce: don't try to wake thread before it exists.

Steven Rostedt (1):
  x86/mce: Defer mce wakeups to threads for PREEMPT_RT

 arch/x86/kernel/cpu/mcheck/mce.c | 98 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 86 insertions(+), 12 deletions(-)

--

-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo <at> vger.kernel.org
(Continue reading)

Luis Claudio R. Goncalves | 26 Feb 21:08 2015

3.18.7-rt: BUG: sleeping function called from invalid context (snd_pcm_stream_lock+0x29/0x70)

Hi!

Right after boot I start seeing these messages to:

[   44.046458] BUG: sleeping function called from invalid context at /home/lclaudio/SANDBOX/kernel-rt/kernel/locking/rtmutex.c:915
[   44.046459] in_atomic(): 0, irqs_disabled(): 1, pid: 2515, name: pulseaudio
[   44.046460] 1 lock held by pulseaudio/2515:
[   44.046469]  #0:  (snd_pcm_link_rwlock){......}, at: [<ffffffffa0323c69>]
snd_pcm_stream_lock+0x29/0x70 [snd_pcm]
[   44.046471] CPU: 3 PID: 2515 Comm: pulseaudio Tainted: G        W      3.18.7-rt2+ #2
[   44.046472] Hardware name: Hewlett-Packard p7-1512/2ADA, BIOS 8.15 02/05/2013
[   44.046474]  0000000000000000 00000000989763cd ffff880189d43cb8 ffffffff8175a165
[   44.046475]  00000000989763cd 0000000000000000 ffff880189d43cd8 ffffffff810a3be9
[   44.046476]  0000000000000011 ffffffffa03359c0 ffff880189d43cf8 ffffffff81760a94
[   44.046477] Call Trace:
[   44.046481]  [<ffffffff8175a165>] dump_stack+0x4f/0x7c
[   44.046484]  [<ffffffff810a3be9>] __might_sleep+0xf9/0x170
[   44.046486]  [<ffffffff81760a94>] __rt_spin_lock+0x24/0x40
[   44.046489]  [<ffffffff817612f7>] rt_read_lock+0x57/0x70
[   44.046494]  [<ffffffffa0323c69>] ? snd_pcm_stream_lock+0x29/0x70 [snd_pcm]
[   44.046497]  [<ffffffffa0323c69>] snd_pcm_stream_lock+0x29/0x70 [snd_pcm]
[   44.046501]  [<ffffffffa0324b06>] snd_pcm_hw_params+0x46/0x3f0 [snd_pcm]
[   44.046504]  [<ffffffff811a700b>] ? memdup_user+0x4b/0x90
[   44.046508]  [<ffffffffa0327327>] snd_pcm_common_ioctl1+0x457/0xe20 [snd_pcm]
[   44.046511]  [<ffffffff810ca49e>] ? put_lock_stats.isra.23+0xe/0x30
[   44.046513]  [<ffffffff810ca7ba>] ? lock_release_holdtime.part.24+0x9a/0x160
[   44.046517]  [<ffffffffa03281db>] snd_pcm_capture_ioctl1+0x1bb/0x2f0 [snd_pcm]
[   44.046518]  [<ffffffff810ca49e>] ? put_lock_stats.isra.23+0xe/0x30
[   44.046522]  [<ffffffffa03283d4>] snd_pcm_capture_ioctl+0x34/0x40 [snd_pcm]
[   44.046524]  [<ffffffff81217a10>] do_vfs_ioctl+0x2f0/0x520
(Continue reading)

Luis Claudio R. Goncalves | 26 Feb 21:05 2015

3.18.7-rt2: NOHZ_FULL warning at boot

Hello,

I have been seeing the warning below at boot time on 3.18.7-rt2. Anybody
else seeing it?

[    1.537207] ------------[ cut here ]------------
[    1.537215] WARNING: CPU: 1 PID: 20 at
/home/lclaudio/SANDBOX/kernel-rt/kernel/time/tick-sched.c:167 can_stop_full_tick+0x1ee/0x2b0()
[    1.537219] Modules linked in: efivarfs
[    1.537224] CPU: 1 PID: 20 Comm: ksoftirqd/1 Not tainted 3.18.7-rt2+ #2
[    1.537225] Hardware name: Hewlett-Packard p7-1512/2ADA, BIOS 8.15 02/05/2013
[    1.537227]  0000000000000000 000000005b4bc999 ffff880191503bb8 ffffffff8175a165
[    1.537228]  0000000000000000 0000000000000000 ffff880191503bf8 ffffffff8107add1
[    1.537229]  ffff880191503be8 ffff880198fce280 ffff880198e00000 0000000000000000
[    1.537229] Call Trace:
[    1.537233]  [<ffffffff8175a165>] dump_stack+0x4f/0x7c
[    1.537235]  [<ffffffff8107add1>] warn_slowpath_common+0x81/0xc0
[    1.537237]  [<ffffffff8107af0a>] warn_slowpath_null+0x1a/0x20
[    1.537238]  [<ffffffff811074fe>] can_stop_full_tick+0x1ee/0x2b0
[    1.537240]  [<ffffffff81107656>] __tick_nohz_full_check+0x76/0xc0
[    1.537241]  [<ffffffff811076ae>] nohz_full_kick_work_func+0xe/0x10
[    1.537243]  [<ffffffff8116ff7c>] irq_work_run_list+0x3c/0x60
[    1.537244]  [<ffffffff8117037e>] irq_work_tick+0x3e/0x90
[    1.537246]  [<ffffffff810f25c4>] run_timer_softirq+0x44/0x3b0
[    1.537248]  [<ffffffff81760855>] ? _raw_spin_unlock+0x35/0x60
[    1.537250]  [<ffffffff810b0b89>] ? vtime_account_system+0x99/0xb0
[    1.537252]  [<ffffffff810b0d9b>] ? vtime_common_account_irq_enter+0x4b/0x80
[    1.537254]  [<ffffffff8107fbd4>] do_current_softirqs.isra.11+0x274/0x550
[    1.537255]  [<ffffffff8107ff86>] run_ksoftirqd+0x26/0x50
[    1.537257]  [<ffffffff810a1a0d>] smpboot_thread_fn+0x18d/0x2d0
(Continue reading)

John Kacur | 24 Feb 15:26 2015
Picon

Re: [PATCH 1/2] Makefile: pi_stress need librttest.a so it should depend on it


On Mon, 23 Feb 2015, Michael Olbrich wrote:

> Signed-off-by: Michael Olbrich <m.olbrich <at> pengutronix.de>
> ---
>  Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index 9e479d3c0dca..da7b419c4bcd 100644
> --- a/Makefile
> +++ b/Makefile
>  <at>  <at>  -73,7 +73,7  <at>  <at>  cyclictest: cyclictest.o librttest.a
>  signaltest: signaltest.o librttest.a
>  	$(CC) $(CFLAGS) $(LDFLAGS) -o $ <at>  $^ $(LIBS)
>  
> -pi_stress: pi_stress.o
> +pi_stress: pi_stress.o librttest.a
>  	$(CC) $(CFLAGS) $(LDFLAGS) -o $ <at>  $^ $(LIBS)
>  
>  hwlatdetect:  src/hwlatdetect/hwlatdetect.py
> -- 
> 2.1.4

Okay,
Signed-off-by: John Kacur <jkacur <at> redhat.com>

will be in the next build 
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
(Continue reading)

Michal Šmucr | 24 Feb 10:46 2015
Picon

Kenrel panic with 3.18.7-rt2 - rootfs at MMC

Hello to all,

I'm getting kernel panics, when I'm using kernel 3.18.7 with recently 
released rt1 and rt2 patches. Platform is x86_64.
It happens only when root filesystem is at SDHC card.
At 3.14.31-rt28 or recent plain vanilla kernels I haven't spotted this 
issue.

Boot panic is possible to workaround by adding kernel parameter: 
"scsi_mod.scan=sync", but it still happens during initrd rebuild using 
update-initramfs at Debian Jessie.

Log from panic is attached.

Thanks for any comment regarding issue or better way for bug-report,

Michal
[    3.184053] task: ffff880036e17170 ti: ffff880036f4c000 task.ti: ffff880036f4c000
[    3.184076] RIP: 0010:[<ffffffff816d2168>]  [<ffffffff816d2168>] rt_spin_lock_slowlock+0x238/0x250
[    3.184082] RSP: 0018:ffff880079403dc8  EFLAGS: 00010046
[    3.184086] RAX: ffff880036e17170 RBX: ffff880075394790 RCX: 0000000000000001
[    3.184090] RDX: 0000000000000000 RSI: ffff880036e17170 RDI: ffff880075394790
[    3.184095] RBP: ffff880079403de0 R08: ffff880036e17170 R09: ffff8800753947a8
[    3.184099] R10: ffff880036e17171 R11: 0000000000000000 R12: ffff880036e17170
[    3.184103] R13: 0000000000000000 R14: ffff88007a9a2e00 R15: ffff880075394600
[    3.184110] FS:  0000000000000000(0000) GS:ffff880079400000(0000) knlGS:0000000000000000
[    3.184115] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[    3.184119] CR2: 00007f2224bd7000 CR3: 0000000036da1000 CR4: 00000000001007f0
(Continue reading)


Gmane