Zygo Blaxell | 3 Mar 16:57 2015

extent-same ioctl hangs holding locks (v3.18.8)

The extent-same ioctl seems to have a locking bug.  My test machines
will run between 0 and 3 days before something gets locked and stays
locked forever.

In the dumps and logs below, 'btrsame' calls the btrfs extent-same ioctl
on its arguments.  As you can see from the trace, it is stuck in this
very ioctl.

'rsync' is famous already and needs no introduction.  It seems
to be stuck just trying to read the file extent-same is reading.

All of these processes are stuck on the same file.  There are lots of
rsyncs because the sender keeps timing out and starting a new rsync,
and the new rsync promptly locks up as soon as it touches the same file
the old ones did.

The rsync processes (and also 'cat' from the command line just now) lock
up when they try to read the same byte range currently being processed by
extent-same.  They are able to read earlier bytes in the file just fine.

Here is /proc/version:

	Linux version 3.18.8-zb64+ (root <at> sneezy) (gcc version 4.9.1 (Debian 4.9.1-19) ) #1 SMP PREEMPT Fri Feb
27 01:09:08 EST 2015

Here are the kernel stacks from currently stuck processes:

	root       583  0.5  0.2 193664 38812 ?        DNs  02:36   2:36 rsync --server -vlHogDtprSze.iLsf --timeout=3600
--delete-during --delete-excluded --partial --numeric-ids --fuzzy --
(Continue reading)

Rich Gannon | 3 Mar 16:15 2015

Re: rsync causes kernel oops


I have applied the patch but once I start making the kernel, the kernel 
oopses and the process freezes.  I have removed all of the btrfs mounts 
from /etc/fstab and tried with no btrfs filesystems mounted.  I still 
get the error upon trying to build the kernel!  I don't understand how 
or why that's happening.  I tried booting up an old, previously 
known-working kernel and the server has not come back up yet.  I don't 
have physical access to it until tomorrow so this may wait until then.


On 03/03/2015 01:44 AM, Liu Bo wrote:
> On Tue, Mar 03, 2015 at 01:28:56AM -0500, Rich Gannon wrote:
>> I should also mention that this is repeatable 100% of the time on this server (only 32-bit box I have) and
once the trace pops up in dmesg, the filesystem will not unmount. It just hangs any process trying to
unmount it. I can not even reboot/shutdown gracefully.
> Could you please try this and post the dmesg log if you can compile your own kernel?
> diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
> index 29850d4..148def3 100644
> --- a/fs/btrfs/extent_io.c
> +++ b/fs/btrfs/extent_io.c
>  <at>  <at>  -2964,6 +2964,9  <at>  <at>  static int __do_readpage(struct extent_io_tree
> *tree,
>   			memset(userpage + pg_offset, 0, iosize);
>   			flush_dcache_page(page);
>   			kunmap_atomic(userpage);
(Continue reading)

Sebastian Thorarensen | 3 Mar 12:47 2015

[PATCH] btrfs-progs: btrfs-convert: Allow setting nodesize

Allow btrfs-convert to use nodesizes other than 4096. It defaults to
max(16384, pagesize), like mkfs. The constant DEFAULT_MKFS_LEAF_SIZE has
been moved to utils.h and renamed to BTRFS_MKFS_DEFAULT_NODE_SIZE for
consistency. convert-tests now test both 4096 and 16384 nodesizes.

Signed-off-by: Sebastian Thorarensen <sebth <at> naju.se>
 Documentation/btrfs-convert.txt |    4 ++
 btrfs-convert.c                 |   80 +++++++++++++++++++++++++++++++++++----
 mkfs.c                          |    5 +--
 tests/convert-tests.sh          |   15 +++++---
 utils.h                         |    1 +
 5 files changed, 89 insertions(+), 16 deletions(-)

diff --git a/Documentation/btrfs-convert.txt b/Documentation/btrfs-convert.txt
index e508c34..5cf9d03 100644
--- a/Documentation/btrfs-convert.txt
+++ b/Documentation/btrfs-convert.txt
 <at>  <at>  -23,6 +23,10  <at>  <at>  Disable data checksum.
 Ignore xattrs and ACLs.
 Disable packing of small files.
+-N <SIZE>::
+Set filesystem nodesize, the tree block size in which btrfs stores data.
+The default value is 16KB (16384) or the page size, whichever is bigger.
+Must be a multiple of the ext2/3/4 block size, but not larger than 65536.
 Roll back to ext2fs.
 -l <LABEL>::
diff --git a/btrfs-convert.c b/btrfs-convert.c
(Continue reading)

Marcel Ritter | 3 Mar 07:02 2015

Regression: kernel 4.0.0-rc1 - soft lockups


yesterday I did a kernel update on my btrfs test system (Ubuntu
14.04.2) from custom-build kernel 3.19-rc6 to 4.0.0-rc1.

Almost instantly after starting my test script, the system got stuck
with soft lockups (the machine was running the very same test for
weeks on the old kernel without problems,
basically doing massive streaming i/o on a raid6 btrfs volume):

I found 2 types of messages in the logs:

one btrfs related:

[34165.540004] INFO: rcu_sched detected stalls on CPUs/tasks: { 3 7}
(detected by 6, t=6990777 jiffies, g=67455, c=67454, q=0)
[34165.540004] Task dump for CPU 3:
[34165.540004] mount           D ffff8803ed266000     0 15156  15110 0x00000000
[34165.540004]  0000000000000158 0000000000000014 ffff8803ecc13718
[34165.540004]  ffffffff8106075a 0000000000000000 0000000000000002
[34165.540004]  00000000ecc13728 ffff8803eb603128 0000000000000000
[34165.540004] Call Trace:
[34165.540004]  [<ffffffff8106075a>] ? __do_page_fault+0x2fa/0x440
[34165.540004]  [<ffffffff810608d1>] ? do_page_fault+0x31/0x70
[34165.540004]  [<ffffffff81792778>] ? page_fault+0x28/0x30
[34165.540004]  [<ffffffff810ae2ce>] ? pick_next_task_fair+0x53e/0x880
[34165.540004]  [<ffffffff810ae2ce>] ? pick_next_task_fair+0x53e/0x880
(Continue reading)

Rich Gannon | 3 Mar 03:43 2015

rsync causes kernel oops

This is on a Dell Poweredge 2650 with dual Xeons.  Running Gentoo x86.  
Kernel 3.18.7 with GRSecurity patches (gentoo's hardened-sources).  
btrfs-progs version 3.18.2

I originally had the drives setup as raid5 with kernel 3.19 and no 
GRSecurity patches (and tried 4.0-rc1)., but kept having these issues so 
I balanced the system to raid10 and this did not help. That's when I 
switched to 3.18.7 kernel as it's been running fine on all of my x86_64 
Gentoo systems in various raid1, 5, and single arrays.

This is the trace I get in dmesg whenever I try to run rsync from the 
local server or remote server that touches any Btrfs filesystem:

[  167.008334]  sdb: unknown partition table
[  176.930717]  sdb: unknown partition table
[  177.319257]  sdb: unknown partition table
[  177.473818] BTRFS: device label secure devid 1 transid 7581 /dev/dm-0
[  189.894615]  sdc: unknown partition table
[  190.266313]  sdc: unknown partition table
[  190.408711] BTRFS: device label secure devid 2 transid 7581 /dev/dm-1
[  203.946583]  sdd: unknown partition table
[  204.312035]  sdd: unknown partition table
[  204.544243] BTRFS: device label secure devid 3 transid 7581 /dev/dm-2
[  217.446332]  sde: unknown partition table
[  227.263168]  sde: unknown partition table
[  227.652513]  sde: unknown partition table
[  227.836720] BTRFS: device label secure devid 4 transid 7581 /dev/dm-3
[  247.868115] BTRFS info (device dm-3): enabling auto defrag
[  247.868121] BTRFS info (device dm-3): disk space caching is enabled
[  247.868123] BTRFS: has skinny extents
(Continue reading)

Josef Bacik | 2 Mar 18:51 2015

[PATCH] Btrfs: remove extra run_delayed_refs in update_cowonly_root

This got added with my dirty_bgs patch, it's not needed.  Thanks,

Signed-off-by: Josef Bacik <jbacik <at> fb.com>
 fs/btrfs/transaction.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index a7a413f..3d017d6 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
 <at>  <at>  -1058,9 +1058,6  <at>  <at>  static int update_cowonly_root(struct btrfs_trans_handle *trans,
 		ret = btrfs_run_delayed_refs(trans, root, (unsigned long)-1);
 		if (ret)
 			return ret;
-		ret = btrfs_run_delayed_refs(trans, root, (unsigned long)-1);
-		if (ret)
-			return ret;

 	return 0;


To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

Josef Bacik | 2 Mar 18:46 2015

[PATCH] Btrfs: fix ASSERT(list_empty(&cur_trans->dirty_bgs_list)

Dave could hit this assert consistently running btrfs/078.  This is because
when we update the block groups we could truncate the free space, which would
try to delete the csums for that range and dirty the csum root.  For this to
happen we have to have already written out the csum root so it's kind of hard to
hit this case.  This patch fixes this by changing the logic to only write the
dirty block groups if the dirty_cowonly_roots list is empty.  This will get us
the same effect as before since we add the extent root last, and will cover the
case that we dirty some other root again but not the extent root.  Thanks,

Reported-by: David Sterba <dsterba <at> suse.cz>
Signed-off-by: Josef Bacik <jbacik <at> fb.com>
 fs/btrfs/transaction.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 038fcf6..a7a413f 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
 <at>  <at>  -1023,16 +1023,22  <at>  <at>  static int update_cowonly_root(struct btrfs_trans_handle *trans,
 	u64 old_root_bytenr;
 	u64 old_root_used;
 	struct btrfs_root *tree_root = root->fs_info->tree_root;
-	bool extent_root = (root->objectid == BTRFS_EXTENT_TREE_OBJECTID);
+	struct btrfs_fs_info *fs_info = root->fs_info;

 	old_root_used = btrfs_root_used(&root->root_item);
 	btrfs_write_dirty_block_groups(trans, root);

 	while (1) {
(Continue reading)

Zhaolei | 2 Mar 13:37 2015

[PATCH] btrfs: Support busy loop of write and delete

From: Zhao Lei <zhaolei <at> cn.fujitsu.com>

 while true; do
   dd if=/dev/zero of=/mnt/btrfs/file count=[75% fs_size]
   rm /mnt/btrfs/file
 Then we can see above loop failed on NO_SPACE.

It it long-term problem since very beginning, because delayed-iput
after rm are not run.

We already have commit_transaction() in alloc_space code, but it is
not triggered in above case.
This patch trigger commit_transaction() to run delayed-iput and
reflash pinned-space to to make write success.

It is based on previous fix of delayed-iput in commit_transaction(),
need to be applied on top of:
btrfs: Fix NO_SPACE bug caused by delayed-iput

Signed-off-by: Zhao Lei <zhaolei <at> cn.fujitsu.com>
 fs/btrfs/extent-tree.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 6c19033..6c1e211 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
(Continue reading)

Robbie Ko | 2 Mar 13:22 2015

Btrfs Send/Receive "no such file or directory" bug


I have been testing btrfs send/receive recently.
I got an error "rename failed: no such file or directory" on receive side.
The followings are simple reproduced steps and related information,
Is there any  idea about what this might be or how to fix it?

uanme -a
Linux ubuntu 4.0.0-rc1-custom #2 SMP Fri Feb 27 22:20:12 CST 2015
x86_64 x86_64 x86_64 GNU/Linux
btrfs --version
Btrfs v3.14.1

 Steps to reproduce:

  $ mkfs.btrfs -f /dev/sdb
  $ mount /dev/sdb /mnt

  $ mkdir -p /mnt/data/n1/n2/p1/p2
  $ mkdir /mnt/data/n4
  $ mkdir -p /mnt/data/p1/p2

  $ btrfs subvolume snapshot -r /mnt /mnt/snap1

  $ mv /mnt/data/p1/p2 /mnt/data
  $ mv /mnt/data/n1/n2/p1/p2 /mnt/data/p1
  $ mv /mnt/data/p2 /mnt/data/n1/n2/p1
  $ mv /mnt/data/n1/n2 /mnt/data/p1
  $ mv /mnt/data/p1 /mnt/data/n4
  $ mv /mnt/data/n4/p1/n2/p1 /mnt/data
(Continue reading)

Qu Wenruo | 2 Mar 04:41 2015

[PATCH v2] btrfs-progs: fsck-test: Add check_sudo to check valid root/sudo privilege

Although fsck-test/012 uses sudo, it uses 'sudo -n', which won't prompt
user to input password and will return 1 if no valid credential is

And this makes test result quite annoying since it fails to mount and
still continue, which will always fail.

This patch will check 'sudo -v -n' and 'sudo -n true' to determine
whether sudo works fine in different version/settings, since in some
setting/version, 'sudo -v -n' will fail even the user is set NOPASSWD.

Also, remove the 'have_root_helper' variant, since there is a
possibility that sudo credential will timeout during the test and
'have_root_helper' won't help to detect such problem.
New '_sudo' command will do credential check if needed to avoid such

Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
  Add support for old sudo.
  Remove 'have_root_helper' variant
  Integrate 'check_root' into root_helper function.

Cc: David Sterba <dsterba <at> suse.cz>
Since I can't reproduce the NOPASSWD setting with 'sudo -v -n' failure,
so only UID==0 and need_validate==1 case, would you please test the
need_validate==0 case? 
 tests/common                                 | 50 ++++++++++++++++++++++++----
(Continue reading)

Chandan Rajendra | 1 Mar 18:20 2015

[PATCH] btrfs/052: Fix test case to work on 64K block size.

The test case passes file offsets which don't align with 64K block size. This
causes btrfs_ioctl_clone() to return with -EINVAL. Fix this by using offsets
which are multiples of 64k.

Signed-off-by: Chandan Rajendra <chandan <at> linux.vnet.ibm.com>
 tests/btrfs/052     |  71 ++++----
 tests/btrfs/052.out | 468 ++++++++++++++++++++++++++--------------------------
 2 files changed, 272 insertions(+), 267 deletions(-)

diff --git a/tests/btrfs/052 b/tests/btrfs/052
index c75193d..462f9ce 100755
--- a/tests/btrfs/052
+++ b/tests/btrfs/052
 <at>  <at>  -59,70 +59,75  <at>  <at>  test_btrfs_clone_same_file()
 	_scratch_mkfs >/dev/null 2>&1
 	_scratch_mount $MOUNT_OPTIONS

-	# Create a file with 5 extents, 4 of 8Kb each and 1 of 64Kb.
-	$XFS_IO_PROG -f -c "pwrite -S 0x01 -b 8192 0 8192" $SCRATCH_MNT/foo \
-		| _filter_xfs_io
+	# Create a file with 5 extents, 4 of 128Kb each and 1 of 1024Kb.
+	$XFS_IO_PROG -f -c "pwrite -S 0x01 -b 131072 0 131072" \
+		     $SCRATCH_MNT/foo | _filter_xfs_io
-	$XFS_IO_PROG -c "pwrite -S 0x02 -b 8192 8192 8192" $SCRATCH_MNT/foo \
-		| _filter_xfs_io
+	$XFS_IO_PROG -c "pwrite -S 0x02 -b 131072 131072 131072" \
+		     $SCRATCH_MNT/foo | _filter_xfs_io
(Continue reading)