Josef Bacik | 19 Sep 16:40 2014

[PATCH] Btrfs: cleanup error handling in build_backref_tree

When balance panics it tends to panic in the

BUG_ON(!upper->checked);

test, because it means it couldn't build the backref tree properly.  This is
annoying to users and frankly a recoverable error, nothing in this function is
actually fatal since it is just an in-memory building of the backrefs for a
given bytenr.  So go through and change all the BUG_ON()'s to ASSERT()'s, and
fix the BUG_ON(!upper->checked) thing to just return an error.

This patch also fixes the error handling so it tears down the work we've done
properly.  This code was horribly broken since we always just panic'ed instead
of actually erroring out, so it needed to be completely re-worked.  With this
patch my broken image no longer panics when I mount it.  Thanks,

Signed-off-by: Josef Bacik <jbacik <at> fb.com>
---
 fs/btrfs/relocation.c | 88 ++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 59 insertions(+), 29 deletions(-)

diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 2d221c4..19726af 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
 <at>  <at>  -736,7 +736,8  <at>  <at>  again:
 		err = ret;
 		goto out;
 	}
-	BUG_ON(!ret || !path1->slots[0]);
+	ASSERT(ret);
(Continue reading)

Rob Spanton | 19 Sep 14:18 2014

Performance Issues

Hi,

I have a particularly uncomplicated setup (a desktop PC with a hard
disk) and I'm seeing particularly slow performance from btrfs.  A `git
status` in the linux source tree takes about 46 seconds after dropping
caches, whereas on other machines using ext4 this takes about 13s.  My
mail client (evolution) also seems to perform particularly poorly on
this setup, and my hunch is that it's spending a lot of time waiting on
the filesystem.

I've tried mounting with noatime, and this has had no effect.  Anyone
got any ideas?  

Here are the things that the wiki page asked for [1]:

uname -a:

        Linux zarniwoop.blob 3.16.2-200.fc20.x86_64 #1 SMP Mon Sep 8
        11:54:45 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

btrfs --version:

        Btrfs v3.16

btrfs fi show:

        Label: 'fedora'  uuid: 717c0a1b-815c-4e6a-86c0-60b921e84d75
        	Total devices 1 FS bytes used 1.49TiB
        	devid    1 size 2.72TiB used 1.50TiB path /dev/sda4

(Continue reading)

Jakob Breier | 19 Sep 10:58 2014
Picon
Picon

Help for creating a useful bugreport

Hi,

my btrfs partition got corrupted. With some trouble I got most of the 
valuable data out of it using `btrfs restore -i` (it crashed a few 
times, but on the fourth or fifth run it reached the stuff I wanted to 
recover). As far as I can tell, the file system broke during normal 
operations without any hardware failures. Before I switch back to ext4, 
I'd like to file a bug report so my troubles were not completely in 
vain. Unfortunately I don't have much to work with. Can you help me with 
extracting enough information to create a useful bugreport?

Regards,
Jakob

$ cat /etc/fedora-release
Fedora release 20 (Heisenbug)

$ uname -a
Linux localhost.localdomain 3.16.2-200.fc20.x86_64 #1 SMP Mon Sep 8 
11:54:45 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ sudo btrfs fi show /dev/dm-1
Label: 'EncJakobExtern'  uuid: 8ccbd085-564d-4022-bfc9-18fd429d0a8d
         Total devices 1 FS bytes used 731.07GiB
         devid    1 size 931.60GiB used 769.04GiB path 
/dev/mapper/luks-a266c492-2360-404d-9ad7-00edc2f0c09d

Btrfs v3.16

$ sudo mount /dev/dm-1 /mnt/diverse/
(Continue reading)

Satoru Takeuchi | 19 Sep 10:45 2014

[PATCH 1/4] btrfs: correct empty compression property behavior

From: Naohiro Aota <naota <at> elisp.net>

In the current implementation, compression property == "" has
the two different meanings: one is with BTRFS_INODE_NOCOMPRESS,
and the other is without this flag.

So, even if the two files a and b have the same compression
property, "", and the same contents, one file seems to be
compressed and the other is not. It's difficult to understand
for users and also confuses them.

Here is the real example. Let assume the following two cases.

  a) A file created freshly (under a directory without both
     COMPRESS and NOCOMPRESS flag.)

  b) A existing file which is explicitly set ""
     to compression property.

In addition, here is the command log (I attached the source of
"getflags" program in this patch.)

=======
$ rm -f a b; touch a b
$ btrfs prop set b compression ""
 # both a and b have the same compression property: ""
$ btrfs prop get a compression
$ btrfs prop get b compression
 # but ... let's take a look at inode flags
$ ./getflags a
(Continue reading)

Satoru Takeuchi | 19 Sep 03:49 2014

[PATCH] btrfs-progs: fix many typos in documents

From: Naohiro Aota <naota <at> elisp.net>

There are many trivial typos in Documentation/*.txt.
All of these use "exist status" to mean "exit status"
by mistake. I guess someone first made this mistake
and it has spread by copy-and-paste :-D

Signed-off-by: Naohiro Aota <naota <at> elisp.net>
Signed-off-by: Satoru Takeuchi <takeuchi_satoru <at> jp.fujitsu.com>
---
 Documentation/btrfs-balance.txt          | 2 +-
 Documentation/btrfs-check.txt            | 2 +-
 Documentation/btrfs-device.txt           | 2 +-
 Documentation/btrfs-filesystem.txt       | 2 +-
 Documentation/btrfs-inspect-internal.txt | 2 +-
 Documentation/btrfs-property.txt         | 2 +-
 Documentation/btrfs-qgroup.txt           | 2 +-
 Documentation/btrfs-quota.txt            | 2 +-
 Documentation/btrfs-receive.txt          | 2 +-
 Documentation/btrfs-replace.txt          | 2 +-
 Documentation/btrfs-rescue.txt           | 2 +-
 Documentation/btrfs-restore.txt          | 2 +-
 Documentation/btrfs-scrub.txt            | 2 +-
 Documentation/btrfs-send.txt             | 2 +-
 Documentation/btrfs-subvolume.txt        | 2 +-
 Documentation/btrfs.txt                  | 2 +-
 16 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/Documentation/btrfs-balance.txt b/Documentation/btrfs-balance.txt
index de93c92..89fd449 100644
(Continue reading)

nick | 18 Sep 22:14 2014
Picon

XFS Tests for Btrfs

Hey Fellow Btrfs Developers,
I am wondering how to run the xfs tests for btrfs as I tried to do it based on a link online
written I believe a few years ago. If someone can help me get set up for testing the btrfs
code using xfs tests that would be great. In addition afterwards I already build kernel 3.17
r5 release candidate and would be glad to run any times you need run to test single drive
config issues.
Thanks Nick     
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Josef Bacik | 18 Sep 17:30 2014

[PATCH] Btrfs: try not to ENOSPC on log replay V2

When doing log replay we may have to update inodes, which traditionally goes
through our delayed inode stuff.  This will try to move space over from the
trans handle, but we don't reserve space in our trans handle on replay since we
don't know how much we will need, so instead we try to flush.  But because we
have a trans handle open we won't flush anything, so if we are out of reserve
space we will simply return ENOSPC.  Since we know that if an operation made it
into the log then we definitely had space before the box bought the farm then we
don't need to worry about doing this space reservation.  Use the
fs_info->log_root_recovering flag to skip the delayed inode stuff and update the
item directly.  Thanks,

Signed-off-by: Josef Bacik <jbacik <at> fb.com>
---
V1->V2: Use fs_info->log_root_recovering instead of adding yet another flag.

 fs/btrfs/inode.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 28e693e..acecf86 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
 <at>  <at>  -3705,7 +3705,8  <at>  <at>  noinline int btrfs_update_inode(struct btrfs_trans_handle *trans,
 	 * without delay
 	 */
 	if (!btrfs_is_free_space_inode(inode)
-	    && root->root_key.objectid != BTRFS_DATA_RELOC_TREE_OBJECTID) {
+	    && root->root_key.objectid != BTRFS_DATA_RELOC_TREE_OBJECTID
+	    && !root->fs_info->log_root_recovering) {
 		btrfs_update_root_times(trans, root);
(Continue reading)

Marc MERLIN | 18 Sep 17:27 2014

btrfs receive: could not find parent subvolume

While debugging a btrfs send/receive slow problem, I now getting this:
legolas:/mnt/btrfs_pool1# btrfs send -p tmp_ggm_daily_ro.20140917_06:29:58
tmp_ggm_daily_ro.20140918_02:48:24 | ssh gargamel btrfs receive -v /mnt/btrfs_pool2/backup/debian64/legolas
At subvol tmp_ggm_daily_ro.20140918_02:48:24
At snapshot tmp_ggm_daily_ro.20140918_02:48:24
receiving snapshot tmp_ggm_daily_ro.20140918_02:48:24
uuid=5d1f0454-1be3-b648-9ea5-dc427cd62d98, ctransid=310713
parent_uuid=d86e69bf-e17f-7f4c-bfb7-e571d5824687, parent_ctransid=308332
ERROR: could not find parent subvolume

The parent is there on the other side, but UUID is different:
gargamel:/mnt/btrfs_pool2/backup/debian64/legolas# btrfs subvolume show tmp_ggm_daily_ro.20140917_06:29:58
/mnt/btrfs_pool2/backup/debian64/legolas/tmp_ggm_daily_ro.20140917_06:29:58
        Name:                   tmp_ggm_daily_ro.20140917_06:29:58
        uuid:                   3d424a2b-69da-244c-bfcc-c283f9cc1f34
        Parent uuid:            05d3b9be-bfe2-bb4a-9f6a-64b9d44896c7
        Creation time:          2014-09-17 06:30:01
        Object ID:              7873
        Generation (Gen):       83621
        Gen at creation:        83476
        Parent:                 263
        Top Level:              263
        Flags:                  -
        Snapshot(s):

Now it seems that the UUID is different on all my snapshots created by
btrfs send, so maybe it doesn't match UUID?

Given that, what is btrfs receive using to get a match?

(Continue reading)

Josef Bacik | 18 Sep 17:27 2014

[PATCH] Btrfs: don't do async reclaim during log replay V2

Trying to reproduce a log enospc bug I hit a panic in the async reclaim code
during log replay.  This is because we use fs_info->fs_root as our root for
shrinking and such.  Technically we can use whatever root we want, but let's
just not allow async reclaim while we're doing log replay.  Thanks,

Signed-off-by: Josef Bacik <jbacik <at> fb.com>
---
V1->V2: use fs_info->log_root_recovering instead, didn't notice this existed
before.

 fs/btrfs/extent-tree.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 28a27d5..44d0497 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
 <at>  <at>  -4513,7 +4513,13  <at>  <at>  again:
 		space_info->flush = 1;
 	} else if (!ret && space_info->flags & BTRFS_BLOCK_GROUP_METADATA) {
 		used += orig_bytes;
-		if (need_do_async_reclaim(space_info, root->fs_info, used) &&
+		/*
+		 * We will do the space reservation dance during log replay,
+		 * which means we won't have fs_info->fs_root set, so don't do
+		 * the async reclaim as we will panic.
+		 */
+		if (!root->fs_info->log_root_recovering &&
+		    need_do_async_reclaim(space_info, root->fs_info, used) &&
 		    !work_busy(&root->fs_info->async_reclaim_work))
(Continue reading)

Josef Bacik | 18 Sep 17:20 2014

[PATCH] Btrfs: remove empty block groups automatically V3

One problem that has plagued us is that a user will use up all of his space with
data, remove a bunch of that data, and then try to create a bunch of small files
and run out of space.  This happens because all the chunks were allocated for
data since the metadata requirements were so low.  But now there's a bunch of
empty data block groups and not enough metadata space to do anything.  This
patch solves this problem by automatically deleting empty block groups.  If we
notice the used count go down to 0 when deleting or on mount notice that a block
group has a used count of 0 then we will queue it to be deleted.

When the cleaner thread runs we will double check to make sure the block group
is still empty and then we will delete it.  This patch has the side effect of no
longer having a bunch of BUG_ON()'s in the chunk delete code, which will be
helpful for both this and relocate.  Thanks,

Signed-off-by: Josef Bacik <jbacik <at> fb.com>
---
V2->V3: Fixed a problem where orphan replay would get angry because bg deletion
would add more orphans.  Fixed it so we don't delete bg's until after we've done
all the open_ctree work.

 fs/btrfs/ctree.h                  |   9 ++-
 fs/btrfs/disk-io.c                |   6 ++
 fs/btrfs/extent-tree.c            | 141 ++++++++++++++++++++++++++++++++++++--
 fs/btrfs/tests/free-space-tests.c |   2 +-
 fs/btrfs/volumes.c                | 115 ++++++++++++++++++++-----------
 fs/btrfs/volumes.h                |   2 +
 6 files changed, 226 insertions(+), 49 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 6db3d4b..d93d598 100644
(Continue reading)

Chris Mason | 18 Sep 17:00 2014

[PATCH] Revert "Btrfs: device_list_add() should not update list when


Johannes and Sam, could you please confirm this patch fixes your mount
regression for now?  Anand, please make sure I kept the generation check
properly.

This reverts commit b96de000bc8bc9688b3a2abea4332bd57648a49f.

This commit is triggering failures to mount by subvolume id in some
configurations.  The main problem is how many different ways this
scanning function is used, both for scanning while mounted and
unmounted.  A proper cleanup is too big for late rcs.

For now, just revert the commit and we'll put a better fix into a later
merge window.

Signed-off-by: Chris Mason <clm <at> fb.com>
---
 fs/btrfs/volumes.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 340a92d..2c2d6d1 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
 <at>  <at>  -529,12 +529,12  <at>  <at>  static noinline int device_list_add(const char *path,
 		 */

 		/*
-		 * As of now don't allow update to btrfs_fs_device through
-		 * the btrfs dev scan cli, after FS has been mounted.
(Continue reading)


Gmane