David Sterba | 23 Jul 22:56 2014

[PATCH] btrfs-progs: wait until all subvolumes are cleaned

Enhance the 'subvolume' subcommand to wait until a given list of
subvolumes or all currently scheduled for deletion are cleaned
completely from the filesystem.

Signed-off-by: David Sterba <dsterba <at> suse.cz>

'wait' seemed too generic, 'sync' is not completely accurate but IMHO better,
I'm open to other suggestions

 cmds-subvolume.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 231 insertions(+)

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 5e821c712e74..07485acbc1bd 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
 <at>  <at>  -1035,6 +1035,236  <at>  <at>  out:
 	return !!ret;

+static const char * const cmd_subvol_sync_usage[] = {
+	"btrfs subvolume sync <path> [<subvol-id>...]",
+	"Wait until given subvolume(s) are completely cleaned",
+	"Wait until given subvolume(s) are completely cleaned after deletion.",
+	"If no subvolume id is given, wait until there are no more snapshots",
+	"to be cleaned. This may take long if new deleted subvolumes appear",
+	"during the sleep interval.",
+	"",
+	"-s <N>       sleep N seconds between checks (default: 1)",
(Continue reading)

Chris Murphy | 23 Jul 22:10 2014

feature request: consider rw subvols ro for send when volume is mounted ro

The use case is when it's possible to mount a Btrfs volume ro, but not rw. Example, a situation where

# mount -o degraded /dev/sdb /mnt
[   71.064352] BTRFS info (device sdb): allowing degraded mounts
[   71.064812] BTRFS info (device sdb): enabling auto recovery
[   71.065210] BTRFS info (device sdb): disk space caching is enabled
[   71.072068] BTRFS warning (device sdb): devid 2 missing
[   71.097320] BTRFS: too many missing devices, writeable mount is not allowed
[   71.116616] BTRFS: open_ctree failed

Yet this works:
# mount -o degraded,ro /dev/sdb /mnt

It would be great if it were possible to send/receive subvolumes to a different btrfs volume. Currently
it's not possible because those subvols aren't ro, and because the mount is ro I can't make ro snapshots first.

Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

David Sterba | 23 Jul 15:16 2014

[PATCH] btrfs-progs: fix build of static target

A user repoted that static buid fails with

utils-lib.static.o: In function `arg_strtou64':
/home/dsterba/labs/btrfs-progs/utils-lib.c:17: multiple definition of `arg_strtou64'
utils-lib.static.o:/home/dsterba/labs/btrfs-progs/utils-lib.c:17: first defined here

utils-lib.o was mistakenly added to linker twice.

Signed-off-by: David Sterba <dsterba <at> suse.cz>
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 8e14d4837520..3853b8734568 100644
--- a/Makefile
+++ b/Makefile
 <at>  <at>  -10,7 +10,7  <at>  <at>  objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
 	  root-tree.o dir-item.o file-item.o inode-item.o inode-map.o \
 	  extent-cache.o extent_io.o volumes.o utils.o repair.o \
 	  qgroup.o raid6.o free-space-cache.o list_sort.o props.o \
-	  utils-lib.o ulist.o qgroup-verify.o
+	  ulist.o qgroup-verify.o
 cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
 	       cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
 	       cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \


(Continue reading)

David Sterba | 23 Jul 14:39 2014

[PATCH] btrfs: wake up transaction thread from SYNC_FS ioctl

The transaction thread may want to do more work, namely it pokes the
cleaner ktread that will start processing uncleaned subvols.

This can be triggered by user via the 'btrfs fi sync' command, otherwise
there was a delay up to 30 seconds before the cleaner started to clean
old snapshots.

Signed-off-by: David Sterba <dsterba <at> suse.cz>
 fs/btrfs/ioctl.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 47aceb494d1d..33ba5d0445b5 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
 <at>  <at>  -5309,6 +5309,12  <at>  <at>  long btrfs_ioctl(struct file *file, unsigned int
 		if (ret)
 			return ret;
 		ret = btrfs_sync_fs(file->f_dentry->d_sb, 1);
+		/*
+		 * The transaction thread may want to do more work,
+		 * namely it pokes the cleaner ktread that will start
+		 * processing uncleaned subvols.
+		 */
+		wake_up_process(root->fs_info->transaction_kthread);
 		return ret;

(Continue reading)

Qu Wenruo | 23 Jul 07:47 2014

[PATCH 1/4] btrfs-progs: Remove fprintf() in find_mount_root().

find_mount_root() function in utils.c should not print error string.
Caller should be responsible to print error string.

This patch will remove the only fprintf in find_mount_root() and modify
the caller a little to use strerror() to prompt users.

Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
 cmds-receive.c | 4 ++--
 cmds-send.c    | 4 ++--
 utils.c        | 6 +-----
 3 files changed, 5 insertions(+), 9 deletions(-)

diff --git a/cmds-receive.c b/cmds-receive.c
index 48380a5..72afe2a 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
 <at>  <at>  -980,9 +980,9  <at>  <at>  static int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd,

 	ret = find_mount_root(dest_dir_full_path, &r->root_path);
 	if (ret < 0) {
-		ret = -EINVAL;
 		fprintf(stderr, "ERROR: failed to determine mount point "
-			"for %s\n", dest_dir_full_path);
+			"for %s: %s\n", dest_dir_full_path, strerror(-ret));
+		ret = -EINVAL;
 		goto out;
 	r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
diff --git a/cmds-send.c b/cmds-send.c
(Continue reading)

Chris Murphy | 23 Jul 03:34 2014

BUGS: bogus out of space reported when mounted raid1 degraded, btrfs replace failure, then oops

btfs-progs 3.14.2

Fortunately this is a test system so it is dispensable. But in just an hour I ran into 5 bugs, and managed to
apparently completely destroy a btrfs file system beyond repair, and it wasn't intentional. 

1. mkfs.btrfs /dev/sda6  ## volume's life starts as single device, on an SSD
2. btrfs device add /dev/sdb1 /  ## added an HDD partition
3. btrfs balance start -dconvert=raid1 -mconvert=raid1
4. clean shutdown, remove device 1 (leaving device 0)
5. poweron, mount degraded
6. gdm/gnome comes up very slowly, then I see a sad face graphic, with a message that there's only 60MB of
space left.

# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda6        26G   13G   20M 100% /
/dev/sda6        26G   13G   20M 100% /home
/dev/sda6        26G   13G   20M 100% /var
/dev/sda6        26G   13G   20M 100% /boot

# btrfs fi df
Data, RAID1: total=6.00GiB, used=5.99GiB
System, RAID1: total=32.00MiB, used=32.00KiB
Metadata, RAID1: total=768.00MiB, used=412.41MiB
unknown, single: total=160.00MiB, used=0.00

# btrfs fi show
Label: 'Rawhide2'  uuid: f857c336-b8f5-4f5d-9500-a705ee1b6977
	Total devices 2 FS bytes used 6.39GiB
(Continue reading)

ronnie sahlberg | 21 Jul 22:34 2014

Testing with flaky disk

List, btrfs developers.

I started working on a test tool for SCSI initiators and filesystem folks.
It is a iSCSI target that implements a bad flaky disks where you
can set precise controls of how/what is broken which you can use to test
error and recovery paths in the initiator/filesystem.

The tool is available at :
and is a modified version of the TGTD iscsi target.

Right now it is just an initial prototype and it needs more work to
add more types of errors as well as making it more userfriendly.
But it is still useful enough to illustrate certain failure cases
which could be helpful to btrfs and others.

Let me illustrate. Lets start by creating a BTRFS filesystem spanning
three 1G disks:
# Create three disks and export them through flaky iSCSI
truncate -s 1G /data/tmp/disk1.img
truncate -s 1G /data/tmp/disk2.img
truncate -s 1G /data/tmp/disk3.img

killall -9 tgtd
./usr/tgtd -f -d 1 &

sleep 3
(Continue reading)

Timofey Titovets | 21 Jul 16:00 2014


I working on readahead in systemd and try to complete todo for it.
One of todos it is:
 readahead: use BTRFS_IOC_DEFRAG_RANGE instead of BTRFS_IOC_DEFRAG
ioctl, with START_IO

Can someone explain what start_io flag in BTRFS_IOC_DEFRAG_RANGE do?
Just force write data after defragment or do something else?
This flag mean what btrfs can guarantee data consistency after defragment?

Thanks for any explanation!


Best regards,
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Qu Wenruo | 21 Jul 11:02 2014

[PATCH] btrfs: Add show_path function for btrfs_super_ops.

show_path() function in struct super_operations is used to output
subtree mount info for mountinfo.
Without the implement of show_path() function, user can not found where
each subvolume is mounted if using 'subvolid=' mount option.
(When mounted with 'subvol=' mount option, vfs is aware of subtree mount
and can to the path resolve by vfs itself)

With this patch, end users will be able to use findmnt(8) or other
programs reading mountinfo to find which btrfs subvolume is mounted.

Though we use fs_info->subvol_sem to protect show_path() from subvolume
destroying/creating, if user renames/moves the parent non-subvolume
dir of a subvolume, it is still possible that concurrency may happen and
cause btrfs_search_slot() fails to find the desired key.
In that case, we just return -EBUSY and info user to try again since
extra locking like locking the whole subvolume tree is too expensive for
such usage.

Reported-by: Stefan G.Weichinger <lists <at> xunil.at>
Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
 fs/btrfs/ctree.h |   2 +
 fs/btrfs/ioctl.c |   4 +-
 fs/btrfs/super.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 116 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index be91397..63fba05 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
(Continue reading)

Anand Jain | 21 Jul 11:03 2014

[PATCH] btrfs-progs: check if there is required kernel send stream version

When kernel does not have the send stream version 2 patches,
the btrfs send with --stream-version 2 would fail with out
giving the details what is wrong. This patch will help to
identify correctly that required kernel patches are missing.

Signed-off-by: Anand Jain <anand.jain <at> oracle.com>
 cmds-send.c | 13 +++++++++++++
 send.h      |  2 ++
 utils.c     | 17 +++++++++++++++++
 utils.h     |  1 +
 4 files changed, 33 insertions(+)

diff --git a/cmds-send.c b/cmds-send.c
index 9a73b32..0c20a6f 100644
--- a/cmds-send.c
+++ b/cmds-send.c
 <at>  <at>  -435,6 +435,7  <at>  <at>  int cmd_send(int argc, char **argv)
 	u64 parent_root_id = 0;
 	int full_send = 1;
 	int new_end_cmd_semantic = 0;
+	int k_sstream;

 	memset(&send, 0, sizeof(send));
 	send.dump_fd = fileno(stdout);
 <at>  <at>  -544,6 +545,18  <at>  <at>  int cmd_send(int argc, char **argv)
 				ret = 1;
 				goto out;
(Continue reading)

Anand Jain | 21 Jul 11:01 2014

[PATCH] xfstest/btrfs: check for matching kernel send stream ver 2

The test case btrfs/049 is relevant to send stream version 2, and
needs kernel patches as well. So call _notrun if there isn't
matching kernel support as shown below

btrfs/047	 [not run] Missing btrfs kernel patch for send stream version 2, skipped this test
Not run: btrfs/047

Signed-off-by: Anand Jain <anand.jain <at> oracle.com>
 common/rc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/common/rc b/common/rc
index 4a6511f..1c914bb 100644
--- a/common/rc
+++ b/common/rc
 <at>  <at>  -2223,6 +2223,11  <at>  <at>  _require_btrfs_send_stream_version()
 	if [ $? -ne 0 ]; then
 		_notrun "Missing btrfs-progs send --stream-version command line option, skipped this test"
+	# test if btrfs kernel supports send stream version 2
+	if [ ! -f /sys/fs/btrfs/send/stream_version ]; then
+		_notrun "Missing btrfs kernel patch for send stream version 2, skipped this test"
+	fi


(Continue reading)