luke | 1 Jun 05:25 2016

Re: [PATCH] btrfs-progs: make btrfs-image restore to support dup


At 06/01/2016 12:01 AM, David Sterba wrote:
> On Tue, May 31, 2016 at 03:14:25PM +0800, luke wrote:
>>>>  <at>  <at>  -68,6 +68,12  <at>  <at>  struct meta_cluster {
>>>>    struct fs_chunk {
>>>>    	u64 logical;
>>>>    	u64 physical;
>>>> +	/* physical_dup only store additonal physical for BTRFS_BLOCK_GROUP_DUP
>>>> +	 * currently restore only support single and dup
>>>> +	 * TODO: modify this structure and the function related to this
>>>> +	 * structure for support raid*
>>> What does it do in case of RAID? Can we do runtime checks and report
>>> potential problems? btrfs-image on multiple device was always somehow
>>> tricky so I'll merge the patch.
>> For multiple device,  if the number of  target devices equal to the
>> number of source devices, obviously we can offer the enough disk size
>> and keep the original offset. However, if restore the multi-device
>> metadata to one disk, we should remap all chunk and use other profile to
>> instead of raid*(for example, raid0 -> single, raid1 -> dup). Currently
>> I have only a rough idea, it may need to refactor a lot of function.
>> We can report some warnings when we find raid* profile in the restore
>> process. Should I add these warnings?
> Yes please, that's what I was asking for. If you're going to do
> intrusive refactorin, please add tests.
>
>
OK. When I started to refactor, I'll add these.

Thanks,
Lu
(Continue reading)

Lu Fengqi | 1 Jun 04:15 2016

[PATCH] btrfs-progs: fix uninit variables in btrfs-image

Signed-off-by: Lu Fengqi <lufq.fnst <at> cn.fujitsu.com>
---
 btrfs-image.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/btrfs-image.c b/btrfs-image.c
index d121951..7c2fa94 100644
--- a/btrfs-image.c
+++ b/btrfs-image.c
 <at>  <at>  -1757,6 +1757,7  <at>  <at>  static void *restore_worker(void *data)
 		if (!mdres->fixup_offset) {
 			while (size) {
 				u64 chunk_size = size;
+				physical_dup = 0;
 				if (!mdres->multi_devices && !mdres->old_restore)
 					bytenr = logical_to_physical(mdres,
 						     async->start + offset,
--

-- 
2.5.5

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Qu Wenruo | 1 Jun 03:51 2016

[PATCH] btrfs-progs: convert-tests: Add test for backup superblock migration

New convert framework uses new and simpler chunk layout, while the cost
is the more complex superblock range migration logical, compared to old
convert.

Enhance the convert test script to create file which will takes up 2nd
backup superblock space, to ensure the superblock migration is working
as expected.

Suggested-by: David Sterba <dsterba <at> suse.cz>
Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
---
 tests/convert-tests.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tests/convert-tests.sh b/tests/convert-tests.sh
index 06d8419..d4e13d6 100755
--- a/tests/convert-tests.sh
+++ b/tests/convert-tests.sh
 <at>  <at>  -124,6 +124,12  <at>  <at>  convert_test() {
 	# create a file to check btrfs-convert can convert regular file
 	# correct
 	run_check_mount_test_dev
+
+	# create a file inside the fs before convert, to make sure there is
+	# data covering btrfs backup superblock range (64M)
+	run_check $SUDO_HELPER dd if=/dev/zero bs=1M count=64 \
+		of=$TEST_MNT/convert_space_holder
+
 	populate_fs
 	run_check $SUDO_HELPER dd if=/dev/zero of=$TEST_MNT/test bs=$nodesize \
(Continue reading)

luke | 1 Jun 03:23 2016

Re: [PATCH] btrfs: fix check_shared for fiemap ioctl


At 06/01/2016 12:15 AM, David Sterba wrote:
> On Tue, May 31, 2016 at 11:08:39AM +0800, luke wrote:
>>>> +};
>>>> +
>>>> +/* dynamically allocate and initialize a ref_root */
>>>> +static struct ref_root *ref_root_alloc(gfp_t gfp_mask)
>>>> +{
>>>> +	struct ref_root *ref_tree;
>>>> +
>>>> +	ref_tree = kmalloc(sizeof(*ref_tree), gfp_mask);
>>> Drop the gfp_mask and make it GFP_KERNEL
>> OK, I'll drop the gfp_mask, but can you tell me why should use
>> GFP_KERNEL instead of GFP_NOFS?
> Because there's no need to narrow the allocation constraints. GFP_NOFS
> is necessary when the caller is on a critical path that must not recurse
> back to the filesystem through the allocation (ie. if the allocator
> decides to free some memory and tries tro write dirty data). FIEMAP is
> called from an ioctl.
OK, I'll update this patch with GFP_KERNEL.
>
>>>>    			disko = em->block_start + offset_in_extent;
>>>>    
>>>>    			/*
>>>> +			 * We need a trans handle to get delayed refs
>>>> +			 */
>>>> +			trans = btrfs_join_transaction(root);
>>> What are the implications of join/end transaction here? It's just
>>> fiemap, I would not expect messing with transaction here.
>> This transaction is used to handle delayed_refs, if trans = NULL, we
(Continue reading)

David Sterba | 1 Jun 01:57 2016

[PATCH] btrfs: advertise which crc32c implementation is being used at module load

From: Jeff Mahoney <jeffm <at> suse.com>

Since several architectures support hardware-accelerated crc32c
calculation, it would be nice to confirm that btrfs is actually using it.

We can see an elevated use count for the module, but it doesn't actually
show who the users are.  This patch simply prints the name of the driver
after successfully initializing the shash.

Signed-off-by: Jeff Mahoney <jeffm <at> suse.com>
[ added a helper and used in module load-time message ]
Signed-off-by: David Sterba <dsterba <at> suse.com>
---

Slightly modified version of Jeff's patch.

 fs/btrfs/hash.c  | 5 +++++
 fs/btrfs/hash.h  | 1 +
 fs/btrfs/super.c | 5 +++--
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/hash.c b/fs/btrfs/hash.c
index aae520b2aee5..a97fdc156a03 100644
--- a/fs/btrfs/hash.c
+++ b/fs/btrfs/hash.c
 <at>  <at>  -24,6 +24,11  <at>  <at>  int __init btrfs_hash_init(void)
 	return PTR_ERR_OR_ZERO(tfm);
 }

+const char* btrfs_crc32c_impl(void)
(Continue reading)

al | 31 May 23:53 2016

Very quick embarrassing q: /var/lib permissions.

Please can someone run:

# ls -l /var/lib/ | grep btrfs

for me (and post to directly or via list as they think fit).

Thank you.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Jeff Mahoney | 31 May 22:41 2016

[RFC] strace patches for new ioctls

Hi all -

Strace 4.12 was tagged for release today and it supports decoding of
btrfs ioctls.  I'd like to propose a requirement that future ioctl
additions come with a patch to strace as well so we don't get out of sync.

I wrote the decoding to help with some issues some of our developers
were seeing while writing various tools.  Rather than describing their
workload or showing us the code, an strace in verbose mode shows us
pretty much exactly what they're doing.

Opinions?

-Jeff

--

-- 
Jeff Mahoney
SUSE Labs

Qu Wenruo | 31 May 10:49 2016

[PATCH] btrfs-progs: convert: Fix bugs in backup superblock migration

New convert has several bugs with backup superblock migration

1) Backup superblocks are not migrated due to bad judgement
   Two wrong judgement cause backup superblocks are not migrated at all

2) Converted ext* image doesn't keep hole for backup superblocks
   Since we are creating file extents according to tmp_used, which has
   wiped out backup superblock ranges.
   In that case, later superblock migration will fail, since migration
   will insert file extent range into ext* image.

Fix above bugs will make convert on ext2 image filled about 100M data
successful.

Reported-by: Satoru Takeuchi <takeuchi_satoru <at> jp.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
---
 btrfs-convert.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/btrfs-convert.c b/btrfs-convert.c
index 30dd6e8..ce4ca1d 100644
--- a/btrfs-convert.c
+++ b/btrfs-convert.c
 <at>  <at>  -1263,12 +1263,32  <at>  <at>  static int create_image_file_range(struct btrfs_trans_handle *trans,
 	struct btrfs_block_group_cache *bg_cache;
 	u64 len = *ret_len;
 	u64 disk_bytenr;
+	int i;
 	int ret;
(Continue reading)

Qu Wenruo | 31 May 09:51 2016

[PATCH] btrfs-progs: convert: Fix a bug that makes convert asserts at scan time

When a ext2 fs filled with a 57M file, it's possible that convert fails
with assert in add_merge_cache_extent().

The problem is that the ext2 used space just takes some of the second
superblock.
And due to a bug in reserving superblock space, it corrupted used space
tree and cause assert.

Fix in by doing better used space merging for case where superblock
range is inside the ext2 used space.

Reported-by: Satoru Takeuchi <takeuchi_satoru <at> jp.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
---
 btrfs-convert.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 74 insertions(+), 7 deletions(-)

diff --git a/btrfs-convert.c b/btrfs-convert.c
index c28dae6..30dd6e8 100644
--- a/btrfs-convert.c
+++ b/btrfs-convert.c
 <at>  <at>  -1928,6 +1928,57  <at>  <at>  static int convert_open_fs(const char *devname,
 }

 /*
+ * Helper for expend and merge extent_cache for wipe_one_reserved_range()
+ * to handle wipe range is inside an existing cache case.
+ */
+static int _expand_extent_cache(struct cache_tree *tree,
+				struct cache_extent *entry,
(Continue reading)

Qu Wenruo | 31 May 04:46 2016

[PATCH] btrfs-progs: convert: Add support to rollback new convert image

For new btrfs-convert, it's less restrict for metadata chunk allocation.
While the may_rollback() function is still following the restrict 1:1
mapping check for all chunks, it will not allow some new convert image
to be rolled back.

Add new per extent check for new convert to allow it to be rolled back.

Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
---
 btrfs-convert.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 78 insertions(+), 1 deletion(-)

diff --git a/btrfs-convert.c b/btrfs-convert.c
index f776a4f..c28dae6 100644
--- a/btrfs-convert.c
+++ b/btrfs-convert.c
 <at>  <at>  -2315,6 +2315,65  <at>  <at>  fail:
 	return -1;
 }

+/*
+ * Check if a non 1:1 mapped chunk can be rolled back.
+ * For new convert, it's OK while for old convert it's NG
+ */
+static int may_rollback_chunk(struct btrfs_fs_info *fs_info, u64 bytenr)
+{
+	struct btrfs_block_group_cache *bg;
+	struct btrfs_key key;
+	struct btrfs_path path;
+	struct btrfs_root *extent_root = fs_info->extent_root;
(Continue reading)

Lu Fengqi | 31 May 04:03 2016

[PATCH] fstests: btrfs: add test for qgroup handle de-refer

Test if qgroup can handle de-reference reallocation. Although current
qgroup can handle it, we still need to prevent any regression which may
break current qgroup.

Signed-off-by: Lu Fengqi <lufq.fnst <at> cn.fujitsu.com>
---
 common/rc           |  4 +--
 tests/btrfs/028     | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/028.out |  2 ++
 tests/btrfs/group   |  1 +
 4 files changed, 96 insertions(+), 2 deletions(-)
 create mode 100755 tests/btrfs/028
 create mode 100644 tests/btrfs/028.out

diff --git a/common/rc b/common/rc
index 51092a0..650d198 100644
--- a/common/rc
+++ b/common/rc
 <at>  <at>  -3284,9 +3284,9  <at>  <at>  _btrfs_get_profile_configs()
 # stress btrfs by running balance operation in a loop
 _btrfs_stress_balance()
 {
-	local btrfs_mnt=$1
+	local options=$ <at> 
 	while true; do
-		$BTRFS_UTIL_PROG balance start $btrfs_mnt
+		$BTRFS_UTIL_PROG balance start $options
 	done
 }

(Continue reading)


Gmane