Timofey Titovets | 30 Jun 13:16 2016
Picon
Gravatar

RAID1: if one disk failed, what errors are expected?

Hi list,
has done some stability test and AFAIK, i see unexpected errors

So:
Take 2 flash devices:
/dev/sdb
/dev/sdc

Format it:
mkfs.btrfs -L TEST -m raid1 -d raid1 /dev/sdb /dev/sdc

Mount:
mount /dev/sdb /mnt

Config test.fio:
[global]
size=1g
filename=/mnt/testfile.fio
numjobs=1
runtime=60
ioengine=libaio
buffer_compress_percentage=15
overwrite=1
end_fsync=1
direct=1
startdelay=30
bs=4k
iodepth=64
rw=randrw
[Disk-4k-randomrw-depth-64]
(Continue reading)

Qu Wenruo | 30 Jun 11:24 2016

[PATCH v5 0/4] Btrfs in-band de-duplication test cases

Btrfs in-band de-duplication test case for in-memory backend.

v5:
  Due to kernel ioctl change, add FORCE flag for "dedupe enable" ioctl call.
v4:
  Due to kernel patchset re-organization, remove on-disk backend test cases
v3:
  Add new test cases for on-disk backend with metadata balance
v2:
  Add new test cases for on-disk backend with full balance

Qu Wenruo (4):
  fstests: rename _require_btrfs to _require_btrfs_subcommand
  fstests: btrfs: Add basic test for btrfs in-band de-duplication
  fstests: btrfs: Add testcase for btrfs dedupe and metadata balance
    race test
  fstests: btrfs: Test inband dedupe with data balance.

 common/defrag       |  13 ++++++
 common/rc           |   2 +-
 tests/btrfs/004     |   2 +-
 tests/btrfs/048     |   2 +-
 tests/btrfs/059     |   2 +-
 tests/btrfs/200     | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/200.out |  22 ++++++++++
 tests/btrfs/201     | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/201.out |   2 +
 tests/btrfs/203     | 110 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/203.out |   3 ++
 tests/btrfs/group   |   3 ++
(Continue reading)

Qu Wenruo | 30 Jun 11:24 2016

[PATCH V9 0/5] In-band de-duplication for btrfs-progs

Patchset can be fetched from github:
https://github.com/adam900710/btrfs-progs.git dedupe_20160630

Inband dedupe(in-memory backend only) ioctl support for btrfs-progs.

User/reviewer/tester can still use previous btrfs-progs patchset to test,
this update is just cleanuping unsupported functions, like on-disk
backend and any on-disk format change.

v7 changes:
   Update ctree.h to follow kernel structure change
   Update print-tree to follow kernel structure change
V8 changes:
   Move dedup props and on-disk backend support out of the patchset
   Change command group name to "dedupe-inband", to avoid confusion with
   possible out-of-band dedupe. Suggested by Mark.
   Rebase to latest devel branch.
V9 changes:
   Follow kernels ioctl change to support FORCE flag, new reconf ioctl,
   and more precious error reporting.

Qu Wenruo (5):
  btrfs-progs: Basic framework for dedupe-inband command group
  btrfs-progs: dedupe: Add enable command for dedupe command group
  btrfs-progs: dedupe: Add disable support for inband dedupelication
  btrfs-progs: dedupe: Add status subcommand
  btrfs-progs: dedupe: introduce reconfigure subcommand

 Documentation/Makefile.in                  |   1 +
 Documentation/btrfs-dedupe-inband.asciidoc | 167 +++++++++++
(Continue reading)

Qu Wenruo | 30 Jun 11:18 2016

[PATCH V12 00/14] Btrfs in-band de-duplication

This patchset can be fetched from github:
https://github.com/adam900710/linux.git wang_dedupe_20160620

And, for any one wants to test it based David's for-next-test-20160620:
https://github.com/adam900710/linux.git wang_dedupe_20160620_for_david
(Several bugs are already reported, although not dedupe bugs)

Hopes this helps David to merge both subpage sector size and dedupe
patchset to for-next branch.

In this update, the patchset goes through ioctl updates.
1) New ioctl re-configure interface
   Suggested by David.

   Unlike old stateless enable ioctl, new re-configure ioctl can only be
   called on dedupe-enabled fs.
   While old enable ioctl will reset any unspecified parameter to
   default value, new re-configure ioctl will only modify specified
   parameters.

   For example:
   # btrfs dedupe enable -b 64K /mnt/test
   # btrfs dedupe ebable -l 1m /mnt/test
   # btrfs dedupe status /mnt/test
   Status:			Enabled
   Hash algorithm:		SHA-256
   Backend:			In-memory
   Dedup Blocksize:		131072 <<< Reset to 128K
   Number of hash: 		[0/1048576]
   Memory usage: 		[0.00B/112.00MiB]
(Continue reading)

Picon
Gravatar

Disk quota exceeded

Hi dear users of the list. 

I'm new to the BTRFS file-system and I am having some problems with quotas 
I would like to share with you what I'm facing about "Disk quota exceeded" it's quite strange to me. 

- Description of the environment 
I have a BTRFS volume "/var/lib/lxd" 
I have some subvolume into "/var/lib/lxd/containers" 

The volume have quota enable 
Every subvolume have their quota groupe created 
Limit of 10Go has been set to every single subvolumes 

-The problem 
Randomly I can't write to some of the subvolumes anymore without having datas added into the subvolume. 
I can't figure why and the subvolumes are randomly not usable with the error : Disk quota exceeded 

I did delete all the quota groups, disable quota on the volume, enable the quota group again on the volume,
check if quota groups were back automatically as btrfs version 4 is supposed to do 
I added back the limit 10Go 
Rescan 
Everything work just fine but some time after, I'm back to over-quota on some subvolumes. 

Thanks for your help on this. 

Below, more details about the config and system point of view : 

uname -a 
Linux lxd-virt-01b 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07 UTC 2016 x86_64 x86_64 x86_64
GNU/Linux 
(Continue reading)

Nikolay Borisov | 29 Jun 08:46 2016

[PATCH] btrfs: Handle uninitialised inode eviction

The code flow in btrfs_new_inode allows for btrfs_evict_inode to be
called with not fully initialised inode (e.g. ->root member not
being set). This can happen when btrfs_set_inode_index in
btrfs_new_inode fails, which in turn would call iput for the newly
allocated inode. This in turn leads to vfs calling into btrfs_evict_inode.
This leads to null pointer dereference. To handle this situation check whether
the passed inode has root set and just free it in case it doesn't.

Signed-off-by: Nikolay Borisov <kernel <at> kyup.com>
---
 fs/btrfs/inode.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

Hello, 

I belive this is fixes the issue reported in 
http://thread.gmane.org/gmane.comp.file-systems.btrfs/57809

diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 4421954720b8..b51723811d01 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
 <at>  <at>  -5159,11 +5159,18  <at>  <at>  void btrfs_evict_inode(struct inode *inode)
 	struct btrfs_root *root = BTRFS_I(inode)->root;
 	struct btrfs_block_rsv *rsv, *global_rsv;
 	int steal_from_global = 0;
-	u64 min_size = btrfs_calc_trunc_metadata_size(root, 1);
+	u64 min_size;
 	int ret;

(Continue reading)

Wang Xiaoguang | 29 Jun 07:15 2016

[PATCH 1/2] btrfs: fix fsfreeze hang caused by delayed iputs deal

When running fstests generic/068, sometimes we got below WARNING:
  xfs_io          D ffff8800331dbb20     0  6697   6693 0x00000080
  ffff8800331dbb20 ffff88007acfc140 ffff880034d895c0 ffff8800331dc000
  ffff880032d243e8 fffffffeffffffff ffff880032d24400 0000000000000001
  ffff8800331dbb38 ffffffff816a9045 ffff880034d895c0 ffff8800331dbba8
  Call Trace:
  [<ffffffff816a9045>] schedule+0x35/0x80
  [<ffffffff816abab2>] rwsem_down_read_failed+0xf2/0x140
  [<ffffffff8118f5e1>] ? __filemap_fdatawrite_range+0xd1/0x100
  [<ffffffff8134f978>] call_rwsem_down_read_failed+0x18/0x30
  [<ffffffffa06631fc>] ? btrfs_alloc_block_rsv+0x2c/0xb0 [btrfs]
  [<ffffffff810d32b5>] percpu_down_read+0x35/0x50
  [<ffffffff81217dfc>] __sb_start_write+0x2c/0x40
  [<ffffffffa067f5d5>] start_transaction+0x2a5/0x4d0 [btrfs]
  [<ffffffffa067f857>] btrfs_join_transaction+0x17/0x20 [btrfs]
  [<ffffffffa068ba34>] btrfs_evict_inode+0x3c4/0x5d0 [btrfs]
  [<ffffffff81230a1a>] evict+0xba/0x1a0
  [<ffffffff812316b6>] iput+0x196/0x200
  [<ffffffffa06851d0>] btrfs_run_delayed_iputs+0x70/0xc0 [btrfs]
  [<ffffffffa067f1d8>] btrfs_commit_transaction+0x928/0xa80 [btrfs]
  [<ffffffffa0646df0>] btrfs_freeze+0x30/0x40 [btrfs]
  [<ffffffff81218040>] freeze_super+0xf0/0x190
  [<ffffffff81229275>] do_vfs_ioctl+0x4a5/0x5c0
  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
  [<ffffffff810038cf>] ? syscall_trace_enter_phase1+0x11f/0x140
  [<ffffffff81229409>] SyS_ioctl+0x79/0x90
  [<ffffffff81003c12>] do_syscall_64+0x62/0x110
  [<ffffffff816acbe1>] entry_SYSCALL64_slow_path+0x25/0x25

From this warning, freeze_super() already holds SB_FREEZE_FS, but
(Continue reading)

Wang Xiaoguang | 29 Jun 07:12 2016

[PATCH 2/2] btrfs: fix free space calculation in dump_space_info()

Signed-off-by: Wang Xiaoguang <wangxg.fnst <at> cn.fujitsu.com>
---
 fs/btrfs/extent-tree.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 8550a0e..520ba8f 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
 <at>  <at>  -7747,8 +7747,8  <at>  <at>  static void dump_space_info(struct btrfs_space_info *info, u64 bytes,
 	printk(KERN_INFO "BTRFS: space_info %llu has %llu free, is %sfull\n",
 	       info->flags,
 	       info->total_bytes - info->bytes_used - info->bytes_pinned -
-	       info->bytes_reserved - info->bytes_readonly,
-	       (info->full) ? "" : "not ");
+	       info->bytes_reserved - info->bytes_readonly -
+	       info->bytes_may_use, (info->full) ? "" : "not ");
 	printk(KERN_INFO "BTRFS: space_info total=%llu, used=%llu, pinned=%llu, "
 	       "reserved=%llu, may_use=%llu, readonly=%llu\n",
 	       info->total_bytes, info->bytes_used, info->bytes_pinned,
--

-- 
2.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Liu Bo | 29 Jun 03:55 2016
Picon

[PATCH] Btrfs: fix read_node_slot to return errors

We use read_node_slot() to read btree node and it has two cases,
a) slot is out of range, which means 'no such entry'
b) we fail to read the block, due to checksum fails or corrupted
   content or not with uptodate flag.
But we're returning NULL in both cases, this makes it return -ENOENT
in case a) and return -EIO in case b), and this fixes its callers
as well as btrfs_search_forward() 's caller to catch the new errors.

The problem is reported by Peter Becker, and I can manage to
hit the same BUG_ON by mounting my fuzz image.

Reported-by: Peter Becker <floyd.net <at> gmail.com>
Signed-off-by: Liu Bo <bo.li.liu <at> oracle.com>
---
 fs/btrfs/ctree.c    | 63 +++++++++++++++++++++++++++++++++++++----------------
 fs/btrfs/tree-log.c |  4 ++++
 2 files changed, 48 insertions(+), 19 deletions(-)

diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index a85cf7d..63510e0 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
 <at>  <at>  -1858,7 +1858,6  <at>  <at>  static void root_sub_used(struct btrfs_root *root, u32 size)

 /* given a node and slot number, this reads the blocks it points to.  The
  * extent buffer is returned with a reference taken (but unlocked).
- * NULL is returned on error.
  */
 static noinline struct extent_buffer *read_node_slot(struct btrfs_root *root,
 				   struct extent_buffer *parent, int slot)
(Continue reading)

Liu Bo | 28 Jun 22:44 2016
Picon

[PATCH] Btrfs: fix double free of fs root

I got this warning while mounting a btrfs image,

[ 3020.509606] ------------[ cut here ]------------
[ 3020.510107] WARNING: CPU: 3 PID: 5581 at lib/idr.c:1051 ida_remove+0xca/0x190
[ 3020.510853] ida_remove called for id=42 which is not allocated.
[ 3020.511466] Modules linked in:
[ 3020.511802] CPU: 3 PID: 5581 Comm: mount Not tainted 4.7.0-rc5+ #274
[ 3020.512438] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014
[ 3020.513385]  0000000000000286 0000000021295d86 ffff88006c66b8f0 ffffffff8182ba5a
[ 3020.514153]  0000000000000000 0000000000000009 ffff88006c66b930 ffffffff810e0ed7
[ 3020.514928]  0000041b00000000 ffffffff8289a8c0 ffff88007f437880 0000000000000000
[ 3020.515717] Call Trace:
[ 3020.515965]  [<ffffffff8182ba5a>] dump_stack+0xc9/0x13f
[ 3020.516487]  [<ffffffff810e0ed7>] __warn+0x147/0x160
[ 3020.517005]  [<ffffffff810e0f4f>] warn_slowpath_fmt+0x5f/0x80
[ 3020.517572]  [<ffffffff8182e6ca>] ida_remove+0xca/0x190
[ 3020.518075]  [<ffffffff813a2bcc>] free_anon_bdev+0x2c/0x60
[ 3020.518609]  [<ffffffff81657a9f>] free_fs_root+0x13f/0x160
[ 3020.519138]  [<ffffffff8165c679>] btrfs_get_fs_root+0x379/0x3d0
[ 3020.519710]  [<ffffffff81e6e975>] ? __mutex_unlock_slowpath+0x155/0x2c0
[ 3020.520366]  [<ffffffff816615b1>] open_ctree+0x2e91/0x3200
[ 3020.520965]  [<ffffffff8161ede2>] btrfs_mount+0x1322/0x15b0
[ 3020.521536]  [<ffffffff81e60e74>] ? kmemleak_alloc_percpu+0x44/0x170
[ 3020.522167]  [<ffffffff8115f5e1>] ? lockdep_init_map+0x61/0x210
[ 3020.522780]  [<ffffffff813a4f59>] mount_fs+0x49/0x2c0
[ 3020.523305]  [<ffffffff813d840c>] vfs_kern_mount+0xac/0x1b0
[ 3020.523872]  [<ffffffff8161dee1>] btrfs_mount+0x421/0x15b0
[ 3020.524402]  [<ffffffff81e60e74>] ? kmemleak_alloc_percpu+0x44/0x170
[ 3020.525045]  [<ffffffff8115f5e1>] ? lockdep_init_map+0x61/0x210
[ 3020.525657]  [<ffffffff8115f5e1>] ? lockdep_init_map+0x61/0x210
(Continue reading)

Otto Kekäläinen | 28 Jun 22:11 2016
Picon
Gravatar

btrfs-check: Fix bitflipped keys from bad RAM

Hello!

A patch with this subject was submitted in May 2014:
http://www.spinics.net/lists/linux-btrfs/msg33777.html

I don't see it among any of the ~360 open issues at
https://bugzilla.kernel.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=btrfs

Unless somebody objects, I'll file it as a NEW issue with patch.

I think the work done by Hugo Mills for this one is important and it
would be a pity if those patches were forgotten. It is one of those
things where btrfs could outperform zfs, which has not bitflip
recovery. Btrfs could have one, and it would be great.

I personally came across a machine with a bitflipped index and I would
love to test these patches. I am however reluctant to invest time in
it if there is no issue in the bug tracker and no visible progress.
Without proper tracking all debugging/feedback would go in vain.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Gmane