Qu Wenruo | 4 Aug 07:52 2015

[PATCH] btrfs-progs: Doc: Add extra notes for qgroup

Add extra explaination on btrfs qgroup on the following two things:
1) Need sync to show accurate qgroup numbers
2) Cow may cause extra quota usage

Although btrfs qgroup is still buggy, especially for limit behavior, but
some strange behavior is not really a bug.
Just add these explaination for end users if they really want to try it.

Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
---
 Documentation/btrfs-qgroup.asciidoc | 40 +++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/Documentation/btrfs-qgroup.asciidoc b/Documentation/btrfs-qgroup.asciidoc
index 57cf012..f2fc514 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
 <at>  <at>  -127,6 +127,46  <at>  <at>  If no prefix is given, use ascending order by default.
 +
 If multiple <attr>s is given, use comma to separate.

+EXTRA NOTES
+-----------
+1. Need sync before *btrfs qgroup show* command
++
+Sync is needed to output correct numbers, especially after kernel v4.2.
+
+2. Copy-on-write(CoW) may cause extra space usage
++
+Cow will cause extra space usage, compared to overwrite filesystem, like ext4
(Continue reading)

Alex | 3 Aug 20:09 2015
Picon

Fwd: Bug report - btrfs hanging

Dear Btrfs devs,

I have an external HD formatted with btrfs, and have noticed that
various operations (copying files, deleting files, etc) hang from time
to time. Here's debug output from the latest hang:

dmesg output:
------------------------
[496960.834080] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
[496960.834192] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
[496960.834261] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
[496960.834334] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
[496960.834357] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
[496960.834658] pci_bus 0000:01: Allocating resources
[496960.834719] pci_bus 0000:06: Allocating resources
[496960.834777] pci_bus 0000:07: Allocating resources
[496960.834807] pci_bus 0000:0c: Allocating resources
[496960.834833] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
[496960.834976] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
[496960.835140] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
[501706.637861] i915 0000:00:02.0: BAR 6: [??? 0x00000000 flags 0x2]
has bogus alignment
(Continue reading)

Qu Wenruo | 3 Aug 08:44 2015

[PATCH] btrfs: qgroup: Fix a regression in qgroup reserved space.

During the change to new btrfs extent-oriented qgroup implement, due to
it doesn't use the old __qgroup_excl_accounting() for exclusive extent,
it didn't free the reserved bytes.

The bug will cause limit function go crazy as the reserved space is
never freed, increasing limit will have no effect and still cause
EQOUT.

The fix is easy, just free reserved bytes for newly created exclusive
extent as what it does before.

Reported-by: Tsutomu Itoh <t-itoh <at> jp.fujitsu.com>
Signed-off-by: Yang Dongsheng <yangds.fnst <at> cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
---
To Chris:

Would you please consider merging this patch in the late v4.2 merge
window?
As it's a huge regression and the fix is small enough and just does
what it did before in __qgroup_excl_accounting() function.

The corresponding test case will follow soon.

And sorry for the regression I introduced.

Thanks,
Qu
---
 fs/btrfs/qgroup.c | 5 +++++
(Continue reading)

Sonic | 2 Aug 19:13 2015
Picon

BTRFS disaster (of my own making). Is this recoverable?

I have had bad dreams about this particular fat finger but after a few
years it has finally happened.

Scenario: 2 drives in a raw btrfs array (no previous partitions and
non-redundant) with various subvols as well. One was sdc the other was
sde, although sde never displays with mount command and blkid is the
same for both.

Thinking I was writing to a flash drive I sent 32MB via dd
===========================
dd if=file of=/dev/sde
===========================
to sde (instead of what I wanted - sdf) and now the volume nor any of
it's subvol's can mount (of course that seems entirely reasonable,
although you can imagine how unhappy I am).

With:
===========================
mount -t btrfs /mnt/butter/
===========================
I get:
===========================
[ 3421.193103] BTRFS info (device sde): disk space caching is enabled
[ 3421.193734] BTRFS (device sde): bad tree block start
8330001001141004672 20971520
[ 3421.193738] BTRFS: failed to read chunk root on sde
[ 3421.203221] BTRFS: open_ctree failed
===========================

If I specify /dev/sdc instead of relying on fstab I get:
(Continue reading)

Hendrik Friedel | 1 Aug 22:09 2015

Data single *and* raid?

Hello,

I converted an array to raid5 by
btrfs device add /dev/sdd /mnt/new_storage
btrfs device add /dev/sdc /mnt/new_storage
btrfs balance start -dconvert=raid5 -mconvert=raid5 /mnt/new_storage/

The Balance went through. But now:
Label: none  uuid: a8af3832-48c7-4568-861f-e80380dd7e0b
         Total devices 3 FS bytes used 5.28TiB
         devid    1 size 2.73TiB used 2.57TiB path /dev/sde
         devid    2 size 2.73TiB used 2.73TiB path /dev/sdc
         devid    3 size 2.73TiB used 2.73TiB path /dev/sdd
btrfs-progs v4.1.1

Already the 2.57TiB is a bit surprising:
root <at> homeserver:/mnt# btrfs fi df /mnt/new_storage/
Data, single: total=2.55TiB, used=2.55TiB
Data, RAID5: total=2.73TiB, used=2.72TiB
System, RAID5: total=32.00MiB, used=736.00KiB
Metadata, RAID1: total=6.00GiB, used=5.33GiB
Metadata, RAID5: total=3.00GiB, used=2.99GiB

Why is there Data single and Raid?
Why is Metadata RAID1 and Raid5?

A scrub is currently running and showed no errors yet.

Greetings,
Hendrik
(Continue reading)

Chris Murphy | 1 Aug 19:29 2015

ext4 convert bugs, wiki warning?

Does someone with wiki edit capability want to put up a warning about
btrfs-convert problems? I don't think it needs to be a lengthy write
up since the scope of the problem is not clear yet. But since there's
definitely reproduced problems that have been going on for some time
now maybe it'd be a good idea to put up a warning until this is more
stable?

I'm thinking of just a yellow warning "sidebar" like thing that maybe
just says there's been a regression and it's breaking file systems,
sometimes irreparably?

--

-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Cornelius van Rooyen | 1 Aug 16:54 2015
Picon

Filesystem unmountable

Good day,

This might be something you are already aware of, but I thought I should
inform someone of what happened to my sytem.

I was running ArchLinux with kernel 4.1.3-1-ARCH #1 SMP PREEMPT x86_64
GNU/Linux and executed btrfs-convert /mnt from an ArchLinux live disk to
convert my ext4 filesystem to btrfs.

The conversion was seemingly successful, but after booting journalctl
--verify reported some corrupt files. I saw on a an online post that I
should try to do a btrfs balance start after converting from ext4, and so I
executed the command and left the laptop to complete the task. The next
morning however I found the system frozen and had to do a hard reset. After
this the filesystem was unmountable.

The initial boot after the hard reset left me in an emergency shell at
which point I tried to mount the system and the shell stopped responding.
Booting a live disk and running btrfsck exited with a segmentation fault.
Attempts to mount the filesystem also exited with a segmentation fault or
the shell seemed to be waiting for the command to complete for very long
and did not respond to CTRL-C.

I know nothing of kernels or filesystem drivers and I hope I am not
waisting any body's time with something that was probably my own fault. I
have since reinstalled my OS and in hindsight I should probably have
attempted to save some of the kernel log files.

Kind regards,
Neels van Rooyen
(Continue reading)

Stefan Priebe | 31 Jul 21:34 2015

btrfs trace / deadlock with 4.2-rc3

Hi,

i got the following error today.

2015-07-31 21:00:19     ---[ end Kernel panic - not syncing: Watchdog 
detected hard LOCKUP on cpu 12
2015-07-31 21:00:19     Kernel Offset: 0x0 from 0xffffffff81000000 
(relocation range: 0xffffffff80000000-0xffffffff9fffffff)
2015-07-31 21:00:18     Shutting down cpus with NMI
2015-07-31 21:00:17     Kernel panic - not syncing: Watchdog detected 
hard LOCKUP on cpu 12
2015-07-31 21:00:13     ---[ end trace f1b45f288cc28a82 ]---
2015-07-31 21:00:13     RSP <ffff880f0cb7fb28>
2015-07-31 21:00:13     RIP [<ffffffffa02152ee>] 
insert_inline_extent_backref+0xde/0xe0 [btrfs]
2015-07-31 21:00:13     Code: 45 10 49 89 d9 48 8b 55 c8 4c 89 34 24 4c 
89 e9 4c 89 fe 4c 89 e7 48 89 44 24 10 8b 45 28 89 44 24 08 e8 a6 ee ff 
ff 31 c0 eb ae <0f> 0b 0f 1f 44 00 00 55 ba c0 00 00 00 65 48 8b 04 25 
80 a8 00
2015-07-31 21:00:12     [<ffffffff81090a30>] ? kthread_worker_fn+0x150/0x150
2015-07-31 21:00:12     [<ffffffff815f9088>] ret_from_fork+0x58/0x90
2015-07-31 21:00:12     [<ffffffff81090a30>] ? kthread_worker_fn+0x150/0x150
2015-07-31 21:00:12     [<ffffffff81090af9>] kthread+0xc9/0xe0
2015-07-31 21:00:12     [<ffffffffa02313f0>] ? open_ctree+0x22e0/0x22e0 
[btrfs]
2015-07-31 21:00:12     [<ffffffffa02315bd>] 
transaction_kthread+0x1cd/0x250 [btrfs]
2015-07-31 21:00:12     [<ffffffffa023390e>] 
btrfs_commit_transaction+0x4e/0xae0 [btrfs]
2015-07-31 21:00:12     [<ffffffffa0221f8b>] 
(Continue reading)

Qu Wenruo | 31 Jul 07:40 2015

Re: mount btrfs takes 30 minutes, btrfs check runs out of memory


John Ettedgui wrote on 2015/07/30 22:15 -0700:
> On Thu, Jul 30, 2015 at 9:52 PM, Qu Wenruo <quwenruo <at> cn.fujitsu.com> wrote:
>> I'm not familiar with ftrace either, but your trace is good enough already,
>> the only thing needed is to avoid using btrfs as root partition(at least
>> /var/).
> I've stopped all journaling services for now, I hope that's enough/
>>
>> My personal recommendation is to use a liveCD or rescue media to do the
>> trace dump.
>>
> If not I'll have to do that, but this computer has no CD drive.
It seems that you're using Chromium while doing the dump. :)

If no CD drive, I'll recommend to use Archlinux installation iso to make 
a bootable USB stick and do the dump.
(just download and dd would do the trick)
As its kernel and tools is much newer than most distribution.

It's better to provide two trace.
One is the function tracer one, with "btrfs:*" as set_event.
The other is the function_graph one. with "btrfs_mount" as 
set_graph_function.

Thanks for your patient to help improving btrfs.
Although I may not be able to check the trace until next Monday... :(

Thanks,
Qu

(Continue reading)

Qu Wenruo | 31 Jul 06:52 2015

Re: mount btrfs takes 30 minutes, btrfs check runs out of memory


John Ettedgui wrote on 2015/07/30 21:09 -0700:
> On Thu, Jul 30, 2015 at 7:34 PM, Qu Wenruo <quwenruo <at> cn.fujitsu.com> wrote:
>>
>>
>> Hi John,
>> Thanks for the trace output.
> You are welcome, thank you for looking at it!
>>
>> But it seems that, your root partition is also btrfs, causing a lot of btrfs
>> trace from your systemd journal.
>>
> Oh yes sorry about that.
> I actually have 3 partition in btrfs, the problematic one being the
> only big one.
>> Would you mind re-collecting the ftrace without such logging system caused
>> btrfs trace?
> Sure, how would I do that?
> This is my first time using ftrace.

I'm not familiar with ftrace either, but your trace is good enough 
already, the only thing needed is to avoid using btrfs as root 
partition(at least /var/).

My personal recommendation is to use a liveCD or rescue media to do the 
trace dump.

Other recommendation is to enable all btrfs trace point, and it seems 
that you have already done it while collecting the trace.
>>
(Continue reading)

Qu Wenruo | 31 Jul 04:34 2015

Re: mount btrfs takes 30 minutes, btrfs check runs out of memory


John Ettedgui wrote on 2015/07/29 18:55 +0000:
> Hello,
> I have the same issue and would like to add myself to this thread.
> My btrfs partition is about 10tb on top of lvm2 and has been taking about a minute to mount in the past few months.
>
>> Qu Wenruo <quwenruo <at>cn.fujitsu.com <http://cn.fujitsu.com>> writes:
>>
>> Quite common, especial when it grows large.
>> But it would be much better to use ftrace to show which btrfs operation
>> takes the most time.
>
>
> I have got a trace file running this command:
> trace-cmd record -e btrfs mount <PARTITION>
>
> Since it is fairly big for an email I have gzipped it.
>
> Thanks!
> John
>
Hi John,
Thanks for the trace output.

But it seems that, your root partition is also btrfs, causing a lot of 
btrfs trace from your systemd journal.

Would you mind re-collecting the ftrace without such logging system 
caused btrfs trace?

(Continue reading)


Gmane