Qu Wenruo | 2 Apr 04:21 2015

[PATCH 1/2] btrfs-progs: convert: Make ext*_image file obey datacsum setting.

Before this patch, ext*_image is always set NODATACSUM inode flag.
However btrfs-convert will set normal file with DATACUSM flag by
default, and generate checksum for regular file extent.

Now, a regular file extent is shared by a btrfs file inode with DATACSUM
and ext*_image with NODATACSUM, and it has checksum in csum tree.
This will cause btrfsck complain about odd checksum, since ext*_image is
set NODATACSUM but has checksum generated from regular file extent.

This patch makes convert completely obey datacsum setting, meaning
btrfs-convert will generate csum for every file extent by default.

Reported-by: Tsutomu Itoh <t-itoh <at> jp.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo <at> cn.fujitsu.com>
---
 btrfs-convert.c | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/btrfs-convert.c b/btrfs-convert.c
index 4dc33b3..d742307 100644
--- a/btrfs-convert.c
+++ b/btrfs-convert.c
 <at>  <at>  -1161,7 +1161,7  <at>  <at>  static int create_image_file_range(struct btrfs_trans_handle *trans,
 				   struct btrfs_root *root, u64 objectid,
 				   struct btrfs_inode_item *inode,
 				   u64 start_byte, u64 end_byte,
-				   ext2_filsys ext2_fs)
+				   ext2_filsys ext2_fs, int datacsum)
 {
 	u32 blocksize = ext2_fs->blocksize;
(Continue reading)

Jim Mills | 1 Apr 23:11 2015
Picon

Test message...

I am attempting to see if this message goes through.  My first one got
returned as I didn't know gmail defaulted to HTML.

My second and third never seemed distributed.

Are attachments allowed?

According to this,
https://btrfs.wiki.kernel.org/index.php/Btrfs_mailing_list, they are,
but other things I have seen seem to say they are not.

--
Jim Mills (JMills007 <at> gmail.com)
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Andy Smith | 1 Apr 17:11 2015
Picon

Is this normal? Should I use scrub?

Hello,

I have a 6 device RAID-1 filesystem:

$ sudo btrfs fi df /srv/tank
Data, RAID1: total=1.24TiB, used=1.24TiB
System, RAID1: total=32.00MiB, used=184.00KiB
Metadata, RAID1: total=3.00GiB, used=1.65GiB
unknown, single: total=512.00MiB, used=0.00
$ sudo btrfs fi sh /srv/tank
Label: 'tank'  uuid: 472ee2b3-4dc3-4fc1-80bc-5ba967069ceb
        Total devices 6 FS bytes used 1.24TiB
        devid    2 size 1.82TiB used 384.03GiB path /dev/sdh
        devid    3 size 1.82TiB used 383.00GiB path /dev/sdg
        devid    4 size 1.82TiB used 384.00GiB path /dev/sdf
        devid    5 size 2.73TiB used 1.13TiB path /dev/sdk
        devid    6 size 1.82TiB used 121.00GiB path /dev/sdj
        devid    7 size 2.73TiB used 116.00GiB path /dev/sde

Btrfs v3.14.2

All of these devices are in an external eSATA enclosure.

A few days ago (I believe) something went wrong with the enclosure
hardware and the SCSI bus kept getting reset over and over. At one
point three of the six devices were kicked out and the filesystem
was left running (read-only) on three devices.

Through some trial and error I determined that the enclosure was
taking exception to one of the devices, and by removing it I was
(Continue reading)

Wang, Zhiye | 1 Apr 14:42 2015

How to avoid certain file being defragmented?

Hello,

I have some files which I hope their on-disk data can be on fixed location of disk. My understanding is that
defragmentation operation can potentially move data blocks of a file.

So, can I avoid certain file being defragmented in btrfs? 

Thanks
Mike​

Filipe Manana | 1 Apr 03:00 2015

[PATCH] Btrfs: avoid syncing log in the fast fsync path when not necessary

Commit 3a8b36f37806 ("Btrfs: fix data loss in the fast fsync path") added
a performance regression for that causes an unnecessary sync of the log
trees (fs/subvol and root log trees) when 2 consecutive fsyncs are done
against a file, without no writes or any metadata updates to the inode in
between them and if a transaction is committed before the second fsync is
called.

Huang Ying reported this to lkml (https://lkml.org/lkml/2015/3/18/99)
after a test sysbench test that measured a -62% decrease of file io
requests per second for that tests' workload.

The test is:

  echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
  echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
  echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
  echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor
  mkfs -t btrfs /dev/sda2
  mount -t btrfs /dev/sda2 /fs/sda2
  cd /fs/sda2
  for ((i = 0; i < 1024; i++)); do fallocate -l 67108864 testfile.$i; done
  sysbench --test=fileio --max-requests=0 --num-threads=4 --max-time=600 \
    --file-test-mode=rndwr --file-total-size=68719476736 --file-io-mode=sync \
    --file-num=1024 run

A test on kvm guest, running a debug kernel gave me the following results:

Without 3a8b36f378060d:             16.01 reqs/sec
With 3a8b36f378060d:                 3.39 reqs/sec
With 3a8b36f378060d and this patch: 16.04 reqs/sec
(Continue reading)

Chen Hanxiao | 1 Apr 09:52 2015

[PATCH] btrfs-progs doc: emphasis that only mounted device works for btrfs device stats

We provided format <path>|<device> in command line.
But btrfs device stats doesn't work if device is not mounted.

Also fix some tailing whitespace.

Signed-off-by: Chen Hanxiao <chenhanxiao <at> cn.fujitsu.com>
---
 Documentation/btrfs-device.txt | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/btrfs-device.txt b/Documentation/btrfs-device.txt
index 66be6b3..3868fd4 100644
--- a/Documentation/btrfs-device.txt
+++ b/Documentation/btrfs-device.txt
 <at>  <at>  -83,14 +83,14  <at>  <at>  Check device to see if it has all of it's devices in cache for mounting.
 *scan* [(--all-devices|-d)|<device> [<device>...]]::
 Scan devices for a btrfs filesystem.
 +
-If one or more devices are passed, these are scanned for a btrfs filesystem. 
+If one or more devices are passed, these are scanned for a btrfs filesystem.
 If no devices are passed, btrfs uses block devices containing btrfs
 filesystem as listed by blkid.
-Finally, if '--all-devices' or '-d' is passed, all the devices under /dev are 
+Finally, if '--all-devices' or '-d' is passed, all the devices under /dev are
 scanned.

 *stats* [-z] <path>|<device>::
-Read and print the device IO stats for all devices of the filesystem
+Read and print the device IO stats for all mounted devices of the filesystem
 identified by <path> or for a single <device>.
(Continue reading)

Qu Wenruo | 1 Apr 09:25 2015

[PATCH 0/3] Use list to replace rb_tree in btrfs_delayed_ref_head->ref_roots.

The old rbtree implement of ref_head->ref_root sacrificed the insert
order to do better delayed_ref_node merging.
However the out of order behavior makes btrfs_find_all_roots() unable to
find correct root, since it needs the insert order to skip later
delayed_nodes.

Without such ability, qgroup can never be accurate (although it is never
accurate anyway, :-( ).

This patchset first fix a small but deadly (only for qgroup) bug in
backref, which will cause btrfs_find_all_roots() return result less than
expected one.

And the second patch do migrate from rb_tree implement to new list
implement, and still maintain the ability to merge delayed_ref_nodes.

The last patch will cleanup the unused rb_tree implement only functions.

The new list implement is in a simpler logic and code base (removes
about 200 lines).

Qu Wenruo (3):
  btrfs: backref: Don't merge refs which are not for same block.
  btrfs: delayed-ref: Use list to replace the ref_root in ref_head.
  btrfs: delayed-ref: Cleanup the unneeded functions.

 fs/btrfs/backref.c     |  15 +--
 fs/btrfs/delayed-ref.c | 311 ++++++++++++-------------------------------------
 fs/btrfs/delayed-ref.h |  18 ++-
 fs/btrfs/disk-io.c     |   8 +-
(Continue reading)

Matt Grant | 1 Apr 04:01 2015
Picon

Python pybtrfs df wrapper script to report btrfs metadata, block, space in df compatible output

Hi!

Use this at work.  Releasing to list to prompt design of output that is
100% df format compatible for automated reporting and graphing.

This is so BTRFS space statistics acn bre reported back to existing
graphing tool chains that many of you would have in place.  Quite useful
for munin, hint, hint :-)

URL for github is:

https://github.com/grantma/pybtrfs.git

There is also a shell script there that can be called from cron.

Please get back to me if you have any questions.

Standard not warranted disclaimers apply to the code.  Its GPLv3
licensed.

Best Regards,

Matt Grant

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

Zach Brown | 1 Apr 01:45 2015
Picon

readdir tree/page lock inversion

We've known for eons that it's not great that readdir holds tree locks
while calling mkwrite by way of dir_emit().  I fiddled around and found
a reliable if goofy deadlock reproducer.  It typically takes a few
seconds on modest hardware here (single package, dual core/ht, single
spindle.)

I made a quick attempt at refactoring readdir to stage entries in a
locally allocated page before releasing the locks and emitting them but
unfortunately my eyeballs fell out and one of the cats ran off with
them.  Maybe someone else will have more luck.

- z

#define _GNU_SOURCE
#include <dirent.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/mman.h>

#define LEN 300
static char junk[LEN];

int main(int argc, char **argv)
{
	int dir_fd;
	void *ptr;
(Continue reading)

Chris Murphy | 1 Apr 00:11 2015

Re: F21 fails to mount root part, btrfs check: Couldn't open file system

On Tue, Mar 31, 2015 at 2:39 PM, Matt Grant
<matt.grant <at> foodstuffs-si.co.nz> wrote:
> Seen this before at home.
>
> You have to mount -o recovery off a 3.19 kernel to fix it...
>
> If you can get the SSD out, attach it to a desktop, as there will be no Install CDs using 3.19 yet.

Fedora 22 Workstation alpha has 4.0.0 (rc1 I think), and the current
TC6 beta has 4.0.0-rc4. It's possible to use the netinstall, which is
much smaller, and use boot param single or rescue to avoid the
installer launching.

And actually 3.19.2 is the stable kernel for Fedora 21, with 3.19.3
just pushed today (take mirrors a day or two to catch up), not 3.18.9
as I reported earlier.

--

-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Filipe Manana | 31 Mar 21:18 2015

[PATCH] Btrfs: add debugfs file to test transaction aborts

At the moment we can not reliably and deterministically test that the
transaction abortion code works as expected. For example in the past [1]
we had an issue where that code returned the pinned extents to the free
space caches allowing fstrim to perform a discard against the physical
locations of the extents, and worse, when the fs was mounted with the
option -o discard it explicitly did a discard on each pinned extent.
This resulted in filesystem corruption, leaving the fs unmountable.

This patch adds a debugfs file named abort_transaction, which has a
default value of 0, can only be written by someone with root privileges
and when a non-zero value is written to it, it makes all subsequent
transaction commits fail at the very end (right before writing the new
superblock), which results in a transaction abortion.
This way we can for example write a deterministic fstest for commit [1]
which looks like:

  _supported_fs btrfs
  _supported_os Linux
  _require_scratch
  _require_btrfs_debugfs "abort_transaction"
  _need_to_be_root

  rm -f $seqres.full

  # We will abort a btrfs transaction later, which always produces a warning in
  # dmesg. We do not want the test to fail because of this.
  _disable_dmesg_check

  _scratch_mkfs >>$seqres.full 2>&1
  _scratch_mount "-o discard"
(Continue reading)


Gmane