headers
Chris Mason | 1 Nov 2011 03:49
Picon
Favicon

new integration-danger branch pushed out (kernel)

Hi everyone,

I've pushed out a new integration branch, but it is not for general use.

I'm still going through and hammering on the new logging code code to
make sure it is properly backwards compatible with older kernels and
that it hasn't introduced any new bugs.

*** Please do not use this code on anything other than test filesystems that
can easily be reformatted. ***

In testing the new sub-transid logging code, I've spent a lot of time
hunting logging bugs, and writing new tests to try and make sure the
logging code is working correctly in general.  Between that and some new
tests from Arne, I've had to make some pretty big changes to the code.

This meant collapsing Fujitsu's original nicely broken out patches into
a larger patch so that I could be sure later kernel bisections wouldn't
run into problems.  All credit to Liu Bo and Fujitsu for getting the
sub-transid work going, any bugs in the pushed patch are probably ones
I introduced.

integration-danger merges in the new sub-transid logging code along with
all of the fixes Dave Sterba has been tracking, and Josef's code.

I'm still merging in trees from Arne and Jan Schmidt, those should get
pushed out tomorrow afternoon.

-chris
(Continue reading)

Permalink | Reply | 16 comments |
headers
Tsutomu Itoh | 1 Nov 2011 04:20
Favicon

[PATCH] Btrfs-progs: fix compiler warning of extent-tree.c

In recovery-beta, following compiler warning was displayed.

 gcc -Wp,-MMD,./.extent-tree.o.d,-MT,extent-tree.o -Wall -D_FILE_OFFSET_BITS=64
-D_FORTIFY_SOURCE=2 -g -O0 -c extent-tree.c
 extent-tree.c: In function 'lookup_inline_extent_backref':
 extent-tree.c:1071: warning: format '%u' expects type 'unsigned int', but argument 3 has type 'long
unsigned int'

Signed-off-by: Tsutomu Itoh <t-itoh <at> jp.fujitsu.com>
---
 extent-tree.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/extent-tree.c b/extent-tree.c
index 5bed3c2..5144d57 100644
--- a/extent-tree.c
+++ b/extent-tree.c
 <at>  <at>  -1068,7 +1068,7  <at>  <at>  static int lookup_inline_extent_backref(struct btrfs_trans_handle *trans,
 #endif
 	if (item_size < sizeof(*ei)) {
 		printf("Size is %u, needs to be %u, slot %d\n", item_size,
-		       sizeof(*ei), path->slots[0]);
+		       (u32)sizeof(*ei), path->slots[0]);
 		btrfs_print_leaf(root, leaf);
 		return -EINVAL;
 	}

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo <at> vger.kernel.org
(Continue reading)

Permalink | Reply | 0 comments |
headers
Miao Xie | 1 Nov 2011 08:39
Favicon

Re: [RFC PATCH 2/2] Btrfs: introduce free space cluster for each node

Any Comment?

On Thu, 08 Sep 2011 16:18:09 +0800, Miao Xie wrote:
> This patch introduce free space cluster for each node in the b+tree. And
> we also simplify the fill method and the space allocation of the cluster:
> - Allocate free space cluster for each node
> - Allocate the free space extent (>=256KB, split a large extent or get the
>   contiguous blocks from the bitmaps) from the block group cache and cache
>   it in the cluster, instead of moving the free space entry from the block
>   group cache to the cluster directly. By this way, we just manage a extent
>   in the cluster.
> - When doing the tree block allocation, we can get the space just by split
>   the extent in the parent's cluster. By this way, we can allocate the tree
>   block concurrently and also can store the child nodes/leaves into the
>   contiguous blocks.
> 
> Beside that, we modify the source of the space cache to make it fit the above
> change. When we write out the information of the free space, we also write out
> the extent information in the clusters. And when we load the free space
> information, we will try to merge the free space fragment at first, because the
> extent in the clusters is small, and may merge them to be a large one.
> (Before applying this patch, we build the free space tree by the cached information
> directly. I think we needn't worry about the compatibility. At the worst, we may
> create lots of small extents which may be merge into a large one, but it will not
> break the use of Btrfs.）
> 
> We did some performance test for this patch, we find it works well, and make the
> small file performance grow up.
> 
> The sequential write performance of the small file:
(Continue reading)

Permalink | Reply | 2 comments |
headers
Arne Jansen | 1 Nov 2011 08:56
Picon

Re: [PATCH 03/21] Btrfs: add BTRFS_AVAIL_ALLOC_BIT_SINGLE bit

On 23.08.2011 22:01, Ilya Dryomov wrote:
> Right now on-disk BTRFS_BLOCK_GROUP_* profile bits are used for
> avail_{data,metadata,system}_alloc_bits fields, which are there to tell
> us about available allocation profiles in the fs.  When chunk is
> created, it's profile is OR'ed with respective avail_alloc_bits field.
> Since SINGLE is denoted by 0 in the on-disk format, currently there is
> no way to tell when such chunks become avaialble.  Restriper needs that
> information, so add a separate bit for SINGLE profile.
> 
> This bit is going to be in-memory only, it should never be written out
> to disk, so it's not a disk format change.  However to avoid remappings
> in future, reserve corresponding on-disk bit.
> 
> Signed-off-by: Ilya Dryomov <idryomov <at> gmail.com>
> ---
>  fs/btrfs/ctree.h       |   12 ++++++++++++
>  fs/btrfs/extent-tree.c |   22 ++++++++++++++--------
>  2 files changed, 26 insertions(+), 8 deletions(-)
> 
> diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> index b882c95..5b00eb8 100644
> --- a/fs/btrfs/ctree.h
> +++ b/fs/btrfs/ctree.h
>  <at>  <at>  -725,6 +725,17  <at>  <at>  struct btrfs_csum_item {
>  					 BTRFS_BLOCK_GROUP_RAID1 |   \
>  					 BTRFS_BLOCK_GROUP_DUP |     \
>  					 BTRFS_BLOCK_GROUP_RAID10)
> +/*
> + * We need a bit for restriper to be able to tell when chunks of type
> + * SINGLE are available.  It is used in avail_*_alloc_bits.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Stephane CHAZELAS | 1 Nov 2011 09:36
Picon
Favicon

ENOSPC on almost empty FS

Hiya,

trying to restore a FS from a backup (tgz) on a freshly made
btrfs this morning, I got ENOSPCs after about 100MB out of 4GB
have been extracted. strace indicates that the ENOSPC are upon
the open(O_WRONLY).

Restoring with:

mkfs.btrfs /dev/mapper/VG_USB-root
mount -o compress-force,ssd $_ /mnt
cd /mnt
pv ~/backup.tgz | gunzip | sudo bsdtar -xpSf - --numeric-owner

That's on a LVM LV with the PV on a USB key.

If I supspend the job and resume it, then the ENOSPCs go away.

The only way I could restore the backup was via rate limiting
the untar:

zcat ~/backup.tgz | pv -L 3000000 | sudo bsdtar -xpSf - --numeric-owner

That 3MB/s wasn't even enough, as 3 files triggered a ENOSPC,
but I did untar them separately afterwards.

That's with debian's 3.0.0-1 amd64 kernel.

Is that expected behavior due to the way allocation works in
btrfs?
(Continue reading)

Permalink | Reply | 1 comment |
headers
Liu Bo | 1 Nov 2011 10:22
Favicon

Re: new integration-danger branch pushed out (kernel)

Hi, 

On 11/01/2011 10:49 AM, Chris Mason wrote:
> Hi everyone,
> 
> I've pushed out a new integration branch, but it is not for general use.
> 
> I'm still going through and hammering on the new logging code code to
> make sure it is properly backwards compatible with older kernels and
> that it hasn't introduced any new bugs.
> 
> *** Please do not use this code on anything other than test filesystems that
> can easily be reformatted. ***
> 
> In testing the new sub-transid logging code, I've spent a lot of time
> hunting logging bugs, and writing new tests to try and make sure the
> logging code is working correctly in general.  Between that and some new
> tests from Arne, I've had to make some pretty big changes to the code.
> 

I'm giving a hard read on the new code, and I notice a slight change:
 <at>  <at>  -3006,14 +3179,11  <at>  <at>  out:
 static int inode_in_log(struct btrfs_trans_handle *trans,
                 struct inode *inode)
 {
-       struct btrfs_root *root = BTRFS_I(inode)->root;
        int ret = 0;

-       mutex_lock(&root->log_mutex);
-       if (BTRFS_I(inode)->logged_trans == trans->transid &&
(Continue reading)

Permalink | Reply | 2 comments |
headers
Arne Jansen | 1 Nov 2011 11:08
Picon

Re: [PATCH 05/21] Btrfs: add basic restriper infrastructure

On 23.08.2011 22:01, Ilya Dryomov wrote:
> Add basic restriper infrastructure: ioctl to start restripe, all
> restripe ioctl data structures, add data structure for tracking
> restriper's state to fs_info.  Duplicate balancing code for restriper,
> btrfs_balance() will be removed when restriper is implemented.
> 
> Explicitly disallow any volume operations when restriper is running.
> (previously this restriction relied on volume_mutex being held during
> the execution of any volume operation)
> 
> Signed-off-by: Ilya Dryomov <idryomov <at> gmail.com>
> ---
>  fs/btrfs/ctree.h   |    5 +
>  fs/btrfs/disk-io.c |    4 +
>  fs/btrfs/ioctl.c   |  107 ++++++++++++++++++++++----
>  fs/btrfs/ioctl.h   |   37 +++++++++
>  fs/btrfs/volumes.c |  219 ++++++++++++++++++++++++++++++++++++++++++++++++++--
>  fs/btrfs/volumes.h |   18 ++++
>  6 files changed, 369 insertions(+), 21 deletions(-)
> 
> diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> index 5b00eb8..65d7562 100644
> --- a/fs/btrfs/ctree.h
> +++ b/fs/btrfs/ctree.h
>  <at>  <at>  -895,6 +895,7  <at>  <at>  struct btrfs_block_group_cache {
>  };
>  
>  struct reloc_control;
> +struct restripe_control;
>  struct btrfs_device;
(Continue reading)

Permalink | Reply | 2 comments |
headers
David Sterba | 1 Nov 2011 11:11
Picon

Warnings (was: Re: new integration-danger branch pushed out (kernel))

Hi,

On Mon, Oct 31, 2011 at 10:49:57PM -0400, Chris Mason wrote:
> I've pushed out a new integration branch, but it is not for general use.

Thank you!

xfstests/083 and 089 cause the following warnings. I've seen them before when
testing josef/for-chris and josef/master branches (reported on irc).

[ 1149.108088] ------------[ cut here ]------------
[ 1149.114384] WARNING: at fs/btrfs/extent-tree.c:3932 btrfs_free_block_groups+0x304/0x320 [btrfs]()
[ 1149.124928] Hardware name: Santa Rosa platform
[ 1149.124931] Modules linked in: loop btrfs aoe
[ 1149.124938] Pid: 23991, comm: umount Not tainted 3.1.0-default+ #64
[ 1149.124941] Call Trace:
[ 1149.124950]  [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 1149.124953]  [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 1149.124973]  [<ffffffffa002bba4>] btrfs_free_block_groups+0x304/0x320 [btrfs]
[ 1149.124985]  [<ffffffffa0036ab9>] close_ctree+0x1e9/0x390 [btrfs]
[ 1149.124991]  [<ffffffff8115e2df>] ? dispose_list+0x4f/0x60
[ 1149.124999]  [<ffffffffa00108dd>] btrfs_put_super+0x1d/0x30 [btrfs]
[ 1149.125003]  [<ffffffff81145202>] generic_shutdown_super+0x62/0xe0
[ 1149.125006]  [<ffffffff81145316>] kill_anon_super+0x16/0x30
[ 1149.125009]  [<ffffffff811461a2>] ? deactivate_super+0x42/0x70
[ 1149.125012]  [<ffffffff811456dd>] deactivate_locked_super+0x3d/0xa0
[ 1149.125015]  [<ffffffff811461aa>] deactivate_super+0x4a/0x70
[ 1149.125019]  [<ffffffff81162972>] mntput_no_expire+0xa2/0xf0
[ 1149.125022]  [<ffffffff811636df>] sys_umount+0x6f/0x390
[ 1149.125027]  [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
(Continue reading)

Permalink | Reply | 2 comments |
headers
Arne Jansen | 1 Nov 2011 11:18
Picon

Re: [PATCH 10/21] Btrfs: usage filter

On 23.08.2011 22:01, Ilya Dryomov wrote:
> Select chunks that are less than X percent full.
> 
> Signed-off-by: Ilya Dryomov <idryomov <at> gmail.com>
> ---
>  fs/btrfs/volumes.c |   33 +++++++++++++++++++++++++++++++++
>  fs/btrfs/volumes.h |    1 +
>  2 files changed, 34 insertions(+), 0 deletions(-)
> 
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index f045615..b49ecfa 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
>  <at>  <at>  -2193,6 +2193,33  <at>  <at>  static int chunk_profiles_filter(u64 chunk_profile,
>  	return 1;
>  }
>  
> +static u64 div_factor_fine(u64 num, int factor)
> +{
> +	if (factor == 100)
> +		return num;

You already have changed this to a range check that always returns
num when <0 or >= 100, but I'd find it more consistent to return 0
when factor < 0.

> +	num *= factor;
> +	do_div(num, 100);
> +	return num;
> +}
(Continue reading)

Permalink | Reply | 0 comments |
headers
Arne Jansen | 1 Nov 2011 11:29
Picon

Re: [PATCH 14/21] Btrfs: save restripe parameters to disk

On 23.08.2011 22:01, Ilya Dryomov wrote:
> Introduce a new btree objectid for storing restripe item.  The reason is
> to be able to resume restriper after a crash with the same parameters.
> Restripe item has a very high objectid and goes into tree of tree roots.
> 
> The key for the new item is as follows:
> 
>     [ BTRFS_RESTRIPE_OBJECTID ; 0 ; 0 ]
> 
> Older kernels simply ignore it so it's safe to mount with an older
> kernel and then go back to the newer one.
> 
> Signed-off-by: Ilya Dryomov <idryomov <at> gmail.com>
> ---
>  fs/btrfs/ctree.h   |  127 +++++++++++++++++++++++++++++++++++++++++++++++++++-
>  fs/btrfs/volumes.c |  105 ++++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 228 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> index 65d7562..b524034 100644
> --- a/fs/btrfs/ctree.h
> +++ b/fs/btrfs/ctree.h
>  <at>  <at>  -85,6 +85,9  <at>  <at>  struct btrfs_ordered_sum;
>  /* holds checksums of all the data extents */
>  #define BTRFS_CSUM_TREE_OBJECTID 7ULL
>  
> +/* for storing restripe params in the root tree */
> +#define BTRFS_RESTRIPE_OBJECTID -4ULL
> +
>  /* orhpan objectid for tracking unlinked/truncated files */
(Continue reading)

Permalink | Reply | 0 comments |

Gmane