headers
dexen deVries | 4 May 2011 18:02
X-Face
Picon
Gravatar

how are files allocated on nilfs?

Hi,

a possibly naive question: how are files allocated on nilfs?

Asking, because I've expected a large-ish (128MB) file to be mostly 
continuous, modulo segment headers. But it doesn't seem so. A small test to 
narrow down the question:
 0) created a new, empty, nilfs, on a separate partition (about 1GB size)
 1) created a file with dd if=/dev/zero of=a.bin bs=$((128*1024*1024)) count=1
 2) ran hdparm --fibmap a.bin
output:
 byte_offset  begin_LBA    end_LBA    sectors
         0        231      16446      16216
     8302592      16503      29094      12592
    14749696      29239      32830       3592
    16588800      32895      49214      16320
    24944640      49255      58158       8904
    29503488      58327      65598       7272
    33226752      65663      81982      16320
    41582592      82007      87230       5224
    44257280      87415      98366      10952
    49864704      98431     114750      16320
    58220544     114759     116302       1544
    59011072     116503     131134      14632
    66502656     131191     145374      14184
    73764864     145519     147518       2000
    74788864     147583     163902      16320
    83144704     163951     174446      10496
    88518656     174607     180286       5680
    91426816     180351     196670      16320
(Continue reading)

Permalink | Reply | 1 comment |
headers
Ryusuke Konishi | 4 May 2011 20:18
Picon
Gravatar

[PATCH 3/4] nilfs2: add truncation routine of segment usage file

When shrinking the filesystem, segments to be truncated must be test
if they are busy or not, and unneeded sufile block should be deleted.
This adds routines for the truncation.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/sufile.c |  112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 112 insertions(+), 0 deletions(-)

diff --git a/fs/nilfs2/sufile.c b/fs/nilfs2/sufile.c
index f4374df..22f2e6e 100644
--- a/fs/nilfs2/sufile.c
+++ b/fs/nilfs2/sufile.c
 <at>  <at>  -98,6 +98,13  <at>  <at>  nilfs_sufile_get_segment_usage_block(struct inode *sufile, __u64 segnum,
 				   create, NULL, bhp);
 }

+static int nilfs_sufile_delete_segment_usage_block(struct inode *sufile,
+						   __u64 segnum)
+{
+	return nilfs_mdt_delete_block(sufile,
+				      nilfs_sufile_get_blkoff(sufile, segnum));
+}
+
 static void nilfs_sufile_mod_counter(struct buffer_head *header_bh,
 				     u64 ncleanadd, u64 ndirtyadd)
 {
 <at>  <at>  -610,6 +617,111  <at>  <at>  void nilfs_sufile_do_set_error(struct inode *sufile, __u64 segnum,
 }
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ryusuke Konishi | 4 May 2011 20:18
Picon
Gravatar

[PATCH 4/4] nilfs2: implement resize ioctl

This adds resize ioctl which makes online resize possible.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/ioctl.c     |   27 ++++++++++++++++++
 fs/nilfs2/nilfs.h     |    1 +
 fs/nilfs2/sufile.c    |   67 +++++++++++++++++++++++++++++++++++++++++++++
 fs/nilfs2/sufile.h    |    1 +
 fs/nilfs2/super.c     |   72 +++++++++++++++++++++++++++++++++++++++++++++++++
 fs/nilfs2/the_nilfs.c |   24 +++++++++++++---
 fs/nilfs2/the_nilfs.h |    2 +
 7 files changed, 189 insertions(+), 5 deletions(-)

diff --git a/fs/nilfs2/ioctl.c b/fs/nilfs2/ioctl.c
index 6f61777..41d6743 100644
--- a/fs/nilfs2/ioctl.c
+++ b/fs/nilfs2/ioctl.c
 <at>  <at>  -698,6 +698,31  <at>  <at>  static int nilfs_ioctl_sync(struct inode *inode, struct file *filp,
 	return 0;
 }

+static int nilfs_ioctl_resize(struct inode *inode, struct file *filp,
+			      void __user *argp)
+{
+	__u64 newsize;
+	int ret = -EPERM;
+
+	if (!capable(CAP_SYS_ADMIN))
+		goto out;
+
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ryusuke Konishi | 4 May 2011 20:18
Picon
Gravatar

[PATCH 2/4] nilfs2: add routine to move secondary super block

After resizing the filesystem, the secondary super block must be moved
to a new location.  This adds a helper function for this.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/super.c |   57 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 57 insertions(+), 0 deletions(-)

diff --git a/fs/nilfs2/super.c b/fs/nilfs2/super.c
index 062cca0..eba5b53 100644
--- a/fs/nilfs2/super.c
+++ b/fs/nilfs2/super.c
 <at>  <at>  -347,6 +347,63  <at>  <at>  int nilfs_cleanup_super(struct super_block *sb)
 	return ret;
 }

+/**
+ * nilfs_move_2nd_super - relocate secondary super block
+ *  <at> sb: super block instance
+ *  <at> sb2off: new offset of the secondary super block (in bytes)
+ */
+static int nilfs_move_2nd_super(struct super_block *sb, loff_t sb2off)
+{
+	struct the_nilfs *nilfs = sb->s_fs_info;
+	struct buffer_head *nsbh;
+	struct nilfs_super_block *nsbp;
+	sector_t blocknr, newblocknr;
+	unsigned long offset;
+	int sb2i = -1;  /* array index of the secondary superblock */
+	int ret = 0;
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ryusuke Konishi | 4 May 2011 20:18
Picon
Gravatar

[PATCH 1/4] nilfs2: add ioctl which limits range of segment to be allocated

This adds a new ioctl command which limits range of segment to be
allocated.  This is intended to gather data whithin a range of the
partition before shrinking the filesystem, or to control new log
location for some purpose.

If a range is specified by the ioctl, segment allocator of nilfs tries
to allocate new segments from the range unless no free segments are
available there.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/ioctl.c         |   34 +++++++++++++++++++++
 fs/nilfs2/sufile.c        |   73 ++++++++++++++++++++++++++++++++++++++------
 fs/nilfs2/sufile.h        |    1 +
 include/linux/nilfs2_fs.h |    2 +
 4 files changed, 100 insertions(+), 10 deletions(-)

diff --git a/fs/nilfs2/ioctl.c b/fs/nilfs2/ioctl.c
index f2469ba..6f61777 100644
--- a/fs/nilfs2/ioctl.c
+++ b/fs/nilfs2/ioctl.c
 <at>  <at>  -698,6 +698,38  <at>  <at>  static int nilfs_ioctl_sync(struct inode *inode, struct file *filp,
 	return 0;
 }

+static int nilfs_ioctl_set_alloc_range(struct inode *inode, void __user *argp)
+{
+	struct the_nilfs *nilfs = inode->i_sb->s_fs_info;
+	__u64 range[2];
+	__u64 minseg, maxseg;
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ryusuke Konishi | 4 May 2011 20:18
Picon
Gravatar

[PATCH 0/4] nilfs2 resize support

This series adds (on-line) resize feature to the nilfs2 filesystem.
Both expansion and shrinking of the filesystem become possible with
this series.

Shrinking of the filesystem is realized with help from a user-land
resize tool "nilfs-resize".  It first sets a limit on the range of
segments that the nilfs kernel code allocates, then moves in-use
segments into the range by utilizing garbage collection API.

A new ioctl command is added to limit the allocation range of
segments.  And, the existent resize ioctl is finally implemented.

The nilfs-resize program is inteded to be a part of nilfs-utils (may
be incorporated into the next util release).  At present, it's only
available from "resize" branch of the following git tree:

  http://git.nilfs.org/nilfs2-utils-devel.git

   -- see http://www.nilfs.org/git/ to know how to get it.

The usage of this program is as follows:

  $ nilfs-resize [option] device [size]

If the size is larger than the current filesystem size, nilfs-resize
extends the filesystem to the given size.  If the size is smaller than
the current filesystem size, nilfs-resize tries to shrink the
filesystem to the given size.  The shrinking will fail if the
filesystem does not have enough free space.
(Continue reading)

Permalink | Reply | 4 comments |
headers
Ryusuke Konishi | 5 May 2011 03:43
Picon
Gravatar

Re: how are files allocated on nilfs?

Hi,
On Wed, 4 May 2011 18:02:16 +0200, dexen deVries wrote:
> Hi,
> 
> a possibly naive question: how are files allocated on nilfs?
> 
> Asking, because I've expected a large-ish (128MB) file to be mostly 
> continuous, modulo segment headers. But it doesn't seem so. A small test to 
> narrow down the question:
>  0) created a new, empty, nilfs, on a separate partition (about 1GB size)
>  1) created a file with dd if=/dev/zero of=a.bin bs=$((128*1024*1024)) count=1
>  2) ran hdparm --fibmap a.bin
> output:
>  byte_offset  begin_LBA    end_LBA    sectors
>          0        231      16446      16216
>      8302592      16503      29094      12592
>     14749696      29239      32830       3592
>     16588800      32895      49214      16320
>     24944640      49255      58158       8904
>     29503488      58327      65598       7272
>     33226752      65663      81982      16320
>     41582592      82007      87230       5224
>     44257280      87415      98366      10952
>     49864704      98431     114750      16320
>     58220544     114759     116302       1544
>     59011072     116503     131134      14632
>     66502656     131191     145374      14184
>     73764864     145519     147518       2000
>     74788864     147583     163902      16320
>     83144704     163951     174446      10496
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ryusuke Konishi | 5 May 2011 06:13
Picon
Gravatar

[PATCH 1/5] nilfs2: use empty_aops for gc-inodes

Applies empty_aops for address space operations of gc-inodes.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/gcinode.c |    5 +----
 1 files changed, 1 insertions(+), 4 deletions(-)

diff --git a/fs/nilfs2/gcinode.c b/fs/nilfs2/gcinode.c
index 1c2a3e2..6f0d60a 100644
--- a/fs/nilfs2/gcinode.c
+++ b/fs/nilfs2/gcinode.c
 <at>  <at>  -48,9 +48,6  <at>  <at> 
 #include "dat.h"
 #include "ifile.h"

-static const struct address_space_operations def_gcinode_aops = {
-};
-
 /*
  * nilfs_gccache_submit_read_data() - add data buffer and submit read request
  *  <at> inode - gc inode
 <at>  <at>  -178,7 +175,7  <at>  <at>  int nilfs_init_gcinode(struct inode *inode)

 	inode->i_mode = S_IFREG;
 	mapping_set_gfp_mask(inode->i_mapping, GFP_NOFS);
-	inode->i_mapping->a_ops = &def_gcinode_aops;
+	inode->i_mapping->a_ops = &empty_aops;
 	inode->i_mapping->backing_dev_info = inode->i_sb->s_bdi;

 	ii->i_flags = 0;
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ryusuke Konishi | 5 May 2011 06:13
Picon
Gravatar

[PATCH 3/5] nilfs2: get rid of NILFS_I_NILFS

This replaces all references of NILFS_I_NILFS(inode)->ns_bdev with
inode->i_sb->s_bdev and unfolds remaining uses of NILFS_I_NILFS inline
function.

Before 2.6.37, referring to a nilfs object from inodes needed a
conditional judgement, and NILFS_I_NILFS was helpful to simplify it.
But now we can simply do it by going through a super block instance
like inode->i_sb->s_fs_info.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/bmap.c    |    4 +++-
 fs/nilfs2/btnode.c  |    9 +++++----
 fs/nilfs2/gcinode.c |    8 ++++----
 fs/nilfs2/inode.c   |    8 ++++----
 fs/nilfs2/mdt.h     |    7 +------
 fs/nilfs2/sufile.c  |    4 ++--
 fs/nilfs2/sufile.h  |    2 +-
 7 files changed, 20 insertions(+), 22 deletions(-)

diff --git a/fs/nilfs2/bmap.c b/fs/nilfs2/bmap.c
index 4723f04..aadbd0b 100644
--- a/fs/nilfs2/bmap.c
+++ b/fs/nilfs2/bmap.c
 <at>  <at>  -34,7 +34,9  <at>  <at> 

 struct inode *nilfs_bmap_get_dat(const struct nilfs_bmap *bmap)
 {
-	return NILFS_I_NILFS(bmap->b_inode)->ns_dat;
+	struct the_nilfs *nilfs = bmap->b_inode->i_sb->s_fs_info;
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ryusuke Konishi | 5 May 2011 06:13
Picon
Gravatar

[PATCH 4/5] nilfs2: always set back pointer to host inode in mapping->host

In the current nilfs, page cache for btree nodes and meta data files
do not set a valid back pointer to the host inode in mapping->host.

This will change it so that every address space in nilfs uses
mapping->host to hold its host inode.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/btnode.c  |    6 ------
 fs/nilfs2/btnode.h  |    1 -
 fs/nilfs2/mdt.c     |    4 ++--
 fs/nilfs2/nilfs.h   |    6 ------
 fs/nilfs2/page.c    |   13 +++++--------
 fs/nilfs2/page.h    |    2 +-
 fs/nilfs2/segment.c |   16 +++++-----------
 fs/nilfs2/super.c   |    2 +-
 8 files changed, 14 insertions(+), 36 deletions(-)

diff --git a/fs/nilfs2/btnode.c b/fs/nilfs2/btnode.c
index 4817560..c353e4f 100644
--- a/fs/nilfs2/btnode.c
+++ b/fs/nilfs2/btnode.c
 <at>  <at>  -34,12 +34,6  <at>  <at> 
 #include "page.h"
 #include "btnode.h"

-void nilfs_btnode_cache_init(struct address_space *btnc,
-			     struct backing_dev_info *bdi)
-{
-	nilfs_mapping_init(btnc, bdi);
(Continue reading)

Permalink | Reply | 0 comments |

Gmane