Ryusuke Konishi | 2 Apr 03:49 2011
Picon

Re: checkpoints to often

On Thu, 31 Mar 2011 21:19:27 +0200, "krzf83@... " wrote:
> There should be a way to make automatic checkpoints less frequent. On
> the busy ssd I have about 7 cp every second. If it were for example
> once every 5 minutes then one could set garbage removal every few
> days.

If your application does't issue fsync, this can be done by increasing
"commit-interval" and "watermark" parameters:

The commit-interval is creation cycle of checkpoints while there is
some change.  The watermark is a threshold count of dirty data blocks.
When the number of dirty blocks exceeds this threshold, a checkpoint
creation will be triggered.

These parameters are adjustable by "nilfs-tune" tool which is included
in the recent nilfs-utils.  For instance, you can change the
commit-interval to 60 seconds by

  # nilfs-tune -i 60 /dev/sdb1

where /dev/sdb1 is the target nilfs partition.

And, watermark is ajustable by

  # nilfs-tune -m <watermark> /dev/sdb1

See manpage of nilfs-tune for details.

> Also garbage removal should have option to clean only if like 90% of
> drive is used. there is no need to remove cp-s when there is still
(Continue reading)

Ryusuke Konishi | 2 Apr 04:47 2011
Picon

[PATCH] nilfs2: fix data loss in mmap page write for hole blocks

Please consider including the attached patch to stable trees between
2.6.30 and 2.6.37.

This is the modified version of the following commit which is already
in upstream.

 commit: 34094537943113467faee98fe67c8a3d3f9a0a8b
 From: Ryusuke Konishi <konishi.ryusuke at lab.ntt.co.jp>
 Date: Sun, 27 Mar 2011 22:50:49 +0900
 Subject: [PATCH] nilfs2: fix data loss in mmap page write for hole blocks

This commit is applicable to 2.6.38, but is not to prior kernels as
is.  It will cause a build error against older kernels due to a change
of nilfs_set_file_dirty function.

I tested the attached patch for every kernel through from 2.6.30 to
2.6.37.  It was all right except that 3-way merge was required for
2.6.30 and 2.6.31.

Thanks,
Ryusuke Konishi

--
From: Ryusuke Konishi <konishi.ryusuke@...>

nilfs2: fix data loss in mmap page write for hole blocks

From the result of a function test of mmap, mmap write to shared pages
turned out to be broken for hole blocks.  It doesn't write out filled
blocks and the data will be lost after umount.  This is due to a bug
(Continue reading)

Alessandro Salvatori | 3 Apr 22:49 2011
Picon

nilfs_cleanerd could use some sensible defaults if its conf file is empty...

Hello,

    my root nilfs2 filesystem filled and trying and editing
/etc/nilfs_cleanerd.conf resulted in an empty file.

I started the cleanerd but it wasn't freeing up any space.
Then I converted all of my snapshots back onto checkpoints and removed
all of my checkpoints.

still nothing.

Then I created a new nilfs_cleanerd.conf with sensible values in my
home directory and pointed nilfs_cleanerd to that.

and finally it worked!!!

maybe nilfs_cleanerd could default to some minimum of -say- the least
between some small amount of gigabytes and some small percentage of
the disk size as the minimum space to free if no setting is found in
the configuration file?

thank you!
-alessandro-
 Here i am, A young man,
 A crashing computer program,
 Here is a pen, write out my name.
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
(Continue reading)

Ryusuke Konishi | 4 Apr 10:06 2011
Picon

[PATCH] nilfs2: get rid of private page allocator

Previously, nilfs was cloning pages for mmapped region to freeze their
data and ensure consistency of checksum during writeback cycles.  A
private page allocator was used for this page cloning.  But, we no
longer need to do that since clear_page_dirty_for_io function sets up
pte so that vm_ops->page_mkwrite function is called right before the
mmapped pages are modified and nilfs_page_mkwrite function can safely
wait for the pages to be written back to disk.

So, this stops making a copy of mmapped pages during writeback, and
eliminates the private page allocation and deallocation functions from
nilfs.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/file.c    |    1 -
 fs/nilfs2/page.c    |   53 +-----------------
 fs/nilfs2/page.h    |    4 -
 fs/nilfs2/segbuf.c  |   12 ----
 fs/nilfs2/segment.c |  153 ++++++---------------------------------------------
 fs/nilfs2/segment.h |    2 -
 6 files changed, 18 insertions(+), 207 deletions(-)

diff --git a/fs/nilfs2/file.c b/fs/nilfs2/file.c
index 397e732..d7eeca6 100644
--- a/fs/nilfs2/file.c
+++ b/fs/nilfs2/file.c
 <at>  <at>  -111,7 +111,6  <at>  <at>  static int nilfs_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
 	nilfs_transaction_commit(inode->i_sb);

  mapped:
(Continue reading)

Ryusuke Konishi | 4 Apr 12:00 2011
Picon

Re: nilfs_cleanerd could use some sensible defaults if its conf file is empty...

On Sun, 3 Apr 2011 13:49:47 -0700, Alessandro Salvatori wrote:
> Hello,
> 
>     my root nilfs2 filesystem filled and trying and editing
> /etc/nilfs_cleanerd.conf resulted in an empty file.
> 
> I started the cleanerd but it wasn't freeing up any space.
> Then I converted all of my snapshots back onto checkpoints and removed
> all of my checkpoints.
> 
> still nothing.

Did you try '-p' option?

nilfs_cleanerd tries to protect recent checkpoints within the period
specified with "protection_period" parameter.

It is set to one hour by default.  So, the effect of converting
snapshots back to checkpoints will not appear soon.

But, you can shorten this period by using the '-p' option for
emergency.

> Then I created a new nilfs_cleanerd.conf with sensible values in my
> home directory and pointed nilfs_cleanerd to that.
> 
> and finally it worked!!!
> 
> maybe nilfs_cleanerd could default to some minimum of -say- the least
> between some small amount of gigabytes and some small percentage of
(Continue reading)

Alessandro Salvatori | 4 Apr 20:16 2011
Picon

Re: nilfs_cleanerd could use some sensible defaults if its conf file is empty...

of course i had already tried -p 0 and also -p 1, in case 0 had some
special meaning ^_^

thank you!
-alessandro-
 Here i am, A young man,
 A crashing computer program,
 Here is a pen, write out my name.

On Mon, Apr 4, 2011 at 03:00, Ryusuke Konishi <ryusuke@...> wrote:
> On Sun, 3 Apr 2011 13:49:47 -0700, Alessandro Salvatori wrote:
>> Hello,
>>
>>     my root nilfs2 filesystem filled and trying and editing
>> /etc/nilfs_cleanerd.conf resulted in an empty file.
>>
>> I started the cleanerd but it wasn't freeing up any space.
>> Then I converted all of my snapshots back onto checkpoints and removed
>> all of my checkpoints.
>>
>> still nothing.
>
> Did you try '-p' option?
>
> nilfs_cleanerd tries to protect recent checkpoints within the period
> specified with "protection_period" parameter.
>
> It is set to one hour by default.  So, the effect of converting
> snapshots back to checkpoints will not appear soon.
>
(Continue reading)

Greg KH | 11 Apr 21:04 2011

Re: [PATCH] nilfs2: fix data loss in mmap page write for hole blocks

On Sat, Apr 02, 2011 at 11:47:11AM +0900, Ryusuke Konishi wrote:
> Please consider including the attached patch to stable trees between
> 2.6.30 and 2.6.37.

Thanks, I've used this version for the .32 and .33-stable trees.

greg k-h

_______________________________________________
stable mailing list
stable <at> linux.kernel.org
http://linux.kernel.org/mailman/listinfo/stable

Ryusuke Konishi | 16 Apr 07:16 2011
Picon

[ANNOUNCE] NILFS utils 2.0.22 released

Hi folks,

nilfs-utils-2.0.22 is now available at:

  http://www.nilfs.org/en/download.html

This release eliminated the need for .nilfs file and supported `block
count per checkpoint' feature which will become available on the
upcoming kernel (2.6.39).

mkfs.nilfs2 still creates the .nilfs file, but the cleaner daemon and
other nilfs tools no longer require it.  This resolves problems that
occurs on the partitions where .nilfs was deleted, especially a stuck
problem on a disk full condition.

See changelog ( http://www.nilfs.org/download/ChangeLog-utils-v2 )
for details.

Thanks,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Ryusuke Konishi | 19 Apr 19:27 2011
Picon

[PATCH] nilfs2: zero fill unused portion of super root block

The super root block is newly-allocated each time it is written back
to disk, so unused portion of the block should be cleared.

Signed-off-by: Ryusuke Konishi <konishi.ryusuke@...>
---
 fs/nilfs2/inode.c         |   10 ++++++++++
 fs/nilfs2/segment.c       |    2 ++
 include/linux/nilfs2_fs.h |    1 +
 3 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/fs/nilfs2/inode.c b/fs/nilfs2/inode.c
index 2cc8c08..699170e 100644
--- a/fs/nilfs2/inode.c
+++ b/fs/nilfs2/inode.c
 <at>  <at>  -596,6 +596,16  <at>  <at>  void nilfs_write_inode_common(struct inode *inode,
 	raw_inode->i_flags = cpu_to_le32(ii->i_flags);
 	raw_inode->i_generation = cpu_to_le32(inode->i_generation);

+	if (NILFS_ROOT_METADATA_FILE(inode->i_ino)) {
+		struct the_nilfs *nilfs = inode->i_sb->s_fs_info;
+
+		/* zero-fill unused portion in the case of super root block */
+		raw_inode->i_xattr = 0;
+		raw_inode->i_pad = 0;
+		memset((void *)raw_inode + sizeof(*raw_inode), 0,
+		       nilfs->ns_inode_size - sizeof(*raw_inode));
+	}
+
 	if (has_bmap)
 		nilfs_bmap_write(ii->i_bmap, raw_inode);
(Continue reading)

Ivan Telichko | 22 Apr 20:27 2011
Picon

Re: Regarding ``problem with nilfs_cleanerd - part 2'' and ``Nilfs_cleanerd err=-17''

Hi,

Same problem here, nilfs_cleanerd dies with these messages in system log:

  Apr 23 15:24:11 : nilfs_ioctl_move_inode_block: conflicting data buffer:
    ino=97606, cno=281, offset=14, blocknr=3526535, vblocknr=1555981
  Apr 23 15:24:11 : NILFS: GC failed during preparation: cannot read source
    blocks: err=-17

It happened after I had to reboot with Alt+SysRq (system suddenly locked up).
Then I booted SysRescueCD, mounted nilfs partition and found it filled up, but
nilfs_cleanerd was working. I left it working for some time, it managed to free
about 1.3Gb, but then died with error above. After that it does not start at
all. Removing all checkpoints changed nothing. Partition works fine, but space
is not freed anymore.

There were unusual messages in the system log just before forced reboot:

  Apr 22 09:26:39 : nilfs_palloc_freev: entry number 5381610 already freed
  Apr 22 09:26:39 : nilfs_palloc_freev: entry number 5381611 already freed
  Apr 22 09:26:39 : nilfs_palloc_freev: entry number 5381612 already freed
  Apr 22 09:26:41 : nilfs_sufile_do_free: segment 2342 is already clean
  Apr 22 09:26:41 : nilfs_sufile_do_free: segment 2343 is already clean
  Apr 22 09:26:41 : nilfs_sufile_do_free: segment 2344 is already clean
  Apr 22 09:26:41 : nilfs_sufile_do_free: segment 2345 is already clean
  ...same thing goes on for about 7 minutes, then system dies.

I am using nilfs-utils 2.0.21 on linux-2.6.32-gentoo-r7 with nilfs2 compiled in
the kernel, nilfs on root partition, with initramfs. Also, there were two
instances of nilfs_cleanerd started on system boot. I thought it was strange,
(Continue reading)


Gmane