headers
Dave Kleikamp | 1 Feb 2007 16:13
Picon

Re: Fragmentation and poor write speeds.

On Wed, 2007-01-31 at 15:30 -0500, Jason Fisher wrote:
> $ > dd if=/dev/zero of=test bs=1024k count=1000
> 1048576000 bytes (1.0 GB) copied, 168.072 s, 6.2 MB/s
> 
> I've tested all variations between 1024 - 1024k with 1-4GB of
> data--doesn't really matter--it's like hitting a wall.

My bad.  Only O_DIRECT writes allocate in bigger chunks.  Normal writes
are still allocated a page at a time.  This is something I want to
address when I get the time.

> You may be right on the 4095 blocksize.  I checked my history and it
> was a bs=10k, count=1 that resulted in 2-3 extents being used every
> 10th write.  But that still means 10% of the time, this jfs couldn't
> find three contiguous blocks on a drive with 800GB+ available. 

So it's not that jfs couldn't find 3 contiguous blocks, but that it
really doesn't look for them.  It's allocating one block at a time, and
will grow into contiguous space if it can.

>  The
> main use was for MythTV, but I did back up a few other systems on to
> it (without tarring first) at one point.  If I created 300,000 smaller
> files and let Myth fill the drive a few times over, I suppose I could
> see how this may snowball.
> 
> I do agree that a defrag tool designed for jfs would probably solve
> this.  Either way, your work on jfs is appreciated and it served me
> well for some time.  I'll be keeping an eye on the project in
> anticipation of a proper defrag tool.
(Continue reading)

Permalink | Reply | 0 comments |
headers
f-jfs | 8 Feb 2007 07:09
Picon
Favicon

files spontaneously vanishing, then reappearing

I just noticed a peculiar problem, which has happened at least twice
in the last 10 weeks---a file that -was- in the filesystem apparently
vanished, then reappeared.  I'm looking for advice on how to debug
this or research it.  [Also, see PS for a slightly different problem
that might or might not be related.]

I'm using the JFS that comes out of the box with Ubuntu Breezy, which
is kernel 2.6.12-10-386.  The filesystem is in a normal partition, NOT
mounted over LVM, and it's not the root filesystem.  It's exported rw
via NFS, but both times a file vanished & reappeared, nothing but the
local host was accessing that filesystem (and that had been true for
many minutes or hours; only one other host can touch that filesystem
anyway).

I noticed the problem because I have some automation that's actually a
large tcsh script.  It always runs with echo & verbose, and logs all
its output to a file.  It tried to remove a file (call it FOO) simply
via
  "rm /blah/biff/FOO"
and got
  rm: cannot remove `/blah/biff/FOO': No such file or directory

I noticed the problem several days later, but FOO is actually there.
To ensure I wasn't misreading something, I marked the entire pathname
(the actual rm used a rooted pathname, e.g., "rm /blah/biff/FOO") in
Emacs and, in a shell buffer, did "ls -alF /blah/biff/FOO" by yanking
the marked pathname.  It shows the file.  "head -c 5 /blah/biff/FOO"
gives me the first five bytes.  Thus, I'm 100% sure that the pathname
the script tried to use and failed with, and the pathname I'm now
using, are the same pathname.
(Continue reading)

Permalink | Reply | 3 comments |
headers
Dave Kleikamp | 8 Feb 2007 20:01
Picon

Re: files spontaneously vanishing, then reappearing

On Thu, 2007-02-08 at 01:09 -0500, f-jfs <at> media.mit.edu wrote:
> I just noticed a peculiar problem, which has happened at least twice
> in the last 10 weeks---a file that -was- in the filesystem apparently
> vanished, then reappeared.  I'm looking for advice on how to debug
> this or research it.  [Also, see PS for a slightly different problem
> that might or might not be related.]
> 
> I'm using the JFS that comes out of the box with Ubuntu Breezy, which
> is kernel 2.6.12-10-386.  The filesystem is in a normal partition, NOT
> mounted over LVM, and it's not the root filesystem.  It's exported rw
> via NFS, but both times a file vanished & reappeared, nothing but the
> local host was accessing that filesystem (and that had been true for
> many minutes or hours; only one other host can touch that filesystem
> anyway).
> 
> I noticed the problem because I have some automation that's actually a
> large tcsh script.  It always runs with echo & verbose, and logs all
> its output to a file.  It tried to remove a file (call it FOO) simply
> via
>   "rm /blah/biff/FOO"
> and got
>   rm: cannot remove `/blah/biff/FOO': No such file or directory
> 
> I noticed the problem several days later, but FOO is actually there.
> To ensure I wasn't misreading something, I marked the entire pathname
> (the actual rm used a rooted pathname, e.g., "rm /blah/biff/FOO") in
> Emacs and, in a shell buffer, did "ls -alF /blah/biff/FOO" by yanking
> the marked pathname.  It shows the file.  "head -c 5 /blah/biff/FOO"
> gives me the first five bytes.  Thus, I'm 100% sure that the pathname
> the script tried to use and failed with, and the pathname I'm now
(Continue reading)

Permalink | Reply | 2 comments |
headers
Liam Carey | 8 Feb 2007 20:52

re : JFS filesystem


 Hello -

 

 We are running RedHat v4 Nahant Update 4 (kernel : 2.6.9-42.0.3.ELsmp ) on an IBM Blade.

 We would like to use JFS to frequently write 5-10 million small files on a 500GB filesystem and I would like to
hear advice/suggestions regarding the performance and if necessary any system tweaks to get good
results. For example : Is fragmentation a problem? What kind of read/write speeds have been seen?
 
 You help and/or suggestions are very appreciated!

 Regards,

 Liam Carey 
 Oakland, CA

The information contained in this message may be privileged and/or confidential. If you are not the
intended recipient, or responsible for delivering this message to the intended recipient, any review,
forwarding, dissemination, distribution or copying of this communication or any attachment(s) is
strictly prohibited. If you have received this message in error, please so notify the sender
immediately, and delete it and all attachments from your computer and network.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
(Continue reading)

Permalink | Reply | 1 comment |
headers
Christian Kujau | 9 Feb 2007 14:56
Picon

Re: re : JFS filesystem


On Thu, February 8, 2007 19:52, Liam Carey wrote:
> We are running RedHat v4 Nahant Update 4 (kernel : 2.6.9-42.0.3.ELsmp
> ) on an IBM Blade.

wow, 2.6.9 was released 10/2004 - I wonder how much they backport to
these vendor kernels...

> We would like to use JFS to frequently write 5-10 million small files
> on a 500GB filesystem and I would like to hear advice/suggestions
> regarding the performance and if necessary any system tweaks to get
> good results.

there are not many tweaks when creating the fs. Putting the log on an
external device (mkfs.jfs -J device=...) sounds like a good idea when
speed is critical. But in any case: before going in production I'd
suggest to test your application with different setups (raid levels,
filesystems) to get relvant details for *your* workload.

> For example : Is fragmentation a problem? What kind of
> read/write speeds have been seen?

fragmentation seems to be an issue, see the archives, e.g.
"Fragmentation and poor write speeds." thread a few days ago.

Christian.
--

-- 
BOFH excuse #36:

dynamic software linking table corrupted
(Continue reading)

Permalink | Reply | 0 comments |
headers
f-jfs | 9 Feb 2007 21:50
Picon
Favicon

files spontaneously vanishing, then reappearing

    > Date: Thu, 08 Feb 2007 13:01:48 -0600
    > From: Dave Kleikamp <shaggy <at> linux.vnet.ibm.com>

    > I guess you know the situation well enough to rule out things like a
    > process recreating a same-named file at some later point, or some
    > component of the path being renamed temporarily and then renamed back.

Yup.  That part of the automation is very simple---machine A generates
a file with a unique timestamp (and its clock -never- runs backwards).
Machine B pulls the file over via NFS using "cp /a/foo/bar/timestamp
/blah/biff/timestamp_human-readable-name".  The file then sits in B's
JFS for a minimum of 30 minutes while it's being processed (and often
a few days).  When processing is done, it's deleted.

Nothing renames it elsewhere, and A can't generate the same filename twice.

    > true > /blah/biff/FOO should give you the space back without wiping out
    > the file itself.

Indeed.  I deleted it anyway.

    > I would agree that nfs probably isn't a factor.   Again, it's not
    > possible that the script at some later point replaced the missing file,
    > is it?

Nope.  Alas---it'd be the nicest explanation.

    > > I have -not- tried running "fsck.jfs -n -v" yet, nor am I sure that's
    > > even the thing to try.  I may try that in a few hours, when I have a
    > > window during which I can unmount it for a few minutes.
(Continue reading)

Permalink | Reply | 1 comment |
headers
f-jfs | 10 Feb 2007 02:29
Picon
Favicon

files spontaneously vanishing, then reappearing

    > Date: Fri, 9 Feb 2007 15:50:23 -0500 (EST)
    > From: f-jfs <at> media.mit.edu

    > All of them were of this flavor, e.g., blocks marked allocated that
    > weren't actually in any file (fortunate that none of them were the
    > reverse!).  The repair reclaimed about 3G of data from a 175G FS
    > (about 2%).  I didn't -thoroughly- check, but it looks as if no files
    > either vanished or appeared as a result of the check.  The last lines
    > of the log were (before the summary statistics):

Btw, the vast majority of files created in this filesystem are large (1-5GB).
A small number of much smaller files get created and, for all practical purposes, 
never deleted.  So it's conceivable that the ~3GB of misallocated space was the
fallout of a single large file getting lost, or one ~1G and one ~2G (the sizes
of files I'm writing tend to be quantized in around 1G steps).

Also:

      Incorrect leaf (485) value detected in L0 page 1.
      Incorrect leaf (489) value detected in L0 page 1.
      Incorrect leaf (491) value detected in L0 page 1.
      Incorrect number of free blocks detected in Block Map Control Page.
      Incorrect number of free blocks in AG 14 detected in Block Map Control Page.
      Incorrect number of free blocks in AG 15 detected in Block Map Control Page.
      Incorrect number of free blocks in AG 17 detected in Block Map Control Page.
      Incorrect number of free blocks in AG 18 detected in Block Map Control Page.
      Descrepancies detected between observed block allocations and pmaps.
-------^

This error message is misspelled.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Joselyn Schwing | 22 Feb 2007 16:44

at tranquilliz

Hi,

Via_aagra $3. 35
Va_aalium $1. 25
Cia_aalis $3. 75
Xan_nnax
Som_mma

http://www. kedrx .com

Remove spaces in the above link

It is too eavy, all zis Ogwarts food,  they heard her saying grumpily
as they left the Great Hall behind her one evening (Ron skulking behind
Harry, keen not to be spotted by Fleur). I will not fit into my dress

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Permalink | Reply | 0 comments |
headers
Ronald Wimmer | 28 Feb 2007 23:57
Picon

Data Recovery (JFS/LVM)

Does anyone know a possibility to recover data from a logical volume 
(LVM2) using JFS?

The problem is that the logical volume spans across 3 harddisks and one 
of them is dead ... is there a possibility to get the logical volume to 
work again without the third disk?

Thank you very much in advance for your answers!

Regards,
Ronald

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Permalink | Reply | 1 comment |

Gmane