headers
Poul-Henning Kamp | 1 Jan 2010 12:26
Picon
Favicon

Re: File system blocks alignment

In message <200912311712.18347.pieter <at> degoeje.nl>, Pieter de Goeje writes:

>Ok, as Miroslav wrote, it does not report 4k sectors:

If you care to, and have the time, try the following:

for N in (0...7)
	for M in (0..2)
		create partition starting at byte offset N*512
		newfs -f 4096 -b 32768 /dev/foo
		mount /dev/foo /mnt
		time restore(8) a filesystem into it.
		unmount /mnt
		time fsck_ffs /dev/foo
		mount /dev/foo /mnt
		time tar cf /dev/null  /mnt
		unmount /mnt

--

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk <at> FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 19 comments |
headers
Poul-Henning Kamp | 1 Jan 2010 23:47
Picon
Favicon

Re: File system blocks alignment

In message <201001012153.44349.pieter <at> degoeje.nl>, Pieter de Goeje writes:

>That yielded some pretty spectacular results. [...]
>
>Performance for restore was abysmal in the unaligned case, easily being 10 
>times slower than aligned restore. Newfs was about 5 times as slow.

That is what I expected, only I didn't expect a factor 14 in performance.

I'm not surprised that newfs and restore take the biggest hits in that
test, those are the hard ones, seen from the disk drive, all the read
only work can be cached and "covered up" that way.

Ideally, newfs/UFS should do a quick test to look for any obvious
boundaries, and DTRT, a nice little task for somebody :-)

Poul-Henning

PS: The reason I asked for 3 iterations, was so we could calculate
a standard deviation (See: ministat(8)) in order to have a statistical
sound conclusion.   With a factor 14 in time difference, I will for
once conceede it unnecessary :-)

--

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk <at> FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
(Continue reading)

Permalink | Reply | 16 comments |
headers
Pieter de Goeje | 2 Jan 2010 01:01
Picon
Gravatar

Re: File system blocks alignment

On Friday 01 January 2010 23:47:12 Poul-Henning Kamp wrote:
> In message <201001012153.44349.pieter <at> degoeje.nl>, Pieter de Goeje writes:
> >That yielded some pretty spectacular results. [...]
> >
> >Performance for restore was abysmal in the unaligned case, easily being 10
> >times slower than aligned restore. Newfs was about 5 times as slow.
>
> That is what I expected, only I didn't expect a factor 14 in performance.

I'm trying to think of reasons why it performs so poorly, because even in 
the "write covers two sectors partially" case a read-modify-write cycle 
shouldn't mess performance up so badly, considering the drive's huge 64MB 
cache. Maybe the firmware is just not that smart.
>
> I'm not surprised that newfs and restore take the biggest hits in that
> test, those are the hard ones, seen from the disk drive, all the read
> only work can be cached and "covered up" that way.
>
> Ideally, newfs/UFS should do a quick test to look for any obvious
> boundaries, and DTRT, a nice little task for somebody :-)

A search for the offset for which newfs (or a simpler test) runs fastest? 
Interesting idea :-)

Technically, the drive's at fault here because it should've reported 4K 
sectors. Perhaps there should be some kind of quirks table :-S for disk 
drives and/or a sectorsize override knob. Or maybe simply selecting a large 
enough power-of-two boundary suffices. That could also be done by gpart 
instead of (or in addition to) newfs.
(Continue reading)

Permalink | Reply | 7 comments |
headers
Ulrich Spörlein | 2 Jan 2010 06:08

Re: File system blocks alignment

On Fri, 01.01.2010 at 22:47:12 +0000, Poul-Henning Kamp wrote:
> In message <201001012153.44349.pieter <at> degoeje.nl>, Pieter de Goeje writes:
> 
> >That yielded some pretty spectacular results. [...]
> >
> >Performance for restore was abysmal in the unaligned case, easily being 10 
> >times slower than aligned restore. Newfs was about 5 times as slow.
> 
> That is what I expected, only I didn't expect a factor 14 in performance.
> 
> I'm not surprised that newfs and restore take the biggest hits in that
> test, those are the hard ones, seen from the disk drive, all the read
> only work can be cached and "covered up" that way.
> 
> Ideally, newfs/UFS should do a quick test to look for any obvious
> boundaries, and DTRT, a nice little task for somebody :-)

Indeed, but newfs is only one small part of the puzzle. Think about
zpools and, more importantly swap partitions. Sysinstall, fdisk, gpart
and bsdlabel should all display some fat warning if partition/label
alignment is not, say at 256kB (a common stripe size, right?) and also
automatically generate that offset if the user uses automatic settings.

But then again, this is all wishful thinking from a users perspective :)

Regards,
Uli
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
(Continue reading)

Permalink | Reply | 5 comments |
headers
Poul-Henning Kamp | 2 Jan 2010 10:42
Picon
Favicon

Re: File system blocks alignment

In message <20100102050843.GI3508 <at> acme.spoerlein.net>, Ulrich =?utf-8?B?U3DDtnJ
sZWlu?= writes:

>Sysinstall, fdisk, gpart
>and bsdlabel should all display some fat warning if partition/label
>alignment is not, say at 256kB (a common stripe size, right?) 

You overlook that MBR/Fdisk requires bootable slices to start at a
"track".  That means that the propper slice-alignmen typically
will be 8*63=504 sectors.

Unless you want to explore how many BIOS'es still are stupid about
this...

--

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk <at> FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 4 comments |
headers
Poul-Henning Kamp | 2 Jan 2010 23:37
Picon
Favicon

Re: File system blocks alignment

In message <4B3FC8A2.1090901 <at> elischer.org>, Julian Elischer writes:
>Poul-Henning Kamp wrote:
>> You overlook that MBR/Fdisk requires bootable slices to start at a
>> "track".  That means that the propper slice-alignmen typically
>> will be 8*63=504 sectors.
>
>No it doesn't, (or at least it didn't) but it has become custom to do so.

Yes it does, for all slices not starting on the first head.

We've been over this maddness in the past multiple times.

If somebody is willing to suffer this breakage on funky old bios'es,
betting that most of those systems will never run FreeBSD-9, I would
say they are being emminently sensible, but still have their work
cut out for them.

--

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk <at> FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 1 comment |
headers
FreeBSD bugmaster | 4 Jan 2010 12:06
Picon
Favicon

Current problem reports assigned to freebsd-arch <at> FreeBSD.org

Note: to view an individual PR, use:
  http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.

S Tracker      Resp.      Description
--------------------------------------------------------------------------------
o kern/120749  arch       [request] Suggest upping the default kern.ps_arg_cache

1 problem total.

_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 0 comments |
headers
M. Warner Losh | 4 Jan 2010 22:40

Re: svn commit: r201534 - in head/sys: amd64/conf arm/conf i386/conf ia64/conf mips/conf pc98/conf powerpc/conf sparc64/conf sun4v/conf

Doug (with arch <at>  cc'd)

I'm sorry that I didn't notice this sooner and comment upon it.  It is
a really bad idea for a lot of reasons..

The DEFAULTS file is not for optional items.  It is only for items
that are required for all (or almost all) users of the system to build
a working kernel.  Including the config file isn't anywhere near
required.  In addition, changing it on a stable branch seems like a
big default change.  Finally, we're planning on deprecating this file
entirely in the future, so no further changes should be made to it
without discussion.

I've reverted this after chatting on IRC with bz <at>  and ed <at> .

I'm posting this to arch <at>  so that the followup discussion can take
place.  *IF* there is wide-spread consensus to add it back, I'll take
care of that leg work (as well as anything I've overlooked in the
back out).  But I don't think there's wide-spread support for this in
DEFAULTS...

Warner

In message: <201001042130.o04LU4ln085140 <at> svn.freebsd.org>
            Warner Losh <imp <at> FreeBSD.org> writes:
: Author: imp
: Date: Mon Jan  4 21:30:04 2010
: New Revision: 201534
: URL: http://svn.freebsd.org/changeset/base/201534
:
(Continue reading)

Permalink | Reply | 3 comments |
headers
Doug Barton | 4 Jan 2010 23:17
Picon
Favicon

Re: svn commit: r201534 - in head/sys: amd64/conf arm/conf i386/conf ia64/conf mips/conf pc98/conf powerpc/conf sparc64/conf sun4v/conf

I originally proposed adding it to GENERIC because my sentiment was
the same as yours on this. After posting the patch for review I was
told by several people (whom I felt were better acquainted with sys/
than I) to add it to DEFAULTS instead, so I changed the patch and
committed it, then MFC'ed it after an appropriate window.

As far as whether to add it to "the default kernel configuration"
(leaving aside what file it should go into for now) the discussion
about that happened on -current in the vicinity of 12/13 where it
received what I felt was pretty good support. Given that disk space is
practically free on modern systems, and given the overwhelming utility
of having at least the basic (without comments) config file included
in the kernel for people who need to do real sysadmin work, I regard
the question of whether or not it should go in as settled. (In fact,
as I said in the thread I thought it had been done a long time ago.) I
am however willing to wait a couple days for those on this list to
offer an incredibly compelling reason why this would be a bad thing.

My preference would actually be that it be added to GENERIC. I'll take
you up on your offer and let you add it there if you have the time to
do so. If you don't and I get to it first, no worries. Either way, I
do want it done in time to be included in 7.3-RELEASE.

In regards to your backing out the change without discussing it with
me first, in this particular case there is no harm done, and
ultimately I agree with your assessment that it's overwhelmingly
likely that the end result will be that the change in DEFAULTS will
not stay. However, this situation is far from an emergency (given that
it had been the way it was for so long). Should a similar situation
arise in the future as a general course of action I would prefer that
(Continue reading)

Permalink | Reply | 1 comment |
headers
M. Warner Losh | 4 Jan 2010 23:54

Re: svn commit: r201534 - in head/sys: amd64/conf arm/conf i386/conf ia64/conf mips/conf pc98/conf powerpc/conf sparc64/conf sun4v/conf

In message: <4B4268EB.3040905 <at> FreeBSD.org>
            Doug Barton <dougb <at> freebsd.org> writes:
: I originally proposed adding it to GENERIC because my sentiment was
: the same as yours on this. After posting the patch for review I was
: told by several people (whom I felt were better acquainted with sys/
: than I) to add it to DEFAULTS instead, so I changed the patch and
: committed it, then MFC'ed it after an appropriate window.
:
: As far as whether to add it to "the default kernel configuration"
: (leaving aside what file it should go into for now) the discussion
: about that happened on -current in the vicinity of 12/13 where it
: received what I felt was pretty good support. Given that disk space is
: practically free on modern systems, and given the overwhelming utility
: of having at least the basic (without comments) config file included
: in the kernel for people who need to do real sysadmin work, I regard
: the question of whether or not it should go in as settled. (In fact,
: as I said in the thread I thought it had been done a long time ago.) I
: am however willing to wait a couple days for those on this list to
: offer an incredibly compelling reason why this would be a bad thing.

Yes.  I missed the original discussion in -current until today.  I'm
sorry about that.  -current is a high volume list and I sometimes
overlook things...

: My preference would actually be that it be added to GENERIC. I'll take
: you up on your offer and let you add it there if you have the time to
: do so. If you don't and I get to it first, no worries. Either way, I
: do want it done in time to be included in 7.3-RELEASE.

I'll move the stuff to GENERIC (and other config files as appropriate
(Continue reading)

Permalink | Reply | 0 comments |

Gmane