headers
Michael McLagan | 1 Jan 2010 20:57

Size limitation?

Hello,

   I'm trying to set up a large array but isn't working.  I tried 
Googling size limits, etc and came up empty.

The problem is that with 10 drives (300GB SCSI), the array is coming up 
with 500GB of space?!?  I did an experiment and when the array size 
exceeds 2TB, it fails/wraps?

Is there a solution (simple or otherwise) for this?  The machine runs 
2.6.29.5 (anything later causes the machine to lock up :( ).  I'm not 
sure if mdadm or the kernel is the problem either.

Any suggestions appreciated (please CC me on replies).

   Michael

-----

Here's the details:

]# mdadm --stop /dev/md1;mdadm --zero-superblock /dev/sd[a-j]1;mdadm --
create --level=5 --raid-devices=6 --assume-clean --auto=yes --force 
/dev/md1 /dev/sd[a-f]1;mdadm --detail /dev/md1
/dev/md1:
        Version : 0.90
  Creation Time : Fri Jan  1 14:15:42 2010
     Raid Level : raid5
     Array Size : 1464841600 (1396.98 GiB 1500.00 GB)
  Used Dev Size : 292968320 (279.40 GiB 300.00 GB)
(Continue reading)

Permalink | Reply | 12 comments |
headers
Kristleifur Daðason | 1 Jan 2010 22:28
Picon
Gravatar

Re: Size limitation?

On Fri, Jan 1, 2010 at 7:57 PM, Michael McLagan <mmclagan <at> invlogic.com> wrote:
> Hello,
>
>   I'm trying to set up a large array but isn't working.  I tried
> Googling size limits, etc and came up empty.
>
> The problem is that with 10 drives (300GB SCSI), the array is coming up
> with 500GB of space?!?  I did an experiment and when the array size
> exceeds 2TB, it fails/wraps?
>

Looked at the mdadm output and ... Wow that's strange. You aren't
supposed to be hitting any size limits there as far as I know, and I
can't see that you're doing anything wrong - The "wraparound" just
shouldn't be happening in my opinion. (But I'm a RAID rookie so don't
take that as an expert judgement. Perhaps rather as some solidarity in
a strange situation.) Anyway ...

> Is there a solution (simple or otherwise) for this?  The machine runs
> 2.6.29.5 (anything later causes the machine to lock up :( ).  I'm not
> sure if mdadm or the kernel is the problem either.
>

That's kinda scary. Is this some unusual architecture or hardware
configuration? I don't want to take this off-topic if you have
determined the kernel version to be a fixed variable, but: Do you know
why a newer kernel won't go? To me, newer-kernels-crashing indicates
an irregularity under the hood.

> Any suggestions appreciated
(Continue reading)

Permalink | Reply | 1 comment |
headers
Kristleifur Daðason | 1 Jan 2010 22:31
Picon
Gravatar

Re: Size limitation?

On Fri, Jan 1, 2010 at 9:28 PM, Kristleifur Daðason
<kristleifur <at> gmail.com> wrote:
> On Fri, Jan 1, 2010 at 7:57 PM, Michael McLagan <mmclagan <at> invlogic.com> wrote:
>> Hello,
>>
>>   I'm trying to set up a large array but isn't working.  I tried
>> Googling size limits, etc and came up empty.
>>
>> Any suggestions appreciated
>

Ah, one more thing,
also try creating an actual filesystem on the weirdly "small" array.
Who knows, the size might only be misreported to mdadm. If you do get
the correct size of filesystem, well, at least it's useful to know.
I'd be uneasy about deploying it into production in that state, but
it's good info to have.

-- Kristleifur
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
Joe Landman | 1 Jan 2010 22:32
Favicon

Re: Size limitation?

Michael McLagan wrote:
> Hello,
> 
>    I'm trying to set up a large array but isn't working.  I tried 
> Googling size limits, etc and came up empty.
> 
> The problem is that with 10 drives (300GB SCSI), the array is coming up 
> with 500GB of space?!?  I did an experiment and when the array size 
> exceeds 2TB, it fails/wraps?

Is this a 32 bit machine?

	uname -a

might help.

--

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman <at> scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
(Continue reading)

Permalink | Reply | 0 comments |
headers
Roger Heflin | 1 Jan 2010 22:42
Picon

Re: Size limitation?

Michael McLagan wrote:
> Hello,
> 
>    I'm trying to set up a large array but isn't working.  I tried 
> Googling size limits, etc and came up empty.
> 
> The problem is that with 10 drives (300GB SCSI), the array is coming up 
> with 500GB of space?!?  I did an experiment and when the array size 
> exceeds 2TB, it fails/wraps?
> 
> Is there a solution (simple or otherwise) for this?  The machine runs 
> 2.6.29.5 (anything later causes the machine to lock up :( ).  I'm not 
> sure if mdadm or the kernel is the problem either.
> 
> Any suggestions appreciated (please CC me on replies).
> 
>    Michael
> 
>

Just guessing it is a mdadm bug.

I have a 32-bit 2.6.27.5 machine with mdadm 2.6.7 and mine is showing 
an array size of 4.5tb correctly, so 2.6.29 should be just fine.

Check mdadm --version and see what version that you have of that, you 
might also mount up the array and do "cat /proc/mdstat" and see what 
size the kernel reports.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
(Continue reading)

Permalink | Reply | 0 comments |
headers
Leslie Rhorer | 2 Jan 2010 00:00
Picon
Favicon

RE: Size limitation?

> Hello,
> 
>    I'm trying to set up a large array but isn't working.  I tried
> Googling size limits, etc and came up empty.
> 
> The problem is that with 10 drives (300GB SCSI), the array is coming up
> with 500GB of space?!?  I did an experiment and when the array size
> exceeds 2TB, it fails/wraps?
> 
> Is there a solution (simple or otherwise) for this?  The machine runs
> 2.6.29.5 (anything later causes the machine to lock up :( ).  I'm not
> sure if mdadm or the kernel is the problem either.
> 
> Any suggestions appreciated (please CC me on replies).

	Well, I know the 0.90 superblock limits the COMPONENT devices to
2TB, but I know for a fact it doesn't limit the array itself to 2TB, at
least not using mdadm 2.6.7.2, because I have created arrays much larger
than 2T using a 0.90 superblock.  There is also a limit of 28 component
devices using a 0.90 superblock, but you are nowhere near that limit.
Nonetheless, why not try using a 1.x superblock, instead?  It can't hurt to
try.  What version of mdadm are you using?  I'm using a 2.6.26-2 kernel, so
I would think 2.6.29.5 would not have any limitations that 2.6.26 doesn't.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
(Continue reading)

Permalink | Reply | 0 comments |
headers
Zdenek Behan | 2 Jan 2010 00:30
Picon
Favicon

Re: Size limitation?

On 01/01/2010 08:57 PM, Michael McLagan wrote:
>    I'm trying to set up a large array but isn't working.  I tried 
> Googling size limits, etc and came up empty.
>
> The problem is that with 10 drives (300GB SCSI), the array is coming up 
> with 500GB of space?!?  I did an experiment and when the array size 
> exceeds 2TB, it fails/wraps?
>   
Just curious, did you compile your kernel with LBD (Large Block Device)
support?

Zdenek
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 6 comments |
headers
Michael Evans | 2 Jan 2010 01:46
Picon

Re: Size limitation?

On Fri, Jan 1, 2010 at 3:30 PM, Zdenek Behan <rain <at> matfyz.cz> wrote:
> On 01/01/2010 08:57 PM, Michael McLagan wrote:
>>    I'm trying to set up a large array but isn't working.  I tried
>> Googling size limits, etc and came up empty.
>>
>> The problem is that with 10 drives (300GB SCSI), the array is coming up
>> with 500GB of space?!?  I did an experiment and when the array size
>> exceeds 2TB, it fails/wraps?
>>
> Just curious, did you compile your kernel with LBD (Large Block Device)
> support?
>
>
> Zdenek
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo <at> vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

I couldn't find it in my config, so I googled:
http://cateee.net/lkddb/web-lkddb/LBD.html  That does look like the
most likely problem.

If at all possible try to get a good dmesg capture of a new kernel,
you might want to explore using netconsole to log via UDP to a remote
system.  Chances are good it's just some chipset/driver issue; I
remember having problems with a lack of timer for some boards; either
forcing it to use the older style timer or using a patched bios which
properly setup the HPET (timer) resolved the issue.
(Continue reading)

Permalink | Reply | 0 comments |
headers
fibre raid | 2 Jan 2010 07:54
Picon

Re: xfs > md 50% write performance drop on .30+ kernel?

Hi Mark,

I'm catching up on my thread-reading and saw your performance report
concerning MD on 2.6.30 (running RAID 0 at 1.7GBps) versus layering
XFS on top, which reduces performance about 50%. Having read (what I
think is) the full thread, it does not seem there was any conclusion
on this. Did you reach a conclusion to determine the cause, etc? I am
curious to see what the issue what be as I'm seeing this issue as well
on my end.

Best regards,
-T

On Tue, Oct 27, 2009 at 3:11 AM, Thomas Fjellstrom <tfjellstrom <at> shaw.ca> wrote:
> On Tue October 27 2009, Thomas Fjellstrom wrote:
>> On Wed October 14 2009, mark delfman wrote:
>> > Hi Chris... we tried the direct DD as requested and the problem is
>> > still there...
>> > 1.3GBsec > 325MBsec  (even more dromatic)... hopefully this helps
>> > narrow it down?
>> >
>> >
>> > Write > MD
>> > linux-poly:~ # dd if=/dev/zero of=/dev/md0 oflag=direct bs=1M
>> > count=20000 20000+0 records in
>> > 20000+0 records out
>> > 20971520000 bytes (21 GB) copied, 15.7671 s, 1.3 GB/s
>> >
>> >
>> > Write > XFS > MD
(Continue reading)

Permalink | Reply | 0 comments |

Gmane