headers
amjad ali | 1 Dec 2009 07:14
Picon

Re: MPI Processes + Auto Vectorization

Hi,
perhaps I could not better ask my question.

My question is that if we do not have free cpu cores in a PC or cluster (all cores are running MPI processes), still the auto-vertorization is beneficial? Or it is beneficial only if we have some free cpu cores locally?

thanks



On Mon, Nov 30, 2009 at 5:50 PM, David N. Lombard <dnlombar <at> ichips.intel.com> wrote:
On Mon, Nov 30, 2009 at 01:24:34PM -0700, amjad ali wrote:
> Hi,
> Suppose we run a parallel MPI code with 64 processes on a cluster, say of 16 nodes. The cluster nodes has multicore CPU say 4 cores on each node.
>
> Now all the 64 cores on the cluster running a process. Program is SPMD, means all processes has the same workload.
>
> Now if we had done auto-vectorization while compiling the code (for example with Intel compilers); Will there be any benefit (efficiency/scalability improvement) of having code with the auto-vectorization? Or we will get the same performance as without Auto-vectorization in this example case?
>
> How can we really get benefit in performance improvement with Auto-Vectorization?

Vectorization takes advantage of the processor's vector instructions to increase data-level parallelism.
How much that benefits your code depends very much on your code; you would need to recompile your code and test.

--
David N. Lombard, Intel, Irvine, CA
I do not speak for Intel Corporation; all comments are strictly my own.

<div>
<p class="MsoNormal">Hi,<br>
perhaps I could not better ask my question.<br><br>
My question is that if we do not have free cpu cores in a PC or cluster (all
cores are running MPI processes), still the auto-vertorization is beneficial?
Or it is beneficial only if we have some free cpu cores locally? <br><br>
thanks</p>

<br><br><div class="gmail_quote">On Mon, Nov 30, 2009 at 5:50 PM, David N. Lombard <span dir="ltr">&lt;<a href="mailto:dnlombar <at> ichips.intel.com">dnlombar <at> ichips.intel.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">
<div>
<div></div>
<div class="h5">On Mon, Nov 30, 2009 at 01:24:34PM -0700, amjad ali wrote:<br>
&gt; Hi,<br>
&gt; Suppose we run a parallel MPI code with 64 processes on a cluster, say of 16 nodes. The cluster nodes has multicore CPU say 4 cores on each node.<br>
&gt;<br>
&gt; Now all the 64 cores on the cluster running a process. Program is SPMD, means all processes has the same workload.<br>
&gt;<br>
&gt; Now if we had done auto-vectorization while compiling the code (for example with Intel compilers); Will there be any benefit (efficiency/scalability improvement) of having code with the auto-vectorization? Or we will get the same performance as without Auto-vectorization in this example case?<br>

&gt;<br>
&gt; How can we really get benefit in performance improvement with Auto-Vectorization?<br><br>
</div>
</div>Vectorization takes advantage of the processor's vector instructions to increase data-level parallelism.<br>
How much that benefits your code depends very much on your code; you would need to recompile your code and test.<br><br>
--<br>
David N. Lombard, Intel, Irvine, CA<br>
I do not speak for Intel Corporation; all comments are strictly my own.<br>
</blockquote>
</div>
<br>
</div>
Permalink | Reply | 2 comments |
headers
Håkon Bugge | 1 Dec 2009 08:54
Picon
Picon

Re: MPI Processes + Auto Vectorization


On Dec 1, 2009, at 7:14 , amjad ali wrote:
My question is that if we do not have free cpu cores in a PC or cluster (all cores are running MPI processes), still the auto-vertorization is beneficial? Or it is beneficial only if we have some free cpu cores locally? 


Amjad,

Vectorization is in x86_64 parlor a compilation technique where the compiler will utilize certain instructions which operate on short vectors. When you execute such a program on a particular core, these vector-instructions will execute on special execution unit _within_ the core you're executing on. Hence, no additional resources or cores are required to use vector instructions and you will benefit from them independent of whether you fully use all cores in your cluster or not.

Håkon
<div>
<br><div>
<div>On Dec 1, 2009, at 7:14 , amjad ali wrote:</div>
<blockquote type="cite"><span class="Apple-style-span"><div><div>My question is that if we do not have free cpu cores in a PC or cluster (all cores are running MPI processes), still the auto-vertorization is beneficial? Or it is beneficial only if we have some free cpu cores locally?<span class="Apple-converted-space">&nbsp;</span><br>
</div></div></span></blockquote>
</div>
<div apple-content-edited="true"><br class="Apple-interchange-newline"></div>
<div apple-content-edited="true"><br></div>Amjad,<div><br></div>
<div>Vectorization is in x86_64 parlor a compilation technique where the compiler will utilize certain instructions which operate on short vectors. When you execute such a program on a particular core, these vector-instructions will execute on special execution unit _within_ the core you're executing on. Hence, no additional resources or cores are required to use vector instructions and you will benefit from them independent of whether you fully use all cores in your cluster or not.</div>
<div><br></div>
<div>H&aring;kon</div>
</div>
Permalink | Reply | 0 comments |
headers
Joe Landman | 1 Dec 2009 19:02
Favicon

Forwarded from a long time reader having trouble posting

My apologies if this is bad form, I know Toon from his past 
participation on this list, and he asked me to forward.

-------- Original Message --------

Dear all,

I've been working on hpux-itanium for the last 2 years (and even
unsubscribed to beowulf-ml during most of that time, my bad) but soon
will turn back to a beowulf cluster (HP DL380G6's with Xeon X5570,
amcc/3ware 9690SA-8i with 4 x 600GB Cheetah 15krpm). Now I have a few
questions on the config.

1) our company is standardised on RHEL 5.1. Would sticking with rhel 5.1
instead of going to the latest make a difference.
2) What are the advantages of the hpc version of rhel. I browsed the doc
but unless having to compile mpi myself I do not see a difference or did
I miss soth.
3) which filesystem is advisable knowing that we're calculating on large
berkeley db databases

thanks in advance,

toon

Toon Knapen toon.knapen <at> gmail.com

-----------------------------------

--

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman <at> scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615
Permalink | Reply | 17 comments |
headers
Gerald Creager | 1 Dec 2009 20:45
Picon
Favicon

Re: Forwarded from a long time reader having trouble posting

Toon, welcome back!

I've been quite happy with CentOS 5.3 and we're experimenting with 
CentOS 5.4 now.  I see good stability in 5.[34] and the incorporation of 
a couple of tools worth having in a distribution for 'Wulf use.  I'd not 
recommend sticking with the old version, but of course, once you're 
established, not carelessly upgrading, either.

gerry

Joe Landman wrote:
> My apologies if this is bad form, I know Toon from his past 
> participation on this list, and he asked me to forward.
> 
> -------- Original Message --------
> 
> Dear all,
> 
> I've been working on hpux-itanium for the last 2 years (and even
> unsubscribed to beowulf-ml during most of that time, my bad) but soon
> will turn back to a beowulf cluster (HP DL380G6's with Xeon X5570,
> amcc/3ware 9690SA-8i with 4 x 600GB Cheetah 15krpm). Now I have a few
> questions on the config.
> 
> 1) our company is standardised on RHEL 5.1. Would sticking with rhel 5.1
> instead of going to the latest make a difference.
> 2) What are the advantages of the hpc version of rhel. I browsed the doc
> but unless having to compile mpi myself I do not see a difference or did
> I miss soth.
> 3) which filesystem is advisable knowing that we're calculating on large
> berkeley db databases
> 
> thanks in advance,
> 
> toon
> 
> Toon Knapen toon.knapen <at> gmail.com
> 
> -----------------------------------
>
Permalink | Reply | 12 comments |
headers
Tom Elken | 1 Dec 2009 20:57

RE: Forwarded from a long time reader having trouble posting

> On Behalf Of Gerald Creager
> I've been quite happy with CentOS 5.3 and we're experimenting with
> CentOS 5.4 now.  I see good stability in 5.[34] 

I have to second the recommendation of 5.3 or 5.4.

Some time ago, we saw significant performance improvements on Nehalem (Xeon X5570) in moving from RHEL 5.2
to 5.3.  So I expect that moving from 5.1 to 5.[34] would also be a significant improvement in performance.

Cheers,
-Tom

> and the incorporation
> of
> a couple of tools worth having in a distribution for 'Wulf use.  I'd
> not
> recommend sticking with the old version, but of course, once you're
> established, not carelessly upgrading, either.
> 
> gerry
> 
> Joe Landman wrote:
> > My apologies if this is bad form, I know Toon from his past
> > participation on this list, and he asked me to forward.
> >
> > -------- Original Message --------
> >
> > Dear all,
> >
> > I've been working on hpux-itanium for the last 2 years (and even
> > unsubscribed to beowulf-ml during most of that time, my bad) but soon
> > will turn back to a beowulf cluster (HP DL380G6's with Xeon X5570,
> > amcc/3ware 9690SA-8i with 4 x 600GB Cheetah 15krpm). Now I have a few
> > questions on the config.
> >
> > 1) our company is standardised on RHEL 5.1. Would sticking with rhel
> 5.1
> > instead of going to the latest make a difference.
> > 2) What are the advantages of the hpc version of rhel. I browsed the
> doc
> > but unless having to compile mpi myself I do not see a difference or
> did
> > I miss soth.
> > 3) which filesystem is advisable knowing that we're calculating on
> large
> > berkeley db databases
> >
> > thanks in advance,
> >
> > toon
> >
> > Toon Knapen toon.knapen <at> gmail.com
> >
> > -----------------------------------
> >
> _______________________________________________
> Beowulf mailing list, Beowulf <at> beowulf.org sponsored by Penguin
> Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
Permalink | Reply | 11 comments |
headers
Greg Lindahl | 1 Dec 2009 20:57
Gravatar

Re: MPI Processes + Auto Vectorization

On Tue, Dec 01, 2009 at 01:14:13AM -0500, amjad ali wrote:

> My question is that if we do not have free cpu cores in a PC or cluster (all
> cores are running MPI processes), still the auto-vertorization is
> beneficial? Or it is beneficial only if we have some free cpu cores locally?

Perhaps you're confusing auto-parallelization and auto-vectorization?

Auto-vectorization does not use any more cpu cores than unvectorized code.

-- greg
Permalink | Reply | 0 comments |
headers
Gerald Creager | 1 Dec 2009 22:04
Picon
Favicon

Re: Forwarded from a long time reader having trouble posting

A combination of mostly kernel improvements, and some useful middleware 
as RedHat and by extension, CentOS, seek to get farther into the cluster 
space.
gerry

Toon Knapen wrote:
> Any idea why it gives better performance? Was it on memory bw intensive 
> apps? Could it be due to changes in the kernel that take into account 
> the Numa architecture (affinity) or ...
> 
> On Tue, Dec 1, 2009 at 8:57 PM, Tom Elken <tom.elken <at> qlogic.com 
> <mailto:tom.elken <at> qlogic.com>> wrote:
> 
>      > On Behalf Of Gerald Creager
>      > I've been quite happy with CentOS 5.3 and we're experimenting with
>      > CentOS 5.4 now.  I see good stability in 5.[34]
> 
>     I have to second the recommendation of 5.3 or 5.4.
> 
>     Some time ago, we saw significant performance improvements on
>     Nehalem (Xeon X5570) in moving from RHEL 5.2 to 5.3.  So I expect
>     that moving from 5.1 to 5.[34] would also be a significant
>     improvement in performance.
> 
>     Cheers,
>     -Tom
> 
>      > and the incorporation
>      > of
>      > a couple of tools worth having in a distribution for 'Wulf use.  I'd
>      > not
>      > recommend sticking with the old version, but of course, once you're
>      > established, not carelessly upgrading, either.
>      >
>      > gerry
>      >
>      > Joe Landman wrote:
>      > > My apologies if this is bad form, I know Toon from his past
>      > > participation on this list, and he asked me to forward.
>      > >
>      > > -------- Original Message --------
>      > >
>      > > Dear all,
>      > >
>      > > I've been working on hpux-itanium for the last 2 years (and even
>      > > unsubscribed to beowulf-ml during most of that time, my bad)
>     but soon
>      > > will turn back to a beowulf cluster (HP DL380G6's with Xeon X5570,
>      > > amcc/3ware 9690SA-8i with 4 x 600GB Cheetah 15krpm). Now I have
>     a few
>      > > questions on the config.
>      > >
>      > > 1) our company is standardised on RHEL 5.1. Would sticking with
>     rhel
>      > 5.1
>      > > instead of going to the latest make a difference.
>      > > 2) What are the advantages of the hpc version of rhel. I
>     browsed the
>      > doc
>      > > but unless having to compile mpi myself I do not see a
>     difference or
>      > did
>      > > I miss soth.
>      > > 3) which filesystem is advisable knowing that we're calculating on
>      > large
>      > > berkeley db databases
>      > >
>      > > thanks in advance,
>      > >
>      > > toon
>      > >
>      > > Toon Knapen toon.knapen <at> gmail.com <mailto:toon.knapen <at> gmail.com>
>      > >
>      > > -----------------------------------
>      > >
>      > _______________________________________________
>      > Beowulf mailing list, Beowulf <at> beowulf.org
>     <mailto:Beowulf <at> beowulf.org> sponsored by Penguin
>      > Computing
>      > To change your subscription (digest mode or unsubscribe) visit
>      > http://www.beowulf.org/mailman/listinfo/beowulf
> 
>     _______________________________________________
>     Beowulf mailing list, Beowulf <at> beowulf.org
>     <mailto:Beowulf <at> beowulf.org> sponsored by Penguin Computing
>     To change your subscription (digest mode or unsubscribe) visit
>     http://www.beowulf.org/mailman/listinfo/beowulf
> 
>
Permalink | Reply | 8 comments |
headers
Andrew M.A. Cater | 1 Dec 2009 22:42
Picon
Picon

Re: Forwarded from a long time reader having trouble posting

On Tue, Dec 01, 2009 at 03:04:32PM -0600, Gerald Creager wrote:
> A combination of mostly kernel improvements, and some useful middleware  
> as RedHat and by extension, CentOS, seek to get farther into the cluster  
> space.
> gerry
>

Maybe also some licensing breaks on large volume licensing. Red Hat is 
primarily a sales and service organisation that also produces a Linux 
by-product :) The HPC variant is targetted at areas which deal in large 
clusters at cheaper than Red Hat Enterprise Linux for servers at 
equivalent volume, IIRC.

> Toon Knapen wrote:
>> Any idea why it gives better performance? Was it on memory bw intensive 
>> apps? Could it be due to changes in the kernel that take into account  
>> the Numa architecture (affinity) or ...
>>
>> On Tue, Dec 1, 2009 at 8:57 PM, Tom Elken <tom.elken <at> qlogic.com  
>> <mailto:tom.elken <at> qlogic.com>> wrote:
>>
>>      > On Behalf Of Gerald Creager
>>      > I've been quite happy with CentOS 5.3 and we're experimenting with
>>      > CentOS 5.4 now.  I see good stability in 5.[34]
>>
>>     I have to second the recommendation of 5.3 or 5.4.

I like 5.2 and 5.4 but my normal experience is on relatively stock IBM 
hardware. As ever, YMMV.

>>      > > Dear all,
>>      > >
>>      > > I've been working on hpux-itanium for the last 2 years (and even
>>      > > unsubscribed to beowulf-ml during most of that time, my bad)
>>     but soon
>>      > > will turn back to a beowulf cluster (HP DL380G6's with Xeon X5570,
>>      > > amcc/3ware 9690SA-8i with 4 x 600GB Cheetah 15krpm). Now I have
>>     a few
>>      > > questions on the config.
>>      > >
>>      > > 1) our company is standardised on RHEL 5.1. Would sticking with
>>     rhel
>>      > 5.1
>>      > > instead of going to the latest make a difference.

Since you have up to date hardware - also check on the necessary version 
of 3Ware drivers and where they are supported. The command line 
utilities are particularly useful.

>>      > > 2) What are the advantages of the hpc version of rhel. I
>>     browsed the
>>      > doc
>>      > > but unless having to compile mpi myself I do not see a
>>     difference or
>>      > did
>>      > > I miss soth.

See above.

>>      > > 3) which filesystem is advisable knowing that we're calculating on
>>      > large
>>      > > berkeley db databases
>>      > >

You get ext3 or Red Hat's cluster filesystem ?? GFS ??, I think. No xfs 
/ Reiser by default. Check also with HP as to what file systems they 
would recommend. 

>>      > > thanks in advance,
>>      > >
>>      > > toon
>>      > >
>>      > > Toon Knapen toon.knapen <at> gmail.com <mailto:toon.knapen <at> gmail.com>
>>      > >

Always happy to pontificate :)

All the best,

Andy
Permalink | Reply | 7 comments |
headers
Tom Elken | 1 Dec 2009 22:51

RE: Forwarded from a long time reader having trouble posting

Toon wrote:  “Any idea why it gives better performance? Was it on memory bw intensive apps? Could it be due to changes in the kernel that take into account the Numa architecture (affinity)”

 

This is very likely it.  Dredging thru old e-mails, I see that, in Jan-09, the application of a then-current 2.6.28 kernel to a RHEL 5.2 system provided about a 2x improvement in 8-thread OpenMP STREAM performance.  Subsequently, moving to RHEL 5.3 and its default kernel provided the same good STREAM performance. 

 

-Tom

 

 

From: Toon Knapen [mailto:toon.knapen <at> gmail.com]
Sent: Tuesday, December 01, 2009 12:53 PM
To: Tom Elken
Cc: gerry.creager <at> tamu.edu; landman <at> scalableinformatics.com; beowulf
Subject: Re: [Beowulf] Forwarded from a long time reader having trouble posting

 

Any idea why it gives better performance? Was it on memory bw intensive apps? Could it be due to changes in the kernel that take into account the Numa architecture (affinity) or ...

On Tue, Dec 1, 2009 at 8:57 PM, Tom Elken <tom.elken <at> qlogic.com> wrote:

> On Behalf Of Gerald Creager

> I've been quite happy with CentOS 5.3 and we're experimenting with
> CentOS 5.4 now.  I see good stability in 5.[34]

I have to second the recommendation of 5.3 or 5.4.

Some time ago, we saw significant performance improvements on Nehalem (Xeon X5570) in moving from RHEL 5.2 to 5.3.  So I expect that moving from 5.1 to 5.[34] would also be a significant improvement in performance.

Cheers,
-Tom


> and the incorporation
> of
> a couple of tools worth having in a distribution for 'Wulf use.  I'd
> not
> recommend sticking with the old version, but of course, once you're
> established, not carelessly upgrading, either.
>
> gerry
>
> Joe Landman wrote:
> > My apologies if this is bad form, I know Toon from his past
> > participation on this list, and he asked me to forward.
> >
> > -------- Original Message --------
> >
> > Dear all,
> >
> > I've been working on hpux-itanium for the last 2 years (and even
> > unsubscribed to beowulf-ml during most of that time, my bad) but soon
> > will turn back to a beowulf cluster (HP DL380G6's with Xeon X5570,
> > amcc/3ware 9690SA-8i with 4 x 600GB Cheetah 15krpm). Now I have a few
> > questions on the config.
> >
> > 1) our company is standardised on RHEL 5.1. Would sticking with rhel
> 5.1
> > instead of going to the latest make a difference.
> > 2) What are the advantages of the hpc version of rhel. I browsed the
> doc
> > but unless having to compile mpi myself I do not see a difference or
> did
> > I miss soth.
> > 3) which filesystem is advisable knowing that we're calculating on
> large
> > berkeley db databases
> >
> > thanks in advance,
> >
> > toon
> >
> > Toon Knapen toon.knapen <at> gmail.com
> >
> > -----------------------------------
> >
> _______________________________________________
> Beowulf mailing list, Beowulf <at> beowulf.org sponsored by Penguin
> Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf

_______________________________________________
Beowulf mailing list, Beowulf <at> beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

 

<div>

<div class="Section1">

<p class="MsoNormal"><span>Toon wrote:&nbsp; &ldquo;</span>Any idea why it gives better
performance? Was it on memory bw intensive apps? Could it be due to changes in
the kernel that take into account the Numa architecture (affinity)&rdquo;<p></p></p>

<p class="MsoNormal"><span><p>&nbsp;</p></span></p>

<p class="MsoNormal"><span>This is very likely it.&nbsp; Dredging thru old e-mails, I see
that, in Jan-09, the application of a then-current 2.6.28 kernel to a RHEL 5.2
system provided about a 2x improvement in 8-thread OpenMP STREAM
performance.&nbsp; Subsequently, moving to RHEL 5.3 and its default kernel provided
the same good STREAM performance.&nbsp; <p></p></span></p>

<p class="MsoNormal"><span><p>&nbsp;</p></span></p>

<p class="MsoNormal"><span>-Tom<p></p></span></p>

<p class="MsoNormal"><span><p>&nbsp;</p></span></p>

<p class="MsoNormal"><span><p>&nbsp;</p></span></p>

<div>

<div>

<div>

<p class="MsoNormal"><span>From:</span><span> Toon Knapen
[mailto:toon.knapen <at> gmail.com] <br>Sent: Tuesday, December 01, 2009 12:53 PM<br>To: Tom Elken<br>Cc: gerry.creager <at> tamu.edu; landman <at> scalableinformatics.com; beowulf<br>Subject: Re: [Beowulf] Forwarded from a long time reader having trouble
posting<p></p></span></p>

</div>

</div>

<p class="MsoNormal"><p>&nbsp;</p></p>

<p class="MsoNormal">Any idea why it gives better
performance? Was it on memory bw intensive apps? Could it be due to changes in
the kernel that take into account the Numa architecture (affinity) or ... <p></p></p>

<div>

<p class="MsoNormal">On Tue, Dec 1, 2009 at 8:57 PM, Tom Elken &lt;<a href="mailto:tom.elken <at> qlogic.com">tom.elken <at> qlogic.com</a>&gt; wrote:<p></p></p>

<p class="MsoNormal">&gt; On Behalf Of Gerald Creager<p></p></p>

<div>

<p class="MsoNormal">&gt; I've been quite happy with
CentOS 5.3 and we're experimenting with<br>
&gt; CentOS 5.4 now. &nbsp;I see good stability in 5.[34]<p></p></p>

</div>

<p class="MsoNormal">I have to second the recommendation of 5.3 or 5.4.<br><br>
Some time ago, we saw significant performance improvements on Nehalem (Xeon
X5570) in moving from RHEL 5.2 to 5.3. &nbsp;So I expect that moving from 5.1
to 5.[34] would also be a significant improvement in performance.<br><br>
Cheers,<br><span>-Tom</span><p></p></p>

<div>

<div>

<p class="MsoNormal"><br>
&gt; and the incorporation<br>
&gt; of<br>
&gt; a couple of tools worth having in a distribution for 'Wulf use. &nbsp;I'd<br>
&gt; not<br>
&gt; recommend sticking with the old version, but of course, once you're<br>
&gt; established, not carelessly upgrading, either.<br>
&gt;<br>
&gt; gerry<br>
&gt;<br>
&gt; Joe Landman wrote:<br>
&gt; &gt; My apologies if this is bad form, I know Toon from his past<br>
&gt; &gt; participation on this list, and he asked me to forward.<br>
&gt; &gt;<br>
&gt; &gt; -------- Original Message --------<br>
&gt; &gt;<br>
&gt; &gt; Dear all,<br>
&gt; &gt;<br>
&gt; &gt; I've been working on hpux-itanium for the last 2 years (and even<br>
&gt; &gt; unsubscribed to beowulf-ml during most of that time, my bad) but soon<br>
&gt; &gt; will turn back to a beowulf cluster (HP DL380G6's with Xeon X5570,<br>
&gt; &gt; amcc/3ware 9690SA-8i with 4 x 600GB Cheetah 15krpm). Now I have a few<br>
&gt; &gt; questions on the config.<br>
&gt; &gt;<br>
&gt; &gt; 1) our company is standardised on RHEL 5.1. Would sticking with rhel<br>
&gt; 5.1<br>
&gt; &gt; instead of going to the latest make a difference.<br>
&gt; &gt; 2) What are the advantages of the hpc version of rhel. I browsed the<br>
&gt; doc<br>
&gt; &gt; but unless having to compile mpi myself I do not see a difference or<br>
&gt; did<br>
&gt; &gt; I miss soth.<br>
&gt; &gt; 3) which filesystem is advisable knowing that we're calculating on<br>
&gt; large<br>
&gt; &gt; berkeley db databases<br>
&gt; &gt;<br>
&gt; &gt; thanks in advance,<br>
&gt; &gt;<br>
&gt; &gt; toon<br>
&gt; &gt;<br>
&gt; &gt; Toon Knapen <a href="mailto:toon.knapen <at> gmail.com">toon.knapen <at> gmail.com</a><br>
&gt; &gt;<br>
&gt; &gt; -----------------------------------<br>
&gt; &gt;<br>
&gt; _______________________________________________<br>
&gt; Beowulf mailing list, <a href="mailto:Beowulf <at> beowulf.org">Beowulf <at> beowulf.org</a>
sponsored by Penguin<br>
&gt; Computing<br>
&gt; To change your subscription (digest mode or unsubscribe) visit<br>
&gt; <a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/mailman/listinfo/beowulf</a><br><br>
_______________________________________________<br>
Beowulf mailing list, <a href="mailto:Beowulf <at> beowulf.org">Beowulf <at> beowulf.org</a>
sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/mailman/listinfo/beowulf</a><p></p></p>

</div>

</div>

</div>

<p class="MsoNormal"><p>&nbsp;</p></p>

</div>

</div>

</div>
Permalink | Reply | 0 comments |
headers
Gerald Creager | 1 Dec 2009 23:05
Picon
Favicon

Re: Forwarded from a long time reader having trouble posting

Andrew M.A. Cater wrote:
> On Tue, Dec 01, 2009 at 03:04:32PM -0600, Gerald Creager wrote:
>> A combination of mostly kernel improvements, and some useful middleware  
>> as RedHat and by extension, CentOS, seek to get farther into the cluster  
>> space.
>> gerry
>>
> 
> Maybe also some licensing breaks on large volume licensing. Red Hat is 
> primarily a sales and service organisation that also produces a Linux 
> by-product :) The HPC variant is targetted at areas which deal in large 
> clusters at cheaper than Red Hat Enterprise Linux for servers at 
> equivalent volume, IIRC.
> 
>> Toon Knapen wrote:
>>> Any idea why it gives better performance? Was it on memory bw intensive 
>>> apps? Could it be due to changes in the kernel that take into account  
>>> the Numa architecture (affinity) or ...
>>>
>>> On Tue, Dec 1, 2009 at 8:57 PM, Tom Elken <tom.elken <at> qlogic.com  
>>> <mailto:tom.elken <at> qlogic.com>> wrote:
>>>
>>>      > On Behalf Of Gerald Creager
>>>      > I've been quite happy with CentOS 5.3 and we're experimenting with
>>>      > CentOS 5.4 now.  I see good stability in 5.[34]
>>>
>>>     I have to second the recommendation of 5.3 or 5.4.
> 
> I like 5.2 and 5.4 but my normal experience is on relatively stock IBM 
> hardware. As ever, YMMV.
> 
>>>      > > Dear all,
>>>      > >
>>>      > > I've been working on hpux-itanium for the last 2 years (and even
>>>      > > unsubscribed to beowulf-ml during most of that time, my bad)
>>>     but soon
>>>      > > will turn back to a beowulf cluster (HP DL380G6's with Xeon X5570,
>>>      > > amcc/3ware 9690SA-8i with 4 x 600GB Cheetah 15krpm). Now I have
>>>     a few
>>>      > > questions on the config.
>>>      > >
>>>      > > 1) our company is standardised on RHEL 5.1. Would sticking with
>>>     rhel
>>>      > 5.1
>>>      > > instead of going to the latest make a difference.
> 
> Since you have up to date hardware - also check on the necessary version 
> of 3Ware drivers and where they are supported. The command line 
> utilities are particularly useful.
> 
>>>      > > 2) What are the advantages of the hpc version of rhel. I
>>>     browsed the
>>>      > doc
>>>      > > but unless having to compile mpi myself I do not see a
>>>     difference or
>>>      > did
>>>      > > I miss soth.
> 
> See above.
> 
>>>      > > 3) which filesystem is advisable knowing that we're calculating on
>>>      > large
>>>      > > berkeley db databases
>>>      > >
> 
> You get ext3 or Red Hat's cluster filesystem ?? GFS ??, I think. No xfs 
> / Reiser by default. Check also with HP as to what file systems they 
> would recommend. 
> 
> 
>>>      > > thanks in advance,
>>>      > >
>>>      > > toon
>>>      > >
>>>      > > Toon Knapen toon.knapen <at> gmail.com <mailto:toon.knapen <at> gmail.com>
>>>      > >
> 
> Always happy to pontificate :)

I believe xfs is now available in 5.4.  I'd have to check.  We've found 
xfs to be our preference (but we're revisiting gluster and lustre). 
I've not played with gfs so far.

gerry
Permalink | Reply | 5 comments |

Gmane