Du Zhihui | 10 Apr 22:05 2000
Picon

What is the meaning of GM_STBAR()

Dear All,
      I find in gmlib that GM_STBAR() is defined as following, I do not
know what is the meaning of it and why and when you should use it.
    Thanks!

#if defined lanai3 || defined lanai7
 /* ": : :" for C++ compat. */
#define GM_STBAR() asm volatile ("! GM_STBAR" : : :"memory")
#elif defined GM_CPU_sparc || defined GM_CPU_sparc64
#ifdef __GNUC__
#define GM_STBAR( ) asm ("stbar": : :"memory") /* ": : :" for C++
compat. */
#elif (defined (__SUNPRO_C) || defined (__SUNPRO_CC)) &&
defined(__sparcv9)
#define GM_STBAR( ) {}
#elif (defined (__SUNPRO_C) || defined (__SUNPRO_CC)) &&
defined(__sparc)
#define GM_STBAR( ) {}
#else
#error Do not know how to emit the "stbar" instruction under this
compiler.

Patrick GEOFFRAY | 12 Apr 02:38 2000
Picon

Re: [myrinet] What is the meaning of GM_STBAR()

Du Zhihui wrote:
> 
> Dear All,
>       I find in gmlib that GM_STBAR() is defined as following, I do not
> know what is the meaning of it and why and when you should use it.

GM_STBAR is a memory barrier for SMP systems. In a SMP, 2 processors can
execute at the same time 2 differents codes on shared datas :
* the order between 2 STORE within a processor is guaranteed (cache).
* the order between 2 STOREs between 2 differents processors is not
guaranteed, i.e. the order of 2 STOREs can be inverse on the memory bus.

You can use a stbar to flush the pending STOREs on the memory bus. After
the GM_STBAR, you are guaranteed about the consistency of shared
variable.

I think it's not a problem on SMP x86, as the write on the memory bus
are ordered, but it's not the case on SMP alphas for example.

Hope it can help you.

Patrick Geoffray
---
Matra Systemes & Information - MS&I
Universite Lyon I - RESAM
http://lhpca.univ-lyon1.fr

Bob Felderman | 12 Apr 06:34 2000

Re: [myrinet] Re: What is the meaning of GM_STBAR()

=> > 
=> > Dear All,
=> >       I find in gmlib that GM_STBAR() is defined as following, I do not
=> > know what is the meaning of it and why and when you should use it.
=> 
=> GM_STBAR is a memory barrier for SMP systems. In a SMP, 2 processors can
=> execute at the same time 2 differents codes on shared datas :
=> * the order between 2 STORE within a processor is guaranteed (cache).
=> * the order between 2 STOREs between 2 differents processors is not
=> guaranteed, i.e. the order of 2 STOREs can be inverse on the memory bus.
=> 
=> You can use a stbar to flush the pending STOREs on the memory bus. After
=> the GM_STBAR, you are guaranteed about the consistency of shared
=> variable.
=> 
=> I think it's not a problem on SMP x86, as the write on the memory bus
=> are ordered, but it's not the case on SMP alphas for example.

Actually, you can have the problem also on a single processor
that allows re-ordering of instructions. The memory barrier 
is a place to be sure all pending writes have taken place, or at least
will happen before any writes that follow in the code.

Patrick GEOFFRAY | 12 Apr 03:25 2000
Picon

Re: [myrinet] Re: What is the meaning of GM_STBAR()

Bob Felderman wrote:
> 
> => >
> => > Dear All,
> => >       I find in gmlib that GM_STBAR() is defined as following, I do not
> => > know what is the meaning of it and why and when you should use it.

> => I think it's not a problem on SMP x86, as the write on the memory bus
> => are ordered, but it's not the case on SMP alphas for example.
> 
> Actually, you can have the problem also on a single processor
> that allows re-ordering of instructions. The memory barrier
> is a place to be sure all pending writes have taken place, or at least
> will happen before any writes that follow in the code.

Hi Bob,

Do you know which architectures re-order instructions (single processor)
?

Patrick Geoffray
---
Matra Systemes & Information - MS&I
Universite Lyon I - RESAM
http://lhpca.univ-lyon1.fr

Daniel Faller | 19 Apr 13:48 2000
Picon

GM on a Ruffian

Hi,

does anyone run GM (1.1.1,2,3) on a 21164 Ruffian running Linux ?

I am facing some installation problems. Running a kernel compiled for a generic machine
GM works fine, but with a kernel compiled especially for a Ruffian the nodes fail to
contact the mapper to get their node id.

If anyone managed to get GM working together with a Ruffian kernel (2.2.14), which should
be no problem according to  help <at> myri.com, I would be very pleased about any comments.

Cheers
     Daniel

_____________________________________________
Daniel Faller
Fakultaet fuer Physik
Abt. Honerkamp
Albert-Ludwigs-Universitaet Freiburg

Tel.: 0761-203-5875
Fax.: 0761-203-5967 
e-mail: Daniel.Faller <at> physik.uni-freiburg.de
URL:    http://webber.physik.uni-freiburg.de/~fallerd         

Paul Lu | 26 Apr 23:00 2000
Picon
Picon

Any experience with SuperMicro PIIIDME motherboard?

Hello:

I would like to build a cluster with Myrinet and the SuperMirco PIIIDME
(i840-based, Dual Pentium III with Ethernet, we won't have ECC memory)
motherboard.

The manufacturer's URL is:

	http://www.supermicro.com/PRODUCT/MotherBoards/840/PIIIDME.htm

Has anybody been able to use these motherboards and their 64-bit/66 MHz
PCI slots with Myrinet?

Thanks in advance for your experience reports.

	...Paul

Bob Felderman | 26 Apr 23:20 2000

Re: [myrinet] Any experience with SuperMicro PIIIDME motherboard?

=> I would like to build a cluster with Myrinet and the SuperMirco PIIIDME
=> (i840-based, Dual Pentium III with Ethernet, we won't have ECC memory)
=> motherboard.
=> 
=> The manufacturer's URL is:
=> 
=> 	http://www.supermicro.com/PRODUCT/MotherBoards/840/PIIIDME.htm
=> 
=> Has anybody been able to use these motherboards and their 64-bit/66 MHz
=> PCI slots with Myrinet?
=> 
=> Thanks in advance for your experience reports.

We've been using one here for a while.
We're unhappy with the PCI read performance and have not found
any way to improve it. I've tried 4 DIMMS. I've been using
PC133 memory.

p2bl 31% hswap.intel_linux -C swap_test7.dat -t
swaping verify = 0x12345678 0x12345678
writing 0x00082400 to pointer register 0 (0x00082400 unswapped)
writing 0x00082420 to pointer register 1 (0x00082420 unswapped)
writing 0x00082440 to pointer register 2 (0x00082440 unswapped)
writing 0x00082460 to pointer register 3 (0x00082460 unswapped)
Using system RAM (0x00470000) as target of transfers
DMA test in progress
Running DMA timing test unit = 0  time = 10 burst = 0xF
LANai code = 'swap_test7.dat'
Dir     Burst   STS     size(B) count   time    BW(MByte/s)

(Continue reading)

Paul Lu | 26 Apr 23:34 2000
Picon
Picon

Re: Any experience with SuperMicro PIIIDME motherboard?

Hello Bob:

Thank you for the reply.  Very interesting.

On Wed, Apr 26, 2000 at 02:20:05PM -0700, Bob Felderman wrote:
> => I would like to build a cluster with Myrinet and the SuperMirco PIIIDME
> => (i840-based, Dual Pentium III with Ethernet, we won't have ECC memory)
> => motherboard.
> => 
> => 	http://www.supermicro.com/PRODUCT/MotherBoards/840/PIIIDME.htm
> 
> We've been using one here for a while.
> We're unhappy with the PCI read performance and have not found
> any way to improve it. I've tried 4 DIMMS. I've been using
> PC133 memory.

According to the first part of this web page:

   http://www.tdl.com/~netex/mb/mb.html

   Contrary to popular belief the 820 and 840 motherboards with the 133MHz
   FSB do *NOT* use PC133 memory, they use PC100 memory.  While you may
   install PC133 memory on any PC100 motherboard it will only run at PC100
   speeds of 100MHz, not 133MHz. Indeed since most of the PC133 memory runs
   at 3-3-3 and not 2-2-2, PC133 memory used on a 440BX, 440GX, 820 or 840
   motherboard will run slower than PC100 memory on the same board. Moreover,
   the AMD K7 also does not support PC133 memory.

Do you think the PC133 vs. PC100 issue could be a factor here?

(Continue reading)

Bob Felderman | 26 Apr 23:39 2000

Re: Any experience with SuperMicro PIIIDME motherboard?

=> According to the first part of this web page:
=> 
=>    http://www.tdl.com/~netex/mb/mb.html
=> 
=>    Contrary to popular belief the 820 and 840 motherboards with the 133MHz
=>    FSB do *NOT* use PC133 memory, they use PC100 memory.  While you may
=>    install PC133 memory on any PC100 motherboard it will only run at PC100
=>    speeds of 100MHz, not 133MHz. Indeed since most of the PC133 memory runs
=>    at 3-3-3 and not 2-2-2, PC133 memory used on a 440BX, 440GX, 820 or 840
=>    motherboard will run slower than PC100 memory on the same board. Moreover,
=>    the AMD K7 also does not support PC133 memory.
=> 
=> Do you think the PC133 vs. PC100 issue could be a factor here?

it might be. When I get on the machine again, I'll try
to replace the pc133 with pc100 and see what happens.
I had some trouble early on with pc100, but maybe it was just 
cheezy memory.

=> Also, do you (or anyone else) have any experience with other i840-based
=> motherboards?  The Tyan boards, for example?

I haven't been able to get a Tyan.
Duke has used a Dell machine.

#
#We just got two Dell PowerEdge 2400 "servers" which have a separate
#64-bit, 33MHz PCI bus.  They use the "RCC Reliance LE 3.0" chipset
#according to Dell.
#
(Continue reading)

David Rauschenbach | 27 Apr 03:05 2000

Optimal systems

I'm impressed that Bob Felderman had hands-on experience with a
motherboard that someone just picked out of a hat.

For those of us that might like Myrinet and haven't bought our systems
yet, are there any known "best picks" in the Intel or UltraSparc
categories?

Thanks,
David


Gmane