headers
Markus Fischer | 2 Nov 2004 09:11

Re: Question about sockets-GM

Hu,

since that number is the number of pending fd's this looks like a bug.

I will run our test suite and will get back to you.

thanks,
Markus

Hu Wei wrote:

> I use sockets-GM-LSP on my myrinet 2000. Everything seems OK, but only one problem.
>
>for example, as following code:
>
>	do
>	{
>		FD_ZERO(&fds);
>		FD_SET((unsigned int)s, &fds);
>		tm.tv_sec = ms / 1000;
>		tm.tv_usec = (ms % 1000) * 1000;
>		ready = select(s + 1, &fds, NULL, NULL, &tm);
>	}
>	while(ready ==0);
>
>if normal ethernet network used, select() would return 0 or 1 to indicate if there are some
>connections. However, when sockets-GM-LSP used, select() returns a very big number,such as
>528060, even there is no connection at all.
>
>What is Wrong?
(Continue reading)

Permalink | Reply | 0 comments |
headers
Tipparaju, Vinod | 12 Nov 2004 19:15

FW: increasing the amount of memory that can be registered


Word of mouth is that it is possible to specify the amount of memory
that is available to be registered as a parameter. Can I increasing the
upper limit for the amount of memory gm can register by specifying it as
a parameter for the gm module? Could you tell me how to do that? I was
unable to find it in the documentation.

Also, how is the upper limit determined?

I was seeing in the documents somewhere that when same memory is
registered for 2 ports, gm realizes it and avoids duplication. Is it
true.

All my questions are in regards to gm2, we have e cards.

Thanks,
Vinod.
Permalink | Reply | 2 comments |
headers
Reese Faucette | 12 Nov 2004 21:36

Re: FW: increasing the amount of memory that can be registered

> Word of mouth is that it is possible to specify the amount of memory
> that is available to be registered as a parameter. Can I increasing the
> upper limit for the amount of memory gm can register by specifying it as
> a parameter for the gm module? Could you tell me how to do that? I was
> unable to find it in the documentation.

When you insmod the driver, specify "gm_max_locked_mbytes=MMM" when MMM is 
the maximum number of megabytes of memory you with to allow GM to register.

> Also, how is the upper limit determined?

There are several factors, but in a system with plenty of memory and no 
IOMMU limitations, the default is 50% of physical memory.

> I was seeing in the documents somewhere that when same memory is
> registered for 2 ports, gm realizes it and avoids duplication. Is it
> true.

I'm not sure what exactly you mean by "avoids duplication", but reference 
counts are kept on registered pages, and the memory will only be 
unregistered when the last instance is "deregistered".  In short, GM handles 
multiple registration calls on the same memory region properly, whether on 
the same port or not.

-reese
Permalink | Reply | 1 comment |
headers
Loic Prylli | 23 Nov 2004 00:34

Re: FW: increasing the amount of memory that can be registered

Reese Faucette wrote:
>> Word of mouth is that it is possible to specify the amount of memory
>> that is available to be registered as a parameter. Can I increasing the
>> upper limit for the amount of memory gm can register by specifying it as
>> a parameter for the gm module? Could you tell me how to do that? I was
>> unable to find it in the documentation.
> 
> 
> When you insmod the driver, specify "gm_max_locked_mbytes=MMM" when MMM 
> is the maximum number of megabytes of memory you with to allow GM to 
> register.
> 

Let me add a few more precisions:

If you load gm through the standard /etc/init.d/gm script (if don't 
insmod manually), you have the possibility of creating a file 
<gm-installation-dir>/etc/gm/module.options containing options to pass 
to the gm modules, in particular you can put there the parameter:
    gm_max_locked_mbytes=MMM
/etc/init.d/gm will read that file and pass that to insmod.

>> I was seeing in the documents somewhere that when same memory is
>> registered for 2 ports, gm realizes it and avoids duplication. Is it
>> true.

Altough the physical page is properly shared in this case, the counter 
of pages registered is currently decremented twice, one time for each 
port. So you will reach the limit specified in gm_max_locked_mbytes=MMM 
(or the default limit of 50% of physical memory) earlier than planned.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ricky Tang Siu Hong | 1 Dec 2004 03:34
Picon

M3-SW16-8E switch with e1000 network card

Hello,

We have M3-SW16-8E modules connected with e1000 card.  But there is no 
traffic between machines we see with "tcpdump".

We use high quality cat 5 cables.  ethtool said it is running 1000 full 
duplex mode.

We also tried to connect two machines using cross-over cable and they can 
talk to each other.

Have anybody give any hint or experience on using the module with e1000 NIC 
?

Thanks.

Ricky.
Permalink | Reply | 0 comments |

Gmane