1 Nov 2006 22:40
1 Nov 2006 23:59
Re: PVFS and 2.6.18
Robert Latham <robl <at> mcs.anl.gov>
2006-11-01 22:59:39 GMT
2006-11-01 22:59:39 GMT
On Wed, Nov 01, 2006 at 03:40:35PM -0600, Robert Olson wrote: > Looking at moving my PPC nodes to 2.6.18; it appears that the 1.5.1 > has problems (I googled & saw the note on the dev list about the > issue in aug); I'm wondering what the right path is for upgrading. Hi Bob We could probably backport the patches for 2.6.18 to 1.5.1. Looking quickly, I can find at least three, but there could be others. We're close to a new release, though. I think the plan is to release around SC, so if you can hold off on the upgrade unti then, that might be the easiest option. ==rob -- -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
2 Nov 2006 00:04
Re: PVFS and 2.6.18
Robert Olson <olson <at> mcs.anl.gov>
2006-11-01 23:04:10 GMT
2006-11-01 23:04:10 GMT
Cool, that works for me. Thanks, --bob On Nov 1, 2006, at 4:59 PM, Robert Latham wrote: > On Wed, Nov 01, 2006 at 03:40:35PM -0600, Robert Olson wrote: >> Looking at moving my PPC nodes to 2.6.18; it appears that the 1.5.1 >> has problems (I googled & saw the note on the dev list about the >> issue in aug); I'm wondering what the right path is for upgrading. > > Hi Bob > > We could probably backport the patches for 2.6.18 to 1.5.1. Looking > quickly, I can find at least three, but there could be others. > > We're close to a new release, though. I think the plan is to release > around SC, so if you can hold off on the upgrade unti then, that might > be the easiest option. > > ==rob > > -- > Rob Latham > Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF > Argonne National Lab, IL USA B29D F333 664A 4280 315B >
1 Nov 2006 23:06
Growing storage
Andrew Pochinsky <avp <at> MIT.EDU>
2006-11-01 22:06:17 GMT
2006-11-01 22:06:17 GMT
Hi, I'm looking to build a new PVFS2 storage and it seems that the price sweet spot these days is on boxes with a lot of local disks (my estimates are 5 to 10 TB per chassis). To provide enough bandwidth to clients, I'm thinking about connecting PVFS2 servers to the switch via 10Ge. (We already have a Cisco switch and can not go IB. GigE and 10G are the only choices, as far as I know.) Did anybody have any experience with such a setup? What pitfalls are there? Thanks, --andrew
2 Nov 2006 17:15
Re: Growing storage
Dean Hildebrand <dhildebz <at> eecs.umich.edu>
2006-11-02 16:15:12 GMT
2006-11-02 16:15:12 GMT
We are using Cisco and Force10 10Gb switches here at Michigan to do the opposite, 10Gb clients accessing many 1Gb storage nodes. One thing we've found (and it is worse with our Cisco switch) is that the switch can run out of buffer space when packets switch from the 1Gb ports to the 10Gb port, and packets are dropped. 1Gb to 1Gb is good, 10Gb to 10Gb is good, but 1Gb to 10Gb doesn't work so well (this is our read throughput benchmark). The best throughput we've been able to get is around 4Gb/s in this scenario. We are currently working with Force10 to try and resolve this issue, but no progress yet. Hope this helps, -- -- Dean Hildebrand Ph.D. Candidate University of Michigan Andrew Pochinsky wrote: > Hi, > I'm looking to build a new PVFS2 storage and it seems that the > price sweet spot these days is on boxes with a lot of local disks (my > estimates are 5 to 10 TB per chassis). To provide enough bandwidth to > clients, I'm thinking about connecting PVFS2 servers to the switch via > 10Ge. (We already have a Cisco switch and can not go IB. GigE and 10G > are the only choices, as far as I know.) Did anybody have any > experience with such a setup? What pitfalls are there? > Thanks, > --andrew > > _______________________________________________(Continue reading)
2 Nov 2006 17:59
PVFS2 & Jumbo Frames
Robert Olson <olson <at> mcs.anl.gov>
2006-11-02 16:59:51 GMT
2006-11-02 16:59:51 GMT
The nodes in my cluster have dual Gig-E nics (tigon3). I'm pondering the installation of a second gigE switch to attach them all with a private storage network, and run it with jumbo frames. Do you think I'd see a significant performance increase with PVFS in this configuration? We're currently running the nodes with standard 1500 byte frames as the nodes are sitting on a shared network with the rest of our group infrastructure. I'm also curious if folks have experimented with the effects of the TCP segment offload stuff that is available on these NICs. thanks, --bob
2 Nov 2006 18:37
Re: PVFS2 & Jumbo Frames
Robert Latham <robl <at> mcs.anl.gov>
2006-11-02 17:37:34 GMT
2006-11-02 17:37:34 GMT
On Thu, Nov 02, 2006 at 10:59:51AM -0600, Robert Olson wrote: > The nodes in my cluster have dual Gig-E nics (tigon3). I'm pondering > the installation of a second gigE switch to attach them all with a > private storage network, and run it with jumbo frames. Do you think > I'd see a significant performance increase with PVFS in this > configuration? Hi Bob We definitely expect jumbo frames to help out pvfs a lot. Ti's had good results on the teragrid, for example [1]. Unfortunately, I don't think anyone has quantified the benefit. > I'm also curious if folks have experimented with the effects of the > TCP segment offload stuff that is available on these NICs. Again, should help PVFS a lot, but we've got no hard data. ==rob [1] http://www.teragridforum.org/mediawiki/index.php?title=TG-11 -- -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
2 Nov 2006 18:45
Re: PVFS2 & Jumbo Frames
Robert Olson <olson <at> mcs.anl.gov>
2006-11-02 17:45:03 GMT
2006-11-02 17:45:03 GMT
Thanks - interesting document! --bob On Nov 2, 2006, at 11:37 AM, Robert Latham wrote: > On Thu, Nov 02, 2006 at 10:59:51AM -0600, Robert Olson wrote: >> The nodes in my cluster have dual Gig-E nics (tigon3). I'm pondering >> the installation of a second gigE switch to attach them all with a >> private storage network, and run it with jumbo frames. Do you think >> I'd see a significant performance increase with PVFS in this >> configuration? > > Hi Bob > > We definitely expect jumbo frames to help out pvfs a lot. Ti's had > good results on the teragrid, for example [1]. Unfortunately, I don't > think anyone has quantified the benefit. > >> I'm also curious if folks have experimented with the effects of the >> TCP segment offload stuff that is available on these NICs. > > Again, should help PVFS a lot, but we've got no hard data. > > ==rob > > [1] http://www.teragridforum.org/mediawiki/index.php?title=TG-11 > > -- > Rob Latham(Continue reading)
2 Nov 2006 19:54
PVFS2 Cluster configuration
Peng Gu <penggu <at> cs.ucf.edu>
2006-11-02 18:54:10 GMT
2006-11-02 18:54:10 GMT
Dear all, We are considering purchasing a 16-32 nodes cluster to serve as a testbed for PVFS2 with GigaE or Infiniband interconnection. Do you have any suggestions on the configuration of the computer nodes and inter-connection hardware? Thanks in advance! Peng
3 Nov 2006 04:26
Re: PVFS2 Cluster configuration
Peng Gu <penggu <at> cs.ucf.edu>
2006-11-03 03:26:46 GMT
2006-11-03 03:26:46 GMT
In addition, we would like know your ideas about the following questions: 1) Whether 64-bit processor like Interl Xeon works fine with PVFS2/future version or not. 2) Will Dell power-edge server cluster works fine? 3) Which OS, Linux 2.6 or other fit PVFS2/future version well. 4) Which vendor/cluster solution do you recommend? Any inputs are appreciated, Peng On 11/2/06, Peng Gu <penggu <at> cs.ucf.edu> wrote: > Dear all, > > We are considering purchasing a 16-32 nodes cluster to serve as a > testbed for PVFS2 with GigaE or Infiniband interconnection. Do you > have any suggestions on > the configuration of the computer nodes and inter-connection hardware? > > Thanks in advance! > > Peng >
RSS Feed