A public mailing list for Parallel Virtual File System v2 users ()

headers Dharani Sankar Vijayakumar | 1 Dec 2007 10:20

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users <at> beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Le Fri, 30 Nov 2007 23:35:35 +0100 vous écriviez:

> I've seen nothing but I may have missed something, perhaps I
> didn't check both nodes of the cluster...

I've checked, and there's nothing in the logs. I tried again today but
unfortunately there was an error at the beginning of the test that
forced me to hard-reboot the nfs server node :

[D 12/03 12:55] PVFS2 Server version 2.7.0pre1-2007-11-29-164557
starting.

[E 12/03 12:57] handle_io_error: flow proto error cleanup started on
0x813ae60,
 error_code: -1073741973

[E 12/03 12:57] handle_io_error: flow proto 0x813ae60 canceled 0
operations, wi
ll clean up.

[E 12/03 12:57] handle_io_error: flow proto 0x813ae60 error cleanup
finished, e
rror_code: -1073741973

[E 12/03 12:57] handle_io_error: flow proto error cleanup started on
0x813fe30,
 error_code: -1073742011

[E 12/03 12:57] handle_io_error: flow proto 0x813fe30 canceled 0
operations, wi
ll clean up.

[E 12/03 12:57] handle_io_error: flow proto 0x813fe30 error cleanup
finished, e
rror_code: -1073742011

[E 12/03 12:57] handle_io_error: flow proto error cleanup started on
0x8111030,
 error_code: -1073742011

[E 12/03 12:57] handle_io_error: flow proto 0x8111030 canceled 0
operations, wi
ll clean up.

[E 12/03 12:57] handle_io_error: flow proto 0x8111030 error cleanup
finished, e
rror_code: -1073742011

Now I have to wait for the software raid-1 to rebuild.

--

-- 
--------------------------------------------------
Emmanuel Florac               www.intellique.com   
--------------------------------------------------

Pete Wyckoff wrote:
> This one looks like changes to method addr ref counting were not
> propagated down into IB.  Here is a trivial hack-around.  TCP
> doesn't call back to BMI in a DROP_ADDR, so maybe it is correct.
> Sam might verify that this is the right thing to do.
>
> 		-- Pete
>   
Thanks for the quick response, it did the trick.

Is there any more information I can help provide regarding the "no space
left on device" issue?

Tad

Tad Kollar wrote:
> Is there any more information I can help provide regarding the "no space
> left on device" issue?
>
>   
A few minutes after sending that, I found that the problem happens when
using "TroveMethod alt-aio". Using "TroveMethod dbpf" appears to work well.

Tad

OK, I've been testing again and again, and I found a weeeeeeird bug.
When I say weird, I mean really really of the black magic flavour.

So : still testing this NFS stuff, I tried with various configurations
to sort it out, and it really a PVFS2 problem (tried different clients,
and exporting non-pvfs2 FS from the same server, all OK).

The NFS server exports the pvfs2 FS this way :

/mnt/pvfs2      *(rw,async,no_root_squash,no_subtree_check)

On the client, I mount it and run this into the share :

for i in `seq 1 50` ; do touch test.$i ; done ; echo "created $i" ;
echo -n "listing " ; ls | wc -l

So I create 50 files from test.1 to test.50 and so what? I tried again
and again, and always the same : test.26 is missing in the ls output!
Not any other file. When I create test.1 to test.65536, test.26 is
still missing.

BUT if you type "ls test.26", it shows up just OK. Yeah, that's weird.
Then I do 
rm * ; ls

and it tells me 
test.26

Uh.... 
If I create 2000 files but starting from 100 (test.100 to test.2100),
when I do

rm *; ls

It gives
test.125

Uh, the 26th file, again... This is a very strange modulo 26 bug
indeed :)

--

-- 
----------------------------------------
Emmanuel Florac     |   Intellique
----------------------------------------

Hi Emmanuel,

This is probably a bug in our readdir code, which may only be  
triggered by NFS doing readdirs in chunks of 26.  I'll look into it.

Thanks,
-sam

On Dec 4, 2007, at 8:49 AM, Emmanuel Florac wrote:

>
> OK, I've been testing again and again, and I found a weeeeeeird bug.
> When I say weird, I mean really really of the black magic flavour.
>
> So : still testing this NFS stuff, I tried with various configurations
> to sort it out, and it really a PVFS2 problem (tried different  
> clients,
> and exporting non-pvfs2 FS from the same server, all OK).
>
> The NFS server exports the pvfs2 FS this way :
>
> /mnt/pvfs2      *(rw,async,no_root_squash,no_subtree_check)
>
> On the client, I mount it and run this into the share :
>
> for i in `seq 1 50` ; do touch test.$i ; done ; echo "created $i" ;
> echo -n "listing " ; ls | wc -l
>
> So I create 50 files from test.1 to test.50 and so what? I tried again
> and again, and always the same : test.26 is missing in the ls output!
> Not any other file. When I create test.1 to test.65536, test.26 is
> still missing.
>
> BUT if you type "ls test.26", it shows up just OK. Yeah, that's weird.
> Then I do
> rm * ; ls
>
> and it tells me
> test.26
>
> Uh....
> If I create 2000 files but starting from 100 (test.100 to test.2100),
> when I do
>
> rm *; ls
>
> It gives
> test.125
>
> Uh, the 26th file, again... This is a very strange modulo 26 bug
> indeed :)
>
> -- 
> ----------------------------------------
> Emmanuel Florac     |   Intellique
> ----------------------------------------
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users <at> beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>

Le Tue, 4 Dec 2007 09:19:12 -0600
Sam Lang <slang <at> mcs.anl.gov> écrivait:

> This is probably a bug in our readdir code, which may only be  
> triggered by NFS doing readdirs in chunks of 26.  I'll look into it.

Great! Tomorrow I'll try again using CIFS to check if it makes any
difference.

--

-- 
----------------------------------------
Emmanuel Florac     |   Intellique
----------------------------------------

Sam,
Are you looking at this bug? If not, I can take a look.
Is this a readdir/pcache bug? Or is this related to the other
lookup/related cleanups that you sent an email
a couple of days back?
thanks!
Murali

On Dec 4, 2007 7:19 AM, Sam Lang <slang <at> mcs.anl.gov> wrote:
>
> Hi Emmanuel,
>
> This is probably a bug in our readdir code, which may only be
> triggered by NFS doing readdirs in chunks of 26.  I'll look into it.
>
> Thanks,
> -sam
>
>
> On Dec 4, 2007, at 8:49 AM, Emmanuel Florac wrote:
>
> >
> > OK, I've been testing again and again, and I found a weeeeeeird bug.
> > When I say weird, I mean really really of the black magic flavour.
> >
> > So : still testing this NFS stuff, I tried with various configurations
> > to sort it out, and it really a PVFS2 problem (tried different
> > clients,
> > and exporting non-pvfs2 FS from the same server, all OK).
> >
> > The NFS server exports the pvfs2 FS this way :
> >
> > /mnt/pvfs2      *(rw,async,no_root_squash,no_subtree_check)
> >
> > On the client, I mount it and run this into the share :
> >
> > for i in `seq 1 50` ; do touch test.$i ; done ; echo "created $i" ;
> > echo -n "listing " ; ls | wc -l
> >
> > So I create 50 files from test.1 to test.50 and so what? I tried again
> > and again, and always the same : test.26 is missing in the ls output!
> > Not any other file. When I create test.1 to test.65536, test.26 is
> > still missing.
> >
> > BUT if you type "ls test.26", it shows up just OK. Yeah, that's weird.
> > Then I do
> > rm * ; ls
> >
> > and it tells me
> > test.26
> >
> > Uh....
> > If I create 2000 files but starting from 100 (test.100 to test.2100),
> > when I do
> >
> > rm *; ls
> >
> > It gives
> > test.125
> >
> > Uh, the 26th file, again... This is a very strange modulo 26 bug
> > indeed :)
> >
> > --
> > ----------------------------------------
> > Emmanuel Florac     |   Intellique
> > ----------------------------------------
> >
> > _______________________________________________
> > Pvfs2-users mailing list
> > Pvfs2-users <at> beowulf-underground.org
> > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
> >
>
>

Hi, Murali,

My bad. The file permission issue was due to my mistake.
After following the steps in the documentation I was able to
manipulate files through Linux VFS aio.

Now I have another question. I've looked through the mailing list
but I wasn't able to find the relevant question (maybe I just
neglected).

I understand that we have to follow the steps to properly configure
kernel interface:

    * Create a mount point on the local filesystem
    * Load the Kernel Module into the running kernel
    * Start the PVFS2 Server application
    * Start the PVFS2 Client application
    * Mount your existing PVFS2 volume on the local filesystem
    * Issue VFS commands

I have managed to automate first 3 steps and the following 2 steps
were performed manually, which successfully allowed me to issue VFS
commands. Now I wonder how I automate pvfs2-client start-up.
I'm guessing that mounting pvfs2 volume can be done through fstab.
The documentation describes the manual pvfs2-client startup but not
automated one.

Thanks,
Ryuta

p.s. I'll post another thread for kernel 2.6.23-ARCH kmod build workaround.

Murali Vilayannur wrote:
> Great to hear that!!
> THe permission issue that you are seeing has been problematic..
> I thought we fixed it by making the root directory sticky.
> Alternatively, you can create a directory, chown it to your uid and then create
> files under that as the regular uid..
> Hope that helps!
> thanks,
> Murali
>
> PS: I think you should share this on the user mailing list so that
> others can benefit as well!
> On Dec 6, 2007 1:06 PM, ryuta <suzu0037 <at> aem.umn.edu> wrote:
>   
>> Hi Murali,
>>
>> I (finally) managed to build kmod on my linux-2.6.23-ARCH of Archlinux
>> kernel.
>> Basically, all I had to do fix kernel header definition in pvfs2-config.h.
>>
>> 1. Add -I/usr/lib/gcc/i686-pc-linux-gnu/4.2.2/include to
>>    src/kernel/linux-2.6/Makefile
>>
>>     ...
>>     EXTRA_CFLAGS = -I/usr/lib/gcc/i686-pc-linux-gnu/4.2.2/include \
>>         -$(aboslute_sr_dir)/ \
>>     ...
>>
>> In pvfs2-config.h
>>
>> 2. set #define HAVE_KZALLOC 1
>> 3. set #define HAVE_XATTR 1
>> 4. set #define HAVE_STRUCT_KMEM_CACHE 1
>> 5. set #define HAVE_AIO_VFS_SUPPORT 1
>> 6. set #define HAVE_VFSMOUNT_GETSB 1
>> 7. set #define HAVE_POSIX_ACL_H 1, instead of HAVE_LINUX_POSIX_ACL_H (bug?)
>> 8. set #define HAVE_LINUX_POSIX_ACL_XATTR_H 1
>> 9. set #define HAVE_PROC_HANDLER_SIX_ARG 1
>>
>> Lastly, we need to set CPATH=/usr/lib/gcc/i686-pc-linux-gnu/4.2.2/include
>> since it seems to be a deficiency on either pvfs2 or archlinux kernel module
>> build script.
>>
>> Actually, from step 2-9 (except for 7) the definitions are missing because
>> configure script fails due to missing stdarg.h which, strangely enough,
>> is not
>> included by gcc. So, I would say if I fix stdarg.h issue and build from
>> scratch
>> again, most failures would be automatically fixed, though I haven't
>> tried, yet.
>>
>>
>> As I pointed out above, HAVE_LINUX_POSIX_ACL_H is only defined by
>> pvfs2-config.h
>> (and hence pvfs2-config.h.in). The closest one is apparently
>> HAVE_POSIX_ACL_H.
>> This definition is relevant to
>>
>> ./src/kernel/linux-2.6/acl.c:23:#ifdef HAVE_POSIX_ACL_H
>> ./src/kernel/linux-2.6/pvfs2-kernel.h:72:#ifdef HAVE_POSIX_ACL_H
>>
>> I almost forgot to mention.
>> I'm building pvfs2 from CVS repository, "2.7.1pre1-2007-12-05-203448".
>>
>>
>> At last, I'd like to ask a simple (dumb) question:
>> Well, I've managed to mount pvfs2 file system and I can do
>> pvfs2-ping, pvfs2-cp, pvfs2-ls, etc. without problem.
>> Now that I loaded kernel module, I can access to /mnt/pvfs2 through
>> VFS command. Since I the mount point is made by root,
>>
>> drwxr-xr-x 2 root  root  4096 2007-12-06 14:18 pvfs2
>>
>> when I tried to cp or create a new file through vim the permission is
>> denied. I wonder if I should chmod 777 or is there other alternative method?
>> I was thinking to modify /etc/fstab from
>>
>> tcp://oroppas:3334/pvfs2-fs /mnt/pvfs2 pvfs2 default,noauto 0 0
>>
>> to
>>
>> tcp://oroppas:3334/pvfs2-fs /mnt/pvfs2 pvfs2 default,user,noauto 0 0
>>
>> but I'm not sure if this will do a trick.
>>
>>
>> Thanks,
>> Ryuta
>>
>>
>>
>>
>>
>>
>>
>> Murali Vilayannur wrote:
>>     
>>> Hmm.. This might be a fallout of some new linux kernel API changes..
>>> I will get to that sometime this weekend unless you want to send a
>>> patch fixing the build error  :)
>>> thanks,
>>> Murali
>>>
>>> On 8/10/07, ryuta <suzu0037 <at> aem.umn.edu> wrote:
>>>
>>>       
>>>>  I've managed to force gcc to include the path. I've gotten another error:
>>>>
>>>>  CC [M]  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.o
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:266: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:269: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:272: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:276: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:282: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:285: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:288: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:292: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:299: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:301: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:307: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:307: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:311: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:311: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:315: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:318: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:321: warning:
>>>> initialization from incompatible pointer type
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c: In function
>>>> 'pvfs2_proc_initialize':
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.c:340: error: too
>>>> many arguments to function 'register_sysctl_table'
>>>>  make[3]: *** [/usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-proc.o]
>>>> Error 1
>>>>  make[2]: *** [_module_/usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6] Error
>>>> 2
>>>>  make[1]: *** [default] Error 2
>>>>  make: *** [just_kmod] Error 2
>>>>
>>>>  I also attached configure kernel check. It produces some error though I
>>>> have no idea if those errors are fatal or not.
>>>>
>>>>
>>>>  Murali Vilayannur wrote:
>>>>  Hi Ryuta,
>>>> hmm.. that is weird. It is probably because gcc does not pick up
>>>> stdarg.h from its location.
>>>> Do you want to try and force gcc to include its default path which may
>>>> be omitted while building the kmods for some reason (perhaps due to
>>>> -nostdinc or -isystem?)
>>>> export CFLAGS=/usr/lib/gcc/i686-pc-linux-gnu/4.2.1/include
>>>> and then redoing the configure and make kmod?
>>>> I don't know..
>>>> thanks,
>>>> Murali
>>>>
>>>> On 8/10/07, ryuta <suzu0037 <at> aem.umn.edu> wrote:
>>>>
>>>>
>>>>  Hi,
>>>>
>>>>  I've successfully built and installed pvfs-2.3.6 on Arch Linux with kernel
>>>> linux-2.6.22-ARCH, however, I encountered a problem building kernel
>>>> interface.
>>>>
>>>>  My configure option is
>>>>
>>>>  ./configure --enable-shared --with-kernel=/usr/src/linux-2.6.22-ARCH
>>>>
>>>>  If you see the attached config.log you can find most of kernel-related
>>>> checking complains on the failure of including one header file "stdarg.h".
>>>>
>>>>  Nonetheless, the PVFS2 Configuration Information displays 2.6.x kernel
>>>> module as configured, and sure enough, "make kmod" simply fails due to
>>>> compilation error as gcc cannot find "stdarg.h"
>>>>
>>>>  [root <at> oroppas pvfs-2.6.3]# make kmod
>>>>  CC [M] /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.o
>>>>  In file included from ./include/linux/moduleparam.h:6,
>>>>  from
>>>> /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-kernel.h:45,
>>>>  from
>>>> /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.c:7:
>>>>  ./include/linux/kernel.h:10:20: error: stdarg.h: No such file or directory
>>>>  In file included from ./include/linux/moduleparam.h:6,
>>>>  from
>>>> /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-kernel.h:45,
>>>>  from
>>>> /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.c:7:
>>>>  ./include/linux/kernel.h:123: error: expected declaration specifiers or
>>>> '...' before 'va_list'
>>>>  ./include/linux/kernel.h:127: error: expected declaration specifiers or
>>>> '...' before 'va_list'
>>>>  ./include/linux/kernel.h:131: error: expected declaration specifiers or
>>>> '...' before 'va_list'
>>>>  ./include/linux/kernel.h:135: error: expected declaration specifiers or
>>>> '...' before 'va_list'
>>>>  ./include/linux/kernel.h:139: error: expected declaration specifiers or
>>>> '...' before 'va_list'
>>>>  ./include/linux/kernel.h:155: error: expected declaration specifiers or
>>>> '...' before 'va_list'
>>>>  In file included from
>>>> /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.c:7:
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-kernel.h:1267: error:
>>>> conflicting types for 'kzalloc'
>>>>  ./include/linux/slab_def.h:55: error: previous definition of 'kzalloc' was
>>>> here
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.c: In function
>>>> 'pvfs2_inode_removexattr':
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.c:1007: error:
>>>> 'XATTR_REPLACE' undeclared (first use in this function)
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.c:1007: error:
>>>> (Each undeclared identifier is reported only once
>>>>  /usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.c:1007: error:
>>>> for each function it appears in.)
>>>>  make[3]: *** [/usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6/pvfs2-utils.o]
>>>> Error 1
>>>>  make[2]: *** [_module_/usr/local/src/pvfs-2.6.3/src/kernel/linux-2.6] Error
>>>> 2
>>>>  make[1]: *** [default] Error 2
>>>>  make: *** [just_kmod] Error 2
>>>>
>>>>
>>>>
>>>>  On my installation, there are several "stdarg.h" just like most
>>>> installation
>>>>
>>>>  /usr/include/c++/4.2.1/tr1/stdarg.h
>>>>  /usr/lib/gcc/i686-pc-linux-gnu/4.2.1/include/stdarg.h
>>>>
>>>> /usr/lib/gcc/i686-pc-linux-gnu/4.2.1/install-tools/include/stdarg.h
>>>>  /usr/lib/klibc/include/stdarg.h
>>>>
>>>>  and gcc by default includes the path
>>>>
>>>>  /usr/lib/gcc/i686-pc-linux-gnu/4.2.1/include
>>>>
>>>>  as most gcc does on typical configuration.
>>>>
>>>>  I'm not quite sure if this issue caused by Arch Linux, GCC, or PVFS.
>>>>
>>>>  Any help will be greatly appreciated.
>>>>
>>>>  Thank you,
>>>>  Ryuta
>>>>
>>>> _______________________________________________
>>>> Pvfs2-users mailing list
>>>> Pvfs2-users <at> beowulf-underground.org
>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>         

Hi Dharani,
PVFS_sys_flush and PVFS_isys_flush should do that job..
There is no implicit flush on close although that would be a nice
thing to have I think.
If you are working on a patch for that, I can help you..
It maybe a good thing to have a flush-on-close flag that
pvfs2-client-core can use to
flush data to server disks for vfs API users atleast.
pvfs2_file_release in src/kernel/linux-2.6/file.c will also need
modifications to send an upcall
to flush file data.
On receiving the flush messages, the servers flush either keyval to db
or data to bstreams...
Hmm.. looking at PVFS_servreq_flush, it looks like we dont initialize
or use the flags member..
Anyways, i hope this helps
thanks,
Murali

On Dec 1, 2007 1:20 AM, Dharani Sankar Vijayakumar
<dharani.viji <at> gmail.com> wrote:
> May I please know:
> How to flush the data written using a pvfs API into the disk immediately?
> - Does PVFS_sys_flush() do the above?
> - Is there any equivalent PVFS api for fclose()? Does it FLUSH data to
> server disks on a close?
>
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users <at> beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>
>

18	May 2013
20	April 2013
13	March 2013
24	February 2013
21	January 2013
18	December 2012
4	November 2012
36	October 2012
17	September 2012
29	August 2012
73	July 2012
8	June 2012
38	May 2012
27	April 2012
43	March 2012
60	February 2012
44	January 2012
55	December 2011
21	November 2011
57	October 2011
31	September 2011
14	August 2011
50	July 2011
43	June 2011
30	May 2011
20	April 2011
43	March 2011
1	February 2011
43	January 2011
16	December 2010
5	November 2010
20	October 2010
1	September 2010
20	August 2010
13	July 2010
18	June 2010
23	May 2010
21	April 2010
32	March 2010
26	February 2010
26	January 2010
34	December 2009
15	November 2009
38	October 2009
16	September 2009
46	July 2009
24	June 2009
19	May 2009
42	April 2009
56	March 2009
30	February 2009
21	January 2009
9	December 2008
39	November 2008
40	October 2008
36	September 2008
17	August 2008
27	July 2008
25	June 2008
67	May 2008
21	April 2008
25	March 2008
29	February 2008
84	January 2008
15	December 2007
44	November 2007
58	October 2007
48	September 2007
29	August 2007
44	July 2007
52	June 2007
27	May 2007
33	April 2007
44	March 2007
66	February 2007
94	January 2007
55	December 2006
28	November 2006
25	October 2006
54	September 2006
35	August 2006
43	July 2006
36	June 2006
78	May 2006
60	April 2006
40	March 2006
29	February 2006
40	January 2006
59	December 2005
47	November 2005

flushing data

Re: NFS trouble : solved (mostly)

Re: 2.7.0/2.7.1pre: Two problems

Re: 2.7.0/2.7.1pre: Two problems

Re: NFS trouble : magic ahead!

Re: NFS trouble : magic ahead!

Re: NFS trouble : magic ahead!

Re: NFS trouble : magic ahead!

Re: building kmod on linux-2.6.22

Re: flushing data

Mon	Tue	Wed	Thu	Fri	Sat	Sun

					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30
31