headers
Marc Eshel | 2 Oct 2005 22:45
Picon
Favicon

pNFS - file commit

What does regular file commit to the metadata server mean for a file with 
a layout? I would like it to mean the same as it means for a file without 
layouts. So the client has the option of calling layout-commit to all the 
data server or have one call to the metadata server that will accomplish 
the same. The benefit of the one call to the MDS is that the metadata will 
not be written multiple times to disk. The MDS must write metadata to disk 
for each DS layout-commit but when it is driven from the MDS with file 
commit it can request all DS to commit their data to disk and when they 
are all done make one update to the metadata on disk.
Marc. 

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www1.ietf.org/mailman/listinfo/nfsv4
Permalink | Reply | 23 comments |
headers
Garth Goodson | 3 Oct 2005 08:38
Picon

Re: pNFS - file commit

Marc Eshel wrote:
> What does regular file commit to the metadata server mean for a file with 
> a layout? I would like it to mean the same as it means for a file without 
> layouts. So the client has the option of calling layout-commit to all the 
> data server or have one call to the metadata server that will accomplish 
> the same. The benefit of the one call to the MDS is that the metadata will 
> not be written multiple times to disk. The MDS must write metadata to disk 
> for each DS layout-commit but when it is driven from the MDS with file 
> commit it can request all DS to commit their data to disk and when they 
> are all done make one update to the metadata on disk.
> Marc. 
> 

LAYOUTCOMMIT is an operation to the metadata server, only.  The backend 
takes care of any synchronization.  COMMIT to the metadata server makes 
sure data written asynchronously through the metadata server is visible 
at the data servers (upon completion).  COMMIT to the data servers does 
what COMMIT normally does.

-Garth

> _______________________________________________
> nfsv4 mailing list
> nfsv4 <at> ietf.org
> https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www1.ietf.org/mailman/listinfo/nfsv4
(Continue reading)

Permalink | Reply | 22 comments |
headers
Halevy, Benny | 3 Oct 2005 07:42
Favicon

RE: pNFS - file commit

Marc, my comments below...

> -----Original Message-----
> From: nfsv4-bounces <at> ietf.org 
> [mailto:nfsv4-bounces <at> ietf.org]On Behalf Of
> Marc Eshel
> Sent: Sunday, October 02, 2005 11:46 PM
> To: nfsv4 <at> ietf.org
> Subject: [nfsv4] pNFS - file commit
> 
> 
> What does regular file commit to the metadata server mean for 
> a file with 
> a layout? I would like it to mean the same as it means for a 
> file without 
> layouts. 

right, COMMIT should mean and do the same thing it does for a
file without layouts.

> So the client has the option of calling 
> layout-commit to all the 
> data server or have one call to the metadata server that will 
> accomplish 
> the same. 

Hmm, the way I read section 3.6 in draft-welch-pnfs-ops-04.txt
the MDS has to flush dirty data to the data servers when it
gets a COMMIT op, but it doesn't say it has to COMMIT it there
as well (or the equivalent in other storage types).
(Continue reading)

Permalink | Reply | 1 comment |
headers
Marc Eshel | 3 Oct 2005 18:13
Picon
Favicon

Re: pNFS - file commit

Yes, I mixed up commit and layout commit. My point was the commit on the 
data server needs to communicate with the metadata server using the 
backend protocol and MDS would have to commit metadata to disk. There 
should be a call to the MDS that commits the entire file by calling commit 
file on all data servers. Is it commit, commit layout or both are needed? 
I don't see the difference between commit layout and commit file for the 
file protocol.
Marc. 

Garth Goodson wrote on 10/02/2005 11:38:24 PM:

> Marc Eshel wrote:
> > What does regular file commit to the metadata server mean for a file 
with 
> > a layout? I would like it to mean the same as it means for a file 
without 
> > layouts. So the client has the option of calling layout-commit to all 
the 
> > data server or have one call to the metadata server that will 
accomplish 
> > the same. The benefit of the one call to the MDS is that the metadata 
will 
> > not be written multiple times to disk. The MDS must write metadata to 
disk 
> > for each DS layout-commit but when it is driven from the MDS with file 

> > commit it can request all DS to commit their data to disk and when 
they 
> > are all done make one update to the metadata on disk.
> > Marc.
(Continue reading)

Permalink | Reply | 21 comments |
headers
Garth Goodson | 3 Oct 2005 18:31
Picon

Re: pNFS - file commit

Marc Eshel wrote:
> Yes, I mixed up commit and layout commit. My point was the commit on the 
> data server needs to communicate with the metadata server using the 
> backend protocol and MDS would have to commit metadata to disk. There 
> should be a call to the MDS that commits the entire file by calling commit 
> file on all data servers. Is it commit, commit layout or both are needed? 
> I don't see the difference between commit layout and commit file for the 
> file protocol.
> Marc. 
> 
Commit need not propagate to the MDS (thus, we have layoutcommit). 
COMMIT on the data server just makes asynchronously written data visible 
at that data server, it does not propagate attributes, that is what 
LAYOUTCOMMIT is for (and hence the difference).  LAYOUTCOMMIT called at 
the MDS is responsible for syncing the metadata between the MDS and all 
DSs.  (At least this is how I have viewed it).  As I stated earlier, 
COMMIT at the metadata server is responsible for pushing out and 
committing data written through the metadata server to the data servers 
(I don't think it needs to COMMIT asynchronously written data that was 
written to the data servers -- that is what COMMIT at the data servers 
is for).

-Garth

> Garth Goodson wrote on 10/02/2005 11:38:24 PM:
> 
> 
>>Marc Eshel wrote:
>>
>>>What does regular file commit to the metadata server mean for a file
(Continue reading)

Permalink | Reply | 2 comments |
headers
Dean Hildebrand | 3 Oct 2005 18:33
Picon
Favicon

Re: pNFS - file commit

I think both are needed.  COMMIT syncs the data to disk.  LAYOUTCOMMIT
commits the layout and updates the server with the last byte written.
For the file layout, the length of the file still need to be updated, so a
LAYOUTCOMMIT is still necessary.  I would assume they can be sent in the
same compound.

Dean

On Mon, 3 Oct 2005, Marc Eshel wrote:

> Yes, I mixed up commit and layout commit. My point was the commit on the
> data server needs to communicate with the metadata server using the
> backend protocol and MDS would have to commit metadata to disk. There
> should be a call to the MDS that commits the entire file by calling commit
> file on all data servers. Is it commit, commit layout or both are needed?
> I don't see the difference between commit layout and commit file for the
> file protocol.
> Marc.
>
> Garth Goodson wrote on 10/02/2005 11:38:24 PM:
>
> > Marc Eshel wrote:
> > > What does regular file commit to the metadata server mean for a file
> with
> > > a layout? I would like it to mean the same as it means for a file
> without
> > > layouts. So the client has the option of calling layout-commit to all
> the
> > > data server or have one call to the metadata server that will
> accomplish
(Continue reading)

Permalink | Reply | 17 comments |
headers
Marc Eshel | 3 Oct 2005 18:30
Picon
Favicon

RE: pNFS - file commit

Writing to the MDS will having layout outstanding is to confusing. I think 
that we should either use layouts for all read/writes that will direct all 
clients to go to the same DS for a given byte range or just do I/O through 
the MDS to satisfy the requirement that all data must be reachable through 
the MDS. I was suggesting that full file commit on the MDS will mean file 
commit on all data servers for data that was written directly to the DS so 
the metadata can be updated once.
Marc. 

nfsv4-bounces <at> ietf.org wrote on 10/02/2005 10:42:21 PM:

> Marc, my comments below...
> 
> > -----Original Message-----
> > From: nfsv4-bounces <at> ietf.org 
> > [mailto:nfsv4-bounces <at> ietf.org]On Behalf Of
> > Marc Eshel
> > Sent: Sunday, October 02, 2005 11:46 PM
> > To: nfsv4 <at> ietf.org
> > Subject: [nfsv4] pNFS - file commit
> > 
> > 
> > What does regular file commit to the metadata server mean for 
> > a file with 
> > a layout? I would like it to mean the same as it means for a 
> > file without 
> > layouts. 
> 
> right, COMMIT should mean and do the same thing it does for a
> file without layouts.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dean Hildebrand | 3 Oct 2005 18:40
Picon
Favicon

Re: pNFS - file commit

Just to clarify, I agree with Marc about full file commit to the MDS
commits all data on all data servers.  I think this could be extended to a
byte-range too.  Am I right to assume it is implementation dependent
whether a client sends his commits through the MDS or directly to the data
servers?

Dean

On Mon, 3 Oct 2005, Dean Hildebrand wrote:

> I think both are needed.  COMMIT syncs the data to disk.  LAYOUTCOMMIT
> commits the layout and updates the server with the last byte written.
> For the file layout, the length of the file still need to be updated, so a
> LAYOUTCOMMIT is still necessary.  I would assume they can be sent in the
> same compound.
>
> Dean
>
> On Mon, 3 Oct 2005, Marc Eshel wrote:
>
> > Yes, I mixed up commit and layout commit. My point was the commit on the
> > data server needs to communicate with the metadata server using the
> > backend protocol and MDS would have to commit metadata to disk. There
> > should be a call to the MDS that commits the entire file by calling commit
> > file on all data servers. Is it commit, commit layout or both are needed?
> > I don't see the difference between commit layout and commit file for the
> > file protocol.
> > Marc.
> >
> > Garth Goodson wrote on 10/02/2005 11:38:24 PM:
(Continue reading)

Permalink | Reply | 16 comments |
headers
Marc Eshel | 3 Oct 2005 18:48
Picon
Favicon

Re: pNFS - file commit

Garth Goodson <Garth.Goodson <at> netapp.com> wrote on 10/03/2005 09:31:36 AM:

> Marc Eshel wrote:
> > Yes, I mixed up commit and layout commit. My point was the commit on 
the 
> > data server needs to communicate with the metadata server using the 
> > backend protocol and MDS would have to commit metadata to disk. There 
> > should be a call to the MDS that commits the entire file by calling 
commit 
> > file on all data servers. Is it commit, commit layout or both are 
needed? 
> > I don't see the difference between commit layout and commit file for 
the 
> > file protocol.
> > Marc. 
> > 
> Commit need not propagate to the MDS (thus, we have layoutcommit). 
> COMMIT on the data server just makes asynchronously written data visible 

> at that data server, it does not propagate attributes, that is what 

I assume you mean it makes it visible from all data servers ? I am not 
sure how you do that without communicating with the MDS.

> LAYOUTCOMMIT is for (and hence the difference).  LAYOUTCOMMIT called at 
> the MDS is responsible for syncing the metadata between the MDS and all 
> DSs.  (At least this is how I have viewed it).  As I stated earlier, 
> COMMIT at the metadata server is responsible for pushing out and 
> committing data written through the metadata server to the data servers 
> (I don't think it needs to COMMIT asynchronously written data that was
(Continue reading)

Permalink | Reply | 1 comment |
headers
Garth Goodson | 3 Oct 2005 18:45
Picon

Re: pNFS - file commit

Dean Hildebrand wrote:
> Just to clarify, I agree with Marc about full file commit to the MDS
> commits all data on all data servers.  

I don't agree.  We want to limit the operations that dictate how the 
control protocol is to be used.  In this case, it is just as simple to 
insert a COMMIT into some compound going to the data servers.

See my previous post (pasted below) for how I believe it should work.

-Garth

Commit need not propagate to the MDS (thus, we have layoutcommit). 
COMMIT on the data server just makes asynchronously written data visible 
at that data server, it does not propagate attributes, that is what 
LAYOUTCOMMIT is for (and hence the difference).  LAYOUTCOMMIT called at 
the MDS is responsible for syncing the metadata between the MDS and all 
DSs.  (At least this is how I have viewed it).  As I stated earlier, 
COMMIT at the metadata server is responsible for pushing out and 
committing data written through the metadata server to the data servers 
(I don't think the MDS needs to COMMIT asynchronously written data that 
was written to the data servers -- that is what COMMIT at the data 
servers is for).

I think this could be extended to a
> byte-range too.  Am I right to assume it is implementation dependent
> whether a client sends his commits through the MDS or directly to the data
> servers?
> 
> Dean
(Continue reading)

Permalink | Reply | 8 comments |

Gmane