Christoph Hellwig | 1 Sep 04:03 2014
Picon

detailed NOTIFY_DEVICEID4_CHANGE semantics?

RFC5661 is very sparse on NOTIFY_DEVICEID4_CHANGE semantics, and the
other layout RFCs don't mention it at all.  I'm particularly worried about
the very imprecise language that talks about outstanding I/O, but does
not mention outstanding layouts, which are the objects that refer to
deviceids in pNFS:

   NOTIFY_DEVICEID4_CHANGE
      A previously provided device-ID-to-device-address mapping has
      changed and the client uses GETDEVICEINFO to obtain the updated
      mapping.  The notification is encoded in a value of data type
      notify_deviceid_change4.  This data type also contains a boolean
      field, ndc_immediate, which if TRUE indicates that the change will
      be enforced immediately, and so the client might not be able to
      complete any pending I/O to the device ID.  If ndc_immediate is
      FALSE, then for an indefinite time, the client can complete
      pending I/O.  After pending I/O is complete, the client SHOULD get
      the new device-ID-to-device-address mappings before sending new
      I/O requests to the storage devices addressed by the device ID.

What is a pending I/O in this case?  As far as the MDS is concerned I/O
operations aren't visible in pNFS, we just have outstanding layouts
that can be commited and/or returned.

For my block layout client and server implementation I'm interpreting the
ndc_immediate = FALSE case in the following way:

 The client may finish any I/O using outstanding layouts that reference the
 previously provided device-ID-to-device-address mapping, and SHOULD use
 GETDEVICEINFO to obtain the updated mapping before requesting new layouts
 for the previously provided device-ID-to-device-address mapping.
(Continue reading)

David Noveck | 31 Aug 16:37 2014
Picon

updated comments re draft-ietf-nfsv4-lfs-registry-00

Supersedes previous version (gmail sent it before it was ready).

Here's my stab at an abstract of the document as I understood  its intent.  You can adopt it, in whole or in part, if it works for you.

In the past Mandatory Access Control (MAC) systems have used particular rigidly-specified policies which were implemented in particular protocols and platforms. As MAC systems become more widely deployed, additional flexibility in mechanism and policy will be required. While traditional trusted systems implemented Multi-Level Security (MLS) and integrity models, modern systems have expanded to include technologies such as type enforcement. Due to the wide range of policies and mechanisms whichneed to be accommodated, it is unlikely that use of a single security label  format and model will be viable.
To allow multiple MAC mechanisms and label formats to co-exist in a network, this document proposes a registry of label format specifications. This registry would contain label format identifiers and would provide for theassociation of each such identifier with a corresponding extensive document document outlining the exact syntax and use of the particular label format.

With regard to the second paragraph of section 1, I don't think you are harsh enough about the complexity.  It is not enough to say the complexity is "unneeded".  I'd say it is actively harmful.  How about the following:


One solution might be to define a single label format which consists of the
union of the requirements of all MAC models/implementations, known at a  given time. This approach is not desirable because it introduces an  environment where many MAC models would either have blank fields for many  of the label's components or where many implementations would ignore
many of values that are present altogether. The resulting complexity
would be likely to result in a confusing situation in which the interaction
of fields that that derive from different MAC models is never clearly
specified and the addition of new models or extension of existing models
is unduly difficult.
An additional consideration is that if a policy authority or identifier  field is specified in the label format it would require a robust  description that encompassed multiple MAC models in an environment in which implementations would be likely to be MAC-model specific

It also seems to me that with some limited reworking the last paragraph above and the existing last two paragraphs of section 1 could be made into a new section 1.1 devoted to policy administration

In section 2, there's a problem with the definition of "label format specification".  This same term has a different definition in section 6, and the definition in section 2 seems to match "Label format selector" in section 6. 
_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
David Noveck | 31 Aug 15:58 2014
Picon

Comments on draft-ietf-nfsv4-lfs-registry-00

Here's my stab at an abstract of the document as I understood  its intent.  You can adopt it, in whole or in part, if it works for you.

In the past Mandatory Access Control (MAC) systems have used particular rigidly-specified policies which were implemented in particular protocols and platforms. As MAC systems become more widely deployed, additional flexibility in mechanism and policy will be required. While traditional trusted systems implemented Multi-Level Security (MLS) and integrity models, modern systems have expanded to include technologies such as type enforcement. Due to the wide range of policies and mechanisms whichneed to be accommodated, it is unlikely that use of a single security label  format and model will be viable.
To allow multiple MAC mechanisms and label formats to co-exist in a network, this document proposes a registry of label format specifications. This registry would contain label format identifiers and would provide for theassociation of each such identifier with a corresponding extensive document document outlining the exact syntax and use of the particular label format.

With regard to the second paragraph of section 1, I don't think you are harsh enough about the complexity.  It is not enough to say the complexity is "unneeded".  I'd say it is actively harmful.  How about the following:

One solution might be to define a single label format which consists of the
union of the requirements of all MAC models/implementations, known at a  given time. This approach is not desirable because it introduces an  environment where many MAC models would either have blank fields for many  of the label's components or where many implementations would ignore
many of values that are present altogether. The resulting complexity
would be likely to result in a confusing situation in which the interaction
of fields that that derive from different MAC models is never clearly
specified and the addition of new models or extension of existing models
is unduly difficult.
An additional consideration is that if a policy authority or identifier  field is specified in the label format it would require a robust  description that encompassed multiple MAC models where implaementation would lock policy administration into the
described model.

In section 2, there's a problem with the definition of "label format specification".  This same term has a different definition in section 6, and the definition in section 2, seems to match "Label format selector" in section 6. 
_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
Matt W. Benjamin | 21 Aug 22:32 2014

Re: LAYOUTERROR vs LAYOUTSTATS [was Re: Capturing my items from Toronto]

Hi Tom,

Thanks for the pointer.  I'm unsure about the reasoning here.

First, while the protocol over which the operations being performed may not be uniquely determined by the
layout (in the case of some layout types), the set of possible protocols is so determined, and can be read
off the layout.  Whatever protocol the client is using to communicate with a DS, the MDS can be expected to
understand its error dialect.

The problem with "just return SERVERFAULT" is that it is just losing the interesting information.  We would
agree that needing to define allowed error returns is an undesirable feature of earlier layout drafts
(it's lossy, and terrible from a future-proofing perspective).  The opaque error return option seems
like the ideal resolution, since no error mappings are required at all in that approach.

Matt

----- "Tom Haynes" <thomas.haynes <at> primarydata.com> wrote:

> Hi Matt,
> 
> 
> BTW - here is the earlier thread on why we are this choice:
> 
> 
> https://www.ietf.org/mail-archive/web/nfsv4/current/msg13020.html
> 
> 
> Thanks,
> Tom
> 
> Hi,
> 
> I'm certainly being dense here, but what of layout types for which
> there is no useful mapping?
> 
> NFS4ERR_SERVERFAULT ?
> 
> The errors being reported are between the client and the storage
> device (which is over a given protocol). The only errors that can be
> reported are ones that the MDS can use to fix something on the storage
> device. Where fix may mean “no longer advertise that storage device in
> the layout”.
> 
> 
> 
> 
> More importantly, why is any mapping useful, when the server endpoints
> indirectly communicating are assured to share (via the layout type) a
> native dialect?
> 
> 
> 
> In the context of flexible files, we define loosely and tightly
> coupled servers. Your question assumes tightly coupled, but even in
> that scenario there is no such assurance. Go look at 5664 for an
> existing example - look at Section 8.1.
> 
> 
> 
> 
> 
> Matt
> 
> ----- "Tom Haynes" < thomas.haynes <at> primarydata.com > wrote:
> 
> 
> 
> The reason LAYOUTERROR does not need one is that in the context of
> LAYOUTRETURN it is already part of a blob. Further, it is the
> responsibility of the client to map storage device errors to NFSv4
> errors. I.e., the mapping layer removes the need for a blob.
> 
> --
> Matt Benjamin
> The Linux Box
> 206 South Fifth Ave. Suite 150
> Ann Arbor, MI 48104
> 
> http://linuxbox.com
> 
> tel. 734-761-4689
> fax. 734-769-8938
> cel. 734-216-5309
> 
> _______________________________________________
> nfsv4 mailing list
> nfsv4 <at> ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4

--

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309 

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
Spencer Shepler | 21 Aug 21:37 2014
Picon

WG last call for Requirements for pNFS Layout Types

 

Hi.  (yes, this is a different WG last call)

 

We are starting a working group last call for the following WG I-D….

 

Requirements for pNFS Layout Types (draft-ietf-nfsv4-layout-types-00.txt)

 

The I-D in your favorite format can be found here:

 

http://datatracker.ietf.org/doc/draft-ietf-nfsv4-layout-types/

 

Please review and provide comments to the author and the WG alias.

 

The last call will run from today through Sept 12th.

 

Spencer

 

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
Spencer Shepler | 21 Aug 21:32 2014
Picon

WG last call for Registry Specification for MAC Security Label Formats

 

Hi.  We are starting a working group last call for the following WG I-D….

 

Registry Specification for Mandatory Access Control (MAC) Security Label Formats (draft-ietf-nfsv4-lfs-registry-00.txt)

 

The I-D in your favorite format can be found here:

 

http://datatracker.ietf.org/doc/draft-ietf-nfsv4-lfs-registry/

 

Please review and provide comments to the authors and the WG alias.

 

The last call will run from today through Sept 12th.

 

 

Spencer

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
Thomas Haynes | 21 Aug 20:31 2014

[PATCH] Clarify the client stateid issue is for locking and not copy

From: Tom Haynes <Thomas.Haynes <at> primarydata.com>

Thanks Trond. Rather than rewrite the last two changes,
I've clarified in the context of the new section...

Signed-off-by: Tom Haynes <Thomas.Haynes <at> primarydata.com>
---
 nfsv42_middle_copy.xml | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/nfsv42_middle_copy.xml b/nfsv42_middle_copy.xml
index 5a83733..6f4da62 100644
--- a/nfsv42_middle_copy.xml
+++ b/nfsv42_middle_copy.xml
 <at>  <at>  -197,6 +197,18  <at>  <at> 
         can achieve this by a combination of OPEN and LOCK operations.
         I.e., either share or byte range locks might be desired.
       </t>
+
+      <t>
+	Note that when the client establishes a lock stateid on the
+	source, the context of that stateid is for the client and
+	not the destination.  As such, there might already be an
+	outstanding stateid, issued to the destination as client
+	of the source, with the same value as that provided for the
+	lock stateid. The source MUST equate the lock stateid as
+	that of the client, i.e., when the destination presents it
+	in the context of a inter-server copy, it is on behalf of
+	the client.
+      </t>
     </section>

     <section title="Client Caches">
 <at>  <at>  -610,17 +622,6  <at>  <at> 
       copy offload operation using a stateid with seqid of 0.  Therefore a
       copy offload stateid with seqid of 0 MUST be considered invalid.
     </t>
-
-    <t>
-      In the context of an inter-server copy, the copy offload
-      stateid is actually issued to the client and not the destination
-      server. As such, there might already be an outstanding stateid,
-      issued to the destination as client of the source, with the
-      same value as that provided for the copy offload stateid. The
-      source MUST equate the copy offload stateid as that of the
-      client, i.e., when the destination presents it in the context
-      of a inter-server copy, it is on behalf of the client.
-    </t>
   </section>

   <section anchor="ss:copy:security" title="Security Considerations">
--

-- 
1.9.3

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

Thomas Haynes | 21 Aug 20:04 2014

[PATCH] Warn implementors of the danger of client caching during a COPY

From: Tom Haynes <Thomas.Haynes <at> primarydata.com>

---
 nfsv42_middle_copy.xml | 29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/nfsv42_middle_copy.xml b/nfsv42_middle_copy.xml
index aaa789d..5a83733 100644
--- a/nfsv42_middle_copy.xml
+++ b/nfsv42_middle_copy.xml
 <at>  <at>  -189,13 +189,28  <at>  <at> 
     </t>
   </section>

-  <section title="Locking the Files">
-    <t>
-      Both the source and destination file may need to be locked
-      to protect the content during the copy operations. A client
-      can achieve this by a combination of OPEN and LOCK operations.
-      I.e., either share or byte range locks might be desired.
-    </t>
+  <section title="Implementation Considerations">
+    <section title="Locking the Files">
+      <t>
+        Both the source and destination file may need to be locked
+        to protect the content during the copy operations. A client
+        can achieve this by a combination of OPEN and LOCK operations.
+        I.e., either share or byte range locks might be desired.
+      </t>
+    </section>
+
+    <section title="Client Caches">
+      <t>
+        In a traditional copy, if the client is in the process of
+        writiting to the file before the copy (and perhaps with
+        a write  delegation), it will be straightforward to update
+        the destination server.  With an inter-server copy, the
+        source has no insight into the changes cached on the client.
+        The client SHOULD write back the data to the source or be
+        prepared for the destination to get a corrupt copy of the
+        file.
+      </t>
+    </section>
   </section>

   <section title="Intra-Server Copy">
--

-- 
1.9.3

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

Thomas Haynes | 21 Aug 19:47 2014

[PATCH] Clarify the use of the copy offload stateid

From: Tom Haynes <Thomas.Haynes <at> primarydata.com>

Signed-off-by: Tom Haynes <Thomas.Haynes <at> primarydata.com>
---
 nfsv42_middle_copy.xml | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/nfsv42_middle_copy.xml b/nfsv42_middle_copy.xml
index 7aff891..aaa789d 100644
--- a/nfsv42_middle_copy.xml
+++ b/nfsv42_middle_copy.xml
 <at>  <at>  -44,7 +44,7  <at>  <at> 
     <t>
       The new operations are designed to copy files. Other
       file system objects can be copied by building on these operations
-      or using other techniques. For example if the user wishes to copy
+      or using other techniques. For example, if the user wishes to copy
       a directory, the client can synthesize a directory copy by first
       creating the destination directory and then copying the source
       directory's files to the new destination directory.
 <at>  <at>  -595,6 +595,17  <at>  <at> 
       copy offload operation using a stateid with seqid of 0.  Therefore a
       copy offload stateid with seqid of 0 MUST be considered invalid.
     </t>
+
+    <t>
+      In the context of an inter-server copy, the copy offload
+      stateid is actually issued to the client and not the destination
+      server. As such, there might already be an outstanding stateid,
+      issued to the destination as client of the source, with the
+      same value as that provided for the copy offload stateid. The
+      source MUST equate the copy offload stateid as that of the
+      client, i.e., when the destination presents it in the context
+      of a inter-server copy, it is on behalf of the client.
+    </t>
   </section>

   <section anchor="ss:copy:security" title="Security Considerations">
--

-- 
1.9.3

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

Christoph Hellwig | 16 Aug 16:35 2014
Picon

[internet-drafts <at> ietf.org: New Version Notification for draft-hellwig-nfsv4-rfc5663bis-00.txt]

I've published a first attempt at starting the bis process for rfc5663.

The major driver for this was that rfc5663 entirely ignores the concept
of volatile write caches on storage devices.  This draft adds simple
language that requires the server to ensure data is on stable storage
after a LAYOUTCOMMIT returns.

The other non-housekeeping change is a much stricter language in section
2.3.5. to enforce the single writer XOR multiple readers exclusion.  The
current text contains a lot of mays but ignores that fact that block devices
generally do not allow byte level access, and thus the server mus
ensure that multiple read-modify-write cycles don't step on each others
toes.

----- Forwarded message from internet-drafts <at> ietf.org -----

Date: Sat, 16 Aug 2014 03:12:27 -0700
From: internet-drafts <at> ietf.org
Subject: New Version Notification for draft-hellwig-nfsv4-rfc5663bis-00.txt
To: Christoph Hellwig <hch <at> lst.de>, Christoph Hellwig <hch <at> lst.de>

A new version of I-D, draft-hellwig-nfsv4-rfc5663bis-00.txt
has been successfully submitted by Christoph Hellwig and posted to the
IETF repository.

Name:		draft-hellwig-nfsv4-rfc5663bis
Revision:	00
Title:		Parallel NFS (pNFS) Block/Volume Layout
Document date:	2014-08-16
Group:		Individual Submission
Pages:		27
URL:            http://www.ietf.org/internet-drafts/draft-hellwig-nfsv4-rfc5663bis-00.txt
Status:         https://datatracker.ietf.org/doc/draft-hellwig-nfsv4-rfc5663bis/
Htmlized:       http://tools.ietf.org/html/draft-hellwig-nfsv4-rfc5663bis-00

Abstract:
   Parallel NFS (pNFS) extends Network File Sharing version 4 (NFSv4) to
   allow clients to directly access file data on the storage used by the
   NFSv4 server.  This ability to bypass the server for data access can
   increase both performance and parallelism, but requires additional
   client functionality for data access, some of which is dependent on
   the class of storage used.  The main pNFS operations document
   specifies storage-class-independent extensions to NFS; this document
   specifies the additional extensions (primarily data structures) for
   use of pNFS with block- and volume-based storage.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

----- End forwarded message -----

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

Fred Isaman | 31 Jul 16:03 2014
Picon

error on OPEN(CLAIM_DELEGATE_CUR)

This popped up on the linux client, but is a general protocol question...

What is the client supposed to do when it receives an error (in this case ADMIN_REVOKED or BAD_STATEID) to an OPEN(CLAIM_DELEGATE_CUR) sent during delegation return?

One option is to try to OPEN(CLAIM_NULL) the file and dump local data, but that (and pretty much every other option I've come up with) seems a recipe for data corruption.

Fred
_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

Gmane