yong | 26 Aug 09:57 2015
Picon

Fwd: New Non-WG Mailing List: Storagesync -- Mechanisms to synchronize client file systems with Internet-based data storage services

Hi all,
 
A new mailing list has been established for Internet Storage Sync (ISS) work. We've made a relevant presentation about this work in Prague (in appsawg session). We will start related discussions (e.g. problems, scope of work and etc.) on it soon and any interested people are more than welcome to subscribe it. Following please find the list address and detailed purpose of the mailing list.
 
Regards,
Yong Cui,
 
Professor, Tsinghua University, Beijing, China
Associate Editor on IEEE TPDS, IEEE TCC
---------- Forwarded message ----------
From: IETF Secretariat <ietf-secretariat <at> ietf.org>
Date: 2015-08-21 0:51 GMT+08:00
Subject: New Non-WG Mailing List: Storagesync -- Mechanisms to synchronize client file systems with Internet-based data storage services
To: IETF Announcement List <ietf-announce <at> ietf.org>
Cc: cuiyong <at> tsinghua.edu.cn, Storagesync <at> ietf.org


A new IETF non-working group email list has been created.

List address: storagesync <at> ietf.org
Archive: https://mailarchive.ietf.org/arch/search/?email_list=storagesync
To subscribe: https://www.ietf.org/mailman/listinfo/storagesync

Purpose:

Network-based storage services, such as cloud storage, attract huge numbers of users and produce a significant share of Internet traffic. But most existing network storage services make use of proprietary sync protocols to achieve data synchronization. Almost all of them have proved not to be efficient enough and should be improved. This mailing list is for discussion of problems caused by inefficient proprietary sync protocols and of alternatives that can be more efficient and standardized.

For additional information, please contact the list administrators.


_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
Benny Halevy | 25 Aug 18:02 2015
Picon

LC comments on draft-ietf-nfsv4-flex-files-06

Hi Tom,

Overall, the document looks great.
Thanks much for your excellent work.

When reviewing this last version in details produced
the comments and issues below.  Nothing earth shattering, though
not all comments are entirely minor....

Thanks,

Benny

* Wordsmithing in Section 2.1. LAYOUTCOMMIT:
- Using "preceded" rather than "proceeded" better describes
the requirement for sending COMMIT to the DSs before LAYOUTCOMMIT
to the MDS.

- COMMIT to the DS is redundant if the DS replied with
stable_how=FILE_SYNC, yet this is not clearly spelled out
in the section.  How about the following? Change
   With a loosely coupled system, a LAYOUTCOMMIT to the metadata server MUST
   be proceeded with a COMMIT to the storage device.  It is the
   responsibility of the client to make sure the data file is stable
   before the metadata server begins to query the storage devices about
   the changes to the file.
To
   It is the responsibility of the client to make sure the data file is stable
   before the metadata server begins to query the storage devices about
   the changes to the file.
   With a loosely coupled system, if any WRITE to a storage device did not
   result with stable_how equal to FILE_SYNC, a LAYOUTCOMMIT to the metadata
   server MUST be preceded with a COMMIT to the storage device.

* 2.2.  Fencing Clients from the Data Server
- nit: s/it forced/it is forced/

* 2.2.1.  Implementation Notes for Synthetic uids/gids
- "An implementation might allow an administrator
   to restrict a range of such ids in the name servers"
   I suggest adding a definition for "name server" in the Definitions section.
   (ditto for further use of "name services")

* 2.3.  State and Locking Models
- "Standalone" and "clustered" are used here with respect to NFSv4.1+
  storage devices.  I suggest adding some text to clarify how these relate
  to loose vs tight coupling.

* 4.1.  ff_device_addr4
-  If ffdv_version equals 3
   then server MUST set ffdv_minorversion to 0 and the client MUST
   access the storage device using the NFSv3 protocol [RFC1813].

   I suggest also adding that the server MUST set ffdv_tightly_coupled
   to false in this case.

- ffdv_tightly_coupled COMMIT semantics:
   I.e., the writes MUST be committed by the
   client to stable storage via issuing WRITEs with stable_how ==
   FILE_SYNC or by issuing a COMMIT after WRITEs with stable_how !=
   FILE_SYNC (see Section 3.3.7 of [RFC1813]).

   The spec provides only two complimentary options, yet it is possible
   (and quite common) that the storage devices always supports stable storage
   regardless of what the client set for stable_how when issuing the WRITE
   call and in this cases the device will reply to the WRITE with "committed"
   = FILE_SYNC.  Note that the storage device MUST do so if the client set stable_how
   to FILE_SYNC in the request but it MAY do so in its own discretion.  The point is
   that what matters is that the data file data and attrs are stable so I'd prefer
   this requirement to be phrased along the following lines:

   I.e., the writes MUST be committed by the
   storage device to stable storage via issuing WRITEs resulting with stable_how ==
   FILE_SYNC or by issuing a COMMIT after WRITEs with stable_how !=
   FILE_SYNC (see Section 3.3.7 of [RFC1813]).

* 4.2.  Storage Device Multipathing
- "This array
   (data type multipath_list4) represents a list of storage device (each
   identified by a network address), with the possibility that some
   storage device will appear in the list multiple times."

  This is a bit confusing...
  a) (nit) "storage device" is mentioned here as singular and not plural
  b) The list does not necessarily represents multiple storage devices
  but rather multiple storage device network addresses that may or may not
  refer to multiple devices.

  Since all the conceptual devices addressed by the array need to recognize
  the filehandle representing the data file (ffda_netaddrs is decoupled from
  ffda_versions/ffds_fh_vers, right?) it is fair to consider them a
  single storage device service, that might or might not be monolithic, clustered,
  or distributed, yet from the client's point of view it's a single logical service.
  So my conclusion is to stick to talking about multiple network addresses to a
  single logical device and refrain from discussing its internal architecture in
  the general case and whether it might be comprised of several, separate devices.

- The last section talks about special consideration for NFSv4.1+ devices.
  I'd leave it for drilling in to the non-monolithic cases. However, the example given:

   For example, the data could be read-only, and
   the data consist of exact replicas.

   Seems to be more elegantly solved by providing a (read only) mirrored layout
   rather than representing the mirrors in the device address - this is also apparent
   in Section 5.2.  Interactions Between Devices and Layouts.
   If we don't have a real use case we can simplify the spec and and allow multipathing
   only for clustered NFSv4.1+ DSs.

* 5.  Flexible File Layout Type
- Reference to the encapsulating NFSv4.1 structures was removed from section 4
  (GETDEVICEINFO). It might be a good idea to remove it here too (struct layout_content4
  and struct layout4).  This could be achieved a-la:

   The ff_layout4 data structure is returned by the server as the
   storage protocol specific opaque field loc_body in the
   layout_content4 structure by a successful LAYOUTGET operation.

- ffl_stripe_unit: is it really needed with mirroring only?  It was clearly needed
  when we supported RAID-0 striping (with the distinction of sparse vs. dense
  striping), but with mirroring I am not sure about its value.
  This also puts to the utility of Section 6 in question, with sparse striping and
  holes, the different data files are no longer mirrored replicas of each other.

* 8.  Mirroring
- nit: s/If a layout segments is being resilvered/If a layout segment is being resilvered/

* 8.2.  Writing to Mirrors
- The text refers to this specific case:
   If all but one copy
   is updated successfully and the last one provides an error, then the
   client needs to return the layout to the metadata server with an
   error indicating that the update failed to that storage device.

  How about the cases where more than one device failed?  How do they differ?
  What's expected from the client and mds in these cases?

* 9.  Flexible Files Layout Type Return
- ditto regarding verbatim copy of NFSv4.1 XDR.

* 9.2.1.  ff_io_latency4
- "Note that LAYOUTSTATS are cummulative, i.e., not reset each time the
   operation is sent."

  Do they reset when the layout stateid resets? (layout revoked or returned
  in full, either explicitly or implicitly with return-on-close)

  If so (and there are good reasons to define it so), this also covers
  the client restart case.

  Also, spelling mistake in "cummulative"?  should be "cumulative" I think...

- "If two RPC calls originating from the same NFS
   client are processed at the same time by the metdata server, then the
   call containing the larger values contains the most recent time
   series data."

  It is technically important to restrict that to the same file (and layout
  stateid.other if it resets the statistics).

  Also, typo in "metdata".

* 9.2.2.  ff_layout update4
- "In the case of multipathing, ffl_fhandle
   indicates which read-only copy was selected."

  As mentioned above, the layout filehandles are decoupled from multipathing
  and are related to multiple nfs versions for same device so this is a bit confusing.

* 10.  Flexible Files Layout Type LAYOUTERROR
  11.  Flexible Files Layout Type LAYOUTSTATS
- "it can use LAYOUT(ERROR|STATS)",
  it MAY? it SHOULD? either would probably be a better choice than "can"
  (I'd vote for SHOULD)

* 13.  Recalling Layouts
- How about the resilvering case?  Is it considered a "sharing conflict"?
  Maybe just spell it out explicitly for completeness...

* 14.  Client Fencing
- "the server MAY revoke client layouts and/
   or device address mappings"

  How would the server revoke device address mappings?

* 15.  Security Considerations
- "The control path contains all the new operations described by this
   extension;"

  copy-paste from RFC5661?

- "When the client acts on I/O operations on
   behalf of its local users, it MUST authenticate and authorize the
   user by issuing respective OPEN and ACCESS calls to the metadata
   server, similar to having NFSv4 data delegations.  If access is
   allowed, the client uses the corresponding (READ or RW) credentials
   to perform the I/O operations at the data file's storage devices."

  This seems to refer only to the tight coupling strategy, not to loose coupling.

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

Spencer Shepler | 24 Aug 22:39 2015
Picon

NFSv4 WG Last Call for Parallel NFS (pNFS) Flexible File Layout (ends Sept 4th)

 

We are starting a last call for the Parallel NFS (pNFS) Flexible File Layout I-D.


This is to cover comments from IESG and to move it to standards track.

 

The text document is located here:

https://www.ietf.org/id/draft-ietf-nfsv4-flex-files-06.txt

 

and the meta-link is located here:

http://datatracker.ietf.org/doc/draft-ietf-nfsv4-flex-files/

 

The last call will end on Sept 4th.

 

Please direct questions/comments to the WG alias.

 

Thanks,
Spencer

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
internet-drafts | 24 Aug 20:18 2015
Picon

I-D Action: draft-ietf-nfsv4-migration-issues-08.txt


A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the Network File System Version 4 Working Group of the IETF.

        Title           : NFSv4 migration: Implementation experience and spec issues to resolve
        Authors         : David Noveck
                          Piyush Shivam
                          Charles Lever
                          Bill Baker
	Filename        : draft-ietf-nfsv4-migration-issues-08.txt
	Pages           : 28
	Date            : 2015-08-24

Abstract:
   The migration feature of NFSv4 provides for moving responsibility for
   a single filesystem from one server to another, without disruption to
   clients.  Recent implementation experience has shown problems in the
   existing specification for this feature.  This document discusses the
   issues which have arisen, explores the options available for curing
   the issues, and explains the choices to be made in updating the
   NFSv4.0 and NFSv4.1 specifications, to properly address migration.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-nfsv4-migration-issues/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-nfsv4-migration-issues-08

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-migration-issues-08

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

internet-drafts | 19 Aug 07:09 2015
Picon

I-D Action: draft-ietf-nfsv4-xattrs-01.txt


A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the Network File System Version 4 Working Group of the IETF.

        Title           : File System Extended Attributes in NFSv4
        Authors         : Manoj Naik
                          Marc Eshel
	Filename        : draft-ietf-nfsv4-xattrs-01.txt
	Pages           : 26
	Date            : 2015-08-18

Abstract:
   This document proposes extensions to the NFSv4 protocol which allow
   file extended attributes (hereinafter also referred to as xattrs) to
   be manipulated using NFSv4.  An xattr is a file system feature that
   allows opaque metadata, not interpreted by the file system, to be
   associated with files and directories.  Such support is present in
   many modern local file systems.  New file attributes are proposed to
   allow clients to query the server for xattr support, and new
   operations to get and set xattrs on file system objects are provided.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-nfsv4-xattrs/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-nfsv4-xattrs-01

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-xattrs-01

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

Mike Kupfer | 18 Aug 02:14 2015
Picon

(mostly) nits in draft-ietf-nfsv4-multi-domain-fs-reqs-04

The one technical concern that I have is with the new text in Section 4
about not using stringified identifiers.  Maybe I'm just confused, but
doesn't the ACE4_IDENTIFIER_GROUP flag tell whether the identifier
refers to a user or a group?  Otherwise, there's no way to distinguish a
user from a group with the same name, even using the name <at> domain form.
So wouldn't the ACE4_IDENTIFIER_GROUP flag work just as well for
stringified identifiers in a stand-alone domain deployment?

Section 2:
  - under "Local representation of identity":
    - "such as a a uidNumber" -> "such as a uidNumber"
    - "gidNumber (GID) [RFC2307], a Windows Security Identifier" ->
      "gidNumber (GID) [RFC2307], or a Windows Security Identifier"
    - "the identifier space for user and groups" -> "the identifier
      space for users and groups"
    - "requiring the one" -> "requiring anyone" or "requiring anything"

Section 3.1:
  - missing period at the end of "interoperate in a multi-domain
    environment"

Section 3.2:
  - "A client obtaining value" -> "A client that receives the values"
    (though I suppose "A client obtaining values" would work).
  - "attributes in a a local form" -> "attributes in a local form"

Section 4 (assuming that the 4th paragraph should be kept at all):
  - "aces cannot contain" -> "ACEs cannot contain"
  - "stringified id's" -> "stringified identifiers" (or "stringified
    IDs")
  - "This is opposed to" -> "This is different from"

Section 4.2:
  - "Another possibility is express" -> "Another possibility is to
    express"
  - "rather than using use" -> "rather than using"

cheers,
mike

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

internet-drafts | 14 Aug 18:25 2015
Picon

I-D Action: draft-ietf-nfsv4-multi-domain-fs-reqs-04.txt


A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the Network File System Version 4 Working Group of the IETF.

        Title           : Multiple NFSv4 Domain Namespace Deployment Guidelines
        Authors         : William A. (Andy) Adamson
                          Nicolas Williams
	Filename        : draft-ietf-nfsv4-multi-domain-fs-reqs-04.txt
	Pages           : 15
	Date            : 2015-08-14

Abstract:
   This document discusses issues relevant to the deployment of the
   NFSv4 protocols in situations allowing for the construction of an
   NFSv4 file namespace supporting the use of multiple NFSv4 domains and
   utilizing multi-domain capable file systems.  Also described are
   constraints on name resolution and security services appropriate to
   the administration of such a system.  Such a namespace is a suitable
   way to enable a Federated File System supporting the use of multiple
   NFSv4 domains.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-nfsv4-multi-domain-fs-reqs/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-nfsv4-multi-domain-fs-reqs-04

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-multi-domain-fs-reqs-04

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4

Spencer Shepler | 12 Aug 20:02 2015
Picon

WG Last call for NFSv4.0 migration: Specification Update - ends Sept 4th.

 

Per Dave’s request, we are starting WG last call for:

 

NFSv4.0 migration: Specification Update

http://datatracker.ietf.org/doc/draft-ietf-nfsv4-rfc3530-migration-update/

 

You will find the text version of the document located here:

 

https://www.ietf.org/id/draft-ietf-nfsv4-rfc3530-migration-update-07.txt

 

Given the end of summer timing, we will run this last call through Sept 4th.

 

Please let me know if this is insufficient time for the review.

 

As always, comments should be directed to the author and the WG alias.

 

Thanks,

Spencer

 

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
David Noveck | 3 Aug 22:04 2015
Picon

review of draft-ietf-nfsv4-xattrs-00

Since the document is unchanged from draft-naik-xattrs-02, the review is almost identical to my earlier one.  There are a few differences that reflect  the discussions we have had, the new situation, and something I just thought of.  Any changed text is in red.

Overall comments
     
I think this document takes a fundamentally correct approach to the issue.  Also, it is fairly far along.  

I think this was a good time to turn this into a working group document.  This is so even though it needs progress on other documents before it could be considered as a standards track document by the IESG.

I have a whole bunch of comments/corrections to be considered listed in the sections below.  Almost all of these are quite minor and the majority are editorial issues.  I've tried to make sure we have the option of making xattrs an extension to v4.2 if there is significant progress on draft-ietf-nfsv4-versioning, while maintaining a fallback to put this into a small v4.3 if that should be necessary.

The only cases in which I discuss potential substantive changes to the protocol are:
  • I think you need to have SETXATTR and REMOVEXATTR return the pre- and post- change times for  the update done by the xattr operation.
  • Proposed changes regarding handling of setxattr_type4.
  • Some sort of revision is needed regarding  ACE4_{GET,SET}_XATTRS
  • Possible additions to deal with ACCESS and xattrs.
There are a couple of cases where I indicate issues that might not need to be dealt with right away but might turn into issues when this goes to the IESG.
  • Internationalization considerations for xattr names :-( 
  • The issue of a potential IANA registry for extended attribute names,
Comments by Section

Abstract:

Suggest replacing the first sentence by:
This document proposes extensions to the NFSv4 protocol which allow file extended attributes (hereinafter also referred to as xattrs) to be manipulated using NFSv4.
My stab at addressing the singular/plural conundrum in the second sentence replaces the sentence by the following two sentences:

Support for xattrs is a file system feature that allows opaque metadata, not interpreted by the file system, to be associated with files and directories. Such support is present in many modern local file systems.

Suggest adding "are provided" to the end of the last sentence.

Introduction:

In the last sentence of the first paragraph suggest replacing "read from, and write to" by "interrogate, and modify".

In the second sentence of the second paragraph, suggest replacing "on the underlying file system"
by  "within the underlying file system".

Suggest replacing the third paragraph by the following:

Extended attributes have previously been considered unsuitable for portable use because some aspects of their handling are not precisely defined and they are not formally documented by any standard (such as POSIX). Nevertheless, it appears that xattrs are widely deployed and their support in modern disk-based file systems is nearly universal.

Suggest replacing the first sentence of the fourth paragraph by the following:
There is no clear specification of how xattrs could be mapped to any existing file attributes defined in RFC 5661 [2]. As a result, most NFSv4 client implementations ignore application-specified xattrs.
Suggest replacing in the second sentence of the fourth paragraph, "This" by "This state of affairs".

Suggest the following in place of the existing fifth paragraph:

There is thus a need to provide a means by which such data loss can be avoided.  This will involve exposing xattrs within the NFSv4 protocol, despite the lack of completely compatible file system implementations.

Suggest the following in place of the existing sixth paragraph.

This document discusses (in Section 5) the reasons that NFSv4 named attributes as currently standardized in [2], are unsuitable for representing xattrs. Instead, it proposes a separate protocol mechanism to support xattrs. As a consequence, xattrs and named attributes will both be optional features with servers free to support either, both, or neither.
Use cases:

In the last sentence of the second paragraph suggest the following changes:
  • Replace "jettison" by "dispense with".
  • End the sentence where the colon now appears and put the material after it into a new sentence.
  • Replace "these" by "this form of access is"
  • Replace "faster to access" by "provides faster access"
  • Replace "readily accessible by any application" by "is readily accessible by applications".

File system Support:

In the first sentence of the second paragraph, suggest adding a comma after "file systems".

Namespaces:

Suggest replacing the second paragraph by the following:

This document only provides for namespaces supported user-managed metadata only, thus avoiding the need to specify the semantics applicable particular system-interpreted xattrs.
The values of xattrs are considered application data just as the contents of named attributes, files, and symbolic links are. Servers have a responsibility to store whatever value the client specifies and to return it on demand.
xattr keys and values MUST not be interpreted by the NFS clients and servers, as such behavior would lead to non-interoperable implementations. If there is a need to specify attributes that servers need to be act upon, the appropriate semantics need to be specified by adding a new attribute for the purpose as provided by [RFC5661] and [draft-ietf-nfsv4-versioning].
I know that I promised to somehow "toughen up" the language. I was unable to do that. Given that the original already used "MUST", there was no real place to go while still staying within the scope of RFC2119. In any case, I hope the above moves things in the right direction
Differences with named Attributes:

Suggest, in the title, replacing "with" by "from".

In the first paragraph, the appropriate reference for the definition of named attributes should be to  RFC7530.

In the first sentence of the second paragraph, suggest replacing "define individual xattrs 'get' and 'set' operations" by "operations to get and set indivdual xatrrs".

Protocol Enhancements:
 
Suggest, in the title, replacing "enhancements" by "extensions".

In the second sentence, suggest deleting "to bitmap4 data type.

In the fourth sentence, suggest replacing "new operations, namely" by "the new operations".

Also in the fourth sentence suggest replacing "xattr key/value" by "xattr keys and values".

Suggest deleting the third sentence and adding a new second paragraph, along the lines of the following:

These changes follow applicable guidelines for valid NFSv4 protocol extension, whether the extensions occur in a minor version (as specified in [2]) or as an extension to an existing minor version (as specified in [draft-ietf-nfsv4-versioning]).
I'm thinking that the reference to the versioning doc would be informational for now but could eventually be made normative, most likely after that document is submitted to the IESG but before approval.

I note that you specify an attribute number and values for certain flags/value but do not specify opcodes for your new operations.  What you might want to do to address that issue is to remove all numerical assignments from where they are now and put them all into a new section 6.x, entitled something like "New values associated with protocol extensions".  Having those in one place would facilitate the process of approving this document as a working group document assuming we follow the procedures now described in draft-ietf-nfsv4-versioning-01.

New Attributes:

In the first sentence suggest adding "per-fs read-only" before "attribute".

Also in the first sentence suggest deleting "with GETATTR".

Attribute 82: xattr_support:

You might want to:
  • Change the title to "xattr_support attribute"
  • In the table replace "82" by "--".
Suggest adding a new paragraph after the current first one, as follows:

Since xattr_support is not a REQUIRED attribute, server need not support it.  However, a client may reasonably assume that a server (or fs) that does not support the xattr_support attribute does not provide xattr support and act on that basis.

New operations:
 
Suggest replacing the first two sentences by the following:

Individual xattrs generally represent separate items of metadata. For various reasons, combining them into single attribute results in clumsy implementations which contain significant functional deficits. In consequence, adding a new attribute to represent the set of xattrs for an object is not an appropriate way to provide support for xattrs.
Suggest replacing the last sentence of the first paragraph by the following:

Such a read-modify-write cycle is subject to updates being lost in the case of simultaneous updates by multiple clients.  In addition, two clients might simultaneously add the same xattr key to the same file with each concluding that it did the inital creation for the common xattr key, when the semantic model implies that only one could have done so.

You need a paragraph to deal with interactions between xattr and stateful operations. Please look at the following attempt and see if it needs any changes:
Xattr operations operate, for the most part, separate from operations related to locking state. Xattrs can be interrogated and modified without a corresponding open file. An open denying write or read does not prevent access to or modification of xattrs. Only in the case of delegations is there an interaction with the state-related logic. Xattrs cannot be modified when a delegation for the corresponding file is held by another client. On the other hand, xattrs can be interrogated despite the holding of a write delegation by another client.

There are a number of issues that arise repeatedly in the subsections of section 6 but are not specifically mentioned below: 
  • In many cases, the word "MUST" is used in contexts it would not be used in corresponding description of READ and WRITE, for example.
  • The indenting of the titles of sections named "ARGUMENTS" seems wrong and could be fixed.
New Definitions:

In the first sentence, suggest replacing "NFS xattr" by "NFSv4 xattr4."

The first sentence of the second paragraph consists of two independent clauses joined by a comma.  To address that and also to simplify capitalization issues, I suggest rewriting this as follows:
  
An xattr4 consists of an xattrname4 which is a string denoting the xattr key name and an attrvalue4 which is a variable-length string that identifies the value of the xattr. The xattr4name's handling of internationalization-related issues is the same as that for NFSv4 file names and named attribute names, as described in [RFC7530].
I don't see the point of the third sentence of the second paragraph. The concept of the combined size of an xattr does not seem to be used anywhere else. I'd suggest deleting this sentence.

In the fourth sentence of the second paragraph, I'd suggest replacing "array of xattr4" by "set of extended attributes".

There are a number of issues with the last sentence of the second paragraph that should be considered:
  • If you do have separate ACE bit for getting and setting xattrs, then ACCESS and OPEN will give you no useful information as to access.
  • ACCESS is the operation designed to determine whether access is permissible. If OPEN does allow you to infer permission that can be mentioned but it should not be on the same level as open.
There may be some other issues that relate to internationalization that need to be addressed. You are best off, if, as previously discussed, you remove any indication that the names are case insensitive. This will enable you to inherit the internationalization-related language from RFC7530, which applies to the names of files and named attributes. Some issues to consider:
  • Although there has been mention of defining xattrname as uft8str_cs, it might be better to define this as component4, since this will make it clearer that everything internationalization-related in RFC7530, applies to xatttr names as well.
  • You are probably going to have to choose whether xattr names are (always) case-sensitive or whether they follow the case-sensitivity of the filesystem of which they are a part. Right now, named attributes use the latter approach but I'm not clear whether this is right of whether this matters.

I'm suggesting you define a structure for pre- and post-operation attributes to be returned by SETXATTR and REMOVEXATTR. Text like the following makes sense at the end of the existing section:

The xachange4 structure is used to communicate the values of the change attribute immediately before and immediately after an operation which modifies an xattr. Having this information available allows a client to validate and retain cached xattr's while setting/removing the value of another xattr.

struct xachange4 {
changeid4 xac_pre;
changeid4 xac_post;
};

Caching:

In the first sentence of the second paragraph,, suggest replacing "an operation" by "some operations".

The way the third paragraph is written is confusing.  I suggest separating the statements about synchronously doing modifications from those about caching per se.  The result would be something like the following:

All modifications to xattrs should be done by requests to the server, since the protocol has no support for writeback caching of of xattrs.  The server should perform such updates synchronously.

Xattrs obtained from the server and xattrs sent to the server may be cached and clients can use them to avoid subsequent GETXATTR requests, provided that the client ensure that the cached value ha not been subsequently modified by another client. Such assurance can depend on the client holding a delegation for the file in question or the client can interrogate the change time, to make sure that any cached value is still valid. Such caching may be read-only or writethrough.

I think that speaking of "incoherency" give the wrong impression.  I suggest rewriting the last paragraph as follows:

The result of local caching is that the individual xattrs maintained on clients may not be up-to-date. Changes made in one order on the server may be seen in a different order on one client and in a third order on another client. In order to limit problems that might arise because by separate operations to obtain individual xattrs and other file attributes, a client should treat xattrs just like other file attributes with respect to caching as detailed in section 10.6 of RFC 5661 [2]. A client may validate its cached version of an xattr for a file by fetching the change attribute and assuming that if the change attribute has the same value as it did when the attribute was cached, then the xattr has not changed. If the client holds a delegation that ensures that the change attribute cannot be modified, that it can dispense with actual interrogation of the change attribute.

GETXATTR - Get an extended attribute of a file


As currently written the first and third sentence of the second paragraph of DESCRIPTION are contradictory in that the first is not conditional on being able to do the operation while third makes due allowance for failure conditions. Suggest replacing, in the first sentence "MUST obtain" by "wiil fetch".


In the last sentence of that paragraph, suggest replacing "

will exceed the channel's negotiated maximum response size" by "is such as to cause the channel's negotiated maximum response size to be exceeded".


The IMPLEMENTATION section is confusing in that it jams client and server considerations together. It would be better to start with the client and proceed like the following:


Clients which have a cached xattr value may avoid the need to do a GETXATTR by determining if the change attribute is the same as it was when the xattr was fetched. If the client does not hold a delegation for the file in question it can do this by do a GETATTR fetching the change attribute and comparing the value to a change attribute value fetched when the xattr value was obtained. This handling is similar to how a client would revalidate other file attributes such as ACLs.
When responding to such a GETATTR, the server will, if there is an OPEN_DELEGATE_WRITE delegation held by another client for the file in question, either obtain the actual current value of these attributes from the client holding the delegation by using the CB_GETATTR callback, or revoke the delegation. See Section 18.7.4 of RFC 5661 for details [2].


SETXATTR - Set an extended attribute of a file


As this is written now, handling the conditional create case (I.e. create the attribute if it does not currently exist) is unduly difficult. One is forced to retry after an error and there is no guarantee that the situation has not changed in the meantime. Some ways of addressing this are:

  • Adding a new value (e.g. SETXATTR4_CONDCR) to setxattr4_type for the conditional create case.
  • Getting rid of setaxattr4_type, making the server always doing a conditional create, and adding a boolean to the response indicating whether the xattr existed before the setxattr was done.
I'm also proposing that you add an xachange4 structure to the NFS4_OK case of SETXATTR4res. doing a GETATTR before the SETXATTR provides a useless value while doing a gETATTR after the SETXATTR leaves a race open in which you can wind up consideriing an out-of-date xattr value as valid.

Which regard to the second paragraph of DESCRIPTION:
  • In the first sentence, suggest replacing "If the xattr key and/or value contained in the client request exceeds" by "If the xattr and value contained in the client request are such that the request would exceed".
  • In the second sentence, suggest you pick another error since NFS4ERR_ENOSPC would most likely indicate you are out of disk space.
The two sentences of the third paragraph of DESCRIPTION disagree regarding the case in which you do a SETXATTR that sets the xattr value to what it currently is. Suggest the following as a replacement:
A successful SETXATTR MUST change the file time_modify and change attributes if the xattr is created or the value assigned to xattr is actually changes. However, these attributes SHOULD NOT be changed in case in which SETXATTR Cause no actual change in the xattr value.
LISTXATTR - List extended attributes of a file:

The struct LISTXATTR4args is missing the definition of la_cookieverf.

REMOVEXATTR - Remove an extended attribute of a file:


As with SETXATTR, I believe you should return an xachange4 structure in the response.


With regard to the second paragraph of DESCRIPTION:

  • Suggest replacing "SHOULD" by "MUST".
  • Suggest deleting the second sentence.
Valid Errors:

This does not address the issues with NFS4ERR_ENOSPC mentioned elsewhere.

With regard to GETXATTR:
  • Don't see why NFS4ERR_GRACE is included.
  • Believe NFS4ERR_{IS,NOT}DIR should be deleted.
With regard to SETXATTR:
  • Believe NFS4ERR_ADMIN_REVOKED should be deleted.
  • Believe NFS4ERR_ATTRNOTSUPP should be deleted.
  • Believe NFS4ERR_BADOWNER should be deleted.
  • Believe NFS4ERR_BAD_RANGE should be deleted.
  • Believe NFS4ERR_BAD_STATEID  should be deleted.
  • Believe NFS4ERR_DELEG_REVOKED should be deleted.
  • Believe NFS4ERR_EXPIRED should be deleted.
  • Believe NFS4ERR_FBIG should be deleted.
  • Don't see why NFS4ERR_GRACE is included.
  • Believe NFS4ERR_LOCKED should be deleted.
  • Believe NFS4ERR_NOTDIR should be deleted.
  • Believe NFS4ERR_OLD_STATEID should be deleted.
  • Believe NFS4ERR_OPENMODE should be deleted.
  • Believe NFS4ERR_UNKNOWN_LAYOUTTYPE should be deleted.
With regard to LISTXATTR:
  • Believe NFS4ERR_ADMIN_REVOKED should be deleted.
  • NFS4ERR_FHEXPIRED is not a valid error.
  • Don't see why NFS4ERR_GRACE is included.
  • Believe NFS4ERR_LOCKED should be deleted.
  • Believe NFS4ERR_{IS,NOT}DIR should be deleted.
With regard to REMOVEXATTR:
  • Believe NFS4ERR_ADMIN_REVOKED should be deleted.
  • Believe NFS4ERR_ATTRNOTSUPP should be deleted.
  • Believe NFS4ERR_BADOWNER should be deleted.
  • Believe NFS4ERR_BAD_RANGE should be deleted.
  • Believe NFS4ERR_BAD_STATEID  should be deleted.
  • Believe NFS4ERR_DELEG_REVOKED should be deleted.
  • NFS4ERR_FHEXPIRED is not a valid error.
  • Believe NFS4ERR_EXPIRED should be deleted.
  • Don't see why NFS4ERR_GRACE is included.
  • Believe NFS4ERR_NOTDIR should be deleted.
  • Believe NFS4ERR_UNKNOWN_LAYOUTTYPE should be deleted

Extensions to

ACE Access Mask Attributes:
You might want to:
  • Delete the definition of the two new values and only indicate that new values are defined.
  • Indicate that the actual value will appears in the new section 6.x.
I don't understand the statement "No additional granularity of control is implied by these constants for server implementations".  Specifically,
  • The previous sentence, in indicating how these are used, does imply a finer-grained access control arrangement.
  • If there were not finer-grained access control, there would be no point in defining these additional bits
If you decide there is to be finer-grained access control, you might want to consider the implications for ACCESS. Some possibilities;
  • Adding bits ACCESS4_XAREAD and ACCESS4_XAWRITE
  • Creating a new ACESS_XATTR operation. This would allow you distinguish permissions for adding, modifying, and deleting xattrs to a file.
  • Creating a new ACCESS_PLUS operation that returns a pair of acemask4's. If this were done, it would be best defined as a feature separate from xattrs since someone not supporting xattr's might still find it useful.
New Section 6.x:

The section title should be something like "Numeric values assigned".

The initial paragraph should be something like the following:

This section lists the numeric values assigned to implement the xattr feature. To avoid inconsistent assignments, they have been checked against the most recent minor version and all extensions currently approved as working group documents. Given that, development of interoperable prototypes should be possible using these value, although the possibility exists that these assignments may be modified before eventual publication as a standard-track document.

After that, I would list the necessary assignments in XDR-style format with comments indicating where each XDR section would by placed in a consolidated XDR for the protocol including the extension.

pNFS Considerations:

Would revise to be more consistent with the fact that xattrs are more likely to be on the metadata server itself and not data servers aka data storage devices. Something like:

All xattr operations are sent to the metadata server, which is responsible for fetching data from and effecting necessary changes to persistent storage.
Security Considerations:

If you are going to reference any previous NFSv4 protocol version, it should be NFSv4.2.  

I think it may make sense to say here that the issues are exactly the same as those relating to the storing of file data and named attributes.  These are all various sorts of application data and the fact that the means of reference is slightly different in each case should not be considered security-relevant.

IANA Considerations:

A better formulation would be:

This document does not require any actions by IANA.

The IESG may raise the issue of the existing IANA maintained registry of named attributes. If you don't think one is required, you may may to explain why this situation is different. In any case, you should refer to the description in RFC7530 as the one in RFC5661 is not correct (may need an errata for that sometime).

References:

I'm pretty sure you need a reference to RFC7530 and I tend to think it should be normative.

I think you need to have a reference to the current NFSv4.2 draft. If this becomes a working group document before the v4.2 RFC is out, you will need to add a note asking the RFC Editor to make sure that the eventual RFC number is inserted.

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
David Noveck | 31 Jul 20:20 2015
Picon

Re: follow-up of xattr discussion

>>    Such caching is write through in that modification to xattrs is
>>    always done by means of requests to the server and should not be
>>    only done locally. Due to the relative infrequency of xattr
>>    updates, it is suggested that all changes be propagated
>>    synchronously. The client MUST NOT maintain a cache of modified
>>    xattrs.
>> 
>> What's the rationale for that MUST NOT?

> Same as the one for all other attributes (except size, time_modify, change): see https://tools.ietf.org/html/rfc5661#section-10.6

The text in question reads as follows:

Other than those three attributes, the client MUST NOT maintain a cache of modified attributes. Instead, attribute changes are immediately sent to the server.
When those two sentences are read together, it seems to me that the intention is/was to forbid writeback caching of attributes. With just the first of them present, it might lead someone to suppose that a writethrough cache of modified attributes is to be forbidden as well. I don't think that is the intention.


On Fri, Jul 31, 2015 at 11:22 AM, Manoj Naik <mnaik <at> us.ibm.com> wrote:

"nfsv4" <nfsv4-bounces <at> ietf.org> wrote on 07/23/2015 11:45:47 AM:

> From: "J. Bruce Fields" <bfields <at> fieldses.org>
> To: David Noveck <davenoveck <at> gmail.com>
> Cc: Christoph Hellwig <hch <at> lst.de>, "nfsv4 <at> ietf.org" <nfsv4 <at> ietf.org>
> Date: 07/23/2015 11:46 AM
> Subject: Re: [nfsv4] follow-up of xattr discussion
> Sent by: "nfsv4" <nfsv4-bounces <at> ietf.org>
>
> On Thu, Jul 23, 2015 at 01:15:22PM -0400, David Noveck wrote:
> > which sound to me like such uses for not allowed.  I'll be reviewing
> > this document in a bit and will propose some toughening up of the
> > language.
>
> One of the practical difference is the caching.  A lot of the
> filesystem-specific attributes look incompatible with client caching.
>
> Looking at the "caching" section: it does say clients may cache reads,
> and that the change attribute can be depended on to validate that cache,
> but:
>
>    https://tools.ietf.org/html/draft-naik-nfsv4-xattrs-02#section-6.2.2
>
>    Such caching is write through in that modification to xattrs is
>    always done by means of requests to the server and should not be
>    only done locally. Due to the relative infrequency of xattr
>    updates, it is suggested that all changes be propagated
>    synchronously. The client MUST NOT maintain a cache of modified
>    xattrs.
>
> What's the rationale for that MUST NOT?

Same as the one for all other attributes (except size, time_modify, change): see https://tools.ietf.org/html/rfc5661#section-10.6

Manoj.

>
> --b.
>
> _______________________________________________
> nfsv4 mailing list
> nfsv4 <at> ietf.org
>
https://www.ietf.org/mailman/listinfo/nfsv4
>


_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4
J. Bruce Fields | 30 Jul 17:53 2015

4.2 label xdr bug

This is confusing--the label attribute isn't an array.  And why does the
.x have two definitions of sec_label4?

Maybe something like the below is better?

(Thanks to Kinlong Mee for spotting this.)

--b.

diff --git a/dotx.d/head.x b/dotx.d/head.x
index 85fb741..3db96ff 100644
--- a/dotx.d/head.x
+++ b/dotx.d/head.x
 <at>  <at>  -501,7 +501,6  <at>  <at>  typedef change_policy4	fattr4_change_policy;
 typedef uint64_t	fattr4_space_freed;
 typedef change_attr_type4
 		fattr4_change_attr_type;
-typedef sec_label4	fattr4_sec_label<>;
 typedef uint32_t	fattr4_clone_blksize;

 %/*
diff --git a/nfsv42_middle_fileattributes.xml b/nfsv42_middle_fileattributes.xml
index c2da218..810e5cd 100644
--- a/nfsv42_middle_fileattributes.xml
+++ b/nfsv42_middle_fileattributes.xml
 <at>  <at>  -196,8 +196,8  <at>  <at>  typedef uint32_t  policy4;
       </t>

       <t>
-        The FATTR4_SEC_LABEL contains an array of two components with the first
-        component being an LFS.  It serves to provide the receiving end
+        The first field of the FATTR4_SEC_LABEL is an LFS, which
+        serves to provide the receiving end
         with the information necessary to translate the security attribute
         into a form that is usable by the endpoint.  Label Formats assigned
         an LFS may optionally choose to include a Policy Identifier
 <at>  <at>  -206,7 +206,7  <at>  <at>  typedef uint32_t  policy4;
         <xref target='Quigley14' />.
         The translation used to interpret the security
         attribute is not specified as part of the protocol as it may depend
-        on various factors.  The second component is an opaque section which
+        on various factors.  The second field is an opaque section which
         contains the data of the attribute.  This component is dependent on
         the MAC model to interpret and enforce.
       </t>

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4


Gmane