Karen | 20 May 18:11 2016

draft-ietf-nfsv4-rpcrdma-bidirection-03 review

Hi Chuck,

Here is some feedback on draft-ietf-nfsv4-rpcrdma-bidirection-03


Abstract section:

The use of the word "recent" seems relative to time.  5 years later, recent will no longer be recent.  Not sure how much of a big deal this is, just seems maybe instead of putting "recent minor versions of NFSv4", a more precise explanation would be "With the introduction of NFSv4.0".


"This traditional form of ONC RPC message passing is referred to as operation in the "forward direction"."

"as operation in the forward direction" seems awkward.  "is referred to as the "forward direction"." seems a bit more concise and clearer.


Same as in section 2.2 in "This form of message passin is referred to as operation in the "backward direction."

Also again the use of the word recently, as in "Not until recently", seems that "Not until NFSV4.0" would be clearer.


"An NFSv4 "delegation" is simply a promise made by a server that it will notify a client before another agent is allowed access to a file"

another agent? seems "a different client" would be clearer.


"Without a backward direction RPC-over-RDMA capability, an NFSv4.1 client must additionally connect using a transport with backward direction capability to use as a backchannel."

I understand the point, and agree, but the sentence seems mumbled.  Maybe its confusing to me because of the use of the word backward direction.  Maybe backward direction needs to be defined as a connection that is initiated by a requestor, but also used to accept requests from the respondor, in which the requestor then reponds?  Not sure if that wording is any clearer, but the idea that the requests are sent from the side that did not create the connection?


"For an RDMA Send operation to work, the receiving peer must have posted an RDMA Receive Work Request (WR) to provide a receive buffer in which to land the incoming message.

why is the posted receive buffer being called a posted receive work request?  Seems this would be clearer

"For an RDMA Send operation to work, the receiving peer must have posted an RDMA receive buffer as the location in which the RDMA send request will land."

And similarly

"If a receiver hasn't posted enough Receive WRs to land incoming Send operations..."

If a receiver hasn't posted enough receive buffers to accept incoming send requests"

(Why is Receive capitalized?)


"When message direction is not fully determined by context"

"fully determined by context" thats confusing, what does that really mean?  I think this means when the rdma header does not directly indicate if the message is a call or reply, but the wording is confusing.


"If a sender transmits a message for a receiver which has no prepared receive buffer"

the word "posted" not "prepared" was used earlier, to be consisted "posted" would read more clearly than "prepared"


"A forward direction RPC-over-RDMA service endpoint"

did you mean "server" instead of "service"?

"To receive incoming backward direction replies, an RPC-over-RDMA server endpoint must pre-post a receive buffer for each backward direction Call it sends"

What prevents a buggy, or hacking client from sending more requests that forward credits and preventing a reply in the backward direction from arriving?  Could a buggy or hacky client cause a denial of service by sending more than credit number of requests?  Guess that could happen in the standard single direction and would result in the connection closing.  Maybe not an issue?


last sentence, "and the header's msg_type field MUST conatin the value CALL",

for more clarification "and the RPC header's msg_type field", this is a nit...


"If an ONC RPC client has no work to do, it may be some time before it re-establishes a transport connection. Backward direction Requesters must be prepared to wait indefinitely for a connection to be established before a pending backward direction ONC RPC Call can be retransmitted."

I don't believe there are currently an non-idempotent backchannel operations, but if there ever were to be, seems that the client should detect if it has any drc replies and re-create the bi-directional, or even one way backchannel connection, right after it detects the connection is  gone.

Not addressed, and obvious to the implementer, but should it be mentioned that send buffers in the backward direction MUST be the same size as the posted receive buffers on the receiving side?  That is the backward connection must use the same send/recv buffer sizes and this can not be negotiated?

nfsv4 mailing list
nfsv4 <at> ietf.org
Mike Kupfer | 14 May 00:33 2016

high-level editorial comments, draft-ietf-nfsv4-versioning-03

Apologies for taking so long to provide feedback.  I plan to be more
ruthless about reserving time for working group activity; hopefully that
will keep me from disappearing for weeks at a time.

Anyway, I've got a bunch of issues in my notes, but I thought I'd start
with a discussion on how to make the document easier to read and digest.

One thing that gets in the way of readability (at least for me) is the
interleaving of specification with discussion (e.g., Sections
4.1.1-4.1.2, or Sections 5.1.2-5.1.3).  If we could more clearly
separate the specification from the discussion, I think that would
improve readability.  One way to do that might be to have all the
history and discussion first, and then follow that with the actual
rules, analogous to the way the specs for NFSv4.0, v4.1, and v4.2 are

Another thing that I had trouble with was the generality and complexity
of the specification.  Here are three examples.

One, Section 5.1.3 has the title "Rules for non-XDR changes", but in
that section we are told that a substantive behavior change *must have*
an associated XDR extension (which seems to conflict with the section
title), and no feature including a behavior change can be introduced as
an extension.  I'm afraid I'm lost there.

Two, Section 6.4 talks about an F-to-E status "with regard to the
feature".  I can understand having a feature introduce multiple protocol
elements.  But this wording seems to imply a many-to-many mapping
between features and elements, even though "every protocol element is
defined as a member of exactly one protocol feature" (bottom of page

Three, the E-to-F statuses SINF and NSINF add complexity, but without
adding any real value (that I can see).  IMO, the spec would be better
off without ever bringing them up.

If it seems useful to address these two issues first, I can go back
through my notes to pick out all the ones that are pertinent.
Alternatively, I could go back through my notes to look for the
technical questions that I had.

Dave, do you have any preference on how to proceed?


nfsv4 mailing list
nfsv4 <at> ietf.org

internet-drafts | 12 May 22:14 2016

I-D Action: draft-ietf-nfsv4-rfc5666bis-06.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Network File System Version 4 of the IETF.

        Title           : Remote Direct Memory Access Transport for Remote Procedure Call, Version One
        Authors         : Charles Lever
                          William Allen Simpson
                          Tom Talpey
	Filename        : draft-ietf-nfsv4-rfc5666bis-06.txt
	Pages           : 52
	Date            : 2016-05-12

   This document specifies a protocol for conveying Remote Procedure
   Call (RPC) messages on physical transports capable of Remote Direct
   Memory Access (RDMA).  It requires no revision to application RPC
   protocols or the RPC protocol itself.  This document obsoletes RFC

The IETF datatracker status page for this draft is:

There's also a htmlized version available at:

A diff from the previous version is available at:

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:

nfsv4 mailing list
nfsv4 <at> ietf.org

Alissa Cooper | 10 May 15:22 2016

Alissa Cooper's No Objection on draft-ietf-nfsv4-rfc3530-migration-update-08: (with COMMENT)

Alissa Cooper has entered the following ballot position for
draft-ietf-nfsv4-rfc3530-migration-update-08: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:


Thanks for addressing my DISCUSS. 


My comments are about the example given in Section 4.9. 

I understand from 4.2 that the requirements for the construction of the
client id are:

1) It should be unique per client.
2) It should persist through reboots.

As best I can tell, embedding the client's IP address in the client id is
not a requirement (but I admit to not fully understanding NFS!). So why
is it suggested that the client's network address be embedded in the
client id?

There are also a few privacy-related issues with the example:

- Some clients will change their IP addresses over time to avoid being
tracked. By suggesting that some prior address be used in the client id,
there is an implied requirement on the client to maintain a history of
previously used addresses, which could be exploited for tracking

- Permanent device identifiers (such as a serial numbers) should not be
embedded in a client id on the wire, again to avoid facilitating tracking
by any other party that knows the serial number.

It seems to me that to avoid all of the issues above, perhaps a better
example to provide would be hash(some client secret, some source of
uniqueness, server identity). That gives the client id a good chance of
being unique without exposing other identifiers related to the client.
And then if the existing guidance about saving and re-using the value is
followed, it won't be necessary to try to shoehorn in an old IP address
or persistent device identifiers when the clientid needs to be

nfsv4 mailing list
nfsv4 <at> ietf.org

David Noveck | 10 May 14:57 2016

Proposed changes to draft-cel-nfsv4-rpcrdma-version-two.

On April 20, when I sent the message credit-related issues in draft-ietf-nfsv4-{rfc5666bis,rfc5666-implementation-experience,rpcrdma-bidirection}, I indicated that there would be corresponding Version Two changes.  Even though some of these changes are, in some sense, motivated by considerations related to bidirectional operation, I think we can discuss these without waiting for the WGLC of draft-ietf-nfsv4-rpcrdma-bidirection to complete.  

This mail includes those credit-related changes and proposals for a bunch of others that I think are needed, based on my work on draft-dnoveck-rpcrdma-xcharext and draft-dnoveck-rpcrdma-rtrissues.

In any case, there isn't a big rush and there would not be a problem waiting until late May, if you are busy with wrapping up Version One.  My main concern is that we get an up-to-date Version Two out in time for the draft cutoff for Berlin, on July 8.  Since I am hoping to be making corresponding changes in draft-dnoveck-nfsv4-rpcrdma-xcharext and the forthcoming draft-dnoveck-nfsv4-rpcrdma-rtrext, I'd hope we can get agreement on the substance of the current round of Version Two changes by the end of June.

So far it looks like all of the necessary changes would go into 3.2. Documentation Requirements.

Credit field ambiguity

To address the fact that, in the context of extensions, it needs to be clear whether the credit field is a request or a grant, I propose that we add a new bullet reading as follows:

For each new opttype code, it must be made clear how it is to be determined whether the existing credit field in the header is to be interpreted as a request or as a grant.   This can include specifying that, for particular codes, the field is to be interpreted as one or the other, that particular fields in the header definition are to direct the interpretation, or that fields in the associated RPC Payload Stream are to direct the interpretation.  In some cases, there may be no valid interpretation and the credit field is to be ignored.  In other cases, the credit field is to be interpreted either as a request or a grant and the complementary information is defined as available elsewhere.

After the header

In the context of many desirable features (e.g. message continuation, send-based DDP), the existing fourth bullet is too restrictive.  I propose the following replacement:
For each new opttype code, it must be made clear whether an RPC Payload Stream or a part thereof follows the Transport Header Stream. It must be made clear whether any material in addition to the transport header is allowed and how such material is to be processed. In cases in which a partial RPC Payload stream is allowed, it needs to be clearly specified how such partial Payload Streams are to be assembled into a complete Payload Stream. Also, when the Payload Stream portion begins at any point other than the byte immediately following the end of the Transport Header, it needs to be stated how the start of the Payload Stream portion is to be found, using the information in the optional header.
This bullet could be broken up using <vspace> if you like.  

That's my preference but I realize that some people aren't comfortable with that.

XDR Model

The current last paragraph of the section seems to be more appropriate to the extension model originally proposed for Version One, than it is to the extension model that is actually described in the current Version Two draft. In the current model, we have a situation like that for pNFS, in which a nominally opaque protocol field is overlaid by an XDR description of that opaque field's contents. As a result, the XDR is not extended as it is in NFSv4.x, by making previously invalid messages valid and providing an XDR description of the newly valid messages. Instead the interpretation of a message which was previously valid according to the existing XDR is refined by the addition of the new overlay.

I propose some rewrites below, in order to bring the treatment in line with the current extension model in the document.

I propose replacing the current first paragraph by the following:
RPC-over-RDMA Version Two may be extended by defining a new rdma_opttype value and providing an XDR description of the corresponding rdma_optinfo content. When this is done, the XDR description provided overlays, for that rdma_opttype value, the nominally opaque field rdma_optinfo. As a result a new header type is effectively created.
I propose replacing the current last paragraph by the following:
Implementers will generally combine the XDR descriptions of the new features they intend to use with the XDR description of the base protocol, extracted from this document. This may be necessary to create a valid XDR input file because:
  • Extension definitions are free to use types (e.g. chunk definitions) defined in the base protocol.
  • Later extensions may use types defined by earlier extensions on which they depend.
Because RPC-over-RDMA is not an RPC program and because we will often have two definitions of the same field (i.e. opaque and non-opaque versions of rdma_optinfo), the way in which XDR tools are normally used will not be available to those using the XDR definition. For example, the base XDR definition may be used to parse the base transport definition while the extension definition may be used provide a more complete interpretation of what would otherwise be treated as opaque data. It may be that either, both, or neither of the levels would use XDR-generated code.
In any case, the combination of the XDR description for the RPC-over-RDMA Version Two protocol combined with that for any selected extensions will provide an adequate human-readable description of the extended protocol.

Feature Structure

As explained in draft-dnoveck-nfsv4-rpcrdma-rtrissues-00, there are some important synergies between message continuation and send-based DDP, making it desirable to describe both together while allowing implementation to independently choose whether to implement each particular feature. I'm not sure if this was the intention, but it seems like the current document is written assuming a one feature-per-extension model. It seems that one feature-per-document is too restrictive in general, 

I'm not sure whether this adds clarity or not but it appears to me that the way the word "feature" is used in this part of the document matches the concept of "feature set" in draft-ietf-nfsv4-versioning., which is the unit of XDR extension there. I don't propose, however, to change the terminology. Still, I think that this use of "feature" conflicts with the colloquial use of that word and things need to be made clearer. BTW, although there are thirteen uses of the word "feature" in the document, I am going to leave the terminology alone for the most part and use "facility" for the more colloquial use of "feature" in this context. YMMV.

In any case, I propose replacing the existing second paragraph with the following:
A set of such new protocol elements may be introduced by a Standards Track document and so be together considered an OPTIONAL feature in that each implementation can be presumed to be either aware of all the protocol elements introduced by that feature or be aware of none of them.
Despite this requirement, particular features may introduce a set of facilities, often colloquially referred to as "features," without requiring that each implementation provide support for either all or none of these. In such cases, it needs to be made clear how the peer implementations determine a common set of facilities supported and interoperate properly when different sets of facilities are supported by the communicating implementations.
Nfsv4 Working Group and IESG review, together with appropriate testing of prototype implementations, should ensure continued interoperation with existing implementations. Similar review and testing should ensure interoperation of those implementing the new feature. In cases in which it is possible that not all implementations implement the same set of facilities, those same practices should ensure the correct interoperation of implementations supporting different subsets of available facilities.
I also propose replacing the existing last bullet by the following two bullets:
  • description of interactions with existing extensions (e.g., requirements that another OPTIONAL feature needs to be present for newly added features to work).
  • description of interactions with facilities introduced by other extensions (e.g. requirements that a particular level of support be provided for some particular facility in the new extension to work).
nfsv4 mailing list
nfsv4 <at> ietf.org
Spencer Shepler | 9 May 21:46 2016

hotel registrations are now available for IETF 96 <eom>


nfsv4 mailing list
nfsv4 <at> ietf.org
Spencer Shepler | 9 May 07:24 2016

Berlin 2016 working group meeting


As you may have noted, I have submitted a working group meeting request for Berlin (July 17-22)


I would still prefer that we have a draft agenda started by May 20th.


I appreciate the discussion to date and if there is other I-Ds that need to be discussed, please send those to the WG alias.





nfsv4 mailing list
nfsv4 <at> ietf.org

nfsv4 - New Meeting Session Request for IETF 96

A new meeting session request has just been submitted by Spencer Shepler, a Chair of the nfsv4 working group.

Working Group Name: Network File System Version 4
Area Name: Transport Area
Session Requester: Spencer Shepler

Number of Sessions: 1
Length of Session(s):  2.5 Hours
Number of Attendees: 20
Conflicts to Avoid: 

Special Requests:
  Please consider scheduling the meeting between July 20-22.

We have a key working group member that will not be available at the IETF until July 19th 

nfsv4 mailing list
nfsv4 <at> ietf.org

Adamson, Andy | 6 May 16:29 2016


Hi Spencer

FYI: The multi-domain draft expired. I fixed one typo and posted version-07. As I’m sure you’re aware,
the WG state is WG Consensus: Waiting for Write-Up with you as the document shepherd.  Perhaps since v4.2
and gssv3 are well on their way, it’s time to finish this draft.

AFAIK there are no current issues….


nfsv4 mailing list
nfsv4 <at> ietf.org
internet-drafts | 6 May 16:24 2016

I-D Action: draft-ietf-nfsv4-multi-domain-fs-reqs-07.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Network File System Version 4 of the IETF.

        Title           : Multiple NFSv4 Domain Namespace Deployment Guidelines
        Authors         : William A. (Andy) Adamson
                          Nicolas Williams
	Filename        : draft-ietf-nfsv4-multi-domain-fs-reqs-07.txt
	Pages           : 15
	Date            : 2016-05-06

   This document discusses issues relevant to the deployment of the
   NFSv4 protocols in situations allowing for the construction of an
   NFSv4 file namespace supporting the use of multiple NFSv4 domains and
   utilizing multi-domain capable file systems.  Also described are
   constraints on name resolution and security services appropriate to
   the administration of such a system.  Such a namespace is a suitable
   way to enable a Federated File System supporting the use of multiple
   NFSv4 domains.

The IETF datatracker status page for this draft is:

There's also a htmlized version available at:

A diff from the previous version is available at:

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:

nfsv4 mailing list
nfsv4 <at> ietf.org

David Noveck | 4 May 19:10 2016

review of draft-ietf-nfsv4-rpcrdma-bidirection-03

General Comments

Overall Evaluation

This document is in good shape and the changes necessary to make it ready to go forward are small. 

A lot of my comments are with regard to inevitable small editorial issues.  A lot of these are matters of preference rather than of correctness.

Another category of comments has to do with the rough edges that have resulted from this document's history.  It has successively been a set of conventions, an optional extension, a part of version One and finally a cross-version guide to bidirectional operation.  I'll leave it to Chuck to determine the level of sanding/filing/buffing that he feels comfortable with.

Important Issues

The only issue that I think needs to be addressed to go forward is the treatment of ULBs for bidirectional protocols, such as NFSv4.  Chuck and I have already discussed this so I think the necessary changes can be made quickly.

Look at 7. Backward Direction Upper Layer Binding for details. It appears that these changes address the issue of contradictions between this document and draft-ietf-nfsv4-rfc5666bis-05, discussed earlier on the list.

Comments by Section

1.  Introduction

In the second sentence of the first paragraph, suugest replacing "in particular pNFS" by "in particular, of pNFS.

In the first sentence of the second paragraph, the phrase "protocol described in this document" is a problem.  This document doesn't describe a protocol.  It uses, quite properly, the protocol described in rfc5666bis (and implicitly future versions).  Suggest replacing "protocol" by "approach".

In the last sentence of the second paragraph, the word "Therefore" is flat-out wrong.  Documents which do not change the XDR can and do update documents that contain XDR without changing the XDR.  Some examples:
  • RFC7530 obsoletes RFC3530 without changing the v4.0 XDR.
  • draft-ietf-nfsv4-rfc3530-migration-update will, when it is finally approved, update RFC7530, without changing the v4.0 XDR.
  • draft-ietf-nfsv4rfc5666bis will, when it is approved, obsolete RFC5666.  It may make some cosmetic XDR changes to the XDR, but that has nothing to do with whether it obsoletes RFC5666.
Some possible ways to address this issue:
  • Just delete the "Therefore".
  • Replace it by "This document does not modify the XDR for RPC-over-RDMA Version One [rfc5666bis] or for any future RPC-over-RDMA version."  
  • Replace it by "This document does not modify the protocol described by [rfc5666bis] or any future RPC-over-RDMA version."
While the statement in the last paragraph is true, it seems incomplete.  Given that we know whose responsibility this is, we should say so, rather than making the reader guess. I suggest replacing the paragraph by the following:
This document does not provide an Upper Layer Binding for NFSv4.x callback operations or any other form of backward direction RPC. Issues related to ULBs for backward direction operation are dealt with in Section 7.
2. Understanding RPC DirectionIn the first sentence of the first paragraph, I don't understand what the word "fundamentally" adds in this context. I suggest deleting it. In the first sentence of the fourth paragraph, suggest replacing "generates" by "results in".In the last sentence of the fifth paragraph, suggest replacing "wait passively for" by "passively await".A general problem is that the basic idea stated in the fifth paragraph, that the client is responsible for establishing transport connections, appears in sections 2, 2.1, 2.2, and 2.3.  As part of cleaning that up, I am suggesting the following replacement for the fifth paragraph:
RPC-over-RDMA is a connection-oriented RPC transport. When a connection-oriented transport is used, ONC RPC client endpoints are responsible for initiating transport connections, while ONC RPC service endpoints passively await incoming connection requests. This same division of responsibility applies in cases in which the client is the Requester (Section 2.1), the Responder (Section 2.2), or both (Section 2.3).
In the last paragraph suggest replacing "considered" by "addressed".
2.1. Forward DirectionIt's not clear what "traditional" means here. I'm going to suggest the following revision of the first paragraph
Normally, an ONC RPC client acts as a Requester, while an ONC RPC service acts as a Responder. This form of ONC RPC message passing is referred to as operation in the "forward direction."
In line with the suggestion in the section above, I suggest deleting the second paragraph.
2.2. Backward DirectionIn line with the suggestion in 2. Understanding RPC Direction above, I suggest deleting the second paragraph.2.3. Bi-directional OperationIn line with the suggestion in 2. Understanding RPC Direction above, I suggest deleting the second sentence of the second paragraph.3. Rationale For Bi-Directional RPC-over-RDMAIn the section title, I suggest replacing "Rationale For" by "Uses of".3.1. NFSv4.0 Callback OperationIn the last paragraph, suggest the following changes:
  • In the first sentence, suggest replacing "are fully functional" by "can function".
  • In the last sentence, suggest replacing "functional correctness" by "correctness".
3.2. NFSv4.1 Callback OperationIn the first sentence of the penultimate paragraph,suggest replacing "Some implementations" by "Implementations often".Suggest replacing the last sentence of the last paragraph by the following:
The pNFS feature raises additional issues. Because NFSv4.1 relies on callbacks to help manage pNFS layout state, pNFS operation is not possible without a backchannel.
4. Flow ControlSuggest the following changes in the first sentence of the first paragraph:
  • replacing "work" by "work properly".
  • replacing "in which to land" by "into which to transfer"
In the second sentence of the first paragraph, suggest replacing "land" by "accommodate"Suggest replacing the last sentence of the second paragraph by the following:
In the case of Version One, this is discussed in Section 4.3 of [I-D.ietf-nfsv4-rfc5666bis].
5. Protocol For Backward OperationSuggest replacing "Protocol For" by "Effecting".Suggest the following rewrite of the only paragraph in this section:
Performing backward direction ONC RPC operations over an RPC-over-RDMA transport connection can be accomplished by using the RPC-over-RDMA protocol in the way described in the following subsections. It may be helpful to refer to the XDR description of the appropriate version of RPC-over-RDMA. In the case of Version One, this is contained in Section 5.1 of [I-D.ietf-nfsv4-rfc5666bis].
6. In the Absence of Backward Direction SupportSuggest moving the current last paragraph so it follows the current third paragraph.Suggest prepending "For example, " to the first sentence of the current fourth paragraph.7. Backward Direction Upper Layer Binding The basic problem in this section is that it concerns itself with ULBs for the backward direction case while the more important case of bidirectional operation is not explicitly addressed (except in 1. Introduction) where it is declared out-of-scope). I'm going to propose changes assuming that the necessary material will be added to this section, although a separate section is possible.Suggest changing the section title to "Upper Layer Binding for Backward Direction and Bi-directional Operation"Suggest adding the following new sentence to the end of the first paragraph:
This applies to both backward direction and bi-directional operation.
Given Chuck's choice regarding how to deal with this issue, I suggest rewriting the third paragraph as follows:
By default, no data items in a ULP are DDP-eligible. If there are no DDP-eligible data items to document, the specification of DDP-eligible items need not be part of the Upper Layer Binding for an Upper Layer Protocol that operates only in the backward direction.
I suggest adding a new paragraph after the existing third paragraph, reading as follows:
An Upper Layer Protocol that operates on RPC-over-RDMA transports and incorporates operation in both directions may have DDP-eligible data items for transfers in either or both directions. All of these are specified in an Upper Layer Binding document, as they would be for a protocol which only operates in a single direction. This applies even when, as in the case of NFSv4. the XDR is constructed so that each direction of operation is defined as part of a separate XDR program.
In the last paragraph, suggest replacing "else may be contained" by "is required".

nfsv4 mailing list
nfsv4 <at> ietf.org