Drafts coming from RDMA Consortium

At the RDDP WG meeting in Yokohama, I said that I
would be working with the RDMA Consortium to get their
work submitted as Internet-Drafts.  That has now come
to pass; the following drafts from the RDMA Consortium
should be appearing on the Internet-Drafts servers in
the near future:

draft-culley-iwarp-mpa-00.txt (MPA framing for TCP)
draft-recio-iwarp-rdmap-00.txt (Remote Direct Memory Access Protocol)
draft-shah-iwarp-ddp-00.txt (Direct Data Placement)

The observant reader will notice that these drafts have
the "wrong" IPR boilerplate on them (NOT in accordance
with Section 10 of RFC 2026, IETF may only publish as an
Internet-Draft).  Subsequent draft versions that have the
"right" boilerplate (subject to all provisions of Section 10
of RFC 2026) will be submitted prior to the Atlanta cutoff
(Nov. 4), and hopefully quite a bit earlier than the cutoff.

The reason for the "wrong" boilerplate on the current versions
is that the legal arrangements under which the drafts were written
includes an <expletive deleted> 30 day legal clock that has
to be run down before the "right" boilerplate can be applied
- it seemed more useful to expose the drafts to the IETF
community for technical discussion now rather than wait 30 days.
Anyone who *really* wants more information on the legal
<expletive deleted>, should send me email directly rather
than posting to one of the WG mailing lists.

I'm reasonably certain that versions of these drafts with the

FW: I-D ACTION:draft-recio-iwarp-rdma-00.txt

FYI, --David

-----Original Message-----
From: Internet-Drafts <at> ietf.org [mailto:Internet-Drafts <at> ietf.org]
Sent: Wednesday, September 18, 2002 7:55 AM
Subject: I-D ACTION:draft-recio-iwarp-rdma-00.txt

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title		: An RDMA Protocol Specification
	Author(s)	: P. Culley, D. Garcia, R. Recio
	Filename	: draft-recio-iwarp-rdma-00.txt
	Pages		: 58
	Date		: 2002-9-17
	
This document defines a Remote Direct Memory Access Protocol (RDMAP) 
that operates over the Direct Data Placement Protocol (DDP 
protocol).  RDMAP provides read and write services directly to 
applications and enables data to be transferred directly into ULP 
Buffers without intermediate data copies. It also enables a kernel 
bypass implementation.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-recio-iwarp-rdma-00.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username

FW: I-D ACTION:draft-shah-iwarp-ddp-00.txt

-----Original Message-----
From: Internet-Drafts <at> ietf.org [mailto:Internet-Drafts <at> ietf.org]
Sent: Wednesday, September 18, 2002 7:57 AM
Subject: I-D ACTION:draft-shah-iwarp-ddp-00.txt

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title		: Direct Data Placement over Reliable Transports
	Author(s)	: P. Culley et al.
	Filename	: draft-shah-iwarp-ddp-00.txt
	Pages		: 35
	Date		: 2002-9-17
	
The Direct Data Placement protocol provides information to Place the 
incoming data directly into an upper layer protocolÆs receive buffer 
without intermediate buffers. This removes excess CPU and memory 
utilization associated with transferring data through the 
intermediate buffers.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-shah-iwarp-ddp-00.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then

STag validation semantics

There are some aspects of STag validation semantics that are
unclear between the rdma and ddp drafts. Additionally, I
believe it is incorrect to address those semantics in the
RDDP document, I believe STag semantics belong as part of
the DDP layer as that it is the DDP layer that executes data
placement.

The specific questions are:

    How is an STag made valid in association with a stream?

    If an STag is valid on multiple streams, what is the
    meaning of an invalidation received on one stream?

The language between the two drafts seems to be consistent
with any of the following interpretations:

    An STag may be associated with exactly one Stream.
    An Invalidation therefore totally invalidates the STag.

    An STag may be associated with exactly one Stream, or it
    may be "unbound". Only an associated STag may be
    invalidated.

    An STag may be associated with any set of Streams
    (either explicitly or through a local interface object
    such as  an InfiniBand Protection Domain).
    An invalidation invalidates the STag only on that
    specific stream, the STag remains valid on any other
    streams that is has been associated with.

RE: STag validation semantics

> I believe these semantics should be clarified, and moved to
> the rddp document. I'll volunteer to do the drafting once a
> concensus is reached on the reflector and the editability of
> the drafts has been established.

There's no need to wait for the latter.  This entire area was
intended to be encompassed by the brief statement in both drafts
that "The STag access control model is defined by a (forthcoming)
document."  The "editability" state of the current drafts should
not be an obstacle to working on that separate draft.

As for the former (i.e., consensus), many thanks to Caitlin for
opening this technical area, please discuss ...

Thanks,
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 249-6449            FAX: +1 (508) 497-8018
black_david <at> emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------

STag Invalidation: local or remote?

Why is the STag to be invalidated part of the Send and
Invalidate message, as opposed to it being part of the local
interface's receive operation?

There are several advantages to keeping this a local
interface issue (even if we mandate its existence):

-- It avoids revealing to the Data Source what the reuse
strategy on STags is. Whether or not an STag has been
delivered multiple times should not be relevant to the Data
Source.

-- It reduces the size of the packet by 32 bits. (Ok, this
isn't very much of a benefit).

-- For most protocols, there will be exactly one STag that
could be invalidated anyway. DAFS for example, delivers one
buffer advertisement per request. The DAFS mapping to DDP
would presumably require the response to invalidate that
tag. The point is: if the value to be supplied was known a
priori to the receiving side, why include it in the packet?

Is there a ULP scenario where there is a pool of advertised
buffers that could be invalidated in an unpredictable order?

If there is, I have no problem with the feature as proposed,
but we should capture the justification in the applicability
statement.

If not, I believe it would make sense to shift this feature

STag Invalidation: local or remote?

Why is the STag to be invalidated part of the Send and
Invalidate message, as opposed to it being part of the local
interface's receive operation?

There are several advantages to keeping this a local
interface issue (even if we mandate its existence):

-- It avoids revealing to the Data Source what the reuse
strategy on STags is. Whether or not an STag has been
delivered multiple times should not be relevant to the Data
Source.

-- It reduces the size of the packet by 32 bits. (Ok, this
isn't very much of a benefit).

-- For most protocols, there will be exactly one STag that
could be invalidated anyway. DAFS for example, delivers one
buffer advertisement per request. The DAFS mapping to DDP
would presumably require the response to invalidate that
tag. The point is: if the value to be supplied was known a
priori to the receiving side, why include it in the packet?

Is there a ULP scenario where there is a pool of advertised
buffers that could be invalidated in an unpredictable order?

If there is, I have no problem with the feature as proposed,
but we should capture the justification in the applicability
statement.

If not, I believe it would make sense to shift this feature

RE: CRC-32 Requirement

DDP draft makes the following requirement:

"LLP MUST provide a strong digest (at least equivalent to CRC32-C) 
to cover at least the DDP Segment. It is believed that some 
of the existing data integrity digests are not sufficient and 
that direct memory transfer semantics require a stronger digest 
than, for example, a simple checksum."

SCTP requires Adler-32 checksum which is "almost as strong as" CRC-32.
I suggest to change the wording to allow an LLP with Adler-32 digest
such as SCTP.

Amir

Re: STag validation semantics

At 12:35 PM 9/19/2002, Caitlin Bestler wrote:
>The first option will not work for DAFS, an STag can be
>delivered on the primary channel and used on the RDMA Read
>channel. Least effort ports of many existing protocols will
>similarly use multile streams and communicate STags that may
>be used on multiple streams within the sesssion.

I disagree - the mechanism by which an STag is delivered has
nothing to do with the scope of the STag's validity. And, DAFS
does not require its STags to have any particular scope - though
the STag used in an RDMA Read must of course be valid at
least on the DAFS RDMA Read channel of the session (or the
DAFS forward channel if the optional RDMA Read channel is
not bound in the session).

>My recomendation is that the method of interfacing be local, 
>or that we suggest something along the model of a InfiniBand 
>Protection Domain. It implements easily, already has 
>solutions and is capable of meeting application 
>requirements.

I agree with you that specifying the scope of the STag is not
a protocol issue, but a local policy. The PD is an excellent
approach to this.

Tom.

RE: CRC-32 Requirement