headers
STEPHAN Emile RD-CORE-LAN | 2 May 2005 11:54

RE: [ipfix-protocol] new version of the IPFIX protocol draft: draft-ietf-ipfix-protocol-12.txt

Dear Benoit,

 

What is the max length of a fixed length field ? Is it 65534 ? Does that make sense as the max length of an IPFIX msg is 65535 ?

 

At large, in the future IPFIX will need to define new kind of field types. The current definition of  'Field Length' must be prepared for that. I propose to reserve the MSB of 'Field Length' for future usage.

 

Regards

Emile

 

De : majordomo listserver [mailto:majordomo <at> mil.doit.wisc.edu] De la part de Benoit Claise
Envoyé : mardi 19 avril 2005 01:43
À : ipfix-protocol <at> net.doit.wisc.edu; me
Cc : Paul Aitken
Objet : [ipfix-protocol] new version of the IPFIX protocol draft: draft-ietf-ipfix-protocol-12.txt

 

Dear all,

I just posted a new version of the IPFIX protocol draft, with the feedback from Paul Aitken's thorough review.

Here are the list of changes:
- Some cross references were wrong
- Flow Record definition exactly similar to the one in [IPFIX-ARCH]
- Template definition exactly similar to the one in [IPFIX-ARCH]
- As deduced from section 9, the source ID definition now contains: the source ID MUST NOT be 0
- The Field Length is completed with: The value 65535 is reserved for variable length Information Element (see section 7). The Field Length MAY NOT 0.
- one mistake corrected in figure O
- correction: In most cases the length of the Information Element will be less than 256 255 octets.
- correction: The IPFIX Message Header 16-bit Length field limits the length of a IPFIX Message to 65536 65535 octets including the header.
- section 10.4.1.2 referred to SCTP while it was a TCP section. Ooops ;) BTW, the TCP sections now speaks of TCP connection as opposed to TCP association
- added the 2 sentences

   When the TCP connection restarts, the Exporting Process MUST resend    all the Template Records.          When an TCP connection is closed, the Collecting Process MUST    discard all templates received over that connection and stop    decoding IPFIX Messages that use those templates. - A couple of mistakes in the examples. I advice to review the changes at http://ietf.levkowetz.com/drafts/ipfix/, once the draft is published.  Regards, Benoit.

 

     

 

 

- A lot of editorial corrections and improvements

Permalink | Reply | 1 comment |
headers
STEPHAN Emile RD-CORE-LAN | 2 May 2005 12:11

[ipfix-protocol] Generalisation of the Reduced Size Encoding

Dear Bryant, Benoit

Protocol-v12 does not generalise the size encoding reduction to string and array. This should be added
because such kinds of fields are defined with a length longer that those used (e.g. wlanSsid).

Regards
Emile

> -----Message d'origine-----
> De : Stewart Bryant [mailto:stbryant <at> cisco.com]
> Envoyé : jeudi 7 avril 2005 13:15
> À : STEPHAN Emile RD-CORE-LAN
> Cc : Benoit Claise; ipfix-protocol <at> net.doit.wisc.edu
> Objet : Re: [ipfix-protocol] draft-ietf-ipfix-protocol-11.txt: 6.2 Reduced
> Size Encoding of Integer Types
> 
> 
> 
> STEPHAN Emile RD-CORE-LAN wrote:
> > Dear Benoit,
> >
> > This section says that an exporter MAY use shorter IE length than in the
> > info model but does not clearly say that in this case it is mandatory to
> > declare these length in the template.
> 
> That is already stated in section 3.2
> 
> "Field Length
>      The length of the corresponding encoded Information Element,
>      in octets.  Refer to [IPFIX-INFO].  The field length may be
>      smaller than the definition in [IPFIX-INFO] if reduced size
>      encoding is used (see section 7.2)."
> 
> 
> 
> >
> > I propose to replace the beginning of this section with:
> >
> > "Information Elements containing integer types in the information
> > model MAY be encoded using fewer octets than those implied by their
> > type in the information model definition [IPFIX-INFO], in this case the
> > exporter MUST indicate these lengths in the template ...
> >
> 
> I don't think that your added text "in this case the
> exporter MUST indicate these lengths in the template"
> 
> is needed since you have to send exactly the number of octets
> that you declared in the template.
> 
> BTW - the reduction would also apply to a fixed length string
> object, say a name, that might be defined as an IE of max length
> X (to tell the collector DB the max size it needs to store)
> but which the exporter knew woud only ever take Y<X to describe.
> We should therefore increase the scope to include any
> primative type where the collector can figure out the semantics
> of a reduced size export.
> 
> Regards
> 
> Stewart
> 
> >
> > Regards
> > Emile
> >
> > --
> > Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message
> body
> > Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> >
> >

--
Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/
Permalink | Reply | 1 comment |
headers
Lutz Mark | 3 May 2005 11:54
Picon
Favicon

[ipfix-protocol] Encoding of IPFIX Data Types


 > 6.1.5   string and octetarray
 >
 >   The data type string represents a finite length string of valid
 >   characters of the Unicode character encoding set.  It is expected
 >   that strings will be encoded in UTF-8 format. The string is sent as
 >   an array of octets.  The length of the string and the octetArray is
 >   encoded as described in Section 7 (Variable Length Information
 >   Element).  The length is followed by that many octets as encoded in
 >   the length.

The section implies that you have to use variable length IEs to
export strings and octetarrays (or does the text mean that you
have to send the length twice? It's not clear to me).
I suggest to change the text to allow the export of strings and 
octetarrays in an IE of fixed length.

For example:

    The data type string represents a finite length string of valid
    characters of the Unicode character encoding set. It is expected
    that strings will be encoded in UTF-8 format. The string is sent as
    an array of octets. The length of the information element
    specifies the length of the octedarray.

Regards,
Lutz

--
Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/
Permalink | Reply | 7 comments |
headers
Lutz Mark | 3 May 2005 12:22
Picon
Favicon

[ipfix-protocol] 9. The Collecting Process's Side


 >   The Collecting Process MUST verify that only one Source ID value is
 >   used inside each stream. If the Collecting Process detects that more
 >   than one Source ID has been received within a stream, it MUST
 >   discard the IPFIX Message, reset the SCTP association, and SHOULD
 >   log the error.

this does not map with the behaviour of using IPFIX over TCP.
With TCP it is possible to export data of multiple observation
domains using only one connection.

Is there a motivation for this restriction that
also applies to TCP?

What about removing this restriction for SCTP?

Regards,
Lutz

--
Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/
Permalink | Reply | 1 comment |
headers
Stewart Bryant | 3 May 2005 14:22
Picon
Favicon

Re: [ipfix-protocol] Encoding of IPFIX Data Types


Lutz Mark wrote:

> 
>  > 6.1.5   string and octetarray
>  >
>  >   The data type string represents a finite length string of valid
>  >   characters of the Unicode character encoding set.  It is expected
>  >   that strings will be encoded in UTF-8 format. The string is sent as
>  >   an array of octets.  The length of the string and the octetArray is
>  >   encoded as described in Section 7 (Variable Length Information
>  >   Element).  The length is followed by that many octets as encoded in
>  >   the length.
> 
> The section implies that you have to use variable length IEs to
> export strings and octetarrays (or does the text mean that you
> have to send the length twice? It's not clear to me).

There are three possible ways to send a string:

1) Fixed length
2) Short variable length
3) Long variable length.

In case 1, you put the length in the template and the string
is always that length, and if your string is shorter you have
to pad.

In case 2 and 3 you use a special length value in the template
which says that the length is encoded at the start of the string.

> I suggest to change the text to allow the export of strings and 
> octetarrays in an IE of fixed length.
> 
> For example:
> 
>    The data type string represents a finite length string of valid
>    characters of the Unicode character encoding set. It is expected
>    that strings will be encoded in UTF-8 format. The string is sent as
>    an array of octets. The length of the information element
>    specifies the length of the octedarray.

This is only true for fixed length strings.

Regards

Stewart

> 
> Regards,
> Lutz
> 
> 
> -- 
> Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message 
> body
> Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
> 
> 

--
Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/
Permalink | Reply | 6 comments |
headers
STEPHAN Emile RD-CORE-LAN | 3 May 2005 15:10

RE: [ipfix-protocol] Encoding of IPFIX Data Types


Dear Stewart, Lutz

I suggest that section 6.2 defines the reduced size encoding not only for integer but for string and array
too. That will clarify the link to section 6.2 in section "3.2 Field Specifier". 

Regards
Emile
> -----Message d'origine-----
> De : majordomo listserver [mailto:majordomo <at> mil.doit.wisc.edu] De la part
> de Stewart Bryant
> Envoyé : mardi 3 mai 2005 14:22
> À : Lutz Mark
> Cc : ipfix-protocol <at> net.doit.wisc.edu
> Objet : Re: [ipfix-protocol] Encoding of IPFIX Data Types
> 
> 
> 
> Lutz Mark wrote:
> 
> >
> >  > 6.1.5   string and octetarray
> >  >
> >  >   The data type string represents a finite length string of valid
> >  >   characters of the Unicode character encoding set.  It is expected
> >  >   that strings will be encoded in UTF-8 format. The string is sent as
> >  >   an array of octets.  The length of the string and the octetArray is
> >  >   encoded as described in Section 7 (Variable Length Information
> >  >   Element).  The length is followed by that many octets as encoded in
> >  >   the length.
> >
> > The section implies that you have to use variable length IEs to
> > export strings and octetarrays (or does the text mean that you
> > have to send the length twice? It's not clear to me).
> 
> 
> There are three possible ways to send a string:
> 
> 1) Fixed length
> 2) Short variable length
> 3) Long variable length.
> 
> In case 1, you put the length in the template and the string
> is always that length, and if your string is shorter you have
> to pad.
> 
> In case 2 and 3 you use a special length value in the template
> which says that the length is encoded at the start of the string.
> 
> > I suggest to change the text to allow the export of strings and
> > octetarrays in an IE of fixed length.
> >
> > For example:
> >
> >    The data type string represents a finite length string of valid
> >    characters of the Unicode character encoding set. It is expected
> >    that strings will be encoded in UTF-8 format. The string is sent as
> >    an array of octets. The length of the information element
> >    specifies the length of the octedarray.
> 
> This is only true for fixed length strings.
> 
> Regards
> 
> Stewart
> 
> >
> > Regards,
> > Lutz
> >
> >
> > --
> > Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message
> > body
> > Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> >
> >
> 
> --
> Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message
> body
> Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/
Permalink | Reply | 2 comments |
headers
Benoit Claise | 3 May 2005 17:07
Picon
Favicon

Re: [ipfix-protocol] 9. The Collecting Process's Side

Lutz,

>
> >   The Collecting Process MUST verify that only one Source ID value is
> >   used inside each stream. If the Collecting Process detects that more
> >   than one Source ID has been received within a stream, it MUST
> >   discard the IPFIX Message, reset the SCTP association, and SHOULD
> >   log the error.
>
> this does not map with the behaviour of using IPFIX over TCP.
> With TCP it is possible to export data of multiple observation
> domains using only one connection.
>
> Is there a motivation for this restriction that
> also applies to TCP?
>
> What about removing this restriction for SCTP?

As far as I recall, this was specify initially to get some fairness 
across multiple source ID sending template records from a single 
exporting process (a distributed platform). However, after thinking 
about it with Stewart Bryant, we think this is an implementation choice.
So we're fine with the removal of this restriction.
Then "The Exporting Process MUST NOT transmit IPFIX Messages with more 
than one Source ID value inside any single stream." in section 8 should 
also go away.

Regards, Benoit.

>
> Regards,
> Lutz
>
>
> -- 
> Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in 
> message body
> Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/
Permalink | Reply | 0 comments |
headers
Benoit Claise | 3 May 2005 17:32
Picon
Favicon

Re: [ipfix-protocol] new version of the IPFIX protocol draft: draft-ietf-ipfix-protocol-12.txt

Dear Emile,

Dear Benoit,

 

What is the max length of a fixed length field ? Is it 65534 ?

The draft says:
Field Length The length of the corresponding encoded Information Element, in octets. Refer to [IPFIX-INFO]. The field length may be smaller than the definition in [IPFIX-INFO] if reduced size encoding is used (see section 6.2). The value 65535 is reserved for variable length Information Element (see section 7). The Field Length MAY NOT 0. So we don't specify what the maximum length is, we simply say that 65535 is reserved. Obviously this is below 65535. Anyway, the length is specify by the information model

Does that make sense as the max length of an IPFIX msg is 65535 ?

IMHO, this offers the maximum of flexibility.

 

At large, in the future IPFIX will need to define new kind of field types. The current definition of  'Field Length' must be prepared for that. I propose to reserve the MSB of 'Field Length' for future usage.

How to export in UDP an IPFIX Message whose length is above 65535 is not that such a trivial issue!
So I propose to start with IPFIX 1.0, get some adoption, see the limitations, and go from there.

Regards, Benoit.

 

Regards

Emile

 

De : majordomo listserver [mailto:majordomo <at> mil.doit.wisc.edu] De la part de Benoit Claise
Envoyé : mardi 19 avril 2005 01:43
À : ipfix-protocol <at> net.doit.wisc.edu; me
Cc : Paul Aitken
Objet : [ipfix-protocol] new version of the IPFIX protocol draft: draft-ietf-ipfix-protocol-12.txt

 

Dear all,

I just posted a new version of the IPFIX protocol draft, with the feedback from Paul Aitken's thorough review.

Here are the list of changes:
- Some cross references were wrong
- Flow Record definition exactly similar to the one in [IPFIX-ARCH]
- Template definition exactly similar to the one in [IPFIX-ARCH]
- As deduced from section 9, the source ID definition now contains: the source ID MUST NOT be 0
- The Field Length is completed with: The value 65535 is reserved for variable length Information Element (see section 7). The Field Length MAY NOT 0.
- one mistake corrected in figure O
- correction: In most cases the length of the Information Element will be less than 256 255 octets.
- correction: The IPFIX Message Header 16-bit Length field limits the length of a IPFIX Message to 65536 65535 octets including the header.
- section 10.4.1.2 referred to SCTP while it was a TCP section. Ooops ;) BTW, the TCP sections now speaks of TCP connection as opposed to TCP association
- added the 2 sentences

   When the TCP connection restarts, the Exporting Process MUST resend    all the Template Records.           When an TCP connection is closed, the Collecting Process MUST    discard all templates received over that connection and stop    decoding IPFIX Messages that use those templates. - A couple of mistakes in the examples.   I advice to review the changes at http://ietf.levkowetz.com/drafts/ipfix/, once the draft is published.   Regards, Benoit.

 

     

 

 

- A lot of editorial corrections and improvements


Permalink | Reply | 0 comments |
headers
STEPHAN Emile RD-CORE-LAN | 3 May 2005 19:06

[ipfix-protocol] info model flexibility

Hi Benoit,

 

The info model (section 2.1) is designed to be managed dynamically by IANA without moving to a new protocol version. So I consider that more flexibility must be given to the specification of Information Elements in the info model. Let's take a real example:

Downcasting is currently defined in the protocol specification. Consequently it will not be possible to use them in the future with new field types. So Downcasting may be moved in the section 2.1 of the info model.

 

What about replacing "dataType - One of the types listed in section 3.1 of this document…" with" dataType - Types listed in section 3.1 of this document…" ? Or what about defining a downcasting keyword in the MAY list of this section?

 

Regards

Emile

De : Benoit Claise [mailto:bclaise <at> cisco.com]
Envoyé : mardi 3 mai 2005 17:33
À : STEPHAN Emile RD-CORE-LAN
Cc : ipfix-protocol <at> net.doit.wisc.edu
Objet : Re: [ipfix-protocol] new version of the IPFIX protocol draft: draft-ietf-ipfix-protocol-12.txt

 

Dear Emile,


Dear Benoit,

 

What is the max length of a fixed length field ? Is it 65534 ?

The draft says:

Field Length             The length of the corresponding encoded Information Element,             in octets.  Refer to [IPFIX-INFO].  The field length may be             smaller than the definition in [IPFIX-INFO] if reduced size             encoding is used (see section 6.2).  The value 65535 is               reserved for variable length Information Element (see             section 7). The Field Length MAY NOT 0.  So we don't specify what the maximum length is, we simply say that 65535 is reserved. Obviously this is below 65535. Anyway, the length is specify by the information model

Does that make sense as the max length of an IPFIX msg is 65535 ?

IMHO, this offers the maximum of flexibility.

 

At large, in the future IPFIX will need to define new kind of field types. The current definition of  'Field Length' must be prepared for that. I propose to reserve the MSB of 'Field Length' for future usage.

How to export in UDP an IPFIX Message whose length is above 65535 is not that such a trivial issue!
So I propose to start with IPFIX 1.0, get some adoption, see the limitations, and go from there.

Regards, Benoit.

 

Regards

Emile

 

De : majordomo listserver [mailto:majordomo <at> mil.doit.wisc.edu] De la part de Benoit Claise
Envoyé : mardi 19 avril 2005 01:43
À : ipfix-protocol <at> net.doit.wisc.edu; me
Cc : Paul Aitken
Objet : [ipfix-protocol] new version of the IPFIX protocol draft: draft-ietf-ipfix-protocol-12.txt

 

Dear all,

I just posted a new version of the IPFIX protocol draft, with the feedback from Paul Aitken's thorough review.

Here are the list of changes:
- Some cross references were wrong
- Flow Record definition exactly similar to the one in [IPFIX-ARCH]
- Template definition exactly similar to the one in [IPFIX-ARCH]
- As deduced from section 9, the source ID definition now contains: the source ID MUST NOT be 0
- The Field Length is completed with: The value 65535 is reserved for variable length Information Element (see section 7). The Field Length MAY NOT 0.
- one mistake corrected in figure O
- correction: In most cases the length of the Information Element will be less than 256 255 octets.
- correction: The IPFIX Message Header 16-bit Length field limits the length of a IPFIX Message to 65536 65535 octets including the header.
- section 10.4.1.2 referred to SCTP while it was a TCP section. Ooops ;) BTW, the TCP sections now speaks of TCP connection as opposed to TCP association
- added the 2 sentences

   When the TCP connection restarts, the Exporting Process MUST resend    all the Template Records.          When an TCP connection is closed, the Collecting Process MUST    discard all templates received over that connection and stop    decoding IPFIX Messages that use those templates. - A couple of mistakes in the examples. I advice to review the changes at http://ietf.levkowetz.com/drafts/ipfix/, once the draft is published.  Regards, Benoit.

 

     

 

 

- A lot of editorial corrections and improvements

 

Permalink | Reply | 1 comment |
headers
Benoit Claise | 4 May 2005 10:03
Picon
Favicon

[ipfix-protocol] One correction in the "octets" information elements

Juergen,

We realized that the Information Element #1 octetDeltaCount, which is in line with NetFlow version 9, is
not correctly specified.

5.9.1  octetDeltaCount
  Description:
     The number of octets since the previous report (if any) in
     incoming packets for this flow at the Observation Point.  The
     number of octets include IP header(s) and IP payload.

-> New Description:
     The number of octets since the previous report (if any) in
     incoming packets for this flow at the Observation Point.  The
     number of octets exclude the data link header and trailer.

This allows to account for the length of MPLS labels, for the ISL bytes, etc...

Several I.E. below the 128 should be updated
octetDeltaCount #1, 
postOctetDeltaCount #23,
octetTotalCount #85,
postMulticastOctetCount #20

And to be consistent amongst the I.E., the I.E. above 128 should be updated as well
postOctetTotalCount #160,
droppedOctetDeltaCount #132,
droppedOctetTotalCount #134 

Regards, Benoit.

--
Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/
Permalink | Reply | 9 comments |

Gmane