Brian Trammell | 8 Mar 2006 17:00
Favicon

[ipfix-protocol] nit: IPFIX over TLS over SCTP

All,

Section 11 (page 47) of proto-19 makes a reference to using TLS over  
SCTP as a transport for IPFIX. However, there is no further  
information (as in TCP, sections 10.4.1.2, 10.4.2) on the interaction  
between TLS and SCTP, and no normative reference to RFC 3436, which  
defines the usage of TLS over SCTP.

Reference to TLS should be made in sections 10.2.4.1 (noting the  
requirement to use reliable bidirectional streams when using TLS as  
in 3436 section 7; and the requirement for the Metering Process to  
establish a TLS session across each stream as shown in 3436 section  
8.1), 10.2.4.2 (noting TLS session teardown on stream shutdown), and  
10.2.4.3 (again, noting unreliable and partially reliable streams are  
not available for TLS). Additional references may be necessary as  
well; I've just taken a first pass here for sake of feedback.

Additionally, RFC 3436 "Transport Layer Security over Stream Control  
Transmission Protocol" should be added to the protocol document's  
normative references.

Thoughts?

- Brian
Brian Trammell | 8 Mar 2006 20:54
Favicon

[ipfix-protocol] semantics of multiple identical Information Elements in a template

Benoit,

Regarding multiple identical information elements, you proposed the  
following a couple of months ago:

> In section 8 "Template Management", after the following paragraph...
>
> If a specific Information Element is required by a Template but is  
> not available in observed packets, the Exporting Process MAY choose  
> to export Flow Records without this Information Element in a Data  
> Record defined by a new Template.
>
> I would add:
>
> If an Information Element is required more than once in Template,  
> the different occurrences of this Information Element SHOULD follow  
> the logical order of their treatments by the Metering Process. For  
> example, if a selected packet goes through two hash functions, and  
> if the two hash values are sent within a single template, the first  
> occurrence of the hash value should belong to the first hash  
> function in the Metering Process. For example, when exporting the  
> two source IP addresses of an IPv4 in IPv4 packets, the first  
> sourceIPv4Address Information Element occurrence should be the IPv4  
> address of the outer header, while the second occurrence should be  
> the inner header one.
>
> In section 9 "The Collecting Process's Side", at the very end, I  
> would add:
>      The Collector MUST support the use of Templates containing  
> multiple occurrences of
(Continue reading)

Lutz Mark | 9 Mar 2006 10:19
Picon
Favicon

Re: [ipfix-protocol] semantics of multiple identical Information Elements in a template


Hi Brian,

find my comments inline.

> Regarding multiple identical information elements, you proposed the 
> following a couple of months ago:
> 
>> In section 8 "Template Management", after the following paragraph...
>>
>> If a specific Information Element is required by a Template but is not 
>> available in observed packets, the Exporting Process MAY choose to 
>> export Flow Records without this Information Element in a Data Record 
>> defined by a new Template.
>>
>> I would add:
>>
>> If an Information Element is required more than once in Template, the 
>> different occurrences of this Information Element SHOULD follow the 
>> logical order of their treatments by the Metering Process. For 
>> example, if a selected packet goes through two hash functions, and if 
>> the two hash values are sent within a single template, the first 
>> occurrence of the hash value should belong to the first hash function 
>> in the Metering Process. For example, when exporting the two source IP 
>> addresses of an IPv4 in IPv4 packets, the first sourceIPv4Address 
>> Information Element occurrence should be the IPv4 address of the outer 
>> header, while the second occurrence should be the inner header one.
>>
>> In section 9 "The Collecting Process's Side", at the very end, I would 
>> add:
(Continue reading)

Brian Trammell | 9 Mar 2006 17:54
Favicon

Re: [ipfix-protocol] semantics of multiple identical Information Elements in a template

Lutz,

Comments, as usual, inline...

On Mar 9, 2006, at 4:19 AM, Lutz Mark wrote:

>> I'm probably a bit late to the party on this, but yes, this seems  
>> exactly the right thing to do for IPFIX version 0x000A. However, I  
>> think we should be more explicit about the semantics implied by  
>> this approach, and more restrictive on the applicability of  
>> multiple information elements in order to avoid any grey areas  
>> that might be used by implementors to overload multiple IE support  
>> in mutually unintelligible ways.
>
> I disagree about this. I prefer to keep Benoit's proposal.
> The IPFIX protocol draft just defines the protocol and does not
> care about the semantics. For instance one can use IPFIX protocol
> with vendor specific information elements and with multiple IEs
> of the same type. The protocol shouldn't restrict this.

Defining IPFIX to be a semantics-free representation makes absolute  
sense when each IE appears only once. Note that even this case is not  
_really_ semantics-free -- "this IE value applies to this Flow  
Record" is an assertion carrying some meaning, however thin. It's  
simple to handle private enterprise IEs in this framework - you may  
not have any idea what a given IE means, but you can unambiguously

However, once you allow multiple IE instances per Flow Record, as we  
have here, it does not appear possible not to make the following  
choices and still have a universally understandable data format:
(Continue reading)

Paul Aitken | 9 Mar 2006 19:45
Picon
Favicon
Gravatar

Re: [ipfix-protocol] semantics of multiple identical Information Elements in a template

Brian,

> if a biflow record has six payload hashes, do the first three apply
> to the forward direction flow, or are the directions interleaved?

If it's not 100% clear and unambiguous, then the biflow design is 
clearly incomplete.

For my 2€:

Presumably the device meansures the same information in both directions
- perhaps it doesn't even care which direction is which.

So divide and conquer: let the device export one template and two records.

The template contains a direction element and three hash elements. 
Define their order as you will (eg, time or location).

Then one record shows "I observed these three hashes in direction A", 
while the other says the same about the other three hashes in direction B.

--

-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.

--
Help        mailto:majordomo <at> net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo <at> net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/
(Continue reading)

Brian Trammell | 9 Mar 2006 19:53
Favicon

Re: [ipfix-protocol] semantics of multiple identical Information Elements in a template

Paul,

On Mar 9, 2006, at 1:45 PM, Paul Aitken wrote:

> Brian,
>
>> if a biflow record has six payload hashes, do the first three apply
>> to the forward direction flow, or are the directions interleaved?
>
> If it's not 100% clear and unambiguous, then the biflow design is  
> clearly incomplete.

My point exactly. As currently specified, multiple identical IEs do  
not appear sufficient for the biflow use case. (The single-record  
biflow export presently advanced in the biflow draft does not make  
use of multiple IEs, for precisely this reason. Direction is  
explicitly specified in the definition of the information elements.)

Regards,

Brian
Brian Trammell | 9 Mar 2006 20:36
Favicon

Re: [ipfix-protocol] semantics of multiple identical Information Elements in a template

Lutz,

Oops. Interrupted myself in mid thought on this one. To finish...

On Mar 9, 2006, at 11:54 AM, Brian Trammell wrote:

> Defining IPFIX to be a semantics-free representation makes absolute  
> sense when each IE appears only once. Note that even this case is  
> not _really_ semantics-free -- "this IE value applies to this Flow  
> Record" is an assertion carrying some meaning, however thin. It's  
> simple to handle private enterprise IEs in this framework - you may  
> not have any idea what a given IE means, but you can unambiguously
...state that the value applies to the record.

Regards,

Brian
Lutz Mark | 12 Mar 2006 23:15
Picon
Favicon

Re: [ipfix-protocol] semantics of multiple identical Information Elements in a template


Brian,

>>> if a biflow record has six payload hashes, do the first three apply
>>> to the forward direction flow, or are the directions interleaved?
>>
>> If it's not 100% clear and unambiguous, then the biflow design is 
>> clearly incomplete.
> 
> My point exactly...

So let's make a well-defined design.

There are definitely multiple methods to export biflow data
via IPFIX protocol. In most cases the method Paul described
would be sufficient. The usage of special Information Elements
for the reverse direction fields is another method. And since
IPFIX allows the use of multiple IEs of the same type within
one flow record, there might be a smart method to exploit
this for the export of biflow records. I can think of further
methods to export biflow data.
Let's collect the alternatives in the biflow draft and
discuss their pros and cons and which method fits best
for different use cases.

Maybe we should also pick this issue and give a
recommendation in the implementation guidelines.

Best regards,
Lutz
(Continue reading)

Benoit Claise | 31 Mar 2006 00:30
Picon
Favicon

Re: [ipfix] AD review for: draft-ietf-ipfix-protocol-19.txt

Bert, all,

Thanks for your comments. See inline.
For clarity purposes, I removed the obvious editorial improvements, which have been inserted already.

Note that I made the following changes:
- I modified all occurrences of Source ID to Observation Domain ID.
- As proposed by Brian Trammell, I clarified:
Add to the end of 10.2.4.1 Association Establishment:
"Each SCTP stream over which TLS [TLS] will be used MUST be established as a fully reliable bidirectional stream as required by RFC 3436."

Insert a new third paragraph to 10.2.4.2 Association Teardown:
"If TLS [TLS] is used, the Process initiating association teardown SHOULD send a close_notify alert over each TLS stream before closing the SCTP association."

Append to the second paragraph to 10.2.4.3 Stream:
"Only fully reliable bidirectional streams are available if TLS [TLS] as used, as required by RFC 3436."
          [RFC3436] added as a normative reference
- Sect 3. States that Template Sets MUST be sent periodically when UDP is used. What if TCP or or SCTP are used? Might want to state what expected behaviour is in those cases as well.
I agree this is confusing, as we don't know yet at this stage that UDP is a MAY.
Since these are examples, since the first two examples don't discuss any transport protocol issues and since the transport protocol is explained at length in the section 10, I have removed the sentences.

- sect 3.1 under SOurce ID I do not understand the last sentence. WHat does it mean? pls clarify in text.
As proposed in http://ipfix.doit.wisc.edu/archive/3145.html, I moved it into the section "3.4.2.1 Scope... where it naturally belongs
- Page 16 under padding s/zero (0) octets/zero (0) valued octets/ or "octets with a value of zero (o)" ??
Done. "the padding octet(s) MUST be composed of zero (0) valued octets. "
Last sentence under "padding" may need a period or semicolon in the middle of the sentence?
Done.
- Sect 5/6 speak about ...DeltaUSeconds, but I cannot find that term in INFO spec. It talks about ...DeltaMicroSeconds. Is that what you mean?
Corrected already. Someone mentioned it on the mailing
- sect 6.1.3 I'd like to see a reference to the IEEE spec for floats.
Corrected already. This was discussed on the mailing list.

 6.1.3   float32
   
   The float32 data type is encoded as an IEEE single-precision 32-bit
   floating point-type, as specified in [IEEE.754.1985].  

 6.1.4   float64
   
   The float64 data type is encoded as an IEEE double-precision 64-bit
   floating point-type, as specified in [IEEE.754.1985].
- sect 6.1.5 I am not sure that by saying "it is expected that strings will be coded in UTF-8 format" will guanrantee interoiperability? Maybe better use MUST (RFC2119) language.
I have right now:
The data type string represents a finite length string of valid characters of the Unicode character encoding set.  The string data type MUST be encoded in UTF-8 format.  The string is sent as an array of octets using an information element of fixed or variable length. The length of the information element specifies the length of the octetarray.
Since this is required for interoperability, I think the "MUST" is more appropriate. So I changed all definitions in 6.1.X: from "is encoded" to "MUST be encoded"


Note: we have the same sentence in [IPFIX-INFO]:
It is expected that strings will be encoded in UTF-8 format, which is identical in encoding for ASCII characters, but also accommodates other Unicode multi-byte characters. I think this should be removed, as the information model should not tell how to encode the data.


Last sentence of 6.1.5 talks about "NUL" characters. That again seems to be ASCII language. Maybe better state "zero valued octets" .
Following some discussions, the sentence "In case of fixed length Information Element, if padding is required, padding MUST be composed of NUL character(s)." has been removed. All padding related info is now in the padding section.
- sect 6.1.6 YOu might want to add a sentence how long this can serve us. I believ it is something like 146 years,
136 years. Added.

The data type dateTimeSeconds represents a time value in units of seconds normalised to the GMT timezone.  It MUST be encoded in a 32-bit integer containing the number of seconds since 0000 UTC Jan 1st 1970. The 32-bit integer allows the time encoding up to 136 years.


- sect 6.2 speaks about signed64/unsigned64 and also for 32 and 16. Where are those types defined??
Previous text:
6.1.1 Integral Data Types Integral data types - octet, unsigned16, unsigned32 and unsigned64 - are encoded using the default canonical format in network byte order. New text: 6.1.1 Integral Data Types Integral data types - octet, signed octet, unsigned16, signed16, unsigned32 signed32, signed64 and unsigned64 - are encoded using the default canonical format in network byte order. Signed Integral data types are represented in  two's complement notation.
- one but last para on page 33 "sufficient time" should be defined/explained somewhat don't you think so?
I added the last sentence.
"The Template Withdraw Message MUST not be sent until sufficient time has elapsed to allow the Collecting Process to receive and process the last Data Record using this Template information.
This time MUST be configurable.  A suitable default value is 5 seconds after the last Data Record has been sent.


- I wonder if the one but last para of section 9 is a nice opening for DoS attacks?
The sentence is: "If the Collecting Process receives a Template Withdraw message for a Template Record it has not received before, it MUST reset the SCTP association, discard the IPFIX Message, and SHOULD log the error as it does for malformed IPFIX Messages."

The point is that we need that statement because it means that the collector is confused about its own status. And the only solution is to shutdown the SCTP association, so that the exporter will resend all the templates with a new SCTP association.


- sect 10.2.6 Where/how is the treshold specified? Same for sect 10.4.1.1
I'm not sure how to answer the questions:
    Where? on the exporter
    How? based on the user appreciation. This is exactly why we have a MAY statement.
We could say "exceeds a locally configured threshold. By default this time is foo secs." However, I have no clue what the value of foo should be.
If the statements are too vague without the default value, I simply propose to remove them.

- sect 10.3.7 It is unclear to me how/where the template lifetimes are specified, defined or configured.
We wrote the following sentence: The Template lifetime at the Collecting Process MUST be at least 3 times higher that the Template refresh timeout configured on the Exporting Process. Isn't it enough?
- sect 11.4 last para I guess that just depends of who/where the attacker is, no? WHat if it is an internal ISP employee?
Old text:
The use of a dedicated network prevents IPFIX Messages from being inspected by an attacker. New text: The use of a dedicated network prevents IPFIX Messages from being inspected by an external attacker.
- sect 12.1 I do not think the IPfix version number is an IANA controlled thing. I also think you want that to be changebale only by an IETF standards action (as opposed to just IETF consensus). Afetr all you want this protocol to be stds track.
It was actually the intend behind the sentence:
"Changes in either IPFIX Version Number or IPFIX Set ID assignments require an IETF Consensus, i.e. they are to be made via RFCs approved by the IESG." Should we simply erase the section 12.1? Or make clearer? "Changes in either IPFIX Version Number or IPFIX Set ID assignments are not controlled by IANA, but by an RFC approved by the IESG."
- sect 12.2 seems to be IPFIX-INFO material and not protocol material??
Section 12.2 erased.

- Sect 12.1 The "IPFIX Set ID" starts (I think) a new registry. So you MUST describe under IANA considerations how to set it up and what the rules are for new assignements. You have that text (more or less) in sect 3.3.2, but you must spell it out here for IANA as well.
New section "12.x Set ID"

Set ID value identifies the IPFIX Set. Set ID values of 0 and 1 are not used for historical reasons [RFC3954]. A value of 2 is reserved for the Template Set. A value of 3 is reserved for the Option Template Set. All other values from 4 to 255 are reserved for future use. Values above 255 are used for Data Sets.
The new Set ID values are defined as "Specification Required" [RFC 2434] prior to the assignment by IANA.

Then you have a set of Set IDs that get assigned by this document and you should list them clearly for IANA so they can do the initial population of the registry. - It seems to me that sect 13 is more an APPENDIX and so non-normative? Might be good to then list it as appendix and state that it is non-normative.
I moved the appendix just before the reference section.
ftp://ftp.rfc-editor.org/in-notes/rfc-editor/instructions2authors.txt doesn't mention that we should mention something about non-normative ... unless I missed something?

- Refrences. - is 1889 really normative?
Moved to informative
!! Missing citation for Informative reference: P060 L046: [RFC3955] Leinen, S., "Evaluation of Candidate Protocols for IP Flow
Removed this reference.
- sect 3.1 remove one of the two words "format"
You migth also want to add a sentence to explain why the version field is 0x000a for this version.
I inserted this proposal

Version

Version of Flow Record format exported in this message.  The value of this field is 0x000a for the current version, incrementing by one the version used in the NetFlow services export version 9 [RFC3954].


- page 33 s/Disused/Unused/ ??
Let me copy the entire paragraph: Disused Templates SHOULD be deleted. Before reusing a Template ID the disused Template MUST be deleted. In order to delete an allocated Template, the Template is withdrawn through the use of a Template Withdraw Message. The term "unused" is already applied in this section with a different meaning. For example "an unused Template ID MUST be used". This means: allocate a new Template ID. However, in the sentence you pointed out: we want to say: the templates that are not used anymore. I propose: "The Templates that are not used anymore SHOULD be deleted. Before reusing a Template ID, the Template MUST be deleted"
Thanks again for your comments. 

Regards, Benoit.


Benoit Claise | 31 Mar 2006 09:48
Picon
Favicon

[ipfix] Re: Source ID -> Observation Domain Id? -> New Definition proposals

Thanks Thomas,

Corrected.

Regards, Benoit.
Dear Benoit, the text is fine. I just pointed at some wording issues inside. Regards, Thomas Am Donnerstag, 30. März 2006 16:56 schrieb Benoit Claise:
Dear all, Considering also that we proposed to change the "Source ID" to "Observation Domain ID", we have to change the two definitions (Observation Domain and Source ID) in all the IPFIX and PSAMP drafts. Here is the old "Observation Domain" definition An Observation Domain is the largest set of Observation Points for which Flow information can be aggregated by a Metering Process. Each Observation Domain presents itself using a unique ID to the Collecting Process to identify the IPFIX Messages it generates. For example, a router line card may be an observation domain if it is composed of several interfaces, each of which is an Observation Point. Every Observation Point is associated with an Observation Domain. Here is the new "Observation Domain" proposal. An Observation Domain is the largest set of Observation Points for which Flow information can be aggregated by a Metering Process. For example, a router line card may be an Observation Domain if it is composed of several interfaces, each of which is an Observation Point. In the IPFIX Message it generates, the Observation Domain includes its Observation Domain ID, which is a unique per
^^^ is unique per
Exporting Process. That way, the Collecting Process can identify the specific Observation Domain from the Exporter that sends the IPFIX Messages. Every Observation Point is associated with an Observation Domain. Here is the old "Source ID" definition Source ID A 32-bit identifier of the Observation Domain that is locally unique to the Exporting Process. The Exporting Process uses the Source ID to uniquely identify to the Collecting Process the Observation Domain that metered the Flows. Collecting Processes SHOULD use the combination the combination of the Exporter (exporterIPv4Address, exporterIPv6Address, or exportingProcessId) and the Source ID field to separate different export streams originating from the same Exporting Process. The Source ID SHOULD be 0 when no specific Source ID is relevant for the entire IPFIX Message. For example, when exporting the Exporting Process Statistics, or in case of hierarchy of Collector when aggregated data records are exported. The Source ID MUST be zero when the IPFIX Message contains data records with different Source ID values defined as scopes. Here is the new "Observation Domain ID" definition A 32-bit identifier of the Observation Domain that is locally unique to the Exporting Process. The Exporting Process uses the Observation Domain ID to uniquely identify to the Collecting Process the Observation Domain that metered the Flows. Collecting Processes SHOULD use the combination the combination of the
^^^^^^^^^^^^^^^^ delete once
Exporter (exporterIPv4Address, exporterIPv6Address, or exportingProcessId) and the Observation Domain ID field to separate different export streams originating from the same Exporting Process. The Observation Domain ID SHOULD be 0 when no specific Observation Domain ID is relevant for the entire IPFIX Message. For example, when exporting the Exporting Process Statistics, or in case of hierarchy of Collector when aggregated data records are exported. The last sentence of the definition, on which Bert had also a comment, has been removed from the definition: - sect 3.1 under SOurce ID I do not understand the last sentence. WHat does it mean? pls clarify in text. It's now part of the section "3.4.2.1 Scope". I agree that having this sentence was not too clear, as the readers don't know the notion of scope at this stage of the draft. The Source ID MUST be zero when the IPFIX Message contains data records with different Source ID values defined as scopes.
I guess you already changed "Source ID" to "Observation Domain ID" here...
Feedback? Note: I will post the next version of the protocol draft by tomorrow, as agreed. Regards, Benoit.
Benoit, I also agree. It makes things even clearer. Regards, Thomas Am Freitag, 24. März 2006 01:27 schrieb Benoit Claise:
Dear all, During his review, Bert was confused by the sourceId being the unique Id for the observation domain - [IPFIX-ARCH] sect 5.4 last line s/Source ID/Domain ID/ ?? Or should the section title be somthing about "Source" ?? I am getting a bit confused about DOmain and Source ID I guess? Protocol doc does indeed speak about SOurce ID. I agree that this confusing, and should be changed. However, this must be changed in ALL IPFIX and PSAMP documents. Information model: sourceId -> observationDomainId Protocol: Source ID -> Observation Domain ID Any objections before starting to change the documents? Regards, Benoit.


Gmane