headers
Benoit Claise | 2 Sep 2011 15:40
Picon
Favicon

Re: [IPFIX] proposal for IPFIX charter update

Hi Juergen,

Double-checking with my meeting minutes, I found some more issues.
See below.
Dear all, At our session in Quebec we discussed candidates for new IPFIX work items. Based on this discussion, Nevil and I drafted an update of our charter that you can find below. Please have a look at it and send us your comments. Thanks, Juergen IP Flow Information Export (ipfix) Description of Working Group The IPFIX working group has specified the information model (to describe IP flows) and the IPFIX protocol (to transfer IP flow data from IPFIX exporters to collectors). Several implementers have already built applications using the IPFIX protocol. As a result of a series of IPFIX interoperability testing events the WG has produced guidelines for IPFIX implementation and testing as well as recommendations for handling special cases such as bidirectional flow reporting and reducing redundancy in flow records. The IPFIX WG has developed a mediation framework, that defines IPFIX mediators for processing flow records for various purposes including aggregation, anonymization, etc. For configuring IPFIX devices, a YANG module has been developed. 1. Having a solid standardized base for IPFIX deployment and operation and several exiting implementations, the IPFIX WG will revisit the IPFIX protocol specifications (RFC 5101) and the IPFIX information element specification (RFC 5102) in order to advance them to draft standard. 2. For giving guidelines to developers of new IPFIX information elements and for better defining the process of registering new information elements at IANA the IPFIX WG will create an information element developers guideline document. 3. The export of IPFIX flow records from IPFIX mediators introduces a set of potential issues at the protocol level, such as the loss of information on the original exporter, loss of base time information, loss of original options template information, etc. The IPFIX WG will define common ways to deal with these issues, by specifying guidelines for the use of the IPFIX protocol on IPFIX mediators. 4. For supporting the aggregation of flow records at IPFIX mediators the IPFIX WG will define how to export aggregated flow information using IPFIX. An aggregated flow is essentially an IPFIX flow representing packets from multiple original Flows sharing some set of common properties. 5. The IPFIX WG will investigate the use of the IPFIX protocol for exporting MIB objects, avoiding the need to define new IPFIX information elements for existing management information base objects that are already fully specified. This method requires the specification of new template set and options template sets to allow the export of MIB objects along with IPFIX information elements. 6. The IPFIX MIB module (RFC 5815) defined a way to register packet selector functions at IANA. The WG agreed that another method would be preferable that requires a minor change of RFC 5815. The IPFIX WG will produce a new version of RFC 5815 with small modifications of the IANA actions and DESCRIPTION clauses in the the MIB modules.
Even if obvious, I believe that a sentence explaining that all new deliverables should be based on RFC5102bis and RFC5101bis
Oct 2011 Publish draft on guidelines for IE doctors Oct 2011 Publish draft on IPFIX use at mediators Oct 2011 Publish draft on intermediate aggregation Oct 2011 Publish draft on exporting MIB objects Oct 2011 Publish draft on data link IEs
I guess you want to mention the IPFIX MIB module
Dec 2011 Publish draft revising RFC 5101 Dec 2011 Publish draft revising RFC 5102 Apr 2012 Submit guidelines for IE doctors for publication as Informational BCP RFC Apr 2012 Submit draft on IPFIX use at mediators for publication as Standards track RFC Apr 2012 Submit draft on intermediate aggregation for publication as Standards track RFC Apr 2012 Submit draft on data link IEs for publication as Standards track RFC
I guess you want to mention the IPFIX MIB module

Regards, Benoit.
Apr 2012 Submit draft revising RFC 5101 for publication as Standards track RFC Apr 2012 Submit draft revising RFC 5102 for publication as Standards track RFC Sep 2012 Submit draft on exporting MIB objects for publication as Standards track RFC _______________________________________________ IPFIX mailing list IPFIX <at> ietf.org https://www.ietf.org/mailman/listinfo/ipfix 

_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 0 comments |
headers
Benoit Claise | 2 Sep 2011 15:24
Picon
Favicon

Re: [IPFIX] proposal for IPFIX charter update

Hi Juergen,

Next to Dan's editorial comments, I have some more.
See in line.
> Dear all,
>
> At our session in Quebec we discussed candidates
> for new IPFIX work items. Based on this discussion,
> Nevil and I drafted an update of our charter that
> you can find below.
>
> Please have a look at it and send us your comments.
>
> Thanks,
>
>      Juergen
>
>
> IP Flow Information Export (ipfix)
>
>
> Description of Working Group
>
>
> The IPFIX working group has specified the information model (to describe
> IP flows) and the IPFIX protocol (to transfer IP flow data from IPFIX
> exporters to collectors). Several implementers have already built
> applications using the IPFIX protocol. As a result of a series of IPFIX
> interoperability testing events the WG has produced guidelines for IPFIX
> implementation and testing as well as recommendations for handling
> special cases such as bidirectional flow reporting and reducing
> redundancy in flow records.
>
> The IPFIX WG has developed a mediation framework, that defines IPFIX
> mediators for processing flow records for various purposes including
> aggregation, anonymization, etc. For configuring IPFIX devices, a YANG
> module has been developed.
>
> 1. Having a solid standardized base for IPFIX deployment and operation
> and several exiting implementations, the IPFIX WG will revisit the IPFIX
> protocol specifications (RFC 5101) and the IPFIX information element
> specification (RFC 5102) in order to advance them to draft standard.
>
> 2. For giving guidelines to developers of new IPFIX information
> elements and for better defining the process of registering new
> information elements at IANA the IPFIX WG will create an information
> element developers guideline document.
There are actually two different audiences. See section 1.1 
http://tools.ietf.org/html/draft-trammell-ipfix-ie-doctors-02
NEW:

For giving guidelines to developers of new IPFIX information
elements, for helping the IPFIX experts appointed as IE-Doctors, and for better defining the process of
registering new
information elements at IANA, the IPFIX WG will create an information
element developers guideline document.

>
> 3. The export of IPFIX flow records from IPFIX mediators introduces a
> set of potential issues at the protocol level, such as the loss of
> information on the original exporter, loss of base time information,
> loss of original options template information, etc. The IPFIX WG will
> define common ways to deal with these issues, by specifying guidelines
> for the use of the IPFIX protocol on IPFIX mediators.
"common ways", "specifying guidelines for the use of the IPFIX 
protocol": actually it's a real protocol specification, which will be 
standard track.
Should we be more specific?
Note that 
http://tools.ietf.org/html/draft-claise-ipfix-mediation-protocol-04 
title is: Specification of the Protocol for IPFIX Mediations

For example:

3. The export of IPFIX flow records from IPFIX mediators introduces a
set of potential issues at the protocol level, such as the loss of
information on the original exporter, loss of base time information,
loss of original options template information, etc. The IPFIX WG will
produce the protocol specifications in order to solve these IPFIX mediation
specific problems.

>
> 4. For supporting the aggregation of flow records at IPFIX mediators
> the IPFIX WG will define how to export aggregated flow information using
s/define/specify
> IPFIX. An aggregated flow is essentially an IPFIX flow representing
> packets from multiple original Flows sharing some set of common properties.
s/Flow/flow

Regards, Benoit.
>
> 5. The IPFIX WG will investigate the use of the IPFIX protocol for
> exporting
> MIB objects, avoiding the need to define new IPFIX information elements
> for existing management information base objects that are already fully
> specified. This method requires the specification of new template set
> and options template sets to allow the export of MIB objects along
> with IPFIX information elements.
>
> 6. The IPFIX MIB module (RFC 5815) defined a way to register packet
> selector functions at IANA. The WG agreed that another method would
> be preferable that requires a minor change of RFC 5815. The IPFIX WG
> will produce a new version of RFC 5815 with small modifications of
> the IANA actions and DESCRIPTION clauses in the the MIB modules.
>
> Oct 2011    Publish draft on guidelines for IE doctors
> Oct 2011    Publish draft on IPFIX use at mediators
> Oct 2011    Publish draft on intermediate aggregation
> Oct 2011    Publish draft on exporting MIB objects
> Oct 2011    Publish draft on data link IEs
> Dec 2011    Publish draft revising RFC 5101
> Dec 2011    Publish draft revising RFC 5102
>
> Apr 2012    Submit guidelines for IE doctors for publication as
> Informational BCP RFC
> Apr 2012    Submit draft on IPFIX use at mediators for publication as
> Standards track RFC
> Apr 2012    Submit draft on intermediate aggregation for publication as
> Standards track RFC
> Apr 2012    Submit draft on data link IEs for publication as Standards
> track RFC
> Apr 2012    Submit draft revising RFC 5101 for publication as Standards
> track RFC
> Apr 2012    Submit draft revising RFC 5102 for publication as Standards
> track RFC
> Sep 2012    Submit draft on exporting MIB objects for publication as
> Standards track RFC
>
>
> _______________________________________________
> IPFIX mailing list
> IPFIX <at> ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 0 comments |
headers
Benoit Claise | 2 Sep 2011 17:18
Picon
Favicon

Re: [IPFIX] template lifetime

Brian,
> Hi, Paul,
>
> My personal opinion is that, if we're not going to deprecate UDP with prejudice (which I realize to my
sorrow we cannot do), we should jettison all this template lifetime nonsense. We spent an inordinate
amount of time on 5101/5102 the first time around trying to come up with sensible guidance, then gave up and
basically said "template lifetimes are configured out of band according to operational environment and
application constraints", which is essentially RFCese for "here, you make this work." Which is actually
sensible: there is no default that covers edge-network, core-network, and
IPFIX-as-SNMP-replacement-on-the-moon use cases.
>
> Half a decade later, there has never been a successful interop test to my knowledge demonstrating the
template management (expiry, withdrawal, reissue) features in IPFIX.
Sure. It's only when you exhausted the number of Template ID than you 
need "reiusse", when then need "withdrawal". Note: I know of such use 
case/implementation.
"Expiry" is a different problem.

Regards, Benoit.
>
> Most export applications use one, two, twenty, thirty templates, and that's all they'll ever use,
because those templates reflect the core data model in use by the application, and the export app wants to
waste as little effort in export transcoding as possible. Sure, collectors in the wild have to handle more
templates than that, but it doesn't take long as a collector operator (or even implementor) to realize
that practically speaking, nobody is ever going to reuse an ID on you without dropping a session, and just
setting your template expiry time to something practically infinite . You interpret templates as they
come in, you replace old with new, and everything works unless you find yourself in an insanely rare corner
case -- and if you do, you write something to the log and wake up 
 the human.
>
> Trying to export template lifetime information from EP to CP adds complexity while pretending that the
system as specified works better than an infinite-lifetime-at-collector with silent replacement,
which IMO is not the case.
>
> I'd propose we use the RFC5655 File Reader template rules (accept anything that would be legal on any
transport in any combination), but I don't know the right way to do this in 5101bis: perhaps filing
technical errata on all the "MUST drop the session" language on the grounds that 1. a receiver cannot
always reliably signal session failure to a transmitter and 2. it violates the "be liberal in what you
accept" principle of protocol design.
>
> Cheers,
>
> Brian
>
> On Aug 18, 2011, at 11:34 AM, Paul Aitken wrote:
>
>> Dear IPFIX experts,
>>
>>
>> RFC5102, section 10.3.7 ("Collecting Process") says:
>>
>>    The Collecting Process MUST associate a lifetime with each Template
>>    (or another definition of an identifier considered unique within the
>>    Transport Session) received via UDP.  Templates (and similar
>>    definitions) not refreshed by the Exporting Process within the
>>    lifetime are expired at the Collecting Process.
>>
>>    ...
>>
>>    The Template lifetime at the Collecting Process MUST be at least 3
>>    times higher than the Template refresh timeout configured on the
>>    Exporting Process.
>>
>>
>>
>> How does a collector determine the correct lifetime to associate with each Template, and how does it know
"the Template refresh timeout configured on the Exporting Process." ?
>>
>> What are the default and acceptable range of lifetime values?
>>
>> Should we have a mechanism which allows an Exporting Process to report Template lifetimes to the
Collecting Process?
>> eg, by exporting an option of { scope = templateId, lifetime = lifeTimeUnits }, where lifeTimeUnits = u32
milliseconds - which allows 1ms to 49.7 days, with 0 = infinite?
>>
>> Thanks,
>> P.
>> _______________________________________________
>> IPFIX mailing list
>> IPFIX <at> ietf.org
>> https://www.ietf.org/mailman/listinfo/ipfix
> _______________________________________________
> IPFIX mailing list
> IPFIX <at> ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 0 comments |
headers
Benoit Claise | 2 Sep 2011 17:19
Picon
Favicon

Re: [IPFIX] template lifetime

Paul,
> Brian, Gerhard,
>
> Thanks for your replies. I know it's an old problem, but someone asked 
> what the default refresh interval should be.
>
> So it seems we should either define a mechanism for Exporters to 
> inform Collectors what their default refresh interval is (eg, define a 
> new standard options template), or re-work (or even remove?) the 
> template timeout sections.
>
> Currently this is a gap which allows an Exporting Process and a 
> Collecting Process to both claim to be IPFIX compliant, while being 
> quite un-interoperable.
Can you please clarify the un-interoperable aspect. I don't get it.

Regards, Benoit.
>
> P.
>
> _______________________________________________
> IPFIX mailing list
> IPFIX <at> ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 1 comment |
headers
Benoit Claise | 2 Sep 2011 17:15
Picon
Favicon

Re: [IPFIX] template lifetime

On 18/08/2011 11:34, Paul Aitken wrote:
> Dear IPFIX experts,
>
>
> RFC5102, section 10.3.7 ("Collecting Process") says:
>
>    The Collecting Process MUST associate a lifetime with each Template
>    (or another definition of an identifier considered unique within the
>    Transport Session) received via UDP.  Templates (and similar
>    definitions) not refreshed by the Exporting Process within the
>    lifetime are expired at the Collecting Process.
>
>    ...
>
>    The Template lifetime at the Collecting Process MUST be at least 3
>    times higher than the Template refresh timeout configured on the
>    Exporting Process.
>
>
>
> How does a collector determine the correct lifetime to associate with 
> each Template, and how does it know "the Template refresh timeout 
> configured on the Exporting Process." ?
Solution 1. Out of band configuration with 
http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10. 
Note: I still envision that IPFIX caches might be configured for a short 
period of time on IPFIX exporters, for troubleshooting reasons.
Solution 2. CP observes 10 (or 20, or whatever number) template refresh 
and conclude the right value
Solution 3: the CP doesn't bother and take a very high value
>
> What are the default and acceptable range of lifetime values?
>
> Should we have a mechanism which allows an Exporting Process to report 
> Template lifetimes to the Collecting Process?
> eg, by exporting an option of { scope = templateId, lifetime = 
> lifeTimeUnits }, where lifeTimeUnits = u32 milliseconds - which allows 
> 1ms to 49.7 days, with 0 = infinite?
This is solution 4.

Regards, Benoit.
>
> Thanks,
> P.
> _______________________________________________
> IPFIX mailing list
> IPFIX <at> ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 5 comments |
headers
Xenofontas Dimitropoulos | 2 Sep 2011 14:32
Picon
Picon

[IPFIX] Traffic Monitoring and Analysis (TMA) 2012 CFP

[Our apologies if you receive multiple copies of this CFP]

--------------------------------
TMA 2012 --- Call for Papers
--------------------------------

4th COST TMA International Workshop on Traffic Monitoring and Analysis
Vienna, Austria, March 12, 2012
Co-located with Passive and Active Measurement (PAM) Conference 2012
http://tma2012.ftw.at/TMA/index.html

After three successful editions of the TMA Workshop organized by the 
Traffic Monitoring and Analysis (TMA) COST Action, we are excited to 
announce the 4th edition that will be held in Vienna (Austria), 
co-located with the Passive and Active Measurement Conference.

**** Scope of the Workshop

We are soliciting network-measurement papers with an emphasis on 
experimental work that relate to the following topics:

Traffic analysis, characterization, and classification
New and improved measurement and monitoring methods and techniques
Measurement-based applications
Measurement-based network management
Large scale monitoring and measurement
Validation and benchmarking of measurement tools
Measurement-based experiments: repeatability tests, data sharing, 
collaborative platforms
Internet topologies
Bandwidth estimation
Software tools for analyzing and visualizing high-volume 
Internet-related datasets
Distributed monitoring architectures
Privacy-preserving network measurement and analysis
Inter-domain and intra-domain routing
Anomaly detection
Measurement-related to data centers and cloud-based services
Measurement-related to Virtualized systems
Home networks measurements

**** Paper Submission
Submissions must not exceed 14 pages (8.5”x11”), including figures, 
tables, and references, using the LNCS format. We recommend using the 
LNCS users' guide to format your paper. Papers that exceed the 14 pages 
limitation will not be reviewed. Accepted papers will be published in a 
volume of the Lecture Notes in Computer Science (LNCS) series by 
Springer Verlag.

**** Important Dates
Abstract Registration: September 25, 2011
Paper Submission: October 9, 2011
Notification of Decision: December 20, 2011
Camera Ready: January 8, 2012
Workshop: March 12, 2012

**** Program Chairs
Antonio Pescapè University of Napoli, Federico II
Xenofontas Dimitropoulos ETH Zurich
Luca Salgarelli University of Brescia

**** Local Arrangements Chair
Philipp Svoboda TU Wien

**** Program Committee
Olivier Bonaventure, Université Catholique de Louvain
Ernst Biersack, Eurecom
Pere Barlet-Ros, Technical University of Catalunya
Kenjiro Cho, IIJ
Konstantina Papagiannaki, Telefonica
Alberto Dainotti, University of Napoli Federico II
Nick Feamster, Georgia Tech
KC Claffy, CAIDA
Pietro Michiardi, Eurecom
Brian Trammell, ETH Zurich
Georgios Lioudakis, National Technical University of Athens
Felipe Huici, NEC Labs Europe
Joseph Gasparakis, Intel
Maurizio Dusi, NEC labs Europe
Grenville Armitage, CAIA
Wenke Lee, Georgia Tech
Maria Papadopouli, FORTH
Gianluca Iannaccone, Intel Labs Berkeley
Marco Mellia, Politecnico di Torino
Steve Uhlig, TU Berlin / T-Labs
Yuval Shavitt, Tel Aviv University
Dario Rossi, TELECOM ParisTech
Fabio Ricciato, FTW and University of Salento
Francesco Gringoli, University of Brescia
Philippe Owezarski, CNRS
Alessio Botta, University of Napoli Federico II
Aiko Pras, University of Twente
Michela Meo, Politecnico di Torino
Louis Plissonneau, Orange Labs
Pierre Borgnat, ENS Lyon
Hyun-chul Kim, Seoul National University
Georgios Smaragdakis, Deutsche Telekom Labs and TU Berlin
Andreas Kind, IBM Research
Marcin Pietrzyk, Swisscom

--

-- 
ETH Zurich
Dr. Xenofontas Dimitropoulos
Lecturer&  Senior Researcher
Computer Engineering and Networks Laboratory (TIK)
ETZ G95, Gloriastrasse 35, CH-8092 Zurich.
http://www.fontas.net
+41 44 632 7004 (phone)

_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 0 comments |
headers
Paul Aitken | 5 Sep 2011 00:10
Picon
Favicon
Gravatar

Re: [IPFIX] template lifetime

Benoit,

> Paul,
>> Brian, Gerhard,
>>
>> Thanks for your replies. I know it's an old problem, but someone 
>> asked what the default refresh interval should be.
>>
>> So it seems we should either define a mechanism for Exporters to 
>> inform Collectors what their default refresh interval is (eg, define 
>> a new standard options template), or re-work (or even remove?) the 
>> template timeout sections.
>>
>> Currently this is a gap which allows an Exporting Process and a 
>> Collecting Process to both claim to be IPFIX compliant, while being 
>> quite un-interoperable.
> Can you please clarify the un-interoperable aspect. I don't get it.

An Exporting Process has a template refresh time in order to satisfy RFC 
5101 / 10.3.6 :

10.3.6.  Template Management

    When IPFIX uses UDP as the transport protocol, Template Sets and
    Option Template Sets MUST be re-sent at regular intervals.  The
    frequency of the (Options) Template transmission MUST be
    configurable.  The default value for the frequency of the (Options)
    Template transmission is 10 minutes.

Meanwhile the Collecting Process follows RFC 5101 / 10.3.7 :

10.3.7.  Collecting Process

    The Collecting Process MUST associate a lifetime with each Template
    (or another definition of an identifier considered unique within the
    Transport Session) received via UDP.  Templates (and similar
    definitions) not refreshed by the Exporting Process within the
    lifetime are expired at the Collecting Process.  If the Template (or
    other definition) is not refreshed before that lifetime has expired,
    the Collecting Process MUST discard that definition and any current
    and future associated Data Records.

Since the CP doesn't know what the EP's template refresh time is, 
perhaps it assumes to follow the "10 minutes" advice in 10.3.6
- and for good measure, it allows three times as much - so, it 
associates a 30 minute lifetime with each template.

However, the EP actually refreshes templates hourly. Or every 90 
minutes. Or daily. Whatever you like, > 30 mins (since, "The frequency 
of the (Options) Template transmission MUST be configurable.").

So both are quite RFC5101 compliant, but fail to interoperate properly - 
because for some period, the CP will have discarded the template and 
won't be able to decode further data until the

P.

_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 0 comments |
headers
Paul Aitken | 5 Sep 2011 00:10
Picon
Favicon
Gravatar

Re: [IPFIX] template lifetime

Benoit,

>> How does a collector determine the correct lifetime to associate with 
>> each Template, and how does it know "the Template refresh timeout 
>> configured on the Exporting Process." ?
> Solution 1. Out of band configuration with 
> http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10. 
> Note: I still envision that IPFIX caches might be configured for a 
> short period of time on IPFIX exporters, for troubleshooting reasons.

This is not mentioned anywhere in IPFIX. It's a hole.

> Solution 2. CP observes 10 (or 20, or whatever number) template 
> refresh and conclude the right value

Packet loss and network jitter make this problematic.

> Solution 3: the CP doesn't bother and take a very high value

That seems reasonable. Why not specify this as the default behaviour?

>>
>> What are the default and acceptable range of lifetime values?
>>
>> Should we have a mechanism which allows an Exporting Process to 
>> report Template lifetimes to the Collecting Process?
>> eg, by exporting an option of { scope = templateId, lifetime = 
>> lifeTimeUnits }, where lifeTimeUnits = u32 milliseconds - which 
>> allows 1ms to 49.7 days, with 0 = infinite?
> This is solution 4.

Then it needs to be written in the protocol so EPs export it and CPs 
expect and action it correctly.

P.
_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 4 comments |
headers
Brian Trammell | 5 Sep 2011 08:34
Picon
Picon

Re: [IPFIX] template lifetime

hi Paul, all, 

Comments on the running discussion inline...

Cheers,

Brian

On Sep 5, 2011, at 12:10 AM, Paul Aitken wrote:

> Benoit,
> 
>>> How does a collector determine the correct lifetime to associate with each Template, and how does it
know "the Template refresh timeout configured on the Exporting Process." ?
>> Solution 1. Out of band configuration with
http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10. Note: I still envision
that IPFIX caches might be configured for a short period of time on IPFIX exporters, for troubleshooting reasons.
> 
> This is not mentioned anywhere in IPFIX. It's a hole.

Is the intention to plug this in 5101bis? The questionable (or at least, not yet demonstrated)
interoperability of the existing spec may give us an opening to do so...

>> Solution 2. CP observes 10 (or 20, or whatever number) template refresh and conclude the right value
> 
> Packet loss and network jitter make this problematic.

Indeed, but assuming a network that is working _well enough_ that the human in charge doesn't reprovision
it due to unacceptable export efficiency, the long term average of template refresh times should
approximate an upper bound reasonably well... 

> 
>> Solution 3: the CP doesn't bother and take a very high value
> 
> That seems reasonable. Why not specify this as the default behaviour?

Infinity seems high enough. :) Coupled with automatic replacement on receipt of a different template with
the same ID (keyed by template validity tied to the export time in the message in which it was received), it
would seem to solve the synchronization problem.

>>> What are the default and acceptable range of lifetime values?
>>> 
>>> Should we have a mechanism which allows an Exporting Process to report Template lifetimes to the
Collecting Process?
>>> eg, by exporting an option of { scope = templateId, lifetime = lifeTimeUnits }, where lifeTimeUnits =
u32 milliseconds - which allows 1ms to 49.7 days, with 0 = infinite?
>> This is solution 4.
> 
> Then it needs to be written in the protocol so EPs export it and CPs expect and action it correctly.

If we go with option 4 (unsurprisingly, I'm not a fan), then I would suggest _explicitly_ tying these
timeouts to the export time in the message header, not any-old-random-clock the CP happens to have
hanging around.

_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 2 comments |
headers
Paul Aitken | 5 Sep 2011 10:53
Picon
Favicon
Gravatar

Re: [IPFIX] template lifetime

Brian,

>>>> How does a collector determine the correct lifetime to associate with each Template, and how does it
know "the Template refresh timeout configured on the Exporting Process." ?
>>> Solution 1. Out of band configuration with
http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10. Note: I still envision
that IPFIX caches might be configured for a short period of time on IPFIX exporters, for troubleshooting reasons.
>> This is not mentioned anywhere in IPFIX. It's a hole.
> Is the intention to plug this in 5101bis? The questionable (or at least, not yet demonstrated)
interoperability of the existing spec may give us an opening to do so...

That's rather the point of this thread.

>>> Solution 2. CP observes 10 (or 20, or whatever number) template refresh and conclude the right value
>> Packet loss and network jitter make this problematic.
> Indeed, but assuming a network that is working _well enough_ that the human in charge doesn't reprovision
it due to unacceptable export efficiency, the long term average of template refresh times should
approximate an upper bound reasonably well...

There may not be any such thing as a "long term" average, since 
templates may come and go at unpredictable times. It may be especially 
difficult to make any predication about the lifetime of short-lived 
templates or frequently redefined templates.

Also, when a template is redefined (ie, same template ID, but different 
fields or different field order), the lifetime calculation has to be 
re-started. Clearly a full discussion is missing and is needed if this 
method is expected to be used.

>>> Solution 3: the CP doesn't bother and take a very high value
>> That seems reasonable. Why not specify this as the default behaviour?
> Infinity seems high enough. :) Coupled with automatic replacement on receipt of a different template
with the same ID (keyed by template validity tied to the export time in the message in which it was
received), it would seem to solve the synchronization problem.

This could work. However, we have to recognise the disadvantage of 
possibly incorrectly decoding some data records where a template 
redefinition has been missed. A finite lifetime would ensure the records 
had to be discarded.

>>>> What are the default and acceptable range of lifetime values?
>>>>
>>>> Should we have a mechanism which allows an Exporting Process to report Template lifetimes to the
Collecting Process?
>>>> eg, by exporting an option of { scope = templateId, lifetime = lifeTimeUnits }, where lifeTimeUnits =
u32 milliseconds - which allows 1ms to 49.7 days, with 0 = infinite?
>>> This is solution 4.
>> Then it needs to be written in the protocol so EPs export it and CPs expect and action it correctly.
> If we go with option 4 (unsurprisingly, I'm not a fan), then I would suggest _explicitly_ tying these
timeouts to the export time in the message header, not any-old-random-clock the CP happens to have
hanging around.

Clearly there are several ways of solving the problem. I don't really 
mind which method is used, as long as it fills what seems to be a hole 
in IPFIX.

Thanks,
P.
_______________________________________________
IPFIX mailing list
IPFIX <at> ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
Permalink | Reply | 1 comment |

Gmane