Benchmarking Methodology (bmwg)

headers Silvija Andrijic Dry (sdry | 10 Jul 2006 15:11

_______________________________________________
bmwg mailing list
bmwg <at> ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg

The IESG has approved the following document:

- 'Terminology for Benchmarking Network-layer Traffic Control Mechanisms '
   <draft-ietf-bmwg-dsmterm-13.txt> as an Informational RFC

This document is the product of the Benchmarking Methodology Working Group. 

The IESG contact persons are David Kessens and Dan Romascanu.

A URL of this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-dsmterm-13.txt

Technical Summary

 This document describes terminology for the benchmarking of                
 devices that implement traffic control based on IP precedence or           
 Diffserv code point criteria.  The terminology is to be applied            
 to measurements made on the data plane to evaluate IP traffic              
 control mechanisms.                                                        

 New terminology is needed because most existing measurements               
 assume the absence of congestion and only a single per-hop-                
 behavior.  This document introduces several new terms that will            
 allow measurements to be taken during periods of congestion.               

 Another key difference from existing terminology is the definition         
 of measurements as observed on egress as well as ingress of a              
 device/system under test.  Again, the existence of congestion              
 requires the addition of egress measurements as well as those              
 taken on ingress; without observing traffic leaving a                      
 device/system it is not possible to say whether traffic-control            
 mechanisms effectively dealt with congestion. 

 The principal measurements introduced in this document are vectors         
 for rate, delay, and jitter, all of which can be observed with or          
 without congestion of the DUT/SUT. 

Working Group Summary

 This document is a product of the BMWG WG.  The WG has consensus              
 to publish this document as an Informational RFC.

Protocol Quality

 This document was reviewed by David Kessens for the IESG.

_______________________________________________
IETF-Announce mailing list
IETF-Announce <at> ietf.org
https://www1.ietf.org/mailman/listinfo/ietf-announce

For folks who intend to participate remotely:

Please let me (not the whole list) know that you intend
to participate from a non-Montreal location.

The slides for all presentations are available now,
and the audio, jabber, and other info is available
on the IETF site under the current meeting page.

http://www3.ietf.org/meetings/text_conf.html

https://datatracker.ietf.org/public/meeting_materials.cgi?meeting_num=66

http://videolab.uoregon.edu/events/ietf/
for audio, we're in room 519 A

Al

BMWG:

A WG Last Call period for the Internet-Draft on

"Hash and Stuffing: Overlooked Factors in Network Device Benchmarking"

will be open from 17 July 2006 through 14 August 2006.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-hash-stuffing-05.txt

A WG Last Call (ending 3 Feb 2006) was issued on version 04
of this draft.  Version 05 addresses comments from
the previous Last Call, the cross-area review, and the ensuing discussion.
In particular, attention should be paid to material in the new
Appendices (B and C).

Please express your opinion on whether or not this Internet-Draft
should be given to the Area Directors for consideration in
progressing the Draft to an Informational RFC.  Send your comments
to this list or acmorton <at> att.com.

BMWG:

A WG Last Call period for the Internet-Drafts on

Terminology for Accelerated Stress Benchmarking
Methodology Guidelines for Accelerated Stress Benchmarking

will be open from 17 July 2006 through 28 August 2006 (6 weeks).

URLs for these Internet-Drafts are:
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-acc-bench-term-09.txt
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-acc-bench-meth-05.txt

These versions incorporate comments resulting from implementation.
The authors believe that all comments from the working group
have been addressed, dating back to June 2003 when version 00
of the terminology draft was published.

These drafts are entering the BMWG Last Call Process. See
http://www1.ietf.org/mail-archive/web/bmwg/current/msg00846.html
At this time, we are seeking volunteers to complete the
BMWG Last Call Review Template in the message of 4/27/04
titled, "Refinements to the BMWG Last Call Process".
Volunteers should contact acmorton <at> att.com
A copy of the template is available here:
http://home.comcast.net/~acmacm/IDcheck/LastCallTemplate.txt

Whether or not you decide to volunteer for the directed review,
please *read* and express your opinion on whether or not this
Internet-Draft should be given to the Area Directors for
consideration as an Informational RFC.  Send your comments
to this list or acmorton <at> att.com 

Hi BMWG-ers,

When reviewing these documents, please keep in mind an excellent suggestion
from Scott Bradner that he made following the BMWG meeting in Montreal last
week.  The Instability Conditions as currently explained in the Methodology
are applied simultaneously.  While this is useful because it increases the
probability of measuring degraded performance, it is difficult to identify
which instability condition, or combination of instability conditions,
actually caused the degraded performance.  Scott's suggestion was therefore
to apply instability conditions in serial fashion to facilitate isolation of
performance degradation to a specific instability condition.  The authors
intend in the next revision to add a test case for this.

Scott

-----Original Message-----
From: Al Morton [mailto:acmorton <at> att.com]
Sent: Monday, July 17, 2006 9:31 AM
To: bmwg <at> ietf.org
Subject: [bmwg] WG Last Call on Accelerated Stress Benchmarking Drafts

BMWG:

A WG Last Call period for the Internet-Drafts on

Terminology for Accelerated Stress Benchmarking
Methodology Guidelines for Accelerated Stress Benchmarking

will be open from 17 July 2006 through 28 August 2006 (6 weeks).

URLs for these Internet-Drafts are:
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-acc-bench-term-09.txt
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-acc-bench-meth-05.txt

These versions incorporate comments resulting from implementation.
The authors believe that all comments from the working group
have been addressed, dating back to June 2003 when version 00
of the terminology draft was published.

These drafts are entering the BMWG Last Call Process. See
http://www1.ietf.org/mail-archive/web/bmwg/current/msg00846.html
At this time, we are seeking volunteers to complete the
BMWG Last Call Review Template in the message of 4/27/04
titled, "Refinements to the BMWG Last Call Process".
Volunteers should contact acmorton <at> att.com
A copy of the template is available here:
http://home.comcast.net/~acmacm/IDcheck/LastCallTemplate.txt

Whether or not you decide to volunteer for the directed review,
please *read* and express your opinion on whether or not this
Internet-Draft should be given to the Area Directors for
consideration as an Informational RFC.  Send your comments
to this list or acmorton <at> att.com 

_______________________________________________
bmwg mailing list
bmwg <at> ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg

BMWG:

Thanks to a truly expert contribution by our note-taker
Tim Alexander, we already have minutes from our session
posted on the meeting materials site:
https://datatracker.ietf.org/public/meeting_materials.cgi?meeting_num=66

Comments or requests for clarification are welcome now through
August 11, 2006.  In particular:

- if you think some critical detail was left out of the summary,
   please indicate it.

- if you volunteered to review a document (and your name isn't listed),
   let me know.

Also note that our new Supplementary web page is available now:
http://home.comcast.net/~acmacm/BMWG/
and any suggestions on this material are welcome.

Al
bmwg co-chair

Great job Tom!

-----Original Message-----
From: Al Morton [mailto:acmorton <at> att.com]
Sent: Monday, July 17, 2006 5:10 PM
To: bmwg <at> ietf.org
Subject: [bmwg] Draft Minutes from BMWG  <at>  IETF-66

BMWG:

Thanks to a truly expert contribution by our note-taker
Tim Alexander, we already have minutes from our session
posted on the meeting materials site:
https://datatracker.ietf.org/public/meeting_materials.cgi?meeting_num=66

Comments or requests for clarification are welcome now through
August 11, 2006.  In particular:

- if you think some critical detail was left out of the summary,
   please indicate it.

- if you volunteered to review a document (and your name isn't listed),
   let me know.

Also note that our new Supplementary web page is available now:
http://home.comcast.net/~acmacm/BMWG/
and any suggestions on this material are welcome.

Al
bmwg co-chair

_______________________________________________
bmwg mailing list
bmwg <at> ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg

I am cross-posting the bmwg and sipping wg lists on purpose.   

There seem to be quite a lot of commonality between the following
Internet-Drafts:

-
http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-03.t
xt
-
http://www.ietf.org/internet-drafts/draft-poretsky-sip-bench-term-01.txt

It would be nice if the authors of the two drafts would work together to
make sure that SIP performance metrics are being defined in one place
(in the IETF at least) and the same definitions are being used for SIP
performance measurement and benchmarking.

Regards,

Dan

This communication and work is taking place to make sure the efforts are
complimentary (and relative to the specific working groups) and do not
overlap.  The differences were discussed in Montreal, but to recap...the
work in Sipping is end-to-end and the work in BMWG is single DUT.

Thanks..--Daryl

On Wed, 2006-07-19 at 01:01 +0300, Romascanu, Dan (Dan) wrote:
> I am cross-posting the bmwg and sipping wg lists on purpose.   
> 
> There seem to be quite a lot of commonality between the following
> Internet-Drafts:
> 
> -
> http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-03.t
> xt
> -
> http://www.ietf.org/internet-drafts/draft-poretsky-sip-bench-term-01.txt
>  
> It would be nice if the authors of the two drafts would work together to
> make sure that SIP performance metrics are being defined in one place
> (in the IETF at least) and the same definitions are being used for SIP
> performance measurement and benchmarking.
> 
> Regards,
> 
> Dan
> 
> 
> _______________________________________________
> Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> This list is for NEW development of the application of SIP
> Use sip-implementors <at> cs.columbia.edu for questions on current sip
> Use sip <at> ietf.org for new developments of core SIP

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use sip-implementors <at> cs.columbia.edu for questions on current sip
Use sip <at> ietf.org for new developments of core SIP

Mon	Tue	Wed	Thu	Fri	Sat	Sun

					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30
31

6	May 2013
8	April 2013
20	March 2013
14	February 2013
15	January 2013
9	December 2012
17	November 2012
9	October 2012
4	September 2012
13	August 2012
12	July 2012
3	June 2012
3	May 2012
15	April 2012
19	March 2012
5	February 2012
8	January 2012
5	December 2011
23	November 2011
30	October 2011
27	September 2011
33	August 2011
24	July 2011
29	June 2011
5	May 2011
8	April 2011
21	March 2011
8	February 2011
4	January 2011
13	December 2010
36	November 2010
16	October 2010
1	September 2010
17	August 2010
47	July 2010
7	June 2010
21	May 2010
20	April 2010
14	March 2010
10	February 2010
8	January 2010
8	December 2009
5	November 2009
33	October 2009
9	September 2009
16	August 2009
37	July 2009
1	June 2009
5	May 2009
13	April 2009
27	March 2009
16	February 2009
9	January 2009
12	December 2008
21	November 2008
23	October 2008
6	September 2008
1	August 2008
22	July 2008
6	June 2008
12	May 2008
47	April 2008
66	March 2008
24	February 2008
3	January 2008
27	December 2007
21	November 2007
5	October 2007
12	September 2007
31	August 2007
61	July 2007
15	June 2007
7	May 2007
12	April 2007
77	March 2007
41	February 2007
6	January 2007
4	December 2006
31	November 2006
38	October 2006
6	September 2006
14	August 2006
31	July 2006
47	June 2006
43	May 2006
19	April 2006
18	March 2006
12	February 2006
10	January 2006
5	November 2005
26	October 2005
3	September 2005
1	August 2005
8	July 2005
22	June 2005
8	May 2005
6	April 2005
6	March 2005
12	February 2005
3	January 2005
1	December 2004
10	November 2004
51	October 2004
1	September 2004
10	August 2004
21	July 2004
11	June 2004
6	May 2004
44	April 2004
4	March 2004
14	February 2004
16	January 2004
2	December 2003
55	November 2003
36	October 2003
5	September 2003
6	August 2003
31	July 2003
18	June 2003
73	May 2003
37	April 2003
75	March 2003
64	February 2003
33	January 2003
10	December 2002
9	November 2002
13	October 2002
9	September 2002
23	August 2002
10	July 2002
7	June 2002
2	May 2002
9	April 2002

Comments for draft-akhter-bmwg-mpls-meth-00.txt

Document Action: 'Terminology for Benchmarking Network-layer Traffic Control Mechanisms' to Informational RFC

Remote participants for the BMWG session

2nd WG Last Call on Address Hash and Bit Stuffing Draft (05)

WG Last Call on Accelerated Stress Benchmarking Drafts

RE: WG Last Call on Accelerated Stress Benchmarking Drafts

Draft Minutes from BMWG <at> IETF-66

RE: Draft Minutes from BMWG <at> IETF-66

SIP Performance Metrics

Re: SIP Performance Metrics