Bob (Robert) McMahon | 23 Nov 00:42 2014

iperf2.0.8 (beta) info & test help request

Hi,

 

We’ve done some updates to the old iperf code base and wanted to inform folks (as well as ask for some testing help.)   This code is in a separate repository from iperf and is a separate effort from iperf3.  We are trying to maintain iperf2 for compatibility and interoperability reasons.  

 

https://sourceforge.net/projects/iperf2/

 

The updates as compared to iperf 2.0.5:

 

o Improved performance (decrease mutex contention)

o Support smaller report intervals (5 ms or greater)

o Support SO_RCVTIMEOUT for server reports regardless of no packets

o Support SO_TIMESTAMP for kernel level packet timestamping

o Support end/end latency in mean/min/max/stdev format (UDP)

      (assumes client and server clocks synched, e.g by Precision Time Protocol)

o Add local port to bind support (-B option) using colon as separator (client)

     (e.g. iperf -c 192.168.100.100 -B 192.168.100.10:6001)

o Fix portability, compile and tested with Linux (Fedora, debian, opensuse) Win10, Win7, WinXP and MacOS

o Support TCP rate limited streams (via the -b) using token bucket

o Support packets per second (UDP) via pps as units, (e.g. -b 1000pps)

o Display PPS in both client and server reports (UDP)

o Support for realtime scheduler as a command line option (--realtime or -z)

o Improve client tx code path so actual tx offerred rate will converge to the -b requested value

o Improve accuracy of microsecond delay calls (in platform independent manner)

    (Use of Kalman filter to predict delay errors and adjust delays per predicted error)

o Display target loop time in initial client header (UDP)

o Suppress unrealistic latency output (-/-/-/-)

o Support SO_SNDTIMEO on send so socket write won't block beyond -t (TCP)

o Use clock_gettime if available (preferred over gettimeofday())

 

Please feel free to email me with comments, issues or suggestions.

 

Thanks,

Bob McMahon

Senior SW Test Engineer (Broadcom)

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@...
https://lists.sourceforge.net/lists/listinfo/iperf-users
Picon

Understanding Iperf

I have logging devices which need to use iperf to test to their home base server.

I need a good layman understanding of Iperf testing. I don't want a highly technical breakdown of Iperf, but
something which is easily understood by practically anyone.

I ask this because searching the net, there seems to be lots of confusion about different results from
client and server side. There are also mentions of being able to do a two way test but I have yet to find a way of
doing that. Would a two way test give me the results of both the client and the server? Would that give me a
better understanding of uploads and downloads speeds?

The server side is
iperf-2.0.5-11

The client side for testing is openwrt running
iperf version 2.0.5 (08 Jul 2010) single threaded

I'm at a loss... help! 

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@...
https://lists.sourceforge.net/lists/listinfo/iperf-users

Picon

Re: protecting iperf behind public server

I forgot to mention that I don't know the IP's of the remotes, they will change all the time. 
However, they do use curl to authenticate to php so have been looking for a way to cobble something together.

I could have php quickly add the IP of the remote to iptables, reload once it is authenticated but since there
would be many remotes connecting at any given time, that seems like an accident waiting to happen.

I also wondered if there is a way that php might be able to control the iperf port or service but that seems to be
a dead end.

I then wondered if there might be a fork of iperf which might allow me to do this but I've not found anything yet.

----- Original Message -----
From: "Sandro Bureca" <sbureca@...>
To: mike@...
Sent: Sunday, November 16, 2014 6:21:29 AM
Subject: Re: [Iperf-users] protecting iperf behind public server

You can use iptables on the machine acting as iper sever if this solution is 
applicable and apply filter on source ip address and the standarf iperf port 5001 or any other at your choice 

On 16 November 2014 00:39, mike@... <
mike@... > wrote: 

I have a number of logging devices which connect home over the internet. Each device calls home using curl,
authenticating part htpasswd then onto a php app. 

I wanted to add iperf as a test to know where there are bandwidth issues but I need to protect iperf to allow
only my own devices to use the server and iperf running as a daemon/service. 

Was wondering if anyone in this list might have to thoughts on how this could be done, so that I could retain my
htpasswd/php authentication and perhaps even use that. 

Might there be a way of having php allow the iperf connection perhaps? 

Thanks kindly. 

------------------------------------------------------------------------------ 
Comprehensive Server Monitoring with Site24x7. 
Monitor 10 servers for $9/Month. 
Get alerted through email, SMS, voice calls or mobile push notifications. 
Take corrective actions from your mobile device. 
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk 
_______________________________________________ 
Iperf-users mailing list 
Iperf-users@... 
https://lists.sourceforge.net/lists/listinfo/iperf-users 

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@...
https://lists.sourceforge.net/lists/listinfo/iperf-users

Picon

protecting iperf behind public server

I have a number of logging devices which connect home over the internet. Each device calls home using curl,
authenticating part htpasswd then onto a php app.

I wanted to add iperf as a test to know where there are bandwidth issues but I need to protect iperf to allow
only my own devices to use the server and iperf running as a daemon/service.

Was wondering if anyone in this list might have to thoughts on how this could be done, so that I could retain my
htpasswd/php authentication and perhaps even use that.

Might there be a way of having php allow the iperf connection perhaps?

Thanks kindly.

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@...
https://lists.sourceforge.net/lists/listinfo/iperf-users

Bob (Robert) McMahon | 1 Nov 23:10 2014

iperf 2.0.8 realease candidate

Hi,

 

I’m working on a iperf2.0.8 release candidate.  If you want to try it out or have any input, please do let me know.  Here’s the info:

 

Source from iperf2 on sourceforge https://sourceforge.net/projects/iperf2/

 

git clone git://git.code.sf.net/p/iperf2/code iperf2-code

 

Features:

    Improved performance

    Support smaller report intervals (5 ms or greater)

    Support SO_RCVTIMEOUT for server reports regardless of no packets

    Support SO_TIMESTAMP for kernel level packet timestamping

    Support end/end latency in mean/min/max format (UDP)

    (assumes client and server clocks synched, e.g by Precision Time Protocol)

    Add local port to bind support (-B option) using colon as separator

    (e.g. iperf -c 192.168.100.100 -B 192.168.100.10:6001)

    2.0.7 only tested with linux as it uses linux kernel specific features

    ======================================================

    **** UPCOMING FOR 2.0.8 BELOW (found in git repository) ****

    ======================================================

    Support TCP rate limited streams (via the -b) using token bucket

    Support packets per second (UDP) via pps as units, (e.g. -b 1000pps)

    Support realtime scheduler as a command line option (--realtime or -z)

    Improve accuracy of microsecond delay calls in os independent manner

    Use of Kalman filter to predict delay errors and adjust delay per predicted error

    Display PPS in both client and server reports (UDP)

    Display target interpacket gap (IPG) in initial client header (UDP)

    Fix final latency report sent from server to client (UDP)

    Fix portability, compile and test with Cygwin and MAC

 

Thanks,

Bob

 

------------------------------------------------------------------------------
_______________________________________________
Iperf-users mailing list
Iperf-users@...
https://lists.sourceforge.net/lists/listinfo/iperf-users
Bruce A. Mah | 14 Oct 23:42 2014
Picon

iperf 3.0.9 is available


ESnet (Energy Sciences Network) proudly announces the public release
of iperf-3.0.9.  This version includes a fairly important bug fix for
a situation in which attempting a UDP test with pathologically large
(and illegal) packet sizes could put the iperf3 server in a state
where it would stop accepting connections from clients, thus causing
the clients to crash when interrupted.

More information on changes can be found in the RELEASE_NOTES
file in the source distribution.

iperf3 is a tool for measuring the maximum TCP and UDP performance
along a path, allowing for the tuning of various parameters and
reporting measurements such as throughput, jitter, and datagram packet
loss.

The original iperf was implemented by NLANR / DAST.  Version 3 is a
complete reimplementation, with the goals of a smaller, simpler code
base, and a library that can be used by other programs.  It also adds
new functionality, such as CPU utilization measurements, zero copy TCP
support, and JSON output.  Note that iperf3 clients and servers are
not compatible with, and will not interoperate with, earlier versions
of iperf.

iperf3 is fully supported on Linux, FreeBSD, and MacOS X.  It may run
on other platforms as well, although it has not received the same
attention and testing.

The source code for iperf 3.0.9 is available at:

http://downloads.es.net/pub/iperf/iperf-3.0.9.tar.gz

SHA256 hash:

40249a2b30d26b937350b969bcb19f88e1beb356f886ed31422b554bac692459  iperf-3.0.9.tar.gz

iperf3 is freely-redistributable under a 3-clause BSD license.  More
information can be found in the LICENSE file inside the source
distribution.

Additional documentation for iperf3 can be found at:

http://software.es.net/iperf

More information about iperf3 (including the issue tracker, source
code repository access, and mailing list) can be found on the iperf3
page on GitHub at:

https://github.com/esnet/iperf

The mailing list for iperf3 development is:

iperf-dev@...

To see the list archives or join the mailing list, visit:

http://groups.google.com/group/iperf-dev

Harry Trieu | 5 Oct 08:35 2014
Picon

Problems Testing Implementation of New TCP Socket Option

Hi there,

I implemented a new TCP socket option in the Linux kernel to turn an experimental TCP option I am developing on/off. I modified iperf to turn the socket option on when the -E flag is passed in, however I get the following error when I run iperf3 -c 10.1.1.3 -E: iperf3: error - unable to initialize stream: Transport endpoint is not connected.

I traced the error back to the call to getpeername() in iperf_init_stream(). Errno after the call to getpeername() is 107 or ENOTCONN.

The other machine that is running iperf and listening for connections prints the following:
Accepted connection from 10.1.1.2, port 36806
iperf3: the client has unexpectedly closed the connection

I'm not sure if the new socket option was correctly implemented. Any ideas as to what might be going on or what I might be doing wrong?

Thanks,
Harry
------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@...
https://lists.sourceforge.net/lists/listinfo/iperf-users
Bruce A. Mah | 30 Sep 22:58 2014
Picon

iperf-3.0.8 is available


ESnet (Energy Sciences Network) proudly announces the public release
of iperf-3.0.8.  This version is functionally identical to the prior
3.0.7 release.  It incorprates some license wording changes to comply
with LBNL tech transfer requirements (the license remains a 3-clause
BSD license), as well as a minor compilation fix.

More information on changes can be found in the RELEASE_NOTES
file in the source distribution.

iperf3 is a tool for measuring the maximum TCP and UDP performance
along a path, allowing for the tuning of various parameters and
reporting measurements such as throughput, jitter, and datagram packet
loss.

The original iperf was implemented by NLANR / DAST.  Version 3 is a
complete reimplementation, with the goals of a smaller, simpler code
base, and a library that can be used by other programs.  It also adds
new functionality, such as CPU utilization measurements, zero copy TCP
support, and JSON output.  Note that iperf3 clients and servers are
not compatible with, and will not interoperate with, earlier versions
of iperf.

iperf3 is fully supported on Linux, FreeBSD, and MacOS X.  It may run
on other platforms as well, although it has not received the same
attention and testing.

The source code for iperf 3.0.8 is available at:

http://downloads.es.net/pub/iperf/iperf-3.0.8.tar.gz

SHA256 hash:

81b8d91159862896c57f9b90a006e8b5dc22bd94175d97bd0db50b0ae2c1a78e  iperf-3.0.8.tar.gz

iperf3 is freely-redistributable under a 3-clause BSD license.  More
information can be found in the LICENSE file inside the source
distribution.

Additional documentation for iperf3 can be found at:

http://software.es.net/iperf

More information about iperf3 (including the issue tracker, source
code repository access, and mailing list) can be found on the iperf3
page on GitHub at:

https://github.com/esnet/iperf

The mailing list for iperf3 development is:

iperf-dev@...

To see the list archives or join the mailing list, visit:

http://groups.google.com/group/iperf-dev

Bob (Robert) McMahon | 26 Sep 06:08 2014

Re: Iperf client sends out less UDP traffic than determined with "-b" flag

Hi Martin,

For me, with traffic control and the txqueuelen set to zero, the iperf client does not seem to get the ENOBUFS
and the tx rate reporting is that per the -b (offered load).   The iperf server is receiving at the qdisc
setting (shaped rate) vs the offered load (-b).  So it does look like there is some interplay between
ENOBUFS, qdiscs, and the txqueuelen that affects tx rate reporting.

Bob
-----Original Message-----
From: Martin T [mailto:m4rtntns@...] 
Sent: Thursday, September 25, 2014 4:10 PM
To: Bob (Robert) McMahon
Cc: Marc Herbert; Andrew Gallatin; iperf-users@...
Subject: Re: [Iperf-users] Iperf client sends out less UDP traffic than determined with "-b" flag

Bob,

I see, thanks!

As a last thing, I played around with qdisc buffer and kernel NIC
driver circular-buffer size. Unfortunately I wasn't able to reproduce
the results I saw on this virtual-machine. Even with no qdisc
buffer("ip link set dev eth0 txqueuelen 0") and lowest NIC driver
buffer supported on my tg3(ver 3.2.0-4-amd64) driver for my Broadcom
BCM5721 chipset, the Iperf client sent out as much UDP traffic as
determined with "-b" option:

root <at> 3:~# ip link show dev eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
mode DEFAULT
    link/ether 00:1d:09:f0:92:ab brd ff:ff:ff:ff:ff:ff
root <at> 3:~# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX:             511
RX Mini:        0
RX Jumbo:       0
TX:             511
Current hardware settings:
RX:             0
RX Mini:        0
RX Jumbo:       0
TX:             55

root <at> 3:~# iperf -c 192.0.2.1 -fm -t 10 -u -b 800m
------------------------------------------------------------
Client connecting to 192.0.2.1, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 0.22 MByte (default)
------------------------------------------------------------
[  3] local 192.168.1.45 port 44172 connected with 192.0.2.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   962 MBytes   807 Mbits/sec
[  3] Sent 686293 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
root <at> 3:~#

Has anyone managed to reproduce the behavior described in my initial
e-mail with decreasing the Tx buffers sizes on (virtual-)machine where
the Iperf client is running?

regards,
Martin

------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@...
https://lists.sourceforge.net/lists/listinfo/iperf-users

Martin T | 21 Sep 22:30 2014
Picon

Re: Iperf client sends out less UDP traffic than determined with "-b" flag

Marc,

> In this case, what throughput number do you get on the server side? Different from the client or always the same?

Throughput I see on a server side was always the same as Iperf client
reported, e.g. if Iperf client reported "[  3]  0.0-600.0 sec  8602
MBytes   120 Mbits/sec" then both Iperf server and switches between
the client and server saw 120Mbps of traffic.

> Is 120Mb/s the UDP maximum you ever see on the client, whatever parameters you use?

Correct.

> What is the maximum UDP sending rate you ever saw reported on this specific hardware and operating system?

It was a VMware virtual-machine(Debian Wheezy was a guest-OS) and
maximum UDP sending rate I received on this particular virtual-machine
was 120Mbps. If I replaced
the guest-OS in this very same virtual-machine with CentOS 7, the same
Iperf release with the same command was able to send traffic at
500Mbps.

Bob,

thank you for this very clear explanation! So for example in those two
write system calls:

[pid 14025] write(3,"\0\0\0\16T\n\331a\0\3\2563\0\0\0\0\0\0\0\1\0\0\23\211\0\0\0\0\0\17B <at> "...,1470)
= 1470
[pid 14025] nanosleep({0, 11071000},  <unfinished ...>
[pid 14026] <... nanosleep resumed> NULL) = 0
[pid 14026] nanosleep({0, 10000000}, NULL) = 0
[pid 14026] nanosleep({0, 10000000},  <unfinished ...>
[pid 14025] <... nanosleep resumed> 0xb74ba2c8) = 0
[pid 14025] gettimeofday({1409997153, 252946}, NULL) = 0
[pid 14025] write(3,"\0\0\0\17T\n\331a\0\3\334\22\0\0\0\0\0\0\0\1\0\0\23\211\0\0\0\0\0\17B <at> "...,1470)
= 1470
[pid 14025] nanosleep({0, 11088000},  <unfinished ...>
[pid 14026] <... nanosleep resumed> NULL) = 0
[pid 14026] nanosleep({0, 10000000},  <unfinished ...>
[pid 14025] <... nanosleep resumed> 0xb74ba2c8) = 0
[pid 14025] gettimeofday({1409997153, 264721}, NULL) = 0

..the number after the equals sign(1470) is the write return value aka
number of bytes accepted by kernel? And Iperf client just sums those
write() return values together and reports the amount of data
transmitted in bytes and in bps?

I also created a small program in C which writes a string into file
and here is the strace comparison when write() is successful:

open("/media/file.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
write(3, "Test string\n", 12)           = 12
exit_group(12)                          = ?

..and when the media is full:

open("/media/file.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
write(3, "Test string\n", 12)           = -1 ENOSPC (No space left on device)
exit_group(-1)                          = ?

regards,
Martin

On Fri, Sep 19, 2014 at 2:58 AM, Marc Herbert <marc.herbert <at> gmail.com> wrote:
>> 2) Iperf client didn't send the data at faster rate than 120Mbps
>> because it received some error-signals from kernel and immediately
>> sent out datagrams less frequently
>
> Whether packets are lost inside the sender or not and no matter how hard it
> tries, the client cannot perform an infinite number of write() per seconds.
> There is a "physical" limit caused by either the CPU, or memory bandwidth,
> or scheduling, etc. What is the maximum UDP sending rate you ever saw
> reported on this specific hardware and operating system?
>
> As already suggested on this list a few times over the years, also try
> varying the socket buffer size and see if it makes any difference.
>
> Marc
>
> 2014-09-18 11:52 GMT+01:00 Martin T <m4rtntns <at> gmail.com>:
>>
>> Marc,
>>
>> I'm afraid I didn't understand your question.. My question has nothing
>> to do with packet loss on the network. In case of packet loss on the
>> network, you see this in server-report if we speak about UDP mode.
>>
>>
>> I'll try to explain once again what I'm confused about. If I start the
>> Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i 60 -u -b 500m"
>> command and thus request Iperf to send data with bandwidth of 500Mbps
>> and Iperf client reports me that it has sent data at 120Mbps instead
>> of 500Mbps:
>>
>> [  3]  0.0-600.0 sec  8602 MBytes   120 Mbits/sec
>>
>> ..then this means that Iperf client somehow had to know that some of
>> the data was either:
>>
>> 1) dropped in the local machine, i.e. Iperf client writes to socket at
>> 500Mbps, but before printing the "[  3]  0.0-600.0 sec  8602 MBytes
>> 120 Mbits/sec" line, it somehow checks that actually not all of the
>> data written to socket was not transmitted
>>
>> 2) Iperf client didn't send the data at faster rate than 120Mbps
>> because it received some error-signals from kernel and immediately
>> sent out datagrams less frequently
>>
>> There is nothing to do with packet loss on network.
>>
>>
>>
>> regards,
>> Martin
>>
>>
>> On 9/18/14, Marc Herbert <marc.herbert <at> gmail.com> wrote:
>> > Martin,
>> >
>> >   What makes you think iperf counts local packet losses differently from
>> > network packet losses?
>> >
>> > Cheers,
>> >
>> > Marc
>> >
>> > 2014-09-18 8:50 GMT+01:00 Martin T <m4rtntns <at> gmail.com>:
>> >
>> >> Andrew,
>> >>
>> >> I'll rephrase my question which I asked earlier. If Iperf ignores the
>> >> errors regarding lack of memory in interface buffers, then how does
>> >> the Iperf client know that under those conditions it should make the
>> >> write calls to socket less frequently? For example if I start the
>> >> Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i 60 -u -b 500m"
>> >> command and thus request Iperf to send data with bandwidth of 500Mbps
>> >> and Iperf client reports me that it has sent data at 120Mbps instead
>> >> of 500Mbps:
>> >>
>> >> [  3]  0.0-600.0 sec  8602 MBytes   120 Mbits/sec
>> >>
>> >> ..then this means that Iperf client somehow had to know that some of
>> >> the data was dropped or Iperf client didn't send the data at faster
>> >> rate than 120Mbps.
>> >>
>> >>
>> >> thanks,
>> >> Martin
>> >>
>> >> On 9/17/14, Andrew Gallatin <gallatin <at> gmail.com> wrote:
>> >> > Actually, there are probably ways you could tell.  But the simplest
>> >> > is
>> >> > to
>> >> > just keep re-sending until you don't get an error, which is what
>> >> benchmarks
>> >> > do.
>> >> >
>> >> > On Wed, Sep 17, 2014 at 7:53 AM, Andrew Gallatin <gallatin <at> gmail.com>
>> >> > wrote:
>> >> >
>> >> >> There is no way to know.  UDP is lossy, after all.
>> >> >>
>> >> >> On Wed, Sep 17, 2014 at 7:47 AM, Martin T <m4rtntns <at> gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>> Andrew,
>> >> >>>
>> >> >>> if Iperf ignores the errors regarding lack of memory in interface
>> >> >>> buffers, then how does Iperf know that it should send out the
>> >> >>> datagrams less frequently, i.e. write to socket less frequently?
>> >> >>>
>> >> >>>
>> >> >>> thanks,
>> >> >>> Martin
>> >> >>>
>> >> >>>
>> >> >>> On 9/17/14, Andrew Gallatin <gallatin <at> gmail.com> wrote:
>> >> >>> > In my (cursory) read of the 3.x code, it looked like it
>> >> >>> > ignores NET_SOFTERROR returns & retries.    It would be nice if
>> >> >>> > it
>> >> >>> reported
>> >> >>> > the amount of ENOBUF errors it received like netperf does..
>> >> >>> >
>> >> >>> > On Wed, Sep 17, 2014 at 4:09 AM, Martin T <m4rtntns <at> gmail.com>
>> >> wrote:
>> >> >>> >
>> >> >>> >> Andrew, Bob,
>> >> >>> >>
>> >> >>> >> did I understand you correctly that while ENOBUFS error code is
>> >> >>> >> not
>> >> >>> >> shown in Iperf client output, the Iperf client actually takes it
>> >> into
>> >> >>> >> account and thus sends data out less frequently?
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> regards,
>> >> >>> >> Martin
>> >> >>> >>
>> >> >>> >> On 9/11/14, Bob (Robert) McMahon <rmcmahon <at> broadcom.com> wrote:
>> >> >>> >> > I thought I remembered hitting the ENOBUFs which is the case
>> >> >>> >> > per
>> >> >>> >> > the
>> >> >>> >> > source.
>> >> >>> >> >
>> >> >>> >> > // perform write
>> >> >>> >> > currLen = write( mSettings->mSock, mBuf, mSettings->mBufLen );
>> >> >>> >> > if ( currLen < 0 && errno != ENOBUFS ) {
>> >> >>> >> > WARN_errno( currLen < 0, "write2" );
>> >> >>> >> > break;
>> >> >>> >> > }
>> >> >>> >> > // report packets
>> >> >>> >> > reportstruct->packetLen = currLen;
>> >> >>> >> > ReportPacket( mSettings->reporthdr, reportstruct );
>> >> >>> >> >
>> >> >>> >> > A minor nit, the currLen may need to set to zero when
>> >> >>> >> > errno==ENOBUFFS
>> >> >>> >> before
>> >> >>> >> > the ReportPacket()
>> >> >>> >> >
>> >> >>> >> > Also, ione could count the number of writes that return
>> >> >>> >> > ENOBUFFS
>> >> >>> >> > but
>> >> >>> >> > not
>> >> >>> >> > sure why that's interesting.
>> >> >>> >> >
>> >> >>> >> > As an FYI, I have a tool that manages multiple, distributed
>> >> >>> >> > iperf
>> >> >>> >> sessions
>> >> >>> >> > which I use for wi-fi testing.   The following shows an 802.11
>> >> >>> >> > AC
>> >> >>> wi-fi
>> >> >>> >> link
>> >> >>> >> > that maxes out around 800Mbs.  The offered load is 2G and the
>> >> iperf
>> >> >>> >> client
>> >> >>> >> > seems to be reporting properly and responding as expected when
>> >> >>> >> > RF
>> >> >>> >> > attenuation is added and then removed.
>> >> >>> >> >
>> >> >>> >> > utf> udp start; udp linkcheck -now;  L1 attn 65; UTF::Sleep
>> >> >>> >> > 0.3;
>> >> L1
>> >> >>> >> > attn
>> >> >>> >> 0;
>> >> >>> >> > UTF::Sleep 0.3; udp stop
>> >> >>> >> > 20:45:08  LOG   sfast-lx2  iperf -s -w 7350000 -l 1470 -u  -i
>> >> >>> >> > 0.1
>> >> >>> >> > -fb
>> >> >>> >> > -p
>> >> >>> >> > 60001
>> >> >>> >> > 20:45:08  HNDLR_ udp-rx     Server listening on UDP port 60001
>> >> >>> >> > (sflx2,file12)
>> >> >>> >> > 20:45:08  HNDLR_ udp-rx     Receiving 1470 byte datagrams
>> >> >>> >> > (sflx2,file12)
>> >> >>> >> > 20:45:08  HNDLR_ udp-rx     UDP buffer size: 8388608 Byte
>> >> (WARNING:
>> >> >>> >> > requested 7350000 Byte) (sflx2,file12)
>> >> >>> >> > 20:45:08  LOG   itx-lx1    iperf -c 192.168.1.42 -w 367500 -l
>> >> >>> >> > 1470
>> >> >>> >> > -u
>> >> >>> >> > -b
>> >> >>> >> 2G
>> >> >>> >> > -B 192.168.1.100  -i 0.1 -fb -S 0x0 -T 255 -t 5356800 -p 60001
>> >> >>> >> > 20:45:08  HNDLR udp-tx
>> >> >>> >> > ------------------------------------------------------------
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR_ udp-rx     [  3] local 192.168.1.42 port
>> >> >>> >> > 60001
>> >> >>> >> > connected
>> >> >>> >> > with 192.168.1.100 port 60001 (sflx2,file12)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     Client connecting to 192.168.1.42,
>> >> >>> >> > UDP
>> >> >>> port
>> >> >>> >> 60001
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:08  INFO  udp-tx     TX start: protocol=UDP Offered
>> >> >>> >> > rate=2G
>> >> >>> >> > pktsize=1470 delay=5us
>> >> >>> >> > 20:45:08  HNDLR udp-tx     Binding to local address
>> >> >>> >> > 192.168.1.100
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     Sending 1470 byte datagrams
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     UDP buffer size: 735000 Byte
>> >> >>> >> > (WARNING:
>> >> >>> >> requested
>> >> >>> >> > 367500 Byte) (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx
>> >> >>> >> > ------------------------------------------------------------
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [  3] local 192.168.1.100 port
>> >> >>> >> > 60001
>> >> >>> >> > connected
>> >> >>> >> > with 192.168.1.42 port 60001 (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [ ID] Interval       Transfer
>> >> >>>  Bandwidth
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [  3] 0.000-0.100 sec  11244030
>> >> >>> >> > Bytes
>> >> >>> >> 899522400
>> >> >>> >> > bits/sec (43602lx1,file13)
>> >> >>> >> > 20:45:08  LOG   L1         attn 65
>> >> >>> >> > 20:45:08  INFO  ::AF-A     Setting attn to 65
>> >> >>> >> > 20:45:08  INFO             Sleep 0.3 sec
>> >> >>> >> > 20:45:08  LOG   itx-lx1    wl0.0: wlc_send_bar: seq 0x17f tid
>> >> >>> >> > 0
>> >> >>> >> > 20:45:08  LOG   itx-lx1    wl0.0: wlc_send_bar: seq 0x17f tid
>> >> >>> >> > 0
>> >> >>> >> (repeated 1
>> >> >>> >> > times)
>> >> >>> >> > 20:45:08  LOG   L1         attn 0
>> >> >>> >> > 20:45:08  INFO  ::AF-A     Setting attn to 0
>> >> >>> >> > 20:45:08  INFO             Sleep 0.3 sec
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [  3] 0.100-0.200 sec  2625420
>> >> >>> >> > Bytes
>> >> >>> >> 210033600
>> >> >>> >> > bits/sec (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [  3] 0.200-0.300 sec  0.00 Bytes
>> >> >>> >> > 0.00
>> >> >>> >> bits/sec
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [  3] 0.300-0.400 sec  0.00 Bytes
>> >> >>> >> > 0.00
>> >> >>> >> bits/sec
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [  3] 0.400-0.500 sec  6251910
>> >> >>> >> > Bytes
>> >> >>> >> 500152800
>> >> >>> >> > bits/sec (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [  3] 0.500-0.600 sec  10406130
>> >> >>> >> > Bytes
>> >> >>> >> 832490400
>> >> >>> >> > bits/sec (43602lx1,file13)
>> >> >>> >> > 20:45:08  HNDLR udp-tx     [  3] 0.600-0.700 sec  10229730
>> >> >>> >> > Bytes
>> >> >>> >> 818378400
>> >> >>> >> > bits/sec (43602lx1,file13)
>> >> >>> >> > 20:45:08  INFO  udp        Kill signal -HUP sent to
>> >> >>> (43602lx1,file13) :
>> >> >>> >> > iperf -c 192.168.1.42 -w 367500 -l 1470 -u -b 2G -B
>> >> >>> >> > 192.168.1.100
>> >> >>> >> > -i
>> >> >>> >> > 0.1
>> >> >>> >> > -fb -S 0x0 -T 255 -t 5356800 -p 60001
>> >> >>> >> > 20:45:09  INFO  udp        Kill signal -HUP sent to
>> >> >>> >> > (sflx2,file12)
>> >> >>> >> > :
>> >> >>> >> iperf
>> >> >>> >> > -s -w 7350000 -l 1470 -u  -i 0.1 -fb -p 60001
>> >> >>> >> > 20:45:09  HNDLR udp-tx     Close actions for event handler
>> >> >>> >> > (43602lx1,file13)
>> >> >>> >> > 20:45:09  INFO  udp-tx     Closed : (43602lx1,file13)
>> >> >>> >> >
>> >> >>> >> > Bob
>> >> >>> >> > -----Original Message-----
>> >> >>> >> > From: Bob (Robert) McMahon
>> >> >>> >> > Sent: Wednesday, September 10, 2014 5:02 PM
>> >> >>> >> > To: Martin T; gallatin <at> gmail.com
>> >> >>> >> > Cc: iperf-users <at> lists.sourceforge.net
>> >> >>> >> > Subject: RE: [Iperf-users] Iperf client sends out less UDP
>> >> >>> >> > traffic
>> >> >>> than
>> >> >>> >> > determined with "-b" flag
>> >> >>> >> >
>> >> >>> >> > Hi Martin,
>> >> >>> >> >
>> >> >>> >> > Just to confirm Andrew, I'm pretty sure I've seen the write
>> >> syscall
>> >> >>> on
>> >> >>> >> linux
>> >> >>> >> > return ENOBUFFs.  I'll check it out in the next few days and
>> >> >>> >> > let
>> >> >>> >> > you
>> >> >>> >> know.
>> >> >>> >> >
>> >> >>> >> >
>> >> >>> >> > Bob
>> >> >>> >> > -----Original Message-----
>> >> >>> >> > From: Martin T [mailto:m4rtntns <at> gmail.com]
>> >> >>> >> > Sent: Wednesday, September 10, 2014 5:12 AM
>> >> >>> >> > To: gallatin <at> gmail.com; Bob (Robert) McMahon
>> >> >>> >> > Cc: iperf-users <at> lists.sourceforge.net
>> >> >>> >> > Subject: Re: [Iperf-users] Iperf client sends out less UDP
>> >> >>> >> > traffic
>> >> >>> than
>> >> >>> >> > determined with "-b" flag
>> >> >>> >> >
>> >> >>> >> > Andrew, Bob:
>> >> >>> >> >
>> >> >>> >> > does Iperf use send() system call? Based on Iperf source code
>> >> >>> >> > and
>> >> >>> >> > tests with strace, it seems to use write() system call to send
>> >> data
>> >> >>> to
>> >> >>> >> > socket and according to write() manual page, it doesn't have a
>> >> >>> similar
>> >> >>> >> > error to ENOBUFS. However, I could easily be wrong here.
>> >> >>> >> >
>> >> >>> >> >
>> >> >>> >> > Andrew,
>> >> >>> >> >
>> >> >>> >> > if Linux kernel silently drops packets in case the device
>> >> >>> >> > queue
>> >> >>> >> > overflows, then how does Iperf client know that? For example
>> >> >>> >> > if
>> >> >>> >> > I
>> >> >>> >> > start the Iperf client with "iperf -c 10.10.10.1 -fm -t 600 -i
>> >> >>> >> > 60
>> >> >>> >> > -u
>> >> >>> >> > -b 500m" command and thus request Iperf to send data with
>> >> bandwidth
>> >> >>> of
>> >> >>> >> > 500Mbps and Iperf client reports me that it has sent data at
>> >> >>> >> > 120Mbps
>> >> >>> >> > instead of 500Mbps:
>> >> >>> >> >
>> >> >>> >> > [  3]  0.0-600.0 sec  8602 MBytes   120 Mbits/sec
>> >> >>> >> >
>> >> >>> >> > ..then this means that Iperf client somehow had to know that
>> >> >>> >> > some
>> >> >>> >> > of
>> >> >>> >> > the data was dropped or Iperf client was restricted to send
>> >> >>> >> > data
>> >> at
>> >> >>> >> > faster rate than 120Mbps. How to explain this behavior?
>> >> >>> >> >
>> >> >>> >> >
>> >> >>> >> > thanks,
>> >> >>> >> > Martin
>> >> >>> >> >
>> >> >>> >> > On 9/8/14, Andrew Gallatin <gallatin <at> gmail.com> wrote:
>> >> >>> >> >> No, just that it is fairly new (added 5 years ago in "6ce9e7b
>> >> >>> >> >> ip:
>> >> >>> >> >> Report
>> >> >>> >> >> qdisc packet drops").  Netperf did not have it either, until
>> >> >>> >> >> I
>> >> >>> >> >> tried
>> >> >>> >> >> to
>> >> >>> >> >> figure out why I was getting wildly inaccurate results from
>> >> >>> >> >> netperf
>> >> >>> >> >> (eg,
>> >> >>> >> >> seeing 20Gb/s on a 1GbE link).
>> >> >>> >> >>
>> >> >>> >> >>
>> >> >>> >> >> On Mon, Sep 8, 2014 at 11:34 AM, Bob (Robert) McMahon
>> >> >>> >> >> <rmcmahon <at> broadcom.com
>> >> >>> >> >>> wrote:
>> >> >>> >> >>
>> >> >>> >> >>>  Hi Drew,
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>> Thanks for the detailed explanation.  Do you know any reason
>> >> that
>> >> >>> >> >>> iperf
>> >> >>> >> >>> doesn’t or shouldn’t set IP_RECVERR on the sending
>> >> >>> >> >>> socket(s)?
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>> Thanks,
>> >> >>> >> >>>
>> >> >>> >> >>> Bob
>> >> >>> >> >>>
>> >> >>> >> >>> *From:* Andrew Gallatin [mailto:gallatin <at> gmail.com]
>> >> >>> >> >>> *Sent:* Monday, September 08, 2014 6:53 AM
>> >> >>> >> >>> *To:* Martin T
>> >> >>> >> >>> *Cc:* iperf-users
>> >> >>> >> >>> *Subject:* Re: [Iperf-users] Iperf client sends out less UDP
>> >> >>> traffic
>> >> >>> >> >>> than
>> >> >>> >> >>> determined with "-b" flag
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>> Please, there are no "dropping" or "non dropping" NIC
>> >> >>> >> >>> drivers,
>> >> >>> unless
>> >> >>> >> >>> some
>> >> >>> >> >>> driver is so broken that
>> >> >>> >> >>>
>> >> >>> >> >>> it silently returns NETDEV_TX_OK & does not stop its
>> >> >>> >> >>> txqueue,
>> >> but
>> >> >>> >> >>> a
>> >> >>> >> >>> driver
>> >> >>> >> >>> that broken would
>> >> >>> >> >>>
>> >> >>> >> >>> not be accepted into the kernel.
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>> What is really happening is that the linux kernel is
>> >> >>> >> >>> silently
>> >> >>> >> >>> dropping
>> >> >>> >> >>> packets when the device's
>> >> >>> >> >>>
>> >> >>> >> >>> queue is full *AND* when the device queue is too small to
>> >> >>> >> >>> hold
>> >> >>> >> >>> the
>> >> >>> >> >>> entire
>> >> >>> >> >>> socket buffer's worth
>> >> >>> >> >>>
>> >> >>> >> >>> of traffic.  If you want blocking behavior, you need to
>> >> >>> >> >>> ensure
>> >> >>> >> >>> you
>> >> >>> >> >>> have
>> >> >>> >> >>> the driver's queue
>> >> >>> >> >>>
>> >> >>> >> >>> large enough (or the socket buffer small enough) so that the
>> >> >>> driver's
>> >> >>> >> >>> queue can hold an
>> >> >>> >> >>>
>> >> >>> >> >>> entire socket buffer worth of traffic (* the number of
>> >> >>> >> >>> active
>> >> >>> sockets
>> >> >>> >> >>> per
>> >> >>> >> >>> queue).  You can change
>> >> >>> >> >>>
>> >> >>> >> >>> the driver's queue size via ifconfig txqueuelen.  It might
>> >> >>> >> >>> be
>> >> >>> better
>> >> >>> >> >>> to
>> >> >>> >> >>> reduce the socketbuffer
>> >> >>> >> >>>
>> >> >>> >> >>> size (and remember that linux multiplies whatever value you
>> >> >>> >> >>> give
>> >> >>> >> >>> it
>> >> >>> >> >>> by
>> >> >>> >> >>> 2,
>> >> >>> >> >>> to account for
>> >> >>> >> >>>
>> >> >>> >> >>> slop...).
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>> Note that any silent drops are done by the kernel, not the
>> >> >>> >> >>> NIC
>> >> >>> >> >>> and
>> >> >>> >> >>> that
>> >> >>> >> >>> this is
>> >> >>> >> >>>
>> >> >>> >> >>> (bizarrely) the expected behavior on linux.  If you don't
>> >> believe
>> >> >>> me,
>> >> >>> >> >>>  read the send(2) man page on linux:
>> >> >>> >> >>>
>> >> >>> >> >>>        ENOBUFS
>> >> >>> >> >>>
>> >> >>> >> >>>               The output queue for a network interface was
>> >> >>> >> >>> full.
>> >> >>> >> >>> This
>> >> >>> >> >>> generally indicates that the
>> >> >>> >> >>>
>> >> >>> >> >>>               interface has stopped sending, but may be
>> >> >>> >> >>> caused
>> >> by
>> >> >>> >> >>> transient congestion.  (Normally,
>> >> >>> >> >>>
>> >> >>> >> >>>               this  does not occur in Linux.  Packets are
>> >> >>> >> >>> just
>> >> >>> >> >>> silently
>> >> >>> >> >>> dropped when a device queue
>> >> >>> >> >>>
>> >> >>> >> >>>               overflows.)
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>> If iperf wants a correct accounting of the packets sent by
>> >> >>> >> >>> the
>> >> >>> stack
>> >> >>> >> >>> vs
>> >> >>> >> >>> dropped, iperf must set
>> >> >>> >> >>>
>> >> >>> >> >>> the IP_RECVERR sockopt on the sending socket.  This stops
>> >> >>> >> >>> the
>> >> >>> kernel
>> >> >>> >> >>> from
>> >> >>> >> >>> silently
>> >> >>> >> >>>
>> >> >>> >> >>> dropping packets, and causes send() to return -ENOBUFS
>> >> >>> >> >>> rather
>> >> >>> >> >>> than
>> >> >>> 0
>> >> >>> >> >>> when
>> >> >>> >> >>>
>> >> >>> >> >>> datagram is dropped due to lack of resources (eg, like BSD
>> >> does).
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>>
>> >> >>> >> >>> Drew
>> >> >>> >> >>>
>> >> >>> >> >>
>> >> >>> >> >
>> >> >>> >>
>> >> >>> >
>> >> >>>
>> >> >>
>> >> >>
>> >> >
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Want excitement?
>> >> Manually upgrade your production database.
>> >> When you want reliability, choose Perforce
>> >> Perforce version control. Predictably reliable.
>> >>
>> >>
>> >> http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk
>> >> _______________________________________________
>> >> Iperf-users mailing list
>> >> Iperf-users <at> lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/iperf-users
>> >>
>> >
>
>

------------------------------------------------------------------------------
Slashdot TV.  Video for Nerds.  Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iperf-users
Andrew Gallatin | 10 Sep 18:14 2014
Picon

gettimeofday

I was looking at the iperf3 source code, and its ... unfortunate ... that iperf3 still seems to be nearly as much of a gettimeofday() benchmark as it is a network benchmark.

For example, if I want to send UDP as fast as possible, I'll do something like:
iperf -c 172.18.126.63 -u -b 9999999M

If I use strace -c on this, I see:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 42.92    0.238950           1    287535           gettimeofday
 41.96    0.233611           3     71567           write
 14.87    0.082772           1     72900           select

The pattern seems to be:

gettimeofday
gettimeofday
select
gettimeofday
gettimeofday
write
gettimeofday
gettimeofday
select
<repeats>

This isn't a problem on *most* OSes (and it is getting better), but it can lead to surprisingly bad results on 10GbE or 40GbE networks on some OSes, especially one that chose an expensive timecounter (eg, one that does a PIO read access on every gettimeofday).  This is the big reason why I still use netperf.

Maybe iperf needs a "go fast" option that dispenses with all timing except to mark the time at the start & end of the test.

Drew

------------------------------------------------------------------------------
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk
_______________________________________________
Iperf-users mailing list
Iperf-users@...
https://lists.sourceforge.net/lists/listinfo/iperf-users

Gmane