headers
Mike Tancsa | 1 Dec 2006 01:41

Re: NetBSD in BSD Router / Firewall Testing

At 06:49 PM 11/30/2006, Thor Lancelot Simon wrote:

>There are some severe problems with the test configuration.
>
>1) The published test results freely mix configurations where the switch
>    applies and removes the vlan tags with configurations where the host
>    does so.  This is not a good idea:

Hi,
The switch is always involved.  The ports are only in trunk mode for 
the trunking tests. Otherwise, its switchport access.  The same 
"limitations" apply to all tested configurations.  When I swapped in 
a faster CPU briefly, I was seeing rates of +1Mpps on RELENG_4 with 
no dropped packets with no firewall in the kernel.  Thats the same 
hardware, so I am not sure how its inadequate hardware on NetBSD 
tests all of a sudden.

>    1) The efficiency of the switch itself will differ in these configurations

Why ?  The only thing being changed from test to test is the OS.

>    2) The difference in frame size will actually measurably impact the PPS.

Framesize is always the same.  UDP packet with a 10byte payload. The 
generators are the same devices all the time. I am not using 
different frame sizes for different setups to try and make something 
look good and other things bad.

>    3) One of the device drivers you're testing doesn't do hardware VLAN
>       tag insertion/removal in NetBSD due to a bug (wm).  Obviously, this
(Continue reading)

Permalink | Reply | 8 comments |
headers
Thor Lancelot Simon | 1 Dec 2006 03:43

Re: NetBSD in BSD Router / Firewall Testing

On Thu, Nov 30, 2006 at 07:41:45PM -0500, Mike Tancsa wrote:
> At 06:49 PM 11/30/2006, Thor Lancelot Simon wrote:
> 
> >   1) The efficiency of the switch itself will differ in these 
> >   configurations
> 
> Why ?  The only thing being changed from test to test is the OS.

Because the switch hardware does not forward packets at the same rate
when it is inserting and removing VLAN tags as it does when it's not.
The effect will be small, but measurable.

> >   2) The difference in frame size will actually measurably impact the PPS.
> 
> Framesize is always the same.  UDP packet with a 10byte payload.

No.  The Ethernet packets with the VLAN tag on them are not, in fact,
the same size as those without it; and for a packet as small as a 10
byte UDP packet, this will make quite a large difference if you actually
have a host that can inject packets at anywhere near wire speed.

> generators are the same devices all the time. I am not using 
> different frame sizes for different setups to try and make something 
> look good and other things bad.

I didn't say that you were, just to be clear.  But that does not mean
that running some tests with tagging turned on, and others not, is
good benchmarking practice: you should run the exact same set of tests
for all host configurations, because doing otherwise yields distorted
results.
(Continue reading)

Permalink | Reply | 7 comments |
headers
Mike Tancsa | 1 Dec 2006 04:15

Re: NetBSD in BSD Router / Firewall Testing

At 09:43 PM 11/30/2006, Thor Lancelot Simon wrote:
>On Thu, Nov 30, 2006 at 07:41:45PM -0500, Mike Tancsa wrote:
> > At 06:49 PM 11/30/2006, Thor Lancelot Simon wrote:
> >
> > >   1) The efficiency of the switch itself will differ in these
> > >   configurations
> >
> > Why ?  The only thing being changed from test to test is the OS.
>
>Because the switch hardware does not forward packets at the same rate
>when it is inserting and removing VLAN tags as it does when it's not.
>The effect will be small, but measurable.

But the same impact will hurt *all* the OSes tested equally, not just 
NetBSD. Besides, supposedly the switch is rated to 17Mpps.  No doubt 
there is a bit of vendor exaggeration, but I doubt they would stretch 
the number by a factor of 10.  Still, even if they did, I would not 
be able to push over 1Mpps on my RELENG_4 setup.

> > >   2) The difference in frame size will actually measurably 
> impact the PPS.
> >
> > Framesize is always the same.  UDP packet with a 10byte payload.
>
>No.  The Ethernet packets with the VLAN tag on them are not, in fact,

I did both sets of tests.  eg . the line

RELENG_6 UP i386 FastFWD Polling
(Continue reading)

Permalink | Reply | 1 comment |
headers
Mike Tancsa | 1 Dec 2006 07:06

Re: NetBSD in BSD Router / Firewall Testing

At 09:43 PM 11/30/2006, Thor Lancelot Simon wrote:

>I note that you snipped the text where I noted that because you're
>testing the wm card with mismatched kernel and ifconfig, you're not
>using its hardware checksum offload.  That's one thing you should
>definitely fix, and if you don't have that turned on for other
>kernels you're testing, of course you should probably fix it there too.

OK, I updated the base as well and rebuilt the kernel. There doesnt 
seem to be much difference, perhaps +5Kpps by turning it on.  But it 
seems to be the driver, as I get FAR better results with the bge nic 
(see below)

# ifconfig wm0
wm0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         capabilities=7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
         enabled=300<IP4CSUM_Rx,IP4CSUM_Tx>
         address: 00:15:17:0b:70:98
         media: Ethernet autoselect (1000baseT 
full-duplex,flowcontrol,rxpause,txpause)
         status: active
         inet 192.168.88.223 netmask 0xffffff00 broadcast 192.168.88.255
         inet6 fe80::215:17ff:fe0b:7098%wm0 prefixlen 64 scopeid 0x5
# ifconfig wm1
wm1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         capabilities=7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
         enabled=300<IP4CSUM_Rx,IP4CSUM_Tx>
         address: 00:15:17:0b:70:99
         media: Ethernet autoselect (1000baseT 
full-duplex,flowcontrol,rxpause,txpause)
(Continue reading)

Permalink | Reply | 4 comments |
headers
Joerg Sonnenberger | 1 Dec 2006 11:49
Picon

Re: NetBSD in BSD Router / Firewall Testing

On Thu, Nov 30, 2006 at 10:15:04PM -0500, Mike Tancsa wrote:
> Stock FreeBSD, or modified FreeBSD ?  With RELENG_4 I can push over 
> 1Mpps.  All of the test setups I used saw input errors when I tried 
> to push too many packets through the box.  I really dont know much 
> about NetBSD but it too will have some sort of limit as to how much 
> it can forward. Once its limit is hit, how does it report that 
> ?  Does it just silently drop the packet ? Or does it show up as an 
> input error ?

Input errors are problems of the hardware, not dropped packets (*). Try
"netstat -q" if you want to see whether the queues dropped packets --
otherwise they are processed.

(*) If the RX processing can't keep up, it might signal it as error, not
sure. That would hint to an interrupt problem though.

Joerg
Permalink | Reply | 0 comments |
headers
Ignatios Souvatzis | 1 Dec 2006 14:52
Picon

Ethersubr fix for netiso

Hi,

when Perry asked for NETISO testing code, I wrote some, and found bugs.
Basically, when ether_input() calls (indirectly) clnp_input(), it does
horrible things. 

Namely, it does an overlapping struct assign (on a memory area
formerly m_adj'ed away!) which might have worked with ancient
compilers, but creates address corruption now. (== netbsd-3-1).

All  to get rid of the intermediate 3-byte 802.x llc field.

As the clnp_input is aware of being called by ethernet anyway (and
sort of needs to), I decided to just give it the full ethernet
header with llc, and let it throw it away itself (it does an m_adj
itself, so this is for free). Saving in the ISO networking path:
two memcpy if done right, one memmove if the current code was 
trivially repaired.

I suspect that any modern compiler would integrate one additional
compare-to-constant and and conditional branch (which are executed 
additionally for the DIX protocols) into the big DIX switch, so no
cost created there, or else, say, 2 instruction cycles (== about one
nanosecond for modern machines).

if_fddisubr and if_tokensubr will need similar patches; at least for
fddi this won't create a penalty because it hasn't any DIX type
field, so there's a convenient switch case already to move the
m_adj() to.
(Continue reading)

Permalink | Reply | 8 comments |
headers
Mike Tancsa | 1 Dec 2006 15:31

Re: NetBSD in BSD Router / Firewall Testing

At 06:06 PM 11/30/2006, Hubert Feyrer wrote:

>[adding tech-net <at>  as I don't really know what to answer...
>
>  Context: adding NetBSD in the benchmark at
>  http://www.tancsa.com/blast.html, with the wm(4) driver in
>  -current, as it's not available in 3.1]
>
>
>On Thu, 30 Nov 2006, Mike Tancsa wrote:
>>Gave it a try and I posted the results on the web page.  The Intel 
>>driver doesnt seem to work too well.  Is there debugging in this kernel ?
>
>That sounds indeed not so bright. I do not know about the wm(4) 
>driver, but maybe someone on tech-net <at>  (CC:d) has an idea. IIRC 
>that's with a -current (HEAD) GENERIC kernel and the wm(4) driver, 
>while bge(4) driver works ok.
>
>What I wonder is: how does the bge(4) driver perform under -current, 
>do you have numbers for that? (Just to make sure it's not -curren that's hosed)

Done and posted.  I also looked at netstat -q and indeed it reports 
dropped packets

# netstat -q
arpintrq:
         queue length: 0
         maximum queue length: 50
         packets dropped: 151
ipintrq:
(Continue reading)

Permalink | Reply | 8 comments |
headers
Steven M. Bellovin | 1 Dec 2006 15:55

Re: NetBSD in BSD Router / Firewall Testing

On Fri, 01 Dec 2006 09:31:23 -0500
Mike Tancsa <mike <at> sentex.net> wrote:

> 
> # netstat -q
> arpintrq:
>          queue length: 0
>          maximum queue length: 50
>          packets dropped: 151

I'm not sure this one matters much in the real world -- I suspect it can
only happen when a large number of addresses are polled in a very short
time.  (OTOH, it might happen if a scanning worm was working through
the router.)

> ipintrq:
>          queue length: 0
>          maximum queue length: 256
>          packets dropped: 133721212

This is the second report we've seen recently of packet drops in this
queue.  We need to understand what's going on, I think.

		--Steve Bellovin, http://www.cs.columbia.edu/~smb
Permalink | Reply | 5 comments |
headers
Mike Tancsa | 1 Dec 2006 16:00

Re: NetBSD in BSD Router / Firewall Testing

At 09:55 AM 12/1/2006, Steven M. Bellovin wrote:

> > ipintrq:
> >          queue length: 0
> >          maximum queue length: 256
> >          packets dropped: 133721212
>
>This is the second report we've seen recently of packet drops in this
>queue.  We need to understand what's going on, I think.

Hi,

I am guessing I am just overwhelming the box no ? Each of my 
generator boxes are blasting about 600Kpps in opposite directions 
through the box 10 byte UDP packets.  Even when doing just the one 
stream in NetBSD, the box (r2) acting as the router is totally 
unresponsive from the serial console and OOB NIC.

         ---Mike
Permalink | Reply | 4 comments |
headers
Steven M. Bellovin | 1 Dec 2006 17:25

Re: NetBSD in BSD Router / Firewall Testing

On Fri, 01 Dec 2006 10:00:09 -0500
Mike Tancsa <mike <at> sentex.net> wrote:

> At 09:55 AM 12/1/2006, Steven M. Bellovin wrote:
> 
> > > ipintrq:
> > >          queue length: 0
> > >          maximum queue length: 256
> > >          packets dropped: 133721212
> >
> >This is the second report we've seen recently of packet drops in this
> >queue.  We need to understand what's going on, I think.
> 
> Hi,
> 
> I am guessing I am just overwhelming the box no ? Each of my
> generator boxes are blasting about 600Kpps in opposite directions
> through the box 10 byte UDP packets.  Even when doing just the one
> stream in NetBSD, the box (r2) acting as the router is totally
> unresponsive from the serial console and OOB NIC.
> 
I'd have expected the problem to show as drops on the output queue, not
ipintrq, unless you're running at near-100% CPU.  The previous case did
not involve CPU exhaustion -- does yours?

		--Steve Bellovin, http://www.cs.columbia.edu/~smb
Permalink | Reply | 1 comment |

Gmane