Re: question about mmap support

Paolo Abeni writes:
> I tried to dig this in the ML archive, without any luck. It look like
> there is a certain interested for the memory map patch by Phil Wood
> (http://public.lanl.gov/cpw/), but it have never been merged, why ?!?
>   

I can't say for sure, but I suspect that Phil Woods' changes, which 
involve a lot of configuration in environment variables are seen as a 
bit too intrusive (and possibly raising security issues - certainly my 
feeling as well).

At our company, we have a privately maintained version of libpcap which 
includes a merge of Alexey Kuznetzov's old Linux fork of libpcap with 
mmap support (with his savefile and structure changes #ifdef'ed out).  I 
suspect that Alexey's gratuitous changes to the structures and savefile 
format etc. may be the source of some resistance to adding support for 
Linux mmap.  I think that it varies by distribution, but I know that Red 
Hat dropped the Kuznetzov fork in favor of tcdpump.org version in RHL 7 
or 8, but I believe that some Linux distros (Gentoo? others?) may still 
have mmap support in their libpcap distros.

I would guess that most people who want improved capture performance on 
Linux either use Phil Woods' version (which tracks the official libpcap 
pretty well) or Luca Deri's PF_RING version (which also requires kernel 
support, and is less flexible, but gives better performance for a 
dedicated capture appliance).  Or else they get dedicated hardware like 
Endace DAG cards, which are supported in tcpdump.org version.

I get the impression that most of the core tcpdump.org people are 
running on *BSD systems, so as long as the Linux version works 

[PATCH] fix some typo in pcap-usb-linux

hello,

the attached trivial patch fix a couple of typos in the comments of
pcap-usb-linux.c. 

I fear that others grammar/spell errors are present in the code I
submitted due to my poor English. I'll try to look after for them...

ciao,

paolo
--------------------------------------------------------------------

CONFIDENTIALITY NOTICE

This message and its attachments are addressed solely to the persons above and may contain confidential
information. If you have received the message in error, be informed that any use of the content hereof is
prohibited. Please return it immediately to the sender and delete the message. Should you have any
questions, please contact us by replying to webmaster <at> telecomitalia.it.

        Thank you

                                        www.telecomitalia.it

--------------------------------------------------------------------

hello,

[PATCH] enable memory mapped access to ethernet device for linux

hi list,

Following the discussion about the memory mapped access for Linux, I
tried to rework some existing code (originally independently created by
Simon Patarin and Phil Wood) to enable this functionality with the
minimum impact on the current pcap code. 

It does not use environment variables to control the memory mapped ring
parameters; instead the requested snap len is used: the low order bytes
are used to select the ring frame size and the high order bytes are used
to select the ring frame number. If the high order bytes is 0, like in
every current libpcap usage, a reasonable default is used. 

The poll function is used to implement the timeout in the read method:
must I add some autoconf check for poll() availability ?!? or use
select() instead ?!?

I really appreciated any comments and or review on this matter (even a
'please don't try to push this thing ...' 

Thanks,

Paolo
--------------------------------------------------------------------

CONFIDENTIALITY NOTICE

This message and its attachments are addressed solely to the persons above and may contain confidential
information. If you have received the message in error, be informed that any use of the content hereof is
prohibited. Please return it immediately to the sender and delete the message. Should you have any

setfilter causes core on Solaris

I'm using pcap_dispatch to call my callback. Inside the callback, I may 
set a new filter. This results in a core dump in bpf_filter.c, line 239. 
Its calling abort because of a bad filter code. This will only happen 
with a live capture.

The bug is actually in pcap-dlpi.c. It keeps a pointer to the filter 
code. Unfortunately the pointer never gets reset as long as there are 
packets to read. Adding:

fcode = p->fcode.bf_insns;

after the callback returns takes care of the issue. I've attached a 
patch and posted it as 1844245.

Regards,

	Andy

*** pcap-dlpi.c.orig    Sun Dec  2 01:23:37 2007
--- pcap-dlpi.c Sun Dec  2 01:25:39 2007
***************
*** 359,364 ****
--- 359,365 ----
                         if (pkthdr.caplen > p->snapshot)
                                 pkthdr.caplen = p->snapshot;
                         (*callback)(user, &pkthdr, pk);
+                       fcode = p->fcode.bf_insns;
                         if (++n >= cnt && cnt >= 0) {
                                 p->cc = ep - bp;
                                 p->bp = bp;

Re: [PATCH] enable memory mapped access to ethernet device for linux

From what you said, you basically changed the behavior of the snaplen 
parameter of pcap_open_live(). At the risk of being annoing, I find it a 
really bad idea. If it's called snaplen, it's the snaplen, period.
Isn't it possible to add a pcap function to set such parameter (or 
eventually create a new pcap_open_live_mmap() function)?

Have a nice day
GV

----- Original Message ----- 
From: "Paolo Abeni" <paolo.abeni <at> telecomitalia.it>
To: <tcpdump-workers <at> lists.tcpdump.org>
Cc: <simon <at> patarin.info>; <cpw <at> lanl.gov>
Sent: Wednesday, December 05, 2007 3:11 AM
Subject: [tcpdump-workers] [PATCH] enable memory mapped access to ethernet 
device for linux

hi list,

Following the discussion about the memory mapped access for Linux, I
tried to rework some existing code (originally independently created by
Simon Patarin and Phil Wood) to enable this functionality with the
minimum impact on the current pcap code.

It does not use environment variables to control the memory mapped ring
parameters; instead the requested snap len is used: the low order bytes
are used to select the ring frame size and the high order bytes are used
to select the ring frame number. If the high order bytes is 0, like in
every current libpcap usage, a reasonable default is used.

Re: [PATCH] enable memory mapped access toethernet device for linux

hello,

On Wed, 2007-12-05 at 08:23 -0800, Gianluca Varenni wrote:
> From what you said, you basically changed the behavior of the snaplen 
> parameter of pcap_open_live(). At the risk of being annoing, I find it a 
> really bad idea. If it's called snaplen, it's the snaplen, period.
> Isn't it possible to add a pcap function to set such parameter (or 
> eventually create a new pcap_open_live_mmap() function)?

The idea behind the snaplen [miss-]use is to permit legacy application
to take advantage of the memory mapped access and permit new ones to
control the ring behaviur. Such goal could be obtained even with other
solution, but I was a bit 'scared' to add a 'Linux only' API to pcap.h.

Please note that any value into the [0-65535] range provided to
pcap_open_live into the snaplen parameter is used exactly in the same
way as before.

Undoubtedly my implementation is not the most clean one; I can rework
the patch adding a function to control the ring size [something like
'pcap_set_ring_size(handle, ring_size)'] and using some default values
for the memory mapped ring during pcap_open_live, if there is some
consensus in this direction.

Thanks for your feedback,

Paolo
--------------------------------------------------------------------

CONFIDENTIALITY NOTICE

Re: [PATCH] enable memory mapped access toethernet

> 
> The idea behind the snaplen [miss-]use is to permit legacy application
> to take advantage of the memory mapped access and permit new ones to
> control the ring behaviur. Such goal could be obtained even with other
> solution, but I was a bit 'scared' to add a 'Linux only' API to pcap.h.

IMHO that's not an issue. Furthermore we have a student working on an
MMAP-pcap for FreeBSD as his master thesis. So it won't be a Linux only
API for long 

> Undoubtedly my implementation is not the most clean one; I can rework
> the patch adding a function to control the ring size [something like
> 'pcap_set_ring_size(handle, ring_size)'] and using some default values
> for the memory mapped ring during pcap_open_live, if there is some
> consensus in this direction.

Using a function to set the ring buffer size sounds like the best choice
to me, as long as these parameters can still be changed after the open
or after packets have been read. If you can't change these parameters
during the capture I would introduce a new pcap_open_live_mmap()
function and make the "old" pcap_open_live() call pcap_open_live_mmap()

just my 2ct

Gregor

Re: [PATCH] enable memory mapped access toethernet

I agree with you.

Consider that under windows, for example, we have a windows-only function to 
set the kernel buffer size 
(http://www.winpcap.org/docs/docs_40_2/html/group__wpcapfunc.html#g124bde25ccd9e39017ff2abec2dda623)
and the kernel buffer in WinPcap is actually a ring buffer (although we do 
not use memory mapped access).

Have a nice day
GV

----- Original Message ----- 
From: "Gregor Maier" <gregor <at> net.in.tum.de>
To: <tcpdump-workers <at> lists.tcpdump.org>
Sent: Wednesday, December 05, 2007 10:27 AM
Subject: Re: [tcpdump-workers] [PATCH] enable memory mapped access 
toethernet

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>>
>> The idea behind the snaplen [miss-]use is to permit legacy application
>> to take advantage of the memory mapped access and permit new ones to
>> control the ring behaviur. Such goal could be obtained even with other
>> solution, but I was a bit 'scared' to add a 'Linux only' API to pcap.h.
>
> IMHO that's not an issue. Furthermore we have a student working on an
> MMAP-pcap for FreeBSD as his master thesis. So it won't be a Linux only

Re: [PATCH] enable memory mapped access to ethernet devices

hello,

Gregor Maier wrote:
> Using a function to set the ring buffer size sounds like the best choice
> to me, as long as these parameters can still be changed after the open
> or after packets have been read. 

In the easier implementation I can think of, changing the ring size will cause the loose of the ring content,
but this should be the only issue. 

I assume that the creation of a new function to set the ring size is the preferred choice, so I'll try to rework
the patch that way. Just a question: I suppose that adding said function will require one more function
pointer in the pcap handle structure, right ?

Thanks again for the feedback,

Paolo
--------------------------------------------------------------------

CONFIDENTIALITY NOTICE

This message and its attachments are addressed solely to the persons above and may contain confidential
information. If you have received the message in error, be informed that any use of the content hereof is
prohibited. Please return it immediately to the sender and delete the message. Should you have any
questions, please contact us by replying to webmaster <at> telecomitalia.it.

        Thank you

                                        www.telecomitalia.it

Re: setfilter causes core on Solaris

On Dec 5, 2007, at 5:18 AM, Andy Howell wrote:

> I'm using pcap_dispatch to call my callback. Inside the callback, I  
> may set a new filter. This results in a core dump in bpf_filter.c,  
> line 239. Its calling abort because of a bad filter code. This will  
> only happen with a live capture.
>
> The bug is actually in pcap-dlpi.c. It keeps a pointer to the filter  
> code. Unfortunately the pointer never gets reset as long as there  
> are packets to read. Adding:
>
> fcode = p->fcode.bf_insns;
>
> after the callback returns takes care of the issue. I've attached a  
> patch and posted it as 1844245.

The same problem exists in some other pcap-XXX.c files.  I fixed it by  
getting rid of the fcode variable, and just passing the fcode.bf_insns  
member of the structure.