Nicholas Mooney | 10 Apr 2012 07:14
Picon

nfcapd not seeing packets

Hi

 

I am having trouble receiving flows at nfcapd.

 

I am exporting version 5 netflows (cflow) from a juniper router. I am export them both to my PC running Wireshark and my nfcapd on port 9996. The interval is 5 and there is traffic on the interfaces involved.

 

I simultaneously send the flows to Wireshark on my pc and it decodes them as version 5 flows properly.

 

However, on the nfcapd I see no data being logged. If I run “nfcapd -E -p 9996 -I FW -l /data/nfsen/test/ -s 5” I don’t see any packets logged to STDOUT, even though I simultaneously see the packets hit the server (tcpport port 9996) and also I get the same flows sent to my PC at the same time.

 

All I get is this:

 

[root <at> ausydmon04 test]# nfcapd -E -p 9996 -I FW -l /data/nfsen/test/ -s 5

File Block Header:

  NumBlocks     =           0

  Size          =           0

 id             =           2

 

Any idea where I could be going wrong? I am running nfcapd as root.

 

[root <at> ausydmon04 test]# nfcapd -V

nfcapd: Version: 1.6.6 $Date: 2012-03-11 11:57:45 +0100 (Sun, 11 Mar 2012) $

 

 

Thanks, Nick.


_________________________________________________________________________________________
This email has been scanned by the MessageLabs Email Security System on behalf of Medibank Health Solutions.
For more information please visit http://www.symanteccloud.com
_________________________________________________________________________________________
------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second 
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
Peter Haag | 10 Apr 2012 10:15
Picon

Re: nfcapd not seeing packets

Hi Nick,
I guess you have some packet filters somewhere on your system.
wireshark reads network data at a very low level. System filters
or SElinux features follow up the chain and nfcapd sits on top of all.

This means something blocks your network data somewhere in your network
data chain.

Hope, this help.

	- Peter

On 4/10/12 7:14, Nicholas Mooney wrote:
> Hi
> 
>  
> 
> I am having trouble receiving flows at nfcapd.
> 
>  
> 
> I am exporting version 5 netflows (cflow) from a juniper router. I am export them both to my PC running
Wireshark and my
> nfcapd on port 9996. The interval is 5 and there is traffic on the interfaces involved.
> 
>  
> 
> I simultaneously send the flows to Wireshark on my pc and it decodes them as version 5 flows properly.
> 
>  
> 
> However, on the nfcapd I see no data being logged. If I run “nfcapd -E -p 9996 -I FW -l /data/nfsen/test/ -s
5” I don’t
> see any packets logged to STDOUT, even though I simultaneously see the packets hit the server (tcpport
port 9996) and
> also I get the same flows sent to my PC at the same time.
> 
>  
> 
> All I get is this:
> 
>  
> 
> [root <at> ausydmon04 test]# nfcapd -E -p 9996 -I FW -l /data/nfsen/test/ -s 5
> 
> File Block Header:
> 
>   NumBlocks     =           0
> 
>   Size          =           0
> 
>  id             =           2
> 
>  
> 
> Any idea where I could be going wrong? I am running nfcapd as root.
> 
>  
> 
> [root <at> ausydmon04 test]# nfcapd -V
> 
> nfcapd: Version: 1.6.6 $Date: 2012-03-11 11:57:45 +0100 (Sun, 11 Mar 2012) $
> 
>  
> 
>  
> 
> Thanks, Nick.
> 
> 
> _________________________________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System on behalf of Medibank Health Solutions.
> For more information please visit http://www.symanteccloud.com
> _________________________________________________________________________________________
> 
> 
> This body part will be downloaded on demand.
> 
> 
> 
> This body part will be downloaded on demand.

--

-- 
--
Be nice to your netflow data

------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second 
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev
Nicholas Mooney | 11 Apr 2012 00:38
Picon

Re: nfcapd not seeing packets

Hi Peter

You were right. I installed the same software nfdump software (version
1.6.6) on a development BSD machine and had the flows capturing and
printing with "-E" within a couple of minutes. Must be something about
my first machine filtering packets after tcpdump.

Nick.

-----Original Message-----
From: Peter Haag [mailto:phaag@...] 
Sent: Tuesday, 10 April 2012 6:15 PM
To: Nicholas Mooney
Cc: nfdump-discuss@...
Subject: Re: [Nfdump-discuss] nfcapd not seeing packets

Hi Nick,
I guess you have some packet filters somewhere on your system.
wireshark reads network data at a very low level. System filters or
SElinux features follow up the chain and nfcapd sits on top of all.

This means something blocks your network data somewhere in your network
data chain.

Hope, this help.

	- Peter

On 4/10/12 7:14, Nicholas Mooney wrote:
> Hi
> 
>  
> 
> I am having trouble receiving flows at nfcapd.
> 
>  
> 
> I am exporting version 5 netflows (cflow) from a juniper router. I am 
> export them both to my PC running Wireshark and my nfcapd on port
9996. The interval is 5 and there is traffic on the interfaces involved.
> 
>  
> 
> I simultaneously send the flows to Wireshark on my pc and it decodes
them as version 5 flows properly.
> 
>  
> 
> However, on the nfcapd I see no data being logged. If I run "nfcapd -E

> -p 9996 -I FW -l /data/nfsen/test/ -s 5" I don't see any packets 
> logged to STDOUT, even though I simultaneously see the packets hit the
server (tcpport port 9996) and also I get the same flows sent to my PC
at the same time.
> 
>  
> 
> All I get is this:
> 
>  
> 
> [root <at> ausydmon04 test]# nfcapd -E -p 9996 -I FW -l /data/nfsen/test/ 
> -s 5
> 
> File Block Header:
> 
>   NumBlocks     =           0
> 
>   Size          =           0
> 
>  id             =           2
> 
>  
> 
> Any idea where I could be going wrong? I am running nfcapd as root.
> 
>  
> 
> [root <at> ausydmon04 test]# nfcapd -V
> 
> nfcapd: Version: 1.6.6 $Date: 2012-03-11 11:57:45 +0100 (Sun, 11 Mar 
> 2012) $
> 
>  
> 
>  
> 
> Thanks, Nick.
> 
> 
> ______________________________________________________________________
> ___________________ This email has been scanned by the MessageLabs 
> Email Security System on behalf of Medibank Health Solutions.
> For more information please visit http://www.symanteccloud.com 
> ______________________________________________________________________
> ___________________
> 
> 
> This body part will be downloaded on demand.
> 
> 
> 
> This body part will be downloaded on demand.

--
--
Be nice to your netflow data

_________________________________________________________________________________________
This email has been scanned by the MessageLabs Email Security System on behalf of Medibank Health Solutions.
For more information please visit http://www.symanteccloud.com
_________________________________________________________________________________________

------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second 
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev
Felipe Garcia | 13 Apr 2012 04:16
Favicon

how to crash nfdump

could be used to create an exploit.. this is a redhat 6.1 box.

/usr/local/bin/nfdump -A srcip -N -f /data/etc/srcfilter -A dstip -N -f /data/etc/dstfilter  -q -o fmt:S,%sa,%byt,%pkt,%fl -Rft-IC02RTR001-ipbill0201-2012-04-11.1815+1000.txt:ft-IC02RTR001-ipbill0201-2012-04-11.1845+1000.txt
*** glibc detected *** /usr/local/bin/nfdump: free(): invalid next size (fast): 0x0000000000754270 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3e728750c6]
/usr/local/bin/nfdump[0x415873]
/usr/local/bin/nfdump[0x402711]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x3e7281ecdd]
/usr/local/bin/nfdump[0x401d79]
======= Memory map: ========
00400000-0042c000 r-xp 00000000 fd:00 542091                             /usr/local/bin/nfdump
0062b000-0062f000 rw-p 0002b000 fd:00 542091                             /usr/local/bin/nfdump
0062f000-00775000 rw-p 00000000 00:00 0                                  [heap]
3d59c00000-3d59c16000 r-xp 00000000 fd:00 663737                         /lib64/libgcc_s-4.4.6-20110824.so.1
3d59c16000-3d59e15000 ---p 00016000 fd:00 663737                         /lib64/libgcc_s-4.4.6-20110824.so.1
3d59e15000-3d59e16000 rw-p 00015000 fd:00 663737                         /lib64/libgcc_s-4.4.6-20110824.so.1
3e72400000-3e72420000 r-xp 00000000 fd:00 654531                         /lib64/ld-2.12.so
3e7261f000-3e72620000 r--p 0001f000 fd:00 654531                         /lib64/ld-2.12.so
3e72620000-3e72621000 rw-p 00020000 fd:00 654531                         /lib64/ld-2.12.so
3e72621000-3e72622000 rw-p 00000000 00:00 0
3e72800000-3e72997000 r-xp 00000000 fd:00 654535                         /lib64/libc-2.12.so
3e72997000-3e72b97000 ---p 00197000 fd:00 654535                         /lib64/libc-2.12.so
3e72b97000-3e72b9b000 r--p 00197000 fd:00 654535                         /lib64/libc-2.12.so
3e72b9b000-3e72b9c000 rw-p 0019b000 fd:00 654535                         /lib64/libc-2.12.so
3e72b9c000-3e72ba1000 rw-p 00000000 00:00 0
3e75000000-3e75016000 r-xp 00000000 fd:00 661916                         /lib64/libresolv-2.12.so
3e75016000-3e75216000 ---p 00016000 fd:00 661916                         /lib64/libresolv-2.12.so
3e75216000-3e75217000 r--p 00016000 fd:00 661916                         /lib64/libresolv-2.12.so
3e75217000-3e75218000 rw-p 00017000 fd:00 661916                         /lib64/libresolv-2.12.so
3e75218000-3e7521a000 rw-p 00000000 00:00 0
7fc73f6bd000-7fc73f6c0000 rw-p 00000000 00:00 0
7fc73f6c7000-7fc73f6c9000 rw-p 00000000 00:00 0
7fff6e3e3000-7fff6e3f8000 rw-p 00000000 00:00 0                          [stack]
7fff6e3ff000-7fff6e400000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
Peter Haag | 15 Apr 2012 08:28
Picon

Re: how to crash nfdump

Thanks for the notification. The crash results from giving twice the -A argument which nfdump was not
supposed to get.
Find appended a patch, which solves this. nfdump now accepts one or more -A arguments.
Although it's a memory corruption, I'm not sure how easy this could be exploited, as no user arguments
directly are
copied into the corrupted memory. Anyway, it's fixed.

Thanks

	- Peter

On 13/4/12 4:16 AM, Felipe Garcia wrote:
> could be used to create an exploit.. [?] this is a redhat 6.1 box.
> 
> /usr/local/bin/nfdump -A srcip -N -f /data/etc/srcfilter -A dstip -N -f
> /data/etc/dstfilter  -q -o fmt:S,%sa,%byt,%pkt,%fl
> -Rft-IC02RTR001-ipbill0201-2012-04-11.1815+1000.txt:ft-IC02RTR001-ipbill0201-2012-04-11.1845+1000.txt
> *** glibc detected *** /usr/local/bin/nfdump: free(): invalid next size
> (fast): 0x0000000000754270 ***
> ======= Backtrace: =========
> /lib64/libc.so.6[0x3e728750c6]
> /usr/local/bin/nfdump[0x415873]
> /usr/local/bin/nfdump[0x402711]
> /lib64/libc.so.6(__libc_start_main+0xfd)[0x3e7281ecdd]
> /usr/local/bin/nfdump[0x401d79]
> ======= Memory map: ========
> 00400000-0042c000 r-xp 00000000 fd:00 542091
> /usr/local/bin/nfdump
> 0062b000-0062f000 rw-p 0002b000 fd:00 542091
> /usr/local/bin/nfdump
> 0062f000-00775000 rw-p 00000000 00:00 0
> [heap]
> 3d59c00000-3d59c16000 r-xp 00000000 fd:00 663737
> /lib64/libgcc_s-4.4.6-20110824.so.1
> 3d59c16000-3d59e15000 ---p 00016000 fd:00 663737
> /lib64/libgcc_s-4.4.6-20110824.so.1
> 3d59e15000-3d59e16000 rw-p 00015000 fd:00 663737
> /lib64/libgcc_s-4.4.6-20110824.so.1
> 3e72400000-3e72420000 r-xp 00000000 fd:00 654531
> /lib64/ld-2.12.so
> 3e7261f000-3e72620000 r--p 0001f000 fd:00 654531
> /lib64/ld-2.12.so
> 3e72620000-3e72621000 rw-p 00020000 fd:00 654531
> /lib64/ld-2.12.so
> 3e72621000-3e72622000 rw-p 00000000 00:00 0
> 3e72800000-3e72997000 r-xp 00000000 fd:00 654535
> /lib64/libc-2.12.so
> 3e72997000-3e72b97000 ---p 00197000 fd:00 654535
> /lib64/libc-2.12.so
> 3e72b97000-3e72b9b000 r--p 00197000 fd:00 654535
> /lib64/libc-2.12.so
> 3e72b9b000-3e72b9c000 rw-p 0019b000 fd:00 654535
> /lib64/libc-2.12.so
> 3e72b9c000-3e72ba1000 rw-p 00000000 00:00 0
> 3e75000000-3e75016000 r-xp 00000000 fd:00 661916
> /lib64/libresolv-2.12.so
> 3e75016000-3e75216000 ---p 00016000 fd:00 661916
> /lib64/libresolv-2.12.so
> 3e75216000-3e75217000 r--p 00016000 fd:00 661916
> /lib64/libresolv-2.12.so
> 3e75217000-3e75218000 rw-p 00017000 fd:00 661916
> /lib64/libresolv-2.12.so
> 3e75218000-3e7521a000 rw-p 00000000 00:00 0
> 7fc73f6bd000-7fc73f6c0000 rw-p 00000000 00:00 0
> 7fc73f6c7000-7fc73f6c9000 rw-p 00000000 00:00 0
> 7fff6e3e3000-7fff6e3f8000 rw-p 00000000 00:00 0
> [stack]
> 7fff6e3ff000-7fff6e400000 r-xp 00000000 00:00 0
> [vdso]
> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
> [vsyscall]
> Aborted (core dumped)
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> For Developers, A Lot Can Happen In A Second.
> Boundary is the first to Know...and Tell You.
> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> http://p.sf.net/sfu/Boundary-d2dvs2
> 
> 
> 
> _______________________________________________
> Nfdump-discuss mailing list
> Nfdump-discuss@...
> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss

-- 
Be nice to your netflow data. Use NfSen and nfdump :)
--- nflowcache.c.orig	2012-04-15 08:23:00.000000000 +0200
+++ nflowcache.c	2012-04-15 08:23:35.000000000 +0200
 <at>  <at>  -637,18 +637,22  <at>  <at> 
 	fmt_len = 0;
 	i = 0;
 	while ( aggregate_info[i].aggregate_token != NULL ) {
+		if ( aggregate_info[i].active )
+			stack_count++;
 		if ( aggregate_info[i].fmt )
 			fmt_len += ( strlen(aggregate_info[i].fmt) + 1 );
 		i++;
 	}
 	fmt_len++;	// trailing '\0'

-	*aggr_fmt = malloc(fmt_len);
+	if ( !*aggr_fmt ) {
+		*aggr_fmt = malloc(fmt_len);
+		(*aggr_fmt)[0] = '\0';
+	}
 	if ( !*aggr_fmt ) {
 		fprintf(stderr, "malloc() error in %s line %d: %s\n", __FILE__, __LINE__, strerror (errno));
 		return 0;
 	}
-	(*aggr_fmt)[0] = '\0';

 
 	FlowTable.apply_netbits  = 0;
 <at>  <at>  -712,7 +716,9  <at>  <at> 
 		while ( a->aggregate_token && (strcasecmp(p, a->aggregate_token ) != 0) )
 			a++;

-		if ( a->aggregate_token != NULL ) {
+		if ( a->active ) {
+			fprintf(stderr, "Skip already given aggregation mask: %s\n", p);
+		} else if ( a->aggregate_token != NULL ) {

 			if ( a->fmt != NULL ) {
 				strncat(*aggr_fmt, a->fmt, fmt_len);
------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
Vincent Magnin | 26 Apr 2012 14:15
Picon
Picon
Favicon

nfdump and flow tagging

Dear Peter and nfdump users,

Is it possible to use nfdump to tag flows for custom aggregation?

Example:
- if a flow srcport or dstport is 80, then tag it as HTTP
- if a flow srcport or dstport is 443, then tag it as HTTPS
- ... (same tagging for a long list of services)
- Then aggregate all these flows by tag (to obtain a list of n top  
used services)

Regards,

Vincent

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Gmane