Re: NFDUMP-1.6.4 and ASA problem NSEL

Hi Adam,
For CISCO ASA please use nfdump-1.5.8-NSEL. AT some point, it will get integrated into 1.6, however the 1.6 features
apply mostly to v9 and FNF which is not an issue for NSEL AA, as this is different anyway. For NfSen

NfSen work up to 1.3.2. If you use any newer version apply this patch:

 <at>  <at>  -311,7 +288,7  <at>  <at> 
                my $channellist  = join ':', keys %{$profileinfo{'channel'}};
                my $subdirlayout = $NfConf::SUBDIRLAYOUT ? "-S $NfConf::SUBDIRLAYOUT" : "";
                my $arg = "-I -t $timeslot -p $NfConf::PROFILEDATADIR -P $NfConf::PROFILESTATDIR $subdirlayout
$NfConf::ZIPprofiles";
-               my $flist = "-L $NfConf::syslog_facility -M $NfConf::PROFILEDATADIR/live/$channellist -r nfcapd.$t_iso";
+               my $flist = "-M $NfConf::PROFILEDATADIR/live/$channellist -r nfcapd.$t_iso";

It backports/removes some newer options of nfprofile. The rest should pretty much work. I will address
this in NfSen 1.3.6

	- Peter

On 9/9/11 9:27, Adam Gill wrote:
> Hi,
> 
> I have a problem with version nfdump-1.6.4.
> Its not support my ASA.
> nfdump get wrong time stamp and all packets are the same size 2.6M.
> 
> When i used version nfsdump-1.5.8-NSEL all data (time stamp, packets) are correct, but problem is with nfsen.
> nfsen does not work profile (ERR Error reading channel stat information. Missing key 'first') and do not
work alerts and bi-directional.
> 

Re: custom CSV output format

Hi Vincent,
nfdump has a fixed CSV format. However, this should be pretty easy to adapt. You will also find a perl stub,
which reads
the format for further processing. This should make it possible to use it right away.

Regards

	- Peter

On 9/13/11 9:46, Vincent Magnin wrote:
> Dear list,
> Dear Peter,
> 
> Is it possible to use nfdump to display flows in a custom CSV format?
> 
> This feature exists with flow-tools:
> 
>> flow-cat /var/flow-tools/data/2011-09-12 |flow-export -f2 -m  
>> doctets,srcaddr,dstaddr,srcport,dstport,prot  
>> #:doctets,srcaddr,dstaddr,srcport,dstport,prot
>> 46,aaa.aaa.aaa.aaa,bbb.bbb.bbb.bb,80,19263,6
>> 99,aaa.aaa.a.aa,bbb.bb.bbb.b,5759,53,17
>> 149,aa.aa.a.aaa,bbb.bbb.bbb.bbb,3750,6257,17
>> ...
> 
> 
> ie.:
> 
>> ./bin/nfdump -R /var/nfdump/data/2011-09-12 -o  
>> "csv:%ibyt,%sa,%da,%sp,%dp,%pr"

Re: Few questions

Hi Greg,

Sorry for the delay. I've been away for holiday and business for several weeks.

As for your question:
First of all, which nfdump version are you using?

I can not really reproduce your findings.  Using lists in nfdump is optimised and should have only little impact.

Example:
nfdump -r /.../nfcapd.201110031000 'not any'
Date flow start          Duration Proto      Src IP Addr:Port          Dst IP Addr:Port   Packets    Bytes Flows
Summary: total flows: 0, total bytes: 0, total packets: 0, avg bps: 0, avg pps: 0, avg bpp: 0
Total flows processed: 3488792, Blocks skipped: 0, Bytes read: 183063912
Sys: 0.564s flows/second: 6185417.6  Wall: 0.577s flows/second: 6040832.2

This is the throughput for nfdump on this host for just reading a file of 68MB in size compressed with 3.5 Mio flows
and denying each flow with a filter. The same can be done with a list filter, which uses a bit more cpu but with
only a
single entry:

nfdump -r /.../nfcapd.201110031000 'ip in [127.0.0.0/24]'
Date flow start          Duration Proto      Src IP Addr:Port          Dst IP Addr:Port   Packets    Bytes Flows
Summary: total flows: 0, total bytes: 0, total packets: 0, avg bps: 0, avg pps: 0, avg bpp: 0
Total flows processed: 3488792, Blocks skipped: 0, Bytes read: 183063912
Sys: 0.660s flows/second: 5285728.1  Wall: 0.661s flows/second: 5277556.4

There is only little difference in speed and throughput.
Lists are pretty much efficient and 5k entries do not really slow down much. Much bigger impact have your system
resources such as I/O throughput and memory availability.

Re: custom CSV output format

Hi Peter,

I've written a custom flow_record_to_csv function which output exactly  
what I need. It was pretty easy as the source code is well documented.

Regards,

Vincent

Peter Haag <phaag@...> a écrit :

> Hi Vincent,
> nfdump has a fixed CSV format. However, this should be pretty easy  
> to adapt. You will also find a perl stub, which reads
> the format for further processing. This should make it possible to  
> use it right away.
>
> Regards
>
> 	- Peter
>
>
> On 9/13/11 9:46, Vincent Magnin wrote:
>> Dear list,
>> Dear Peter,
>>
>> Is it possible to use nfdump to display flows in a custom CSV format?
>>
>> This feature exists with flow-tools:
>>

Invalid Time window

Hi all,

we have noticed the problem regarding the time window setting in nfdump.
No matter the settings of original nfdump command the resulting time
window is often set to:

Time window: 2038-01-19 04:14:07 - 1970-01-01 01:00:00

without apparent cause. From our point of view this behaviour is more or
less random, but we often spot this when the result of the query is
empty (no flows). An example is attached.

Currently we use:

nfdump: Version: 1.6.4 $Date: 2011-07-19 12:43:31 +0200 (Tue, 19 Jul 2011) $

but we encountered this problem in latest versions as well.

Does anybody know how to fix it?

Best regards,

Tomas Plesnik
CSIRT-MU

--

-- 
Tomas Plesnik                                       plesnik@...
CSIRT-MU, Network Security Department	       http://www.muni.cz/csirt
Institute of Computer Science, Masaryk University, Brno, Czech Republic

Total Link utilization

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct

_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss

juniper inline-jflow

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct

_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss

Re: juniper inline-jflow

On Monday, October 17, 2011, Nitzan Tzelniker wrote:

> does nfdump support juniper inline-jflow (ipfix for trio based cards ) 
> http://www.juniper.net/techpubs/en_US/junos11.3/topics/task/configuration/inline-flow-monitoring.html

JunOS supported netflow v9 since JunOS v9... I have not tried it but 
plan to try in near future.

--

-- 
Regards,
Sergey

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct

Filter from command line

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning <at> Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev

_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss

Re: custom CSV output format

Hi Peter,

On Mon, Oct 03, 2011 at 12:29:44PM +0200, Peter Haag wrote:
> nfdump has a fixed CSV format. However, this should be pretty easy to adapt. You will also find a perl stub,
which reads
> the format for further processing. This should make it possible to use it right away.

Where do we find the perl stub that you mention?

Thanks,
Dave

--

-- 
plonka@...  http://net.doit.wisc.edu/~plonka/  Madison, WI

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning <at> Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev