nftest usage

Dear all,

In the mailing list archive of March 2008 I found a message talking about "compression in 1.5.7": http://comments.gmane.org/gmane.network.nfsen.general/975

I'm interested in comparing nfdump performance in a setup with and within compression enabled. The
message linked above suggests that nftest could help me to find this out. The message also says "nftest is
only in the src directory and built, when doing a 'make test'". However, I haven't got it to compile, yet.
When I run 'make test', make complains about a missing 'test' target.

Does anyone know how to use nftest?

Best regards,

--
Rick Hofstede
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev

Filter to only see ICMP type 3, code 4 messages

I'm using Nfsen and am trying to view flows that contain icmp type 3 
code 4 messages (Needs fragmenting but DF bit set).

I tried

proto ICMP and icmp[0]=3 and icmp[1]=4

but it doesn't appear to work. Is there a specific filter for that?

-- 
Regards,

Giles Coochey, CCNP, CCNA, CCNAS
NetSecSpec Ltd
+44 (0) 7983 877438
http://www.coochey.net
http://www.netsecspec.co.uk
giles@...
	

------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. A cloud service to automate IT design, transition and operations
2. Dashboards that offer high-level views of enterprise services
3. A single system of record for all IT processes
http://p.sf.net/sfu/servicenow-d2d-j

Fwd: Re: nfdump on Centos 6 problem

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may

_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss

nfdump on Centos 6 problem

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may

_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss

nfdump cluster question

Has anyone written a method for distributing nfdump query processing to a compute cluster?
We have a compute cluster (condor based) that I'd like to utilize when processing nfdump data.

>From what I understand, some sort of wrapper would be required that would instruct N number of compute
nodes to map certain data files, process them, return values, then another process would aggregate the
values based on the original request and generate the output the user was looking for.

Perhaps I'm thinking about this all wrong?

Thoughts? Comments?

/Ryan

Ryan Harden
Senior Network Engineer
University of Chicago - AS160
P: 773-834-5441

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

nfdump-1.6.10 released

Hi List,
Just to let you know - 1.6.10 is out:

Maintainance/bugfix release.
You should update, if you use IPFIX, or ASA/NSEL

- Fix SPARC compile/optimise bug
- Add output packet/bytes counter to global stat for NSEL flows ASA > 8.5
  Fixes stat problems in NfSen
- Add NSEL filter options xnet
- Modify extension descriptor code for nfdump1.7.
  Still use 1.6 extension map layout for compatibility
- Add prototype for nfpcapd - pcap -> nfdump collector.
  Converts traffoc directly to nfdump files. - experimental - not enabled
- Fix bug in ipfix module: uninitialised variable
- Cleanup syslog/LogError calls
- Fix minor non critical bugs and compile issues

	- Peter
--

-- 
--
Be nice to your netflow data

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

[nfdump 1.6.9] -- Dst IP option remove IP SOURCE

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss

Perl module for working with nfdump files

Hi nfdump community,

    as the nfdump is a great tool we desperately missed the additional
functionality allowed us either modify or create the records in those
files. So we decided to develop own perl module. After using it in our
several projects we decided to publish the module to whole community.
The module is available via CPAN site 
(http://search.cpan.org/~tpoder/Net-NfDump/lib/Net/NfDump.pm). It uses
standard DBI-like conventions, so it should be quite easy to use.  Maybe
someone will find it useful for yourself.

Best regards
    Tomas

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

vSphere 5.1 distributed switch to nfcapd with IPFIX

Hi,
     I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server.  (nfsen v 1.3.5)

I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter.

I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice
that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of
the packet in "received at".

NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are
formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time.

I notice this has been raised on various sites but I have not seen a fix.  I don't mind testing some patches if
they become available to fix up this timestamp issue.

# nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R
2013/05/03/nfcapd.201305031040 -c 100 -o raw

Flow Record: 
  Flags        =              0x06 FLOW, Unsampled
  export sysid =                 2
  size         =                72
  first        =                 0 [1970-01-01 10:00:00]
  last         =                 0 [1970-01-01 10:00:00]
  msec_first   =                 0
  msec_last    =                 0
  src addr     =    110.175.94.222
  dst addr     =      192.168.64.6
  src port     =             58464
  dst port     =               443
  fwd status   =               157
  tcp flags    =              0x00 ......
  proto        =                 6
  (src)tos     =                 0
  (in)packets  =                 9
  (in)bytes    =              1500
  input        =              1678
  output       =              1799
  ip router    =         10.1.4.39
  received at  =     1367541600163 [2013-05-03 10:40:00.163]

Flow Record: 
  Flags        =              0x06 FLOW, Unsampled
  export sysid =                 2
  size         =                72
  first        =                 0 [1970-01-01 10:00:00]
  last         =                 0 [1970-01-01 10:00:00]
  msec_first   =                 0
  msec_last    =                 0
  src addr     =     101.163.67.76
  dst addr     =      192.168.64.6
  src port     =              2735
  dst port     =               443
  fwd status   =               255
  tcp flags    =              0x00 ......
  proto        =                 6
  (src)tos     =                 0
  (in)packets  =                 1
  (in)bytes    =                40
  input        =              1678
  output       =              1799
  ip router    =         10.1.4.39
  received at  =     1367541600163 [2013-05-03 10:40:00.163]

Kind Regards,
                          David
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may

UNIX timestamps without using '-o pipe'

Dear Peter,

It would be really useful to have an nfdump output format (e.g. to use with '-o fmt:') that allows to output
UNIX timestamps for flow record start and end times. To the best of my knowledge, outputting UNIX
timestamps is currently solely possible using '-o pipe'. However, this output format provides way more
information than we need.

We plan to use this in an NfSen plugin, where we would like to parse a few fields from the nfdump files,
including start and end times as UNIX timestamps.

Would it be possible to include something like this in an upcoming version of nfdump?

Kind regards,

--
Rick Hofstede
University of Twente, The Netherlands
------------------------------------------------------------------------------
Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1

nfdump extension problem

Dear all,

I'm trying to implement an extension for nfdump that stores and processes data from
(enterprise-specific) NetFlow fields. In total, I've defined 6 fields and all but one (i.e. no. 2) are
working fine. The fields have the following properties:

1: unint8
2: unint32 <== problem
3-6: uint32

Since the extension has to be 64-bit-aligned, I've defined the corresponding masks:

1: 0xFF00000000000000LL
2: 0x00FFFFFFFF000000LL
3: 0xFFFFFFFF00000000LL
4: 0x00000000FFFFFFFFLL
5: 0xFFFFFFFF00000000LL
6: 0x00000000FFFFFFFFLL

Again, field 1, 3, 4, 5 and 6 are working perfectly fine. The only difference between field no. 2 and the
others is that its shifted by another value than 32 bits. It needs to carry a UNIX timestamp, which is
correctly 'received' by nfcapd (I've verified that). However, after reading the file by nfdump, the
value is wrong. For example, the HEX value '516C5DE4' (1366056420) results in the decimal value after
processing '81'.

Does anyone of you have a clue what may be the (direction of the) problem? Any idea is appreciated!

Kind regards,

--
Rick Hofstede
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter