Discussion of the NFSen Netflow Sensor

Alexandro Marcelo Zacaron | 24 May 06:58

Frontend - Backend version missmatch!

Hi,

I'm instaled:
- nfdump 1.6.6 with options:
./configure \--prefix=/usr \--sysconfdir=/etc \--mandir=/usr/share/man
\--enable-nfprofile \--enable-nftrack \--enable-sflow

in sequence nfsen 1.3.6p1

when access web is show the message:
Frontend - Backend version missmatch!

Then found the link below:
http://sourceforge.net/mailarchive/forum.php?thread_name=CC36D0621A8F654D988D6C3542FF6ADC0434FC356C%40EXCHANGE2.grove.ad.uconn.edu&forum_name=nfsen-discuss

but when change the options of link above, my nfsen in browser don't
display nothing.

I'm using SUSE 64 bits as virtual machine in VMware, is there any
particularity in case to virtual machines or specific configuration or
the version specific for NFDUMP and NfSen in case 64bits?

thanks,

--

-- 
Alexandro Marcelo Zacaron
+55 45 9942 8561

------------------------------------------------------------------------------
Live Security Virtual Conference

(Continue reading)

headers

John Elliot | 24 May 00:46

Another question on flow-tools -> nfsen/nfdump migration.

Hi Guys,

We often receive requests from EC's to provide traffic analysis when there usuage is "abnormal"

Typically, with flow-tools it is analysis of a days flow data (24hours), and we provide:

Total Octets

Top port usage

Top src/dst IP

With flow-tools, we create a specific acl to only provide analysis on an EC's IP(could be a /32 or larger subnet)

Is the following the correct way to provide similar reports in nfdmp? (i.e. No acl, all inclusions/exclusions are added in command line?)

nfdump -R /data/nfsen/profiles-data/live/ASR1006/2012/05/21/ 'dst net 10.1.1.0/24' -s dstip/bytes -s port/bytes -s record/bytes -n 20| more

Thanks in advance.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss

headers

Alexandro Marcelo Zacaron | 18 May 18:17

error install nfsen (Lookup and AbuseWhois)

Hello,

I have problem in the execution of NfSen as below
----------------------------------------------------------------------------------
/usr/local/nfsen/bin#./nfsen start
Subroutine Lookup::pack_sockaddr_in6 redefined at
/usr/lig/perl5/5.14.2/Exporter.pm line 67.
at /usr/local/nfsen/libexec/Lookup.pm line 43
Subroutine Lookup::unpack_sockaddr_in6 redefined at
/usr/lig/perl5/5.14.2/Exporter.pm line 67.
at /usr/local/nfsen/libexec/Lookup.pm line 43
Subroutine Lookup::sockaddr_in6 redefined at
/usr/lig/perl5/5.14.2/Exporter.pm line 67.
at /usr/local/nfsen/libexec/Lookup.pm line 43
Subroutine AbuseWhois::pack_sockaddr_in6 redefined at
/usr/lig/perl5/5.14.2/Exporter.pm line 67.
at /usr/local/nfsen/libexec/AbuseWhois.pm line 42
Subroutine AbuseWhois::unpack_sockaddr_in6 redefined at
/usr/lig/perl5/5.14.2/Exporter.pm line 67.
at /usr/local/nfsen/libexec/AbuseWhois.pm line 42
Subroutine AbuseWhois::sockaddr_in6 redefined at
/usr/lig/perl5/5.14.2/Exporter.pm line 67.
at /usr/local/nfsen/libexec/AbuseWhois.pm line 42
Subroutine AbuseWhois::pack_sockaddr_in6 redefined at
/usr/local/nfsen/libexec/AbuseWhois.pm line 44
Subroutine AbuseWhois::unpack_sockaddr_in6 redefined at
/usr/local/nfsen/libexec/AbuseWhois.pm line 44
Subroutine AbuseWhois::sockaddr_in6 redefined at
/usr/local/nfsen/libexec/AbuseWhois.pm line 44
NfSen is alredy running!
----------------------------------------------------------------------------------

The browser return the message Frontend - Backend version missmatch!
or sometimes works, but not present flows, zero records..

The files are being written, but empty.

I'm using openSUSE 12.1 (64), nfdump-1.6.6, NfSen-1.3.6p1

output of instalation:
----------------------------------------------------------------------------------
Check for required Perl modules: All modules found.
Upgrade from version '1.3.6p1' installed at Wed May 16 21:05:19 2012
Setup NfSen:
Version: 1.3.6p1: $Id: install.pl 53 2012-01-23 16:36:02Z peter $

Perl to use: [/usr/bin/perl] Found /usr/bin/nfdump: Version: 1.6.6
$Date: 2012-03-11 11:57:45 +0100 (Sun, 11 Mar 2012) $
Setup php and html files.

Copy NfSen dirs etc bin libexec plugins doc ...
Copy config file 'etc/nfsen.conf'

In directory: /usr/local/nfsen/libexec ...
Update script: AbuseWhois.pm
Update script: Log.pm
Update script: Lookup.pm
Update script: NfAlert.pm
Update script: Nfcomm.pm
Update script: NfConf.pm
Update script: NfProfile.pm
Update script: NfSen.pm
Update script: NfSenRC.pm
Update script: NfSenRRD.pm
Update script: NfSenSim.pm
Update script: Nfsources.pm
Update script: Nfsync.pm
Update script: Notification.pm
In directory: /usr/local/nfsen/bin ...
Update script: nfsen
Update script: nfsend
Update script: RebuildHierarchy.pl
Update script: testPlugin

Cleanup old files ...

Setup diretories:

Use UID/GID 1001 8
Exists: /usr/local/nfsen/var
Exists: /usr/local/nfsen/var/tmp
Exists: /usr/local/nfsen/var/run
Exists: /usr/local/nfsen/var/filters
Exists: /usr/local/nfsen/var/fmt
Exists: /usr/local/nfsen/profiles-stat
Exists: /usr/local/nfsen/profiles-stat/live
Exists: /usr/local/nfsen/profiles-data
Exists: /usr/local/nfsen/profiles-data/live

Profile live: spool directories:
Exists: gw-ipfix
Rename gif RRDfiles ... done.
RRD DB 'gw-ipfix.rrd' already exists!
Use existing profile info for profile 'live'

Reconfig: No changes found!
Setup done.

* You may want to subscribe to the nfsen-discuss mailing list:
* http://lists.sourceforge.net/lists/listinfo/nfsen-discuss
* Please send bug reports back to me: phaag@...
----------------------------------------------------------------------------------

I hope someone can help me

thanks

--

-- 
Alexandro Marcelo Zacaron
+55 45 9942 8561

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

headers

Nikolaos Milas | 17 May 14:53

logrotate

Hello,

Is there a suggested logrotate script for nfsen (on CentOS 5.8) ?

I was thinking of something like:

/var/log/nfsen.log {
     daily
     rotate 10
     missingok
     notifempty
     sharedscripts
     postrotate
         /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> 
/dev/null || true
         /bin/kill -HUP `cat /var/run/rsyslogd.pid 2> /dev/null` 2> 
/dev/null || true
     endscript
}

Would this be OK, or would it also be necessary to restart nfsen (to 
avoid possible hangs) ?

         if test -n "`ps acx|grep nfsend`"; then
                 /data/nfsen/bin/nfsend stop
                 /data/nfsen/bin/nfsend start
         fi

Any experiences?

Thanks,
Nick

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

headers

Mohamed Elzaki | 16 May 11:50

Problem with a channel deleting!

Hello,

I have something wrong with deleting any channel !
When I click "Edit channel" -> "Delete channel" -> it shows (Are you sure to delete channel 'DownLink') -> OK ... Then, the page refreshed, and nothing happened!
The channel is still exists!

Would you please advise as necessary?

Thanks in advance,
Conan

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss

headers

John Elliot | 16 May 07:17

flow-tools -> nfdump/nfsen

Hi,

We currently use a number of flow-tools servers, and are looking to migrate to nfdump/nfsen due to lack of development of flow-tools(It has served us well for 10years)

We predominantly use flow-tools for IP billing, and basic traffic analysis.

With our current flow-tools deployments, we store 40Gb of historic flow data (./flow-capture -w /netflow/oar/krc3.v5 -E40G ...), once the flow data reaches 40Gb in this dir the oldest data is removed/deleted - Is this housekeeping feature available in nfcapd? (The 40G gives us ~1month of raw flow data history if we need to perform traffic analysis for a client)

We also run a cron job every morning just after midnight, that dumps the previous 24 hours flow data into the following file format:

# src IPaddr dst IPaddr flows octets packets

We then import this into sql/billing system

Can nfdump produce something "similar" to this? (And is it possible to have the flow data directory structure as /YYYY/MM/DD/flow data in 5 or 10 min file?)

Thanks in advance.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss

headers

djwil mer | 14 May 23:33

Time Window issue

Today I created a time window request from 7:00 to 7:10 but for some reason there are entries from 3/25/2012 as the start date. Why is this happening? Is this some type of bug?

Thanks.

** nfdump -M /var/nfsen/profiles-data/test/test1:test2:test2:test1 -T -R 2012/05/14/nfcapd.201205140700:2012/05/14/nfcapd.201205140710 -n 10 -s record/flows
nfdump filter:
ip 10.10.30.3
Aggregated flows 368
Top 10 flows ordered by flows:
Date flow start          Duration Proto      Src IP Addr:Port          Dst IP Addr:Port   Packets    Bytes Flows
2012-05-14 06:59:33.262   865.084 TCP      10.10.25.163:3322 ->      10.10.30.3:44574       57     4707    11
2012-05-14 06:59:33.325   865.149 TCP        10.10.30.3:44574 ->    10.10.25.163:3322        57     4872    11
2012-05-14 07:00:00.010   781.825 TCP        10.10.30.3:58997 ->   10.10.78.151:3750      4473    1.2 M    11
2012-03-25 13:58:27.496 4295747.373 TCP      10.10.78.59:3714 ->      10.10.30.3:49502       45     4140    10
2012-03-25 13:57:12.743 4295749.100 TCP     10.10.78.151:3750 ->      10.10.30.3:58997     2632   852472    10
2012-05-14 06:59:44.719   870.268 TCP        10.10.30.3:49502 ->    10.10.78.59:3714        59     5054    10
2012-03-25 13:57:12.935 4295749.102 TCP        10.10.71.54:3778 ->      10.10.30.3:64588       30     2375     7
2012-05-14 07:00:00.203   781.889 TCP        10.10.30.3:64588 ->      10.10.71.54:3778        30     2425     7
2012-05-14 07:09:01.641     0.000 TCP      10.10.25.170:3322 ->      10.10.30.3:47887        1       46     1
2012-05-14 07:13:53.545     0.000 TCP      10.10.25.170:3322 ->      10.10.30.3:55910        1       46     1

Summary: total flows: 437, total bytes: 2.1 M, total packets: 7745, avg bps: 3, avg pps: 0, avg bpp: 272
Time window: 2012-03-25 13:57:12 - 2012-05-14 07:14:14
Total flows processed: 437, Blocks skipped: 0, Bytes read: 23252
Sys: 0.004s flows/second: 109250.0 Wall: 0.002s flows/second: 211724.8

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss

headers

Mallow, Christopher R. | 14 May 22:45

Busted PMs with Ubuntu dist upgrade

Greetings,

Just an FYI regarding something I found today. I had initially installed on Ubuntu 11.10 and thus my install was running perl 5.12 with rrdtool 1.4.7. Ran the upgrade to 12.04 LTS today only to discover that the RRDs.pm module ended up broken. The issue, apparently, is that just enough has changed in perl 5.14.2 (which is what comes on 12.04) that the previously compiled stuff wasn’t happy. (Also note that I had to custom-compile RRDtool originally because I couldn’t get the version that came via apt-get to function with nfcap/nfsen, for some reason.)

Long story short, I flushed all of the old RRDtool install and recompiled 1.4.7 from source and this repaired it. I don’t know if anyone else had run into this, but just in case…

-------------------------------------------------------------------

Chris R. Mallow, CISSP, GCFA, ACE

IT Forensic Analyst - CSIRT

The University of Oklahoma

O: 405.325.4991

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss

headers

Simon Woodhead | 27 Apr 14:00

Porttracker plugin - graphs and data, but not together!

Hi folks,

We have the Porttracker plugin installed on 1.3.6p1 but it isn't quite working.

We have data in the tables and we have graphs with correctly identified top ports showing (as attached). We just have no data on the graphs!

Any pointers appreciated.

cheers

Simon

-- ***** Email confidentiality notice ***** This message is private and confidential. If you have received this message in error, please notify us and remove it from your system. Simwood eSMS Limited is a limited company registered in England and Wales. Registered number: 03379831. Registered office: c/o HW Chartered Accountants, Keepers Lane, The Wergs, Wolverhampton, WV6 8UA. Trading address: Falcon Drive, Cardiff Bay, Cardiff, CF10 4RU.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss

headers

johan lotter | 25 Apr 22:17

graphing individuals usage per ip address

Hi

I have nfsen gathering stats from my Mikrotik Router but would like to
drill down to get usage per user ip on my lan.

How can I graph / get totals for individual users on my lan?

Thanks

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

headers

Adrian Popa | 25 Apr 11:30

Re: sfcapd problem with 3com 4800g router

Malformed packets might be due to the bad capture options. If you are capturing via tcpdump, it truncates packets to 64 bytes by default. You would need to use the -s 1500 parameter to specify the capture length.

The bad checksums may not be bad. Some NICs are doing TCP/UDP checksum offloading and may be calculating the checksum as part of the driver, which might be displayed differently than what wireshark shows.
If you get the same reports for valid traffic (e.g. TCP traffic that is ok and doesn't show retransmissions), you can ignore the checksum check (there's even an option in wireshark).

Please keep the discussion on the list, so that others may benefit of your findings as well.

On Wed, Apr 25, 2012 at 9:19 AM, Johannes Lavre <johannesl <at> vfk.no> wrote:

The collector has been on over night now and I see some flows coming in my nfsen/nfdump box. Problem is now finding out how the router behaves because I don’t see much coming in. Also in my pcap dump a lot of the sflow packets are malformed packets and I am loosing about 3 out of 7 packet because of bad checksums. I keep investigating this until I figure it out. Thank you very much for some pointers and good advice in troubleshooting.

Fra: Adrian Popa [mailto:adrian.popa.gh-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]
Sendt: 24. april 2012 10:08
Til: Johannes Lavre
Kopi: nfsen-discuss-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
Emne: Re: [Nfsen-discuss] sfcapd problem with 3com 4800g router

There are some strange segfaults in your messages - they may be the cause of the problem...

However, in order for nfdump to process and save flows in its files, it needs to understand the flows being sent. The router should periodically export a flow template packet that describes the fields exported in the flow. Once that packet is processed, the flows should be recorder.

The export interval for such a packet varies from router to router - can be every second, or once in 30 minutes.

To see if such a packet is exported, do a packet capture on your server and load it up in wireshark. Choose Decode As -> cflow and if you can see individual fields in the packets (e.g destination prefix, counters, etc), then the template packet is exported. If you don't get granular information, then the packet was not captured.

Good luck

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss

34	May 2012
38	April 2012
29	March 2012
23	February 2012
44	January 2012
18	December 2011
44	November 2011
30	October 2011
13	September 2011
12	August 2011
36	July 2011
10	June 2011
41	May 2011
19	April 2011
19	March 2011
18	February 2011
22	January 2011
22	December 2010
18	November 2010
8	October 2010
26	September 2010
25	August 2010
31	July 2010
61	June 2010
6	May 2010
50	April 2010
34	March 2010
12	February 2010
22	January 2010
13	December 2009
23	November 2009
17	October 2009
15	September 2009
8	August 2009
36	July 2009
65	June 2009
19	May 2009
19	April 2009
22	March 2009
16	February 2009
29	January 2009
25	December 2008
25	November 2008
30	October 2008
45	September 2008
55	August 2008
36	July 2008
20	June 2008
45	May 2008
75	April 2008
70	March 2008
65	February 2008
11	January 2008
34	December 2007
31	November 2007
35	October 2007
24	September 2007
112	August 2007
28	July 2007
60	June 2007
54	May 2007
56	April 2007
76	March 2007
54	February 2007
58	January 2007
34	December 2006
29	November 2006
47	October 2006
78	September 2006
26	August 2006

Mon	Tue	Wed	Thu	Fri	Sat	Sun

	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26	27
28	29	30	31