headers
Allen Chan | 18 May 2013 02:24
Favicon

Nfsen scalability and data aggregation on graphs

Nfsen has been running for 3 days now and we are loving it. Still working my way around the UI to understand the data.
I have a few questions that I cannot find an obvious answer for.
  1. How does one scale Nfsen product to handle thousands of flows/sec? All the commercial products have ways to scale (horizontally mostly). I wonder if nfsen supports that. Multiple collectors with central nfsen server?
  2. How is data aggregation handled? Most products start aggregating data after a certain amount of time. What is the raw data period for Nfsen and when does the product start aggregating data for graphs? Is it configurable?
Thanks,
Allen Chan


CONFIDENTIALITY NOTICE: This e-mail and any files attached may contain confidential information of Five9 and/or its affiliated entities. Access by the intended recipient only is authorized. Any liability arising from any party acting, or refraining from acting, on any information contained in this e-mail is hereby excluded. If you are not the intended recipient, please notify the sender immediately, destroy the original transmission and its attachments and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Copyright in this e-mail and any attachments belongs to Five9 and/or its affiliated entities.
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
Permalink | Reply | 0 comments |
headers
Frank Meier | 15 May 2013 14:27
Picon

nfsen and sflow

Hi there,

I want to capture netflow an sflow.

So I installed:
nfdump-1.6.9
/configure --enable-sflow --enable-nfprofile --enable-nftrack

nfsen-1.3.6p1

On my Linux Notebook I installed ipt_NETFLOW
and hsflowd for testing.

Netflows are capured, but nor sflow.

hsflow.conf:
DNSSD = off
  collector {
    ip = 10.66.20.204
    udpport = 2057
  }

On 10.66.20.204 nfsen.conf:

%sources = (
    'credne_netflow'    => { 'port' => '2055', 'col' => '#0000ff',
'type' => 'netflow' },
    'credne_sflow'    => { 'port' => '2057', 'col' => '#ff0000', 'type'
=> 'sflow' },
    'switch_sflow'    => { 'port' => '2056', 'col' => '#4b0082', 'type'
=> 'sflow' },
);

ps says:

nfsen    26635  0.0  0.0  17088  2632 ?        S    14:02   0:00
/bin/sfcapd -w -D -p 2056 -u nfsen -g omd -B 200000 -S 1 -P
/usr/local/nfsen/var/run/p2056.pid -z -I switch_sflow -l
/usr/local/nfsen/profiles-data/live/switch_sflow
nfsen    26642  0.0  0.0  17088  2628 ?        S    14:02   0:00
/bin/sfcapd -w -D -p 2057 -u nfsen -g omd -B 200000 -S 1 -P
/usr/local/nfsen/var/run/p2057.pid -z -I credne_sflow -l
/usr/local/nfsen/profiles-data/live/credne_sflow
nfsen    26651  0.0  0.0  18172  2760 ?        S    14:02   0:00
/bin/nfcapd -w -D -p 2055 -u nfsen -g omd -B 200000 -S 1 -P
/usr/local/nfsen/var/run/p2055.pid -z -I credne_netflow -l
/usr/local/nfsen/profiles-data/live/credne_netflow

The data in
profiles-data/live/credne_sflow/2013/05/15/
are all 276 bit.

Any clue where the problem is?

Thanks
-- 
Mit freundlichen Grüßen

Frank Meier
UNIX-Basis

Hamm Reno Group GmbH
Industriegebiet West | D-66987 Thaleischweiler-Fröschen
T.: +496334 444-8322
Frank.Meier@... | www.reno.de
___________________________________________________________________

Sitz: Am Tie 7 | D-49086 Osnabrück

Handelsregister Osnabrück HRB 19587

Geschäftsführer: Hans-Jürgen de Fries,
Jens Gransee, Manfred Klumpp, Robert Reisch, Uwe Niemann,
Jens Rauschen, Uwe Wesemann

Diese E-Mail sowie eventuelle Anhänge enthalten vertrauliche und / oder
rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat
sind
oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte
sofort den
Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren oder
Speichern
sowie die unbefugte Weitergabe dieser E-Mail sind nicht gestattet.

This e-mail and any attachments may contain confidential and / or privileged
information. If you are not the intended recipient or have received this
e-mail in error, please notify the sender immediately and destroy this
e-mail
. Any unauthorized copying, storing, disclosure or distribution of the
contents of this e-mail is strictly forbidden.

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
Permalink | Reply | 0 comments |
headers
Allen Chan | 10 May 2013 02:32
Favicon

Issues installing Nfsen

Hi everyone,

I got nfcapd working and writing the flow data to file. nfdump is also showing data.
I am trying to install nfsen for the UI part of it.

I am getting this error:

[root <at> xx nfsen-1.3.6p1]# ./install.pl ./etc/nfsen.conf 
Check for required Perl modules: Failed
Required nfsen modules not found
Can't locate Mail/Header.pm in <at> INC ( <at> INC contains: libexec ./libexec ./installer-items /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at ./install.pl line 640.

[root <at> xx nfsen-1.3.6p1]# find / -name 'Header'
/usr/local/lib/perl5/5.16.2/x86_64-linux-thread-multi/Encode/MIME/Header
/usr/lib64/perl5/Encode/MIME/Header

Pretty sure i have all the requirements so this is a little puzzling. Googling the error has mostly just gotten advice to install the cpan packages...

Thanks
Allen Chan


CONFIDENTIALITY NOTICE: This e-mail and any files attached may contain confidential information of Five9 and/or its affiliated entities. Access by the intended recipient only is authorized. Any liability arising from any party acting, or refraining from acting, on any information contained in this e-mail is hereby excluded. If you are not the intended recipient, please notify the sender immediately, destroy the original transmission and its attachments and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Copyright in this e-mail and any attachments belongs to Five9 and/or its affiliated entities.
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
Permalink | Reply | 7 comments |
headers
Skept | 3 May 2013 07:59
Picon
Gravatar

Graphing specific asn

Dear list,

We are planning to graph traffic to top asn's. Currently we are exporting traffic from a switch mirrored port. The port is connected to the Linux system hosting nfsen and the flows are exported via nprobe.

I figured the obvious choice would be src as and DST as, but graphs with those parameters are turning up empty.

I searched around and found a three part script on nfsen list detailing procedure to graph the top thousand asn's. The link is here.

http://comments.gmane.org/gmane.network.nfsen.general/1242

I couldn't figure out what the top directory means in the first part.

Also, I guess the question boils down to if the core router is not doing bgp, how do I graph specific asn's?  Look up each IP address block, add them to an ASN and then graph traffic to and from that ASN? Are there any implementations of it? 

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
Permalink | Reply | 1 comment |
headers
Wilkinson, Alex | 26 Apr 2013 09:24
Picon

Message Details For Email Alerts ... ?

Hi all,

Any progress on email body contents/details for alerts ? As question here in
2010:

http://www.mail-archive.com/nfsen-discuss-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f <at> public.gmane.org/msg01866.html ?

   -Alex

************** IMPORTANT MESSAGE *****************************       
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential. 
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries. 
We can be contacted through our web site: commbank.com.au. 
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line. 
**************************************************************


------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
Permalink | Reply | 0 comments |
headers
Naim Shafiev | 24 Apr 2013 11:33
Picon

Non-latin names in profiles

Hello.Is there way to make a non-latin(ascii) names in profile name?
------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
Permalink | Reply | 0 comments |
headers
Abel Guzmán Sánchez | 24 Apr 2013 08:06
Picon

nfsen 1.3.6p1 not doing graphics on debian 6!

Hello, I'm new to nfsen, I've installed fprobe, nfdump and nfsen on a Debian 6 box, every thing seems to be working fine and I tested to run nfcap as follows:
/bin/nfcapd -w -D -p 23456 -B 200000 -S 1 -z -I Linux-Host-1-eth0 -l /var/netflow/

And it's able to get the responses from the client side... I've run a nfdump comand with the -r option with the path to the file and it shows every thing that was captured...
But nfsen is unable to show any thing on the graphics and I don't know how to get any proper information about why is it not working or how to troubleshoot it...
Can you please give me some light on this issue?
Kind Regards and thank you in advanced.



------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
Permalink | Reply | 2 comments |
headers
Wilkinson, Alex | 24 Apr 2013 05:26
Picon

"flow-export: destination with same IP already exists" ... ?

Hi all,

I am successfully exporting flows from one interface on multiple ASAs. Today I
went to export flows from another interface and was greeted with the following
error:

   ASA(config)# flow-export destination dmz x.x.x.x 2055
   ERROR: flow-export: destination with same IP already exists

Can I not export flows from multiple interfaces on a single device ?

Regards

   -Alex

************** IMPORTANT MESSAGE *****************************       
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential. 
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries. 
We can be contacted through our web site: commbank.com.au. 
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line. 
**************************************************************


------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
Permalink | Reply | 0 comments |
headers
Abel Guzmán Sánchez | 23 Apr 2013 12:07
Picon

About nfsen not doing graphics!

Hello, I'm new to nfsen I've installed fprobe, nfdump and nfsen on a Debian 6 box, every thing seems to be working fine and I tested to run nfcap as follows:

/bin/nfcapd -w -D -p 23456 -B 200000 -S 1 -z -I Linux-Host-1-eth0 -l /var/netflow/

And it's able to get the responses from the client side... I've run a nfdump comand with the -r option with the path to the file and it shows every thing that was captures...
but nfsen is unable to show any thing on the graphics and I don't know how to get any proper information about why is it not working, how to troubleshoot it...
Can you please give me some light on this issue?
Kind Regards and thank you in advanced.


------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
Permalink | Reply | 0 comments |
headers
Wilkinson, Alex | 23 Apr 2013 09:45
Picon

nfdump-1.6.9 + ASA (8.2) - Packets(%) - Empty (zero) ... ?

Hi all,

Firstly superb piece of software Peter!

I have two questions:

Question one:
~~~~~~~~~~~~~

I am successfully using nfdump-1.6.9/nfsen-1.3.6p1 on FreeBSD 9.1-STABLE to
monitor ASAs running Version 8.2(5)33. Things seem to work well, except for the fact
that "Packets(%)", "pps" and "bpp" are all zero and never increment e.g

  Top 10 IP Addr ordered by packets:
  Date first seen          Duration Proto           IP Addr    Flows(%)     Packets(%)       Bytes(%)         pps      bps   bpp
  2013-04-23 17:08:53.859   191.039 any       x.x.x.x       11( 0.0)        0( 0.0)    73797( 0.0)        0     3090     0
  2013-04-23 17:04:23.717    71.253 any       x.x.x.x        7( 0.0)        0( 0.0)    33930( 0.0)        0     3809     0
  2013-04-23 17:04:58.374   195.439 any       x.x.x.x        9( 0.0)        0( 0.0)   906003( 0.1)        0    37085     0
  2013-04-23 17:18:13.639   313.166 any       x.x.x.x       15( 0.1)        0( 0.0)   528703( 0.1)        0    13506     0
  2013-04-23 17:13:18.240    29.137 any       x.x.x.x        2( 0.0)        0( 0.0)      287( 0.0)        0       78     0
  2013-04-23 17:11:57.899     0.000 any       x.x.x.x        1( 0.0)        0( 0.0)      203( 0.0)        0        0     0
  2013-04-23 17:12:04.468   233.405 any       x.x.x.x       14( 0.1)        0( 0.0)   531998( 0.1)        0    18234     0
  2013-04-23 17:12:34.695    62.923 any       x.x.x.x        3( 0.0)        0( 0.0)   131622( 0.0)        0    16734     0
  2013-04-23 17:05:26.531   246.503 any       x.x.x.x       21( 0.1)        0( 0.0)     4735( 0.0)        0      153     0
  2013-04-23 17:08:34.931    64.883 any       x.x.x.x        4( 0.0)        0( 0.0)    56680( 0.0)        0     6988     0

I was under the impression that the NSEL fork is no longer needed since it has been merged into nfdump-1.6.9 ?
(The reason I ask this is because I have seen in the archives others with same problem and the solution was the
NSEL fork).

So can anyone suggest how I can troubleshoot the aforementioned issue ?

Question two:
~~~~~~~~~~~~~

Apparently Cisco wrote and released a plugin called "NSELTracker", however, I cannot see it here: http://sourceforge.net/apps/trac/nfsen-plugins/.

Is the "NSELTracker" plugin still relevant ? If yes, can someone tell me where to get it from ?

Regards

  -Alex

************** IMPORTANT MESSAGE *****************************       
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential. 
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries. 
We can be contacted through our web site: commbank.com.au. 
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line. 
**************************************************************


------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
Permalink | Reply | 2 comments |
headers
Dp Singh | 23 Apr 2013 06:21
Picon

Fwd: nfsen


Kindly advice.


---------- Forwarded message ----------
From: Dp Singh <dpsingh1812-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date: Mon, Apr 22, 2013 at 8:26 PM
Subject: Re: [Nfsen-discuss] nfsen
To: Tomas Plesnik <plesnik <at> ics.muni.cz>


Hi,

Thanks for your reply,

its done i have installed nfdump its working now, but i want to add all my LAN device into it, its possible and how to do.

Thanks & Regards
Devendra Prasad


On Mon, Apr 22, 2013 at 8:22 PM, Tomas Plesnik <plesnik-8qz54MUs51PtwjQa/ONI9g@public.gmane.org> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

On 22.4.2013 14:11, Dp Singh wrote:
> hi,
>
> I am new to nfsen, i am trying install nfsen as user manual in
> online.
>
> while i am at this stage
>
> nfdump tools installation error: 'nfcapd' not found in
> '/usr/local/bin' at ./install.pl <http://install.pl> line 197,
> <STDIN> line 1.

first of all you must have installed NFDUMP tools
(http://nfdump.sourceforge.net/), which collect and process netflow
data on the command line and they are part of the NfSen project.

Tomas

>
> kindly advice.
>
> Thanks & Regards Devendra Prasad
>
>
> ------------------------------------------------------------------------------
>
>
Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for
> building apps and a phenomenal toolset for data science. Developers
> can use our toolset for easy data analysis & visualization. Get a
> free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
>
>
>
> _______________________________________________ Nfsen-discuss
> mailing list Nfsen-discuss-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>


- --
Tomas Plesnik                                       plesnik-8qz54MUs51PtwjQa/ONI9g@public.gmane.org
CSIRT-MU, Network Security Department          http://www.muni.cz/csirt
Institute of Computer Science, Masaryk University, Brno, Czech Republic
PGP key ID: 0x9D3722F3





-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAlF1TsoACgkQGA/bT503IvOvMQCeM56GzNchqgv2x/PgQ8LgqzXD
QNAAoIEn1bqf7BZ+yIxUtJNL1PCD7CTa
=rzKz
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss


------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@...
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
Permalink | Reply | 0 comments |

Gmane