Discussion of flow-tools software

headers Kirk Olson | 6 May 2013 22:49 · /rmtserv1/

_______________________________________________
Flow-tools mailing list
flow-tools@...
http://mailman.splintered.net/mailman/listinfo/flow-tools
On my freebsd box:

#uname -rimp
9.1-STABLE amd64 amd64 GENERIC
flow_tools:

> pkg_info -x flow
Information for flow-tools-0.68_7:

Collector is ng_flow, started with

    /usr/sbin/ngctl mkpeer ipfw: netflow 30 iface0
    /usr/sbin/ngctl name ipfw:30 netflow

    /usr/sbin/ngctl msg netflow: setdlt {iface=0 dlt=12}
    /usr/sbin/ngctl msg netflow: setifindex {iface=0 index=5}
    /usr/sbin/ngctl msg netflow: settimeouts {inactive=15 active=150}
    /usr/sbin/ngctl mkpeer netflow: ksocket export inet/dgram/udp
    /usr/sbin/ngctl msg netflow:export connect inet/127.0.0.1:9995
And ipfw rule:

02750  59239017674  33111253913522 ngtee 30 ip from any to any via em0
Exported with flow_fanout for flow_capture.

# ps axww | grep flow
15106 ??  Ss        2:50,08 /usr/local/bin/flow-fanout -p
/var/run/flow-capture/flow-fanout.pid 127.0.0.1/0.0.0.0/9995
127.0.0.1/127.0.0.1/9556
16367 ??  Ss       11:28,63 /usr/local/bin/flow-capture -n 95 -N 3 -z
5 -S 5 -E270G -w /var/netflow -p
/var/run/flow-capture/flow-capture.pid 127.0.0.1/0.0.0.0/9556

In log files i see :
Mar 28 09:05:00 rubin flow-capture[16367]: STAT: now=1364439900
startup=1364227269 src_ip=127.0.0.1 dst_ip=65.121.97.1 d_ver=5 pkts=1
flows=30 lost=0 reset=0 filter_drops=0
Mar 28 09:05:00 rubin flow-capture[16367]: STAT: now=1364439900
startup=1364227269 src_ip=127.0.0.1 dst_ip=255.127.0.0 d_ver=5
pkts=1458 flows=43711 lost=21989 reset=1395 filter_drops=0
Mar 28 09:05:00 rubin flow-capture[16367]: STAT: now=1364439900
startup=1364227269 src_ip=127.0.0.1 dst_ip=109.112.100.32 d_ver=5
pkts=446 flows=13380 lost=15933 reset=401 filter_drops=0
Mar 28 09:05:00 rubin flow-capture[16367]: STAT: now=1364439900
startup=1364227269 src_ip=127.0.0.1 dst_ip=12.79.228.1 d_ver=5 pkts=4
flows=120 lost=0 reset=3 filter_drops=0
Mar 28 09:05:00 rubin flow-capture[16367]: STAT: now=1364439900
startup=1364227269 src_ip=127.0.0.1 dst_ip=105.110.100.44 d_ver=5
pkts=465 flows=13950 lost=16443 reset=411 filter_drops=0
Mar 28 09:05:00 rubin flow-capture[16367]: STAT: now=1364439900
startup=1364227269 src_ip=127.0.0.1 dst_ip=8.0.0.0 d_ver=5 pkts=88
flows=2611 lost=210 reset=85 filter_drops=0
Mar 28 09:05:00 rubin flow-capture[16367]: STAT: now=1364439900
startup=1364227269 src_ip=127.0.0.1 dst_ip=82.111.119.115 d_ver=5
pkts=449 flows=13412 lost=11044 reset=409 filter_drops=0

What is that ips in dst_ip 65.121.97.1 , 255.127.0.0, 109.112.100.32 etc?

I was tryed start flow_capture without flow_fanout and nothing was changed.

Is it flow_capture bug, or collector bug? Or maybe its  my fault in some config?

_______________________________________________
Flow-tools mailing list
flow-tools@...
http://mailman.splintered.net/mailman/listinfo/flow-tools
_______________________________________________
Flow-tools mailing list
flow-tools@...
http://mailman.splintered.net/mailman/listinfo/flow-tools
If you use Cisco products, you may be interested that they've announced the impending end for "traditional
netflow" on the ASR1000 line of routers.  Here is the announcement, which includes a link to a whitepaper
for migrating traditional to flexible netflow:

  http://www.cisco.com/en/US/prod/collateral/routers/ps9343/eol_C51-718332.html

Traditional netflow is what you have if your interfaces have any of these commands:

 ip route-cache flow
 ip flow ingress
 ip flow egress

The announcement only affects the ASR 1000 series of routers right now, but it points out that newer
platforms like the Catalyst 6500/Sup2T, the Catalyst 4500/Sup7E, and the Catalyst 3850 are exclusively
supporting flexible netflow. I suspect that the next generation of ISR routers follows suit.

-Craig

_______________________________________________
Flow-tools mailing list
flow-tools@...
http://mailman.splintered.net/mailman/listinfo/flow-tools
hello, list!
Plese, give:
how get flow fith flow-nfilter for incoming and outgoing traffic per 
some net for one request?
like nfdump tool:
nfdump 'ip xx.xx.xx.xx'?

--

-- 
WBR, Konstantin V. Krotov
CJSs "Information Systems"
mailto: kkv@...
phone: +7 (8332) 51-35-95

>From the lack of activity and a few of the last entries, I am slowly
figuring out that flow-tools community support has pretty much
dissolved.  So what are people doing instead of flow-tools?  Has
everyone gone and bought Lancope Stealthwatch or stayed with open
source and gone to SiLK (http://tools.netsa.cert.org/index.html)??

I have quite a lot of time and effort wrapped up in using flow-tools
and both Dave Plonka's FlowScan and Joe Loiacono's FlowViewer.  If I
need to eventually convert away from flow-tools to SiLK, I would
ideally want a simple way to convert both historical data in
flow-tools format and both of these higher level tools' feeds.

Or maybe someone will pick up the pieces and continue with flow-tools
development?

Thoughts???

Cheers,
Dave

Hi,  I am currently running a CentOS 5.7 server with flow-tools-0.68.5.1,
FlowViewer 3.3.1, and JKFlow 3.5.2 with FlowScan 1.006.  This is kind of an
odd server as it receives all the NetFlow data via scp from 2 other servers
also running flow-tools.  Every 5 minutes when these other servers process
their newly captured files, the files are copied to this server and a shell
script with the following lines in it is triggered:

    $flowpath/flow-cat $flowdir/rmtserv1/$infiles > /tmp/flowfile1
    $flowpath/flow-cat $flowdir/rmtserv2/$infiles > /tmp/flowfile2
    $flowpath/flow-merge -z2 /tmp/flowfile1 /tmp/flowfile2 >
$flowdir/$infile

The first 2 lines combine any unprocessed files from each of the remote
collector servers to temporary files,
then the 3rd line uses flow-merge to properly combine those temporary
files.

The problem is that somewhere in this process the file header timestamps
are lost so that

    flow-cat -t "strtime" -T "endtime" $flowdir

doesn't work.  This was working fine with the original uncombined flow
files.

Doing this command (note the debug level > 5):

    flow-cat -t "03/13/2012 08:58:59" -T "03/13/2012 12:00:01" -d 9 2>&1
$PREFIX/saved  | more

gives a bunch of lines like this:

    flow-cat:  i=0
    flow-cat: name=/var/local/flows/saved/ft-v05.2012-02-29.231500-0500
size=4971055  time=0
    flow-cat: name=/var/local/flows/saved/ft-v05.2012-03-02.095500-0500
size=7335032  time=0
    ...

where the times at the end are all zeros.

Although the doc for flow-merge has a line in it that describes the "-p"
switch to preload headers, the command does not actually support this.

So the questions is - how can I combine flow files from 2 different capture
servers with the proper header timestamps?

Thanks for any help.

Cheers,
Dave

Hello,

As was mentioned earlier today, flow-tools is a little long in the
tooth and it looks like lots of other projects have popped up to
fill the void.

That said, of the remaining list subs, what are you folks using or
considering moving to?

I'm particularly interested in any collector that's able to provide
some of the less technical staff with the ability to look at flow
data easily.  My wish list would include the following data somewhat
easily accessible:

-being able to tell a customer what's eating up their bandwidth ("oh
look, you've got 20Mb/s of outbound BitTorrent traffic, time to
write an IT policy on that").
-accurate traffic counts, ideally something that could even be used
for billing purposes.
-some basic alerting, such as a huge jump in PPS to any particular host.
-network-wide usage info (ie: X% of traffic is http, Y% is nntp, etc.)

Being able to do the one-offs listed above in a browser is a big plus...

Thanks,

Charles

_______________________________________________
Flow-tools mailing list
flow-tools@...
http://mailman.splintered.net/mailman/listinfo/flow-tools

3	May 2013
1	March 2013
2	December 2012
5	October 2012
4	July 2012
11	April 2012
1	March 2012
4	February 2012
10	January 2012
7	December 2011
1	November 2011
12	October 2011
9	September 2011
7	July 2011
7	June 2011
1	May 2011
5	April 2011
2	March 2011
11	February 2011
17	January 2011
1	December 2010
7	November 2010
11	October 2010
6	September 2010
3	August 2010
5	July 2010
11	June 2010
6	May 2010
5	April 2010
30	March 2010
7	February 2010
6	January 2010
5	December 2009
3	November 2009
1	October 2009
1	September 2009
6	August 2009
1	July 2009
5	June 2009
6	May 2009
3	April 2009
3	March 2009
7	February 2009
6	January 2009
1	December 2008
9	November 2008
10	October 2008
1	September 2008
9	August 2008
3	July 2008
8	June 2008
16	May 2008
15	April 2008
5	March 2008
31	February 2008
13	January 2008
19	December 2007
8	November 2007
33	October 2007
38	September 2007
12	August 2007
8	July 2007
13	June 2007
7	May 2007
32	April 2007
10	March 2007
26	February 2007
36	January 2007
17	December 2006
16	November 2006
2	October 2006
7	September 2006
32	August 2006
12	July 2006
33	June 2006
31	May 2006
25	April 2006
30	March 2006
44	February 2006
48	January 2006
38	December 2005
28	November 2005
22	October 2005
82	September 2005
51	August 2005
33	July 2005
14	June 2005
110	May 2005
46	April 2005
7	March 2005

http flow request not displaying flows

Strange ip addresses in flow_capture log files

Flowdumper not working after installing Cflow

(no subject)

Good bye, traditional netflow

Netflow v5 pretty much dead what is everyone migrating to?

how get flow fith flow-nfilter for incoming and outgoing traffic per some net?

Where did the community go?

flow file header timestamps

Other options

IPFIX

Mon	Tue	Wed	Thu	Fri	Sat	Sun

					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30