headers
Dave Plonka | 10 Feb 2003 19:20
Picon

using flow-tools for ad hoc flow reports (was "Re: toptalkers over a longer timespan")


FlowScan users,
[Argus users, please forgive the cross-post.
 I think you'll find it pertinent towards the end though.]

Over the past years, a number of you have asked for additional FlowScan
reports, such as this:

   On Wed, Jan 08, 2003 at 10:43:39AM -0500, Matthew Deatherage wrote:
   > Any suggestions on generating a TopTalkers report for a given time 
   > span?  I'd be interested in a report on the top talker for a week or month.

Such ad hoc reports can be generated fairly easily using the flow-stat
reporting utility supplied with Mark Fullmer's excellent flow-tools
package.  The flow-tools package is available here:

   http://www.splintered.net/sw/flow-tools/

For instance, to produce a "Top Talkers" report for a whole day,
sorted, descending by bytes, one can run:

   ft_flows$ flow-cat ft-v05.2003-02-10.*0 | flow-stat -f9 -S2 >/tmp/flow-stat_2003-02-10.txt

I've attached the first 22 lines of that output file as a sample which
show the "Top Ten Talkers" (anonymized IP addresses), please check it out.

flow-stat's "-f9" option selects a report by source IP address, and
"-S2" causes it to sort descending by column 2, which is bytes for this
report.  Do "man flow-stat" to see all of the reports and options.
(Continue reading)

Permalink | Reply | 4 comments |
headers
Andrew Pollock | 13 Feb 2003 05:56
Picon
Favicon

Splitting a ludicrously large Argus log

Hi,

Our log rotation let us down and we've got an enormous (8.5 Gb) argus.log 
that ra can't read. How else can we split this into manageable chunks 
without losing anything?

Andrew
Permalink | Reply | 1 comment |
headers
Carter Bullard | 13 Feb 2003 06:16

RE: Splitting a ludicrously large Argus log

Hey Andrew,
   Can any of the ra* programs read the whole thing, or
are you getting some system error?

Carter

> -----Original Message-----
> From: owner-argus-info <at> lists.andrew.cmu.edu 
> [mailto:owner-argus-info <at> lists.andrew.cmu.edu] On Behalf Of 
> Andrew Pollock
> Sent: Wednesday, February 12, 2003 11:56 PM
> To: argus-info <at> lists.andrew.cmu.edu
> Subject: Splitting a ludicrously large Argus log
> 
> 
> Hi,
> 
> Our log rotation let us down and we've got an enormous (8.5 
> Gb) argus.log 
> that ra can't read. How else can we split this into manageable chunks 
> without losing anything?
> 
> Andrew
>
Permalink | Reply | 0 comments |
headers
Andrew Pollock | 18 Feb 2003 07:42
Picon
Favicon

Re: Splitting a ludicrously large Argus log

On Thu, Feb 13, 2003 at 02:56:14PM +1000, Andrew Pollock wrote:
> Hi,
> 
> Our log rotation let us down and we've got an enormous (8.5 Gb) argus.log 
> that ra can't read. How else can we split this into manageable chunks 
> without losing anything?

For the record, I worked around this by catting the ludicrously large file 
through ra, rather than getting ra to open it directly, e.g.:

cat bloody_huge_file | ra -r - -t 02/01 -w 2003-02-01

instead of

ra -r bloody_huge_file -t 02/01 -w 2003-02-01

regards

Andrew
Permalink | Reply | 0 comments |
headers
Scott A.McIntyre | 20 Feb 2003 10:07

NetFlow (ra -C)

Hi,

Perhaps I've not had enough coffee yet today, but I can't seem to get a 
ra-client to attach to a netflow source, specifically, I've got running 
a cflowd, cflowdmux and the other caida tools all up and running, 
however if I use ra -C -S localhost:5555, I get:

ArgusError: ra[5033]: usage: -C and -S not compatible.

And if I flip the order around, ra -S localhost:5555 -C ...:

ArgusError: ra[5052]: ArgusAddHostList: format error -S no port value.

So, how does one do this properly?

Thanks,

Scott
Permalink | Reply | 2 comments |
headers
Yann Berthier | 20 Feb 2003 10:37

Re: NetFlow (ra -C)

On Thu, 20 Feb 2003, Scott A.McIntyre wrote:

> Hi,
> 
> Perhaps I've not had enough coffee yet today, but I can't seem to get a 
> ra-client to attach to a netflow source, specifically, I've got running 
> a cflowd, cflowdmux and the other caida tools all up and running, 
> however if I use ra -C -S localhost:5555, I get:
> 
> ArgusError: ra[5033]: usage: -C and -S not compatible.
> 
> And if I flip the order around, ra -S localhost:5555 -C ...:
> 
> ArgusError: ra[5052]: ArgusAddHostList: format error -S no port value.
> 
> So, how does one do this properly?

   Do you try to get data from a Cisco NetFlow _and_ an argus source at
   the same time ? If yes I don't think this is possible as stated by
   the error message you get (please correct me if i'm wrong)

   The way to get netflow and argus data are mutually exclusive: with
   neflow source you bind on a udp local port waiting for udp datagrams
   sent by you netflow source, with argus source you connect to the
   remote tcp socket of the argus server (kind of push vs pull, well not
   really but you get the point)

   The -C flag indicates to ra() that you bind on port 9995/udp
   listening for netflow input (change the port with -P). All you have
   to do is to say to your favorite netflow generator to send the
(Continue reading)

Permalink | Reply | 1 comment |
headers
Scott A.McIntyre | 20 Feb 2003 15:13

Re: NetFlow (ra -C)

Hi,

>    Do you try to get data from a Cisco NetFlow _and_ an argus source at
>    the same time ? If yes I don't think this is possible as stated by
>    the error message you get (please correct me if i'm wrong)
>
>    The way to get netflow and argus data are mutually exclusive: with
>    neflow source you bind on a udp local port waiting for udp datagrams
>    sent by you netflow source, with argus source you connect to the
>    remote tcp socket of the argus server (kind of push vs pull, well 
> not
>    really but you get the point)

Ah, this may indeed explain what it is I'm not seeing that I thought I 
could see.

What I was hoping for was one of two things:

1)  I am using cflowd from caida; so the ability to use the ra-client 
to attach to the cflowdmux via tcp and run in a similar fashion as "ra 
-S argushost" and use various other ra-tools to get at the cflow 
exported data from our routers.  This would be handy for ratop, and 
other live-analysis tools where it's not possible to run a real 
argus(8).

2)  The ability to use the ra-tools to parse a cflowd created file and 
use the plethora of other ra-based scripts and front ends I have to 
parse the flow files.   This is probably more of a job for argus(8) 
than ra anyway.  Perahps the ability for argus(8) to connect to the 
cflowd itself rather than interfaces would be useful; but, yes, I know
(Continue reading)

Permalink | Reply | 0 comments |
headers
Carter Bullard | 20 Feb 2003 15:22

RE: NetFlow (ra -C)

Hey Scott,
   The syntax is:

      ra -CP 9999

All the ra clients understand the Cisco netflow file format,
just run "ra -r filename" against one of these files and
it should run fine.  I belive that cflowd uses the same
format, but if you have any problems, just send mail.

Carter

> -----Original Message-----
> From: owner-argus-info <at> lists.andrew.cmu.edu 
> [mailto:owner-argus-info <at> lists.andrew.cmu.edu] On Behalf Of 
> Scott A.McIntyre
> Sent: Thursday, February 20, 2003 9:14 AM
> To: Yann Berthier
> Cc: argus-info <at> lists.andrew.cmu.edu
> Subject: Re: NetFlow (ra -C)
> 
> 
> Hi,
> 
> 
> >    Do you try to get data from a Cisco NetFlow _and_ an 
> argus source at
> >    the same time ? If yes I don't think this is possible as 
> stated by
> >    the error message you get (please correct me if i'm wrong)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Alexander Bochmann | 21 Feb 2003 16:27
Picon

Re: using flow-tools for ad hoc flow reports (was "Re: toptalkers over a longer timespan")

...on Mon, Feb 10, 2003 at 12:20:14PM -0600, Dave Plonka wrote:

 > ARGUS AND CFLOWD USERS:
 > You can use flow-tools' flow-stat reports *even if* you are using
 > cflowd or argus rather than flow-tools' flow-capture as your flow
 > collector.  With my flowdumper utility, supplied with the Cflow

Would it also be possible to convert flowtools capture 
files to cflowd format, so that argus can understand them?
Or even make argus read that format directly?

Personally, I find the argus clients' usage much more 
intuitive than some of the flow-tools for simple tasks, 
but then that may only be because I have already used 
argus.

Alex.
Permalink | Reply | 3 comments |
headers
Mark Fullmer | 21 Feb 2003 17:05

Re: using flow-tools for ad hoc flow reports (was "Re: toptalkers over a longer timespan")

On Fri, Feb 21, 2003 at 04:27:08PM +0100, Alexander Bochmann wrote:
> ...on Mon, Feb 10, 2003 at 12:20:14PM -0600, Dave Plonka wrote:
> 
>  > ARGUS AND CFLOWD USERS:
>  > You can use flow-tools' flow-stat reports *even if* you are using
>  > cflowd or argus rather than flow-tools' flow-capture as your flow
>  > collector.  With my flowdumper utility, supplied with the Cflow
> 
> Would it also be possible to convert flowtools capture 
> files to cflowd format, so that argus can understand them?
> Or even make argus read that format directly?

Yes.  Use flow-export -f0.

mark
Permalink | Reply | 2 comments |

Gmane