headers
Russell Fulton | 11 Jun 2003 04:35
Picon
Picon
Favicon

ragator questions

I too am exploring ragator to do some reporting.  I want to aggregate
traffic by port for both tcp and udp:

#label   id    SrcCIDRAddr        DstCIDRAddr         Proto  SrcPort  DstPort   ModelList  Duration

Flow     100       *                  *                tcp      *        *        210        100000
Flow     101       *                  *                udp      *        *        210        100000

# TCP and UDP Flow Model Definitions
# label  id      SrcAddrMask     DstAddrMask      Proto  SrcPort  DstPort

Model    210    0.0.0.0         0.0.0.0            yes       no       yes

This works up to a point but I get multiple lines for some ports:

09 Jun 03 23:59:03    tcp         0.0.0.0.*      ->           0.0.0.0.139   5369014  5804876   1862159581   4294730450  RST
10 Jun 03 00:00:26    tcp         0.0.0.0.*      ->           0.0.0.0.80    3096583  4066172   1206524197   4294502846  RST
10 Jun 03 12:30:56    tcp         0.0.0.0.*      ->           0.0.0.0.139   4567659  5105243   1717844692   4292788911  RST
10 Jun 03 14:51:26    tcp         0.0.0.0.*      ->           0.0.0.0.139   3991806  4801433   750558675    4275893922  RST
10 Jun 03 16:21:49    tcp         0.0.0.0.*      ->           0.0.0.0.139   3022618  3923831   734860325    4268835146  RST
10 Jun 03 17:08:37    tcp         0.0.0.0.*      ->           0.0.0.0.139   2690251  3752698   492063501    4270547227  RST
10 Jun 03 13:12:06    tcp         0.0.0.0.*      ->           0.0.0.0.80    3676245  4635555   1414504177   4294358018  RST

One possibly relevant point, there were multiple input files covering on
days traffic.

I am also seeing records like this:

10 Jun 03 14:24:24    tcp         0.0.0.0.65535  ->           0.0.0.0.43091 26       44        1412         60227       RST
10 Jun 03 19:20:50    tcp         0.0.0.0.65535  ?>           0.0.0.0.2865  1        1         54           54          FIN
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dave Plonka | 11 Jun 2003 17:08
Picon

Queue Exceeded Maximum Limit, Terminating process (was "Re: DoS woes....")


Hello argus users,

One of our argus instances is periodically terminating one of its
processes with these messages " Queue Exceeded Maximum Limit",
"ArgusWriteOutSocket failed Input/output error", then "Terminating process".

It's argus-2.0.5 on this platform:

   # cat /etc/redhat-release
   Red Hat Linux release 8.0 (Psyche)
   # uname -r               
   2.4.18-14smp

This argus is watching all IP traffic for a relatively quiescent
class-A network (5,000-10,000 packets-per-second).

When the problem occurs, two other argus processes continue to run, but
no more content is written to the argus output file.  Once we discover
the problem, we typically kill the argus parent at this point, and
restart it, and it runs for another week or two.

Here is a typical series of argus syslog messages leading up to the
termination:

   Jun  9 03:52:26 localhost argus[19729]: ArgusWriteOutSocket(0x8148ae0) Queue Count 249238 
   Jun  9 03:52:56 localhost argus[19729]: ArgusWriteOutSocket(0x8148ae0) Queue Count 254228 
   Jun  9 03:53:26 localhost argus[19729]: ArgusWriteOutSocket(0x8148ae0) Queue Count 259423 
   Jun  9 03:53:51 localhost argus[19729]: ArgusWriteOutSocket(0x8148ae0) Queue Exceeded Maximum Limit 
   Jun  9 03:53:51 localhost argus[19729]: ArgusHandleData: ArgusWriteOutSocket failed Input/output
(Continue reading)

Permalink | Reply | 0 comments |
headers
John Hermes | 17 Jun 2003 14:40

Log "tear-off" mechanism

Howdy,

I am hoping someone might be able to help me understand how
Argus deals with having the open logfile pulled out from under
the daemon (using the argusarchive script for example). If you
could decribe it in terms of file descriptors and pointers,
that would be most helpful. For instance, does Argus get
an I/O error and create a new file in response? Why does the
OS (Linux, FreeBSD, etc.) allow me to move an open file? Why
can't I remember this stuff from college? :-)

I have a watchdog daemon that I configured to stat the
logfile on a continuous basis, and now I need to write a
compatible monitor that won't mind that the file is missing
for the few seconds before Argus creates a new logfile. Any
info related to moving open files would be a great help!

Thanks!

--

-- 
John Hermes
Systems Engineer
Infoglobe, Inc
937-225-9999 x317
937-226-1623 Fax
jhermes <at> infoglobe.com
Permalink | Reply | 1 comment |
headers
Peter Van Epp | 17 Jun 2003 17:01
Picon
Picon
Favicon

Re: Log "tear-off" mechanism

	While reading the source or waiting for Carter to comment would be 
best, as I recall argus stats the output file and when the stat changes (as
when the file gets renamed) closes and reopens the logfile (using the file name
it was given on invocation to open the new logfile) while puffing its cheeks up 
in memory with the current data until the new logfile is open and writable to.
	For your watchdog what you want to do is have the argusarchive script
write the current time when it renames the log file in to a file. When your
watchdog wakes up and finds the output file missing it looks for the file
from argusarchive, reads the time that the change was made and uses the 
elapsed time to see if there is a problem or not (if insufficient time has 
elapsed to be sure there is a problem, sleep for a while and check again to
see if a new logfile has been created). Once a new logfile is running the 
watchdog deletes the file created by argusarchive ready for the next log roll.

Peter Van Epp / Operations and Technical Support 
Simon Fraser University, Burnaby, B.C. Canada

On Tue, Jun 17, 2003 at 08:40:21AM -0400, John Hermes wrote:
> Howdy,
> 
> I am hoping someone might be able to help me understand how
> Argus deals with having the open logfile pulled out from under
> the daemon (using the argusarchive script for example). If you
> could decribe it in terms of file descriptors and pointers,
> that would be most helpful. For instance, does Argus get
> an I/O error and create a new file in response? Why does the
> OS (Linux, FreeBSD, etc.) allow me to move an open file? Why
> can't I remember this stuff from college? :-)
> 
> I have a watchdog daemon that I configured to stat the
(Continue reading)

Permalink | Reply | 0 comments |
headers
Carter Bullard | 17 Jun 2003 15:04

RE: Log "tear-off" mechanism

Hey John,
   Argus calls stat() on the output filename before writing
any record.  stat() will indicate if the file has been
renamed, and if it has, argus will recreate it, and write
an argus data file header into the file and then proceed
to write out the new record.

   Because this is data driven, and argus could be idle for
quite a while, based on traffic on the wire/interface, 
the upper bound on how long the file could be missing is
the ARGUS_MAR_STATUS_INTERVAL.

Carter

> -----Original Message-----
> From: owner-argus-info <at> lists.andrew.cmu.edu 
> [mailto:owner-argus-info <at> lists.andrew.cmu.edu] On Behalf Of 
> John Hermes
> Sent: Tuesday, June 17, 2003 8:40 AM
> To: argus-info <at> lists.andrew.cmu.edu
> Subject: Log "tear-off" mechanism
> 
> 
> Howdy,
> 
> I am hoping someone might be able to help me understand how
> Argus deals with having the open logfile pulled out from under
> the daemon (using the argusarchive script for example). If you
> could decribe it in terms of file descriptors and pointers,
> that would be most helpful. For instance, does Argus get
(Continue reading)

Permalink | Reply | 0 comments |
headers
Carter Bullard | 17 Jun 2003 17:14

RE: Log "tear-off" mechanism

Hey Peter,
   Thanks, I forgot to mention that we closed the
datafile, before creating the new one, which is of
course very important.

Carter

> -----Original Message-----
> From: owner-argus-info <at> lists.andrew.cmu.edu 
> [mailto:owner-argus-info <at> lists.andrew.cmu.edu] On Behalf Of 
> Peter Van Epp
> Sent: Tuesday, June 17, 2003 11:01 AM
> To: argus-info <at> lists.andrew.cmu.edu
> Subject: Re: Log "tear-off" mechanism
> 
> 
> 	While reading the source or waiting for Carter to 
> comment would be 
> best, as I recall argus stats the output file and when the 
> stat changes (as
> when the file gets renamed) closes and reopens the logfile 
> (using the file name
> it was given on invocation to open the new logfile) while 
> puffing its cheeks up 
> in memory with the current data until the new logfile is open 
> and writable to.
> 	For your watchdog what you want to do is have the 
> argusarchive script
> write the current time when it renames the log file in to a 
> file. When your
(Continue reading)

Permalink | Reply | 0 comments |
headers
Russell Fulton | 20 Jun 2003 00:20
Picon
Picon
Favicon

ra in clients distribution

Hi All,
	I've been trying to look for traffic from the new trojan that sends syn
packets with specific window size and options set.  I can do this with
raxml but it is a pain because 
     A. its slow (lots of formatting) and 
     B. output is spread over multiple lines so I cant post process
        using grep.
However I notice that the -s switch on the ra in the client distro can
be used to display window and option information in normal display. The
problem is that I could not make it work.  Is it supposed to at the
moment?  I admit I did not spend much time on it...

--

-- 
Russell Fulton, Network Security Officer, The University of Auckland,
New Zealand.
Permalink | Reply | 0 comments |
headers
Carter Bullard | 20 Jun 2003 04:42

RE: ra in clients distribution

Hey Russell,
   You can get last window size for both the src and dst
diretion using the '-s win' option.  IP options come up
in the 'ind' (indicator) field, and have to be parsed.
If we can come up with a decent representation of the
options, we could have a separate field printed just for
them.  Currently, the argus record has all the options
that were observed as a bit map, so we can report
any/most of them as they occur.

Carter

> -----Original Message-----
> From: owner-argus-info <at> lists.andrew.cmu.edu 
> [mailto:owner-argus-info <at> lists.andrew.cmu.edu] On Behalf Of 
> Russell Fulton
> Sent: Thursday, June 19, 2003 6:20 PM
> To: argus-info <at> lists.andrew.cmu.edu
> Subject: ra in clients distribution
> 
> 
> Hi All,
> 	I've been trying to look for traffic from the new 
> trojan that sends syn
> packets with specific window size and options set.  I can do this with
> raxml but it is a pain because 
>      A. its slow (lots of formatting) and 
>      B. output is spread over multiple lines so I cant post process
>         using grep.
> However I notice that the -s switch on the ra in the client distro can
(Continue reading)

Permalink | Reply | 0 comments |
headers
Mitesh P Choksi | 20 Jun 2003 21:49

Repository of argus clients

Dear All,

I use argus for recording what flows went to and from a router to 
audit/identify what could have happened when traffic peaks occur. Analysis 
usually find out the IP address giving the problem or the type of protocol. 
Only sometimes it is number of flows/sec.

I would like to hand over this task of digging into the data and make it 
available for my brother who does not know linux.

I wanted to know if there are set of scripts/cgi/client software available 
for web based interface to dig in information from argus and graphically 
representing it.

I don't know perl so am not capable of writing the scripts on my own.

Any help will be appreciated even if it is pointers. If there are any windoze 
based utilities available that allow digging information from argus, I will 
try it out.

Regards,

Mitesh
Permalink | Reply | 2 comments |
headers
Peter Van Epp | 20 Jun 2003 23:58
Picon
Picon
Favicon

ra (or equivelent) in daemon mode?

	Before I look at possibly reinventing the wheel, was there any 
resolution to the question of how to have argus on one box writing only 
to a socket and ra (or something else since ra seems overkill) on another 
box that listens on the socket and writes the data to disk? Basically what
I'm after is argus_linux spread across two machines. One collecting / 
processing (but doing no disk I/O) and the other one writing the data to 
disk and being rotated by argus archive (and possibly running ra against 
the data in the archive). Linux is up and listening to a fdx link on a pair
of bonded 3c905Bs (and currently writing to disk on the same machine) now I
need to move the disk I/O to another machine in preparation for changing to 
Gig.

Peter Van Epp / Operations and Technical Support 
Simon Fraser University, Burnaby, B.C. Canada
Permalink | Reply | 1 comment |

Gmane