Rodney McKee | 9 Dec 06:12
Favicon
Gravatar

color graphs

How do I get something like this to give a result in color?

ragraph proto trans -nr argus.out-investigate -M 1m -w proto.png


Rgds Rodney
Carter Bullard | 9 Dec 17:39

Re: color graphs

Hey Rodney,
The output is a color graph.  You don't see any colors?

Carter

On Dec 9, 2009, at 12:12 AM, Rodney McKee wrote:

How do I get something like this to give a result in color?

ragraph proto trans -nr argus.out-investigate -M 1m -w proto.png


Rgds Rodney


Attachment (smime.p7s): application/pkcs7-signature, 3815 bytes
Matt Sheridan | 9 Dec 17:43
Picon

Re: Rasplit compression support

Hi Carter -

I was wondering what your thoughts might be on this moving forward? The defunct processes continue to pile up.

Not sure if its related, but if I run a basic ra on a file, I am getting some other process complaints:

ra -r ./argus.2009_12_09_1100.out.gz - host 10.10.10.10
ra[4935]: 11:42:56.648522 ArgusReadStreamSocket (0x41fc) record length is zero

gzip: stdout: Broken pipe

On Tue, Nov 17, 2009 at 10:51 AM, Matt Sheridan <mattmail5050 <at> gmail.com> wrote:

I have an update. It seems that what is becoming a zombie is the script called by the "-f" option (you may have already figured that out, I just did ;)"

I attempted to use the default contrib script included in the clients package, and it also zombied.


But as you can see:

root      1716 31417  8 10:46 pts/1    00:00:10 rastream -S localhost:561 -M time 1m -B 10s -f /opt/IDS/argus/etc/rastream-orig.sh -
root      2057  1716  0 10:47 pts/1    00:00:00 [rastream-orig.s] <defunct>

it was the rastream-orig.sh script, called by the -f option (which is used to zip up the files) that zombied. So it is possible that the problem is not within rastream, but in the way the called script is handled?



On Mon, Nov 16, 2009 at 9:32 PM, Peter Van Epp <vanepp <at> sfu.ca> wrote:
On Mon, Nov 16, 2009 at 04:38:38PM -0500, Matt Sheridan wrote:
> I was able to get it figured out. I admit not being fully versed in my
> compile skills :). Adding the .debug to the compile directory was the
> clarification that I missed.
>
> Here are the results from stdout. Zombie processes were created when looking
> at ps:
>
> rastream -S localhost:561 -M time 10m -B 10s -f
> /opt/IDS/argus/etc/rastream.sh -w
> /opt/IDS/argus/log/\$srcid/argus.%Y_%m_%d_%H%M.out -D1
> rastream[14067.a08e7288f12a0000]: 16:03:10.090959 main: reading files
> completed
> rastream[14067.4019a54100000000]: 16:03:10.091388 Trying 127.0.0.1 port 561
> Expecting Argus records
> rastream[14067.4019a54100000000]: 16:03:10.091652 connected
> rastream[14067.4019a54100000000]: 16:03:10.091768 ArgusGetServerSocket
> (0x87df0010) returning 3
> rastream[14067.a08e7288f12a0000]: 16:10:15.474102 ArgusRunScript(0x87d41010,
> 21198c10) filename /opt/IDS/argus/log/127.0.0.1/argus.2009_11_16_1600.out
> rastream[14067.a08e7288f12a0000]: 16:10:15.474201 ArgusRunScript(0x87d41010,
> 0x21198c10) scheduling  /opt/IDS/argus/etc/rastream.sh -r
> /opt/IDS/argus/log/127.0.0.1/argus.2009_11_16_1600.out
> rastream[14067.a08e7288f12a0000]: 16:10:15.474240 ArgusRunScript(0x87d41010,
> 21198c10) returning  /opt/IDS/argus/etc/rastream.sh -r /opt/IDS/argus/log/
> 127.0.0.1/argus.2009_11_16_1600.out
> rastream[16367.a08e7288f12a0000]: 16:10:16.074641 ArgusRunScript calling
> /opt/IDS/argus/etc/rastream.sh -r /opt/IDS/argus/log/
> 127.0.0.1/argus.2009_11_16_1600.out
> deleting
<snip>

       While I haven't used rastream (let alone run scripts from it :-)), can
you tell what is zombieing? My first guess (although I'm not an expert on
zombies either :-)) would be that your script isn't returning a result to
rastream which is I think what causes a zombie. It would be instructive to
know what the original pid of one of the zombies is (as the pids are showing
in the argus debug output, although I don't seem to have left one :-)). That
should point to what task is causing the zombie.

Peter Van Epp


Carter Bullard | 9 Dec 19:40

Re: ra: window difference ?

Hey Julien,
What filter(s) are you using to generate your numbers?

Carter

On Nov 29, 2009, at 10:06 AM, julien wrote:

> Hello Carter,
> 
> Carter Bullard wrote on 27/11/09 16:41:
>> Argus will report the last window advertisement seen for the src and dst direction
>> in each status report interval and it will also indicate if the window went to zero
>> during that time.   This zero state is the TCP flow control indicator, and you will
>> see a "S", "D" or "@" indicator in column 5 of the flags field.
> [...]
> 
> thanks a lot for all this details
> 
>> 
>> So, what are you most interested in in this packet trace?  Is there a need to capture
>> more in the TCP windows metric?
> 
> that's not really a problem of capture. More about interpreting different results between Wireshark and
Argus from the same data.
> 
> When I speak about IP and Window, I will assume it's the same for everyone but here count/proportions are
reversed so ...
> 
> 
>> On Nov 26, 2009, at 3:35 PM, julien wrote:
>>> does someone know the difference between Wireshark "Window Space" (tcp.window_space) and Argus
"Window Advertisement" (swin/dwin) ?
>>> 
>>> I'm currently investigating a pcap representing a kind of DoS Synflood attack. The former returns
about 25k packets with size 0 a
>>> nd 230k with size<n>, the latter returns 130k&  25k (swin only) ???
>>> 
> 
> Why Wireshark would return 9% of packets with size 0 and the others with 0 (filter with tcp.windows_space
== 0 or n)
> and Argus returns 84% of flows with size 0 and the others with size 0 ? (with ra)
> 
> thanks
> 
> 

Attachment (smime.p7s): application/pkcs7-signature, 3815 bytes
julien | 9 Dec 23:02
Picon

Re: ra: window difference ?

Carter Bullard wrote on 09/12/09 19:40:
> What filter(s) are you using to generate your numbers?

here for wireshark

>> Why Wireshark would return 9% of packets with size 0 and the others with 0 (filter with tcp.windows_space
== 0 or n)
>> and Argus returns 84% of flows with size 0 and the others with size 0 ? (with ra)

and for argus, I make a chart with data from the following command:
ra -n -s swin -r $src_log

thanks

Carter Bullard | 10 Dec 17:36

Re: ra: window difference ?

Hey Julien,
In the file you uploaded, 131803 of the flows that are reporting source windows of
zero are flows that are simple single packet flows, so you and I are getting consistent
numbers, so I suspect that your numbers are real.

I can't explain the discrepancy, as I have looked at the code and your sample
file and there shouldn't be a problem.  With the numbers switched, you should be
able to find a packet in your file  where wireshark thinks there is a window number,
and the resulting argus record reports zero.

If you could create a packet file that has a packet or two that wireshark thinks
has a src window value but argus reports as zero, then I can debug.

Carter

On Dec 9, 2009, at 5:02 PM, julien wrote:

> Carter Bullard wrote on 09/12/09 19:40:
>> What filter(s) are you using to generate your numbers?
> 
> 
> here for wireshark
> 
>>> Why Wireshark would return 9% of packets with size 0 and the others with 0 (filter with
tcp.windows_space == 0 or n)
>>> and Argus returns 84% of flows with size 0 and the others with size 0 ? (with ra)
> 
> and for argus, I make a chart with data from the following command:
> ra -n -s swin -r $src_log
> 
> thanks
> 

Attachment (smime.p7s): application/pkcs7-signature, 3815 bytes
Carter Bullard | 10 Dec 17:42

Re: Rasplit compression support

Hey Matt,
I can't recreate your condition on any machines I have, and if you 
look at the code, and I think the debug output shows this, we are doing the
waitpid() needed to harvest the zombies, so I don't know how to
proceed.  If you want to provide me with access to the machine
I'll be happy to try to debug the problem.

This problem could be as simple as a poor PATH definition for the script.
Do you have a PATH definition or full path specification for commands
called in the script?

Carter

On Dec 9, 2009, at 11:43 AM, Matt Sheridan wrote:

Hi Carter -

I was wondering what your thoughts might be on this moving forward? The defunct processes continue to pile up.

Not sure if its related, but if I run a basic ra on a file, I am getting some other process complaints:

ra -r ./argus.2009_12_09_1100.out.gz - host 10.10.10.10
ra[4935]: 11:42:56.648522 ArgusReadStreamSocket (0x41fc) record length is zero

gzip: stdout: Broken pipe

On Tue, Nov 17, 2009 at 10:51 AM, Matt Sheridan <mattmail5050 <at> gmail.com> wrote:

I have an update. It seems that what is becoming a zombie is the script called by the "-f" option (you may have already figured that out, I just did ;)"

I attempted to use the default contrib script included in the clients package, and it also zombied.


But as you can see:

root      1716 31417  8 10:46 pts/1    00:00:10 rastream -S localhost:561 -M time 1m -B 10s -f /opt/IDS/argus/etc/rastream-orig.sh -
root      2057  1716  0 10:47 pts/1    00:00:00 [rastream-orig.s] <defunct>

it was the rastream-orig.sh script, called by the -f option (which is used to zip up the files) that zombied. So it is possible that the problem is not within rastream, but in the way the called script is handled?



On Mon, Nov 16, 2009 at 9:32 PM, Peter Van Epp <vanepp <at> sfu.ca> wrote:
On Mon, Nov 16, 2009 at 04:38:38PM -0500, Matt Sheridan wrote:
> I was able to get it figured out. I admit not being fully versed in my
> compile skills :). Adding the .debug to the compile directory was the
> clarification that I missed.
>
> Here are the results from stdout. Zombie processes were created when looking
> at ps:
>
> rastream -S localhost:561 -M time 10m -B 10s -f
> /opt/IDS/argus/etc/rastream.sh -w
> /opt/IDS/argus/log/\$srcid/argus.%Y_%m_%d_%H%M.out -D1
> rastream[14067.a08e7288f12a0000]: 16:03:10.090959 main: reading files
> completed
> rastream[14067.4019a54100000000]: 16:03:10.091388 Trying 127.0.0.1 port 561
> Expecting Argus records
> rastream[14067.4019a54100000000]: 16:03:10.091652 connected
> rastream[14067.4019a54100000000]: 16:03:10.091768 ArgusGetServerSocket
> (0x87df0010) returning 3
> rastream[14067.a08e7288f12a0000]: 16:10:15.474102 ArgusRunScript(0x87d41010,
> 21198c10) filename /opt/IDS/argus/log/127.0.0.1/argus.2009_11_16_1600.out
> rastream[14067.a08e7288f12a0000]: 16:10:15.474201 ArgusRunScript(0x87d41010,
> 0x21198c10) scheduling  /opt/IDS/argus/etc/rastream.sh -r
> /opt/IDS/argus/log/127.0.0.1/argus.2009_11_16_1600.out
> rastream[14067.a08e7288f12a0000]: 16:10:15.474240 ArgusRunScript(0x87d41010,
> 21198c10) returning  /opt/IDS/argus/etc/rastream.sh -r /opt/IDS/argus/log/
> 127.0.0.1/argus.2009_11_16_1600.out
> rastream[16367.a08e7288f12a0000]: 16:10:16.074641 ArgusRunScript calling
> /opt/IDS/argus/etc/rastream.sh -r /opt/IDS/argus/log/
> 127.0.0.1/argus.2009_11_16_1600.out
> deleting
<snip>

       While I haven't used rastream (let alone run scripts from it :-)), can
you tell what is zombieing? My first guess (although I'm not an expert on
zombies either :-)) would be that your script isn't returning a result to
rastream which is I think what causes a zombie. It would be instructive to
know what the original pid of one of the zombies is (as the pids are showing
in the argus debug output, although I don't seem to have left one :-)). That
should point to what task is causing the zombie.

Peter Van Epp


Attachment (smime.p7s): application/pkcs7-signature, 3815 bytes
Carter Bullard | 10 Dec 17:44

Re: Rasplit compression support

Hey Matt,
If you have data files that can't be read, if you can upload the file to
ftp://qosient.com/incoming (which is a blind repository) I can debug
the file.

It is possible that grabbing the latest copy of argus-clients-3.0.2.tar.gz (which
is frozen now)can solve the problem.

Carter

On Dec 9, 2009, at 11:43 AM, Matt Sheridan wrote:

Hi Carter -

I was wondering what your thoughts might be on this moving forward? The defunct processes continue to pile up.
Not sure if its related, but if I run a basic ra on a file, I am getting some other process complaints:

ra -r ./argus.2009_12_09_1100.out.gz - host 10.10.10.10
ra[4935]: 11:42:56.648522 ArgusReadStreamSocket (0x41fc) record length is zero

gzip: stdout: Broken pipe

On Tue, Nov 17, 2009 at 10:51 AM, Matt Sheridan <mattmail5050 <at> gmail.com> wrote:

I have an update. It seems that what is becoming a zombie is the script called by the "-f" option (you may have already figured that out, I just did ;)"

I attempted to use the default contrib script included in the clients package, and it also zombied.


But as you can see:

root      1716 31417  8 10:46 pts/1    00:00:10 rastream -S localhost:561 -M time 1m -B 10s -f /opt/IDS/argus/etc/rastream-orig.sh -
root      2057  1716  0 10:47 pts/1    00:00:00 [rastream-orig.s] <defunct>

it was the rastream-orig.sh script, called by the -f option (which is used to zip up the files) that zombied. So it is possible that the problem is not within rastream, but in the way the called script is handled?



On Mon, Nov 16, 2009 at 9:32 PM, Peter Van Epp <vanepp <at> sfu.ca> wrote:
On Mon, Nov 16, 2009 at 04:38:38PM -0500, Matt Sheridan wrote:
> I was able to get it figured out. I admit not being fully versed in my
> compile skills :). Adding the .debug to the compile directory was the
> clarification that I missed.
>
> Here are the results from stdout. Zombie processes were created when looking
> at ps:
>
> rastream -S localhost:561 -M time 10m -B 10s -f
> /opt/IDS/argus/etc/rastream.sh -w
> /opt/IDS/argus/log/\$srcid/argus.%Y_%m_%d_%H%M.out -D1
> rastream[14067.a08e7288f12a0000]: 16:03:10.090959 main: reading files
> completed
> rastream[14067.4019a54100000000]: 16:03:10.091388 Trying 127.0.0.1 port 561
> Expecting Argus records
> rastream[14067.4019a54100000000]: 16:03:10.091652 connected
> rastream[14067.4019a54100000000]: 16:03:10.091768 ArgusGetServerSocket
> (0x87df0010) returning 3
> rastream[14067.a08e7288f12a0000]: 16:10:15.474102 ArgusRunScript(0x87d41010,
> 21198c10) filename /opt/IDS/argus/log/127.0.0.1/argus.2009_11_16_1600.out
> rastream[14067.a08e7288f12a0000]: 16:10:15.474201 ArgusRunScript(0x87d41010,
> 0x21198c10) scheduling  /opt/IDS/argus/etc/rastream.sh -r
> /opt/IDS/argus/log/127.0.0.1/argus.2009_11_16_1600.out
> rastream[14067.a08e7288f12a0000]: 16:10:15.474240 ArgusRunScript(0x87d41010,
> 21198c10) returning  /opt/IDS/argus/etc/rastream.sh -r /opt/IDS/argus/log/
> 127.0.0.1/argus.2009_11_16_1600.out
> rastream[16367.a08e7288f12a0000]: 16:10:16.074641 ArgusRunScript calling
> /opt/IDS/argus/etc/rastream.sh -r /opt/IDS/argus/log/
> 127.0.0.1/argus.2009_11_16_1600.out
> deleting
<snip>

       While I haven't used rastream (let alone run scripts from it :-)), can
you tell what is zombieing? My first guess (although I'm not an expert on
zombies either :-)) would be that your script isn't returning a result to
rastream which is I think what causes a zombie. It would be instructive to
know what the original pid of one of the zombies is (as the pids are showing
in the argus debug output, although I don't seem to have left one :-)). That
should point to what task is causing the zombie.

Peter Van Epp



Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York  10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax



Attachment (smime.p7s): application/pkcs7-signature, 3815 bytes
Carter Bullard | 10 Dec 18:25

Re: Question about states

Hey Matt,
Found the problem.  Seems that you have a packet that is reporting a window size of
32M+ bytes, which is bigger than I expected.  My bad.  Here is a patch, and I'll include
this patch in the official argus-clients-3.0.2 release package which is now officially
frozen.

Give this patch a try, or download the newest argus-clients-3.0.2.tar.gz from the website.
http://qosient.com/argus.

Sorry for the inconvenience,

Carter

On Dec 1, 2009, at 2:31 PM, Matt Brewer wrote:

> Hi Carter,
> 
> The packets that cause the overflow are available online.  http://www.ddtek.biz/ctf_dc17_packets.tbz.torrent
> 
> If you convert the ctf_dc17.pcap000 to an Argus file then attempt to read it with RA with at least adding
swin and dwin fields it will crash.  I followed the instructions provided by Peter and nothing changed the output.
> 
> ===========================
> | Matt Brewer
> | CCNA
> | www.sheridantutorials.com
> ===========================
> 
> ----- Original Message -----
> From: carter <at> qosient.com
> To: "Matt Brewer" <hilather <at> gmail.com>
> Sent: Tuesday, December 1, 2009 11:46:52 AM GMT -05:00 US/Canada Eastern
> Subject: Re: [ARGUS] Question about states
> 
> Hey Matt,
> Need the packet file that is causing problems.
> Carter 
> Sent from my Verizon Wireless BlackBerry
> 
> -----Original Message-----
> From: Matt Brewer <hilather <at> gmail.com>
> Date: Fri, 27 Nov 2009 19:49:55 -0500 (GMT-05:00)
> To: Carter Bullard<carter <at> qosient.com>
> Subject: Re: [ARGUS] Question about states
> 
> Hi Carter,
> 
> Sorry for taking so long to reply, I've had the flu and been out of action for the last few days.  
> 
> I really like the way the -z flag works, translating the state changes to the TCP flag equivalent.  The only
thing I would change would be to have lower case letters represent the source flags and uppercase
represent destination flags.
> 
> Heres an example.  A TCP conversation:  
> Syn -> 
>   <- Synack
> Ack -> 
>   <-Fin
> Rst ->
> 
> Would be represented as sSFr
> 
> If its not too much to ask could you work this into a command line switch?
> 
> Also, Monday I'll be in the office again and should be able to perform the necessary debugging for the
buffer overflow. 
> 
> ===========================
> | Matt Brewer
> | CCNA
> | www.sheridantutorials.com
> ===========================
> 
> ----- Original Message -----
> From: "Carter Bullard" <carter <at> qosient.com>
> To: "Matt Brewer" <hilather <at> gmail.com>
> Cc: "argus-info" <argus-info <at> lists.andrew.cmu.edu>
> Sent: Tuesday, November 24, 2009 11:23:31 AM GMT -05:00 US/Canada Eastern
> Subject: Re: [ARGUS] Question about states
> 
> Hey Matt,
> In most cases, the only reason a field can't be printed the way you want,
> is because we don't have a command line name of letter to apply the
> option, usually because we didn't get around to doing it, or nobody asked
> for it.
> 
> OK, understand that the default status fields are simply a convenient way of
> presenting the TCP state progression in a simple fashion. We have more
> information in the Argus records, as we track the perceived TCP states per
> side of the connection, but how would you like to print them?
> 
> In raxml(), which is now obsolete (replaced with the '-M xml' option), we
> printed out the entire contents of the ARGUS_TCP_DSR.  That was argus-2.x.
> In argus-3.0, we have more and less TCP information available, depending
> on how you setup your argus().  So if you want some new TCP printing support,
> we should discuss what the information is, what it means, and how to print it.
> 
> Currently, the combined TCP status field (default TCP output with the -z option),
> tracks the progression of the TCP state machine, and the direction of the TCP control
> packets that establish the various TCP states is indicated by the assignment of Src
> and Dst for the flow.   Who sent the SYN ('s') ?  The Source.  Who sent the
> SYN_ACK ('S')?  The Destination.  In fact that is how the Source and Destination
> are determined, and in some cases of TCP mediated scans, the direction seems
> confused, because the scanner is violating the TCP state machine to get the
> targets to respond.   Notice that the use of lower case and upper case tracks the
> source and destination.
> 
> The states for TCP are derived from the indications that Argus stores in the TCP
> DSR.  We capture "SAW_SYN", "SAW_SYN_ACK", "ESTABLISHED", "SAW_FIN",
> "SAW_FIN_ACK", "SAW_RESET" on each side of the connection, as well
> as the state that argus() thinks the TCP connection is in.  And with each status
> report, we zero most of these bits, so we can know what control packets were sent,
> when.
> 
> So, with all of that, we can print out all that you ask, just need to know what syntax
> to use to print the fields.
> 
> The flags format, because its simply the OR'ing of the TCP flags during
> the status interval, you lose massive amounts of information, like direction
> etc.....   However, when you print the status field using the "-Z" option, you have
> the flags for the source, the an '_' and then the flags for the destination.
> 
> Maybe you haven't seen this because your default status field width isn't large
> enough to print the whole indicator?
> 
> "man" records are argus management records.  They convey status of the argus
> data source, which can be argus() or radium(), and act as keep alives for the
> argus transport stream.   I believe that they are described in the man pages.
> 
> Carter
> 
> On Nov 21, 2009, at 10:53 PM, Matt Brewer wrote:
> 
>> Hello,
>> 
>> I have a few questions and ideas I'd like to share.
>> 
>> Some of the flows in my captures are marked EC (This is with the -z option to show mimic tcp states) and I
haven't been able to determine what the C stands for, any idea what it means?
>> 
>> Also, I've been working with the states field a lot with the -z option, and its come to my attention that the
sSEfFR orientation of the flags doesn't actually give me indication of who sent what packets with what tcp
flags.  I'm aware I can remove the -z and use the -Zs or -Zd option and it will give me an output of with tcp flags
were set for each direction, but it would be nice to actually have both options displayed.  If I use the -Zb
option, it looks like it attempts to print out both states deliminated by an underscore, however the field
appears to be truncated to 5 characters and is often cut off. And even if it wasn't cut off, the tcp flags
aren't displayed it order of occurrence.
>> 
>> Is there any reason the state field cannot keep the occurrence of flags in order? It would be incredibly
useful to display the state of the flow in order. Ideally I would like to see the tcp flags as lower case
letters as the source produced flags, and upper case letters as destination produced flags.  A system like
this would allow us to easily determine which side of the flow tore down the TCP session, either via a FIN or RST.
>> 
>> Also, I've recently seen the "man" protocol in my captures.  What exactly is this supposed to indicate?
Some of the "man" flows are marked with STP or STA in the state field.  I am assuming that "man" refers to
switch management protocols? Correct me if I'm wrong.
>> 
>> Also, the "man" protocol flows with the state "STA" have zero destination or source packets. Which is
very bizarre, I do not understand how a flow could exist without packets.
>> 
>> Oh also, what is the "offset" field for? The man pages are rather vague "record byte offset in file or stream."
>> 
>> Thanks in advance, argus is a wonderful tool.
>> 
>> ===========================
>> | Matt Brewer
>> | CCNA
>> | www.sheridantutorials.com
>> ===========================
>> 
> 
> 
> 
> 
> 
> 
> 

Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York  10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax

Attachment (smime.p7s): application/pkcs7-signature, 3815 bytes
CS Lee | 11 Dec 01:19
Picon

Rastream call script zombie

hi matt,

May i know which OS platform are you on where you have zombie process for rastream call script, I remember I have it on FreeBSD but not on linux, therefore please do let me know further.

Cheers

--
Best Regards,

CS Lee<geek00L[at]gmail.com>

http://geek00l.blogspot.com
http://defcraft.net


Gmane