headers
Carter Bullard | 3 May 2012 00:57

argus-clients-3.0.6 rabins, rastream memory leak

Gentle people,
We've got a memory leak in the newest versions of rabins.1 and rastream.1.
I'm working on them now, but I have confirmed that long lived rastream processes
do collection the memory.  If you are using rastream.1, please consider testing
the patches when I make them available.

Should have a fix for you later tomorrow, which I will send to the list.

Hope all is most excellent,
Carter
Attachment (smime.p7s): application/pkcs7-signature, 4367 bytes
Permalink | Reply | 0 comments |
headers
Carter Bullard | 7 May 2012 18:45

patch file for rabins and rastream memory leak fix

Gentle people,
I have uploaded  argus-clients-3.0.6.1 to the developers site, and I
have uploaded a patch, for those that like patches, that fixes the reported
problems with rabins and rastream.  The fixes addresses the "-f racluster.conf"
issues with rabins.1, as well as the memory leak problems with long
term use of rabins and rastream, which occur primarily with ARGUS_EVENT
records.  

Apply the patch by being in the parent directory that contains argus-clients-3.0.6,
and then:

   % patch -p0 < patch-memory-leak-clients-3-0-6.diff

For those that have a patch that is sensitive to RCS, SCCS, Perforce, whatever, try:

   % patch -g0 -p0 < patch-memory-leak-clients-3-0-6.diff

The you will just need to make, as there are no configure.ac changes.

   % cd argus-clients-3.0.6; make

The patch will upgrade argus-clients-3.0.6 to argus-clients-3.0.6.1.

If you use rastream.1, you will want to upgrade to argus-3.0.6.1.
For those that want to use rabins.1 with the "-f racluster.conf", you will also
need to upgrade.

Please take a look at these, and if you have any issues, please holler !!!!
I'll be "releasing" argus-clients-3.0.6.1 after the patch has gone through some
testing, possibly at the end of the week.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Carter Bullard | 9 May 2012 00:11

updated patch for memory leaks on the server

Gentle people,
I have uploaded a new patch-all-memory-leak-clients-3-0-6.diff to the developers site,
which adds a memory leak fix for radium, when it is configured to generate geo labels
for flows its processing.

This fixes all known memory leak issues in argus-clients-3.0.6.
I've updated a new argus-clients-3.0.6.1.tar.gz with these new memory leak fixes.
If you could give these a try on your systems, that would be great.  

I will release these final tarfiles and patches tomorrow or Thur.


Apply the patch by being in the parent directory that contains argus-clients-3.0.6,
and then:

   % patch -p0 < patch-memory-leak-clients-3-0-6.diff

For those that have a patch that is sensitive to RCS, SCCS, Perforce, whatever, try:

   % patch -g0 -p0 < patch-memory-leak-clients-3-0-6.diff

The you will just need to make, as there are no configure.ac changes.

   % cd argus-clients-3.0.6; make

The patch will upgrade argus-clients-3.0.6 to argus-clients-3.0.6.1.

If you had already patch your files, you should still be able to use the new patch
file to add the changes.  It may complain a bit about already having the patch, but
that would be fine.

Carter

Attachment (smime.p7s): application/pkcs7-signature, 4367 bytes
Permalink | Reply | 0 comments |
headers
Russell Fulton | 9 May 2012 02:47
Picon
Picon
Favicon

problems using ARGUS_OUTPUT_FILE and 'filter'

When I use this filter on the command line it works as expected but when applied in the -F argus.conf I get all
sorts of stuff that I does not match the filter?

argus-3.0.6.

Makes no difference if I put the filter on the FILTER = or the ARGUS_OUTPUT_FILE.

ARGUS_OUTPUT_FILE=/home/sensors/data/wireless/argus/user-data "tcp and src net ( 172.23.0.0/16 or
172.24.0.0/18 ) and not dst net ( 172.23.0.0/16 or 172.24.0.0/18 )  and dst port 80"

see attached file...

I've used this before often so I am very puzzled - on another sensor I have an identical config but with a (
130.216.0.0/16 )  rather than the 172 addresses.

Clearly I'm doing something stupid, but what?

Any ideas?

Russell

PS:

sample of captured flows with -F argus-user-data.conf (attached)

   12:12:31.589938  e         tcp       207.46.61.90.80        ?>    130.216.181.194.57572         1         60   RST
   12:12:31.766180  e         tcp     74.125.237.102.443       ?>     130.216.215.70.1629          1         60   RST
   12:12:31.810594  e         tcp      203.97.30.168.80        ?>    130.216.211.104.64352         1         60   RST
   12:12:32.025105  e         tcp     67.195.186.237.80        ?>     130.216.180.70.63388         1         60   RST
   12:12:32.055496  e         tcp    108.166.124.101.8081      ?>     130.216.185.49.49962         1         60   RST
   12:12:32.284281  e         tcp    203.167.141.160.80        ?>     130.216.45.214.50546         1         60   RST
   12:12:32.128061  e         tcp    203.167.141.160.80        ?>    130.216.235.214.3909          1         60   RST
   12:12:32.313530  e         tcp     74.125.237.103.443       ?>     130.216.215.70.1630          2        120   RST
   12:12:32.407612  e         tcp       157.56.52.37.443       ?>    130.216.138.119.55450         1         60   RST
   12:12:32.419595  e         tcp      203.97.30.166.443       ?>    130.216.138.180.56112         2        120   RST
   12:12:33.058135  e         tcp    203.167.141.160.80        ?>     130.216.64.209.49393         1         60   RST
   12:12:33.125684  e         tcp       193.1.122.17.80        ?>     130.216.109.13.1509          1         60   RST
   12:12:33.216804  e         tcp       96.7.131.235.80        ?>     130.216.73.113.62942         1         60   RST
   12:12:32.840228  e         tcp      203.97.30.172.80        ?>    130.216.211.104.64353         1         60   RST

sample from
sudo /usr/sbin/argus -w  data/wireless/argus/user-data -i eth1 - tcp and src net \( 172.23.0.0/16 or
172.24.0.0/18 \) and not dst net \( 172.23.0.0/16 or 172.24.0.0/18 \)  and dst port 80

   12:30:11.657737  e         tcp      172.23.210.58.52203     ?>     205.251.203.89.80           16       1056   CON
   12:30:11.657816  e         tcp     172.23.200.226.50630     ?>    208.117.254.157.80           43       4484   CON
   12:30:11.658591  e         tcp      172.23.63.238.62230     ?>     119.31.253.194.80            6        396   CON
   12:30:11.658597  e         tcp     172.23.189.245.63528     ?>    203.167.141.154.80            4        288   CON
   12:30:11.658740  e         tcp      172.23.51.133.56828     ?>        91.121.5.64.80           72       5732   CON
   12:30:11.659003  e         tcp      172.23.191.23.49648     ?>     208.43.117.186.80          105     151586   CON
   12:30:11.659135  e         tcp      172.23.97.109.54534     ?>      183.60.157.73.80           36       2410   CON
   12:30:11.659852  e         tcp      172.24.12.211.52737     ?>    205.196.120.161.80           50       3054   CON
   12:30:11.659857  e         tcp      172.24.12.211.52681     ?>    205.196.122.242.80           51       3060   CON
   12:30:11.659868  e         tcp      172.24.12.211.52750     ?>     199.91.152.194.80           43       2670   CON
   12:30:11.659870  e         tcp      172.24.12.211.52761     ?>     199.91.152.194.80           38       2316   CON
   12:30:11.660339  e         tcp     172.23.151.151.38450     ?>     188.132.173.12.80           22       1452   FIN
Attachment (argus-user-data.conf): application/octet-stream, 7156 bytes
Permalink | Reply | 1 comment |
headers
Russell Fulton | 9 May 2012 06:48
Picon
Picon
Favicon

Re: problems using ARGUS_OUTPUT_FILE and 'filter'

This gets weirder -- my argus instance on other sensors are also ignoring the filter on the
ARGUS_OUTPUT_FILE line too.  I just never noticed that I have a heap more flows than I should in all my argus
files :(

Russell

On 9/05/2012, at 12:47 PM, Russell Fulton wrote:

> When I use this filter on the command line it works as expected but when applied in the -F argus.conf I get all
sorts of stuff that I does not match the filter?
> 
> argus-3.0.6.
> 
> Makes no difference if I put the filter on the FILTER = or the ARGUS_OUTPUT_FILE.
> 
> ARGUS_OUTPUT_FILE=/home/sensors/data/wireless/argus/user-data "tcp and src net ( 172.23.0.0/16
or 172.24.0.0/18 ) and not dst net ( 172.23.0.0/16 or 172.24.0.0/18 )  and dst port 80"
> 
> see attached file...
> 
> I've used this before often so I am very puzzled - on another sensor I have an identical config but with a (
130.216.0.0/16 )  rather than the 172 addresses.
> 
> Clearly I'm doing something stupid, but what?
> 
> Any ideas?
> 
> Russell
> 
> PS:
> 
> sample of captured flows with -F argus-user-data.conf (attached)
> 
>   12:12:31.589938  e         tcp       207.46.61.90.80        ?>    130.216.181.194.57572         1         60   RST
>   12:12:31.766180  e         tcp     74.125.237.102.443       ?>     130.216.215.70.1629          1         60   RST
>   12:12:31.810594  e         tcp      203.97.30.168.80        ?>    130.216.211.104.64352         1         60   RST
>   12:12:32.025105  e         tcp     67.195.186.237.80        ?>     130.216.180.70.63388         1         60   RST
>   12:12:32.055496  e         tcp    108.166.124.101.8081      ?>     130.216.185.49.49962         1         60   RST
>   12:12:32.284281  e         tcp    203.167.141.160.80        ?>     130.216.45.214.50546         1         60   RST
>   12:12:32.128061  e         tcp    203.167.141.160.80        ?>    130.216.235.214.3909          1         60   RST
>   12:12:32.313530  e         tcp     74.125.237.103.443       ?>     130.216.215.70.1630          2        120   RST
>   12:12:32.407612  e         tcp       157.56.52.37.443       ?>    130.216.138.119.55450         1         60   RST
>   12:12:32.419595  e         tcp      203.97.30.166.443       ?>    130.216.138.180.56112         2        120   RST
>   12:12:33.058135  e         tcp    203.167.141.160.80        ?>     130.216.64.209.49393         1         60   RST
>   12:12:33.125684  e         tcp       193.1.122.17.80        ?>     130.216.109.13.1509          1         60   RST
>   12:12:33.216804  e         tcp       96.7.131.235.80        ?>     130.216.73.113.62942         1         60   RST
>   12:12:32.840228  e         tcp      203.97.30.172.80        ?>    130.216.211.104.64353         1         60   RST
> 
> sample from
> sudo /usr/sbin/argus -w  data/wireless/argus/user-data -i eth1 - tcp and src net \( 172.23.0.0/16 or
172.24.0.0/18 \) and not dst net \( 172.23.0.0/16 or 172.24.0.0/18 \)  and dst port 80
> 
>   12:30:11.657737  e         tcp      172.23.210.58.52203     ?>     205.251.203.89.80           16       1056   CON
>   12:30:11.657816  e         tcp     172.23.200.226.50630     ?>    208.117.254.157.80           43       4484   CON
>   12:30:11.658591  e         tcp      172.23.63.238.62230     ?>     119.31.253.194.80            6        396   CON
>   12:30:11.658597  e         tcp     172.23.189.245.63528     ?>    203.167.141.154.80            4        288   CON
>   12:30:11.658740  e         tcp      172.23.51.133.56828     ?>        91.121.5.64.80           72       5732   CON
>   12:30:11.659003  e         tcp      172.23.191.23.49648     ?>     208.43.117.186.80          105     151586   CON
>   12:30:11.659135  e         tcp      172.23.97.109.54534     ?>      183.60.157.73.80           36       2410   CON
>   12:30:11.659852  e         tcp      172.24.12.211.52737     ?>    205.196.120.161.80           50       3054   CON
>   12:30:11.659857  e         tcp      172.24.12.211.52681     ?>    205.196.122.242.80           51       3060   CON
>   12:30:11.659868  e         tcp      172.24.12.211.52750     ?>     199.91.152.194.80           43       2670   CON
>   12:30:11.659870  e         tcp      172.24.12.211.52761     ?>     199.91.152.194.80           38       2316   CON
>   12:30:11.660339  e         tcp     172.23.151.151.38450     ?>     188.132.173.12.80           22       1452   FIN
> 
> 
> <argus-user-data.conf>
>
Permalink | Reply | 0 comments |
headers
Carter Bullard | 9 May 2012 19:05

new client patches for cygwin compile problems

Gentle people,
We are now in a period where I'm fixing gotcha's after the official release of argus-3.0.6.
We now have two sets of fixes, a set of memory leaks and, now, a set of fixes for cygwin.
This looks to be the only issues right now.

The next round of released code will be argus-clients-3.0.6.1, and because
these are show stopper problems, I'll officially release this next version soon.

Until I hear from the list, my strategy for testing is to put up a single patch file 
that represents the 3.0.6 to 3.0.6.1 upgrade, patch-argus-clients-3-0-6-1.diff, and
to provide an accompanying argus-clients-3.0.6.1.tar.gz file.   These are only
on the developers side of the site.  When all is well and good, I'll move these two
files to the stable side of the server.

As a result, I've uploaded patch-argus-clients-3-0-6-1-diff to the developers site,
and a new argus-clients-3.0.6.1.tar.gz.  The patch file should be applied to the
original argus-clients-3.0.6 distribution.  But, you can apply it to a previously
patch distribution, if that is your preference.  Just type carriage return in response
to patch's complaints about a previously patched chunk. 

Again I will release these final tarfiles and patches tomorrow being Thur if no other
issues come up.

   http://qosient.com/argus/dev/patch-argus-clients-3-0-6-1-diff

Please send any comments / opinions / complaints / whatever to either me or the list !!!!
Hope all is most excellent !!!!!!

Carter 
Attachment (smime.p7s): application/pkcs7-signature, 4367 bytes
Permalink | Reply | 0 comments |
headers
Torbjorn Wictorin | 16 May 2012 14:01
Picon
Picon
Favicon

Cisco non-flow data.

Hello,

how are the plans regarding argus handling of cisco non-flow data?
In common/argus_import.c there is a routine ArgusParseCiscoRecordV9Data 
that I assume is supposed to do that, but it is a dummy in 3.0.6.
Any plans for the (reasonably near) future?

Torbjörn Wictorin,
Uppsala university
Permalink | Reply | 1 comment |
headers
Carter Bullard | 16 May 2012 15:57

Re: Cisco non-flow data.

Hey Torbjorn,
Yes netflowV9 and full sflow support is a todo item for argus-3.0.8, which should start in June.
I restructured all the "foreign" flow support into the argus-import.c file in 3.0.6, and put a huge
push into flow-tools (100%) and sflow support (60%).  I actually have much of the netflowV9
stuff done, but the sample netflowV9 data that I was using had some errors in it, so I didn't
have it ready for the 3.0.6 release.

So, do you want to be a test site?

I'm hoping to have all of this done in a developers release by the end of June.

Carter

On May 16, 2012, at 8:01 AM, Torbjorn Wictorin wrote:

> Hello,
> 
> how are the plans regarding argus handling of cisco non-flow data?
> In common/argus_import.c there is a routine ArgusParseCiscoRecordV9Data 
> that I assume is supposed to do that, but it is a dummy in 3.0.6.
> Any plans for the (reasonably near) future?
> 
> Torbjörn Wictorin,
> Uppsala university
Attachment (smime.p7s): application/pkcs7-signature, 4367 bytes
Permalink | Reply | 0 comments |
headers
Matt Brown | 25 May 2012 03:00
Picon
Gravatar

Full docs about ra output?

Hello,

I see the man page for ra, but it seems lacking for some DSR value output.  For instance, there are somethings that aren't implicit, but appear like they should/were intended to be.

Specifically, I see this with 'dir's possible values.

The cases if confusion are <? And ?>.  How can argus "not" know the direction of the transaction "sort of?"

Thanks,

Matt

Permalink | Reply | 1 comment |
headers
Mark Poepping | 25 May 2012 03:39
Picon
Favicon

Re: Full docs about ra output?

Taking a stab (trying to relieve Carter of some of the burden)…

 

For directionality specifically, if it’s a well-defined protocol and argus saw most (if not all) of the packets from the beginning, it will know the direction, but there are many examples of ordinary and hybrid protocols where you won’t necessarily know the direction in all cases: peer-to-peer, ICMP, UDP can all make it hard to understand direction – or direction might not have meaning.  Packet loss (esp. packet sampling) often causes this output, and multi-path routing will ‘look like’ packet loss too, depending on where you’re watching and how your paths are advertised or have evolved over time.

 

On a simple, lightly loaded network (my house), long-running argus probes generally get the directionality right.

At my work, it’s not so simple so it helps to interact with questions that we have for the data and considerations of probe location and efficiency given the use cases.

 

Hope that helps some, it takes a little getting used to.  If you have specific questions or confusions, it does help to snap a packet capture that displays your confusion – that way others may be work with them directly and try to help you (with no explicit promises, of course).

Mark.

 

 

From: argus-info-bounces+poepping=cmu.edu <at> lists.andrew.cmu.edu [mailto:argus-info-bounces+poepping=cmu.edu <at> lists.andrew.cmu.edu] On Behalf Of Matt Brown
Sent: Thursday, May 24, 2012 9:00 PM
To: argus-info <at> lists.andrew.cmu.edu
Subject: [ARGUS] Full docs about ra output?

 

Hello,

I see the man page for ra, but it seems lacking for some DSR value output.  For instance, there are somethings that aren't implicit, but appear like they should/were intended to be.

Specifically, I see this with 'dir's possible values.

The cases if confusion are <? And ?>.  How can argus "not" know the direction of the transaction "sort of?"

Thanks,

Matt

Permalink | Reply | 0 comments |

Gmane