Carter Bullard | 8 Aug 2007 14:53

argus instability problems

Gentle people,
I have new code on the server that attempts to address a number
of instability problems that were in the last release, primarily on
64-bit architectures.  I working the problem, we have improved
the threads model for argus and the clients, and so the code is
much better structured and will perform much better under load,
especially when there are multiple processors.

In particular this version should correct the "Zero length" error,
which did appear to be an argus() bug, rather than a radium() bug,
performance problems with radium(), where it would go idle
for many seconds at a time, and better handling of reliable connections
to multiple remotes for all the ra* programs.

Please give this a run, and hopefully this will be the final release
candidates for argus-3.0.0.

Carter

CS Lee | 8 Aug 2007 15:21
Picon

argus clients

Carter,

When compiling argus-clients 3RC 46, I encountered this error -

./argus_util.c: In function `ArgusNtoH':
./argus_util.c:12293: error: structure has no member named `ArgusCurrentInput'
./argus_util.c:12294: error: structure has no member named `ArgusCurrentInput'
./argus_util.c:12295: error: structure has no member named `ArgusCurrentInput'
./argus_util.c: In function `ArgusHtoN':
./argus_util.c:12745: error: structure has no member named `ArgusCurrentInput'
./argus_util.c:12746: error: structure has no member named `ArgusCurrentInput'
./argus_util.c:12747: error: structure has no member named `ArgusCurrentInput'
*** Error code 1

I'm on FreeBSD 6.2 Release.

--
Best Regards,

CS Lee<geekooL[at]gmail.com>

Carter Bullard | 8 Aug 2007 15:58

Re: argus clients

Yes, I screwed up and put up a version from the wrong machine.
A working copy is up now!!  I thought I had sneaked it in before
anyone got it.

Very sorry for the goof up

Carter

CS Lee wrote:
> Carter,
>
> When compiling argus-clients 3RC 46, I encountered this error -
>
> ./argus_util.c: In function `ArgusNtoH':
> ./argus_util.c:12293: error: structure has no member named 
> `ArgusCurrentInput'
> ./argus_util.c:12294: error: structure has no member named 
> `ArgusCurrentInput'
> ./argus_util.c:12295: error: structure has no member named 
> `ArgusCurrentInput'
> ./argus_util.c: In function `ArgusHtoN':
> ./argus_util.c:12745: error: structure has no member named 
> `ArgusCurrentInput'
> ./argus_util.c:12746: error: structure has no member named 
> `ArgusCurrentInput'
> ./argus_util.c:12747: error: structure has no member named 
> `ArgusCurrentInput'
> *** Error code 1
>
> I'm on FreeBSD 6.2 Release.
>
> -- 
> Best Regards,
>
> CS Lee<geekooL[at]gmail.com> 

Carter Bullard | 8 Aug 2007 16:21

Re: argus clients

Gentle people,
Already I have found some issues with racluster() in reading large
numbers of files recursively, with this new set of clients (inconsistent
total counts for bytes, suggesting that not all the records are
processed).  Please test this version of clients (rc.46) in a limited
way until I can do a thorough job of getting this new threads model
down. 

But please, do not hesitate to use the new argus(), as it is the primary
focus of this round of testing and should be stable.

Thanks, and sorry for the inconvenience!!!!!

Carter

Carter Bullard wrote:
> Yes, I screwed up and put up a version from the wrong machine.
> A working copy is up now!!  I thought I had sneaked it in before
> anyone got it.
>
> Very sorry for the goof up
>
> Carter
>
> CS Lee wrote:
>> Carter,
>>
>> When compiling argus-clients 3RC 46, I encountered this error -
>>
>> ./argus_util.c: In function `ArgusNtoH':
>> ./argus_util.c:12293: error: structure has no member named 
>> `ArgusCurrentInput'
>> ./argus_util.c:12294: error: structure has no member named 
>> `ArgusCurrentInput'
>> ./argus_util.c:12295: error: structure has no member named 
>> `ArgusCurrentInput'
>> ./argus_util.c: In function `ArgusHtoN':
>> ./argus_util.c:12745: error: structure has no member named 
>> `ArgusCurrentInput'
>> ./argus_util.c:12746: error: structure has no member named 
>> `ArgusCurrentInput'
>> ./argus_util.c:12747: error: structure has no member named 
>> `ArgusCurrentInput'
>> *** Error code 1
>>
>> I'm on FreeBSD 6.2 Release.
>>
>> -- 
>> Best Regards,
>>
>> CS Lee<geekooL[at]gmail.com> 
>
>

Carter Bullard | 8 Aug 2007 17:02

Re: argus clients

I have just now fixed the racluster() recursion problem and upload
a new rc.46 clients.  Please test this version with some care, just
as a conservative approach, but the basic thread model should now
be solid and ready to go.  I would love to be more confident, but
hopefully, we'll not find any major issues :o)

Thanks, and sorry for any confusion,

Carter

Carter Bullard wrote:
> Gentle people,
> Already I have found some issues with racluster() in reading large
> numbers of files recursively, with this new set of clients (inconsistent
> total counts for bytes, suggesting that not all the records are
> processed).  Please test this version of clients (rc.46) in a limited
> way until I can do a thorough job of getting this new threads model
> down.
> But please, do not hesitate to use the new argus(), as it is the primary
> focus of this round of testing and should be stable.
>
> Thanks, and sorry for the inconvenience!!!!!
>
> Carter
>
>
> Carter Bullard wrote:
>> Yes, I screwed up and put up a version from the wrong machine.
>> A working copy is up now!!  I thought I had sneaked it in before
>> anyone got it.
>>
>> Very sorry for the goof up
>>
>> Carter
>>
>> CS Lee wrote:
>>> Carter,
>>>
>>> When compiling argus-clients 3RC 46, I encountered this error -
>>>
>>> ./argus_util.c: In function `ArgusNtoH':
>>> ./argus_util.c:12293: error: structure has no member named 
>>> `ArgusCurrentInput'
>>> ./argus_util.c:12294: error: structure has no member named 
>>> `ArgusCurrentInput'
>>> ./argus_util.c:12295: error: structure has no member named 
>>> `ArgusCurrentInput'
>>> ./argus_util.c: In function `ArgusHtoN':
>>> ./argus_util.c:12745: error: structure has no member named 
>>> `ArgusCurrentInput'
>>> ./argus_util.c:12746: error: structure has no member named 
>>> `ArgusCurrentInput'
>>> ./argus_util.c:12747: error: structure has no member named 
>>> `ArgusCurrentInput'
>>> *** Error code 1
>>>
>>> I'm on FreeBSD 6.2 Release.
>>>
>>> -- 
>>> Best Regards,
>>>
>>> CS Lee<geekooL[at]gmail.com> 
>>
>>
>
>

Ting-Fang Yen | 9 Aug 2007 21:09
Picon

TCP header fields

Hi,

Since Argus produces flow records, I am wondering how the TCP header
fields, such as TTL, window size, sequence number, etc, are
determined. Is it set according to the first packet in the flow?

Thanks,
Ting-Fang

Carter Bullard | 9 Aug 2007 21:59

Re: TCP header fields

Hey Ting-Fang ,
For flow attributes that are generally not a part of the flow key, such
as mac addresses when you're doing classic 5 tuple tracking, you will  
get the
mac addresses seen in the first packet. We do this to minimize copying
of potentially large objects on every packet, and this rule applies  
generally
to encapsulation identifiers, such as addresses, mpls labels, vlan  
tags, etc.

Generally, for metrics we report the value in the last packet  
monitored on
the flow, with a few exceptions.   Some metrics, like stime, are  
specific
for the value in the first packet, and the TCP base sequence number.
But for general attributes, such as ttl, tos, and ip_id, these are all
from the last packet observed.  The ip_id is something that changes on
each packet (except for v4 fragments), so we want to know what the
last value is.  In some cases the ip_id can be used as a pseudo sequence
number, and so getting the last is a good thing.  Changes in ttl and tos
can indicate path changes for a flow, so we want to know the last value
seen.  For values that are generated through an accumulation method,
such as what encapsulations are used on the flow, counters and status
fields, i.e. TCP flags values where we OR the flag values into the  
reported
value, the value represents all observed until the last packet seen.

The few exceptions to the last packet seen rule, is the TCP window size
which is not updated from RST or FIN packets, as these usually have
zero as the window value.  Reporting that value is pretty useless.  So
for window, you should see the last valid number in during the  
observation period.

There are quite a number of sequence numbers for TCP, base, last obs,  
last
ack'd, etc.. so the base seq number that argus tracks comes from the  
first packet
seen in a reporting interval, and others are generally the last value  
seen.

May seem complicated but it gets the semantics that are useful for a  
set of
problems.  If you seen something that seems odd, don't hesitate to  
discuss it
in email.

Carter

Ting-Fang Yen wrote:
> Hi,
>
> Since Argus produces flow records, I am wondering how the TCP header
> fields, such as TTL, window size, sequence number, etc, are
> determined. Is it set according to the first packet in the flow?
>
> Thanks,
> Ting-Fang
>
>

Peter Van Epp | 10 Aug 2007 18:26
Picon
Picon
Favicon

Re: argus instability problems

	I'm still seeing problems (time dsr not set, followed by apparant
argus malloc corruption) on my test 64 bit PowerPC argus sensor. Carter and
I are poking at this to see if we can identify why, but I expect it will be
of interest to know if anyone else is seeing the problem as well and if so
what architecture (Intel, PPC), OS and size (32 or 64 bit) would be useful. 
The problem I'm seeing involves a syslog message like this (and the argus
shutting down which in my case involves a seg fault in argus_free):

Aug  9 13:54:32 hcids argus[2945]: 09 Aug 07 13:54:32.047332 ArgusGenerateRecord: time dsr not set

	If I'm the only one seeing this it may be something local but I don't
think I am the only one (this is the cause of the 0 length records in the
clients as well I believe). 

Peter Van Epp / Operations and Technical Support 
Simon Fraser University, Burnaby, B.C. Canada

Russell Fulton | 15 Aug 2007 23:42
Picon
Picon
Favicon

Weird behaviour from argus

I have some weird problems, this (or something very similar) is
happening on two sensors.  When I run ra on a file (any file) I get back
just two records and then get the error:

 ra[20871]: 09:18:05.640627 ArgusReadStreamSocket (0xb740d954) record
length is zero

either on the console or (as in this case) in syslog.

In both cases I am running argus-3.0.0 as the server and the RC46 and
RC39 clients.

Luckily (or was it good planning ;) I have other sensors on the path of
the traffic I was tracking and this on is working fine.

I admit not paying much attention to the list traffic recently so my
apologies if this has come up and I missed it.

Carter: I can upload one of these files to the ftp server if you want to
do diagnostics.

OH yes! the platform is FC6.

Russell.

Carter Bullard | 16 Aug 2007 01:23

Re: Weird behaviour from argus

Hey Russell,
Yes, Peter and I are working on this very problem, and in the process
have generated quite a bit of dust, but we're getting it cleaned up.
If you can upload the file, I am working on putting automatic recovery
in the file data processing so we can recover from this error (basically
we're putting out some zero's when we're not really suppose to, and
good records start again at the end of the string of zeros, at least  
that
is the behavior I've seen so far).

Sorry, we should have an update hopefully very soon!!!

Carter

On Aug 15, 2007, at 5:42 PM, Russell Fulton wrote:

> I have some weird problems, this (or something very similar) is
> happening on two sensors.  When I run ra on a file (any file) I get  
> back
> just two records and then get the error:
>
>  ra[20871]: 09:18:05.640627 ArgusReadStreamSocket (0xb740d954) record
> length is zero
>
> either on the console or (as in this case) in syslog.
>
>
> In both cases I am running argus-3.0.0 as the server and the RC46 and
> RC39 clients.
>
> Luckily (or was it good planning ;) I have other sensors on the  
> path of
> the traffic I was tracking and this on is working fine.
>
> I admit not paying much attention to the list traffic recently so my
> apologies if this has come up and I missed it.
>
> Carter: I can upload one of these files to the ftp server if you  
> want to
> do diagnostics.
>
> OH yes! the platform is FC6.
>
> Russell.
>


Gmane