Re: Nondeterministic output
2010-09-30 23:21:09 GMT
Thanks. Look forward to testing the fix....my whole process is stdout pipelines...On Sep 30, 2010 2:34 AM, "Carter Bullard" <carter <at> qosient.com> wrote:
> Hey George,
> Well, there are a number of things going on at the same time, but I'm getting a grip
> on this issue. When writing to a file, argus seems to be very consistent. When
> writing to stdout, it seems to be having some issues with getting all the records out
> of the engine, and it seems that when we zero out a record (when we write a flow
> status record, we maintain the cache, but zero out the metrics) things may not behave
> as we would like, ....., but only when we have to queue records for output. We do this
> when we are delivering flow records to the output socket/file descriptor faster than they
> can be written out the device. When we have partially written a record, and we are
> still queuing outgoing flow records, we get into a bad situation where we only clear
> one record every turn, and we have a turn every 0.020 seconds, so we get really slow.
> That is where the trouble then begins.
> So the short story is, when you write to disk, all is good, when you write to stdout, all
> is not. I'm working this now, but it is a head scratcher and so it may take a few days.
> Sorry for the inconvenience,
> On Sep 21, 2010, at 11:05 AM, George Jones wrote:
>> The following command produces different output:
>> cat foo.pcap | argus -U 64 -r - -w /tmp/foo.ar
>> cat foo.pcap | argus -U 64 -r - -w /tmp/bar.ar
>> cksum(1) shows the content differs (but byte count is the same).
>> More disturbing is different numbers of records output from identical runs on the same input, etc.
>> cat foo.pcap | argus -U 64 -r - -w - | racluster -r - -w - | ra -r - | tee /tmp/1.out
>> cat foo.pcap | argus -U 64 -r - -w - | racluster -r - -w - | ra -r - | tee /tmp/2.out
>> results in slightly different output. Sometimes there are slight differences in the flgs (packet ordering, I think),
>> but in a file of several thousand records, I'm getting 6 or so additional records in one output file vs the other.
>> ---George Jones