headers
K K | 1 Jul 2008 02:51
Picon

Re: migrate over to PCRE?

I prefer PCRE, have no problem converting to PCRE only.

I've been told PCRE runs faster, a definite consideration for
high volume sites. We've gone out of our way to simplify regexes
to avoid performance hits, with a faster expression parser,
 I could back down on optimizations.

What other modules do you foresee?

Kevin
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
Permalink | Reply | 0 comments |
headers
Balazs Scheidler | 1 Jul 2008 14:07
Picon

Re: Latest RHEL5 build doesn't contains fix for PID

On Tue, 2008-07-01 at 14:47 +0300, Kostyantyn Gushtin wrote:
> Hi,
> I just download and try latest build 2.1beta2
> (syslog-ng-2.1beta1-1.i386.rpm , libdbi8-0.8.2bb2-2.i386.rpm). 
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Are you sure you downloaded beta2? Can you show me the file size,
modification date, etc?

> But I have the same problem with PID macro as before, it contains non
> valid integer values(some text). I try syslog-ng rpm's for RHEL5.

--

-- 
Bazsi

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
Permalink | Reply | 2 comments |
headers
Kostyantyn Gushtin | 1 Jul 2008 15:31
Picon

Re: Latest RHEL5 build doesn't contains fix for PID

Balazs Scheidler wrote:
> On Tue, 2008-07-01 at 14:47 +0300, Kostyantyn Gushtin wrote:
>   
>> Hi,
>> I just download and try latest build 2.1beta2
>> (syslog-ng-2.1beta1-1.i386.rpm , libdbi8-0.8.2bb2-2.i386.rpm). 
>>     
>    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Are you sure you downloaded beta2? Can you show me the file size,
> modification date, etc?
>
>   
>> But I have the same problem with PID macro as before, it contains non
>> valid integer values(some text). I try syslog-ng rpm's for RHEL5.
>>     
>
>
>
>
>   
I download it again and install on clean machine but result was the same.
Some info from package :

Name        : syslog-ng                    Relocations: (not relocatable)
Version     : 2.1beta1                          Vendor: Balabit IT Ltd.
Release     : 1                             Build Date: Fri 18 Apr 2008 
07:31:08 PM EEST
Install Date: (not installed)               Build Host: sun64.balabit
Group       : System Environment/Daemons    Source RPM:
(Continue reading)

Permalink | Reply | 1 comment |
headers
Balazs Scheidler | 1 Jul 2008 19:27
Picon

Re: Latest RHEL5 build doesn't contains fix for PID

On Tue, 2008-07-01 at 16:31 +0300, Kostyantyn Gushtin wrote:
> Balazs Scheidler wrote: 
> > On Tue, 2008-07-01 at 14:47 +0300, Kostyantyn Gushtin wrote:
> >   
> > > Hi,
> > > I just download and try latest build 2.1beta2
> > > (syslog-ng-2.1beta1-1.i386.rpm , libdbi8-0.8.2bb2-2.i386.rpm). 
> > >     
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > 
> > Are you sure you downloaded beta2? Can you show me the file size,
> > modification date, etc?
> > 
> >   
> > > But I have the same problem with PID macro as before, it contains non
> > > valid integer values(some text). I try syslog-ng rpm's for RHEL5.
> > >     
> > 
> > 
> > 
> > 
> >   
> I download it again and install on clean machine but result was the
> same. 
> Some info from package :

> But I think now the problem is that there is no beta2 package for
> RHEL-5.
> For example for RHEL-4 there is syslog-ng-2.1beta2-1.i386.rpm, but for
> RHEL-5 there is syslog-ng-2.1beta1-1.i386.rpm
(Continue reading)

Permalink | Reply | 0 comments |
headers
Joe Shaw | 1 Jul 2008 19:52
Gravatar

Odd behavior with very small log_fifo_size()

Hi,

I was doing a little bit of testing with syslog-ng today, and I think
that with a very small log_fifo_size() it was dropping messages but
not telling me.

If I set my log_fifo_size to 1, here are the results:

[jshaw <at> ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ ./loggen -r 1000000
-s 1024000 -I 60 localhost 2000
average rate = 96.48 msg/sec, count=5789
[jshaw <at> ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ ls -lh /ita/jshaw/messages
-rw-------  1 jshaw ita 101M Jul  1 13:30 /ita/jshaw/messages

However, 1024000 * 5789 = 5.9 GB

Grepping for drop messages in the output file:

[jshaw <at> ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ grep -i drop
/ita/jshaw/messages

returns nothing.  I have stats_freq(5) in my config file, and that
messages file is the only output.  I do have other statistical lines,
which imply to me that no messages are dropped:

Jul  1 13:49:52 s_local <at> ars1dev7 syslog-ng[6344]: Log statistics;
processed='center(queued)=6045', processed='center(received)=6045',
processed='destination(d_file)=6045', processed='source(s_local)=6045'

But it seems clear to me that they are -- or loggen is lying.
(Continue reading)

Permalink | Reply | 5 comments |
headers
Ed Ravin | 2 Jul 2008 00:19
Picon
Favicon

syslog-ng-2.1beta1 on NetBSD 3.0, duplicate suppression docs?

I encountered two problems building syslog-ng-2.1beta1 on NetBSD 3.0:

1) I needed to patch loggen.c (stole this patch from a FreeBSD site,
apparently it happens on that OS too)

--- tests/loggen/loggen.c.orig  2008-01-13 00:04:22.000000000 -0800
+++ tests/loggen/loggen.c       2008-03-26 08:27:43.000000000 -0700
 <at>  <at>  -226,7 +226,9  <at>  <at> 
           memset(&hints, 0, sizeof(hints));
           hints.ai_family = AF_UNSPEC;
           hints.ai_socktype = sock_type;
+#ifdef AI_ADDRCONFIG
           hints.ai_flags = AI_ADDRCONFIG;
+#endif
           hints.ai_protocol = 0;
           if (getaddrinfo(argv[optind], argv[optind + 1], &hints, &res) != 0)
             {
-------------------------------------

2) I built syslog-ng with --enable-tcp-wrappers, which needlessly put
"-lwrap" on the LIBS line in the Makefiles for programs that did not
actually use TCP wrappers.  This causes link-time errors, since
libwrap requires you to declare two symbols in your program.  The fix
was to run this in the source directory:

    find tests/ -name Makefile -exec perl -p -i -e 's/-lwrap//' {} ";"

-------------------------------------

I also have a couple of observations/questions:
(Continue reading)

Permalink | Reply | 5 comments |
headers
chris packham | 2 Jul 2008 01:04
Picon
Favicon

Re: syslog-ng-2.1beta1 on NetBSD 3.0, duplicate suppression docs?

Hi Ed,

On Tue, 2008-07-01 at 18:19 -0400, Ed Ravin wrote:
> 4) I'm looking for info on how to use the duplicate suppression
> option - it doesn't seem to be in the docs included in the source.
> Can someone give me a quick summary?  This was (IMHO) the only
> extant flaw in syslog-ng and I'm looking forward to trying it out!

You just need to add suppress(<seconds>) to your destination config.

e.g. to suppress duplicate messages that occur within 60 seconds

destination d_file { file("/var/log/messages" template(t_logfile)
suppress(60)); };

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
Permalink | Reply | 501 comments |
headers
Balazs Scheidler | 2 Jul 2008 11:13
Picon

Re: Odd behavior with very small log_fifo_size()

On Tue, 2008-07-01 at 13:52 -0400, Joe Shaw wrote:
> Hi,
> 
> I was doing a little bit of testing with syslog-ng today, and I think
> that with a very small log_fifo_size() it was dropping messages but
> not telling me.
> 
> If I set my log_fifo_size to 1, here are the results:
> 
> [jshaw <at> ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ ./loggen -r 1000000
> -s 1024000 -I 60 localhost 2000
> average rate = 96.48 msg/sec, count=5789
> [jshaw <at> ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ ls -lh /ita/jshaw/messages
> -rw-------  1 jshaw ita 101M Jul  1 13:30 /ita/jshaw/messages
> 
> However, 1024000 * 5789 = 5.9 GB
> 
> Grepping for drop messages in the output file:
> 
> [jshaw <at> ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ grep -i drop
> /ita/jshaw/messages
> 
> returns nothing.  I have stats_freq(5) in my config file, and that
> messages file is the only output.  I do have other statistical lines,
> which imply to me that no messages are dropped:
> 
> Jul  1 13:49:52 s_local <at> ars1dev7 syslog-ng[6344]: Log statistics;
> processed='center(queued)=6045', processed='center(received)=6045',
> processed='destination(d_file)=6045', processed='source(s_local)=6045'
>
(Continue reading)

Permalink | Reply | 4 comments |
headers
Balazs Scheidler | 2 Jul 2008 13:04
Picon

Re: syslog-ng-2.1beta1 on NetBSD 3.0, duplicate suppression docs?

On Tue, 2008-07-01 at 18:19 -0400, Ed Ravin wrote:
> I encountered two problems building syslog-ng-2.1beta1 on NetBSD 3.0:
> 
> 1) I needed to patch loggen.c (stole this patch from a FreeBSD site,
> apparently it happens on that OS too)
> 
> --- tests/loggen/loggen.c.orig  2008-01-13 00:04:22.000000000 -0800
> +++ tests/loggen/loggen.c       2008-03-26 08:27:43.000000000 -0700
>  <at>  <at>  -226,7 +226,9  <at>  <at> 
>            memset(&hints, 0, sizeof(hints));
>            hints.ai_family = AF_UNSPEC;
>            hints.ai_socktype = sock_type;
> +#ifdef AI_ADDRCONFIG
>            hints.ai_flags = AI_ADDRCONFIG;
> +#endif
>            hints.ai_protocol = 0;
>            if (getaddrinfo(argv[optind], argv[optind + 1], &hints, &res) != 0)
>              {

Thanks. I've committed a fix for this issue.

> -------------------------------------
> 
> 2) I built syslog-ng with --enable-tcp-wrappers, which needlessly put
> "-lwrap" on the LIBS line in the Makefiles for programs that did not
> actually use TCP wrappers.  This causes link-time errors, since
> libwrap requires you to declare two symbols in your program.  The fix
> was to run this in the source directory:
> 
>     find tests/ -name Makefile -exec perl -p -i -e 's/-lwrap//' {} ";"
(Continue reading)

Permalink | Reply | 2 comments |
headers
Joe Shaw | 2 Jul 2008 19:21
Gravatar

Re: Odd behavior with very small log_fifo_size()

Hi,

On Wed, Jul 2, 2008 at 5:13 AM, Balazs Scheidler <bazsi <at> balabit.hu> wrote:
> syslog-ng does not log message loss on file destinations, because that'd
> clutter the log statistics line, on the asssumption that no messages are
> can be lost there.
>
> However this is not always the case, if you log_fifo_size() is smaller
> than the window size of incoming sources, messages can be lost.

Could something be added if -v were passed to syslog-ng then?  It's
(very) useful diagnostic information when you're tuning an
installation and I probably wouldn't have noticed if not for the data
that loggen outputs and the fact that the log file seemed small.

> Here is a description of how flow-control and window sizes work in
> syslog-ng:
>
> http://www.balabit.hu/dl/html/syslog-ng-admin-guide_en.html/ch08s03.html
>
> Please let me (and the doc team) know if the description could be made
> clearer somehow.

I hadn't seen this before -- I was only using the docs shipped with
the tarball -- and it's good information.

My setting of the log_fifo_size() in testing was for two reasons: (1)
to see how setting it affected memory usage when being hit with many
large messages and (2) to demonstrably see what its behavior was if it
was overloaded and had to drop messages.  I would have liked a little
(Continue reading)

Permalink | Reply | 1 comment |

Gmane