bugzilla | 23 Apr 16:31 2014

[Bug 279] New: Syslog-ng central loging server seg fault gentoo

https://bugzilla.balabit.com/show_bug.cgi?id=279

           Summary: Syslog-ng central loging server seg fault gentoo
           Product: syslog-ng
           Version: 3.4.x
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: critical
          Priority: unspecified
         Component: syslog-ng
        AssignedTo: bazsi <at> balabit.hu
        ReportedBy: hlavacek <at> gmx.com
Type of the Report: bug
   Estimated Hours: 0.0

Hello, 

we have set up the following topology: 
OS gentoo: 
Linux syslog1 3.8.12-hardened #1 SMP Wed Sep 18 16:30:57 CEST 2013 x86_64 Intel(R) Xeon(TM) CPU 3.00GHz
GenuineIntel GNU/Linux 

syslog: 
syslog server v3.4.7 and 40 syslog clients v3.x.x. 

problem: 
After a few days (5 or 10) server experienced segfault: 
[309942.605750] syslog-ng[17117]: segfault at 8 ip 00007fdb01e66609 sp 00007fdafe1ccac0 error 6 in
libsyslog-ng-3.4.7.so[7fdb01e2b000+8f000] 
(Continue reading)

David Hauck | 18 Apr 17:40 2014

Pattern DB Parser "Default Values"

Hi,

I was wondering if there was a way to specify default values for pattern DB parsers that include a value, but
where the parsed value is <null>[/empty]?

In particular if I have something like the following:

          <pattern>test message; field1= <at> ESTRING:field1:  <at> field2= <at> ESTRING:field2:  <at> field3= <at> ESTRING::
 <at> field4= <at> ESTRING:field4:  <at> </pattern>

I'd like to be able to do something like either, 1:

          <pattern>test message; field1= <at> ESTRING:field1<foo>:  <at> field2= <at> ESTRING:field2<bar>:
 <at> field3= <at> ESTRING::  <at> field4= <at> ESTRING:field4<beef>:  <at> </pattern>

Or 2:

          <pattern>test message; field1= <at> ESTRING:field1:  <at> field2= <at> ESTRING:field2:  <at> field3= <at> ESTRING::
 <at> field4= <at> ESTRING:field4:  <at> </pattern>
        <values>
          <value name="field1.default">foo</value>
          <value name="field2.default">bar</value>
          <value name="field4.default">beef</value>

Just curious...

Thanks,
-David
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
(Continue reading)

Bendler, Ehren | 16 Apr 15:46 2014

syslog-ng does not start if destination host not found

In syslog-ng 3.5.4.1, this seems to be a recurrence of an issue that has appeared several times before over
the years, most recently said to be fixed/changed in v3.3.5 here
(https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660897). We had previously been using
syslog-ng 3.0.4, which would fail this test gracefully and retry the connection based on the
time_reopen() parameter.

We don't use DNS, but we generate a hard-coded /etc/hosts file after startup that associates internal
names with a customer chosen subnet address so that we can collect logs without editing the
syslog-ng.conf file while we are running.  As in, we know that the primary controller will always be
present with the name "CTXP", but we can't promise that it will have a particular address because the
customer can change it, so we have to write it out to /etc/hosts at a point in the startup process *after*
syslog-ng is started.

  * Starting syslog-ng                      
 Error resolving hostname; host='CTXP' 
 Error initializing message pipeline;
..[FAIL(2)]

If this is the intended behavior, that's fine too. We can deploy our own patch to the afsocket module if it
isn't going to be changed in a release.

Another thing I need help with (when I use a hard-coded IP in syslog-ng.conf for testing) is how to get this
message to go away:
  * Starting syslog-ng                      
 WARNING: window sizing for tcp sources were changed in syslog-ng 3.3, the configuration value was divided
by the value of max-connections(). The result was too small, clamping to 100 entries. Ensure you have a
proper log_fifo_size setting to avoid message loss.; orig_log_iw_size='10',
new_log_iw_size='100', min_log_fifo_size='12000'
[OK]

(Continue reading)

David Hauck | 16 Apr 01:40 2014

pdbtool 'patternize'

Hello,

Does anyone have an explanation for why a "pdbtool patternize" generated pattern db indicates it is
version '3'? I'm running the latest version of syslog-ng (3.5.4.1) so I was expecting that this would
produce a version '4' pattern db. Easy enough to change in the generated XML, just wondering why the latest
generator wouldn't create the latest version.

Also, what is the nominal format for the log messages that the 'patternize' command is able to process
(i.e., would this be logs that contain the nominally formatted syslog-ng output - e.g., via the default
template: template("$ISODATE $HOST $MSGHDR$MSG\n");). I've seen some output that appears to suggest
there's some nominal decoding of the input log messages.

Thanks,
-David
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

David Hauck | 15 Apr 19:31 2014

Millisecond Resolution Timestamps

Hello,

I'm using the following global options in order to format messages with millisecond resolution:

  ts_format(iso);
  frac_digits(3);

Although the initial (syslog-ng starting) message appears to include sub-second resolution subsequent
messages do not:

20140415 10:19:23.590 notice syslog(syslog-ng):syslog-ng starting up; version='3.5.4.1'
...
20140415 10:19:33.000 notice authpriv(su):FAILED su for test by root
...
20140415 10:20:11.000 notice user(root):test

This includes messages originating from any number of sources (including all processes that log via
syslog()), *except* messages originating from the kernel (these always seem to have sub-second
resolution). Does anyone have any ideas what might be going on here?

Thanks,
-David

PS: the timestamp formatting above is done via a simple template. Regardless of this (re-)formatting
nominal iso messages also exhibit this limitation. For e.g.,:

2014-04-14T15:23:48.000-07:00 host99738728 nasysconfd: exit code 0 for
/netacquire/bin/sysconf/osinfo read

______________________________________________________________________________
(Continue reading)

bugzilla | 15 Apr 07:53 2014

[Bug 278] New: Many memory leak problems happen when do configuration reloading

https://bugzilla.balabit.com/show_bug.cgi?id=278

           Summary: Many memory leak problems happen when do configuration
                    reloading
           Product: syslog-ng
           Version: 3.5.x
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: unspecified
         Component: syslog-ng
        AssignedTo: bazsi <at> balabit.hu
        ReportedBy: xufeng.zhang <at> windriver.com
Type of the Report: ---
   Estimated Hours: 0.0

Created an attachment (id=95)
 --> (https://bugzilla.balabit.com/attachment.cgi?id=95)
the config and script files to reproduce this problem

I found many memory leak problems on syslog-ng_3.5.4.1, I have resolved some
of them, but there are still some others need to be resolved:
1). mutex problem such as:
==1077== 40 bytes in 1 blocks are definitely lost in loss record 514 of 636
==1077==    at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==1077==    by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0)
==1077==    by 0x36886024FE: ??? (in /lib64/libgthread-2.0.so.0.2600.0)
==1077==    by 0x3688266B23: g_static_mutex_get_mutex_impl (in /lib64/libglib-2.0.so.0.2600.0)
==1077==    by 0x523D83C: affile_dw_queue (affile-dest.c:263)
(Continue reading)

Xufeng Zhang | 14 Apr 08:35 2014

[Ask for help] How to resolve these two memory leak problem? thanks!

Hello all,

I still met below two memory leaks problem when using remote udp connection:

==11004== 20 bytes in 2 blocks are definitely lost in loss record 18 of 596
==11004== at 0x4A05F58: malloc (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==11004== by 0x305927C6B1: strdup (strdup.c:43)
==11004== by 0x305E6529F2: _cfg_lexer_lex (cfg-lex.l:199)
==11004== by 0x305E6284C1: cfg_lexer_lex (cfg-lexer.c:759)
==11004== by 0x305E65BF76: rewrite_expr_parse (rewrite-expr-grammar.c:2959)
==11004== by 0x305E654732: T.99 (cfg-parser.h:83)
==11004== by 0x305E655294: main_parse (cfg-grammar.y:584)
==11004== by 0x305E625EE2: cfg_run_parser (cfg-parser.h:83)
==11004== by 0x305E625FF5: cfg_read_config (cfg.c:384)
==11004== by 0x305E642A6E: main_loop_init (mainloop.c:680)
==11004== by 0x401774: main (main.c:246)

_cfg_lexer_lex (cfg-lex.l:199) is: yylval->cptr = 
strdup(yyextra->string_buffer->str);

==11004== 40,960 bytes in 40 blocks are definitely lost in loss record 
596 of 596
==11004== at 0x4A05F58: malloc (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==11004== by 0x305DA478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0)
==11004== by 0x305E641778: log_writer_flush (logwriter.c:950)
==11004== by 0x305E64185C: log_writer_work_perform (logwriter.c:130)
==11004== by 0x305E6418C7: log_writer_io_flush_output (logwriter.c:210)
==11004== by 0x305E6605E6: iv_fd_poll_and_run (iv_fd.c:167)
(Continue reading)

Xufeng Zhang | 14 Apr 08:24 2014

Fix several memory leaks caused by configuration reloading

Hello all,

Patch 1 is try to resolve below valgrind memory leak:
==25354== 26,112 bytes in 32 blocks are definitely lost in loss record 619 of 619
==25354== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25354== by 0x3A05A478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0)
==25354== by 0x3A036416D2: log_writer_flush (logwriter.c:1020)
==25354== by 0x3A03641812: log_writer_deinit (logwriter.c:1138)
==25354== by 0x523AF4A: affile_dw_deinit (logpipe.h:268)
==25354== by 0x523B7EB: affile_dd_deinit (logpipe.h:268)
==25354== by 0x3A0362961E: cfg_tree_stop (logpipe.h:268)
==25354== by 0x3A036420DF: main_loop_reload_config_apply (mainloop.c:498)
==25354== by 0x3A036613FA: iv_signal_event (iv_signal.c:170)
==25354== by 0x3A0365FE48: iv_event_raw_got_event (iv_event_raw_posix.c:89)
==25354== by 0x3A03660511: iv_fd_poll_and_run (iv_fd.c:163)
==25354== by 0x3A03660C93: iv_main (iv_main_posix.c:117)

Patch 3 is try to resolve below valgrind memory leak:
==25354== 1,107 (176 direct, 931 indirect) bytes in 1 blocks are definitely lost in loss record 594 of 619
==25354== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25354== by 0x3A05A478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0)
==25354== by 0x3A0362FFEF: g_process_set_argv_space (gprocess.c:502)
==25354== by 0x40164C: main (main.c:196)

The memory leak problem which resolved by patch 2 is very clear.

The only concern from me is patch 1, would it cause any side effect?

Thanks!
______________________________________________________________________________
(Continue reading)

David Hauck | 11 Apr 19:18 2014

Correct Usage of Multiple 'pattern' Databases

Hello,

I've only recently dug into some more intricate 'syslog-ng' configurations and had a question regarding
'log' construct blocks where multiple 'parser' references exist. I've been trying to do something like
the following (testing with the supplied example pattern databases):

log {
   filter(f_auth);
   parser("login");
   parser("sshd");
   parser("su");
   parser("sudo");
   log {
      filter(f_class_system);
      ...
   };
};

The problem I'm having is that extracted values from matched rules appear to be lost when the matched rule
exists in a pattern db *other than the last referenced parser() db*. Specifically, if a rule is matched in
the 'sshd' db above the following 'f_class_system' filter (which attempts to match
'.classifier.class') *does not* match; however, if a rule is matched in the 'sudo' db above the
'f_class_system' filter *does* match.

I'm sure this is perfectly explainable, but I can't find any documentation/Google references
specifically outlining this behaviour. Given the above and in order to work around this I assume I would
have to, either: 1) combine all of the rules into a single db file, or 2) break out each 'parser' reference
into a separate embedded 'log' construct (not ideal since the filtering et mechanics in each would be
identical and for maintenance reasons I'd like to consolidate these into a single 'log' construct). Both
options are less than ideal. Is there a better way?
(Continue reading)

Bendler, Ehren | 10 Apr 15:27 2014

Syslog-ng build issue (Autoconf?)

Our build environment:
Autoconf 2.68
Automake 1.11.3
m4 1.4.16
libtool 2.4.2
gcc 4.6.3 (cross compiling to PPC)

We get this error when building syslog-ng 3.5.4.1:
./configure: line 12794: syntax error near unexpected token `-Wno-pointer-sign,'
./configure: line 12794: `AX_CFLAGS_GCC_OPTION(-Wno-pointer-sign, CFLAGS_NOWARN_POINTER_SIGN)'

When I comment out that line in configure.ac (since we don't use that flag), I get an error from libmongo
complaining about not having automake 1.14+ and that it only tries to build it because I changed the AC
file. I tried to get around that by disabling mongodb support in the configure flags, but it still tries to
build that library.

I'd prefer to just get rid of the initial error. At the moment I am thinking it is related to our GNU Autotools
versions, but I need confirmation of that before I can go ask the powers that be for updates. Or if that isn't
the problem, alternate suggestions are appreciated.

-Ehren Bendler
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Yakup Kaya | 10 Apr 14:31 2014

syslog-ng uses 100% cpu

Hello everyone,

My problem is syslog-ng is using 100% CPU. When I trace the process for about 1 minute I get output as follows. As you can see epoll_wait and clock_gettime system calls are causing the problem. My syslog-ng version is 3.3.4. In the older versions I did not have such problems (e.g. 2.0.9). I would like to ask what exactly these system calls are used for and is there a configuration option or parameter to disable these calls or tune them?

top command output;

top - 15:28:46 up 4 days, 16:44,  7 users,  load average: 1.33, 1.35, 1.33
Tasks: 245 total,   3 running, 240 sleeping,   2 stopped,   0 zombie
Cpu(s):  9.2%us, 16.5%sy,  0.0%ni, 72.4%id,  1.9%wa,  0.1%hi,  0.0%si,  0.0%st
Mem:   4029172k total,  3958124k used,    71048k free,   322556k buffers
Swap:  4192928k total,   479076k used,  3713852k free,  1613548k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                   &nb sp;                               & nbsp;&nbs p;   
10192 root      20   0  8304 3908 2452 R 99.4  0.1 850:33.93 syslog-ng


strace command output:

strace -f -c -p `pgrep -f '/sbin/syslog-ng'`

Process 10192 attached - interrupt to quit
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 54.37    0.073938           0   5740561           epoll_wait
 45.54    0.061930           0   5740562           clock_gettime
  0.03    0.000035           0      1884           fcntl64
  0.01    0.000020           0      1488       370 read
  0.01    0.000018           0       408           writev
  0.01    0.000015           0      2166           epoll_ctl
  0.01    0.000008           0       172        78 accept
  0.01    0.000007           0       797           write
  0.01    0.000007           0       848       182 setsockopt
  0.00    0.000006           0       489           gettimeofday
  0.00    0.000006           0       408           _llseek
  0.00    0.000000           0        92           close
  0.00    0.000000           0       190           alarm
  0.00    0.000000           0        15           stat64
------ ----------- ----------- --------- --------- ----------------
100.00    0.135990              11490080       630 total

 
-- -- Yakup KAYA (B.Sc., CCNA) Kıdemli Sistem Destek Uzmanı/Senior System Support Specialist Labris Teknoloji A.Ş. Silikon Blok 1 NK 24 ODTÜ-Teknokent / Ankara, TURKEY Tel: +90 312 210 11 13 Fax: +90 312 210 14 92 yakup.kaya <at> labrisnetworks.com yakup.kaya <at> labris.eu www.labrisnetworks.com
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq


Gmane