Mason, Tron | 28 Aug 18:43 2015

Can't get syslog-ng to start...

All, I keep getting the following when attempting to start syslog-ng:


/usr/local/etc/syslog-ng/syslog-ng.conf: line 10: <at> version:: command not found

/usr/local/etc/syslog-ng/syslog-ng.conf: line 12: syntax error near unexpected token `('

/usr/local/etc/syslog-ng/syslog-ng.conf: line 12: `options { use_dns(yes);'


Not sure why I’m getting these messages when I’m running a later version:


bash-3.2# /usr/local/sbin/syslog-ng -V

syslog-ng 3.2.4

Installer-Version: 3.2.4

Revision: ssh+git://bazsi <at> git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.2#master#ef7b91e4a1b1f9628c66138b4ae83de7e4c697c6

Compile-Date: Jun  9 2011 05:24:53

Enable-Threads: off

Enable-Debug: off

Enable-GProf: off

Enable-Memtrace: off

Enable-Sun-STREAMS: on

Enable-IPv6: on

Enable-Spoof-Source: on

Enable-TCP-Wrapper: off

Enable-SSL: on

Enable-SQL: off

Enable-Linux-Caps: off

Enable-Pcre: on

Enable-Pacct: off


bash-3.2# more /usr/local/etc/syslog-ng/syslog-ng.conf

# Syslog-ng example configuration file for Solaris


# Copyright (c) 1999 Balazs Scheidler

# $Id: syslog-ng.conf.solaris,v 1.2 1999/11/15 12:30:41 bazsi Exp $

# Current syslog-ng binary version 1.5.16


# Solaris 2.5.1 and below uses the STREAMS driver, above extends it

# with doors. For 2.5.1 remove the door() option from the source declaration.


<at> version: 3.2.4


options { use_dns(yes);

#       use_time_recvd(yes);


        log_fifo_size(1000); };


#source local { sun-streams("/dev/log" door("/var/run/syslog_door")); };

source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); };

#source s_tcp { tcp(port(5014) max-connections(100); };

#source s_tcp { tcp(ip( port(5014)); };

source s_udp { udp(); };

# source s_udp { udp(ip( port(514)); };






Member info:

Budai, László | 26 Aug 13:03 2015

Some development notes on 3.7


after the 3.7 is out we would had to create a new 3.7/master branch, but I would postpone this step at a later point. 

Why? Because I know that there are some fixes in the queue what we would have to forward port to 3.8 and some of them to 3.6. 

This branching will be a lazy one: we will branch it when we have to, but not earlier.

Member info:

Peter Volkov | 24 Aug 18:43 2015

How to disable DNS resolution in syslog-ng?

Hi guys. Whenever syslog-ng start without networking it hangs trying to resolve hostname it runs on (checked with tcpdump). Now I've tried to add use_dns(no); into configuration file but delay still in place:

options {

I've tried to google, but still failed to find answer: Is there any way to disable this hostname resolution delay?

Thank in advance,
Member info:

Saurabh Shukla | 22 Aug 02:55 2015

Forwarding system startup messages

I am running syslog-ng 3.6.4 and I have the following destination and log path configured that forwards all messages to the destination:

destination remote {
    network("" port(514) transport(tcp) log_fifo_size(2048));
log { source(s_all); destination(remote); flags(flow-control);};

When the system reboots, I see that startup messages from the kernel are logged into /var/log/syslog.
syslog-ng establishes a connection to the remote destination around 10 sec after the first message was logged into /var/log/syslog. However, it fails to forward any message that was logged into /var/log/syslog during the first 10 seconds even though I have the output buffer and flow control configured.

Is this a bug in syslog-ng or am I missing some configuration steps?

-- Saurabh
Member info:

Giovanni Mancuso | 19 Aug 17:49 2015

Advice on the right destination

I am writing to ask your advice on a solution I'm thinking.

I have 12 servers with postfix, amavisd-new and other custom software that manage the e-mail system and I was working in a web interface to analysis the logs and correlation with the ability to search for certain fields (from, to, message-id , date).

All applications send the logs to a centralized syslog-ng, and I was trying to understand which type of "destination"  is better to use to ensure the rapid search. I was analyzing the possibility of using elasticsearch, but I don't know neither it or its performances.

The quantity of data is very high, about 3TB of data monthly or each machine, with 2 years of retention.

What do you think about? Have you any suggestions?

Giovanni Mancuso
System Architect
T 06.9826.9600 M +39.340.65.80.739 F 06.9826.9680
P.zza S.Benedetto da Norcia, 33 - 00071 Pomezia (RM)
CONFIDENZIALE: Questo messaggio ed i suoi allegati sono di carattere confidenziale per i destinatari in indirizzo.
È vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale.
Se ricevuto per errore, l'uso del contenuto è proibito; si prega di comunicarlo al mittente e cancellarlo immediatamente.

Attachment (giovanni_mancuso.vcf): text/x-vcard, 319 bytes
Member info:

Jacek Drewniak | 18 Aug 08:57 2015

Multiple failures while inserting this record to the java destination, message dropped

Hello, from yesterday passing logs to elasticsearch is strange. It works some times, for example by 1 hour, and then nothings happens. I had changed parsing logs to syslog-protocol, but I tried with the default file parsing too.  No errors, warnings, nothing. I left my station for 24h  and I can see that log appears in the night, but this time ends with this errors:

MESSAGE:Failed to send message: failed to parse; 
ISODATE:August 17th 2015, 23:34:07.000 

MESSAGE:Multiple failures while inserting this record to the java destination, message dropped; number_of_retries='3'
ISODATE:August 17th 2015, 23:34:07.000

Jacek Drewniak

emailjacek.drewniak <at>

mobile: +48 696 151 670



Bluetooth Breakthrough Award Finalist
CES 2015 Envisioneering Innovation & Design Award Winner
Tech Trailblazers Awards Winner
Most exciting company at Bluetooth Media Event in New York 2014
Polish Agency for Enterprise Development Award Winner

Member info:

Budai, László | 17 Aug 16:46 2015

unofficial syslog-ng 3.7.1 Debian packages


I've just created my unofficial OBS repo[1] for syslog-ng 3.7.1 series (current version: 3.7.1-1 ).

List of supported OSs (i386/amd64):
* Debian 7.0
* Debian 8.0
* Ubuntu 12.04
* Ubuntu 14.04
* Ubuntu 14.10
* Ubuntu 15.04


example: Debian 8.0

1. get release key

2. add repo to APT sources
eg.: /etc/apt/sources.list.d/syslog-ng-obs.list

Then `apt-get update` and `apt-get install syslog-ng-core=3.7.1-1`

You can replace Debian_8.0 to
 * Debian_7.0
 * xUbuntu_12.04, 
 * xUbuntu_14.04
 * xUbuntu_14.10
 * xUbuntu_15.04

(`x` before Ubuntu is not a typo ;-) )

Available packages:
 * syslog-ng-core
 * syslog-ng-dbg
 * syslog-ng-dev
 * syslog-ng-mod-stomp
 * syslog-ng-mod-amqp
 * syslog-ng-mod-elastic
 * syslog-ng-mod-geoip
 * syslog-ng-mod-graphite
 * syslog-ng-mod-hdfs
 * syslog-ng-mod-http
 * syslog-ng-mod-java
 * syslog-ng-mod-java-common-lib
 * syslog-ng-mod-journal
 * syslog-ng-mod-json
 * syslog-ng-mod-kafka
 * syslog-ng-mod-mongodb
 * syslog-ng-mod-python
 * syslog-ng-mod-redis
 * syslog-ng-mod-riemann
 * syslog-ng-mod-smtp
 * syslog-ng-mod-sql
 * syslog-ng-mod-stomp

Notes to the java destinations

About the 

Error opening plugin module; module='mod-java', error=' cannot open shared object file: No such file or directory'

problem. is linked to the library.
If your java installation doesn't create a symlink in /usr/lib directory to then 
you have to modify your or expand your LD_LIBRARY_PATH with the containing library
before starting syslog-ng. 



Laszlo Budai

Member info:

Budai, László | 17 Aug 16:45 2015

syslog-ng 3.7.1 released


New dependencies

OpenSSL is now a required dependency for syslog-ng because the newly added 
hostid and uniqid features requires a CPRNG provided by OpenSSL.
Therefore non-embedded crypto lib is not a real option, so the support of 
having such a crypto lib discontinued and all SSL-dependent features enabled
by default.

Library updates

  • Minimal libriemann-client version bumped from 1.0.0 to 1.6.0.
  • Added support for the monolithic libsystemd library (systemd 209).
  • RabbitMQ submodule upgraded.


Language bindings

  • Java-destination driver ported from syslog-ng-incubator.
    Purpose of having Java destination driver is to make it possible
    to implement destination drivers in the Java language (and using
    'official' Java client libraries).

  • Python language support is ported from syslog-ng incubator and 
    has been completely reworked. Now, it is possible to implement template
    functions in Python language and also destination drivers.
    Main purpose of supporting Python language is to implement a nice
    interactive syslog-ng config debugger for syslog-ng.

New drivers

New Java destination drivers

ElastiSearch, Kafka and HDFS destination drivers are implemented by using
the 'official' Java client libraries and syslog-ng provides a way to set
their own, native configuration file. Log messages generated by the client 
Java libraries are redirected to syslog-ng via our own Log4JAppender which
means that those logs are available as internal syslog-ng messages.

  • ElasticSearch
  • Kafka
  • Hadoop/HDFS
  • HTTP


  • Added a geoip() parser, that can look up the country code and
    latitude/longitude information from an IPv4 address. For lat/long to
    work, one will need the City database.

  • New parser, extract-solaris-msgid() added for automatically extracts
    (parses & removes) the msgid portion of Solaris messages.

  • Extended the set of supported characters to every printable ASCII's except
    ., [ and ] in extract-prefix for json-parser().

  • Added string-delimiters option to csvparser to support multi character
    delimiters in CSV parsing.

  • A kv-parser() introduced for WELF (WebTrens Enhanced Log Format) that 
    implements key=value parsing. The kv-parser() tries to extract 
    key=value formatted name-value pairs from the input string.

  • value-pairs: make it possible to pass --key as a positional argument
    From now it is possible to use value-pairs expressions like this:

    $(format-json MSG DATE)

    instead of

    $(format-json --key MSG --key DATE)


  • Added IPv6 netmask filter for selecting only messages sent by a host whose IP address belongs to the specified IPv6 subnet.


  • Added a new macro, called HOSTID which is a 32-bit number generated by
    a cryptographically secure PRNG. Its purpose is to identify the
    syslog-ng host, thus it is the same for every message generated on the same

  • Added a new macro, called UNIQID which is a practically unique ID generated
    from the HOSTID and the RCPTID in the format of HOSTID <at> RCPTID. 
    Uniqid is a derived value: it is built up from the always available hostid
    and the optional rcptid. In other words: uniqid is an extension over rcptid.
    For that reason use-rcptid has been deprecated and use-uniqid could be
    use instead.


  • welf was renamed to kvformat
    As this reflects the purpose of this module much better, WELF is just
    one of the format it has support for.

  • $(format-cim) template function added into an SCL module.

  • It is possible to create templates without braces.

SMTP destination

  • The afsmtp driver now supports templatable recipients field. Just like the subject() and body() fields, now the address containing parameters of to(), from(), cc() and bcc() can contain macros.

Unix Domain Sockets

  • Added pass-unix-credentials() global option for enabling/disabling unix 
    credentials passing on those platforms which has this feature. By default
    it is enabled.

  • Added create-dirs() option to unix-*() sources for creating the
    containing directories for Unix domain sockets.

Riemann destination

  • Added batched event sending support for riemann destination driver which
    makes the riemann destination respect flush-lines(), and send event
    in batches of configurable amount (defaults to 1). In case of an error,
    all messages within the batch will be dropped. Dropped messages, and
    messages that result in formatting errors do not count towards the batch
    size. There is no timeout, but messages will be flushed upon deinit.

  • A timeout() option added to the Riemann destination.


  • Earlier, in patterndb, the first applicable rule won, even if it was only a partial match. This means that when rules overlapped, the shorter match would have been found, if it was the first to be loaded. A strong preference introduced for rules that match the input string completely. The load order is still applicable though, it is possible to create two distinct rules that would match the same input, in those cases the first one to be loaded wins.

Miscellaneous features

  • New builtin interactive syslog-ng.conf debugger implemented for syslog-ng.
    The debugger has a Python frontend which contains a full Completer
    (just press TABs and works like bash)

  • Added a reset option to syslog-ng-ctl stats. With this option the non-stored
    stats counters can be zeroed.

  • New parameter added to loggen: --permanent (-T) wich is for sending logs 

  • Loggen uses the proper timezone offset in generated message.

  • The ssl_options inside tls() extended with the following set:
    no-sslv2, no-sslv3, no-tlsv1, no-tlsv11, no-tlsv12.

  • Added syslog-debug bundle generator script to make it easier to reproduce bugs
    by collecting debug related information, like:

    • process information gathering
    • syscall tracing (strace/truss)
    • configuration gathering
    • selinux related information gathering
    • solaris information gathering (sysdef, kstat, showrev, release)
    • get information about syslog-ng svr4 solaris packages, if possible


  • New utf8 string sanitizers instead of old broken one.

  • syslog-ng won't send SIGTERM when getpgid() fails in program destination

  • In some cases program destination respawned during syslog-ng stop/restart

  • syslog-ng generates mark messages when mark-mode is set
    to host-idle.

  • Using msg_control only when credential passing is supported in socket 
    destination (afsocket).

  • Writer is replaced only when protocol changed during reload in socket
    destination (afsocket).

  • Fix spinning on EOF for unix-stream() sockets. Root cause of the spinning
    was that a unix-dgram socket was created even in case of unix-stream.

  • When the configured host was not available during the initialization of
    afsocket destination syslog-ng just didn't start. From now, syslog-ng
    starts in that case and will retry connecting to the host periodically.

  • Fixed BSD year inference in syslogformat. When the difference between the
    current month and the month part of the timestamp of an incoming logmessage
    in BSD format (which has no year part) was greater than 1 then syslog-ng
    computed the year badly.

  • In some cases, localtime related macros had a wrong value(eg.:$YEAR).

  • TLS support added to Riemann destination

  • Excluded "tags" from Riemann destination driver as an attribute which 
    conflicts with reserved keyword

  • When a not writeable/non-existent file becomes writeable/exists later,
    syslog-ng recognize it (with the help of reopen-timer) and delivers messages
    to the file without dropping those which were received while the file was
    not available (affile).

  • Fixed a crash around affile at the first message delivery when templates
    were used (affile).

  • Fixed a configure error around libsystemd-journal.

  • Removed syslog.socket from service file on systems using systemd.
    Syslog-ng reads the messages directly from journal on systems with systemd.

  • Fixed compilation where the monolitic libsystemd was not available.

  • Fixed compilation failure on OpenBSD.

  • AMQP connection process fixed.

  • Added DOS/Windows line ending support in config.

  • Retries fixed in SQL destination. In some circumstances when
    retry_sql_inserts was set to 1, after an insertion failure all incoming
    messages were dropped.

  • Transaction handling fixed in SQL destination. In some circumstances when
    both select and insert commands were run within a single transaction and
    the select failed (eg.: in case of mssql), the log messages related to
    the insert commands, broken by the invalid transaction, were lost.

  • Fixed a memleak in SQL destination driver.
    The memleak occured during one of the transaction failures.

  • Memory leak around reload and internal queueing mechanism has been fixed.

  • Fixed a potential abort when the localhost name cannot be detected.

  • Security issue fixed around $HOST.
    Tech details:
    When the name of the host is too long, the buffer we use to format the
    chained hostname is truncated. However snprintf() returns the length the
    result would be if no truncation happened, thus we will read uninitialized
    bytes off the stack when we use that pointer to set $HOST
    with log_msg_set_value().

    There can be some security implications, like reading values from the stack
    that can help to craft further exploits, especially in the presense of
    address space randomization. It can also cause a DoS if the hostname length
    is soo large that we would read over the top-of-the-stack, which is probably
    not mmapped causing a SIGSEGV.

  • Journal entries containing name-value pairs without '=' caused syslog-ng
    to crash. Instead of crashing, syslog-ng just drop these nv pairs.

  • Fixed the encoding of characters below 32 if escaping is enabled in 
    templates. Templated outputs never contained references to characters below
    32, essentially they were dropped from the output for two reasons:

    • the prefixing backslash was removed from the code
    • the format_uint32_padded() function produced no outputs in base 8
  • Fixed afstomp destination port issue. It always tried to connect to the port 0.

  • Fixed memleak in db-parser which could happen at every reload.

  • Fixed a class of rule conflicts in db-parser:
    Because an error in the pdb load algorithms, some rules would conflict which
    shouldn't have done that. The problem was that several programs would use 
    the same RADIX tree to store their patterns. Merging independent programs 
    meant that if they the same pattern listed, it would clash, even though

    their $PROGRAM is different.

    There were multiple issues:

    • we looked up pattern string directly, even they might have contained
      <at> parser <at> references. It was simply not designed that way and only

      worked as long as we didn't have the possibility to use parsers

      in program names

    • we could merge programs with the same prefix, e.g.
      su, supervise/syslog-ng and supervise/logindexd would clash, on "su",
      which is a common prefix for all three.

    The solution involved in using a separate hash table for loading, which
    at the end is turned into the radix tree.

  • pdbtool match when used with the --debug-pattern option used a low-level
    lookup function, that didn't perform all the db-parser actions specified
    in the rule

  • Max packet length for spoof source is set to 1024 (previously : 256).

  • A certificate which is not contained by the list of fingerprints is
    rejected from now.

  • Hostname check in tls certificate is case insensitive from now.

  • There is a use-case where user wants to ignore an assignment to a name-value
    pair. (eg.: when using csv-parser(), sometimes we get a column we really
    want to drop instead of adding it to the message). In previous versions an
    error message was printed out:
    'Name-value pairs cannot have a zero-length name'.
    That error message has been removed.

  • Fixed a docbook related compilation error: there was a hardcoded path that
    caused build to fail if docbook is not on that path. Debian based
    platforms did not affected by this problem.
    Now a new option was created for ./configure that is --enable-manpages
    that enables the generation of manpages using docbook from online source.
    '--with-docbook=PATH' gives you the opportunity to specify the path for
    your own installed docbook.


syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Adam Arsenault, Adam Istvan Mozes, Alex Badics, Andras Mitzki, 
Balazs Scheidler, Bence Tamas Gedai, Ben Kibbey, Botond Borsits, Fabien Wernli,
Gergely Nagy, Gergo Nagy, Gyorgy Pasztor, Kristof Havasi, Laszlo Budai,
Manikandan-Selvaganesh, Michael Sterrett, Peter Czanik, Robert Fekete, 
Sean Hussey, Tibor Benke, Toralf Förster, Viktor Juhasz, Viktor Tusa, 
Vincent Bernat, Zdenek Styblik, Zoltan Fried, Zoltan Pallagi.

View it on GitHub.

Member info:

Thanh Dat | 17 Aug 12:51 2015

Patterndb to get timestamp from log record in file

Dear all syslog-ng user,

I'm new with syslog-ng. Now I'm using patterndb to parse the log which I get from a log file.
For example, I have this pattern:

<pattern> <at> ESTRING:postfix.qid:: <at> message-id= <at> QSTRING:postfix.msgid:&lt;&gt; <at> </pattern>

and it work perfectly with this log:

Jul 27 00:04:41 mail1 postfix/cleanup[19856]: 246F21FE3C: message-id=<20150726170441.246F21FE3C <at> mail1>

Now I want to get the timestamp of this log record (Jul 27 00:04:41). How can I do this ?

Thank you so much for all of your help.

Dat Tang Thanh Signature <!-- /* Font Definitions */ <at> font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:1627400839 -2147483648 8 0 66047 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:""; margin:0mm; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman","serif"; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast;} a:link, span.MsoHyperlink {mso-style-noshow:yes; mso-style-priority:99; color:blue; text-decoration:underline; text-underline:single;} a:visited, span.MsoHyperlinkFollowed {mso-style-noshow:yes; mso-style-priority:99; color:purple; text-decoration:underline; text-underline:single;} p.MsoAcetate, li.MsoAcetate, div.MsoAcetate {mso-style-noshow:yes; mso-style-priority:99; mso-style-link:"Balloon Text Char"; margin:0mm; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:8.0pt; font-family:"Tahoma","sans-serif"; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast;} span.BalloonTextChar {mso-style-name:"Balloon Text Char"; mso-style-noshow:yes; mso-style-priority:99; mso-style-unhide:no; mso-style-locked:yes; mso-style-link:"Balloon Text"; mso-ansi-font-size:8.0pt; mso-bidi-font-size:8.0pt; font-family:"Tahoma","sans-serif"; mso-ascii-font-family:Tahoma; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Tahoma; mso-bidi-font-family:Tahoma;} span.SpellE {mso-style-name:""; mso-spl-e:yes;} .MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; font-size:10.0pt; mso-ansi-font-size:10.0pt; mso-bidi-font-size:10.0pt;} <at> page WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt; mso-header-margin:36.0pt; mso-footer-margin:36.0pt; mso-paper-source:0;} div.WordSection1 {page:WordSection1;} -->

Best Regards.

 Tang Thanh Dat (Mr.) | System Administration Department
18 Hoang Quoc Viet, Cau Giay, Hanoi,Vietnam
(T)+84-4-37562227, (F)+84-4-37 561 888, (M)+84-(0)-9 32336692
(E) <at> (W)
NetNam - one of the best ISPs and Solutions Providers in Vietnam,
specialized in corporate networks, managed services & security solutions.
Your Net, We Care! 

Member info:

Jacek Drewniak | 14 Aug 14:40 2015

Syslog-ng message formating


I am new in logging world.

I am using syslog protocol.

For example I am logging this:
But it is parsed to fields (I can see this on kibana) : 

Can You tell me what I am doing wrong? 

Jacek Drewniak

emailjacek.drewniak <at>

mobile: +48 696 151 670



Bluetooth Breakthrough Award Finalist
CES 2015 Envisioneering Innovation & Design Award Winner
Tech Trailblazers Awards Winner
Most exciting company at Bluetooth Media Event in New York 2014
Polish Agency for Enterprise Development Award Winner

Member info:

Giovanni Mancuso | 13 Aug 11:02 2015

syslog-ng split long rows

i try to do a simple configuration of syslog-ng:
Konsole output

source mailet_log {

       file("/opt/myApp/mylog.log" log_msg_size(81920));

destination mailet_test {

log { source(mailet_log); destination(mailet_test); };

It works, but if i have in a source file a row that has Konsole output70410 70410 char (for example),  in the destionation file i have two different row (first: 65024 char, second: 5433 char).

Is it normal? can i configure syslog-ng to not split the lines?


Attachment (giovanni_mancuso.vcf): text/x-vcard, 319 bytes
Member info: