Peter Czanik | 1 Aug 10:23 2014
Picon

syslog-ng with mongodb support in EPEL7

Hi,
syslog-ng with mongodb support was successfully built for EPEL7. It 
should be on the mirrors by the weekend. If you are really inpatient, 
you can download libmongo-client and the new build directly from the 
build system:
libmongo-client: http://koji.fedoraproject.org/koji/buildinfo?buildID=549386
syslog-ng: http://koji.fedoraproject.org/koji/buildinfo?buildID=549647
Have a nice weekend!
Bye

--

-- 
Peter Czanik (CzP) <peter.czanik <at> balabit.com>
BalaBit IT Security / syslog-ng upstream
http://czanik.blogs.balabit.com/
https://twitter.com/PCzanik

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Smith, Jerry Don II | 30 Jul 20:34 2014
Picon

Can you have multiple non-standard TCP ports as sources

Good Afternoon,


I am trying to setup a syslog-ng aggregation server and I can currently get it to listen on standard 601/514 ports.


source remote {
        udp(ip(0.0.0.0));
        tcp(ip(0.0.0.0) port(601) max-connections(200));
};


and

source remote_2 {
        tcp(ip(0.0.0.0) port(514) max-connections(200));
};


[root <at> ms30snllx opt]# netstat -pan |grep syslog
tcp        0      0 0.0.0.0:601                 0.0.0.0:*                   LISTEN      22113/syslog-ng     
tcp        0      0 0.0.0.0:514                 0.0.0.0:*                   LISTEN      22113/syslog-ng  

udp     3328      0 0.0.0.0:514                 0.0.0.0:*                               22113/syslog-ng 



But when I add a new source and have it listen on 1514 for example:


source remote_3 {
        tcp(ip(0.0.0.0) port(1514) max-connections(200));
};


I don't get a listener on 1514 as well when looking via netstat .


Is there something I am missing?


Thanks in advance,


______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

DeeKaeOne | 30 Jul 18:45 2014
Picon

Problem with mongodb

Hi,

I have a fresh CentOS 7 minimal installation and enabled the EPEL 7 repo. Then installed syslog-ng and
syslog-dbi in the following versions:

Installed Packages
Name        : syslog-ng
Arch        : x86_64
Version     : 3.5.5
Release     : 1.el7
Size        : 1.6 M
Repo        : installed
From repo   : epel
Summary     : Next-generation syslog server
URL         : http://www.balabit.com/network-security/syslog-ng

Installed Packages
Name        : syslog-ng-libdbi
Arch        : x86_64
Version     : 3.5.5
Release     : 1.el7
Size        : 61 k
Repo        : installed
From repo   : epel
Summary     : libdbi support for syslog-ng
URL         : http://www.balabit.com/network-security/syslog-ng

All documentations say that this version should support mongodb, but after some debugging I found this
output while failing to start syslog-ng:

Adding include file; filename='/etc/syslog-ng/conf.d/mongodb.conf'
Starting to read include file; filename='/etc/syslog-ng/conf.d/mongodb.conf', depth='1'
Error parsing destination, destination plugin mongodb not found in
/etc/syslog-ng/conf.d/mongodb.conf at line 2, column 5:
                                                     included from /etc/syslog-ng/syslog-ng.conf line 68, column 1

    mongodb(
    ^^^^^^^

Where my /etc/syslog-ng/conf.d/mongodb.conf looks like this:

destination d_mongodb {
    mongodb();
};
log { source(s_sys); destination(d_mongodb); };

Am I missing something or is that rpm compiled without mongodb support?

Glad about any ideas!

Cheers,
Dennis

--

-- 

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Scot Needy | 28 Jul 19:19 2014
Picon

syslog-ng-3.5.5 bison: invalid argument `no-yacc' for `--warnings'


Sorry if this is a dupe. 

 Looked through my mail archives and didn’t see any hits. 

Trying to compile syslog-ng on CentOS 6.5 final and make fails on bison arguments. 

[root <at> syslog-ng-3.5.5]# make
  YACC     lib/rewrite/rewrite-expr-grammar.c
bison: invalid argument `no-yacc' for `--warnings'
Valid arguments are:
  - `none            - no warnings'
  - `midrule-values  - unset or unused midrule values'
  - `yacc            - incompatibilities with POSIX YACC'
  - `all             - all of the above'
  - `error           - warnings are errors'
  YACC     lib/rewrite/rewrite-expr-grammar.h
….
…..
….
bison: invalid argument `no-yacc' for `--warnings'
Valid arguments are:
  - `none            - no warnings'
  - `midrule-values  - unset or unused midrule values'
  - `yacc            - incompatibilities with POSIX YACC'
  - `all             - all of the above'
  - `error           - warnings are errors'
make --no-print-directory all-recursive
  CC       lib/lib_libsyslog_ng_la-afinter.lo
  CC       lib/lib_libsyslog_ng_la-apphook.lo
  YACC     lib/block-ref-grammar.h
bison: invalid argument `no-yacc' for `--warnings'
Valid arguments are:
  - `none            - no warnings'
  - `midrule-values  - unset or unused midrule values'
  - `yacc            - incompatibilities with POSIX YACC'
  - `all             - all of the above'
  - `error           - warnings are errors'
  CC       lib/lib_libsyslog_ng_la-block-ref-parser.lo
lib/block-ref-parser.c:26:31: error: block-ref-grammar.h: No such file or directory
lib/block-ref-parser.c:43: error: 'LL_CONTEXT_BLOCK_REF' undeclared here (not in a function)
make[2]: *** [lib/lib_libsyslog_ng_la-block-ref-parser.lo] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

============== ./configure output =================== 

configure: creating ./config.status
config.status: creating librabbitmq.pc
config.status: creating Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands

rabbitmq-c build options:
	Host: x86_64-unknown-linux-gnu
	Version: 0.2.0
	64-bit: no
	Tools: no
	Documentation: no

syslog-ng Open Source Edition 3.5.5 configured
 Compiler options:
  compiler                    : gcc -std=gnu99
  compiler options            : -g -O2 -Wall -pthread  -pthread -I/usr/include/glib-2.0
-I/usr/lib64/glib-2.0/include   -I/usr/local/include/eventlog       
-I$(top_srcdir)/lib/ivykis/src/include -I$(top_builddir)/lib/ivykis/src/include
-D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
  linker flags                :
  prefix                      : /usr/local
  linking mode                : dynamic
  embedded crypto             : no
  __thread keyword            : yes
 Submodules:
  ivykis                      : internal
  libmongo-client             : internal
  librabbitmq                 : internal
 Features:
  Debug symbols               : no
  GCC profiling               : no
  Memtrace                    : no
  IPV6 support                : yes
  spoof-source support        : no
  tcp-wrapper support         : no
  Linux capability support    : no
  PCRE support                : no
  Env wrapper support         : no
  systemd support             : no (unit dir: none)
 Modules:
  Module search path          : /usr/local/lib/syslog-ng
  Sun STREAMS support (module): no
  SSL support (module)        : yes
  SQL support (module)        : no
  PACCT module (EXPERIMENTAL) : no
  MongoDB destination (module): yes
  JSON support (module)       : yes
  SMTP support (module)       : yes
  AMQP destination (module)   : yes
  STOMP destination (module)  : yes
  GEOIP support (module)      : no
  Redis support (module)      : no
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

jrhendri | 25 Jul 22:09 2014

syslog-ng redis destination problem

Hi (again) -

  This one has me a bit puzzled. I am trying to write to redis using the example from the syslog-ng documentation:

destination d_local_redis {
 redis(command("HINCRBY" "${HOST}/programs" "${PROGRAM}" "1"));
};

but I am getting "-ERR value is not an integer or out of range"

I have tried this on a couple systems. This particular one is:

[n0142566 <at> VDDP13E-F1A47ED src]$ /usr/local/sbin/syslog-ng --version
syslog-ng 3.5.4.1
Installer-Version: 3.5.4.1
Revision: 
Compile-Date: Jul 22 2014 12:30:06
Available-Modules: afsocket-notls,afprog,csvparser,system-source,dbparser,redis,syslogformat,afuser,afstomp,afmongodb,linux-kmsg-format,json-plugin,basicfuncs,afamqp,affile,cryptofuncs,afsocket,confgen
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: off
Enable-TCP-Wrapper: off
Enable-Linux-Caps: off
Enable-Pcre: off
[n0142566 <at> VDDP13E-F1A47ED src]$ 

And I have captured the packets to see what is being sent and see this following the TCP stream with Wireshark
(pcap attached)

*1
$4
ping
+PONG
*4
$7
HINCRBY
$24
VDDP13E-F1A47ED/programs
$16
ArcSightLogs_PDC
$2
 *
-ERR value is not an integer or out of range

Note: when I use ("INCR" "testkey") it works - seems like the integer increment is not being sent correctly?

Is this something I am getting wrong or is it a bug ?

Thanks (again)

Jim

Attachment (redis6.pcap): application/octet-stream, 1535 bytes
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Jean Faye | 23 Jul 15:13 2014
Picon

Plugin module not found and Error parsing source, source plugin pipe not found

confirm a6d843f7ad7d1dbf1fefb6f12432e54941a680a9

Hi all,

When I run syslog-ng on my platform I faced the issue below. Could you please tell me why this issues happen and how can I fixed them?

I am using syslog-ng-3.2.5 and eventlog-0.2.13.

See in below the syslog-ng.conf I used.

Thank you in advance

Ismael

/**************Error Logs*******************/
Starting syslog-ng:Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='syslogformat'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='basicfuncs'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='afsocket'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='affile'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='afprog'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='afuser'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='dbparser'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='csvparser'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='syslogformat'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='basicfuncs'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='afsocket'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='affile'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='afprog'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='afuser'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='dbparser'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='csvparser'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='confgen'
Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng', module='confgen'
Error parsing source, source plugin pipe not found in /etc/syslog-ng.conf at line 26, column 2:

        pipe("/tmp/pipe" pad_size(2048));
        ^^^^

syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng



confirm a6d843f7ad7d1dbf1fefb6f12432e54941a680a9


syslog-ng.conf

<at> version: 3.2
<at> include "scl.conf"
#
# Syslog-ng configuration file, compatible with default Debian syslogd
# installation. Originally written by anonymous (I can't find his name)
# Revised, and rewrited by me (SZALAY Attila <sasa <at> debian.org>)

# First, set some global options.
options { long_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no);
      owner("root"); group("adm"); perm(0640); stats_freq(0);
      bad_hostname("^gconfd$");create_dirs(yes);
};

########################
# Sources
########################
# This is the default behavior of sysklogd package
# Logs may come from unix stream, but not from another machine.
#
#source s_src { unix-dgram("/dev/log"); internal();
#                file("/proc/kmsg" program_override("kernel"));
#};

######FIJ######
source s_mysource {
    pipe("/tmp/pipe" pad_size(2048));
    #file("/var/log/ldb/fij_source.log" create_dirs(yes));
};
###############

# If you wish to get logs from remote machine you should uncomment
# this and comment the above source line.
#
#source s_net { tcp(ip(127.0.0.1) port(1000) authentication(required) encrypt(allow)); };

########################
# Destinations
########################
# First some standard logfile
#
#destination d_auth { file("/var/log/auth.log"); };
#destination d_cron { file("/var/log/cron.log"); };
#destination d_daemon { file("/var/log/daemon.log"); };
#destination d_kern { file("/var/log/kern.log"); };
#destination d_lpr { file("/var/log/lpr.log"); };
#destination d_mail { file("/var/log/mail.log"); };
#destination d_syslog { file("/var/log/syslog"); };
#destination d_user { file("/var/log/user.log"); };
#destination d_uucp { file("/var/log/uucp.log"); };

# This files are the log come from the mail subsystem.
#
#destination d_mailinfo { file("/var/log/mail/mail.info"); };
#destination d_mailwarn { file("/var/log/mail/mail.warn"); };
#destination d_mailerr { file("/var/log/mail/mail.err"); };

# Logging for INN news system
#
#destination d_newscrit { file("/var/log/news/news.crit"); };
#destination d_newserr { file("/var/log/news/news.err"); };
#destination d_newsnotice { file("/var/log/news/news.notice"); };

# Some `catch-all' logfiles.
#
#destination d_debug { file("/var/log/debug"); };
#destination d_error { file("/var/log/error"); };
#destination d_messages { file("/var/log/messages"); };

# The root's console.
#
#destination d_console { usertty("root"); };

# Virtual console.
#
#destination d_console_all { file("/dev/tty10"); };

# The named pipe /dev/xconsole is for the nsole' utility.  To use it,
# you must invoke nsole' with the -file' option:
#
#    $ xconsole -file /dev/xconsole [...]
#
#destination d_xconsole { pipe("/dev/xconsole"); };

# Send the messages to an other host
#
#destination d_net { tcp("127.0.0.1" port(1000) authentication(on) encrypt(on) log_fifo_size(1000)); };

# Debian only
#destination d_ppp { file("/var/log/ppp.log"); };

######FIJ######
########################Vérifier l'option create_dirs(yes)######################
destination d_GEN {
                #file("/var/log/ldb/GENTrace.log" create_dirs(yes));
                file("/var/log/ldb/GENTrace.log");
};

destination d_SU {
                #file("/var/log/ldb/SUTrace.log" create_dirs(yes));
                file("/var/log/ldb/SUTrace.log");
};

destination d_WAN {
                #file("/var/log/ldb/WANTrace.log" create_dirs(yes));
                file("/var/log/ldb/WANTrace.log");
};

destination d_CPL {
                #file("/var/log/ldb/CPLTrace.log" create_dirs(yes));
                file("/var/log/ldb/CPLTrace.log");
};
###############

########################
# Filters
########################
# Here's come the filter options. With this rules, we can set which
# message go where.

#filter f_dbg { level(debug); };
#filter f_info { level(info); };
#filter f_notice { level(notice); };
#filter f_warn { level(warn); };
#filter f_err { level(err); };
#filter f_crit { level(crit .. emerg); };

#filter f_debug { level(debug) and not facility(auth, authpriv, news, mail); };
#filter f_error { level(err .. emerg) ; };
#filter f_messages { level(info,notice,warn) and
#                    not facility(auth,authpriv,cron,daemon,mail,news); };

#filter f_auth { facility(auth, authpriv) and not filter(f_debug); };
#filter f_cron { facility(cron) and not filter(f_debug); };
#filter f_daemon { facility(daemon) and not filter(f_debug); };
#filter f_kern { facility(kern) and not filter(f_debug); };
#filter f_lpr { facility(lpr) and not filter(f_debug); };
#filter f_local { facility(local0, local1, local3, local4, local5,
#                        local6, local7) and not filter(f_debug); };
#filter f_mail { facility(mail) and not filter(f_debug); };
#filter f_news { facility(news) and not filter(f_debug); };
#filter f_syslog3 { not facility(auth, authpriv, mail) and not filter(f_debug); };
#filter f_user { facility(user) and not filter(f_debug); };
#filter f_uucp { facility(uucp) and not filter(f_debug); };

#filter f_cnews { level(notice, err, crit) and facility(news); };
#filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); };

#filter f_ppp { facility(local2) and not filter(f_debug); };
#filter f_console { level(warn .. emerg); };

######FIJ######
filter f_GEN {
                facility(local0) and filter(nom_du_composant_applicatif);
};

filter f_SU {
                facility(local0) and filter(nom_du_composant_applicatif);
};

filter f_WAN {
                facility(local0) and filter(nom_du_composant_applicatif);
};

filter f_CPL {
                facility(local0) and filter(nom_du_composant_applicatif);
};
###############

########################
# Log paths
########################
#log { source(s_src); filter(f_auth); destination(d_auth); };
#log { source(s_src); filter(f_cron); destination(d_cron); };
#log { source(s_src); filter(f_daemon); destination(d_daemon); };
#log { source(s_src); filter(f_kern); destination(d_kern); };
#log { source(s_src); filter(f_lpr); destination(d_lpr); };
#log { source(s_src); filter(f_syslog3); destination(d_syslog); };
#log { source(s_src); filter(f_user); destination(d_user); };
#log { source(s_src); filter(f_uucp); destination(d_uucp); };

#log { source(s_src); filter(f_mail); destination(d_mail); };
#log { source(s_src); filter(f_mail); filter(f_info); destination(d_mailinfo); };
#log { source(s_src); filter(f_mail); filter(f_warn); destination(d_mailwarn); };
#log { source(s_src); filter(f_mail); filter(f_err); destination(d_mailerr); };

#log { source(s_src); filter(f_news); filter(f_crit); destination(d_newscrit); };
#log { source(s_src); filter(f_news); filter(f_err); destination(d_newserr); };
#log { source(s_src); filter(f_news); filter(f_notice); destination(d_newsnotice); };
#log { source(s_src); filter(f_cnews); destination(d_console_all); };
#log { source(s_src); filter(f_cother); destination(d_console_all); };

#log { source(s_src); filter(f_ppp); destination(d_ppp); };

#log { source(s_src); filter(f_debug); destination(d_debug); };
#log { source(s_src); filter(f_error); destination(d_error); };
#log { source(s_src); filter(f_messages); destination(d_messages); };

#log { source(s_src); filter(f_console); destination(d_console_all);
                    destination(d_xconsole); };
#log { source(s_src); filter(f_crit); destination(d_console); };

# All messages send to a remote site
#
#log { source(s_src); destination(d_net); };

######FIJ######
log { source(s_mysource); filter(f_GEN); destination(d_GEN); };
log { source(s_mysource); filter(f_SU); destination(d_SU); };
log { source(s_mysource); filter(f_WAN); destination(d_WAN); };
log { source(s_mysource); filter(f_CPL); destination(d_CPL); };
###############
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Peter Czanik | 23 Jul 11:06 2014
Picon

syslog-ng is now available in EPEL7

Hi,
syslog-ng 3.5.5 just arrived to EPEL7: 
https://czanik.blogs.balabit.com/2014/07/epel-7-now-contains-syslog-ng/ 
As far as I can see, not all mirrors carry it yet...
Bye,

--

-- 
Peter Czanik (CzP) <peter.czanik <at> balabit.com>
BalaBit IT Security / syslog-ng upstream
http://czanik.blogs.balabit.com/
https://twitter.com/PCzanik

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

John Cole | 21 Jul 22:20 2014

syslog driver parse bug?

Apologies if I missed a bugfix in my web searching and manually scanning the changelogs.

 

I’m running syslog-ng (syslog-ng-3.2.5-3.el6.x86_64) on EL6 with the syslog driver as my network source:

source s_network {

        syslog(ip(0.0.0.0) transport("udp") port(514));

        syslog(ip(0.0.0.0) transport("tcp") port(514));

};

 

I have an application that does not have an internal synchronized clock source.  Per RFC5424, “A syslog application MUST use the NILVALUE as TIMESTAMP if the syslog application is incapable of obtaining system time.”   And, the grammar shows TIMESTAMP = NILVALUE / FULL-DATE "T" FULL-TIME

 

When I specify the “-“ NILVALUE in the syslog message, the syslog driver does not seem to be able to parse the message and does not log anything.

 

If I hardcode a time value, all message fields seem to post appropriately.

 

Is this a new issue, or did I miss a version that addressed the handling of NILVALUE?  Given RedHat lagging on versions, and Fedora’s subsequent changes, I haven’t yet begun the effort of retrofitting the RPM source in RAWHIDE to test, with the hope that someone might recognize the bug, or have a pointer to a EL6 SRPM so I can test against latest…

 

Thanks for any/all pointers for a quick resolution!

 

John

 

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

jrhendri | 21 Jul 22:15 2014

compiling problems --enable-mongodb --enable-json --enable-redis

Hi,

  I am trying to get syslog-ng to be the front-end for a larger log collection architecture including
mongodb, redis, elasticsearch, possibly logstash & kibana.

  Have been trying to build syslog-ng with the necessary modules and failing miserably. 

  I know it must be me - that said, I have spent several hours trying to get this to compile on both RHEL 5 and
Ubuntu 14.04

I ran into a few things that might be useful (or not), but since the platforms and libraries installed are
different, yet the end error is the same I thought I would ask for other eyes / brains to help here...

Thanks for any assistance!

Jim

RHEL 5:
I have installed json-c & redis-stable which provides these libraries:

./redis-stable/deps/hiredis/hiredis.h
./redis-stable/deps/hiredis/libhiredis.a

and created links here to try and get around issues:
/usr/lib/hiredis
/usr/lib/hiredis/libhiredis.a
/usr/include/hiredis
/usr/include/hiredis/hiredis.h

configure succeeds with this:
 Modules:
  Module search path          : /usr/local/lib/syslog-ng
  Sun STREAMS support (module): no
  SSL support (module)        : no
  SQL support (module)        : no
  PACCT module (EXPERIMENTAL) : no
  MongoDB destination (module): yes
  JSON support (module)       : yes
  SMTP support (module)       : no
  AMQP destination (module)   : yes
  STOMP destination (module)  : yes
  GEOIP support (module)      : no
  Redis support (module)      : yes

Make fails here:

  CC       modules/redis/modules_redis_libredis_la-redis-grammar.lo
  CC       modules/redis/modules_redis_libredis_la-redis.lo
  CC       modules/redis/modules_redis_libredis_la-redis-parser.lo
  CCLD     modules/redis/libredis.la
/usr/bin/ld: cannot find -lhiredis
collect2: ld returned 1 exit status
make[2]: *** [modules/redis/libredis.la] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
[n0142566 <at> VDDP13E-F1A47ED syslog-ng-3.5.4.1]$ 

Ubuntu:
I have installed json-c, redis-stable & libredis:
/usr/local/lib/libredis.so.1.0.0
/usr/local/lib/libredis.1
/usr/local/lib/libredis.la
/usr/local/include/redis.h

configure succeeds with this:

 Modules:
  Module search path          : /usr/local/lib/syslog-ng
  Sun STREAMS support (module): no
  SSL support (module)        : yes
  SQL support (module)        : no
  PACCT module (EXPERIMENTAL) : no
  MongoDB destination (module): yes
  JSON support (module)       : yes
  SMTP support (module)       : no
  AMQP destination (module)   : yes
  STOMP destination (module)  : yes
  GEOIP support (module)      : no
  Redis support (module)      : yes

Make fails with:

  CC       modules/redis/modules_redis_libredis_la-redis-grammar.lo
  CC       modules/redis/modules_redis_libredis_la-redis.lo
  CC       modules/redis/modules_redis_libredis_la-redis-parser.lo
  CCLD     modules/redis/libredis.la
/usr/bin/ld: cannot find -lhiredis
collect2: ld returned 1 exit status
make[2]: *** [modules/redis/libredis.la] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
[n0142566 <at> picard:~/src/syslog-ng-3.5.5$ 

===========================================

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Nicholas Radonicich | 21 Jul 15:53 2014

redis/tcp destination throughput and dropped messages

Hello, hoping someone might be able to help... I am attempting to build a syslog server with outputs to Logstash/Elasticsearch via redis but am having issues with dropped messages in such that i cannot get over ~4000 messages per second out to a destination (TCP or redis) without the rest being dropped. I have tried various tuning on the tcp output with flush_lines() and threaded() to no avail, as well as some things on the inputs. Both redis and logstash (used for tcp inputs) were on the localhost.

 

redis-benchmark has no issues doing >70000 LPUSH/second and the file based destinations of syslog-ng are not dropping messages (we are sending almost 80k/s) so I don’t know where I am going wrong.

 

loggen was accepting > 110000mps its just on the outputs it seems.

 

 

 

Thanks, Nick


The information in this message, including in all attachments, is confidential or privileged. In the event you have received this message in error and are not the intended recipient, you are hereby advised that any use, copying or reproduction of this document is strictly forbidden. Please notify immediately the sender of this error and destroy this message, including its attachments, as the case may be.

L'information apparaissant dans ce message électronique et dans les documents qui y sont joints est de nature confidentielle ou privilégiée. Si ce message vous est parvenu par erreur et que vous n'en êtes pas le destinataire visé, vous êtes par les présentes avisé que toute utilisation, copie ou distribution de ce message est strictement interdite. Vous êtes donc prié d’en informer immédiatement l’expéditeur et de détruire ce message, ainsi que les documents qui y sont joints, le cas échéant.

 

Ce courriel provient de Nicholas.Radonicich <at> cogeco.com . Pour assurer la livraison de futurs envois, veuillez inclure la présente adresse courriel à votre carnet d’adresses ou votre liste d’expéditeurs autorisés.

Si vous ne souhaitez plus recevoir de messages promotionnels de la part de Cogeco, veuillez envoyer un courriel à desabonnement <at> cogeco.com.

Politique en matière de protection des renseignements personnels de Cogeco et Engagement en matière d’anti-spamContactez-nous

Cogeco Câble Canada, 5 Place Ville-Marie, Bureau 1700, Montréal, Québec, H3B 0B3

--

This email is from Nicholas.Radonicich <at> cogeco.com . To ensure the delivery of future emails, please add the current email address to your address book or safe senders list.

If you no longer wish to receive promotional emails from Cogeco, please send an email to unsubscribe <at> cogeco.com.

Privacy Policy and Anti-spam Commitment - Contact us

Cogeco Cable Canada, 5 Place Ville-Marie, Suite 1700, Montreal, Quebec, H3B 0B3

 

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Anwar El fatayri | 21 Jul 10:47 2014
Picon

Syslog-ng using an old Template

<!-- .hmmessage P { margin:0px; padding:0px } body.hmmessage { font-size: 12pt; font-family:Calibri } -->
Hey,

I was using this template for my tests : 

template t_lyraJBossFormat {

                template("${DATE} ${FULLHOST} jboss_vad - ------------ - SD : ${SDATA} - ------------ - MSG :
 ${MSGONLY}\n");
                template_escape(no);
};

This template is a Format that I use for JBoss Logs. The syslog packet (IETF) is created and then sent to syslog-ng via a named pipe. 

After Debugging all the problems, the following template was created to be used for all Jboss Logs :

template t_lyraJBossFormat {

                template("${DATE} ${FULLHOST} jboss_vad - ${SDATA} - ${MSGONLY}\n");
                template_escape(no);
};


The problem is that syslog-ng is still using the old template. (I restarted the syslog-ng service of course) 

Any ideas ?

Thanks in advance.

El Fatayri Anwar


______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq


Gmane