Frank Wilkinson | 26 Aug 19:48 2014
Picon

really need help...not writing files

Please forgive me if this has already been addressed. If so will you point me to it?

Syslog-ng will all of a sudden, stop writing files.

 

I’m running syslog-ng 3.5.3

Installer-Version: 3.5.3

Revision: ssh+git://algernon <at> git.balabit/var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#ccb05a22408ba4c837d998b2538854d994f845a5

Compile-Date: Jan  8 2014 13:35:02

Available-Modules: afsocket,afprog,dbparser,system-source,affile,syslogformat,linux-kmsg-format,csvparser,afmongodb,afsocket-tls,confgen,afuser,afstomp,afsocket-notls,basicfuncs,cryptofuncs,afamqp

Enable-Debug: off

Enable-GProf: off

Enable-Memtrace: off

Enable-IPv6: on

Enable-Spoof-Source: off

Enable-TCP-Wrapper: on

Enable-Linux-Caps: off

Enable-Pcre: on

 

 

The service status is showing running but not writing log files.  We are logging  udp from about 2400 devices

When it dies strace shows:

 

epoll_ctl(3, EPOLL_CTL_DEL, 10, {0, {u32=19726648, u64=19726648}}) = 0

write(110, "\1\0\0\0\0\0\0\0", 8)       = 8

epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 11, 3414) = 1

read(6, "\1\0\0\0\0\0\0\0", 8)          = 8

futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1

futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1

fcntl(10, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)

fcntl(10, F_GETFL)                      = 0x802 (flags O_RDWR|O_NONBLOCK)

setsockopt(10, SOL_SOCKET, SO_OOBINLINE, [1], 4) = 0

write(110, "\1\0\0\0\0\0\0\0", 8)       = 8

epoll_ctl(3, EPOLL_CTL_ADD, 10, {0, {u32=19726648, u64=19726648}}) = 0

epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 12, 3413) = 1

read(6, "\1\0\0\0\0\0\0\0", 8)          = 8

futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1

futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1

futex(0x14e25a0, FUTEX_WAKE_PRIVATE, 1) = 1

epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 12, 0) = 1

read(6, "\1\0\0\0\0\0\0\0", 8)          = 8

futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1

futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1

write(110, "\1\0\0\0\0\0\0\0", 8)       = 8

futex(0x12d02c0, FUTEX_WAIT_PRIVATE, 2, NULL) = ? ERESTARTSYS (To be restarted)

--- SIGTERM (Terminated) <at> 0 (0) ---

write(17, "\1\0\0\0\0\0\0\0", 8)        = 8

rt_sigreturn(0x7fe21cfab740)            = 202

futex(0x12d02c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>

 

here is where I did a restart

+++ killed by SIGKILL +++

 

top - 12:45:56 up 133 days, 23:11, 13 users,  load average: 1.06, 1.13, 1.14

Tasks: 634 total,   2 running, 632 sleeping,   0 stopped,   0 zombie

Cpu(s):  4.2%us,  2.0%sy,  0.0%ni, 93.6%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%s

Mem:  32898840k total, 31285296k used,  1613544k free,   128188k buffers

Swap: 16777212k total,   684800k used, 16092412k free, 29249028k cached

 

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

1631 root      20   0 1725m 104m 2892 S 28.4  0.3  10:38.46 syslog-ng        2843 root      20   0 1725m 104m 2892 S 17.5  0.3   0:04.35 syslog-ng

2795 root      20   0 1725m 104m 2892 S 15.8  0.3   0:11.99 syslog-ng        2842 root      20   0 1725m 104m 2892 S 13.9  0.3   0:02.68 syslog-ng

2793 root      20   0 1725m 104m 2892 S 13.5  0.3   0:14.54 syslog-ng        2855 root      20   0 1725m 104m 2892 R 13.5  0.3   0:00.41 syslog-ng

2776 root      20   0 1725m 104m 2892 S 12.2  0.3   0:18.57 syslog-ng       43203 root      20   0  359m 101m  10m S 11.9  0.3  15:35.10 splunkd

2794 root      20   0 1725m 104m 2892 S  9.6  0.3   0:14.62 syslog-ng        2791 root      20   0 1725m 104m 2892 S  9.2  0.3   0:11.89 syslog-ng

2697 root      20   0 1725m 104m 2892 S  6.3  0.3   0:31.74 syslog-ng       43204 root      20   0  359m 101m  10m S  4.9  0.3   8:01.72 splunkd

2825 root      20   0 1725m 104m 2892 S  2.3  0.3   0:07.73 syslog-ng        2841 root      20   0 1725m 104m 2892 S  1.6  0.3   0:03.30 syslog-ng

 

Also, one other problem I have is the syslog-ng log file says:

Aug 26 11:48:49 sopher1 syslog-ng[488]: Input is valid utf8, but the log message is not tagged as such, this performs worse than enabling validate-utf8 flag on input; value='758AARULOCAL01'

 

My config specifies flags(validate-utf8):

 

source s_udp { udp( port(514) so_rcvbuf(15000000) log_iw_size(50000) log_msg_size(65535) log_fetch_limit(50000) flags(validate-utf8));};

Frank Wilkinson

(205)934-3540 w

 

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

James Lay | 25 Aug 23:47 2014
Picon

Re: Quick set of eyeballs on this

On 2014-08-25 15:39, Balint Kovacs wrote:
> Hi,
>
>  just a guess, but as far as i can remember, the ASA-* part is 
> usually
> in the program field, try the program() filter instead of message() 
> in
> f_firewall.
>
>  Balint
>
> On 2014. augusztus 25. 22:48:05 CEST, James Lay
> <jlay <at> slave-tothe-box.net> wrote:
>
>> Can anyone see anything blatantly wrong with this? The goal is to
>> syslog as usual, but to forward firewall messages to a different
>> server.
>> Thanks for looking all.
>>
>> James
>>
>>  <at> version:3.3.5
>> options {
>> use_dns(no);
>> flush_lines(0);
>> stats_freq(43200);
>> };
>>
>> source s_local {
>> unix-stream("/dev/log");
>> udp(ip(0.0.0.0) port(514));
>> tcp(ip(0.0.0.0) port(514));
>> file("/proc/kmsg");
>> };
>>
>> destination d_file {
>> file("/var/log/messages");
>> };
>>
>> destination d_syslogserver { udp ("x.x.x.x", port(7514)); };
>>
>> filter f_syslogfilter {
>> not (
>> message("0x0004")
>> or message("169.254.")
>> or message("192.168.")
>> );
>> };
>>
>> filter f_firewall {
>> message("ASA-4-71005")
>>
>> or
>> message("ASA-2-106100")
>> };
>>
>> log {
>> source(s_local);
>> filter(f_syslogfilter);
>> destination(d_file);
>> };
>>
>> log {
>> source(s_local);
>> filter(f_firewall);
>> destination(d_syslogserver);
>> };
>>
>> -------------------------

Ah thank you...I've just been testing using logger.  I'll adjust and 
try this with program().  Thanks again.

James

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

James Lay | 25 Aug 22:48 2014
Picon

Quick set of eyeballs on this

Can anyone see anything blatantly wrong with this?  The goal is to 
syslog as usual, but to forward firewall messages to a different server. 
Thanks for looking all.

James

 <at> version:3.3.5
options {
         use_dns(no);
         flush_lines(0);
         stats_freq(43200);
};

source s_local {
         unix-stream("/dev/log");
         udp(ip(0.0.0.0) port(514));
         tcp(ip(0.0.0.0) port(514));
         file("/proc/kmsg");
};

destination d_file {
         file("/var/log/messages");
};

destination d_syslogserver { udp ("x.x.x.x", port(7514)); };

filter f_syslogfilter {
         not (
                 message("0x0004")
                 or message("169\.254\.")
                 or message("192\.168\.")
         );
};

filter f_firewall {
         message("ASA-4-71005")
         or message("ASA-2-106100")
};

log {
         source(s_local);
         filter(f_syslogfilter);
         destination(d_file);
};

log {
         source(s_local);
         filter(f_firewall);
         destination(d_syslogserver);
};

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Jason Long | 24 Aug 19:33 2014
Picon

I want to forward Windows server 2008 R2 logs to Linux syslog-ng.

Hello all.
I have a Windows Box that want to forward all Even logs to my Linux box. I install Snare on Windows(172.30.10.19) and configure it to forward logs to Linux  and my Linux box receive it properly. When I use " tcpdump udp "port 514" ", Tcpdump show me that Snare sending Logs to Linux but Syslog-ng can't write it to log files :(. I paste my syslog-ng configure :

options {
flush_lines (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (no);
use_fqdn (no);
create_dirs (no);
keep_hostname (yes);
};

source s_sys {
file ("/proc/kmsg" program_override("kernel: "));
unix-stream ("/dev/log");
internal();
#udp(ip(0.0.0.0) port(514));
};


#source s_net {
#udp (ip(172.30.10.19) port(514));
#};
source s_net { udp(); };
filter f_openwrt { host("172.30.10.19");};
destination df_openwrt { file("/var/log/winlog/win.log"); };
log { source ( s_net ); filter( f_openwrt ); destination ( df_openwrt ); };


destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };

filter f_kernel     { facility(kern); };
filter f_default    { level(info..emerg) and
                        not (facility(mail)
                        or facility(authpriv) 
                        or facility(cron)); };
filter f_auth       { facility(authpriv); };
filter f_mail       { facility(mail); };
filter f_emergency  { level(emerg); };
filter f_news       { facility(uucp) or
                        (facility(news) 
                        and level(crit..emerg)); };
filter f_boot   { facility(local7); };
filter f_cron   { facility(cron); };

#log { source(s_sys); filter(f_kernel); destination(d_cons); };
log { source(s_sys); filter(f_kernel); destination(d_kern); };
log { source(s_sys); filter(f_default); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_auth); };
log { source(s_sys); filter(f_mail); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_cron); };

# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:



Can you tell me how can I solve this problem? 

Cheers.
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Scot Needy | 20 Aug 13:11 2014
Picon

create_dir and "dynamic" destinations.

	

Hi,  

  I was wondering if anyone has tried to work around syslog_ng. My destinations are psudo dynamic in that I
have one per subnet but I don’t want to pre-create every destination directory until a syslog message is
received for that destination. 

I have 3 conf files (Dfilters.conf Ddestination.conf and Dlog.conf ) that are built using an API to our
IPNMS (IP Network Management System). The purpose is to use syslog-ng stats to quickly report on subnets
and also provide per-subnet options for reporting, forwarding …  

Example Destination 
	destination d_192_168_9_0 { file(/opt/syslog-ng/logs/192_168_9_0/$YEAR$MONTH$DAY.$HOST.log);};

Example Filter 
	filter f_192_168_9_0 { netmask(192.168.9.0/26);};

Example Log 
	log { source(s_net); filter(f_192_168_9_0); destination(d_192_168_9_0);};

My question is can anyone think of a way to get the create_dir option to dynamically create my subnet
directories as needed ?  
For now I Need to pre-create every directory or syslog-ng complains which causes me to create a bunch of
empty directories. 

Thanks  
Scot 

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Czanik, Péter | 19 Aug 15:58 2014

syslog-ng 3.5 for RHEL6

Hi,
Recently many people asked for syslog-ng 3.5 rpms for RHEL6. What I
did breaks most of the packaging rules, but works. It's a wild mixture
of the current EPEL6 and EPEL7 packages. So it has most of the
features of the EPEL7 package (except UDP spoofing), but has sysvinit
style init script. It's installed to /usr/sbin instead of /sbin, has
SSL support enabled, and uses bundled libraries instead of system
libraries (as those are missing from RHEL6 and EPEL6).
To use the packages, you need to enable EPEL6 repositories for
dependencies ( https://fedoraproject.org/wiki/EPEL ) and also my Copr
repository: http://copr.fedoraproject.org/coprs/czanik/syslog-ng35epel6/
It's ugly as hell, but survived my quick test on CentOS 6.5. Use at
your own risk but let me know if you run into any trouble!
Bye,
--

-- 
Peter Czanik (CzP) <peter.czanik <at> balabit.com>
BalaBit IT Security / syslog-ng upstream
http://czanik.blogs.balabit.com/
https://twitter.com/PCzanik
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

devel | 19 Aug 14:59 2014
Picon

syslog-ng 3.6.0alpha3 has been released

------------------------------------------------------------------------------
PACKAGE             : syslog-ng
VERSION             : 3.6.0alpha3
SUMMARY             : new alpha release
DATE                : Aug 19, 2014
------------------------------------------------------------------------------

DESCRIPTION:

  The third alpha version of syslog-ng Open Source Edition (3.6.0alpha3) has
  been released. This being an alpha release, testing is most welcome, but
  production use is not recommended!

CHANGES:

3.6.0alpha3
	Released: Tue, 19 Aug 2014 14:33:00 +0200

This is the third alpha release of the upcoming syslog-ng OSE 3.6
branch. It is expected to be the last alpha release, with the first
beta in about two weeks. This release contains a number of important
features and bugfixes:

Changed defaults
----------------

* The `flush-lines()` setting now defaults to *100*, rather than *1*,
  for increased speed.

Features
--------

* The `system()` source will now parse ` <at> cim` marked messages as JSON,
  if the JSON module is available at run-time. This improves
  inter-operation with other software that uses the *Common
  Information Model*.

### Features from the [Incubator][incubator]

* The `$(or)` template function that returns the first non-empty
  argument is now included in syslog-ng itself.

* The `$(padding)` template function, to pad text with custom padding
  to a given length is also included.

* The `$(graphite-output)` template function, to be used for sending
  metrics to [Graphite][graphite] was ported over from the Incubator.
  The `graphite()` destination SCL block is also available now, to
  make it even easier to talk to Graphite.

* The `riemann()` destination, which allows sending metrics to the
  [Riemann][riemann] monitoring system was also ported over from the
  Incubator.

 [graphite]: http://graphite.wikidot.com/
 [incubator]: https://github.com/balabit/syslog-ng-incubator
 [riemann]: http://riemann.io/

### Threaded destinations

A number of features were implemented for all threaded destinations:
`amqp()`, `mongodb()`, `redis()`, `riemann()`, `smtp()` and `stomp()`.

* The destinations gained support for `SEQNUM` persistence: the
  counter will be preserved across reloads and restarts.

* A new option called `retries()` was implemented for all of these,
  which controls how many times a message delivery is retried before
  dropping it.

* The `throttle()` option is now implemented, and works for all of the
  aforementioned destination drivers.

* The message delivery loop was optimised to do less sleep/wakeup
  cycles, which should make the drivers not only faster, but more CPU
  friendly too.

Bugfixes
--------

* The basicfuncs module was fixed to work correctly on 32-bit
  architectures.
* The `stored` statistics is no longer incremented by various drivers
  when they mean `processed`.
* The type hinting feature is now more picky about what kind of type
  hints it accepts, allowing one to use template functions in - for
  example - `$(format-json)` pairs.

Miscellaneous changes
---------------------

* We now ship a "Contributors Guide" in the `CONTRIBUTING.md` file.

Credits
-------

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Balazs Scheidler, Fabien Wernli, Gergely Nagy, Laszlo
Budai, Peter Czanik, Robert Fekete, Tibor Benke, Viktor Juhasz, Viktor
Tusa.

DOWNLOAD:

  You can download the source or binary packages from:

    http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.6.0alpha3/

  The documentation of the syslog-ng Open Source Edition is available in
  The syslog-ng Open Source Edition Administrator's Guide at

    http://www.balabit.com/support/documentation/

------------------------------------------------------------------------------
PACKAGE             : syslog-ng
VERSION             : 3.6.0alpha3
SUMMARY             : new alpha release
DATE                : Aug 19, 2014
------------------------------------------------------------------------------

DESCRIPTION:

  The third alpha version of syslog-ng Open Source Edition (3.6.0alpha3) has
  been released. This being an alpha release, testing is most welcome, but
  production use is not recommended!

CHANGES:

3.6.0alpha3
	Released: Tue, 19 Aug 2014 14:33:00 +0200

This is the third alpha release of the upcoming syslog-ng OSE 3.6
branch. It is expected to be the last alpha release, with the first
beta in about two weeks. This release contains a number of important
features and bugfixes:

Changed defaults
----------------

* The `flush-lines()` setting now defaults to *100*, rather than *1*,
  for increased speed.

Features
--------

* The `system()` source will now parse ` <at> cim` marked messages as JSON,
  if the JSON module is available at run-time. This improves
  inter-operation with other software that uses the *Common
  Information Model*.

### Features from the [Incubator][incubator]

* The `$(or)` template function that returns the first non-empty
  argument is now included in syslog-ng itself.

* The `$(padding)` template function, to pad text with custom padding
  to a given length is also included.

* The `$(graphite-output)` template function, to be used for sending
  metrics to [Graphite][graphite] was ported over from the Incubator.
  The `graphite()` destination SCL block is also available now, to
  make it even easier to talk to Graphite.

* The `riemann()` destination, which allows sending metrics to the
  [Riemann][riemann] monitoring system was also ported over from the
  Incubator.

 [graphite]: http://graphite.wikidot.com/
 [incubator]: https://github.com/balabit/syslog-ng-incubator
 [riemann]: http://riemann.io/

### Threaded destinations

A number of features were implemented for all threaded destinations:
`amqp()`, `mongodb()`, `redis()`, `riemann()`, `smtp()` and `stomp()`.

* The destinations gained support for `SEQNUM` persistence: the
  counter will be preserved across reloads and restarts.

* A new option called `retries()` was implemented for all of these,
  which controls how many times a message delivery is retried before
  dropping it.

* The `throttle()` option is now implemented, and works for all of the
  aforementioned destination drivers.

* The message delivery loop was optimised to do less sleep/wakeup
  cycles, which should make the drivers not only faster, but more CPU
  friendly too.

Bugfixes
--------

* The basicfuncs module was fixed to work correctly on 32-bit
  architectures.
* The `stored` statistics is no longer incremented by various drivers
  when they mean `processed`.
* The type hinting feature is now more picky about what kind of type
  hints it accepts, allowing one to use template functions in - for
  example - `$(format-json)` pairs.

Miscellaneous changes
---------------------

* We now ship a "Contributors Guide" in the `CONTRIBUTING.md` file.

Credits
-------

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Balazs Scheidler, Fabien Wernli, Gergely Nagy, Laszlo
Budai, Peter Czanik, Robert Fekete, Tibor Benke, Viktor Juhasz, Viktor
Tusa.

DOWNLOAD:

  You can download the source or binary packages from:

    http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.6.0alpha3/

  The documentation of the syslog-ng Open Source Edition is available in
  The syslog-ng Open Source Edition Administrator's Guide at

    http://www.balabit.com/support/documentation/

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Riyas Ahamed | 19 Aug 08:52 2014

Syslog-ng log rotation problem

Hi Team,

 

I have configured syslog-ng 3.2.5 with backend as mysql  Ver 14.14 Distrib 5.6.20,

 

I have able to fetch logs from the backend to frondend.

 

Please help me to how to rotate syslog-ng logs on daily basis.

 

 

 

 

Regards,

N.B.RIAZ AHMED

 

 

https://www.csscorp.com/email-disclaimer
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

VMI X | 18 Aug 23:06 2014
Picon

Recommended Data base

Hi,
Currently we're logging everything to text files for a few LAN clients.
We're considering using a database instead and have a few questions to help us decide:
  • Would a database be a good option to replace existing text files for long term storage considering storage space?
  • Would mongo OR mysql be better suited for storing system logs? 
I understand answers to these questions can vary depending on specific use case but seeking a general recommendation to see what's typically being used and what the most stable/supported options would be.


--
Nullius In Verba
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Czanik, Péter | 14 Aug 12:19 2014

insider 2014-08: EPEL; graphite; PCI DSS;

Dear syslog-ng users,

This is the 36th issue of the syslog-ng Insider, a monthly newsletter
that brings you syslog-ng related news.

FEATURED NEWS

EPEL 7 now contains syslog-ng

-----------------------------

RHEL 7 was released over a month ago and CentOS 7 not much later, but
one piece of software was still missing: syslog-ng. Not any more.
EPEL, which stands for Extra Packages for Enterprise Linux, is a
software collection containing additional packages for Enterprise
Linux and derivatives. Now its latest version, EPEL 7 also contains
syslog-ng, version 3.5:

http://czanik.blogs.balabit.com/2014/07/epel-7-now-contains-syslog-ng/

Introducing syslog-ng PE 5F1

----------------------------

The latest version of the syslog-ng Premium Edition, 5F1, adds support
for the popular NoSQL database MongoDB. Along with support for
MongoDB, we have added support for Java Script Object Notation (JSON),
a text-based open standard designed for human-readable data
interchange. You can read more at
http://gyp.blogs.balabit.com/2014/08/introducing-syslog-ng-premium-edition-5f1/

Performance monitoring using syslog-ng and graphite

---------------------------------------------------

For most of its history, syslog-ng could only be used for collecting,
processing and storing log messages. Not any more. The Redis and
Riemann destinations are already a step into the direction of
metrics-based monitoring, and the monitoring source combined with
Graphite template support are the next.

https://czanik.blogs.balabit.com/2014/07/how-to-setup-syslog-ng-quickly-for-performance-monitoring-using-graphite-inside-docker/

Introducing syslog-ng store box 3F2

-----------------------------------

We recently released a new version of our log management appliance,
the syslog-ng Store Box. 3F2 is the latest feature release and
includes one major new feature and a major improvement to an existing
one. First, we have added a RESTful API which opens up all sorts of
possibilities for accessing log data in SSB. Second, we have revamped
the search interface on the web-based user interface making searching
and troubleshooting much easier. You can read more at
http://gyp.blogs.balabit.com/2014/08/syslog-ng-store-box-3-f2-released/

Syslog-ng incubator 0.3.3 released

----------------------------------

The syslog-ng incubator is a set of tools and modules for syslog-ng,
which are not (yet) available in the official release. This version of
incubator works with the latest stable syslog-ng (v3.5.5+) and fixes
many problems of the initial 0.3 incubator release.

https://czanik.blogs.balabit.com/2014/07/syslog-ng-incubator-0-3-3-is-available/

Log management and the Verizon 2014 PCI Compliance Report

---------------------------------------------------------

Recently, the eagerly anticipated Verizon Data Breach Investigations
Report for 2014 was published. With more than 63,000 security
incidents, 1,300 confirmed data breaches and 50 contributing global
organizations, it provides the most comprehensive insight to state of
IT security around the world. Drawing on data from the Data Breach
Investigation Report, Verizon also publishes a lesser known but very
interesting report on the state of compliance of with the Payment Card
Industry Data Security Standard (PCI DSS), perhaps the most
widely-adopted security standards globally. Read, what requirements
PCI DSS has towards log management:

https://jluby.blogs.balabit.com/2014/06/30/key-log-management-takeaways-from-the-verizon-2014-pci-compliance-report/

NEW RELEASES

syslog-ng OSE 3.5.6:
https://lists.balabit.hu/pipermail/syslog-ng-announce/2014-August/000192.html

syslog-ng OSE 3.6 alpha2:
https://lists.balabit.hu/pipermail/syslog-ng-announce/2014-August/000193.html

syslog-ng PE 5LTS (5.0.5a):
https://lists.balabit.hu/pipermail/syslog-ng-announce/2014-June/000189.html

syslog-ng incubator 0.3.3

Your feedback and news tips about the next issue is welcome at
documentation <at> balabit.com To read this newsletter on-line, visit:
http://insider.blogs.balabit.com/

--

-- 
Peter Czanik (CzP) <peter.czanik <at> balabit.com>
BalaBit IT Security / syslog-ng upstream
http://czanik.blogs.balabit.com/
https://twitter.com/PCzanik
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Simão Mata | 12 Aug 17:53 2014

connection broken every few minutes

Hello,

Our syslog-ng clients are losing their connection to our central log
server every few minutes.

In the client I have the following log:

2014-08-12T15:44:02+00:00 wimdu-app07 syslog.notice syslog-ng[14032]:
Syslog connection broken; fd='10', server='AF_INET(<our ip>:514)',
time_reopen='1'
2014-08-12T15:44:03+00:00 wimdu-app07 syslog.notice syslog-ng[14032]:
Syslog connection established; fd='10', server='AF_INET(<our ip>:514)',
local='AF_INET(0.0.0.0:0)'

I am using the following configuration on the clients:

destination d_net_wimdu {
    syslog(<our ip>
    transport("tcp")
    port(514)
    log-fifo-size(2048));
};

log { source(s_src); source(s_net); destination(d_net_wimdu); };

And on the server:

source s_net {
    syslog(ip(0.0.0.0)
    transport("tcp")
    max-connections(300)
    log-iw-size(3300)
    so_rcvbuf(8000000000)
    port(514));
};

log {
    source(s_net);
    destination(d_file_w);
};

destination d_prog_logparser {
    program("< logparser_bin >"
    template(t_w_filetemplate)
    log_fifo_size(90000000)
    flags(no-multi-line));
};

log {
  source(s_net);
  destination(d_prog_logparser);
  flags(flow-control);
};

Any idea how I can even debug this?

Thank you

-- 

[image: Wimdu] <http://www.wimdu.com/>*Wimdu GmbH* - Voltastraße 5, 13355 
Berlin, Germany
*Managing Directors* - Arne Bleckwenn, Hinrich Dreiling
*Commercial Register Number* - 129773 B Berlin[image: Contact] 
<contact <at> wimdu.com>[image: Blog] <http://blog.wimdu.com/?wt_vi=signature>[image: 
FB] <http://www.facebook.com/wimdu>[image: TW] 
<http://www.twitter.com/wimdu>[image: G+] <http://www.google.com/+wimdu>[image: 
YT] <http://www.youtube.com/wimtubechannel>[image: GH] 
<https://github.com/wimdu>[image: LI] 
<http://www.linkedin.com/company/wimdu-gmbh>[image: XI] 
<http://www.xing.com/companies/wimdu>
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq


Gmane