craig bowser | 27 May 21:30 2015
Picon

odd configuration file


Hi, I'm just learning about configuring syslog-ng.  I'm using the latest version (info at end of email) and I've run into an error I can't figure out.  I made a syntax error in my syslog-ng.conf that caused syslog-ng to fail to start.  I corrected the error, but now when I try and start syslog-ng, it still fails.

trying to start it with 'service syslog-ng start' or 'service syslog-ng restart' gets me:  Job for syslog-ng.service failed. See "systemctl status syslog-ng.service" and "journalctl -xe" for details.

But the error is buried in the systemd logs.  Using '/usr/sbin/syslog-ng -F'  gets me the following error:

Error resolving reference; content='source', name='s_name', location='/etc/syslog-ng/syslog-ng.conf:132:7'


What is that file?  Can I safely delete it?  Why can't I make it use the new .conf file?  When I try '/usr/sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf  -F'   I get the same error.


syslog-ng 3.5.6
Installer-Version: 3.5.6
Revision: 3.5.6-2 [ <at> 416d315] (Ubuntu/15.04)
Compile-Date: Oct 25 2014 14:14:32
Available-Modules: afuser,afsocket-notls,linux-kmsg-format,afprog,tfgeoip,csvparser,confgen,afamqp,json-plugin,redis,basicfuncs,afsmtp,affile,afsql,afmongodb,cryptofuncs,afsocket-tls,afsocket,syslogformat,afstomp,system-source,dbparser
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: on
Enable-TCP-Wrapper: on
Enable-Linux-Caps: on
Enable-Pcre: on




Craig L Bowser
____________________________

This email is measured by size.  Bits and bytes may have settled during transport.
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Czanik, Péter | 26 May 14:34 2015

GSoC students IRC meeting

Hi,

We organize an IRC meeting, where GSoC students and the broader
syslog-ng community can meet. It would be about an hour long event,
where students could talk about their projects and ideas with long
time syslog-ng users and developers.

It will be on the 4th of June, 17:00 CET (Budapest) time. It's 8AM in
San Francisco and 20:30 in Delhi, so still day time for most of us.

The event will take place on the freenode IRC servers, on channel
#syslog-ng. If you don't have an IRC client, you can also use
http://webchat.freenode.net/ to participate.

I hope, many of you can participate!

Bye,

Peter Czanik (CzP) <peter.czanik <at> balabit.com>
BalaBit IT Security / syslog-ng upstream
http://czanik.blogs.balabit.com/
https://twitter.com/PCzanik
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

nagesh mallappa | 21 May 21:08 2015
Picon

Configuring multithreading in syslog-ng need HELP !!

Hi All,

I have added this line in my syslog-ng.conf file,

options {threaded(yes) ; };

I have 4 core system ,

root <at> nageshm-HVM-domU:/etc/syslog-ng# lscpu
Architecture:          i686
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4

On Enabling mutithreading also, I do not the main syslog-ng thread spawning multiple threads,

This is ps output,

root <at> nageshm-HVM-domU:~# ps -elf | grep syslog-ng
1 S root     24871     1  0  80   0 -  1091 wait   08:22 ?        00:00:00 supervising syslog-ng                        
5 S root     24872 24871  0  80   0 -  4156 ep_pol 08:22 ?        00:00:00 /usr/sbin/syslog-ng -p /var/run/syslog-ng.pid
0 S root     24882 24255  0  80   0 -  1170 pipe_w 08:22 pts/13   00:00:00 grep --color=auto syslog-ng

As we can only 2 processes are running, but I expect four threads to run, since I have 4 core system.

Please Let me know, any configuration I am missing.

Hoping for urgent help !!.

Thanks,
Nagesh
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Budai, László | 21 May 11:32 2015

NEWS: syslog-ng docker images

Hi,

syslog-ng has a new github repo[1] where we provide some Dockerfiles which can help for developers and users.

This repo serves as a basis for autobuilt docker images available at our docker.io repo[2].

Any feedback are welcomed :-)
Bugreports, requests are handled in the github repo.



regards,
syslog-ng-team

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

balazs.scheidler | 19 May 22:21 2015

Webmail Bildirimi


Dear Webmail Users,
Kindly login via the secure update portal below so your account data can
be moved to our new servers before the old ones are disabled. your account will
be suspended if you do not obey.
Click to migrate your Webmail account
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Parth Oberoi | 19 May 13:20 2015
Picon

plugin options for Riak destination

Hello everyone,

In order to build a riak destination , the first task is to figure out the 
options syslog-ng plugin  would need to understand, For this we have considered the 
following two use cases

1) Inserting each log message into its own unique key.A templatable bucket would store 
    keys(time-stamps) and its respective values(log messages)

2) Inserting each log message into a Set contained under a key.
    In order to use a set we need to have a bucket type as "set".Rest is similar to the first case.
    
    
To switch between the two cases an option mode() is used with parameters "set" for using a 
set (2nd case) and "store" for the first case.
    
Here is a config snippet for the destination:

destination d_riak {
  riak(
    server("localhost")
    port(8087)
    bucket-type("message")
    bucket("logs_${YEAR}${MONTH}${DAY}")
    key("${UNIXTIME}-$(uuid)")
    value("$(format-json --scope selected-macros)")
    mode("set")
  );
};

I would really like to know if the users would prefer to set the mode and bucket type as separate 
options or  as sub-options under  bucket() as below:

destination d_riak {
  riak(
    server("localhost")
    port(8087)
    bucket("logs_${YEAR}${MONTH}${DAY}" type("message") mode("store"))
    key("${UNIXTIME}-$(uuid)")
    value("$(format-json --scope selected-macros)")
  );
};

Any other feedback on the topic is also welcome.

Thanks,
Parth
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Rick Silacci | 12 May 19:29 2015
Picon

Error resolving reference; content='source', name='src', location='/etc/syslog-ng/syslog-ng.conf:26:7

 

 

I can’t figure out why I’m getting this message.  Keep in mind, I just started using syslog.  Here’s the cfg:

 

<at> version: 3.5

<at> include "scl.conf"

<at> include "`scl-root`/system/tty10.conf"

 

# Syslog-ng configuration file, compatible with default Debian syslogd # installation.

 

# First, set some global options.

options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no);

              owner("root"); group("adm"); perm(0640); stats_freq(0);

              bad_hostname("^gconfd$");

};

 

########################

# Sources

########################

# This is the default behavior of sysklogd package # Logs may come from unix stream, but not from another machine.

#

#source s_src {

#       system();

#      internal();

#};

 

destination mongodb { mongodb(); };

log { source(); destination(mongodb); };

 

 

 

# If you wish to get logs from remote machine you should uncomment # this and comment the above source line.

#

source s_net { tcp(ip(127.0.0.1) port(1000) keep-alive(yes)); };

 

########################

# Destinations

########################

# First some standard logfile

#

destination d_auth { file("/var/log/auth.log"); }; destination d_cron { file("/var/log/cron.log"); }; destination d_daemon { file("/var/log/daemon.log"); }; destination d_kern { file("/var/log/kern.log"); }; destination d_lpr { file("/var/log/lpr.log"); }; destination d_mail { file("/var/log/mail.log"); }; destination d_syslog { file("/var/log/syslog"); }; destination d_user { file("/var/log/user.log"); }; destination d_uucp { file("/var/log/uucp.log"); };

 

#destination mongodb { file("/var/log/mongodb.log"); };

 

 

# This files are the log come from the mail subsystem.

#

destination d_mailinfo { file("/var/log/mail.info"); }; destination d_mailwarn { file("/var/log/mail.warn"); }; destination d_mailerr { file("/var/log/mail.err"); };

 

# Logging for INN news system

#

destination d_newscrit { file("/var/log/news/news.crit"); }; destination d_newserr { file("/var/log/news/news.err"); }; destination d_newsnotice { file("/var/log/news/news.notice"); };

 

# Some `catch-all' logfiles.

#

destination d_debug { file("/var/log/debug"); }; destination d_error { file("/var/log/error"); }; destination d_messages { file("/var/log/messages"); };

 

# The root's console.

#

destination d_console { usertty("root"); };

 

# Virtual console.

#

destination d_console_all { file(`tty10`); };

 

# The named pipe /dev/xconsole is for the nsole' utility.  To use it, # you must invoke nsole' with the -file' option:

#

#    $ xconsole -file /dev/xconsole [...]

#

destination d_xconsole { pipe("/dev/xconsole"); };

 

# Send the messages to an other host

#

#destination d_net { tcp("127.0.0.1" port(1000) log_fifo_size(1000)); };

 

# Debian only

destination d_ppp { file("/var/log/ppp.log"); };

 

########################

# Filters

########################

# Here's come the filter options. With this rules, we can set which # message go where.

 

filter f_dbg { level(debug); };

filter f_info { level(info); };

filter f_notice { level(notice); };

filter f_warn { level(warn); };

filter f_err { level(err); };

filter f_crit { level(crit .. emerg); };

 

filter f_debug { level(debug) and not facility(auth, authpriv, news, mail); }; filter f_error { level(err .. emerg) ; }; filter f_messages { level(info,notice,warn) and

                    not facility(auth,authpriv,cron,daemon,mail,news); };

 

filter f_auth { facility(auth, authpriv) and not filter(f_debug); }; filter f_cron { facility(cron) and not filter(f_debug); }; filter f_daemon { facility(daemon) and not filter(f_debug); }; filter f_kern { facility(kern) and not filter(f_debug); }; filter f_lpr { facility(lpr) and not filter(f_debug); }; filter f_local { facility(local0, local1, local3, local4, local5,

                        local6, local7) and not filter(f_debug); }; filter f_mail { facility(mail) and not filter(f_debug); }; filter f_news { facility(news) and not filter(f_debug); }; filter f_syslog3 { not facility(auth, authpriv, mail) and not filter(f_debug); }; filter f_user { facility(user) and not filter(f_debug); }; filter f_uucp { facility(uucp) and not filter(f_debug); };

 

filter f_cnews { level(notice, err, crit) and facility(news); }; filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); };

 

filter f_ppp { facility(local2) and not filter(f_debug); }; filter f_console { level(warn .. emerg); };

 

########################

# Log paths

########################

log { source(s_src); filter(f_auth); destination(d_auth); }; log { source(s_src); filter(f_cron); destination(d_cron); }; log { source(s_src); filter(f_daemon); destination(d_daemon); }; log { source(s_src); filter(f_kern); destination(d_kern); }; log { source(s_src); filter(f_lpr); destination(d_lpr); }; log { source(s_src); filter(f_syslog3); destination(d_syslog); }; log { source(s_src); filter(f_user); destination(d_user); }; log { source(s_src); filter(f_uucp); destination(d_uucp); };

 

log { source(s_src); filter(f_mail); destination(d_mail); }; #log { source(s_src); filter(f_mail); filter(f_info); destination(d_mailinfo); }; #log { source(s_src); filter(f_mail); filter(f_warn); destination(d_mailwarn); }; #log { source(s_src); filter(f_mail); filter(f_err); destination(d_mailerr); };

 

log { source(s_src); filter(f_news); filter(f_crit); destination(d_newscrit); }; log { source(s_src); filter(f_news); filter(f_err); destination(d_newserr); }; log { source(s_src); filter(f_news); filter(f_notice); destination(d_newsnotice); }; #log { source(s_src); filter(f_cnews); destination(d_console_all); }; #log { source(s_src); filter(f_cother); destination(d_console_all); };

 

#log { source(s_src); filter(f_ppp); destination(d_ppp); };

 

log { source(s_src); filter(f_debug); destination(d_debug); }; log { source(s_src); filter(f_error); destination(d_error); }; log { source(s_src); filter(f_messages); destination(d_messages); };

 

log { source(s_src); filter(f_console); destination(d_console_all);

                                                    destination(d_xconsole); };

log { source(s_src); filter(f_crit); destination(d_console); };

 

# All messages send to a remote site

#

#log { source(s_src); destination(d_net); };

 

###

# Include all config files in /etc/syslog-ng/conf.d/ ### <at> include "/etc/syslog-ng/conf.d/*.conf"

 

 

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Ray Van Dolson | 12 May 05:42 2015

3.2.5 and Multiline(?) messages from Solaris

Admittedly haven't done enough searching or testing on this, but am
hoping someone might have a quick answer.

Recently moved from the 2.x verions to 3.2.5 (as part of EPEL on
RHEL6).  Have noticed that we're no longer getting the full messages
from some Solaris boxen using the tcp() and udp() source definitions.

Messages like this:

May 10 02:29:30 dev-zfs2 scsi: [ID 365881 kern.info] /pci <at> 0,0/pci8086,3410 <at> 9/pci15d9,400 <at> 0 (mpt_sas0):
May 10 02:29:30 dev-zfs2        Log info 0x31080000 received for target 24.
May 10 02:29:30 dev-zfs2        scsi_status=0x0, ioc_status=0x804b, scsi_state=0x0

Come through looking like this:

May 10 02:29:30 dev-zfs2 scsi: [ID 365881 kern.info] /pci <at> 0,0/pci8086,3410 <at> 9/pci15d9,400 <at> 0 (mpt_sas0):

(Only the initial line)

However, messages like this one:

May  9 04:12:57 dev-zfs2 scsi: [ID 243001 kern.warning] WARNING:
/pci <at> 0,0/pci8086,3410 <at> 9/pci15d9,400 <at> 0 (mpt_sas0):
May  9 04:12:57 dev-zfs2        mptsas_handle_event_sync: IOCStatus=0x8000, IOCLogInfo=0x31110610

.. do seem to be coming through "whole" (I do note that the priority
is different in both).

Relevant config items are as follows:

log {
    source(remote);
    filter(syslog);
    destination(hosts_syslog);
};

source remote {
    udp();
    tcp();
    # udp(ip(0.0.0.0) port(514));
    # tcp(ip(0.0.0.0) port(514));
};

destination hosts_syslog {
    file("/logs/hosts/$HOST/$YEAR/$MONTH/syslog.$HOST.$YEAR.$MONTH.log"
        create_dirs(yes));
    pipe("/logs/hosts/everything.fifo");
};

filter syslog {
    (not facility(mail)
    and not filter(f_ucgw)
    and not filter(f_esx));
};

Will try and do some packet captures to confirm Solaris is, in fact,
sending the entire message (I believe it is since this worked on
syslog-ng 2.x).

Thanks,
Ray
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

ZeroUno | 8 May 16:19 2015
Picon

unix-dgram for Linux syslog stream

Hi,
I've read some old discussions about unix-dgram vs unix-stream for 
getting the /dev/log syslog stream on Linux (e.g. 
https://bugs.archlinux.org/task/22153), but some years have passed by 
and I'd like to be sure my configuration is safe.

I'm using syslog-ng 3.2.5 on a system composed of different Linux 
(RHEL6) machines. I cannot install a different version.
There is a central log facility, but each machine collects and sends its 
logs using the following source config:

source src {
	file ("/proc/kmsg" program_override("kernel: "));
	unix-dgram ("/dev/log" flags(no-multi-line));
	internal();
};

It was originally using unix-stream(), but I need to change it to 
unix-dgram() because some custom applications are sending multiline 
messages which need to be converted into single line, and unix-stream() 
does not support this flag.

Can I be sure that no messages risk to be lost due to this change?
This is a rather critical application.

Thank you very much.

--

-- 
01

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Parth Oberoi | 2 May 14:48 2015
Picon

Feedback for GSoC project - RIak Destination for Syslog-ng

Hello Everyone,

My proposal[0] for adding Riak as a destination to Syslog-ng was accepted for GSoC2015, so before my coding period begins i.e. 25th May, I wanted to do some ground work and take some feedback from the potential users.
The proposal[0] contains all the initial implementation details which might require improvement .
Therefore if some of you could take a look and give me some feedback on it, would  help me greatly. 
Below are a few links to important snippets of the proposal.







Thanks in advance.

Regards,
Parth Oberoi 
( <at> htrap)

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Czanik, Péter | 30 Apr 14:11 2015

insider 2015-04: 3.7 beta1; RedHat Summit; Hadoop; GSoC; RFC5424;

Dear syslog-ng users,

This is the 41th issue of the syslog-ng Insider, a monthly newsletter
that brings you syslog-ng-related news.

NEWS

syslog-ng 3.7 beta1 released

----------------------------

In the first beta of syslog-ng OSE 3.7, the Java and Python
destinations were migrated from the syslog-ng incubator to the
syslog-ng core. Also, syslog-ng now includes an interactive
syslog-ng.conf debugger.

Read about known problems and how to get started at
https://czanik.blogs.balabit.com/2015/04/first-beta-of-syslog-ng-3-7-released-with-java-and-python-support-interactive-configuration-debugger/

syslog-ng at RedHat Summit

--------------------------

BalaBit is a general sponsor of the Red Hat Summit. The event will be
in Boston, June 23-26. http://www.redhat.com/summit/ Visit the BalaBit
booth to talk about syslog-ng and log management!

syslog-ng Hadoop support

------------------------

With the release of syslog-ng PE 5F3, support for Hadoop arrived. It
will also be part of the upcoming syslog-ng OSE 3.7 release. Read more
about how syslog-ng can send logs to a HDFS destination:

https://jluby.blogs.balabit.com/2015/04/08/sending-logs-to-hadoop-using-syslog-ng/

With this solution BalaBit became a certified MapR technology partner:
https://www.mapr.com/partners/partner/balabit-it-security

syslog-ng in Google Summer of Code 2015

---------------------------------------

This year we participate in GSoC again. The syslog-ng team received
many good proposals, it will be a tough choice to decide on the
finalists. We plan to have an IRC meeting with the participation of
the winning GSoC students and members of the syslog-ng community. A
separate e-mail will announce the exact time.

The project and idea list is available at
https://github.com/balabit/syslog-ng/wiki/GSoC2015-idea-&-project-list

Forwarding name value pairs using RFC5424

-----------------------------------------

Recently many users asked how to forward the name of a file source
over the network. It can easily be done using the RFC5424 syslog
protocol. The same method can also be used to create and forward name
value pairs between servers in general. Read more about it at
https://czanik.blogs.balabit.com/2015/03/using-rfc5424-syslog-to-forward-file-names/

syslog-ng  <at>  LOADays 2015

------------------------

This year syslog-ng was presented again at LOADays, a small but very
exciting Linux admin conference in Antwerp. The basics of syslog-ng,
the importance of name value pairs and the latest developments in
syslog-ng were presented. For details, check:

https://czanik.blogs.balabit.com/2015/04/loadays-2015-logging-elasticsearch-home-automation-and-other-sysadmin-topics/

NEW RELEASES

syslog-ng 3.7.0beta1:
https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.7.0beta1

Your feedback and news tips about the next issue is welcome at
documentation <at> balabit.com To read this newsletter on-line, visit:
http://insider.blogs.balabit.com/

Peter Czanik (CzP) <peter.czanik <at> balabit.com>
BalaBit IT Security / syslog-ng upstream
http://czanik.blogs.balabit.com/
https://twitter.com/PCzanik
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq


Gmane