Jason S. Whitwill | 1 Feb 16:21 2011

PropertyConfigurator::configure file encoding

I'm passing a file to PropertyConfigurator::configure to initialize my log4cxx logging configuration.
Inside the configuration file I have a rolling file appender with the file property set to ${APPDATA}/example.log.

For example:
log4j.rootLogger=info, R
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=$(APPDATA)/example.log
log4j.appender.R.MaxFileSize=100KB
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%r %p %t %c - %m%n

If the log configuration file is encoded in ansii, everything works fine as long as the APPDATA path doesn't
contain any characters outside of the normal ascii range. However, if the path contains Chinese
characters for example, I see no output file. I attempted to save the log configuration file with Unicode
encoding but unfortunately, no matter what the contents of the file, log4cxx would fail to read it. I am
passing the path to the configuration file as a wchar_t*. I'm presuming that the rolling file appender
supports outputting to a path containing international characters as the constructor accepts a
LogString parameter.

Is there a particular encoding that would make it such that international characters could be interpreted
correctly when parsing the configuration file using PropertyConfigurator::configure?

Thank you for your time.

Jason
_______________________
Jason Whitwill
Software Designer

(Continue reading)

Curt Arnold | 3 Feb 06:32 2011
Picon

Re: PropertyConfigurator::configure file encoding

Property files in Java are by definition in ISO-8859-1 which cannot support Chinese characters without
using escape characters (see
http://download.oracle.com/javase/6/docs/api/java/util/Properties.html).  log4cxx follows
this convention so that it is compatible with log4j configuration files.

However, the issue is the substitution of the contents of the APPDATA environment variable into the
evaluation of the configuration which should occur after the properties file in parsed and should happen
in LogString (aka Unicode) space.

I'm guessing things are failing since the evaluation of APPDATA does not match an existing directory and
therefore the appender fails.  It would be interesting to experiment with an environment variable for the
file name (not the path) to see how the name is mangled.

There are a couple of things that would be very useful to know:

What operating system and version is being used?
What is the default character encoding (control panel or $ locale charmap)?
What settings are used to build log4cxx?
What is the observed behavior when using environment variables for the filename (not the path)?  What were
the expected behavior?

I'm pretty confident that the property files are correctly always interpreted as ISO-8859-1 regardless
of the default encoding.

log4cxx depends on APR to get the environment variables and for file IO, so something unexpected could be
happening there or log4cxx could be mangling the substitution.

Jason S. Whitwill | 3 Feb 18:24 2011

RE: PropertyConfigurator::configure file encoding

Thank you very much for your reply.

The operating system is Windows 7 64 bits English version.
Under region and language in the control panel, everything is set to English (United States)
The user name has Chinese characters which are interpreted correctly in Windows Explorer and in my Unicode
MFC application.
Log4cxx was built with Visual C++ 2005 and I don't recall changing anything in the default configuration
when it was built (it has been a while).
The log4cxx version is 0.10.0
When using environment variables for the filename (not the path) I get a file output but the characters of
the file name are garbled (from some other language).

If I execute the following code snippet, it interprets the environment variable correctly and the file is
output with the correct path and file name.

    wchar_t lPathPtr[2048];
        size_t lPathSize = 0;
        _wgetenv_s( &lPathSize, lPathPtr, 2048, L"APPDATA" );
    if ( lPathPtr != NULL )
    {
        wchar_t lPath[2048];
        StringCbPrintfW(lPath, 2048, L"%s\\unicode_test.txt", lPathPtr);

        log4cxx::LayoutPtr layout(new log4cxx::SimpleLayout());
        log4cxx::FileAppenderPtr appender(new log4cxx::FileAppender(layout, lPath, true));

        LoggerPtr logger(Logger::getLogger("MyApp"));
        logger->addAppender( appender );

        wchar_t lString[512];
(Continue reading)

Curt Arnold | 7 Feb 06:21 2011
Picon

Re: PropertyConfigurator::configure file encoding

I'd suggest looking at System::GetProperty in system.cpp, specifically the last section that calls
apr_get_env.  It appears that the problem is specific to the introduction of environment variables.  I'd
suggest a native Windows alternative that calls GetEnvironmentVariableW which bypasses the
conversion to and from an single byte representation.  Alternatively, you could look at the
implementation of apr_get_env and try to determine whether log4cxx called it incorrectly or whether it
has a bug or limitation.

I would appreciate it if you could file a bug report at http://issues.apache.org/jira.

On Feb 3, 2011, at 11:24 AM, Jason S. Whitwill wrote:

> Thank you very much for your reply.
> 
> The operating system is Windows 7 64 bits English version.
> Under region and language in the control panel, everything is set to English (United States)
> The user name has Chinese characters which are interpreted correctly in Windows Explorer and in my
Unicode MFC application.
> Log4cxx was built with Visual C++ 2005 and I don't recall changing anything in the default configuration
when it was built (it has been a while).
> The log4cxx version is 0.10.0
> When using environment variables for the filename (not the path) I get a file output but the characters of
the file name are garbled (from some other language).
> 
> If I execute the following code snippet, it interprets the environment variable correctly and the file is
output with the correct path and file name.
> 
>    wchar_t lPathPtr[2048];
>        size_t lPathSize = 0;
>        _wgetenv_s( &lPathSize, lPathPtr, 2048, L"APPDATA" );
>    if ( lPathPtr != NULL )
(Continue reading)

Jason S. Whitwill | 7 Feb 14:43 2011

RE: PropertyConfigurator::configure file encoding

I managed to work around the issue by parsing the file outside of log4cxx, substituting the environment
variables and creating a log4cxx::helpers::Properties object to pass to that overload of PropertyConfigurator::configure.

I created bug https://issues.apache.org/jira/browse/LOGCXX-378

I'm OK for now (I have an acceptable workaround) so prioritize the bug however you like.

Thanks again for your time.

Cheers,
Jason

-----Original Message-----
From: Curt Arnold [mailto:curt.arnld <at> gmail.com] On Behalf Of Curt Arnold
Sent: Monday, February 07, 2011 12:22 AM
To: Log4CXX User
Subject: Re: PropertyConfigurator::configure file encoding

I'd suggest looking at System::GetProperty in system.cpp, specifically the last section that calls
apr_get_env.  It appears that the problem is specific to the introduction of environment variables.  I'd
suggest a native Windows alternative that calls GetEnvironmentVariableW which bypasses the
conversion to and from an single byte representation.  Alternatively, you could look at the
implementation of apr_get_env and try to determine whether log4cxx called it incorrectly or whether it
has a bug or limitation.

I would appreciate it if you could file a bug report at http://issues.apache.org/jira.

On Feb 3, 2011, at 11:24 AM, Jason S. Whitwill wrote:

> Thank you very much for your reply.
(Continue reading)

secvet | 16 Feb 09:42 2011
Picon

SyslogAppender not working


Hi,

I'm trying to use log4cxx to log messages to a remote syslog server.
To begin with, I tried to log the message to the local syslog server for a
simple testing.

I made the following configuration in my log4cxx configuration, but nothing
had been logged to /var/log/message.

I tried DAEMON, SYSLOG, LOCAL1, LOCAL7 for the
log4j.appender.Syslog.Facility but none of them was working.

Did I miss something?
Any suggestions are greatly appreciated.

Thank you very much.

####################
log4j.rootLogger=INFO, Console, Syslog

log4j.appender.Console=org.apache.log4j.ConsoleAppender
log4j.appender.Console.layout=org.apache.log4j.PatternLayout
log4j.appender.Console.layout.ConversionPattern=%5p [%d] [%t](%F:%L)- %m%n

log4j.appender.Syslog=org.apache.log4j.net.SyslogAppender
log4j.appender.Syslog.syslogHost=localhost
log4j.appender.Syslog.Facility=DAEMON
log4j.appender.Syslog.layout=org.apache.log4j.PatternLayout
log4j.appender.Syslog.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss:SSS}
(Continue reading)

security veteran | 16 Feb 10:09 2011
Picon

SyslogAppender not working

Hi,

I'm trying to use log4cxx to log messages to a remote syslog server.
To begin with, I tried to log the message to the local syslog server for a simple testing.

I made the following configuration in my log4cxx configuration, but nothing had been logged to /var/log/message.

I tried DAEMON, SYSLOG, LOCAL1, LOCAL7 for the log4j.appender.Syslog.Facility but none of them was working.

Did I miss something?
Any suggestions are greatly appreciated.

Thank you very much.

####################
log4j.rootLogger=INFO, Console, Syslog

log4j.appender.Console=org.apache.log4j.ConsoleAppender
log4j.appender.Console.layout=org.apache.log4j.PatternLayout
log4j.appender.Console.layout.ConversionPattern=%5p [%d] [%t](%F:%L)- %m%n


log4j.appender.Syslog=org.apache.log4j.net.SyslogAppender
log4j.appender.Syslog.syslogHost=localhost
log4j.appender.Syslog.Facility=DAEMON
log4j.appender.Syslog.layout=org.apache.log4j.PatternLayout
log4j.appender.Syslog.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss:SSS} %p [%c] %m%n

Jacob L. Anawalt | 16 Feb 18:40 2011

Re: SyslogAppender not working

On 2/16/2011 2:09 AM, security veteran wrote:
> Hi,
>
> I'm trying to use log4cxx to log messages to a remote syslog server.
> To begin with, I tried to log the message to the local syslog server
> for a simple testing.
>

I use a SyslogAppender, but my configuration is different so I can't 
do a quick apples to apples comparison.

Verify that you can log locally and remotely using the logger command 
(assuming some kind of UNIX like environment) or similar.

http://unixhelp.ed.ac.uk/CGI/man-cgi?logger+1

Try a simple log4cxx program that configures it all programatically. 
The following works for me:

// g++ -o syslogAppender syslogAppender.cpp -llog4cxx
//
#include <log4cxx/logger.h>
#include <log4cxx/basicconfigurator.h>
#include <log4cxx/patternlayout.h>
#include <log4cxx/net/syslogappender.h>

int
main(void)
{
         log4cxx::BasicConfigurator::configure();

         log4cxx::LayoutPtr l(new log4cxx::PatternLayout(
                 "%-4r %-5p %c %x - %m"));
         log4cxx::net::SyslogAppenderPtr a(
                 new log4cxx::net::SyslogAppender(l
                 ,log4cxx::net::SyslogAppender::getFacility("DAEMON")));
         a->setSyslogHost("127.0.0.1");
         a->activateOptions();

         log4cxx::LoggerPtr logger(log4cxx::Logger::getLogger("MyApp"));
         logger->addAppender(a);

         while(true) {
                 LOG4CXX_INFO(logger,"beep");
                 sleep(15);
         }

         return EXIT_SUCCESS;
}

--

-- 
Jacob Anawalt
Gecko Software, Inc.
janawalt <at> geckosoftware.com
435-752-8026

Rhys Ulerich | 27 Feb 20:25 2011
Picon

Static destruction fiasco?

Hi,

I've noticed that this little test program segfaults beautifully under 0.10

   #include <log4cxx/logger.h>

   log4cxx::LoggerPtr logger;

   int main()
   {
       logger = log4cxx::Logger::getLogger("someName");
       LOG4CXX_WARN(logger, "Stuffstuffstuff");
       // logger = 0;
      return 0;
   }

but if I uncomment the "logger = 0" statement it runs just fine.  Presumably
that's because "logger = 0" forces the ObjectPtrT instance behind LoggerPtr to
destruct logger before other static destruction occurs.

That said, which of
   1) Bug
   2) Don't do that
does that segfault fall into?

Thanks,
Rhys

Rhys Ulerich | 27 Feb 21:04 2011
Picon

Re: Static destruction fiasco?

> I've noticed that this little test program segfaults beautifully under 0.10

The "logger = 0" fix from my last email of course assumes regular program exit.

In case anyone wants something a little cleaner, atexit() can save you when
your program unwinds for other reasons:

    #include <cstdlib>
    #include <log4cxx/logger.h>

    static log4cxx::LoggerPtr logger;

    static struct DestructLoggerRegistration {
        DestructLoggerRegistration() { atexit(&destruct_logger); }
        static void destruct_logger() { logger = 0; }
    } destructLoggerRegistration;

    int main()
    {
        logger = log4cxx::Logger::getLogger("someName");
        LOG4CXX_WARN(logger, "Stuffstuffstuff");
        return 0;
    }

where (at least for me) atexit() seems to make its callbacks prior to
static instance destruction occurs.

- Rhys


Gmane