headers
Mats Erik Andersson | 2 Nov 2010 23:16
Picon

An issue with unlocked-io.h and bootstrapping.

Hello,

I have made a strange observation concerning

    lib/unlocked-io.h

which appears when running the bootstrap script
for GNU Inetutils.

Until very recently, "unlocked-io" was included
as a dependency in our "bootstrap.conf". Upon my
complaint, Alfred M. Szmidt remove that entry,
after which the following issue vanished completely.
Therefore we suspect there be a bug in GNUlib on
this matter.

Now, running "bootstrap" on a fresh cheackout of
GNU Inetutils, and then happily running

   rsync -Ca . ../temp/

one would expect a source tree where configuration and
compilation works. Right? But alas, it does not.

An empty soft link

    lib/unlocked-io.h

remains, which is pointing at the non-existent
location
(Continue reading)

Permalink | Reply | 2 comments |
headers
Eric Blake | 2 Nov 2010 23:28
Picon
Favicon
Gravatar

struct timespec on mingw vs. pthreads-win32

The <pthreads.h> shipped with pthreads-win32 is horribly broken; it
pollutes the global namespace with more than just bad macros (such as
the incorrect macros for asctime_r that we are already working around in
time.in.h), but also by including a <config.h> specific to how
pthreads-win32 was configured, which may clash horribly with how the
current package is being compiled.

Rather than making our <time.h> drag in the botched pthreads-win32
header just for struct timespec, is there a way that we can instead keep
the namespace clean by providing our own struct timespec, and then take
stronger measures in our <pthread.h> replacement to work around the
broken <pthread.h> header without penalizing packages like libvirt that
intentionally avoid pthreads-win32 for other reasons?  It looks like the
pthreads-win32 will avoid declaring struct timespec if
HAVE_STRUCT_TIMESPEC is already defined; therefore, it seems like our
replacement <time.h> could define this macro, and our replacement
<pthread.h> could be sure that our replacement <time.h> is included
_prior_ to the include_next of the system <pthread.h> (for just mingw).

--

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org
Permalink | Reply | 0 comments |
headers
Jim Meyering | 3 Nov 2010 10:13
Gravatar

glibc's snprintf is a pig; fix or replace ?

glibc's snprintf is next-to-useless in an application that must accept
arbitrary inputs and/or arbitrary format strings, and that cannot afford
to allocate 2GiB of heap.

Did you know that glibc's snprintf can fail due to ENOMEM?  Your reaction
to that possibility should be disbelief.  But it's true.  Why does
snprintf allocate space from the heap for a format string like "%*d"?
I reported it as http://bugzilla.redhat.com/441945, but amazingly
enough, it was closed "NOTABUG".  The justification was that the POSIX
specification permits the current senseless behavior.  The letter of the
spec may permit it, but the spirit must not, so if indeed POSIX allows
the current behavior, the spec should be adjusted.

What's the problem?
Consider a single format directive.
If I give snprintf a buffer to print into, should it first allocate
(potentially from the heap) enough space to render the entire result,
and then copy that (or whatever fits) into the user-supplied buffer?
In many cases it's ok, albeit wasteful (unwarranted allocation and
an extra copy).  In others, it's a needless source of failure, and
a ridiculous abuse of resources.  Do you want your code to take 10-30
seconds to allocate 2GiB of heap and fill that space with 2 billion '0's
only to learn that snprintf(NULL, 0, "%0*d", INT_MAX, 0) returns INT_MAX ?
Often, when I use snprintf, it's because I want to avoid using as*printf.

More reasonably, imagine a low-memory scenario.
You want to determine whether a small static buffer
is large enough to hold the result of an snprintf call.
If not, you must take other measures, but you cannot allocate
memory from the heap.  You would use
(Continue reading)

Permalink | Reply | 8 comments |
headers
Gary V. Vaughan | 3 Nov 2010 10:41
Picon
Gravatar

Re: glibc's snprintf is a pig; fix or replace ?

Hi Jim,

On 3 Nov 2010, at 16:13, Jim Meyering wrote:
> glibc's snprintf is next-to-useless in an application that must accept
> arbitrary inputs and/or arbitrary format strings, and that cannot afford
> to allocate 2GiB of heap.

Many years ago I wrote a complete reimplementation of the printf functions,
and with some help from Bruce Korb and Paolo Bonzini it ended up in both
autogen and GNU smalltalk.

    https://savannah.nongnu.org/projects/libsnprintfv/

We already proposed cannibalizing it into an abstract string type, a
self-extending string type and then the self-contained *printf* 
implementation built atop those for inclusion in gnulib:

    http://lists.gnu.org/archive/html/bug-gnulib/2007-02/msg00221.html

I (and I believe others) would still like to see this happen, but as is
usual for such things I believe the main thing holding it back is a lack
of time from concerned parties, and overcoming the inertia to get it
started.  Your message adds another good reason to the list of why doing
this would be useful...

Cheers,
--

-- 
Gary V. Vaughan (gary <at> gnu.org)
Permalink | Reply | 1 comment |
headers
Jim Meyering | 3 Nov 2010 10:53
Gravatar

Re: An issue with unlocked-io.h and bootstrapping.

Mats Erik Andersson wrote:
> Hello,
>
> I have made a strange observation concerning
>
>     lib/unlocked-io.h
>
> which appears when running the bootstrap script
> for GNU Inetutils.
>
> Until very recently, "unlocked-io" was included
> as a dependency in our "bootstrap.conf". Upon my
> complaint, Alfred M. Szmidt remove that entry,
> after which the following issue vanished completely.
> Therefore we suspect there be a bug in GNUlib on
> this matter.
>
> Now, running "bootstrap" on a fresh cheackout of
> GNU Inetutils, and then happily running
>
>    rsync -Ca . ../temp/
>
> one would expect a source tree where configuration and
> compilation works. Right? But alas, it does not.
>
> An empty soft link
>
>     lib/unlocked-io.h
>
> remains, which is pointing at the non-existent
(Continue reading)

Permalink | Reply | 1 comment |
headers
Pádraig Brady | 3 Nov 2010 10:59

Re: glibc's snprintf is a pig; fix or replace ?

On 03/11/10 09:13, Jim Meyering wrote:
> glibc's snprintf is next-to-useless in an application that must accept
> arbitrary inputs and/or arbitrary format strings, and that cannot afford
> to allocate 2GiB of heap.
> 
> Did you know that glibc's snprintf can fail due to ENOMEM?  Your reaction
> to that possibility should be disbelief.  But it's true.  Why does
> snprintf allocate space from the heap for a format string like "%*d"?
> I reported it as http://bugzilla.redhat.com/441945, but amazingly
> enough, it was closed "NOTABUG".  The justification was that the POSIX
> specification permits the current senseless behavior.  The letter of the
> spec may permit it, but the spirit must not, so if indeed POSIX allows
> the current behavior, the spec should be adjusted.

Well heap allocation is blatantly silly/lazy/dangerous.
Has that always been the case? It's been the case for long
enough that a replacement is warranted I suppose.
Will we eventually replace all of glibc as it accretes bugs over time :(

It's a pity that bug is closed.
What's the process for appealing this?
I notice no voting feature for example.

cheers,
Pádraig.
Permalink | Reply | 2 comments |
headers
Jim Meyering | 3 Nov 2010 11:12
Gravatar

Re: glibc's snprintf is a pig; fix or replace ?

Pádraig Brady wrote:
> On 03/11/10 09:13, Jim Meyering wrote:
>> glibc's snprintf is next-to-useless in an application that must accept
>> arbitrary inputs and/or arbitrary format strings, and that cannot afford
>> to allocate 2GiB of heap.
>>
>> Did you know that glibc's snprintf can fail due to ENOMEM?  Your reaction
>> to that possibility should be disbelief.  But it's true.  Why does
>> snprintf allocate space from the heap for a format string like "%*d"?
>> I reported it as http://bugzilla.redhat.com/441945, but amazingly
>> enough, it was closed "NOTABUG".  The justification was that the POSIX
>> specification permits the current senseless behavior.  The letter of the
>> spec may permit it, but the spirit must not, so if indeed POSIX allows
>> the current behavior, the spec should be adjusted.
>
> Well heap allocation is blatantly silly/lazy/dangerous.
> Has that always been the case?

Yes.

> It's been the case for long
> enough that a replacement is warranted I suppose.
> Will we eventually replace all of glibc as it accretes bugs over time :(

It would be a real shame/waste to have to replace glibc's snprintf forever,
but I suspect that if someone writes a good patch, they'll apply it.

> It's a pity that bug is closed.
> What's the process for appealing this?
> I notice no voting feature for example.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Jim Meyering | 3 Nov 2010 11:23
Gravatar

Re: glibc's snprintf is a pig; fix or replace ?

Pádraig Brady wrote:

> On 03/11/10 09:13, Jim Meyering wrote:
>> glibc's snprintf is next-to-useless in an application that must accept
>> arbitrary inputs and/or arbitrary format strings, and that cannot afford
>> to allocate 2GiB of heap.
>>
>> Did you know that glibc's snprintf can fail due to ENOMEM?  Your reaction
>> to that possibility should be disbelief.  But it's true.  Why does
>> snprintf allocate space from the heap for a format string like "%*d"?
>> I reported it as http://bugzilla.redhat.com/441945, but amazingly
>> enough, it was closed "NOTABUG".  The justification was that the POSIX
>> specification permits the current senseless behavior.  The letter of the
>> spec may permit it, but the spirit must not, so if indeed POSIX allows
>> the current behavior, the spec should be adjusted.
>
> Well heap allocation is blatantly silly/lazy/dangerous.

Actually it's not quite as bad as you might think.
It tries to allocate first from the stack, and if that fails,
it tries to allocate from the heap.  So both heap and stack
limits have to be taken into account.

But where/how it allocates is moot.  The point is that snprintf need not
allocate *at all* for all but a tiny fraction of format-string/value
combinations.
Permalink | Reply | 0 comments |
headers
Mats Erik Andersson | 3 Nov 2010 11:18
Picon

Re: An issue with unlocked-io.h and bootstrapping.

onsdag den  3 november 2010 klockan 10:53 skrev Jim Meyering detta:
> Mats Erik Andersson wrote:
> > Hello,
> >
> > I have made a strange observation concerning
> >
> >     lib/unlocked-io.h
> >
> > which appears when running the bootstrap script
> > for GNU Inetutils.
> >

> [...]

> > remains, which is pointing at the non-existent
> > location
> >
> >     ../gnulib/unlocked-io.h
> 
> Thanks for the report.
> 
> There is already code in gnulib's bootstrap script designed to
> remove dangling symlinks from m4/ and lib/-style directories:
> 
>   # Remove any dangling symlink matching "*.m4" or "*.[ch]" in some
>   # gnulib-populated directories.  Such .m4 files would cause aclocal to fail.
>   # The following requires GNU find 4.2.3 or newer.  Considering the usual
>   # portability constraints of this script, that may seem a very demanding
>   # requirement, but it should be ok.  Ignore any failure, which is fine,
>   # since this is only a convenience to help developers avoid the relatively
(Continue reading)

Permalink | Reply | 0 comments |
headers
Eric Blake | 3 Nov 2010 18:25
Picon
Favicon
Gravatar

Re: bug#7324: coreutils on Solaris 10

[adding bug-gnulib]

On 11/03/2010 10:51 AM, Eric Blake wrote:
>> The problem appears to be happening when trying to set the times on
>> the new file.
>>
>> I suspect it's an updated patch which Oracle has provided, which has
>> broken this.
> 
> Thanks for the report.  Have you tried with the latest snapshot?
> http://lists.gnu.org/archive/html/coreutils/2010-11/msg00004.html
> since it includes some recent gnulib fixes for at least some Solaris 10
> bugs?
> 
> Also, can you provide a truss report of a failing command, so we can see
> exactly how Oracle has broken futimens/utimensat, assuming that you are
> correct that it is a bug in setting file times?

I've confirmed the failure on:

$ uname -a
SunOS xxx 5.10 Generic_142900-06 sun4u sparc SUNW,Sun-Fire-V440 Solaris

via gnulib's test-fdutimensat:

../../gltests/test-fdutimensat.c:56: assertion failed
/bin/bash: line 5: 25716 Abort                   EXEEXT=''
srcdir='../../gltests' MAKE='make' ${dir}$tst
FAIL: test-fdutimensat
skipping test: setting fd time not supported on this file system
(Continue reading)

Permalink | Reply | 7 comments |

Gmane