headers
Santiago Vila | 3 Sep 2006 14:55
Picon
Favicon

Bug#385720: m4: INTERNAL ERROR: recursive push_string (fwd)

Hello.

I received this report from the Debian bug system:

Andreas Schultz <aschultz <at> warp10.net> writes:

  m4 crashes with "m4: INTERNAL ERROR: recursive push_string"

Later, he adds:

  It seems that the problem is reproducible with every automake project
  that has a) AC_OUTPUT on the last line in configure.ac and b) no
  terminating newline on that line. Adding a newline will make the
  problem disappear.

and in fact, I can reproduce it using GNU hello 2.1.1 by removing the
last two lines of configure.ac, removing also the newline from the
AC_OUTPUT final line, and then trying "./configure; make configure".

This is with m4 1.4.6, and it did not happen with m4 1.4.4.

Thanks.
Permalink | Reply | 15 comments |
headers
Eric Blake | 4 Sep 2006 00:03
Gravatar

Re: Bug#385720: m4: INTERNAL ERROR: recursive push_string (fwd)


[adding autoconf to cc]

According to Santiago Vila on 9/3/2006 6:55 AM:
> Andreas Schultz <aschultz <at> warp10.net> writes:
> 
>   m4 crashes with "m4: INTERNAL ERROR: recursive push_string"
> 
> Later, he adds:
> 
>   It seems that the problem is reproducible with every automake project
>   that has a) AC_OUTPUT on the last line in configure.ac and b) no
>   terminating newline on that line. Adding a newline will make the
>   problem disappear.
> 
> and in fact, I can reproduce it using GNU hello 2.1.1 by removing the
> last two lines of configure.ac, removing also the newline from the
> AC_OUTPUT final line, and then trying "./configure; make configure".
> 
> This is with m4 1.4.6, and it did not happen with m4 1.4.4.

Thanks for a useful bug report.  I will spend time narrowing down what is
causing this regression, and hope to get m4 1.4.7 out within a month,
after I see if any other bugs are reported.  But be advised that POSIX
only requires m4 to process text files, and a file that does not end in a
newline is not a text file (unless it is empty), so the bug is only
triggered when you are violating POSIX.  On the other hand, as a quality
of implementation issue, I totally agree that this is not good practice
for GNU software.  In the meantime, perhaps Autoconf should document that
all autom4te input files should always end in newline.
(Continue reading)

Permalink | Reply | 9 comments |
headers
Paul Eggert | 4 Sep 2006 10:16
Favicon

Re: Bug#385720: m4: INTERNAL ERROR: recursive push_string (fwd)

Eric Blake <ebb9 <at> byu.net> writes:

> In the meantime, perhaps Autoconf should document that
> all autom4te input files should always end in newline.

Naaaah.  Let's just fix the bug in M4.  It's clearly a bug.
The GNU tradition is to handle arbitrary input files,
and not to insist on "text" files in the POSIX sense.
Permalink | Reply | 5 comments |
headers
Eric Blake | 4 Sep 2006 15:24
Gravatar

Re: Bug#385720: m4: INTERNAL ERROR: recursive push_string (fwd)


According to Paul Eggert on 9/4/2006 2:16 AM:
>> In the meantime, perhaps Autoconf should document that
>> all autom4te input files should always end in newline.
> 
> Naaaah.  Let's just fix the bug in M4.  It's clearly a bug.
> The GNU tradition is to handle arbitrary input files,
> and not to insist on "text" files in the POSIX sense.
> 

OK, I found the culprit; the regression crept in on Aug 1 (post 1.4.5).
My fix to debian bug 175365 (remembering the current file and line, rather
than printing NONE:0: when diagnosing incomplete input) was not expecting
the last token in a file to be a macro expansion with no arguments.  In
addition to the workaround of adding the newline, you can also fix things
by calling AC_OUTPUT with arguments (fortunately, AC_OUTPUT will ignore
those arguments, so you aren't changing the syntax of your configure.ac),
or introducing any other non-macro token (such as an empty string [],
extra whitespace, etc.).  Here's the patch I'm applying, if debian wants
to release 1.4.6-2 instead of waiting for 1.4.7.  And wow is it hard
adding files to the testsuite that don't end in newline, so instead I used
changequote to allow newline in a macro name to trigger a similar failure
in the 'make distcheck' testsuite, to ensure we don't regress again.

2006-09-04  Eric Blake  <ebb9 <at> byu.net>

	* src/input.c (peek_input): Fix regression in handling macro
	without arguments as last token in file; debian bug 385720.
	(next_token): Always consume an input character.
	Reported by Andreas Schultz.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Santiago Vila | 4 Sep 2006 18:05
Picon
Favicon

Re: Bug#385720: m4: INTERNAL ERROR: recursive push_string (fwd)

On Mon, 4 Sep 2006, Eric Blake wrote:

> [...] Here's the patch I'm applying, if debian wants to release 1.4.6-2
> instead of waiting for 1.4.7. [...]

Yes, I have just uploaded 1.4.6-2 fixing src/input.c.
Thanks a lot!
Permalink | Reply | 0 comments |
headers
Ralf Wildenhues | 7 Sep 2006 14:42
Picon
Picon

HEAD: defn test fails

CVS M4 fails test 60 for me on GNU/Linux x86_64:
What else do you need to know?
(Or maybe this is a known failure and I just didn't see it?)

Cheers, please Cc: me on replies,
Ralf

#                             -*- compilation -*-
60. generated.at:514: testing ...
../../m4/tests/../m4/doc/m4.texinfo:1721:
../../m4/tests/generated.at:528: m4 -b -d  input.m4 < /dev/null
../../m4/tests/../m4/doc/m4.texinfo:1738:
../../m4/tests/generated.at:541: m4 -b -d  input.m4 < /dev/null
../../m4/tests/../m4/doc/m4.texinfo:1750:
../../m4/tests/generated.at:556: m4 -b -d  input.m4 < /dev/null
../../m4/tests/../m4/doc/m4.texinfo:1768:
../../m4/tests/generated.at:575: m4 -b -d  input.m4 < /dev/null
../../m4/tests/../m4/doc/m4.texinfo:1787:
../../m4/tests/generated.at:590: m4 -b -d  input.m4 < /dev/null
../../m4/tests/../m4/doc/m4.texinfo:1802:
../../m4/tests/generated.at:611: m4 -b -d  input.m4 < /dev/null
--- -	2006-09-07 14:30:58.066470000 +0200
+++ /tmp/m4/build/tests/testsuite.dir/at-stdout	2006-09-07 14:30:58.000000000 +0200
 <at>  <at>  -2,6 +2,6  <at>  <at> 

 

-a0
-0a
+0
(Continue reading)

Permalink | Reply | 1 comment |
headers
Eric Blake | 7 Sep 2006 15:00
Gravatar

Re: HEAD: defn test fails


According to Ralf Wildenhues on 9/7/2006 6:42 AM:
> CVS M4 fails test 60 for me on GNU/Linux x86_64:
> What else do you need to know?
> (Or maybe this is a known failure and I just didn't see it?)

Known failure, and I'm already in the middle of a big patch trying to fix
it.  The problem is that we don't (yet) properly concatenate builtins and
text into a single token.  But the fix is pretty invasive - it has to
touch the input engine, the macro parser, the symbol table, and the defn,
dumpdef, define, and pushdef builtins.

--
Life is short - so eat dessert first!

Eric Blake             ebb9 <at> byu.net
Permalink | Reply | 0 comments |
headers
Mikhail Teterin | 20 Sep 2006 15:06
X-Face

Updating FreeBSD port

Hello!

In preparing an update of the FreeBSD port of m4 (currently at the 1.4.4) to 
1.4.6, I noticed the following minor nits:

	. Although FreeBSD's getopt.h is detected as sufficiently capable
	  by configure, the getopt.c and getopt1.c are still compiled and
	  linked into m4, instead of relying on the libc's versions

	. FreeBSD's -lgnuregex is ignored, requiring an explicit
	  `--without-included-regex' and minor patching to src/Makefile.in
	  to add -lgnuregex to LIBS

	. mkstemp_safer appears to be redundant -- how can mkstemp possibly
	  return the STDIN_FILENO?

	. Although *snprintf code is compiled and linked into libm4, none of
	  these functions make it into the m4 executable -- are the files
	  obsolete?

	. The comment in strcase.m4 states, that no existing system provides
	  multi-byte correct strcasecmp -- is there a test one can run to check
	  a particular system? If I were to patch m4 to use the libc's strcasecmp,
	  would running `gmake check' detect a problem, if there were one?

Any comment? Thank you very much!

	-mi
Permalink | Reply | 42 comments |
headers
Eric Blake | 20 Sep 2006 15:28
Gravatar

Re: Updating FreeBSD port


Hi,

According to Mikhail Teterin on 9/20/2006 7:06 AM:
> Hello!
> 
> In preparing an update of the FreeBSD port of m4 (currently at the 1.4.4) to 
> 1.4.6, I noticed the following minor nits:
> 
> 	. Although FreeBSD's getopt.h is detected as sufficiently capable
> 	  by configure, the getopt.c and getopt1.c are still compiled and
> 	  linked into m4, instead of relying on the libc's versions

No, the FreeBSD's getopt is NOT sufficiently capable.  It has subtly
different semantics than GNU which break the option parsing done by m4.
Cygwin is another platform that has the BSD-flavored getopt_long but must
use gnulib's getopt for the testsuite to pass.  Read the comments in
getopt.m4.

> 
> 	. FreeBSD's -lgnuregex is ignored, requiring an explicit
> 	  `--without-included-regex' and minor patching to src/Makefile.in
> 	  to add -lgnuregex to LIBS

That would also be a gnulib issue; but there I am not sure of the details
of -lgnuregex to know if it is up-to-date enough (even glibc's regex
functions have bugs that only gnulib has fixed).

> 
> 	. mkstemp_safer appears to be redundant -- how can mkstemp possibly
(Continue reading)

Permalink | Reply | 39 comments |
headers
Mikhail Teterin | 20 Sep 2006 16:59
X-Face

Re: Updating FreeBSD port

On Wednesday 20 September 2006 09:28, Eric Blake wrote:
= > 	. Although FreeBSD's getopt.h is detected as sufficiently capable
= > 	  by configure, the getopt.c and getopt1.c are still compiled and
= > 	  linked into m4, instead of relying on the libc's versions
= 
= No, the FreeBSD's getopt is NOT sufficiently capable.  It has subtly
= different semantics than GNU which break the option parsing done by m4.
= Cygwin is another platform that has the BSD-flavored getopt_long but must
= use gnulib's getopt for the testsuite to pass.  Read the comments in
= getopt.m4.

I see -- would you have a test-case, that detects this difference? Getopt_long 
was introduced into *BSD for the sake of compatibility with GNU software -- 
any incompatibilities in semantics are a bug, which should be fixed. It can 
not be fixed, however, if it is not reported...

= > 	. FreeBSD's -lgnuregex is ignored, requiring an explicit
= > 	  `--without-included-regex' and minor patching to src/Makefile.in
= > 	  to add -lgnuregex to LIBS
= 
= That would also be a gnulib issue; but there I am not sure of the details
= of -lgnuregex to know if it is up-to-date enough (even glibc's regex
= functions have bugs that only gnulib has fixed).

Since configure tries to auto-detect a regex library, this point was not about 
compatibility of FreeBSD's own, but of m4's not knowing to check for the 
different name (gnuregex) :-)

That said, the version we currently use is obtained by:
(Continue reading)

Permalink | Reply | 37 comments |

Gmane