Schwarz, Konrad | 6 Aug 09:54 2007
Picon

defn() with multiple arguments

Hi,

I am not sure if a future version of GNU m4 intends to become compliant with POSIX in the matter of defn() with multiple arguments.

However, I think the following example motivates this feature, should you in fact have need of motivation.

In my application, I build up program text in macros; more exactly, I build up nested `foreach()' loops when expanding macros.  Each input line consists of a single macro invocation, each such line adds a level of nesting to the resulting program text, which is expanded when a section of such lines has been read.  The program text is held in certain macros, such as LOOP_BEGIN, LOOP_END, LOOP_BODY, etc., which are concatenated at the end of a section.  (As an aside, these macros typically have an unbalanced set of quotes and require local changes to the quote characters to be processed correctly.)

Anyhow, my code for concatenating/rereading/executing the resultant program text currently looks like this:

define(`ALL', changequote([,])defn(LOOP_BEGIN)[]defn(
                LOOP_BODY)[]defn(LOOP_END)changequote(`,'))ALL
The local change of quotes is required because the different loop parts have unbalanced quotes.  My hope would be that with a multiple-argument defn(), this could be simplified to:
define(`ALL', defn(`LOOP_BEGIN', `LOOP_BODY', `LOOP_END'))ALL
In other words, that defn() be implemented to concatenate its arguments first, then to quote the resulting string as a whole.

Regards,
 
Konrad Schwarz
Attachment (Konrad Schwarz.vcf): text/x-vcard, 880 bytes
_______________________________________________
Bug-m4 mailing list
Bug-m4 <at> gnu.org
http://lists.gnu.org/mailman/listinfo/bug-m4
Eric Blake | 6 Aug 14:19 2007
Picon

Re: defn() with multiple arguments


According to Schwarz, Konrad on 8/6/2007 1:54 AM:
> Hi,
> 
> I am not sure if a future version of GNU m4 intends to become compliant
> with POSIX in the matter of defn() with multiple arguments.

Yes, we are aware that m4 is not compliant, and yes, we intend to make m4
2.0 compliant.  CVS head already supports multiple non-builtin arguments;
however, correctly supporting the mix of builtin and non-builtin arguments
is a bit more difficult.  It is on my TODO list, but seems rather far off
before I will have time to code it.

--
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9 <at> byu.net
STEVE555 | 7 Aug 23:35 2007
Picon

Compiling M4-1.4.10


Hi to all,
           I've just tried compiling the latest M4.I have applied the latest
patch :xstrtol-upstream-change.
However,I went to the M4 folder,opened Konsole from there,ran ./configure
then make.

I got this error message when make got into the source directory:

make[1]: Entering directory `/home/steve/Desktop/Software/m4-1.4.10/src'
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT m4.o -MD -MP -MF
.deps/m4.Tpo -c -o m4.o m4.c
mv -f .deps/m4.Tpo .deps/m4.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT builtin.o -MD -MP -MF
.deps/builtin.Tpo -c -o builtin.o builtin.c
mv -f .deps/builtin.Tpo .deps/builtin.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT debug.o -MD -MP -MF
.deps/debug.Tpo -c -o debug.o debug.c
mv -f .deps/debug.Tpo .deps/debug.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT eval.o -MD -MP -MF
.deps/eval.Tpo -c -o eval.o eval.c
mv -f .deps/eval.Tpo .deps/eval.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT format.o -MD -MP -MF
.deps/format.Tpo -c -o format.o format.c
mv -f .deps/format.Tpo .deps/format.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT freeze.o -MD -MP -MF
.deps/freeze.Tpo -c -o freeze.o freeze.c
mv -f .deps/freeze.Tpo .deps/freeze.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT input.o -MD -MP -MF
.deps/input.Tpo -c -o input.o input.c
mv -f .deps/input.Tpo .deps/input.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT macro.o -MD -MP -MF
.deps/macro.Tpo -c -o macro.o macro.c
mv -f .deps/macro.Tpo .deps/macro.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT output.o -MD -MP -MF
.deps/output.Tpo -c -o output.o output.c
mv -f .deps/output.Tpo .deps/output.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT path.o -MD -MP -MF
.deps/path.Tpo -c -o path.o path.c
mv -f .deps/path.Tpo .deps/path.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT symtab.o -MD -MP -MF
.deps/symtab.Tpo -c -o symtab.o symtab.c
mv -f .deps/symtab.Tpo .deps/symtab.Po
gcc -std=gnu99    -I../lib -I../lib   -g -O2 -MT stackovf.o -MD -MP -MF
.deps/stackovf.Tpo -c -o stackovf.o stackovf.c
mv -f .deps/stackovf.Tpo .deps/stackovf.Po
gcc -std=gnu99  -g -O2   -o m4 m4.o builtin.o debug.o eval.o format.o
freeze.o input.o macro.o output.o path.o symtab.o stackovf.o ../lib/libm4.a
../lib/libm4.a(regex.o): In function `wcstouq':
/usr/include/wchar.h:585: multiple definition of `wcstouq'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:585: first defined here
../lib/libm4.a(regex.o): In function `wcstoq':
/usr/include/wchar.h:580: multiple definition of `wcstoq'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:580: first defined here
../lib/libm4.a(regex.o): In function `wcstold':
/usr/include/wchar.h:574: multiple definition of `wcstold'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:574: first defined here
../lib/libm4.a(regex.o): In function `wcstof':
/usr/include/wchar.h:569: multiple definition of `wcstof'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:569: first defined here
../lib/libm4.a(regex.o): In function `wcstoul':
/usr/include/wchar.h:562: multiple definition of `wcstoul'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:562: first defined here
../lib/libm4.a(regex.o): In function `wcstol':
/usr/include/wchar.h:558: multiple definition of `wcstol'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:558: first defined here
../lib/libm4.a(regex.o): In function `wcstod':
/usr/include/wchar.h:554: multiple definition of `wcstod'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:554: first defined here
../lib/libm4.a(vasnprintf.o): In function `wcstouq':
/usr/include/wchar.h:585: multiple definition of `wcstouq'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:585: first defined here
../lib/libm4.a(vasnprintf.o): In function `wcstoq':
/usr/include/wchar.h:580: multiple definition of `wcstoq'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:580: first defined here
../lib/libm4.a(vasnprintf.o): In function `wcstold':
/usr/include/wchar.h:574: multiple definition of `wcstold'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:574: first defined here
../lib/libm4.a(vasnprintf.o): In function `wcstof':
/usr/include/wchar.h:569: multiple definition of `wcstof'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:569: first defined here
../lib/libm4.a(vasnprintf.o): In function `wcstoul':
/usr/include/wchar.h:562: multiple definition of `wcstoul'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:562: first defined here
../lib/libm4.a(vasnprintf.o): In function `wcstol':
/usr/include/wchar.h:558: multiple definition of `wcstol'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:558: first defined here
../lib/libm4.a(vasnprintf.o): In function `wcstod':
/usr/include/wchar.h:554: multiple definition of `wcstod'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:554: first defined here
../lib/libm4.a(printf-args.o): In function `wcstouq':
/usr/include/wchar.h:585: multiple definition of `wcstouq'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:585: first defined here
../lib/libm4.a(printf-args.o): In function `wcstoq':
/usr/include/wchar.h:580: multiple definition of `wcstoq'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:580: first defined here
../lib/libm4.a(printf-args.o): In function `wcstold':
/usr/include/wchar.h:574: multiple definition of `wcstold'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:574: first defined here
../lib/libm4.a(printf-args.o): In function `wcstof':
/usr/include/wchar.h:569: multiple definition of `wcstof'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:569: first defined here
../lib/libm4.a(printf-args.o): In function `wcstoul':
/usr/include/wchar.h:562: multiple definition of `wcstoul'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:562: first defined here
../lib/libm4.a(printf-args.o): In function `wcstol':
/usr/include/wchar.h:558: multiple definition of `wcstol'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:558: first defined here
../lib/libm4.a(printf-args.o): In function `wcstod':
/usr/include/wchar.h:554: multiple definition of `wcstod'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:554: first defined here
../lib/libm4.a(printf-parse.o): In function `wcstouq':
/usr/include/wchar.h:585: multiple definition of `wcstouq'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:585: first defined here
../lib/libm4.a(printf-parse.o): In function `wcstoq':
/usr/include/wchar.h:580: multiple definition of `wcstoq'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:580: first defined here
../lib/libm4.a(printf-parse.o): In function `wcstold':
/usr/include/wchar.h:574: multiple definition of `wcstold'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:574: first defined here
../lib/libm4.a(printf-parse.o): In function `wcstof':
/usr/include/wchar.h:569: multiple definition of `wcstof'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:569: first defined here
../lib/libm4.a(printf-parse.o): In function `wcstoul':
/usr/include/wchar.h:562: multiple definition of `wcstoul'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:562: first defined here
../lib/libm4.a(printf-parse.o): In function `wcstol':
/usr/include/wchar.h:558: multiple definition of `wcstol'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:558: first defined here
../lib/libm4.a(printf-parse.o): In function `wcstod':
/usr/include/wchar.h:554: multiple definition of `wcstod'
../lib/libm4.a(quotearg.o):/usr/include/wchar.h:554: first defined here
collect2: ld returned 1 exit status
make[1]: *** [m4] Error 1
make[1]: Leaving directory `/home/steve/Desktop/Software/m4-1.4.10/src'
make: *** [all-recursive] Error 1

I hope someone can help me with this one.

Regards,
           Steve
--

-- 
View this message in context: http://www.nabble.com/Compiling-M4-1.4.10-tf4232929.html#a12042989
Sent from the Gnu - M4 - Bugs mailing list archive at Nabble.com.
Eric Blake | 8 Aug 00:34 2007
Picon

Re: Compiling M4-1.4.10

STEVE555 <stevenward666 <at> hotmail.com> writes:

> 
> 
> Hi to all,

Hi Steve,

Thanks for the report; however, you left out several important details.

>            I've just tried compiling the latest M4.I have applied the latest
> patch :xstrtol-upstream-change.

Huh?  Where did you get this patch?  Are you referring to this thread [1]?  If 
so, that was a patch applied to CVS head, and it is NOT applicable for 1.4.10.  
Are you trying to build from the 1.4.10 tarball directly from the GNU site, or 
did you get 1.4.10 plus patches through a distribution?  Or are you trying to 
build M4 directly from CVS?

[1] http://thread.gmane.org/gmane.comp.gnu.m4.patches/815

> However,I went to the M4 folder,opened Konsole from there,ran ./configure
> then make.

What platform are you on?  I'm guessing a Linux variant, based on your mention 
of Konsole; but what distro?

> 
> I got this error message when make got into the source directory:
> 
> gcc -std=gnu99  -g -O2   -o m4 m4.o builtin.o debug.o eval.o format.o
> freeze.o input.o macro.o output.o path.o symtab.o stackovf.o ../lib/libm4.a
> ../lib/libm4.a(regex.o): In function `wcstouq':
> /usr/include/wchar.h:585: multiple definition of `wcstouq'
> ../lib/libm4.a(quotearg.o):/usr/include/wchar.h:585: first defined here

Nowhere in the M4 source is there a mention of wcstouq.  I'm suspecting that 
your /usr/include/wchar.h has a function that is _supposed_ to be inline, but 
in reality is getting extern'd.  Hence, the multiple definition errors, when it 
gets linked into every .o file that used your broken system wchar.h.  Can you 
please post your config.log; also, post your system's /usr/include/wchar.h (or 
at least the relevant lines around where wcstouq is declared).

Also, what version of gcc are you using?  Did you recently upgrade gcc?  There 
is a known change in semantics in 'extern inline' functions, where gcc used to 
be non-compliant with C99, but where gcc 4.3 (not yet released) changes the 
behavior; in the process, functions that used to be local to a compilation unit 
are now exported.  You can check this by seeing if __GNUC_STDC_INLINE__ is 
defined by your compiler.

> 
> I hope someone can help me with this one.

We probably can, but it will take some feedback from you along the way.

--

-- 
Eric Blake
STEVE555 | 8 Aug 01:03 2007
Picon

Re: Compiling M4-1.4.10


Hi,
   First of all yes that was the patch I was referring to,I'm sorry I had no
idea it was for the CVS version of M4.The M4 I'm compiling with is form the
tarball sorurce from GNU.

Second,the O.S I'm using is Linux Kubuntu Feisty Fawn version 7.04.

Third, the config.log is here:
http://www.nabble.com/file/p12044224/config.log config.log .

Fourth,the GCC version is 4.3.0 from the SVN repository.I had applied the
updates for it and re-compiled it yesterday.Here is the ouptut using gcc -v:

Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: /home/steve/Gcc/trunk/configure
Thread model: posix
gcc version 4.3.0 20070806 (experimental)
steve <at> x1-6-00-c9-00-02-c6-36:~$

5th,Here is my wchar.h : http://www.nabble.com/file/p12044224/wchar.h
wchar.h 

Lastly,I'm very sorry I wasn't very specific,I hadn't realized I needed to
provide more information.

Regards,
           Steve 

 
--

-- 
View this message in context: http://www.nabble.com/Compiling-M4-1.4.10-tf4232929.html#a12044224
Sent from the Gnu - M4 - Bugs mailing list archive at Nabble.com.
Eric Blake | 8 Aug 02:35 2007
Picon

Re: Compiling M4-1.4.10


According to STEVE555 on 8/7/2007 5:03 PM:
> Hi,
>    First of all yes that was the patch I was referring to,I'm sorry I had no
> idea it was for the CVS version of M4.The M4 I'm compiling with is form the
> tarball sorurce from GNU.

It seems a little bit inconsistent that you want bleeding-edge unreleased
gcc, but are willing to stick with the 1.4.10 tarball rather than going
with CVS, but I suppose that's okay.

> 
> Second,the O.S I'm using is Linux Kubuntu Feisty Fawn version 7.04.
> 
> Third, the config.log is here:
> http://www.nabble.com/file/p12044224/config.log config.log .

Thanks.  Obviously, we currently don't do a good job of detecting outdated
system headers in the presence of bleeding edge compilers.

> 
> Fourth,the GCC version is 4.3.0 from the SVN repository.I had applied the
> updates for it and re-compiled it yesterday.

Bingo - just as I guessed.  You need to upgrade your glibc installation
before you can use gcc 3.4.x:
http://sourceware.org/bugzilla/show_bug.cgi?id=4022

> 5th,Here is my wchar.h : http://www.nabble.com/file/p12044224/wchar.h
> wchar.h 

Just as I thought - you have an old header where wcstouq is still declared
extern inline.

A word of advice - there is nothing inherently wrong with using
bleeding-edge unreleased tools (such as the gcc 4.3.0 svn snapshot); after
all, somebody has to test it before it can be released as a stable
product.  However, using unreleased software is something that generally
takes an extreme amount of experience and knowledge, not to mention a
close monitoring of the development mailing lists, so that changes in
behavior will not be a total surprise to you.  The fact that you reported
your problem to the m4 list, rather than the gcc or glibc lists, shows
that you probably aren't prepared for bleeding edge gcc development yet.
If it were me, I would downgrade your gcc back to the pre-compiled 4.2.x
shipped with your distro, rather than trying to use an unreleased gcc 4.3;
when 4.3 is finally released, I would then wait until Kubuntu releases it
as a precompiled package, because then you will be guaranteed to also pick
up any prerequisite upgrades such as system headers.  But if you plan on
continuing with gcc 4.3 right now, then it is not m4's bug that you can't
compile; rather, it is the mismatch between your outdated system headers
and your new compiler.

--
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9 <at> byu.net
Eric Blake | 9 Aug 06:31 2007
Picon

Re: defn() with multiple arguments


According to Schwarz, Konrad on 8/6/2007 1:54 AM:
> However, I think the following example motivates this feature, should
> you in fact have need of motivation.

Actually, on further thought, you brought up a valid use for this
POSIX-mandated feature.  When I became the M4 maintainer, the M4 1.4.4
manual said this:

System V m4 supports multiple arguments to `defn'.  This is
not implemented in GNU m4.  Its usefulness is unclear to me.

I deleted the sentence (first person disclaimers in a manual is rather
tacky) but did nothing about the lack of implementation.

But since POSIX requires it, and since it can, indeed, be used to
concatenate macros that would otherwise contain unbalanced quotes if
expanded in isolation, I think it is worth adding to M4 1.4.11.  Patch to
follow soon.  However, for the 1.4.x series, I will simply bail out if you
try to use defn to concatenate a builtin with a text, as the GNU m4
plumbing isn't currently built to handle that (I still hope to get it in
for m4 2.0, but that's another story).

> In other words, that defn() be implemented to concatenate its arguments
> first, then to quote the resulting string as a whole.

Sorry, but for compatibility with other implementations, it can't quite
work like that.  Each macro definition is quoted, then those quoted
strings are concatenated.  However, because the entire concatenation
option happens before any reparsing is done, you can get away with it as
long as you finally balance things back again.  Whereas, as you noted,
using defn on a single macro with unbalanced quotes causes problems the
moment it is reparsed, prior to any subsequent text it would be
concatenated with.

Here's a test case I tried on Solaris' m4, as well as CVS head, to make
sure the handling of the concatenated quotes is done correctly:

$ /usr/xpg4/bin/m4
define(l,<a[a>)define(r,<a]a>)define(a,A)dnl
changequote([,])dnl
define([q1],defn([l])defn([r]))
])

define([q2],defn([l],[r]))

dumpdef([q1],[q2])dnl
q1:	<a[a>]defn([r]))

q2:	<a[a>][<a]a>
q2
<Aa><aA>

Note how for q1, defn([l]) expands to [<a[a>], which is then rescanned as
an unterminated string containing a nested [a>]; then the text defn([r]))
is read in as part of the string, and I had to add another ]) to close out
the string.  But for q2, defn([l],[r]) expands to [<a[a>][<a]a>], with an
embedded end-of-string start-of-string; which, after rescanning, leaves
unquoted <a, the string [a>], the string [<a], and another unquoted a> as
the desired definition for q2.

--
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9 <at> byu.net
Schwarz, Konrad | 9 Aug 09:08 2007
Picon

RE: defn() with multiple arguments

> -----Original Message-----
> From: Eric Blake [mailto:ebb9 <at> byu.net] 

> I deleted the sentence (first person disclaimers in a manual is rather
> tacky) but did nothing about the lack of implementation.
> 
> But since POSIX requires it, and since it can, indeed, be 
> used to concatenate macros that would otherwise contain 
> unbalanced quotes if expanded in isolation, I think it is 
> worth adding to M4 1.4.11.  Patch to follow soon.

Thank you very much.  That's great news.

On another tack, the otherwise excellent m4 manual is silent on what
happens if you change the quoting characters to open parenthesis, close
parenthesis.  I would expect things to break, since these characters
enclose a macro's arguments; perhaps it would be worthwhile to add a
warning to the manual not to do this, or to prohibit it outright, or
perhaps to clarify the resulting situation.  (Similar, but more
theoretical, questions arise for use of comma, dollar, and number sign
as quoting characters.)

Regards,

Konrad Schwarz
Eric Blake | 9 Aug 14:10 2007
Picon

Re: defn() with multiple arguments


According to Schwarz, Konrad on 8/9/2007 1:08 AM:
> On another tack, the otherwise excellent m4 manual is silent on what
> happens if you change the quoting characters to open parenthesis, close
> parenthesis.  I would expect things to break, since these characters
> enclose a macro's arguments; perhaps it would be worthwhile to add a
> warning to the manual not to do this, or to prohibit it outright, or
> perhaps to clarify the resulting situation.  (Similar, but more
> theoretical, questions arise for use of comma, dollar, and number sign
> as quoting characters.)

In CVS head (currently labeled m4 1.9a), we have added the changesyntax
builtin, which lets you change which characters play the role normally
used by ( and ).  And in 1.4.10, the manual already mentions the
following, in the changequote node, exactly addressing your concern:

| Quotes are recognized in preference to argument collection.  In
| particular, if `start' is a single `(', then argument
| collection is effectively disabled.  For portability with other
| implementations, it is a good idea to avoid `(', `,', and
| `)' as the first character in `start'.

In my opinion, having changequote force a prohibition of certain tokens
would only be an artificial limitation of the source code, so I don't see
any reason to change the current behavior, even if it does mean the user
can shoot themselves in the foot.

--
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9 <at> byu.net
Eric Blake | 10 Aug 22:45 2007
Picon

Re: m4 1.4.10


[Adding bug-m4, since that is the list for M4 problems, and bug-gnulib,
since this particular problem comes from the gnulib file fseeko.  Feel
free to drop bug-gnu-utils on replies]

According to Ancient_Hacker on 8/10/2007 11:02 AM:
> Hi, I'm trying to compile m41.4.10 on a kinda dated OpenBSD system,
> using gcc 2.95.3
> 
> I get the message:
> 
>  In file included from /usr/include/stdio.h:46,
>                  from stdio.h:31,
>                  from fseeko.c:21:
> /usr/include/sys/types.h:79: warning: empty declaration
> fseeko.c: In function `rpl_fseeko':
> fseeko.c:72: structure has no member named `_ext'
> *** Error code 1
> 
> Looking at the source, it looks like there's a "quad_t" type that isnt
> defined .

Can you figure out which header on your platform defines quad_t, so that
we can make sure it is included prior to sys/types.h?  Also, it looks like
your stdio.h is older than the version that Bruno was looking at when he
wrote the code to use FILE._ext; I wonder if using fp->_ub from two lines
later in fseeko.c, rather than the fp_ub macro associated with __OpenBSD__
would make a difference.

--
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9 <at> byu.net

Gmane