antlr C target regression - 3.1.2+?

Hi,

I believe I've found a regression in the ANTLR C target. If I try to 
compile the polydiff example on 3.1.1, I get the following tree as first:

(* 2 (^ x 3)) + (^ x 5) + (* 4 x) + (* 10 x) + (* 8 x) + x + 2

However, with antlr 3.1.2+, I get:

( 2 (^ x 3)) + (^ x 5) + ( 4 x) + ( 10 x) + ( 8 x) + x + 2

(the '*'s are missing!) Seems that one cannot use imaginary tokens?

Also, I'm trying to convert my own antlr2 grammar into antlr3 grammar, 
which turned out into a nightmare. I've currently a semi-working grammar 
(see attached, input text could be e.g. [word="a"]), which, however, 
produces no trees (cqpAST.tree = NULL, "No tree produced" in my code 
which is attached too), although it finishes without any errors. Any 
idea what might cause that? Don't know if this is somehow related to the 
above mentioned problem (I'm unable to try my grammar with 3.1.1 because 
of another error which I can't explain, but which vanishes in 3.1.2+).

I spent with the move from antlr2 to antlr3 a disgusting amount of time 
and now it seems to be totally wasted:(

Thank you very much in advance for any advice!
Milos Jakubicek

//  Copyright (c) 1999-2008  Pavel Rychly

Re: antlr C target regression - 3.1.2+?

I am sorry you feel you have wasted your time and spent disgusting  
amounts of it - a conversion should not take much more than a day or  
so if you know what you are doing. :)

Please download the 3.1.4 snapshot and let me know if that wastes a  
little less of your precious time. If your time is still wasted then I  
will look to fix it and obvously I will be sending you a check to  
cover all the time I have wasted for you :)

Seriously though, please try 3.1.4 and report back.

Jim

On Sep 3, 2009, at 5:45 PM, Milos Jakubicek <xjakub@...> wrote:

> Hi,
>
> I believe I've found a regression in the ANTLR C target. If I try to  
> compile the polydiff example on 3.1.1, I get the following tree as  
> first:
>
> (* 2 (^ x 3)) + (^ x 5) + (* 4 x) + (* 10 x) + (* 8 x) + x + 2
>
> However, with antlr 3.1.2+, I get:
>
> ( 2 (^ x 3)) + (^ x 5) + ( 4 x) + ( 10 x) + ( 8 x) + x + 2
>
> (the '*'s are missing!) Seems that one cannot use imaginary tokens?
>
> Also, I'm trying to convert my own antlr2 grammar into antlr3  

Re: antlr C target regression - 3.1.2+?

Hi Jim,

I'm sorry for being that helpless, but the move from antlr2 to antlr3 is 
(especially in case of a C++ project) really not that easy.

I've tried the 3.1.4 snapshot of antlr C runtime (didn't find any 
snapshot of the tool itself), it solves the problem for the polydiff 
examples ('*' are back there:), but not the one with my grammar.

The parsing still finishes without any errors, but the there are no 
trees. This shouldn't be possible normally, should it (I guess that from 
the examples code? What might cause that? Is it a problem with the 
grammar or with my code interacting with the C runtime?

Thank you again for any help. I can promise that for your wasted time 
with me I'll waste time with antlr maven dependencies to get antlr (and 
antlrworks and all their maven dependencies) building offline from 
source with maven to be able to update (and in case of antlrworks and 
some maven deps submit as new) the packages in Fedora repositories to 
3.1.3 and forwards on;)

Milos

P.S.: If you'd look into my grammar, the REGEXP token needs a tiny fix 
to prevent the "multiple alternatives warning":

REGEXP: '"' ('\\' . |~('"'|'\\'))* '"'

Dne 4.9.2009 16:38, Jim Idle wrote:
> I am sorry you feel you have wasted your time and spent disgusting

Writing out ASTs / Preserving comments / Handling erroneous tokens

<div>

<p dir="LTR"><span lang="en-gb">I currently have a set of 3 or 4 grammars that are a work in progress that read stored procedures from various DB vendors and output a fairly generic AST. The AST from each vendor is then re-written via a tree grammar and lots of tree re-writing (again a work in progress) into a standard IngresDB AST, which a final tree grammar then prints out in Ingres DB Procedure syntax. I am currently working towards something which will eventually be released as an open source project and wondered if the community might be able to help me solve a few issues.</span></p>

<p dir="LTR"><span lang="en-gb"></span></p>

<p dir="LTR"><span lang="en-gb">1) What is the best way for my final tree grammar to output the specific syntax</span><span lang="en-gb"> I</span><span lang="en-gb"> need. At the minute</span><span lang="en-gb"> I</span><span lang="en-gb"> have a tree grammar which has lots of {System.out.print($a.text)} in and amongst</span><span lang="en-gb"> the tree nodes and</span><span lang="en-gb"> if statements which check input.LA(1) and add comma's etc, etc.&nbsp; It makes the grammar harder to read. Is there a better way of doing this? The StringTemplates looked like they might be the solution but that is another thing to learn and</span><span lang="en-gb"> I</span><span lang="en-gb"> also don't understand how the string templates print out your syntax once you have converted your AST to a StringTemplate?</span></p>

<p dir="LTR"><span lang="en-gb"></span></p>

<p dir="LTR"><span lang="en-gb">2) I pass the comments in the initial grammars to the hidden stream. Can anyone point me at the right article to show how</span><span lang="en-gb"> I</span><span lang="en-gb"> can output these comments from the hidden stream at the right point in my final grammar?</span></p>

<p dir="LTR"><span lang="en-gb"></span></p>

<p dir="LTR"><span lang="en-gb">3) As this is an organically growing process there are many things my grammars can not yet cope with, there are also features that some vendors support and others do not. I want to be able to pass these things into each grammar and kind of 'TODO' tree node so that</span><span lang="en-gb"> I</span><span lang="en-gb"> can output them in the final grammar, probably commented out with a message to the developer to point out they have some manual effort to perform. Should</span><span lang="en-gb"> I</span><span lang="en-gb"> use the error handling to do this? Can anyone point me at a good article or offer any advice on how</span><span lang="en-gb"> I</span><span lang="en-gb"> should achieve this?</span></p>

<p dir="LTR"><span lang="en-gb"></span></p>

<p dir="LTR"><span lang="en-gb">Thanks in advance.</span></p>

<p dir="LTR"><span lang="en-gb">Stuart - A new ANTLR enthusiast!</span><span lang="en-gb"></span></p>

</div>