Germán Diago | 1 Nov 16:46
Picon

Writing a grammar with "subgrammars"

Hello. I'm trying to write a parser for a language and I have something like this:


class MyPieceOfGrammar : public Grammar<MyPieceOfGrammar>
{

   template <class ScannerT>
   struct definition {
        rule<parserT> rule1, rule2, rule3;

        definition(const MyPieceOfGrammar & self)
        {
           rule1 = ...;
           rule2 = ...;
         }

         const rule<ScannerT> & start() const
         {
             return rule1;
         }
   };

};

So now, I have a little parser for one piece of my grammar. But the problem is that I want to write many little grammars
and I want to reuse some rules for different grammars, rules that I want to be stored in a "main" grammar object or so
so that those rules are visible from my little grammars. After that I want to be able to integrate all those little grammars in
the big grammar, which is the whole language. . I want something like this:

class AllTheGrammar : public grammar<AllTheGrammar> {
  
   ///Uses all my little grammars and my little grammars should be able to reuse rules from this one, because if I make
      a change in a rule of this one, the change will be propagated to all the other grammars.
 };

And I want to write little pieces of grammar in different files to build a modular parser. If I don't do this, I get a big piece of monolithic
software, and it's too big to do it in a monolithic way.

I don't know if I have expressed myself well enough, I expect so. Can anyone help on this, please?






-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general
Joel de Guzman | 2 Nov 00:38
Picon
Favicon

Re: Writing a grammar with "subgrammars"

Germán Diago wrote:
> Hello. I'm trying to write a parser for a language and I have something 
> like this:
> 
> 
> class MyPieceOfGrammar : public Grammar<MyPieceOfGrammar>
> {
> 
>    template <class ScannerT>
>    struct definition {
>         rule<parserT> rule1, rule2, rule3;
> 
>         definition(const MyPieceOfGrammar & self)
>         {
>            rule1 = ...;
>            rule2 = ...;
>          }
> 
>          const rule<ScannerT> & start() const
>          {
>              return rule1;
>          }
>    };
> 
> };
> 
> So now, I have a little parser for one piece of my grammar. But the 
> problem is that I want to write many little grammars
> and I want to reuse some rules for different grammars, rules that I want 
> to be stored in a "main" grammar object or so
> so that those rules are visible from my little grammars. After that I 
> want to be able to integrate all those little grammars in
> the big grammar, which is the whole language. . I want something like this:
> 
> class AllTheGrammar : public grammar<AllTheGrammar> {
>   
>    ///Uses all my little grammars and my little grammars should be able 
> to reuse rules from this one, because if I make
>       a change in a rule of this one, the change will be propagated to 
> all the other grammars.
>  };
> 
> And I want to write little pieces of grammar in different files to build 
> a modular parser. If I don't do this, I get a big piece of monolithic
> software, and it's too big to do it in a monolithic way.
> 
> I don't know if I have expressed myself well enough, I expect so. Can 
> anyone help on this, please?

Spirit is designed to be modular. You can have as many fine-grained grammars
as you wish. In fact, that's the recommended strategy. You simply use
your small grammars inside your bigger grammars, that's it.

Regards,
--

-- 
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Germán Diago | 2 Nov 09:27
Picon

Re: Writing a grammar with "subgrammars"



2007/11/2, Joel de Guzman <joel <at> boost-consulting.com>:
Germán Diago wrote:
> Hello. I'm trying to write a parser for a language and I have something
> like this:
>
>
> class MyPieceOfGrammar : public Grammar<MyPieceOfGrammar>
> {
>
>    template <class ScannerT>
>    struct definition {
>         rule<parserT> rule1, rule2, rule3;
>
>         definition(const MyPieceOfGrammar & self)
>         {
>            rule1 = ...;
>            rule2 = ...;
>          }
>
>          const rule<ScannerT> & start() const
>          {
>              return rule1;
>          }
>    };
>
> };
>
> So now, I have a little parser for one piece of my grammar. But the
> problem is that I want to write many little grammars
> and I want to reuse some rules for different grammars, rules that I want
> to be stored in a "main" grammar object or so
> so that those rules are visible from my little grammars. After that I
> want to be able to integrate all those little grammars in
> the big grammar, which is the whole language. . I want something like this:
>
> class AllTheGrammar : public grammar<AllTheGrammar> {
>
>    ///Uses all my little grammars and my little grammars should be able
> to reuse rules from this one, because if I make
>       a change in a rule of this one, the change will be propagated to
> all the other grammars.
>  };
>
> And I want to write little pieces of grammar in different files to build
> a modular parser. If I don't do this, I get a big piece of monolithic
> software, and it's too big to do it in a monolithic way.
>
> I don't know if I have expressed myself well enough, I expect so. Can
> anyone help on this, please?

Spirit is designed to be modular. You can have as many fine-grained grammars
as you wish. In fact, that's the recommended strategy. You simply use
your small grammars inside your bigger grammars, that's it.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net

Yes, I know, but the problem is that I don't know how to declare rules outside of a grammar's definition.
   For example, I have a rule<ScannerT> arule

    Now I want to make a grammar (another one) which reuses a rule,  but when I  do this, I don't know the
    way to access rule<ScannerT> arule, because a rule is stored inside another grammar. I want to have
    "global rules" that can be reused. Which is the correct way to do it? I don't know how to do it because
    the template parameter gets me confused.
    Thanks in advance.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general
Picon
Favicon

Re: Writing a grammar with "subgrammars"

Hi there,

First of all thanks a lot for this amazing tool ! I have a lot of fun using it !
Though I think that have more or less the same problem than German in that I have solved it but in a not completely satisfactory way. Indeed the solution I have found lead to some slightly complicated type definitions
Here is an excerpt of my main grammar:

     // Litteral grammars
        typedef LiteralGrammar<bool> bool_literal_t;
        bool_literal_t boolLiteral;
        typedef LiteralGrammar<double> double_literal_t;
        double_literal_t doubleLiteral;
        typedef LiteralGrammar<int> int_literal_t;
        int_literal_t intLiteral;
        typedef LiteralGrammar<std::string> string_literal_t;
        string_literal_t stringLiteral;

        // wrapping input nodes tables in grammars
        typedef GrammarWrapper<input_node_table<double>::type> input_double_t;
        input_double_t doubleInputTable;
        typedef GrammarWrapper<input_node_table<std::vector<double> >::type> input_node_double_vector_t;
        input_node_double_vector_t doubleVectorInputTable;       
        typedef GrammarWrapper<input_node_table<int>::type> input_int_t;
        input_int_t intInputTable;
        typedef GrammarWrapper<input_node_table<bool>::type> input_bool_t;
        input_bool_t boolInputTable;

        // wrapping output nodes tables in grammars
        typedef GrammarWrapper<output_node_table<double>::type> output_double_t;
        output_double_t doubleOutputTable;
        typedef GrammarWrapper<output_node_table<std::vector<double> >::type> output_node_double_vector_t;
        output_node_double_vector_t doubleVectorOutputTable;
        typedef GrammarWrapper<output_node_table<int>::type> output_int_t;
        output_int_t intOutputTable;
        typedef GrammarWrapper<output_node_table<bool>::type> output_bool_t;
        output_bool_t boolOutputTable;

        // all integer output nodes
        typedef JointGrammar2<int_literal_t, output_int_t> all_int_output_t;
        all_int_output_t allIntOutputs;

        // integer expressions
        typedef ExpressionGrammar<all_int_output_t> int_expr_t;
        int_expr_t intExpression;

        // index array grammars
        typedef IndexArrayGrammar<output_node_double_vector_t, int_expr_t,
                                  OutPutNode<double> > output_double_node_index_array_t;
        output_double_node_index_array_t doubleOutputArrayElement;
       
        typedef IndexArrayGrammar<input_node_double_vector_t, int_expr_t,
                                  InPutNode<double> > double_input_index_array_t;
        double_input_index_array_t doubleInputArrayElement;

        // all double outputs
        typedef JointGrammar3<double_literal_t,
                              output_double_node_index_array_t,
                              output_double_t> all_double_output_t;
        all_double_output_t allDoubleOutputs;

        typedef JointGrammar2<double_literal_t,
                              output_double_node_index_array_t> all_output_double_t;
        all_output_double_t allDoubleOutput;

        //double expression grammar
        typedef ExpressionGrammar<all_double_output_t> double_expr_t;
        double_expr_t doubleExpression;

        // comparisons grammars
        typedef ComparisonGrammar<double_expr_t> double_comp_t;
        double_comp_t doubleComparison;
        typedef ComparisonGrammar<int_expr_t> int_comp_t;
        int_comp_t intComparison;

        // all bool usable by bool expression grammar
        typedef JointGrammar4<bool_literal_t, output_bool_t,
                              double_comp_t, int_comp_t> all_output_bool1_t;
        all_output_bool1_t allBoolOutputs;
        typedef JointGrammar3<bool_literal_t, double_comp_t, int_comp_t> all_output_bool_t;
        all_output_bool_t allBoolOutput;

        //bool expression grammar
        typedef ExpressionGrammar<all_output_bool1_t> bool_expr_t;
        bool_expr_t boolExpression;

        // to string grammars
        typedef ToStringGrammar<double_expr_t> to_string_double_t;
        to_string_double_t toStringDouble;
        typedef ToStringGrammar<int_expr_t> to_string_int_t;
        to_string_int_t toStringInt;
        typedef ToStringGrammar<bool_expr_t> to_string_bool_t;
        to_string_bool_t toStringBool;
        // all strings grammars
        typedef JointGrammar4<string_literal_t, to_string_double_t, to_string_int_t, to_string_bool_t> all_strings_t;
        all_strings_t allStrings;

        //string expression grammar
        typedef ExpressionGrammar<all_strings_t> string_expr_t;
        string_expr_t stringExpression;

        // debug grammar
        typedef DebugGrammar<string_expr_t, bool_expr_t> debug_t;
        debug_t debug;

        // all assignable doubles
        typedef JointGrammar2<double_input_index_array_t, input_double_t> all_double_inputs_t;
        all_double_inputs_t all_double_input_t;

        // assignment grammars
        typedef AssignmentGrammar<all_double_inputs_t, double_expr_t> double_assgt_t;
        double_assgt_t doubleAssignments;
        typedef AssignmentGrammar<input_int_t, int_expr_t> int_assgt_t;
        int_assgt_t intAssignments;
        typedef AssignmentGrammar<input_bool_t, bool_expr_t> bool_assgt_t;
        bool_assgt_t boolAssignments;
       
        // all assignments
        typedef JointGrammar4<debug_t, double_assgt_t, int_assgt_t, bool_assgt_t> all_assgt_t;
        all_assgt_t assignments;

        // all control flows
        typedef ControlFlowsGrammar<all_assgt_t, bool_expr_t> control_flows_t;
        control_flows_t controlFlows;
 
As you may notice I have to precise quite precisely the type of my subgrammars. I would be happy if I could use some plain old polymorphism so as to be a bit more general.
best regards,
François

----- Original Message ----
From: Joel de Guzman <joel <at> boost-consulting.com>
To: spirit-general <at> lists.sourceforge.net
Sent: Friday, 2 November, 2007 12:38:49 AM
Subject: Re: [Spirit-general] Writing a grammar with "subgrammars"

Germán Diago wrote:
> Hello. I'm trying to write a parser for a language and I have something
> like this:
>
>
> class MyPieceOfGrammar : public Grammar<MyPieceOfGrammar>
> {
>
>    template <class ScannerT>
>    struct definition {
>        rule<parserT> rule1, rule2, rule3;
>
>        definition(const MyPieceOfGrammar & self)
>        {
>            rule1 = ...;
>            rule2 = ...;
>          }
>
>          const rule<ScannerT> & start() const
>          {
>              return rule1;
>          }
>    };
>
> };
>
> So now, I have a little parser for one piece of my grammar. But the
> problem is that I want to write many little grammars
> and I want to reuse some rules for different grammars, rules that I want
> to be stored in a "main" grammar object or so
> so that those rules are visible from my little grammars. After that I
> want to be able to integrate all those little grammars in
> the big grammar, which is the whole language. . I want something like this:
>
> class AllTheGrammar : public grammar<AllTheGrammar> {

>    ///Uses all my little grammars and my little grammars should be able
> to reuse rules from this one, because if I make
>      a change in a rule of this one, the change will be propagated to
> all the other grammars.
>  };
>
> And I want to write little pieces of grammar in different files to build
> a modular parser. If I don't do this, I get a big piece of monolithic
> software, and it's too big to do it in a monolithic way.
>
> I don't know if I have expressed myself well enough, I expect so. Can
> anyone help on this, please?

Spirit is designed to be modular. You can have as many fine-grained grammars
as you wish. In fact, that's the recommended strategy. You simply use
your small grammars inside your bigger grammars, that's it.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general


For ideas on reducing your carbon footprint visit Yahoo! For Good this month.
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general
Jose | 2 Nov 16:14
Picon

Spirit vs Spirit2 - docs

Hi,

I just signed for the list. Can anybody point me to a doc that explains the differences between Spirit and Spirit 2 ?

Also, is there any PDF doc with the Spirit2 docs ?

thanks and regards
jose

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general
Jens Weller | 2 Nov 16:21
Picon
Picon

Re: Spirit vs Spirit2 - docs

Afaik, there is currently no official (HTML) Documentation of spirit2 as it exists for spirit.

So, best place to go visit and see some of the new and cool stuff spirit2 does is the SVN:
http://spirit.svn.sourceforge.net/svnroot/spirit/trunk/final/

Under doc you also find the presentation which was held by Joel and Hartmut this spring at the boost conference:
http://spirit.svn.sourceforge.net/svnroot/spirit/trunk/final/libs/spirit/doc/

Under Example you find easy-to-read spirit2 code, most of the code should be easy to understand as its well commented.

And last but not least, you always can search the mailinglist.

If you speak german, I wrote a beginners tutorial for spirit2 in german:
http://c-plusplus.de/forum/viewtopic-var-t-is-189058.html

regards

Jens Weller

-------- Original-Nachricht --------
> Datum: Fri, 2 Nov 2007 16:14:37 +0100
> Von: Jose <jmalv04 <at> gmail.com>
> An: spirit-general <at> lists.sourceforge.net
> Betreff: [Spirit-general] Spirit vs Spirit2 - docs

> Hi,
> 
> I just signed for the list. Can anybody point me to a doc that explains
> the
> differences between Spirit and Spirit 2 ?
> 
> Also, is there any PDF doc with the Spirit2 docs ?
> 
> thanks and regards
> jose

--

-- 
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! 
Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Jose | 2 Nov 18:24
Picon

Re: Spirit vs Spirit2 - docs

thanks. exactly what i was looking for.

when is the spirit2 release expected ? in 3 months, 6 months or later ?

thanks

On 11/2/07, Jens Weller <JensWeller <at> gmx.de> wrote:
Afaik, there is currently no official (HTML) Documentation of spirit2 as it exists for spirit.

So, best place to go visit and see some of the new and cool stuff spirit2 does is the SVN:
http://spirit.svn.sourceforge.net/svnroot/spirit/trunk/final/

Under doc you also find the presentation which was held by Joel and Hartmut this spring at the boost conference:
http://spirit.svn.sourceforge.net/svnroot/spirit/trunk/final/libs/spirit/doc/

Under Example you find easy-to-read spirit2 code, most of the code should be easy to understand as its well commented.

And last but not least, you always can search the mailinglist.

If you speak german, I wrote a beginners tutorial for spirit2 in german:
http://c-plusplus.de/forum/viewtopic-var-t-is-189058.html

regards

Jens Weller

-------- Original-Nachricht --------
> Datum: Fri, 2 Nov 2007 16:14:37 +0100
> Von: Jose <jmalv04 <at> gmail.com>
> An: spirit-general <at> lists.sourceforge.net
> Betreff: [Spirit-general] Spirit vs Spirit2 - docs

> Hi,
>
> I just signed for the list. Can anybody point me to a doc that explains
> the
> differences between Spirit and Spirit 2 ?
>
> Also, is there any PDF doc with the Spirit2 docs ?
>
> thanks and regards
> jose

--
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general
Germán Diago | 3 Nov 11:33
Picon

Re: Writing a grammar with "subgrammars"



2007/11/2, François du Vignaud <francois_du_vignaud <at> yahoo.com>:
Hi there,

First of all thanks a lot for this amazing tool ! I have a lot of fun using it !
Though I think that have more or less the same problem than German in that I have solved it but in a not completely satisfactory way. Indeed the solution I have found lead to some slightly complicated type definitions
Here is an excerpt of my main grammar:

     // Litteral grammars
        typedef LiteralGrammar<bool> bool_literal_t;
        bool_literal_t boolLiteral;
        typedef LiteralGrammar<double> double_literal_t;
        double_literal_t doubleLiteral;
        typedef LiteralGrammar<int> int_literal_t;
        int_literal_t intLiteral;
        typedef LiteralGrammar<std::string> string_literal_t;
        string_literal_t stringLiteral;

        // wrapping input nodes tables in grammars
        typedef GrammarWrapper<input_node_table<double>::type> input_double_t;
        input_double_t doubleInputTable;
        typedef GrammarWrapper<input_node_table<std::vector<double> >::type> input_node_double_vector_t;
        input_node_double_vector_t doubleVectorInputTable;       
        typedef GrammarWrapper<input_node_table<int>::type> input_int_t;
        input_int_t intInputTable;
        typedef GrammarWrapper<input_node_table<bool>::type> input_bool_t;
        input_bool_t boolInputTable;

        // wrapping output nodes tables in grammars
        typedef GrammarWrapper<output_node_table<double>::type> output_double_t;
        output_double_t doubleOutputTable;
        typedef GrammarWrapper<output_node_table<std::vector<double> >::type> output_node_double_vector_t;
        output_node_double_vector_t doubleVectorOutputTable;
        typedef GrammarWrapper<output_node_table<int>::type> output_int_t;
        output_int_t intOutputTable;
        typedef GrammarWrapper<output_node_table<bool>::type> output_bool_t;
        output_bool_t boolOutputTable;

        // all integer output nodes
        typedef JointGrammar2<int_literal_t, output_int_t> all_int_output_t;
        all_int_output_t allIntOutputs;

        // integer expressions
        typedef ExpressionGrammar<all_int_output_t> int_expr_t;
        int_expr_t intExpression;

        // index array grammars
        typedef IndexArrayGrammar<output_node_double_vector_t, int_expr_t,
                                  OutPutNode<double> > output_double_node_index_array_t;
        output_double_node_index_array_t doubleOutputArrayElement;
       
        typedef IndexArrayGrammar<input_node_double_vector_t, int_expr_t,
                                  InPutNode<double> > double_input_index_array_t;
        double_input_index_array_t doubleInputArrayElement;

        // all double outputs
        typedef JointGrammar3<double_literal_t,
                              output_double_node_index_array_t,
                              output_double_t> all_double_output_t;
        all_double_output_t allDoubleOutputs;

        typedef JointGrammar2<double_literal_t,
                              output_double_node_index_array_t> all_output_double_t;
        all_output_double_t allDoubleOutput;

        //double expression grammar
        typedef ExpressionGrammar<all_double_output_t> double_expr_t;
        double_expr_t doubleExpression;

        // comparisons grammars
        typedef ComparisonGrammar<double_expr_t> double_comp_t;
        double_comp_t doubleComparison;
        typedef ComparisonGrammar<int_expr_t> int_comp_t;
        int_comp_t intComparison;

        // all bool usable by bool expression grammar
        typedef JointGrammar4<bool_literal_t, output_bool_t,
                              double_comp_t, int_comp_t> all_output_bool1_t;
        all_output_bool1_t allBoolOutputs;
        typedef JointGrammar3<bool_literal_t, double_comp_t, int_comp_t> all_output_bool_t;
        all_output_bool_t allBoolOutput;

        //bool expression grammar
        typedef ExpressionGrammar<all_output_bool1_t> bool_expr_t;
        bool_expr_t boolExpression;

        // to string grammars
        typedef ToStringGrammar<double_expr_t> to_string_double_t;
        to_string_double_t toStringDouble;
        typedef ToStringGrammar<int_expr_t> to_string_int_t;
        to_string_int_t toStringInt;
        typedef ToStringGrammar<bool_expr_t> to_string_bool_t;
        to_string_bool_t toStringBool;
        // all strings grammars
        typedef JointGrammar4<string_literal_t, to_string_double_t, to_string_int_t, to_string_bool_t> all_strings_t;
        all_strings_t allStrings;

        //string expression grammar
        typedef ExpressionGrammar<all_strings_t> string_expr_t;
        string_expr_t stringExpression;

        // debug grammar
        typedef DebugGrammar<string_expr_t, bool_expr_t> debug_t;
        debug_t debug;

        // all assignable doubles
        typedef JointGrammar2<double_input_index_array_t, input_double_t> all_double_inputs_t;
        all_double_inputs_t all_double_input_t;

        // assignment grammars
        typedef AssignmentGrammar<all_double_inputs_t, double_expr_t> double_assgt_t;
        double_assgt_t doubleAssignments;
        typedef AssignmentGrammar<input_int_t, int_expr_t> int_assgt_t;
        int_assgt_t intAssignments;
        typedef AssignmentGrammar<input_bool_t, bool_expr_t> bool_assgt_t;
        bool_assgt_t boolAssignments;
       
        // all assignments
        typedef JointGrammar4<debug_t, double_assgt_t, int_assgt_t, bool_assgt_t> all_assgt_t;
        all_assgt_t assignments;

        // all control flows
        typedef ControlFlowsGrammar<all_assgt_t, bool_expr_t> control_flows_t;
        control_flows_t controlFlows;
 
As you may notice I have to precise quite precisely the type of my subgrammars. I would be happy if I could use some plain old polymorphism so as to be a bit more general.
best regards,
François

----- Original Message ----
From: Joel de Guzman < joel <at> boost-consulting.com>
To: spirit-general <at> lists.sourceforge.net
Sent: Friday, 2 November, 2007 12:38:49 AM
Subject: Re: [Spirit-general] Writing a grammar with "subgrammars"

Germán Diago wrote:
> Hello. I'm trying to write a parser for a language and I have something
> like this:
>
>
> class MyPieceOfGrammar : public Grammar<MyPieceOfGrammar>
> {
>
>    template <class ScannerT>
>    struct definition {
>        rule<parserT> rule1, rule2, rule3;
>
>        definition(const MyPieceOfGrammar & self)
>        {
>            rule1 = ...;
>            rule2 = ...;
>          }
>
>          const rule<ScannerT> & start() const
>          {
>              return rule1;
>          }
>    };
>
> };
>
> So now, I have a little parser for one piece of my grammar. But the
> problem is that I want to write many little grammars
> and I want to reuse some rules for different grammars, rules that I want
> to be stored in a "main" grammar object or so
> so that those rules are visible from my little grammars. After that I
> want to be able to integrate all those little grammars in
> the big grammar, which is the whole language. . I want something like this:
>
> class AllTheGrammar : public grammar<AllTheGrammar> {

>    ///Uses all my little grammars and my little grammars should be able
> to reuse rules from this one, because if I make
>      a change in a rule of this one, the change will be propagated to
> all the other grammars.
>  };
>
> And I want to write little pieces of grammar in different files to build
> a modular parser. If I don't do this, I get a big piece of monolithic
> software, and it's too big to do it in a monolithic way.
>
> I don't know if I have expressed myself well enough, I expect so. Can
> anyone help on this, please?

Spirit is designed to be modular. You can have as many fine-grained grammars
as you wish. In fact, that's the recommended strategy. You simply use
your small grammars inside your bigger grammars, that's it.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net

I found a way to decouple rules from my grammars and semantic actions from grammars too, in an easy way. Here is my solution. I have a global (Singleton) RulesStore from where I reuse the rules I want to. When I write a piece of grammar I reuse the rules from the RulesStore if I have to, and I write my piece of grammar own rules inside the grammar if I need to. This way if I change a global rule, that rule will be propagated to all my pieces of grammar without touching any of them. The implementation is something like this:

 //It's a singleton
 template <class ScannerT>
 struct RulesStore
{
    rule<ScannerT> r1, r2;
};

struct MyLittleGrammar : public mygrammar<MyLittleGrammar>
{
   template <class ScannerT>
   struct definition
   {
       rule<ScannerT> rule1, rule2, localrule;

       RulesStore<ScannerT> & globalrules;

      definition(const MyLittleGrammar & self) : globalrules(RulesStore<ScannerT>::instance())
      {
           rule1 = globalrules.r1;
           rule2 = globalrules.r2;
           localrule = ...;
      }
      //.......
   }
};

//This is the class where I use MyLittleGrammar and attach semantic actions to it.
class MyLittleGrammarAnalyzer  {
  MyLittleGrammar lg;

  void action(const char * first, const char * last) { //... }

  MyLittleGrammarAnalyzer()
  {
    lg.rule1[tr1::bind(&action, this, _1, _2)];
  }
};

I don't know if this code is exactly correct, but you get the idea.
The only thing I'm not sure about is that if I attach an action to lg.rule1 and rule1 stores a globalrule, will the action be attached
to globalrules.r1 or just to mylittlegrammar::definition::rule1?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general


For ideas on reducing your carbon footprint visit Yahoo! For Good this month.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general
Germán Diago | 3 Nov 11:57
Picon

Re: Writing a grammar with "subgrammars"



2007/11/3, Germán Diago <germandiago <at> gmail.com>:


2007/11/2, François du Vignaud <francois_du_vignaud <at> yahoo.com >:
Hi there,

First of all thanks a lot for this amazing tool ! I have a lot of fun using it !
Though I think that have more or less the same problem than German in that I have solved it but in a not completely satisfactory way. Indeed the solution I have found lead to some slightly complicated type definitions
Here is an excerpt of my main grammar:

     // Litteral grammars
        typedef LiteralGrammar<bool> bool_literal_t;
        bool_literal_t boolLiteral;
        typedef LiteralGrammar<double> double_literal_t;
        double_literal_t doubleLiteral;
        typedef LiteralGrammar<int> int_literal_t;
        int_literal_t intLiteral;
        typedef LiteralGrammar<std::string> string_literal_t;
        string_literal_t stringLiteral;

        // wrapping input nodes tables in grammars
        typedef GrammarWrapper<input_node_table<double>::type> input_double_t;
        input_double_t doubleInputTable;
        typedef GrammarWrapper<input_node_table<std::vector<double> >::type> input_node_double_vector_t;
        input_node_double_vector_t doubleVectorInputTable;       
        typedef GrammarWrapper<input_node_table<int>::type> input_int_t;
        input_int_t intInputTable;
        typedef GrammarWrapper<input_node_table<bool>::type> input_bool_t;
        input_bool_t boolInputTable;

        // wrapping output nodes tables in grammars
        typedef GrammarWrapper<output_node_table<double>::type> output_double_t;
        output_double_t doubleOutputTable;
        typedef GrammarWrapper<output_node_table<std::vector<double> >::type> output_node_double_vector_t;
        output_node_double_vector_t doubleVectorOutputTable;
        typedef GrammarWrapper<output_node_table<int>::type> output_int_t;
        output_int_t intOutputTable;
        typedef GrammarWrapper<output_node_table<bool>::type> output_bool_t;
        output_bool_t boolOutputTable;

        // all integer output nodes
        typedef JointGrammar2<int_literal_t, output_int_t> all_int_output_t;
        all_int_output_t allIntOutputs;

        // integer expressions
        typedef ExpressionGrammar<all_int_output_t> int_expr_t;
        int_expr_t intExpression;

        // index array grammars
        typedef IndexArrayGrammar<output_node_double_vector_t, int_expr_t,
                                  OutPutNode<double> > output_double_node_index_array_t;
        output_double_node_index_array_t doubleOutputArrayElement;
       
        typedef IndexArrayGrammar<input_node_double_vector_t, int_expr_t,
                                  InPutNode<double> > double_input_index_array_t;
        double_input_index_array_t doubleInputArrayElement;

        // all double outputs
        typedef JointGrammar3<double_literal_t,
                              output_double_node_index_array_t,
                              output_double_t> all_double_output_t;
        all_double_output_t allDoubleOutputs;

        typedef JointGrammar2<double_literal_t,
                              output_double_node_index_array_t> all_output_double_t;
        all_output_double_t allDoubleOutput;

        //double expression grammar
        typedef ExpressionGrammar<all_double_output_t> double_expr_t;
        double_expr_t doubleExpression;

        // comparisons grammars
        typedef ComparisonGrammar<double_expr_t> double_comp_t;
        double_comp_t doubleComparison;
        typedef ComparisonGrammar<int_expr_t> int_comp_t;
        int_comp_t intComparison;

        // all bool usable by bool expression grammar
        typedef JointGrammar4<bool_literal_t, output_bool_t,
                              double_comp_t, int_comp_t> all_output_bool1_t;
        all_output_bool1_t allBoolOutputs;
        typedef JointGrammar3<bool_literal_t, double_comp_t, int_comp_t> all_output_bool_t;
        all_output_bool_t allBoolOutput;

        //bool expression grammar
        typedef ExpressionGrammar<all_output_bool1_t> bool_expr_t;
        bool_expr_t boolExpression;

        // to string grammars
        typedef ToStringGrammar<double_expr_t> to_string_double_t;
        to_string_double_t toStringDouble;
        typedef ToStringGrammar<int_expr_t> to_string_int_t;
        to_string_int_t toStringInt;
        typedef ToStringGrammar<bool_expr_t> to_string_bool_t;
        to_string_bool_t toStringBool;
        // all strings grammars
        typedef JointGrammar4<string_literal_t, to_string_double_t, to_string_int_t, to_string_bool_t> all_strings_t;
        all_strings_t allStrings;

        //string expression grammar
        typedef ExpressionGrammar<all_strings_t> string_expr_t;
        string_expr_t stringExpression;

        // debug grammar
        typedef DebugGrammar<string_expr_t, bool_expr_t> debug_t;
        debug_t debug;

        // all assignable doubles
        typedef JointGrammar2<double_input_index_array_t, input_double_t> all_double_inputs_t;
        all_double_inputs_t all_double_input_t;

        // assignment grammars
        typedef AssignmentGrammar<all_double_inputs_t, double_expr_t> double_assgt_t;
        double_assgt_t doubleAssignments;
        typedef AssignmentGrammar<input_int_t, int_expr_t> int_assgt_t;
        int_assgt_t intAssignments;
        typedef AssignmentGrammar<input_bool_t, bool_expr_t> bool_assgt_t;
        bool_assgt_t boolAssignments;
       
        // all assignments
        typedef JointGrammar4<debug_t, double_assgt_t, int_assgt_t, bool_assgt_t> all_assgt_t;
        all_assgt_t assignments;

        // all control flows
        typedef ControlFlowsGrammar<all_assgt_t, bool_expr_t> control_flows_t;
        control_flows_t controlFlows;
 
As you may notice I have to precise quite precisely the type of my subgrammars. I would be happy if I could use some plain old polymorphism so as to be a bit more general.
best regards,
François

----- Original Message ----
From: Joel de Guzman < joel <at> boost-consulting.com>
To: spirit-general <at> lists.sourceforge.net
Sent: Friday, 2 November, 2007 12:38:49 AM
Subject: Re: [Spirit-general] Writing a grammar with "subgrammars"

Germán Diago wrote:
> Hello. I'm trying to write a parser for a language and I have something
> like this:
>
>
> class MyPieceOfGrammar : public Grammar<MyPieceOfGrammar>
> {
>
>    template <class ScannerT>
>    struct definition {
>        rule<parserT> rule1, rule2, rule3;
>
>        definition(const MyPieceOfGrammar & self)
>        {
>            rule1 = ...;
>            rule2 = ...;
>          }
>
>          const rule<ScannerT> & start() const
>          {
>              return rule1;
>          }
>    };
>
> };
>
> So now, I have a little parser for one piece of my grammar. But the
> problem is that I want to write many little grammars
> and I want to reuse some rules for different grammars, rules that I want
> to be stored in a "main" grammar object or so
> so that those rules are visible from my little grammars. After that I
> want to be able to integrate all those little grammars in
> the big grammar, which is the whole language. . I want something like this:
>
> class AllTheGrammar : public grammar<AllTheGrammar> {

>    ///Uses all my little grammars and my little grammars should be able
> to reuse rules from this one, because if I make
>      a change in a rule of this one, the change will be propagated to
> all the other grammars.
>  };
>
> And I want to write little pieces of grammar in different files to build
> a modular parser. If I don't do this, I get a big piece of monolithic
> software, and it's too big to do it in a monolithic way.
>
> I don't know if I have expressed myself well enough, I expect so. Can
> anyone help on this, please?

Spirit is designed to be modular. You can have as many fine-grained grammars
as you wish. In fact, that's the recommended strategy. You simply use
your small grammars inside your bigger grammars, that's it.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net

I found a way to decouple rules from my grammars and semantic actions from grammars too, in an easy way. Here is my solution. I have a global (Singleton) RulesStore from where I reuse the rules I want to. When I write a piece of grammar I reuse the rules from the RulesStore if I have to, and I write my piece of grammar own rules inside the grammar if I need to. This way if I change a global rule, that rule will be propagated to all my pieces of grammar without touching any of them. The implementation is something like this:

 //It's a singleton
 template <class ScannerT>
 struct RulesStore
{
    rule<ScannerT> r1, r2;
};

struct MyLittleGrammar : public mygrammar<MyLittleGrammar>
{
   template <class ScannerT>
   struct definition
   {
       rule<ScannerT> rule1, rule2, localrule;

       RulesStore<ScannerT> & globalrules;

      definition(const MyLittleGrammar & self) : globalrules(RulesStore<ScannerT>::instance())
      {
           rule1 = globalrules.r1;
           rule2 = globalrules.r2;
           localrule = ...;
      }
      //.......
   }
};

//This is the class where I use MyLittleGrammar and attach semantic actions to it.
class MyLittleGrammarAnalyzer  {
  MyLittleGrammar lg;

  void action(const char * first, const char * last) { //... }

  MyLittleGrammarAnalyzer()
  {
    lg.rule1[tr1::bind(&action, this, _1, _2)];
  }
};

I don't know if this code is exactly correct, but you get the idea.
The only thing I'm not sure about is that if I attach an action to lg.rule1 and rule1 stores a globalrule, will the action be attached
to globalrules.r1 or just to mylittlegrammar::definition::rule1?

   Oops!! I was wrong about the semantic actions part. I cannot assign semantic actions to
   MyLittleGrammar because rules are stored inside of  definition. Can anyone give me a     solution? I want to be able to reference rules of my grammars to be able to decouple parsing from actions.

Thanks in advance.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general


For ideas on reducing your carbon footprint visit Yahoo! For Good this month.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general



-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Spirit-general mailing list
Spirit-general <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/spirit-general
Joel de Guzman | 3 Nov 16:12
Picon
Favicon

Re: Writing a grammar with "subgrammars"

Hi Germán,

Just a quick note: Please limit your quote to as few lines
as possible to get the context right. Thanks!

Regards,
--

-- 
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Gmane