headers
john skaller | 1 Apr 2009 02:55
Picon

Re: [felix-lang] Re: irc meetup on the future of felix


On 01/04/2009, at 3:00 AM, Emmanuel Onzon wrote:
[]

I have a view that Dypgen's technology is worth trying
to see what obstacles come up in practice.

It is already clear that GLR is not as powerful as a novice like myself
might believe from the mathematical capabilities: conflict resolution
happens at unexpected places. It seems Felix is about to undergo
some serious feature chopping and downsizing, and a move away
from trying to appeal to C/C++ programmers. This may allow some
redesign of the grammar to be a bit more consistent and less
likely to lead to conflicts. There are a couple of really bad cases
(where a C style declaration looks like an expression ..)

I'm particularly interested in second order dynamics, that is:
at the moment a *fixed* grammar is used to load the Felix
grammar. Let's call that fixed grammar the meta-grammar.

At present, the meta-grammar can't extend itself. That would
be interesting to implement! In principle this only requires the
Ocaml function calls that extend the grammar to be wrapped in
Scheme. Of course doing that makes a stable Dypgen interface
more important (because the Scheme is hard to debug).

Why would you want this? Well I have already got a bit bored
changing a basic BNF grammar into a EBNF meta-grammar,
that is, a grammar with "regular expressions" added on top.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Erick Tryzelaar | 2 Apr 2009 08:03

Irc meeting starting soon.

You can find it [here](irc://irc.freenode.net/#felix).

------------------------------------------------------------------------------
Permalink | Reply | 0 comments |
headers
Rhythmic Fistman | 2 Apr 2009 08:18
Picon
Gravatar

Re: irc meetup on the future of felix

2009/3/30 Erick Tryzelaar <idadesub@...>:
> On Mon, Mar 30, 2009 at 4:39 AM, Rhythmic Fistman
<rfistman@...> wrote:
>>> Anyone out there interested in having a meeting on the future of felix? I'm tentatively thinking
Wednesday at 7pm PST (I believe 1pm-ish on Thursday for the Australians) on [#felix](irc://irc.freenode.net/#felix).
>>
>> I'm up for it. However I think that's 4am here.
>
> Which timezone are you and skaller in? I was guessing UTC+11/AEST.
> It's currently 2:45pm UTC. I'm in UTC-7, so it's 7:45am. That means
> that it should be 1:45am in UTC+11 on Tuesday. So, if we add 12 hours,
> it'd be 7:45pm here, and 1:45pm there the next day. So, I think that
> that means that the meeting would be at:
>
> 7pm UTC-7 Wednesday
> 2am UTC Thursday
> 1pm UTC+11 Thursday
>
> But I'm happy to reschedule to work around skaller's sailing event.
>
>> P.S. How do I use Aye Are See?
>
> It depends on what os you're using. If you're doing mac stuff, then
> I'd recommend http://colloquy.info. To connect do:
>
> * File -> New Connection: enter in irc.freenode.net
> * File -> Join Chat Room: use the irc.freenode.net connection, and
> enter #felix as the chatroom.

Hey, John - you coming?
(Continue reading)

Permalink | Reply | 0 comments |
headers
Emmanuel Onzon | 3 Apr 2009 21:43
Picon

Re: irc meetup on the future of felix

2009/4/3 Erick Tryzelaar <idadesub-Rn4VEauK+AKRv+LV9MX5urNAH6kLmebB@public.gmane.org.net>
On Mar 30, 2009, at 9:24 AM, Emmanuel Onzon wrote:

I've started updating dypgen to the latest released version, and I'm running into a couple problems. First off, it looks like dypgen now generates and compiles a temporary file, and is erroring out:

This temporary file is not compiled but it is used with
  ocamlc -i
to help generate the .mli file.
Note that the flag --no-mli prevents from calling ocamlc -i
(but it doesn't prevent from creating the temporary file)
 

> build/src/compiler/dyp/dypgen/dypgen --ocamlc "-I build/src/compiler/dyp/dyplib -I build/src/compiler/sex" build/src/compiler/sex/sex_parse.dyp
File "build/src/compiler/sex/sex_parse.temp.ml", line 1, characters 0-1:
Warning X: bad source file name: "Sex_parse.temp" is not a valid module name.
File "build/src/compiler/sex/sex_parse.dyp", line 1, characters 8-22:
Error: Unbound module Sex_token

I'm not sure if we can do anything about the warning, but the error is happening because my build system can't determine that build/src/compiler/sex/sex_parse.dyp depends on build/src/compiler/sex/sex_token.mli and build/src/compiler/sex/sex_types.mli, and doesn't know that it needs to compile the .cmi file first. After manually creating it, I'm running into this:

> build/src/compiler/dyp/dypgen/dypgen --pv-obj --noemit-token-type --ocamlc -I build/src/compiler/dyp/dyplib -I build/src/compiler/flx_lex build/src/compiler/flx_lex/flx_preparse.dyp
File "build/src/compiler/flx_lex/flx_preparse.dyp", from l:103, c:0 to l:104, c:0 :
Lexing failed with message: lexing: empty token

Several incompatible changes of the syntax have been
introduced since 2007. I will look at the files to tell you more
about this error.
 

Oh, and I see that dypgen dropped support for the '--prio-pt' flag. I have to admit I'm not sure what does that flag mean. The CHANGES file mentioned that there are alternative ways of getting that functionality. So is that related to the error and what do I have to do about it?

You don't need this flag anymore, it is irrelevant. There use to be
two ways of managing priorities, but now there is only one.
I don't know if it is related to the "Lexing failed" message.
I am going to look at the file flx_preparse.dyp.




Thanks for the help.

------------------------------------------------------------------------------

_______________________________________________
Felix-language mailing list
Felix-language@...
https://lists.sourceforge.net/lists/listinfo/felix-language
Permalink | Reply | 0 comments |
headers
Emmanuel Onzon | 3 Apr 2009 21:47
Picon

Re: irc meetup on the future of felix

2009/4/3 Erick Tryzelaar <idadesub-Rn4VEauK+AKRv+LV9MX5urNAH6kLmebB@public.gmane.org.net>

> build/src/compiler/dyp/dypgen/dypgen --pv-obj --noemit-token-type --ocamlc -I build/src/compiler/dyp/dyplib -I build/src/compiler/flx_lex build/src/compiler/flx_lex/flx_preparse.dyp
File "build/src/compiler/flx_lex/flx_preparse.dyp", from l:103, c:0 to l:104, c:0 :
Lexing failed with message: lexing: empty token

The lines that are causing the error are:
 
%global_data_type <unit>
%local_data_type <unit>

But the keywords %global_data_type and %local_data_type don't
exist anymore. You can remove these two lines.
They are not needed anymore.
------------------------------------------------------------------------------

_______________________________________________
Felix-language mailing list
Felix-language@...
https://lists.sourceforge.net/lists/listinfo/felix-language
Permalink | Reply | 0 comments |
headers
Emmanuel Onzon | 3 Apr 2009 22:28
Picon

Re: irc meetup on the future of felix

2009/4/3 Erick Tryzelaar <idadesub-Rn4VEauK+AKRv+LV9MX5urNAH6kLmebB@public.gmane.org.net>

I've started updating dypgen to the latest released version,

I have looked quickly at flx_parse.dyp:

The constructor Keep_grammar has no argument anymore,
you must replace :
    Keep_grammar true
with just
    Keep_grammar

The constructor Priority_data doesn't exist anymore.
It is replaced with Relation. Adding new priorities is much
simpler, you don't have to use insert_priority, you just use
the priority in a new rule or in a relation (with the string
of its name).

In the following code:

let priority_data =
        fold_left (fun pd prios ->
          let pd,plist = fold_left (fun (pd,plist) pri ->
            let pd,pr = insert_priority pd pri in
            pd,(pr::plist)
          ) (pd,[]) prios
          in
          match plist with
          | []
          | [_] -> pd
          | _ ->  add_list_relations pd (rev plist)
        ) dyp.priority_data prios
      in

      let add_rules =
 List.map (extend_grammar dyp priority_data) rules in
      let local_data =
 { m with loaded_dssls = to_install <at> m.loaded_dssls } in
      (0,sr,Snull),[Add_rules add_rules; Local_data local_data; Priority_data priority_data]

You can remove entirely the block:
    let priority_data = ... in
And just replace:
    Priority_data priority_data
with:
    Relation prios

------------------------------------------------------------------------------

_______________________________________________
Felix-language mailing list
Felix-language@...
https://lists.sourceforge.net/lists/listinfo/felix-language
Permalink | Reply | 34 comments |
headers
Emmanuel Onzon | 3 Apr 2009 23:08
Picon

Re: irc meetup on the future of felix

More about the update of dypgen:

The function define_scheme is called by extend_grammar which is called by a user action of the parser.

In define_scheme the following block is outdated:

let lhs =
    try dyp.add_nt name "Obj_sexpr"
    with Dyp.Constructor_mismatch _ -> failwith ("CONSTRUCTOR MISMATCH " ^ name)
  in

add_nt is not available anymore. To bind a constructor to a new non terminal you use Bind_to_cons in the list of parser commands that is returned by the user action:

    Bind_to_cons [(name,"Obj_sexpr")]

In define_scheme:

    match o with
      | STRING _ ->
        Dyp.Ter Dyp_symbols.t_NAME
      | SCHEME_CODE _ ->
        Dyp.Ter Dyp_symbols.t_NAME

Terminal symbols are now just strings instead of int. The previous code must be changed:

    match o with
      | STRING _ ->
        Dyp.Ter "NAME"
      | SCHEME_CODE _ ->
        Dyp.Ter "NAME"

Non terminals too became strings:

      | NONTERMINAL (sr,s,p) ->
          let nt = mapnt s in
          let nt =
            try dyp.add_nt nt "Obj_sexpr"
            with Dyp.Constructor_mismatch _ -> failwith ("CONSTRUCTOR MISMATCH " ^ s)
            | Dyp.Bad_constructor _ -> failwith ("BAD CONSTRUCTOR FOR " ^ s)
          in
          let ntpri = cal_priority priority_data p in
          Dyp.Non_ter (nt,ntpri)

becomes:

      | NONTERMINAL (sr,s,p) ->
          let nt = mapnt s in
          let ntpri = cal_priority p in
          Dyp.Non_ter (nt,ntpri)

and cal_priority must become:

let cal_priority p =
  match p with
  | `No_prio -> No_priority
  | `Eq_prio p -> Eq_priority p
  | `Less_prio p -> Less_priority p
  | `Lesseq_prio p -> Lesseq_priority p
  | `Greater_prio p -> Greater_priority p
  | `Greatereq_prio p -> Greatereq_priority p

Instead of:

let cal_priority priority_data p =
  let cp p =
    try find_priority priority_data p
    with Not_found -> failwith ("Priority " ^ p ^ " not defined!")
  in
  match p with
  | `No_prio -> No_priority
  | `Eq_prio p -> Eq_priority (cp p)
  | `Less_prio p -> Less_priority (cp p)
  | `Lesseq_prio p -> Lesseq_priority (cp p)
  | `Greater_prio p -> Greater_priority (cp p)
  | `Greatereq_prio p -> Greatereq_priority (cp p)

Off course it is probably better to code with the regular variants directly instead of the polymorphic ones and get rid of cal_priority.

Watch out that in the new code:

      | NONTERMINAL (sr,s,p) ->
          let nt = mapnt s in
          let ntpri = cal_priority p in
          Dyp.Non_ter (nt,ntpri)

The binding of nt to "Obj_sexpr" has disappeared (because it is not done with add_nt anymore). You must not forget to bind this nt to "Obj_sexpr" with Bind_to_cons if nt is new. This must be done when the calling user action returns its parser command list.

This:
      | s ->
        let name = Dyp_symbols.str_token s in
        Dyp.Ter (Dyp_symbols.get_token_name s)
becomes:
      | s -> Dyp.Ter s

This code:

  let priority = match prio with
  | `Default -> Dyp_priority_data.default_priority
  | `Priority p ->
    try find_priority priority_data p
    with Not_found -> clierr sr ("Priority " ^ p ^ " not found")
  in
  let rule  = lhs,(List.map f rhs),priority in

Becomes:

  let priority = match prio with
  | `Default -> "default_priority"
  | `Priority p -> p
  in
  let rule  = lhs,(List.map f rhs),priority,[] in

Remark the empty list at the end of the 4-tuple that makes the rule. It is a list of options for the rule (it is only useful when the lexer generator is dypgen see 6.1 of the manual for more info).

In addition to the new rule and the new action, define_scheme and extend_grammar should return the list of string couples (nt,cons) where nt is a new non terminal bind to the constructor cons, to allow the user action to return the appropriate command Bind_to_cons.

------------------------------------------------------------------------------

_______________________________________________
Felix-language mailing list
Felix-language@...
https://lists.sourceforge.net/lists/listinfo/felix-language
Permalink | Reply | 9 comments |
headers
john skaller | 4 Apr 2009 05:23
Picon

Re: irc meetup on the future of felix


On 04/04/2009, at 7:08 AM, Emmanuel Onzon wrote:

> and cal_priority must become:
>
> let cal_priority p =
>   match p with
>   | `No_prio -> No_priority
>   | `Eq_prio p -> Eq_priority p
>   | `Less_prio p -> Less_priority p
>   | `Lesseq_prio p -> Lesseq_priority p
>   | `Greater_prio p -> Greater_priority p
>   | `Greatereq_prio p -> Greatereq_priority p
>
> Instead of:
>
> let cal_priority priority_data p =
>   let cp p =
>     try find_priority priority_data p
>     with Not_found -> failwith ("Priority " ^ p ^ " not defined!")
>   in
>   match p with
>   | `No_prio -> No_priority
>   | `Eq_prio p -> Eq_priority (cp p)
>   | `Less_prio p -> Less_priority (cp p)
>   | `Lesseq_prio p -> Lesseq_priority (cp p)
>   | `Greater_prio p -> Greater_priority (cp p)
>   | `Greatereq_prio p -> Greatereq_priority (cp p)
>
> Off course it is probably better to code with the regular variants  
> directly instead of the polymorphic ones and get rid of cal_priority.

I think there is a reason to decouple the value used by Felix and that  
used by Dypgen:
it keeps the typing decoupled. That my be important, remember Felix  
may *store*
grammar productions (including any priority relations) on disk and  
reload them.

>
> Remark the empty list at the end of the 4-tuple that makes the rule.  
> It is a list of options for the rule (it is only useful when the  
> lexer generator is dypgen see 6.1 of the manual for more info).
>
> In addition to the new rule and the new action, define_scheme and  
> extend_grammar should return the list of string couples (nt,cons)  
> where nt is a new non terminal bind to the constructor cons, to  
> allow the user action to return the appropriate command Bind_to_cons.
>

What happens if the string is not new? add_nt just ignored it so it  
was always safe..

--
john skaller
skaller@...

------------------------------------------------------------------------------
Permalink | Reply | 1 comment |
headers
john skaller | 4 Apr 2009 05:27
Picon

Re: irc meetup on the future of felix


On 04/04/2009, at 6:28 AM, Emmanuel Onzon wrote:
> In the following code:
>
> let priority_data =

> You can remove entirely the block:
>     let priority_data = ... in
> And just replace:
>     Priority_data priority_data
> with:
>     Relation prios
>

The thing to note is that priorities didn't work before (dynamically)  
so the
code to use them never got debugged properly. The Felix grammar used
to allow encoding priorities somehow but I don't think this ever
got finished and it made the (meta) parser difficult.

--
john skaller
skaller@...

------------------------------------------------------------------------------
Permalink | Reply | 33 comments |
headers
Erick Tryzelaar | 4 Apr 2009 05:40
Picon

Re: irc meetup on the future of felix

On Apr 3, 2009, at 8:27 PM, john skaller wrote:

>
> On 04/04/2009, at 6:28 AM, Emmanuel Onzon wrote:
>> In the following code:
>>
>> let priority_data =
>
>> You can remove entirely the block:
>>    let priority_data = ... in
>> And just replace:
>>    Priority_data priority_data
>> with:
>>    Relation prios
>>
>
>
> The thing to note is that priorities didn't work before (dynamically)
> so the
> code to use them never got debugged properly. The Felix grammar used
> to allow encoding priorities somehow but I don't think this ever
> got finished and it made the (meta) parser difficult.

John and Emmanuel, I just got things to compile but it doesn't seem to  
work. Mind if I commit it and could you see what I did wrong?

------------------------------------------------------------------------------
Permalink | Reply | 32 comments |

Gmane