headers
David Ponce | 1 Jan 2003 11:21
Picon

ANNOUNCE: Semantic 1.4.2 is released!

Hi All,

Semantic 1.4.2 is released!
Happy new year 2003!

David

* What's new

   This release adds namespaces support to the C/C++ grammar,
   improves docstrings, and fixes bugs found since 1.4.1.
   See the Change Log at end for details.

* Requirements

   Semantic works with Emacs 20.x, 21, XEmacs 20.x, and 21.x.

   Semantic's speedbar interface requires the latest version of
   speedbar (1.4beta4).  The Semantic Database, Semantic Analysis, and
   semantic charts tools require the latest version of EIEIO (0.17).

* Availability

   You can download Semantic, speedbar and EIEIO from the CEDET project
   home page at: <http://sourceforge.net/projects/cedet>.

   The home page of Semantic is at:
   <http://cedet.sourceforge.net/semantic.shtml>.

* Patches, bug reports & feature requests
(Continue reading)

Permalink | Reply | 0 comments |
headers
ryk | 7 Jan 2003 00:13

wisent-python.wy and wisent-python.el now uses new lexer

I finally got around to finishing the migration of python lexer to use
the new wisent-lex rather than relying on old wisent-flex.
I checked in the two files.  Not much change in the .wy file, but
the .el file is now much smaller!

It seems like wisent-flex is no longer used by any sematnic parser
after this check in.

For now I set semantic-lex-depth to nil for python so that full parse
is done.  The next thing for me to do is to use the "error" mechanism
in the wisent parser to do what java does, i.e., use something like
"block" non-terminal to rely on error handling to create semantic-list
token for inner blocks.

The rest are miscellaneous comments/questions that I jotted down in the
past few days:

* semantic-lex-c-if does not account for ``#elif''.

* semantic-lex is defined using define-overload.
  Shouldn't it be defined using defun instead?
  First no semantic code over-rides this.
  Second the body of semantic-lex simply calls the function
  bound to semantic-lex-analyzer variable, i.e., there is already
  one level of indirection via this variable.
  Do we need a second method of indirection?

* semantic-lex-keyword-p doc-string needs to be changed to
  indicate that it must return the keyword symbol rather than simply
  saying that non-nil is returned.
(Continue reading)

Permalink | Reply | 1 comment |
headers
David Ponce | 7 Jan 2003 20:56
Picon

Re: wisent-python.wy and wisent-python.el now uses new lexer

Hi Richard,

[...]
 > I finally got around to finishing the migration of python lexer to use
 > the new wisent-lex rather than relying on old wisent-flex.
 > I checked in the two files.  Not much change in the .wy file, but
 > the .el file is now much smaller!

Great.  Thanks for your effort!

 > After this check in, it seems like wisent-flex is no longer used at
 > all.  It could be now deleted if desired.

For now I would prefer to keep it ;-)

 > For now I set semantic-lex-depth to nil for python so that full parse
 > is done.  The next thing for me to do is to use the "error" mechanism
 > in the wisent parser to do what java does, i.e., use something like
 > "block" non-terminal to rely on error handling to create semantic-list
 > token for inner blocks.

Yes that would be a big improvement.  But IMO it would be even better
if we could use `semantic-repeat-parse-whole-stream' instead of the
error recovery mechanism of the LALR parser.  It is more robust and
dramatically simplify writing of grammar rules (you can easily see the
difference by comparing the wisent-java.wy and wisent-java-tags.wy
grammars!).

The main problem with python is that blocks (semantic-list) are based
on indentation instead of open/close brackets, braces or
(Continue reading)

Permalink | Reply | 0 comments |
headers
Eric M. Ludlam | 9 Jan 2003 01:48
Gravatar

Re[2]: `describe-function' for lexical analyzers.

Hi David.

  Sorry 'bout the delay in my response.  This looks great.  The
  use of defun is clever.  I'm glad you knew to do that.  That will
  help a lot.

Thanks
Eric

>>> "David PONCE" <David.Ponce <at> wanadoo.fr> seems to think that:
>Hi Eric,
>
>I finally got a few time to reply to your suggestion!
>
>[...]
>>>(defmacro define-lex-analyzer (name doc condition &rest forms)
>[...]
>>>(defsubst semantic-lex-analyzer-p (symbol)
>[...]
>>>(defun semantic-lex-analyzer-at-point ()
>[...]
>>>(defadvice describe-function (around semantic-help activate)
>[...]
>
>> Regarding these functions.  This seems like a pretty logical path to
>> take given our existing implementation.  I just had a thought that
>> might be interesting.  Right now in `define-lex-analyzer', we mock
>> up a setq.  If we ALSO added this:
>> 
>>   (fest ',name `(lambda () ,doc (when ,condition , <at> forms)))
(Continue reading)

Permalink | Reply | 0 comments |
headers
Eric M. Ludlam | 9 Jan 2003 02:15
Gravatar

Re[1]: wisent-python.wy and wisent-python.el now uses new lexer

>>> ryk <at> dspwiz.com seems to think that:
>I finally got around to finishing the migration of python lexer to use
>the new wisent-lex rather than relying on old wisent-flex.
>I checked in the two files.  Not much change in the .wy file, but
>the .el file is now much smaller!

Good News!

  [ ... ]
>* semantic-lex-c-if does not account for ``#elif''.

Good find.  I'll fixed that up.  It reminded me to ask: 
Does everyone have this in their .emacs file? ;)

(add-hook 'write-file-hooks 'copyright-update)

>* semantic-lex is defined using define-overload.
>  Shouldn't it be defined using defun instead?
>  First no semantic code over-rides this.
>  Second the body of semantic-lex simply calls the function
>  bound to semantic-lex-analyzer variable, i.e., there is already
>  one level of indirection via this variable.
>  Do we need a second method of indirection?

Originally, semantic-flex was a one-size fits all lexer.  Later we
moved to the new format so that lexers could be made fast for simple
syntax, and complex for python. ;)

Anyway, old style lex configuration was set up with lots of
variables.  New lex was a custom function for the language.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Eric M. Ludlam | 9 Jan 2003 19:05
Gravatar

Re[2]: `describe-function' for lexical analyzers.

>>> "David PONCE" <David.Ponce <at> wanadoo.fr> seems to think that:
>Hi Eric,
>
>>   Sorry 'bout the delay in my response.
>
>No problem ;-)
>
>>   This looks great.  The use of defun is clever.  I'm glad you knew
>>   to do that.  That will help a lot.
>
>Thanks!  I checked the changes in.
>
>[...]
>>>* semantic-bovinate-nonterminal-db-nt doc-string refers to
>>> semantic-bovinate-nonterminal which does not exist.
>> 
>> 
>> David renamed much of the core parser functions, and this is left
>> over bits.  I think it wants to refer to `semantic-bovinate-stream',
>> but I'll have to double check.  What it is doing is certainly a
>> hack.
>
>I also checked-in a doc fix to semantic-bovine.el.
  [ ... ]

Thanks
Eric

--

-- 
          Eric Ludlam:                 zappo <at> gnu.org, eric <at> siege-engine.com
(Continue reading)

Permalink | Reply | 0 comments |
headers
ryk | 14 Jan 2003 06:26

an idea for generating semantic-list tokens in python

Hi Eric and David,

I would like to share with you a work in progress which attempts to
make python parser generate semantic-list tokens properly.

Current semantic/wisent parser employees two methods of of generating
semantic-list tokens.  (Please correct me if I'm wrong.)

One is via the wisent-skip-block function, e.g., the "block"
non-terminal found in wisent-java.wy.  This relies on writing a
grammar that intentionally forces an error condition which is handled
via wisent-skip-block function which in turn generates a semantic-list
token as part of handling the error.

In the second method, the semantic-list tokens are generated by the
lexer even before the parser begins parsing, e.g.,
define-lex-block-analyzer.

Both methods rely eventually on Emacs' built-in primitive function
called scan-lists to skip matched blocks deliniated by parenthesis,
braces and brackets.  This takes advantage of the emacs syntax tables.

Since python relies on indentation rather than braces for block
structure, the built-in scan-lists function is useless.

However, it seems like it is possible to mimick scan-lists for python
with a bit of elisp code shown below.

First, we advise scan-lists to intercept python-mode and take special
course of action.  Note that current implementation ignores the three
(Continue reading)

Permalink | Reply | 1 comment |
headers
Eric M. Ludlam | 14 Jan 2003 19:21
Gravatar

Re[1]: an idea for generating semantic-list tokens in python

Hi,

  I think what you have done is a great idea.  So much so, that you
will find a similar API for it already in semantic-ctxt.el.  Having
the lexical analyzer use the semantic-ctxt apis for block analysis
instead of a built in function also makes some sense.

  To avoid the override method overhead, the build in block analyzer
function could check to see what function is installed for any mode,
and cache and use that function during the lexical analyzer code
generation phase.  (This would unfortunately make a language file
order dependent.)

  An added benefit is new languages will have added incentive to full
out their semantic-ctxt methods, which means the analyzer will work
for more languages.

Eric

>>> ryk <at> dspwiz.com seems to think that:
>Hi Eric and David,
>
>I would like to share with you a work in progress which attempts to
>make python parser generate semantic-list tokens properly.
>
>Current semantic/wisent parser employees two methods of of generating
>semantic-list tokens.  (Please correct me if I'm wrong.)
>
>One is via the wisent-skip-block function, e.g., the "block"
>non-terminal found in wisent-java.wy.  This relies on writing a
(Continue reading)

Permalink | Reply | 0 comments |
headers
Eric M. Ludlam | 15 Jan 2003 14:06
Gravatar

Re[2]: an idea for generating semantic-list tokens in python

>>> "David PONCE" <David.Ponce <at> wanadoo.fr> seems to think that:
>Hi,
>
>>   I think what you have done is a great idea.
>
>Absolutely!  Thanks Richard!
>
>> So much so, that you will find a similar API for it already in
>> semantic-ctxt.el.  Having the lexical analyzer use the semantic-ctxt
>> apis for block analysis instead of a built in function also makes
>> some sense.
>
>As semantic-ctxt is a high level API to handle "contexts", maybe it
>would make more sense if the context API used the lexical analyzer to
>find context bounds.
  [ ... ]

Hmmm, also a clever thought.  Right now the Ctxt stuff does
everything from forward/backward block/symbol, to finding out what
local variables there are.

It is probably appropriate to move the block analysis stuff (as
identified by Richard) into the lexer in a way that the -ctxt stuff
could just call it w/out using the entire lexer.  Hmmm, I like that
idea more.  Unfortunately things like "go backward to the beginning
of this context block" is not related to what a lexical analyzer would
use.

Perhaps our lexical analysis package should define a super-set of a
syntax table that has more complex definitions in it.  Hmmm.  I'm not
(Continue reading)

Permalink | Reply | 2 comments |
headers
Eric M. Ludlam | 15 Jan 2003 16:23
Gravatar

Re[2]: an idea for generating semantic-list tokens in python

>>> "David PONCE" <David.Ponce <at> wanadoo.fr> seems to think that:
  [ ... ]
>> Perhaps our lexical analysis package should define a super-set of a
>> syntax table that has more complex definitions in it.  Hmmm.  I'm
>> not sure.
>
>I am not very at ease with Emacs syntax tables :-( It seems it isn't
>possible to use the syntax table to parse blocks based on line
>indentation.  Is it?
  [ ... ]

Correct, that is why we would need a super-set, meaning a syntax
table that does more.

The nice thing with the syntax table is that it has simple C code that
can zoom around a buffer based on that syntax.  We need something that
lets us arbitrarily choose complex "thing" delimiters, and perhaps new
classes of "thing" based on static data.  As a syntax table
contributes to regular expression evaluation, it would be nice to
devise our plan to not use regular expressions in case we have an
opportunity to port changes into Emacs C code.

And that, is a real trick.

Eric

--

-- 
          Eric Ludlam:                 zappo <at> gnu.org, eric <at> siege-engine.com
   Home: http://www.ludlam.net            Siege: www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: www.gnu.org
(Continue reading)

Permalink | Reply | 0 comments |

Gmane