headers
Thomas Gries | 1 May 2012 02:08
Picon
Favicon

Re: everything Etherpad - general status, where are we going, questions

Am 30.04.2012 20:19, schrieb Ryan Lane:
> These open more questions:
> - Can we preserve all old etherpads and URLs when switching to -lite ?
> when upgrading ?
> Etherpad-lite has a migration utility. 
https://github.com/Pita/etherpad-lite/wiki/How-to-migrate-the-database-from-Etherpad-to-Etherpad-Lite
No clue how good it is.
Permalink | Reply | 0 comments |
headers
Tim Starling | 1 May 2012 09:15
Picon

Lua: parser interface

I've written up a proposed interface between the MediaWiki parser and Lua:

<https://www.mediawiki.org/wiki/Extension:Scribunto/Parser_interface_design>

In summary: the Lua function is called with a single argument, which
is an object representing the parser interface. The object is roughly
equivalent to a PPFrame.

The object would have a property called "args", which is a table with
its "index" metamethod overridden to provide lazy-initialised access
to the parser function arguments with a brief syntax:

{{#invoke:module|func|name=value}}

function p.func(frame)
   return frame.args.name --- returns "value"
end

There would be two methods for recursive preprocessing:

* preprocess() provides basic expansion of wikitext
* callTemplate() provides an API for template invocation, since I
imagine that would otherwise be a common use case for preprocess().
Using preprocess() to expand a template with arbitrary arguments would
be difficult.

Like a normal parser function, the Lua function returns text which is
not modified any further by the preprocessor.

Please see the wiki page for a more detailed description, including
(Continue reading)

Permalink | Reply | 9 comments |
headers
Bjoern Hassler | 1 May 2012 18:03

Re: exporting from LibreOffice to MediaWiki

+1 for this one.

Also, it would be really nice if the output could be improved in terms
of formatting, as well as to cope with images.

Bjoern

On Mon, Apr 30, 2012 at 11:58 AM, Amir E. Aharoni
<amir.aharoni <at> mail.huji.ac.il> wrote:
> Hi,
>
> One of LibreOffice killer features is exporting documents in MediaWiki
> format. It's not perfect, but it's nevertheless very useful for
> converting papers written by academics and their students to Wikipedia
> articles.
>
> Unfortunately, it's broken in the latest LibreOffice releases:
> https://bugs.freedesktop.org/show_bug.cgi?id=46509
>
> Is there, by any chance, anybody here who is familiar with the world
> of LibreOffice development that can help get it fixed?
>
> --
> Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
> http://aharoni.wordpress.com
> ‪“We're living in pieces,
> I want to live in peace.” – T. Moore‬
>
> _______________________________________________
> Wikitech-l mailing list
(Continue reading)

Permalink | Reply | 0 comments |
headers
Victor Vasiliev | 1 May 2012 19:24
Picon

Re: Lua: parser interface

Thank you for bringing those issues to the public discussion, Tim,
they are really worth it.

On Tue, May 1, 2012 at 11:15 AM, Tim Starling <tstarling <at> wikimedia.org> wrote:
> I've written up a proposed interface between the MediaWiki parser and Lua:
>
> <https://www.mediawiki.org/wiki/Extension:Scribunto/Parser_interface_design>
>
> In summary: the Lua function is called with a single argument, which
> is an object representing the parser interface. The object is roughly
> equivalent to a PPFrame.
>
> The object would have a property called "args", which is a table with
> its "index" metamethod overridden to provide lazy-initialised access
> to the parser function arguments with a brief syntax:
>
> {{#invoke:module|func|name=value}}
>
> function p.func(frame)
>   return frame.args.name --- returns "value"
> end

I like this part. Also, I really enjoy the idea of making a separate
parser frame for script instead of running it in the parent template's
frame.

I am a bit leery though about the part where you suggest that
name-value arguments ({{#invoke:module|func|param=value}}) should be
parsed by engine, not the script. Don't you have to expand those
arguments in order to parse them, hence making any form of
(Continue reading)

Permalink | Reply | 3 comments |
headers
Gabriel Wicke | 1 May 2012 20:51
Favicon

Re: Lua: parser interface

On 05/01/2012 09:15 AM, Tim Starling wrote:
> In summary: the Lua function is called with a single argument, which
> is an object representing the parser interface. The object is roughly
> equivalent to a PPFrame.

+1 for the abstract frame object.

> The object would have a property called "args", which is a table with
> its "index" metamethod overridden to provide lazy-initialised access
> to the parser function arguments with a brief syntax:
> 
> {{#invoke:module|func|name=value}}
> 
> function p.func(frame)
>    return frame.args.name --- returns "value"
> end
> 
> There would be two methods for recursive preprocessing:
> 
> * preprocess() provides basic expansion of wikitext

An alternative to a wikitext-specific preprocess() method and plain-text
argument values could be a conversion / expansion method on an opaque
'parser value' object:

  frame.args.name.expandTo( 'text/x-mediawiki' ) --- returns "value"

This would make it possible to work with other formats apart from wikitext.

I recently added an API like this in Parsoid (the method is called 'as'
(Continue reading)

Permalink | Reply | 4 comments |
headers
Jeroen De Dauw | 1 May 2012 22:10
Picon
Gravatar

Git repo holding the extensions

Hey,

Setting up a git repo holding all extensions hosted by WMF was one of the
things on the git migration roadmap IIRC. AFAIK there is none yet - is this
being worked on?

Right now it's rather hard to get an overview of all changes made
everywhere and to do analytics on the overall codebase. You can of course
write a script to work with all the different repos, but that's rather less
convenient :)

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
Permalink | Reply | 1 comment |
headers
Chad | 1 May 2012 22:12
Picon

Re: Git repo holding the extensions

On Tue, May 1, 2012 at 4:10 PM, Jeroen De Dauw <jeroendedauw <at> gmail.com> wrote:
> Hey,
>
> Setting up a git repo holding all extensions hosted by WMF was one of the
> things on the git migration roadmap IIRC. AFAIK there is none yet - is this
> being worked on?
>

This is just waiting on the upgrade to 2.3 which is slated to
happen this week.

-Chad
Permalink | Reply | 0 comments |
headers
Tim Starling | 2 May 2012 02:21
Picon

Re: Lua: parser interface

On 02/05/12 03:24, Victor Vasiliev wrote:
> I am a bit leery though about the part where you suggest that
> name-value arguments ({{#invoke:module|func|param=value}}) should be
> parsed by engine, not the script. Don't you have to expand those
> arguments in order to parse them, hence making any form of
> lazy-expanding impossible?

No, you don't have to expand the arguments in order to extract equals
signs for name/value pairs. The equals signs are already identified by
the preprocessor's parser, for the purposes of lazy expansion of
template arguments. See PPFrame::newChild() and the implementation of
the #switch parser function.

[...]
> This is the part which I strongly oppose. Providing direct
> preprocessor access to Lua scripts is a bad idea. There are two key
> reasons for this:
> 1. Preprocessor is slow.

We can limit the input size, or temporarily reduce the general parser
limits like post-expand include size and node count. We can also hook
into PPFrame::expand() to periodically check for a Lua timeout, if
that is necessary.

The preprocessor is slow now, it won't become slower by allowing Lua
to call it.

> 2. You would have to work out many very subtle issues with time out
> and nested Lua scripts. This includes timeout subtleties caused by the
> preprocessor slowness (load a slow template, and given the small Lua
(Continue reading)

Permalink | Reply | 2 comments |
headers
Victor Vasiliev | 2 May 2012 03:28
Picon

Re: Lua: parser interface

On Wed, May 2, 2012 at 4:21 AM, Tim Starling <tstarling <at> wikimedia.org> wrote:
> We can limit the input size, or temporarily reduce the general parser
> limits like post-expand include size and node count. We can also hook
> into PPFrame::expand() to periodically check for a Lua timeout, if
> that is necessary.
>
> The preprocessor is slow now, it won't become slower by allowing Lua
> to call it.

What I meant is that one of the goals of Lua project is to improve the
performance of template system, and by invoking the preprocessor you
slow it down because of parser overhauls.

>> 2. You would have to work out many very subtle issues with time out
>> and nested Lua scripts. This includes timeout subtleties caused by the
>> preprocessor slowness (load a slow template, and given the small Lua
>> time limit, it will cause PHP to show a fatal error due to emergency
>> timeout; even if you fix it, the standalone version uses ulimit, and
>> it may be more difficult to fix).
>
> The scenario you give in brackets will not happen. If a Lua timeout
> occurs when the parser is executing, the Lua script will terminate
> when the parser returns control to it. The timeout is not missed.

But the parser working time would still be included in normal Lua time limit?

> It doesn't matter if there are several levels of parser/Lua recursion
> when a timeout occurs. LuaSandbox is able to unwind the stack efficiently.

What I meant is that it should be able to handle the time limit
(Continue reading)

Permalink | Reply | 1 comment |
headers
Tim Starling | 2 May 2012 04:01
Picon

Re: Lua: parser interface

On 02/05/12 04:51, Gabriel Wicke wrote:
>   frame.args.name.expandTo( 'text/x-mediawiki' ) --- returns "value"
> 
> This would make it possible to work with other formats apart from wikitext.

I can see how that would make sense when you're writing a parser, but
given the target audience for the Lua API, I think I would prefer to
provide an abbreviated interface.

How about frame.args.name as an abbreviation for
frame:getArgument('name'):expandTo( 'text/x-mediawiki' ) ?

And how about frame.plainArgs.name as an abbreviation for
frame:getArgument('name'):expandTo( 'text/plain' ) ?

> I recently added an API like this in Parsoid (the method is called 'as'
> there), and liked the way that worked out for parser functions. I am
> currently using the 'text/plain' type to retrieve a text expansion with
> comments etc stripped, and 'tokens/x-mediawiki' for expanded tokens
> (~list of tags and strings). 

I know you're not really asking for a review of Parsoid and its
interfaces, but I worry as to whether your use of text/plain to
indicate wikitext with comments stripped is appropriate.

In MediaWiki, PPFrame::expand() takes any combination of 5 boolean
flags, and modifies its behaviour based on which of the Parser's 4
output types is selected, so if you were to match it for flexibility,
you would need 128 MIME types.
(Continue reading)

Permalink | Reply | 3 comments |

Gmane