headers
Neil Kandalgaonkar | 11 Aug 2010 04:01
Picon

Altering message string output for Javascript

I've been trying to figure out if it's possible to slightly change how 
message strings are processed, for the particular case of sending them 
to a Javascript-heavy application.

The WMF has been working on a lot of interface improvements, and these 
things necessarily lean heavily on Javascript.

There are many cases where we want to do some late message parsing on 
the client, like

    '[$1 Click here to advance to the next item]'
    'You have uploaded $1 {{PLURAL:$1|file|files}}'

However there is a lot of other kinds of parsing that could and should 
be done on the server, like

    'From {{SITENAME}}'
    '[[Special:SpecialPages|{{int:specialpages}}]]'

So we have two options, as far as I can tell:

OPTION 1:

Include a wikitext parser in Javascript, that also knows how to fetch 
int:strings and other such values when it needs them. This is 
undesirable for obvious reasons.

Michael Dale actually did a lot of this already for his JS2 project, and 
his solution works. But I'm taking another look at the problem.
(Continue reading)

Permalink | Reply | 8 comments |
headers
Trevor Parscal | 11 Aug 2010 09:51
Picon
Gravatar

Re: Altering message string output for Javascript

  On 8/11/10 12:40 AM, Roan Kattouw wrote:
> 2010/8/11 Neil Kandalgaonkar<neilk <at> wikimedia.org>:
>> OPTION 2:
>>
>> Figure out some way to alter parsing on the server for Javascript messages,
>> such that things like {{SITENAME}} are parsed, but {{PLURAL}} isn't (or, the
>> parse results emits an identical {{PLURAL}} template call).
>>
> What I think would be the easiest solution is to simply /mark/
> messages as needing server-side parsing. If such a flag is set, we'd
> run the message through parsemag (or maybe even parse, in some cases)
> before sending them to the client; if not, we'd send it to the client
> verbatim and the client-side parser (with its very limited wikitext
> support) would handle it.
>
> Roan Kattouw (Catrope)
Just wanted to add some clarity.

The point here is not that detecting which messages need to be parsed 
fully or partially is hard, it's that extending the parser to render 
{{PLURAL}} and {{GENDER}} differently has been proving to be 
problematic. The concept is, some processing could be differed to the 
client, such as which plural or gender case to use, in which case we 
would want to process the entire message, except {{PLURAL}} and {{GENDER}}.

Marking messages as "render me on the server" or "render me on the 
client" doesn't solve the problem - we want to process many messages on 
both, such as "{{int:something}} tests {{PLURAL:$1|this thing|these $1 
things}}", which would be transformed to "Something tests 
{{PLURAL:$1|this|these $1 things}}", so the client can finish processing
(Continue reading)

Permalink | Reply | 0 comments |
headers
Michael Dale | 11 Aug 2010 10:39
Picon
Gravatar

Re: Altering message string output for Javascript

On 08/10/2010 07:01 PM, Neil Kandalgaonkar wrote:
>
> OPTION 2:
>
> Figure out some way to alter parsing on the server for Javascript 
> messages, such that things like {{SITENAME}} are parsed, but 
> {{PLURAL}} isn't (or, the parse results emits an identical {{PLURAL}} 
> template call).
>
> Then a much simpler parser -- probably based on regular expressions -- 
> can be deployed to the client.
>
> Roan Kattouw and Trevor Parscal are working on a new Resource Loader 
> that may be able to do some of the above. Also, Niklaus Laxstrom is 
> working on a new Message class that looks like it will be far more 
> amenable to this sort of thing.
>

One potential problems with simple regex is msg key values like:
'category-subcat-count'          => '{{PLURAL:$2|This category has only 
the following subcategory.|This category has the following 
{{PLURAL:$1|subcategory|$1 subcategories}}, out of $2 total.}}

If you can do a regEx that will work for such substitution that would be 
great, or the server could do some transform that represented the 
transforms in JSON so that the template parsing could be avoided .. ( If 
thats the goal ) then it could relatively easily be done that way. The 
JSON representation the server creates ends up looking a lot like the 
intermediate object the client side template parser creates ( more 
verbose than the wikitext )... or ... we can just do the template
(Continue reading)

Permalink | Reply | 7 comments |
headers
Niklas Laxström | 11 Aug 2010 15:22
Picon
Gravatar

Re: Altering message string output for Javascript

On 11 August 2010 11:39, Michael Dale <mdale <at> wikimedia.org> wrote:
> In general direct substitutions like {{SITENAME}} should move towards
> normal $1 substitution with a uniform way to package php variables into
> javascript.

Why? We can and should just replace those in the server side.

--

-- 
Niklas Laxström
Permalink | Reply | 1 comment |
headers
Neil Kandalgaonkar | 11 Aug 2010 19:43
Picon

Re: Altering message string output for Javascript

There are a lot of apparent benefits to doing it all in Javascript, but 
it still bothers me because it's a huge violation of Don't Repeat Yourself.

You're probably right that we will want a wikitext parser on the client 
someday, especially for the editor. So perhaps I'm struggling for no 
good reason.

But, duplicating all the parser functionality *and* transferring the 
entire state of the MediaWiki server to the client still seems like the 
wrong thing.

I have some more radical ideas like making the parser emit JS closures 
when appropriate, but those are probably even more unrealistic until 
Niklaus' new Message class lands (and maybe not even then).

Anyway, after denting my head on my desk a few times I'm starting to 
think that wfMsg* is just unworkable for this idea.

So unless anyone else has any bright ideas I'll just give up and go with 
Michael Dale's approach of sending a parser along, at least for now.

On 8/11/10 1:39 AM, Michael Dale wrote:
> On 08/10/2010 07:01 PM, Neil Kandalgaonkar wrote:
>>
>> OPTION 2:
>>
>> Figure out some way to alter parsing on the server for Javascript
>> messages, such that things like {{SITENAME}} are parsed, but
>> {{PLURAL}} isn't (or, the parse results emits an identical {{PLURAL}}
>> template call).
(Continue reading)

Permalink | Reply | 4 comments |
headers
Platonides | 11 Aug 2010 23:51
Picon

Re: Altering message string output for Javascript

> it seems that this won't work either since so many of the 
> ParserOptions are retrieved from global settings in strange ways.

Strange? They are loaded from either initialiseFromUser() or one of its
setters...

> If you can do a regEx that will work for such substitution that would be 
> great, or the server could do some transform that represented the 
> transforms in JSON so that the template parsing could be avoided ..

I remember some time ago we discussed in wikitech-l the possibility of
doing a late replace of plural: and {{sitename}} to have the rest of the
message parsed in a cache.

It may be worth to bring back that idea if such light parser is going to
be done in JS too.

For transforming the templates into a JSON tree, the DOM preprocessor
may be easy to transform.
Permalink | Reply | 3 comments |
headers
Michael Dale | 12 Aug 2010 10:03
Picon
Gravatar

Re: Altering message string output for Javascript

On 08/11/2010 06:22 AM, Niklas Laxström wrote:
> On 11 August 2010 11:39, Michael Dale<mdale <at> wikimedia.org>  wrote:
>    
>> In general direct substitutions like {{SITENAME}} should move towards
>> normal $1 substitution with a uniform way to package php variables into
>> javascript.
>>      
> Why? We can and should just replace those in the server side.
>
>    

Because we should avoid lots of wikitext in messages, and variable 
substitution gives us better cache expire management. {{SITENAME}} is 
not such a good example since its pretty static .. but in general its my 
understanding that lots of wikitext template calls in interface msgs 
should be avoided if possible.

--michael
Permalink | Reply | 0 comments |
headers
Michael Dale | 12 Aug 2010 10:12
Picon
Gravatar

Re: Altering message string output for Javascript

On 08/11/2010 02:51 PM, Platonides wrote:
> I remember some time ago we discussed in wikitech-l the possibility of
> doing a late replace of plural: and {{sitename}} to have the rest of the
> message parsed in a cache.
>
> It may be worth to bring back that idea if such light parser is going to
> be done in JS too.
>
> For transforming the templates into a JSON tree, the DOM preprocessor
> may be easy to transform.
>
>    
I think the we should do server side msg parse and cache for everything 
it can, with variable substitution arguments like {{PLURAL:$1|value 1| 
value 2}} being delivered to the client in a way that it can manage 
relevant in-line substitutions. I am open to that template being 
represented as a wikitext string or as a JSON tree. I lean towrds 
wikitext string since it will be helpful for javascript to be able to 
deal with 'light' wikitext strings.

--michael
Permalink | Reply | 2 comments |
headers
Petr Kadlec | 12 Aug 2010 13:30
Picon

Re: Altering message string output for Javascript

On 12 August 2010 10:12, Michael Dale <mdale <at> wikimedia.org> wrote:
> I think the we should do server side msg parse and cache for everything
> it can, with variable substitution arguments like {{PLURAL:$1|value 1|
> value 2}} being delivered to the client in a way that it can manage
> relevant in-line substitutions.

I might be stating the obvious, but please do not forget that
{{PLURAL}} handling (and number of its parameters) is
language-dependent. The client-side JavaScript code handling this
would have to be defined in LanguageXx.php.

-- [[cs:User:Mormegil | Petr Kadlec]]
Permalink | Reply | 1 comment |
headers
Neil Kandalgaonkar | 12 Aug 2010 20:12
Picon

Re: Altering message string output for Javascript

On 8/12/10 4:30 AM, Petr Kadlec wrote:

> I might be stating the obvious, but please do not forget that
> {{PLURAL}} handling (and number of its parameters) is
> language-dependent. The client-side JavaScript code handling this
> would have to be defined in LanguageXx.php.
>

Michael created partial analogs of each LanguageXx.php as LanguageXx.js.

 
http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/JS2Support/mwEmbed/languages/classes/

--

-- 
Neil Kandalgaonkar   ) <neilk <at> wikimedia.org>
Permalink | Reply | 0 comments |

Gmane