Gabriel Wicke | 3 May 2012 17:04
Favicon

Parsoid expanding [[:en:Barack Obama]]

Hello,

template expansion performance in Parsoid has made great strides
recently. [[:en:Barack Obama]] now expands in relatively reasonable time
and memory:

  echo '{{:Barack Obama}}' | time node parse.js  > obama.html

  42.18user 2.04system 0:46.23elapsed 95%CPU (0avgtext+0avgdata
  1528480maxresident)k
  848inputs+0outputs (8major+691968minor)pagefaults 0swaps

Sample output: http://dev.wikidev.net/gabriel/tmp/obama.html

The time utility reports the maximum resident memory inflated by a
factor of four (some bug in time afaik), so this works out to 382M
resident max which matches with top. This is measured on my Intel i3
M370 laptop, and in the same ballpark as the PHP parser running on the
cluster.

There are a few tokenizer issues visible in the output. Some of them,
such as the Navbox templates near the bottom, contain constructs like this:

{{#if:foo|<tr><th><th style="foo;|<th style="}} bar;">

Supporting this would be very messy, so the template should be changed
to use a saner nesting:

{{#if:foo|<tr><th>|}}<th style="{{#if:foo|foo;|}} bar">

(Continue reading)

Christian Williams | 4 May 2012 22:49
Favicon

Re: Editing surface/UX: Inline menus

Wikia has hackathons every quarter or so and I once built a keyboard shortcut legend interface. It could either supplement the toolbars or stand on its own. I stress, this is a hack (not localized, not optimized, may not match your keyboard layout, might not work in your browser, etc). :)


To use the keyboard shortcuts, hold Ctl-Alt-[key]. To make text bold, use Ctl-Alt-B. If you hesitate for too long (holding only Ctl and Alt), the keyboard shortcuts legend will appear.

-Christian

On Apr 23, 2012, at 7:26 PM, Erik Moeller wrote:

How can we make the Visual Editor as efficient as, or more so than,
wikitext editing when it comes to handling complex tasks like
citations, templates, etc.? I know it's a bit early to talk about
detailed aspects of the editing surface, but this may have some impact
on architectural considerations for plug-ins and such, so I wanted to
raise it on this list.

I don't think it's that much of a stretch goal to be more efficient
than wikitext editing for a lot of common cases. For example, using
complex templates manually is extremely inefficient, usually requiring
a visit to a Template: namespace to read the template documentation,
then apply the correct syntax and hope you're not making a typo in the
process.

A few models:

* Wikia's current-generation visual editor offers panels with access
to templates and categories, autocompleting as you type. If our goal
is to create an editing surface that focuses on the content and gets
out of the way (and IMO it should be), these types of panels add
clutter which is likely unacceptable.

* Most RTE editing surfaces have toolbars, and some also add desktop
application style pulldowns. The pulldown/toolbar combination has many
known usability issues as software complexity increases -- they become
cluttered, and keyboard shortcuts for anything but the stuff you use
all the time become hard to remember.

* Code IDEs use a lot of hinting/autocompletion based on intelligence
gathered from parsing the file you're editing, or any other files.
This works well when what gets rendered in the editing surface is
identical to what the user is typing. It's easy to auto-complete
"myObject.getSomeStuff()", but it's less discoverable that I should
start typing "{{Infobox country" to get a beautiful right-aligned
table.

* vi is the best example of a powerful modal editor which can perform
complex operations by essentially entering an in-editor command-line.
It's also well-known to be very difficult to learn and master, in
large part because its UX is so different from other applications
people commonly use.

What other models ought to be/are being considered?

A couple of ideas:

== Inline menus ==

You could have a menu key, say Ctrl+I or, if we want to be more
radical, a single compose key like "\", which triggers an in-editor
menu structure with associated keyboard hints.

E.g. if the user types <menu key>c, they see an overlay expanding at
cursor position:

   w - cite a website
   b - cite a book
   n - cite a newspaper
   j - cite a journal

So by typing: <menu key>cw, I could enter whichever the appropriate
dialog is for citing a website (which itself could be rendered in-line
if that gives us additional efficiency).

This type of system could intelligently trigger auto-completion where
appropriate. Say "y" is the shortcut for category, and I type:

   <menu key>yChurches in <cursor down><cursor down><cursor down><enter/tab>

to insert the category "Churches in the United Kingdom".

Or say "t" is short for template, so I could type:

   <menu key>tInfobox a<cursor down><cursor down><enter/tab>

to insert the template "Infobox album" and invoke the appropriate dialog.

One could make such a feature discoverable by adding hints to the UI
in appropriate places, e.g. if the most obvious invocation method is
through a tabbed dialog, each dialog could have a small hint
indicating how to quickly invoke it inline.

== Inline markup completion in visual mode ==

Another option would be to just intelligently detect use of existing
markup, e.g. to magically complete "[[Category:Churches in" or "{{Cite
web" or "{{Infobox co" even when in visual mode. This is beneficial
for people who already know wiki markup, but is arguably less
efficient/user-friendly than a menu/command structure that's designed
from scratch to be maximally efficient, discoverable and intuitive. It
also would tie us permanently to wiki markup.

What do you think? How can we make the visual editing surface
maximally efficient for frequent use?

All best,
Erik

--
Erik Möller
VP of Engineering and Product Development, Wikimedia Foundation

Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate

_______________________________________________
Wikitext-l mailing list
Wikitext-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

_______________________________________________
Wikitext-l mailing list
Wikitext-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l
Erik Moeller | 4 May 2012 23:11
Picon
Gravatar

Re: Editing surface/UX: Inline menus

On Fri, May 4, 2012 at 1:49 PM, Christian Williams
<christian <at> wikia-inc.com> wrote:
> Wikia has hackathons every quarter or so and I once built a keyboard
> shortcut legend interface. It could either supplement the toolbars or stand
> on its own. I stress, this is a hack (not localized, not optimized, may not
> match your keyboard layout, might not work in your browser, etc). :)

Definitely nifty! I like the affordance that it comes up if you're
hesitating for a long time, indicating that you've forgotten the
correct keyboard shortcut. :)

For VE, I think coming up with a single coherent & efficient
interaction model for completing some of these actions (e.g.
templates) will be important for it to gain acceptance among markup
editors (which in turn would help its continued evolution and
success).
--

-- 
Erik Möller
VP of Engineering and Product Development, Wikimedia Foundation

Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
Jim Safley | 19 May 2012 21:26
Picon
Gravatar

Visual Editor & WikiEditor Integration outside MediaWiki

Will I be able to implement the Visual Editor outside MediaWiki? For
instance, could I install it as a JavaScript library in a separate
application?

I notice that the original edit toolbar and the toolbar that comes
with the WikiEditor extension are coupled so tightly to MediWiki that
it's near impossible to integrate them into a separate application.
(If I'm wrong, please educate me.) I wonder if the same will apply to
the rich-text editor.

Thanks,
Jim
Trevor Parscal | 21 May 2012 21:46
Picon
Gravatar

Re: Visual Editor & WikiEditor Integration outside MediaWiki

VisualEditor is designed for MediaWiki, but technically is still fairly stand-alone. The idea of using it outside of MediaWiki seems to glaze over the back end systems that are going to be required to edit actual pages.


VisualEditor is the front-end, it does not parse Wikitext, it reads in and emits out annotated HTML. It depends on Parsoid to convert from Wikitext to annotated HTML and back again after editing. If you wanted to use VisualEditor to edit arbitrary HTML, that should work (soon) with some restrictions because our concept of annotated HTML has certain limitations including only supporting certain types of elements and not supporting all conceivable HTML-isms.

I would love to see our work on VE be used elsewhere, but it's not a central goal at this time to make it generic enough for general HTML editing.

- Trevor

On Sat, May 19, 2012 at 12:26 PM, Jim Safley <jimsafley <at> gmail.com> wrote:
Will I be able to implement the Visual Editor outside MediaWiki? For
instance, could I install it as a JavaScript library in a separate
application?

I notice that the original edit toolbar and the toolbar that comes
with the WikiEditor extension are coupled so tightly to MediWiki that
it's near impossible to integrate them into a separate application.
(If I'm wrong, please educate me.) I wonder if the same will apply to
the rich-text editor.

Thanks,
Jim

_______________________________________________
Wikitext-l mailing list
Wikitext-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

_______________________________________________
Wikitext-l mailing list
Wikitext-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l
Jim Safley | 21 May 2012 22:37
Picon
Gravatar

Re: Visual Editor & WikiEditor Integration outside MediaWiki

Thanks for the explanation. My application interfaces MediaWiki and
various content management systems using the MediaWIki API. Put
simply, it adds a textarea to a CMS where users can edit a specific
MediaWiki page. I'm looking for some editing tool that facilitates the
authoring of wiki markup in that textarea. Do you think VisualEditor,
properly configured, will be able to do this? Are there any other
options?

Jim

On Mon, May 21, 2012 at 3:46 PM, Trevor Parscal <tparscal <at> wikimedia.org> wrote:
> VisualEditor is designed for MediaWiki, but technically is still fairly
> stand-alone. The idea of using it outside of MediaWiki seems to glaze over
> the back end systems that are going to be required to edit actual pages.
>
> VisualEditor is the front-end, it does not parse Wikitext, it reads in and
> emits out annotated HTML. It depends on Parsoid to convert from Wikitext to
> annotated HTML and back again after editing. If you wanted to use
> VisualEditor to edit arbitrary HTML, that should work (soon) with some
> restrictions because our concept of annotated HTML has certain limitations
> including only supporting certain types of elements and not supporting all
> conceivable HTML-isms.
>
> I would love to see our work on VE be used elsewhere, but it's not a central
> goal at this time to make it generic enough for general HTML editing.
>
> - Trevor
>
> On Sat, May 19, 2012 at 12:26 PM, Jim Safley <jimsafley <at> gmail.com> wrote:
>>
>> Will I be able to implement the Visual Editor outside MediaWiki? For
>> instance, could I install it as a JavaScript library in a separate
>> application?
>>
>> I notice that the original edit toolbar and the toolbar that comes
>> with the WikiEditor extension are coupled so tightly to MediWiki that
>> it's near impossible to integrate them into a separate application.
>> (If I'm wrong, please educate me.) I wonder if the same will apply to
>> the rich-text editor.
>>
>> Thanks,
>> Jim
>>
>> _______________________________________________
>> Wikitext-l mailing list
>> Wikitext-l <at> lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitext-l
>
>
>
> _______________________________________________
> Wikitext-l mailing list
> Wikitext-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitext-l
>
Trevor Parscal | 21 May 2012 23:32
Picon
Gravatar

Re: Visual Editor & WikiEditor Integration outside MediaWiki

As long as VisualEditor is being used to edit a MediaWiki page, there shouldn't be much difficulty getting it to do so from any arbitrary page on the internet. VisualEditor will however use things like ResourceLoader, so it's not going to be completely stand-alone without some work.


- Trevor

On Mon, May 21, 2012 at 1:37 PM, Jim Safley <jimsafley <at> gmail.com> wrote:
Thanks for the explanation. My application interfaces MediaWiki and
various content management systems using the MediaWIki API. Put
simply, it adds a textarea to a CMS where users can edit a specific
MediaWiki page. I'm looking for some editing tool that facilitates the
authoring of wiki markup in that textarea. Do you think VisualEditor,
properly configured, will be able to do this? Are there any other
options?

Jim

On Mon, May 21, 2012 at 3:46 PM, Trevor Parscal <tparscal <at> wikimedia.org> wrote:
> VisualEditor is designed for MediaWiki, but technically is still fairly
> stand-alone. The idea of using it outside of MediaWiki seems to glaze over
> the back end systems that are going to be required to edit actual pages.
>
> VisualEditor is the front-end, it does not parse Wikitext, it reads in and
> emits out annotated HTML. It depends on Parsoid to convert from Wikitext to
> annotated HTML and back again after editing. If you wanted to use
> VisualEditor to edit arbitrary HTML, that should work (soon) with some
> restrictions because our concept of annotated HTML has certain limitations
> including only supporting certain types of elements and not supporting all
> conceivable HTML-isms.
>
> I would love to see our work on VE be used elsewhere, but it's not a central
> goal at this time to make it generic enough for general HTML editing.
>
> - Trevor
>
> On Sat, May 19, 2012 at 12:26 PM, Jim Safley <jimsafley <at> gmail.com> wrote:
>>
>> Will I be able to implement the Visual Editor outside MediaWiki? For
>> instance, could I install it as a JavaScript library in a separate
>> application?
>>
>> I notice that the original edit toolbar and the toolbar that comes
>> with the WikiEditor extension are coupled so tightly to MediWiki that
>> it's near impossible to integrate them into a separate application.
>> (If I'm wrong, please educate me.) I wonder if the same will apply to
>> the rich-text editor.
>>
>> Thanks,
>> Jim
>>
>> _______________________________________________
>> Wikitext-l mailing list
>> Wikitext-l <at> lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitext-l
>
>
>
> _______________________________________________
> Wikitext-l mailing list
> Wikitext-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitext-l
>

_______________________________________________
Wikitext-l mailing list
Wikitext-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

_______________________________________________
Wikitext-l mailing list
Wikitext-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l
Gabriel Wicke | 23 May 2012 19:06
Favicon

Parsoid web service

Hello,

I have just set up a very basic web service for Parsoid at

  http://parsoid.wmflabs.org/

It gives you the ability to browse the English Wikipedia as Parsoid
currently sees it, and provides forms for wikitext -> HTML DOM and HTML
DOM -> wikitext conversion tests.

Round-tripping is brand new and very limited, and does not support the
preservation of variable whitespace, templates and other complex
constructs yet. Currently 154 parser tests are passing in the new
--roundtrip mode that Subbu added last week.

Gabriel
Trevor Parscal | 23 May 2012 19:24
Picon
Gravatar

Re: Parsoid web service

This is awesome.


With this functionality rapidly growing in maturity, the VisualEditor will be editing real wiki pages in no time!

Great work guys, this work is critical to the success of VE.

- Trevor

On Wed, May 23, 2012 at 10:06 AM, Gabriel Wicke <wicke <at> wikidev.net> wrote:
Hello,

I have just set up a very basic web service for Parsoid at

 http://parsoid.wmflabs.org/

It gives you the ability to browse the English Wikipedia as Parsoid
currently sees it, and provides forms for wikitext -> HTML DOM and HTML
DOM -> wikitext conversion tests.

Round-tripping is brand new and very limited, and does not support the
preservation of variable whitespace, templates and other complex
constructs yet. Currently 154 parser tests are passing in the new
--roundtrip mode that Subbu added last week.

Gabriel


_______________________________________________
Wikitext-l mailing list
Wikitext-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

_______________________________________________
Wikitext-l mailing list
Wikitext-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l
Erik Moeller | 23 May 2012 20:22
Picon
Gravatar

Re: Parsoid web service

On Wed, May 23, 2012 at 10:24 AM, Trevor Parscal <tparscal <at> wikimedia.org> wrote:
> Great work guys, this work is critical to the success of VE.

Agreed. This is the first time I'm actually trying Parsoid -- it's
definitely shaping up very nicely. Thanks for setting this up.

Erik
--

-- 
Erik Möller
VP of Engineering and Product Development, Wikimedia Foundation

Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate

Gmane