headers
Eli Zaretskii | 16 Jan 17:54
Picon

Handling invisible text in bidirectional display

This is another design decision I needed to make -- how to handle
invisible text in the display engine modified to support bidirectional
text.

The current unidirectional display code simply skips all the
characters that have the invisible property (or are covered by an
overlay with such a property).  The iteration then resumes at the
first character that does not have the invisible property.  Thus,
Emacs behaves as if the invisible characters simply did not exist in
the buffer.

But with bidirectional display, the invisible property can begin
and/or end in the middle of text that is reordered for display.  For
example, suppose the buffer includes the following text (capital
letters stand for right-to-left characters):

  abcABCxyz

This would normally be displayed like this:

  abcCBAxyz

Now imagine that we have an invisible property on characters C and x.
The current code would skip C and x, and stop on y.  But because C is
encountered when moving backwards from C to A, that causes us to miss
A and B entirely and display

  abcyz

which is clearly incorrect.  There are additional complications when
(Continue reading)

Permalink | Reply | 31 comments |
headers
David Kastrup | 16 Jan 20:15
X-Face
Picon
Picon

Re: Handling invisible text in bidirectional display

Eli Zaretskii <eliz <at> gnu.org> writes:

> This is another design decision I needed to make -- how to handle
> invisible text in the display engine modified to support bidirectional
> text.

I think a reasonable model is to display the text as if the invisible
characters are not there.

Invisibility has other consequences except display, like what cursor
positions it allows.  Have not thought about those.

--

-- 
David Kastrup
Permalink | Reply | 8 comments |
headers
Eli Zaretskii | 16 Jan 21:32
Picon

Re: Handling invisible text in bidirectional display

> Date: Sat, 16 Jan 2010 19:43:47 +0100
> From: martin rudalics <rudalics <at> gmx.at>
> CC: emacs-devel <at> gnu.org, emacs-bidi <at> gnu.org
> 
> But what should we do with overlays with a display property?

If you mean how would their characters be reordered for display, I
described that in my mail Re: "Bidirectional editing in Emacs -- main
design decisions", back in October.  I reproduce the relevant excerpt
below.

If you mean overlays with display properties in some combination with
invisible text, please describe the situation in more detail, because
I'm not sure I see the difficulty.

Here's what I wrote in the mail mentioned above:

6. Reordering of strings from `display' properties

   Strings that are values of `display' text properties and overlay
   properties are reordered individually.  This matters when such
   properties cover adjacent portions of buffer text, back to back.
   For example, PROP1 is associated with buffer positions P1 to P2,
   and PROP2 immediately follows it, being associated with positions
   P2 to P3.  The current design calls for reordering the characters
   of the strings that are the values of PROP1 and PROP2 separately.
   An alternative would be to feed them concatenated into the
   reordering algorithm, in which case the characters coming from
   PROP2 could end up displayed before (to the left) of the characters
   coming from PROP1.  However, this alternative requires a major
(Continue reading)

Permalink | Reply | 6 comments |
headers
Eli Zaretskii | 16 Jan 21:37
Picon

Re: Handling invisible text in bidirectional display

> From: David Kastrup <dak <at> gnu.org>
> Date: Sat, 16 Jan 2010 20:15:35 +0100
> Cc: emacs-bidi <at> gnu.org
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > This is another design decision I needed to make -- how to handle
> > invisible text in the display engine modified to support bidirectional
> > text.
> 
> I think a reasonable model is to display the text as if the invisible
> characters are not there.

I think the resulting change of the visual order will surprise the
users.  It also complicates implementation, which for me is an
important downside.

> Invisibility has other consequences except display, like what cursor
> positions it allows.

For now, I have only implemented strictly logical cursor motion.  That
is, when the next character in the logical (a.k.a. reading) order is
far away on display, C-f causes the cursor jumps to that place, then
subsequent C-f's cause it move to the left.  With this cursor motion,
invisible text in a bidi buffer behaves exactly like it does in Emacs
23.
Permalink | Reply | 7 comments |
headers
Eli Zaretskii | 16 Jan 21:53
Picon

Re: Handling invisible text in bidirectional display

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Date: Sat, 16 Jan 2010 14:56:10 -0500
> Cc: emacs-bidi <at> gnu.org, emacs-devel <at> gnu.org
> 
> > This is another design decision I needed to make -- how to handle
> > invisible text in the display engine modified to support bidirectional
> > text.
> 
> Great to hear you progress on bidi support!

I never stopped.  Progress is slow, but steady.

Things that work so far:

 . Display of bidirectional text, including line truncation and
   continuation
 . Scrolling
 . Cursor positioning according to point
 . Text insertion and deletion
 . Left-to-right and right-to-left paragraphs, including dynamically
   determined paragraph direction
 . Horizontal cursor motion
 . Text properties and overlays that affect faces

All this works only on a text terminal for now.  I know very little
about the GUI display back-ends we support, so others will have to
come on board and make the necessary changes there, when the time
comes.

> > Therefore, I decided to modify the code which skips invisible
(Continue reading)

Permalink | Reply | 12 comments |
headers
Richard Stallman | 17 Jan 17:04
Picon
Picon

Re: Handling invisible text in bidirectional display

    > Great to hear you progress on bidi support!

    I never stopped.  Progress is slow, but steady.

    Things that work so far:

Could you post a TODO list of the things not implemented?
Permalink | Reply | 6 comments |
headers
Richard Stallman | 17 Jan 17:05
Picon
Picon

Re: Handling invisible text in bidirectional display

Your description of the interaction of invisible with bidi seems right
to me, but I was surprised by your response to this

    > I think a reasonable model is to display the text as if the invisible
    > characters are not there.

    I think the resulting change of the visual order will surprise the
    users.  It also complicates implementation, which for me is an
    important downside.

because I don't see a differnce between "display the text as if the
invisible characters are not there" and what you described (in the
no-ellipsis case).  What you described

    One consequence is that a run of invisible characters can now be split
    into several non-contiguous runs.  For example, this text:

      abcABCxyz

    with c and A covered by an invisible property will be displayed as

      abCBxyz

seems to agree entirely with that description.  Is there something I
have misunderstood?

However, the invisible characters do have some effect when they
generate an ellipsis.  In that case, multiple ellipses as you've
described seems right.
(Continue reading)

Permalink | Reply | 6 comments |
headers
Eli Zaretskii | 17 Jan 19:24
Picon

Re: Handling invisible text in bidirectional display

> From: Richard Stallman <rms <at> gnu.org>
> CC: monnier <at> iro.umontreal.ca, emacs-bidi <at> gnu.org, emacs-devel <at> gnu.org
> Date: Sun, 17 Jan 2010 11:04:13 -0500
> 
>     > Great to hear you progress on bidi support!
> 
>     I never stopped.  Progress is slow, but steady.
> 
>     Things that work so far:
> 
> Could you post a TODO list of the things not implemented?

I will do that in a few days, but please note that I don't have an
exhaustive list of all the jobs that need to be done, or even
something close.  Just to make such a list is a very large job which
involves examining many features for possible implicit reliance on
text direction.

What I'm trying to do is prepare some minimal infrastructure upon
which others could build more complex features and applications.  But
even for that much smaller job, the list of things to take care is
highly dynamic, because I find out about hidden problems as I go
along.  I keep notes about issues I discovered; I will use that to
generate some kind of TODO.
Permalink | Reply | 3 comments |
headers
Eli Zaretskii | 17 Jan 20:06
Picon

Re: Handling invisible text in bidirectional display

> Date: Sun, 17 Jan 2010 09:59:17 +0100
> From: martin rudalics <rudalics <at> gmx.at>
> CC: emacs-devel <at> gnu.org, emacs-bidi <at> gnu.org
> 
>  >   abcABCxyz
> 
> Suppose I put an overlay from the position between "b" and "c" to the
> position between "A" and "B" (just like your invisibility property) and
> give that overlay a display property, say the character "D".
> 
>  > This would normally be displayed like this:
>  >
>  >   abcCBAxyz
> 
> Should we now see "abDxyz", "abDAxyz", "abDCBDxyz", or something else?

"abDCBxyz", I believe.  c and A are displayed as D, but the rest of
the characters should be visible.

> Note that the "D" is quite similar to the ellipses for invisible text
> you mentioned earlier.

No, it's different.  The ellipsis shows that some text is hidden, so
if we have several stretches of hidden text, it is reasonable to show
an ellipsis for each stretch.  By contrast, a display property says,
in effect, to display the string _instead_ of the entire text covered
by the overlay, so it should be displayed only once.

> BTW, I'm not even sure whether
>
(Continue reading)

Permalink | Reply | 4 comments |
headers
Eli Zaretskii | 17 Jan 20:29
Picon

Re: Handling invisible text in bidirectional display

> From: Richard Stallman <rms <at> gnu.org>
> CC: dak <at> gnu.org, emacs-bidi <at> gnu.org, emacs-devel <at> gnu.org
> Date: Sun, 17 Jan 2010 11:05:03 -0500
> 
> Your description of the interaction of invisible with bidi seems right
> to me, but I was surprised by your response to this
> 
>     > I think a reasonable model is to display the text as if the invisible
>     > characters are not there.
> 
>     I think the resulting change of the visual order will surprise the
>     users.  It also complicates implementation, which for me is an
>     important downside.
> 
> because I don't see a differnce between "display the text as if the
> invisible characters are not there" and what you described (in the
> no-ellipsis case).  What you described
> 
>     One consequence is that a run of invisible characters can now be split
>     into several non-contiguous runs.  For example, this text:
> 
>       abcABCxyz
> 
>     with c and A covered by an invisible property will be displayed as
> 
>       abCBxyz
> 
> seems to agree entirely with that description.  Is there something I
> have misunderstood?
(Continue reading)

Permalink | Reply | 5 comments |

Gmane