[ft] Stray pixel from FT_Outline_Get_Bitmap()?

Hi all,
I believe I'm seeing a regression between 2.3.9 and 2.3.11 when
rendering monochrome text, resulting in a stray pixel being
rendered - See the attached screenshots.  Note that I feel this
is a regression since the 2.3.11 version looks wrong, while the
older 2.3.9 looks correct.  This is on a 64-bit Mandriva 2010.0
system (GCC 4.4.1).  Note that no problems are seen when the
text is rendered using AA.  Also, hinting is disabled.

I had initially encountered this via Cairo, but after some
debugging and discussions with them, suspicion is on the
FT_Outline_Get_Bitmap() function that Cairo uses as part of its
rasterization of text.

Unfortunately, I'm not able to give out this custom font nor view
it in ftview, so I'm at a bit of a loss as to how to best pass this
problem over to someone for further investigation - Suggestions
welcome!

Meanwhile, I can provide the raw geometry of the problematic
diamond shape (from the font file), in the hopes that it can be
used somehow:
      -35, 368   (LL)
     -111, 444   (UL)
      -22, 444   (UR)
       54, 368   (LR)
      -35, 368   (LL)
My suspicion is on the 45-degree angle the lower-right corner forms,
with that angle triggering a problem somewhere in
freetype-2.3.11/raster/ftraster.c

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

> I believe I'm seeing a regression between 2.3.9 and 2.3.11 when
> rendering monochrome text, resulting in a stray pixel being rendered
> - See the attached screenshots.

Just for reference: Have you tested 2.4.1 also?  I don't expect any
improvement, but...

> Unfortunately, I'm not able to give out this custom font nor view it
> in ftview, so I'm at a bit of a loss as to how to best pass this
> problem over to someone for further investigation - Suggestions
> welcome!

You can't view it in ftview...  Interesting.  Can you elaborate for
curious people like me?

In case it is in a standard format you might always use FontForge to
erase everything from the font but the offending shape; this would be
the easiest solution for me to debug the problem...

> Meanwhile, I can provide the raw geometry of the problematic diamond
> shape (from the font file), in the hopes that it can be used
> somehow:
>
>      -35, 368   (LL)
>     -111, 444   (UL)
>      -22, 444   (UR)
>       54, 368   (LR)
>      -35, 368   (LL)
>

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

On 04/08/10 09:54 AM, Werner LEMBERG wrote:

>> I believe I'm seeing a regression between 2.3.9 and 2.3.11 when
>> rendering monochrome text, resulting in a stray pixel being rendered
>> - See the attached screenshots.
>
> Just for reference: Have you tested 2.4.1 also?  I don't expect any
> improvement, but...

No, I haven't.  I confess that being on an RPM-based system, I
usually just use the system-provided RPMs, and I don't see anyone
packaging 2.4.x by default (yet).
If it becomes crucial, I can explore trying to upgrade...

>> Unfortunately, I'm not able to give out this custom font nor view it
>> in ftview, so I'm at a bit of a loss as to how to best pass this
>> problem over to someone for further investigation - Suggestions
>> welcome!
>
> You can't view it in ftview...  Interesting.  Can you elaborate for
> curious people like me?

Sorry, I knew what I meant, but didn't convey it well...  :P

I actually have a custom (Legacy) font file format that I'm
supporting, using a custom (Non-sanctioned) FT_Driver (Yep, it's
me again! :)  This C++ driver uses our framework, ICU, etc, so it
wouldn't be possible to redistribute it.

FWIW, I'm reasonably confident that it's not the cause of this

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

> I confess that being on an RPM-based system, I usually just use the
> system-provided RPMs, and I don't see anyone packaging 2.4.x by
> default (yet).  If it becomes crucial, I can explore trying to
> upgrade...

Well, 2.4.1 has a lot of fixes to prevent crashes with invalid fonts,
but this is probably not an important issue for you.

> - Our EM is 8000 (for some historical reason - the comments suggest
>   its related to preventing overflow).
> - Those points I gave are an integral/scaled mm size.  We take the
>   [floating point] mm size and scale it fit into the EM box.
> - For this face, the root.bbox ends up being (-4265,-4000),
>   (4265,4000)
> - In my "slot load CB", based on these inputs:
>        face->scale = 521.4579965583772
>        size->metrics.x_scale = 18579
>     I return these values for the points:
>        -131, 1382
>        -417, 1667
>         -83, 1667
>         203, 1382
> 
> I'm not sure if that's clear, or if it's enough info.  If not,
> please let me know, and I'll dig further.  Similarly, if there's
> something specific I could look at or try, please let me know...

This should be enough info, thanks.  Unfortunately, these days I'm
moving from Germany back to Austria (after almost 15 years abroad...),
so I don't have enough time to provide something by myself I could

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

On 04/08/10 11:32 AM, Werner LEMBERG wrote:

> This should be enough info, thanks.  Unfortunately, these days I'm
> moving from Germany back to Austria (after almost 15 years abroad...),
> so I don't have enough time to provide something by myself I could
> debug.

Understood - Good luck on the move!
Funny though - From Canada, both Germany and Austria are 'abroad' ;)

FWIW, this isn't critical (yet...) - I'm just pursuing it now while
I've been into it for several days, and while it's fresh in my
mind...

> However, there is this example:
>
>    http://freetype.sourceforge.net/freetype2/docs/tutorial/example3.cpp
>
> Could you adapt this to render your shape?

Seems suitable - Thanks.  I've been able to tweak the numbers+points
in there to get my parallelogram to draw.  Unfortunately, it hasn't
reproduced the problem I was seeing, so I'm going to have to look
closer at what that example is calling verses what was being done
under my Cairo case (Specifically FT_Outline_Get_Bitmap()).
I fear it may be something to do with a particular size/resolution
or something, meaning I may need virtually the same settings in order
to reproduce it...

Meanwhile, I've been looking through raster.c, and confess that it

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

> Might there be a key place that does some sort of equivalent of a
> "PutPixel(i, j, value)" that I could watch while debugging my
> Cairo-based case?  Or does the algorithm not lend itself to that?  I
> figure someone must be turning on that offending pixel, and
> detecting that case might illuminate the problem.

The two central routines are Vertical_Sweep_Span and
Horizontal_Sweep_Span in combination with the drop-out control
functions Vertical_Sweep_Drop and Horizontal_Sweep_Drop.  The generic
calls to them happen in the function Draw_Sweep.  The first pass
handles the vertical sweep, the second pass (if asked for) does a
horizontal sweep which basically only checks for drop-outs.

For each scanline, the start and end points of spans are calculated,
sorted, and passed to the *_Sweep_Span routines.  They blacken all
pixels fully covered by the spans – this is trivial – and then decide
how to render the partially covered ones and what to do with gaps –
this is tricky.  I've tried to improve the drop-out handling in 2.3.10
(the documentation in the TrueType specification is very bad
unfortunately); I've probably worsened your particular case.


    Werner

_______________________________________________
Freetype mailing list
Freetype <at> nongnu.org
http://lists.nongnu.org/mailman/listinfo/freetype

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

On 05/08/10 02:09 AM, Werner LEMBERG wrote:

>> Might there be a key place that does some sort of equivalent of a
>> "PutPixel(i, j, value)" that I could watch while debugging my
>> Cairo-based case?  Or does the algorithm not lend itself to that?  I
>> figure someone must be turning on that offending pixel, and
>> detecting that case might illuminate the problem.
>
> The two central routines are Vertical_Sweep_Span and
> Horizontal_Sweep_Span in combination with the drop-out control
> functions Vertical_Sweep_Drop and Horizontal_Sweep_Drop.  The generic
> calls to them happen in the function Draw_Sweep.  The first pass
> handles the vertical sweep, the second pass (if asked for) does a
> horizontal sweep which basically only checks for drop-outs.
>
> For each scanline, the start and end points of spans are calculated,
> sorted, and passed to the *_Sweep_Span routines.  They blacken all
> pixels fully covered by the spans – this is trivial – and then decide
> how to render the partially covered ones and what to do with gaps –
> this is tricky.  I've tried to improve the drop-out handling in 2.3.10
> (the documentation in the TrueType specification is very bad
> unfortunately); I've probably worsened your particular case.

Ok, thanks for the overview.  I've been stepping through this stuff,
adding print statements (See attached), etc, but have a couple of
questions/comments...

- Note that I cut the "A" part out of my font, so I'm only rendering
   the accent part at the top.  It seems to be 4 scanlines, matching
   some of the attached output.

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

> - Even though I'm getting different rasterization results between
>   2.3.9 and 2.3.11, I can't seem to see anything different being
>   done.  The output I attached is the same for both versions,
>   leading me to suspect I'm not looking at the relevant code.

I suggest that you build your product with both 2.3.9 and 2.3.11 and
do two parallel gdb sessions, inserting breakpoints in the sweep
functions to find out where the additional pixel gets set.  Then you
should be able to repeat it with the standalone version of the
rasterizer.

> - I found the spot(s) in Vertical_Sweep_Span() where some pixels are
>   being set (See attached), but I can't seem to find where the rest
>   of the pixels are being set.

Basically, it doesn't matter: the remaining pixels are all set to
black and are completely uninteresting.

>   I had expected the 'while' loop at the bottom to be doing it,
>   setting 0xFF into *target, but that loop is never entered...

The leftmost and rightmost 8 pixels are handled within `f1' and `f2',
respectively.  The loop is used only if the width of the current span
exceeds 8 pixels.

> - Vertical_Sweep_Drop() is never called.

Then there aren't any drop-outs to consider during the vertical sweep
phase.

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

On 06/08/10 02:47 AM, Werner LEMBERG wrote:
>> - Even though I'm getting different rasterization results between
>>    2.3.9 and 2.3.11, I can't seem to see anything different being
>>    done.  The output I attached is the same for both versions,
>>    leading me to suspect I'm not looking at the relevant code.
>
> I suggest that you build your product with both 2.3.9 and 2.3.11 and
> do two parallel gdb sessions, inserting breakpoints in the sweep
> functions to find out where the additional pixel gets set.  Then you
> should be able to repeat it with the standalone version of the
> rasterizer.

(sigh) My sincere apology for that last post, as I see now it was
completely useless...
Too many simultaneous things on the go, too many VMs, too many
hack-jobs to system libraries...  In the end, it seems I was
debugging two separate 2.3.9 builds... (Duh!)

Now, on to _real_ 2.3.9 vs 2.3.11 info...   :P

>> - Horizontal_Sweep_Drop() doesn't do anything when called (Bails
>>    early due to leftmost/rightmost stub tests)
>
> Then there aren't any drop-outs to consider during the horizontal
> sweep phase.  This is strange, however, since one of the drop routines
> should be responsible for your additional pixel, I believe.

As you suspected, Horizontal_Sweep_Drop() seems to be the culprit,
and the 2.3.11 changes seem to be producing different results in
the second/final call to it.

Re: [ft] Stray pixel from FT_Outline_Get_Bitmap()?

> As you suspected, Horizontal_Sweep_Drop() seems to be the culprit,
> and the 2.3.11 changes seem to be producing different results in the
> second/final call to it.
> 
> I'm not going to try to understand the logic, nor suggest how to fix
> it (Not my area of expertise!).  Instead, I've tried to print out
> all the relevant information [See attached] to hopefully allow you
> to ponder how to best proceed.

Thanks, will have a look soon.

> If anything isn't there, let me know and I can add more output.

As you've written, you would me help most if you could provide a
modified example3.cpp so that I can do the single-stepping by myself.

    Werner