Quiring, Sam | 11 Feb 17:22 2009

AccessibleComponent_getAccessibleAtPoint(): doesn't work for all cases?

Greetings,
 
Over in the Mozilla accessibility mailing list (dev-accessibility <at> lists.mozilla.org) I asked how to determine the topmost object under the mouse, the object that would get the event if the user clicked?  Aaron Leventhal answered "MSAA and AT-SPI have a way to get the accessible object at a point".  With some help I eventually discovered the at-spi routine AccessibleComponent_getAccessibleAtPoint(), which has worked great until today.
 
I have a GTK window on the screen and the mouse is hovering over a Text box.  Below is a dump of part of the at-spi accessibility tree representing that area of the window.  The mouse is at (x,y) = (1060, 538).  This is inside the text object shown at the last line of the dump.
 
I call AccessibleComponent_getAccessibleAtPoint(...,1060, 538) on the "page tab list" object and it returns NULL.  Given the data in the dump, I can see how this might have happened: there are exactly 2 children of "page tab list" and neither contain (1060, 538):
 
child 2/2 (n:,d:,r:page tab list) Box[756,508,506,170] [actions: ] [subs: Component Selection ] (children: 2)
  child 1/2 (n:,d:,r:page tab) Box[-2147483648,-2147483648,0,0] [actions: ] [subs: Component Text ] (children: 1)
  child 2/2 (n:,d:,r:page tab) Box[756,508,0,0] [actions: ] [subs: Component Text ] (children: 1)
but if we look further, we can see that deeper descendents do contain (1060, 538), including the text box at the bottom of the dump, which is what I was looking for.
 
Is this a bug in AccessibleComponent_getAccessibleAtPoint()?  If not, is there another routine I should be calling to get the accessible object at a point?
 
-Sam
 
 
key: Box[x, y, width, height]
child 2/2 (n:,d:,r:page tab list) Box[756,508,506,170] [actions: ] [subs: Component Selection ] (children: 2)
child 2/2   - [states: enabled, sensitive, showing, visible]
  child 1/2 (n:,d:,r:page tab) Box[-2147483648,-2147483648,0,0] [actions: ] [subs: Component Text ] (children: 1)
  child 2/2 (n:,d:,r:page tab) Box[756,508,0,0] [actions: ] [subs: Component Text ] (children: 1)
    child 1/1 (n:,d:,r:filler) Box[756,508,506,170] [actions: ] [subs: Component ] (children: 2)
      child 1/2 (n:,d:,r:scroll pane) Box[762,514,494,122] [actions: ] [subs: Component ] (children: 2)
        child 1/2 (n:,d:,r:viewport) Box[762,514,476,122] [actions: ] [subs: Component ] (children: 1)
          child 1/1 (n:,d:,r:panel) Box[764,516,472,385] [actions: ] [subs: Component ] (children: 12)
            child 1/12 (n:,d:,r:panel) Box[1162,528,68,56] [actions: ] [subs: Component ] (children: 1)
            ...
            child 11/12 (n:,d:,r:panel) Box[912,528,250,25] [actions: ] [subs: Component ] (children: 1)
              child 1/1 (n:,d:,r:text) Box[912,528,250,25] [actions: activate] [subs: Action Component EditableText Text ] (children: 0)
_______________________________________________
Gnome-accessibility-devel mailing list
Gnome-accessibility-devel <at> gnome.org
http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
Nagappan A | 11 Feb 20:38 2009
Picon

Re: AccessibleComponent_getAccessibleAtPoint(): doesn't work for all cases?

Hi Sam,

2009/2/11 Quiring, Sam <Sam.Quiring <at> windriver.com>
Greetings,
 
Over in the Mozilla accessibility mailing list (dev-accessibility <at> lists.mozilla.org) I asked how to determine the topmost object under the mouse, the object that would get the event if the user clicked?  Aaron Leventhal answered "MSAA and AT-SPI have a way to get the accessible object at a point".  With some help I eventually discovered the at-spi routine AccessibleComponent_getAccessibleAtPoint(), which has worked great until today.
 
I have a GTK window on the screen and the mouse is hovering over a Text box.  Below is a dump of part of the at-spi accessibility tree representing that area of the window.  The mouse is at (x,y) = (1060, 538).  This is inside the text object shown at the last line of the dump.
 
I call AccessibleComponent_getAccessibleAtPoint(...,1060, 538) on the "page tab list" object and it returns NULL.  Given the data in the dump, I can see how this might have happened: there are exactly 2 children of "page tab list" and neither contain (1060, 538):

I have noticed incorrect value with respect to Nautilus, when I try to get the values using AccessibleComponent_getExtents. Its the value provided by individual application, where the object is located and not the job of at-spi (I guess).

Thanks
Nagappan

 
 
child 2/2 (n:,d:,r:page tab list) Box[756,508,506,170] [actions: ] [subs: Component Selection ] (children: 2)
  child 1/2 (n:,d:,r:page tab) Box[-2147483648,-2147483648,0,0] [actions: ] [subs: Component Text ] (children: 1)
  child 2/2 (n:,d:,r:page tab) Box[756,508,0,0] [actions: ] [subs: Component Text ] (children: 1)
but if we look further, we can see that deeper descendents do contain (1060, 538), including the text box at the bottom of the dump, which is what I was looking for.
 
Is this a bug in AccessibleComponent_getAccessibleAtPoint()?  If not, is there another routine I should be calling to get the accessible object at a point?
 
-Sam
 
 
key: Box[x, y, width, height]
child 2/2 (n:,d:,r:page tab list) Box[756,508,506,170] [actions: ] [subs: Component Selection ] (children: 2)
child 2/2   - [states: enabled, sensitive, showing, visible]
  child 1/2 (n:,d:,r:page tab) Box[-2147483648,-2147483648,0,0] [actions: ] [subs: Component Text ] (children: 1)
  child 2/2 (n:,d:,r:page tab) Box[756,508,0,0] [actions: ] [subs: Component Text ] (children: 1)
    child 1/1 (n:,d:,r:filler) Box[756,508,506,170] [actions: ] [subs: Component ] (children: 2)
      child 1/2 (n:,d:,r:scroll pane) Box[762,514,494,122] [actions: ] [subs: Component ] (children: 2)
        child 1/2 (n:,d:,r:viewport) Box[762,514,476,122] [actions: ] [subs: Component ] (children: 1)
          child 1/1 (n:,d:,r:panel) Box[764,516,472,385] [actions: ] [subs: Component ] (children: 12)
            child 1/12 (n:,d:,r:panel) Box[1162,528,68,56] [actions: ] [subs: Component ] (children: 1)
            ...
            child 11/12 (n:,d:,r:panel) Box[912,528,250,25] [actions: ] [subs: Component ] (children: 1)
              child 1/1 (n:,d:,r:text) Box[912,528,250,25] [actions: activate] [subs: Action Component EditableText Text ] (children: 0)

_______________________________________________
Gnome-accessibility-devel mailing list
Gnome-accessibility-devel <at> gnome.org
http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel




--
Linux Desktop (GUI Application) Testing Project - http://ldtp.freedesktop.org
http://nagappanal.blogspot.com
_______________________________________________
Gnome-accessibility-devel mailing list
Gnome-accessibility-devel <at> gnome.org
http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
Li Yuan | 12 Feb 08:28 2009
Picon

Re: AccessibleComponent_getAccessibleAtPoint(): doesn't work for all cases?

Quiring, Sam wrote:
> Greetings,
>  
> Over in the Mozilla accessibility mailing list 
> (dev-accessibility <at> lists.mozilla.org 
> <mailto:dev-accessibility <at> lists.mozilla.org>) I asked how to determine 
> the topmost object under the mouse, the object that would get the 
> event if the user clicked?  Aaron Leventhal answered "MSAA and AT-SPI 
> have a way to get the accessible object at a point".  With some help I 
> eventually discovered the at-spi 
> routine AccessibleComponent_getAccessibleAtPoint(), which has worked 
> great until today.
>  
> I have a GTK window on the screen and the mouse is hovering over a 
> Text box.  Below is a dump of part of the at-spi accessibility tree 
> representing that area of the window.  The mouse is at (x,y) = (1060, 
> 538).  This is inside the text object shown at the last line of the dump.
>  
> I call AccessibleComponent_getAccessibleAtPoint(...,1060, 538) on the 
> "page tab list" object and it returns NULL.  Given the data in the 
> dump, I can see how this might have happened: there are exactly 2 
> children of "page tab list" and neither contain (1060, 538):
>  
> child 2/2 (n:,d:,r:page tab list) Box[756,508,506,170] [actions: ] 
> [subs: Component Selection ] (children: 2)
>   child 1/2 (n:,d:,r:page tab) Box[-2147483648,-2147483648,0,0] 
> [actions: ] [subs: Component Text ] (children: 1)
>   child 2/2 (n:,d:,r:page tab) Box[756,508,0,0] [actions: ] [subs: 
> Component Text ] (children: 1)
Probably because these component doesn't have the right value. Are these 
text view?

Li
> but if we look further, we can see that deeper descendents do contain 
> (1060, 538), including the text box at the bottom of the dump, which 
> is what I was looking for.
>  
> Is this a bug in AccessibleComponent_getAccessibleAtPoint()?  If not, 
> is there another routine I should be calling to get the accessible 
> object at a point?
>  
> -Sam
>  
>  
> *_key: Box[x, y, width, height]_*
> child 2/2 (n:,d:,r:page tab list) Box[756,508,506,170] [actions: ] 
> [subs: Component Selection ] (children: 2)
> child 2/2   - [states: enabled, sensitive, showing, visible]
>   child 1/2 (n:,d:,r:page tab) Box[-2147483648,-2147483648,0,0] 
> [actions: ] [subs: Component Text ] (children: 1)
>   child 2/2 (n:,d:,r:page tab) Box[756,508,0,0] [actions: ] [subs: 
> Component Text ] (children: 1)
>     child 1/1 (n:,d:,r:filler) Box[756,508,506,170] [actions: ] [subs: 
> Component ] (children: 2)
>       child 1/2 (n:,d:,r:scroll pane) Box[762,514,494,122] [actions: ] 
> [subs: Component ] (children: 2)
>         child 1/2 (n:,d:,r:viewport) Box[762,514,476,122] [actions: ] 
> [subs: Component ] (children: 1)
>           child 1/1 (n:,d:,r:panel) Box[764,516,472,385] [actions: ] 
> [subs: Component ] (children: 12)
>             child 1/12 (n:,d:,r:panel) Box[1162,528,68,56] [actions: ] 
> [subs: Component ] (children: 1)
>             ...
>             child 11/12 (n:,d:,r:panel) Box[912,528,250,25] [actions: 
> ] [subs: Component ] (children: 1)
>               child 1/1 (n:,d:,r:text) Box[912,528,250,25] [actions: 
> activate] [subs: Action Component EditableText Text ] (children: 0)
> ------------------------------------------------------------------------
>
> _______________________________________________
> Gnome-accessibility-devel mailing list
> Gnome-accessibility-devel <at> gnome.org
> http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
>   
Quiring, Sam | 12 Feb 18:39 2009

Do all references to the same object have the same value?

Greetings,
 
My application uses AT-SPI to examine an application.  Suppose the target application has a check box on it.  If the target application's GUI does not change, will every Accessible *reference to the check box, no matter how it is obtained, have the same value?  For example, supposed I obtain a reference for the check box by enumerating the entire AT-SPI tree.  Then the user clicks on the checkbox and a reference to the check box is delivered to my handler for the "focus:" event.  Will these two references be equal?
 
-Sam
_______________________________________________
Gnome-accessibility-devel mailing list
Gnome-accessibility-devel <at> gnome.org
http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
Willie Walker | 12 Feb 18:56 2009
Picon

Re: Do all references to the same object have the same value?

If I understand your question correctly, to be guaranteed equality 
between separate events, you need to explicitly ref() the object.

Will

Quiring, Sam wrote:
> Greetings,
>  
> My application uses AT-SPI to examine an application.  Suppose the 
> target application has a check box on it.  If the target application's 
> GUI does not change, will every Accessible *reference to the check box, 
> no matter how it is obtained, have the same value?  For example, 
> supposed I obtain a reference for the check box by enumerating the 
> entire AT-SPI tree.  Then the user clicks on the checkbox and a 
> reference to the check box is delivered to my handler for the "focus:" 
> event.  Will these two references be equal?
>  
> -Sam
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Gnome-accessibility-devel mailing list
> Gnome-accessibility-devel <at> gnome.org
> http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
Willie Walker | 12 Feb 18:59 2009
Picon

Re: Do all references to the same object have the same value?

PS - Are you ready to let the world know what your mystery project is 
yet?  A fear I have is that we may all be spending lots of time helping 
you develop a proprietary product.  That might be a bummer for us open 
source folks.

Will

Quiring, Sam wrote:
> Greetings,
>  
> My application uses AT-SPI to examine an application.  Suppose the 
> target application has a check box on it.  If the target application's 
> GUI does not change, will every Accessible *reference to the check box, 
> no matter how it is obtained, have the same value?  For example, 
> supposed I obtain a reference for the check box by enumerating the 
> entire AT-SPI tree.  Then the user clicks on the checkbox and a 
> reference to the check box is delivered to my handler for the "focus:" 
> event.  Will these two references be equal?
>  
> -Sam
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Gnome-accessibility-devel mailing list
> Gnome-accessibility-devel <at> gnome.org
> http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
Quiring, Sam | 12 Feb 19:32 2009

Re: Do all references to the same object have the same value?

Will,

Sorry.  As I've told you before, my hands are tied at this point.

I'm new to the open source community so if I'm overstepping the bounds
of etiquette, please let me know.  I've searched the available
documentation as well as I know how to find answers to my questions.
Believe me, I wish I could find the answers there.  If I ask a question
that is answered in some documentation I've missed, please point me at
that documentation.

Is it the desire of open source software developers to only help open
source usages of the software?

Your email begins with "PS", was there more at the beginning?

-Sam

-----Original Message-----
From: William.Walker <at> sun.com [mailto:William.Walker <at> sun.com] 
Sent: Thursday, February 12, 2009 10:00 AM
To: Quiring, Sam
Cc: gnome-accessibility-devel <at> gnome.org
Subject: Re: [g-a-devel] Do all references to the same object have the
same value?

PS - Are you ready to let the world know what your mystery project is
yet?  A fear I have is that we may all be spending lots of time helping
you develop a proprietary product.  That might be a bummer for us open
source folks.

Will

Quiring, Sam wrote:
> Greetings,
>  
> My application uses AT-SPI to examine an application.  Suppose the 
> target application has a check box on it.  If the target application's

> GUI does not change, will every Accessible *reference to the check 
> box, no matter how it is obtained, have the same value?  For example, 
> supposed I obtain a reference for the check box by enumerating the 
> entire AT-SPI tree.  Then the user clicks on the checkbox and a 
> reference to the check box is delivered to my handler for the "focus:"
> event.  Will these two references be equal?
>  
> -Sam
> 
> 
> ----------------------------------------------------------------------
> --
> 
> _______________________________________________
> Gnome-accessibility-devel mailing list 
> Gnome-accessibility-devel <at> gnome.org
> http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
Quiring, Sam | 12 Feb 19:35 2009

Re: Do all references to the same object have the same value?

I read my mail in most-recent order.  I get the "PS" now.  Thanks for
the info.  Is this documented somewhere?

-Sam 

-----Original Message-----
From: William.Walker <at> sun.com [mailto:William.Walker <at> sun.com] 
Sent: Thursday, February 12, 2009 9:57 AM
To: Quiring, Sam
Cc: gnome-accessibility-devel <at> gnome.org
Subject: Re: [g-a-devel] Do all references to the same object have the
same value?

If I understand your question correctly, to be guaranteed equality
between separate events, you need to explicitly ref() the object.

Will

Quiring, Sam wrote:
> Greetings,
>  
> My application uses AT-SPI to examine an application.  Suppose the 
> target application has a check box on it.  If the target application's

> GUI does not change, will every Accessible *reference to the check 
> box, no matter how it is obtained, have the same value?  For example, 
> supposed I obtain a reference for the check box by enumerating the 
> entire AT-SPI tree.  Then the user clicks on the checkbox and a 
> reference to the check box is delivered to my handler for the "focus:"
> event.  Will these two references be equal?
>  
> -Sam
> 
> 
> ----------------------------------------------------------------------
> --
> 
> _______________________________________________
> Gnome-accessibility-devel mailing list 
> Gnome-accessibility-devel <at> gnome.org
> http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
Willie Walker | 12 Feb 20:03 2009
Picon

Re: Do all references to the same object have the same value?

Hi Sam:

> Is it the desire of open source software developers to only help open
> source usages of the software?

For the open source culture, my general opinion is that the 
accessibility community is very small with very few resources.  This 
leaves more room for cooperation than competition, and the culture is 
one of being open and sharing.  I also have a strong desire that time 
spent helping someone will result in positive payback to the 
accessibility community.  That's the culture that makes people help you 
for free.

So, my question to you is more of getting to when and how your work will 
contribute back to the open source accessibility community so it can 
continue to grow and help other people like you become part of it.

In addition, as the de facto lead for the GNOME accessibility project, I 
would also benefit by knowing what you're doing to figure out how it 
fits in with the overall scheme for GNOME accessibility.

Will
Willie Walker | 12 Feb 20:05 2009
Picon

Re: Do all references to the same object have the same value?

Quiring, Sam wrote:
> Is this documented somewhere?

It's kind of documented.  The Accessibility_Accessible interface extends 
the Bonobo::Unknown interface: 
http://svn.gnome.org/svn/libbonobo/trunk/idl/Bonobo_Unknown.idl.

There's some ref counting docs here:

http://svn.gnome.org/svn/libbonobo/trunk/doc/refcounting.txt

(Sorry - I always tend to read sources since docs tend to usually be out 
of sync with reality, so I might end up pointing you to raw sources and 
docs).

Will

> 
> -Sam 
> 
> -----Original Message-----
> From: William.Walker <at> sun.com [mailto:William.Walker <at> sun.com] 
> Sent: Thursday, February 12, 2009 9:57 AM
> To: Quiring, Sam
> Cc: gnome-accessibility-devel <at> gnome.org
> Subject: Re: [g-a-devel] Do all references to the same object have the
> same value?
> 
> If I understand your question correctly, to be guaranteed equality
> between separate events, you need to explicitly ref() the object.
> 
> Will
> 
> Quiring, Sam wrote:
>> Greetings,
>>  
>> My application uses AT-SPI to examine an application.  Suppose the 
>> target application has a check box on it.  If the target application's
> 
>> GUI does not change, will every Accessible *reference to the check 
>> box, no matter how it is obtained, have the same value?  For example, 
>> supposed I obtain a reference for the check box by enumerating the 
>> entire AT-SPI tree.  Then the user clicks on the checkbox and a 
>> reference to the check box is delivered to my handler for the "focus:"
>> event.  Will these two references be equal?
>>  
>> -Sam
>>
>>
>> ----------------------------------------------------------------------
>> --
>>
>> _______________________________________________
>> Gnome-accessibility-devel mailing list 
>> Gnome-accessibility-devel <at> gnome.org
>> http://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
> 

Gmane