Re: eBook Reader user interface
Marco Pesenti Gritti <mpg <at> redhat.com>
2007-05-05 08:41:51 GMT
On Fri, 2007-05-04 at 23:08 -0700, Don Hopkins wrote:
> Goal for improving the eBook reader user interface:
>
> I've been doing some exploratory programming with GTK and Sugar,
> trying to improve the user interface of the eBook reader, and make
> it useable in book mode with the gamepad.
>
> + Support the game keypads in eBook mode.
>
> + Low level game keypad support
>
> Need to remap low level keyboard scan codes to Linux keyboard codes.
>
> The setolpckeys.c program remaps the keys and gamepad buttons.
>
> Currently it maps both gamepads to the numeric keypad keys (KEY_KP8, etc),
> which the X server and GDK translates to directional keys (GDK_Up, etc).
>
> I tried to map them to buttons (BTN_A, etc), but the X server seems
> to ignore keycodes in that range.
>
> The xorg.conf file has a keycode mask that looked like it might help,
> but I couldn't get it to work.
>
> Need to have unique keycodes reported for each of the two gamepads,
> which are not the same as any keyboard keys, without any predefined meanings
> like arrow keys have.
>
> Need to define special purpose keycodes just for the OLPC gamepad,
> instead of trying to reuse existing but not appropriate keycodes.
>
> What is the process for defining new keycodes in <linux/input.h>?
>
> Here's my strawman proposal for some new keycodes.
>
> Use keys ("KEY_*") instead of buttons ("BTN_*"), since they
> seem to work better.
>
> The 0x1b* range seems to be unused in <linux/input.h>,
> and it's between other groups of keycodes, so I'll
> propose using that range for the OLPC.
>
> The UP/DOWN/LEFT/RIGHT keys correspond to the directional
> keypad.
>
> #define KEY_XO_GAMEPAD_UP 0x1b0
> #define KEY_XO_GAMEPAD_DOWN 0x1b1
> #define KEY_XO_GAMEPAD_LEFT 0x1b2
> #define KEY_XO_GAMEPAD_RIGHT 0x1b3
>
> The NORTH/SOUTH/EAST/WEST keys correspond to the other
> buttons. Those names are agnostic to the button labels,
> which may change from the current Playstation buttons
> (X/O/Triangle/Square). Can anyone suggest better names for
> the four buttons on the right?
>
> #define KEY_XO_GAMEPAD_NORTH 0x1b4
> #define KEY_XO_GAMEPAD_SOUTH 0x1b5
> #define KEY_XO_GAMEPAD_EAST 0x1b6
> #define KEY_XO_GAMEPAD_WEST 0x1b7
>
> While we're at it, we could define keycodes for the other
> OLPC buttons and switches on the screen. I think there are
> some other sensor switches that could generate keycodes,
> like opening the screen, rotating it around, and putting it
> into book mode, so I will make some guesses at names for
> them, just to get the discussion rolling.
>
> #define KEY_XO_SCREEN_ROTATE 0x1b8
> #define KEY_XO_SCREEN_POWER 0x1b9
> #define KEY_XO_SCREEN_OPEN 0x1ba
> #define KEY_XO_SCREEN_CLOSE 0x1bb
> #define KEY_XO_SCREEN_IN 0x1bc
> #define KEY_XO_SCREEN_OUT 0x1bd
>
> Is there an exhaustive list of all buttons and switches and
> events on the OLPC? Are any more planned? Which ones should
> be assigned keycodes?
>
> Rewrote setolpckeys.c code in Python (just uses ioctl, but needs to know keycodes).
> Writing utilities like that in Python instead of C makes it easier to
> reconfigure the keys on the OLPC without a C compiler.
>
> + High level action support.
>
> GTK uses "Actions" to define the actions available in an
> application, independent of the user interface used to invoke
> them. Actions can be bound to user interface widgets and
> keyboard accelerators, and they can hide, show, enable and
> disable the corresponding parts of the interface. You can
> subclass Action to define custom toolbar buttons and menu items.
>
> We need to define a generic way of navigating and executing the
> application's actions from the gamepad.
>
> We can make Sugar specific actions that create the appropriately
> styled and customized Sugar user interface widgets.
>
> Actions can be used to support navigation and operation of the
> toolbar components from the gamepad:
>
> Actions have a list of their "proxy" components (toolbar
> buttons, menu items, etc).
>
> Actions know how to execute a callback function, so the user
> interface components tell the action to activate, instead of
> calling the function themselves.
>
> The actions also know their labels, icons and tooltips.
>
> Actions can be shown and hidden, and all their proxies show and
> hide.
>
> Actions can be made sensitive or not (disabled), and all their
> proxies enable or disable.
>
> Actions have methods to construct toolbar buttons and menu
> items, which subclasses can override to customize the user
> interface. The higher level GTK UIManager calls these methods in
> actions to create the user interface, although you can still use
> Actions without the UIManager, by creating the components
> yourself.
>
> There are three standard kinds of actions: Action, which creates
> a normal button or menu item, ToggleAction, which creates a
> toggle button (checkbox), and RadioAction, which creates a radio
> button (multiple choice).
>
> The GTK toolbars and menus support keyboard navigation of
> sensitive buttons, as well as showing tooltips on sensitive
> buttons, but it won't show a tooltip on unsensitive (disabled)
> buttons, tabbing skips over disabled buttons, and it kicks you
> out of buttons that get disabled while you're using them, by
> moving the focus into the next button (whatever that happens to
> be).
>
> All inactive items should show tooltips telling WHY they are
> inactive, and WHAT you have to do before using them.
>
> Unfortunately you can't get a tooltip on an inactive item.
> This needs to be fixed, so we can display helpful tooltips and
> documentation on any item whether active or not.
>
> Unfortunlatey you can't navigate to an inactive item. This
> needs to be fixed, so the user can use the gamepad to navigate
> to an inactive item, to find out what it is and why it's
> inactive. (Also, so you can navigate the interface predicably
> with muscle memory, because the number of presses required to
> navigate doesn't change depending on the state of the
> application.)
>
> A problem with the current GTK keyboard navigation behavior of
> not allowing inactive items to have the input focus, is that it
> violates the principle of least astonishment, makes the
> interface less predictable, and interferes with type-ahead:
>
> If you're focused on the "back page" button, and select it
> until you get to the first page, then it will become inactive
> (because you can't go back from the first page), which
> results in the input focus being kicked out of the "back
> page" button and throwing you into the "next page" button.
>
> Imagine how hard this "astonishing" behavior would be to use
> if you were visually impared. It would not be very obvious
> that you had "bounced off" the first page and suddenly were
> moving forward. Other toolbars might arbitrarily relocate the
> input focus to an even more confusing button, when the one
> you're using gets disabled.
>
> It would be much less astonishing if the input focus simply
> remained in the "back page" button when you hit the first
> page, and a tooltop popped up saying "You can't go to the
> previous page, because you are at the first page."
>
> I've made a simple API for changing the sensitivity of a
> component, that takes a "reason" string (translated text) to
> show to the user in the tooltip of a disabled control (after
> the normal text). I've changed the code in the eBook reader
> that disables actions to figure out the most important reason
> (there might be several reasons to disable a control, but
> usually one is most important). It passes that reason to the
> API that disables the action, which currently just stores it
> away in a dictionary. Now it needs to be hooked up so it
> actually shows the tooltip with the reason when disabled.
>
> GTK has an "AcceleratorList" class that keeps a list of keyboard
> accelerators which invoke actions. The UIManager helps manage
> the accelerators for you automatically.
>
> I think we should use accelerators for normal keyboard
> acceleration of application actions, but implement the gamepad
> navigation stuff as a higher level framework that uses a more
> semantically meaningful model. (Not just low level tab/backtab
> focus navigation, or simple global keyboard accelerators, but
> an actual framework specifically designed to support browsing
> and executing arbitrary Actions via the game controller, and
> providing feedback about the reasons controls are disabled.)
>
> The pygtk library has some methods on action_group to add
> actions "action_group.add_actions(action_list)", toggle
> actions "action_group.add_toggle_actions(action_list)", and
> radio actions "action_group.add_radio_actions(action_list,
> cur_value, callback, *args)". These all take an action_list
> that's a list of action specification tuples. That interface
> is more brittle and less extensible than it should be, because
> it takes tuples instead of dictionaries, and it only makes
> stock GTK actions.
>
> We need a more flexible API than add_actions and its ilk, which
> supports custom Sugar actions, and uses dictionaries instead
> of tuples, so we can easily pass in additional optional
> parameters without changing the API.
>
> I have taken a first cut at rewriting pygtk's ActionGroup's
> add_actions, add_toggle_actions and add_radio_actions
> functions in Python, and changing them to use custom
> SugarAction, SugarToggleAction and SugarRadioAction classes.
>
> But I still don't think the add_actions_esque interface is
> flexible enough. It needs an easy way to add other custom
> widgets (like the search text input field), and it should make
> it easy to customize the user interface classes and configure
> the objects by passing in additional optional parameters and
> configuration dictionaries. (For example, the action
> specification should be a dictionary that has an optional key
> to specify the toolbar button and menu item classes, and it
> should be possible to plug in different kinds of user
> interface controls than toolbars and menus, like pie menus and
> gamepad specific interfaces).
>
> GTK has an "atk" module that interfaces to the accessibility toolkit.
>
> I'm not clear on just what this does and how it can help us,
> and I would love to hear from somebody who's familiar with it.
>
> I think we can do what we need to support the gamepad without
> using the atk, but maybe it could be helpful. But my
> impression is that it's more geared towards screen readers, is
> tied closely to the user interface layout (declaring that this
> label is associated with that control, etc), and it looks like
> it takes a lot of lines of code to use (unless you use some
> kind of framework that supports it automatically).
>
> GTK has a "UIManager" that knows how to build menu bars, menus,
> toolbars and buttons from XML files, and tie together actions,
> accelerators, menus and toolbars. The XML describes the
> structure and layout of the user interface, but refers to the
> actions by name to configure the buttons and menu items with
> labels, icons, tooltips, accelerators, callbacks, and custom
> component classes. There is nothing in the XML about what label,
> icon, tooltip or widget class to use -- just an Action
> name. It's up to the Action to figure out the visual appearance
> and behavior of the gui. That's why it should be easier to use
> custom Action classes.
>
> I think we need to write our own UIManager and accessibility
> framework that addresses our specific needs (like the
> focus/tooltip/disable reason issues discussed above, custom
> sugar controls, gamepad support, etc), because I don't think
> the UIManager (plus the pygtk utilities that go along with it)
> are flexible enough to support our needs. I think it would be
> easy to implement it in Python, in a way that would be a lot
> more flexible and easier to extend that the current C
> implementation of GTK's GuiManager.
GTK UIManager also does not support tabbed toolbars and palettes (which
will display the accellerators).
Marco
RSS Feed