shell icon indicating copy to clipboard operation
shell copied to clipboard

Screen reader/accessibility: Does not read search results

Open personalizedrefrigerator opened this issue 3 years ago • 1 comments

Summary

  • The launcher openable with super+/ doesn't read search results when a screen reader is enabled! Keyboard shortcut super+slash in Pop!_Shell Keyboard Shortcuts window
  • I am unable to determine how to get the screen reader to read search results.

Steps to reproduce

  • Enable the screen reader in settings
  • Press super+/
  • Type settings
  • Press the up and down arrows
  • Press tab
  • Press shift+tab
  • Delete everything and type =1+1
  • Press Ctrl+1

Expected Behavior

  • Screen reader should read the currently selected suggestion when I move the focus up/down with arrow keys (0:44 in the recording below).
  • Screen reader should announce changes to the input box (e.g. changing from =1+1 to =2 when I press ctrl+1)
  • Dialog/text entry should be appropriately described when it first appears (see 0:36 in the recording below)

Recording

https://user-images.githubusercontent.com/46334387/167257951-c5fbf896-3c09-411e-a8ba-28acab5996ef.mp4

To hear the screen reader in the recording above, please ensure that the video is unmuted. GitHub may mute it by default: mute button at bottom right of video

https://github.com/pop-os/shell/blob/b299641fb472f47f062d6457252b072360cedca2/src/search.ts#L323

Emitting a signal-value-changed here would make sense, but I haven't figured out how to do that...

Here's what I've found so far:

  • We probably want to be looking at Cally and Atk (documentation)
  • Calling get_accessible() on a Clutter Actor returns a Cally Actor.
    • Has/inherits set_description and set_relationship methods.
      • A Cally Actor inherits from both Atk#Object and Atk#Action, both of which have a set_description method.
        • It might be necessary to use imports.gi.Atk.Object.prototype.set_description.call(callyActor, descriptionStr) to use the one-argument version of set_description.
  • imports.gi.Atk gives access to enums like StateType.