Allow users to change the order in which object properties are reported in speech and braille

[LeonarddeR]

There have been opened some issues in the past regarding the order of speech and braille output. For example, we have issue #4629 which covers Reading Column & Row Titles before the content instead of after. Furthermore, we have #4195 and #6901 regarding displaying column and row headers in braille. To add more, I've received several questions regarding this subject from people who wanted to change the order of object speech, e.g. "to report "Not checked I agree check box" instead of "I agree checkbox not checked", etc. Personally, I would stick with the current speech order but would change the default braille order significantly, e.g. by moving cell coordinates up a hole lot. Last but not least, we have ticket #214 that covers braille verbosity

So I'd like to open this ticket as an umbrella ticket and come up with a use case which covers everything here and to use it as a discussion ticket for implementation. The idea I'd like to implement is the ability to change the order object properties are reported in speech and braille, initially using NVDA's configuration file, but preferably using the UI.

Under de hood part

For this part, here are the steps I propose:

1. Add new items to the presentation config spec for speech and braille, e.g.

   speechPropertiesOrder = string_list(max={objectPropertiesCount},default=name, role, value, description, keyboardShortcut, cellCoordsText, rowHeaderText, rowNumber, columnHeaderText, columnNumber, rowCount, columnCount, current, positionInfo)

   I have discussed other possibilities for this with @DerekRiemer, this one seems to be the most easy to implement and maintain, at least better than a dictionary kind of approach where the position is either key or value. It also allows someone to just keep some strings out of the order, which will automatically stop reporting them. This means that the underlying behaviour of reportKeyboardShortcuts, reportObjectDescriptions and reportObjectPositionInformation can be changed, just remove those from the string list

2. In speech.getSpeechTextForProperties, add all the speech parts to a dictionary, e.g. called textDict. The key is one of the strings from the config spec example above

3. Than, loop through the items in config.conf['presentation']['speechPropertiesOrder'] and add text chunks to the textList using that order. Make sure that level will always be first for tree view level changes, regardless of custom speech order

4. Thanks to @DerekRiemer who came up with this: Provide, or at least document, the fact that every new property (e.g. aria-current has been added recently) needs a config spec update. New properties can be included in every desired position in the properties order. For custom property orders, try to find newProperty-1 in the custom order and insert the new property after that item. If newProperty-1 does not exist in the custom order, try newProperty-2, newPoperty-3, etc.

UX

1. Create a new class in guiHelper called sortableListHelper. This class will add a list control to the gui, along with move up and move down buttons which change order
2. Add a sortableListHelper control to object presentation to reorder speech properties

Things that need discussion

1. There is a difference in property order between object presentation and document presentation. What to do with these?

2. Ideally, I'd like list items to be checkable (i.e. like the list control in Windows Disk cleanup). This would allow for enabling/disabling items on the fly. That would have some major impact on the gui though, e.g. three options from object presentation can be removed and cell coordinates, column and row numbers could be disabled from this new list control. I think this shouldn't be a part of the initial implementation

... Probably more

CC @JCSTeh @DKager @JosephSL

[LeonarddeR]

It seems that especially the control type reporting that is used for documents and browse mode will be quite hard, since there are lots of edge cases. For example, links have the states reported before the link, "visited link", while check boxes have the state after the role, "checkbox not checked". I think we should come up with a solution for these edge cases before further work will be invested in this.

The implementation for focussing objects won't be that difficult.

[dkager]

What would you say is the priority of this feature?

[LeonarddeR]

@dkager commented on 31 Jan 2018, 19:28 CET:

What would you say is the priority of this feature?

I belief that there are several requests in this area, see the referenced tickets. However, I think that the desire to disable some properties in braille and enable them in speech is more important. Of course, this also applies the other way around.

[Adriani90]

#7887

[Adriani90]

#7886

[LeonarddeR]

@josephsl: Thanks for pointing me at RearrangeCtrl in https://github.com/nvaccess/nvda/issues/8117#issuecomment-375843530. That will certainly make things easier. However, I wonder whether this control supports the checkable list mixin to enable/disable items if desired.

[jiangtiandao]

@leonardder Talkback provide three entries state name type checked or clickable is split from type as state

[jiangtiandao]

I ran into similar issue when editing subtitles on a web page NVDA keeps announcing edit multi line blank when I use tab to switch between edit box containing subtitles which annoys me a lot. My workaround is to edit corresponding translation entry in nvda.po change them to empty string .

[Adriani90]

Since this will significantly improve user experience, I suggest adding the possibility to turn off display element type and state in braille as well. Many people use braille and speech at the same time. Apart from that, smal displays of 20 cells cannot be properly used for reading because some element states or typles use nearly the whole display (i.e. MS Outlook when a message has an attachment).

Finally, the possible implementation of #6614 has to be considered. So we need much more flexibility on what is shown on a braille display.

[LeonarddeR]

@Adriani90 commented on 24 Mar 2018, 11:35 CET:

(i.e. MS Outlook when a message has an attachment).

I'm afraid this isn't the best example here. Outlooks attachment state is exposed in the name of the mail item object, not as a state.

[dpy013]

hello What is the current status of the issue? thank

[LeonarddeR]

Thanks for poking. Currently pretty much out of my view.

[Novalis7747]

Just 1hr ago it took me crazy when jumping with H from one youtube result to the next and trying to find a visited link. I had to listen to the full heading before I hear “visited link”…

From: Leonard de Ruijter [email protected] Sent: Donnerstag, 26. September 2019 15:09 To: nvaccess/nvda [email protected] Cc: Ralf Kefferpuetz [email protected]; Manual [email protected] Subject: Re: [nvaccess/nvda] Allow users to change the order in which object properties are reported in speech and braille (#7232)

Thanks for poking. Currently pretty much out of my view.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnvaccess%2Fnvda%2Fissues%2F7232%3Femail_source%3Dnotifications%26email_token%3DAH37BHER2XMMOO3FKSY576LQLSYAHA5CNFSM4DNMX5V2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7VP6XA%23issuecomment-535494492&data=02%7C01%7C%7Cdbd0295e69fd449907d008d74282c0b2%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637051001645876797&sdata=wDQ5KqfT4Wlr%2Bu3w3aHGpNj89NmWpxLqO8i9MNNGdGY%3D&reserved=0, or mute the threadhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH37BHBHJ7OOM5ASUYIXNBTQLSYAHANCNFSM4DNMX5VQ&data=02%7C01%7C%7Cdbd0295e69fd449907d008d74282c0b2%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637051001645886796&sdata=SPuNVii8BZ%2B5FcGP9V2euBBamEXUktderv1ze8%2FDIDk%3D&reserved=0.

[dpy013]

If this issue can be solved, it can improve the browsing efficiency of NVDA. This is my opinion thank

[akash07k]

thanks @leonardder for pointing me to this issue. So, any possibility whether it is going to be implemented soon or not? If no, then any expert who can develop an addon for this? I'm even ready to pay..

[akash07k]

I've logged one more issue which requests to implement this: https://github.com/nvaccess/nvda/issues/11102

[cary-rowen]

What is the latest status of the issue? Thanks.

[akash07k]

I wish this issue gets some attension soon.

[lijianwei2019]

@seanbudd I hope you can pay attention to this issue. I wish this issue gets some attension soon. Thanks.

[travisroth]

Where is the code in NVDA that controls the order of object attributes presentation? Would it be possible to override it with a Global add-on, as a workaround to meet immediate individual user needs? I notice Braille Extender can override the dispalying of attributes such as Bold, Underline, using it's add-on so it seems this may be possible? But I have not located the order of object attributes presentation in NVDA source to see what may be possible. Any suggestions where to look?

[AAClause]

@travisroth the Braille Extender add-on (dev release only for now) allows to reorder braille properties. Go to Braille Extender Settings > Object Presentation > Order Properties....

[ABuffEr]

Would it be possible to override it with a Global add-on, as a workaround to meet immediate individual user needs? I notice Braille Extender can override the dispalying of attributes such as Bold, Underline, using it's add-on so it seems this may be possible? But I have not located the order of object attributes presentation in NVDA source to see what may be possible. Any suggestions where to look?

See speech.getPropertiesSpeech, that provides the main speech sequence according to a passed dictionary of properties to process (name, role, etc). I don't studied BrailleExtender solution yet, but, in my opinion, any solution is quite tricky at the moment, because the order is determined by instructions sequence in above method (that returns a list of strings, not a dict of (property: string) pair, for example, which could be more manipulable by an add-on).

[davetaylor2112]

I would very much welcome this, for the following reasons. Most often I want the important information first, and if I want to know the control type at all, I would like it last. Two examples. Windows Task manager, i don't need to hear that I am in a list all the time, or what that list is, I only need that once, but I always want to know what app or process is highlighted, so I want that information first. Web Browser: I want to know control types as I am getting to know a website, so I do want to hear them, but once I know a site, and I know what I'm looking for, I no longer want to hear them, so for me, the compromise would be text, then landmark name, control type and all those things. Experience with other screen readers in the past tells me that I would actually make better use of control information than I do now if it were always after the text

[cary-rowen]

If this is achieved, it would be a great advancement in NVDA.

[ABuffEr]

#14934

[marrie]

I would personally love this. there are times when I want to hear the edit box or the thing in the control first. Please consider this in a future update.

[XLTechie]

I am also in favor of this being configurable. I am most annoyed by checkboxes, wherein the label is a "brief" terms of service. You have to read through 10 paragraphs just to find out you're on a checkbox. Fine once you know a website, but not fine when you're trying to understand a site.

On the other hand, there are lists of controls where the information being first will be most critical.

[josephsl]

Hi,

I propose 2024.1 as the optimal development milestone as we can work with newer wxPython releases and see rearrange control has received some "accessibility love" (probably not but won't hurt to try), as that control and the checkable list box (housed in object presentation settings panel) is the most sensible solution. Another reason is that this work will require vast refactoring of control types, speech and braille processing, and reordering object properties announcements, potentially breaking add-ons in the process - we should avoid breaking add-ons in 2023.3 if possible unless it is justified and critical).

Thanks.

[marrie]

Yes agreed. A toggle or an order on a per profile basis would be wonderful in that I might know I’m on a website wherein the check box is labeled with the TOS. I do not want to hear that. That would be a good use case for this option of verbosity

[erdemYigitoglu]

Hello, I just want to let you know that this issue is very important for me as well. reading the text first improves my productivity significantly. I hope this issue to be solved soon. I am a new python user and if I can help to solve this issue in any way, I would love to do so. best regards