Rich accessibility + data navigation

[frankelavsky]

Provide a general summary of the feature here

Every interactive data visualization should be accessible to screen readers and other "navigational" assistive technologies.

However, Vega only provides minimally descriptive support for visualizations and only when they are rendered using SVG.

Rich interactivity and robust, intelligent navigation design is not possible using Vega out of the box. At best (when using SVG and a screen reader), this adds a burden on users of assistive technology: navigation is tedious + un-opinionated (defaults to rendering order and includes every element). At worst (when using canvas to render, including any interactivity in a chart, or the user has any assistive tech other than a screen reader), the user is excluded from interaction with the visualization.

🤔 Expected Behavior?

We would expect that data visualization interactivity and navigation is made available to users who use any of the following assistive technologies: screen readers, sip-and-puff devices, "keyboard-only" techniques, text- or voice-only input, or any other "navigational" assistive technology.

If a chart can be hovered, clicked: a user should be able to do this with just a keyboard. If a chart can't easily be summarized in just a couple of sentences: a screen reader user should be able to navigate the data and hear alt text for each element. If a chart is being used for important decisions or is in an exploratory context: a user should be able to navigate and explore the data in a meaningful, structured way without requiring the use of a mouse or eyesight.

Presently, these experiences are not possible.

💁 Possible Solution

There is empirical research that establishes some great patterns for navigation and interaction as well as solid industry examples as well.

Research: "Data Navigator" (2023) by Elavsky et al (that's me!) "Rich Screen Reader Experiences" (2022) by Zong et al "Polyfilling accessible chemistry diagrams" (2016) by Sorge et al

Providing a structure similar to a tree, where data points can be nested and organized (and thus skipped if high volume) yet optionally explored in detail is the common pattern established in the research space. This gives users freedom to dig deep if they want, but also skim through data at a high level too. You don't want users to feel trapped in a navigation that requires them to press 1000 times to get through a scatter plot with 1000 points.

Industry examples that use this tree-like pattern of progressive disclosure: Highcharts Visa Chart Components

One of the limitations of every existing solution is that this "tree-like" structure isn't able to handle graph visualizations, maps, spatial visualizations, and alluvials/sankeys, and other diagrams very well.

To add to this, it is difficult to add navigation and interaction functionality into an existing library without building everything from scratch.

The difficulty of this work combined with the limits of a purely tree-based approach is precisely why I made my research project, Data Navigator. This tool creates a node-edge network of relationships in your data that can then be navigated using assistive technology. (I'm proposing Data Navigator as a possible solution.)

Data Navigator will allow RSC core developers and Spectrum Designers to have opinions on what a good assistive tech (AT) navigation and interaction experiences should be while downstream developers can get smart defaults out of the box.

🔦 Context

Ultimately, this feature is moving towards more accessible interactive data interfaces for anyone who interacts with React Spectrum Charts. Users with disabilities should be empowered with data to make informed decisions in analytical contexts. Right now, they are largely excluded. Our goal and ultimate measure of success comes down to whether or not we are including people with disabilities and providing an equal opportunity for them to create meaning out of data.

💻 Examples

I have a working prototype of Data Navigator that shows off many different input technologies interacting with an otherwise static png chart.

I have a working prototype of Data Navigator automatically scaffolding navigable vega-lite visualizations that have been rendered using canvas.

I also have a test environment that shows some of the newer features of data navigator (the "dimensions" api), visualizing the navigable structures using a node-edge diagram.

I also have a PR pending with a prototype of data-navigator and RSC as well.

🧢 Your Company/Team

Adobe, React Spectrum Charts