Accessibility

Written by Mark Billinghurst

Summary

Accessibility for XR devices means making XR experiences available for as many people as possible regardless of their abilities. There are a range of accessibility issues including device comfort, support for long term use, cost, input mappings, and more. With visual impairments affecting 2-3% of the population, hearing deficiencies affecting 10-12%, and with physical movement limitations common, designing for accessibility is important to ensure that XR is available for all (Ciccone 2023). Designing for accessibility also improves the XR experience for all users, regardless of their ability.

Definition

Accessibility in XR refers to designing XR environments, devices, and experiences so that they can be usable and enjoyable for everyone, including people with a wide range of abilities. This means considering a wide range of abilities and potential limitations, so that everyone can participate in XR experiences. XR accessibility is about inclusive design that ensures a positive experience for everyone, regardless of their background or abilities.

Types of Accessibility

There are are range of different types of accessibility, or aspects that should be considered in accessibility, including:

• Usability for all: An accessible XR experience should be usable by people with a wide range of disabilities, including visual, auditory, motor, and cognitive impairments.

• Multiple access points: Providing alternative ways to interact with and perceive information in the XR environment is crucial. This could involve offering options for visual, auditory, and haptic feedback.

• Delightful experience: Accessibility isn't just about making XR usable, it's about making it an enjoyable experience for everyone. This means considering user comfort and avoiding experiences that might cause nausea or disorientation.

• Physical Interaction: Many XR experiences rely on physical movement and dexterity for control. For example, using two handed gestures to manipulate virtual content, or navigating virtual environments through physical movement in the real world. This can be difficult for people with motor impairments or limitations.

• Sensory Issues: XR experiences can be visually overwhelming, with fast-paced movements and complex 3D environments. This can be challenging for people with visual impairments, vestibular disorders (balance problems), or those prone to motion sickness.

• Audio Accessibility: Some XR applications rely on spatial audio cues, which can be confusing for people with hearing impairments. Additionally, background noise in the real world can make it difficult to hear audio elements within the XR experience.

• Cognitive Challenges: People with cognitive disabilities may find complex interfaces or fast-paced experiences overwhelming. Additionally, understanding and navigating unfamiliar virtual environments can be difficult.

• Lack of Awareness: XR is a relatively new technology, and there is still a lack of awareness about accessibility considerations among developers and content creators.

Examples

Free-hand gestures are becoming normal as an input mechanism in VR interfaces. Hand tracking and gesture recognition techniques are used to enable users to control VR interfaces by using one or both of their hands and a set of distinct gestures. However, many people are not able to perform hand gestures, or have limited gesture ability (such as people with Spinal Muscular Atrophy). One way to provide accessibility in this case is to perform input remapping where limited gesture input is mapped onto the range of input commands needed, or other input modalities can be used to replace gesture input. For example, Wang et al. (Wang 2018) show how face expression can be used to control VR applications for people with motor disabilities. Gaze can also be used in conjunction with limited gesture input to provide a full range of input (Yu 2012). Tian et al. (Tian 2024) describes how gesture elicitation can be performed with people with Spinal Muscular Atrophy to see what range of gestures are possible.

XR experiences are primarily visual, making it difficult for people with visual disabilities to enjoy AR or VR applications. A range of different haptic (Espinosa-Castañeda 2020) and audio techniques (Guerreiro 2023) can be used to represent immersive VR spaces, and support user interaction. For example, Espinosa-Castañeda and Medellın-Castillo (Espinosa-Castañeda 2020) describe a system where a force-feedback system can be used to allow completely blind people to feel the shape of simple virtual shapes and so correctly identify them. Similarly Guerreiro et al. (Guerreiro 2023) describe how spatial cues could be added to a VR boxing game to allow people who are legally blind to still be able to play. Ghali, et al. (Ghali 2012) provides a summary of research into techniques enabling blind or visually impaired people to use VR systems, while Li et al. provide a scoping review for of research that employ AR and VR HMD to enhance the visual sense of people with visual impairments (Li 2022).

For Neurodiverse (ND) individuals, such as people with attention deficit hyperactivity disorder (ADHD), or autism spectrum disorder (ASD), using XR systems can be a challenge. Lukava et al. (Lukava 2022) point out that neurodivergent individuals process sensory stimuli in a way that is significantly different from neurotypical people, and so can experience sensory overload when using XR technologies. However, Newbutt et al (Newbutt 2016) conducted a study with ASD individuals, and found that they appeared not to experience sensory issues any more severe than neurotypical users. Other researchers have shown that AR can be used to add virtual cues to the real world to help children with ASD to remain focused (Escobedo 2014), or that VR experiences can be modified to reduce the sensor overload depending on the user's needs (Rossi 2019). Glaser and Schmidt (2022) provide a systematic literature review of VR research with individuals with ASD, finding that there is still a lot of research that needs to be done in this area.

Concerns

Although accessibility is regarded as important, there has been relatively little work undertaken around the design and development of more inclusive immersive experiences. In addition, most of the research work that has been done, hasn’t been translated into the commercial marketplace yet.

Creed et al. (Creed 2023) identify a need for key stakeholders to integrate disability as a core consideration across all elements of their work, increasing disability representation within research studies, and building stronger collaborative partnerships with disability and accessibility groups. They also outline a comprehensive research agenda where further work is required in relation to specific forms of disabilities to develop inclusive AR and VR systems.

Similarly there are some important user groups that have had very little research conducted into their accessibility needs. For example, Lukava et al. (Lukava 2022) point out that while research has been conducted into using XR for treatment of neurodiversity, there is very little work on XR design for neurodiversity. Other important groups include the elderly, the young and people with varied cognitive abilities.

One of the other concerns is that people access XR experiences though a wide range of devices, each of which have different accessibility issues. For example, interacting with AR content on a mobile phone is very different from interacting in a head mounted display. Thus it is difficult to provide universal guidelines for XR accessibility, using a hardware abstraction layer or similar..

Mitigations

Unlike current mobile phones and computer interfaces, there are currently no accessibility tools built into XR operating systems. Low level support for accessibility should be included by device manufacturers, such as including support for screen readers or magnifying tools to support this with visual impairments. Lessons from decades of developments in adding accessibility support to mobile and desktop operating systems should be applied in the XR space.

The notion of accessibility should be broadened beyond just physical disability, to include neurodiversity, cognitive impairment, and other aspects that may make XR systems less inclusive. Rupp (2023) discusses many of the aspects that should be considered with making XR accessible

XR experiences should support input remapping allowing the user to decide which input methods are going to map to the XR input commands. Systems which rely exclusively on input from gesture or speech should have the capability to support input from additional hardware devices for people with differential abilities. Similar to desktop systems, a Human Interface Abstraction layer needs to be developed that will allow any HIA compliant device to provide input into the XR experience.

There is a need for accessibility guidelines that make it easy to improve accessibility for XR experiences, and demonstrations showing how to apply them. Research papers such as those from Ciccone (2023) provide examples of design recommendations for accessible AR systems, while Dudley et al. (Dudley 2023) reviews guidelines for Accessible VR systems.

Tools should be developed that make it easy to incorporate best practice accessibility designs into XR experiences. For example, Unity or Unreal plug-ins that enable VR scenes to be rendered in colors not affected by color blindness, or adding support for audio tools that can vocalize a description of everything in an XR scene.

Work needs to be done to identify the core areas where additional research is urgently required to facilitate more inclusive AR and VR experiences. The work of Creed et al. (Creed 2023) provides an example of this type of work, identifying areas for research to support users with users with physical, audio, visual and cognitive impairments. These include the need for further research on customisable device interfaces, movement filtering, audio-based navigation, and many more.

Independent organizations should be established that can provide accessibility audits and validation for XR products and experiences. This could be a resource for companies to provide improved accessibility compliance in their products. Existing associations such as the XR Association (XRA) or academic research consortiums such as XR Access could help fulfill this need.

Relevant Papers and Links:

https://xraccess.org/ - a research consortium based at Cornell Tech with the goal to make XR inclusive of all, regardless of abilities.

https://xra.org/xr-and-accessibility/ - XR Association Accessibility working group

Apple VisionPro accessibility guidelines and usage link

Ciccone, B. A., Bailey, S. K., & Lewis, J. E. (2023). The next generation of virtual reality: recommendations for accessible and ergonomic design. Ergonomics in Design, 31(2), 24-27.

Creed, C., Al-Kalbani, M., Theil, A., Sarcar, S., & Williams, I. (2023). Inclusive augmented and virtual reality: A research agenda. International Journal of Human–Computer Interaction, 1-20.

Dudley, J., Yin, L., Garaj, V., & Kristensson, P. O. (2023). Inclusive Immersion: a review of efforts to improve accessibility in virtual reality, augmented reality and the metaverse. Virtual Reality, 27(4), 2989-3020.

Escobedo, L., Tentori, M., Quintana, E., Favela, J., & Garcia-Rosas, D. (2014). Using augmented reality to help children with autism stay focused. IEEE Pervasive Computing, 13(1), 38-46.

Espinosa-Castañeda, R., & Medellín-Castillo, H. I. (2020). Virtual haptic perception as an educational assistive technology: A case study in inclusive education. IEEE Transactions on Haptics, 14(1), 152-160.

Ghali, N. I., Soluiman, O., El-Bendary, N., Nassef, T. M., Ahmed, S. A., Elbarawy, Y. M., & Hassanien, A. E. (2012). Virtual reality technology for blind and visual impaired people: reviews and recent advances. Advances in Robotics and Virtual Reality, 363-385.

Glaser, N., & Schmidt, M. (2022). Systematic literature review of virtual reality intervention design patterns for individuals with autism spectrum disorders. International Journal of Human–Computer Interaction, 38(8), 753-788.

Guerreiro, J., Kim, Y., Nogueira, R., Chung, S., Rodrigues, A., & Oh, U. (2023). The design space of the auditory representation of objects and their behaviours in virtual reality for blind people. IEEE Transactions on Visualization and Computer Graphics, 29(5), 2763-2773.

Heilemann, F., Zimmermann, G., & Münster, P. (2021). Accessibility guidelines for VR games-A comparison and synthesis of a comprehensive set. Frontiers in Virtual Reality, 2, 697504.

Li, Y., Kim, K., Erickson, A., Norouzi, N., Jules, J., Bruder, G., & Welch, G. F. (2022). A scoping review of assistance and therapy with head-mounted displays for people who are visually impaired. ACM Transactions on Accessible Computing (TACCESS), 15(3), 1-28.

Lukava, T., Morgado Ramirez, D. Z., & Barbareschi, G. (2022). Two sides of the same coin: accessibility practices and neurodivergent users' experience of extended reality. Journal of Enabling Technologies, 16(2), 75-90.

Newbutt, N., Sung, C., Kuo, H. J., Leahy, M. J., Lin, C. C., & Tong, B. (2016). Brief report: A pilot study of the use of a virtual reality headset in autism populations. Journal of autism and developmental disorders, 46, 3166-3176.

Rossi, H., Prates, R., Santos, S., & Ferreira, R. (2019). Development of a virtual reality-based game approach for supporting sensory processing disorders treatment. Information, 10(5), 177.

Rupp, M. A., Gluck, A., Derby, J., Gable, T., Kelling, N., & Van Ommen, C. (2023, September). Towards Making XR 100% Accessible: A Discussion Panel. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 67, No. 1, pp. 1489-1494). Sage CA: Los Angeles, CA: SAGE Publications.

Tian, J., Wang, Y., Yu, K., Xu, L., Xie, J., Li, F. M., ... & Fan, M. (2024, May). Designing Upper-Body Gesture Interaction with and for People with Spinal Muscular Atrophy in VR. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-19).

Valakou, A., Margetis, G., Ntoa, S., & Stephanidis, C. (2023, July). A Framework for Accessibility in XR Environments. In International Conference on Human-Computer Interaction (pp. 252-263). Cham: Springer Nature Switzerland.

Wang, K. J., Liu, Q., Zhao, Y., Zheng, C. Y., Vhasure, S., Liu, Q., ... & Mao, Z. H. (2018, December). Intelligent wearable virtual reality (VR) gaming controller for people with motor disabilities. In 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) (pp. 161-164). IEEE.

Ye, G., Corso, J. J., Hager, G. D., & Okamura, A. M. (2003, October). Vishap: Augmented reality combining haptics and vision. In SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme-System Security and Assurance (Cat. No. 03CH37483) (Vol. 4, pp. 3425-3431). IEEE.

Yu, D., Lu, X., Shi, R., Liang, H. N., Dingler, T., Velloso, E., & Goncalves, J. (2021, May). Gaze-supported 3d object manipulation in virtual reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-13).