REDEFINING ACCESSIBILITY IN VR EXPERIENCES
TEAM
Camila Morales (Thesis Advisor)
Camila Morales
(Thesis Advisor)
ROLE
UX Researcher
UX Designer
TOOLS
Unity
Figma
Meta Quest 3
PROJECT OVERVIEW
This project started out with the question of: how can we leverage Virtual Reality technology in a way that is accessible for different types of users in order to ultimately enhance the overall user experience? This question came about when my mother was diagnosed with Carpal Tunnel Syndrome and wasn’t able to experience VR. This project won 2nd place at the NYU Tandon Research Excellence Exhibit 2024
ABSTRACT
InclusiveRealms is a Virtual Reality (VR) experience that aims to tackle the problems of Physical Accessibility in a virtual environment. Despite VR’s ability to help people, VR interactions are plagued with complex controllers, intricate button configurations, and physically demanding actions. The project aims to tackle the challenges to accessibility in a VR environment by developing intuitive hand-based gestures as an alternative to traditional interaction methods.
InclusiveRealms is a Virtual Reality (VR) experience that aims to tackle the problems of Physical Accessibility in a virtual environment. Despite VR’s ability to help people, VR interactions are plagued with complex controllers, intricate button configurations, and physically demanding actions. The project aims to tackle the challenges to accessibility in a VR environment by developing intuitive hand-based gestures as an alternative to traditional interaction methods.
InclusiveRealms is a Virtual Reality (VR) experience that aims to tackle the problems of Physical Accessibility in a virtual environment. Despite VR’s ability to help people, VR interactions are plagued with complex controllers, intricate button configurations, and physically demanding actions. The project aims to tackle the challenges to accessibility in a VR environment by developing intuitive hand-based gestures as an alternative to traditional interaction methods.
THE PROBLEM
Individuals with disabilities, limited mobility, or cognitive impairments may find it hard to navigate current interfaces. As a result of this, many potential users are excluded from fully accessing and benefiting from VR technology.
Individuals with disabilities, limited mobility, or cognitive impairments may find it hard to navigate current interfaces. As a result of this, many potential users are excluded from fully accessing and benefiting from VR technology.
Individuals with disabilities, limited mobility, or cognitive impairments may find it hard to navigate current interfaces. As a result of this, many potential users are excluded from fully accessing and benefiting from VR technology.
LACK OF CONSISTENCY
User Interface in Virtual Reality is not standardised
User Interface in Virtual Reality is not standardised
UNDERDEVELOPED ACCESSIBILITY GUIDELINES
UNDERDEVELOPED ACCESSIBILITY GUIDELINES
The current guidelines for virtual reality are not thoroughly developed and focus largely on only digital accessibility.
The current guidelines for virtual reality are not thoroughly developed and focus largely on only digital accessibility.
HARDWARE DIFFCIULTY
There is an over emphasis on motion-controllers
There is an over emphasis on motion-controllers
RESEARCH QUESTION
RESEARCH QUESTION
How can we leverage Virtual Reality technology to be accessible to different types of users in order to enhance the overall user experience?
How can we leverage Virtual Reality technology to be accessible to different types of users in order to enhance the overall user experience?
How can we leverage Virtual Reality technology to be accessible to different types of users in order to enhance the overall user experience?
HOW DID I GET HERE?
METHOD 1
Literature Analysis
METHOD 2
Expert & Stakeholder Interviews
METHOD 3
Surveys
METHOD 4
Observational Study
KEY INSIGHT 1
Create a standard for accessible Virtual Reality.
Create a standard for accessible Virtual Reality.
Create a standard for accessible Virtual Reality.
KEY INSIGHT 2
Create a method for displaying error
states for users in virtual environments.
Create a method for displaying error
states for users in virtual environments.
Create a method for displaying error states for users in virtual environments.
KEY INSIGHT 3
Create a uniform onboarding process.
Create a uniform onboarding process.
Create a uniform onboarding process.
STORYBOARDING
STORYBOARDING
IDEATION
IDEATION
FINAL PROJECT GOAL: the minimum viable prototype
FINAL PROJECT GOAL: the minimum viable prototype
Develop a comprehensive VR accessibility standard incorporating gesture recognition for sign languages, error state displays to reduce guidance and onboarding times.
Develop a comprehensive VR accessibility standard incorporating gesture recognition for sign languages, error state displays to reduce guidance and onboarding times.
Develop a comprehensive VR accessibility standard incorporating gesture recognition for sign languages, error state displays to reduce guidance and onboarding times.
FEATURE 1
Custom Personalised Gestures
FEATURE 2
Accessibility Guide
Accessibility Guide
FEATURE 3
ASL TUTORIAL
ASL TUTORIAL
USER PERSONAS
USER PERSONAS
INFORMATION ARCHITECTURE
INFORMATION ARCHITECTURE
TASK FLOW
TASK FLOW
WIREFRAMES
WIREFRAMES
LOW-FI PROTOTYPE
LOW-FI PROTOTYPE
MVP CONCEPT: PART 3
MVP CONCEPT: PART 3
A gamified experience teaching users the sign-language alphabet
A gamified experience teaching users the sign-language alphabet
A gamified experience teaching users the sign-language alphabet
TESTING AND FEEDBACK
The feedback received on this prototype was very useful in the direction of the way this project should move forward. There was quite a range of feedback received a lot of which was concerned about how niche this part of the project becomes. Experts desired that this part of the project to be narrowed down to a specific user group instead of keeping it quite general with the aim of making it accessible to everyone. The first idea should be to create for a specific user group and then move on to a broader audience. Another aspect is looking at the tweaking of the hand gestures in the environment themselves, as the technology is still quite new it isn’t perfect but can be studied further to make it perfect. Based on this testing and feedback, the high-fidelity prototype focuses on tackling these areas and making it a more well-rounded project.
The feedback received on this prototype was very useful in the direction of the way this project should move forward. There was quite a range of feedback received a lot of which was concerned about how niche this part of the project becomes. Experts desired that this part of the project to be narrowed down to a specific user group instead of keeping it quite general with the aim of making it accessible to everyone.
The first idea should be to create for a specific user group and then move on to a broader audience. Another aspect is looking at the tweaking of the hand gestures in the environment themselves, as the technology is still quite new it isn’t perfect but can be studied further to make it perfect. Based on this testing and feedback, the high-fidelity prototype focuses on tackling these areas and making it a more well-rounded project.
HIGH-FI PROTOTYPE
HIGH-FI PROTOTYPE
MVP CONCEPT: PART 1
MVP CONCEPT: PART 1
A simplified and detailed onboarding process that allow users to customise and personalise the hand-gestures to make their process enjoyable
A simplified and detailed onboarding process that allow users to customise and personalise the hand-gestures to make their process enjoyable
A simplified and detailed onboarding process that allow users to customise and personalise the hand-gestures to make their process enjoyable
BRAINSTORMING THE GESTURES
BRAINSTORMING THE GESTURES
Based on the research conducted there were 8 gestures that I created to help with the processes. The users would start out in a room with an onboarding tutorial after which they could move onto the gesture learning room, in which they would be presented with a video for each gesture and they could perform the same to learn those particular gestures. If they are further intrigued to learn the gestures there is a gamified experience waiting for them. The reason that there are 2 ways to perform a few of the gestures is to allow users to choose and customise what feels more natural for them.
User testing was then conducted to understand:
If the users could perform these gestures or not
If they liked the gestures presented to them
Based on the research conducted there were 8 gestures that I created to help with the processes. The users would start out in a room with an onboarding tutorial after which they could move onto the gesture learning room, in which they would be presented with a video for each gesture and they could perform the same to learn those particular gestures. If they are further intrigued to learn the gestures there is a gamified experience waiting for them. The reason that there are 2 ways to perform a few of the gestures is to allow users to choose and customise what feels more natural for them.
User testing was then conducted to understand:
If the users could perform these gestures or not
If they liked the gestures presented to them
Based on the research conducted there were 8 gestures that I created to help with the processes. The users would start out in a room with an onboarding tutorial after which they could move onto the gesture learning room, in which they would be presented with a video for each gesture and they could perform the same to learn those particular gestures. If they are further intrigued to learn the gestures there is a gamified experience waiting for them. The reason that there are 2 ways to perform a few of the gestures is to allow users to choose and customise what feels more natural for them.
User testing was then conducted to understand:
If the users could perform these gestures or not
If they liked the gestures presented to them
THE FINAL GESTURES
THE FINAL GESTURES
GESTURE 1a) | HAND-FLIP MENU
Two-Handed Users: Flip the non-dominant hand upwards and look at the palm to activate the menu.
One-Handed Users: Flip the dominant hand upwards and look at the palm to activate the menu.
Flip the dominant hand upwards and create a closed palm gesture. Slowly open it, imitating something being opened, or keep the hand facing as it is and perform the same gesture to open the menu. Flip the dominant hand up
Two-Handed Users: Flip the non-dominant hand upwards and look at the palm to activate the menu.
One-Handed Users: Flip the dominant hand upwards and look at the palm to activate the menu.
Create a method for displaying error states for users in virtual environments.
GESTURE 1b) | HAND-DRAWER MENU
Two-Handed Users: Flip the non-dominant hand upwards and create a closed palm gesture. Slowly open it, imitating something being opened, or keep the hand facing as it is and perform the same gesture to open the menu.
One-Hand users: Flip the dominant hand upwards and create a closed palm gesture. Slowly open it, imitating something being opened, or keep the hand facing as it is and perform the same gesture to open the menu.
Create a method for displaying error states for users in virtual environments.
GESTURE 2a) | POINTER TELEPORTATION
Two-Handed or One-Handed User: With the dominant hand make a pointer gesture, slowly curly it back, and pinch it to your thumb to release the teleport. This gesture plays upon the current way of teleporting but modifies it a little to make it easier and more intuitive for the user. by the designers & developers.
This gesture is a completely new gesture that creates a new form of interaction that can make it easier for the users.
Two-Handed or One-Handed User: With the dominant hand make a pointer gesture, slowly curly it back, and pinch it to your thumb to release the teleport. This gesture plays upon the current way of teleporting but modifies it a little to make it easier and more intuitive for the user. b
Create a method for displaying error states for users in virtual environments.
GESTURE 2b) | PORTAL TELEPORTATION
Two-Handed or One-Handed User: There are two ways to perform this create a circle with your finger or pinch your fingers together and create a circle to trigger the portal. The portal then allows users to choose destinations via designated checkpoints/ waypoints defined by the designers & developers.
This gesture is a completely new gesture that creates a new form of interaction that can make it easier for the users.
Create a method for displaying error states for users in virtual environments.
GESTURE 5 | PHYSICS-SIZE GRAB
Two-Handed or One-Handed User: Imitate the size of the object when close to activate and pick it up.
This gesture is a play on the current way of performing the gesture but creates a clear indication for the users to perform this interaction.changing the inteethod to make it easier for users to perfo
Two-Handed or One-Handed User: Imitate the size of the object when close to activate and pick it up.
This gesture is a play on the current way of performing the gesture but creates a clear indication for the users to perform this interaction.c
Create a method for displaying error states for users in virtual environments.
GESTURE 6 | NEAR-FAR PULL
Two-Handed or One-Handed User: Create a triangular gesture with the index finger and thumb of the dominant hand and focus on an object to bring it close and interact with it.
This is a play on the current way of performing the gesture but changing the interaction method to make it easier for users to perform this action.
Create a method for displaying error states for users in virtual environments.
GESTURE 7 | ON-SCREEN HAPTICS
It is almost impossible to receive feedback for actions being performed through gesture tracking as the haptics received from the VR controllers. Therefore, this isn’t a complete gesture but a way to provide feedback to the users. When users are performing an action, if it is performed correctly, they will be met with visual and auditory feedback to help guide them through the experience however, if they don’t receive any feedback this means that the action wasn’t performed correctly and they need to adjust their angle before the gesture is valid again. is valid again. is vali
Create a method for displaying error states for users in virtual environments.
GESTURE 8 | THUMB JOY-STICK
Two-Handed or One-Handed user: Create a thumbs-up gesture with the dominant hand and move it left, right, forward, or backward to snap-turn 45 degrees in that direction.
This gesture allows users to turn their dominant hand into a controller, creating freedom and flexibility in their actions. This form of interaction also allows them to easily rotate in their environment without actually turning, this helps them in small places or in a situation where turning is quite difficult.
Create a method for displaying error states for users in virtual environments.
FINAL PROTOTYPE DEMO
FINAL PROTOTYPE DEMO
MVP CONCEPT: PART 2
MVP CONCEPT: PART 2
An accessibility guide created for designers and developers to keep certain parameters in consideration when designing and building to ensure applications are accessible to different types of users
An accessibility guide created for designers and developers to keep certain parameters in consideration when designing and building to ensure applications are accessible to different types of users
An accessibility guide created for designers and developers to keep certain parameters in consideration when designing and building to ensure applications are accessible to different types of users
BRAINSTORMING
Through the course of this research it was found that guidelines for building in a virtual environment aren't thoroughly built out. Even WCAG last updated their virtual guidelines in 2021 at the time of this research which doesn't help users as the virtual world has developed considerably since then.
Through the course of this research it was found that guidelines for building in a virtual environment aren't thoroughly built out. Even WCAG last updated their virtual guidelines in 2021 at the time of this research which doesn't help users as the virtual world has developed considerably since then.
Through the course of this research it was found that guidelines for building in a virtual environment aren't thoroughly built out. Even WCAG last updated their virtual guidelines in 2021 at the time of this research which doesn't help users as the virtual world has developed considerably since then.
VIEW THE FULL ACCESSIBILITY GUIDE HERE
VIEW THE FULL ACCESSIBILITY GUIDE HERE
TESTING AND FEEDBACK
The user testing started out by asking the users to perform a certain set of tasks. The first one asked them to go through the gesture testing experience and the second one involved them going through the accessibility debug guide. While the testers are going through the experience they are asked to use the speak-aloud method to describe what they are doing, and what they are feeling in that particular moment.
During the user testing, it could be said that the users fairly enjoyed the experience however they were a little confused about what was happening, or the task had to be performed. They were able to perform the tasks at hand but when they reached the gestures some gestures confused the users. Users thought that the accessibility guide was very useful however they didn’t think it was necessary to see it in the experience itself and wished it was more of a digital experience. On the other hand, some users said that the debug guide would be very useful for designers and developers and believed it would be very useful to have.
The user testing started out by asking the users to perform a certain set of tasks. The first one asked them to go through the gesture testing experience and the second one involved them going through the accessibility debug guide. While the testers are going through the experience they are asked to use the speak-aloud method to describe what they are doing, and what they are feeling in that particular moment.
During the user testing, it could be said that the users fairly enjoyed the experience however they were a little confused about what was happening, or the task had to be performed. They were able to perform the tasks at hand but when they reached the gestures some gestures confused the users. Users thought that the accessibility guide was very useful however they didn’t think it was necessary to see it in the experience itself and wished it was more of a digital experience. On the other hand, some users said that the debug guide would be very useful for designers and developers and believed it would be very useful to have.
The user testing started out by asking the users to perform a certain set of tasks. The first one asked them to go through the gesture testing experience and the second one involved them going through the accessibility debug guide. While the testers are going through the experience they are asked to use the speak-aloud method to describe what they are doing, and what they are feeling in that particular moment.
During the user testing, it could be said that the users fairly enjoyed the experience however they were a little confused about what was happening, or the task had to be performed. They were able to perform the tasks at hand but when they reached the gestures some gestures confused the users. Users thought that the accessibility guide was very useful however they didn’t think it was necessary to see it in the experience itself and wished it was more of a digital experience. On the other hand, some users said that the debug guide would be very useful for designers and developers and believed it would be very useful to have.
CONCLUSION & LEARNINGS
CONCLUSION & LEARNINGS
This project has highlighted the impact and the need for accessibility in VR experiences and other XR worlds alike. It shows the importance of physical accessibility and how potential users can easily be left out. Therefore, this project aimed to include those users and create a memorable experience for them.
The custom gestures, although still under modification to meet its final design because of different views gained during the user testing, indicate that there is a potential and a market for this to truly help people. The accessibility guide is one step further into creating a potential for good user experience in virtual environments which is currently a lack thereof as displayed in the research above. The ASL tutorial is still at its beginning phase as currently you can only learn the letters but is a start for people to be mindful about including people with hearing disabilities and create a place where they can communicate more efficiently and inclusively.
Finally, this project aims to reach people of various abilities and create a place that is inclusive and enjoyable for all through the use of good user experience and mindful strategies in design.
This project has highlighted the impact and the need for accessibility in VR experiences and other XR worlds alike. It shows the importance of physical accessibility and how potential users can easily be left out. Therefore, this project aimed to include those users and create a memorable experience for them.
The custom gestures, although still under modification to meet its final design because of different views gained during the user testing, indicate that there is a potential and a market for this to truly help people. The accessibility guide is one step further into creating a potential for good user experience in virtual environments which is currently a lack thereof as displayed in the research above. The ASL tutorial is still at its beginning phase as currently you can only learn the letters but is a start for people to be mindful about including people with hearing disabilities and create a place where they can communicate more efficiently and inclusively.
Finally, this project aims to reach people of various abilities and create a place that is inclusive and enjoyable for all through the use of good user experience and mindful strategies in design.
This project has highlighted the impact and the need for accessibility in VR experiences and other XR worlds alike. It shows the importance of physical accessibility and how potential users can easily be left out. Therefore, this project aimed to include those users and create a memorable experience for them.
The custom gestures, although still under modification to meet its final design because of different views gained during the user testing, indicate that there is a potential and a market for this to truly help people. The accessibility guide is one step further into creating a potential for good user experience in virtual environments which is currently a lack thereof as displayed in the research above. The ASL tutorial is still at its beginning phase as currently you can only learn the letters but is a start for people to be mindful about including people with hearing disabilities and create a place where they can communicate more efficiently and inclusively.
Finally, this project aims to reach people of various abilities and create a place that is inclusive and enjoyable for all through the use of good user experience and mindful strategies in design.
NEXT STEPS
NEXT STEPS
ONBOARDING & GESTURES
More support during the onboarding. Including support for finger types, such as missing fingers/ amputated fingers
More support during the onboarding. Including support for finger types, such as missing fingers/ amputated fingers
More support during the onboarding. Including support for finger types, such as missing fingers/ amputated fingers
ACCESSIBILITY GUIDE
In version 2, the plan is to include a more extensive set of guidelines that are not currently listed in the book to make it the most comprehensive version
In version 2, the plan is to include a more extensive set of guidelines that are not currently listed in the book to make it the most comprehensive version
In version 2, the plan is to include a more extensive set of guidelines that are not currently listed in the book to make it the most comprehensive version
ASL TUTORIAL
In terms of the ASL tutorial, the aim is to include the vast library of gestures available innto the experience and create closed captioning responses in reaction to the actions as it currently only supports the alphabet
In terms of the ASL tutorial, the aim is to include the vast library of gestures available innto the experience and create closed captioning responses in reaction to the actions as it currently only supports the alphabet
In terms of the ASL tutorial, the aim is to include the vast library of gestures available innto the experience and create closed captioning responses in reaction to the actions as it currently only supports the alphabet
You've come to the end of my design journey! If you need any additional information or just want to say hello, don’t hesitate to reach out!
FROM COLLABORATIONS TO COFFEE, IT IS ALWAYS A PLEASURE TO CONNECT WITH FELLOW DESIGNERS AND DEVELOPERS.
You've come to the end of my design journey! If you need any additional information or just want to say hello, don’t hesitate to reach out!
FROM COLLABORATIONS TO COFFEE, IT IS ALWAYS A PLEASURE TO CONNECT WITH FELLOW DESIGNERS AND DEVELOPERS.
You've come to the end of my design journey! If you need any additional information or just want to say hello, don’t hesitate to reach out!