INCLUSIVE REALMS

INCLUSIVE REALMS

INCLUSIVE REALMS

REDEFINING ACCESSIBILITY IN VR EXPERIENCES

TEAM

Camila Morales (Thesis Advisor)

Camila Morales

(Thesis Advisor)

ROLE

UX Researcher

UX Designer

TOOLS

Unity

Figma

Meta Quest 3

PROJECT OVERVIEW

This project started out with the question of: how can we leverage Virtual Reality technology in a way that is accessible for different types of users in order to ultimately enhance the overall user experience? This question came about when my mother was diagnosed with Carpal Tunnel Syndrome and wasn’t able to experience VR. This project won 2nd place at the NYU Tandon Research Excellence Exhibit 2024

ABSTRACT

InclusiveRealms is a Virtual Reality (VR) experience that aims to tackle the problems of Physical Accessibility in a virtual environment. Despite VR’s ability to help people, VR interactions are plagued with complex controllers, intricate button configurations, and physically demanding actions. The project aims to tackle the challenges to accessibility in a VR environment by developing intuitive hand-based gestures as an alternative to traditional interaction methods.

InclusiveRealms is a Virtual Reality (VR) experience that aims to tackle the problems of Physical Accessibility in a virtual environment. Despite VR’s ability to help people, VR interactions are plagued with complex controllers, intricate button configurations, and physically demanding actions. The project aims to tackle the challenges to accessibility in a VR environment by developing intuitive hand-based gestures as an alternative to traditional interaction methods.

InclusiveRealms is a Virtual Reality (VR) experience that aims to tackle the problems of Physical Accessibility in a virtual environment. Despite VR’s ability to help people, VR interactions are plagued with complex controllers, intricate button configurations, and physically demanding actions. The project aims to tackle the challenges to accessibility in a VR environment by developing intuitive hand-based gestures as an alternative to traditional interaction methods.

THE PROBLEM

Individuals with disabilities, limited mobility, or cognitive impairments may find it hard to navigate current interfaces. As a result of this, many potential users are excluded from fully accessing and benefiting from VR technology.

Individuals with disabilities, limited mobility, or cognitive impairments may find it hard to navigate current interfaces. As a result of this, many potential users are excluded from fully accessing and benefiting from VR technology.

Individuals with disabilities, limited mobility, or cognitive impairments may find it hard to navigate current interfaces. As a result of this, many potential users are excluded from fully accessing and benefiting from VR technology.

LACK OF CONSISTENCY

User Interface in Virtual Reality is not standardised

User Interface in Virtual Reality is not standardised

UNDERDEVELOPED ACCESSIBILITY GUIDELINES

UNDERDEVELOPED ACCESSIBILITY GUIDELINES

The current guidelines for virtual reality are not thoroughly developed and focus largely on only digital accessibility.

The current guidelines for virtual reality are not thoroughly developed and focus largely on only digital accessibility.

HARDWARE DIFFCIULTY

There is an over emphasis on motion-controllers

There is an over emphasis on motion-controllers

RESEARCH QUESTION

RESEARCH QUESTION

How can we leverage Virtual Reality technology to be accessible to different types of users in order to enhance the overall user experience?

How can we leverage Virtual Reality technology to be accessible to different types of users in order to enhance the overall user experience?

How can we leverage Virtual Reality technology to be accessible to different types of users in order to enhance the overall user experience?

HOW DID I GET HERE?

RESEARCH & BRAINSTORMING

RESEARCH & BRAINSTORMING

METHOD 1

Literature Analysis

METHOD 2

Expert & Stakeholder Interviews

METHOD 3

Surveys

METHOD 4

Observational Study

KEY INSIGHT 1

Create a standard for accessible Virtual Reality.

Create a standard for accessible Virtual Reality.

Create a standard for accessible Virtual Reality.

KEY INSIGHT 2

Create a method for displaying error

states for users in virtual environments.

Create a method for displaying error

states for users in virtual environments.

Create a method for displaying error states for users in virtual environments.

KEY INSIGHT 3

Create a uniform onboarding process.

Create a uniform onboarding process.

Create a uniform onboarding process.

STORYBOARDING

STORYBOARDING

IDEATION

IDEATION

FINAL PROJECT GOAL: the minimum viable prototype

FINAL PROJECT GOAL: the minimum viable prototype

Develop a comprehensive VR accessibility standard incorporating gesture recognition for sign languages, error state displays to reduce guidance and onboarding times.

Develop a comprehensive VR accessibility standard incorporating gesture recognition for sign languages, error state displays to reduce guidance and onboarding times.

Develop a comprehensive VR accessibility standard incorporating gesture recognition for sign languages, error state displays to reduce guidance and onboarding times.

FEATURE 1

Custom Personalised Gestures

FEATURE 2

Accessibility Guide

Accessibility Guide

FEATURE 3

ASL TUTORIAL

ASL TUTORIAL

USER PERSONAS

USER PERSONAS

INFORMATION ARCHITECTURE

INFORMATION ARCHITECTURE

TASK FLOW

TASK FLOW

WIREFRAMES

WIREFRAMES

LOW-FI PROTOTYPE

LOW-FI PROTOTYPE

MVP CONCEPT: PART 3

MVP CONCEPT: PART 3

A gamified experience teaching users the sign-language alphabet

A gamified experience teaching users the sign-language alphabet

A gamified experience teaching users the sign-language alphabet

TESTING AND FEEDBACK

The feedback received on this prototype was very useful in the direction of the way this project should move forward. There was quite a range of feedback received a lot of which was concerned about how niche this part of the project becomes. Experts desired that this part of the project to be narrowed down to a specific user group instead of keeping it quite general with the aim of making it accessible to everyone. The first idea should be to create for a specific user group and then move on to a broader audience. Another aspect is looking at the tweaking of the hand gestures in the environment themselves, as the technology is still quite new it isn’t perfect but can be studied further to make it perfect. Based on this testing and feedback, the high-fidelity prototype focuses on tackling these areas and making it a more well-rounded project.

The feedback received on this prototype was very useful in the direction of the way this project should move forward. There was quite a range of feedback received a lot of which was concerned about how niche this part of the project becomes. Experts desired that this part of the project to be narrowed down to a specific user group instead of keeping it quite general with the aim of making it accessible to everyone.


The first idea should be to create for a specific user group and then move on to a broader audience. Another aspect is looking at the tweaking of the hand gestures in the environment themselves, as the technology is still quite new it isn’t perfect but can be studied further to make it perfect. Based on this testing and feedback, the high-fidelity prototype focuses on tackling these areas and making it a more well-rounded project.

HIGH-FI PROTOTYPE

HIGH-FI PROTOTYPE

MVP CONCEPT: PART 1

MVP CONCEPT: PART 1

A simplified and detailed onboarding process that allow users to customise and personalise the hand-gestures to make their process enjoyable

A simplified and detailed onboarding process that allow users to customise and personalise the hand-gestures to make their process enjoyable

A simplified and detailed onboarding process that allow users to customise and personalise the hand-gestures to make their process enjoyable

BRAINSTORMING THE GESTURES

BRAINSTORMING THE GESTURES

Based on the research conducted there were 8 gestures that I created to help with the processes. The users would start out in a room with an onboarding tutorial after which they could move onto the gesture learning room, in which they would be presented with a video for each gesture and they could perform the same to learn those particular gestures. If they are further intrigued to learn the gestures there is a gamified experience waiting for them. The reason that there are 2 ways to perform a few of the gestures is to allow users to choose and customise what feels more natural for them.


User testing was then conducted to understand:

  1. If the users could perform these gestures or not

  1. If they liked the gestures presented to them

Based on the research conducted there were 8 gestures that I created to help with the processes. The users would start out in a room with an onboarding tutorial after which they could move onto the gesture learning room, in which they would be presented with a video for each gesture and they could perform the same to learn those particular gestures. If they are further intrigued to learn the gestures there is a gamified experience waiting for them. The reason that there are 2 ways to perform a few of the gestures is to allow users to choose and customise what feels more natural for them.


User testing was then conducted to understand:

  1. If the users could perform these gestures or not

  1. If they liked the gestures presented to them

Based on the research conducted there were 8 gestures that I created to help with the processes. The users would start out in a room with an onboarding tutorial after which they could move onto the gesture learning room, in which they would be presented with a video for each gesture and they could perform the same to learn those particular gestures. If they are further intrigued to learn the gestures there is a gamified experience waiting for them. The reason that there are 2 ways to perform a few of the gestures is to allow users to choose and customise what feels more natural for them.


User testing was then conducted to understand:

  1. If the users could perform these gestures or not

  1. If they liked the gestures presented to them

THE FINAL GESTURES

THE FINAL GESTURES

GESTURE 1a) | HAND-FLIP MENU

Two-Handed Users: Flip the non-dominant hand upwards and look at the palm to activate the menu.

One-Handed Users: Flip the dominant hand upwards and look at the palm to activate the menu.

Flip the dominant hand upwards and create a closed palm gesture. Slowly open it, imitating something being opened, or keep the hand facing as it is and perform the same gesture to open the menu. Flip the dominant hand up

Two-Handed Users: Flip the non-dominant hand upwards and look at the palm to activate the menu.

One-Handed Users: Flip the dominant hand upwards and look at the palm to activate the menu.

Create a method for displaying error states for users in virtual environments.

GESTURE 1b) | HAND-DRAWER MENU

Two-Handed Users: Flip the non-dominant hand upwards and create a closed palm gesture. Slowly open it, imitating something being opened, or keep the hand facing as it is and perform the same gesture to open the menu.

One-Hand users: Flip the dominant hand upwards and create a closed palm gesture. Slowly open it, imitating something being opened, or keep the hand facing as it is and perform the same gesture to open the menu.

Create a method for displaying error states for users in virtual environments.

GESTURE 2a) | POINTER TELEPORTATION

Two-Handed or One-Handed User: With the dominant hand make a pointer gesture, slowly curly it back, and pinch it to your thumb to release the teleport. This gesture plays upon the current way of teleporting but modifies it a little to make it easier and more intuitive for the user. by the designers & developers.


This gesture is a completely new gesture that creates a new form of interaction that can make it easier for the users.

Two-Handed or One-Handed User: With the dominant hand make a pointer gesture, slowly curly it back, and pinch it to your thumb to release the teleport. This gesture plays upon the current way of teleporting but modifies it a little to make it easier and more intuitive for the user. b

Create a method for displaying error states for users in virtual environments.

GESTURE 2b) | PORTAL TELEPORTATION

Two-Handed or One-Handed User: There are two ways to perform this create a circle with your finger or pinch your fingers together and create a circle to trigger the portal. The portal then allows users to choose destinations via designated checkpoints/ waypoints defined by the designers & developers.


This gesture is a completely new gesture that creates a new form of interaction that can make it easier for the users.

Create a method for displaying error states for users in virtual environments.

GESTURE 5 | PHYSICS-SIZE GRAB

Two-Handed or One-Handed User: Imitate the size of the object when close to activate and pick it up.


This gesture is a play on the current way of performing the gesture but creates a clear indication for the users to perform this interaction.changing the inteethod to make it easier for users to perfo

Two-Handed or One-Handed User: Imitate the size of the object when close to activate and pick it up.


This gesture is a play on the current way of performing the gesture but creates a clear indication for the users to perform this interaction.c

Create a method for displaying error states for users in virtual environments.

GESTURE 6 | NEAR-FAR PULL

Two-Handed or One-Handed User: Create a triangular gesture with the index finger and thumb of the dominant hand and focus on an object to bring it close and interact with it.


This is a play on the current way of performing the gesture but changing the interaction method to make it easier for users to perform this action.

Create a method for displaying error states for users in virtual environments.

GESTURE 7 | ON-SCREEN HAPTICS

It is almost impossible to receive feedback for actions being performed through gesture tracking as the haptics received from the VR controllers. Therefore, this isn’t a complete gesture but a way to provide feedback to the users. When users are performing an action, if it is performed correctly, they will be met with visual and auditory feedback to help guide them through the experience however, if they don’t receive any feedback this means that the action wasn’t performed correctly and they need to adjust their angle before the gesture is valid again. is valid again. is vali

Create a method for displaying error states for users in virtual environments.

GESTURE 8 | THUMB JOY-STICK

Two-Handed or One-Handed user: Create a thumbs-up gesture with the dominant hand and move it left, right, forward, or backward to snap-turn 45 degrees in that direction.


This gesture allows users to turn their dominant hand into a controller, creating freedom and flexibility in their actions. This form of interaction also allows them to easily rotate in their environment without actually turning, this helps them in small places or in a situation where turning is quite difficult.

Create a method for displaying error states for users in virtual environments.

FINAL PROTOTYPE DEMO

FINAL PROTOTYPE DEMO

MVP CONCEPT: PART 2

MVP CONCEPT: PART 2

An accessibility guide created for designers and developers to keep certain parameters in consideration when designing and building to ensure applications are accessible to different types of users

An accessibility guide created for designers and developers to keep certain parameters in consideration when designing and building to ensure applications are accessible to different types of users

An accessibility guide created for designers and developers to keep certain parameters in consideration when designing and building to ensure applications are accessible to different types of users

BRAINSTORMING

Through the course of this research it was found that guidelines for building in a virtual environment aren't thoroughly built out. Even WCAG last updated their virtual guidelines in 2021 at the time of this research which doesn't help users as the virtual world has developed considerably since then.

Through the course of this research it was found that guidelines for building in a virtual environment aren't thoroughly built out. Even WCAG last updated their virtual guidelines in 2021 at the time of this research which doesn't help users as the virtual world has developed considerably since then.

Through the course of this research it was found that guidelines for building in a virtual environment aren't thoroughly built out. Even WCAG last updated their virtual guidelines in 2021 at the time of this research which doesn't help users as the virtual world has developed considerably since then.

VIEW THE FULL ACCESSIBILITY GUIDE HERE

VIEW THE FULL ACCESSIBILITY GUIDE HERE

TESTING AND FEEDBACK

The user testing started out by asking the users to perform a certain set of tasks. The first one asked them to go through the gesture testing experience and the second one involved them going through the accessibility debug guide. While the testers are going through the experience they are asked to use the speak-aloud method to describe what they are doing, and what they are feeling in that particular moment.


During the user testing, it could be said that the users fairly enjoyed the experience however they were a little confused about what was happening, or the task had to be performed. They were able to perform the tasks at hand but when they reached the gestures some gestures confused the users. Users thought that the accessibility guide was very useful however they didn’t think it was necessary to see it in the experience itself and wished it was more of a digital experience. On the other hand, some users said that the debug guide would be very useful for designers and developers and believed it would be very useful to have.

The user testing started out by asking the users to perform a certain set of tasks. The first one asked them to go through the gesture testing experience and the second one involved them going through the accessibility debug guide. While the testers are going through the experience they are asked to use the speak-aloud method to describe what they are doing, and what they are feeling in that particular moment.


During the user testing, it could be said that the users fairly enjoyed the experience however they were a little confused about what was happening, or the task had to be performed. They were able to perform the tasks at hand but when they reached the gestures some gestures confused the users. Users thought that the accessibility guide was very useful however they didn’t think it was necessary to see it in the experience itself and wished it was more of a digital experience. On the other hand, some users said that the debug guide would be very useful for designers and developers and believed it would be very useful to have.

The user testing started out by asking the users to perform a certain set of tasks. The first one asked them to go through the gesture testing experience and the second one involved them going through the accessibility debug guide. While the testers are going through the experience they are asked to use the speak-aloud method to describe what they are doing, and what they are feeling in that particular moment.


During the user testing, it could be said that the users fairly enjoyed the experience however they were a little confused about what was happening, or the task had to be performed. They were able to perform the tasks at hand but when they reached the gestures some gestures confused the users. Users thought that the accessibility guide was very useful however they didn’t think it was necessary to see it in the experience itself and wished it was more of a digital experience. On the other hand, some users said that the debug guide would be very useful for designers and developers and believed it would be very useful to have.

CONCLUSION & LEARNINGS

CONCLUSION & LEARNINGS

This project has highlighted the impact and the need for accessibility in VR experiences and other XR worlds alike. It shows the importance of physical accessibility and how potential users can easily be left out. Therefore, this project aimed to include those users and create a memorable experience for them.


The custom gestures, although still under modification to meet its final design because of different views gained during the user testing, indicate that there is a potential and a market for this to truly help people. The accessibility guide is one step further into creating a potential for good user experience in virtual environments which is currently a lack thereof as displayed in the research above. The ASL tutorial is still at its beginning phase as currently you can only learn the letters but is a start for people to be mindful about including people with hearing disabilities and create a place where they can communicate more efficiently and inclusively.


Finally, this project aims to reach people of various abilities and create a place that is inclusive and enjoyable for all through the use of good user experience and mindful strategies in design.

This project has highlighted the impact and the need for accessibility in VR experiences and other XR worlds alike. It shows the importance of physical accessibility and how potential users can easily be left out. Therefore, this project aimed to include those users and create a memorable experience for them.


The custom gestures, although still under modification to meet its final design because of different views gained during the user testing, indicate that there is a potential and a market for this to truly help people. The accessibility guide is one step further into creating a potential for good user experience in virtual environments which is currently a lack thereof as displayed in the research above. The ASL tutorial is still at its beginning phase as currently you can only learn the letters but is a start for people to be mindful about including people with hearing disabilities and create a place where they can communicate more efficiently and inclusively.


Finally, this project aims to reach people of various abilities and create a place that is inclusive and enjoyable for all through the use of good user experience and mindful strategies in design.

This project has highlighted the impact and the need for accessibility in VR experiences and other XR worlds alike. It shows the importance of physical accessibility and how potential users can easily be left out. Therefore, this project aimed to include those users and create a memorable experience for them.


The custom gestures, although still under modification to meet its final design because of different views gained during the user testing, indicate that there is a potential and a market for this to truly help people. The accessibility guide is one step further into creating a potential for good user experience in virtual environments which is currently a lack thereof as displayed in the research above. The ASL tutorial is still at its beginning phase as currently you can only learn the letters but is a start for people to be mindful about including people with hearing disabilities and create a place where they can communicate more efficiently and inclusively.


Finally, this project aims to reach people of various abilities and create a place that is inclusive and enjoyable for all through the use of good user experience and mindful strategies in design.

NEXT STEPS

NEXT STEPS

ONBOARDING & GESTURES

More support during the onboarding. Including support for finger types, such as missing fingers/ amputated fingers

More support during the onboarding. Including support for finger types, such as missing fingers/ amputated fingers

More support during the onboarding. Including support for finger types, such as missing fingers/ amputated fingers

ACCESSIBILITY GUIDE

In version 2, the plan is to include a more extensive set of guidelines that are not currently listed in the book to make it the most comprehensive version

In version 2, the plan is to include a more extensive set of guidelines that are not currently listed in the book to make it the most comprehensive version

In version 2, the plan is to include a more extensive set of guidelines that are not currently listed in the book to make it the most comprehensive version

ASL TUTORIAL

In terms of the ASL tutorial, the aim is to include the vast library of gestures available innto the experience and create closed captioning responses in reaction to the actions as it currently only supports the alphabet

In terms of the ASL tutorial, the aim is to include the vast library of gestures available innto the experience and create closed captioning responses in reaction to the actions as it currently only supports the alphabet

In terms of the ASL tutorial, the aim is to include the vast library of gestures available innto the experience and create closed captioning responses in reaction to the actions as it currently only supports the alphabet

You've come to the end of my design journey! If you need any additional information or just want to say hello, don’t hesitate to reach out!

FROM COLLABORATIONS TO COFFEE, IT IS ALWAYS A PLEASURE TO CONNECT WITH FELLOW DESIGNERS AND DEVELOPERS.

You've come to the end of my design journey! If you need any additional information or just want to say hello, don’t hesitate to reach out!

FROM COLLABORATIONS TO COFFEE, IT IS ALWAYS A PLEASURE TO CONNECT WITH FELLOW DESIGNERS AND DEVELOPERS.

You've come to the end of my design journey! If you need any additional information or just want to say hello, don’t hesitate to reach out!

FROM COLLABORATIONS TO COFFEE, IT IS ALWAYS A PLEASURE TO CONNECT WITH FELLOW DESIGNERS AND DEVELOPERS.

©2024 | Yesha Shah