Abstract
Flower Delivery is a interactive 3D environment prototype developed as a team project for Virtual Worlds, a graduate-level course in spring 2024. The project explores the use of audio and haptic feedback to assist blind or low-vision players as they navigate through a virtual 3D environment. The primary goal of the project was to create reproducible techniques that could be applied to future games developed in indie or academic environments. My specific contributions included project management along with UX research, design, and implementation.
Developed with Unreal Engine 5
Key Findings
Strategic Audio
Sound cues were strategically placed throughout the level to mark key locations for the player. For example, the central location of the map, which served as the player's home base, was marked by the sound of wind chimes. Dynamic footsteps were created using physical materials, and the level design was carefully constructed to incorporate footstep audio feedback for player orientation. Items throughout the level utilized blueprints to integrate collision audio and haptic feedback. During testing, both audio and haptic feedback levels were meticulously adjusted to aid player navigation while preserving the immersive audio experience.
Text-to-Speech
Unreal Engine's Text-to-Speech plugin was repurposed from its original function to assist in-game item identification and audio descriptions. Players could press a button that would read aloud the name of the object in front of them. Tools such as actor components and blueprint interfaces were utilized to allow the name-to-speech tool tip be seamlessly integrated into future projects. Trigger volumes were also used to provide audio descriptions as the player entered new areas of the level.
Interactable Objects
Several unique features were integrated into interactable objects to enhance accessibility for players with limited or no vision. Interactable objects were designed to work seamlessly with our name-to-speech system. When not picked up, the item emitted a subtle clicking sound, with the pitch lowering as the player moved closer to the object. To interact with the object, the game utilized a large collision area instead of ray tracing, accompanied by distinct audio cues when the item was picked up or placed down. Additionally, the delivery zone was expanded to reduce frustration in locating a small target.
Camera Redirection
Initial playtesting revealed that players were experiencing difficulty orienting the camera towards their objective. To address this issue, a redirection system was implemented, allowing players to press a button and automatically face the next waypoint in the game. This system also provides audio feedback. For instance, when triggered by the player the audio may announce "50 feet away from waypoint windmill". Additionally, the system notifies the player when they have arrived at their waypoint destination with audible text similar to "You have reached the windmill".
Disclaimer: While preproduction research included published perspectives from subject matter experts, including those apart of the blind community, the testing phase was conducted with sighted players due to limited university resources. Continued testing with blind and low-vision players is still required in order to embody the phrase "nothing about us without us".