Loud and Clear: Improving Accessibility for Low Vision Players in 'Cosmonious High'

[Music] hello and welcome to loud and clear imp improving accessibility for Low Vision players in cosmon high my name is Peter Galbraith I'm the senior accessibility engineer at Alchemy Labs I've been with Alchemy for seven and a half years and have worked on all of the VR titles our team has shipped I am also the primary programmer and one of the main designers for the update we will be discussing today and hello everyone I'm Jasmine Cano the senior accessibility product manager at Alchemy Labs where I've worked for two years I own the production on the project we'll be discussing today I mainly worked on design and leading PL testing Alchemy Labs is an amazing team with around 70 L's it's thanks to everyone's efforts and commitment to accessibility that we can be here today to share this with you accessibility is

core to what we do and is a big part of alchemy's mission of VR for everyone you might recognize Alchemy Labs from our previous VR games job simulator vacation simulator while those games contain a variety of accessibility features we're here today to talk to you about a specific update to our most recently released game cosmon is high cosmon is high is a VR game where you crash land into your first day at an alien High School you unlock powers that help you compete class or sorry complete classes make friends and save the school from cosmic chaos and this Talk's Focus will be about the game's Vision accessibility update we'll go over the challenges we faced and the design choices we made we also want to acknowledge that when most people think of VR they think of putting on a headset and being immersed in the visual experience it makes sense why they imagine this the headset goes over their eyes after after all so why even

try to add Vision accessibility to a VR game well the answer to that is VR has been largely inaccessible to Blind and low vision players but we believe it doesn't have to stay that way there's more to VR than simply what is on the screen just like there's more to life than what can be experienced through visuals alone and everyone should be able to experience all of what VR has to offer so with that in mind we set ourselves a lofty goal we wanted to allow people with partial or total vision loss to play and enjoy all of cosmon as high without sighted assistance when we started this project we weren't sure if we could make the entire game accessible to blind or low vision players but in our mission to make VR for everyone we knew we had to try so we started by asking ourselves the big questions what challenges do

blind players face when trying to enjoy VR and what barriers can we remove for them before we started the project we reached out to Steve sailor to be our primary primary accessibility consultant on the project in our first meeting he outlined potential challenges blind players might face information they might need and possibly interactions to try and VR this helped inform many of the early versions of features we'll be talking about today since no one had ever tried something like this on the scale of a commercial VR game before we knew Theory would only get us so far what was most clear to us was that well from that meeting was that blind players had less access to important information than cited players so the first problem we decided to tackle was the inequality of knowledge blind players were missing crucial information about the world due to how much was conveyed through visuals

alone from the type of objects and interactions available to whether or not the player was holding an item there was a clear Gap in knowledge that was going to hold players back if we were going to make the game accessible we knew we needed to level the playing field when we started asking ourselves what information do players need we also asked ourselves how do we deliver that information to them the former would turn out to be a complex question fortunately though we already had an idea for the latter our proposed solution was to provide detailed information about the virtual world to the player through descriptive audio but unlike movies where a single unchanging descriptive audio track could be used to Aid in the experience we were going to need audio to react dynamically to wherever the player chose to go and whatever the player chose to do and the obvious candidate tool to help us achieve that

was Text to Speech Text to Speech has been a tool used in digital accessibility for many years almost every modern computer or smartphone has built-in screen reader functionality and more recently we have begun to see text to speech used in games to help aid players through setup and menu navigation impressed by the work others have done with TTS in their games we wanted to find out if we could take these ideas a step further we decided to try to implement our own made for VR version of the accessibility object model aom for short aom is a framework designed to make web pages compatible with assisted Technologies however instead of the contents of a 2d web page our implementation would attempt to expand the concept to a fully immersive and interactive 3D World we first started this this process

by implementing a tool for working with text to speech we then made it so that we could send that tool information that uh about the items a player could pick up anytime a player hovered over grabbed dropped or pointed at an object with either of their hands that object would have its text sent to the text of speech tool this text was actually pulled from a pre-existing system in our game called the world item system which contained information such as the name and a comedic in-game description of the objects then we would play back the generated audio and voila we now had audio describing all of our interactive objects well not all of our interactive objects we quickly realized that we had other interactive objects that weren't able to be picked up but still required descriptions so we got those hooked into the tool as well at the same time we

also developed a new system that we called the describable item system that would allow texts that appeared on objects to be read such as the text on posters book pages and journals found around the school this also allowed us to describe other interactive objects where adding a world item wasn't possible this was our equivalent of adding alt text to images so now we could have TTS on nearly all interactive and important objects in the environment at this point surely we were done right well when we began internal testing we realized that we had overlooked some major aspects of the game the most notable example was our tutorial popups that were just animated pictures and didn't use the world item system and wouldn't work with the describable item system so after a bit of reworking and tweaking we got tutorials to trigger a TTS description automatically upon being

shown to the player and instantly ran into another blocker during testing we hadn't yet accounted for how the player would know where they were as they tried to navigate around their space we decided to make it so that while the player was aiming the teleport cursor the system would provide information about key locations of Interest we would also provide an additional text to speech confirmation after teleporting to let the player know where they had arrived uh and when they arrived at the desired destination however once we began running external tests it became apparent that we still hadn't provided enough location information internally we knew the layout of our game but external accessibility testers who were not familiar with the game were getting lost because there weren't enough marked up areas so we went back and fleshed out even more locations throughout the game with additional teleport zones

unlike some of the other aspects of getting this text of speech working on objects there wasn't a simple way of automating this process and these zones had to be hand authored by our technical designers shout out uh but the good news is that after the new zones had been added the frequency with which our testers got lost decreased so we took that as a win finally we had achieved a pretty thoroughly described World there was some content that was less detailed than we would have liked but this part of the project took much more time than initially expected so we reduced our scope to focus on what was required for the player to reach the end of the game by the end of this process a huge amount of the game had been hooked up to play TTS descriptions we did it players could learn nearly anything they wanted about the game world unfortunately merely offering

descriptions uh on objects wasn't enough players had to be able to find the objects we identified early on that it would be difficult or even impossible for blind players to locate objects in the game world we knew we were going to have to provide more information to the player in order for them to play the game sightless players were a particular concern at this point since one of the main ways sited players players found objects was by simply looking around the room and even if the players had some Vision it was important that they could distinguish interactive objects from the environment with object interactions being a core part of alchemy games we knew that it if we couldn't provide people with the resource they needed to play then the dream of making our game work for blind players would be over before it ever really started fortunately our previously our previous except accessibility knowledge

gave us a general idea on what we could do and what was needed to make this happen inaccessibility and across all design it's generally a good idea to convey information in multiple ways this increases the chance that users will be able to access your content in a spatial Computing environment like VR this becomes even more important for players to locate objects we were going to have to take advantage of every feedback channel available to us and that started with auditory feedback though virtual 3D environments present an access present accessibility challenges that aren't present in other mediums VR affords us some unique opportunities as well traditional screen readers don't indicate the position of text on the screen because they don't really need to but for our implementation we realized that we could provide more location context without

adding more text to our spoken descriptions as it turns out people are pretty decent at detecting the direction a sound is coming from additionally volume is a good indicator of how far away or close an object is if you've ever lost your phone and tried finding it by listening to it ringing uh you've used these techniques now this isn't going to be perfectly accurate but by using spatialized audio that plays from the location of an object in the world we could give our players a general idea of the location of an object so that's what we did and with that players could gain a sense of the approximate position of objects in the virtual world which meant that we could begin using other methods to help them narrow their search now onto haptic feedback in the physical world people can reach out and feel nearby objects one of the limitations of VR is that there's no

physical resistance that allows you to feel that those objects are there in the virtual world since we can't change that we chose to take a lesson from white cane users some blind folks use white canes to help them navigate the environment as they're walking they may sweep their cane across the space in front of them failing for obstacles and changes in texture or elevation detecting the change from one object or Surface to another was something that we could do as well well in an attempt to recreate the functionality we added a haptic pulse for when the player moved the teleportation cursor into a different area by doing this we were able to Signal the boundaries of those zones to the player in an attempt to recreate the functionality I just said that sorry about that furthermore we made it so that when a player moved their open hand over something it could be interacted with and the controller would also give a short vibration this little bit of added context meant

that players knew that their hand was near something that they could attempt to interact with or get more information about it too there was still one feedback Channel we had yet to address visual feedback Steve sailor suggested we Implement a high contrast mode since blindness doesn't always mean sightlessness it was important to add clear visual indicators to the world as well some of the most accessible implementations of high contrast modes drastically changed the visuals of an entire game to help highlight the most important things on screen uh and though we wanted to do this we didn't have the available resources for a feature like this thankfully a complete reskinning isn't the only way to improve contrast and object definition since as early as job simulator we used a system that added a light blue Highlight around objects that

could be grabbed and since there are static objects that the player needed to know about we decided to extend that system to add a light green highlight on important environmental objects we chose light green is the color because it stood out well against the mostly purple in the en the mo the mostly purple environment but more more important than the Hue is making sure that the contrast is clear enough to show that something changed now that players could position themselves near interactive objects that were providing different forms of feedback we had a new problem assisted descriptions were being constantly and unintentionally triggered and this was happening with just about any inter any interaction the player took so for example a player could grab their backpack when they're teleporting or even simply reaching their hand out there is very little a player could do

without setting off the TTS now that players could learn about anything and everything and essentially had no other choice the synthesized speech became a dominant and cacophonous Nightmare it drowned out subtle sounds in all the audio and it made it sound incomprehensible it was an unpleasant experience and it felt like drinking from an auditory fire hose it was simply too much it was like happening all at once all the time and something had to be done to prevent players from being constantly bombarded with information that eventually eventually just became noise so our plan was to improve player agency by giving them more direct control over their individual experience but what could we do to put the power back into the hands of the player what tools did we have at our disposal

lucky for us we had an as up Our Sleeve for just such an occasion for all our supported platforms and their different controllers we had reserved one button per hand that had no existing functionality these Reserve buttons would quickly find their new purpose as the assist button the assist button was designed to be a simple yet context sensitive action button that would allow the player to be in control of what information they received and when they received it now instead of automatically playing audio when hovering over or grabbing an object players could choose if and when to have the objects description read aloud by simply tapping the assist button and more importantly the player could also now tap the same button to cancel currently playing TTS descriptions this meant that players no longer had to listen to the entire description before getting back to whatever they were doing now we did briefly experiment with

having the assist button function as a toggle rather than a tap but during testing play players would often forget that they had toggled it on on one hand but not the other and this led to the original problem where they would be accidentally triggering audio at unwanted times again so we chose to revert back to using the original tap Behavior instead overall though the introduction of the the assist button appeared to be a notice ible Improvement to the experience however this still didn't quite solve the problem of overlapping audio it was definitely frustrating to have a character speak while trying to listen to assist of information but delaying a character speaking wasn't always an option for us and delaying the assist of audio until after all the characters were finished speaking would make the feature feel unresponsive and broken so in sticking with the principle of

resp respecting players choices we made it so that when assist of information was playing we would duck the rest of the game's audio lowering the volume so that the audio from the text to speech description could be heard clearly this allowed players to still be aware of other audio playing and cancel the assist of information if they wanted to hear what was being said or they could choose to ignore NPC dial and instead listen to the information they had chosen to seek out and since the game already allowed players to wave it an NPC to get a reminder of what they are likely supposed to be doing uh the exact audio might be missed but important game information wouldn't be lost as an added benefit of using audio ducking the player now knew when the description had finished playing because the game audio would return to normal levels and in circumstances where the

audio could take a little longer to be generated the player could tell that their request was actually being processed and wouldn't imp impatiently tap the assist button over and over unsurprisingly testers were Universal universally on board with these new additions players were now able to use assistive audio as much or as little as they wanted they wouldn't have to worry about accidentally missing information now that players had greater control over their experience a new issue began to reveal itself in testing testers weren't activating assistance on objects very often at first we were concerned that they were forgetting which button on the controller they had a press but during post PL test feedback sessions it became clear that using assistance didn't always feel beneficial testers noted that it was hard to determine what information in the the assist of audio was actually

important to them and it was a valid criticism in our efforts to bridge the knowledge gap for these players we had overcorrected and provided more information than they really needed another issue was that some testers would try to patiently wait to hear every part of the description so that they wouldn't miss crucial information but it became a problem since some of the important information was sometimes mixed in with Superfluous information that means that if a player got distracted or disinterested partway through they'd have to relisten to the whole thing all over again we figured the solution to this problem was just to make sure we presented the information more clearly and concisely to do that first we had to identify what specifically needed clarifying and where we can make changes without losing information the first thing we identified was that

the order we were presenting uh the information mattered we knew we wanted to present the most relevant and valuable information first and decided it would be helpful to standardize the order that information was read it made sense to put information that was the shortest and always present first so interaction State information like whether or not the object was grabbed and the object's name were prioritized next we added a short description that contained the most commonly needed information about the object and would play that immediately after the object's name finally the in Universe descriptions were placed at the end due to typically being the longest component of the description and including a lot of lore uh and this format was more logical and flowed more nicely uh the audio AIO that would play

uh or as an example the audio that would play uh you can look at the uh if you see the uh image on this slide uh it would now sound something like hovering star Shades sunglasses with star-shaped lenses clearly the only accessory for the next ultraviolet but even when placed at the end that in Universe description proved to be a source of confusion and frustration some players would get hung up thinking that jokes and flavor text were important Clues to beating the game and after talking with James wrath who was both one of our testers and an accessibility consultant in his own right he recommended that we follow the guidelines used for image alt text on the web and remove information that wasn't important at first we were hesitant to take this suggestion since since this information was available to

cited players through other uh areas and means in the game but when we took a step back to think about it it was clear that these in Universe descriptions provided very little actual value to the player and were doing more harm than good so we opted to remove them entirely this ended up being a worthwhile trade because in addition to improving the signal to noise ratio in the assist of descriptions this also meant that less text would have to be turned into speech speeding up the audio processing time and making everything feel more responsive it was a win-win after reorganizing the information and trimming the excess we observed that players were interacting with the feature and the game at large with a higher degree of confidence and speed knowing the information sorry knowing the information uh order allowed players to feel confident and canceling the audio description sooner and allowed players

to stay in that flow State additionally text to speech processing time and audio clip length were now shorter which seemed to keep players more engaged so all of these features seemed like they worked and we believed that they would help our players but there was one major outstanding problem that had been presented from the beginning turning on this assist mode it still required cited assistance and what good is an assist mode if it's inaccessible to the people who need it the mode was enabled by a light switch like toggle in the game uh in the game world where the player initially started as well as one that was available in the player's virtual backpack but players couldn't simply feel around for this switch like they could in the real world even if they knew it was there our work around for this uh during the earlier stages of accessibility testing was to provide a keyboard hot key that could be used to turn this mode on which worked well

enough for players using VR on their PC but we planned for this feature to ship on Standalone and console headsets where it was all but guaranteed that a player would not have a keyboard throughout the development process this issue weighed on our minds we needed a way to turn this mode on with only things that were common across all platforms and we wanted it to work with our existing accessible accessibility feature like our one-handed mode so we took stock of what we still had left to work with which amounted to one spatially tracked controller and one spatially tracked headset it didn't seem like a lot until we turn the problem on its head if players couldn't figure out where they were in our world maybe we could figure out where they were in their own we had forgotten one of the things that makes VR special proprioception for those who are

unfamiliar with the term proprioception is the sense a person has of where their body is in relation to itself it's why if I asked you to close your eyes and touch your nose you could once we began considering a player's propri reception it meant that both the game and the player could know where the controller was relative to the headset with this information we could ask the player to perform uh an action relative to their own body attempt to detect that action and turn on the vision assistance mode for them but what kind of actions could we consistently identify gestures were one option around the time of the vision accessibility update being in development I had been slowly playing through Horizon forbidden West in my spare time and one day while I was playing it hit me in the game the main character uses a small device worn near her ear that she Taps to get additional

context uh information about the world the answer had been in front of my face for months we could infer the approximate location of the player's head and have the player perform an action near their ear we already were using a similar action to allow the player to pull their backpack from over their shoulder though so grab wasn't an option we could however have the player double tap the assist button this idea worked and the result was that players now could activate the mode without sighted assistance kind of in truth we should have done a better job making sure players knew right away about this method of activating the vision assistance mode there is explanatory audio that tells the player about this shortcut however it doesn't trigger automatically upon starting the game so unless a player had heard about this action through Word of Mouth they might still need sighted assistance and though this may not have been the

ideal situation blind players faced another challenge before even that one launching the game unfortunately no matter what we did with this update certain players would still require assistance to navigate the process of setting up a headset and launching the game so with complete Independence still unavailable to some players we decided that we would keep this feature as is for the time being we had to ship it and hope it still provided benefit to some players which meant it was time for us to face the scariest question was it all worth it would this update actually help blind and low vision players who wanted experience cosmon as high while conducting external play tests with people who were legally blind or had low vision we sent them a survey afterwards to get their sentiment about this mode we were ecstatic to see that when asked if they would recommend this feature to a friend or colleague who also has low vision their results on

a scale from 1 to five with five being highly likely to recommend were mostly five no one was on the lower range of one and two when talking to accessibility experts we heard a lot of praise and got a lot of great feedback that would help us create even better verions and have better insights for future games looking at March 14th of last year that's when we pushed the button we shipped it we shipped the vision accessibility update to steam for PC VR Quest 2 and Playstation VR to from that point forward we've had a steady amount of people reach out to us over time people who played cosmon as high and played with the feature they let us know that they found the feature useful and have expressed their gratitude for making a game that they are able to play we receiv received some periodic updates from meta about how many TTS requests the system had received a

request is made each time a player uses that accessibility button to hear a description when they teleport with that Vision assistance on and for March through September 2023 9 million text to speech requests were made and that's just coming from Quest data alone I've heard other developers ask why even do this if they can't see it maybe it's not meant for them and I would point out to those people that first of all excluding people because of their disabilities is just wrong people who are blind also play games additionally we have proof that blind people want to play VR too and I'm pretty sure we are all here because we like to make games we want as many people to enjoy them and experience that feeling that comes with gaming so going back to what we said early on we at Alchemy Labs want to make VR for

everyone and everyone includes people with disabilities so let's go over with uh go over where we at now with VR accessibility is VR accessible to everyone no absolutely not the good news is that it's improving and we're seeing progress all around we learned so much from this experience and there's way more that we didn't have time to cover our hope is that by educating you about some of the ways we approached solving the challenges that blind and low vision players face you'll be able to work uh uh to build upon this work and help us make VR a more inclusive medium thank you if anyone has any questions or wants to connect with us we'll be in the rup rooms outside he

[Music]