Improving Games Accessibility in Children's Games Through QA

[Music] hi and welcome to this talk titled improving games accessibility in children's games through qa which now i'm trying to say it out loud is a bit of a mouthful um i hope you're all having a great gdc so far my name is hannah bunce and i am a test engineer at the bbc over in the uk and just some housekeeping before we begin please turn all mobile phones tablets anything that might make any noise to silent and as i'm sure you're all aware you will be sent evaluation and surveys for session evaluations at the end of this session please fill that in especially if you like what i'm talking about and want to see more of this in future so a bit of an overview of the session i'm going to give a quick introduction to myself accessibility and the bbc children's

games team an overview of our problem areas within our accessibility testing and qa our current solutions to these problems and then some i'm going to leave you with some future tips and tricks for safeguarding against these problems and how hopefully you might be inspired to do the same in your studios and companies so um to introduce as i said to introduce myself my name is hannah bunz um i have been a qa tester for about four years now i have been working at the bbc as both a test engineer and what is known as an accessibility champion for 18 months and i've been interested in accessibility for the better part of a decade now when i started discovering that i needed subtitles to hear what was going on in games so what is accessibility um accessibility has a number of definitions

in terms of uh general terms accessibility is about allowing people to access a thing regardless of what their thing is and regardless of their abilities disabilities age race gender anything that might be a barrier to entry and in terms of games it's about allowing people from all walks of life to access and to play games and which as i'm sure quite a few of you might be aware of and agree with that the more people who play your games the better it is for everybody especially if you make money from people playing those games so what is games at children's bbc for those of you who might not be aware or who have no knowledge of the bbc it stands for british broadcasting corporation and it is a publicly funded organism

corporation in the uk it has an obligation to provide entertainment information and education to the british public through a variety of platforms and for audiences of all ages we do a lot more than just make doctor who and sherlock i can assure you of that and you might be aware of some of our products like iplayer the bbc sport websites apps we cover a wide range of content and for a wide range of audiences but what we specifically do at children's bbc with regards to games is we provide games for children of children aged between two and twelve through these two brands cbbs and cbbc cbvs is aimed at preschool kids so between the ages of two to six and cbc is aimed at slightly older audience of between six and twelve of course part of this service is

providing entertainment including people with disabilities so in order to do this we have to make sure that our games are as accessible as possible so to give you an idea of how busy we are and how many games we do we on average we release three games a month which adds up to quite a lot and uh recently our budgets have been expanding and as these have been expanding so have the breadth and complexity of the games naturally this has had an effect on qa and qa resourcing and also on top of developing and testing these three games a month we also develop and maintain an in-house interaction engine called iva any british people in the audience might get that joke i didn't when i first started working there and so ah crap sorry um

so when we started looking at our accessibility qa we noticed a problem which is with this game called something special the looking game this was released in the spring of 2016 and it was designed to be a completely bright blind friendly game primarily through the use of screen readers and in-game audio the game which if we were in the uk i'd be able to actually show you unfortunately can't access the game outside of the uk it's a very simple hidden object game uh the main character mr tumble who is a hit for kitten with kids of all ages and also some adults with various disabilities absolutely love this character and he decide well he doesn't decide to lose objects he just loses objects um and he's enlisted your help to find them around his house

when you click on an object to find you're shown a photorealistic representation of the object and then also the makaton symbol for this object the object is then animated to hide behind one of three buttons which are a red blue and green button we also provide a colorblind mode of this game where the object will hide behind a square circle or triangle button so we don't rely just on color so when this game went live we thought great we were shouting about it from the rooftops like this was our first ever completely blind friendly game so a couple of weeks later we received a complaint from a blind player stating that the game could not be accessed via voice over on ios it was fine on talkback on android but just not voice over on ios so after much panicked investigation between myself the project manager and my test

lead who were the only people really assigned to the project at the time we discovered that the final build we tested actually broke the screen order functionality this final build that we received came in three hours before the game had to go live so basically zero time for proper regression testing or any kind of testing at that point we only had time to say yep this game loads on a few devices it doesn't just crash and burn so we went live with it um so this was is a horrible thing for our team to have to go through it did kickstart the improvement period that we'd so desperately needed around the same time development started on a new cbb's app called playtime island again it's only available in the uk i apologize as it's quite a fun app um so the purpose of this game was to bring our html5 games into the hands of

audiences so parents could download our games put them in the app and then take their kids on long car journeys and keep them quiet so now we not only needed to consider games accessibility but also app accessibility and at times these can be two completely different beasts so this introduction of the app also doubled our qa requirements one certification period for games to go on the website and one certification period to go into the app and that's not including the actual app testing itself so naturally we ramped up recruitment and within a few months we went from a team of three testers to six which in turn meant doubling the accessibility test in training and whilst the recruitment was going on we also had some issues behind the scenes with guidelines so the bbc as a whole has a wealth of

guidelines for digital accessibility these are used throughout our entire digital portfolio however very few of these guidelines actually apply to games which caused a lot of confusion for some of our developers a lot of confusion for the designers and naturally a lot of confusion for us testers so our a lot of our improvements actually came from compiling a games accessibility document from the ground up with the help of the bbc's digital accessibility team and the resources available from sites such as games accessibilityguidelines.com and we also found it was actually a great way to get ux and qa working together as these two departments very rarely interact so qa is the final boss before going live but it doesn't need to be we now have qa looking at projects from kickoff meetings through to prototypes to alpha stage

beta and then final release candidate so qa are involved from the very beginning and now we also have time dedicated and kick-off meetings for qa and also accessibility and then accessibility qa so this was one of the major improvements that we started was we got buy-in from the project managers the product managers and our content producers to get qa in from the ground up so one of our other solutions at the moment is a lot of documentation so not much has changed in the practical aspect of our testing we still do our manual testing much the same way as we did before and we are currently looking at automation solutions so the more effective changes have come from implementing more robust processes we adopted a certification period very similar to what you would

see from sony microsoft and nintendo we implemented this about 12 months ago and it has been a major hit with all of our developers um so these are just basically a set of technical tests that must be passed before game is clear to go live and the biggest of these tests the biggest section is accessibility which is what this first screenshot is is just a snippet of our accessibility test cases in test rail these test cases just live there they are generic enough to be used across multiple projects at multiple times and they are written in such a way that any tester can come in and could just understand what needs to be done straight away the second screenshot i have there is just the title page of our children's games accessibility guidelines and recommendations which is a document that gets sent out to any of our third-party developers

our internal developers our ux designers and basically anyone involved in the project and these all um these documents are all hosted in a place where anybody who might even be thinking about doing a game for us can actually read them and get on board with our ideals before development even begins so more uh improvements that we've made um these are what i tend to call the four c's because they're just four things that will be called the letter c and the first of which is collaboration so over the last year we have undertaken two massive research projects in a crossover between qa and ux the first of these research projects was in collaboration with the university of edinburgh in scotland with experts in autism the second project that we did was a collaboration with two external accessibility consultants

to do an audit of all of our games which at the time there were over 200 games that these um consultants helped us audit to see how many of our games could be played with one or two switches the first project we found some interesting things that i can't talk about too much today unfortunately with how to design for kids with autism and how to keep them included we went to a bunch of special schools and we gained some major insights and it was worthwhile having a qa tester there to go back and report back to um the rest of the qa team because we found this helps increase the understanding of how and why things have been designed in a particular way which in turn improves the quality of bugs that developers get and just all around improves the quality of the project so the second thing is to champion accessibility at all levels at the bbc we have a accessibility

champion network which at the moment is over 100 people strong and these are people from all walks of the project hierarchy so from heads of departments all the way down to developers and testers and um consist these people consist of a wide range of technical backgrounds as well so whilst creating a network might be impractical within a game studio there's nothing wrong with being a champion of accessibility yourself the way we like to think of things is that if you can see a way for your games or your product to be more accessible for your players speak up the results are nearly always worth it and then finally on to consistency we found that having documents written in consistent ways inconsistent layouts helps everybody to understand what we were trying to pivot and portray

in these documents and then communication communication is absolutely key in qa and accessibility because whilst common sense normally prevails this is a point that most tend to forget the open communication lines between the games team and the apps team and then the apps teams are third-party developers and the apps team and the qa team basically see the picture with open communication lines with all different teams and they've been unparalleled in helping everybody improve the accessibility of our games and since implementing all of these we've not had a single accessibility complaint and we have released well over 50 games since that original campaign came in so as i said earlier accessibility testing and accessibility design are now actual agenda items on project kickoff meetings and we have found that accessibility issues found during certification of these projects

of these games have decreased especially with developers who show enthusiasm for accessibility during these kickoff mixins another iqa team has uh is now larger than it's ever been before we used to have monthly qa meetings they're only about 10 to 20 minutes long just just a quick catch of say oh what have you been testing this month and et cetera et cetera we have now increased this to weekly and we've increased them to 45 minutes um these allow us to go into more technical details of what we've been testing and how we've been testing it and it also allows for um for us to share new accessibility tips and tricks and to be passed on to the team um yeah so um the future there's obviously always more room for improvement and whilst we're in a better position than we've ever been with regards to accessibility and also

with regards to qa we are always looking to make things better so the first thing that um we want to do is more training we are ramping up recruitment again and naturally we want people to be on board with our accessibility ideals from day one so what we're looking to do is when we have when we hire a new tester as well as getting them on board with all of our normal processes you can go through them with accessibility testing give them a few tasks to get their feet wet and just basically help them from day one this should be common sense but again maintaining documentation is something that our games team struggle with and i'm sure is a ongoing issue with many other studios we have taken it on ourselves in qa to maintain these documentations because normally we're the ones that need it the

most and if no one else is going to do it we might as well do it so something that is quite what i think is quite unique to the bbc is having specialized working groups i founded a working group called alicats which stands for accessibility champion testers in the summer of last year people who are invited to this group are people who are test engineers qa analysts but are also part of the bbc's accessibility champion networks so we get together on a bi-monthly basis and we learn things from each other about like uh so for example um sport some testers from sport had a new way of doing automation accessibility testing which then they shared with people from another department and now that other departments that their products accessibility has just skyrocketed

and we like to emphasize communication and and said earlier we found that with having open lines of communication whether it's just be going up to someone's desk or through more formal means having these open lines of communication is just helping everybody and then finally we want to review our processes on a more regular basis i recommend something between two to three months to review your processes because if you review them say every two weeks that's not not enough time for new processes to sink in for you to actually see if they're working or not thank you i did go through that quite quickly i do apologize any questions if you have any questions there's a microphone there please use it

hi there hey uh first of all thanks for your talk thank you um i was wondering um i'm from linebridge game services we work a lot with localization and other languages so i'm kind of wondering um how much you put under this umbrella accessibility as far as people who speak languages other than english and that is a very good question that i actually answered um afterwards so after a talk i did on monday i should actually put it in this sorry so as part of our remit to help the british public we also produce games for welsh gaelic and um irish and these do present unique challenges mostly to do with subtitles because um i don't know if you're familiar with welsh at all but compared to english there's a lot more characters so and we've had some unique uh challenges with regards to the size of subtitles and where to place

them on screens and i do have some amazing screenshots of a game that was just basically covered in subtitles in the welsh language you couldn't see the game at all um so with that we basically edited the script a little bit um obviously we got permission from all the editorial staff and the brand owners to do that but that's the only unique challenge we've had with regards to language is just with subtitles i guess part of my question is when it comes to folks who are there in immigrant populations who speak a language that isn't an official language yeah um so is that just a totally different department that works on that or is that part of your no we only look at english language things and then about um five or six games per year will be translated into welsh gaelic and irish which are the official languages in the uk gotcha okay thank you thanks hi i'm stephanie from vision game labs

we're based in wales actually so well she's a thing for us um we i was just wondering really when you were developing the things for visual impairment whether or not you had used anything to determine the level of visual impairment or or whether that's something that you would like to work on together and that is definitely something we would like to look at and we do a lot of testing around colour contrasting to help with people with low vision and we obviously because also with games for kids they're full of bright colors and high contrast anyway so we're sort of lucky in that regard that we're making games for kids and that's what kids like and the color contrast is our main thing but um also like the size of icons and everything because yet again for kids children's games for kids sorry um we like to keep the art quite simple and well defined which

again is kind of cheating for people with low vision but it helps everybody and i hope that answers your question about it yeah yeah yeah thank you bob and if you want to have a chat after please okay i'll be in the backup room afterwards thank you any other questions yep excellent thank you [Applause]