Superstar Session: Why QA Input During Development is Critical

[Music] thank you [Music] so this is a superstar session why QA input during development is critical with my guest Rafael torner QA manager gaming Solutions at transperfect nice to me nice to meet you as well you're all miked up yeah the clicker is right here perfect hopefully you have everything you need so which one is uh green for girl green to go back all right perfect sweet thank you all right great hi everyone nice to nice to see you here A

bunch of good looking people um I'll have to tell you that it's my first time speaking in front of complete stranger so sorry if there's anything uh that doesn't work out very well but here we go so my name is Rafael I am the QA manager for Trans perfect gaming Solutions I have been in QA localization quality assurance and functionality quality assurance for nearly 12 years now um and I've worked for a couple of vendors for a couple of developers and now I'm I'm heading operations at transperfect um so yeah we are here today to talk about the importance of QA in games uh or the import the importance of the inputs from the QA people that work in your projects uh this is something that you might have heard before and it's not really I'm not

here to tell to tell you all you have to listen to us or you know it's not about me trying to get you to just listen to QA for the sake of it there's obviously advantages and disadvantages and there's good times and bad times for QA now um well I guess I should have started with this slide um now okay the question what do I do to help developers mobile developers so we work with a plethora mobile developers all sizes from one man teams to larger Studios 20 40 50 100 people and you know some of the larger ones that you all know and love and we we give them the the knowledge that we bring the knowledge that we have in QA and other services as well but in my case it's QA and we bring this knowledge to them so they can uh uh so so they can skip

the the that learning process which takes often a lot of times and it uh it you know a learning process will cause you to have failures as well so we help them skip those failures and they go straight to having successful and and experienced QA people working on their projects um so as I mentioned uh I'm not here to talk about I'm not here to tell you just listen to QA but some of you if you're already developing or if you're familiar with development you'll know that QA uh a lot of times can be put into a category uh that you know can be seen as like it says in here that it's an obstacle something just to get through something that's not as desirable if you'll notice uh a lot of times QA people are not even

put in games credits and this is obviously to uh developers discretion of course I don't think there's a problem with that but just to show you that there is sometimes a a separation between the QA teams and the development teams so um to differentiate QA I think some developers think of it as just meeting the minimum requirements for release whether it's on Console whether it's on a PC but there's no real certification on PC so you can pretty much release anything you want and on mobile which you do have an entry point for for your game's quality right so a lot of developers just want to get to that minimum uh minimum level um and you also know that there's others other games other developers other Publishers

that do aim for that excellence in their game so there's obviously a difference in in the objectives of each game on each developer and it has to do with their return of investment what do they want out of their QA what do they want out of their results do they want to just release the game cash in and go on to the next one or do they want to release a game that works properly works as expected and keep keep working on it and keep releasing content for it and keep that product to life for a long time um so again it's it's about uh developers choose what do they want based on what do they want from their game right their return um so oh I think we're missing some text in there I'm gonna try to ah there you go some animation

um so for all games you will always have that same uh uh you know this this is true right when you have core functionality your people your games are not going to be happy it doesn't matter how hard uh you worked in your game it doesn't matter how much they enjoy the games that you make what publisher you are I mean you can for sure remember a few examples from the last couple of years of games that were supposed to be the most groundbreaking games in the uh in the decade and uh they their their functionality and the the QA was so poor that you know they got sued by in the by their investors right they got the games got pulled from the from the from the stores Microsoft said you know this game you can buy this game but it's crap right so how how how does that happen

on a game that's supposed to be the next best thing um so obviously when you have poor quality you have less player enjoyment which leads to poor reviews we gotta remember that uh it's way more likely for someone with a negative experience to leave a review than for someone with a positive experience to live a review because someone with a positive experience it's kind of what's expected right to have a positive experience a negative experience no one wants a negative experience so most likely you guys whenever you've had a negative experience you've left a bad review whether it's a restaurant a game a survey Etc so it's way more likely that you'll get bad reviews and when you get bad reviews people are not going to want to buy your game so in the end lack of uh QA lack of a lack of QA and a lack of this process or poor QA will directly impact your game and and

will lead to to poor sales and as we can see here uh this is this is something that I found out while I was doing the research um 20 of people will just leave your game with one bad experience I mean I have there's been some games that I played that I'm like nope this is not happening right so um that's that's the ultimate impact is that you're getting your your game is going to suffer right and uh your bottom line whatever it is it's going to suffer um same with uh linguistic SMS with functional with functional QA we also have localization QA or linguistic QA and um it's also a step that we see a lot of times people skipping especially on the mobile world people think okay you know it's in English everyone speaks English um I have news for you not everyone speaks English the fact that we uh uh uh a move around maybe France or in

an industry where we all speak English and and it's a uh uh it's a language that unites us a lot of people just don't do it or are not comfortable even if they understand it on TV or whatnot or they can understand or speak a little bit it doesn't mean that they're comfortable I work with plenty of people that are fully bilingual or trilingual and they will still play their games in French or in Italian or whatever language they they speak whatever's their main language right so it's important to have also the the localization process your Translation your QA um uh and yeah you know having those done poorly especially linguistic QA um can take you out of the experience right I mean this is this is an actual example of a released game on mobile right so you tell me if that doesn't

take you out of the experience it does right um so you know I think as as the takeaway and it looks like I'm reading but I'm not I did practice this a little bit um in the end when your QA done is correctly no one will notice it no one will see QA no one will see the benefits of it because things will work as expected no one will notice the placeholders that are not there no one will notice the game breaking bugs that are not there right that's the goal QA is invisible to the player and when it's visible then that's a problem because it wasn't done correctly uh uh and I think it's very important I mean if you guys are in development or thinking about it um that takeaway is I think probably the best thing right the a player will not notice the good QA

uh and uh um and that that that was changed that initially said actually um a good game uh good QA doesn't make a game fun but bad QA can make a good game bad right so I think that's the that's that's the takeaway that that you guys need to to leave with um so I think I need to hurry up um after I spoke to several uh game developments uh and Publishers uh management teams QA management teams and although this is really not a figure that anyone can verify because it's not really out there usually this is kind of kept in-house for a lot of these people um about one in 10 games are released with a QA saying yes your game's good to go so that means that the other nine QA

said I wouldn't think it's ready but there's obviously other things to take into account right like there's other decision makers that uh need to take a lot of other things into account so um games are actually being released without QA giving uh giving that green light or giving that uh approval if you will um so why is this happening right the decision makers they usually um can be people that are not really in the QA world or have no experience in QA and that's totally fine um but I think there's a misunderstanding and it says in there about the importance of QA a lot of times other aspects are giving a priority which is again totally fine like everyone releases their game as they want and everyone has their own priorities but

um to to to Really um to really listen to QA feedback uh you know it's it's important so um yeah it's it's it's weird it's a weird thing um so really quickly I'm gonna go through a quick storytelling uh we had uh a developer a mobile developer approaches they had some issues with uh with their game [Music] um where they were having issues with certain devices which then they later found out that it was lower specification devices um they missed this because they thought QA is not important or we can do a little bit of QA and that should be good so they used people that were not experts in QA that did not have the

tools that did not have the processes that did not have the equipment um so but you know they thought it's going to be good we tested on a few devices and it works well and you're good to go uh the problem is that you couldn't get past a certain point of their game and they really couldn't figure it uh they couldn't figure it out so they they came to us they uh they brought us this problem they told us what they needed what they wanted and we uh you know we have way more devices than they would so we went uh we went ahead did the test provided the the the information that they wanted um they had some test plans that they wanted us to to follow and ultimately it ended up being something that they did not or could not fix uh uh you know some things you can some things you can't um but what ended up happening is that they increase the minimum requirements for

the game and basically this problem disappeared because people that were having the issue were no longer able to access the game now I don't know what happened if they had any you know they had to return money or uh for microtransactions maybe um but the game went from a 1.5 stars or 1.5 reviews to 3.8 I think in the span of a few weeks right so more people were playing their game than they uh than they were when they had let's say a larger base right so they're base got smaller but more people were playing their game uh than than when they first released um so what do we need what do we need for for the future I think it's important that you guys and everyone that's that's developing that we are here for like we have the same goal

right like QA and any other department uh our goal is for your game to be the best that it can I've never met anyone in Korea that says I don't want that game to be successful I'm not going to report a bug no that we all want your game to succeed right so uh we're seeing a lot of people uh announcing this this change in the last couple years specifically we're seeing a lot of a lot of developers actually start bringing in QA earlier because they've probably noticed in the last decade so from the early 2010s to the late 2010s uh there was this trend of poor QA and then relying on patches and and you know just things really we got accustomed to games releasing like garbage and then it would take a year for a game to be uh properly played I mean you can still see them today but this is this seems to be changing people

uh we're getting a lot more requests to bring people to bring testers into uh into the development process earlier uh which are usually uh uh yes almost almost ready which are usually called Dev testers so we're seeing a lot of people uh ask for Dev testers ask for embedded testers uh we have teams that we've been working with developers from you know two two and a half years before the release of the game when usually QA takes place in the last year last six months or so so uh we're speaking to people to developers that want you know our inquiring about QA and they're three years from actually starting proper production right so there's uh there's definitely a shift in mentality uh why this is I don't know I think it has to do with just multiple failures from everyone uh uh but but there's change there's a change happening and I think us as Gamers because I'm I am a QA person but I'm I'm also a huge gamer and I think

I'm tired of just buying a game and it's not performing how it should so I think all the decision makers and everyone working on their games they're noticing this so there's uh there's this is changing um yeah so I mean I don't think I have any more time so uh that's it uh well I did miss a couple of things in there from the presentation animations are not good for presentations um oh that's a huge one really quick sorry so as you as you go right when you're bringing in earlier your QA team your QA will probably be all those things faster cheaper than better and better right so you're not going to have to have this massive team for three months and your your uh your development team is gonna have to go through a

backlog of things and and get swamped right so if you get a couple of people in early they can work next to your development team and you can address things in a more uh natural way right and it's going to be cheaper you're not going to have such a massive team and your end product is going to be better so that one's great that one's really really good uh any questions I think we're short on time yeah we've got to move on do catch up with Raphael on the show any questions find out what transperfect can do for your game speak to the man right here but uh thank you for that thank you thank you everyone and do make sure to uh visit us at the trans perfect booth and if you go back here you should find some cookies and some goodies so please uh please go ahead thank you very much for your time thank you have a lovely uh have a lovely day thank you for that