So You Want To Be in QA - The Test Chamber - Extra Credits

Quality assurance is a frequently misunderstood, often maligned branch of the games industry, but more goes into testing than even some industry folks realize. Also, this is Josh. He's doing the art today, so say hi. From the outside, people often mistake game testing for playing games all day until something breaks. But really, much of the testers job happens outside the game itself. To do this topic justice, we have brought in veteran QA specialist David Gilland to help us explain the various kinds of quality assurance jobs out there, QA's future, and how games testing is a literal science. First, let's talk about how testing works. The simplest way that a tester finds bugs is the same way that players do. They play the game until something breaks. This is called ad hoc testing, and it's what most outsiders imagine the entire QA job to be. But this type of testing is actually fairly rare. QA folks don't just play games aimlessly. They go in

there with a laser focus on specific scenarios. If they're testing an RTS, maybe they'll be tasked with using every unit on every map. For a shooter, maybe they'll have to fire every gun and make sure it goes pew pew. But breaking the game is actually where the job starts. Once a tester finds a bug, it's important that they be able to describe exactly what happened, where, and when. But even more so, it's important that they be able to explain why. And while it is impossible for a tester to have a master level understanding of the games programming, scripting, and art, most of the time you can use deductive reasoning to come to a pretty good guess as to why a bug might have occurred. Enter the scientific method. As a tester, it is your job to gather evidence, form hypotheses, and then test those hypotheses until you can formulate a working theory of that bug source. Imagine you're testing a level. You jump over a fence and you see an enemy. You shoot it the foot and the game crashed. All right, why exactly did the game

crash? Was it jumping over the fence that did it? Maybe that was a place the player wasn't supposed to be able to reach, or maybe it was something to do with the enemy AI, or one of literally hundreds of other systems that could easily crash even a simple game. So, you start to gather evidence. You try to repeat exactly what you did leading up to the bug as closely as possible. Just like when scientists repeat experiments to gather more data. Then, once you've done that for a while and can reliably reproduce the result of your experiment, in this case the game crashing, you can then try to isolate which variable is responsible. So, you mix things up a bit. Maybe you shoot the enemy with a different weapon, or you lob a grenade instead so you don't have to jump over the fence first. Your goal is to find the shortest, most precise number of steps that will reliably reproduce the bug. These are what are called repro steps. Then, once you have those, you can file a bug in the database where it'll be handed off to somebody on the development team to fix. Then, eventually, once the dev has checked in a fix for the problem, that report will

come back to you as resolved, at which point it's up to you to go verify that fix and either close the ticket or send it back to the developer to be fixed again. So, how does one get into quality assurance? When it comes to breaking into the industry, most of these discipline specific episodes have focused on the skills needed for that discipline, but this one's going to be a little bit different. See, just landing any games testing job isn't super hard. Heck, some testing labs out there will hire anybody who can pass a drug test. A lot of these employers will promise an exciting industry job saying, "Come on, play games for a living." before dropping those new hires in front of a game and just expecting bugs to pop out. This is kind of like being offered the role of a swashbuckling pirate only to end up chained to an oar and rowing for eternity. Do not work at these places. I promise, it's not worth it. They may seem like a good place to get your foot in the door, but they often don't give you any of the skills or connections you'll need to be successful in a long-term testing or game development

career. Besides, QA doesn't have to be a miserable job. Testing is probably the most varied discipline in the games industry, and the best way to find a gig you're happy with is to connect with the right team. A lot of people make the rookie mistake of applying only to studios that make their favorite games, but imagine watching your favorite movie two or three times a day, every day for a year. I don't care who you are, no one likes Star Wars that much. Prioritize team over title, you'll be glad you did. Better still, find a QA job where you'll be working in the same building as the developers. Half the job is really about communicating, and being able to meet with developers in person will let you create the working relationships you need to be truly productive. Unfortunately, QA is often the first group of people to be outsourced to another office, or even another country, especially as dev teams become increasingly globalized. This results in a lot of developers never getting to know the people who send them bugs, and even viewing them in an adversarial

manner. If you've been working 60-hour weeks for 3 months trying to implement a feature, only to get sent a bunch of bugs from some anonymous tester living in a city you've never heard of, it's pretty easy to start getting irritable. Likewise, it's easy for testers to start taking it personally when developers or producers decide not to fix their bugs. Obviously, development resources aren't infinite, and everybody's got deadlines, but testers can grow attached to their bugs in much the same way that developers grow attached to their own work. And if you have spent days perfectly reproducing and documenting a problem that the developer then says they won't be fixing, it's easy to get cranky. But, that's just a bug in human nature, so try not to be that coworker. Anyway, now that you know what a QA person does and how to find a good QA job, let's talk about the future of testing. Games QA is shifting very quickly right now. Some studios like Valve don't even have dedicated testers. Instead, the developers spend a portion of each week playing the game, discussing it, and calling out any bugs

they see. It's always been a standard part of the development process to a certain extent, but it's increasingly becoming part of the quality assurance process at other studios as well, both large and small. Combine that with the increasing use of automation in testing, and I suspect that those large testing pins I warned you about earlier will start looking a lot smaller in 5 to 10 years. Now, don't get me wrong, there will probably always be a handful of dedicated testers working on any large project, but the brute force demand for armies of inexperienced testers is likely going to decline with time. But, that said, it's still a great time to hone your skills and become one of the experienced testers, because QA can be incredibly rewarding. Taking a level or a feature you've been assigned and working to make it as polished as it can be is often just as fulfilling for the tester as a bit of brilliant design, code, or art can be to a dev. QA is a difficult career. A lot of testers either transition to different game dev disciplines or leave the industry

entirely before too long. But, if you find a good fit on the right team, this game career can be as rewarding and satisfying as any other. Thanks again to David and Josh for the help. We'll see you next week.