QA Career Path: Evolution from an Industry Entry Point

[Music] so welcome to QA as a career path Evolution from an industry entry point I am Bryce bradrick and my this is my fifth year in the game industry and I throughout my career I've been in a few different positions but mostly I've had a through line of QA work so starting from Testing mobile games in college to arcade games to party games and to now AAA development on an action game known as Destiny 2 at uh Bungie I am a test engineer now and all the skills that I'm going to be sharing we're both going to be sharing in this talk I've used personally in my career to set up my different progression so it's all going to be applicable skills I hope to all of you and hello I'm Christopher Rios and currently I'm the raids and dungeons

test lead over at Bungie I started off working in mobile tech where I got to test out some mobile apps then later became a product specialist within that field and then I started to take a lot of the Knowledge and Skills I learned from that to apply to game daav and I'm running around the foure plus Mark uh right now at Bungie and it's been absolutely fantastic yeah so as you can tell from this picture from 1960 we're a bit of a dynamic duo um maybe not that much but uh we do employ a people first Approach at Bungie for the QA field where we actually work with our functional testers to ensure that they are working on their career growth and the things that they want to achieve in the discipline it's not just a revolving door of testers it is US individually working with each to make sure that they are working on that skill progression we're going to be talking about in this talk and they're achieving the growth opportunities that they want to work towards um and this actually goes with

uh different things like giving them tools to succeed and ensuring one-on-one contact so that they are working in the right ways and that actually touches on a lot of the agenda of the talk today that we're going to go through which is we're going to start with how QA the QA career is changing from that industry entry point to something that could sustain a long-term career then we're going to go into why you would consider a QA career why would you like to do Game Dev in QA your entire career and we're going to present you with three clear benefits that we've used in our uh career as well as other Professionals in the industry then we're going to go into sustaining that career if you get a job in testing you start what is going to keep you going in the field what is going to progress you to that next step and we're also going to give you tools for finding your correct job and what you would like to do in the industry yeah and I'm really excited for this too because we're going to be talking about some of our stories of how

we got here we're going to talk about some of the things that like we have done specifically at Bungie as well that kind of correlate to the the many talking points here and then wrap things up to where you feel like you have a lot of like resources and guidance to take the next step if you see that this is a really fun thing to do for you yeah so to start off what do I actually mean when I say QA as an entry point what is the misconception in the industry right now is that the primary purpose of QA is to go into to the discipline earn skills and then transfer out it's only a stepping stone in the industry it is not something that can fulfill that that 20-year career in Game Dev so there are different opportunities for growth that lots of people are not seeing at a perception level right now um this is kind of theorized due to the low barrier of Entry of the discipline where QA doesn't require a lot of technical skills to get in or shipped game experience which I would argue is a strength of the discipline but this can

lead to a lot of people wanting to do the work and when there's a large supply of people that want to do work companies can devalue the work that we do and the value is apparent to us that work in the field but this perception needs to change and so lots of companies feel they don't need to invest in the discipline because they can just get more people at this base level and that's where we see this lack of upper mobility and low pay it doesn't have to be this way and as you can see some people people think oh I'll just get into QA and then I'll be the director of the next AAA Blockbuster um it doesn't exactly work like that and it doesn't need to yeah as we like get into kind of just like how the perception has affected the discipline because like bry's talking about there's a conception where it's like you just go in there you run into a wall while you're play testing for 700 hours and it's like okay what do I really do from here is there really any growth opportunity in this field and there is and the spaces for those exist

uh and that's what we want to bring visibility about too because we see that a lot of times that folks go into the uh QA career path they stay for like maybe like a year or so uh and then they just start to transition out to like different disciplines um and that's not to say that you shouldn't be transitioning to other disciplines but we want to make sure that there is that visibility for that sustainable career path so instead of developing out of the discipline we get to show you the benefits of developing in the discipline to where you have that long-term like sustained growth and really good thing about this too is that the discipline has been evolving it does continue to evolved and it's not So Much Anymore of the perception of okay QA just kind of just like looks at like the end product and they just like help us like close the project we're at the beginning of the project we add that value at the beginning of the project um for the entire completion of it by quate creating these Quality Systems and becoming a force multiplier for the teams and we're going to be touching on that point a little bit it's a force multiplier that's going to be the key

word uh in a lot of this yeah so if you wonder how we stand on this discussion everyone in QA as a game developer and you know who said that Cade 6 and that is Canon now so um I mean don't don't challenge me on that because I don't think I got legal sign off but anyway um narrative is going to yell at us what I want to touch on is just that QA is being included in more facets of production because people see the value just at at Bungie we have all these specifications of QA that can nurture your interests and give you an equal seat at the table of all these different disciplines it's not so much now where we are attached to the end as Creos is saying we are actively in from pre-production to ship and all of these different places are seeing the value and it's expanding out I've seen it other places other than Bungie but this is just at Bungie all the different specifications you can go into um and this actually touches on one

of our first major benefits of working in QA which is the flexibility of it QA is a really unique where there you don't need the technical experience to do to create Assets in order to meaningfully contribute to different work um you can actually do it with the interest to explore it so you don't need that technical expertise when other disciplines you generally need enough technical experience to create an asset that's not the case in QA intro knowledge and interest is really helpful for the work that we do um so adding how learning how a system works and adding that to your core QA responsibilities will create really important input that you can work with others to uh affect the production and this is really allowing the disciplines to work on that creative aspect while you were ensuring the quality of the products they create and this actually gives you a really unique ability to explore within

the QA field so you if you have other interests in different disciplines you don't need to transfer out of QA to explore those you can actually meaningfully contribute to all of those products while still being in the QA field so you you can nurture all the different interests that you have and work in QA your entire career yeah and the keyw for like this slide in particular is curiosity CU curiosity especially as you're starting out in the discipline and growing within that discipline that's going to be the big propeller in your advancement into the career because curiosity is going to spark your interest and exploring these problem spaces is really going going to have you engage in working with people and really like diving into like unique solutions to creative and complex problems curiosity is going to be like the biggest part uh in that so a way that I've actually used this in my uh career is uh when I worked on the jackbox party pack 8 and the game weapons drawn I came into jackbox with a

a screenwriting degree I I studied screenwriting school and this was something that really interested me at jackbox because it is a very narrative focused company you know we have our different uh narrators and stuff like that and jokes and so it's very dialog focused and this I had a big interest in that and when I was kind of dig digging around in dialogue tables and trying to learn this different things writers took notice of my interest and we worked together in a technical way to learn the software that they were using so then I was able to apply that interest in that new technical work to my QA work where now I was looking for more meaningful results in test cases and I was Finding fixes and bugs much more streamlined it was making the The Writer's jobs much easier and this would led to more um collaboration in the future where I was actually able to work with the quality of the narrative content coming out as I

was now a point of contact for that kind of work and this is all within QA but I did eventually even write on a game because I was working this closely with narrative so if you ever stab someone in weapons drawn you can thank me a little bit for it yeah now I get to talk a little bit about how I came into the career and how I had a little bit of that flexibility as well because when I first started off at Bungie specifically I started off at testing the strikes and the public events if you're familiar with Destiny 2 but a lot of the open world activities a lot of the activities that you launch into and during that time I saw a really like really interesting area of investment which is typically the rewards that you get the achievements that you earn the player pursuit of things and the quests that you engage in and I saw that that was an opportunity that I was really curious in and with that Curiosity I remember going to my test discipline manager and talking to

them about like oh I really want to like dive more into this space and just like how to uh explore the systems with it how to debug them how to see how uh the things are made and the problem spases for it and that's what the Curiosity led me to is to start to build that understanding of the systems that the content designers use and learning their workflows and how we can uh benefit uh the ways of reducing the amount of bugs that we find or building book strategies for our life support or during the development of our game and that's where it's like we get into the Nuance understanding of it to where you can really apply and start to see in real time your impacts to the game and then being able to master that understanding as well to where you can start to improve the efficiency of the development of the content um by making it easier more efficient and let's work overall to start building these systems uh for it

and that leads us into career benefit number two player advocacy which is one of our personal favorites uh for this um and BR and I are going to be able to talk about a lot about this to where we got to talk about like some of the quality systems that we were able to implement in some of our testing uh at Bungie and when we get into player advocacy we really get to explore the player lens a lot more with it and bring that player lens into the project itself at the very beginning of it so we get to work with our discipline peers to start to Define these unique experiences that we want to have players engage in and we get to not only verify that the thing is being built and that it works but we get to validate it as well are we building the right thing uh for it and starting to build qualitative metrics around those things of not only just saying like yep we fixed a trillion bucks this is perfect but those trollan bugs entail

to the overall experience of it if you're making a heart encounter are the things like increasing the difficulty of it are the combatants engage are you feeling pressured in multiple aspects of the fight things like that those are the experiences that we want to advocate for as we're building the content and providing that unique lens uh for it and you might have seen this popular video before of a very very classic example of someone building something with a specific intent in mind and then someone else being being able to add that unique perspective that unique lens to how players can approach uh these problem spaces for it uh and that's exactly one of the reasons why we're here uh to help provide that digal perspective uh for our content development because implementation is one thing but being able to validate that implementation is a whole separate thing alog together uh and we add the value for that and as we start to build the confidence uh for the decisions for or the player uh we really

get to explore these Quality Systems uh to where we get to align on a lot of these things with our uh cross discipline peers and again build these Quality Systems and what we're going to be talking about like specifically in a moment here is that uh for rids and dungeons which Bryce was also on my team during that time uh where we got to work on cotus end together uh for our content so we're going to be able to talk about kind of just like the explorations of building a quality system to ensure that we're meeting the challenge bar for right we heard the community that they want the challenge and I think we were able to deliver it I think Joe Blackburn our prior game director he made a funny tweet of cro just doing like a deadlift Squad uh it's like yeah he he went to the gym for sure uh to make sure that there was like a good uh challenge being presented uh to players uh and Bryce is able to talk a lot about this because he was part in being able to refine and develop this quality system that we have been using uh which we dubbed folcrom which is just a hardcore audience of

players within the company that we get to uh come together for a play test ritual to where we validate those experiences yeah so as Creos mentioned this fulcrum was like a collaboration between QA and design to understand what a player means really when they say what is hard like if this is hard what do they mean by that and how do we use design ideas to enhance that experience so in the place of croa uh we were looking to make it much harder because the original one was kind of a pushover you could solo it in Destiny one we didn't want to carry this along to Destiny 2 we wanted to present a new challenge but we wanted to do this with mechanics we didn't want to just boost the difficulty of all the enemies and then you know that's not really challenging the player in meaningful ways so what we did was we started with sourcing the the people that we wanted to challenge which was uh these high skill players that are used to highly

individual uh play where they don't actually need their teammates to succeed they can do things like soloing dungeons or even low Manning raids so once we sourced this pool of people we presented a simple design idea to challenge the way that they play the game which was what if you have to play Hot Potato while moving through a raid and in this situation it's immediately contrasting how they like to play where they have to actively work with their their teammates to succeed and once we found the ways that they were failing we could track these ways so all these different you know uh situations they were getting themselves into we could track how they're failing if it is frustrating or fun and if it is challenging in an engaging way and once we kind of saw these different things in the play test we could enhance these experiences so what if we threw a trap here would that frustrate them would that be really fun um and it really allowed us to on the same understanding between QA and design

so we could start making those design decisions to enhance that difficulty we were achieving so finally uh we we really dug into this and I felt that we made a lot of good design decisions and we validated them right through these different play tests to make sure that the player was having a good experience and as Creo said I think that we achieved that croa was really well received in Destiny 2 and it was much much harder and meaningful ways uh to our player base yeah an example for this as well is that for design and QA and all the disciplines that come together we come together into the ruin and we Define okay we want this experience to require a lot more Co-op uh collaboration to go through the encounter space design gets to really enforce that idea and they get to implement the idea and when it comes into like the QA role in all this too and how we like build these Quality Systems is that we start to analyze the player trends of how this group of folr is gr through the content even outside of fol Room 2 right to where we start to

see um what types of player behaviors and player cohorts are going to be engaging in this type of content and how they can circumvent it and know Bryce was able to find like a lot of opportunities during the raid two where just like yeah I could still solo this for example and find a a complete unique way to go through the content of it right uh so Bryce was really able to have that Prov provide that additional player lens in that perspective being able to analyze the trends that we were seeing in our play test to help refine and accomplish the goal that all of us set out to have yeah and this kind of goes into our third QA benefit which is cross-discipline collaboration um QA is in an interesting spot where everything we touch and everything we will be applying our work to will be other disciplines content we don't work in a box and this is unique because other disciplines generally have to do a specific collaboration you know maybe you're you're a designer and you're taking some audio and you're putting it into your encounter and

that's that's kind of a one interaction that you do but for QA it happens all the time because everything we're touching is other people's assets so this allows constant communication with different departmental leads and if you ensure that this communication is direct sustain and effective it will establish a trust that is really important and taking that next step from testing to Quality Management and this forms into super important production work so as I kind of mentioned the unique impact that we Define with this trust is what we best Define it as is qual that quality management so instead of verifying the completed work as it comes out you are impacting the creation by managing the quality of what's being created um Bungie actually does this at a studio level where we have um a body of QA in different disciplines that all dictate what is going into a certain release so will assign risk to all work we will advocate for fixes or different

things that will be fixed based on the trust that we've been given from other disciplines and this actually makes it so that there is much more ability for the the disciplines to focus on creation and then we can make sure they're hitting their quality standards um based on our judgment and trust that we've gained a good example of this I'm showing on the screen right here is the Spire of The Watcher dungeon uh we found an out of environment issue where there was this really flat big flash of light that was coming very suddenly I've slowed it down here but that is generally something we wouldn't fix it's out of environments not many people are going to hit but we in QA identified it as a photosensitivity risk so when we did that identification we were able to advocate for it and get it fixed when generally something like that may not be fixed yeah and I will say for a lot of this too it goes into the topic of the force multiplier aspects of development where you're really engaging like these teams and you're being able to build up

that trust that Bryce was talking about of how you go through risk escalation how you go for player advocacy and seeing the real time impacts of that is all about being a force multiplier for the team for the project uh and for all the experiences that you're making and yeah now we get to talk about uh the QA job crash course so we talked about a lot of the benefits in this we gave a perspective of our stories and how we do things in development and now we get to talk about about okay how do I start in this what does the career path really entail and look like uh in this and we get to start to explore uh like how to uh begin that train for it of how you can start to really foresee the future of not only just like the first and second year but how you go through the different positions in QA and starting to see that a lot of the aspects of career development you can start building now uh for it you can start to build up that foundation for it and for some of the following examples in this we can think about it as as you're going into let's say an associate

tested role you might have the opportunity to start to own something and that's going to be the reoccurring theme Here is that ownership is going to be really powerful in this whether it is owning a small feature let's say like one encounter inside a raadar or dungeon or you get to own the entire feature and and say okay I'm going to own all the encounters for this or you can say I'm going to own all the raids and dungeons uh in this case and that's how you start to to build up the ownership of those uh aspects and you can see like through the flowchart of this is that we go through testing the product we start to familiarize ourselves with the content to become that subject matter expert in it and that's how you start to become that point of contact you get to build up the relationship with your cross- discipline peers and you can start to uh have those reports built in you can really start to Define and to showcase the quality and the trends of how the project is going and start to manage it a lot more and through the cycle of ownership

you're really taking advantage of four unique things you're taking advantage of the planning aspect of it how do you plan for the coverage from the beginning to the end of the project and then you start to go into the execution of it is that you got your plan you put it on your notebook and now you want to execute it you want to bring it to reality and just start to uh provide work items and actionable things to start to do through those Milestones through the development of the project for it and when you start to do that you have the opportunity to analyze saying okay how did these uh these things go once I implemented it in once I saw the results of it from the execution and then you start to iterate on it that cycle continues endlessly it continues for every single piece of work that you do and it demonstrates the horizontal growth that you get to achieve as you're going through this I know for a lot of the things that I've uh done on raising dungeons when I started off as an associate tester on the team one example exle was owning like parts of the uh

secrets in volto glass of how you achieve the weapon and the weapon Catalyst and started to own that system for it to where we get to really Define uh the bug outputs of things if we're building the right experiences uh for the players in mind keeping the secrecy intact keeping it fair and really correlating our goals uh to the actual outputs of the project and being able to Loop those conversations in with the feature owners uh for the project the design the engineers the artists the uh audio team the VFX team and everyone and that's how you really start to build up that trust and become the point of contact yeah so as Creos is talking about this skill progression is really important for all of your expanding out in the discipline so in order to hit this sometimes the company that you choose will be really beneficial to starting that skill progression and then keeping it going so I've designed this tool this is my example you can take a picture if you'd like uh we're going to

tell you how to set up your own but basically uh this keyword tool helps you find if a company is right for you and it will be good for your skill progression where it actually identifies based on common language and things like job descriptions um what it hints at how companies treat QA and if that aligns with your goals and values for the discipline so um this will actually how you set this up is that you can start with what's important to you in QA write all this down put in your goals put in how you want to work and then condense this to the common terminology the keywords once you condense this down it makes it very easy to compare it to job descriptions to score and see if this job is right for you is this hitting all those important aspects of QA work in your personal aspect um and once you can do this you can actually use this for a cover loader so you have already what is important to you and what is important to to the company if it aligns throw it

into a cover letter bring it up in an interview and of course the last step is dominate dominate everything in your path um also what's important for advocating for yourself in different ways is to know your unique QA impact and articulate that how do I articulate my specific impact this is another uh thing that you can use to to show that and what we call a mission St statement so the QA character sheet will help you find your unique impact in that mission statement it will tell companies what I can do that have not been taught what can I do that are soft skills that are really important to the work that QA does um and the character sheet will help you Market all of your your goals and your soft skills in a bite-sized form um that way you can kind of use it for the interview or cover letter of your choice uh this is what mine kind of generated out was I test games to this statement this is a bite-sized picture

of all my different soft skills all my different uh QA values and goals and it is really easy to take this and expand out into something like a cover letter or a job interview yeah and I'll say with that too that for the cover letter aspect of it that is frequently going to be the most compelling thing that recruiters and hiring managers see especially if you're entering an early in career position of it because that's your opportunity to really talk about your passion and your love for not only the company for the position itself and how you can add value to the project to the company with that uh those things in mind for like player advocacy uh for cross discipline collaboration and everything that we've been talking about that's our opportunity to really talk about it and we have another resource here as well uh big credits uh go to Willam dethol and uh Kath kman for the resources that they were able to make for this of having portfolio examples for QA and being able to see a job

posting uh Resource as well so you can start to review some companies as well so that way you have the opportunity to go into the next steps of seeing examples of working Professionals in the industry and how you can start to apply for these places as well um and yeah yeah so I just want to outline this with the content of this talk has a lot of how the industry views our work how does industry view QA and how our role is perceived it it allows them to value or devalue our our work and we know that the value is here and working in the ways that we've talked about in this talk will show that our apparent value is in QA we know it we need to everyone else to see it so working on your skill progression regardless of how much experience you have in the industry you are adding value to the the discipline so we want to see all of this different things in your career progression it is great for what we do so the future of QA is in this room and it's in the industry

right now so if you want to continue on your skill progression the structure is here and great QA professionals are here to help you go forward so if this interests you you are doing such important work for the discipline and for the perception of the industry yeah and to wrap things up for the takeaways in this so number one QA is a forc multiplier hopefully we got to see a little bit of that in action uh for this and it's not about just testing for the sake of just like finding bugs but the quality of that as well and for the last takeaway is that quality is an ownership for everyone that you work with right and for QA unique position is that you be get to Shepherd that quality you help to guide that quality for Everyone by building out those Quality Systems and one more time for everyone in the room uh everyone in QA is a game developer thank you Kate 6 so much thank you everyone we'll also be able to talk in the side

rooms up ahead uh thank you all so much for attending this uh this talk yeah if you have any questions we'll be doing QA in the side rooms just because we're out of time thank you all for coming [Music]