Building An Intelligent Game Testing System in Netease MMORPG Games

[Music] hi everyone an invention from 40 lab in daddy's company I'm pleased to be here to show you some contesting work with my partner so Lee who is problem security department the title of our presentation is building in Tunisian team testing system in that is MMORPG okay now let's begin foo CLM is the first organization focusing on the research and application of technology in human error the lab is founded in September 2017 our vision is to indict him with a ax we have 5 min search directions including reinforcement learning gyro profile computer vision virtual human and natural language processing besides we also build our date and computing

platform to support all the area research and application in testing is a in session part of gabled ramen it's kimchi medium test even testers are response for the team count in the to Emeril periods and our team published in tester can award operation accident bugs indeed we are just gonna spoil the game experience and cause economic loss for example Bioshock 2 a great deal it's bought several platform including Windows PlayStation and suppose however fire start to suffered from bad issues and in the year of 2010 2k games along that they will not provide patch to face these issues and stop to offer the turnable contest since the even can't work around this audacious and another lessons from China total ancient popular

game developed by LEDs the company in Shanghai over holidays the game officer is preview did a farewell to all the play as you however this time they made a big mistake they plan to give every player 20 sovereigns gold and 15 talent but the amount what we worst they give each player conscious RS diamonds and 50 course the diamonds were very valuable in the key this mistake caused our economic loss of five million so all the test shows that the Kim test is very important however on the other hand the Kim test often complained with their work the task is repeat and Iquitos the workload is hard and in the sternum is nonsense fast and so luckily current AI and machine learning technical progress

rapidly all these technicals such as evolutionary algorithm and reinforcement learning and a deep neural network may offer new possibility to build a more intelligent in testing system and this also what we want to share with all of you we explored and tested our measure in bad MMORPG game named justice it has became one of the most popular game in China since the notes from 2018 the main in place are in quest and game dungeons in dungeons you'll include PvE and PvP most of the a fight ancillary the testing work of Justice on AI is very hard the building system which generates two reverses invented each times not there are many doors involved including programmers designers and artists dollars there are more than

400 commit to him was not ideally all these three words needed to be tested invented and it will include more than 100 story and more than 160 hundred in place the workload is so heavily in practice it is impossible for our cutie to finish it with only seven so it shows the new launch always Anthony modified task to test however this may need is a game at risk me with his human neighbor company we have already achieved some with that and experienced on the regression test of quest and the class Benison test so today our presentation will focus on this two type of test the work okay let's begin with rather recent testing of quest I will invite my partner Tony to share the background of this work hi

my name is Shelly and I'm from QA Department and that is company yes I'm one of the QA for justice online next I will talk about the general testing work in Amity and the difficulties encountered for regression testing of class quest design in the cornerstone of MMO RPG game which helped players involved in the story and the familiar with the game world when you create a new character and enter the game the first thing you encounter is a cat or a mission it will teach you how to operate

how to walk and even how to kill a master besides you can learn many interesting stories by finishing them at the same time there are male 3 types of plot or until class injustice online main story side story and the encounter story no matter which type their main purpose is to tell available story two players the implementation of this class is relatively simple and at the same time the difficulty for players to complete is relatively low however to construct a colorful game world there

are a great number of days quest so it's really a big challenge for our QA to deal with these Kratz first the number of the class is large we have 200 LAN team master race without 171 side stories and 475 encounter stories each quest consists of several task steps step means an independent unit such as a code in you cast like talk with and PC o Q some masters for example a class may require players to talk with a PC a Q Master P and then go to somewhere this

means that the class constants three task steps our steps are in a certain order so the total number of steps in justice online is over 15,000 it takes 200 hours to complete all these quests just one time second the task steps are modified frequently especially for the lately design Kratz implementations between quests are covered even different class might influence each other unexpectedly so it's very

important to do regression testing regression testing for all paths is time-consuming and we don't have enough time and the labor to do this however if we don't do enough regression testing facts may be hidden in our game this is a contradiction in addition to manual testing we also have some automated methods to ensure the quality of the class for example we write test scripts to run test cases automatically to check if there are bugs similarly these scripts need to be written manually and the because of the frequent iterations

this script also need to be mental manually which is demanding this is an example how different casts may fool ins each other unexpectedly one day the came designer modifying the boundary of the map formation a unfortunately that map was also used in another game play a mission B because of the modification one of the kind of DC was out of boundary of the map in mission B so players couldn't walk to the NPCs position therefore mission B was blocked actually this park is very easy to be

detected if we to regression testing we have a complete set of automated testing process after a new-build version is packed a new task task is established it will deploy test environments load the script library and then execute the test scripts the title real results will be collected after all cases are done at last it will generate a test report and listen to everyone in the project in the script library there are not only scripts forecasts but also script place each script covers the

process of class I can play and have some checkpoints when the script cannot proceed as expected all the checkpoint fails we think there may be a bug and we need to check the problem manually most of our scripts are running on the tenth client and they simulate human players operation let's look at an example of how the test script works this is a part of test script for cut this crap require players to go to the river to fetch water and then come back and pour the water into a packet in our script we use

actions sequence to simulate player's actions player need to go to the river press FK to fetch the water with for fetching and then go back to the market use Adam to pour the water at last with four pouring like a flow chart will we read to the test script each action is written as shown in the picture however this method also have several disadvantages only one hand one passed Chris bounce to one test script so totally we need over 1000 attack scripts just for cats

yeah our current situation the coverage is very low and showing in the table unless some of Locris are covered especially for side story and encounter stories besides new cast in new test script in one of the past expansion pack updates we updated nearly 100 new cats so completing all scripts is a highly and continuous work on the other hand scripts are sensitive to iterations when designers modify the past test the script should be updated for example in the past one tasks tab

require players to talk with NPC a but now it has changed to an pcp under the script and the test script may not be able to fund an tcp and it will be blocked this mean that man turning test script is also time consuming so we really need a new solution to deal with this simple plotted or plot oriented hats okay thanks for Shaw's introduction so we did a loose lesson to deal with this simple and a Florentine quest where los zetas old tested is is a essentially code base or rule base we should write a scrip for all quest and if we can create a virtual

test who can complete the quest but itself yes will be great in order to implement such a water test we need to remodel problems of completing quests in gates we motor problem from the view of human player as the left port picture shows the human player plays a game by observing the screen state and performing actions through mouse and keyboard so first we define state and actions state start of current game informations they should be distinguishable between difference progress of quest actually I watch the player can to in the key injustice or not state concept of task info than max items in the back pan amis and so on all this can be reflected in the screen and

as they are important to decide which actions will be performed it should be loaded that the item in state get from the internal team engines gone from screen image which makes our measures more efficient in practice the action divided what the player can do in games it is not a simple operations about mouse and keyboard passes the function interface such as run to object which means go to the operation of of eject a candidate action including two part exit art and optional action parameters all actions can be called by our Edwin's and the implementation of these functions is provided by to a team okay our solutions of progress and tests can be divided into three words the first phrase know how to complete the quest

story inserting and a building quest graph at the same time the second of race where it directly use the graph to get the best path for quest and I replaced the action sequence in best path to perform a test in third place we speed up the searching for new quest since the problem is more than a Markov decision processes we can build a graph of quest by Monte Carlo tree search algorithms and shoo-in the left figures the load a is this start low and the loaded D is the terminal node mass rod and a low D it means that the quest is completed during the search interest if the envelope is allowed by the algorithm we can calculate the best path from A to D based on the graph in conclusion the best path is the shortest path which can

complete the quest with minimal steps in either example as shown in the right figures the shortest path is from A to B to C and Rob and at T we assume that if the game code is not chilled we can really complete the quest for me in the best pass again the key of the 30 a briam is to choose the candidate action AI for load as here we adopt the up competence bound string G in MCTS which chooses the maximum actions values of t i TI is initialized with eco provinces and updated according to the UCB ecosan the CI in the equation is indicating the children time of action area so the probability of actually I will decreased if you see I increase the

advantage of this the entity are twofold first it can't rid of the exploration and exploitation which is the historic problem and CTS second it can avoid infinitely loop between to load the number of all the candidates actions is more than 14 so such in the whole action space it's time-consuming in fact based on the simple rulers we can reduce the access base or not here is the example in the load informations we can see that the flag of the you are open is false so the Glickman's action is not available and if there is a item in the bag of tasks we should try use item actions and their parameter of these actions is the item id another is

problems if we will found two enemies around the player we may try to attack them all these rulers are not complicated but they show great effect in practice and crystal or debt this rulers give up all the legal actions of available actions and the counter law but all the legal actions has the equal initial probabilities in the next slide we will use an erp technicals to assign some more potential action with high properties among all the legal actions okay this is a workflow of our searching procedures where is all the information from the game and extract some key information to create state alone and generate action space for each lot the candidate actions including action type and its parameters and then which other actions to perform in the game two

functions interface and then the procedure is repeated again okay now for the second phrase how to use the load graph to perform regression test that is simple we just is to replay the best path in detail when we enter the load a which is the action well then we will enter that so the B and C and final arrival D if everything goes okay we will allow a determinant law and prove that the quest will not be blocked and it passed in the perkinston testing however in some case the situation is not always as expected when Todd planted cookies is there the game environment are non deterministic you may found that after you to X somewhere in load a you a lot and to the load P you into an

antelope to solve these problems we'll take two measures y2 is that we were a lot the side action in the best pathway the flow properties and we also give other actions to some smaller provenance on the other aspect us reload the property of all the legal actions are average and this make testing work more most meanwhile we did mind well across the pasture local testing according to rhetoric our request is not completed in the plain models in fifteen minute it is done not means that there is a park but at least if it means that there are some problems because the quest may also block human player for fifteen minute which is not the game desires world as the premier mentioned

where Antony adopted the shortest path in the graph to performance a realistic test it is a real noble but not always valid here is the gas map of one quest we in count two complete Qwest the playoffs first should go to NPC one and then optometry and is then erupted at a pressure of implicit tool and this is also the shortest past to complete this quest however some players tend to first jump down tree unfortunately we found that they were able to for the clip again and is this situation we are lost reproduce isn't healthy so this case shows that the rotation test was just passed only prove that there as exit a way to complete the

quest but can't counted the park absurd as a shortest path now with the third phrase speeding up the search for new quest although the prevail method can complete the quest it was not smart enough because it is to try every new additions and waste a lot of time but human play away a lot human player will read a native and in the description of quest first and then they try the most potential actions so enlightened by these ideas can waste it up the building of testing using table and a description a scream the answer is yes any option technical skill our measure is illustrated in the figure first with Easter Egg the text of demographic here is a example in justice not even for our

meeting with Jesus on the army camp is a place and she saw song is the name of increases in the team well human read this text the main will ask that they should go to the operation of our care and before this they may need to financial NPCs she saw some we follow the same idea of human first we need to extract entities in the Texas likes I'm here and she's also the two entities are important and we can retrieve more little infor information such as operations and IDs from the timid then they look for the world in the text you for well know that you for it means to go to some well so the action wrong to person and nonpolar object are the most precious actions although the common law reads is simple

it is ative code to rock rollers here we need to implement two models one is to distract entities it's a lot of code we adopt a Chinese word segmentation to us she will cover so with the customer detectors the other models is to maps as opposed to pretentious actions Labour's in session it is a classification task emotionally to me it's a lot more difficult real task however it needs real difference so well we can correct label Dipset to clean our classification neural network fortunately we already get the Dipset when we finished the first phrase the best path includes text and action neighbors pairs after collection date and tweeny we can ferry into text and up to the prediction properties of all the go access and this any optimal dose is easy to integrate with premiere agreements the predictions

probabilities are used to image initials the pi of all interactions and it means that the legal actions were a lot arranged to the chains and the position actions where was a large change to be selected so the certain speed is illustrated the dessert is our expected it works which in euro euro network whether they said including our two servers and several hundred samples and test each our validation set with about six hundred and eighteen samples the accuracy of entitled extractions models is 98% and top five will call of clarifications models is merely 88% and in the real case it speed up the decision of you class for three times faster which

[Music] [Music] regression test is completed okay and the last so we're reports and result in the real application of reduce testing in that is game justice shall welcome next I will talk about our new automating the regression testing architecture based on AI in the past of automated testing architecture only

content one team server and the server came class because most of our scripts are running on a cleanse now we use two major parts in our new architecture architecture to do our daily testing work one a static environment and the other one is training environment testing environment is used to do daily regression testing there are one team server several game clients and one algorithm server in clients Kim summer since the state's two algorithm server as input which folder actions as output then send them to the client and the client executed action this cycle looks until all tested cases

are finished at last and will generate a tester report in this environment the algorithm server uses the best path for cuts which have been chained to finish the class as stable as possible now we use three computers for clients one computer one compliant we really use other resources except the basically environment for compliance and the game server but when we add a new class to the test cases of some quests are modified we neither training environment for this class there is no random aid information and then the awkward observer is based on churning and

starting to calculate the best path for these paths in your word charlie environment is to generate a data path which is used in the testing environment for regression testing when we want to add a new quest in our regression tactic we just need to put it in the tunnel government and after chilling added to the testing environment when a quest are modified the testing environment also has a certain searching function when a change is simple there is a great possibility to pass a test there is its own adoption if the change is complex and tasks tab is blocked it will be put

in their training environment to update its best path this is an example of our tester report report content test a time version pass rate and other related information the red circle shows the month has a suit of our task regression testing in this report all cases are past this report contains field cases the table shows which tasks and the which task step is blocked for example here there are two steps blocked number 7 and number 701 we

can also click the link to get to the lock and the square shoots of them to check the problem in this page I will give the result of our work and then show some important data a word inner regression testing now covers all the main stories totally in 290 Kratz and 4030 steps in most time all the cases are past normally and accidentally there may be one or two step locked so the step passing rate is over ninety nine point ninety four percent for each time of regression testing this matter will see you about ten hours compared with

manual testing and we found five real box using this method besides we also use training environment to test a new quest we launched a new series of monster rate engine called Chile enjoy its contents 15 quest and 348 steps for the first time or Charlie five steps were locked and the four of them will cost by box and during the entire testing process we totally fund six real bucks compared with the old method of automated regression testing the new method has a

lot of advantages such as saving more time on the labor and surely in a table in the past we need one to two weeks to write a new test script for a new series of cut underspend have to do this to matenda and now we don't need to write a new test script or matenda the only thing we should do is to put them in the training environment our tailing regression testing now covers all my stories beside side stories are turning now and they will be added into daily testing as soon as possible the step passing rate of the

side story now our 96.4% so in the future we extracted all class to be covered moreover the framework is easy to integrate MMORPG can always have some common features so this method can also be applied in other things in the future we plan to use the method in other MMO RPG games to improve testing efficiency the above in the work for regression testing of class thanks

ok let's begin the second about class belliston testing there are six classes in justice on I at is first much and a world class claimin no me was released for monitors Givens class in game a tease us with its own create and skill system Crispina is great show for MMORPG games well as the palace is broken many players better chosen a predominant classes and as a team will be lack of diversity and interest however to make the transparency it's difficult emitted as many relies on human testing before nudge or adjusting according to player feedback after night it's empirical and health heterotic and we propose a transparent testing framework based on a plate any calls in reinforcements no need to mess at the show ends of snide we create where was this world arena and

set up new memory venues for two classes and then the two classes a tree with aim to win the opposite start the twin result with give some informations to the gym desires of whether the true class append advanced the preceding officer play is to freedom bankers and trim the other trucks until training classes ratio stable be rid of 75 percent and then exchanges the two classes and repeaters in the training Co is very interesting the we read of class a first increase since it's not as it turns and they need to decrease since it is wall prison the Tiwi is time-cost lacrosse they reached the limit of 75% Easter's another t2 is the time cost that the class be rich in the win rate of 75

percent at Eastern the t1 is shorter than t2 in this product so that the Class A is easy to start for player on the other hand we also get very anxious from the final image the final result of Class A is higher than P it means the American values of the skill system is higher at last we will make a report based on this result for game designers and they decide whether to modify the numerical value the way measures before we knew released classes and so how to assure the balance of mooie with the fitted classes which was three classes to test whether the new class interest so we let su mo sensor initial to compare with another dessert shows that

the normal can eat all the three classes with the win rate higher than nine percent in the most case finishes the better in 20 seconds and remain and the remainder nearly half of although the new me is a dps class it's much stronger than expected so the desires making these classes before released okay this last time we just do some other wise of the new class after that we decided to make a comprehensive evaluation of the balance of all classes so it has all the 49% of the all classes in Justice released orders that in these tables the intersections of Zoar and column is the will read and at reintegration we read difference indicates the combat power of two

classes in games research indicates the degree of easy control for a certain class from table we found that the last column class sua is a very weak in was well combat and it is in current services effect over the game because the class is a healer you can hear is teammate in team fighting but it is fragile in valve is one better okay to make a summary we introduce to lower measures to perform a revolution test and a class Pelisson test we help them in the justice team and get some inspiring dessert and in future we will try to improve all these two measures for revolution testing our last pair is to extend these measures to more MMORPG game and now we get in the shortest action sequence of the revision testing

but it makes a lot of big best because the shortest path it's a lot of always the player's choice some players may complete a quest with their own styles the shortest path may I go to call these cases and other examples if the game tester want to make a full courage of the testing the shortest is obviously a little best beside we still want to improve the robustness of these measures for ready for class venison testings currently the cost of training is a lot of sheep it'll eat a lot of time to treat or models and also it justice should bow for worse as well Moses well combat and a lot for purity team balancing nastily since lastly thanks for all our core goals we are from two different Department we have difference doughnuts

back wrong but we make happy and there are fruit for corporations okay Oh also thanks for your time if you have any questions please email to me you