Balancing Nightmares: An AI Approach to Balance Games with Overwhelming Amounts of Data

[Music] thank you for coming today i'd like to talk about the system we developed to support game balancing using ai i'm kazuko i'm working at the squadron japan i worked as a programmer on the development of some console games before currently i'm an ai engineer and my work is about quality assurance especially in the area of game balance using ai i'll cover all of this presentation today but before it please let me introduce a member of our team sierra allergy here is the change of data visualization and data analysis in this project our project would never have succeeded without him shigeru can you introduce yourself hello i'm shigeru awagi i'm online back-end

engineer in my previous career i designed and created general windows desktop applications and databases and i made a career change to the game industry by using that knowledge especially in the last three years i have done data analysis work with over 20 projects using ui under simulation results we created created total to visualize the characters behaviors there will be time for gnda at the end of this presentation and i will try to answer them as best as we can okay let's start this is the outline of the presentation first i briefly introduce the game targeted for the research next i explain how we used ai to balance again with the different sexes and failures we had this is the main

topic in this presentation so i will give a detailed explanation of the ai algorithm and the results after that i'll show how to configure the system finally i talk about the future plan of our research our research was about finding a method to balance the game green's notes repeat it's an action battle game for smartphones and the main game loop is the following one at first prepare the party before the battle choose the characters and their equipment then the battles it's automated so the player cannot do anything end post battle the player receives rewards based on the result which allows him to become stronger

then repeat party consists of four main characters each chapter is supported by two heroes and only the heroes will participate in the battles there are few hundreds of heroes each hero can have some equipment such as weapon and accessory for example there are about 100 kinds of weapons and each weapon can utilize up to five enhanced items out of 65 available choices the shearer's behavior in battle depends on its hero ai equipment each hero ai contains a predetermined rule set of actions

which can be customized by the player it evaluates condition from the top until any condition is matched and execute its associated action though the system is simple there are few dozens of conditions and actions and their actual behaviors change depending on heroes properties as a result it's difficult to find the best hero ais for each hero but it's a high potential feature as it can allow the tractor to beat stronger characters until now i explained how to configure a hero eventually a party is built by eight heroes like this figure

so i think you starting understand why balancing green's notes is a nightmare but just think yes let's see the number of possible combinations for a party to give us a better idea we have eight main churches 80 heroes 60 weapons three costumes etc and the number of possible combination is temple 182 a real night to me you see furthermore the game is actually updated every one or two weeks with new churches weapons and so on and we need to find the bounce breakers before each update impossible right illustration that amount of data continues to increase over time it's very difficult to

adjust game balance by hand our idea for the problem is let's try to make the strongest party if their element that breaks the balance of the game the strongest party must have these elements a really simple idea this methodology of such is called optimization based falsification it uses global optimization methods to guide the test toward a possibly small region in the input space that leads to the incorrect system behavior in the case about game balance breaker explain an algorithm to find the base combination of characters and equipment first i'll show the results a little

the left party is created by our team and the right party is opponent the upper birth shows each health point a party that have more half points is the winner she is the strongest hero in her party she is a good weather weapon that heals her every time she kills an enemy and due to her hero ai settings she uses frequently her ultimate skill it repositions how far from the enemy so it's hard to get damaged every time she uses her ultimate skill she consumes two points in the video as soon as there are two points available

the hero detects it and directly uses its ultimate we never told our method the effect of the ultimate skills or the meaning of the hero air conditions and actions nevertheless the method was able to optimize the play of this hero by only observing the battle results to create parties we use genetic hours which will be referred to as ga in this presentation ga is a method for combinational optimization based on biological evolution g assimilates evolution of individuals in an environment the evolution proceeds by crossover of

individuals between each other in this process the better individuals will have a higher probability of being chosen as parents individuals born from good parents tend to be good too in the result after some generations all individuals get better in general this is a general explanation for ga so let's confirm the correspondence with grim smarts the environment is a problem we want to solve so it is a battle in the case an individual is a candidate to solve the problem it's a party

each individual has a fitness volume which is a score that represents the degree of adaptation against the environment as the environment is the battle the fitness is a better result there are many other times in ga but this is enough for today as a side note we chose g8 not because it's the best algorithm to find the balance breakers there are three reasons first a ga is a simple algorithm it's easy to understand and implement second ga has a long history we can find many understandable explanations on different media this helps us to save our time to study

and implement ga third we can get the sub optimal solution in limited time in game development this changes often changes suddenly right stuff ga outputs an optimal solution at the time in conclusion we use ga because it's suitable for product development now let's see the process of ga after the initialization we evaluate the parties we select the best parties we cross breed them and mutate a few of them this process is down for each generation and after specified conditions have been

reached or a specified time has elapsed the algorithm stops from here i'll explain them step by step step one initialization the initialization is a simple part ghs charges and equipment randomly and create parties using them these randomly created individuals will be the individuals of our first generation step 2 evaluation of the parties in this step the algorithm will evaluate each party on how good they are against the environment which is indicates the battles so the fitness of each party will be computed based on the result

of their practice in greens notes there are different types of butters we focused on two types of them pbe and pvp in pbe the player fight against predetermined parties from the game on the other hand in pvp the parties are fine-tuned by other players in both cases the buttons are automated i summarize the differences between pve and pvp these differences look very small but they affect the evaluation methods

players fight in pve to get many materials no amount of materials could satisfy the players there is no constraint of number of challenges and no defeat penalty consequently it's very important to win quickly so we use these axes for evolution in case of victory we are going to use a battle time in case of defeat the time is not a good expression of how good was the party we use instead the damage dealt by the party as you can see these evolution policies are very specialized for the game

next for pbk the players aim to get more arena points i want to explain what are exactly area points but the important point is that they can be on or lost depending on the result of the battle and because the number of battle is limited the player will want to make the most out of them these facts indicate that important point for pvp is high and stable winning presentation but the player cannot choose the opponent to measure the winning percentage we let the generated party to fight

against some of the top players from the online leaderboard we use the average of the result as evaluation values these top players have different fighting styles as a result we can assess the ability of the generated party to win any pvp match so this is the final version of evolution policy in pvp for both victory and defeat we use a difference of remaining life points in the case of close matches there is little difference in score between victory or defeat therefore we add the fixed amount of bonus point to see a bigger difference between victory and defeat

this policy is also very specialized for the game the evolution equation is very important and sensitive for not only g but as other learning methods it takes a little time to design a good evaluation equation we need to think about what is the essence of the game we decide not to generalize but specialized equation to our game to get more accurate evaluation value in case of pve the victory and the defeat evaluation functions are based on two different units the time in case of victory and the damage dealt in case of defeat ga needs to compare evaluation values

but it's difficult to compare the values if their units are different therefore we mapped this value to one axis for pvp we handle the evolution value in the same way now let's advance to step 3 in ga select the best parties after evaluation process ga choose parties to be inherited to the next generation as evaluation values are on one dimension now we can know easily which parties are better we decided that we keep top 80 parties for the next generation and dispose the others this policy keeps

a diversity in parties and avoid falling to a local optimum step 4 cross-breed the selected parties we represent a party composition as 3 structure it select point to cut in each tree randomly and swap the clip symmetry like this figure this is a one method one point crossover step 5 mutate a part of the parties we expect the crossover improves the parties entirely but it's a probably probabilistic method sometimes good elements can disappear

also in the parties randomly generated in the first generation there might be no overpowered elements our mutation method compensates these disadvantages the mutation occurs with a five percent probability in the mutation after selecting a point to cut randomly it replaces the subtree with a new randomly generated subtree as this is also random operation in most of the case mutated parties get worse however really very good elements appear and they are inherited to seeking generation i guess you feel a bit afraid of the mutation work that depends on luck that sorry

i was afraid of the same thing too as an an experiment we ran ga repeatedly in the same settings except random seed there was a hero that always appeared in the final generations but sometimes that hero was not in the initial generations this experiment proved that this mutation works well and is an important process for the coverage of the elements until now i explain the evaluation of the practice the selection of the best parties their crossover and their mutations by repeating it many times she improves the parties gradually

this is a more simplified feature of the overview our system repeats the button evolution and improvement steps as evolution and improvement are located outside of the game we need a communication pipeline with the game we use an api to communicate with the game because we delegate the development of the game to an external company therefore using an api is a suitable method for us but if all members are in the same development team it might be okay to communicate directly in the course before and after the improvement step there is a conversion process

this looks redundant work but it allows improvement steps to be independent from the game and so that it can be applied to any games in short the bubble part is almost the same as actually and the improvement process by ghmb share if we customize the communication pipeline the evaluation the gene conversion process we can apply this method to other games actually the evolution and the conversion are small functions so most of the required customization work is in the communication pipeline this is one of the results we got for

pbe a point shows a party blue means victory and green means defeat the evolution proceeds from left to right and the vertical axis shows the battle time at the beginning of the evolution all parties lost their battles after the first winning party appeared the number winning protest increases generation after generation and the battle time decreases pve battles tend to be easy to win so this party evolved without any trouble this is a result for pvp please note that vertical axis changed from battle time to fitness value so the higher the fitness

value the better the party the fitness score improved continuously too on the other hand there is some sudden growth sometimes it's more difficult to win pvp battles than pbe battles so these sudden growth happened when generated parties sold one of the reasons they were losing the battle before this is the best party in this evolution however because fitness value is computed based on the result of 15 bottles there is a second measure of error to confirm the party's strengths we made it fight in 300 bottles the upper number is a winning rate against party that were used for the

evolution the lower number is a winning rate against the other parties both of them shows high wind presentation and they're almost the same this result indicates the best partition win against various parties as planned this is a video of a battle where the best party created by ga against the opponent the game version is different from the video i showed just a while ago so the pretty composition was changed because new characters and pieces of equipment have been added please watch strongest party [Music] i guess some of you have never played

this game before so you may not understand what is happening on the screen but it's the same for me although we can understand just a little we cannot follow all of the actions by the eight heroes actually [Music] and to share this presentation i explain the detail of the g algorithm ga showed as a strong parties however it's too complex to understand video of battles are also very hard to understand we need a better way to visualize this information from now i'll introduce our visualization method to find valuable information from

enormous amount of data every time the analysis progresses every time the game is updated what you want to check and what you want to know will change therefore the visualization is required to be startable at low cost easy to be customized we are using open source apache sampling various languages and data sources are available for this tool we can add and change the display such as table and graph anytime we want apache spark that runs in the background has scalability that can that its memory and cpu usage to the available cpu

and memory with clusters we recommend it as its convenient tool can be used immediately for free in this table we can see the result of 15 matches exceeded by one party each match results are compressed and displayed in a row specifically it shows the settings of ai the type of weapon equipped and the special activity that occurred in the battle the column shows the fitness so the table also serves as a ranking upgrade each row follows this structure the first part displays a summary of the battle results

it is then followed by more detailed information about each chart but i omit it partially in this figure let's see the method to make information easier to see their dimensionality reduction and abstraction first dimensionality reduction finding the best party is very useful for designers sometimes you want to check how strong is a party with a setting composition or you want to find similar proteins to the best ones until now the only way to find similar parties was to take a look at the party composition tables and find similar

products from here this table is one of them this table only shows the list of weapons used per party each row corresponds to a party and each column to the number of times a specific type of weapon is equipped in the party and as you can see it is already very difficult to find this similar practice we needed a better way to visualize this kind of information in this case we use the dimension reduction method called principle component analysis for plotting multi-dimensional data with this method we reduce the dimensionality of our previous table into only two dimensions without losing the features that date have

by performing this conversion it is now possible to plot 20 dimensional data into two dimensional x y coordinate system which is easier to understand intuitively in this case the ground shows the similarities between the parties for instance if you want to find parties similar to one of the parties at the top the green parties loaded at the top are all very similar to each other each number corresponds to party id on the other hand if we want to find a party that is largely different from a green party the part is loaded at the opposite in the in this case at the bottom left

will be great candidates it takes a little time to use this method because it requires some data preparation but important point is that this method can help you solve this kind of problem effectively next this is an example using abstraction the content of the frame is status such as the number of ultimate points attack power etc for example if a hero has 74 thousand health points is it high is it low this exact number has no meaning what is important is to know how high or low it is relative to the other

here we explain relative evaluation using standard deviation standard deviation is a measure that is used to quantify the amount of variation of a set of data values the average is also a well-known measure to show how relative is a value to the other values but it will be affected too much by a few outliers to express the data properly alone standard deviation helps ranking in accordance with distribution of the data in this slide c and d are valencian and hp 747s belong to the rank d which is a little lower than the

average value standard deviation replaces the two detailed numbers with expression that can be understood at a glance what i'll show you next is change of granularity the part highlighted in the future is the settings of the sheer ai which compressed into the single letters what was the hero eye as explained earlier in this presentation here we are as a rural set that decides heroes behavior in the ultimate battle there are many kind of settings we focus only on the actions in ignored conditions

to represent the equilibrium settings at the simplest level in action ai we also compress the meanings so that the pattern stays very simple for example there are some pieces of the hearing aid that trigger ultimate skill ultimate stereo immediately malevolent close range long range and on the same line we grabs all of them into the category u it is the same for all other pieces of the hair ai if features can be understood at a glance comparison becomes easier for example we can know easily that first hero in these four parties has similar hero ai by comparing only six letters

letter u means ultimate skew and f means full combat x so we can understand that this here is very aggressive of course a part of information is lost when we abstracted hear ai but tendencies are not for us in most of the cases if the shortlist makes us want to know more about this party we can open the page of its detailed information there is no information about the conditions of hearing aid but we can see a little information by the order of the letters actions at left side have higher priority and importance because the conditions associated with the more left side actions will be evaluated first we can know a certain amount of information about roads played in the battles by

considering not only the compressed stakes data but also the order this is another example with a change of granularity of information these marks show the battle results that is used for the calculation of the fitness in the ga evolution in the evolution i showed today a fitness value is calculated with the result of 15 battles each party had five different opponents and default each of them three times the symbol at the bottom are the average result of the multiple games for each opponent in the example the winning percentage against enemy 4

is 66 percent so a double sec is shown here the battles in the game have a lot more randomness battle results often change easily therefore the exact numbers in the winning percentages do not have big importance if the two had a feature that showed color easily we wouldn't have used color to show the winning percentage like hit them up because it's somehow cool actually we can understand the rough winning percentage by the symbols and it's enough for us like this example it saves a bit of our limited time by removing insignificant information

the third method is summarization which extracts important information from the large amount of data in the example here you can see the content of the bottle which had been done during the g evolution heroes from the party can be tagged automatically according to the roles played in the bottles for instance in the middle frame there is a tag of ace which is given to the hero that deals relatively high damage in the party a and h is an abbreviation of attack and hill which will be tagged to the hero who both attack and heal the frame on the right said this hero

did not join the battle thanks to the tag we can know that we do not have to check this field carefully we use the visualization tool at first to know which path we should focus on then we watch the video if required now that we know what we should focus let's see the video again this hero named shahiravat was tagged ace which means top damage dealer her data also indicates that she's aggressive and she both attacks and heals her normal attack is not very strong

she used her ultimate skill here her ultimate heals her ally first and after that a long-range attack occurs she use it again the ultimate the damage is much higher than the damage by the normal attack and she use it again as a summary she keeps her putty alive while dealing high damage to the opponents

actually outside the previously introduced hero we found a few other strong heroes let me introduce one of them this hero despite being strong is not very popular among the player base her ultimate skill cost very little compared to the previous characters our two indicate that she's one of the key heroes of some specific parties let's check her in details her ultimate scale is slow and remain for a long time as a result thanks to its wide collision box it hits many enemies a lot of times the damage is not so high but her

ultimate push back all the opponent on its path and prevent them from doing anything consequently despite the opponent having very strong hearers they're unable to fight back in the end the video are not what were recorded we have a function to replay the battle it needs to specify a random seed and running with fixed frame rate the proceeding of the bottle is always the same if we specify the same random seed and the same protein composition by json file this function makes us possible to watch the battles in the evolution later without recording

if ga and visualization 2 are used alone it will not be useful even if it find bounce breakers no game design i believe that it is indeed a failure or problem if the only thing they see is numbers and texas they need a visual proof in other words our visualization to only detect potential balance breakers and the replay function proves them let's check this process with the previously introduced space chart named shahrazad a sheer bronze breaker actually this chapter was a popular character with high evaluation from veteran players as the player did our ga and visualization 2 also found that she is a balance breaker

however it's very difficult to get this hero in the game after multiple conversations with the designers it was concluded that her strength was as designed and the balance as a whole is working as expected when we want to find other balance breakers we can do it with a simple operation we rerun ga with specific settings that prevent the use of attraction in any party ga was such a strongest party with this condition this section explained the overall

system configuration and the processing flow from simulation execution to seeing the result first build the game and deploying it to the outplay environment and second the designer exchange ga with the tool third they check the results with a web visualization tool they do not have to care about the content in the old play environment inside of the old play environment we can find all the things i explained earlier after the building computers for the game have the binary files there is a master server to manage auto

buttons the service stores battle tasks to be run the slave computers fetch and run tasks and return the result after the battles the visualization tool uses the data stored in the server to show the result to the designer that's all for the system configuration as much as possible we made each element of the system independent to each other so that we can easily replace one of them later if we need to what about performance for the system ga needs to simulate many battles to find strong parties

how much time does it take we tried different approaches we created the first simulator to solve this solve the problem it is what calculates the result of the bottle numerically by simulating the proceeding of the bottle the running time was only one second for one battle however we were unable to prove that the simulator was at rate at 100 percent because the simulator was a new implementation of the game loop so it may not be exactly the same game loop as it is in the original game especially when the game was updated we need to update manually to simulate the game

so it was another heavy button and there was a possibility that we even forget update simulator when we update the game we decided to stop using the simulator second idea is to connect the ga directly to the in-game logic and actually play the battle in the game this idea can guarantee that battle progress will be the same as in the game also as i showed the designer can see the bottle using the replay function on the other hand it takes much time and many computing resources we concluded that accuracy was the most

important aspect for us and currently we use this solution as a conclusion the requirement of the earth player as follows using in-game logic it is a necessary condition to guarantee the accuracy the following ones are options but we strongly recommend satisfying them supporting acceleration the update cycle of games became much shorter than it used to be the shorter the play time the better executing the program in the cloud i said our system is may compare resources it will be a problem or budget but

if the program runs in the cloud we can easily start using many resources and dispose of them soon after it reduces the cost to the minimum and we don't have to sweat from the fans blowing at us and of course these all need to be updated automatically or by very little work finally i'd like to talk about what we want to realize in this feature believe that ideally the balance of the game should be spiced up time to time so the strangest hero is always changing to do so we created a cycle in the correlation of

parties strength however currently we have no tool to check if the changes we do to the balance of the game are in line with the cycle of purchase strength we are aiming for we can already let amit participate each other but we will need new data analysis and data visualization tools to confirm the cyclic balance also it will be good to propose party compositions to players using the result of the data analysis of course there are still many possibilities in the current system we will continue to improve it especially we focus on the data visualization the current tool is specialized to visualize only the generated party

the next major goal is to visualize the reasons that you contract the parties we are considering using clustering method called k-means the clustering makes us possible to see the past progress of the triangle error by the gs so we can know what the ga valued the sensing point is how should we abstract the party composition for the clustering on the other hand being abstract means that it makes it difficult to observe the individual strengths of a party visible as a measure for it we are studying the use of the visualization method called box plot box plots express abstractly while leaving important potential such as maximum value median value

and outlier value of graph as a whole we would like to continue to report to you about this research let's wrap up the presentation with takeaways if you want to balance again with nightmare amount of data finding the strongest elements of your game will help you today i introduce gendered algorithm but it's only one of the methods to find strong airlines no matter that method you used we recommend data visualization and analysis to understand the strongest of the strongest elements an autoplay environment is also very

important if you implement it for your game you should use the same logic as actual game to guarantee the accuracy in addition the outplay function must be updated automatically according to the actual game update the most important point is preparing them only in the development i busy with the game development even if you're too busy now it's most effect for the budget and the development of your game destroyed by sydney's presentation i'm sorry but we're still used to sharing english so we'd like to ask you speak slowly and simply

it's serious thank you very much it was great uh how long did it take to fully design and implement this system thank you [Music] i could understand wow [Music] the first prototype finished after starting a half of here and we continued to improve the whole system so now it's

the final version finished by one year what one year about one year he agreed [Laughter] yes um have you considered any options for automatically highlighting the statistical outliers in battle during the replay sorry can i can you change my motion through what i'm sorry so during the replay it's uh very difficult to see what's actually happening have you tried uh finding automatic methods

of highlighting uh those statistical outliers [Music] visualize it i think video is best the video is best and so we we make visualization for such a video and planner want to look a video sorry so uh i i want to make more visualization but then right now i haven't run out won't do only a only video so um

um well video isn't a very good visualization for designer sorry other than it static statistics statistics method any other methods did you think is it a question it's okay thank you sorry okay sorry it's a time for me thank you for it