Addressing Exploit Abuse in EVE Online with Customer Care

[Music] thank you very much for coming so my name is Davide Newsome and I'm a senior project lead of player experience at CTP games I've been with CCP for 11 years now and I want to tell you about one of the most interesting challenges that I worked on in this time and exploit named ghost training now EVE Online has a lot of legacy the game is actually turning 15 later this year so to make sure you have all the necessary context I'm going to start with a crash course in Internet spaceships so what is EVE Online well first of all there's possibly as many answers to this question as there are Eve blares it is a pretty complex game and it has a multitude of different play styles it's also a very competitive game now if you ask me what Eve is I'd say that at its core Eve is a game of economics the way I see it Eve's player driven economy is what gives meaning to all player achievements

in EVE and I consider it to be the most important core system of the game virtually every mechanism and system in the game directly or indirectly touches the economy in some way you could say that the rest of the game is essentially built on top of it and whether they realize it or not our players interact with the economy in every game session now another core system which every eve player interacts with pretty in pretty much every session is the skill training system even lines unique real-time progression system last but not least Eve is a single short game everyone is playing together on the same shear at global universe let's talk a little bit more about the economy so first off most items in the game are manufactured and sold by the players themselves this includes the spaceships the fly they mate with the weapons and mining lasers they equip to their ships and the ammo that the suited pirates it even includes the Space Station's which the doctor ship set there's also no soul bound

items in EVE every item in EVE is tradable if you get tired of a ship just sell it to the highest bidder now to facilitate trades we have a dynamic player driven market broker system it works sort of a stock market where players creates pie orders and sell orders there's not just one market either each space station in the game has its own unique market so in other words there's no central auction house in Eve instead market helps form organically based on the landscape of the universe every day there are around 1 million transactions made by players on the market so again the market is a player driven ecosystem and it's not regulated by CCP in any way since we don't set the price or a price range for any item that value is determined by the players themselves and simple economics supply and demand now as an aside as an example an in-game war can increase

demand for certain spaceships and in turn make the prices skyrocket all over the universe the outcome of conflicts is also sometimes even determined by attrition whoever has the biggest pockets wins all in all the market system is also deep enough that there are some Eve players who never undock their ships from stations all they do is sit in stations and play on the market also another important thing of note is that Eve has permanent asset loss any item in the game can be destroyed or stolen by other players if a player spaceship is destroyed it's gone forever and depending on a dice roll items carried on the ship can either be destroyed or drop this loot drop items can then be stolen by any player that's in the area and it's because of this that the number one rule often quoted by Eve players is to never fly what you cannot afford to lose now on one hand permanent asset loss keeps the economy healthy since there's always new demand for new ships and items on

the other hand it makes PvP combat very consequential and makes it important to always have capital on hand to replace lost assets now Eve is primarily a PvP game and at the end of the day the most meaningful achievements in the game often come at the expense of another player a staple economy is in other words a critical component to keeping Eve balanced let's talk about character rushon so progression in eve is based around training skills skills unlock new functionality such as fly a new ship classes and they also offer various bonuses such as increasing weapons damage or reducing broker fees on the market now skill training is performed passively and in real time players don't even have to be locked into progress training continues while you're locked out of the game now depending on the skill they can take anywhere from a couple minutes to over a month of real-world time to train and players

also have access to a training cue which can be used to cue up and train skills in the preferred order the training cue looks like this this is actually one of my characters and as you can see on the slide I've got skills queued up for the next 122 days some of these skills take a relatively short time to train while other more advanced skills take longer as an example the skill on the bottom capital repair systems is going to take me 33 days to train from a lower standpoint it works sort of like in the film The Matrix in order to learn data is downloaded into the character's head it just 10 it just tends to take quite a bit longer to learn kung-fu than it took neo I'm not sure exactly why that is but my thought is that maybe there's just no fiber optic connections in Eve now Eve has no set character classes players are free to progress in any direction they choose it's common for players to

specialize in certain areas such as combat or industry but there are also other players which choose to be Jack's of all trades access to high-end skills is gated by requiring prerequisite skills here is an example in order to train this skill which is a battleship skill you must first train a character to fly cruisers and battle cruisers the speed of a character's training can also be influenced by in-game means there's in-game items for one such as performance-enhancing implants and there's also an attribute system now the biggest effect on the player speed of progression is however decided by whether you play for free or pay for your subscription which brings us to clone states now clone states are our version of open access lust free to play which was launched in November 2016 free to play users are called alpha clumps and paid users are called Omega clones converting to paid status grants perks and lifts various

restrictions as you can see on this slide most of the benefits of a paid account concerned skill training or character progression now the perks they include doubling the training speed which significantly speeds up progression and alpha clones are also limited to a certain pool of skills which you can think of as a level cap of sorts at the time of the ghost training exploit the pool of skills available to alpha users was approximately 5 million skill points which is the equivalent of around 5 months of skill training at maximum speed alpha users are also limited to a 24 hour long training queue so as you can imagine doubling the speed of training and bypassing the skill restrictions are by far the most important drivers for free-to-play conversion while on the subject of free-to-play and paid access Plex is worth the mention so Plex or pilot's license extension is an in-game item representing Omega time which can be bought from CCP this item behaves

exactly like any other item in the game it can be sold on the market and it can also be destroyed or stolen by other players if you're not careful Plex also enables Omega commercient with in-game cash known as is K or is with enough is an alpha user can operate their account Omega status but by implication the market from another player now since Plex is bought for real-life money and traded on the Indian market we can also establish an exchange rate of sorts between real-life money and disc and since all items in the game are also tradable we can in a roundabout way assign a real-world value to items in the game this is how we reach dollar values in headlines like this that being said Plex is a one-way street and real money trade is strictly prohibited in EVE so numbers like this are the theoretical now the last game system I'm going to cover is skill trading in

early 2015 we added new items to the game called skill extractors and skill injectors these items facilitate trading of skill points two skill points an intern character progression or in other words an in-game commodity skill extractors are bought from CCP for real-life money and when a skill extractor is activated on a character it consumes 500,000 skill points and is turned into a skill injector a skill injector on the other hand grants up to 500,000 skill points to a character when is consumed now these two items essentially enable conversion of character progression to is and vice versa so as you can imagine that become an expensive and very heavily traded commodity they have also created very clear overlap between EVE Online 2 core systems the economy and character progression so what have you learned about EVE Online we've established that the economy is the central game system which ties everything together we've

also established that Eve is primarily a PvP game with great wealth confers great advantage and we've established the free-to-play conversion is largely driven by enhanced character progression available to paying users finally wealth is somewhat interchangeable with progression in EVE due to the skilled trading system now let's talk about the ghost training exploit itself so what is the ghost training exploit in short it was a bug which enabled a non-paying alpha you sure to obtain pretty much all of the benefits of a paid on maker user you could in other words go from this to this you could get double training speed with no skill or training queue restrictions no Omega subscription required unfortunately the Reaper steps were fairly simple as well making this exploit highly susceptible to abuse

let's go over the Reaper steps to trigger the book you had to fulfill the following requirements have an account with Omega status active set a long skill queue the longer it is the better let the account so make a subscription run out and lapse to alpha wait for a few months without logging into the game and finally upgrade to Omega status first before logging in the result of doing this is that after logging in at Step five the user skill training progress as if the account was in paid on maca status for the entire absence in step four no matter how long that absence was you could also potentially chain the exploit moving straight back to step one after step 5 also keep in mind that since Plex can be purchased for in-game money and used to activate on macro status real money payment wasn't actually necessary to trigger either step 1 or step 5

one thing of note that I want to highlight is that unlike most exploits we've dealt with the ghost training exploit had a direct effect on both our game and on our business on one hand it undermined the integrity of the games economy by abusing the exploit and utilizing skill trading a player or an organized group of players could get very very rich on the other hand it compromised our newly launched free-to-play model with knowledge of the bug a player could effectively bypass the restrictions of a free account removing much of the incentive for converting to paying status so what made all of this possible well when we identified the roof'd cost it came down to two factors the first is how skilled training is handled in the backend skill training progress isn't calculated or recorded while the user is logged out of the game instead a calculation is made when logging into the game and it's at this point that any progress made well

offline is recorded to the DP now I know this isn't a perfect analogy but since the real estate of an offline characters training isn't determined until after logging in I like to call it shredding or skill training now the other root cause of the exploit was a critical bug in our free-to-play implementation we basically wrote a new system which is independent of the FTP to manage clone state entitlements you can think of these entitlements of as expiry dates we call the system CIS which stands for customer information system in short and keep in mind that this is a little bit of a simplification the bug can be boiled down to a communications issue between CIS and the FTP let's look at a couple hypothetical scenarios in this first example are used to stop playing in the third week of January and as you can see on the slide their last login was about three days before running out

of o maker subscription this user logged back into eve in the final week of february while still in alpha state and then upgraded to omega the next day on this first login after the five week absence the FDP asked CIS for the users clone state entitlement and the answer given is this accounts clone state is currently alpha now as far as the DP is concerned the accounts last clones last known clone state is omega since there's a discrepancy the game checks for the Omega expiry date before calculating skill training between these two logins so the end result is basically that the character is credited with the remaining three days of a maker training time and 24 hours of alpha training time in other words everything worked as it should things however did not go planned in this second example it's essentially identical to the first scenario with one key difference after taking a five-week break the usual aggregate upgraded to

Omega before logging into the game so on this first login after the break the FDP are CIS for the same constant entitlement as before the answer given is that this accounts clone state is currently Omega now according to the evap the accounts last non clone state is also Omega now since there's no discrepancy the DP just assumes that the account was in omega state for the whole time so despite having elapsed to alpha the user is credited with 5 weeks we'll make a skill training time the user has in other words ghost train do you think that Schrodinger's cat analogy is starting to make some sense now but the reaper steps were fairly specific and in order to pull off the exploit you had to stay locked out of the game so who did we think was actually exploiting this bug well our main concern was about what if players

call skill farms so what's a skill farm well as we covered before skill injectors became a very valuable commodity after skilled trading was introduced and some clever players noticed an opportunity the idea was to create dedicated characters to Train extract and sell skill points for a profit and this turned out to be a great money-making idea now most players who operate skilled firms do so essentially for free by maintaining their own maker subscription with Plex and when properly setup is skilled from can turn a really high profit even after calculating the cost of Plex there is one drawback though operating a skill farm is very time-consuming and labor-intensive they don't scale particularly well regarding time spent however by exploiting the coach training exploit the operating cost in Plex could be greatly reduced and since the user didn't constantly have to extract and

sell skill points to fund more Plex the process also became less labor-intensive it was basically a win-win situation last time spent more profit but scale farms weren't the only thing we were concerned about let's look at the timeline so the bug was introduced on November 15 2016 now at this time we did receive a handful of reports in support tickets and bug reports about the ghost training exploit but the Reaper steps were regrettably unknown until later in other words we knew it was possible and to certain unknown circumstances to train Omega skills as an alpha user but what we didn't know is that it could be triggered on purpose and so easily so the problem therefore didn't get the attention it deserved flew under the radar fast forward to May the following year a post dismayed on reddit which exposes the easy-to-follow Reaper steps that's when we realized we

had a major issue on our hands so this threat which was posted on May the 12th it got a lot of attention and others threats post popped up in the following days with various theories and speculation about who might have benefited from these from this exploit to be quite honest our players were furious with us and rightfully so it had after all taken us over six months to address this problem so now on top of everything else we had we were faced with a massive PR problem in retrospect we really should never have let this happen but on the bright side we looked at this as a learning opportunity and we've already addressed the internal processes which failed and allowed this to happen but back to the timeline so in early June we deployed the first iteration of a fix first thing first we ran a DP script as a stopgap measure this script post skill training for

characters likely to be ghost training the idea was that since the setup to make the expert work takes roughly around a month the script would put an end to any current abuse in progress since a permanent fix could be deployed in that time now while the intent was solid the script actually ended up causing and more problems than it solves now this is an important detail players can normally only pause skill training while locked into the game and this is also an action which causes skill training progress to be recorded in the game DP the script however did not force a lock an event meaning that the CIS system did not provide the game DB with the clone state update now remember that any account potentially ghost training must be elapsed to alpha in CIS and must have been an Omega state during the players last play session that was basically the criteria we were targeting with the script

now since these characters hadn't logged in yet their Omega 2 alpha labs had not been communicated to the DP do you see where this is going let's look at the repo steps again these were the five steps be an Omega state set along training queue laps to alpha state and stay locked out for a few months lastly make sure to upgrade to Omega before logging in to ensure that the B thinks the account was in omega state during the absence in step four now what the script essentially did was bypassed at five thanks to Schrodinger skill training any character who had reached the fourth step when we run the script essentially goes trained without having to fulfill stat 5 now as you may imagine this greatly increased the number of characters affected since the fifth step was by far the most obscure one the first four steps are actually not an uncommon pattern for a player who's just taking an extended break from the game additionally there was a bug in

the script itself we post skilled training for far more characters than we intended over 200,000 characters actually many of these characters were even active and most certainly not ghost training so this in itself cost quite a bit of drama on top of the whole ghost training situation but now shortly after we ran the script the relevant stakeholders sat down to discuss the next steps fixing the underlying bug had naturally already become our top priority and we also immediately committed to addressing damage caused to the economy by heavy abuse of the exploit at this point in time however we had no concrete data available yet regarding the extent of the abuse what we did have were the results of a manual review by a player experience team of a few prominent skill firms this investigation validated that there were indeed some players who had greatly benefited from the exploit now armed with this knowledge combined with

the Reaper steps being fairly simple it was easy for us to conclude that the potential for abuse was very high it was really never a question of if we'd address the damage but how so we explored the options available to us the preferred option was naturally some form of automated solution such as a DP script maybe we could automatically reverse ghost train skills to an untrained state this option was however ruled out after some debate for one it was inherently risky we were after all talking about character progression a system we prefer not to mess with at all if it can be avoided it didn't help either that we were coming fresh off that other DB script which went terribly wrong and Anand had already anchored some legitimate players we were also concerned about the possible negative effect of removing prerequisite skills in the worst case scenario doing so could potentially render the game

practically unplayable take for example a hypothetical user who ghost trained for three months now imagine that this player returned to the game and played for another three months legitimately training skills for that whole time if we automatically remote removed ghost train skills from this player it could and most likely would render many of the legitimately trained skills unusable we had some other concerns but these were the main ones so long story short we wait our options and found out that an automated solution was not going to work because it had far too much potential to cause problems in the end i Wallen tears to lead a team which would attempt to address the worst damage manually now I've told this story a few times in private settings and at this point I often get asked the question why why go through all that trouble there's already a commitment to add to fix the

bug so this won't happen again why not just leave things as they are now that's a valid question for sure and I totally understand that manually addressing the damage seems like it could be a lot of effort with unproven payoff first and foremost though we felt responsible for cleaning up this mess that the we ourselves had created letting the exploit exist in our game for over six months was our mistake and there was a strong collective feeling within the company or having led letter players down we intended to set things right furthermore we see ourselves not just as game developers but caretakers of a universe it may sound a little cheesy but the even universe has taken on a life of its own and we see it as our our responsibility to protect the integrity of the game it's also simply it also simply just felt like the right thing to do as caretakers of this universe or

space janitors if you will we always make an effort to have as little direct influence on the political landscape of the game as possible it's our job to create the systems which drive conflict not to interfere with what the players are doing we also see it as your duty to ensure a clean and level playing field for all of our players it's a big taboo for us to directly give a player or a group of players an advantage over their enemies the way we basically saw it is that we'd be overlooking a massive potential advantage given to a select number of players if we let things stay as they were and that just didn't sit right with us but that's not to say we had a work cut out for us first thing first we needed data on who exactly had benefited so one of CCP's experts data analysts went to work some time later he emerged from his cave with a list of a list in spreadsheet format of individuals who had benefited from the

exploit now I scrubbed some player identify herbal data and removed some other sensitive information but this is what the list looked like each row on this spreadsheet shows a single individual the amount of skill points that go strength and on how many different characters the training took place the list is also sorted so the most severe abuse is visible on the top of the list now as we had suspected there were indeed people who had benefited massively the person at the top of the list for example had ghost trained on over 300 different characters and in total obtained more than 1.2 billion skill points that's the equivalent of about 50 year of still training at the fastest possible speed for Omega users and as you can imagine that's a whole lot of free money when it's converted to skill injectors and salt on the market so now we had the data we needed and the numbers confirmed a massive amount of skill points had been ghost rain

thankfully there was a little bit of a silver lining these skill points were spread across tens of thousands of accounts and the vast majority of those accounts hadn't ghost trained enough skill points to be of much significance if you ask me it's probably because many of the users who go strained did so because of the script and it didn't have long training to set before the labs to alpha state that being said there were over a hundred people who had obtained absolutely massive benefit and they had to be addressed now let's backtrack a little bit well the data was being gathered I sat down and created a standard procedure which our team would follow once we had the necessary data now we've dealt with our share of experts in the past and I saw no need to reinvent the wheel the procedure was therefore modeled to be quite similar and consistent with our approach in our approach to serious game exploits in the past now I assumed that any player with greatly benefited from

the exploit would most likely have abused it on purpose and the action plan was therefore largely geared towards punitive action let's go over the procedure the first step was establishing intent did the player intend to abuse the expert or did the benefit and intentionally in case of intentional abuse the next step was to find out what had happened to the ghost train skill points following the money if you will we basically forced her to possible scenarios for these cases the skilled points would either still be in the character's head or they'd been extracted and likely already sold on the market for profit in case of unintentional benefit no action was to be taken this was because I didn't expect any accidental abuse to be massive in scale nevertheless I did instruct the team to keep an eye out for massive unintentional benefit just on the off chance that it was there if they

came across any such cases they were to note them down and discuss them with the rest of the team finally the importance of context was stressed regarding the level of punitive action taken thinks it is attempt to hide the abuse or hide the trail of laundered benefits were to be considered the intent behind this was to be a bit more lenient towards those who perhaps didn't quite realize that they were breaking the rules furthermore some leniency was also be given to players who proactively turned themselves in now the action plan itself was also broken down depending on what what happened to the ghost train skill points we intended to issue a permanent ban to any skill from account still in possession of the ghost train skill points the idea was that skill from accounts had probably been created just to abuse this exploit anyway and the damage could therefore be contained by closing these accounts down the players main accounts on the other

hand were to be suspended on a three level scale fourteenth a thirty day or permanent balance depending on the players history the permanent pants were reserved for those that were involved in prior exploit abuse now in case of skills having already been extracted and circulated on the market the approach was to be pretty much the same with the added caveat that is core assets should be confiscated to account for profit this confiscation was to happen was to be done from the players main accounts we didn't actually have a concrete plan for players who had benefited unintentionally I can I didn't really expect we would come across many or any accidental abusers near the top of the list so we were just going to cross that bridge if we got to it I just didn't see a need to create a process in advance for non-existent cases so in summary

establishing intent was the first step of the manual review process and a primary theme of the investigation we intend to go through each individual on the list and take action according to what had happened to the proceeds now that we had all the data we needed and the process it was time for the team to start investigating I instructed the team to not take any action for the first few days the order of business was to only review and mark the users in the worksheet without any direct action taken just yet the idea was that these investigations would likely be complex and we might learn a lot along the way so in interest of ensuring a consistent approach I wanted the team to be well familiarized with any possible nuances before taking punitive action the thought was also that if for some reason we had to change her approach it's much easier to do so before directly involving the player now this turned out

to be a very good call now once we got started the team held daily read groups for about half an hour each day on the second day it became apparent that many of our assumptions had been completely wrong so comforta to our expectations the majority of players didn't appear to have abuse the exploit on purpose at all on one hand we had underestimated how easy it was to accidentally fulfill the criteria for the exploit on the other hand we had underestimated the effect of the skill cube parsing script in hindsight I'd estimate that around 80 to 90% of the people on our radar benefited unintentionally the team also noted that many of the top offenders weren't actually operating skill firms another assumption which turn out to be wrong now our expectations had been shattered but the fact remained that while these

players had benefited unintentionally the scale of the benefits was far too large for us to look the other way we however also knew where the standard procedure wasn't going to be fair it was way too focused on punitive action and planning so we went back to the drawing board to be quite honest with you we were a bit lost at this point we thought we were getting into a fairly routine investigation but that wasn't the case at all so we brainstormed in the end we decided to shift her focus to confiscation of benefits instead of bands bands were to be used only as a last resort in cases of clear and very serious abuse so we had a starting point we knew what we wanted to do but we didn't quite have the how nailed down yet we knew the chasing down and confiscating assets in every case would be extremely time-consuming far too time-consuming to be realistic for a

problem of the scale communication was also a concern if we removed assets or money out of the blue it was bound to cause player confusion and frustration which could result in support tickets and increased load on the support team so being mindful of these concerns we decided to do we decided to proactively contact the beneficiaries of the exploit the idea was to give a transparent clear and honest explanation on the situation and appealed to the players cooperative side in order to set things right so what we did is sent an email to the players on our list this email began by explaining what the ghost training expert is it also clearly stated that while we were fairly certain it was done unintentionally they had benefited - - way too far way too large a degree for us to ignore we also made it clear that we did not intend to issue any kind of pan or account suspension the message we

sent also included the exact number of skilled points go string by the player and highlighted the iske value of these skill points in other words the potential value or potential profit which could be obtained by selling these skill points overall the tone of the message was apologetic making it abundantly clear that we considered this primarily an issue of our own making a mistake which we sincerely regretted but needed to sell right finally we entered the email by offering a choice to the player on how to set things right three different options were proposed the first option was to hand over the summer is already highlighted as the potential profit gained from the xplit the player could also choose to hunt over assets any number of in-game items which represented the same value the last option was for players to manually extract the coach trained exploit the ghost trains guild points

with skill extractors provided by CCP this last option was mainly intended for players who hadn't extracted the skill points yet now players were furthermore not limited to just one of these options they were able to mix and match at will and to establish confidence in our data we gave a copy of the same data that we were using to calculate the potential profit to any player who asked for it so how did it go in fact it went really well most of the players showed a great deal of understanding that the benefits of this exploit had to be removed the vast majority of the contacted players were quick to respond with her preferred option cooperating with us throughout the whole process now we certainly did speak to a handful of frustrated players but there were far fewer encounters of this kind than we anticipated based on our past experiences we expected far more resistance and turn out to be the case

overall this approach turned out to be a major success to put a number to the success we looked at the results of our customer satisfaction survey or CSAT which was sent to each contacted player now the way we sort of structure our cset survey is that were you satisfied yes or no question not by the end of this process we found that over 80% of players who answered the survey chose yes now considering that these ratings weren't for traditional support tickets but cases where we were where we were removing money from the player we found this percentage to be extremely high we also had a pretty decent response rate of 21% now to put these percentages into context our overall cset score for 2017 was 9 one point 16% it's a fair bit higher for sure but again consider the difference between a support case and these ghost training communications so why did it

work well I personally for the most part attribute the success to honest and transparent communication I believe that by making it clear from the get-go that this was a result of our own mistake and that we didn't intend to issue bans we managed to break down the us-versus-them barrier and set a positive and cooperative tone to the conversation you could also say that within that initial communication we managed to to manage to subvert the player's expectation there is somewhat of a perception of heavy-handedness within our player base and after seeing the word exploit in the subject of an email from CCP the initial expectation may have been let me take a shoot first and ask questions never approach that of course didn't turn out to be the case now I also believe that offering players the choice of how to proceed played a critical role in making this a success being directly involved in the solution

rather than being dictated to by the space police was something our customers genuinely appreciated we even had a few examples of players explicitly thanking us for being in touch since since they most certainly didn't want anything to do with an unfair advantage so again this was an overall a major success and we definitely made some serious mistakes maybe even a series of serious mistakes but we managed to make the best of a bad situation and we made sure to learn from these mistakes and use these learnings to address problems on many fronts of our company now let's backtrack a little bit and talk about potential profit so we talked a lot about the potential profit which ghost train skill points represented in in disk how did we determine and calculate this value the answer a skill trading since Gil points are extracted and sold on the market in

increments of 500,000 skill trading enabled us to assign a monetary value to each such increment first we looked at the income market value of a skill injector and subtracted the value of a skill extractor this difference represented the potential profit made from each 500,000 skill points ghost trained by a player now at the time or at the time of our investigation skill injectors cost 740 million s and skill extractors cost 350 million disc which is a difference of 390 million s and that is the Esk value we assign to each 500 thousand skill point increment now now to put these numbers into context it takes an Omega user approximately eight days to Train 500,000 skill points at the maximum possible training rate so in one month a character cannot most train approximately two million skill points now plaques representing one month of

Omega game time cost 19.99 u.s. dollars at the time of the ghost training exploit this amount of Plex was worth a little over one point four billion esque on the games market with these values we can also assign a dollar value of 5.44 u.s. dollars to 390 million discs again though keep in mind that since is cannot actually be legally traded into real money these numbers are largely theoretical it does make for a fun thought experiment though for example remember the the number one player on our sheet he goes train around 1.2 billion skill points with a potential profit value of about nine hundred and sixty billion esque using this conversion the skill points he goes train were worth about thirteen point four thousand US dollars now you may have noticed that there's a bit of a flaw in the options that we gave to our

players if skill points can only be extracted in increments of five hundred thousand how did we treat skill points which exceeded five hundred thousand but didn't reach the next million now that's a great question and here's a real example to illustrate this is another spreadsheet which we made in order to visualize ghost training done by a single individual this is actually the same spreadsheet we gave to players if they asked for a copy now the player in this example and this is a real example ghost rain on 14 different characters and as you can see in column C some of his characters didn't even cross the 500k threshold at all while the other characters had varying amounts of XS it was a now how did we deal with this the answer is simple we let them keep the XS it was a compromise which led to sweeten the deal a little bit as a token of our appreciation for the players understanding and cooperation so in

other words this player in this example goes trained around thirty seven point seven million skill points with a potential profit value of twenty nine point two five billion esque however when we add the only skill points in increments of 500k the total is 34.5 million skill constraint and the potential profit goes down to twenty six point nine billion esque so all in all this particular player got to keep about eight percent of the proceeds for other users the access generally accounted for between five and ten percent now we consider this to be an acceptable compromise which also no doubt contributed to players work it walking away relatively satisfied now to be consistent we also extend that this leniency to the is scanned asset confiscation options and this was all communicated in our initial correspondence to the player now I want

to show you show you some stats with some upper actions during the investigation first off we looked at a lot of users we investigated 6036 unique characters which were operated by a hundred and nineteen different people we permanently banned 796 accounts operated by 23 different individuals in terms of in-game money we removed a total value of just over 8 trillion discs from the economy to put this value into context using the same calculation as before a trillion is corresponds to just under one hundred and twelve thousand US dollars if left in the game this amount of is would definitely have given a very significant advantage to the players involved now I also thought it would be interesting to look up some customer retention and revenue stats what I discovered is that seven months after the investigation concluded fifty-seven point eighty-five percent of the accounts with an active

omakr subscription during the investigation still have an active subscription I also looked at Omega gametime purchases in this same seven month time frame the accounts we investigated made a total or 6531 gametime purchases now some of these purchases were from multiple months in total seven thousand four hundred and fifty six months of Omega gametime were bought by these users now I unfortunately wasn't able to determine the currency used for each purchase but these months represent a revenue of just under a hundred and forty thousand US dollars or Euros and this is after accounting for both purchase discounts now keep in mind that while this isn't necessarily a large amount of number in the large scheme of things this is a revenue we would definitely have lost if we'd gone with a trigger-happy and ban heavy approach also keep in mind that its revenue from just one hundred and

nineteen different people in a seven month period or just under twelve hundred US dollars producer it's pretty clear to us that the users in question have a very high customer lifetime value which further justified the time spent on our investigation I also believe that this is yet another indicator over of our approach largely having been perceived positively by the players involved many of them are play still playing after all but what did we learn from this experience that can help us in the future now I know it sounds like common sense but the most important learning for us was just how important open and on communication is to player satisfaction we were honest and we took responsibility for letting the expert stay in the game for too long our players appreciated this we also strongly highlighted the purpose of removing the benefits of the exploit which was protecting the integrity of the game the players involved understood

and agreed with his principle our our eyes were also open to just how important first impressions are far more important than we realized before now in the past we've been pretty reactionary when it comes to explaining the explaining the purpose and reasons for our actions this time around we made both the why and the how clear up front before taking any action so basically rather than making this into a one-sided rule enforcement situation we established a personal and human dialogue right off the bat and this greatly increased the likelihood of a collaborative relationship I'm pretty confident that the players perception would have been far worse had we shot first and ask questions later or never now we also learned that directly involving the player in the solution can be great for customer satisfaction the

player is much more likely to be invested in what's happening and with increased by and by the player the chance of a mutually understood and agreed solution is much higher last but not least and somewhat tying the rest of the points together we validated just how important direct and human engagement with our customers is in fact we got the idea of offering a choice to the player and letting them extract skill points on their own after talking to the player in the number one spot on our spreadsheet so before I continue the story I want to show you a letter or letter of appreciation which was sent to us by this player and the cset survey result from another player who goes train I'm not going to read them out loud but I think they really underlined how appreciative these players were that we engaged with them to this extent now to make a long story short the player in the number one spot contacted us proactively he basically came forward

about having obtained benefits from this exploit now remember when I said that we felt a bit lost after we before we found her final approach during this time I was engaged in a conversation with this player he gave his side of the story which showed that while he had benefited a lot it had been unintentional I made it clear that he'd benefited too far too large the degree for us to ignore but that I wanted to work with him to find us painless a solution as possible so I floated some ideas and he responded I floated more atiast and he responded again he even floated some ideas on his own now after talking for a couple of days we decided together to try the skill extraction method now this Intel individual expressed clear appreciation that we were willing to work with him and propose various different solutions based on our interactions with this guy

we got the idea to give everyone else a choice of the path and making manual skill extractions as one of those choices so the point of the story is engaged with your players even if the subject matter is inherently unpleasant if I hadn't done so we probably would have settled for a much less customer friendly approach now this is the final slide and as a closing comment before we go to Q&A I want to leave leave you with a few points on areas I think we as an industry can grow in I believe there's a lot of room for improving communication with our customers to achieve this I recommend focusing on increased honesty and transparency take responsibility for your missteps such as bugs and exploits and do so publicly also pushed down that ban hammer whenever possible let's also improve engagement by increasingly involving the customers in the things that we're doing ask them for

feedback on how they would like things to be done present them with choices when you're deciding between a few predetermined options this may of course not always be possible but it's worth keeping in mind apply it when a situation allows for it also none of this necessarily has to be in a crisis or our customer service setting either it could be a thing that letting players vote on a few different options for a balancing change now these were the main points that I wanted to get across today and I hope this talk gave you some ideas and inspired improvement in these areas for your organization before we go to Q&A I also want to encourage all of you to fill out the evaluation survey sent by GDC about this session and if you can't see it in your inbox please check your spam folders so I'm now opening up the floor for questions there's a couple microphones in the middle I think we have time for two or three questions if

there are any hi thank you for the talk fascinating as so many stories from afar thank you I am one of those people who constantly regret not having as much time as I'd like to play again but I'm just curious about the whole single shard the big interconnected economy thing what was the third party impact of all the money because he talked about solutions directed at people who still had the money or taking away something from them did anyone do anything really interesting with profits like maybe influence the war or anything like that so we didn't that's a really good question by the way thank you so we didn't see anyone influence a war but we did see quite a few cases where the player didn't have any of the is left yet because had already spent it somewhere or tried to launder it and we actually went to pretty great lengths to try to track down the is in almost every case we were able to do that but there were a couple cases where we just couldn't figure out where the benefits

went so we ended up banning the players main accounts kind of in hopes that they'd negotiate with us and tell us what happened together a counseling ban in the end they actually didn't but I guess the several lining is that none of the guys in the top 15 slots that I showed you guys before were these people these were players who benefited a little bit less maybe around 30 billion is K which is a lot but not as much as 960 billion thank you I think you were next thank you very much for your talk it was really interesting Q I had a question about was there a call from the larger player community the ones who didn't participate in the exploit for some type of compensation due to like just fluster e in the community or if they found out that you were kind of sweetening the deal were people who the exploit or say well we should get something too and how do you handle those kinds of situations well I don't remember hearing anything

out like that there were a lot of there was a lot of unrest in the player base especially on reddit where some of our most critical users are based but as far as you remember I didn't see a single ticket or like a social media post asking for some sort of compromise to other players for the most part the the people who got their deal sweetened a little bit they were guys that well they had circumstances like the SK the skillpoints had already been extracted the iskut in spann so they actually had to meet this in the middle as well there was one guy who had deployed a safe to tell which is a huge Space Station and he had to take it down in order to fund the the amount of risk that we needed to take away so really no the answer is no I don't remember that happening but its

food for thought I mean I'm sure if we had message out more clearly that we were Sweeting and sweetening the deal that players might have asked for that this information was out there but we didn't like post on you side I'm saying that hey we're sweetening the deal for everyone but yeah good question thank you thank you I was kind of wondering if you think you would have done like the same thing like an honest transparent communication if the community would not have figured the exploit in the first place like is it worth it to like admit the mistake to your community or if they didn't figure it out or if they don't figure it out do you do something else well my opinion that's a great question by the way my opinion is I think we should be transparent because I think if players here about the exploited from you it's much better than the potential of them hearing about it from somewhere else and in this case the back knowledge of the exploit was out in the open before we

had made any sort of comment and that was one of the reasons that players were so frustrated and upset with us so I would say in these kinds of scenarios if you know something went terribly wrong on your mistake go out there tell people that it happened and tell them that you're addressing it that way you can also control the messaging a little bit you're able to basically make it clear that you are gonna take this seriously before the speculation on social media goes around like wildfire that your company doesn't care about the game etc because I think all of us care it just might not always look like that and when someone's posted the thing that's blown up on social media it's really hard to change that message so that's my recommendation thank you I wish we had taken that approach to go straining thank you really good question okay I think we're actually right on time so thank you very much for

attending the talk really appreciate you guys coming if you have any further questions in the next few days please don't hesitate to get in touch you can find me on LinkedIn also if you see me in the conference hall feel free to poke me and we can have a chat about evil or talk about this exploit thank you very much for listening and being a great audience