How a reborn determinism will disrupt multiplayer games development

[Music] [Music] hello guys I'm Chris I'm the founder and CTO of exit games were known as we rebuilt just one product so it's really confusing called Photon and I talked about determinism and how it was one of the first multiplayer techniques techniques ever and it becomes a second wave now and I want to talk about it because I think it's particularly interesting just before I start so who's who is technical in this room I'm getting a little bit technical

not too deep I think so I actually explained it my-my father who's a DSO should should be fine so I'm I'm Chris about 14 years ago I entered into the the multiplayer space middleware space and founded the company exit games and back in the days it was a time of Nokia phones and so on we built a back-end as a service for for mobile phones so running on on these Nokia phone series 40 series 60 and so on that was pretty visionary and great and but it was a little bit too early so we had to pivot into something else and then we moved into the real-time multiplayer space and that's about things six years we are focusing on this we have now at one would fit more than 250,000 registered developers 250 million players on our servers running worldwide and we try to run pretty much on on

every platform so multiplayer developing multiplayer is in the end pretty simple so you just have to synchronize the state on all devices across any machine without any lag right so that's that's pretty straightforward so actually it's not right so it's it's pretty located so what I thought I guide you through the process of building like like a simple game I called it car soccer so let's say it's a game of two to eight players has a couple of NPCs in it is cross-platform it's real-time it's a top-down game where you move your car and you actually have the controls WASD to move your cars and play the ball and

bully the other players and so on so so far so good it's a pretty simple concept so let's see how traditionally or the most modern way to build this type of game these days is state sync so what do you do you have two players just to simplify everything one player on the left one player on the right and you X you have between them the packages are traveling through the internet so it takes between 30 milliseconds and 300 milliseconds to get data to the other side that can forth so what do you do well you're synchronizing everything that's on on your screen right so you're serving the positions of the cars you're thinking the ownership of the objects you're synchronizing rotations and velocity you have to decide with a ball

so who is owning a certain object so can you grab something and let's say the other person is grabbing something too so you have to make sure that there's a certain ownership concept so you have to think about an authoritative piece who's actually deciding who did what so and all that can be pretty tricky especially if you have collisions of cars if you have collisions of the car and the ball and so this is just with two players so now imagine you have eight players and imagine you have that internet connection and then this can get pretty much a nightmare so this is pretty much what multiplayer development is right now so most games that you play on the Internet actually written this using this technique using States and sometimes you have like like a simulation running on the server side

which is the Master Chief and then deciding stuff so I'm I don't want to talk about this today but I want to to feature a completely different approach and it took place in the internet end of the 90s or Starcraft or beginning of the United stood us technique so instead of synchronizing state you synchronize input so everything you do on your local machine so which is WASD you go left you go right you shoot you jump or you click your mouse and so on so what would happen if we take that auto soccer sample and make it lockstep so as humilating see of about 40 milliseconds or we are playing probably within europe and you adjust synchronizing input back

and forth so input in in my game in my simplified game is just the controls going left right up down so here you see like a timeline so in there are ticks so let's say your game would display at 60 Hertz so your game is refreshing 60 times a second so that every little tick would be 1,000 milliseconds divided by 60 so that's around 17 milliseconds and you are getting the keyboard input at every tick you're taking the keyboard input so you see player 1 is going up for 3 ticks then it presses up and down which doesn't make sense two ticks and then he's not doing

anything which is now so now this sounds simple so first off you need your game needs to be capable of using that input from the other player putting it into your game simulation and producing the exact same state so now let's take the example I'm this guy and I'm I'm pressing right so now I this moment when I go to the right I don't have the information what the other player is doing right so it needs to travel 40 milliseconds to me so I have to to do something and what lock-step did back in the days because you don't have that information you're actually delaying your own input so what you're doing is instead of I press right and my car does move to the right right away rather it waits for let's say 50 milliseconds so

50 milliseconds is not a lot so you it will be hardly recognizable but you will recognize a little lag right so you do something and your character doesn't move right away so this is what most online gamers would call lack but it's fine and the cool thing is the bandwidth you're sending around it's just very tiny right you're just taking the input and no matter how many objects are flowing around on the screen it doesn't matter it everything will be totally fine and the bandwidth wouldn't be impacted by that so one very important piece why it's not well the principle is very simple but actually what's pretty complicated is determinism because taking that input and feeding it into your game you need to make sure that it produces 100% the same result on

all clients no matter if it's a Mac PC a mobile phone a console and that can be a little bit tricky especially since floating point numbers random numbers are not deterministic at all so in the slightest difference might lead to simulations which are totally out of sync so for example I take unity in my example because it's the most popular game engine right now and if you would want to build a lock stack framework inside unity it would be super tough because unity is not deterministic all over the place so you have the physics engine which is Nvidia physics using floating-point numbers so you can't use it you have the animation system Meccan 'm which is not deterministic you have you use floating-point numbers in your code it's not a finished deterministic you use random numbers it's not

deterministic so and what's not so nice as well in unity you have the view and the simulation so your code is attached to your objects and everything so it feels pretty much spaghetti so unit is working on this with a new entity component system and job system all that stuff to clean it up and just make it fast but right now it's pretty messed up so just to sum up the pros and cons of lockstep so the pros he has super low bandwidth you're just sending around the input it's very easy to replay a complete version of your game because you just replay the input and your game will magically completely replay the whole game without the complex you don't have to record a video or Aeon you don't have to record State and of course you can have very high number of moving units and the cons are of course determinism

can be super tricky you have the lag you press and your character doesn't move right away but it's it's lagging and it's pretty complicated to do late joints and if you use asset store code as its stock code is not that humanistic all over the place so 2011 a guy from the US called Tony cannon had an idea and said so he was a professional fighter fighter and developer so fighting games and he developed the technology called a gGPO which was famous in the fighting space where and especially in the fighting space you won't have a lag free experience and he created this gGPO framework which is allowing an ear necklace on an experience and in fact what he did is

lockstep but he extended it with a pretty clever concept so again to two out of soccer we have the same situation so I'm in frame 60 46 and let's say I'm player one and I do something on my keyboard so now I don't have the information what the other player is doing right now so in the traditional lock step you would actually wait to execute your own input and have this laggy experience so what gGPO introduced it's actually predicting the input of the other player based on the last input you got so let's say the last input I got from the player is now so he didn't do anything then the engine will assume that in all the ticks

before this guy continues to do nothing and you are simulating your simulation without any lag so what's happening you're you're steering your car I'd say this is player one and everything you do will happen right away on your screen and the other car will move based on the predictions so now you will say okay that sounds interesting but what happens if the prediction was wrong so if you receive so you continuously receive input from the other player so now let's say you predicted wrong and you get a package which says okay 60 milliseconds ago you basically assumed the other guy didn't do anything but in fact he went up so what happens here is a so-called rollback so the state of the simulation is rolled back to that frame

the right input is injected and the complete simulation is done again this too basically the time now so in this case the null was recognized to be a different signal and then it's replicated into so it completely changes the simulation but the cool thing is every state everything is staying 100% in sync and you have zero lag so now the view of course because you you did something wrong 60 milliseconds ago you have to deal with it somehow in the view so you have to fix it somehow so the ideal deterministic framework let's say that you could wish for in unity would be like a clean simulation which is completely not unity dependent which

would contain the physics engine which would contain pathfinding which would contain an animator to actually capture the state which would be super fast because you have to imagine that you are 60 times a second you are predicting and rolling back all the time you're completely simulating 60 times a second to have a smooth experience and you want to have a very clean interface to the view and actually we we believed in this in this solution and in the past three years we built a solution for this called quantum so kind of a reference implementation if you want and so I'm showing now a video where we have one player located in Sweden with connected with 50 milliseconds latency on the right side there will be a player in

Brazil who has 270 milliseconds latency so together they have more than 300 milliseconds latency you will still see it's a super fast action game mobile title and the game will still look perfectly in sync and every player has a luxury experience so there's no lag at all so here you see the ping is assumed out there you see the local frame number you see the simulation speed so the simulation could actually run up to 10,000 times a second so it's optimized in a way that it can be very fast so now there's a little slow motion to to track it better but the thing is you know I mean in multiplayer you don't put two screens next to each other and we'll try to spot the differences so this is kind of a little bit unfair but the core experience it's always about yourself so

you are trying to do something and everything needs to make sense on your screen so and it's so the framework we've built is exactly what I described so it's it's a clear simulation which is written purely in c-sharp it can be C++ or anything in our case it's just C sharp it's using the predict roll back mechanism so the concept of of predict roll back is very very simple and but the results what you can do and what type of game suddenly can build a multiplayer is really really powerful so how did we achieve that performance so to have this if you want to roll back and simulate forward 60 times a second you need a very fast performance so what

we did is we we went very low level and started to track the state and completely memory memory align structs so we are putting on the state in a very compact memory format so it can be easily consumed by the CPU and we are pre allocating like a rim ring buffer of states so if we if we roll back ten frames it's basically just a mem copy so it's super fast and then the simulation because we've written the physics engine in a way that it's very simple only 2d but super fast we can simulate 10,000 times a second we can run the engine easily so it's a little bit nasty because as a developer you don't want to handle with low-level pointers and States and so on so we've written like a convenient layer on top so you are actually modeling the state in a special language the domain specific language

DSL and then based on this so in this example you have resources like health and energy and then you have character array of thirty-two and then you can link resources and based on this the code is generated where you actually implement your your simulation yeah and with this technique suddenly multiplayer development becomes like single-player because your actual game code is not aware that it's actually networked a multiplayer because everything is predicted by the framework around it so you can actually write a single-player game or a multiplayer game it doesn't really matter yeah that's it I hope it was interesting and I personally think this will influence

multiplayer games in the next years to come and will enable I mean there are reasons why there are no soccer games basketball games or it's really hard to build mobile because it's really a tough problem to challenge but this technique and we are talking to a lot of studios say yeah this is exactly what we started we believe this is the right approach to do it but what has been a problem for us is simply the performance and we've well we we think we've correct the performance issue and that's let's say the critical part to to get it done but once you have that writing multiplayer game suddenly becomes really simple and that's our claim as always we make multiplayer simple but in fact it's it's not it has not always been true all right thank you [Applause] [Music]

[Music]