Crash Course in Online Features for Programmers

[Music] good day hi everyone so if we could start off with some housekeeping first of all if you could turn off your cell phones for the next hour just mute them or whatever and at the end of the talk i'm going to ask you this again but if you could answer the feedback email that gdc sends through to you okay with that getting let's get started hi i'm claire i'm from sony i work out of london and i work as a senior online consultant so mostly i work with first party titles and occasionally i'm lucky enough to work with some indies through our strategic content or extent teams so what is this talk and what isn't it first of all this talk is for generalists and indies people without network experience at the end of this talk hopefully you should know all the things you don't know about networking and you can take maybe one or two of those topics and you can research them in your

own time and find out what you need to do for your game okay so the first rule is everyone connects even if you are making a single player title you will be connecting to platform services so uh we're gonna go through a bunch of stuff that you can do in a single player and a multiplayer title first of all you need to leverage the power of the platform if you're on steam if you're on xbox or if you're on playstation there is a whole range of services that are provided to you for free to use and abuse now the difficult part about networking is actually to make it cheap to make it scalable and to make it fast oh there we go okay so the other thing i love about networking is with a little bit of networking you can actually do really powerful features but these are the kind of things that a designer can't necessarily come up with themselves and that's where your technical

expertise and networking is really going to help and if you're an indie where you've got multiple of this stuff in your head that's kind of a superpower so let's start off with trophies traditionally not thought of as a networking feature every title has it now the interesting thing about trophies is that they're stored on the server and they're shared so this is a water cooler moment that you can create this is a shared social landmark so without writing any real network code you already can have this common reference point and you can say hey your friend has already achieved this trophy or you can highlight activity that's going on in the game it's a very simplistic example but it's a good example of how a traditionally non-network library can be used for networking features leaderboards are perhaps one of my favorite little hacks and usages high score tables were kind of the first reason we invented save states and

persistence right they're the first long-term data store that we started doing in games but then once we took those same leaderboards and we made them online they're now shared online persistent state and that's really exciting that is a world view we all share now the cool thing about leaderboards is first of all find out if your platform has fixed or flexible leaderboards fixed leaderboards can be a bit of a pain in the ass because they require pre-set up and flexible leaderboards you can do all sorts of things like create leaderboards on demand for challenges and features that are hyper-specific then the really cool thing is your leadable data does not need to match the ui you display so let's say i get a score of a hundred well if instead of posting a hundred i post a thousand and one then that one can be used to represent that i played a rogue on that particular mission

and i can surface that data up and as long as a good score is a higher number the leaderboard will still function you are responsible for always displaying your display in your leaderboards so that's a really cool little hack and using the least significant bits you can pack a lot of data in and that can give you a much more interesting leaderboard the other thing you can do is you can put a simple checksum in your leaderboard school posting now that sounds like a bit of redundancy but by putting a simple checksum bit in you will avoid a lot of the hacks that happen on pc leaderboards and mobile leaderboards where they do large scale score posting and if your score has some intrinsic intrinsic little bits to it like some custom code that's yours you it's very easy to pick out like when someone's posting 99999 it's a simple example but i like it the other thing is most modern leaderboard systems have a binary data attachment now that

school trick i mentioned is still super useful because when you retrieve a page of leaderboard results you're typically retrieving a page of scores whereas the binary data stuff is typically a separate follow-on call so that binary date is really handy because you've got the shared persistent state that binary data could be a challenge it could be a ghost it could be the particular loot you finished it with there's all sorts of things you can do with that once you've wired it up and supported it and exposed it up to your designers and plan for leaderboard bugs because you are going to have them find out what your platform's policies are in regards to fixing a single leaderboard entry fixing multiple entries and bulk or potentially resetting the leaderboards have contingency plans in place because i guarantee you you will have to fix some leaderboard entries so this is the most basic example pretty much all of these platform features that i'm talking about in these social features are built on rest apis this isn't

complicated stuff but it's a good starting point so curl is something that's going to be available on pretty much every platform you ever work with and before you write a single line of code to a new web api you really should look at just poking it with curl and figuring some bits out now as you move on from curl you can look at tools like postman which are a suite of curl like tools to do automated testing or bulk calls now postman is just one example there's a whole suite of rest api stuff out there as we get on to real time networking later in the talk i'll mention wireshark again but i'm going to mention it here because you can use wireshark to debug rest style calls and to look at the actual payloads that are being sent but that's a very simple example where we're posting some json data through to an end point now there's a big gotcha here about memory allocation and runtime performance most web tools and most of these restful api

sets and libraries that are written are not written with performance in mind so especially when they're pausing xml or json and those kind of things they'll often do a lot of runtime allocations that'll cause you big performance issues with got your garbage collector so it's worth looking for a gaming specific alternative library or just analyzing in your profiler what your networking libraries are doing because they can cause you a lot of problems okay but now here's a really simple um unity example that's a very small amount of code to actually post a screenshot to an end point so that's posting some binary data to web endpoint and that's really powerful so really you need to look at what platform and middleware is available to you you don't want to reinvent the wheel because if you actually go behind the scenes of what this is doing if you look at it like a c plus place plus lib you'll see multi-part form processing where you've actually got to split up

this data can be actually quite a headache okay so let's use pixel to go into our next little bit on real time now sockets are something i find a lot of people get scared off by but they're actually really simple as a concept a socket is the basic tool we use in real-time networking where you set it up you get it to listen and then you loop it and now listening is a blocking operation and that's kind of where our problems start we'll get back to that in a sec your three kinds of sockets you can get what really two kinds tcp are the standard internet socket that you'll be used to this is where there's a lot of overhead to ensure the data that you're sending is arriving and is definitely being confirmed udp is much more low level and faster and generally what you want to use in games of course the old joke with udp is i'd tell you a udp joke but i don't care if you get it but as you move into games you'll

probably find that you want to use some flavor of udp like reliable udp in this case it'll have a bit of extra helper functionality that's added in by a library or some middleware to allow you to do say reliable ud pay packets so you know they've been delivered or allows you to do in order packet delivery you can set this on a per packet basis which is really handy because if you're sending all of your traffic as in order reliable udp you've kind of lost all the benefit of working in udp so it's worth thinking that that comes with an overhead web sockets are super cool if you're doing in anything with mobile or if you want a companion app there's a whole bunch of functionality there if you're doing um your game and you may be deploying to console or something where you might not always have access to uh the build so let's say you're taking a build to pax and you've got like a shipping build because you don't want all your debug stuff in there you can actually put like a little http server in there that's doing a little

tcp socket and you can connect from your laptop and you can do like some basic turn on stuff and debugging stuff and that's really handy so web sockets have a lot of hidden uses and all they really are is a tcp socket with a few standards wrapped around them so they are a bit safer because the browser's a bit concerned about security and as i said the gotcha with sockets isn't so much sockets they're actually quite easy and you should just play around with them um you'll pick them up very quickly the gotcha is concurrency and obviously every programmer who's ever worked with concurrency knows the overheads that comes with you've got to think about your mutex locks and you've got to think about how you're dealing with your memory and how you're going to stomp on top of each other the good news is if you're working in c sharp or a modern language there's a lot of tools that really help you out with concurrent programming c plus is getting better as well okay so the most common kind of real-time networking you're going to come across

it's actually one of the most complex implicit networking systems or object replication systems are the unity one real default so this is when you go into your unity networking tutorial or your unreal networking tutorial and then tell you let's make an object networked and they hide away a lot of the complexity the first thing to realize is object replication is actually quite a complex technique under the hood and it is very state heavy and it is a bandwidth hog the most common mistake i see newer network programmers doing is replicating everything only replicate what you need to replicate so a classic example is let's say you've got this object in your scene that has a whole bunch of state and scripts and everything attached to it but all you really care about is how it moves about in the scene it's better to parent another object to that or to have a hierarchical relationship or if you're able to replicate say just a single component that's the position component on the object because so many times i've

seen people basically replicate the entire complex state of this object and all they cared about was the position so that's a classic um problem the other big gotcha i see with replication systems catches people all the time is often a zero and null or a default value won't be replicated this sounds like a weird edge case but you'd be amazed how many bugs i've seen come up from that the other more classical style of networking and i'm going very simplistic here as an introduction is sort of input only so this is when you're saying i've pushed a button and i'm sending that button across the network the first bit of advice is don't send the button make it abstracted out so i am pushing i choose to accelerate at this time and you timestamp that the first implementation you can look at this is a lock step implementation what this means is we gather the inputs from every single client we look at them and we go okay we can

move to the next frame now that sounds really basic but it's super reliable it has no weird complexity to it you can always ensure it's fair and outside of some horrible lag spikes which is where you're going to get your main problems like when the lag spikes up to 500 800 milliseconds most of the time your latency is going to be sort of around the 200 300 range and you can easily hide that in your game if you're dealing a turn based or a slower action game so as i say it's very low bandwidth it's very performance but it is sensitive to spikes and it's worth looking at uh if you can rewind your state because that gets us onto alternative facts now i could do a whole nother talk on this but the laws of physics i'm afraid cannot be broken the speed of light is a thing we actually have to deal with um it's worth realizing that even computers in the same data center talking to each other add significant overhead so the first

thing you want to do is think about not what is the truth but what the client sees of the truth whether they think they hit the target so some common techniques you'll come across in research for instance if you're doing a shooter you should look at relative target aim systems there's some very good talks on that where basically the the concept is rather than caring about where i shot the bullet out and doing a simulation that all the clients agree on on whether or not i hit i'm saying do i as a client think i hit or i'm saying i'm shooting relative from my position at this at this angle now obviously this is open to cheating and exploitation and it's where a lot of this hitbox problems come is where a lot of gaming of lag and stuff comes in but it is a good technique and if you want to make a performance shooter is really one of the basic techniques you're going to use also um smoothing or correction remember how i was talking about lag spikes

in the old days you get a situation where you're playing quake or something and you'd get your character sort of jumping about the level and that was because there was no um smoothing or correction these days when we get lag spikes and the truth drifts away from our alternative truth we need to kind of sort of bring those together without the player noticing and so we do that over several frames using smoothing techniques and correction also if you look at a lot of libraries they will have some kind of projectile motion option available often and this will basically do a lot of this smoothing and correction for you over the network and it'll handle a lot of the negotiation all of these topics are worth investigating individually if they think you if you think they fit your game but most importantly i would say if a library offers it take the libraries or the middleware's offer as a starting point

because there's a lot of really good code that's been written out on these areas and you can always replace it later with your own which comes me to my final topic in the real time section rolling your own networking please please don't do it until you feel very confident in networking a much better approach is to mix and match you can start with an object replication system doing 100 of your networking and for a lot of smaller games that is actually sufficient but then what you can do in the engine is you can say okay first step i need to stand my game state so you go off and you take some post-it notes and whiteboard it and document it and you understand at any given time in your frame what data really matters to you and then you decide what data you need to send across and you may find that there is some data that you care about more than others and in those situations you can

basically say for this particular object for this particular class i'm just going to not use object replication but i'm going to write my own implementation for that usually talking directly to the middleware because the middleware will have some kind of exposed layer for low level and that's very very handy then the next question is can you rewind and replay this um first of all there's a cool feature if you have demos right if you can actually save off the game state in a later time playback the game state that's a cool single player feature anyway but if your network code if your game at an under a basic level supports this you're in a really great state for networking there's a lot of advanced techniques that open up to you everything from service date verification to um rolling back the state when you're doing host migration there's a lot of cool stuff that comes up if your game supports a rewind and a replay state and that's no networking card so if you're a confident coder

that's a good place to start when you're trying to optimize your networking and different channels some platforms will offer you or some middlewares will offer you the option of prioritizing your traffic that's a big win a lot of the times where you can go okay i really want to prioritize this above this so for instance voice channels are something we usually put on a very low priority or if you're doing a vr game and you care about players position a lot but you might not care about their exact fine grain motion tracking until they're right up close okay so because we can't change the laws of physics the best thing we can do to help networking is to send less data and to do more with that data so the first thing you can do is throw away data you don't need to send right so in the classic shooter situation where you've got eight players battling it out but there's three players across the map rather than simulating the whole physics and everything that's going off where those three players are fighting you can say i don't care about those

three players update their position once every second and if there's things like rocket explosions and gunfire going off just tell just as custom networking calls send off the fact that there are explosions and rocket fire going off in that general direction and that's a much more optimal way to handle that particular situation you'll find a lot of cases where you can just throw away huge reams of data that previously you were just sending by default c plus plus unions are a very basic topic but i find too often in networking because people haven't come from a performance critical background or a memory critical background they might not realize how much um using an n32 where an int 8 would suffice will actually cost them and in a situation where you can actually use a c plus plus union or many other languages equivalent to bit pack a structure you'll get a lot more bang for buck you know if if you're sending something every single network frame

and you're sending maybe eight instances of the player's data right because it's an eight player game and you're able to shave 16 bits off that you'd be amazed how much that all adds up so the next thing i'd say with compression is we're not talking about writing a better zip program what we're talking about is writing domain specific compression so if you have ugc for instance where people have a custom hair color instead of sending a full rgb value over you might just have a selection of colors that you're sending over that can be compressed right down or you might move your custom colors to say a hue saturation value space decide you only need t increments of hue decide when the value is too low that you just throw away the hue there's a whole bunch of cool stuff you can do and if you're interested in some more advanced compression techniques that will really help your networking i highly recommend the google compressor head videos as a good introduction to a wide range of networking techniques of

compression techniques okay social features one of my favorites uh we'll use vertex to go into this one okay messaging this is top of my list um we have all heard the story of playing a game and finding a message or a present from a loved one or someone you haven't spoken to right we've all heard that story that tugs the heartstrings the vast majority of those players sending your game are going to be yo they're going to be simple don't really care but out of that minutiae of messages you're going to have some really really special moments and every single platform has an option to send sort of messages with a bit of binary data to another player for free with no server on your site so why not use that that's a really powerful thing when you can attach a bit of binary data to something like this you can build whole systems around

messaging you can not only send someone a message you can send them an item you can send them a challenge you can send them all kinds of stuff messaging is a really really powerful system and especially if you've got a smaller indie game that doesn't have a huge player base it doesn't mean that everyone has to be online at the same time as well it's i really am a big fan of messaging forums and communities one of the things everyone who plays your game has in common is they play your game wiring in communities or wiring in a platform's forums into your game is technically not as hard a task as it sounds many times there will be helper libraries and tools that will assist you in this and when that's when those social tools are brought into your game your players are much more likely to engage around the game a very 90s concept i like is shout boxes yes the sharp box may just be you going to your players hey look at this thing

that's happened whatever and it could be an rss feed on your personal website it could be on dropbox you know there's all sorts of places or you could put it on a platform data store but you could also maybe allow your players to use the shout box there are a whole range of ways you can approach that and with the gotcha of moderation if you are relying on platform systems you can on the platform for moderation and that's a big win because if you have to take on the moderation task it can get a bit daunting you may be fortunate enough that you're making an 18 plus game and you say oh well free speech but for a lot of people moderation becomes a very big topic so if you use the platform services moderation gets taken out of your hands and you don't have to worry about it help share sharing right um in this day and age everyone is taking photos and videos of your game the first step is make those photos and videos look good the second step is don't send them

out into the wilderness like these files to be lost forever give them some metadata you know that photo was taken when they had a hammer equipped and they were on level five and they were fighting this monster right you you can pack all that data in you can put it in the image as well but you can put it in the xf data or the metadata of the image another thing that's becoming a trend that hasn't surfaced up a lot but more and more we're talking about temporal tagging in videos where you're able when you're recording a video to say at this moment the player scored a goal at this moment the player killed an enemy and actually tagging that in the vtt track of a video and that's very interesting because all of this data that your game is generating goes out and then you can use platform services or you can even hook up to uh youtube or images api and stuff and actually pull the stuff back into your game and sort of show the community

all the cool stuff that is happening around your game now you don't have to go the full final fantasy level but we all are kind of prompted at the top there and we kind of suck at taking photos and if you give players help if you when they post they don't see this white box but you've primed the social pump with a personalized message that says i know you are fighting pirates on the sea and in your piratey game you know you actually get your copywriter to go yar and do that you know flavor text that's very cool also if you look at ai captioning i'm not suggesting using ai captioning but it's a good inspiration point because you can see the stuff they're doing just from an image you have the frustum you know what's in the camera's view you know what the player was locked on when the photo was taken you can do a much better job of captioning that image and making a story from that image then any ai algorithm can do at the moment with just a basic image

conflict in the cloud so patching is really hard but uh changing your config is easy so you can store a json blob in again like dropbox or your own server preferably or a platform data store that has some basic balance values in right so this means that if you discover an enemy is too overpowered or a particular level needs a tweak you can just update this json and very quickly push that out that's a very powerful tool as you rebalance on the fly you can also unlock or trigger events so a special halloween event to a christmas event using this kind of config a big gotcha is no execution i know some people try sneak lure or something in these kind of systems it's a bad idea the reason why we're so careful with patching on platforms is because this kind of injection point can be where your game gets broken open in a horrible way some other gotchas is

make sure the game is playable with whatever local version of the config the player may currently have don't rebalance the entire game on the player mid mission think about when you're going to update those configs and it's worth having in a versioning system and noting that in your save files and such like that because you'll get situations where players pull old saves to new machines or maybe a new save to a machine that isn't 100 update because they've still got the save data installed but they've put in the disk and they've never connected to the internet so they haven't pulled that stuff down so there's a whole range of stuff around there that you want to think about platform data oh i love platform data you can do so much with a bit of binary data in a store that you can access and it's 10 times better if you're not paying for the storage it's a big bonus feature if you can take this dump you dump it on a server and you can get your friends data dumps the moment you've got that

you've got again the shared persistent state that i love talking about and you can first of all check if you have atomic access to that so some of these data store systems will provide atomic access functions so for instance if you want to get a number and increment it they may have a low level function that allows you to do it that's very efficient and it means that you're not stomping over each other if you don't have atomic access be very aware of when a player is getting data from that who owns that data so you don't get a situation where multiple people are writing on top of each other so that's a good thing to look at if the particular data store you're using has some atomic access functionality now with this basic binary data it doesn't sound super exciting but you can build an entire trade and loot system out of this you can build a basic auction house you can build seasonal events you can do your mission editor and user generated content by putting stuff in there and you don't even have to go stream as a full letter

level editor even if you're um let's say you've got a customization system where your character's picking particular loot and stuff you can just upload the person's gear their current loadout to that and then you've mimicked the blizzard armory system in your little game using platform data and all of this won't cost you a sec so it's super super powerful player signs and dark souls messages is another obvious one that's not hard to do and i think that also adds a lot of vitality to a lot of games i know it's a commonly used function now but i think it's still a powerful one online saves so online saves our basic promise here is never lose progress that's the key headline here so the first thing you want to do is some basic sanity checking on your saves be assured that you always have multiple saves in your game whenever you're pulling down a save you verified that you've got one or two save files that are good

so you're not overwriting anything don't have a monolithic save state right when you're doing online saves and when you're syncing this kind of stuff it's very good to think of it in a version control mindset right so first of all if you can break this monolith up to for instance if you're tracking how many monsters they kill break out the monster killing bits to the story progression bits right because if you're doing a story based game and i'm on chapter five on one machine and chapter four on another yeah clearly i want chapter five but maybe i was killing just a lot more monsters here now this is down to a design decision and you need to decide for your game what's the right whether you just let them take whichever monster kills the highest whether you just add both in you've got to figure that out for your particular game but you need to make these intelligent decisions about how you merge data and if your saved data has a delta like you know what's changed between each save state it makes this kind of

merging behavior much easier obviously as you have more backup systems for online saves and as you open this up more it is a bigger exploit potential for cheating so that is worth thinking about and as a fallback whenever online save data goes wrong always ask the player make it their decision okay so getting together is one of the hardest topics because as you're a smaller developer or if you've got a smaller community hell even triple a titles worry about getting enough of their community together to play so the first thing is we all love your game we all play your game that's what brings us together so your community features need to be in your game so think about the journey from stranger to acquaintance to friend and how you can enable that the first step is to think about when are we going to play if you can have functionality in your game that

helps other players decide when they want to play a game that's extremely useful if you can highlight that another hack i quite like is irc chat so frozen synapse did this quite well and i've seen a few other games do it where they've basically just put some irc client code into their game and they connect up to a chat room on afternet or free node check the policies of whatever irc server you're using and then you have a community chat in game so even when people are playing the single player maybe they're playing the multiplayer and they're still in the middle of a match but you're a matchmaking or whatever it's a basic route for people to see the rest of the community and to sort of hook in and maybe someone is struggling with the game and they can just go in and yes the chances are that there's going to be some toxic chat in there or it's going to totally derail and stuff but that's not your problem you can do community management and stuff like that on the purely technical side

getting irc in is a cool function to explore um and that brings me to my next topic social apis a lot of the stuff i'm not mentioning here is the smaller stuff it very it varies platform to platform whether you're on google games whether you're on steam whether you're on playstation there's little features social apis that every platform has it's really worth reading those and putting them in the game even if your game doesn't end up using them doing the technical implementation for these features tends not to be very hard it gives the designers something to work with something to put in and platform holders love it when you use their social api stuff and they will potentially give you a bit more of a leg up and the other cool thing is a lot of these features are often overlooked so when someone goes into some section of a social functionality on a platform that's not widely served they're much more likely

to see your game so that's a big feature events and calendars again that's becoming a more common api we've seen on platforms and that's a big helper for your community to get together and to to gather so social matchmaking matchmaking is a hard topic it's a really tough topic there is a way to think about matchmaking that's much more open and that's around social matchmaking so this is how do you make that journey from acquaintance to friend right of stranger acquaintance friend whenever i have a match whenever i compete against you it is a story right the first time it's not much of a story the first time the story is simply oh i beat you you were really heavy on the rockets i had really good aim right not much of a story but the next time we match up you can say hey do you know you played this person on friday oh yeah you beat them but they were really favored rockets or their aim was really good

and you can start telling the story you can start building on that and as you save this relationship up there's more data there to do storytelling and if you're the kind of programmer who's into narrative and procedural storytelling stuff there's some very cool stuff you can do there you don't have to save this on a server you can save this locally obviously if you're saving on a server there's more functionality you can do with it but even saving locally it's very powerful and graphs are really your friend here as a data structure uh don't go mad with some crazy graphing api graph data structure is literally node edge node right it's the simplest thing and you can make graph operations really efficient when you think about it in a memory sensitive way again that's a whole nother talk but graph operations are really cool and if you're interested in how you can do uh nifty stuff with your matchmaking and

build these shared stories so the next time i play against a stranger they're not a stranger they're an acquaintance and maybe then we have like three games over the period of a week and we maybe start chatting or we maybe start you know interacting more and if a friendship is built around your game that's very powerful and friends remember that they remember how they meet and they often go back to the game and that's how you build real fans that is a very powerful networking feature also spectating matches um if someone i know is playing your game and i can watch in on that that's that's again another way to sort of bring that stuff together especially if you're doing this with acquaintances because it's a way to just sort of see oh yeah that person that beat me two games ago is playing up against someone else i've played against or playing against a friend very very useful invites okay so invites could be as simple as a phone text message to someone saying you know you

want to play this game but it can be a hand crafted message and that's the really fun and interesting bit so the invite is the thing that lures someone into your game most platforms support uh attaching an image and some text right and again this is a basic rest api so you can actually do some really cool stuff rather than sending generic test jet text and generic image you can actually select a specific image for your invite and you can generate specific text there and if you want to do some cool stuff with the programmatic images this is a really cool opportunity now the first thing i want you to do with your invites after you've thought about that before you even touch the networking libraries is i want you to make your game boot up using command line arguments and go straight to a level go straight to a particular matchups case skip all the menu guf now once you

have the command line functionality in place it's much easier to integrate the invites system of whatever particular platform you're working on and so when someone loads up this invite they go straight to the game they skip loading the menus they skip a whole bunch of guff and that can really speed it up because you every time you do anything with matchmaking we care about if matchmaking is 10 seconds or 15 seconds or 30 seconds or a minute right we really care about that but we don't often talk about oh matchmaking was only one minute for me but then i had a five minute load screen you know there is a whole path there a whole journey that you need to go through and it's not something that's easy to retrofit so it's really up to you to handle that as early as you can and really death to the dell lobby there's so much more interesting things you can do with a lobby and if you can skip the lobby altogether you're in a much better situation

for your networking flow okay matchmaking concurrency is king everyone worries about concurrency from small titles to very large titles concurrency is one of your key cost factors that you can use to determine but it's also one of your key success metrics with networking and is the thing that smaller titles really struggle to get that critical mass for matchmaking so your peak concurrency is probably going to be just off after launch but be very aware that your game may be much more active on the weekends you may have tuesday as your most active game think about what regions are act and how you can action that and improve that for your community give the community actionable information right if you say to people 5 000 people are playing the game great i feel awesome well that information is kind of useless to me but then you tell those same people six people are playing the game well i'm bummed out

what are the chances of me finding a good match that's not actionable information what's much more useful is to tell your players hey did you know tuesday 7 p.m is our most active time or hey do you know your friends currently playing single player you know like bring them in or hey this match that you with someone that you know is ending soon like here's the score right maybe go spectate it until the end there's loads of cool actionable information you can give the player drop in his queen if if concurrency is king drop in his queen the factor with drop-in is based on your session length and based on your concurrency numbers drop-in becomes either not a really critical feature or an absolute must-have if you've got low concurrency numbers and your session time for a game is say 10 minutes what will happen is people get matched up they'll go into that game

and if your matchmaking queues are slightly longer they're much more likely to jump straight back into that game straight back into the queue and get matched up very quickly and so you'll have these bursts of activities and you'll have this cascade effect which means sometimes your matchmaking can be 80 percent of your session time so you can get some really bad behavior if you've got a longer session time especially if you've got a strategic game where session time is say 30 minutes you can get into really black hole with that so drop-in becomes extremely useful the other big gotcha is unless you are supremely confident about your concurrency numbers one bucket just have a quick play bucket shove everyone in it if they want to do custom games or whatever let them use the invite functionality for that put in other social tools for them to do those custom games every time you add another queue or another bucket for your matchmaking to

go in you drastically increase the matchmaking time and it could be that you actually have a fairly active community but they're going into different queues and the other problem is you may have a game mode that players don't enjoy as much as other modes but because the matchmaking times are faster they go into those modes and then your premiere mode or the mode that you think is the most fun actually suffers because people are going into 1v1 say so that's a big functionality there okay so jumping straight onto servers pixel again very shocked because servers are quite shocking i love my cats i'm sorry so peer-to-peer peer-to-peer is extremely complicated okay but it is the default option and the reason is the default option is because it keeps things cheap peer-to-peer is going to always produce a sub-optimal topology right unless

you're one-to-one there's going to be a situation where things are not the best okay but the existing frameworks for peer-to-peer multiplayer are very powerful and well written nat punch through basically relies on a third-party server somewhere doing that for you so if you're using for instance platform matchmaking they've got a server doing that but chances are you have to rely on like a steam server or playstation server or microsoft server or a unity offer that an option for it as well to actually do that for you so that's something you have to consider and the other thing is but going back to that sending less data it's very important in a peer-to-peer situation where if you've got eight people playing a game together don't send all data to all peers throw away data as much as possible right because uh if you have to propagate data to all peers you can get into some pretty

horrible situations now the cost of this is if you've got partial game state shared between all your peers and you get into a situation of host migration or you get into a situation where people who hold key parts your game state drop off your mesh you can end up in a situation where you're like how do i recreate the state again a bit more of an advanced topic but it's worth knowing that that's a black hole you can fall down it sounds like i'm really dissing peer-to-peer i'm not i'm advising you it's technically complex it comes with a lot of compromise but for most titles it is the correct solution because it's cheap and um it's very easy to end up in a situation where you might as well put two dollars in an envelope and send it to every player playing your game so it's it's to be avoided um sorry it's to be used so tiny server the first thing about servers is i would

say a very small server can be very powerful you can write a simple c plus or node.js or golang server that does some basic rest api functionality and this can do all the kind of stuff i've been talking about with platform services but even more right it can be really powerful a raspberry pi scale server for most smaller indie games can serve most of its needs for um basic low low cadence networking that's very powerful and you can later move that small server to the cloud and such but it's a much lower commitment the other thing to think about with this kind of server is that if it's providing additional functionality sort of premiere functionality if it's down can play people still play your game because if the server goes down and all that suffers there's some social features in your game but people can still play the core

experience you're in a very good shape however if the server goes down and it hinders gameplay in a significant way you're basically in a devops situation so that's worth thinking about the other thing is when you write a small server like this you should be writing tests and restart scripts and stuff even if it seems a bit overkill for your small application it's really useful i have a app on my phone which pings my home servers and stuff just a few small ones and some actual proper servers as well and so when a server goes down my phone just buzzes and if the server doesn't come up in a bit i you know call home and i'm like what's happened the cats have a network cable again right replace that so it's good idea to have these tests and restarts you'd be amazed at how many times to fix a problem on a server we just restart it and we just restart it because putting in the engineering time to fix it is expensive infrastructure so now to go from that small server

to go to a situation where you are spinning up game instances on demand running um game logic on the server side is really shifting from a battery to a power grid you're now in the infrastructure business right and running game state server side is expensive as hell i know it sounds really appealing to say oh i'm just going to run unity on my server or i'm going to run unreal on my server you can run up some big server bills that way it may be the right option for your game but be aware it will cost you a lot of money and the moment you go into that situation you can't use platform networking typically you have to have a standalone matchmaking solution you have to manage spinning up your own instances your bills can spike horribly and your devops becomes a lot more complex but there is a bit of a tractable getaway if you're pc dev

if your pc dev player servers are kind of awesome because you just provide a server executable and you say your players hey do you mind running my server for me and you'd be amazed at how good players are doing at doing that the big caveat is you've now lost control of your service piracy is much more likely and you know you may have a whole secondary market the thing is if you've got an indie game and it's competitive no one's going to run better servers than your community they're going to spend out the zoo to make sure they get no latency so pc servers and player run servers are actually a good sort of trap door exit if you really care about performance and you're willing to make those sacrifices the other thing i want to mention is local for development which is you need to run your infrastructure locally especially as you get to a more complicated structure something that runs over multiple instances and stuff and uses a bunch of different servers and

technologies and you need to document what that's setting up your local system is you know how do you run cassandra locally you know what is your particular lamp stack there's probably like 20 different pieces of technology you've installed locally and you need to know what that is because you're going to onboard another programmer at some point or your machine's going to break and you're going to need to move to a new machine and that's usually going to happen when you're having server trouble and you don't want to be in that situation the other thing i'll mention and i can do a whole nother talk in fact i have done a whole nother talk on this configuration outperforms single threaded so in the modern infrastructure of networking we have a tendency to go wide to have a lot of server redundancy and to use a lot of complex middleware and often this is the correct solution but many times a single threaded c plus server can drastically outperform

these broader implementations if you google that you'll find a bunch of interesting talks around it but for one example there was a graph processing situation where someone found an old laptop would run at orders of magnitude faster than a full-scale like multi-instance aws deployment with the top tier professional graph pricing software and this was something a guy had written in c plus plus in like a weekend because they're overhead in communication so that's an important topic to look at as well one of my favorite stories to tell is there's actually a aaa title inside sony that deploys um pretty much one server and then one server for redundancy for a pretty major title and that's because of that cost of single threaded configuration that outperforms single threaded okay so cheetahs you're going to have cheaters in your game

it's a given state verification is really good if you can do some basic checking to see if the state they've gotten to is valid building peer trust if you have multiple peers and you're able to communicate between those and find out who's the bad peer that's a common technique replay verification if you've got the functionality to play back those replays i mentioned in the past that's a really solid technique for detecting a lot of behavior machine learning machine learning sounds like something only a really big company would use but you'd be surprised at the tools and services available out there at the moment to basically use machine learning to detect aberrant behavior and in the worst case figure out what your worst case is you should excuse me you shouldn't have credit card numbers but you might have embarrassing karaoke videos that people don't want to leaked so moving to the cloud everyone who is aware of cloud stuff aws shut the bed yesterday and that was fun

because that was in the middle of a big game launch um it is someone else's machine it will typically be more reliable than your machine but it is someone else's machine moving a small server like the raspberry pi instance i explained earlier to the cloud is actually very simple and i would encourage it as your first foray into cloud computing but building up infrastructure is much more complicated compare service providers aws is kind of the default choice but you may find that azure or the google offering will actually be better for you and if you're talking to the sales people of these particular platforms you would be amazed the deals you can get so understand what that platform offers you because what they're offering you isn't hard where they're not offering your computers the cloud offering is really about the tools and services they're providing like aws is elasticsearch for example okay and devops i've mentioned already which is a huge cost so in conclusion rest apis and platform

features really are the key thing to look at they're the bread and butter stuff it's not super complicated and they're basically free to get in there's a lot of core social functionality you networking you can get into the game that isn't a huge headache real time again start with the basic middle west provided stuff optimized later and servers if you go in small scale it's not a huge headache if you go into the infrastructure it's pretty big find two things in the talk and follow that up so as i say you will be getting a feedback email pixel and vertex asks kindly that you fill it out and with that if any of you want to come to the microphones and ask questions or if you want to poke me on social media my twitter is there thank you very much you