Skill Progression, Visual Attention, and Efficiently Getting Good at Esports

[Music] hi everybody my name is anders frank i'm sitting in a cool room with the zelda in the background and i'm going to spend the next 30 minutes to talk about visual attention and getting good at esports my talk has a key message and that is that this attention matters it can it can support better game performances as it provides the opportunity to see the possibilities of actions in the game environment and to establish that claim i would like to introduce a theory from psychology that helps us understand the relationship between gameplay and visual attention but i also want like to share some kind of work we're doing in the game league of legends before i get heading on on that topic i would like to introduce give a little bit background on myself

i've been interested in in use cases for for quite a long time it started actually in the 90s when i was given the opportunity to head a virtual reality laboratory everybody was of course technical optimistic about technology at that time and i worked at the vr laboratory for the swedish military to try to find use cases and how to use that sort of technology for the swedish military quite interesting head mounted display oversight in the end of the 90s it was the technology was not obvious or was not that trendy anymore and it became replaced by games and game technology and so on so i shifted to look at that one which was quite nice for me because i love games and uh so we start using the commercial games engines and also game commercial games

to adapt them to force release military and we try to figure out ways to use that technology for other purposes and just combat and violence so we did um for instance a peacekeeping mission trainer called foreign ground for the ongoing u.n mission in liberia and interested i became so um interested in the training aspect of it so you can actually train by games and learn from games so i i shift focus to do that in office training trying to address that phenomena of war gaming now one how to imagine remember that wargaming has been around in cadet and officer training for centuries but my my sort of interest was not the only the good side but also the back side of it the bad sides which actually became the thesis uh the topic of my thesis where i just thought to introduce

a concept called gamer mode which is basically when you play the game and not training for the real stuff it is the it's just the backside of the game because it becomes so powerful that you actually lose the concert while you play the game so that became the topic of my phd but since uh 2017 i'm in a toby working on other technology in in this sense i'm working on eye tracking and i work as a position as a game experience researcher and once again i'm trying to identify the user values uh but now with eye tracking and i've been working with two with many well-known games to influence the games with eye tracking for instance shadow tomb raider or the division

um where we really trying to define aspects of the game and influence the game with eye tracking to move the camera to aim with your eyes and also to to create immersion effects and lightning and so on but since a while back we are more focused on on esports which is kind of another aspect of eye tracking because it's analytical use to use the eye tracker technology to become better and a couple of years ago we did a study among professional dota 2 players and we interview them to try to figure out what kind of skills are needed to be good at the game due to two and the arguments and the logic behind that was if we're going to introduce eye tracking to support that skill level we need to first of all we need to understand what kind of skills are

needed to play it again to play it again good so this is uh all these answers interviews was then analyzed and categorized into different groups of skill levels and we could see in this graph how the different categories are changing over as you move from novices to experts and the first category is mechanical skills which is basically learning how to play the game the moving the camera control and their hitting and the creep control harass stuff in your body abilities and very mechanical skills which is a skill level that's quite easily been learned in the beginning but it's never ever learned finally learned to the extent that you don't learn anymore so the graph of that one moving from novice to experts actually goes from have a steep learning curve and then plateaus but it doesn't

it still increases the second uh category of uh skills is what we can call awareness skills and awareness skills is when uh actually you would would like to see to read the game if you survive and not keep being killed constantly are you looking at the team positioning yourself and also the opponents and the incoming threat you're being aware of what time it is where the time awareness and cooldown of your abilities but also what kind of level the opponents are and that is a skill that's quite not not really introduced and learned when you're novice it becomes aware of it but it is being more associated to intermediate than expert players and the last category is what we can call higher cognitive skills for instance be able to really really really understand the game in depth to understand to predict what's happening what are the

enemy intentions what are our current tactics what to do but also in the meta level what items and abilities or heroes should i choose given the given current situation and that skill is mostly associated to those players who are really good at the game um so having this graph in the end gave us a rough estimate that um having the eyes on the game really changes depending on what kind of skills level you are trying to sort of enhance so the question is done how do players extract meaningful information from the game because this sense it is different meaningful information of tactics that is for actually creep control and the clock and so on so how does actual visual attention and gaming relate how can the player extract meaningful

information and one aspect of that one is to introduce the theory of ecological psychology one of the proponents or of those is psychologist james j gibson he was his battle's life interesting in visual perception and in his work he coined the term fornices which is which i would spend some time to to talk about a fornices is uh the mutual relation between the organism that is of course humans and the environment and one need to keep in mind that the environment contains everything uh it is the background is the society it's the forest it's the objects in the city it is the humans and the animals so it's everything and all those things exist in relation to each other in in a layer structure which

consults changes as events occur humans and organisms organisms are actively changing the in the environment interacting with it and the phone is is actually when the environment how that offers to humans it is the relationship between an agent and the environment and i will give you a couple of examples here to really simplify what i mean if we take an object as we usually call a share it offers most humans a place to sit on but it doesn't offer that to a very small human for instance an infant who's not able to do that or animals for that set take for instance an extreme value as a fish fish cannot sit on this one so we cannot say that the this sort of object is something you sit on it applies to certain kind of

of organism or humans even if we design and create objects to offer actions to most humans and one must remember this always relative to an agent so this scene of the cockpit uh as i am and i'm two meters and four centimeters offers something else than this than the place to sit and a restaurant and move around for me it's more like offers torture cannot stand sitting in an airplane that's for sure another example is that we have something arabic called we can call a hammer which has a certain some kind of a qualities to it it offers something to grasp it offices you can hit with it and you can also throw it but it does but that's only relevant for those individuals who can grasp the hammer and

and for instance humans without arms is that's not an object that offered that same sound qualities properties for other humans and what about learning then well um part of it is that it is constrained by the physical properties of of the agents or ourselves as interacting with environment but many other things can actually be learned and there's abilities that can be developed and one when you need to keep in mind that the the the perceiving of affordances or the possibilities of actions are essential for survival it is what our perception system has been about to do so learning is about different differentiating and making finer distinctions about the world to discover new affordances that was in visible by experience training time and explorations

and it usually comes to that extreme when an organism is totally compatible to an environment we can call it attune to the environment and then in that sense the experts become a sea of furnaces that's more or less investible for for for novices for instance if we take this beautiful part of the forest for me it it's a place that offers walking about in taking a rest enjoying the smell and sound of it perhaps my stepmother looking at the same situation or same image will see other forms you will perceive other perfor affordances than i am she will immediately respond to say that this is something where i can pick mushrooms and she has done that of course to make the financings and of course and have experienced similar situations

and cultivated her perception to see those fullnesses and same goes for new sports situations where a good soccer player can see the opportunities when to pass when to shoot she or he are able to read the game and interesting uh for us as spectators of the sport is that it's enjoyable to see those really good soccer players in actions we can if we can see those acting upon their fullnesses which is kind of interesting because it doesn't only applies to perceiving the what to do is also performing what to do which take us to the difference between perceiving and acting on our furnaces and according to the theory uh there is actually no difference seeing an act in our fullness is deeply intertwined

we act to perceive and we perceive to act we move our heads to see better what's going on and we shuffle off objects to perceive newer furnaces we move a wall to babysit behind that we we construct rows and level out the environment we do constant exchanges to the environment to proceed newer furnaces to to make a roadway drivable and so on and the theory also claims uh describes two different kind of actions to that one and the first actions aspect is exploratory and that is a part when you explore the sort of the perceived possibilities information gather all the information what kind of forces are in their surroundings and the other one is actually performing it as i said in the earlier example of the performatory soccer player he or she is able to see the opportunity but also act on it and to give in another example for this

one in the in the human activity of playfulness and game world we have a game of chess gamer chess is a game that's more or less designed around the exploratory function performing the moves that is changing the the chess pieces change the position of the chess pieces is more or less trivial but what to change his own seed opportunities is the hard stuff in the other end of the spectrum we have a tracking field for instance 100 meter dash relay which is kind of a trivial way to understand what to do you should just run in a straight line 400 meters but it's super hard to actually perform that in a good way and if we go through the video games for instance counter-strike that's something that's always usually false in between you not only need to move around to see the opportunities of what to do and so on you also need to perform quite well so when you get an opportunity for a shot you need to

really be fast enough and do it really really quickly so most games have those kind of two aspects of actions already designed in them to play the game so now i'm going to show you a video uh from a influencer called back jackie so when he explains how to play pubg and we will see he's using eye tracking in the in this video so we can see the bubble moving around where he's looking at and he will use uh show us how how a technique to spot the enemy and as you will see in the video he will constantly move to be able to perceive the opponents but we also will see that he's quite effective to performing and realizing their foreigners whilst he when he gets the the kills opportunity for kills obviously let's see if i can get it off here

welcome guys i've said it before being good at spotting enemies is the single most important skill to have in pubg the reasons are obvious spotting your enemies before they spot you allow you to move in open areas with low risk allow you to take the first shot in a gunfight or let you decide if it's better to avoid confrontation when executing stealth tactics the art of spotting enemies lies mainly in the way you use your eyes where you focus and how you scan your surroundings i illustrated this manually in my old spotting video but now thanks to my sponsor tobi gaming i am able to show you this using tobii eye tracking technology what you will see here is as real as it gets as i have been able to record exactly where i look during gameplay the content of this video is scan types

focus resource management and detecting movement the first scan type is the horizontal scan the horizontal scan is by far the most important way to spot enemies when running in the open my most frequently used strategy is to stay completely away from vehicles and instead simply move out directly in the open the reason i feel comfortable doing this is because of the effectiveness of horizontal scanning horizontal scanning is a method you want to use when you want to be able to consume as much visual information at the fastest possible rate which is also super useful when holding a position or trying to identify the location of an enemy while looking in the direction you just heard him fire a shot in horizontal scanning should be used when an enemy would naturally stand out in relation to his surroundings by contrast alone using the tobii eye tracker 4c i was able to record this heat map of my

focus which is really cool because you can clearly see in which areas of the screen i looked the most take a moment to study this recording and as i said earlier the horizontal scan the wacky jacket goes through and more example in the video i don't have time to show it is a technique to really see the affordances horizontal scan is not the performance itself it's actually what's uh it's a technique to discover their furnaces and since battery royale is more or less a game but if you see an enemy if you see a moving avatar it's it's the enemy it could be the team of course but but it's mostly the case it is this is a game so it's mostly enemy so there are certain techniques to really discover those performances when it comes inside hit the enemy um so the horizontal scan is a technique to

discover the affordances which comes up to the summing up to the ecological psychology and gaming and i borrowed this quote from eunice linderot who wrote in his paper that gameplay is to perceive and act on and transform their furnaces that are related to a game system or other players in the game and in this sense it's interesting because it's a definition of game play and gaming with this theory in mind that doesn't actually care of what what form it is being played in it could be board games it could be video games and it can even include sports which is funny because there's given the current debate debate about whether esports is a real sport or not so what about then visual attention how to improve that skills can we support the visual attention the ability to

to attend to your visual perception yes we can and we just started this one in this journey we can provide metrics we can track the visual attention in some aspect and by having mattress we can explain to players how the strength and weaknesses of their ability to to to take advantage of the information visions of the game environment and tubia has together with mobilitics created a visual attention metrics in the sort of companion apps that's been developed by mobilitics and that companion map gaming performance index contains many many aspects of other of playing league of legends now we are precisely right now we are introducing visual attention to an eye tracking to that sort of metric existing metrics but the key challenge is to transform

eye tracking data into meaningful and game specific metrics and last year we spent some serious time to create kind of understanding on that field so we collect case data from a vast amount of different players from different ranks and analyze that together with to to get some kind of a contextualizations of what's going on so it became meaningful and from that we obtain the fine metrics and those five metrics are first of all the mean and map awareness that's free because you look at the minimap and how how much time you spend on that minimum app the second one is tunnel vision television is a bad aspect of it because you're focused on a specific reading and you're sort of zoning out on something you neglect the other parts of the game so it's a metric that tells you how often you do that information processing indicates how

much rapidly you can play can process information which is kind of a the average is calculated because it is really associated with professional performance if you are information processing information quite rapidly and to those who is really aware about the theory he will say that well wait a minute the information processing is not compatible to the theory of hilarious psychology well i can't count to that i know i know i know the fourth aspect concerns fight preparation it is a measurement how aware you are with the surroundings before you enter a fight look at the minimap you look at us around your abilities and so on before entering a fight and pick awareness is that you are um when you're getting picked off it's usually because of you don't pay attention to the surroundings so uh we can measure how if that happens

and if you are paying attention to the to the game uh i would like to show you a example from video i'm going to hide my screen over here and that's this is an image of oh sorry video a player who played two twisted fate and he's plays mid lane and he's about to get his ultimate and before he gets an ultimate he scans and see that opportunity to in the bot lane and we will see in the video how often he checks the bottle and what's going on and so on and immediately he's got to get the ability for the ultimate he will go there so he's and also look at if they're in the right moment so let's see the video let's put out sound though and the red rectangle over there is actually the the measurement of the fixations of the eyes so it's actually where we can quite easily see when the

eyes are on the minimap and we should perhaps take that again because now it's over more or less so what's happening in the beginning here is that he is he or she actually is playing the game and is constantly looking at the bot lane over there making fixation makes a fixation and soon he will get his ultimate bam but i'm checking out there he goes and decides where to go bam so he constantly checks if it's the right and preparedness to do that before we actually enter the fight what are the benefits for this kind of system where first of all we have the ability to track how players gather information and speak for us with the

eye tracker and the storing of the gaze data and by that we can analyze that and provide feedback to the players and tell what how to improve their information gathering skills in through the metrics and then we lay the foundation for players to make use of what they see to improve their decision making if james gibson would be around he would say that well you know what players are supporting this quest to discover fornices he wouldn't use that kind of a thing of decision making he would say that players are supported in the quest to for instance which you cannot need however the metrics support discovery of their furnaces the metrics is not the affordances by themselves at the same time as the horizontal scan is not a fullness by itself it's a technique to discover their furnaces

so the metrics tell us what part of the screens you're looking at not on what items specifically and to understand to really understand what you're looking at you need to understand you need to experience and training to as i said earlier earlier on for instance if you have a situation on a minimap is that a threat or is that an opportunity that depends she depends on the skill level it can also depends what was happening of course to really unlock the uh potential uh for this one i i encourage people to uh to take on the journey to to to develop future companion app that will assist in perceiving act on more furnaces which is tailored on player skill level for instance scanning we can if we have an affordable system live in in the in the game app we can see we

can measure how often you were killed by an opponent that you didn't spot before and that is if that's live and that can be user measured if we can have we can introduce ideas that you you need we need to change your the ways you scan your or horizon will actually scan the game or we can make predictions of vortices uh encouragement for instance if things are building up to become a threat and to be able to detect if the players actually see those things and and or highlight that that's something's going on and so on to really nurture and train the the players to perceive those kind of performances when the situation is built up and the same goes for planning to avoid taking riskier actions for instance roaming around in league of legends uh when you are quite uncertain or unaware if that could be measured live that's good that could actually encourage you to change your behavior as having this instructor behind them

and saying that well what are you doing what are you doing before we realize that of course we need some kind of advanced systems we need machine learning computer visions or preferably we can integrate this kind of a trainer metrics or analytical metrics in the uh in the in the game itself sourcing creates that because then we can make make use of the full api to really make sense of what's going on in the contextual situation of the game to summer up some this presentation up a little bit i first introduced the gibson ideas from affordables which provides quite valuable insights how visual tension relates to successful game performances um the theory itself combines uh the exploration phase i mean the ability to see what opportunities are uh before environment and acting on it the performatory aspect of it in an interesting way

and player discovery and acting on upon those phones are are essential to improve in the game performances not only seeing what to do but also doing it and by tracking visual attention we can support this discovery for instance with eye tracking and the journey has just begun um i encourage all of you to pick a game and let's get started because there's so much work to do and by that i would like to say thanks for listening to this presentation i hope you learned something about that and uh hope you since nestor get this here bye