Immersing a Creative World into a Usable UI

[Music] hey everyone good afternoon I'm going to talk to you about my specific process of how I go about immersing a creative world into a game user interface my name is Jeff Chow I am a UX UI consultant for games and I'm really excited to be here my first GC was back in 2014 and it was such a great experience to attend these sessions and the fact that I'm standing here giving my own talk is truly unbelievable so I really hope you folks get something good out of it today to give a short background on myself before entering the game the game industry I worked in the field of graphic design primarily with branding consumer products and in it for entertainment I then joined the game industry back in 2012 with dizzy games to help out with art direction and UI UX and even after my time at Disney I was able to work on other big brands with my consulting work for both new and live games and so I've

had the opportunity to work with really terrific brands that revolve around unique worlds and for today I have a couple goals I hope I can achieve with you I hope that at the end of this talk you will have a knowledge of the impact UI can have on your games overall branding you will have learned a method of how to go about producing creative UI elements that complements your games world and last but not least an understanding of the balancing act required between your narrative UI and usability so with my history embedded and working with art direction in UI UX for games I learned a lot of what it means to work on games backed by great IP and as I worked on these games I agree to realize that one of the reasons why players play these games is not just because the game is fun itself but it's because of the worlds these IPS stem from that fans grow to love for years and sometimes even their entire life and when I say worlds I mean the fictitious universe created from mediums like film

games and have their own individual culture visual rules and aesthetics these worlds are the things people fall in love with because of the unique perspective they present that of our world and when it comes to games and when it comes to games the entire screen is an opportunity to help immerse a player in that world so within a world there are so many different visuals that one can take reference from when developing their game and their user interface its with the details that go beyond character art and environments like typography iconography color palettes the help keep a player immersed and engaged in a game and in my years of working on interface really would have aimed for with art direction for UI UX is a branded experience creative worlds presented in games can make for a game's holistic brand vision and journey and that's what you can achieve with in a games world and experience a truly creative and exciting experience that through visuals but also through

functionality while we often talk about user experience through via functionality this player experience along with world immersion becomes defining of a great brand experience take for example BioShock Infinite game by Irrational Games yes the game is a first-person shooter set in the early 1900s but while you're not shooting up enemies are engaging in psychokinetic powers you're still able to be immersed within the world while navigating crucial UI that enables the game to be played by the player in the first place and that brings me to in point Porton point just as important for a game to be visually immersive games as interactive experiences need to be usable and easy to use we want as little friction to happen so the player can enjoy the game's world because it plays into the bigger picture of defining a great brand experience a pretty game is nothing with a bad user experience so it's imperative that you focus on creating a UI that

simultaneously usable and immersive even with the dead space series by EA the team successfully created a diegetic user interface the had yourself and the main character immersed in the same world and seeing the same UI there is still though a simultaneous focus however on how everything is laid out and design then enables the player to frictionlessly navigate the game's content so when we talk about immersive elements with your game which equals out to the entire user experience UI has the same capability to immerse a player into a world just like characters visual effects and environments UI is an opportunity to reflect your games world and strengthen your games visual branding through narrative graphics and of course usability so in a compact amount of time I will discuss how to embed a game's world in its UI via these three phases research exploration and iteration so step one let's create the

foundation powerup and research your world what is the world you are presenting to the player what is the world influenced by culturally what visuals patterns textures can you take on with your UI this is the time to seek out the visual elements that make your world distinct and it's quite possible that the same references your games world is inspired by can be the same references that your UI can be inspired by but there's so many different places your UI can get inspiration from you I can be inspired by history most notably how the Bethesda's fallout series took inspiration from American 1950s post-war signage and technology interfaces the Fallout series has a remarkably branded image through its UI typography color textures all of it plays a part of keeping your your your universe immersed into the game the UI evokes a powerful brand well the player in acts with contact in a way that allows them a fun experience UI can even be inspired by

nature as seen here in King's paradise Bay you can see how the buttons color palette and treatment is reminiscent of oceans and natural rocks UI can even be inspired by say a subculture where it's very clear how games like Nintendo splatoon - gets an inspiration from Punk and urban references while this universe is out of this world their UI draws actually from very familiar fun color palettes that you will see often in graffiti and other fashionable Punk patterns so big-picture UI doesn't have to just be a bright juicy green button are plenty of ways to dress it up so in a natural way that fits with your games narrative if you desire it now all those games that I showed were already live and for example of myself with this process when I was working on more on his island life I had spent hours researching myself in the Oceania culture which is what the movie itself is based off of and when you're

researching you start to not only find keywords that define your world but you start seeing inspiration for your UI you start to see how icons can be designed how silhouettes can be used or what even kind of color palettes or typography you can dot - these three key words and that is going to allow you to start researching handful of fonts that can complement your world start developing a UI color palette that guides the user naturally while still maintaining a branded experience or even provide inspiration of what kind of pop-up container shapes you can house your UI content in so after researching your world picking up visual cues and palettes you can then take this all into exploration for a couple of sample screens and by doing so you can see what portions of your research is going to work and maybe not work so well my suggestion would be to apply this this exploration with something like perhaps a main game HUD a

character screen so to recap within research that acts as your foundation for your UI immersion I would recommend looking into reference visuals and their details that inspire your games world if you can do that beyond Google by say watching movies museum music museum visits that are similar to the world that has inspired your game that makes for even better reference and finally once you have all this research you can then start taking it to the exploration phase so then when we go into the exploration phase this is our chance to build out our UI on perhaps a couple of key screens and we want to answer the question of how do you immerse the world into your UI and it's exciting because all you have to do is just explore I think this is a great opportunity to be able to explore all different kind of spec terms of visuals when you are setting out for your UI direction to me it's almost looks like let's say baking a cake there's so many different

ingredients to play with but not only not only influences the way that your cake tastes but also how it looks there's an endless amount of variations that you can perceive with but this is a time with the range of UI elements for your game but you can think about how your player will enjoy the most but what's important is that you explore the spectrum offered by your ingredients these UI ingredients can allow for immersion but within each one there are different spectrums you can experiment with during your exploration for example dietary Qi or non-diegetic Qi which one will work better for your game and as a quick recap diegetic UI is interface that is included in the game's world for example it can be seen and heard by the game characters non-diegetic however is interface that is rendered outside the game world only visible and audible to the players in the real world so with diegetic you are able to gain a fully immersive narrative visually it's

great when the narrative is very natural like let's say looking at a map the same way that Campo Santos Firewatch does or even in dead space because you and your character are in a in a spaceship ultimately you're looking at the same heads-up display but there's also risk to diegetic UI if you have a lot of information just show there is a chance that crucial information can get buried due to the narrative UI compost tax complex tasks in the game may have a friction a lot of friction due to a heavy cognitive load required by these tasks and when you commit to a diegetic look you're committing to preserving the fourth wall which could prove later difficult pending on your players tasks in the game and non-diegetic shouldn't be considered a loss of narrative as it's there to help help guide the player through compost complex tasks and strategies and as you see with even games such as riots league of legends or atlases persona series the to keep to

non-diegetic because they have complex tasks and content that should be clearly separated so the player can easily play the game but of course the risk of non-diegetic is pending on how it is laid and is laid out and designed because the way if it's badly laid out you can risk further complicating your complex task and if it's not designed correctly then it will distract from the world being presented in addition to just breaking the fourth wall so how do you explore the decision to have a UI that is more diabetic or non-diegetic well it depends both can be narrative this should be more of a decision based on your games mediums and mechanics for simple mechanics or console games you may not need a non-diegetic UI and can project through characters or other diegetic methods but perhaps your game is significantly complex and with strategy that creating the UI layer in between the player will

make for a frictionless experience between a player and a game's world so pending on how you pursue via diegetic or non-diegetic UI there are still several other ingredients you can play with that immerse your world into the UI skeuomorphic UI creates a sense of familiarity by emulating materials while flat design stays true to its medium often feeling minimal and utilitarian but Blizzard but for blizzards hearthstone the developer commits to ASCII morphic UI that enforces their narrative for a player literally playing with cards in this tavern that's in the game the game is a card game yep through its UI elements we are kept in a world that preserves such a narrative that feels tangible to the user and while flat design stays true to its minimal style it can help focus the player on the what the bigger focus of the world is in the game for example Nintendo's Pokemon go slowly wants to

focus on the pokemon so they aim to keep their UI rather minimal and flat in order to focus consistently on the pokemon and then you have things like aura games like Nintendo's Zelda breath of the wild where the minimal flat design allows the player to focus on Linc to focus on his inventory which plays a huge part of the aventuras world within the game and while these few examples are extremities you often see many games will adopt almost a mixture but they do so in a way that tries to complement the world in a consistent manner and to not look fractured within their UI so when going through your exploration even testing out designs through a similar visual scale can help you create a wide variety of choices well I tend to do the specific exercise just with one layout even different layouts are worth exploring as well when looking at the visuals scale of each design layout as

well has the ability to promote easy usability and world immersion within nintendo's Fire Emblem heroes the way they lay out their gotcha feature plays in line with their narrative of using magical stones to summon heroes through a mystical gateway the layout of the orbs is not only easy to interact it with by simply tapping but they're also able to clearly explain the benefits of each Orban what kind of hero you can gain while summoning the stones hero another fun example is with dragon ball z dokkan battle when you're about to go into battle not only is the screen laid out in this exciting action-packed among that inspired layout but it also clearly communicates who you are fighting against or even the way that they decide to display their characters that even though it's just about communicating basic stats they do so in mimicking a traditional trading card layout style which is very familiar to the brand's long history of selling trading cards

and then if we even go beyond just static visuals but animation animation of your UI can still keep the player immersed within the world here in Quantic Dream's heavy rain the developers clearly develop clearly develop an animation UI that helps communicate timing urgency of specific commands that were pinned to their related actions such as punching and dodging and even with EAS Mirror's Edge the team was able to use animation with their UI to not only direct the in the right direction but simultaneously promote the game spirit of being a fast-paced action game so in short there is so much you can explore in these phases with just using these four UI elements when you're trying to figure out how to immerse a game's world into the UI just know that variety is king in this phase and exploring different varieties of each ingredient will provide for a wholesome building process but then once you build out

these UI options how do you pick in the iteration phase you get to decide the balance between narrative UI and its impact on the overall user experience you've done a bunch of exploration of what your UI can be but in the end of the day we need to remember that UI needs to enable interaction with the game's world not distract from it sometimes this decision can be very clear but I have found that focusing on these four main elements often helped me edit revise and decide a solid direction when it comes to critical UI visuals for a game readability am i sacrificing readability for style am i looking at the game's info or the info styling consider the risk you could be over designing your UI perhaps due to color or more visual iteration in the decoration you could be distracting the player from the content you're presenting to them personality

can I see my worlds key words when I look at the UI even without the visual support of characters and environments what's important to see is in those games brands or keywords are visually implied in the designs so visuals have to bill you stand on their own but still reflect the game's brand implication our interactive elements distinct from non interactive elements how much can i edit down to a more essential core look your UI doesn't have to be literal to your references perhaps color palettes and light iconography is enough to communicate a brand in a world also I need to make sure that the player can tell the difference between interactive objects and interactive objects and finally scale does the visual UI link UI language have legs to expand to other UI elements can I scale without breaking the memory bank within the game can I possibly scale

this distinct look to even perhaps marketing or community assets you may find that perhaps some of your ideas while looking really cool have a really heavy memory load or can't even scale let's say from a 9 slicing perspective so when you look and start to iterate your designs you just need to be judicious about these 4 categories you need a balance between narrative impact and player usability remember during our process how good UI should enhance and promote a frictionless experience for a player and the world they are interacting with UI is just one of the many visual elements that can be included in your game to enhance the game's world and once you have chosen a UI direction likely you need to keep iterating on it too and when that happens it's always good to keep some key principles in mind to make sure your UI's narrative isn't jeopardizing usability as I am creating UI for an entire entirely new game I

will always refer to these 10 usability heuristics provided by the Nielsen Norman group to help me revise my immersive UI and even though a lot of the examples I had were very much console or mobile based I just wanted to give some thoughts on as well of how I can see UI immersion continuing on with some new mediums within games so for AR and UI immersion I think the there's going to be a very interesting balancing act since ARS focus with games will be based on the camera in real world interactions I feel like the user interface will be kept to a minimal to balance out with the camera visuals especially if the intent is to blend a player's environment with the with game visuals such as characters a focus on that experience would require minimal interface on your screen and when it comes to VR I think that there's a huge opportunity for Dai Jetta Qi with Villar the additional asks access to

haptic audio and animation features that provides good player feedback mechanisms can also be considered for world immersion opportunities and you'll also likely need to still have some kind of visual interface like type or perhaps even buttons for accessibility but there's going to be some really exciting stuff that goes on I think between VR and immersion for UI so to conclude its really crucial during the entire process that you keep a very open mind of how you can inject story into your UI there are visual ways both diegetic and non-diegetic to do so and by doing so you're going to keep a player immersed in a games world which will be beneficial to your overall brand it is just your job as a UI UX designer to make sure that this UI does not distract from the world and you can do that using heuristic principles and this is just even more important because we live in a time in a market where not invest

investing in your you eyes ability to add on to your brand it to pass up on that opportunity is a mistake there are so many user interfaces that are generic and can be mistaken for any other let's say sci-fi game or even any other farm building game your UI is an opportunity to be as unique as your games worlds so take advantage of it so if you only take a smidgen away from today's presentation here's a few things to to remember to acquire a well-designed immersive UI select your process into research exploration and iteration and then during your exploration make sure to test a design out with a variety of designs a variety of layouts think about how animation can play a part in your UI and then finally when you want to decide that balance between your UIs immersion and usability you can use these four

elements to iterate and improve your UI so that's it for me thank you everyone for listening and hopefully I can take some questions [Applause] first of all thank you very much for delivering this incredible speech now with regard to diuretic and non-diegetic aui I know that there are games that do it fully on one end or the other and then there are games like the Metro series I kind of have a balance between that in terms of stealth and objectives for dodge for in-game objects and things like ammo cans for instance being non-diegetic do you think that there's a risk of a conflict between a merc kind

of a conflict between immersion readability and all those aspects that you mentioned when you try to strike a middle ground so your question is is there a risk in going for a more diegetic UI and risking usability for immersion sake basically if immersion is going to be impacted somewhat if you try to go in the middle if you try to incorporate components from both ends I would say if you try and strike a balance that's in between those two you have to do so in a consistent manner and pending on the user task whether it's something that's really complex the more complex a task is diegetic UI may not be died rock you eye is useful when it feels very first nature to the player and so if it's a very complicated task that a user doesn't say experience in the real world and will be tough but you should always explore if you can thank you kindly

hi Steph I'm Kevin I'm a designer at Google Maps and I love what I do but I'm really torn when it comes to in game navigation UI and one of the things I've always found conflicting was reaching your destination or seeing in an abstract simplistic 2d view of your goals or enemies on the map and sometimes that would break immersion and sometimes I would actually play games very differently turning off the UI entirely games like Far Cry Assassin's Creed syndicate my first experience turning off UI was the radar in Metal Gear Solid 2 and it completely blew my mind as curious as to if you have any insights the balance between reaching your goal and objectives versus immersion in nav UI right so in regards to you said Google Maps yeah so for Google Maps I would

just turn to the user for your your answer why is your user using Google Maps it's because they're trying to get somewhere they're not necessarily looking for an immersive experience like let's say with games and so I think if being able to optimize for that process of for them the experience will be much better if it's more clear to them just like I just want to get from point A to point B but then it's always thinking about I guess you would say the users intent of what why they're doing that task okay cool thank you very much hi so you mentioned a few times about animated UI elements I was just wondering if you had any specific methods or tools for kind of communicating the intended animation to developers or clients when you're kind of exploring right yeah I mean it really depends on your resources at that point sometimes pending on your games team you have people who are naturally animators sometimes you have only resources let's say like PowerPoint right and I say that

from it really depends on your comfort level sometimes even for myself I will just use like a quick keynote pending on the animation just to demonstrate ok this UI element should move over here once I this acquire this goal so I would say it really depends on your resources um yeah thanks do you recommend to any practices or pieces of work that new a UX designer should try and add to their portfolio if they're just trying to get into the industry right um I would say that's an interesting question and I would say from that perspective it I think the best thing you can do as a UI designer starting out is just to create create your own projects and to do so in a way that I think what's always a lovely demonstration is when you show something

like a game HUD and a character profile screen because that demonstrates your skill of layout your skill of colors and typography and you want to be able to tell them that you can balance in between screens that have a lot of information that needs to be digested versus very minimal information thank you so does localization ever play a factor I know in my personal experience I've artists paid great fonts that fit the world and they don't have any European characters so we can't localize and maybe on the flip side you need to do something like an Ikea manual that there's no text at all but that might fit the world it's a that's a really good question so in terms of localization I think from an exploration perspective you should be when you're when you're creating these UI screens it should be very inspirational and so from that perspective you should try and find fonts that that feel true to your brand but then and then afterwards you can

take that as kind of your your goal and then look at fonts that are that are able to be like localized for the your game and so I would say for your research and exploration part you just look at what you feel like it's the best font and then after that you go into localization fonts and see and then work with your your your font vendor to see which font will match closest to it do you have any like ideas or guidelines or recommendations for say user interfaces and user experience and games or immersive environments where there isn't necessarily a fourth-wall where instead of playing a character you are more or less the character in something that is like either set the augmented reality or an alternate reality that is still may call back to the current reality like a lot of way like to do is in those situation environments what there's a little bit of a blur between game reality and reality so UX and you either for me gets

a little confusing because I'm not quite sure what to do because there's not really a fourth-wall and the only recommended to you is entirely separate and also the same as the character as you are the character right um so in that respect I would say that it depends on your game's nature right if it's a very story driven game then I think that you want to not necessary like UI is important but then you want to make sure it's not distracting as well from the main experience so Express especially if you're doing something like mixed reality augmented reality I would say that you want if anything you want user interface at that point to be very minimal in terms of resources I would say just being able to really being able to I think like research about just like any other games or have been working on is just seeing their findings as well would be a good idea because with

especially with AR like there's going to be new usability issues that you're going to be finding yourself so it's a process I think thank you and I think actually we're out of we're out of time right now and so I'll be just on the side if you have more questions and thank you so much for listening you