Building the Interface of The Elder Scrolls: Blades in Landscape and Portrait

[Music] good afternoon ux summit this is my first tdc talk so whether you think it was terrible or decent please fill out your forms we want your feedback also mute your phones please my name is marija smay i've been in ui in game ui design for 10 years now and last year i joined a team at bethesda game studios with an interesting mandate to create an elder scrolls experience on your phone the vision for the elder scrolls blades was to create intricate role-playing game systems with depth set in environments that would push the edge of visual quality on the small devices

while fully embracing portability and accessibility now it may not be apparent at a glance but this vision for blades contains a dilemma should the game be built in the portrait mode or in the landscape mode blades ended up being built for both and today we'll dig a little deeper into the vision that led us to choose dual orientation support then we'll look at the ui design process and how we made it happen and finally we'll analyze the player response we have so far considering we're still in early access close beta actually um so but it's still interesting to look at today so we're building an elder scrolls

game on mobile in essence mobile gaming liberates the context of play away from the living room or home pc and takes it anywhere on the couch at the park on the bus at the airport on the toilet or during a meeting or a gdc talk notice a few things here in some of these contexts people only have one free hat so if gaming is to be made even possible there portrait is a must in fact for some players the one hand situation is a little more persistent for others it might be even permanent

in other contexts the public eye is upon our player and in those situations portraits can act as some sort of a stealth mode where people may not necessarily appear to be playing a game our creative director todd howard affectionately calls portrait mode the meeting mode so looking at the context of play we must conclude that if mobility is a requirement then a portrait orientation is the only way to go however remember that for blades navigation in stunning environments is also a must external research done on the way we hold our devices reveals that while portrait holes are relatively diverse a lot of people hold

their device with both hands or just one hand depending on context but the landscape hold is almost universal people have both thumbs firmly placed on each side of the screen towards the bottom the physiognomy of people or the ergonomy of the landscape hold enables us to separate controls for camera and navigation in two so um so yeah the the alternative in portrait is the one finger tap to move but this inevitably binds camera to the eyes effectively reproducing the feeling of walking with a very stiff neck and perhaps more importantly navigation options for humans and khajiit and oregonians alike if you're familiar with the elder scrolls are usually lateral above is mostly ceiling and sky below is mostly ground

and floor not very interesting for navigation if you're a regular human not spiderman as a result when so in this example you see two doors right so you could go left you could go right you have choices you have agency but when viewing this same environment in portrait you can only see one door so effectively what happens is that your navigation options are hidden under the fold to borrow a term from web design so basically what it means is you need to scroll to be able to see your options and this combined with the stiff neck phenomenon creates feelings of claustrophobia when navigating a 3d environment in portrait hence we come to the ground-breaking revelation of this day at gdc landscapes are

better experienced in landscape so let's do a little experiment if you personally had to choose a device orientation to build the elder scrolls blades which one would you go for those who vote for the portrait mode with better accessibility and mobility raise your hands a few people and those who choose the more immersive mode oh wow that's interesting a lot of immersive people here actually this was an endless debate at bethesda so we took the opportunity here to turn lack of consensus into uh an opportunity for player agency so we let people flip their phones whenever they

please and yes we knew it would be expensive and painful and we had never done this before what are we waiting for let's get started to begin with we make design documents and menu flows and we make only one set of them so to preserve our sanity uh we go through the regular design process but trying to keep the information presented to the players uh the exact same as much as possible just in two different layouts so a bit like mirror layouts um so what this means is that the ux or design team had minimal overhead for dual orientation support but the ui team needs to produce each mock-up or wireframe twice on blades this added about 20 to 60 percent time to ui conception it doesn't uh double the time

because most visual communication problems have already been solved when you tackle the alternate mode but the extra time invested in conception did yield some early benefits to provide an example this is one of the earliest building menus we designed and here i list all the materials people need to craft certain buildings being as generous with information as i can so that players can make their own crafting decisions and paper tests were relatively successful so if i ask can you craft the smithy people would say um no because i'm missing a lumber and potatoes successful but then came the time to adapt the layout to portrait and while our list now

nicely held a large number of items and this was nice we wanted to keep it this way but there was no way all the information would ever fit the narrow space so it forced us to adapt right so we started disclosing information depending on context so here in the repair section of the smithy i show the item condition and in the crafting side section i show information about the materials required but notice here that the materials information has been distilled to a red x or green check mark and when we paper tested this again the distilled version in portrait the responses were noticeably more confident so can you craft a steel dagger sure but it's not worth it it has bad stats so it was way better so we realized that the compromises we had made

for portrait were actually way more efficient so we adopted them for landscape as well the takeaway here is that the extra constraints on the ui team allowed blades to identify certain issues early on and it saved us a lot of time down the line thank heavens it did because down the line is where time became excessively scarce the implementation was time consuming first dual orientation still means each of the two layouts need to adapt to the various aspect ratios notches and punch holes of the mobile market today so even if all our documents were produced in the 16x9 aspect ratio they needed to stretch vertically on ipads and uh

horizontally on larger newer phones so our integration notes got quite complex and our prefabs in unity got even more so but the real dual orientation party happened in quality assurance because remember that players not only own differently shaped and sized sizes of devices they are also free to rotate them whenever they damn well please so qa to test this needed to flip their phones in every possible game situation in the middle of a combat move during a loading during a server call during animations between a button press and a button release yeah bug festival ensued in this instance the ui disappeared entirely

we fixed it but fixing a dual in dual orientation is a tedious and delicate affair our systems were sometimes redundant it was the case of our animation system so keyframes are basically copy pasted that meant we needed to fix issues twice and most of the times they were instantiated so the same widget would be used in both orientations but that was delicate because it mean it means fixing a bug in one mode might break it in the other mode we did get the interface to work though and uh we got a first label demo in the hands of players in dual orientation as early as we could getting the game in human hands as early as possible was crucial to the dev process of blades

don't try to retrofit dual orientation do it from the start you will uncover flaws in your designs and you need to have enough time ahead of you to course correct to provide an example here we have a shield icon smack in the middle of the screen in landscape and it's a critical combat mechanic allowing players to protect themselves with their shield the icon was terribly positioned and my punishment as a designer for not having noticed earlier was to be forced to stand completely silent as i watched players struggle physically to even reach it and die one of the many ways i've killed people with ui we fixed it of course we moved the shield and we added a few player options

in a process so the left-handed could enjoy the game as much as anyone else but our first hands-on demo also opened our minds to an opportunity we had overlooked with the two different screen modes we could cater to different play styles so as a design experiment we began optimizing the landscape mode for immersion allocating a good portion of the screen real estate to 3d models the player's avatar etc and in the portrait mode we got rid of all that it became our gameplay first mode we even cluttered the portrait mode 3d model with extra non-essential information that was nonetheless useful this this is a worst case scenario by the way but the the takeaway here is that the diverging layouts allowed us to be a lot bolder with our

design decisions since we were catering to a bit narrower subset of our players for each mode while it's still early to draw any conclusions as to whether this grand dual orientation experiment was all worth it the results we have so far are interesting first if we take a look at uh the free form comment our close beta players are sending us through a generic feedback email so we give them a pop-up thank you for playing blades send your feedback here and they do and we witness something unprecedented in ui apart from bugs the majority of ui mentions are positive from my past experience what typically

typically happens is in freeform comment is that ui doesn't even get noticed or it gets noticed but that's when people tell the developer what's wrong with it and how to fix it what's interesting though is that landscape mode gets the usual treatment people don't mention it it's the portrait mode that gets all the love our players seem to perceive it as some sort of a bonus mode that we've generously built for them and landscape appears to be considered default or taken for granted dinging a little further into our letters of portrait mode appreciation we realized that the design assumptions we made about people with different play styles choosing a mode or the other to best suit them were partly wrong so we do have some players who

stick to one mode but quite a few of our play testers have gone to great length to describe their use of the portrait mode and it's all about the context of play so more like what i talked about earlier some of the associated keywords uh in the feedback associated to portrait are going and walking i know walking don't play video games while walking people and um telemetry data sort of confirms this for instance and take these with a grain of salt we're still enclosed beta but we clearly see though that the session length on average is much shorter in portrait so people play less long and overall it's slightly shorter in the day time when people are going about their daily business versus longer sessions in landscape

mostly past 8 pm local time another curious finding from digging into a device orientation telemetry data is the gradual adoption slow but gradual adoption of the landscape mode by the same players over time so it's as if our players kind of naturally realize that the experience is a bit better in landscape and they kind of adopt it however the adoption is not a global people play the quests dungeons more in landscape over time but in menus it remains around 65 percent portrait 45 to landscape kind of consistently over time so globally the trend we're observing is that many of our players will tend to go about their daily business

in in their town in short daytime sessions in portraits so they'll go get their gear repaired temper a sword brew a few potions and then they'll come back later to the game in the evening for longer sessions to explore dungeons fight dragons or explore the abyss outside of town so in a way dual screen orientation support has positioned the elder scrolls blades in the best of both worlds it remains portable and accessible throughout the day while making no concessions on the stunning rpg experience it provides the grand take away here is that if well planned and i need to insist on this if well planned and you've done from the beginning both technically and production wise the

return on investment of portrait mode support on a 3d mobile experience is well worth the extra costs the the extra design constraints actually help improve the experience and our data shows our players appreciate it and they use it when it suits them if you'd like to try it for yourselves you can download the game it will be available this spring to all it's free give it a run and send us your feedback this is a feedback address if you want to follow me i rant about ui and if you witness your colleagues playing blades in meetings share your stories i want to hear them thank you

we do have a bit of time for questions okay um about uh how many i guess iterations did you go through for the ui when you started to realize hey we can kind of put these things together like we can use check marks instead of red numbers etc um so it depends on which parts of the ui some uh were pretty good from the get-go others were very much iterated on the the game actually it's interesting because it's a smaller team than i'm used to i come from triple a and it changed radically throughout the dev process i'll share

this one of the things we discovered in our first play tests last year was that people were obsessed with cheese and we were like oh my god cheese is one of the elder scrolls core pillars and people couldn't interact with cheese they were very really sad so we added cheese and now you can eat cheese in the other scrolls blade but yeah some parts so got actually really really changed radically it depends on the the ui crafting is one of the ones that changed a lot okay thank you wonderful talk i have a bit of a question sort of in the theme of one of our earlier talks in projects like this one of the hardest things to do is to get buy-in so when you're starting a project like this generally speaking you need some data to say we can invest all of this time because this is how our players are going to feel how did you tackle convincing

your team and your company to be able to do this in invest in the time to be able to do in portrait and landscape that's an excellent question and yes it's true you get you need buy-in we were lucky because it was the other way around on our project it's the creative director who believed in dual orientation support for planes and he needed to convince us because the doers were like but it's gonna cost this much money and and he convinced us amazing but but that's easier than the other way around yes um hi so when you have your phone normally it's in portrait like that's how all the apps are displayed yes um when you boot up the app and you have the splash screen that usually sets the precedent for whether they want you to turn your phone or not what how did you make the decision to make whatever the default whether it was portrait or landscape so the default is whatever the player is is booting in and one of the things we're considering now

is maybe tutorializing the fact that you can even flip your phone we we have some uh hardcore portrait users you saw like the about 65 percent of people are in menus and portrait and we think maybe it's because some people don't even know they can flip their phone because it's not very common yet in north america so we we're considering it thank you hi yes um i have a question for smaller companies with smaller budget for instance is it an option to play the game in landscape and the menu in portrait is it something that you already you have yes it could be an option or did you evaluate this approach on is it a valid approach that's interesting but it would force users to flip their phones at certain times is what they do i'm thinking but i don't

think i've ever seen this approach oh me neither but uh it's great to do both but let's say that is it something that you analyze doing play test or ask no we wanted to give full control to the players yeah and don't you think that it could be an option or not at all i i don't know i think it might be frustrating to be forced to to rotate your phone when you don't necessarily want to but that's that's just a hunch because i don't have an example of a case where it's been done no i mean either thank you uh just a quick question because you talk about the all rotation scenario with your qa team and i'm just curious what was the process with that or with them because i mean it's can well i i think they might have lost their minds at some level because there was so many scenarios and the follow-up question on that is

how much data did you get from the qa on the ui ux perspective uh great question yes they needed a hug at some point but yeah for qa it's uh yeah i said so it's the they carry basically the heaviest part of the burden because they needed to test every possible case for localization it was twice so every every language needed to be tested twice not all the the possible aspect ratios because you just need to test your smallest possible aspect ratio and that's everything twice uh but for yeah testing flipping their phones and they kind of liked it though because they're sadistic like i flipped my phone in the middle of this yeah button press and release was like why would you do that because qa okay um so yeah the the buy-in

again to come back to the buyer needs to be global right the qa needs to be aware we're doing this we're in this together and yes they need excellent support yes hi my name is david do you have any data on whether it was like harder to play in portrait you were talking about how you'd sort of have a tunnel vision effect you probably still get through the levels does it take longer to get through or more tries do you know this uh that's a very good question i it doesn't take longer to get through with the same uh level but uh yeah it's mostly a question of uh how you feel through it and we we know also it's actually really hard to get any data on this because people are are in different contexts they don't have time they're kind of busy and portrait usually they they want to go

faster i don't think you would notice it's harder if you're playing the the landscape people will explore a bit more go to the side path discover the hidden treasure so it's it's very different ways of playing the same game actually unfortunately we have not a lot of data because we're still in closed beta but that's what we observe is people are a bit more in a hurry and you saw the session length is much shorter in portrait so it's just a different way to experience the same game thank you hi i had a question uh you mentioned at the end you validated that there was a good return on investment in doing both but i guess what were you looking in terms of data to kind of validate that thank you so was it a good return on investment what we are looking right now to say that is mostly the reception of the players and also the media and it's interesting i didn't put it on

my slides but we even have headlines uh in our reviews that are like wow like elder scrolls blades is getting one thing right the portrait mode and we're like we're a headline like ui is not used to be so to get so much limelight but that's also all the good press the people are grateful for the existence of the portrait mode we think we think that's really well worth it it's hard to calculate isn't it because we also don't know exactly how much it cost costed us it's hard to evaluate because we did it from the get go in both orientations but yeah we believe it was worth it thank you thank you that's all the time we have thank you very much you