The Design of HoloVista: Crafting Mechanics for Immersive Storytelling

[Music] thank you hi everyone good morning uh thank you all for being here my name is Scott John Siegel I was lead designer on Holo Vista and I want to tell you the story of how it was designed and developed this is a story about an unconventional design process and what it took for us to find a unique structure for expressing a beautiful narrative through gameplay it's also 76 slides in 30 minutes so please buckle up silence your cell phones and let's get started uh I've been designing games professionally for the last 14 years almost exclusively in the Casual free-to-play space and after years of full-time employment and a brief attempt to depart the industry entirely I started freelancing and Consulting in 2016. and then in 2017 I was introduced to Star and Nadia of akonite Nadia Lev is a professional photographer writer and artist SARS Saint Germain is an illustrator musician poet and graphic

designer and together they founded the company aconite in order to expand their combined creative skills into a more interactive medium I still remember our first meeting in a it was a sunny December in a fancy San Francisco coffee shop I ordered a chai they got matching matcha lattes and we sat and discussed their company and their aspirations star and Nadia wanted my help designing a game I asked what kind and they said they didn't know what they did know was what they wanted the game to achieve akonite was founded on the idea of crafting immersive experiences that would blur the lines between fiction and reality and they wanted their first game to tell a compelling story about technology and culture to them the game's genre didn't matter at least not yet that game would eventually become Hollow Vista but what holla Vista was changed throughout our creative process and finding the shape of hollow Vista required us to be flexible on what kind of game it was in order to achieve akonites goals and this genre fluid approached this mechanical agnosticism is what allowed us to find the structure we ultimately landed on and I want to walk through the steps we took to get there and the most important things that

we learned that game design is more than just mechanics and for story focused games a thematic's first approach lets you consider mechanics that will help tell the type of story you want to tell rather than trying to cram your type of story into a predefined structure but thematics first doesn't mean story first and sometimes finding the best narrative structure means not tying it to any one particular plot and instead crafting a structure that can tell the type of story you want to tell and only then crafting a narrative that works best within that structure and that narrative can be best served through gameplay when contrivance is embraced that allowing for game-like structure doesn't come at the cost of immersion and can actually provide better scaffolding for expression than an adherence to realism and lastly that the best possible game you can make is one you can actually ship we compromised heavily on the ambition of Holo Vista but those compromises led us to the thing we made which is exactly what it needed to be and if there's a fifth learning to be had from this talk is this document your process our ability to reflect back on holiday vista's development owes to us

embracing documentation from a very early stage it's how we were able to learn from every choice good and bad it's how I was able to write this talk um before diving in I do want to take a moment to acknowledge that what Holo Vista is everything that it accomplishes owes to a great deal more than just its mechanical design and though this talk doesn't cover these other aspects it would feel unusual and wrong to not give credit for the game's holistic shape to its other critical components the incredible story crafted by our narrative designers strix Beltran and Corey O'Brien with additional writing by Sarah Carr The Surreal vaporwave aesthetic brought to life by our environmental artist Blake Catherine and Andrew Morgan our haunting soundtrack by electronic artist mariod atmospheric sound design by John lydiker and our accessibility efforts spearheaded and evangelized by producer Meredith Hall to say nothing of our code crafted by Jay treat and Dexter Lonas tireless QA by Cindy ijima and additional literally everything by associate producer Andy hegates and of course the work of star and Nadia who not only co-directed the

game but also contributed art animation code design and music alongside the team this was a small but formidable group of incredibly talented people and The Game Wouldn't exist as it does today without every single one of them okay so what is hola Vista hello Vista is a little hard to describe sometimes maybe it's simplest to call it a immersive hidden object and storytelling game you could also call it a social media simulation or a visual novel it's also a VR game but without VR goggles in augmented reality game without the reality um halavista is styled as a fictional instagram-like social media app you play as Carmen Razzo an aspiring architect living in New York in the nearest Future and you experience the story entirely through the literal lens of her social media when you open the camera in Carmen's app you are viewing a virtual 3D environment a place that Carmen has been and you can move your device to look around as though you were looking through your phone's own camera and of course you take photographs which you post to Carmen's feed to progress the story and through these mechanisms

holovista expresses a beautiful haunting narrative about memory identity and authenticity no spoilers in this talk but if you want to play it yourself and you should it only takes around three hours to complete that was the game that we shipped in 2020 but in 2017 none of that was known all that we knew was the highest of high level so star Noddy and I would meet regularly at an artist Co-op space in downtown San Francisco and our earliest discussions focused around star and Nadia's interests both in modes of Storytelling and topics prevalent in technology and culture news of the moment and this is what I mean by thematics first they shared short stories that evoked the worlds they wanted to play in we discussed common topics of Interest ranging from Ai and computer vision to divination and Magic mechanical Turks and Yahoo answers but the one thing we weren't discussing at this stage were specific game Concepts we weren't in a rush to lock down a genre or core mechanics not before we knew what our thematic Focus was going to be and this has since become kind of a core thesis of mine the idea that

holistically game design is a function of all aspects of a game's development we may think of design as being just about the game's mechanics but game development is a multi-faceted discipline with each facet informing and being informed by every other whether they are the game's thematics it's themes setting narrative goals and story its Aesthetics the art style look and feel Muse music sound design it's technology the code engine infrastructure and even its business the underlying business model as well as its production schedule resourcing and the expectations of investors or other stakeholders there are always interlocking dependencies and for any type of design problem at any stage of development it can be useful to consider the design through all these facets not just Mechanics for us our Focus was on thematics first and foremost which led us to start establishing establishing core pillars of our thematics what we wanted to come across through whatever narrative we expressed and whatever mechanics we used to express it we wanted the player to have agency both in self-expression and in their impact on the game's systems we wanted the experience to be immersive to

blur fiction and reality and we wanted there to be a community aspect emphasizing empathy and connection to others and at this stage we were seeking both thematic and aesthetic inspiration looking at interesting storytelling structures particular themes of Interest as well as art styles imagery Vibes that we wanted to evoke one Topic in particular was Cambridge analytica the political consulting firm that would become embroiled in controversy in 2018 and as we discussed its implications more and more the micro targeting predictive algorithms Big Data fueled by personal information we realized that we had a lot to say so we decided to make social media our focus and if we wanted to blur fiction to comment on reality then we would speak to social media through its own lens and this is when the mechanical conversation started in Earnest social media was a thematic rapper what kind of game would live within it well fortunately social media is already heavily systematized and suggestive of a lot of mechanics so we saw at our disposal a wide range of levers followers as indicators of both success and audience the social graph as a

mechanism for the visibility of content whether public or private but dopamine hits of likes and reactions comments as a mode of discourse around posted content and actions that could have both local and Global impact lots to work with where do we start our early Concepts were heavily systematized one used a rain style mechanic where liking commenting and reblogging would cause social status changes that could lead to fail States another had a friendly AI who would help you go viral by suggesting the types of images and captions that could attract different audiences these Explorations were interesting but not quite compelling enough around this time Nadia was using proto.io to build our various prototypes and discovered that they had added an interactive walkthrough feature where you could use your phone's gyroscope to view a 3D space and similar Tech was being used for real estate listings but Nadia being a photographer took the feature and added a shutter button and suddenly you're not just looking around a space suddenly you're looking through a camera and the possibilities for this were immediately apparent it tied to our

themes of social media the VR feeling was immersive and reality blurring players had agency through self-expression and there were plenty of mechanical implications we evolved the Prototype with the ability to detect what you took photos of kind of a Zillow meets Pokemon Snap and with that we had a new mechanic that tied to arithmetics and could give the player agency in our story great so what are we going to do with it well we could really do anything the player could visit different locations and take photos and we could evaluate the photos based on their content and we could have the story change depending on what they took photos of so we envisioned this as a loop of taking photos picking captions reading the feed chatting with characters and then traveling to new locations to take photos and so on our first gdd described a system where player choices could influence the game's world and story Direction depending on what the things the player photographed and the things they said in chat The Narrative would unfold in one way or another and because this wasn't already complex enough we then expanded the concept further to have more types of player choice influence parts of the system and it's

easy now to look back and realize that this was ambitious especially for a new team and especially given that as lead designer I never actually worked on a branching narrative game before um so it was for the best that we decided to build out a full vertical of this prototype to demo and play test at the internet Art Festival XOXO in September of 2018 it imposed a deadline on her work which was incredibly useful given the scope of the concept this version called filter framed the player character as an up-and-coming influencer getting paid to travel to new locations and take photos of places and events the player would open the camera into a location they'd be able to look around and take a photo which they would apply a filter to and then pick one of two captions which were related to what was in the photo the photo would then get posted to the feed which would also populate with posts from other characters from around the same time period the player can read those posts to get a sense of the world and the larger story at the same time characters would reach out over chat and the player had choices as to how to respond and depending on the choices the characters might suggest the player visit one location or another so the player would

use the location feature built into the app to visit those new locations where they would take another photo apply another filter add a caption and post it resulting in more feed and another character messaging them etc etc that's the loop this was the narrative map for just our prototype build with five locations to visit and all this variance in Choice structure amounting to about 15 minutes of gameplay so how'd it go uh well players loved the look and feel with massive credit to star's graphic design and the moment of opening the camera into a virtual space was a repeated highlight but they didn't know what they should be photographing they didn't understand the significance of their actions and most players weren't bothering to read the content in the social media feed so they either didn't follow the story or if they did it didn't care enough to get invested in part because the idea of playing as a social media influencer was also unappealing and more testing after XOXO reinforced the feedback we received what we had wasn't resonating how could we make it work uh well players were having a hard time following the story and knowing what to do next so we added a journaling

Quest system to help guide them and the you're an influencer story itself wasn't engaging so we started experimenting with different narrative ideas our only constant in this exploration was the structure we'd already built how could we create something more compelling around the idea of photo taking and posting so every team member went off on pitching their own version of the game maybe you were posting photos you took in your dreams or you were an amnesiac revisiting places you had been and taking photos to recover your memory or you were a detective using social media to solve a murder case we kept trying to make the structure work by tweaking The Narrative we wanted to know what the story of the game was and if we could crack the story we could figure out the right Mechanics for it so why were you taking photos what were the places you were visiting why is there a feed what is it that I'm trying to do how do I win how do I lose can I lose and most critically why would player want to do any of this what was their emotional investment we struggled to find answers to these questions that we found satisfying and we struggled to find a shape of the game that worked and we were running out of ideas and we were running out of time my first meeting

with star and Nadia had been at the end of 2017 and by the end of 2018 we still had no real certainty about what the game was going to be after 12 whole months on a direction that wasn't working the project was in a bit of an existential spiral after exploring so many ideas trying so many things there was a real fear that we might never figure it out and the game might not ever ship so we entered 2019 in kind of a dark place and we did what any reasonable humans would do in the face of overwhelming despair we watched YouTube Star had been playing Return of the oberdin and wanted us all to watch the episode of Mark Brown's game Maker's toolkit on how the game works if you haven't played it oberden is an adventure game set in the 1800s where you have to uncover the fate of 60 people on board a merchant ship that had mysteriously vanished five years prior you do so by using a mystical pocket watch to rewind time to the moment of each character's death and you use their dialogue the environment and deductive reasoning skills to identify each person in the ship's manifest and how they died or disappeared it's an incredible game and I credit it and game Maker's toolkit

with saving our project so thank you Lucas Pope and thank you Mark Brown in the video Mark Brown was interested in how oberden succeeded as a detective game despite not having traditional detective e mechanics and he called particular attention to the mechanics of the pocket watch Whose Behavior was never fully explained and the player's Journal that only confirms correct answers in sets of three and he likened these to similar mechanical conceits and limitations in sambarlo's her story and he refers to these mechanics as contrived but he doesn't mean it in a bad way as he puts it the worlds of oberden in her story are not trying to be realistic instead they have stories gameplay systems and means of interactivity that exist purely for the purposes of allowing for deductive reasoning without taking power from the player's hands and this for me was a Galaxy brain moment because the greatest impediment we were perpetually up against was narrative justification for mechanics why would the player be doing this how do you explain this in the world of the story does this interaction make sense in the context of a social media app this had led us to prioritizing various similitude or the appearance of being

true or real because for us we believe that immersion depended on the mechanics of the game being believable in the context of the game's World the game is styled as the app itself and if the app feels too much like a game or behaves in game-like ways we believed it would break immersion and this also meant that all of our UI had to be diegetic or embedded in the fiction rather than breaking the fourth wall so everything the player does had to make sense as things you do in a social media app but this Devotion to supposed realism was boxing us into a corner there were more things that we couldn't do than things we could so what would change if we allowed ourselves contrivance what if we let the game be a game and trusted the player to excuse contrivances for the sake of making the story and gameplay work what if we let ourselves get really contrived so here's what we knew players liked the act of opening the camera and looking around but they didn't know what they should photograph and they liked the Thematic rapper of social media but they didn't understand how their choices matter and they weren't following the story because they weren't reading the feed but there was one other thing we

knew one moment in the demo that folks responded well to and that was the Sacred Circle at what point in the XOXO build the character Valerie invites the character uh the player to her apartment to attend a sacred ritual but tells them to not photograph the ritual itself and this moment came up in a number of play tests because it's the only time a character explicitly calls out a specific element of a scene so obviously everyone going into the scene immediately looks for the ritual some people photographed it others didn't but at least for that one moment everyone understood the ramifications of their choice and we credited this as the one time we were making the choice space explicit the one moment our branching narrative through photography worked but maybe that wasn't the real takeaway maybe the takeaway was that the game was more interesting when players knew what to look for and what they should be photographing so maybe we should just tell them what to find in each scene we could even give them a list of things to find what kind of game does that start to sound like so one of the most successful games I ever worked on was a game called Gardens of time and Gardens of time is a hidden

object game and you know the genre cluttered 2D scenes lists of things to find and click little bits of story interspersed between linear progression it's simple but very effective and more importantly fun but I wanted my work with akonite to be different than everything else I'd done I didn't want to say oh this immersive narrative experience this Artful vehicle for storytelling let's just make it a hidden object game like that other thing I did that one time but here I was being resistant to contrivance because this was a structure that actually worked really well for us giving players a list of objects to photograph provides them with clear Direction it forces them to observe the environments more closely and gets them to spend more time in each place and we could also use the spaces themselves more creatively we could employ environmental storytelling and use the object list as a way to guarantee that the player would notice certain things but this was also us arguing for the most seismic shift in the game's Direction yet we weren't just changing the story we were changing the genre while still trying to preserve the overall structure we needed to prove that it was going to work so we created a new prototype to see if

Hidden Object gameplay was actually effective and we called it ftn or the narrative because at this stage I didn't want to discuss story at all this was just about finding a fun core mechanic we couldn't worry about why the player would be taking photos or why there's a list so fun first fiction later no plot just play ftn was easy to build using the same scenes as the XOXO demo with a little extra non-diegetic UI for listing objects and Crossing them off as you photographed them it was obvious how to play and fun to look around the scenes and take photos and players were even picking up on more details that they hadn't noticed in earlier play tests ftn was a success but the problems extended Beyond just making it a hidden object game we needed players to actually read the social media feed and we needed them to understand how to progress and we needed them to understand the story through the structure so we created a three-step prototype plan to prove this new Direction could work ftn just needed to answer one question is hidden object gameplay fun with our scenes step two was about getting players to read the feed and

step 3 was about closing the loop and testing whether all this contrivance actually did a better job telling a story so what's wrong with the feed well it was easy for players to miss it entirely the moment they landed on the feed of the chat icon was badged pulling their attention away and the feed had no mechanical purpose there was nothing to do on it we wanted players to just read it out of sheer interest without us adding an incentive to do so and since they could skip right to chat it didn't actually need to be read at all in order to progress so let's get contrived again for our second prototype we turned the feed into a reading comprehension puzzle the posts that the player character would uh take uh the post from the player character were already in the feed with captions and comments but the images themselves would be missing so the player must open their photo drawer and from the photos they just took in Hidden Object gameplay select the one that matches the caption and comments it's know her story or over din but it forced players to prove they'd read the feed to progress and it made it more fun to engage as well as connecting to the photos they'd actually taken and now it was time to bring it all together we already knew that Hidden

Object gameplay would unlock new feed content but now the feed could gate chat requiring players to complete the feed puzzles in each section to unlock new messages and only then after reading those messages does the next Hidden Object scene unlock and each of the three parts of the structure Hidden Object feed chat would move in a loop always in the same direction and we could use diegetic badges on the UI as breadcrumbs to make sure it was always clear what the player had to do next but this direction meant entirely in abandoning branching narrative which was the right call this pivot was about finding the right structure but it was also about scope the complexity of what we expect was never realistic but it was in our best interest to remove the complexities our branching's narrative system entirely to simplify our development we brought our final prototype to GDC 2019 to test with some trusted highly critical peers in this version We reintroduced an actual story a short two-scene plot where the player character attends an open mic night and meets a small coven of millennial witches hashtag hex the patriarchy

proving the structure meant not only proving that it functioned as a game but that it also functioned as a storytelling mechanism and it worked it was fun the loop made sense and our testers were able to follow the story we'd finally cracked the game's structure and we were back on track with a path to production and what's more we knew exactly how a story could be told within our structure through the places visited by the player the photos that they would take the posts they would make in the feed along with the comments on those posts posts from other characters and lastly the private conversations had over chat this is also what let us find our narrative designers we attended GDC 2019 with a robust narrative design document laying out our exact structure of where and how narrative could be delivered and we were also able to generate a wealth of other pre-production documentation outlining the remaining open questions anticipating every possible roadblock that could arise during our production we felt confident that every contingency was covered but of course you know what they say about hindsight yes some of our challenges in 2020 were

absolutely pandemic related and others are just part of game development our working capacities were severely diminished some of us had to take time off and we ran into technical issues that ate into our engineering estimates but we also received invaluable feedback from sensitivity readers that resulted in some story rewrites which also required other necessary and worthwhile content changes and play testing feedback also led to a few late stage design tweaks and cutting of a few additional features it took more time than we wanted it always does but the game we now knew as holla Vista slowly started to take shape you would open the camera into each scene seeing through Carmen's eyes finding and photograph the objects listed on screen and completing each scene would take you to Carmen's social media feed where you'd match Carmen's captions with the photos just taken to complete her posts revealing comments as well as posts from other characters in the story establishing a world and building a cast and as you finished each section of feed those characters would reach out in private messages direct dialogue that would reveal more about

Carmen's relationships and her interior world and then the next scene would unlock and you'd find yourself traveling once again to a new location finding and photographing in each new place as you experience five very strange days in the life of Carmen Razo on September 30th of 2020 we released Holo Vista on iOS roughly 32 months after matcha lattes with star and Nadia after our first spirited discussions about making some sort of game who knew what kind that would blur the lines between fiction and reality halavista was finally out and available to the world and I continue to be blown away by how well halvis has been received I've been moved tears on multiple occasions by some of the honors bestowed on this game and looking back at our earliest thematic pillars you can see that even with some divergences our aspirations were reached in the spirit of the law if not the letter certainly halavista is immersive and maybe if we sacrificed agency in the sense of embracing a linear narrative over player choice but

our camera feature still allows for a degree of player expression just as Carmen explores her own agency over the events unfolding and though we didn't impart a sense of community Through multiplayer as originally intended holla Vista is still a story about social interaction isolation and vulnerability and the importance of finding connection to others and yourself we achieve these goals because we allowed ourselves to remain genre fluid mechanically agnostic for the sake of what we wanted the game to say and I would argue that this mechanical agnosticism extended Beyond just a focus on thematics it was technology that brought us the AR camera that became the game's centerpiece without which we never would have found our eventual genre our Aesthetics provided our unique look and feel and the talents of Blake Andrew and star in creating beautiful scenes and locales led us to gameplay a narrative that focused around environments and architecture and influenced the story's slow dive into surrealism and The Uncanny and even business our pivot to Hidden Object was a production-driven decision we needed a way to work with what we had to come up with a solution that didn't require

starting over and that this design shares a structure with earlier versions is entirely due to the constraints of needing to still ship in a reasonable time frame even after a significant shift in Direction so what did we learn Holo Vista would have never found its shape if we allowed mechanics alone to dictate the shape of the game pivoting genres entirely required us thinking of mechanics as only one facet of game design to be considered and that pivot was necessary because our design wasn't helping us tell a story we got stuck in late 2018 trying to find the right kind of story that would make our gameplay work but the reality was that our gameplay didn't work and no narrative was going to fix that finding the right solution required us putting aside story entirely as with ftn and focusing on finding core mechanics and a loop that we're engaging and building a solid narrative structure out from there only then did we explore writing new story to fit that structure and part of our challenge came from striving for realism in our game's mechanics immersion can easily become an excuse for things being obtuse or non-intuitive it's okay for games to be games holla Vista still looks like a

social media app even if it has a level select screen the scenes are still beautiful and draw you in even if they list the things you need to find and photograph our immersion is aided by the contrivances that make it intuitive and functional nothing is less immersive than not knowing how a system works or what you're supposed to do and ultimately our design pivoted as it did because we needed a path to actually putting it out into the world good design is working within constraints and for professional game development there's no more critical constraint than the realities of production we could have gone down a hole in 2019 trying to make our branching narrative structure work we could have thrown the entire thing out and started with an entirely different concept but the thing that got us to ship was finding out how to work with what we had pivoted and scoped to create a visible end to development and that's why hola Vista exists as it does today it is exactly the shape that it needed to be I frequently described my role on Holo Vista as creating a narrative scaffolding my job was always in support of

akonite's vision I was in support of star and Nadia's aspirations strix and Corey's story Blake and Andrew's art Mario's music the proudest I've ever felt as a designer is knowing that I helped create a structure that allowed something beautiful to be expressed and I hope this talk helps all of you create beautiful things and enables you to support more beautiful expression in our industry uh thank you all so much for your time [Applause] I'm told we have three minutes because I went slightly faster than I'd rehearsed so if anyone has any uh questions they can ask them here or we'll go across the hall when our time is up uh are there mics or am I just repeating the questions there is a mic

hi that was really great it's really an amazing adventure that you've that you've taken uh have you ever have you thought about uh think back and and think about the times that you've gotten the support that you have uh I've seen a lot of people who've made indie games and they've they've had an ambition and a lot of them have never even thought like would I even play this game myself you know or you know it's is this is this a game and so some people have been able to make their game with the support and love of their friends who are like well they encourage their vision but would you have also thought would you have gone back in time and changed the way that you had gotten feedback would you have also gone to other places to have people who maybe not supporting you personally but are supporting you uh by giving you feedback on what I play this game is just you know getting would you change the way that you gotten feedback in the past um I think we always would have welcomed more feedback I feel very grateful that we got the feedback we did uh when we

did um and and that feedback continued to come in throughout development and as frustrating as it is to be mid-development and for people to be giving you feedback that doesn't work that's exactly the feedback we needed to hear to know how to change it um right I I was very happy to be soliciting feedback from people who were not afraid to be honest with us about how they felt about the game um because people could have been polite and kind and said the XOXO version was was fun and delightful and maybe we wouldn't have gone through the process we did in terms of asking ourselves you know maybe this isn't the right direction yeah you kind of need both yeah cool thank you polite and unkind yes there's there's kindness and unkindness sometimes uh I think that's our time but we'll be going across the way uh and I can chat more and answer more questions there thank you all so much have a good day [Music]