Why Most New Games Still Face-Plant — and How Player Insight Can Flip the Script

Hey Sean and welcome. So in the games industry today, the environment for new game launches has become increasingly difficult. And we've seen a number of games that have launched to kind of underwhelming player expectations or even complete failure. And some of these games have lost over $200 million in total development and marketing costs. And so for new game launches, we've seen so many failures relative to successes over the past few years. And I think there's a number of reasons for this, but I think one issue that does seem to be a major contributing factor is that games today do not seem to be properly vetted or have enough player feedback and validation. So now I want to transition to you Sean and what LiveAware is doing. Does Liveaware solve this problem? Well, we certainly believe LiveWire is a great tool to add uh to a team's toolbox. You know, it takes it takes a

lot to overcome some of the uh inertia that you talked about earlier. Um and we're seeing that across the industry, which is which is pretty unprecedented, but um you know, I'm a professional game developer. I've been making games uh for well over two decades. And you know, we've felt the pain firsthand. Um, you know, we at the on the AAA scale, you know, when we were working at companies like Activision and Idos and uh Microsoft, you know, we had a ton of resources, UXR teams, marketing, publishing. Uh, but even with all of those resources, um, we always felt siloed from real player experiences, real player expectations. And I think a lot of teams today are working kind of um in in that silo, in that in that echo chamber. Um, and so, you know, I think one of the reasons why we're seeing so many of these failures is around this gap in in the gap between what we think an audience wants and what they are actually expecting a game to to do or to do for them or deliver.

A big gap between a team's vision uh for the experience that they want to make and what players are actually how they're actually experiencing that uh that what they've made there. And so, you know, bridging that gap is really what we're trying to help solve at Liveaware. Um, so we're building tools and workflows so that teams uh of all shapes and sizes can really reduce that uh that um the gulf between what we're building and what players are expecting. And and Sean, if you were to characterize like what are some of the major problems or issues that a lot of game studios or designers face when it comes to getting that feedback and understanding, you know, whether you're on the right track in terms of like the internal game vision and things like that. Um what do you think those problems are and then what aspects of that does LiveAware solve? And maybe you could even talk a little bit about specifically how the service works as well. Since launching Liveaware uh last year, we've talked to hundreds and hundreds of game teams um across the industry, all shapes and sizes, and we

kind of were we've identified a number of pain points that I think many of us are are going through as we build these games. Um one is um you know, teams have over time come to rely heavily on quantitative data. That's like numerical data that can be measured or analyzed statistically. And that's really helpful for identifying uh that an issue is happening with the play. Um but to understand why that those issues are happening and what to do about them, we really need to start to leverage qualitative data and you know that kind of descriptive subjective information where players are expressing their opinions and experiences um uh really come into to the fold. Um qualitative data is really hard to capture and organize especially at scale. Um you know it's very cumbersome. It can be a very manual process. Um and analyzing that feedback can take you know hundreds of hours and specialized um you know specialized skills. And so um one of the things liveware is trying to do is automate the entire process for capture

analysis and sharing that evidence so that even small teams can can do this very fast. Um you know we're we're seeing this across the board you know especially this last you know 18 months where teams are shrinking. We're having to do more with less right. Um and with the rise of hybrid and remote work, it's becoming increasingly challenging to collaborate uh you know and to to realize the creative vision. So um what we're finding is many teams just stop the process of ingesting feedback. Um they often work um for many months sometimes years uh in in that echo chamber. Um and it's really to their detriment I think. Um, so if we can make that experience of bridging bridging the gap between uh game teams and and players easier um that more teams we find are doing it um and they're building that habit of regular listening which we think is so important just for the audience so they kind of understand exactly how Liveaware works. Could could you kind of like, you know, maybe more

specifically describe what the service does and then and then maybe try to bridge that gap because if I'm hearing you correctly, it sounds like what you're saying is that the thing that's not coming across in a lot of player feedback sessions when when game studios are trying to do player validation is understanding the nuances with some of the quality qualitative feedback that they're getting. Maybe that's getting lost and maybe there's more of a focus on quantitative. So if you could also just kind of highlight where liveaware could potentially like highlight some of those things that of regular you know using a regular player feedback system whether it's you know um uh whether it's a focus group or whether it's using you know uh some of these like cloud-based play testing services what that wouldn't do totally um yeah liveware um we built it first and foremost to uh we I mentioned earlier that liveware we want to automate the entire process of capture, analyze and share. And so we started um with a vision of um being

able to capture any play experience from any device anywhere in the world. So this is high quality streaming recording from desktop, from mobile, console, XR. Um uh very high-end streaming, so it's highly optimized. Um doesn't interfere with gameplay at all. one button click and you can be privately streaming anybody uh can be privately streaming that experience uh to a private uh web portal where teams can view and analyze that material. Um we encourage the team and players to use a thinkaloud method so they're verbally annotating commenting as they play. Um and the system automatically transcribes uh translates in the case we support 35 languages and we can also ingest things like discord conversations survey data a lot of other uh that kind of data that can come in and then as soon as videos hit the platform the system automatically gets to work analyzing that material. Um, so looks for things like bugs, areas of improvement, uh, technical issues and presents that

information to a team in a really easy to use dashboard. Um, and that's meant to surface uh, insights for any member of the game team. So whether you're an artist or a designer or a programmer based on the themes and the topics that you're interested in most um libraryware is surfacing those details to you and which you can drill down to see what exactly the player was or players were saying and go right to moments in in the play experience. So you can understand what was happening uh through integrations with the tools that devs already use like Jira and things like that. you could teams can pass that information right to the people who can take that information and make make good use of it and make make fixes. Um and so it's really the first tool that's built for game developers by game developers um to integrate into their process from a very earliest stages of development. So Liveware for instance is really useful before you're even ready to go out to to players. You can use it for um uh internal build reviews, uh feature reviews. Really helpful for uh ensuring

that teams can be cohesive and and work very closely together, especially if they're remote and hybrid. And then it grows with the team as they begin to start to work with friends and family, um external players. Um the the service is able to expand to take that in. Right. And so I I mean I think you've mentioned quite a lot. So maybe just to specifically break down a little bit more, but the first thing you said is that you know it's it acts as a way of kind of recording a bunch of um play like player gameplay. And so to uh on the the one thing that it does is it'll um it will record and store um players who are game testing a game. And then the second part that you mentioned is that it will analyze uh that gameplay and be able to create a dashboard and be able to provide some kind of analysis on on that dashboard to potentially highlight some insights that normally would otherwise take a producer or a

product manager or some kind of analyst to sit down watch all that stuff and try to figure that stuff out. So there's like a time-saving component there. The the other thing you mentioned is that it does some kind of sounds like sentiment analysis. So whether you're attaching some kind of live aaware agent to Discord or Slack or or maybe it goes into certain forums and just analyzes a bunch of the text or videos and things like that to also try and draw insights not just from play test but from maybe some uh communities or websites or things YouTube videos or things of that nature to try and kind of draw stuff in and and also highlight specific insights that could be used to help a team with understanding where there may be some issues. Is that my That's absolutely correct. Yeah, it's exactly right. Yeah, that multi- channelannel feedback is really where it gets very powerful for teams because um feedback is living in so many different places, right? It's the real player experience as they play. It's in the Discord conversations uh

before uh before, during, and after release of u new builds uh in reviews uh in public forums like u YouTube and Twitch videos and those comments. And so we're really building Liveware to be a multi- channelannel um be able to monitor that feedback across multiple channels, bring that all in together in one place so that teams can have a very easy way to find and analyze that information. And and maybe something that hasn't been explicitly stated by you, but that's kind of coming across is that there's a lot of sort of automation pro possibly AI. I I assume you guys are doing a lot of this stuff using AI and and one of the thesis that I have is that I have a belief that in this age of AI that the most critical factor for success will be essentially product velocity meaning both the direction and the speed at which companies can make a good product. And so to the extent that you have this kind of tool that helps you determine the

direction to pivot when when you have when you're getting uh you know game feedback and to be able to do all this stuff very quickly I think you know in theory will really help game development teams make the right kinds of decisions and could be very valuable. But I wanted to have you maybe opine a little bit on that aspect in terms of AI and how much AI integration is in your service. How much do you guys leverage AI to do some of these things that you were talking about? Yeah, it's actually it's it's a critical component of of the product in that um uh you mentioned earlier um the big pain point in the kind of the old way of of handling a lot of this stuff was some somebody had to sit and watch this material. Uh you know so an hour of gameplay could take three hours to sit through and hand document key areas of the of the play. um they might have to manually make clips um and share that information through a variety of different communications tools. Um so AI

can help um radically improve or make more efficient that whole process. So um a system can our system can go through and analyze gameplay footage uh can analyze the transcripts can pull in these conversations and look for themes across the that data. Um and so really it's not only uh handling the manual kind of uh minutia of the of those basics but um it would be nearly impossible for an individual uh to to look at the corpus of data at scale. So if you have hundreds of hours of play, um so for even medium-sized teams, large sized teams or that don't have a dedicated UXR group, um this is just it's a game changer in terms of uh team's ability. And for us, one of the things that we really want to to one of our big missions is to democratize this information so that as an individual on the team, as an artist or as a designer, um they have access to this as part of their regular uh process of development. We think this is a a massive enabler for

individuals to be able to to to make good decisions about how to improve their part of the game. Yeah. Actually, that's something I hadn't thought about, but um to the point that you guys are recording all these videos and you're able to create different kinds of uh visibility, right? Like whether it's dashboards or things like that. And it doesn't necessarily need to just be for a product manager, producer, or game designer, but to your point, like there could be aspects that are tailored for different disciplines within the company. Like, you know, somebody like, you know, you might have hundreds of hours of gameplay and there are certain comments buried kind of needle in the haystack style somewhere about art style and things of that nature that should be going to an artist. Um, and so that basically through the use of AI and things like that, you could basically call those specific comments out and then be able to make some of that stuff visible to people. That's absolutely that's exactly right. And that's the main mission is we've seen as I

mentioned earlier, we've seen a lot of this information is siloed. It's in in the hands of a UX research team and it's delivered to teams through reports. But what if an individual on the team uh could get a digested version from the last two weeks of play? um here are the topics that I care about as an artist. So, I want to know about the vis visual quality of the boss fights or the um lighting conditions or things like that. That individual can express what they're what they're interested in and have have that information served to them so they're constantly being um have a stream of feedback coming in from the various places so they can make better decisions. That's a that's a real um enabler. Yeah. I was actually just coincidentally just yesterday I was talking to a friend of mine who's a maybe I shouldn't specifically call him out but he's a very famous person who was essentially the creator of one of the biggest shooter games uh in the in the industry and one of the things that he told me was that he felt that for a

lot of games when you when you think about when certain games fail there's like this oversimplification in terms of like ascribing a causality. Oh, this game failed because of this or that reason on in and generally in like some kind of strategic context. But what he was saying is but sometimes it's actually not that. It's actually that you have hundreds of these small decisions, these micro decisions that are being made. in that sometimes the the reason why some of these games fail is because of some of these micro decisions that the you know management teams leads of some of these uh game studios aren't actually aware of but it's somebody like deep down that has that's responsible for a small thing and so I guess in theory if your service could could help surface some of these micro issues Because what he's saying is like it's not the macro, it's the micro. Right.

Essentially, that's what he was saying. I mean, millions of tiny decisions, right? Right. That that that would actually um give a lot of development studios potentially a lot of visibility and a lot of power to be able to affect change in their game. Absolutely. Yeah. And and I think the sooner that um um that a team is conditioned or builds the habit of really being thoughtful about listening and being open to that feedback and uh using feedback as part of their uh normal mechanisms for improvement. Um that that can pay dividends in the long run. So those micro decisions are being made earlier. Uh there's earlier course correction. you don't get to what traditionally happens with games is to the end of a kind of the cycle we're going we're going into release and now we're going to start to check this against uh player expectations when it's way too late to make uh to make major course corrections. So I'm a firm believer in that and also a firm believer in um can we empower individuals on the team to

have the visibility into these problems so that they can uh make improvements um more rapidly. And not only do you have a happier team that's more engaged because they're they're they're seeing the results of their work, um but they're more inclined to to lean forward and to uh to make those improvements on their own. So we've seen a massive with teams using live or a massive acceleration. Um smaller teams are are are iterating faster as a result. Right. And for maybe somebody in the audience who's potentially from a game studio who's thinking about, oh, actually we do need a better solution when it comes to player validation and feedback. Could you like characterize for us a bit? How would you characterize like um let's say a typical team that's trying to get player feedback and validation relative to then a team that's using LiveAware? How would you compare those maybe from a staffing or economic perspective? Uh could you characterize that for us? Yeah, absolutely. Um well, we find most

teams have the desire to incorporate uh basically build their community early and be inclusive and sort of wanting to get their feedback as early as possible, but um oftentimes they end up deferring that for for for many many months because it's hard to do and you get you get kind of embedded in in the normal cycle of development. Um, and so they do it irregularly. Um, when they do those feedback sessions, they tend to use things like postplay surveys or Discord queries and the information that comes back can be very thin. Players um often are very poor at self-reporting things. They have a poor they don't remember the gameplay session. They can't give a lot of details. Um, and they and then teams don't have the evidence to cor correlate to that to that feedback. So what we're seeing is many teams deferring that process until very late. Um they are um that information is being siloed through a product manager, through a community manager, through a producer. So it's not getting down into the teams fast enough.

Um and so um with Liveware, we're we're obviously um uh it's a it's a tool that devs can use very early on and um as I mentioned, the entire team can benefit. So a team like uh Brain Jar, they're working on a really novel game called Dead is Disco um which you may have heard of. Um um they have incor like 12 12 folks have incorporated liveware as part of their early process. They went from um really having a hard time with their even their internal build reviews getting team members to capture and to share feedback to where now weekly every member of the team is playing at least an hour. um those team members are verbally annotating as they play and so they're getting tremendous data from the the team itself on on things that they're seeing and how they can improve and then the team was conditioned to use that workflow. And so when they started to go out to friends and family and then eventually to u more full-scale external players um the team was was really uh already adapted to to leveraging the

tool to get what they needed out of it. And so, right, and I I get the kind of value proposition around, especially for a small team, you don't have a lot of people, you don't have a lot of time, you don't have a huge budget. So then, you know, at some point you probably do want to get player validation and feedback. And so by using Liveaware, you know, it's going to be a lower cost, lower effort, lower amount of people. like if if you were at EA or Zinga, do could you characterize what would that consumer insights team look like in terms of people, staffing, time and then or or we don't need to go into that or maybe just speak to and and then maybe you could talk to your your customers, the the the customers you have now that are using LiveAware. What does that team look like? Is it one person? Is it five people? You know, and you know, if you could help characterize that, that'd be awesome. Absolutely. Well, so in a AAA setting in the the sort of um uh large publisher scale or AAA dev teams, they

often have a dedicated UX research team um uh so they have UX researchers on staff um that are that are there to take builds, get them into play community, do all the player logistics and stuff like that and then come back with with reports. The problem is that's often a shared resource. So teams are vying for a time slice of that of th those research teams time. Um and often there are long cycles to be able to get a build and then come back with a with a a report. Um so we we find that many teams feel like that that intermediation can be cumbersome. It's good in that it's formalized and it's uh very professionally done but they want a more rapid uh way to do that kind of on their own. Um and so um you know we have a range of customers from Craftton at the high-end scale to a number of uh large uh ventureback doubleA and AAA teams folks like Bonfire believer gardens um GSC game worlds another customer that just released Stalker 2. What we're finding that's very interesting is they're using Liveware for very

different use cases which were which is great. Live ends up being a very flexible tool for a lot of different things. In the case of GSC Game World for instance, um they were using for two great uh use cases. One, quality assurance. Um so they were rapidly releasing new new fixes um into the community and their community was using Liveaware to um to send back captures of the play. Um and they were able to identify and spot bugs uh within their community in a very fast way. So, as soon as players uh the system itself was picking up on these and they would be able to go into a liveware and see a corpus of uh clips and highlights that would show them exactly what was going on. Um they also started using it for analysis of public streamer um commentary. So when they would release a new patch um they would you can identify uh videos in YouTube for instance like co Carnage did a did a whole series on stalker 2 play um and instantly see from across the 16 hours of that co-played um

problems that he was having insights from his kind of expert play perspective. Um so th that was two really great those great use cases. Um other teams like Treehouse have been using it for competitive analysis. So, they have been playing competitor games um and watching streams of competitor games, ingesting that information into LiveWire to be able to see, okay, what are players saying about uh comps um for for the game that we're that they're building. Um and they found that very helpful to be able to to kind of have a very clear picture of what kind of market they were going into and make adjustments to their play to differentiate. Um so, those are just a couple of examples uh of uh how teams are are leveraging it. But in all cases, what I think teams are finding the value in is that they have control. Um it's a tool that they can use. They don't have to wait. Um they don't have to schedule time um for that information. They they have it at their fingertips, which is really critical no matter what scale of team you are. Right now, one of the issues that I've personally found when

trying to get player feedback is like this this issue around signal versus noise. I'm wondering if you could comment on that, right? because like yeah I think it's very difficult sometimes to understand what is good feedback what is not good feedback I mean and and to some degree you know may maybe that's not on live aware right like your your service does something and it might be more the the the the game studio taste makers or designers that need to interpret it but is is there something in terms of you know potentially being able to highlight specific types of feedback that may be more relevant or important relative to other types of feedback that may be less. So, it's super great point and in fact um we often call it the feedback tsunami and teams teams kind of get inundated with feedback from all different directions and it's hard really to differentiate what to pay attention to, what not to pay attention to. Um so I think it's both an art and a science. Um we often recommend some things to teams especially ones that are

less experienced in in in uh kind of being on the front lines of this kinds of feedback. Um, one of them is to to ensure that you're setting clear objectives when you set out to um, capture and analyze feedback. You know, teams at any given point really make sure that you're focused on specific questions that you're trying to answer or specific things that you're trying to test. Um, and um, have those in mind when you're when you're reviewing the feedback. So, you're really filtering for those specific things as opposed to coming in completely general. Um, you know, teams ought to look for patterns rather than specific outliers. So individual complaints from specific like vocal minority can really skew your your decision-m if you pay too much attention. So you're really looking for themes across many different players uh many different contexts across time. Um and be judicious about uh how you react to those things is really important. Um we find that uh prioritizing superengaged players. of players who have leaned in, um, who clearly have a

vested interest in in the in your success, um, who understand the genre of the game you're building. Um, those kinds of players can often, uh, provide more valuable feedback because it's more in-depth. Um, and then you can balance that with newer players who have a fresh perspective on it. And taking into account those things can be very valuable. Um and then of course I talked earlier about com um qualitative and quantitative but when you can combine those two kinds of data together it really um can um form a very powerful um uh net in terms of how how to um how to deal with that stuff. So um those are some of the some of the kind of examples of things we we recommend. Right. Yeah. I I think the the the tricky part about player feedback and validation is that it's it's very much an art, right? It's very and it's very difficult to try to parse through and understand how to react to player feedback at times. But I I thought I could ask you a question where I kind of step back and you know

kind of like not specifically related to live aware but more of a philosophical question about player feedback and validation be since you're in this space and I assume you've spent a lot of time thinking about it but kind of like the way that I've characterized player feedback and validation is at the simplest level I kind of think about two primary schools of thought and I I I call it top of pyramid and bottom of pyramid. Right? So if you imagine a a pyramid at the um at the bottom layer is kind of like what I would call like the the the riot philosophy like player first being driven by what the players say and and really like leaning towards the players to tell you how you should uh make changes and to uh optimize gameplay for them. Now part of that is also conditioned upon understanding what the player persona is and finding the right kinds of players. But the philosophy there is players drive the

feedback drive the design and decision-m. Now, top of the pyramid is more I would I I I kind of characterize it and I don't know if they would agree with me or not, but old school Blizzard where it's in old school Blizzard was like we are the gods. We are the taste makers. We know what players want. If I like it, they will like it, right? And it's actually kind of similar to I don't know if you've read Rick Rubin the creative act but yeah Rick Rubin's like you need to make something that you love. Yeah. And so there's there's these two schools of thought. Yes. Um and I'm wondering like where do you fall on that? like based upon Liveaware, I would assume you're more oriented towards the bottom of the pyramid, but just just again, you know, not not specific to your service, but just more of a philosophical question about player feedback and validation. I'd love to hear your thoughts. It's a great observation and that that continuum from

a to completely audience uh led. It's a really fascinating uh uh through line and and the more teams we talk to we realize that everyone's got a little bit of a different philosophy on this as you said the older school and the newer school. Um, I I personally as a developer, I I think it's very important that a uh a team have a very strong creative vision for what it is they're trying to build, the experience that they want players to have, a very clear idea of how mechanically and from a presentation perspective. Um, you're going to deliver that. Um but then um within that vision um the being humble enough to understand that you're not going to get it right you know the first time all the time and that it really takes there's a lot of benefit to building an audience early and making them feel included in the process and having avenues for them to be able to help uh guide you and your team to refine within that vision. Right. Um, personally I found I and I've

and and I've built games on both ends of the spectrum as the outdoor shut everybody else out in the world and and hope for the best. It's a very risky proposition, especially if you have a finite amount of time and budget and you have to you're forced to release. If you have endless budget and time, you know, go for it. um um those are great for very highly personal projects and where I would say the artist kind of um the purity of that vision can can come come through today's market though I think audiences are diversifying tremendously um and it's harder especially for us as as we represent kind of a specific demographic our if we're our age and our our cultural background um to I think it's imperative to kind of um condition a team to be able to to listen uh and do that throughout the the life cycle of the product. So, and and maybe, you know, I'm just kind of thinking about this and as you were talking, I was kind of thinking about maybe to some degree that this kind of the the two models

that I talked about may not necessarily be mutually exclusive. They could potentially be complimentary in the in the sense that like the creative act could potentially help characterize and define a new kind of gameplay model, but then the optimization of that would be bottom of the pyramid. So a specific example would be Dota mod for Warcraft 3 that gave us MOBAs is the creative act. Y but then the Dota mod to League of Legends is the is the bottom of the pyramid. Yeah. So So yeah, may maybe they are kind of compatible to some degree. Yeah. And and I I think there's also like um maybe even like a middle layers because one of the friends that that I speak to formerly from Riot has also suggested that the other thing that he's careful to do is not just take because you know I think Riot's big on player personas. You find your player

persona and then you get feedback from them. But then I think his philosophy is also not just the player persona but like very specific people going for higher quality rather than quantity. So like y finding the like two three four five people that are you know really understand a the specific vision of the game and what you're trying to achieve and have a lot of context and are taste makers and getting the feedback from the taste makers relative to a broader set of people. Yeah. But I I guess in either either of those cases, your tool would work the same, right? Yeah. It's funny when when we set out to make Liveware, it was really we consciously said, "Hey, we'll make we're going to make a tool set that would cater no matter where you are on the spectrum." Right. It's And the other thing that we're really careful about is LiveWare doesn't um doesn't prescribe what you should do with this information. It just tries to surface the insights to you in a way that's convenient and um highly customizable. So you can filter uh for what you want to use it for. But we try

not to um exert an influence on the process itself for teams because we recognize that teams are are um approach this with very different philosophies in mind. Right. And sorry I I wanted to go back because I don't think I closed the loop on this question around uh for example if a team that might be interested in Liveaware is interested in using your tool. Yeah. Yeah. Could you talk about for some of your current customers like how many people does it take to run play tests using LiveAware and then what is your you know like like your pricing model? How much would it cost a small studio to be able to use Liveaware? Just just for some people who might be interested in in the service. Yeah. So one of the things we tried to set out to do when we designed Liveware to make was to make it a zero uh integration platform. So very easy to get up to speed and start using the platform. no engineering or technology required to be able to to administer play tests. Um it's very low friction. So um when working with external

players, we have a play test uh management system that handles scheduling. It's got a built-in CRM to uh handle contactive details, send out information when a play test starts with links so that players can get in and play. And so a very uh small number of people oftentimes uh a producer or community manager can run a large scale play test using liveware. Um so I think um that's very helpful. We've actually also uh done an integration with a tool called First Look, which is now owned by Pragma. Yeah. Yeah. Chen, exactly. Yeah. So Pragma um it's like two great tastes that taste great together. First look and liveware um work really well together. First book handles things like Steam code distribution. Um uh legal document uh signing and things like that. Um uh player attribution. Um and so teams are combining um that sort of supercharged onboarding through first look. Um we did an integration so data comes through clean and a player choose

liveware to do the capture portion. Um so that's working really well. Um, in terms of the u pricing model, we have a flexible uh set of plans um that run anywhere from a newly introduced pay as you go uh plan that's meant for indie developers. We're we're looking to democratize uh liveware so that more more and more teams can use it. Um and so it's a pay as you go usage with very small fees for for video ingest and uh video storage. Um, it's unlimited seats, so we want to make sure everybody in your team can use it. Anyone within your community can use it and not have to worry about nickeling and dime on seats. Um, the plans go up into we have a basic tier that runs um it's like uh 399 a month. Um pro tiers at $799 a month and then up into enterprise tiers which open up new features uh and additional capacity for teams to use. So, um, no matter what kind of team you are and, um, what kind of stage you're at, we're looking to be able to support support, uh, you with the right price. And so,

maybe the next question I can ask is, so you've you've launched this service, you've got some customers using it, and and you had talked about some specific case studies with some of your customers, and I actually recognize most of your your customers. So, so I should probably ask ask them about it later, but like uh you know uh I guess since launching, could you talk about maybe some of the lessons learned working with your customers, getting clear feedback with the service? Uh maybe you could walk us through like some spec uh some additional specific case studies or but more focused on like was there something interesting that came out of working with these customers and using Liveaware that you could share with us? Yeah, absolutely. Um, you know, it's one of the best parts of the job, I would say, is getting to work with so many great teams and being a small part in their success. I think for me as a a longtime game dev, um, at this stage in my career, um, being able to inspire the new generation of game developers build great games is a real um, compelling

mission for all of us here. And so, uh, working with teams like Brain Jar and Treehouse and Gardens and Bonfire had been have been really fantastic. um the level of innovation that's happening within the industry even at a very hard time is just inspiring to see every day. And so um you know one of a lot of great lessons one um was we set out to make sure that Liveaware was something that we were using uh to improve our own service. And so we take great pride in the level of support and connectivity we have with the teams that are on the platform and constantly listening constantly taking in feedback from them on how to improve the system. And so Liveware itself is kind of a case study for um uh building that habit uh of regular of regular feedback. So we're kind of living our we're eating our own dog food as it as it were. Um one of recently we released three case studies, Brain Jar, Treehouse, and Lunchbox Entertainment that are up on the website now. Um Brain Jar is one of our favorites. Detis Disco is a really fantastic kinetic. It's a music video

meets fighter uh where the action is is highly synchronized to any music that you feed feed to it. And so in a game like that where um reaction time and animation is so important, you can imagine um all of the ways in which that system can break um and not be responsive in a way that's natural for players. And so they've been really successful at using liveware to spot spot issues small deficiencies in the presentation or the feel the mechanics um um uh in a very fast way and then to have as I mentioned earlier have the evidence that they can support to give to an engineer so that you don't run into things like cannot reproduce will not fix you know need more information an engineer or a dev has exactly what they need to to make those um uh problems go away and so the iteration loops have gotten faster and faster. Um, you know, uh, James the producer there talks a lot about prior to using Liveware, he just couldn't get the team into the habit of regular build reviews.

The idea of, um, putting it out to players was very foreign to them. They had a lot of resistance from the team that like, hey, it's not ready yet. Um, and with Liveaware, um, not only did they build that habit internally, but they it gave them the confidence to be able to go out to players and to know that that feedback was going to be useful. wasn't going to be a waste of time, which is a huge game changer for for them. For a team like Lunchbox, they're working on a a MOA uh it's called Sroco. Um and they had a very difficult to to understand problem with their progression. Um that their prior surveys were really wasn't the players weren't articulating what problems they were having with the progression in the surveys in a way that the team could pick up. And when they started using Liveaware and getting that commentary in real time, um they were able to pinpoint exactly what was um happening. Um and then they ended up redesigning their progression as a result and really changing the the trajectory of the game. Um so we feel that's one of the of our kind of our proud achievements is we were able to provide a vehicle for them

to be able to get at that those kinds of insights in a at a time in a time when they could make make those changes. So just a couple of examples there, right? And maybe one last question that's more specifically focused on the service itself and and usage of the service would be a question really around when you think about the the development life cycle. I I think certainly it it's kind of a bit more obvious to me how you would use LifeAaware when you're kind of like in alpha stage, but if you were to think about the other stages of development um earlier on when you're, you know, from the concept stage to when you're just starting to stand up certain features to the point where you're, you know, uh feature complete and then just trying to um you know, kind of fine-tune or iterate like would Would you say that where where would you recommend your customers to start using Liveaware and does it have relevance in maybe some of the earlier stages of game development

as well? Great great question. For us, we wanted to make sure that Liveware was was useful at all stages and we have found we recommend and have found um our teams are getting the best value when they start at the conceptual phase. So way be before they're ready to go out to players that the teams themselves are building that habit of weekly uh reviews, feature reviews. So if an engineer and artist is delivering a new uh on their deliverable, they'll record that and send it into Liveware along with annotation verbal annotations for that. Um uh the weekly build reviews with their team, especially with the distributed or hybrid teams where you have people all over the world. um uh reviewing regularly, having those individual team members feel like they have a have a voice in the in the improvement. Um that's been a real game changer. Partner feedback when you're when you're delivering early milestones to your publisher, your investor, having them jump into LiveWare, be able to provide that feedback so that everything's in one place. Um that's been really powerful. And as I mentioned

um when um teams kind of uh get accustomed to using liveware at that early stage as they grow and they widen that funnel of uh folks that are kind of viewing and giving giving feedback. It's just a really natural evolution. Um so um through concept pre-production production we find teams are using it for those build reviews um uh and uh feedback quality assurance is another big use case. So QA teams are able to punch way above their weight. Um smaller QA teams instead of having to stop play, you know, write down the bug, go back, try to reproduce it, make the clip. Um those those QA teams are just using LiveAware, they have a hotkey to save clips, they verbally annotate instead of writing down. Um and they can instantly send send those things into Jira. So um what we're finding is that teams that are having to either use their team as QA or smaller smaller QA teams are able to be way more efficient. Um, and then as you go as you start to go to friends and family, as you start to go into these larger contexts, liveware scales.

Um, so as I mentioned that player the player logistics uh, workflows uh, to be able to schedule those things. Um, the the desktop recorder is a very lightweight, easy to download, easy to run um, application that the players use. It runs in the background, is able to collect that information. Um, and then as you get to launch and post-launch, the addition of being able to pull in data from Discord, um, pull in data from YouTube and monitor live live, uh, sentiment, um, is really helpful. So, we built Liveware to be a kind of an end toend solution from the early stages all the way through to post-launch. Right. And you kind of rushed through like the part where you talked about Jur, but I I do think that's kind of a small thing, but kind of also very relevant and kind of important in the sense that to the extent that you have the the AI that looks at the game play that can, you know, potentially find specific issues and then you have like this dashboard where you see all these issues and the juror tickets are ready to go and maybe a a a human

producer PM can just look at these say yes yes yes like how much time do you save, right? It's there is potentially a lot of cost savings there where liveware with AI then can directly in integrate into a workflow. So I I think that's that's awesome to hear. Yeah, it's and we're really at the start of that now. So we built the initial integration to be able to channel evidence and channel uh commentary into into Jer and other project management tools. But as you as you mentioned it it where we're going with this is the system to be able to as you mentioned write the Jira ticket itself. Um document the reproduction steps. Um uh basically get it packaged up in a way where it's one button click and it's in and they have what they need. Um today our Jer integration is one direction. So you send information to Jira. We're working on a a birectional communication so that the liveware knows what's in your jer database and so when a new issue comes up it can add to an existing ticket or

it can tell you hey this is something that was resolved in a previous milestone and now it's back you I mean I think an ideal scenario too would be like hey here's a new build did these your tickets actually get resolved or not that sounds really right you're you're essentially failing yeah failing a a check yeah um so really it's early days already. We're seeing a lot of teams get benefit from it but uh you're uh you're absolutely right in your vision how that how it can evolve with the weed use of AI. Yeah. And uh yeah, no, the one other interesting thing that you mentioned that that you kind of rushed through, sorry, was like that the the case study and yeah, not I mean I I'm just saying for for me it's I heard it, but then as I think about it, I'm like, "Oh, actually that's pretty cool." Was like when you mentioned how um I forget the specific company was kind of externalizing their QA, right? by using liveware um you basically have like a um like a capability QA that's typically locked within a game studio. But then

what you're saying is with LiveAware now you've kind of have like this extra net of dedicated players or people in the community who you can then lock in and become like an externalized QA for your company. So I I thought conceptually that was pretty cool. But yeah, I just yeah, you you said it, but in but you know, my mind didn't click on it until until just now. But what's nice in Liveware is it'll it'll pick up on those signals from many different sources. So, and it organizes them in the same place. So, you can say, hey, over the last two weeks um uh essentially show me all of the bugs related to this particular like u mechanic or boss fight for instance. And it will pull the insights from your QA team. It will pull insights that came from like an external play test. It'll pull insights from your publisher and bring them together, organize them into one place so you can see them all in one in one easy to see view. Um so that kind of aggregation is really powerful when it casts a much wider net than um typically for teams this information is

siloed depending on the department or where where it's coming from. Great. So so now I just wanted to shift gears and may maybe end with a little bit more background on you and the company. So the first question I have is like I know you come studio side and you've you know had a long career longtime game industry veteran worked on games like Tomb Raider to Never Alone and so I just wanted to kind of like maybe step back a bit in your career from the point where you were on the studio side to thinking about this current opportunity you know developing LiveAware. Could you talk about what was that origin story like? How did you make the transition from being on the studio side to kind of making this kind of a tool? Yeah, I think there's um three three kind of key moments in my career that kind of set us up for LiveWare. Um and the first I'll start in the sort of when when I was working in AAA, I was running Crystal Dynamics for a number of years on the Tomb Raider series. My my partner and CTO David Burgerer was the uh partner engineering director for 343.

So he worked on Halo 45 and Infinite. and we had both had very similar experiences coming from AAA where you know we're on a development team we have tons of resources we have UXR a research team at our disposal you know marketing and publishing um and yet we still felt like as teams we were cut off from from contact and and cut off from that um direct kind of feel of what how players were experiencing our our work. We also had this problem where a lot of times at that time we were dis not only discouraged but prohibited from sharing early versions of of our work uh with with players and so you're kind of flying blind at that point especially after you know developing a game for two three years hoping that the thing that you're making is going to is going to resonate. So um we felt that to be pretty problematic. Um when I got uh up to Seattle about 11 years ago, 10 years ago, um I was asked to work on this very special project called Never Alone, which is an indie uh cooperative

platforming game. Um and um it was special in that unlike the games that I had worked on in the past where we were um you know, big budget, you know, we were the experts, um this was a game that was designed to be a celebration of a specific culture, an Alaskan Native culture called the Anupic. And um the the project was designed to make put them at the center of the development uh effort. So they were collaborators uh from the very beginning. And our jobs as game developers was really to extend our craft in a way that could help them share their uh cultural mythology and spiritual and um also worldview with with the worldwide gaming audience. So we I learned very that was the first time where we learned about the importance of like checking your ego at the door. um really developing good listening skills um and uh working to cultivate a community very early on early on in development. So I think the inception of LiveWare started there uh in earnest to kind of we were really

successful in in integrating a community and helping helping us to build a game that would um you know be memorable for years to come. um where the turning point happened. Um I have a small game studio in Seattle called Very Very Spaceship and we worked for years with Niantic on geospatial games. Um we had this problem where our teams could not um did not have a feel for what players were experiencing when they were running around in the neighborhoods playing games like Pokémon Go. Um, so we developed a really novel backpack, streaming backpack that would beam back the device display of the of the phone, the camera feeds and their their commentary so that um we could build geospatial games faster and that was really the genesis of Liveaware. Um, we shared that with teams in the Seattle dev scene and we really got a lot of encouragement to generalize it for all games and to productize it and that's when we spun liveware out. That was about two years ago. Maybe the one other question I could ask is since you started Liveaware two years ago and more more from a company perspective, could

you talk about maybe some of the biggest challenges you've had over the last two years and then maybe some of the biggest lessons learned? I would say challenges in general started maybe five years ago with COVID. And when I look back at kind of COVID was the sort of before COVID and after COVID, but um you know, CO started a range of I think cultural events that have been very difficult for not only just everybody, but game developers in in general tend to be pretty um thoughtful and um you know, empathetic folks. Um, but you we had COVID and George Floyd and January 6th and Ukrainian invasion and all of those events really um um made it more and more challenging I think to develop games. We've had now market changes uh recently with pullback and funding and and downsizing and stuff like that. And so um it's really one of the things that I've noticed is um as we go to GDC and DICE this year is kind of like the last

kind of uh big market downturn in games. we're kind of seeing who's who are the players in the industry that are really here for the right reasons, you know, and you're looking around the table and it's like these are folks who have dedicated their entire lives to making great games and making great experiences. And so, um, uh, for us, um, that's been heartening to see those who are those who are still in the game, those who are working hard against this stuff, um, and helping each other out through this time has been really, uh, really important. Um, so for us, you know, um, looking at a lot of game developers, they're having to do a lot more with less. Um, they're having to, smaller teams are having to punch way above their weight. Um, and so the discovery is really how can we outfit teams with tools and tech? um how can we leverage things like AI to allow these creators u to make amazing experiences but do it at a at a scale that's more in line with what um sort of the funding apparatus and the publishing apparatus is allowing us to do. And that's where we really felt like this is our time to to be able

to contribute in a way that could help help the industry through through this. And we thought and we think that, you know, this idea of feedback management, connecting with players, um arming teams with information that they can make better decisions is really a um a really worthwhile uh type of mission. Yeah. And it seems like it's a very important function within a game studio. So, in theory, it seems like it should resonate with most game studios, but from the folks that you reach out to, what what would be the biggest push back? Why would a studio not use LiveAware? That's great. there a lot of always have ex yeah lots of um I guess a no matter who we pitch to I think we we find that folks are um engaged and they everyone's having this problem no matter you're at AAA all the way down to indie same same exact pain points um I think right now teams are um many teams are running kind of um there there's uncertainty in the

market there's uncertainty with the amount of capital they have folks who raised in like 2020 2021 are getting to the kind to the end of that that funding and new funding is is hard to come by. So teams are having to rush into launches and things like that. Um so spending additional money is is tough for a lot of teams. Um a lot of times introducing a new team on a development with a development group can can be challenging. There's a lot of philosoph philosophy and inertia on established workflows that need to be overcome. Um so that can take time. Um and devs are busy. they got they have looming milestones. You know, there's always something around the corner that's can take higher priority on something like this. So, our job is to make it as easy as possible to be like, "Try this thing." What we find is when people actually give it a shot, we have a a one month free trial. Yeah. They're like, "Okay, man. This is saving me so much time. How can I how can I get in?" So, that's really been that's really been our goal. Okay. All right. And so what what's I I I think you alluded to some of the things when we talked about the geo tickets that might might be coming up, but are there any major upcoming

milestones in terms of capabilities or features and how are you thinking about the future of Liveaware just like further further down the line? Further further down the field. Well, the thing I'm most excited about is um one of the things I'm most excited about is our new uh pay as you go uh plan. So it's completely self-service. a team can come in to the website, sign up, get a workspace, get going absolutely free, and then uh only pay as much as they use. So for us, this is really significant because we see a lot of smaller scale teams now um who could really use a a product like this and just need a a method to do that that's affordable to them. So for us, that's a really huge milestone. Um on the feature side, we're really um very passionate about this idea of multi- channelannel ingestion of of feedback. So we're we're building a new channels, new wherever feedback lurks, we're finding a way to get it into the platform and get it uh so that it can be counted as part of the information that's available for devs. So um we started with a dev uh discord

in just pass. So you can there's a bot you can apply uh to your discord uh set the channels and set the time frame and it'll send that information over. We just have a path now to be able to analyze YouTube videos and we're we're looking to continue to multiply that so that no matter you have survey data, you've got steam reviews and wherever your feedback lives, you can get it in. Um, and I think that's going to be a real gamecher. Um, uh, the insights dashboard itself is so powerful, but it's continuing to refine that in a way that makes it so that individual team members highly personalized data comes to you and does in a very digestible way. So that's a a big innovation happening on the on the dashboard side. Um where this is going I believe uh liveware is going to be an indispensable tool for all dev teams. Um that is something that people just come to expect. Um it works seamlessly with all of your dev tools. So you don't have to contort yourself or change a your uh telemetry system or change any of your product management tools. We just we

just integrate and fit in. Um, and it's something that people come to rely on. So that in the future, you just can't we can't imagine a world where we're operating without that, um, um, that visibility into into into how real people are experiencing your game. And so, um, I think we're on a great path to toward, uh, toward achieving that. All right. Well, Sean, that was the last question that I had. So, maybe just to wrap, do you have any final message for the audience or any words of advice or anything else you'd like to end on? maybe something I missed. One message I would have you know we talked to a lot of teams and it is a very challenging time in the industry. Um, but what it's great to see is so much innovation happening still. Um, so many people dedicated to the craft of games and I see so much happening with cross team and cross company support. And that is really powerful to see in an industry like ours that the the real passion for this come out. And so, you know, if it's anything, it's a words of positivity and and hope that like we're we're going to

come out stronger on the back end of this. And you can already see it with some of the great um uh the great products that are hitting now. Um, if you're interested in checking out Liveware, come to liveawware.io. Um, you know, hit us up for we'll give you a quick demo. Uh, talk through your your workflows. You can sign up for a free workspace and get started right away. And and, uh, yeah, happy developing. All right. Awesome. Well, thank you so much for your time and insights and good luck with Liveware, Sean. Thanks, Joseph. Appreciate it. All right. Bye. Take care. [Music] [Music]