Bayesian Thinking for Startups with @SebPark

hey everybody i am here back with seb park who is the co-founder of infinite canvas as well as a venture partner at bitcraft now seb you had talked to me before about a really interesting concept about basically how do you adapt your company's strategy practices in decision making based upon new information and you had a concept that i think you're calling process-oriented thinking or bayesian culture maybe you can clarify that for me but could you first start off with kind of like the high level perspective on what this concept is and potentially the types of applications that you imagine it to have for sure it's it's one of the coolest parts about startups in general is that we're exploring these new unseen unknown terrains and so especially when you're working in

industries like web 3 or use generated gaming or the edges of gaming in general across different regions there aren't a lot of examples as to why people should do one thing or another and so often the job of an entrepreneur and the founder is to figure out the strategies and the tactics to do so what's strange and i think one of the things that people make huge mistakes around is that they think they'll figure out one thing and that's good forever that they're they understand this new loop they understand this new approach and we're not good to go and infinite them until the company ipo as it goes become successful but that's not the case is that when you are operating in new environments you have such a imperfect information set that every new piece of information you receive should be dramatically changing your confidence levels as well as your approach both on the strategy and tactics of your business the the best way to like think about this from an academic level is something

called bayes theorem or bayesian thinking which is effectively the conditional probabilities of events changing as a function of the amount of knowledge you had prior to the event occurring what's interesting about startups is you don't have a lot of prior information so the new input the new x input it should have a much larger effect on how you view your strategy or tactics then let's say you're a google or a activision blizzard or a company that has like such an established practice that for them because of their history every new piece of information is a smaller effect on their overall strategy or tactics and that i think is one of the coolest things that great founders i have worked with do inherently without even knowing the theory behind but something that we should like try to teach everyone in terms of how you operate your company's strategy and thinking right and maybe i could just react to some of what you have talked about right now said in in just for the audience just to kind of clarify a few things as

well so bayes theorem meaning conditional probability just as an example for our audience so like and correct me if i'm wrong but uh so that would be for example if you were to look at the potential i know the potential for you to die in any spec specific year but then depending on your age right so like that would be the conditional probability your mortality rate based upon but you know if you're 15 versus 25 versus 80 would be different and so what you are saying is that in a similar way if you think about a specific business practice your uh you know the the type of game that you're working on the kind of feature that you're working on or things like that that that should be conditioned upon different information that you're able to get from you know from the market for example is that yeah yeah yeah and i think just to

give a little more nuance there this is stuff that we as humans do inherently but we're just bad at estimating it verbally or visualizing in our head okay but to think the best way to think about it and i love the poker example right which is if you've played let's say you studied some poker and you played one hand of poker and you won or lost money on it it's just hard to tell probabilistically whether you're good bad lucky or unlucky if you've played ten thousand hands of poker or ten thousand hours of poker and you're losing lots of money like a hundred thousand dollars a one-two poker there's a chance you got unlucky but there's like no chance you're good there's like no chance probabilistically given the information we have that you're particularly great at poker and in the business sense it's very similar in hey if this practice is doing x well and the next time you do it it does why well and y is larger than x that's a directional informational gain you've gotten

but if it's been doing well for like 10 years and y is now lower than x you don't suddenly blow up the company you like figure out hey like is this a is this randomness what's happening here whereas if y is the second input you have where the only historical information you have was that one prior input you should consider a wise value to be much higher because of the potential new world you're exploring right yeah and maybe maybe another way of describing this for our audience and again feel free to correct me if i'm wrong is just similar to kind of like the the way in which development teams have shifted from not everybody but some development teams have shifted from a waterfall methodology to an agile methodology where you're basically developing in smaller increments and then at the end of every increment you try to take new information and then and then adjust your your development based upon new information at every increment

so similarly you're saying for your business strategy for any kind of decision making to try and adapt to new information that's coming from the market and then to adjust accordingly and so then if if i could ask examples of like the kinds of decisions would it be things like features that you put into a game would it be you know more design oriented or would it be things like okay given idfa deprecation here are you know that's a new thing so then how do we adapt um maybe you could provide some specific examples of of of how to to apply this um bayesian thinking or process oriented thinking to decisions or strategy yeah so i mean i think a great example of it will just be like company focus over the course of time right kevin quack wrote a great piece about this i think a couple years ago on why figma wins where he describes how they

started as a cloud tech company then use that loop to find out like hey like with this information that we have now it's clear that our targeted audience ought not be consumers but rather enterprise sales like enterprise is a better use case for figma given the information we have so that's like focus on enterprise sales features and then now we with this information and with the stuff we had before on the cloud infrastructure like hey network effects are actually really easy to achieve here let's focus on that next and that sort of type of updating based information from a game feature standpoint i think there's a really fun example if you are creating poke if you have pokemon as your game you have that core game loop of your catching pokemon battling getting gym badges and one of your new games after 20 years one of the new games of the gym dials underperforms you shouldn't suddenly throw away the core loop of catching pokemon and doing

gym battles we have 20 years of experience that this has been an incredibly valuable and highly selling game you don't need to change that right away whereas on the flip side if you are a web 3 games company and you are one of the you are explicitly you've only been around for like if you're actually been around for three years but for everyone else you haven't been around for more than like order magnitude of months right and the first game goes to zero because for some reason your game gets about like a thousand players and your play to earn model no longer works and it starts to break down well you don't have the 20 years of pokemon's experience to be like hey this might be its jitter in our model you have your quarter this new information that the game doesn't work at 1000 should be taken and be like okay in that case we should switch up something and dramatically change how the pte or some other ecosystem will work here and then move from there yeah it's no super strange as a result okay yeah one other i was just as you were

talking i was just kind of thinking about the other way of thinking about bayes theorem for example in terms of the conditional probability and i think something that in particular companies that are more audience oriented are good at is really trying to take the context of the conditional probability part for example so if you're trying to develop features for a game i think the companies like right what they're good at is understanding the player so what are the features for this game given conditioned upon this player type being our primary player target for example and so um i don't know if that's how you think about it but i i do think that that's something where as an industry we could do a better job of trying to understand well you know it's not so taking the situational context into mind taking that conditional probability aspect of whether it's um for this game given this

audience and maybe for web3 it's like you know because you not only have a game audience but you also have like in you know you also have a crypto a rich crypto audience buying stuff based upon speculation and so like then i'm trying to understand who are you making the features for who is your ultimate target and then base trying to understand how to develop features given that um but i can kind of see how that kind of thinking that the conditional probability part could be generalized to more more kinds of decisions for sure for sure and i think one of the the cool parts about this thinking structure is that people often think game developments conflate strategy and tactics right you can have a strategy and your taxes can look very different from that strategy and they often do and so what's really cool about working through the process of building out a culture that thinks this way is that it helps every person be it the founder the middle manager the the like uh the entry-level engineer

start to ask some of these questions and thus develop the tactics or develop the culture of being like hey let's iterate faster because based on this information and then make those decisions for with the i think the hard thing and this is one of the things that i think should go so much the pokemon example but also fact of the blizzard and some of the older companies is that the reason why startups succeed is not necessarily purely because larger companies are stale or archaic or unwilling to iterate it's that by definition when you have so much more data and so much more history of things working you should in fact make decisions slower that actually that is the con like the conditional probabilities that you have are just a different set than that of a startup and i think that is like the interplay where it's like yeah like your best practices from your you know your riot games or your activision blizzards may not work

necessarily as well on these new frontier ecosystems mostly as a function of just a difference of your understanding of your audience your understanding of your players you're understanding your target ecosystems i think that's really cool and i think that's why startups are awesome but it's also one of the reasons why it's often hard for people to make the jump back and forth consistently a lot of people who are startup founders have a hard time going into full-on uh enterprise industries and vice versa in part because if you spend your entire career learning one style of thinking it's hard to then like deviate back and forth and the ones who do the best job are the ones who i believe already inherently understand even without necessarily the vocabulary describe it this idea of conditional probabilities this idea of bayesian updating and this idea of then building out your process and your approach to game making and company development as a function of this yeah and to your point about the large companies right like the conditional probability of making a huge change to

an existing successful product that cost is dramatically higher than a new startup with a new product just doing some crazy just then their cost is like with with few users or no users is basically zero yeah but opportunity costs are certainly a huge factor into it as well right and just speaking to this notion about startups and i i do think that to your point like one of the the key advantages of startup is going to be speed and um and making decisions based upon imperfect or lack of information so then i do think that being able to integrate new information and to apply that well could be a very important strategic advantage for startups or new companies and if i'm understanding like the basis of what you're calling let's say bayesian culture is what you're calling it then it seems like there's kind of two aspects to it right which is

one taking in and adapting to new information and two understanding your set of conditional probabilities and how that impacts how you make your decisions or strategy or things of that nature but taking it to a more practical level if you were to give advice to a company okay so we we have this this kind of high level concept around um process oriented thinking beijing type of culture or thinking or approach but now on a practical level if you were to tell a startup ceo or founding team okay now that you understand the concept here are some things that you can do practically to try and apply this what what would you what would be like a good first step for a founding team to try and do to kind of integrate this thinking yeah i mean i think the first step is to your point there's two parts there's understanding conditional probabilities which is hard but the thing that you should first start with is just writing out your

priors writing out right in like like type it up write it on a piece of paper but be honest about here's what we believe and here's why we believe it at this moment in time like once you have that base then you can circle back and do it over and over again so my co-founder tal and i we run a remote company but effectively once every eight weeks we got together for a week and we got together not for necessarily anything operational just to go over our priors it's just a meeting where we're like hey this is what we believe when you set down the company this is what we believe six months ago this is what we believe three months ago let's just write our hypotheses what's new information how much should it be changing each of our philosophical approaches in terms of the company moving forward so that i think is like the easiest first step to go about think of it as founders from yeah i was just about to say so just uh just to try to summarize that for for our audience and for me like so maybe

one thing you could do is every call it two months or eight weeks you write down all of your assumptions maybe assumptions and key risks about your business you try to understand all of the new information that has occurred in the last eight weeks and then as you have written down your assumptions and maybe for us were really key risk driven we would look at those and we would try to understand not only the impact of new information on all of that stuff but in addition understanding whether the way that we're thinking about conditional probabilities against all those assumptions and key risks is correct or not right right it's important for sure because there are two macro mistakes people make in decision making some people over react to new information and some people under react to new information right and the the bayesian approach helps you figure out the proper response rate to the information which is depending on magnitude and depending on how much existing information you have you should change up your response rates and the

best way to dial that in is to just write it out because it will help you disentangle the general human inability to think through conditional probabilities because humans like everyone from doctors and mathematicians are all bad at it like we're just all horrendous at it and the easiest way to like solve for things when you're bad at it at least in my opinion is just to write it out so you have these points of time to do so right and then the second thing i think is really important from a practical standpoint is then applying that culture to the greater company right and i think that actually comes out in like i said culture which is i think a really interesting nuance here an example of this is that we have a release manager who uh you know we we gave the ability to pull the trigger on the new game's release and in the post-mortem we were like hey this was a bit conservative you could have pulled that trigger you know a week earlier two weeks earlier well now we

have one piece of information and so you write that down and then the next time this when she has another chance to pull the release strategy we'll see if she or is she very aggressive this time is she even more conservative less conservative and we can take that information to then judge like hey we need you to adapt faster or we need you to like take what you believe subtract three days or take what you believe at three days improve your own process and be okay with being wrong insofar as that we're seeing fast iterative improvements in terms of decision making moving forward which then allows her to actually go from being like a junior release manager to a mid-level to a senior release manager in a much faster time period i'd imagine than your typical like years of progression right and maybe the one other question i can ask you seb is that in terms of like you know whether it's a practice or a process it's easier to kind of document that and to roll it out

and then as you're doing the specific work like you know whether whether it's a development process or something like that you're you're kind of engaging with it on a very regular basis and you know there's a lot of people working on that kind of process together so it's easy to install but when you're talking about a way of thinking in terms of like the bayesian approach how do you how would you think about or how would you try to and to your point you know disseminate or scale this thinking across the organization when it's more abstract or more conceptual it's it's hard and it's something that part of reason i want to talk more about is i'm sure people have even better ideas our strategy and my strategy has been taking almost a uh evangelical approach to it which is just constantly talking about it just constantly having a conversation being like hey here are my priors here's what i believe here's how the probabilities are

changing here's i think why we should do this outcome and just doing that over and over again until the next person does it the next person do it and that's why i keep saying the word culture because i do think it's a culture driven environment that you can infect this type of thinking to the entire team such that it's no longer just the senior management team it's not long just middle managers it's people who come into the company realizing this is the part the culture is we are going to be very quickly updating all of our information in order to make this thing happen i've seen a couple of other approaches right i've seen uh you know when i worked in in north american basketball at the mba and then the esports side we just had a lot of people track data on ability in zero sum ecosystems like basketball or in esports it's actually pretty easy to show that people are bad like it's like hey like here's here's literally the data sheet that says you thought it was going to be x but it was y yeah and that was super helpful we actually had all of our out

of our scouts and managers and coaches votes on what they thought the outcome of every game would be ahead of time and we tracked it over the course of multiple seasons being like look you're clearly overestimating these guys or underestimating these guys let's figure out why let's update your priors based on this information we have conditional towards counter logic gaming or conditional towards evil geniuses right and so it's so much easier to do this and this type of process is already being done at the best front offices in sports because they have zero sub environments where they can see their wins and losses over time it's certainly harder to implement i think in traditional business but i think one of the best learnings that we can bring over and i certainly also think it's a vocabulary issue because one of the coolest things i've seen and talking to business leaders over the last few years is that the best ones already do it they just only hire people who also do it and so it's they're like oh they're like oh

this person doesn't make good decisions what's less of a they're not a good decision maker in the vacuum it's that they don't respect the condition probabilities and so they fail over a period of time and so therefore they move on to the next guy or the next woman who and they tried them out and then over time over 20 years now they have a team that's actually very good at it but they had to like cull large percentages of the team yeah i think that's really interesting how you talk about grading internal people i it really reminds me of ray dalio and his book principles where he talks about how they have a believability weighted index for how they do decision making where they actually grade people based upon different aspects of of their business and then based upon the decision everyone kind of votes but then they all have different kind of weighting based upon how good or bad they are at something but i do think it's um it's interesting because in the games industry that's so hard to do and we don't we don't do believability weighted um decision

making at our company just because you know the like like in in finance you have a lot of transactions right you invest in a stock it goes up where it goes down or you have a bunch of games in the nba you win or you lose and you've got a bunch of these transactions to give you all the data points but then for a for a game like how many games does someone work on in their career right you know with like two two to three to four year life cycles right then it's it's like the the number of transactions is so low that uh maybe the other thing i maybe i'm going too far afield right now but the other thing that would be interesting is how do you evaluate or judge people based upon proxies rather than specific outcomes given the long life cycles within our industry it is so hard like and that i think is why it's the the the revealed and stated preferences for people just so different because when it's hard to judge i think

a lot of people come to conclusion it's not worth measuring when in reality the conclusion you should come to is like we should try harder yeah like we should try to measuring more things see if we can find out things uh what i'll i'll give a fun anecdote which is we've been iterating and empty canvas our release cadence as a function of inputs like testing as an input working as an input feature development and future ideations input and the thing that we've seen in the data and that's like unequivocally the most important thing is how often do you update like it it matters even less for everything else the two things that matter the most are number one how often you update and number two how often do you break everything if you don't break everything and you update frequently it's just massively positive and it's not necessarily because the updates themselves are positive is that the feedback loops get smaller and smaller and so people are getting more information into their system and they're making better and better decisions because they like now have

better inferences and so the the external inputs are a little bit lower and lower uh it's amusing to me now because you know modern machine learning basically has to abstract to the way a lot of these things are they and especially with reinforcement learning you're trying not to bias the inputs you're trying to just like let the inputs run and tell you what's right i think especially in our industry where we're spending a lot of time trying to figure out new games and spending two three four years on them we're going you have to trust yourself to bias it a little bit and then like move it in that direction but certainly i'm sure someone's working on a really cool reinforcement learning bias or move process to make new games but not quite there for at least for infinite canvas okay and for our audience if they wanted to learn more about this are there any resources out there or is this just kind of like a is this like a sub you know concept or have you written about this at all yes i've written a couple of times about it and if you follow me on twitter at

southpark like that's a decent place to see it but i would also just recommend reading a lot of the behavioral economics literature out there that was my undergrad area of study it's something i really care a lot about but i think for example daniel kahneman thinking fast and slow is a very good book for a lot of people i think ray dalio's book has a lot of bayesian updating and weighting that you see and a lot of that stuff in there i'd also just honestly and it's gonna sound ridiculous i actually think for a lot of people if you're not a reader find some youtube videos like i kid you not like if you just go to youtube and type in bayes theorem and watch the top 10 videos and 10 different youtube educators trying to explain this incredibly complex topic to you it will help a ton in terms of getting you into the right mindset where it clicks and then you can make good decisions around it yeah so for the audience if you haven't subscribed make sure to subscribe here well we'll be sent back to talk about more topics

um okay so i don't have any other questions do you have any last message for our audience or and also besides uh you know following you on twitter at seb park is there any other um way people can get in touch with you is that the main mean way that's the main way i think i have like an email on my website if you want to go look it up feel free to do that just sebastianpark.com the one thing i will say is that this is still for me at least we're still iterating on this idea and so what there's so many unknown unknown things we don't know we don't know and would love to hear from people i like how people develop companies one of my favorite conversations of all time was this conversation i had with jens one of the gps at bitcraft where we were just arguing and debating the rate at which you scale companies from like 20 to 50 in a 50 to 100. and it was a ton of fun because i have my priors he has his priors and then

you come up with better systems by merging them together and figuring out conditional to these xyz qt thing what do we do and i think similarly if you have thoughts on how you've done it how you're making decisions or how you're trying to expand it doesn't have to be this idea but any ideas part of your culture i would love to hear it in part because i think that'll help us all improve our thinking around it right all right seb well thanks so much uh for jumping on again i hope to have you on again to talk about more interesting concepts like this one and for our audience we'll catch you next time thanks up cheers [Music]

you