Physiological Effects of Crunch: A Look at the Science

[Music] hello all right so the story behind this talk is that ever since the ea spouse letter i've asked everyone i could think of like when they're talking about crunch like is there actual any actual scientific research showing the effects of crunch some people glorify it some say it's unfortunate but necessary some say it's terrible and detrimental but when i ask anyone in any category okay is there any peer-reviewed research on this it's crickets right or someone swears i saw a study once but when i asked them for a citation suddenly they can't find it so i went ahead and did the research myself so you don't have to these slides will have numbers in them i will give a bibliography at the end and a link to the slides so you can check them out because i think we can do better at this point than just gut feelings

so just to lay down some boundaries what are we talking about when we say crunch this doesn't have an industry standard definition for this talk when i say crunch really i'm using it as a synonym for any amount of unpaid overtime past eight hours a day or 40 hours a week now this can be short it can be long it can be light it can be brutal i'm talking about all of these we do usually talk about this in the context of salaried employees where extra hours are not compensated there's a that's significant because there are federal laws requiring at least time and a half compensation for overtime specifically to make it more cost effective that if you're going to make a habit of this you know you're going to have to pay for it it's more cost effective to just hire more workers and work them less hours but salaried workers are exempt from those laws so on the surface it means that a company appears to just get more work out of other people for free there are some edge cases to that but they generally don't apply to the game industry the other term that i'll say i'll use in this talk is death march which is just

our colloquial term for a situation where a team is behind schedule they're asked to work longer hours to catch up that makes them even further behind they work even longer hours with crunch getting worse and worse over time and the project getting further and further behind until the entire project just fails causing high degrees of burnout and other problems so that's kind of like one extreme end of the scale but we'll be looking at the entire range so one thing that i did not include in my definition is calling crunch mandatory unpaid overtime that is usually how it's used but this has actually been studied and whether overtime was strictly voluntary with a worker putting in more hours because they just wanted to and were passionate versus it being required by management the issues arising from crunch that we'll be discussing here actually are the same either way with one small exception i'll get to at the end of the talk so if you work long hours you will have certain fatigue effects period regardless of the situation that put you there so i'll start with a study that was

measuring cognitive function not productivity but just basic simple tests like the ability to memorize and repeat a word list or do simple math that kind of thing and it turns out that your ability to process things mentally is not actually constant throughout a work week it starts off low gets better in the first part of the week as you get into the swing of things and then decays until you get a break for working hours up to about 25 a week an increase in working hours has a positive impact on cognitive function you get into the swing of things you get into a rhythm after that cognition slowly declines and this study actually looked at gender differences and found none so this is basically the same curve for everyone once you get past 60 hours your cognitive function is actually worse than someone who wasn't working at all so there is actually scientific evidence that mondays suck but if you talk about wednesday being hump day that's actually not supported by science you peek early thursday we knew that already since wednesday since most people actually do it on the weekends

so um this was actually consistent with a number of other studies that didn't look hour to hour but did confirm that longer hours do show performance degradation when averaged out over the entire week per hour i'd also like to draw your attention to the bottom study here there is a difference in effects of crunch depending on the type of work done knowledge work and creative work are actually hit harder than things like rote repetitive tasks while i would love to give you a spreadsheet with just a bunch of numbers to optimize to say here's exactly what's going to happen at hour number 48 um there's just too much variability between people between different kinds of tasks and many many other variables so this as much as you i would love to spreadsheet this out it's not actually possible so now let's talk week to week so suppose you just work a 40 hour work week no overtime at all and we just track your weekly performance instead of hourly yes you'll have some good weeks you'll have some bad weeks going too many weeks without time off can degrade performance you do get more experienced

and skilled over time so your performance will increase like year to year but over the course of say a few months we can treat this as sort of constant and under crunch uh conditions there is a degradation of productivity over time that continues to go down the longer that crunch uh happens now there isn't a single study that measures this overall because there's just too many variables but there are lots of studies that show this in different narrow contexts and they all tell the same story so this is a repeated pattern across a number of fields countries and studies that comes through consistently um this is an example of what an actual productivity cr oh i also i drew this as like a straight line down um it's not necessarily um it's not necessarily in a linear shape um so this is an example of an actual productivity curve this is not from game development it's from but it is from real data collected over 10 years from a processing and manufacturing plant um calculating productivity as the time it takes to do short-term tasks during crunch compared

to how long it takes those under normal baseline conditions um so you can see like you do get a big fall off at early on then it kind of goes a little bit back up as people go into the swing of things then it drops a bit uh more you know that's in a non-game dev context but it's an example of what a curve might look like so it's not linear um and yes this can actually go negative and the reason is because the more fatigued someone is the more likely they are to make catastrophic errors so what zero productivity looks like is you spend three hours fixing a simple bug and you create another bug in the process that's zero productivity um negative productivity is you're tired you see a deprecated file in the git repository and you go to remove it to clean things up and by accident you actually delete the entire git repo and cost the entire company like a person month's worth of work uh because a bunch of people have just lost their most recent check-ins and everything had to be restored from last night's backup that's what negative productivity looks like and this is what happens during a death march the project is behind

schedule people work longer hours but if it happens for too long this can actually put the team further behind if you're unlucky because someone just happens to make a really big blunder while they're because of the fatigue effects and if that happens too many times then the entire project just ends up collapsing under its own weight there's also some amount of recovery time that it takes you to get back to your original baseline productivity after crunch we talk about comp time again this this isn't necessarily linear either i'm just drawing it here to show that there there is some recovery period um you know so you can see very clearly here that a short-term crunch followed by a recovery period is actually a net productivity loss compared to just working continual 40-hour weeks short-term sometimes you need this to hit a milestone at the end of this week with the understanding that long-term is a little bit less efficient but sometimes you need to squeeze a little bit more hours into a little bit less time in the short term that can be fine

long term there's really no science i could find showing any productivity benefits of doing extended crunches in fact just the opposite it's less efficient you get less done than just working 40 straight um so there's there's really no justification for crunch beyond like a month or so um now it seems intuitive to say well if people people work longer hours you get more out of them but that's just not supported by reality because humans aren't machines we don't have a constant output per hour we get tired so so it gets worse when you're not looking not just at productivity but the actual effects on human physiology so this um this is from a pretty big study of over 10 000 americans over 13 years they got about 90 000 person years worth of on-the-job data which let the study authors control for age gender occupation industry and religion and within that we're willing we're able to show a very clear linear relationship

between hours worked above 8 a day or 40 a week with greater incidence of injury or illness so yes you don't typically deal with workplace injury in game development beyond a paper cut but this does include things like heart attack stroke stomach ulcer and other medical conditions that can either kill you or at least take you out of the game for a while and this is consistent with two other studies shown here that showed a link between overtime hours and heart disease stroke and injury hazard rate another study here looked at a survey of almost 4 000 canadians and asked them lifestyle questions and also whether they were consistently working short normal or long hours and basically found that you've got increased chances of mental health issues decreasing healthy habits like eating well and exercise and increasing unhealthy habits like or starting on healthy habits like smoking drinking or illicit drug use if you are consistently working longer hours you're just more likely to have those problems if you're consistently working long hours and this is also backed up by

three other studies that show increased incidence of anxiety and depression disorders increased as well as increased anger and hostility at work on a mood scale which then leads to greater incidence of interpersonal conflict on a team and resulting lower team morale now there is an actual mechanism for why overtime hours affect mental and physical health and that mechanism is an increase in stress there is there is scientific evidence that long working hours is related to stress because and that's not surprising that long hours are stressful it's not surprising that chronic stress is known to cause all kinds of health problems um i do want to be clear when i say stress here i'm not for the purpose of these studies i'm not just talking about like self-reporting or observational stress i'm talking about biological stress um there are numerous biomarkers for chronic stress in you know in in blood studies including it changes your levels of cortisol catecholamines glucose hba1c

triglycerides cholesterol prolactin oxytocin and a whole bunch of other things so when i say chronic stress here i'm actually talking about alterations in your blood chemistry and there's a vast body of literature that links those biomarkers with a number of physical illnesses mental illnesses and cognition deficits i could give an entire separate presentation just showing biochemical pathways that get you from stress to poor outcomes so this when i say stress i'm not saying this in a way of well we humans fold under crunch hire people who are tough this is a physiological response to stress that you cannot get around with just you know toughness and bootstraps um now you some people might be thinking well okay these are just the sacrifice these are unfortunate but they're the sacrifices we have to make in order to make good games um i think we all know that game development is a bit more messy than that you can't draw a clear line between a single variable like crunch and the outcome uh but there was a study on gamasutra back before it was game developer specifically relating people's self-reports of whether they worked overtime or crunched on a project

versus the project's metacritic score on the left their financial roi in the center and an aggregate score that the study authors calculated from a combination of factors on the right and as you can see the data is all over the place each dot on these graphs is the game's score versus how much crunch there was some games did a lot of crunch did really well others did poorly some ga same for games with no crunch but spread across a large number of games spanning the range from aaa to small indie there is a very mild negative correlation between working overtime and making a successful game um yeah and if you thought that was the worst of it it actually gets even worse because if you're working at really long hours these hours have to come from somewhere and the chances are pretty good they're going to come from your sleep and so there is a there was a key study i found which showed that people who work more than 55 hours a week compared to those who work 35 to 40 are twice as likely to experience shortened sleep hours almost four times as likely to

find it difficult to fall asleep in the first place and twice as likely to wake up without feeling refreshed and repeated exposure to long work hours greatly increased those odds so that's where it starts and it gets worse from there so there is actually a strong correlation between overtime and sleep deprivation so let's talk about sleep most humans need somewhere around eight hours of sleep per night to function optimally there's some variation here we follow along a bell curve but if anyone here is thinking that yeah well i can function just fine on six hours of sleep i'm one of the lucky ones at the lower end of the bell curve odds are actually extremely high that you're fooling yourself and you really can't and honestly you are none of us is in a position to actually know how much sleep is optimal for us unless you've had this measured in a lab in an actual sleep study because as we're about to see your self-perception about this is not great under normal conditions and it gets even worse with sleep deprivation so going without sleep it turns out physiologically is almost indistinguishable from drinking alcohol other than the effect on a breathalyzer test this is literally what sleep loss

does it reduces motor function reaction times ability to think judgment all kinds yeah all the things that getting drunk make you do um so yeah there's actually studies that show like exactly how much sleep deprivation versus how many shots of alcohol it takes so you know so the next time you hear someone talk about like pulling an all-nighter or staying up late working on a project just mentally translate that uh to drinking alcohol right you know you'd say if someone said oh that ian he came in and rebalanced our entire game economy overnight last night he's such a hero he does the all-nighters a lot right just mentally translate that to oh that ian last night he got totally drunk then he balanced our game economy he's a hero he does that a lot right that that sounds crazy when i say it like that but if you say stayed up all night that is really literally what is happening here um and kind of like being drunk right if you've ever been in a bar and seen someone who's like falling off their stool and saying i'm fine give me the keys to drive right and everyone else is

like dude you're you're not fooling anyone um right you you like being drunk you are always think you're less drunk than you actually are and being tired is like that too right one of the first things to go when you start getting tired or drunk is your ability to self-assess um how tired or drunk you are so however tired you ever think you are it's probably worse it turns out this is actually what the second wind is if any of you have experienced that where you're like late up at night and maybe two or three in the morning you suddenly reach this point where um where you get the sun rush of energy right that's actually the point where the part of your brain that detects how tired you are is too tired to function and has shut off um it doesn't feel that way to be sure you would need to measure this in a lab with cognition tests but if you do that what you find is that the second wind is pure perception your actual abilities are follow a a progressive smooth slide downward and the point at which you say i'm having my second wind does not show up on that

graph at all also this surprised me when i found this if you pull an all-nighter and then take a full day of rest you are still not back to baseline so this can be extremely dangerous and inefficient this was from a study where experienced night shift workers had mental performance measured at baseline after a night shift and again after a full day of recovery and found that even after a full night's sleep and a full day off test subjects were not recovered all the way back to their baseline levels now i know what you're thinking oh i know a cheat code for this in some ways yes caffeine does have effects that counteract the effects of being tired but there are a whole bunch of caveats and limitations and the most important one that most people are not aware of is that you reach caffeine tolerance shockingly quickly it takes about a week meaning that one week of regular caffeine use and that amount of caffeine use is no longer giving you any kind of stat bonuses at all it's just bringing you back to the baseline you were prior to starting your use of

caffeine sorry coffee drinkers so if you don't use caffeine normally and you need a boost for a short period of time that is covered under caffeine like one week or less you can do this to make up for some of the adverse mental effects of short-term crunch as long as you stop using it immediately after that week and get back to a non-caffeine baseline for anything longer than that no caffeine will actually not help you so you might say okay ian is there any good news here is there anything you can do to improve your performance at work and yes um yo if you want to optimize time whether you're in crunch or not short attention breaks uh so yeah a 10 second attention break stretching every few minutes taking a five minute break to walk around every hour that can refresh your attention and keep you fresh throughout the day also taking at least 20 minutes when you take a meal to just eat in a relaxed and slow way actually improves the nutrition of the food and improves digestion and relaxation and on top of that of course any

techniques you can use to reduce stress because as we heard you know the the main bad effects of extended crunch are from from levels of stress so stress reduction of course is a huge field that could be an entire talk of its own but if you have noticed for example lots of companies offering things like yoga meditation and mindfulness classes that actually does have a solid basis in science for positive health outcomes due to stress reduction it's just that it comes across as a bit disingenuous when the company offering stress reduction programs is also the primary cause of your stress in the first place um another thing that uh was found across multiple studies and this is the exception that i that i was referring to at the very beginning is that allowing people to choose their own work hours so allowing people flexible time over their schedules whether in crunch or not has basically nothing but positive effects under normal baseline conditions or crunch conditions so that is also something you can do is letting people set their own hours helps

with job satisfaction health well-being productivity efficiency just basically everything across the board now another thing you may be wondering there may be an elephant in the room here that i want to address which is you're saying like well ian you're co-founder of global game jam everyone knows that game jams take place over a sleepless weekend are you the biggest hypocrite getting up here and telling us about this so i will say game jams in general and global game jam in particular do have an image problem a lot of people think that these glorify and encourage crunch culture i have actually talked with global game jam leadership about fixing this and having them make some very public statements about crunch so far that hasn't happened i will keep pushing on it but i do want to address this right now so first off as we've seen the worst health and like physiological and psychological effects of crunch happen in the long term not the short term uh with game jams this is time boxed um you know you can't you can short yourself on sleep over 48 hours but that won't have nearly the same effects uh of depression and in

illness and injury that a longer crunch period would have so a weekend jam just isn't a fair comparison to the long-term damage that a six-month death march would do but more importantly than that you don't have to crunch at game jams right co-founder of global game jam telling you you do not have to crunch at game jams right in fact knowing what i've just shown you about the effects of long hours and sleep deprivation i would say that if you treat a game jam as a short-term crunch you are doing it wrong you can work an eight-hour day in a game jam two days on a weekend and probably make a better overall project than if you work two 16-hour days and definitely better than if you pulled two adjacent all-nighters um if you want you can do empirical research on yourself to prove this to yourself game jams are actually a good way to do this relatively safely because they are so time compressed just schedule several game jams that are at least a month apart to give yourself time to recover time it so that you're going to get each game jam equally fresh

and then try one jam where you only work eight hours a day one where you work until you're subjectively tired and one where you go with as little sleep as possible and push yourself to your limit and then compare what's the quality of the work that you did on those games how much did you get done how did you feel during and after each jam just let your team know when you do this on a game jam with a team so they can scope accordingly and if you do that that's not really glorifying crunch it's using game jams to learn more about yourself and the effect that crunch has on you so that you can make better personal decisions when your actual job is on the line and i just want to address also every now and then you'll see a thread where people will post pictures of jammers asleep on the floor or at their desks just keep in mind that this floor is comfy tumbler isn't glorifying crunch it's mocking it this is not aspirational don't treat it as such um so so if you take away nothing else from today these are the points that i really want you to remember know that your productivity per unit time decreases

continually and linearly if you work long hours or you're short on sleep and these effects stack with each other and get worse until you catch up in the short term if you do this long enough that you can no longer catch up in the short term the excess is taken off your lifespan with permanent health damage and if you're fatigued enough you'll also be in the territory where you can sometimes if you're unlucky create more messes than you can clean up and you're actually a net liability to your team if your entire team is under these conditions you're in a death march and even if you're short on and also even if you're short on sleep for just a single day remember that during the time when you are sleep deprived you are operating as if you are drunk so avoid anything where you can kill someone with a mistake try not to drive for god's sake stay off the critical path of your project and there are other things i didn't even get to cover here because i just have a half hour the effects of crunch on family life and relationship stress decrease in job satisfaction those can

lead to increased absenteeism and employee turnover and employees if they turn over are quite expensive to replace burnout is also a factor that doesn't just increase employee turnover but also costs the entire industry senior level experience that leaves so there's more research on that that i just didn't have time for so the next time you see this sign in moscone west uh your first thought should probably be um you know which creative is wasting money at 10 pm probably whichever one is in the office working so um to leave you with some final thoughts uh you might wonder okay well how do we avoid crunch if we get behind schedule then you know that's all fine and good to say this is bad what's the alternative um let's be clear that we don't usually get behind schedule because the engineering team was lazy and playing games all day um or because the designers were unskilled and didn't know how to to make the game right those things both happen a lot on student projects but in the industry we tend to filter those people out when we're hiring professional developers actually tend to be pretty good at what we do

um so the cause of crunch is usually not uh is usually not a skill problem or a laziness problem it's an unrealistic schedule expectations are too high that's a project management fail and even if there is a technical skill problem where you do have say engineers that just aren't skilled enough to build the thing as fast as they should that's still arguably a project management fail because it's the project manager's job to know the capabilities of their team if you estimate your overestimate the team skills that's on the person who did the overestimation so if you do find your project behind schedule with too much work and not enough time to do it the solution is really not work harder or work longer hours because you are just as likely to cause damage to the project as to advance it yes if you're very lucky and happen to avoid random catastrophic errors you might get a little more done but the odds are against you right the solution really here is to in order to maximize your chances of success is to reassess how much you can realistically

get done and if you can't get done what's currently specced in the time that you have redo the specs so that it is possible to get done or knock back the release date to give the team extra time to get more done or hire more people onto the team with enough ramp up time that they're a net productivity gain that one does have some delay so yes there are real citations for all of the sources here these are on the comment slide and individual slide as well as the slide here if anyone wants to look these up for themselves or do more research on this topic you can get your slides here and with that i will take questions we have mics here and here hi there um my name is lauren and um i'm an art student at academy of art but i'm also a college softball player and so you can imagine the amount of crunch i get put under having to wake up at like 6 a.m to do

weights for you know an hour and then play softball for three hours so it it definitely um the hours only come from sleep i can definitely uh agree with that um i found it interesting when you talked about um depression because i in the last couple you know years i've realized my increase in depression and experiencing anxiety almost exclusively when i'm during like when i'm in season and so when i have like a lot of obligations all of a sudden it just keeps piling on but with the increase of awareness from mental health as we move forward into the future i wanted to ask how do you think the work week will change in the future for the betterment of mental health and what is your ideal work week so ideal work week um

like you said you know there's definitely you know you peak at 25 hours so you don't want to stop there right there you do still get more um yes i'm you know i think you know 40 hour you know 40 hours was ideal back in henry ford's time for people working on an auto assembly line doing rote tasks the ideal work uh work week length for knowledge tasks i have not found yet um like specific i did not cover specific things here with with relation with relationship i'm guessing it's probably between 30 and 35 now whether we will get there as an industry when we are very very stubborn about these kinds of things i i'm not sure that i have that kind of of confidence you know at scale but we actually have seen some uh recent things with that like you know some companies experimenting with four day work weeks and not a four by ten like a four by eight um and repeat and reporting positive results from that um so i think like many things in the

industry there are positive things we can do some studios will do them some of them won't and i can only just hope that the um you know that the companies that are doing things better will eventually you know over time make better games and just kind of displace the ones that aren't so but we but yeah game development is chaotic enough occasionally a toxic studio you know in any number of ways can still produce a hit game um just yeah by random chance not necessarily because they uh because they um you know because that's part of the process um you know and sometimes good studios will will produce bombs um you know but hopefully over time yeah the yeah you know it's there's there's a die roll there's a die roll modifier the ones that give positive die roll modifiers are overall going to be you know more successful on average and hopefully we can have that also with a shift in the culture you know as more people understand that that's that was kind of my hope with this talk that it's just you know it gets us past just talking

about anecdotes and feelings and well i think and here's my you know experience and really talk about the science that we know uh so that we can kind of have a more elevated conversation about this topic so thanks for one more yeah i think so that was a great question because it's in relation to mine uh this conversation about sleep it's a very equitable conversation so from a privileged perspective i mean there's folks that are working more have more jobs and things like that so it's great to find from a scientific standpoint the causes of some of these downfalls but moving forward i'm curious what your solutions might be from a process perspective like there's the pomodoro technique and and things like that where you're taking breaks in between your um your work days so i'm curious your thoughts on that and how we can use this information to move forward with companies to recognize that not everybody has the same number of hours in a day and while this information is very valuable i think uh there's some

sets of minorities that will struggle with implementing it yeah yeah as far as far as helping studios to do to do better with this you know that's that's part of why i'm giving this talk right so that you'll have a resource to be able to bring up and point to and say look here's here's what science actually says um you know and and you know again so so that people you know so the yes i mean if you're if you're like a junior designer at a company you have absolutely no institutional power um and there you do not have a say over whether you crunch or not and your producer says all right everyone's crunching it's mandatory right there's there's limited things you can do other than maybe like look for additional work but i want to at least have it so that you know individuals in that situation at least understand this is what's going to happen to your mind and body under these conditions if it goes for long enough and so that you can make a more well-informed decision about do i stay do i leave do i push back which parts of the rest of my life do i protect right i would definitely say if you are in crunch conditions prioritize sleep as much as you can

um because that's because because if you do get short on sleep from that that's just going to amplify all the other negative effects um and make you even more dangerous so yeah and yes you know there like i said there was a slide where i did show um you know some you know small things that you can do you know during the day to improve productivity this is this is under crunch conditions or non-crunch conditions um but obviously there is there are limits right the the effects of an extended crunch will stack and overpower any positive things that you do uh you know given enough time right so it's mostly about just optimizing and then of course if there's anyone here who is a key decision maker who does have a say over you know do we crunch do we not crunch how much do we crunch that my hope is that you cannot use this as a better understanding to understand okay here are the effects that are going to happen to individuals and to teams under crunch conditions so that you can make more informed decisions about when do we crunch you know in the short term because we have a milestone due next week and we're a little bit behind um yeah and i recognize that uh that

it's important to hit that milestone if because if we miss it then you know the entire project gets canceled and our studio closes all right go ahead um but just understand that you will want to give people comp time afterwards to get them back to baseline that it is inefficient to do that regularly um you know and then as far as extended crunches go that's where you should really be looking at your project timelines and your goals and your scope instead um you know so for p you know and for people who aren't in that position who maybe are junior or mid-level um you know if you stay in the industry for long enough and get into positions of power where this is your call just remember this so that when you do have the ability to to make those kinds of decisions on behalf of a team that you're you're making decisions based on you know based on actual science so i think we're out of time there um thank you again for coming i i will be still around and going to the speaker wrap-up area where wherever that is if anyone wants to continue the conversation thank you so much for coming [Music]