What To Do When You've Been Hacked

so uh my name is Dave Weinstein with me is Angela gun and we're going to talk about what to do in the event of a security issue now my background is in development and security including services at scale and I'll be talking to you about operational roles and planning Angela's background is in journalism and Communications and she'll be talking to you about crisis Communications which are not quite the same thing as marketing PR or Community Management but overlap all of them all right bluntly this is a talk about operational and Communications planning and emergency response this is not a technical talk and we're not going to talk to you about how to do security right we are in fact just assuming that you're doing security right please be doing security right so our intent in this talk is to give you tools you can put into action you should be able to apply these techniques immediately and with the resources you already have on your team

I'm going to talk about logical roles in this in many cases especially for small shops people will wear multiple hats we're going to talk about planning and training including how to build playbooks for incident response and how to use those even before an incident occurs to help identify critical paths and resource issues angel will talk to you about the security Community which you're probably not in touch with and crisis Communications and Outreach now there is an important thing I am not your lawyer in fact if you want to get right down to it I'm not actually a lawyer and if you really want to get technical I don't have a degree so Angela has a degree that's the good news um the bad news it's in philosophy so totally do not get Socrates as your defense attorney and neither one of us stated holiday and express last night so you are totally 0 for four now unless you happen to have a lawyer turned game developer on your team the odds are very good you don't have an house counsel and

that's fine whenever I talk about legal roles in the scope of this what I'm really talking about is a person who's wearing that as an additional hat um this is someone on the team who's familiar with your contractual and other legal obligations these may be the terms you have with a credit card company about protecting card information and pii it may be a relationship you have with your sales Channel and distribution Channel or with an IP holder whose work you've licensed and in the case of things like The Children online privacy and protection act actual legal requirements imposed by the government by Statute that being said when you're working on playbooks that involve legal issues you probably do want to consult with an actual attorney brand attorney what we're going to do is we're going to take a look at the operational roles that the model uses many people will fall into more than one logical role especially in smaller companies and that's expected uh one of the things that happens in any small company is people wear multiple hats you usually find that out when they go somewhere

else and you discover that you actually needed two or three people to do the work they were doing to make this work we're going to go over a set of terms and roles and other things that will help frame the discussion and the first one is the notion of a dri or directly responsible individual and the drri model is a little different than what you're used to on most teams either the producer or the lead programmer will make a decision about whether or not an update goes live uh whether something makes it into the build uh whether the build is safe to prop to Beta all these decisions in a drri model are made by the drri this person calls the shot usually a week at a time so if you are the dri you are the one wearing the Beeper you are the one on call if something goes wrong at 4 in the morning and you are the one who says wow this is really really buggy I don't think we want to push it to Beta or even to internal testing or uh friendly testers the drri does not need to be a programmer any member of the team should be able to fulfill this role after all we're historically used to having producers ERS fill fulfill a lot of this

role and most of them are not programmers and the ones that were programmers haven't been programmers as a general rule for long enough that their skills are rusted out anyway as far as that goes you don't need to be technical for this role it's a matter of training and responsibility and that's something that we're going to talk about throughout this discussion is that it's really important if you're going to put people in a position to make decisions that have real impact to train them and not just put them out there to fail the drri does not make decisions in a vacuum decisions about the status of bills are going to be taken after discussion with the leads the drri is responsible for the decisions they make but you need to support them don't hang your people out to dry and you want the drri role to rotate it's stressful and you need anyone to be able to act in that role as needed otherwise you can't take a vacation go to a wedding uh get married have any kind of a life if you are always the person who has to be there and again we're going to see that more when we look at critical paths finally you probably want to have an internally own internal only email address that's

just drri that can be reached by anyone on your team and then you don't have to worry about who is drri this week if something comes up you send it to drri at your domain anytime now we're going to talk a lot about Communications and this role internally and externally or more specifically Angela is going to talk a lot about this role now assuming you're large enough to actually have a comms team what this means is your technical staff don't actually have to learn to speak to the outside world if they don't want to some of them will generally speaking because of the demographics of the industry most of them are introverted and really don't want to talk to other human beings if they can possibly help it now for smaller teams and certainly for single person shops this doesn't apply you'll be wearing this hat among many others but the whole idea is again we're trying to give people responsibilities that they are comfortable with and trained for and so you'll find people who are like I'm perfectly fine with making decisions about whether or not the build goes out but I don't want to be the one to talk to the players base and tell

them there's no update this week and you'll have other people who don't want to make the decision but are fine putting things into terms that the players or the press or anyone else can understand um and in fact comms in addition to keeping the dri in the loop of anything they find comms will often often be the transition point between multiple dris if it's rolling off at the end of the week saying hey this is what we were working on last week just so you know uh in a smaller shop you'll probably know anyway general staff this is everybody else now there's an old folk expression you have two ears and one mouth for a reason I've not personally done well with that advice but it's there the key point is that you want to have a way for your team members to be ears for the company and you don't want to have them speaking for the company accidentally if you're speaking as a representative of the company it should be something planned and deliberate now there are a couple ways to do this the biggest one is to have an internal only email address essentially

fire or fire is a good one um basically something has gone wrong and this something doesn't have to be technical it can be hey there's a report of a bug on the internet are we vulnerable it can be hey someone is pirating our game it can be hey someone got upset about a design decision we made and Reddit is exploding they don't have to be technical issues it's just a way for people on your team who happen to be out looking at something on Twitter or Reddit or anywhere else to go oh they're talking about us I don't know what to do they email to fire fire goes usually to the drri and comms and whoever else is responsible for monitoring things at the moment now as a practical standpoint unless you have really strong reason to believe that someone is deliberately pranking your internal fire account there can be no penalties for a false alarm what you want here is for your people to have a way to let whoever has to be the adult this week to know about it and if you penalize them for not

doing the research ahead of time well they're not going to tell you what you need to know and you don't want to drown out that signal as always you want to support your people with training if they're going to be speaking for communications or other role for the company if you're a small company you may actually have no general staff who aren't leadership in which case support yourselves you should be good to yourselves which brings us to leadership now Angel's going to talk about how this can go wrong in more detail when we talk about crisis Communications but seriously the biggest thing for leadership to do is Do no harm even off-the cuff comments by someone who is perceived as having a strong identity as a founder or leader of the company can do damage that your Communications and Community Management people cannot overcome if the head of the company says something that drives people away or offends them or otherwise causes problems no one is going to believe that the community manager saying oh no that was a problem has any Authority whatsoever they're clearly there as a puppet saying whatever they were supposed to say to make every body happy

if you are in a leadership role strongly identified with the company what you say has disproportionate weight on the flip side you can use this power to back up your people if you've got someone saying wow yeah we know we promised you a patch this week but it turned out it was a really difficult problem and it's going to be next month well that's not good news you want to be giving out but someone's giving it out you're probably going to catch a lot of grief from your customer base of people calling them everything but you know a a Sweet Child of God and that's okay but now you can come in as a leader with your established Authority and the weight of your reputation and say no this is a really good person they were doing the right thing we just misjudged we're sorry and you can lend that credibility there it's good to be able to have that but it's more important to just not make things worse and we've talked enough about training we should probably talk in more detail about training so I love playbooks flat out they're wonderful one of the nice things

about playbooks is that they are a fail soft solution having one Playbook is always better than having none Two Is Better Than One and so on this means you don't need to solve the problem right away you can order in Pizza have a staff party occasionally at the end of the week and say hey let's brainstorm a problem and then do a playbook for it playbooks need to be focused and short if they're complex they won't get used the end result of a Playbook is either an action or a pointer to another Playbook now the other thing you'll find is if you're making playbooks they're going to shape both how you manage your infrastructure and build your software and they'll have real ramifications when you realize what your critical paths are and we're going to see that as we walk through one right now and yes if you've ever done customer support A playbook looks an awful lot like a customer support flowchart all right the example I'm going to use is heart bed and the reason I'm using heart bed as an example is because the developers didn't actually do anything wrong

they used cryptography they used it appropriately they did not invent their own they did not even Implement their own they used the industry standard library for their platform everything we tell people to do from a security perspective but because of a vulnerability in the platform they were absolutely Flatout vulnerable and the fix was horrific because the keys were compromised or potentially compromised and so we're going to use that as an example because the whole point of this is your ability to protect yourself from being compromised is pretty much gone you can make it harder but something like heart bed there was nothing you could do as a developer unless you are a cryptographer who was auditing Heartbleed aiting open SSL and found it you didn't really have an option and so the whole purpose of this talk is to plan for what happens afterwards and that's why I said it was not a security talk or a technical talk per se so let's walk down the tree there are reports of

a critical vulnerability in a library or component and the first question is do we use it and you'll note that don't know is defaulting to we're vulnerable we'll talk about this again when we talk about planning and assets but this not only puts us in a safer security posture overall it also acts as negative feedback against skimping on your upfront analysis because suddenly you have more work because if we know that we don't use that you log it is not affected and you're done you can go back to sleep it's 4 in the morning on a Saturday and you're over problem solved well if we use it or we don't know we go to page two now this may come as a surprise coming from a security specialist but not everything is drop everything now if the attack is not being exploited in the wild it can wait until the start of the next business day however this should be a monitored situation if it does start being exploited this decision point should also be Revisited and again you have to look at the ramifications uh for those familiar with the recent Drew Paul ageddon bug uh Drew Paula said basically if you did not patch your site within 7

hours of the update being available they consider it almost certain that your site was compromised because 7 hours later they were seeing automated attacks hitting servers they manage now obviously as we said earlier most Studios do not have in-house lawyers or legal staff so the legal role is likely to be someone who is informed as to the Studio's contractual legal obligations and is authorized to call a real lawyer if the case requires it so it's not being exploited in the wild we're going to schedule a meeting with our drri our QA are legal engineering and coms for the start of the next business day otherwise well we go straight to the meeting and this is all Hands-On deck so first of all we need to produce tests that verify the vulnerability and the reason for this is first you want to make sure you have it and second of all you want to make sure you fixed it so right now verify that you've got it while that's going on engineering Ops and QA need to start preparing the fix and prepare for deployment and at the same time Ops coms and legal are busy working on the scope of damage

Communications business partners and any other remediation now you'll note this Playbook does not have replace all of our website certificates and we probably didn't have a what to do when our SSL for our site is now completely compromised but in the wake of heart bleed we probably want to sit down and write another playbook for that and we do that as a separate call out we'd check for uh remediation specific Playbook specific remediation playbooks now there's something you may have noticed here we've got QA and Engineering into Ops in two and comms and legal in one and if you only have one person in each of those roles you have a problem it may not be a problem you can get away with but or that you can you can avoid because you're a very small Shop but you want to look at what your critical path looks like when things go pair shaped and the critical path says I need these people and if I only have one of them and they need to be doing two things at once that's a problem and that may indicate cross training or it may indicate a hiring need now we'll cover it a little bit more when we talk about critical paths

but there's an anti- pattern that crops up in development very often and I just call it build boy no one wants to be stuck doing the build so whoever does it first tends to get stuck with it now that's problematic in the best cases but having your critical path for all updates go through one person is an absolute disaster waiting to happen so how long does a build test cycle take if it takes 96 hours for you to do a build and complete regress that's the max the minimum amount of time even if the patch were available before you could have everything out how automated is your build test cycle does someone have to keep puttering at it or you can simply put something up there and fire it up and let automated test start running and who is available to build and deploy now deployment of updates is not necessarily something you control some sales channels give you the opportunity to update whenever you feel like if you want to update twice a day and three times on Sunday that's fine with them others you have a real approval process the time to find out if there's a way to

expedite that is not when your house is on fire so during your planning reach out to your sales partners and figure out is there a way if things were really bad that we could get an expedited review so we could get this up there if things are catastrophically bad there's a vulnerability in the software that's letting you compromise backend servers you may need to shut things down can you do that if you had to in an emergency what are the ramifications can you just shut down multiplayer or would you have to shut down the whole thing all of this is planning for deployment that you should do as part of making your playbooks and also just ahead of time know what your requirements and capabilities are if you show up that you have a problem that we can't get an update deployed fast enough well that has to go back and close the loop and inform your playbooks because now we have more of a lead time before we can fix something as we saw in the Playbook example we need to know what we use and what we protect when building our playbooks one thing to recall is that some of the incidents that would trigger a Playbook will also involve some of our assets being unavailable having the

Playbook or asset lists on a Wiki is nice but if the incident is internal Network failure or VPN failure that Wiki is suddenly a lot less useful you probably actually do want to have hard copies of your asset lists in your playbooks and at least have the drri have them now another thing we're talking about what customer data do we have dis space is cheap liability is not when you're looking at things like customer data ask yourself what is the business case for having it because if you don't have it they can't steal it just because you think you might need something someday does not make keeping it now a good idea for example it may be that we don't want to take the credit cards ourselves we want to use a third-party payment whether that's Apple or Google or Microsoft or pay by cash or any one of a number of other places PayPal now all of those places will take a cut they're going to step in and they're going to take a cut what you're basically doing is buying yourself a hedge against

liability but here's the thing if I don't need their credit card number then I may not need their real name and physical address if I don't need it don't keep it again if you look at the data breaches what are people worried about losing credit cards addresses email addresses depending on the uh platform you're on you may not even need the real email address again unless you have a business case to hold on to something when you're your planet try and get rid of as many consumer assets as you possibly can because that's your liability if our database doesn't have anything that the consumer would be hurt if it got out it doesn't have their email it doesn't have their address it doesn't have their credit card then even if someone pops the backend database I don't need to say anything about it I don't need to do go into a comm's emergency because we didn't lose anything that anybody else cares about great they got our source code they got our assets whatever that's our loss it's not your loss as a customer and therefore I don't need to tell you

and as I said earlier one of the things that comes out of this is we're trying to find out what roles need to be filled and we may discover that there going to be some categories of problems that only have one person who can act now sometimes this is unavoidable either the studio is small enough that this is inevitable or the role requires someone to speak formally on behalf of the company which is by definition a limited number of people but often it indicates a dangerous critical path if only one person can release a new build you are in a precarious situation vacations illness bus offer letter this is a problem this will inform your need to cross train and hire and that's good for one thing it's really easy to say look you know we're going to make everybody learn how to do the build so that everyone can go on vacation you don't necessarily have to say so that if anyone leaves we don't discover we have no ability to ever update again and when we get down to the rest of the roles we're talking about Communications and so I'll turn things over to

Angela thanks Dave again my name is Angela and I'm here to talk about a role that lies outside the comfort zone of most tech folk and that's Communications because comms has some unique dealings with not just developers but with legal and the outside world and because comms functions as Dave mentioned as a handoff entity between dris uh there's some things we need to talk about separately so here I am most of us are pretty good communicators under normal circumstances thanks to ebook and Facebook and Tumblr and text and the twitters and the phones and having conversations now and then with normal human beings which is nice but in a crisis Communications becomes something that benefits from a lot of structure and as much planning and expertise as possible um I assume most of you would rather be working on the technical problem as opposed to explaining it to the least technical most hostile user who can manage to personally reach you

probably at 3 in the morning at home am I right believe it or not some folks do actually specialize in handling Communications in crisis situations and my half of the talk explains what those folks need to function and what you will need to do if there's no one around to do that job uh before we start does anybody here watch Scandal familiar with Scandal no oh that's so good we're going to live in the real world and not be TV crisis coms people they dress so much better than me okay the job of the crisis Communications person is to tell your story in difficult situations uh one of the leading crisis comps practitioners describes his job as making terrible news just bad and bad news good um so let's look at the slide um questions answers that should be clear enough right uh give your comps people the facts trust them to tell your story and give them the facts this is really important um we have current events to talk about in this one crisis comps folk

do not have the privilege communication status that you would have with say legal council but they still need to be operating with as much of the truth as as they possibly can um anybody F following the uh John gesi scandal up in Canada right now uh fell who uh broadcaster for CBC before that he ran a band called Moxy Fus um anyway he's in he's in some personal trouble right now and uh the the guy was not forthright to his exceedingly expensive crisis comms team and um went out on Facebook with a long thing about how he didn't do what he's accused of doing and no no it didn't happen and it didn't happen the way they said and because he's a really articulate Charming guy um it sounded good and they were winning and then more effects came out and they were losing and then the crisis comps team f HED their client here's a word of advice if

your crisis cons team that people who specialize in trouble don't want your brand of trouble you have a very interesting brand of trouble Outreach and response uh Outreach is to use a terrible word proactive uh response is reacting to questions and statements made by others uh C spaces outward to the public inward to the company again lawyers execs as well as devs and again as Dave said we can't always count on on lawyers being in house but they will be dealing with probably whoever you have to be lay is on to real counsel and real counsil itself now to be honest comms is a game there are objectives there are success conditions and because this is a crisis you have time pressures you have achievement pressures um in the gesi case you can win the new cycle you can get your story out there and then you can lose the next new cycle or you can lose all the new cycles and things the snowball keeps getting bigger most importantly there is a Terrain map for those process and even if you're not familiar with it everyone else you will

deal with in a crisis is uh the Press especially has its way of doing things and the better you are at understanding encountering how they operate and how social networks operate with your story and so forth the more effective your comms effort will be you cannot beat the system that exists all you can do is convince it that you're winning okay I want to take two slides and run through the components you will need to build in a crisis comps plan you will need these for both Outreach and reactive uh this first slide covers the three musthave items after that we're going to go through a few fancier additions and I want to go back and disagree with something Dave said about playbooks um in the case of comms I'm a big fan of set it and forget it as much as you possibly can build your playbook and don't touch it unless you really unless there is an extraordinary circumstance H because if you have a flowchart memorized for that if you have your playbook memorized everything's going to flow trust me coms will have

enough drama dealing with the specifics of the incident including trying to understand it your comps people are probably not technical learn the components do not reinvent the wheel here okay talking points um these are the basis of everything else you would develop they are sentences your spokespeople and anyone else you engage with anybody on the company who might be speaking should be able to pretty much recite um word for word I'm not joking about that they are your point uh comps person on average spends more time in care per word building talking points than on any other of the many components you will see um each of these sentence conveys one point they are simple unambiguous precise they are short uh 20 words a piece is too many which means all my slides fail my own my own uh metric there um your coms team should be doing everything in its power to make sure that these statements get out perfectly are not misquoted um I'm going to give you a test now what do you think would be the problem with a talking

point that says there are reports that our Foo game is causing eye blisters but no proof exists that this is true what I do wrong there anybody oh you're so quiet the problem is the words I put words in a row that said our food game is causing eye blisters that's awesome and that's the quote the news picks up uh present the facts um we are aware of reports that customers are experiencing issues with one of our popular games really really dry but do you see how the gotcha quote people can't do anything with that if somebody's going to hang you with that where do they start you gave them no hook uh it's business writing uh there's a place for memorable language in this process the talking points ain't it statement on the other hand is it um the statement's interesting because it's the one thing that sounds like humans are speaking um it's also the first and often the only thing you can send out intact to press or anybody asking you questions um it's a nice paragraph or you can do two

whatever that you can mail to reporters post it on your website put it on social media anywhere you need to make you know the statement uh clear accurate quotable serious respectful positive um your goal is to assure people that you are on the case and as much as you can to give them the information they need to settle the problem in their own mind um as we the gesic case we just discussed um the statement was beautiful memorable positive and was not written with enough knowledge to get out in front of what the next news cycle would be so it was long it was extremely long but most importantly it lacked understanding of what was to come wouldn't have said what was to come but would have been aware and started countering think chess uh some folks use the statement to bring in third party voices who are independent that can support what you're saying either in a quote in the statement itself or more usually by getting permission to uh include their names and contact information in mailings to the Press not social media but the Press um the right independent

voice can make a big difference to a reporter who's just sifting through a lot of opinions on the situation yours is only one even if it is your product um tough part about doing this is that someone's got to go line that guy up and talk to them figure out what they might say um a good example of how this works well in the tech world is that there are certain companies that make a point of briefing analyst firms before a major release especially with security bulletins they brief them they figure out who who's kind of on the same page as they are with what's going on and they will suggest to the press that oh talk to this analyst this analyst and this analyst now when I was a reporter I did not trust those because I knew how the how the sausage was made but you will find that especially in a crisis situation where things are moving quickly other reporters have different opinions on that so always good um a fact or FAQ as we seem to be saying these days um some comps folk will call this a Q&A but that kind of implies a linearity that doesn't exist uh this consists basically of pretty

much every question a comps person can get uh with a well crafted full sentence answer to all the questions possible um I actually really like doing this component when I when I have these little projects because it helps everybody see what the parameters are and sometimes raises issues that you know the drri or legal or or others haven't really you get a lot of really somebody's going to ask that yes really they are um how long has your food game been shipping uh complete sentence response uh will you be offering refunds for buyers of the food game complete sentence response how could the fu game have shipped without review from the centers from Disease Control whatever full sentence response um not necessarily an answer um the goal is to stand informed about the progress of the investigation and to give information that has been internally vetted as accurate and safe to discuss you never want to say we don't know or please God no comment don't do this um the end of

the FAQ is always I'll have to check on that and get back to you and then do it um as you have guessed from all that not for publication you use the parts you need and you keep the rest under your hat quickly on these these are some specialized tools available um if your team already does video like if you put a lot of demos up on YouTube and you feel you have something that's communicated well uh I think Dave's interface Dave's uh design you know this is not exploded it's just different is a good example of that um that might be a way you can make your point very efficiently um I'm not personally a fan of podcasts for for this purpose but there are a few podcasters um kind of from the Nerdist on down worth talking to uh for crisis purposes you treat those guys as press um an open letter if you had like a big name on your leadership team um very big name on leadership team this is why you pay the money to go out and be this guy um the corollary to you know Dave's you put it

putting your leadership out out there to to add veracity to whatever you're saying um the the trick here is that a lot of times your coms folk are writing it you are sock puppeting your leadership team um your comps people write it the person whose name is going on it goes yeah that sounds like something I'd say and you proceed uh gives them more time to actually work the problem uh tweets post social um more on that in a second but the short thing here is if you are already active on on those that's great if you are not now is really really not the time to start um I threw backgrounder on here I go back and forth on whether I should have this is a really bold tactic if you're in the best defense is a good offense stance um if you have someone on your staff who is well-versed on a topic let's take uh open SSL to use Dave's example on Heartbleed and if your company is not actively involved in the crisis um you know heart bed but you knew you weren't affected if you feel

you have the bandwidth to speak out and be experts you know what that's marketing to me but some people feel that's a good time to act and let's move on okay Staffing Personnel access you know what you need uh who does the work how does it get to the people that it needs to see um first off about this slide these venue bullets whatever are an ascending order of both the E with which coms can be done and um the lack of control you will have over the output once it's out there uh most importantly these are really really fluid these days the Pressed read Twitter uh random email Guy can put your entire correspondence on blast on on Tumblr uh things move around and every one of these levels of Engagement it's important to again know your map have your plan um email we talked about the uh having a drri box we've talked about some public facing um public facing emails um every everybody comfortable with the idea of having a public facing email that sometimes says

stuff to the outside world it's quiet out there um keep an eye on it um the the email addresses Dave mentioned such as fire I think those are a wonderful idea you don't make those public but um by that you need to have something where people can tell you things um some companies use the secure at metric or um I think eBay and PayPal do spoof for some of their things you get the idea anyway um that's also a useful tool if you're set up to notify CH users of changes or whatnot through that address um uh active or at least actively monitored presidents on social media is important and I think most of you probably do that to some extent um how many of you have somebody on your staff who just kind of does it anyway they blog or they're posting to Tumblr or they're Facebooking or they they're tweeting about work stuff on a regular basis this may have a massive intersection with your rock stars um anybody have that guy aha we're going need to talk about that guy um and the rest of you pay attention

because you've probably got that guy and you don't know it um then there's a press if you have PR folk who are already in touch with the Press um they have the contact information the business relationship to handle the outputs um if not press will probably reach out to you through social media or through any phone or contact information they can find that thing you put on your your home you know your your main web page a million years ago is probably going to get a workout um if you are contacted by press in a strange situation ask for their contact info tell them you were phone or email them back Google them for the love of God Google them to find out if they are who they say they are and what sort of you know what sort of entity they are what they cover learn a little bit about this person and then Reach Out The Objective in any case is to funnel all this action to a manageable field of play so that's going to be managing where you engage where your coms engages and what you're saying and putting out there that's your messaging um good

comps people can usually prioritize Outreach and response efforts and have the patients to remember that the loudest voice is not always the truest measure of landscape so that's how so who uh so Proprietors here's your list of hats be strong with have three roles uh first role coms um you may have someone in house with some are all the expertise as I said it's a specialized field a lot of companies aren't staffed like that um related to crisis comms you have PR public relations actually it's more accurately press relations um some of you may do that in-house already for those of you who who use a firm outside they may in turn have crisis comps folks on staff to specialize in in meltdowns and then further a field of marketing I don't find marketing folk as a role very comfortable with these activities because bad things are happening and you know how marketing folk get and things get weird um but this may be your first line of defense on things like Wordsmith thing or uh social media presence um com

spoke are good at communicating information but you do not want them working in a vacuum so this means that not only are comms folks talking to the outside world they have got to have access to someone on staff who has a deep understanding of the problem now that should be your drri uh these conversations I will warn you ahead of time are not always fun for the dri you are tired you are stressed and you feel like the questions you're getting are the dumbest thing you've ever heard um but you know bear with that the comps person probably agrees but they can't tell you that right now and uh as a reminder as Dave pointed out if you keep comps looped in you get a payback because it's the continuity you need to hand off between dris um you have a class of creatures with comps that lives to document stuff so use that superpower of theirs um in some situations you're going to want a public spokesperson uh that could be super simple just having some quotes out there attributed to one of your leadership team or whatever um and again the comps people tend to sock puppet leadership teams on those on the

other end of the scale you could have a spokesperson who's talking directly to reporters you may have them on camera um and if somebody in your company is already media trained in that process fantastic that's your guy do not get cute and decide that the current crisis would be a good time for one of your leaders to do his first television appearance um as Dave put it do not put your people in a position to fail and if you're not sure about this here's a rule of thumb if if you do not know what pivoted talking point means by the time I get to the comma you are not ready to speak in public you talk about that later but if you have any doubts no you are not ready um sole Proprietors you poor kids um if you have no comms team have the Playbook at least um the parts I listed before are what a crisis comps team would develop on your behalf and stick to pretty closely if you have the presence of mind to develop them and stick to them you you might be okay it's business writing like any other thing and you can learn to do this it is a learnable skill and not a difficult learnable skill that

said uh I want to say that there are occasions on which um you take a deep breath and you buy some expertise um and you don't have to keep them on staff this is this is ad hoc expertise you're buying um when I was a a reporter I was covering a story in which a very small business that posited itself as familyfriendly managed to hire a guy who had a thing for teenage girls um very small company and it was going to be an absolute disaster when the news got out and they called their lawyers and their lawyers told them to hire crisis comps I never talked to the people the family who ran the company without the crisis comps person acting as a mediator there and um that person had enough of enough distance on the subject to be able to handle it to be able to handle the fam's you know extreme distress at what this this person had done to their business to handle the you know different legal questions that were coming up to handle the fact that I was one of about 40

reporters who felt they needed to talk to them every hour on the hour um kept everything calm kept everything clear and in the end the business survived that was not a foregone conclusion going into that one by any stretch um the family got to present their side of what happened and they dealt with the things they needed to deal deal with didn't let themselves get sidetracked you know I can't do that cost benefit analysis for you but it is something to keep in mind that sometimes you're going to drop a dime and it's if your company is at stake this is this is worth spending the money on so let's talk about how you know you're in trouble um call is coming from inside the house possible um that fire ad address Dave discussed will be your first sign of trouble um you'll hear about something in the news later that day somebody walks up and says guess what um given all your given that's really not bad um it gives you the Head Start and for comms that's really crucial um Dave talked about speed of response and this sort of conforms to the idea of wait till the next business day um first thing everybody on staff

needs to do to quote the wise men of Insane Clown posy is stop talking I believe it might have been do nothing or sit down or something but anyway first thing you need to do is stop talking outside you have your playbooks in place it's time to work it everybody Above All Things needs to understand this is not the time for tweeting not the time for thinking aloud online in your blogs on Twitter with friends um you know not saying anything flippant or casual or even talking about how bad your workload is today you remember that guy we talked about the one who kind of does his own thing tweets about work stuff on his own yeah go find him tell him no tell him to sit down put it as clearly as you can um because somebody's going to ask him if he's out there and getting good feedback from this this is as a reported that was the first guy I went looking for I didn't want to talk to PR people I wanted to talk to the loudmouth um this is sometimes by the way when you figure out who on your staff is looking out for

their own brand rather than yours uh #f firing offense the uh the advantage is you gain again it's framing your narrative choosing your playing field and that Head Start um these are all preferable start conditions to what could happen oh those pleasant calls from the friendly hackers people want Dave's group for instance here's a friendly hacker now doesn't he look it um so it's not someone on your staff you get an email from someone like Dave who says hey I found a problem with your product you know it's not ideal but you still have a really good chance of everything being okay as long as you keep your cool keep the conversation civil and keep it private um on the other hand in-house behind the scenes you are accelerating your activity at this point all comm's hands should be on deck and briefed but you know at like you know parade rest or whatever you know your hair is not yet on fire um one person probably Dr should be engaging with this uh with this this uh researcher and that should either be by email or if you must by phone or by

Skype ask them for confidentiality but assume that anything goes wrong your conversations will be made public or even if it didn't go wrong sometimes you'll get somebody who's so excited to hear from a company they really care about they will go out on Twitter and put on blast everything you thought you said privately about the bug um it hurts but it happens um keep your cool you know in-house again you're keeping coms looped in you're keeping legal looped in I will point out that both comms and legal are not just good at documents but very good at keeping quiet when things are on speakr phone just saying um when dealing with Outsiders do not whine do not lie to them and don't gloat if it turns out the researcher was wrong this is really important try above all things to keep in mind that this person reached out to you um which by definition they're your friend or at least your ally at least for now um if your investigation finds it the problem isn't your fault you be clear with them and be very specific about that treat

them as collaborators is what I tell people am I understanding the thing you found correctly anything in our our anal you know anything our analysis missed did you see something we're not seeing um in in in most cases I advise more carrot and less stick with these guys you know they know you have lawyers and they know you have power in this situation if they start making details public in in an aggressive way they start demanding money or whatever that's different and they know that that um at that point your legal folk are going to have some thoughts on how to proceed but above all things that remember that that reach out that reaching out to you that you know that call that says hey I found something in your product no matter how you know aggressively worded no matter how you know it's not an aggressive act these are your friends these people are not your friends um once the press and the general public have taken an interest in your situation uh the risk of damage is higher if you play your cards right you may at least come out no worse than you

are and by the way yes your hair is on fire at this point you need your comps plan locked and loaded um and A good rule of thumb in this case is the wider the interest in your situation spreads the simpler you're going to have to make your message so you may be going back and making it even simpler adding talking points going a video game is you know you'd be amazed um odds are really good that interest in your situation will stay with the Press you're already used to the gaming press you maybe computer press but if the mainstream for some reason takes an interest New York Times comes calling prepare to explain everything as simply as possible um if they don't know your product if they don't know the gaming space at all there are a lot of things you're going to have to spell out be patient um especially since you're going to find some of them are very arrogant about their their lack of knowledge um always be polite always be responsive always you know I'll get back to you instead of I don't know never say no comment we talked about that um for the

most part if things get to this condition you're respon in response only mode you're not doing Outreach there is one exception to that if you are forced to shut down some or all of your uh servers or any any kind of access as part of deployment as Dave was describing I think um you want to communicate that to anybody who will listen no surprises on outages it looks so much worse if they don't know it's coming um if you're and by the way if you're drri fails to warn the comps person about an outage the comps person is legally allowed to kick them really hard you can look that up it's true um friends and fire away places if you've previously had good interactions with somebody in the press or somebody in the industry who's high-profile as we discussed consider reaching out with reporters that can be as simple as as an email so hey I heard know you probably heard about this thing and I wondered if you had any questions for me you don't be pushy just once one email is more than sufficient do not mail bomb your reporters um but if you have that good credit built up at this point it might

be a really good time to spend that um maybe you got lucky Maybe by now your investigation your guys have come back it's like nope not us we're not affected it's not a bug it's somebody else's bug um say so just say so you write another statement says the investigation has concluded here's the situation you make sure it gets to all the people and places that you have interacted with in this progress process again your coms person's keeping a list of that or should be um I advise you um through sad experience to say that you'll continue to monitor the situation because it's that's the safest thing to say in case it decides to claw its way back out of the Grave um but I'm not here to tell you what that might mean in your case you just put it out there for closure though so there's one more possibility and you know I sa The Best For Last with any luck none of the things on the slide will ever happened to you um plan works the same but even your best

efforts in these situations are probably not going to come to more than a draw sorry about it um you can break into jail and put yourself in a bad situation um anybody uh follow SAA Nella talking at the Grace Hopper conference couple weeks ago our good our good CEO of Microsoft decided to get in there and talk about being nice and having good karma when asked about how to how to improve salaries for women it did not go well at the women in chat conference for him and the saddest thing about that by the way was um about 2 days before that SAA had been the subject of a really positive piece from the from Vanity Fair and for coms that's that's a huge win that is that is your entire year of of you know we won you know it's an elaborate process there's interviews there's factchecking there's background work with the reporter it is the highest profile thing your boss's mom reads Vanity Fair and it all blew up in two days because their boss opened his

mouth if the security version of this happens to you you work the map um you get a decent open letter out there it may take several as it did with uh with Microsoft and you you just kind of hope the new cycle goes and does something else pretty soon um you could be the victim of an actually hostile security reporter as opposed to the socially inept security reporter from before um uh those of you who don't know about past bin it's a site where some of your more blackheaded researchers will drop information about vulnerabilities or more pertinent here perhaps uh names and password and other information they've ganked from uh from insecure sites or Services um in cases like those you have some work to to figure out what happened over on the tech side but a lot of your comms will will be following the lead of legal um and in some cases the the regulations covering breach notifications if you didn't find your way clear to not not keeping the credit

card info um let your lawyers lead just follow finally there are some pretty random events where all you can do is work the plan and start thinking about how to rebuild um everybody here know what Arby's is the fast food chain restaurant yeah okay uh they're having a really weird year the um both John Stewart and Steven colar have been using them as a punchline for a little minute now um saying some really mean funny things about the quality of the food and Arby's clearly has some really smart crisis comps people on retainer because instead of lawyering up and making them stop they started sending over sandwiches here's some food free food total charm offensive and it seems to be working in the last few messages you know storting col Bear say you oh hey they sent over these sandwiches they're they're forthright about it but the comments have been Kinder and so now Arby's is in this interesting transition between being this this mean punch line to this sort of you recipient of free advertising these mean things are said but the edge is definitely off um I

wouldn't say it's positive yet but as far as making lemonade out of beef lemons sorry it's uh you know it's not bad that's and sometimes that is exactly what a wind looks like um and I to the end of my slides questions questions for Dave especially he has an open mic I think no would you like to go stand at this micophone we'll both stand up here because someone might actually have questions but there's an edge who will win the edge of the of the podium The Edge will win so are there any questions there's a microphone up there or you can raise your hand and we will ask and then we will repeat the question any questions terrify them into submission or there's nothing after lunch it's one of the two all right well oh for a second I thought someone was going to ask a question well in that case thank you very much and have a great show