Heading 1


Cyber In security News


A year later, a lawyer talks about what he learned battling the largest known attack that crippled the infrastructure of an American city.
It’s easy to understand why a company in the throes of a cybersecurity crisis might want to call in Roy Hadley, Jr. Not only is he knowledgeable and experienced in this area, he seems like a man who knows how to keep calm. And he has a straightforward way of expressing himself. “You can’t keep it private if you can’t keep it secure,” he says, discussing the link between privacy and cybersecurity. Later, explaining why he chairs the board of the Information Security Society for the Technology Association of Georgia, he says, “You need to understand the tech side of it to understand the legal side of it.” Last year Hadley, a special counsel at Adams and Reese, got a call from an entity a little bigger than most of his clients. It was the city of Atlanta, which had just been hit by a ransomware attack that crippled the city for days.
     That was in March 2018. A lot has happened in cybersecurity since then. But a year later, Atlanta remains the best-known example of an attack on the infrastructure of an American city—the kind of event that makes government cybersecurity officials shiver. Though responding to an attack on a city is obviously much more complicated than one on a company, and the stakes for entities entrusted with public safety are infinitely higher, Hadley’s recollections and insights have equally obvious relevance to general counsel everywhere.

CyberInsecurity News: Would you say the attack on Atlanta was the largest “successful” attack on U.S. infrastructure to date?
Roy Hadley, Jr.: I would say that the attack on Atlanta was the largest known criminal attack on a local government to date. And I say that because that’s what we know. There’s always the question of what you don’t know. The Atlanta attack came to prominence partly as a result of the decision of Mayor Keisha Lance Bottoms not to pay the ransom and to be transparent about the impact of the attack and the city’s response.

CIN: And ransomware was never paid, right?
RH: Correct. And that was confirmed by the FBI and the Department of Justice in their indictment against the perpetrators.

CIN: How much damage was done? Was any data lost? Any damage that could not be remediated?
RH: I have to thread a line, because I am the attorney for the city. There are some things that are attorney-client privileged. And we have state statutes in Georgia that say that you don’t have to disclose information about your systems that might impact your security. So I have to make sure that I don’t say anything that might potentially highlight a vulnerability or divulge something that may diminish the security of Atlanta. Now, was some data unrecoverable? Yes. What I will say, though, is that the city took this opportunity to really take a deep dive into its systems and its architecture and make sure that, going forward, the city would be in a better place. Were there impacts that we couldn’t recover from? No. The city did a very good job initially in assessing the impact of the ransomware attack, and then immediately putting workarounds in place. Some technological workarounds, some manual processes. For example, we had to put in place manual processes with the court system, so that you had defendants adjudicated even though certain software programs that the court utilizes were not operable initially.

CIN: I read that patrol car videos were lost and were not able to be recovered.
RH: I really can’t comment on that, for various reasons. But what I will say is that the position of the police department is that they were able to continue to do their jobs protecting the public. What that means is that the police were able to continue investigations and prosecutions despite this attack.

CIN: Can you think of other cyberattacks or ransomware attacks in the U.S. that you would consider comparable?
RH: I don’t know that any were comparable. You had a fair number of attacks in 2018. Atlanta was unique in that it was very public. And part of that was Mayor Bottoms’ desire to be very transparent. You had a fair number of reported and unreported attacks in the United States in 2018. For example, Colorado had a significant attack. A lot of universities—not really reported—had significant attacks in 2018. What people need to understand is that cyberattacks in the beginning were typically done by kids, hackers who wanted to deface your website, put a goblin on there and say: “Ha, ha, ha, you’ve been hacked!” Now it’s much more insidious, and so you’re starting to see more and more very sophisticated attacks against cities, states, municipalities, because that’s where the data is. That’s where you’ve got a lot of information about people. You’ll start seeing them again at the universities because, again, that’s where the data is. Also, you are seeing ransomware attacks against institutions that interface with the public—governments, hospitals and schools—because the bad guys know that these institutions have to continue to provide services on a daily basis. They have no choice.
     And another thing that we’re starting to see is attacks that undermine the integrity of data. We always assume that data is going to be taken and utilized for nefarious purposes. But you’re starting to see data being manipulated, especially in universities, research institutions, governmental research facilities, manipulating the data in order to slow down research. In order to cast doubt on outcomes of research. It used to be that you were worried about data being exfiltrated from your systems. Now you also have to worry about the integrity.

CIN: Can you give me examples?
RH: This was a while back. But a public example, and I’m trying to think of the company. Basically it was a health care company where cancer diagnoses were being manipulated. So you go in and you have a cancer screening. And you come back, and it may or may not be right.

CIN: What would be the purpose? Was this by a competitor?
RH: A competitor—if you are, for example, developing artificial intelligence systems. And let’s just say a foreign government—China, Iran, North Korea—is also trying to develop those systems or comparable systems. If I can go in and manipulate your data, and slow down your progress, then I can get there first. And there are market advantages to that. There are national security advantages to that. And so you’re starting to hear a lot more around that kind of cyberattack that is much more nefarious and insidious, because it doesn’t readily pop up to say, “Oh, we’ve been hacked.” The whole purpose is to get in there and not to be figured out.
     And I’ll bring it back to government. All elections are done at the state and local level. Even a federal election—all voting is done at the state and local level. So the states and municipalities are in charge of those voting systems. Here in Georgia, we don’t have paper backups. What does that mean? If you can get in and manipulate that data, then you can manipulate election outcomes. People are saying the Russians did that in the last federal election. But you’re starting to hear more rumors around those types of attacks. And because there is no backup, there really is no legitimate way to check the data. If you challenge the election, they just hit the button and recalculate. But the recalculation is going to use the same data.

CIN: Globally, the biggest ransomware attacks that I’m aware of are WannaCry and NotPetya . Are those the two biggest that are known?
RH: Those are the two biggest that are known. But SamSam —which was the variant that hit Atlanta—is also up there.

CIN: At the time of the Atlanta attack, the SamSam perpetrators had garnered something like $850 million in ransom payments. Is that accurate?
RH: It’s always difficult to validate those kinds of numbers. It could be higher. Somebody could have paid it and not reported it. And I would think that there are a lot of small players that will pay $1,000, $5,000, $400, whatever it may be to get their data back, because it’s cheaper to do that than recreating the data—assuming that the criminal does what they say they’re going to do. And that is increasingly not happening, because the people that created these variants of malware were “professionals,” and so there was kind of a code of honor—honor among thieves—that if you pay the ransom, you will get your data back. The problem is that you now have a lot of other actors that are using these malware variants who are not “professional thieves.” They’re buying these tool kits on the dark web, deploying them, and sometimes they get around to sending you the [decryption] keys, sometimes they just take the money and walk away. What used to be… I’m not going to say “certain,” but more likely than not—that if you paid the ransom, you would get the keys and your data back—is not necessarily the case now.
     Another thing that people have to remember is that when your data is locked up, there isn’t necessarily one key that’s going to unlock it all. A lot of these variants will utilize multiple keys, maybe even hundreds of keys. You may get 10 keys for your $100,000. Those 10 may unlock 30 percent of your data. And then you may come back and get another request: “You see that we have the keys, and we’re acting in good faith. I know we said $100,000. But if you want the other 70 percent, we’re going to need another $100,000.” Then what do you do? Chances are, you pay. There are no guarantees. So when people say, “Why didn’t you pay the ransom?”—it’s not that simple.

CIN: In the Atlanta SamSam attacks, two Iranian nationals were indicted. They don’t have a connection to the Iranian government, according to the indictments. Correct?
RH: That’s correct, as far as the intelligence community and the cyber threat community can determine.

CIN: When did you first get a call about the ransomware attack on Atlanta?
RH: The day of. The city attorney called me, and I was called because I’ve worked with the city before as an outside counsel in other capacities. I’m familiar with the city and its operations, which is a plus. But, secondly, I’ve been working in the cybersecurity and privacy areas for more than 15 years. That was why the initial call was made to me. Once I got there, I added additional value, because I’ve been in-house, which has helped me understand how organizations work. But I’ve also spent a lot of time educating myself on cyber threats and solutions and vendors—all those sorts of things that can help. I also bring relationships to the table with vendors, such as SecureWorks and Mandiant , and other big vendors that play in the cybersecurity space. So I was able to get in, help the city understand, in layman’s terms, what was going on. And then, as it progressed, what we needed to do and what assistance we needed.

CIN: What was your role at the point of your engagement, and did they know your expertise at that time?  
RH: Outside counsel was my role. It is always a best practice to retain outside counsel for this type of issue from the get-go. And yes, there were lawyers in the city that knew of my expertise in this area. They had attended events that I had sponsored or participated in. The outside counsel role is a very important one, because it allows you to do a lot of things under the auspices of attorney-client privilege. Now with governments you have open records acts, so there’s a level of transparency that’s going to be there by statute. But it does allow you to do some things under the privilege, which allows you to protect very sensitive information and discussions.
     When you bring vendors on, it is a best practice to have outside counsel retain those vendors. Then the discussions with the outside vendors also potentially are subject to the attorney-client privilege. You balance that with transparency and trying to keep the public informed. But you do have to balance to make sure what you’re doing remains secret to some extent, because if I’m the bad guy and I lock up a city’s systems, the first thing I would do is start watching the news and see what they say. If they say, “We’re doing X, Y, Z,” that’s giving me intel as to what else I may need to do to keep it locked up.

CIN: When you first arrived, who were you reporting to and working with?
RH: When I first got the call to help, I was asked how soon could I be down at City Hall, to which I replied, “I am on my way.” These types of events often require an immediate response. When you get the call, you drop everything and get prepared for some really long days. I report to Nina Hickson, the current city attorney. She’s equivalent to the general counsel of a company. But my role was to support the city: the mayor, the chief operating officer, Richard Cox. I had direct contact with them to keep them abreast of what was going on. Initially we were in a conference room, trying to figure out what was going on. We quickly got some outside support from vendors. SecureWorks came in and was on the ground very quickly and began to do a forensic analysis to determine what was happening, what was impacted. That allowed us to begin to assess the damage and start to formulate plans.

CIN: Who was in charge?
RH: Ultimately Mayor Bottoms was in charge. But the chief operating officer took the day-to-day, minute-to-minute reports from everybody. That’s important, because you have to have somebody in charge to make those decisions. Ultimately the big decisions are going to be made by the governance structure—either by the mayor or by the city council, which is independent of the mayor. And for day-to-day, the chief operating officer in conjunction with the city attorney would make certain calls. That’s important, because you’ve got to have somebody to make the call. You’re going to have a lot of decisions that are going to have to happen very quickly in order to stop the hemorrhaging, assess the issues, bring in support—whatever those decisions may be. If the bullets are flying past you, you can’t say, “Oh, let’s wait a couple of days and determine what we’re going to do.” The bullets are flying, and you have to make those decisions right then and there.

CIN: Did Atlanta have an incident response team and plan?
RH: Yes. They were critical to what I deem to be the success of the city in responding to this incident. People knew who to call. People knew who should be in the room. So the city was able to implement its response plan. Ria Aiken, who is the director of emergency preparedness and oversees the incident response on a more macro level for the city, was very involved in the minute-by-minute implementation of that incident response plan. She did a fabulous job. All of the city’s personnel were fantastic.

CIN: What were the biggest challenges for Atlanta?
RH:  The biggest challenges for Atlanta, like any large organization in a ransomware or cyberattack, is to figure out what happened and what the impact is. In terms of the virus, we were quickly able to determine what it was, isolate it, stop the hemorrhaging. Then we began the process of getting services back online. That was the biggest challenge. The city delivers water, fire services, police services, public safety. It delivers the municipal court system and 311—the help line where citizens can call in. There’s legal services, building permits, trash collection. And so the biggest challenge was trying to figure out what was impacted, and then figuring out how long they’re going to be impacted, whether you can put a technology workaround in place or whether you need to put a manual processing workaround in place in order to get the services back up and running. And one of the things that the mayor said from the get-go was, “We’re not going to let this impact the delivery of services to the citizens of Atlanta.” So under that directive and under her leadership, we were tasked with: “What do we need to do in order to maintain these services?” Whether it’s standing up a separate call center in order to get calls, or standing up a manual process for officers to write tickets to get them all the way through the court system so that people can pay fines. That was probably the greatest challenge.

CIN: What were the other lawyers who were involved doing, and what were their positions?
RH: The city has a fantastic legal department, with a lot of very experienced lawyers. As we hired vendors to come in and help, you had contracts that you had to deal with. Negotiating those contracts. Some vendors had existing contracts, but you had to do amendments to them. We had open records requests coming in, media requests. You had lawyers who were helping with the courts—standing the courts back up. And now you’re talking about people’s constitutional rights to hearings and stuff like that. You had lawyers involved in public safety responses to make sure things were done in an appropriate legal manner. In cyber responses, a lot of time the lawyer’s role is unsung. You really don’t hear about them. But when you really roll up your sleeves, there’s so much going on, in terms of how an entity has to quickly respond for procurement of vendors and equipment, and delivery of police and fire services. You have to have lawyers involved to make sure that they’re done in accordance with city or county or state procurement statutes.

CIN: Some of this would be similar to a company under attack. But much more complicated, because the ramifications involve public safety, and it’s so much larger.
RH: Absolutely. So much larger and so much more complex. And so much more outwardly facing. So, for example, with the Equifax issue, they were able to operate—and I’m not saying this in a bad way—they were able to do a lot without having to divulge it to the public as they were doing it. Nobody was calling them and saying, “Is my trash going to get picked up today? If I call 911, will somebody answer? My son is in jail. Can I pay his bail and get him out? Is an ambulance going to come and get my mother who’s having a heart attack?” Those sorts of questions don’t happen when you talk about a private company being breached or impacted. But when you’re talking about a municipality, you’re sometimes talking about life and death situations. So it is more complex and more impactful on people’s lives and well-being.

CIN: In January 2018, there was a city auditor’s report that revealed that Atlanta’s state of preparedness for a cyberattack was poor. That report was publicly released. I don’t want to sound like I’m questioning the idea that the auditor’s report should be made public. I’m a big believer in public accountability, and I’m a journalist who believes in the public’s right to know. On the other hand, does a report like this increase the vulnerability of Atlanta to just this kind of attack?   
RH: I don’t know. It’s difficult to say whether the release of that report affected this particular event. As part of normal processes, however, the city has taken a more critical look at the release of that type of information. I think you will find that Atlanta and other municipalities are going to be more circumspect before that kind of information in that level of detail is released going forward.

CIN: Did Atlanta have cyber insurance in place at the time of the attack?
RH: Yes. The risk management department had the foresight to put cyber insurance in place. So Atlanta did have insurance, and it’s working right now with the carrier to recover under that policy. I’m helping the city with that also. We’re still in the midst of it, so I really can’t say much about it. But it’s going well. One of the things that was done throughout the response and recovery process that I think is a great practice for any organization is to document what happened and what you’re doing. Document services that you procured. Document what they’re doing, what hours they were there. When you then start talking about trying to make an insurance recovery, it becomes easier because you have documentation. Another really good practice is, as you’re going through this thing, have continual updates with your insurance carrier. Let them know what’s going on. That gives them the ability, if they think something’s going askew, to say something sooner rather than later so that you can adjust if necessary. Those kinds of open conversations can be helpful in ultimately recovering.

CIN: The attack could have been a lot worse, couldn’t it?
RH: Yes. [laughs] Attacks can always be worse, no matter how bad you think they are.

CIN: I’m thinking about targets. Like Colorado’s transportation system, which was attacked. There are so many critical pieces of the infrastructure that could really affect public safety in a direct way, and that didn’t seem to be the case here.
RH: That is correct. A large part of that—despite the impact that it did have on the city—there were things in place to prevent those types of things from happening. I can’t really tell you about them. But I can say that there were things in place to prevent impacts and to prevent the spread of impacts. As much as the city was impacted, it was not operating from a position of naivete. That’s why you had segregated systems. So even though certain systems were impacted, many were not. The focus has always been on what went wrong. What I will say is, even in that situation, a lot went right. You can’t really talk about it. You can’t really report on it. You can’t divulge, because those go directly to the security posture of the city. But I will say, in general, that a lot went right. Which was why the city was able to, the next day, continue to deliver services, including critical services like 911 and things like that.

CIN: Do you think Atlanta is now prepared for attacks?
RH: You’re never “prepared,” because that assumes 100 percent. Atlanta is much more resilient than it was when that attack happened. Also, the city has taken this opportunity to harden itself. The city has said, “We’re going to really do the work necessary.” And so work continues to go on at the highest levels, but it is in a much better place than it was a year ago. And a year from now, it will be in a better place than it is now. What the city understands is that you never will be totally secure. You never will be totally prepared. Because those are absolutes. And this business is not a business of absolutes.

CyberInsecurity News: You’ve worked in-house. You’re a lawyer. But you’ve also been a chief privacy officer. How has your background helped prepare you for this kind of work?
Roy Hadley, Jr.: The in-house experience allows me to understand better how organizations work. And the stresses of organizations. For example, there is no such thing as a blank checkbook. Things are always going to have to be prioritized, whether it’s in a corporation or a municipality. Understanding that helps me, when I go in to help an entity, to say: “In a perfect world, if we had $15 million, we could be prepared. But we don’t have 15. We may have 5. So we’ve got to figure out how best to prioritize to where we’ll get the most bang for the buck.” Being in-house helped me understand that. It’s not just theoretical to me. I have been there. I helped make those decisions. That’s going to give me a level of insight when I’m helping a client that another lawyer might not have.
     The chief privacy officer experience helps because you understand how data works, how it flows, how it’s secured. You’ve worked with CISOs, you’ve worked with CIOs, you’ve worked with upper management. So you understand what needs to happen, from a response standpoint. Those are things that a lot of lawyers may not have. It helps the client because you can talk to them in a language that they understand, and help them make decisions in a language that they understand. You’re translating it in real time for them, and helping them synthesize the input and the information coming from the security vendor, from the FBI, from the Department of Justice, from their own CIO. And from finance, because you’ve got to talk about how you’re going to fund this thing.

CIN: In-house lawyers are not specialists. How can they prepare to help their companies shore up their cybersecurity before the next attack?
RH: You hire outside help to come in before something happens, to give you assessments to help you understand what’s going on. And then your outside vendors should be doing seminars for you every year or every six months about what threats are out there, what best practices are out there. How do you, as a lawyer, help your client? Your outside lawyers and outside vendors should be helping the organization do that. That’s what’s happening with the city of Atlanta. But that’s how you have to learn, because you’re not a specialist, and you don’t really have time to be a specialist.

CIN: Any other big lessons for companies from the Atlanta experience?
RH: You have to continuously be talking about this, thinking about it, and acting on it. An organization at a minimum should have a tabletop exercise around a cyber incident and response once a year. Best practice, biannually. Having conversations between the CIO and the CISO and the general counsel and the chief operating officer quarterly about where we are. Cyber historically has people lumped in under IT. I’d say the biggest takeaway for all organizations is that it’s not just an IT issue. It’s an operations issue, it’s a finance issue, it’s a legal issue, it’s an HR issue. All of these stakeholders need to get together regularly and talk about it, so that they can understand better what they need to be doing to keep the organization safe.