The Committee on Oversight and Government Reform released a fascinating 231-page report detailing the how and why behind the epic breach at the United States Office of Personnel Management.
Richard Spires, the former CIO of the IRS and DHS, remarked on OPM’s failure to take a data-centric approach to information security:
“[I]f I had walked in there [OPM] as the CIO―and, you know, again, I’m speculating a bit, but―and I saw the kinds of lack of protections on very sensitive data, the first thing we would have been working on is how do we protect that data ? OK? Not even talking about necessarily the systems. How is it we get better protections and then control access to that data better? ” What data was taken?A picture of the damage inflicted by the OPM breach is painted through a series of powerful quotes, like this one from James Comey, Director of the FBI:
“My SF-86 lists every place I’ve ever lived since I was 18, every foreign travel I’ve ever taken, all of my family, their addresses. So it’s not just my identity that’s affected. I’ve got siblings. I’ve got five kids. All of that is in there.”
It’s hard to refute the argument that this is the most devastating breach of all time given the scale and sensitivity of the data that was stolen:
4.2 million personnel files of former and current government employees 21.5 million security clearance background investigation files 5.6 millionThe background investigation files include things like mental health history, alcohol abuse, gambling issues, and other deeply personal information.
How OPM happenedThe landmark event that everyone thinks of when they hear “OPM breach” is the theft of 21.5 million background investigation files from the Personnel Investigations Processing System (PIPS) a legacy mainframe that stores the organization’s crown jewels. This breach was disclosed in 2015.
However, a file share breach disclosed back in 2014 appears to have played an instrumental role in the eventual PIPS breach. In fact, investigations showed that hackers had access to OPM’s network since July of 2012 and were discovered only after advanced monitoring was enabled in March of 2014.
Regrettably, we’ll never know the extent of documents exfiltrated prior to March 2014.
On March 20, 2014, the Department of Homeland Security’s Computer Emergency Response Team (US-CERT) informed OPM’s own response team that a hacker had exfiltrated OPM data from the network.
To “better understand” the threat posed by the hacker (referred to as Hacker X1), OPM monitored the adversary’s movements for two months until they discovered a second hacker (Hacker X2) who gained initial access using a contractor’s stolen credentials.
Brendan Saulsbury, an OPM contractor with OPM’s IT Security Operations, says:
“So we would sort of observe the attacker every day or, you know, every couple of days get on the network and perform various commands. And so we could sort of see what they were looking for. They might take some documentation, come back, and then access, you know, somebody else’s file share that might be a little bit closer or have more access into the system.”
Hikit and SMBHacker X2 dropped Hikit malware to establish a backdoor, escalate privileges, and perform keylogging. Hikit was found on numerous systems and was beaconing back to a C2 server. OPM sniffed the hacker’s traffic to determine what was being exfiltrated.
Activity logs showed that the hackers would logon between 10 p.m. and 10 a.m. ET using a compromised windows domain administrator account and search for PII on file shares using SMB commands.
OPM watched a hacker exfiltrate documents from a file share which contained information that described the PIPS system and how it is architected.
Appendix D of US-CERT’s June 2014 incident report describes the stolen file-share data:
OPM’s Director of IT Security Operations, Jeff Wagner, testified:
“In 2014, the adversary was utilizing a Visual Basic script to scan all of our unstructured data. So the data comes in two forms. It’s either structured, i.e., a database, or unstructured, like file shares or the home drive of your computer, things of that nature. All the data that is listed here, all came out of personal file shares that were stored in the domain storage network.”
The value of the data known to be exfiltrated was initially dismissed as being fairly inconsequential, but the US-CERT investigation report makes it clear that the hackers were doing reconnaissance on OPM’s file-sharing infrastructure in order to get closer to PIPS:
“ The attackers primarily focused on utilizingSMB [Server Message Block] commands to map networkfile shares of OPM users who had administrator access or wereknowledgeable of OPM’s PIPS system. The attacker would create a shopping list of the available documents contained on the network file shares . After reviewing the shopping list of available documents, the attacker would return to copy, compress and exfiltrate the documents of interest from a compromised OPM system to a C2 server.”When asked if the documents exfiltrated from the file shares would yield an advantage in future attacks, Wagner replied:
“It gives them more familiarity with how the systems are architected. Potentially some of these documents may contain accounts, account names, or machine names, or IP addresses that are relevant to these critical systems.”
Not so trivial after all.
After conceding that the hackers were getting “too close” to PIPS, security ops decided to “boot” the hacker in an operation called the “Big Bang.”
They successfully booted Hacker X1 in late May 2014, but Hacker X2 maintained a foothold, traversing thecyber kill chain en route to the famous PIPS breach:
“Beginning in July through August 2014, the Hacker X2 exfiltrated the security clearance background investigation files. Then in December 2014, personnel records were exfiltrated, and in early 2015, fingerprint data was exfiltrated.”
A stunning lack of visibilityUS-CERT identified numerous gaps in the OPM’s centralized logging strategy:
“Gaps in OPM’s audit logging capability likely limited OPM’s ability to answer important forensic and threat assessment questions related to the incident discovered in 2014. This limited capability also undermined OPM’s ability to timely detect the data breaches that were eventually announced in June and July 2015.”
The big takeaway from US-CERT’s gap analysis is that traditional security strategies have a severe vulnerability when it comes to insider threats. By Jeff Wagner’s own admission, OPM had focused heavily on perimeter security, but lacked the technology necessary to detect and stop attackers who were already inside.
The report outlines OPM’s history of inadequate security controls and failed audits:
2005 the Inspector General (IG) gives OPM a bad security grade, says they’re vulnerable to hackers FY 2013-2015 OPM’s IT spending is at the bottom of all federal agencies 2014 the IG says “material weaknesses” have become “significant deficiencies” 2015 despite a mandate, only one percent of OPM employee and contractor accounts were required to use multi-factor authentication 2015 (post-breach) IG still sees an “overall lack of compliance that seems to permeate the agency’s IT security program.” Why all CISOs need to pay attention to what happened at OPM
