Slashdot Mirror
Target Ignored Signs of Data Breach
puddingebola writes "Target ignored indications from its threat-detection tools that malware had infected its network. From the article, 'Unusually for a retailer, Target was even running its own security operations center in Minneapolis, according to a report published Thursday by Bloomberg Businessweek. Among its security defenses, following a months-long testing period and May 2013 implementation, was software from attack-detection firm FireEye, which caught the initial November 30 infection of Target's payment system by malware. All told, up to five "malware.binary" alarms reportedly sounded, each graded at the top of FireEye's criticality scale, and which were seen by Target's information security teams first in Bangalore, and then Minneapolis.' Unfortunately, it appears Target's security team failed to act on the threat indicators."
In Target's defense, FireEye said it would have to restart the computer to remove the threats.
How can I believe you when you tell me what I don't want to hear?
I'd wager there's about an 80% chance someone said the following:
"There's no way someone could have infected the POS systems; must be something wrong with this stupid FireEye thing..."
Well, there you go.
It isn't clear (at least to me) how many false alarms they got before they got the real one. The key to a good security monitoring system is not just to catch all the real threats, but to not flag imaginary or minor ones.
Maybe they're just fucking idiots, with an IT department that either is utterly inept or had been so marginalized by MBA morons and sociopaths.
The world's burning. Moped Jesus spotted on I50. Details at 11.
Well it was a real knee slapper. /sarc
I'd wager it wasn't the security team that dropped the ball. I work in the same role (I'm the most senior member of the security team), and I can tell you first hand that I don't have the authorization to act in matters of that scope independent of the executive team in situations like those. I have to forward my recommendations up the chain and get approval.
That causes delays. Often times, things then get lost in the executive level. Whenever there are contractors involved it's even worse as they spend a week or so arguing over whose responsibility it is, who is going to pay for it, how much down time it's going to represent, how much money they're going to lose, etc,etc, etc. Executives are also really bad at judging risk when it comes to security. They'll expose themselves and their companies to staggering amounts of risk - if for no other reason - than the fact that the failure/security breach/what-have-you isn't impacting business "right now" but shutting down an ecommerce system to patch it will impact the bottom line *right now* and they would rather risk "maybe" losing money at some future date than know they're losing money "right now".
Executives will mortgage their companies futures at every possible opportunity for a few extra dollars today.
The number of times I've taken a GLARING security issue up only to have the "how long can we leave it before it impacts business" be their main concern. If it's a vulnerability on a production, WAN facing system - but we don't have evidence of it being actively exploited - it's not considered to be as critical as taking that system offline for an hour to patch/test it. The certainty of lost revenue in that hour is more meaningful than the potential of abuse at a later date. Worst part of it all is that when that later date does come around and things get really bad, they all point their collective fingers at the security team and none of them take any responsibility whatsoever.
You're damned if you do, damned if you don't and blamed all the way around.
Corporate InfoSec is a very, very frustrating occupation. I feel for those poor guys at Target.
The security team should have a license to kill from the executive team. We do, our instructions are if we believe we breach is in progress, "shut it down".
Mind you we have never done it. We came very very close to doing so once on a false positive. The operations team failed inform us of some activity they were going to be doing. Fortunately the guy answered his phone, but otherwise we would have pulled the plug and islanded the entire dmz ecommerce and the corporate home page and all.
After reviewing the after action report the executive team agreed and would've been right to do it given what we knew.
That is how it should work
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
I think the big problem is that 24 x 7 monitoring tends to be outsourced. It's not a good model. SIEM systems or good if anything to deserve human attention. But they either get so over tuned they don't really detect much of anything or they throw a lot of false positives.
Long as your in-house cert team is watching the SIEM that works they know the network. They recognize that radius server is likely to produce a lot of multiple authentication failed followed by authentication succeeded events against the domain controller because of the nature what it does. that's one to ignore but if it happened with some other server it might be a serious issue.
Now that monitoring gets outsourced to some CallCenter. They don't know the network. they escalate tickets for both events. Employees responsible those tickets are no longer 24 hour but they come in all day every day and all night. Most of them are crap how long until those guys stop jumping up from the dinner table to go check their PCs every time the phone vibrates?
Serious incidents get missed or not acted on until the next morning
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
Maybe they're just fucking idiots, with an IT department that either is utterly inept or had been so marginalized by MBA morons and sociopaths.
Or, with a name like 'Target', they were pretty much asking for it?
Their alerts are the closest thing to security magic I have ever seen. Their false positive rate is astronomically low and they really do detect brand new malware.
On the FireEye system I use at work if it alerts we take action. Always. For URLs they sometimes get it wrong but we see 1 false positive a year with binaries. That's way beyond impressive when protecting tens of thousands of particularly gullible users, it's downright witchcraft. We often find another systems like URL filtering, IPS or endpoint protection prevented a true infection but we always do the homework when FireEye triggers. When you have real confidence the security threat is real doing legwork to confirm infection is easy.
For Target to have ignored FireEye's data borders on criminally negligent. It's really common to dig back through IPS logs once you know something was wrong and find a trove of data about the attack. FireEye is something else altogether; it's the most actionable security intelligence I have ever seen. It's truly astonishing technology since it's so effective. It captures binaries and URLs from the wire (IPS-style), email (SMTP MTA) and file shares and runs them in VMs. If enough malicious activity is detecting like deleting itself, changing registry keys, or contacting suspicious or blacklisted IPs (along with lots of other things) the binary is flagged in an alert. It's prefect for filling in the gaps left by traditional antivirus and the noise of intrusion prevention.
Not only InfoSec, most warnings from the people who know up to the people who don't know, but have authority to act, or spend money are just ignored.
Several years ago I told Data Center management that a vital piece of hardware had reached end of life and needed to be replaced else we'd be at risk for a total system outage that might last for days.
They didn't want to spend the $30,000 dollars until they absolutely had to, so they ignored my recommendation. In the end, nothing bad happened, but it very easily could have and we'd have lost revenue in the millions of dollars, just so as not to spend money before they absolutely had to.
At comcast that is how we worked from 1998 to 2007 when I was there. Security breach? I can tell the CEO to fuck himself to his face and yank the plug. And at many times I saw executives escorted out of the data center by guards because they were being idiots demanding we restore internet access. Management are clueless morons, they must be left out of the loop for security.
It's why Cops dont have to call the mayor when they see a guy running into a bank with a gun in his hand and a big sack with a dollar sign painted on it.
Do not look at laser with remaining good eye.
It still amazes me that companies are willing to outsource or "right shore" their critical IT development and functions to third parties like this. Still, Target Management who have now been sent packing are ultimately held responsible, except of course the CEO and the Board who probably rubber stamped the deal because it could "save them money." At one time I held a senior position at a major transportation company and the first question during budget reviews with our CIO was "what are we going to outsource this year?" It wasn't about did it add value or add a capability we didn't already have, it was one of his initiatives that he received a bonus for meaning if he outsourced X% of what was considered "IT administration" he'd get his fat bonus.
There are a lot of competent IT outsourcing firms out there and they exist because IT isn't viewed as a value added function within a business like it used to be. Unfortunately with the competent ones there's a ton of incompetent ones. The real problem is the perception that it's just a few PCs and Servers in a closet and we don't change that much so why do we have all of this staff and budget. Therein lies the problem because the person responsible for that, the CIO, hasn't done his/her job of communicating effectively to their peers and the board about the role that IT plays in the organization. Sure, are there bloated IT organizations or functions that can be outsourced or eliminated? In most cases yes but that doesn't mean wholesale outsourcing is always called for or should be done at all. In this case Target fucked up and didn't have the proper management structure in place to address the problem when it was being pointed out to them.
Harrison's Postulate - "For every action there is an equal and opposite criticism"
These IPS/IDS systems literally generate more alerts (usually including a bunch of false positives) than you could possibly read in a day. Heck, it would take a year or two to learn up on in detail on each signature/threat they have in their catalogue; only people who specialize in security and keep up to date daily can make the calls as to what alarms are noise and what's indicative of real activity (no the default "levels" shipped with the product don't cut it, because if you only look at the "red" ones you'll miss important crap and there are too many "yellow" ones to look at). Those people generally tend to work at places that produce IPS/IDS products, not in support IT. So that means you almost inevitably end up with some misconfigurations or bad calls.
What generally happens is the PHBs buy oodles of this security software and vastly underestimate the amount of manpower and expertise needed to actually use the software. Some places just plop these things on the network team and somehow expect them to magically work even though zero man-hours have been allocated to read the logs and continue the ongoing process of tuning the event filters/reactors -- because after all if they installed it and got it working, the network team must know how to run it, right? Larger outfits may actually have dedicated "security" personnel. If those personnel are not busy implementing security measures internally and are of the strange types that won't shoot themselves in the head if they have to stare at logfiles continuously for several hours a day, that might work. What could also work is hiring professional services from the IDS/IPS company to tune your filters for you.
Someone had to do it.
The breach started two days before Black Friday. What incentive would management have to do anything that would jeopardize their ability to sell all the way until Christmas?
Levy a fine against them equivalent to their entire profit from November 27 until December 19 when they finally admitted the breach. Maybe companies will think twice before trying to sweep these things under the rug.
It could easily be alarm fatigue. After the 500 billionth 'red alert' that turned out to be someone checking their bank balance during lunch, a warning or 2 about a suspicious attachment can easily fly under the radar.
It happens in hospitals too and sometimes people die as a result.
It's funny how IT is a pure cost center right up until it suggests shutting down one of those pure costs for 5 minutes. Then suddenly it's "OMG NO! we'll loose bazillions!"