Slashdot Mirror
Target's Internal Security Team Warned Management
david.emery writes "According to this story, Target's own internal computer security team raised concerns months before the retailer lost millions of credit card numbers in an attack. (Quoting a paywalled story in the Wall Street Journal.) Target's management allegedly 'brushed them off.' 'At least one analyst at the Minneapolis-based retailer wanted to do a more thorough security review of its payment system.' This raises a more general question for the Slashdot community: how many of you have identified vulnerabilities in your company's/client's systems, only to be 'brushed off?' If the company took no action, did they ultimately suffer a breach?"
Yes, there are horrible security flaws where I work. Things as basic as changing passwords on a regular basis have been brought up repeatedly, and the answer is always, "we can't make people do that", or "that's something to keep in mind for the future, but we have more important things to worry about"
I have several times. It was never much of a concern of the client. Luckily we were never breached to our knowledge, but several others around us and in our field were breached and made big national headlines.
Oh well... we speak, they don't listen, screw them.
Years ago I worked for one of the two big American cable companies currently merging. I identified a security flaw in the public facing side of their customer service portal, essentially giving access to all the config files, which contained admin credentials in plain text. I proposed simple solutions, like not allowing directory listings of folders, among others.
They shrugged it off, and to the best of my knowledge, last year the vulnerability was still accesaible
Predicting which concerns will be used in an attack is the real game.
Actually I should have said... once those breaches occurred, our bosses were uber-concerned. For about 2 weeks. Then nothing again.
None of them care about security unless you're willing to fix it for free...
Posting as Anonymous Coward, for obvious reasons! (Hi NSA analyst! It's me again!)
Large company here, sales in the hundreds of millions of dollars.
Me: "OK, we need to make an audit of that B2B web application of yours... Something does not look quite right..."
VP: "What do you mean an audit? This application has been working without any problem for the past 3 years!! Stop bothering me with your lame ass paranoia, you slacker!"
Me: "Errr... One of your main clients juste contacted me, and wanted to know why every order he has ever made through that site can be downloaded in PDF from this directory? Unencrypted?".
VP: "Still not a major problem! Let me know when you have something serious!"
I still work there by the way.
This has all the hallmarks of the beginnings of a civil suit for negligence, and if it can be proven that the flags were raised based on actual break-ins and were ignored, possibly criminal negligence. The only place in Target I'd want to be right now is in their legal office - they're gonna be putting in some overtime soon.
There are security concerns in every company, without exception. Obviously, even the NSA itself had inadequate security!
Yes, many times security concerns are brought up, and brushed off. But this is not necessarily an indication of a problem. Every security risk must be weighed based on the likelihood of occurrence, and the severity of the impact, should it occur. Many of these calculations are inexact, and must be based on incomplete information.
Should Target have protected themselves better? Probably. But hindsight is 20/20. The difficult part is to anticipate the problems that might occur, without crippling your organization through impossibly tight security.
There is there problem they are fairly computer illiterate, I've dealt with many FBI computer forensic specialists whatever's that are dumbfounded by a .tgz, unix line endings. Hire out of the Secret Service they understand computers.
No sir I dont like it.
They'll run with the vuln until they get caught. It's a risk worth valuable money.
I remember one system someone was trying to break into. I was sitting in my office, with a coworker, watching the traffic and everything. Very entertaining. They had walked into our honey pot.
A feeling of having made the same mistake before: Deja Foobar
Stupid cookie-cutter MBA pindicks.
They were the jocks in school who got ahead because of their aggro and ego, but not their brains.
Guess what? They're now our bosses.
Years ago I worked for a government IT department. A vendor wanted us to try out a product. The device plugs directly into the Internet connection, and monitors every packet, in real time, looking for strings matching an array of string that you provide. We ran queries against our internal databases, and compiled a list of SSNs and CCNs. The vendor programmed that data into their device, which from what I can tell used an FPGA to perform deep packet inspections.
We expected that we might see maybe an email every week or two where someone accidentally sent that kind of information.
First hit occurred 12 seconds after turning the device on.
Second occurred .47 seconds later.
Etc. Etc. Etc.
Within an hour, we had overrun the quota on the network directory where we were logging this data.
We found hundreds of separate systems that were transmitting this kind of data without authorization. We were planning a massive internal sweep to find and fix them all, when the following came down from management:
Shut it down. Remove the device. Destroy all logs, emails, EVERYTHING. Offer the vendor a payment in return for signing an NDA. All employees required to sign secrecy docs (unenforceable at that level of govt, but still.)
I believe this is how the acronym SNAFU came into existence.
--- Generation X: The first generation to have SIG lines inferior to their parents... ---
I got my first job in the industry due to that sort of screw-up. A network administrator was "let go" following a server crash and loss of months' worth of data. The backup system hadn't been working. I was hired shortly thereafter to get things back in order.
Now, that would be the end of the story, except that I was good friends with this administrator. The embarrassing subject of his dismissal didn't come up for about three years, but when it did, and I mentioned my surprise at a fairly intelligent guy allowing backups to lapse for that amount of time, he dug up an e-mail he'd sent to the president of the company, cc'ing the head of HR (who was more or less running the show, for some reason), pointing out the various problems they had - their "server," an old workstation, had been running for two years on a three-month evaluation copy of Windows Server 2000, there were no backup tapes working, and so on. The only excuse they could have had was that the backup thing was buried in a page-long list of serious issues. But when it blew up in their faces, they pinned it on the closest available peon. Assholes.
We have all recognized security breaches or system vulnerabilities and been given the brush off. Nobody in the business world wants to be proactive. If a business has never been hacked then security will remain lax until that company is finally hacked. Even then most companies will just do enough to take away (or make it seem that they have taken away) that particular attack vector. (Hope nobody minds that I spoke for all of us).
No sigs in BETA. Beta SUCKS.
This is a frequent occurrence. I used to get upset about it. These days I have seen enough of these exact type of situations blow up that I am content to document my observations, report them to the appropriate people (always a direct supervisor), and then move on with my life. When things blow up, I am covered.
Situations like this are why, although I understand security, I will never work in a security position. There is too much risk and liability, and not enough support.
A lot of companies flat out ignore security concerns and think that a simple firewall is sufficient and the latest security setup. I worked for a company that was storing a lot of PII and had basically no security (let alone understanding of security). They wanted to render arbitrary images received through email and I had to explain to them that images in fact can contain viruses and other bad things.
I picked up maintenance of an application that had been built by one of the military business units. For the longest time I couldn't figure out how it was passing user credentials and session state, until I found it all contained in a 2,000 character URL string. That string included the administrator username and password, in plain text.
Instead of being grateful that I raised a red flag on the application security, they tried to insinuate that I was blaming the previous developer. They also insinuated I was being unethical.
That's what happens when you try to do the right thing.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
I'm sure that Target will address the issue by firing all of the management that brushed off the security researcher's concerns and will promote that security researcher to the head of a new task forced aimed at increasing their security and give him a huge pay increase (and maybe a pony).
But what really irks me the testimony that retailer's CTOs gave before congress.
Neiman Marcus CTO:
"I think what we've learned ... is that just having the tools and technology isn't enough in this day and age," Neiman Marcus Chief Information Officer Michael Kingston told the panel. "These attackers again are very, very sophisticated and they've figured out ways around that."
Translation: "We did everything we possibly could, those hackers are just too damn smart. You should probably pass some laws to make knowing how to hack illegal."
Target CTO on if they knew about the attack before they were notified:
"Despite significant investment in multiple layers of detection that we had in our systems, we did not," Mulligan replied.
Translation: "It isn't that we got caught with our pants down, we were doing our best, honest!"
There is just no accountability! Why were there even congressional hearings if congress didn't even do an investigation and call in experts to find out why Target fucked up so badly? Senator Tech. Illiterate (D) and Representative STICKYKEYS (R) don't know enough to call bullshit.
There is no penalty for ignoring your engineers when they bring up problems. Investing in security is a well known joke amongst CTOs. Target's bottom line isn't going to be affected by this in a year. The business world learned a lesson recently: you can lose 100 million people's credit card data and nothing bad will happen.
Can't speak to Target, but for future people who are in this predicament, now you have a great case study and example to point to!
Given that all that was done was to re-issue credit cards to the 45% of Americans affected. What does Target have to pay? And so what if a fine is paid? The end result is, "What do you remember?" Try Target, and Credit Card. How much is that free advertising worth? Billions?
We ran queries against our internal databases, and compiled a list of SSNs and CCNs.
Everything one does with the Government - Taxes to job applications - requires one to enter their SSNs, put it on forms and even put it on their resumes. If HR was sending Resumes to managers or forms or anything, of course that software had all those hits.
In other words, from what I can see, unless there is a policy to remove SSNs or CCNs (whatever those are), I don't see what the deal was outside of the requirement that everything needs an SSN - which is by law, so it's Congress' fault there.
There will be reports, studies etc. that all pointed to this retarded situation within Target. Cripes, any myopic goofball from Deloitte or Accenture could have spotted the problems from 1000 miles from space but it just goes to show how stupid management can be because ultimately it'll wind up on their doorstep. You'll obviously have a few sacrificial lambs too from the cyber-security team and management and bad news for other companies they're probably updating their resumes now. Yes retarded security professionals are available for hire in your area! Shit, we're screwed.
There's probably holes in their infrastructure that you could drive a truck through. How the hell can an HVAC contractor's credentials be used to eventually access their payment infrastructure? It's absurd in and of itself points to the fact that these idiots were doing it wrong and should be fired. Of course there will be fines from the Feds and those banks that have had to now had to deal with all the card re-issuing and the credit monitoring. I've had two cards swapped out by my bank because I shopped, one time, at Target during the supposed breech window. I haven't been back since and that in and of itself will probably do the most harm to them because if they're not secure with my data, especially payment information, then I won't be a customer.
Harrison's Postulate - "For every action there is an equal and opposite criticism"
This exact same story could be told at any company that has been compromised. There is always someone in security trying to push management for investments in security. It is very difficult as management to determine the real need, because clearly the real need is not the $ and resource amount security is proposing, and not zero either.
When you have lot's of outsourcing / contracting / subcontracting they don't want to pay the costs of doing stuff right no they want fast / cheap.
Places where I've worked that users were required to change their password regularly invariably had the same password but with an incremented number at the end every time they needed to change the password. This allowed them to remember it more easily, be effectively meant they were using the same password.
The more stringent that the password requirements become, the more likely it is that users are going to start writing them down somewhere or trying to come up with workarounds so that they can remember them. And in turn, you have another security issue.
Everywhere I have worked has also have a review of brute force password hacking attempts. :-)
We had complex installations of Linux servers that were so old that patching them often required a lot of work to be able to compile the fixes.
After a steady flow of layoffs and cut downs, I was no longer able to keep up with even just the maintenance tasks and the list of critical things that needed fixes grew longer. And forget about trying to find time to do proactive things like planning new systems or capacity planning, since I now had to do everything myself.
So I had informed my bosses of the problems, even had it in writing although I hate that CMA crap. But I ended up quitting because even though a hacked web server would not be my fault, I just could not sleep well at night.
Of course then there also was the problem with the rest of the company growing tired of the lack of progress and quality of the IT department. They quickly forgot that the staff had been reduced to half and still expected the same service they got earlier on even though the official word was that everyone would be understanding that we didn't have time to do as much as we used to.
They offered me a raise if I stayed but it was really not about the money but about my health.
Dutch government tax system; only specific (high clearance) government employees should be allowed to see income info for VIPs (celebrities, high officials, etc.). Instead, anybody who could access non-VIP info could bypass this additional authentication without effort.
Warned multiple officials about this, none of them found it important enough.
This was a few years ago, so it may have been fixed. Then again; the leak was already in there for a few years when I found it.
As long as you keep them in your wallet then writing them down is fine.
You're MUCH more likely to be aware when someone steals your wallet than when someone steals your password. So keep your passwords in your wallet if you cannot remember them.
Similar for home systems. Keep them safe at home. Criminals breaking into your home to steal stuff are not USUALLY going to be looking for a piece of paper with your passwords on it.
I have brought up several cases like this to various employers over the years. Typically it is just struggled off. A couple of times I've basically be called a liar (probably so they can pretend ignorance of the problem if a breach happens). Sometimes I can understand management not wanting to tackle securiy matters, there is a cost/benefit balance to be made. But sometimes I have pointed out a serious flaw that would have no side-effects if we fixed it and offered to do the work on my own time. The result was I was shot down and told to leave the issue alone. That bothers me and I cannot figure out why management would intentionally want insecure machines running when a fix would be cheap and easy.
but when you work with HVAC vendors who sub work out / are not really IT people. Then they may have a few fixed passwords / login's that they need to give out to all the people in the field it's much easier to have fixed one then giving each field tech own log in's that they may not even need day to day or even working at target all the time.
Keeping track of who works for each Contractor / Subcontractor down the line is hard and can be a lot of need less work of adding / removing users who may not even be on a target site but may work for a place that does some target work. Or let's say you have a tech who does not go to target sites all the time and the password times out on there next visit? or you have a tech who does not do target but needs to fill for the tech that does as they are tied up on another job and some needs to cover?
As a former US Navy nuclear engineer, I informed management of material and procedural problems related to the nuclear reactor plant on board the USS La Jolla on a weekly basis. Have you ever gone to your boss with a technical manual that perfectly explains the "unexplainable problem" he's having, have him brush you off, and less than a week later that problem destroys a major system, causing millions of dollars in damage and endangering the entire ship? I have. I'm pretty sure none of my complaints were ever addressed except on the one or two occasions where I threatened to bypass management and complain to a newspaper. That's pretty standard Navy leadership. When you're dealing with a culture where everyone starts at the bottom, the best and brightest leave, and whatever's left gets promoted, that's the kind of technical management you get.
The vulnerability used will be the easiest/first one that the attacker can find.
That sounds flippant but it is true. Most attackers won't even bother to map your network/systems. They'll just try whatever they have and use the first thing that works.
About 10 years ago, among other bad practices, managers in this one department of SAIC had a single FTP account that was shared by many personnel, and even outside parties, which was used as a dropbox of sorts. Essentially some very sensitive data was easily accessible to many people who shouldn't have access to it. Customers could see other customer's data, etc. I had mentioned it was a bad idea, but didn't push it, as previously when I had taken a hard line about handling of some other username/passwords in the past(leaving the entire staff's windows name/passes on a printout on a table for anyone to see), I nearly got fired because the managers were offended that I told them it was a terrible practice.
Couple years after I left, I heard that someone(probably one of our military clients) found out this FTP account was going on, and things hit the fan. SAIC brought in a bunch of lawyers and interviewed the dozen or so staff in this department under the premise of "protecting the employee's interests". After about two weeks of this, they cleaned house, fired every last person in the department.
So they get their own network that does not touch the production network.
Probably just a *DSL/cable from a local ISP.
With a firewall that you control. Heavily locked down. No need for them to hit Facebook from the HVAC, is there? No need for inbound access from 99.9% of the IP addresses out there, is there?
Then paint it and label it and make sure no one else can touch it. Use super-glue on the ports.
The downside of changing passwords is that it makes them harder to remember. And if you force people to change their passwords regularly, they're more likely to choose easy-to-remember -- and easy-to-guess -- passwords than they are if they can use the same passwords for many years.
My current day job requires a new password every 60 days or something.
I finally got tired of having to call the Helpdesk to get my password reset every time I forgot the new one (EVERY TIME I was forced to change it), so the last time I called them, they reset my password to "Helpdesk1". Now I just increment the password's digit by one when I'm forced to change.
I'm not going to memorize a new password every n months. You want it secure? Let me use a software authenticator or a hardware thing like SecureID.
How many of anyone has been brushed off by management for what they thought were serious safety or security concerns. I know I've been there for both cases. Management however was more interested in other stupid crap than doing things right.
Someone please tell me what HVAC contractors could possibly need to do on a corporate network. This sounds totally insane. If a company has 74 million people's credit card information on the same network that HVAC contractors can access, something more powerful than flamethrowers are needed to clean up that kind of crazy.
Changing passwords regularly has been found to be more of a security risk than not changing them and having a more restrictive password policy. I've had informal discussions with a number of people at various companies where password changing every 3 months causes more problems with lost passwords and written down passwords.
Yeah, let's install winzip on all our unix systems so we can use Windows Explorer to view the archives. What, winzip doesn't work on unix? Then let's install WINE so we can use WinZip and Windows Explorer together!
Problem solved!
I find things like XSS, CSRF, etc. during my normal line of work, report them, and they get fixed. 80% of the time that I find these things in our web apps, it's due to a developer "duh", and it gets fixed during the testing cycle of the app, BEFORE the app is in production.
I don't deal with retail/POS systems, but I do deal with web apps that have financial and/or personally identifiable information processed through them, and in my experience, management has a very mature attitude toward these kinds of things. First of all, when the shit hits the fan, nobody gets fired. Second, the team generally treats security defects in two categories: exploitable, and best practices. If it's exploitable, there's usually a fire drill to get it fixed and updated on the live site ASAP (or if the vulnerability isn't live, ensure that it's fixed before it goes live). If it's not directly exploitable, sometimes it goes into production with "weaknesses" or "best practices" issues, and remains that way for several years sometimes until it's fixed. Note that some of these weaknesses can lead to breaches if they are combined with other weaknesses or exploits, but in general, there needs to be some kind of injection, XSS or CSRF vuln as an entrypoint, and these three classes of exploits are treated very seriously and expeditiously addressed.
Maybe we're an outlier, but we pay people to test for security issues and we react to them in a mostly sane way. I haven't ever heard of someone getting brushed back or told to hush if they find a security problem. Worst case, you report it and the risk assessment folks decide that the risks are not great enough to warrant the effort of fixing it -- but that entire decision-making process is recorded, so if it blows up in their face, we know who to blame.
In some cases out side vendors / contractors have shaded / fixed accounts / passwords.
1: Target is paying for new cards and for any fraud perpetrated by the cards that got hacked because the card carrier network sure as hell ain't going to be paying for that; they have the numbers used at target, it's a quick SQL query for them and then you crossreference costs of successful chargebacks. Target is being told in backroom meetings either they pay or they get dropped. And believe you me, they will pay, because if they get dropped, business will either switch to another network or more than likely go where their card is taken. Imagine if tomorrow Visa dropped Target. Would you go get a master-card to shop at target or just go to wal-mart?
2: There's a concept of 5 Nine's for security as well, and most companies buy 3. Unless you are a bank you are not buying 5. The additional 9 is a clean double to quadruple of cost, and one of the things you do as a business is you buy 9's until the insurance company tells you to stop . Also, their business insurance just doubled.
3: The market is more than capable of using the courts to clear small-scale financial fraud. I hope you realize, Involving congress in most things these days has negative results.
4: If I were in Target's IT department I'd be job shopping right now, because the first thing execs do is blame everyone down the hill. I repeat: it does not matter if you have CYA Material, if the blame falls on the CTO, they'll find a way to make it your fault. Lookit all those IT positions open on their website (hint: search for business analyst). They just cleaned house.
http://targetcareers.target.com/search?q=IT&filter=true&locale=en_US&title=analyst
5: Businesses always skirt spend on security because the cost justification is just not there. I will guarantee you nobody in their IT department ever called up VISA and asked them what the cost would be of a complete breech, and stuck the e-mail into a power-point. At best you're the hardest target and get some business when your competition gets hacked.
I have never experienced this. It's mostly because if we DID have a security flaw, the implications would be far worse than stolen credit card numbers. Think hacking/controlling critical infrastructure. If I think there is a security flaw, I will be taken seriously.
The bad side of this is that we have some pretty paranoid information systems and policies in place which can interfere with productivity, but usually we can find reasonable tradeoffs and accommodations to make everyone mostly happy.
There's the default way -- self-absorbed managers deliberately ignoring and not understanding security warnings, wanting to keep earning bonuses for all the money they saved, etc.
Then there's the alternate explanation, IT security people seeing threats without any conclusive proof, wanting to spend a metric ton of money, expand their empire and cause a bunch of disruption that might not even accomplish anything but create chaos and complexity.
I've seen both. It's easy to see how this could be a combination of both with neither side really able to claim they were right. While there were obviously security problems, were these specific vectors the ones the security people saw? Or did they want to go on some kind of fishing expedition with little to show for it or implement a bunch of costly changes "because security"?
While management is easy to caricature as self-serving and incompetent, Target is generally a well-run company and it's hard to see their management purposefully ignoring concrete security weaknesses that could cost them maybe billions.
My guess is its probably a long-term case of all of the above. Too many managers exposed to 3Li73 53CUrI7y who just made things difficult with no concrete improvements or any attempt at usability and too many hard-working IT/security people who put up with managers that cover for weak security simply because they don't understand it and don't want to spend the money to fix it because it will either cost them personally or professionally.
That was at a big investment bank.
They offered me a raise if I stayed but it was really not about the money but about my health.
Only noobs and suckers take the counteroffer, unless it comes with a contract and golden parachute (e.g., 3-6 months salary upon termination, regardless of cause).
More money can make a crap job okay...for a while. Like a meth addict, you'll need to keep amping up (even more money for the same job) or you'll burn out, hard.
Which is a perfect example of incompetence.
*raises hand* ooh! ooh! Pick me! Pick me! Been there! Done that!
Two things:
1. It's not that they need access to the CORPORATE network. It's that they need access to the INTERNET so that the machinery can report back to the vendor when something starts to go wrong. That's usually in the service agreement. The sooner detected the sooner fixed without problem.
2. For managers who like to look at stuff. There is usually an internal web server on the HVAC. You go there and it displays things like the temp and the humidity and blah blah blah.
Thus, dumb managers (I've dealt with them) want them on the corporate network. It's easier for everyone.* Including the crackers who are looking for these exact vulnerabilities.
*Security people are not included in this definition of "everyone" in this case.
4 months before Microsoft SQL Slammer http://en.wikipedia.org/wiki/S..., we let management know about it. They still railroaded the engineers who found it. They probably would have been better off if they had not found it before release.
I strongly suspect this is not a hindsight problem whatsoever. The problem is that long term risks are usually weighted against short term gains: personal bonus clauses/promotions triggered by a run of street-beating financial quarters.
There's also the problem of risk hacking, where management willing trades the possibility of a huge setback against the likelihood of a good run of beating par.
With a long enough track record of success, even the big boom which erases more than your accrued margin over par is all too easily swept away under the hindsight carpet.
The only way to get correct risk trade-offs is where the people making these decisions are stuck on "long term hold" in their reward structure. This usually ends up being the founding entrepreneurs and first round employees who are quietly vesting. While these groups have influence, it's not usually enough to deflect the Venture Capitalist's hand-selected upper management team, hugely incentivised around servicing the VC's priority access to the sell-high exit ramp.
Unless you think Target was an inside job, your appeal to the NSA's woes (self-inflicted for entirely different reasons) falls a little short here.
There is very nearly no defense possible against the insider perfect crime. Of course you can always find some neighbour who describes the fellow as a bit suspicious. These are the same people who believe in the nun bun.
Perhaps the brain scan will be soon invented where this worrisome component of free human will can be exorcised from the system with 20/20 foresight. This won't be a good development for human society, in my humble opinion. 20/20 foresight is the planet of the damned.
The entire ecosystem of credentials is a catastrophe. The correct system is NTSC: never twice the same credential. Then when Target leaks the unique credential upon which your transaction stream is based, it would be conceptually possible to permit class action lawsuits against damages incurred, both direct (cash out of pocket) and indirect (hassle and time).
There would still need to be centralized certificate authorities, but these organizations would have no other business model than getting security right. Suffering a Target breach would amount to an existential threat. Then the NSA becomes the correct standard of comparison.
Not in the same category, but I once convinced my boss that a certain type of attack was real by taking his user ID in our app. I'm mostly satisfied with how work treats security (and scalability, and other "no visible problem until things go haywire"-type things). It's been harder to convince folks to worry about UI polish and tidying up user-facing (and dev-facing) annoyances. Isn't an awful problem to have, I guess.
Years ago I noticed bad default passwords on a professional industry website. Think doctors or bar association, that kind of thing. So basically every one in the country along with their dues payment info and personal profiles are accessible through a simple mangling of their name.
I reported it and was ignored. It's still like that. Professionals indeed.
Cwm, fjord-bank glyphs vext quiz
Yabbut...yabbut!!! Wouldn't some big company like Target have someone on staff who knows how to firewall off a network just for the HVAC? Huh? Huh?
VISA had sent out warnings twice in March and August of 2013 after investigation into Barnes & Noble hack from previous year (2012).
They probably have several people who can do that. It requires some expertise but not a lot.
I can do that. And I still push for a completely separate Internet connection.
Because once it is on the corporate network it becomes very easy to make mistakes. People think they know more than they really do. Or that they understand the situation when they do not. And the processes that can be put in place to catch those mistakes require additional expertise.
ALWAYS design the network so that the next admin will not have to be as smart as you or as experienced as you or as knowledgeable as you.
"Alright, you've covered your ass now."
Management doesn't listen to technical staff recommendations!
That's why I use tin cans and string.
Time to cue: Getting the Brush.
Get thee glass eyes, and, like a scurvy politician, seem to see things thou dost not.--King Lear
Bleh... *why didn't you sue*
I mean, yeah, the US system is absurdly litigation-happy, but refusing to participate in it just gets you run over by it, and that seems to be what happened here.
You don't actually need to file the lawsuit, most likely - just point out that you told them this was coming, and they refused to do anything about it, and that you now hold documentation showing that you were terminated for something that was demonstrably not your fault (your boss's fault, in fact, though that's not necessarily something you need to bring up). Consult a lawyer about it beforehand, if you want, but it sure sounds like a cut-and-dried case of wrongful termination. Demand severance, not "for cause", vesting of remaining stock options, etc. in exchange for making their little screw-up go away, then go find a better job while living on the proceeds.
The wonderful thing is, this works even in at-will states. They could let you go with basically no justification at all (just as you could walk out the door at any time), but they can't fire you *for cause* when that cause is demonstrably untrue, or they put themselves on the hook to keep paying your salary for a long, long time.
There's no place I could be, since I've found Serenity...
I was the responsible IT manager, over all devs. admins, ops and security.
Reviewed all contracts and implementations, upon taking over the job.
Discovered some seriously, bad stuff.
Developed plan to *quietly*, discretely, repair over short time period.
"Rebury the bodies"
Turned out the responsible party was the CEO's favorite, "baby shark".
Got cardboard boxed. Out day after board presentation.
So it goes.
Interesting point:
All of those devs, techs and security people who moan about the lack of management support?
How many of you have ever supported or somehow defended *any* manager who tried to help you, to do the right thing?
Speaking personally, I would guess ... None of you. "Not my problem" attitude, up and down.
Maybe you have all been luckier.
(R)ule in Hell or (S)erve in Heaven [R]?
Department of Corrections... Prisoners using the DOMAIN ADMIN account to logon to PCs! I got in trouble for reporting it.
Hey, I work for a fairly unknown little marketing firm, but we recently deployed a website for a major airline that uses vanilla HTTP for user login to the site. That is, the username and password are sent to our site unencrypted. Said site provides the ability to book flights and to CANCEL them. I flat out told my boss that the setup we had was "patently non-secure", and that it made our company (in which I hold stock) vulnerable to huge liabilities.
He told me the site was "good enough". In many ways, my boss is more intelligent than even I am, but it just floored me when he told me (and my peer, who generally respect my opinions) that the client would have to complain before we would do anything about it.
Sound about right?
The management who 'brushed off' the security staff should be held criminally liable. This goes beyond mere negligence.
'He who has to break a thing to find out what it is, has left the path of wisdom.' -- Gandalf to Saruman
yup and unencrypted dump file for the POS of sale system at Mc donalds (I was the IT tech) includes card details, sales and the stores VPN network ip addresses and passwords.
I was told "not to worry about it"
Its still there 5 years later with about 8 - 9 years of data the files over 15gb now.
A case in point is a phone guy who came in that used a UPS as a drink coaster (he came so close to being a crispy critter) and wanted telnet access to his device from the internet. The device had a username, which was the company name, and no password. Anyone who found the thing would have been able to reap the reward of international phone calls changed to the poor suckers that had bought the equipment if he has got his way.
There's a lot of people far out of their depth in such environments.
But unfortunately it's what all the cool kids use
See, this is the problem with companies like Target not having legal liability for such things.
Because if they were legally responsible for it, they couldn't just brush it off, do nothing, and then let millions of credit cards get compromised.
To me, the company should be paying a huge fine for what can really only be called indifference to security. If you can't safeguard our financial information, you should be penalized.
Otherwise there's never any incentive for them to give a damn.
Lost at C:>. Found at C.
So working for Orbitz World wide a few years ago we questioned why a comment out section of code was there. The code itself was a bity nasty - it was a function that would copy the users credit card details to a cookie to use for a later comparision (something we did in uat apparently to get around CC checks).
We raised it, got ignored until the function was implemented in production, anyway long story short a quick investigation by the operations team and we discovered that every user on the hotelclub sites were having their credit card numbers, ccv, expiry, name stored in a plain text cookie on their computer. Anyone who logged in later to the local machines could expose these details.
Yes, when I had just started at my company the tyrant CEO let me have view access of the firewall, but no access to make changes. I noticed one Friday afternoon that the firewall was getting hammered with attacks from Korea, Russia, China, etc..., so I reported it to the CEO. He happened to be out having dinner with his wife, and rushed through dinner to get home. By the time he did, the attacks had stopped, and I got my ass reamed for being paranoid and making stuff up.
Needless to say, our voip system had been hacked, and a few days later was shut down after they racked up a $15k bill.
No apology from him.. nothing... But I still laugh about it to this day. I also don't report anything,else suspicious, fuck them.
So let's say I work^h^h^h^hknow a guy who works for a company that has a large and very important public facing system.. the kind that he can guarantee everyone reading this interacts with most days, and it has some gaping holes in it that management have decided to not fix because it would cut in to their massive profits... what is he supposed to do to protect you without losing his career and livelihood in the process?
They probably have several people who can do that. It requires some expertise but not a lot.
Of course they have people who CAN do that. The better question is - do any of those people have the political clout to require Target to spend money and inconvenience managers and "essential" vendors to prevent a "theoretical" security attack.
If your children ever found out how lame you are, they'd murder you in your sleep
Target Corp.'s computer security staff raised concerns about vulnerabilities in the retailer's payment card system at least two months before hackers stole 40 million credit and debit card numbers from its servers, people familiar with the matter said.
Members of Target's computer-security staff raised concerns about vulnerabilities in the retailer's payment-card system before the massive hacking occurred. Danny Yadron has details on the News Hub.
At least one analyst at the Minneapolis-based retailer wanted to do a more thorough security review of its payment system, a request that at least initially was brushed off, the people said. The move followed memos distributed last spring and summer by the federal government and private research firms on the emergence of new types of malicious computer code targeting payment terminals, a former employee said.
The suggested review also came as Target was updating those payment terminals, a process that can open security risks because analysts would have had less time to find holes in the new system, the employee said. It came at a difficult time—ahead of the carefully planned and highly competitive Black Friday weekend that would kick off the holiday shopping period.
It wasn't clear whether Target did the requested review before the attack that ran between Nov. 27 and Dec. 18. The specific nature of the feared security holes wasn't immediately clear, either, or whether they allowed the hackers to penetrate the system.
The sheer volume of warnings that retailers receive makes it hard to know which to take seriously. Target has an extensive cybersecurity intelligence team, which sees numerous threats each week and could prioritize only so many issues at its monthly steering committee meetings, the former employee said.
"It is everyone's worst-case scenario," the former employee said. "As an intelligence analyst, there is only so much you can do."
Target declined to confirm or comment on the warning.
The breach has caused headaches for Target customers who have dealt with fraudulent charges and have had millions of credit and debit cards replaced by issuers. Investigators and card issuers haven't quantified damages from the attack.
The new details, culled from interviews with former Target employees, people with knowledge of the post-breach investigation and others who work with large corporate networks, show that the breach wasn't entirely a bolt from the blue, but instead a sophisticated attack on a known point of vulnerability.
Retailers last year had received a number of indications of dangers. In addition to the alerts from the government, Target and other retailers saw a "significant uptick" in malware trying to enter their systems, people familiar with the investigation said.
Still, the discovery of the intruder that ravaged Target's systems came as a surprise. Chief Financial Officer John Mulligan told Congress last week that the company wasn't aware the malicious computer code that carried out the attack was in its system until contacted by federal investigators late last year.
The U.S. Secret Service declined to comment. It and several private companies are investigating the attack.
At last week's congressional hearings, Mr. Mulligan said Target passed an audit in September that certified its compliance with payment industry requirements for protecting card data.
More broadly, Target may have not done enough to wall off its payment systems from the rest of its vast network, people who work with large corporate networks said. The company has since moved to isolate its different platforms and networks to make it harder for a hacker to move between them, a Target executive said.
The hackers, still unnamed, originally gained access to Target's network by stealing the access credentials of a refrigeration contractor in Pennsylvania. The contractor, Fazio Mechanical Services, has confirmed it was breached and is cooperating with the Secret Service investigation.
Fazio said i
For years, I worked at a company that sold its software to very large companies, some of which were government contractors handling top secret materials on airgap-isolated networks.
Our client and server communicated with each other essentially (but not exactly) by opening a connection directly to the database. You could do things like... set your own is_system_admin flag to "yes". Fetch data you shouldn't have had access to. Delete data that should have been retained for auditing purposes. All without any sort of logging.
We made management aware of this issue ASAP, and reminded them every time client/server communication came up. We made it very clear that, were the problem discovered and published by one of our customers, it would probably end up with lots of our customers leaving us. Also, it was just Very Bad Practice. ... But they never prioritized it high enough to get any real work done. What was always a higher priority was that next sexy feature that would land us the next customer.
Funny thing -- many of our customers were very security conscious and ran automated security analysis tools against our software (mostly the web server) all the time. They would find some minor issues, and a WHOLE LOT of false positives, but never the glaring security hole I detailed above. But, they'd gone through their bureaucratic process, so our software was deemed "secure" and let onto their networks.
Software security is a joke.
This is a general QA problem. It's hard to get management to listen to on going quality problems. They don't want to spend time on things that do not translate into a quantifiable cost savings or income generation. It take a lot of effort and time to sell the problem.
I see the glass as full with a FoS of 2.
I've been doing security analysis stuff for close to six years now. And I've got to say this article doesn't surprise me in the least. We'll notify customers for months and years on end about serious and silly flaws in their system. We so rarely see any real effort to fix stuff that it is always shocking when someone actually loads a quarterly patch, even if it is nearly a year out of date. I always have to give a nervous giggle when our leadership brags on how secure our systems are, because one day I know I'm going to walk into the office and find that some major system was attacked through a known security hole.
Where I used to work had a policy like that, and you are right, the number of post-it notes with !t$Feb2014 or similar you could find stuck around was incredible.
All companies do the brush off. Profitability is a balancing act between recovery, production, quality and safety. None can be 100% without impacting something else, even safety. Those of us in the support field are responsible for making suggestions and pointing out issues, but at the end of the day, all we can do is stand back and watch as the shit hits the fan, and hope we're not in the line of fire.
I'm going to post as a lovely AC, because I am fully aware of a security issue at my company.
Customer passwords are stored as plaintext and openly accessible to all employees. Personally, I have access to at least two individual remote access points that would grant me undetectable access to systems with credit cards using other employee's accounts. I pointed out the weaknesses to the last hire in IT, and he got canned in the first 90 days because he brought up the issue.
From what I've see of the financials, I'm sure someone is embezzling, but its a privately held, family owned company who don't want to pull their heads out of the sand. I just hope I can stick around until this house of cards collapses.
It is not just secuity, but software quality in general.
The good programmer is never given enough time to do the job the way they know it should be done.
The bad programmer costs less per hour, and is given more hours, but the result is even worse.
Add the changes management makes when the task is half done and you begin to see the full picture.
On the other hand, I have seen programmers sequestered for a couple years and come back with nothing.
It just inconvenience users, and encourages them to write down passwords on sticky-notes and such.
Target can not claim that they are not responsible for damages that have taken place when they have denied requests by qualified experts to make the system secure as it was inadequate. Big bucks may flow from this irresponsible company to the victims of the breach.
We moved into a new build office several years ago and one of the things I specified for the computer room was a locking door. As the fitters were finishing up I pointed out that the lock hadn't been fitted and was told that it was dropped to save money. It would have cost about £120 with fitting. About six months later someone wandered into the computer room one evening while the cleaner was the only one in the building and stole our nice shiny new server. The computer room lock was fitted before the replacement server was delivered.
>Make the CIO, CFO, and CEO cough up a few million per breach and they will be stopped. Close companies that are breached repeatedly, and make the directors reimburse the other stockholders out of their own pockets
I didn't see any mention of the analyst raising specific issues that needed to be addressed. Without those, I can totally understand them getting brushed off - I work with a lot of utterly useless people in a large financial company's IT department - these are people with liberal arts degrees who have never written a line of code in their life, yet somehow they've infiltrated IT as "project manager" types who don't even understand the thing they are "managing". Anyway, they manage to stick around by having 2-3 "go to" questions/responses for everything. - "I think we need to add more time to testing", "I'd like to be more inclusive and invite everyone to the meeting" (to justify wasting engineers' time on endless status calls, which is all these people do), or "We should make sure we have security review this change" for every change, no matter how small, even to systems that don't hold any data of value.
People who constantly cry wolf without naming a specific addressable are actually a detriment to security, and they numb everyone to real security threats and turn security into a CYA exercise where no one wants to surface issues for fear one of the wolf cryers will start the blame game, as opposed to coming up with real solutions.
Target, like all large modern businesses, is run by babies. Managers in American companies are no longer grown-ups. Proof is that they never take responsibility even though they are responsible.
Even if it is someone else's fault they are STILL responsible because they are in charge. That's the difference between a manager and an employee.
Grown-ups take responsibility. Children blame others.
Real security costs a lot in both in productivity and in dollars.
Customers, for the most part, are unable to tell the difference between a supplier that has good security and bad security. (And yes, some high cost suppliers have terrible security, there's no guarantee that better price means better security).
Guess which companies can offer a cheaper product?
From observation, I'd say that most suppliers who took security seriously have gone under. Everybody promises excellent security, so from the customer perspective, they're identical, and thus naturally the customer chooses the cheapest. If there's a security issue, then the customer gets ticked, and switches to another low-cost supplier.
And of course, the same applies whether the customer is a business or an end-user. When was the last time you checked the on-line security of the store where you went shopping? The only survival strategy is to go cheap, and pray you don't get unlucky.
(Of course, the reality is a somewhat more nuanced, but the pressures are absolutely in the direction I describe.)
In a few places I've worked - large financial corporations - the process for setting up contractors was smooth, far more than for regular employees. Part of that 'smooth' process appears to be a carte blanche with regards to a *lot* of security process. At one institution, I started as a contractor, and made the jump to full-time a year and a bit later. Over the next several years, discovered I had better access to nearly everything - physical access, systems, heck, even admin access to production applications. Scary. So much for the 'You only have access to what we explicitly give you access to.' The concept for contractors seems to be 'Give them everything; they'll be busy on one project; then we dump them before problems arise.'
There's also the problem with CYA. People/groups become obsessed with following the letter of the bureaucracy, rather than doing what is correct. Security is - mostly - manned by near-technical incompetents ... you don't want to give that kind of power to people who could abuse it, do you? And so it becomes harder and harder to get them to actually do anything.
A line from one of my first security courses becomes more and more relevent: When security becomes overly onerous, people will develop ways around it.
Pathetic password 'rules' leading to perfect examples: people writing passwords down; shared IDs/passwords; and 'rules' that make cracking passwords easier
A system I worked on had passwords that *could* be any length, up to 16 characters. The rules specified 8 characters, one number, one capital, and one of @, #, %. The first gives something like 5*10^29 combinations, the second - with rules - only 9*10^9 ... because, hey, the rules make things more secure, right? (Worse, some users connections automatically translated two of the symbols, which precluded their use. Dropping the number of valid combinations by a factor of three)
Our company recently was hit thru our PBX VOIP phone system. The attackers got in through a flaw in the configuration of unused voicemail extensions and the "feature" of allowing users to dial out from their voicemail. All unused extensions had their default passwords left set and the attackers seemingly were transferring data 20-30 minutes at a time to various Caribbean and South Pacific islands. For 7 hours, 3 pm eastern to 10 pm eastern, all lines were dialing out. I suggested to the System Admin that we power down the PBX, but they simply rebooted it. The phone company eventually flagged the behavior and disabled international calling until the PBX installer could test the patch and update the system. He admitted this type of attack has been happening more often with some of his other customers. Live and learn I suppose. We can't wait to see the phone bill!
I worked at save-a-lot and found an ordering bug where you could partially pay with food stamps, ask for a void on a certain item ( I forget if it would be food or non-food ).. and the total would start bumping in your direction. Eventually the computer could tell you "Here's your $5.96 change on $0.00, also take these groceries. Have a nice day". ...
I told the managers, the GM was apathetic and one of the assistant managers almost cared. i told him I posted it to my web, I gave him the address... but he viewed it in IE and I had an unclosed html tag so the whole page rendered blank for him. =)
I would tell some of the more pathetic food stampers about it, noone ever seemed care.
-qe2e
Identified multiple security holes/issues? Check.
Brought them to management's attention (multiple times)? Check
Brushed off or, usually, 'acted' like "oh that's serious, send me an email on it"... and then "lost" the email apparently (I sent them the same email 2 years straight, every 4-6mo's because I would mention it again and get "send me that list again")? Check
... end result? I got laid off, the bosses were still there last I knew, and number of those holes/issues fixed? Probably zero.
Posting AC for obvious reasons.
Worked for a small company as their only admin.
Found out that the lead dev wasn't sanitizing inputs for their main web application. The application was taking credit card info, and was incredibly hackable.
So, I fired off an email, explaining that working with CC info required that security needs to be the top priority when developing.
Got back a response from the lead dev that I don't set his policy or priority. Boss decided to stick up for him, so I pretty much said, I'll have to clean up the mess after you're hacked, but at least you're paying me by the hour.
8 Months later, SQL injection, someone took over the whole app. Lead dev quits. I start to clean up the mess with no help from the developers. 4 days later, another break in.
Didn't get an apology, but I got paid.
I work at a Fortune 500 company, and we are required to do verified static analysis every six months on all of our source code (with 30 to 60 days to fix anything considered a serious risk -- XSS, SQL injection, etc. -- which we avoid anyway), and manual pen testing bi-annually. We also have to adhere to PCI standards (via regular Qualys scans). It's a hassle, but it protects our data and our clients, so I'm cool with it.
Me: Since those users don't need shell access and Pop3 passwords are sent in clear text over the internet, be sure to change to /bin/true or /bin/false or something.
Linux Admin: [very coldly] I *know* how to secure a computer.
------ Some Months Later ------
Same Admin: They sshed in as one of my Pop users and then they somehow managed to escalate to root and then....
I used to work at a major software firm that produced time and attendance software and hardware. I found a way for a "hacker" to manually insert arbitrary time records into the central database without the possibility of detection. This meant that a person could write a simple Perl/PHP or whatever script to enter time records into the system as if they'd been at work even if they hadn't, and there was no way to audit them. Management's response upon seeing the vulnerability and proof-of-concept script? "Nobody's going to go to that much trouble."
I can related many specific examples of companies I've worked for that have done exactly this. From:
The list goes on. IMHO, IT security would be a lot easier and more secure if they got rid of an awful lot of managers.
Sure enough, the cow costume was hanging up next to the superhero outfit and sailors uniform. (S,Spud)
No it's not. I set up accounts for vendors with set passwords that can not be changed. Here you go this is you user name and password. The account is enabled for the next 2 hours and will then automatically be disabled. If you need it after that call me and I will re-enable it.
Actually, I'd love to plug that kind of data into my zabbix instance, so I can plot temperatures, power usage, on/off cycles and analyse what's going on and why. But that should be strictly separate from any POS or corporate network - setup a VLAN or two for vendor stuff. Ideally each should be separate anyways. There's plenty of subnets under 10.x.x.x - use them!
. Define sqrt(x) as something really evil like (x / rand()), and bury it deep. Watch your coworkers go nuts.
When we went from 2G to 3G, I noticed our security protocols weren't updated appropriately. At first I was blown off, I followed up for a few weeks, and then set back for awhile (1-2 months). With the public launch coming soon and the issue not being addressed, I changed tact. I had root permissions and access to the most sensitive servers, the billing server feeds, and these servers will break careers if mismanaged. So, I took a screen shot of a tracert from the public side of the network with a billing server as the successfully reached end point and emailed it to the responsible group. No explanation, just the tracert screenshot inserted at the top of the e-mail string dismissing my initial concern. Problem fixed in under 2 days from last E-mail sent.
and they brush you off. You did your job and the people that brush you off meets the consequences.