Ask Slashdot: Application Security Non-existent, Boss Doesn't Care. What To Do?

An anonymous reader writes "I am a senior engineer and software architect at a fortune 500 company and manage a brand (website + mobile apps) that is a household name for anyone with kids. This year we migrated to a new technology platform including server hosting and application framework. I was brought in towards the end of the migration and overall it's been a smooth transition from the users' perspective. However it's a security nightmare for sysadmins (which is all outsourced) and a ripe target for any hacker with minimal skills. We do weekly and oftentimes daily releases that contain and build upon the same security vulnerabilities. Frequently I do not have control over the code that is deployed; it's simply given to my team by the marketing department. I inform my direct manager and colleagues about security issues before they are deployed and the response is always, 'we need to meet deadlines, we can fix security issues at a later point.' I'm at a loss at what I should do. Should I go over my manager's head and inform her boss? Approach legal and tell them about our many violations of COPPA? Should I refuse to deploy code until these issues are fixed? Should I look for a new job? What would you do in my situation?"

Comments

EASY

[houbou]

Document your correspondences to your boss when you notify vital security issues. Make sure your e-mails are not only backed-up, but you get read receipts or something showing your boss opened the e-mail (and might have read it). Keep those receipts archived. When poop hits the fan, at least, you are protected.

Re:EASY

[Penguinisto]

All that, and it wouldn't hurt to print off copies of those emails (and his responses!) and take those home for personal storage. That way, if poop-meets-fan and they suddenly perp-walk you out (before you have a chance to reach for your backups or suchlike) you still have usable documentation - this is in case any governmental authorities get involved, a lawsuit springs from it, etc..

Printing also gives you the advantage of having backups that you can walk out of the building with and not set off any alarms, since many tightly-regulated companies lock down the use of USB sticks, external hard disks, and etc. (my last employer -- a web-banking software house-- would literally fire you on the spot if you got caught using a geek stick or external drive on their desk/laptop equipment or servers - at least if you do it w/o prior written manager authorization and only on authorized devices.)

To top that off, the printed copies are protection against an 'oops - our retention is only set to two weeks and the backups were corrupted somehow; sorry, sucker!' move. F500 firms generally blow away anything in the inbox that's more than a couple of weeks old anyway, so if you forget to archive it off to a .pst or another folder, it's usually gone by week 3, with no recourse.

Meanwhile, it wouldn't hurt to have a bit of a side conversation with someone in legal (for a start), then escalate it to formal conversations with them via email (again, print those suckers off) should nothing get resolved.

Re:EASY

[MillerHighLife21]

This. My last job was at an after market buy/sell/trade website where I got to take over the whole project mid-rebuild after the previous staff walked out/botched the job/etc. The user base was under constant attack from phishing, fraud, scams doing literally everything you could imagine including hacking accounts. The users complained about it constantly, people were losing trust in the site.

The owners only concerns were that I add new functionality. One of them wanted me to build a blog in the midst of all this. Also were totally willing to sell user information to ad companies if it meant better ad deals.

The core of the entire business was the part that was under attack. Being the only programmer there and realizing that there would not be a job left to complain about if I didn't do what needed to be done, I finally just started doing everything once all attempts at communicating the level of importance had failed. Built and integrated security features that had been present in the previous platform. Developed anti-phishing tools. Added intrusion detection for accounts. Built my own anti-spam system. By the time I was done with it, user complaints had nearly stopped and people were significantly more comfortable. Trading went back up. Crisis was over.

Owners didn't think I was working hard enough.

In the end I collected enough numbers to measurably illustrate the impact that my work had on the company, so I resigned with an awesome resume addition in hand that promptly landed me a muuuuuuuch better job with a better company.

Moral of the story: Do your due diligence. Try to communicate the importance. If you can provide numbers that put things in perspective for somebody more business minded - do it. At the end of the day though, owners who don't understand probably won't care. In this particular situation, if I didn't take the action that I did the company would have gone under. Others may be different though, so you need to be able to measure the cost of a breach in financial terms because that is the ONLY thing the owners will care about.

Outside of that, C.Y.A.

Re:EASY

[Jeremiah Cornelius]

Find another job.

These are not the only problems, just the ones you have seen.

Re:EASY

[Garridan]

Cover your ass BEFORE you talk to somebody in legal. The legal department is there to protect the company and NOT its employees. A good legal dept will say "hey, this employee is trying to reduce our liability" -- but a bad one will say "this employee is a liability" and shoot the messenger.

Re:EASY

[jonnyj]

I agree, but I wouldn't be underhand and I certainly wouldn't use read receipts. That looks horribly like the very worst kind of arse covering.

You shouldn't go over your boss's head. Juggling a large number of conflicting priorities is what managers are paid to do, and you won't do yourself or anyone else any favours by undermining your boss's judgement in that way. But you should also consider the risk that she consciously has her own best interests at heart rather than the business's interests. She might have the view that, in the event of a security debacle, she will pretend that the team messed up and failed to follow instructions, and simply ride out the storm. In the meantime, she looks efficient and appears to gets jobs done quickly with a minimum of fuss.

Instead, you should sit down with her and clearly express your concerns. You should then follow up your meeting with a very clear email that summarises the conversation. You need to start with an assertive but non-hostile comment that leaves no-one in any doubt what has happened - something like this, "As we discussed earlier, these are the security issues where I believe that we are falling short of regulatory expectations..." Print out that email and take it home with you.

At that point, your boss has three options. 1. She can fix things. 2. She can escalate up the food chain, so that someone bigger than her can decide whether poor security is really in the company's best interests. 3. At huge personal risk, she can quietly ignore you.

Middle managers tend to have pretty strong survival instincts, so option 3 is very unlikely to to fly. Option 2 is pretty likely, and her manager might well say that security is too expensive/awkward/boring/inconvenient. If that happens, you're probably better off working some place else where you can be proud to turn up in the morning.

Re:EASY

[Nefarious Wheel]

Marketing is driving the software?
They don't care about security?
System administration is outsourced?

Quit. Leave now. Take only your jacket. Your adrenals will thank you later.

Re:EASY

[TheCarp]

Potentially good advice, potentially bad.

I work at a large company where this wouldn't fly for one reason: We have a security policy that specifically forbids it. Under the security policy, we have specific guidance for who must be told and, very specifically, that it should not be discussed or divulged beyond that.

So check the policies first, because, just sending a message out to a large group of people may get you in hot water itself for violating policy.

oh and.... fuck that shit entirely, get yourself back on the market, if you have to hammer them to get them to take real security issues seriously, its not worth it.

Re:EASY

[MugenEJ8]

If you don't let me fix them, you will have to take the blame.

Word to the wise. Don't ever tell your boss what they need to do. I've been in the work force for over fifteen years, and this holds true for small business all the way up to large enterprise.

Best case, you've aggravated them and they will retaliate somehow. Worst case, you've aggravated them and they will retaliate somehow.

Re:EASY

[Grishnakh]

No, don't leave. Find a new job, get an offer, accept it, then leave.

It's extremely unlikely they're going to get into any criminal legal trouble in that time, and even if they do, it won't be traced to you. Get out and just find a new job. Don't try to be a hero: America hates whistleblowers, and there are zero protections for them here. If you reveal the problems, you'll never get a job again, because you'll be seen as a liability. Anyone who's ever blown the whistle on anything will tell you this. It just isn't worth it. The only way to blow the whistle is to do it anonymously somehow, so it doesn't taint you with a reputation as a "rat fink".

Go on .. tell us who

[OzPeter]

And I guarantee that all your problems will be solved very quickly by the dedicate volunteers who visit this site.

But you may need to brush up your resume first.

Call Elbonia

[Chemisor]

There are some newly unemployed hackers in Elbonia, made deaf and blind by viewing Wally's browsing history. Be a good sport and hire a few of them to break into your website. They are cheap and, being deaf and blind, would not be able to actually see anything useful for identity theft, but will sure be able to get your boss to see the light.

Don't ask /.

[Dishwasha]

I'd start by not advertising to a large public forum containing a lot of people with security exploit experience and motive about your companies web security vulnerabilities where your synopsis easily reduces the attack vector to significantly less than 500 potential targets. How many fortune 500 companies exist that target kids, let alone ones that have a female web software development manager? Also, it should be fairly easy for somebody in the industry to discover which fortune 500 kid targeted companies outsource their system administration.

At this point, I would do nothing. If they aren't hacked within a week after you posting this article then the security vulnerabilities don't really matter.

Re:Don't ask /.

[paavo512]

At this point, I would do nothing. If they aren't hacked within a week after you posting this article then the security vulnerabilities don't really matter.

Maybe this was the strategy of OP? In that case, brilliant!

Paper trail

[bugnuts]

Plain and simple, keep your old emails, offline. If you get cornered for a conversation in person or phone, no problem... just dash off an email stating "You know how you were telling me at lunch not to worry about the security vulns? This still really bothers me. There's got to be a way to mitigate it without affecting deadlines. Imagine the missed deadlines if we lose our infrastructure to an easy hack."

Don't sound like a troublemaker, but rather, a concerned worker.

Make it clear you're the professional, and in your professional opinion and that of industry standards, security is sorely lacking. Itemize the issues you have in an email. Keep that email.

Support their decisions, and live with it.

Finally, if the shit hits the fan and anyone points fingers at you, refer them to that email. If they fire you for it, that's when you become a troublemaker.

Integrity Hotline

[MNNorske]

If you're working for a Fortune 500 company there likely will be some form of internal integrity hotline. I know my own corporation has one. Document your concerns and contact them. I recently had to report a concern raised about one of the major offshore contractors we use to our integrity hotline and it was actually a very good experience from my side. After submitting the issue it took a few days but an investigator from our legal department contacted me and we had a phone conversation, and then I forwarded him some additional details I had held back from the initial correspondence. I did that mostly to protect an individual from the contractor who brought the concerns to my attention.

I would make sure that the correspondence you send to your legal department includes copies of some of the email chains you have with your managers, peers, etc... raising the concerns. Be sure to specify any regulations you suspect are being violated. If the legal team determines there is concern you can bet that change will happen. If they determine otherwise, then you've done your due diligence and reported it within the means your company gives for you to report it.

Re:Da fuq?

[tsa]

He knows what his problem is. Why is your comment rated insightful?

Re:It won't be a problem until it's a problem...

[epe]

Leave, ASAP.. quit:
it is a problem of ethics.. don't work in an environement that does not adjust to your ethics. That's it.

Re:B'OH!

[TapeCutter]

This isn't a Dr. Evil plot, the boss isn't hiding anything from anyone, the boss simply believes other things are more important than a secure web site. "Web sites are cheap but a secure one is expensive" - is probably closer to the level of thought running through the boss' head. Programmers are not automatically "right" every time the say something needs doing. The boss in TFA probably sees the programmer as a loyal employee who's concerned about the quality of his work but is blowing the problem out of proportion.

It's a hard life lesson for geeks to learn that "correct" is not sufficient evidence to convince others to follow your lead in the real world. Of course you should cover your arse, but if that is your only motivation then your no better than the DR. Evil you describe in your post. If you turn the issue into a battle of wills, or a gotcha moment, then you will more than likely lose the argument and it will become more difficult to raise the subject in the future. Nobody benefits from that, least of all the programmer.

OTOH arseholes do exist and if you have one as a boss in a small to medium sized business there is little you can do about it other than to walk out. Don't think of it as quitting, think of it as sacking the boss.

Disclaimer: Developer with 20+yrs experience, computers are easy, people are difficult.

Bring boss facts and a tech recommendation, don't

[raymorris]

I would extend that to say don't ever tell the boss what they need to do in a way that implies they don't know how to do their own job. That can be tricky if you are recommending that they reverse their own decision. Don't "act like you're smarter than the boss".

What has worked for me and people working for me is to bring facts along with a "from a programmer's perspective this option looks attractive" recommendation. Change "programmer's perspective" to whatever is appropriate. For many years I did IT security. CxOs would sometimes ask "should we do this" or "what should we do". I try to remember to answer "that's a business decision that's up to you, but FROM A SECURITY PERSPECTIVE ...".

The idea is to recognize and explicitly state that you are looking at it from a specialist's perspective, focusing mostly on one aspect of it. What you don't know, but the boss may know, as if they are planning on scrapping the entire project next month anyway. I can't tell the boss that we should upgrade X, because as far as I know the entire division that uses X may be getting laid off tomorrow. What I can tell the boss that that an upgrade to X would provide benefits Y and Z, at a cost of A.

I'd leave Microsoft

[TheGoodNamesWereGone]

I'd leave Microsoft and get another job