Ask Slashdot: Application Security Non-existent, Boss Doesn't Care. What To Do?
An anonymous reader writes "I am a senior engineer and software architect at a fortune 500 company and manage a brand (website + mobile apps) that is a household name for anyone with kids. This year we migrated to a new technology platform including server hosting and application framework. I was brought in towards the end of the migration and overall it's been a smooth transition from the users' perspective. However it's a security nightmare for sysadmins (which is all outsourced) and a ripe target for any hacker with minimal skills. We do weekly and oftentimes daily releases that contain and build upon the same security vulnerabilities. Frequently I do not have control over the code that is deployed; it's simply given to my team by the marketing department. I inform my direct manager and colleagues about security issues before they are deployed and the response is always, 'we need to meet deadlines, we can fix security issues at a later point.' I'm at a loss at what I should do. Should I go over my manager's head and inform her boss? Approach legal and tell them about our many violations of COPPA? Should I refuse to deploy code until these issues are fixed? Should I look for a new job? What would you do in my situation?"
Document your correspondences to your boss when you notify vital security issues. Make sure your e-mails are not only backed-up, but you get read receipts or something showing your boss opened the e-mail (and might have read it). Keep those receipts archived. When poop hits the fan, at least, you are protected.
Find a new job. Thread over.
Explain the possibility of liability. Let them investigate the risks. Problem will then resolve itself from the top down.
Frequently I do not have control over the code that is deployed; it's simply given to my team by the marketing department.
Well there's your problem.
And I guarantee that all your problems will be solved very quickly by the dedicate volunteers who visit this site.
But you may need to brush up your resume first.
I am Slashdot. Are you Slashdot as well?
Write an email to your manager listing out the security vulnerabilities and your concerns. CC your manager's boss.
Assuming you are with a large company there should be published legal and compliance policies. You can innocently ask that group for advice about the situation and you will probably find they carry much more weight on what should be done. If they approve of what your manager decided then nothing else for you to do.
There are some newly unemployed hackers in Elbonia, made deaf and blind by viewing Wally's browsing history. Be a good sport and hire a few of them to break into your website. They are cheap and, being deaf and blind, would not be able to actually see anything useful for identity theft, but will sure be able to get your boss to see the light.
are about the only two choices. It's extremely unlikely you'll be able to change anything until the business case makes it something they want to do. Sure you "could" get hacked. What are the odds? The business folk are willing to play those odds. You'll only lose your hair trying to convince them otherwise..
Have a written copy (email) of your exchanges with the boss. Advise him/her of the security risk and what consequences could occur if the software were compromised. If there's no response on the matter forward the communication to the legal department.
...then it will be your problem no matter how well you perform due diligence in this case. This is why I'm making it a rule that if I have to be responsible for making decisions, I want irebokable severance going forward so I can do the right thing by the stockholders without fear of retaliation due to butt-hurt bosses...
I'll address these concerns you have one by one:
* If there's any failure, willful or not, by your company to comply with any laws you should notify legal immediately. They'll appreciate that.
* You receive your code from marketing!? I hope I read that wrong.
* If your job is to deploy code, you have a duty to refuse to deploy code until such fixes are put in place. If your boss overrides that decision, speak with someone higher up the chain. If push comes to shove, document the security issues and your thoughts to protect yourself.
* If you feel you cannot perform the job you were hired into, you should find a new job. It sounds like you can't if no one is listening to you.
I often found that when someone doesn't want to listen to you and you know what you have to say is important, then the solution to it is create more noise until your heard.
In your case, which would be very risky as you "could" lose your job but at this point I would do it since no one wants to listen, its to create a problem LIVE and let the company go in nightmare mode.
To be more precise, let them think a hacker got all the info off of one of the vulnerable issues and because of that they got some sensitive information..just don't let them know its not true. Let them think it's real and let them freak out a bit.
Then, after they freak out, calm them down and explain how to fix it...at that point, they will listen and undertand your issue...some people are just plain fucking idiots. That especially happens on the higher end of a company, its like they live on another planet and only looks at numbers and statistics only.
Does it ever work out well for the whistle blower? Document your concerns then move on... it's better then being unemployed.
I'd start by not advertising to a large public forum containing a lot of people with security exploit experience and motive about your companies web security vulnerabilities where your synopsis easily reduces the attack vector to significantly less than 500 potential targets. How many fortune 500 companies exist that target kids, let alone ones that have a female web software development manager? Also, it should be fairly easy for somebody in the industry to discover which fortune 500 kid targeted companies outsource their system administration.
At this point, I would do nothing. If they aren't hacked within a week after you posting this article then the security vulnerabilities don't really matter.
Plain and simple, keep your old emails, offline. If you get cornered for a conversation in person or phone, no problem... just dash off an email stating "You know how you were telling me at lunch not to worry about the security vulns? This still really bothers me. There's got to be a way to mitigate it without affecting deadlines. Imagine the missed deadlines if we lose our infrastructure to an easy hack."
Don't sound like a troublemaker, but rather, a concerned worker.
Make it clear you're the professional, and in your professional opinion and that of industry standards, security is sorely lacking. Itemize the issues you have in an email. Keep that email.
Support their decisions, and live with it.
Finally, if the shit hits the fan and anyone points fingers at you, refer them to that email. If they fire you for it, that's when you become a troublemaker.
Go home, drink a bottle of whiskey, cry a little, go to work the next day and stop worrying about it :)
Cover your own arse. Document that you were the one reporting the problems and violations. You may lose your job anyway. Prepare for alternative employment. This is always easier while you are still employed. Once you have a reasonable plan for alternative employment you can start making demands. You may either be the hero, or you may end up in the other job.
I should use this sig to advertise my book ISBN-13 : 978-1501515132.
To me it is all based on what your own conscious demands. I spent years battling with my employers about their testing methods (the solution to the program crashing is the user should never enter that combination of values... yet you aren't going to prevent them from doing just that?) and got nowhere. At this point I put in my 40 a week, document the rejection of my recommendations (e-mail archives are your friend) and take pride in what I do outside of work.
If your conscious wont allow for that... ask someone else.
It's his responsibility to protect the company from idiots. Alternatively speak to the auditors, who also have a duty to report concerns. But on the whole you are probably screwed; whistle blowers tend to be shot on principle even if they have done the right thing - a new job is probably the best solution.
I've been told repeatedly here on slashdot and elsewhere that private companies, especially big ones, don't have IT problems, only the goverment does because everything the goverment does is terrible while everything the private sector does is perfect. So either you are lying or they are wrong.
However it's a security nightmare for sysadmins (which is all outsourced)
So it is the security nightmare that is outsourced? Finally someone got outsourcing right.
Ezekiel 23:20
Can you get budget to hire a security penetration tester? There are companies which will do penetration testing and then give you a report documenting all of the vulnerabilities they found. With that in hand you have a much stronger case to convince management to fix the problem because now it is a highly qualified security expert that has documented explicit problems.
When information is power, privacy is freedom.
So let's say it gets hacked. Are we talking minor embarrassment, or serious privacy violations? All big companies patch stuff all the time, after they deploy. Adobe probably has a big list of things that need fixing when they get around to it, which maybe explains why there are constantly updates.
When a hacker eventually steals and publishes all the little kids' info, are you the one who will get blamed? If you are, then find a new job now.
If you're working for a Fortune 500 company there likely will be some form of internal integrity hotline. I know my own corporation has one. Document your concerns and contact them. I recently had to report a concern raised about one of the major offshore contractors we use to our integrity hotline and it was actually a very good experience from my side. After submitting the issue it took a few days but an investigator from our legal department contacted me and we had a phone conversation, and then I forwarded him some additional details I had held back from the initial correspondence. I did that mostly to protect an individual from the contractor who brought the concerns to my attention.
I would make sure that the correspondence you send to your legal department includes copies of some of the email chains you have with your managers, peers, etc... raising the concerns. Be sure to specify any regulations you suspect are being violated. If the legal team determines there is concern you can bet that change will happen. If they determine otherwise, then you've done your due diligence and reported it within the means your company gives for you to report it.
Get numerous written reports with security concerns and documentation. Get written notice from your superiors to make no changes. Leak some information to 4chan.
Eat popcorn.
Not your problem.
Take your money and run to the ... sofa.
I did the smart thing; put my paper on the street (immediately) and started searching for a better, smarter, place to work (and found it). When shops abandon all the lessons and experience learned over decades of maturing our industry; It is unlikely to matter. "Agile" has been and is often used as an abomination to do a way with pesky issues such as quality control, proper coding, release strategies, and requirements (dont be haters, Agile used correctly is a powerful tool for rapid development). Turning everything into a "beta" product that is ripe for failure and abuse and releasing it to the public, and the burden of the results or responsibility will not fall on the shoulders of those who made that decision. Thats why they made it. Since your in a Fortune 500, I would look for greener pastures inside as well as talk with a few 'good/effective' recruiters.
If your company is breaking the law you should report it to your legal department via email AFTER discussing it with your manager and cc him and his boss. Alternatively visit your HR representative, who's job it is to protect the company, not watch out for you, and discuss it with them. If you do not, one day the hammer will come down and you will be thrown beneath it in the interests of mitigating damage to the company. If they decide to hammer you for reporting the issue then its not a company you should work for is it?
Or you can give notice and explain your reasons for leaving at the exit interview.
You should get advice from an attorney. You COULD be held responsible if something happens. Do you think your boss would stand by you and say you did your job, but she told you to wait?
Prosicutors would pin it on you because you failed to report it, and those truely guilty would use you as a scapegoat. Be smart, talk to an attorney, then at the very least you need hard evidence that you went to your boss, several times, and even over her head. If you have plausable deniability, then you are mostly covered.
Watch your p's and q's. dot your i's and cross your t's.
Credit travels up, blame travels down. Make sure you are up and out before it happens.
that's pretty much what I did for several years... (well, that & pay off our house so it wouldn't matter that much if I got blamed)
I even coined a Dilbert-esqe term for it: "the rapt* principle - no cube dweller ever got rewarded for being right about someone in a corner office being wrong..."
*long story I'll spare everyone
it's definitely the corporate Kobayashi Maru...
that said (& as others have noted): DOCUMENT! DOCUMENT! DOCUMENT! it won't save you from corporate scape goatting but could from a legal/PR/future job hunt problem...
what's the worst thing that can happen if the site is hacked? any CC info? how much money will be lost
not every site and data should be treated like fort knox. keep your emails for CYA purposes and keep doing what you are doing
I wrote a memo laying out all the issues in layman's terms and proposing solutions. Then I gave it to my boss. A little while later with no further movement on the problem, I quit.
A year passed and the system was hacked. Publicly. Embarrassingly. Folks here on Slashdot asked what the sysadmins could possibly have been thinking. So, I published a copy of the memo I had written.
Your mileage may vary.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
move on to a company that knows what they're doing.
A fortune 500 company that deals with any area that has Federal compliance laws like COPPA, HIPPA, etc should have a compliance officer. They would be the person to contact for issues like this and contacting them should address all your issues.
1) It gives a paper trail showing you raised the issue and should prevent you from being the scape goat when something happens.
2) It should give you someone who understands the relative compliance laws and the risks associated with not complying.
3) The compliance officer should then have the juice to get something done if they determine this is a legitimate issue. If they determine it isn't an issue then their neck is on the line not yours.
This happened to me when I was contracting for the USDA. Developers were pulling SQL statements in url strings. No... I'm not kidding. Literally "SELECT * FROM .
1) keep a copy of every email you sent.
2) evaluate the situation from an objective point of view. Should security be breached... what would be the possible fallout?
If personal information loss is part of this, immediately take your concerns to your legal team. In my case, I was told by several individuals it was not a problem and it was safe followed by my supervisor who told me it would be fine. I was okay with it until I realized I could pull anyone private information this way including social security numbers.
The legal team was very easy to work with. We had to self report 56 violations and my supervisor and two developers were terminated.
He's of the opinion that you give your opinion once. If they choose not to listen to you well fuck them. (Admittedly my uncle is very smart, has an ivy league degree. Anybody that ignores his advice is royal fucked.) I'm guessing the best thing to do is start looking for a new job because some how I doubt they'll suddenly get smart. (They'll probably just manage the company into the ground and then blame you for it.)
Did you know 80 to 90% of the moderators on slashdot wouldn't recognize a troll even if one dragged them under a bridge.
But given that we have the IT professional community that we have:
Incidentally, your case neatly demonstrates the near-uselessness of the IEEE-ACM Software Engineering Code of Ethics, which is very long on what the ethical obligations of a software engineer are, but has nothing useful to say about what you should do where others are ordering you to act unethically.
Any sufficiently advanced technology is indistinguishable from a rigged demo
--Andy Finkel (J. Klass?)
I suggest you anonymously post the issues to some very public place.
The stockholders should deal with it, due to the huge liability issues.
Sure send your notification emails and cya.. once that's done it's more a game of wait for the overtime, because when, and I mean when, it goes down it will be like Oprah came by with.. And overtime for you, and for you, and for you.. overtime for everyone until we fix this!!
Step by step, so a non-technical type can understand just what the issue is. "Security" for some folks is a vague amorphous issue with no real consequence. I've been stunned by some of the malware and lack of security I've seen on people's computers. They don't "get it." They don't understand the risk and the damage.
Help your boss "get it" if that's the issue. Explain the consequences of a breach, and the damage to the brand. Show with other examples in the media.
My $0.02.
Give up and come work for me.
Your first job will be to make a couple of security attacks on your old company.
I have a couple of million I can use to short those idiots....
Document the issues so that it is clear you are aware and tried to do something about them. Bring them up verbally to your boss - without being obnoxious about it. Once you've done those than you need to the hardest thing of all which is to let it go. If you make too big of a deal about it you will be seen as a troublemaker. If you do nothing you will be seen as complicit or incompetent if there is a violation.
Now in certain industries you may have requirements (possibly enforced by law) that require you do to more. Most of the time that isn't the case and you have to let it go and move on with other things. Often times disasters are the only way that people higher up the food chain can and will learn.
I recall when Nimda was making it's rounds in 2000. I was aware of the worm, had the patches downloaded, instructions printed and had requested permission to patch servers. Permission was denied. I asked again, it was denied again. I had awareness of the issue, my statement of the severity and denial all in writing.
I watched a fortune 25 company go down for 2 days and lose $100 million dollars and countless workers get sent home when their facilities were rendered useless. As a result an inflexible policy was changed and any number of people were fired or disciplined. Because I had documented everything I was just about the one person nobody faulted.
#1: Document every security problem you find and rank them in severity as far as how much they'd hurt the business if they were exploited. Document steps to exploit them from the outside if you know how, or if only exploitable from the inside, document how that could be done too.
#2: Notify appropriate management of all these documented issues, particularly the ones most damaging to the business that would be easy to exploit from outside.
#3: Explain the consequences if the exploit occurs. It might not be a bad idea to find news stories of other organizations that have been compromised to show the fallout from such problems.
#4: Document steps to rectify the exploits if you know how to, in as much detail as you can and preferably with time estimates.
#5: If all of this falls on deaf ears, go higher up. Find another job before doing so (at least get an offer) if you believe you will receive backlash for going above management's head. Honestly if the management above your manager is competent, they will greatly appreciate your efforts.
#6: You can also publish this list to any communally-accessible location and send it to all the developers in your company who are creating software that has security holes or could have them. Knowledge is power and I doubt all your engineers know they're creating dangerous security problems.
#7: Do what you can with the code you control. Lock down and secure whatever is most important. Let the small problems slide if it means the big ones get plugged. This is why the severity ranking is important, to help you and others prioritize.
#8: You should also log all these issues as defects and assign them to the appropriate person/team as ship-stopping defects, so that the software CAN'T be released until they're fixed. At least, that's how it works in a healthy development shop (which it sounds like you're not part of at the present time).
Your company is Sony?
Slashdot, fix the reply notifications... You won't get away with it...
If it's dealing with children and you are that concerned and management has done nothing to change it, blow the whistle on them.
I am Bennett Haselton! I am Bennett Haselton!
You should include the business owner on your emails to your boss outlining what is wrong AND how to fix the problem. Include in the what is wrong part, why the app is vulnerable.
Since you state that you came into the migration towards the end of the process, state that you are just now understanding that these issues even exist.
Webkinz?
He said CC and he meant it. Part of the logic (he even said it explicitly) is that the boss sees "Oh crap, now all these other people in the company know what's going on, and will be watching to see what I do about it."
Difficult to imagine the powers that be caring much about application security if they're willing to outsource sysadmin duties. And yes, I know that's common. But that doesn't make it sensible from a risk management viewpoint.
So you've got a vulnerable web app that can't be fixed with new vulnerabilities being introduced all the time.
That's what web application firewalls are designed for. Installing one takes less schedule time than doing things right would take, and it might work better than nothing.
Though of course this is not a technical problem, it's easier to paper over a people problem with a technical patch than it is to fix people.
Document the problems, report them up your chain appropriately and thoroughly, backup that documentation to personal storage resources to CYA and get out of there before the inevitable implosion happens. The management shakeup that will occur during and after the implosion will sweep away people regardless of who was aware of it and reporting it properly. The CYA is in case there are legal repercussions which draw you in,
Or to put it another way, nothing will get fixed as long as the software architect is as gutless as his management and just posts as an anonymous coward and helps conceal the problem. Sure, you don't have to commit carer suicide by saying "I'm the guy in the third office on the east wall and I've been reporting all of these problems to Bob but he just lets them slide, here's how to hack our toys", but you could put minimal effort into letting the problems slip out and help the public become aware of them. The hackers likely know about them anyway, management has decided that they don't care, as long as the public doesn't know. When the public knows they will become interested in fixing it.
I'm an American. I love this country and the freedoms that we used to have.
CLIENT: 'Let's email credit card details from our SSL website to lots of different locations'
When I said Uhh nooooo, I was unhelpful and not at all a celtic tiger and another company ended up doing all the transaction work. I don't work with them anymore.
I could literally walk into any internet cafe anywhere in the world and with a one line SQL injection attack put my last employer (also a well known publicly quoted company) out of business within 24 hours. I documented the attack, I documented several different ways to fix it. Senior management were not interested, they just wanted new features.
Walk. Get a new job before the ceiling caves in.
You don't want to apply for a job and have the hirer glance down your CV and see you worked for that corporation that just went so messily bankrupt.
Probably the task furthest from experience as an engineer/architect, but when it's not enough to tell them (boss, executives, legal) that it's a "potentially bad thing," also include some dollar figures.
As a tangent, you should also always have the right to contact Legal without supervision. In this case, you could even tell that person in the legal department you're doing a risk-impact report (without lying) and need an estimate for how much it would cost for the company to legally defend or settle a class-action violation of those COPPA guidelines/regulations. Because that suddenly becomes the development budget for making sure everything is in compliance.
Yes, we understand these tags always apply: fud, dupe, typo, slashdotted, topic name
Fortune 500 company? Mattel perhaps?
I find it hard to swallow a question like this.
Security is a tradeoff between the amount of time/money available, and the the features/productivity required to keep the company afloat and paying your salary (Surely everyone on /. is aware that security is a continuum with many shades of gray, right??)
Mangers (i.e. NOT YOU) are paid to figure out where that tradeoff is, and ensure that the company continues to make money so that everyone gets a paycheck and there are not mass layoffs. Yet....somehow....it seems like there's always some low level twit like you who believes he knows where the "CORRECT" tradeoff is, and nobody else could possibly have a differing viewpoint with validity.
You voiced your opinion - your opinion was not accepted. Move on and accept the fact that this is a business decision made by managers - i.e. NOT YOU.
You're at a Fortune 500 company, right? So you have a Chief Security Officer. Take it them, highlighting your bosses brushing aside of serious security concerns.
Your company has P&P in place for a reason, follow it rather than asking Slashdot what to do.
That "gotta meet the deadline at all costs!" attitude is probably coming from the top down... going over your boss' head would be yield nothing but anger from above, and you'd be branded not as someone trying to do the right thing, but as a troublemaker. Can almost guarantee it. If your employer is so obsessed with doing it quickly as opposed to doing it correctly, and you're obviously of a different mindset, you need to look for new opportunities. Let them deal with the blow-ups while you give your talent to another company, one who'll appreciate your attention to security. That's how the market is supposed to work, in theory...
Here I thought we was using his Credit Card.
They'll understand the consequences soon enough. Its the same with other companies as well.
You're at a Fortune 500 company.
Document issues, why they are regulatory violations, document that your raised them to your boss and got pushback (agree with the "offsite hardcopy" backup).
Then call the Ombudsman and raise the concern to them. That's what they are there for- Issues where the company itself could be screwed by individuals trying to make numbers any way possible.
Don't even think of "helping".
I have tried doing just that, at least 3 times. It's never appreciated by mgmt. They either already know, or have been actively avoiding learning about these problems, or are unwilling to spend money on it. They will either frown, deny your facts, not follow up on your suggestions, or just plain fire you for not being a team player. Been there, tried to "help", got canned twice, now I don't squawk about the SQL injection issues, phishing, spam, etc, etc, etc.
Just start looking for another job, with luck, at a less clueless place.
"Frequently I do not have control over the code that is deployed; it's simply given to my team by the marketing department."
If this hasn't prompted the answer "run as fast as your legs will carry you" then asking slashdot is not going to be of any use to you whatsoever.
Sometimes I think these Ask Slashdot stories just get made up by the editors when they're bored to see what replies come in, and the above quoted sentence is a particularly good piece of evidence for this.
Here I thought we was using his Credit Card.
In that case he could just sic a PCI compliance audit team on them.
http://www.newsday.com/news/health/gut-feelings-might-be-best-predictors-of-marital-bliss-1.6512381
If Slashdot were chemistry it would look like this:Cadaverine
Surely there's a infosec or security group at your company. Let them know. Otherwise, fire a note to your boss and cc'd your second level manager.
Don't have the email be one where you are blaming your boss, but if the security issues are beyond your manager's command and control span, then it's probably under your next level manager/director. Something as simple as "I've noticed some odd security practices taking place within the application... what group is responsible for setting the methodology...?"
Sounds harmless but gets the point across.
Others have mentioned the need to cover your assets I suggest doing so then consider what useful options you have. Please don't use the below list until your exposure is minimized by documentation and strategic copies showing your knew and reported the problem. Cover your assets !
1) Your skill-set is very valuable, if you don't want or care for a messy fight engage a recruiter and start a job search. Use the exit interview to vent. Feel free to take your favorite coworkers with you -done carefully no one need know you pouched them. IF they like you possess ethics and morals your company may thank you for removing such troublemakers!
2) If you want to make this right, prepare your documentation. Call your internal ethics hotline. Pitch this as serious risk the company faces. Upfront cost vs a very embarrassing civil or criminal investigation. If the ethics line fails you consider locating a Board chairman - someone with enough stock in the game to have the power to protect you, then prepare a overview of your concerns and meet with them. Done with respect and discretion you may not only survive but flourish . But the key is to stay inside the corporate process.
2a) Take a subset of your concerns - the most likely items to be exploited and take those issues , prioritize their cost to implement vs amount of vulnerabilities it will close. The easiest to code that has the highest impact should be number 1, etc.
3) Accept the harsh lesson - security does not matter in most cases. never has, never will. Too much security increases costs, reduces flexibility, makes deadlines slip.
I've ignored nuclear options - ratting your company out never ever ends well. Witnesses, whistle blowers have little protections and the world is awash in talented Senior Architects working at tech support firms for us$30k per year.
Good luck!
You can report unethical to the ombudsman designated under the sox act. I once worked at a well known technology company. The company reduced our salary after giving us lay off notice, termination almost a year out. That violated employment standards act in our jurisdiction. As once notice is given they cannot change your pay for the remainder of your employment.. One of my colleagues reported this unethical behaviour to the ombudsman, the net result was that we got back pay, salaries were restored, the responsible HR people and the managers were reprimanded.
If you work for a fortune 500 company will will have a corporate compliance hotline that is completely anonymous. It is likely posted by your HR department. Calling that will protect you and make changes happen without anyone needing to know.
Leak to the media and run to Russia a la Snowden. There's an IT job over there you can get
I didn't "lose" the job any more than I "lose" a defective computer when I throw it in the trash. Indeed it would be very hard to consider it a loss when six months later I was earning $10k more per year.
Nor did I put myself in any legal jeopardy. I'll spare you the lengthy analysis.
Best way to handle the problem? Burning bridges rarely is. But sometimes it has a moral righteousness that's hard to defy.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
This isn't a Dr. Evil plot, the boss isn't hiding anything from anyone, the boss simply believes other things are more important than a secure web site. "Web sites are cheap but a secure one is expensive" - is probably closer to the level of thought running through the boss' head. Programmers are not automatically "right" every time the say something needs doing. The boss in TFA probably sees the programmer as a loyal employee who's concerned about the quality of his work but is blowing the problem out of proportion.
It's a hard life lesson for geeks to learn that "correct" is not sufficient evidence to convince others to follow your lead in the real world. Of course you should cover your arse, but if that is your only motivation then your no better than the DR. Evil you describe in your post. If you turn the issue into a battle of wills, or a gotcha moment, then you will more than likely lose the argument and it will become more difficult to raise the subject in the future. Nobody benefits from that, least of all the programmer.
OTOH arseholes do exist and if you have one as a boss in a small to medium sized business there is little you can do about it other than to walk out. Don't think of it as quitting, think of it as sacking the boss.
Disclaimer: Developer with 20+yrs experience, computers are easy, people are difficult.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Ideally, one would backup the entire mail store, take a hash, document the whole process in detail, and quickly get a read-only copy on CD to a trusted third party such as an attorney.
More likely, it won't be a problem. A civil case is decided by the preponderance of the evidence - which ever appears most likely. If you have a copy and they don't show a copy of a different version, it's most likely your copy is correct. Of course that depends on which side seems more trustworthy - judges and juries, like all people, have a feel for who is lying to them.
The other side can either a) claim you edited the messages, which they know because "here's the original copy" or b) claim the conversation never happened. If they do a), they'd need to falsify evidence themselves. Any party who goes to the extreme of creating false evidence will probably out themselves as full of BS somewhere along the line. They may well say "I didn't read that email", at which point you pull out their reply. Wham, you've just proven they are being untruthful.
I would extend that to say don't ever tell the boss what they need to do in a way that implies they don't know how to do their own job. That can be tricky if you are recommending that they reverse their own decision. Don't "act like you're smarter than the boss".
What has worked for me and people working for me is to bring facts along with a "from a programmer's perspective this option looks attractive" recommendation. Change "programmer's perspective" to whatever is appropriate. For many years I did IT security. CxOs would sometimes ask "should we do this" or "what should we do". I try to remember to answer "that's a business decision that's up to you, but FROM A SECURITY PERSPECTIVE ...".
The idea is to recognize and explicitly state that you are looking at it from a specialist's perspective, focusing mostly on one aspect of it. What you don't know, but the boss may know, as if they are planning on scrapping the entire project next month anyway. I can't tell the boss that we should upgrade X, because as far as I know the entire division that uses X may be getting laid off tomorrow. What I can tell the boss that that an upgrade to X would provide benefits Y and Z, at a cost of A.
Any senior engineer can secure an app. Sounds like you just suck...
Your house isn't secure-- anyone can walk by and throw a brick through your window. Why don't they? There are other ways besides an attacker-proof house to discourage the attacker from throwing the brick! :)
If you enjoy your job, stay on. Save a copy of the emails you send documenting the security issues in a non-company controlled space. They might be useful when you part ways with your employer.
Sell that shit.
Also, FUCKING Name Names.
Review the Employee Manual. There must be a section on Compliance. Contact the Compliance Office and let them know you are concerned not only about the issues you have raised but that you would also prefer to remain anonymous to the parties responsible for the debacle. Your management has already spoken.
Also, you must, clearly and concisely, enumerate the issues their implications, and proposed resolutions (as much as you know about that). It seems that no site is perfectly secure, so you need evidence that your site is far worse than the norm. Bullet points or a tabular format make it easy and get directly to your point.
You want to be careful not to come off all "ranty". That weakens your message.
Fortune 500? Publicly traded company?
Then there is an code of ethics violation reporting mechanism. Contact them, contact internal audit, or contact corporate legal.
Reporting to the code of ethics violation provides you the strongest protection, because there is a stated policy that you cannot be retaliated against (still no guarantee that you will not be, just that it will help you in the subsequent multi-million dollar lawsuit you can bring). Make sure you mention the violation of COPPA and ask THEM to contact corp legal.
Also understand that you will not be seen as a hero. You will be branded as a troublemaker, so better be ready to switch jobs.
(Yes, I have been in a very similar position)
PS: I see some advice about documenting your interaction with the manager for the time when the shit hits the fan. Trust me, will not help you a whit if it came to that.
First, contact an attorney that has experience in whistle blowing.
Second, start lining up another job before attempting anything.
Third, try the CEO or President. Remember Sarsbane- Oxley? You'r CEO's ARSE is in the noose if any of this 'could' and 'does' impact the financial statements of the company. Should the information become public and it has a financial impact, you CEO may face jail time if as he has to sign off on financial statements, software exposure, etc. IF he is aware of the problem and does not deal with it, you may be able to visit him at the Federal pen. once a month.
I'm sure that CEOs like it when others remind them that they are only a few key clicks away from prison. So CYA.
I had basically the same situation at my last job. I fought it for years, talk about 'working on your nerves'. Anyway, I finally quit. After that they got some other guy to take my position, and he quit 3 weeks later. They eventually had to restructure the company, but did so in a way to keep the stupidity that caused us to quit. They're a failing company now, and I've moved on. Now I'm self-employed and am able to pay my bills. Aside from being able to sustain my life with money, I'm also able to sustain my sanity, and I'm a lot better to be around now, I hear.
Politics; n. : A religion whereby man is god.
Lots of what other people have said is good.
Approach legal and tell them about our many violations of COPPA?
Ask legal what framework you should be working under, and what laws and compliance are going to be required as part of doing your job. You aren't really sure what your personal obligations are in this regard, because you understand that there are regulations but you aren't sure who is responsible for implementing what exactly, and you've gotten conflicting or confused responses from your superiors.
... and why is he annoyed when someone tells someone else about him?
CLI paste? paste.pr0.tips!
I would email the bosses involved, advise them of the security issues, turn in my one month notice, go to a hackers forum, give them all the information, wait until the site is hacked (oddly enough just shortly before my one month is up) and when they come crying to me to save them I'll say "I already have another offer since I quit".
Then tell them I'll take double my old salary to be rehired, and tell them how lucky they are to have me.
Yes... I'm joking.
"If any question why we died, Tell them because our fathers lied."
You can't care more about security than your boss does. You'll always be frustrated. Time to move on.
The good news is that people are looking for good security engineers and it sounds like you're passionate and knowledgeable. Go out and find a job you love.
I was not even a developer at my old company (whom I shall not name, but I can assure you its products in some form are on almost 100% of windows machines).
I started in tech support with a bunch of pretty brilliant guys and gals who needed a tech job during a recession. Most are developers now, one gal went on to get her PHD, I decided to become a Consultant as I was not as smart as some of them, but had good people skills.
I complained for YEARS (even in Support) about the security of our web application. Not even real complicated stuff like buffer overflows, but just how passwords were stored (vanilla SHA1) and a few other things like blanking passwords from logfiles and such. When I became a consultant, I wrote a whitepaper detailing how one could spend about $10k and build a machine that would crack practically EVERY password in the application's database, and explaining in detail why this was likely to be a bad thing.
To my knowledge, nothing ever came from this. But I will say, I have moved on to a much better paying company since then--one that listens when I talk.
Is there personal health information (HIPPA/HITECH), or credit card information (PCI) at risk? If so get another job immediately and if personal circumstances permit, give notice immediately. This may be also advisable if you have information about specific minors.
If the above is not the case, the company's reputation is at stake and the millions that would be spent on PR firms to patch a PR mess should be forestalled. Tell the boss that a little CYA may not be a big deal. Take a look at Nessus, Metasploit and WireShark. Use the free trials if you have to. A pro will put in some extra hours to learn these tools. This should readily uncover the egregious risks.
Today anybody who doesn't make reasonable efforts to bake security into their code should be held accountable. Since you've outsourced the work, the vendor should stand behind their work. They are likely obliged to under their master services agreement - but don't wait too long.
Don't you have a Chief Security Officer or an information security policy? Discreetly tell those kind of guys your concerns and I bet you get action, especially if you have some reports from the scanning programs I mentioned.
Greed is the root of all evil.
Your boss is a /person/. They may not be prepared to accept 'you're an idiot' in a formal context but /might/ if approached with alcohol+confidential environment. (I say this to alert other geeks to more touchy-feely-social possibilities in their own bear-pits.)
Regulation will not be taken seriously unless it has teeth. COPPA violations have netted huge fines - as much as a MILLION DOLLARS!!!!1111!!!! - for people violating it. You claim you work for a Fortune 500 - a million bucks is pocket change, and easily paid. How much do they lose by not releasing the software as quickly? Probably more than a million dollars. Given the choice between "Earning 10 million dollars this month" and "earning 20 million this month, and paying a 1 million dollar fine for rolling out software quick & dirty," most companies will choose the option that puts 19 million dollars in their pocket at the end of the month, instead of 10 million.
You're not going to jail for "COPPA violations." Your company may eat a fine, and if you have the ability to show that you did your due diligence in reporting these possible violations and advocating for them to be addressed, then you're not going to jail, or eating a fine.
Compare with what will happen if you just flat-out refuse to deploy code until the issues are fixed: Great, enjoy looking for a new job. The company will still make 19 million dollars this month.
The problem is, if you want to actually fix the issues, you're whining to the wrong people. Your colleagues and your manager, like you, are powerless in this organization - you've admitted it when you say you're just handed shit by marketing and told to publish it to the web site. So, you can either find an ally in marketing or other upper corporate management functions, or you can whistleblow to an external agency. Those are your options to fix the issue; If you don't care to do that, then nut up, hand in your resignation, and take your skills elsewhere.
Just get it in writing (email) to CYA and chill. It's their business to run into the ground, not yours. If they want to put themselves at risk, that's their choice. If you don't agree with the poor security decisions, it's a free country (find a new job).
Assuming the website really violates COPPA, Google "COPPA violations" and grab some links to articles showing where the FTC sued over such violations and got big settlements. Then email those links to the boss (keeping copies of all this as others have suggested) and say something like "these guys got sued by the FTC and had to pay some big $, do you want to see our company get sued?"
If the boss takes an "I dont care" attitude or ignores the emails, go to the legal department or compliance officers with the same thing and say "I pushed this to my superiors and they chose to ignore it, I dont want to see our company held liable by the FTC, what should I do about it?"
If that doesn't work, consider packing up and leaving. Any company where the legal department doesn't care that the company is violating such a law and is one tip-off away from an FTC investigation (which could be a PR nightmare especially for a site that targets kids specifically) isn't a good company to work for.
I'd leave Microsoft and get another job
Quit and go work somewhere people/management aren't such dickheads! This company's systems are going to be pwned PDQ, and you DO NOT want to be there when they are!
Blow the whistle and move to russia
Document the vulnerabilities and the impossibility to fix them so that nobody can tell you did not make your job. Then hire a friendly hacker to break the product without doing any harm beyond shame. And never tell anyone you did that!
There are a number of network security firms out there who's whole reason for being is testing how vulnerable a company is, teaching the IT staff about how to plug their holes, and most importantly, providing the IT department (or often CTOs) with the ammunition required to convince stubborn CEOs that they need to put on the breaks and sink some serious resources into network security. These firms have all the facts and figures to be able to say, "you've got x y and z security holes, it should take this many man hours to fill them and if you don't and a hacker gets through your security, you'll not only need to fill the holes then but you'll also have to do a bunch of audits which are pricey, you'll lose rep, your brand will be damaged, and if they get to personal information of your clients you could be blackmailed for millions."
It's not always easy to convince people to hire a firm to do some security testing but generally its a lot easier than requesting the resources to fix security issues without a third party telling you it needs to be done.
I suppose the other way to fix it is to just publish the name of your company alongside your question... Shouldn't take long for you to get a free audit from some hackers that'll convince your boss...
I was fortunate to learn the correct attitude to this as a third-grade child.
My brother and I were raised in two different countries, 2000 miles apart.
So when he came to visit, I wanted to show him "my school".
The Principal very patiently explained to me -- taking all the time it took to make sure the lesson was as deeply implanted in my mind -- that "my" in "my school" had different shades of meaning, and in this case it simply meant "the school I went to" but it was really *his* school, and it was not appropriate to bring my brother to see.
Much later in life, I was taught that "my company" was not really "my company".
So yeah -- cover your ass so your boss can't shift blame onto you, and get on with life.
then run to russia
Go look at the company's 10-K. Look at the numbers for goodwill and marketing budgets. Most of the powerful people in the company cares about assets and risk.
Depending on what you want your career path to be, choose one of the three following options. A) Do nothing and document the hell out of your escalations and communications so that your hind parts are insulated. B) Find another job. C) Grab this opportunity to demonstrate that you can "contribute to business outcomes by being proactive" and put a proposal that both identifies a problem and a solution with a financial analysis of cost vs. risk.
It's your life.
When your boss is the President. We all know what application you're talking about.
I want to delete my account but Slashdot doesn't allow it.
... and get them to sign up to Sony and Adobe.
It took me three years of emailing Slashdot links to the Development Manager and Product Manager as each data breach story came out. But they only finally took action when their own personal account (and credit card) details got compromised in the Adobe breach. Now we have salted and hashed user passwords plus a full-time person working on checking and fixing security issues.
CxOs would sometimes ask "should we do this" or "what should we do". I try to remember to answer "that's a business decision that's up to you, but FROM A SECURITY PERSPECTIVE ...".
Too wordy.... "We would recommend X, because Y and Z."
A true security professional is not going to look on everything, purely from a perspective of maximizing technical security -- they will be concerned about the whole Risk management / Usability tradeoff thing.
That's what is called pertinent critical information; that you had to know to make a reasonable recommendation in the first place.
Obviously, you have to be informed about such known/planned things to make reasonable recommendations about anything they may affect.
do the CYA....document the recommendations and the responses....
happily take the company's money until they fail from their own stupidity or you find another job....
I have seen leaked code from many very successful products. Typically it is shoddy and full of security holes (hence why it was public) yet the companies behind it were and generally are doing quite well. Personally I am a fussy about making my products solid and secure; but I hate to say it but quick and dirty makes for a better business model.
So as many people have advised, document your worries, but even better find the various security problems and come up with a solid recovery plan. Then when the day comes and things go to hell you will be able to save the day. The only thing to do with your documents is not to play the blame game ( you will lose to some MBA asshole who will take you out before you can do any damage to his career) but to be able to show the MBA looking for a scapegoat that you aren't going to be easy prey and for them to find some other person to blame.
So in the end you will be a hero, not a scapegoat, and will have all the resources to fix everything for a while at least. What you will never be is vindicated.
You can also now smoke all the weed you want.....maaaaaaan....
If you give notice and leave you'll get branded a quitter. If you speak up to anyone you'll get branded a troublemaker.
Can you personally be held liable for anything? If so, then jump ship before it sinks with you on it.
Chances are you're already screwed no matter what.
Work the system and treat the boss just like you would handle a system bug or limitation.
Step 1: Get it into your official procedure, to do some kind of acceptance test or quality checks for software delivered by 3rd parties. This can often be done innocently and disguised as a formality.
Step 2: Improve the acceptance test procedure so that the pieces of garbage with security holes will fail Here, make sure the improved tests become official and rubber stamped.
Step 3: When at delivery the tests fail, raise a critical ticket with the delivering company. This works best if you managed in step 2 to make the test part of the acceptance. Now people will start to feel the pain, because a failed acceptance and a piece of software marked as "Not Ready for Deployment" will have commercial impacts. People will curse and try to force it through.
Step 4: While the shit is flying your way, make sure you stay reasonable, helpful and stick very closely to the official company procedures. Get acquainted with the QC department and ISO-whatever proceedings. Don't be controversial, never bad-mouth anyone. At the same time, document your cases, print out the mails where people attach your message to their replies.
Step 5: The software will be rolled out no matter what you said, but now you have a proper documentation of how your boss and the marketing department bend and break the holy official rules nobody want to keep.
Step 6: Various outcomes
a: People in marketing hate your guts now and avoid you as much as they can because you're branded as difficult. Problem solved for you.
b: They want you to do it again next month. Some chances are that the delivering organisation learned that releases are smoother if the software doesn't fail the test devised by that crazy lunatic in software engineering (this means you). A slow increase of security will ensure.
Step 7: Somewhere down the road there's a big chance the company will get into troubles because of their faulty software. Make sure, the people investigating that get access to your documentation.
Plant kiddie porn on your boss's computer and get him fired.
If the security nightmare is going to cause problems for the company and you care about the risks to the company, then wistleblowing is a good option. But if you don't care about the company but would like to protect your backside, then you need to ensure that you professionally raise the issue with the boss once or maybe twice and CYA. Of course, it can still come back to bite you, so as insurance you might want to start looking for another job - which does not necessarily mean your next boss will be any better.
I don't know what the OP's particular situation is wrt business perspective --- could it be that the bosses actually are looking at a tradeoff "ship now with internally known security problems, or try to fix them and not ship at all, and fail as a business"? If this is the case, one should probably think how to gradually integrate better security in long-term. Certainly, if there is a criminal negligence going on, then the "ship with known problems" is not an option! It is very easy to over-hype security, but remember that, in the end, it's all down to business bottom line. If you have a supermarket chain with some casual shoplifting happening, sometimes you want to invest $$$ not into more security guards and anti-theft tech that frustrates the customers, but into everything else --- maybe opening a couple of more locations --- and in the end turning more profit from the same investment.
VKh
Here in Sweden we have a service called "Techleaks" (https://www.techleaks.se/sa-kontaktar-du-oss-via-techleaks.html), use Google Translate to get another language if needed.
This seems to be a way if nobody in top management listens.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
I don't know what the structure of your company is, but many of the larger companies i've worked for has had some kind of 'complaints department', although that was never what it was called. In one company, if you saw something bad happened, you went to the CEO's assistant. In another company, it was the head of HR. I don't think any of this was officially stated, but people generally knew, if you're having a serious problem, this is person is the release valve. It's the person who you go to and say, "I don't want to go over my boss's head, but...," or "I don't know who to talk to about this, but..."
In a big bureaucratic company, they should have some person, or some kind of mechanism, for complaints about your own boss that isn't breaking the chain of command. They might not be able to fix the problem, but they might be able to give you advice on what to do, from the perspective of someone who knows your company.
Ask Betty?
I developed security controls for one particular technology in a Fortune 50 company and ran into this situation many times. Your path forward is to check the employee code-of-conduct documentation. In my company, it said that security vulnerabilities could be reported either to the security department or to management. I made sure management received reports about the vulnerabilities and I made sure that they knew I did not have the authority/resources to close those vulnerabilites. (All of this via a documented trail as well as during meetings in which multiple employees participated . . . I had my butt covered, legally.)
Ultimately security comes down to this phrase: everybody is responsible for their own sandbox. Security issues that can be solved only by management action must be solved by managers. If a manager chooses not correct issues, that's his responsibility. In one case, I advised my division manager that according to company security practices he did not have the authority to make a particular security decision, and if he wanted to protect himself, he should get permission in writing from his own manager. Which he did.
Show marketing a few high profile breach stories, tell them that you won't be surprised if you're next. They probably aren't aware of the risk, and will re-prioritize initiatives if you educate them.
Disneyguideline.com
it's a question of professional ethics.
That's what makes this tricky. From the point of view of professional ethics, avoiding harm to the users is paramount. But some of the things that are justified by professional ethics can be bad for your career, like going around your boss's back to your boss's boss. Depending on the corporate culture and how old you are, it could be a career ending move. If you're under 30 and obviously have marketable skills, go for it. If you're over 40 and have a family to support, you want to bring your spouse in on the decision. In fact you probably want a lawyer too.
From a pure CYA standpoint, documenting everything but not rocking the boat might well be the safest position to take *for you personally*. It may be very bad for your company and your product's users.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
Mangment usualy dont care much for detail. IT Security risks sound to them as one gory blah blah blah.
So right after covering your ass. Invite a Proffesional IT security risk Assesment, ask them to tag a price to each potential risk and potential safeguard.
Then arrange a mangment meeting and let the outside experts to present it, you will do the sum up.
It is very important that you stay positive, start with the good things, and present a proposed plan (with budget and priorities) to close the gaps.
Good luck.
Read receipts? Most e-mail (service/host)s and users disable this feature due to prviacy reasons.
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
Find the IT security group and tell them. They'll be very interested in what you have to say, and since they're outside your direct chain of command you're not going over your boss's head, but you're also not going outside the organization. Also the security group should be able to help articulate the issues to management, and can help marketing (or whoever is writing the code) fix the issues or mitigate the vulnerabilities with other controls. Depending on how your organization is structured, they may be under the IT group, or they could report separately, perhaps through a name like "compliance". If you can't find something analogous, at least try finding someone in internal audit who does IT audit - every internal audit group I've ever met does some IT audit.
As others have said, save all copies of email between you and your boss informing them of the flaws. Make sure all future emails are return receipt requested. Print them out. Take copies home... and then buy a safety vault in a bank, nothing less, and store them there. That way, they can't be made to disappear so easily.
Yes, I knew someone who lost their federal job, and the documentation at home disappeared.
mark
And then from a Starbucks, using their wifi, post a meme picture saying that "X website has no security- don't use a critical password or personally identifying information for your kids unless you want them kidnapped by a sex abuser" on the App's facebook page.
Never own up to starting the meme, but watch things change VERY quickly.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
What is all this about backing up to physical media? Back it up in your personal email. Back it up in a cloud storage solution. Just because your company controls the digital resources on their LANs doesn't mean they do so on the entire internet. Burning cds, sealing them in envelopes and clutching them as you're per walked out is a lot more conspicuous than sending yourself an email, strolling out with empty hands and not a care to give, and downloading it at home.
I would instead say that computers are predictable, people are not.
Making a web front-end work like a desktop GUI *is* difficult if we target many browser versions. However, we have a general idea of how long doing such well would take. A "difficult" person on the other hand can be very difficult to predict and communicate with, making life too unpredictable.
True, a desktop metaphor may not be appropriate or economical for a given organization, and convincing the boss or customer of that may require some solid people skills. It's two factors intertwining: technical issues (UI) and people issues (expectations of UI). They both can be difficult, but the second is less predictable.
Table-ized A.I.
Get a new offer, then leave. I have done consulting gigs and full-time work with some of the more-visited sites on the Net, and many of them showed blatant disregard for their customers' security. I actually had a CTO of a major net company tell me "there is no such thing as Android malware, you are making it up" and "my engineers write secure code" without any code reviews or even basic security awareness training. This year. Some refused to admit they could have security problems even after I demonstrated them. Just go. You cannot win.
Oh, and HR/Legal WILL hose you without a doubt. I recall talking to an HR person at a company I was with about what she would do if someone came to her for substance abuse problem counseling. She, without batting an eye, said she would do everything she could to fire the employee as they are nothing but a liability. Most view their job as protecting the company vs. advocating for employees.
Life sucks, move on. Laugh when they get owned.