Slashdot Mirror
Third Party Code Review?
An Anonymous Coward asks: "It looks like our sale-person is about to land a big contract with a very large US Bank, however there is a large catch in that the bank is demanding that we let them do a full audit on the source code of the software application we are selling them. After the recent rash of identity thefts of credit card and other personal info, they now mandate that all internet facing applications that store potentially private information have to have a full source code audit. This includes software from 3rd party vendors such as my company. They want to run our Java code through some software called Fortify (we looked up the price -- around $80,000) and also do a manual analysis of the code. This software is our company's life-blood. We would be ruined if it fell into a competitor's hands. We aren't storing private information about their customer's; all of the information can be found from government county auditor web sites. I understand their point of view, but it is a very scary step for us to take. Has anyone else done this and how did it work out?"
then it sounds like you are in the business of selling disks with programs on them. in that case, you're already sunk. you need to move NOW to a model where you make your money deploying and supporting software.
show them the bleeding source code, you pansy.
This is why your company has lawyers.
Handing over the source code should be part and parcel of any non-retail software package. Like Free Software, but without all the philosophical bullshit.
Get the bank to sign an NDA, and sue the pants off of them if they leak your source.
After all, I am strangely colored.
You'll sue. You'll win. You'll be fine. Your competitor will be ruined. The person who leaked the code will be ruined.
As for the use of a $80k code audit tool. If the bank's paying for it, that's how it's going to happen. BTW, expensive niche software companies don't always like it when their quotes become public knowledge. Companies like that often try to guess what each customer is willing to pay, within reason.
I was almost sympathetic until I re-read:
.. )
... "
very large US bank
What, pray tell did you expect? It looks as though you blundered into a pot of gold and kept going despite the fact that you're not large enough yet to carry it away.
Of course they'd demand third party review. I hope *my* bank would! What I also don't see mentioned is any mention of a three inch NDA that would be signed.
Established companies like Microsoft can sell stuff with some (or all) of the hood welded shut. They are an authority. They dictate who our browsers trust. They're huge and they could afford to pay for resulting damages (good luck pinning any on them
If you really want to use this as a spring board I'd let them have at the code. Unless you're in the middle of an "Oh SHIT we gotta re-code all that GPL stuff we used
Why would you worry if there wasn't anything to worry about? And why risk your "life's blood" on one single venture?
Order happy meal first. Big mac later.
Off my soapbox.
Non Disclosure Agreements and Really Good Lawyers, that's what it's all about. And if you think it's too much of a risk, just turn the job down. Big fat contracts are look appealing when they arrive on your doorstep, but if they come with massive provisions which are too risky for your business then don't be scared in turning them down. Especially when it's your business, your life and your way of thinking/sanity which is exposed.
Of course, there is the bargaining position of if they are really in need of your software, then you could be in a good position to strike up a trust and maybe negotiate your way out of being audited.
I've done a few defence contracts where they've demanded the same type of auditing, and in a few I've managed to get out of the auditing process for non-mission-critical systems by negotiation.
Task Mangler
The only thing missing is the names of local variables and the comments. If you're distributing your program unobsfucated (and studies have shown that 99% of companies do) then they already have your source code.
How we know is more important than what we know.
I'm assuming the bank and its software is running on a closed operating system, like Windows. Does this bank require source code from Microsoft?
The system will fail security at its weakest link, whether it be your banking software or the operating system it runs on.
All my liberal friends think I'm a conservative, all my conservative friends think I'm a liberal.
You get paid no matter how you do on the audit, right?
If I were your competitor, and i wanted your code that badly, I would have already disassembled it by now. If it's Java, I would have had a really easy time of it.
As others have pointed out just get a really good NDA (which I'm sure the bank has probably already insisted on anyhow). The benefit of 3rd party code review is that they just might actually turn up a vulnerability in your software. This is just like having someone pay you to help you validate your software. At the end of this you can say your software has been validated by Fortify.
Author of Enyo: Up and Running from O'Reilly Media
I'm a manager at a fortune 10 company, and parts of our business run SAP. Maintenance licensing runs into the tens of millions of dollars per year. So it would make sense to try to steal this product, right? Wrong.
If we somehow got hold of the source to SAP R/4 and MySAP, outside of a quality review, it would be worthless to us. The support, maintenance fixes, and configuration assistance SAP provides are worth far far more than the code. And the risk that comes with internally-compiling the code (and we can do it) could cost the business far more than $10M.
Net: Don't sweat it.
Why doesn't your company audit the auditor?
OK, this is flamebait, but what the hell. It annoys me when people claim something is proven, without actually supplying any proof.
As a thought experiment, try this out. Start off with a software project that forks into two branches. One branch is developed with an open source model. The other branch becomes closed source. Assume that the original software project had a certain number of bugs. Now I ask you, as time goes on, which branch would contain less bugs? The closed source, or the open source?
It seems obvious to me that the open source project would quickly have its bugs fixed, because of two reasons. The coders on the project would have ego on the line, and would therefore be much more careful with publicly available code. Due to the immense availability of peer review, issues are discovered and reported much more quickly and thoroughly.
It also seems obvious that the closed source project would have many more bugs, for many different reasons. Programmers are more likely to ignore issues, because "nobody can see it". The pressure of getting a release out means that things are quickly cobbled together without much thought for security. If there is a review of code, it is not likely to be as thorough as what is available to open source.
If the code is hidden, a program is not more secure. Otherwise, how do you explain the numerous security issues found in closed source software every year?! Whoever found the bugs, didn't have access to the source, yet they were still found. And once it is found, how can you fix it? I'd say that closed source software is far more insecure than open source, because not many have access to the source.
* Closed source is proven to be far more secure in the real world than source that has been picked through by numerous people.
:) lighten up.
OK, this is flamebait,
Well the gp wasnt flamebait but funny
That's how the mainframe COBOL guys used to work; they put the source up on the really big bank's mainframe and work with the banks systems people to compile it and customize it. I've worked for two companies that followed that model and let me tell you, it is really really lucrative. It is ingrained in some really big banks' cultures, be it a holdover from the true "big iron" days, but so are the wonderful amounts that you are expected to charge them. Now, you and I realize that times have changed a bit, but many banks still call it DP, not IT and consider this; once you are in, they begin to build systems and protocols around your software. Before it even goes live, they will have their technical writers creating binders just for interoperation with your system. Your software becomes part of the machine. Do you have any idea how expensive it is after a couple of years to rip all of that out and start over? Like others have said, get a good non-disclosure written by pit bull lawyers and laugh all the way to the bank. Just take care of them (the banks decision makers) by doing a good job and catering to their whims (and charging them for it) and they will take care of you.
I've done quite a lot of this over the years, and I can see how you'd find it scary. here's the key things:
* get a good tight NDA from the auditors
* get a well-respected firm to do it, one that has something to lose. Someone like Ernst-Young.
* insist on it being done on your site, and that you receive all work products at the end of the audit. This won't keep someone from walking off with a copy of the code anyway (not when you can buy a 2 gig USB key for a hundred bucks) but will strengthen your case if anything does get pirated.
* Look for a firm that doesn't have a software business in your area of expertise. You don't need to be buildign bank apps to audit the code; if you pick someone who doesn't have bank apps in their product line, and they suddenly start some after the audit, you'll have a good hint that there's a balrog in the woodpile.
Simply allow them their audit: Here is a room, we've prepared for your audit. It doesn't have any communications links, leave your cell phone at the door. There are two computers with the source code. Feel free to audit it all you want.
I contracted with an electronic voting systems company last summer; one task was preparing code for an audit as mandated by the FEC. This was to be a manual audit (versus an automated audit like Fortify), conducted by a 3rd party government contractor.
:)
Notes from the experience:
* We requested examples of code that met specific auditing criteria, and received back several somewhat-anonymized methods, apparently taken from competitor's products. You should verify that the bank has appropriate "handling procedures" for protecting 3rd party source code.
* Our audit criteria was spelled out in an FEC ruling in decent detail. We found that 50% of the rules could be easily expressed as existing Checkstyle "checks" [http://checkstyle.sourceforge.net/ ]; it was pretty easy to build custom "checks" to catch another 30%. We then used an Eclipse plugin [http://eclipse-cs.sourceforge.net/ ] to get real-time highlighting of detected issues (plus Ant scripts for command-line checking).
In your case, Fortify "rulepacks" appear quite proprietary/complex, so using their product is probably your only option for pre-audit auditing. If licensing is out of the question, and you can't strike a cross-promotional bargain (i.e. you market with "Secured by Fortify", they use you as a case study, you get a discount), try and get access to the tool through the bank before the official audit, or negotiate an appropriately flexible window of time in which to address any discovered issues.
* You're not "innocent until proven guilty" in an audit. In *many* situations, we had to argue against rules that were nonsensical in Java, or false-positive issues discovered by the audit. Some we won, most we lost; we faced an uphill battle on all.
* Our auditor was apparently not fluent in Java, and flagged several issues regarding the method names on classes in java.lang.*. Be thankful for automated auditing
Good luck!
Scott Severtson
Senior Architect, Digital Measures
The whole Sarbanes-Oxley regulations really leave the bank (or any financial institution) with little choice; they are legally required to guarantee the security and accountability of their systems, without being able to audit your code, they cannot give such guarantee and thus cannot use your system whilst still following Sarbanes-Oxley regulations.
So you've only got two choices: "Let them audit the code" or "Lose a customer".
FYI, I work as a programmer at a bank.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
Just make the application in Obfuscated C! by the time the audit has finished, new computers will be able to play DNF!
>> Non Disclosure Agreements and Really Good Lawyers, that's what it's all about.
> Spot on.
Tell that to Cisco
3.243F6A8885A308D313
What kind of things can an automated process do for auditing, anyway? This is Java we're talking about. 90%+ of things that are security problems in other languages aren't even an issue in Java, as the compiler and/or the assembly language verifier already do that.
The main issues in Java are going to be logic errors and misimplementing security protocols. Things like bad packet handling in a network server. There is NO WAY an automated system can detect problems like this: it is the Halting Problem.
So what can this program do? All I can imagine it doing is checking to make sure that you're not using any function calls that Fortify's authors consider "unsafe", no matter whether the particular context makes it safe. It probably will also yell at you for using variable names that don't follow its stupid rules.
I can imagine how things like this exist. They approach these security-paranoid companies with offerings of a magic solution that will allow them to verify that their system is secure. Extremely afraid of being the next target of a class-action lawsuit, they are eager to pay large sums of money. The people who make the decisions aren't trained in computer science, so they don't understand that an automated system such as this is truly impossible.
It is the small companies who have to deal with this that suffer. The magic oracle says that you used a single letter as a variable name, so you absolutely must change it, with no excuses. You spend a lot of time and money "fixing" it to please the oracle, when you have done absolutely nothing for true security.
Melissa
"Screw Sun, cross-platform will never work. Let's move on and steal the Java language." - Visual J++ Product Manager
I work for a lab that does seurity reviews and evaluations. There are a few things you might want to consider:
http://erichsieht.wordpress.com/category/english/
That is what I thought about.
:)
Allow audit of the source under full NDA and without leaving any technical means to circumvent the NDA.
As an extra: advise and provide help of lead programmers of your company - they will be able to immediately explain any doubts, possibly fix immediately all easier security bugs found by the audit (no waiting for feedback: "here's an error, fix it", week later "here's a fixed version, audit it", another week later "the fix is not satisfactory, fix again" and so on), and in the meantime they will look at the strangers' hands, making sure no piece of software leaves the room without their knowledge.
If they want to be sure the binaries they get are the result of the code they audit, finish the audit with compilation of the newly-audited code and burn a CD with the newly-compiled binaries, handing it to them, all while they look and see you're not doing any tricks.
Some profits:
- both sides learn a bit.
- your code is safe.
- potential bugs may be fixed immediately
- audit is more efficient (thanks to communication, the auditors have a better understanding of the code)
- You are sure they don't make up anything to screw you
- they are sure you don't hide anything from them.
The minus is that they need to bring their own audit software and install it on your computers. Of course, deleting it afterwards
Generally, neither side trusts the other enough, so the only solution is that both sides look at each other's hands all the time.
Anagram("United States of America") == "Dine out, taste a Mac, fries"
As a buyer of software I would not only expect (not demand: expect) access to the code for an audit, I would also expect you to keep the source code in escrow, in case you fold and are no longer able to support the software.
As the other poster said, if you're in the business of moving bits on discs, you're already ruined. You're just waiting for the time delay to kick in.
While your code is very, very important to you, to that bank, it's just a program they want to buy. To the auditor (if it's a separate organization, as it probably legally has to be), it's just a bunch of code that the bank wants checked out.
Your company, virtually certainly, isn't even vaguely important enough to them to mess with. If that code leaked and the word got out, the reputation of both the bank and the auditor would be badly damaged...a financial loss greatly in excess of the net worth of your entire company.
If for some reason they wanted to leak the code, it would be a lot cheaper for them to just buy you out, lock, stock and barrel.
Use your brain. Give them the damn code. They'll probably treat it better than you guys do. They have a lot more to lose if they don't.
Code auitor: "Hahahahaha! this code is so bad! I'm going to post it on TheDailyWTF.com so everyone can laugh at it."
You have the bank sign some legal document that says they pay for the harm if your source code gets out due to their fault. The lawyers know how to take care of this. Get one. Problem solved.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
The reason that closed source software has more bugs is mentioned in your post, but isn't because it's closed source. It's because of A) Programmers who don't care and think that since nobody will see it that they don't have to make it look good. And B) Managers rushing the software out the door before it's ready. You could have an open source project where the programmers really don't care but it happens less often. You could have a manager pushing an open source project out the door before it's ready, but it doesn't usually happen. Also I'd like to point out that when closed source software has a bug, there's only the people at the company who can fix it. With open source, anybody who wants to fix the bug can fix the bug.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
I would believe that some company that puts their whole reputation on the line by handling people's lifeblood code in their hands would make sure to do a little more then just check names of variables...
I would think first that they have a big database of source code (of which they probably put a disclaimer for in their license agreement and put your code in their db for further use...) and compare to see if you have not already used someone else's code signature or even GPLed code for that matter.
Second, they would also probably put a stress test to any network type interaction within a honeypot to observe the demands of the software in question...seeing as it is possible to tie up multi threads quite nicely with poorly designed code and create a DDOS attack within a network....
Thirdly but not least, they also probably stress factors such as math rounds, nested function calls, and memory leaks....and no way you can also guarantee that running a highly intensive program on a server that already has a big charge would allow for someone to not purposely leave out the part about requiring a certain amount of resources (thereby freezing up your whole server). This is what they would end up finding out for you, once all this has been checked into on your behalf (if you are the purchasing party needing such services).
And again, once you have been through it once, and have been certified,
it ends up working in your favor for future endeavors , and can make a very nice plaque on the wall!
L.A. Senior developer
He mentioned the ego factor. If your source code is ugly, crufty, and buggy, and you let it out into the open, people will mock and deride you and your skills. While open source advocates like to talk about "thousands of eyes," I suspect pride and socialization have just as much to do with the code quality.
I guess I'll tone down the "mock and deride" statement a little. In the best world, people will constructively help you improve your skills. If you refuse or fail to learn, then they'll mock and deride - and ignore you, which is just as bad.
The living have better things to do than to continue hating the dead.
As for the `far more secure' claim, there is some truth to it. (Or were you saying that your comment was flamebait rather than the post you were replying to?) If you have a closed source project (and even an open source project may be similar), it probably has lots of bugs -- some you know about, many you don't. The source code may even be riddled with FIX THIS! BIG SECURITY HOLE! type comments. If the source gets out somehow, then people who go over this code may be looking for security holes to use against you and your customers. Which isn't automatically bad, but there's two differences between this model and the traditional open source model -- 1) nobody is supposed to have your source, so anybody who does is pretty much by definition `bad', and 2) bugs found are not likely to be reported back to you, so you can't go and fix them unless you're able to detect and analyze an exploit actually being used.
That said, the loss of your company's source code isn't as big a deal as some might think. Yes, depending on the software, crackers might interested in using it to find holes in your product. However, if your competitors are legitimate companies, they're not going to touch your source with a 10' pole. Even if they could learn all sorts of neat stuff from it, it could also easily lead to corporate ruination -- all it takes is one disgruntled employee to report it to you or the authorities and provide proof. And really, your software may already be out there -- it only takes one employee and a portable hard drive to take it all off site. (Of course, he'll have a hard time selling it for the reasons I gave above ...)
That, and just having the source is not everything. Your company probably also provides support and professional services. For large projects, this is really important, and software that doesn't come with support often isn't very useful.
In any event, even if you do give out your source to this customer, a full code audit is not likely to happen. They'll use their automated tools, they'll look at key parts, but it's very unlikely that they'll have the resources or time to do a full audit like the OpenBSD team did when they forked from NetBSD -- instead, they're just looking for low lying fruit, and are likely to find only a small percentage of the bugs. On the other hand, they'll probably report what they find back to you so you can fix it.
But as for the dangers, for starters, make sure your legal team makes a iron-clad NDA for the other company to sign. If your company is too small to have a dedicated legal team, get a lawyer for this. Make sure that only a small group of people will have access to the code, and that it's deleted when it's done, with big penalties if this is not done. Perhaps the audit could be done on your premises, on your hardware, supervised by your employees?
It seems obvious to me that the open source project would quickly have its bugs fixed, because of two reasons. The coders on the project would have ego on the line, and would therefore be much more careful with publicly available code. Due to the immense availability of peer review, issues are discovered and reported much more quickly and thoroughly.
It also seems obvious that the closed source project would have many more bugs, for many different reasons. Programmers are more likely to ignore issues, because "nobody can see it". The pressure of getting a release out means that things are quickly cobbled together without much thought for security. If there is a review of code, it is not likely to be as thorough as what is available to open source.
From my personal experiences, I felt the opposite. When I worked on commercial, closed code, it was something I took pride in. Not many people can say something they worked on made it to store shelves. I wanted it to be as good as could be, as it reflected on me.
When I've contributed to open source projects, I've felt less concern about the quality. I've pretty much had it in the back of my mind that if I make a mistake, someone else will see it and correct it. Realisticly though, whenever I submit a patch to a project I get very little feedback on it. Generally any feedback I do get is on the functionality of the patch rather than the implementation. If I submit a large patch, people are probably only going to catch the fairly obvious flaws unless they have a good reason to mess with the code I just submitted. When you're talking about something really important like the Linux Kernel, I'm sure patches will get stricter reviews. But when you start talking stuff like small utilities and random KDE/Gnome apps, people aren't going to care that much.
I worked for a company that sold software to banks, and we had to satisfy the same requirements. We ended up negotiating an arrangement where a consultant from IBM who would show up on our premesis, check out a laptop with no network or removable storage support that had a copy of our source code on it, read source code all day, then check it back in and go home.
We thought it was pretty draconian, but the bank thought it was a great idea; bank IT staff by necessity live in a paranoid world, so I guess they didn't mind so much.
Often companies will use a 3rd party auditor - particularly if they don't have the skills themselves. We've been through that several times, no problems. They can also help by spotting things you weren't.
dominionrd.blogspot.com - Restaurants on
OK, With the NDA you may have legal grounds to sue their asses off, but what happens when you have to do this with some other company, and the one after that? How can you know who leaked it? Just do what the CIA/NSA/DOD et al have been doing for years. Make small almost insignificant modifications, (comments, variable names, etc) so that each version is slightly different. Get the names of who will actually be handling the code from their side and supply each with a different tainted version. So, when and if your source does show up in the wrong hands, you have a higher probablity of identifying specifically where it came from.
"Can there be a Klein bottle that is an efficient and effective beer pitcher?"
Let's review real quick:
1. you are a software developer with big clients such as US banks
2. you must submit your source code to a 3rd party auditor
3. you think slashdot is a good source of advice
Well, seeing as you're in business I'll assume #3 is just a temporary brain fart on your behalf. If you're really worried about your code falling into the wrong hands, then have a lawyer write up a nice contract that expressly forbids the bank and any other party from using the code for purposes other than security auditing, and throw in a time limit while you're at it. That way you have a leg to stand on if your code were to be spread. This is called protecting your work and intellectual property.
Just name any big software company, and try to figure out what they would do if their code was stolen. Yeah, they'd sue your ass into the ground. Why should your company be any different ?
-Billco, Fnarg.com
You'll sue. You'll win. You'll be fine. Your competitor will be ruined. The person who leaked the code will be ruined.
This is it exactly. I don't understand how it got to be this way, but the "Oh, no! Our source code must be kept secret or we'll be ruined! Ruined, I say!!" attitude is almost as silly as it is widespread. That's what copyright is *for*: You can hand out copies of your source all over the place and no one is even allowed to compile it without your permission. Certainly no one is going to be able to set up shop and compete with you using your own code.
In some, rare, cases, there's something in the code that is so fiendishly clever and novel that even allowing your competitors to read your code and pick up your ideas actually would really damage your business. That's what trade secret law and NDA contracts are for. Patents, too. But in most cases, that really *isn't* the case. Odds are, your software doesn't do anything that someone else couldn't write software to do, given the time and the resources, and odds are that your competitors wouldn't be able to benefit much from direct infringement without you knowing and suing their pants off.
IMO, one of the best things about the Free Software and Open Source movements is that they're exposing more people to the idea that code doesn't need to be secret. You can even publish your code on the Internet *and* still sell it (c.f. Trolltech, Sleepycat, MySQL AB, etc.). And even if you don't go so far as to allow its use for free by some people, you still don't need to be so frightened that someone might see it.
Unless, of course, it includes illegal copies of someone else's code. *Then* you have a reason for keeping it secret.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Give them access to a another machine if they need Internet access.
To be just as fair, I never claim to prove anything
Whether it's open source or closed source, it is impossible to say software is 'secure'. My only argument, is that open source software is generally more secure than closed source software. Sure, there'll be open source projects that are horribly insecure, and there'll be closed source projects that are very secure. But these are the minority, in my argument.
Send the bank a hard copy of all the posts on this topic, sign the contract.
I eat my grapes at room temperature, cuz the cold ones hurt my teeth
Big financial markets, certain medical devices (huge liability), anything nuclear and intelligence/defense.
'Most of the code audits though, are for the low hanging fruit only, since a huge codebase can take a very long time to become familiar with, and the auditors never attain the same level of understanding as the original authors' (paraphrased as told to me by a defense code auditing guru).