De Raadt Doubts Alleged Backdoors Made It Into OpenBSD

itwbennett writes "In follow-up to last week's controversy over allegations that the FBI installed a number of back doors into the encryption software used by the OpenBSD operating system, OpenBSD lead developer Theo de Raadt said on a discussion list Tuesday, that he believes that a government contracting firm that contributed code to his project 'was probably contracted to write backdoors,' which would grant secret access to encrypted communications. But that he doesn't think that any of this software made it into the OpenBSD code base."

Comments

Audit necessary

[dewarrn1]

I hope that he's right, but without a thorough audit, who can say?

Re:Audit necessary

[CAPSLOCK2000]

Even with a thorough audit you will never be sure. That's the beauty of these kinds of accusations, no matter what you do, you can never 100% sure.
OpenBSD is among the best audited code in the world. People have been looking for this backdoor specifically for an entire week and nothing fishy has been found yet.

Re:Audit necessary

[Anonymous Coward]

Well, great way to halt the actual development, right?

Remember how Microsoft accused ReactOS of copying NT code?

They spent LOTS of time auditing.

Re:Audit necessary

[Anonymous Coward]

I hope that he's right, but without a thorough audit, who can say?

It is physically impossible that a backdoor makes it past De Raadt's ego into the kernel.

Re:Audit necessary

[milonssecretsn]

OpenBSD does have an ongoing code audit

Perhaps not as thorough as you were suggesting. However, I think for others who are not familiar with OpenBSD's ongoing code audit, the above link will be essential for fully understanding these stories.

Re:Audit necessary

[Eil]

As unlikely as it is that any backdoors have made it into OpenBSD, even an audit cannot conclusively prove that there are no backdoors in the code. Witness the Underhanded C Code Contest. The goal of the contest is to write a chunk of code that does something, well, underhanded that is difficult to detect even upon close examination of the code. The winners have been quite successful. Even with only 15-20 lines of code, it's a challenge to locate the underhandedness even when you know exactly what you're looking for. The phase "microscopic needle in a galactic hacksack" comes to mind when imagining the challenge of finding malicious code that may or may not even be there, in a code base thousands or millions of lines long.

Re:Audit necessary

[mysidia]

I hope that he's right, but without a thorough audit, who can say?

The whole scare behind crypto backdoors is they can include sidechannel leakage, and they can include subtle leakage through the underlying drivers. Which can amount to elaborate timing vulnerabilities and other types of vulnerabilities intentionally introduced that are poorly understood by developers in general.

Remember... even though the crypto in the SSH protocol was perfectly sound, as you were typing a password in SSH; a timing attack could be used to assist an attacker in guessing the password typed. For example, the minute timing between keystrokes can identify some passwords that are much more likely to have been typed than others, reducing the attack required to something much easier than brute force.

You can have a backdoor without even revealing the key material or having an obvious vulnerability; all the 3 letter agencies need is a mechanism of reducing the work to crack the key to something much less than brute force. If the operation of the cryptosystem in any way makes the key easier to get than brute force, then the attacker's work is massively reduced.

In other words, it's so subtle that even a thorough audit cannot say, and a complete rewrite of the code would be required to guarantee no intentionally backdoors by the original authors (though it won't guarantee no backdoors by the new authors. and it definitely won't guarantee no subtle vulnerabilities)

It's possible can be no visible error for an audit to discover, and yet, the way the code is structured, could cause information to still be vulnerable through essentially a form of compromising virtual emissions.

Re:Audit necessary

[ThePhilips]

The goal of the contest is to write a chunk of code that does something, well, underhanded that is difficult to detect even upon close examination of the code.

First two examples on the front page haven't passed even through my shallow code review.

The third sample failed at readability (ambiguous operator precedence) and I would have immediately subjected it to re-factoring.

It is not that difficult to detect the problems.

My first, the most generic rule of code review: code works much like the way it looks. And I know for a fact that OpenBSD folks use that rule too.

P.S. The 3 samples I looked at are the winners from the year 2008.

Re:Audit necessary

[drinkypoo]

And while you are entirely correct, the differentiating factor between OpenBSD and basically any other operating system is that it is under continual code review for things that might cause security problems, which has famously rendered OpenBSD immune to a number of attacks to which other systems are vulnerable, including systems which started with the same common codebase. As such OpenBSD seems least likely of all possible projects which could have absorbed this code.

Re:Audit necessary

[Rhaban]

The third sample failed at readability (ambiguous operator precedence) and I would have immediately subjected it to re-factoring.

The operator precedence have nothing to do with the maliciousness of the code (if the third sample is the code from John Meacham). This part could have been refactored any way you'd like, the exploit is in the algorithm itself.

Re:Audit necessary

[Anonymous Coward]

That's not the point. The point is that every discussion these days ends in "citation needed" when there are no more arguments.

The _fact_ that BSD gets audited constantly is can be found easily, it's not obscure knowledge.

But, let me give you an example why this is annoying: You say that the burden of proof lies on the guy making the bold statement. Well, is that a fact? Can you cite some references for that? How are you so sure? Then you state that OpenBSD is an irrelevant niche OS. Well, that's your opinion, I think, unless you can point to some peer-reviewed research on the matter. And I could go on.

See how you can't have a normal discussion when one party doesn't bring arguments, but only shouts "citation needed"?

Re:Audit necessary

[Chris Burke]

To me, it doesn't matter where in the implementation the bug is, since it has to be rewritten anyway for readability reasons.

Which is a fallacious viewpoint, because when you reject the patch, the author could easily recode it within the appropriate coding guidelines yet the bug would remain. In fact, you could have refactored the code yourself and yet still kept the malicious payload.

Code style is important and it's right to reject a patch with it. It's wrong to say this negates the need to actually find the bug. Which you didn't.

It also BTW would trigger another alarm in the eyes of seasoned code reviewers: in the "isdigit() == true" branch it looses the read character, printing '0' instead.

And then someone would say "No, that's the [intended, benign] purpose of the routine".

So with the style issues resolved, and the thing you thought was the bug not being a bug at all, on what basis would this "seasoned code reviewer" reject the patch? At this point the only reason is because you know it's malicious. But if you didn't, it looks like this would have passed your review.

Don't feel bad about that, though. Feel bad about thinking finding flaws in deliberately crafted malicious code is so easy when real seasoned code reviewers know it isn't.

Re:Audit necessary

[toofishes]

$ man ssh-copy-id

Re:Audit necessary

[jc42]

They spent LOTS of time auditing.

Looking for code taken from somewhere else is relatively simple when you have access to both sets of code

So did MS actually show the ReactOS people the supposedly stolen code? A few years ago, when MS made similar accusations of stolen Windows code in linux, there were lots of calls for MS to tell us exactly what code they were talking about. MS simply stonewalled those requests, and continued to make vague, non-specific public accusations that couldn't be validated. It was widely understood to be a marketing ploy, to put the fear of Microsoft's lawyers into potential linux customers' minds.

If a company is serious about infringements, the laws generally require that the accusers state explicitly what is being infringed where, and give the culprits a chance to remove the offending infringement. An accusation without the specifics is legally worthless, since nobody can stop doing something if they don't know what the something is.

There was also the suspicion that, if there was common code in both OSs, it was because MS "stole" the publicly-published linux code rather than the other way around. But, while that's more credible (due to the difficulty in getting a copy of MS's source code), it's a different story than we're talking about here.

There was at least one bit of humor in the "linux stole from Windows" story. At one point, a MS rep mentioned a line count for the stolen code. Someone did a count, and said that the number matched the number of "/*" and "*/" lines in the linux kernel source. This might sound frivolous, but it goes along with the famous story of the Sys/V version of /bin/true, which was a shell script consisting solely of a blank line and an AT&T copyright notice. MS claiming copyright ownership of comment delimiters would be roughly similar to AT&T claiming copyright ownership of a blank line.

Re:Audit necessary

[firewrought]

It is not that difficult to detect the problems...

Maybe not for an experienced code reviewer who's examining 20 lines of code for an extremely simple security need. In the real world it takes extraordinary resources (talent, discipline, passion) at both the individual and organizational level to produce "logically" secure software. Even then, it usually takes academic/hacker security research to find subtle, indirect attacks that depend on power consumption, network behavior, and other such complexities [the SSH packet-timing vulnerability springs to mind here].

Every professional software developer can, at some point in their life, benefit from studying material similar to Meacham's vulnerability. Maybe you are so far beyond that level that you do not appreciate the Underhanded C Contest's educational potential. Pity that it's not better established, but maybe that will come in time.

Re:Audit necessary

[snowgirl]

All of these would be sore thumbs in a code review. Getting this into production code would have to rely on your co-contributors being nitwits.

Working at a Very Highly Notable Computer Operating System Producing Company, I was hit by a number of reviews that likely would not have caught any of this code, because no one I worked with cared particularly hard about code-reviews at all. I would constantly get code reviews back that state: "looks good", and then after performing my own code review, I would pull up some crazy stupid easy-to-catch bug that anyone should have seen if they actually looked over my code review.

And when I gave these code reviews to others, what I received back were people being pissed and upset that I would nitpick their code.

As for using totally stupid and inefficient algorithms, I hit across a number of those while working on a bug, and I attempted to refactor them when I could. One of the most egregious issues I dealt with was a config file reader that listed magic constants THREE TIMES throughout the code. Once for a a validity check, the next for a more robust check, and then a huge IF-THEN-ELSE block to define behavior. After refactoring this to use ONE set of definitions for these magic constants, the code was so altered that it could not be backported at all.

I absolutely don't trust code reviews from anyone outside of the open source community anymore. They have their own job to do, and they rarely consider code reviews of someone else's code to be their "job". They view it as being non-productive and non-work... like going to meetings. Volunteers at least take the code-reviews seriously, as it is their own time that they're spending on it.

Re:Audit necessary

[youn]

It took a while but they actually audited the whole code and they documented how they came out with most stuff... they really did a good job... they are moving slowly but doing phenomenal work

Re:Audit necessary

[Plunky]

Citation needed.

Ok then, the first hit from "openbsd auditing" leads to an OpenBSD Security page which has a section claiming that OpenBSD has a continual audit process and that it is successful..

I'm not necessarily thinking the opposite, but is OpenBSD really that much audited? Are we talking about the kernel? The network stack? Or the encryption protocols?

As I understand it, OpenBSD refers to the whole release, everything they ship.

Now, I'm not sure if claims from the OpenBSD marketing department actually translates to a citation.. I am of the feeling that an "audit" would imply that there were specific procedures followed (searching for particular algorithms known to be problematic for instance) and specific records kept of the results but I have never been able to find public records of those.. Yes, there is the OpenBSD CVS repository but that includes things which are not part of any audit. I found a quote from Theo de Raadt "Most bugs in software are the same ten to fifteen mistakes made over and over" but I don't know what he thinks those mistakes actually might be.

So in conclusion I [as a NetBSD devloper] would be interested to see such records.. I know that many open source projects are subject to Coverity scans which are more public, though it seems that OpenBSD is not listed at this time..

Re:Strange how much fuss...

[tomz16]

...can be made over something so obvious. OpenBSD's code has been screened again and again. If something was amiss somebody would have noticed it . . .

Yeah just look for the parts commented //super secret FBI backdoor, shhh!

You obviously have not seen things like this http://underhanded.xcott.com/

Re:Strange how much fuss...

[bejiitas_wrath]

The OpenBSD source as is known is the best for security as everything is screened and checked, this would have been discovered in that process. Microsoft, who checks their source? They could have any backdoors installed and how would you know? The Open Source way is the best way.

Lose that misconception, please

[Anonymous Coward]

First, most "open source" code is written by employees working for a corporation.

Second, nobody reviews it outside a very small number of people. It's easy to miss things like well-hidden back doors. And that's not even getting into the politics of open source review and the insular cliques of developers - just try and get anyone to listen to you when you start saying you found a back door.

Third, it's cryptographic code. There are probably an uncountable number of "back doors" that could be incorporated into the code that would get by almost all very experienced and very good cryptographic programmers. Just write the code in such a way that you remove a little bit of randomness. Hell, maybe you can write what looks like perfect code but rely on a quirky compiler optimization to do your work for you. It won't matter how many times you screen the source code for something like that. And how many good, experienced cryptographic coders spend their spare time reviewing BSD code in detail anyway?

Link to the ACTUAL FREAKING POST

[brunes69]

Since the useless summary did not include one

http://marc.info/?l=openbsd-tech&m=129296046123471&w=2

Re:Sorry, but how..?

[vbraga]

One of the problems is the lack of people with enough knowledge and time to review, for free, something as cryptographic code.

What about the law?

[DoofusOfDeath]

If the FBI did this without a court order, wouldn't they have been in breech of laws regarding attempted wiretapping and/or unauthorized computer access?

If so, have we just accepted that the FBI, CIA, and NSA break laws with impunity, and that there's nothing we can do about it?

Re:What about the law?

[DragonWriter]

If the FBI did this without a court order, wouldn't they have been in breech of laws regarding attempted wiretapping and/or unauthorized computer access?

Probably not; AFAIK -- even assuming wiretapping laws would apply -- there is no law prohibiting the FBI from contracting others to build in a capacity that could be used for wiretapping. The only time they would need a warrant is to actually make use of the facility.

Link directly to Theo's post

[martyros]

A link to Theo's post on the subject is much more informative.

Highlights:

• Two of the guys named in the original allegation did work on the security stack, but
• Almost certainly didn't check in any malicious code, and
• "wrote much code in many areas that we all rely on. Daily. Outside the ipsec stack."

Also:

I believe that NETSEC was probably contracted to write backdoors as alleged. If those were written, I don't believe they made it into our tree. They might have been deployed as their own product.

Interesting approach to security,,,

[Giant Ape Skeleton]

"I doubt it, therefore it's not true": Security through incredulity!

The Spine Defense

[Giant Ape Skeleton]

I think you must really have no spine if you except money from the FBI to backdoor crypto software.

"I needed the money to pay for my prosthetic spine!"

The code doesn't even have to be in the source

[brokeninside]

If they can get a backdoor built into the compiler used to build the binaries for the general releases, the backdoor doesn't have to be anywhere in the source.

So, yeah, an audit isn't foolproof.

Re:The code doesn't even have to be in the source

[jc42]

This is why they should rebuild the compiler from source for every release, and make sure to publish the source code to that compiler,

Okay then. What should they use to rebuild the compiler? Do they need to rebuild the compiler compiler? And what happens if the compiler compiler compiler compiler compiler has been compromised?

It's turtles all the way down.

Nah, probably not. The techniques for doing this tend to be variants of the famous example that Ken Thompson published back in 1983, and consists of a compiler routine that recognizes a specific chunk of code somewhere in the victim software, and adds the "backdoor" to the output. The meta stage consists of the compiler also recognizing the section of its own source code where this is done, and inserting the backdoor-insertion code there. This then allows you to remove the actual backdoor code from all the software, and recompiles will continue to insert it even though the code do do this no longer exists in source form anywhere.

The conventional scheme to defeat this is to use multiple compilers to compile each other. The more compilers the better, since if you have N compilers, the insertion code has to be developed for each compiler, and each of the N compilers must recognize the appropriate insertion point in all N compilers. If you randomize the use of compilers, a single instance of compiler i not correctly inserting the backdoor-insertion code into compiler j will break the loop, and after a few compiles, the backdoors will all evaporate.

This is actually a case where non-open code has a use. If you have one or more tightly-held compilers that you use as part of the random rotation, you can make it effectively impossible for an outside agency to successfully insert a backdoor-insertion routine into your other compilers, or into your system's binaries. This is most effective if you can keep these internal compilers a secret, of course, because the outside agency will attempt to bribe your people to get the backdoor-insertion code into those compilers, too.

But each independently-developed compiler makes the intruder's job exponentially more difficult. Even a few compilers would suffice to defeat most existing "outside agencies", especially since it would be very difficult to hide the massive communication and bribery needed to keep the backdoor code installed and functional. And it's especially difficult with open-source compilers, where the hacker community has a strong motive (reputation) to find and expose any mysterious, undocumented code in the code base.

You get what you pay for.

[Dcnjoe60]

Hah, that's just like the government contractor -- write a backdoor into a system that doesn't actually work. Since the so called announcement, and the source being available. If this back door were true, wouldn't there be a patch issued for it?

Personally, I think that the leak got it wrong, it's not about making OpenBSD insecure, it was to openly create the BSoD in another well known operating system.

Re:You get what you pay for.

[0123456]

Hah, that's just like the government contractor -- write a backdoor into a system that doesn't actually work.

Does this mean that the government can demand their money back?

Re:Sorry, but how..?

[0123456]

Read this for an idea, someone hacked in some well crafted code that appeared innocent, had the machine not been hacked it probably would have stayed

That code is neither innocent nor well-crafted. Setting uid to zero is not 'innocent' and using '&& (x = 0)' is not well-crafted since it will always evaluate to false. I don't know whether the compiler will generate a warning in that case, but it should, and while a brief look through the code might miss that it's using = instead of ==, any kind of code review worthy of the name would spot it and flame the developer who wrote it.

Yeah, right...

[curious.corn]

Backdoors, who needs backdoors?

Forgetting to close an attack vulnerability on all but the software encryption implementation is a much more dramatic and questionable error. Anyone that has taken the trouble to add hardware acceleration to their encryption stands a good chance to have something to protect from undesired access.

But, by doing so they have exposed themselves to the vulnerability itself. Brilliant!

Re:Sorry, but how..?

[TheRaven64]

Not to mention the fact that most of the bugs are hidden in idioms that OpenBSD's style(7) explicitly prohibit. These would be refactored before being committed, and the hidden bugs would probably be fixed without anyone noticing that they were there...

Re:He thinks?

[pete_norm]

Until then, refrain from using any other programs and operating systems because the best anyone can say is that they think their code is secure.

Thoughts

[DaMattster]

FUD is already getting spread around about OpenBSD, see the following article from Linux Journal, "Allegations of OpenBSD back doors may be true" This came as a comment from within the article. The journalist rambles on about far reaching impacts and doomsday scenario for the project. Okay, reality check. If backdoors are found, (a) Theo and company immediately release patches closing the back doors and (b) the FBI would get another black eye for being caught in a major public lie. Far reaching, my ass. In the end, the only changes made will be to close off core commit privileges to all US-based, OpenBSD contributers. Only certain trusted individuals will have core commit privileges. Say what you want about Theo, the man sticks to his principles like cement. Even if back doors are found, I'll still continue to trust OpenBSD as the most secure OS in the world. Why? For every security hole in OpenBSD found, I'll bet that there are several hundred in other operating systems. A 1/~250 ratio is not bad at all!

Re:Thoughts

[durdur]

Certainly OpenBSD has a good track record and finding and fixing security flaws. But in this case, I wouldn't assume the flaw, if any, can be found quickly and fixed. The post alleging it was certainly not very detailed about it.

Not convinced ..

[0dugo0]

Paranoid mickey's take on it .. Interesting read.
http://mickey.lucifier.net/b4ckd00r.html

Yes, you are right...

[PaulBu]

"Reflections on trusting trust", by Ken Thompson:

http://cm.bell-labs.com/who/ken/trust.html

Paul B.