Slashdot Mirror
Free Can Make You Bleed: the Underresourced Open Source
jones_supa (887896) writes "After the Heartbleed fiasco, John Walsh brings attention to the lack of proper manpower and funding to run various open source projects. Free is not usually a bad thing, but it can be when it causes the software your business depends on to be under resourced. 'OpenSSL for example is largely staffed by one fulltime developer and a number of part-time volunteer developers. The total labor pool for OpenSSL maybe adds up to two fulltime developers. Think about it, OpenSSL only has two people to write, maintain, test, and review 500,000 lines of business critical code. Half of these developers have other things to do.' Theo de Raadt has also spoken about too much donations coming from the little people instead of companies, and not too long ago even the OpenBSD project almost couldn't pay its power bills. Walsh goes on to ponder security of open source software, the 'many eyes' phenomenon, dedicating people to review code, and quality control."
It is over fragmented
If a bad actor, such as a government or an illegal organization wanted to inject a zero day flaw, the current system makes it awfully cheap. Heck open-source developers aren't even required to say a loyalty oath before submitting their changeset.
If your business relies "critically" in its functions on such a piece of software, how would you as a business owner ensure the continuity of the "critical" function?
A. Hire someone to maintain and work on that software.
B. Whine about someone not giving you their time for free.
C. Buy a commercial solution which costs you 50 k USD a year and has at most same level of support as OpenSSL (though better packaged and you get to chat with the smooth sales rep)
What do you do?
From a bit different perspective (largely unix-practical) -- when not having enough resources, you are forced to keep stuff simple. That's usually good, isn't it?
Anyway, I always wondered why is OpenSSL such a bloated pile of code. It does one god damn gazillion things tightly packed. Now, TLS implementation itself is pretty simple, Key management tools are pretty simple, PKCS verification tools are pretty simple, mathematics behind that is pretty simple, commandline tools for quickusing the maths are simple, relationship between those entities ("APIs") are well-defined and usually clear. Who stuffed all of it into one project?!
PS. Bonus paranoia&FUD I saw today: http://pastebin.com/gjkivAf3
The author works for the actual SSH company that sells commercial SSH software. Though the points may largely be valid, a lot of the slant in the article is meant to tell people "this is what happens when you don't pay for software, so buy our commercial stuff today. Because it can't POSSIBLY suffer from the same kind of mistake, right? Right guys? ...guys?"
SSH programmers make mistakes. The article writer has an agenda and it's quite obvious. There is no reason to assume SSH is of any better quality than OpenSSH. He even shoots his implication in the foot: "are you going to review two year old patches for errors? No, of course not." This is no different in paid software. If it gets missed during any sort of review, the hole remains. See the recent IE 0-day hole (which has only been around for over a decade) for proof that this is true.
If your business is depending *critically* on a piece of free software then don't be such a cheapass git. Hire a developer or allocate some of your budget to fund the project.
Problem solved.
SJW n. One who posts facts.
Open source software makes sense. Free (as in gratis) software makes no sense and the proposition that people shouldn't be paid for the software they develop is the stupidest thing I've ever heard. No other profession gives away its services like that (where can lawyer to handle my divorce for free?).
Often corporations are the main beneficiaries of your free labor when they should have paid for your services. A much preferable alternative is software released for free for personal use, but with a modest cost per seat for commercial use.
And yes, when I was young I contributed to several Open Source projects, before the term had even been coined.
How many programmers does Microsoft have? Are their products bug free as a result?
This article is nothing but pure propaganda.
Free software may not be perfect, but, from a security standpoint, it easily beats microsoft, and most other proprietary software.
Microsoft's never let anybody dump a bunch of memory from my server. They've never done http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-0166 either.
Open sourcing software can be a much better model than proprietary - not if it's not resourced though.
Do you really think that the required functionality of OpenSSL should be implemented in a way that requires more than two full time developers? A code base of that size for a piece of software with a very concise mission hints at an obese specification or feature creep in the implementation. In particular, OpenSSL should probably be split into the parts which are necessary for an actual SSL/TLS implementation and the parts which are used for creating CSRs and signing them, conversion of data formats and other aspects which are not used online in a TLS implementation. Remember the Unix philosophy: Each program should do one thing and do it right.
So sad to see /. is in M$ back pocket so deeply. Disappointed.
Free is not usually a bad thing, but it can be when it causes the software your business depends on to be under resourced.
Of course, paying money for closed source software is no guarantee that it's going to be adequately resourced either. Compare the two most recent, high-profile flaws, both very similar, in that they deal with memory allocation issues:
- Heartbleed on SSL has a team of 2, was extant for 2 years, was patched in 6 days, and the patch was available to anyone who used the software
- CVE-2014-1776 on Internet Explorer. Don't know how many people the team, was extant for 13 years, was patched in 6 days, and the patch was originally going to be denied to users who hadn't upgraded recently.
This does not seem to be an issue with closed vs open source development models - both have had major vulnerabilities extand for far too long, and both can turn around fairly rapid patches when needed. Doling out cash to Microsoft is no more effective at securing your applications than using free open source products.
Just because you're paranoid doesn't mean there isn't an invisible demon about to eat your face
The only way corporations are going to carry their fari share of the burden is if they are legally required to. The only way to do that is to make them pay with $, its all they understand.
Libre software is being used by corporations to build gold-plated cages for consumers. Its time to stop playiong their game.
Our glorious leaders are fundamentally wrong on the concept that software tshould be "free to use for any purpose", it should be free to use for the purpose of ensalving us.
sort of a racist post, don't you think?
The problem is not that the software is free (as in open). The problem is that people (and companies even more) perceive it as free (as in beer). That that's the main misconception.
Companies want to cut corners by using OSS. They don't do it because it's easier to review, easier to adapt or easier to find someone who can audit it sensibly. They want it because they can grab it and use it without having to pay anyone for it.
And that simply won't fly. Because that entails the "can't someone else do it?" attitude. Yeah, the code should be reviewed. But someone else will do that, we needn't spend money on that. And it should be audited, but can't someone else do it and we save some money?
Funny enough, the fact that anyone can review, audit and fix things is also the reason why nobody does it. It's a bit like that job in your company that anyone could do, and since anyone can do it, everyone relies that someone else will. There's so many who can, at least ONE of them will. Right? RIGHT?
And since the fact that it is "cost neutral" (to avoid saying the ambigious free) is one of the criteria, if not actually THE criterion, why an OSS product is chosen 999 out of 1000 times in a corporation environment, you may rest assured that the same cheapskates that chose OSS because they can pinch a penny will not spend it on auditing it.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Ignorance is bliss.
http://cve.mitre.org/cgi-bin/c...
More specifically related to SSL:
http://cve.mitre.org/cgi-bin/c...
http://cve.mitre.org/cgi-bin/c...
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
I just have to comment that if you have 10 of 1/10 developers, you don't have 1 developer. Instead you have 10 developers that are not focused on that project. If that is hard to understand, lets take a car analogy. We have a car that is driving by one driver. After a while, the driver is teleported away and another person is teleported to the car to continue driving from there. And the same is repeated multiple times. Would you like to be a passenger in a car like that? If not, why do you think having non 100% developer is a good idea.
Microsoft's never let anybody dump a bunch of memory from my server.
While this may be true, consider Code Red (2001 worm). Setting aside for a moment that it attacked an exploit Microsoft had patched a month prior, the mere existence of an IIS overflow bug, allowing the installing of a worm, meant the attacker could have gained access not only to all the input and output on the server, but also to all the data sitting on the server. Patches servers were safe, but that's assuming the flaw was not being exploited before it was patched.
It's of course not just Microsoft. Back on the open source side, my own favorite web server software, Nginx, had patches for buffer overflows. You're going to find this in both proprietary and open source software until static code analysis tools get good enough to catch all the buffer overflows, and programmers are using these tools as part of their standard programming process.
Heartbleed's bad, but by no means is it necessarily worse that what can be done with a buffer overflow, except that Heartbleed leaves no commonly visible trail (unusual files on the hard drive, or process running in memory).
Timing is pretty convenient. We have a tale of two exploits:
-Heartbleed. Open source project. Huge catastrophic bug, existed as of beginning of 2012. Fix available pretty much immediately upon discovery. As a result, significant resources are pouring in to proactively examine OpenSSL, some fixing and some forking OpenSSL. One way or another, the fix was immediate and concerned parties are empowered to do what they think is needed and the open source world will have risks mitigated as well as closed source being able to make their own call since it is BSD licensed.
-MSIE vulnerability. Closed source. Analagously large bug (albeit client side instead of server side by sheer luck), has existed since at the very latest 2008, but probably as of 2001. Fix was over a week in coming after disclosure. If you are an organization standardized on IE, you were largely SOL with respect to a fix (though mitigation through tedious security settings was possible). Maybe MS ramps up an internal effort to root out more of these, maybe they don't. They seem to have been in a more vigilant stance as a matter of course and that wasn't enough to stop it.
So in other words, very important projects with huge responsibilities can cock up. They can be open source, they can be closed source. The practical lower bound of resources to address issues in both cases will be small when no one knows something is wrong, but the upper bound when concern happens is much higher in open source.
Some have argued that the 'any bug is shallow with enough eyes' was proven wrong with heratbleed. Discovering security bugs are always more tricky than the bug intended to be considered in that philosophy, but even then once discovered, the bug was very very shallow.
XML is like violence. If it doesn't solve the problem, use more.
Who woulda' thunk it? You Get What You (Collectively) Pay For. It doesn't matter if "payment" is in the form of money or manpower. If software you use is built by labor effort much smaller (in cost and/or size) than is usually needed for a project of a particular size or complexity, it should come as no shock that the product ends up not being the quality it needs to be.
He just observes westeners. He does not attack or insult them in any way. It's not a racist post.
Oh dear, yet another Harvard BizSch / Big Business canard that FOSS is governed by the contraints of closed software. Resources only matter when they are restricted -- ie only [certain] employees can fix the code.
FOSS does not have this restriction _AT_ALL_ and that is one of it's greatest strengths. Torvald's "With enough eyeballs, all bugs are shallow." Yes, limited resources may mean it takes a while for an offical patch (still quicker than MS) but unofficial patches can and are generated by anyone interested and capable within minutes.
This has absofuckinglutely NOTHING to do with lack of resources and everything to do with BAD engineering practices being applied.
One follows from the other. If your Free license says that anybody that works on your product is required to give away their efforts for free-beer free, it should not be surprising that it's difficult to find companies to spend money on something (like paying a developer) that won't give them a competitive advantage. This, incidentally, is why we have taxes; it forces people (and companies) to pay for the common good. We wouldn't have much in the way of public works if they relied solely on charitable donations and user fees.
This is a persistent weakness of Free software, but you'll never get RMS to admit that money to pay for programmers does not magically fall from the sky. People are cheap, and if they can get something for free, it's no shock that few of them will pay for it.
In my mind, an ideal software license would have the following;
1) Mandatory Code Release (This gives you some software Freedom)
2) Payment required to copy and/or use the software.
3) Some sort of revenue sharing scheme so that any contributors to the code receive a portion of the funds collected.
Think of it like a "software co-op license"
(This, incidentally, is how industry standards commonly work in the hardware business. You want to implement the IEEE 1234.567 standard? You pay up a standard fee per implementation, and that's doled out to the contributing companies.)
They overlap and interact in unexpected ways, along with the theft economy and the subsistence economy. OpenSSL is a prime example of these overlaps and the complexities of trying to manage all that socially. Should the planned government economy make the code work via tax-supported staff of a government agency? Should businesses exchange money for more development work and support services specific to their needs? Should more developers just donate their time or individuals donate their funds to make OpenSSL work better? What mix makes sense? Especially for software of such global importance?
I talk about the interaction of those five types of economic transactions in general in a youtube video.
https://www.youtube.com/watch?...
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
So how many developers does OpenSSL need for maintaining the code base?
Troll is not a replacement for I disagree.
When you have a widely-used, yet complex, product that nobody has to pay for, doesn't require tech support (unlike, say, an OS), doesn't have any provisions for proprietary (i.e. non-free) features, and isn't really much fun to work on (unlike, say, a compiler), it should come as no shock that it's somewhat difficult to recruit enough eyeballs to look for all those bugs.
Yep, a patch can be issued quickly, but a project with sufficient access to resource ahead of time breaks less to begin with.
Heartbleed was the result of someone adding an optional and useless feature to OpenSSL that should never have been added in the first place. That's not a problem with having insufficient resources, it's a problem with poor management.
If it has anything to do with resources, it's a sign that people on the project have too much free time on their hands, because if there had been anything important to be done, people wouldn't have the time to add this feature.
Understaffed to save money with a huge backlog, insane deadlines, cut corners, and massive scope creep. So what's his point?
putting the 'B' in LGBTQ+
Inadequate resources is hardly the exclusive domain of Open Source projects. Nor is a failure to adequately vet code particularly reflective of an open development model. The insecure, buggy code in the devices used in the world's infrastructue display those facts, perfectly. Device manufacturers tend to focus on hardware, underfund and understaff their software development and demand unrealistic delivery times. These are, by and large, proprietary endeavors.
Except in the imbedded markets where there are still 386's. We just had one bank buy out branches in our area and guess what, the ATM they replaced had still used a 386 as the core gui driver. They endedup totaly replacing it because they could not find anyone who knew how to change the codes between the gui section and the data line (that they changed from POTS to ISDN but that is another 'we are 20 years behind' deal). SSL is wrote in C to keep being used by old garbage like this becuse it costs to much to upgrade them and many people do not even know they are still there.
AC because I forgot what my /. account was 10 years ago.
There are people who would love to have everyone paid evenly but work to the best of their ability. I had a 'job' at such a place one time. I ran from there very fast. Sadly waiters still endure this type of crap (shared tips)
The point is a closed source project isn't going to accept patches from some random dude on the internet.
I don't trust "free" software. And never will. If you pay for software and it does not perform, you probably have a comeback on the vendor. If "free" software does not perform, you have no comeback on anyone, OpenSSL being exhibit one. And in case anyone was wondering, the ONLY reason "free" software has been so widely accepted by corporations is because it is supposedly "free". IMHO, free = crappy.
...where the CEO's idea of the time it takes to develop anything is off by a factor of five, and every developer is also an IT guy and half a dozen other things, how exactly?
Open source is not different from communism... Open source is principles of communism applied to software.
It's a little odd that the go-to company to contrast with open source is Microsoft, given that they just open-sourced all of .NET, and a number of their other software projects (e.g., Entity Framework, F#, etc.) have been open source for some time. In fact, many of the big tech players have open-sourced big projects, some from the very beginning.
Open source is turning into slave labor where corporations exploit open source projects to make their billions and build their walled gardens to destroy the freedoms that open source was supposed to guarantee in the first place. Forget H1-B visas, the real exploitation is open source itself as Google, Apple, and others build their empires off the backs of people who contribute to open source. (At least Microsoft hates open source and has a bad NIH complex.) So why do people donate their time and effort to make corporations richer? They don't get any of the billions. The whole concept of open source is beginning to break down. I don't know why I would spend one second of my time on an open source project that made Google or Apple richer.
OSI Certified(tm) open source software is distributed under a license that conforms to what are essentially the Debian Free Software Guidelines. And these guidelines include free redistribution.
This is exactly why I only use Microsoft products!
They make software which is well tested and secure through their thousands of well paid developers!
Let's get it out the door for the absolute least amount of investment, is common.
The ones now complaining about how risky it is to rely on open-source software are exactly thos thousands of companies who just use open-source and never give anything back, usually not even provide a patch ever. It is not the fault of open-source it is the fault of greedy assholes who just take, never give back and then complain and bash.
paying them means at least they would be invested for the paycheck. someone sitting there 9-5 would have to do some sort of work.
Except that 'blaming the vendor' does not work. Nobody every sued microsoft with any success, even though their software is full of security holes bigger than heartbleed and there's a boatload of viruses taking advantage of this fact.
No, I prefer my source open. I can fix bugs myself that way - or more often, ask the community. Usually, the bug is known and has a workaround.
Their "fair share"? Well, that is very little for something as prevalent as openssl.
Not that it matters, corporations are required to carry some burdens. They are supposed to take care of their assets (such as password databases). Putting core sw through the occational test is one way. They can do it on their own, or opt to fund the maintainers, or some software testing institute. Oh, and don't trust commercial sw. With that, you don't have sources to check. It is still possible to see how it responds to protocol abuses though...
This article is a direct attack against OpenSSH by a commercial competitor. OpenSSH is part of the OpenBSD project, a project with one of the most stringent code review practices known to mankind, and this accusation is utterly ridiculous!
Companies routinely underfund development staff, overwork them, and then end of life products in order to shove 2.0 out. I don't know why everyone's making a big deal out of 'heartbleed'. There have been exploitable faults in open and closed software many times before, some of which have been worse.
I've been critical of the often blase attitude of open source fans. I maintain there's an almost mystical belief in the merits of FOSS, one which reflexively feels the need to deflect and minimize the weaknesses of the model. Which is not to say there aren't a lot of strengths too.
However I'd like to comment on the quantity of personnel in the project. This is the "...one fulltime developer and a number of part-time volunteer developers" bit.
I've always been amazed at how few the number of people there are tasked with any given piece of programming work in any given organization. Also how few alternates are available, should an alternate be needed. And it's not relevant how big the organization is either. The number is always tiny, never more than 10 and routinely less than 5. My belief is that this is a function of organizations finding the optimally smallest number of people needed based upon cost. Even when a larger group of people could theoretically do the work, there are micro-specializations and varying skill levels going on all the time. These can be formally acknowledged on an org chart but often it's done by a manager and handled simply by work assignments.
As a result, I'm not terribly surprised that there's only one full-time developer working on OpenSSL. That's a commonplace finding in my experience. Even in global organizations employing hundreds of thousands of people. Therefore using this information as some kind of indicator of how under-resourced the OpenSSL project is, is flawed. It may be under-resourced but you need a different metric to establish that.
Sure thing. Having spent the last 2 years rewriting a 1+ million line 50% MIT/BSD license open source 50% closed source project with a large number of cobbled together components, many open source projects are 1 to 2 developers away from being unmaintained.
That is why we used 1 small 5K lines open source library and 1 third party UI controls library for the rewritten system. The old system had challenges in
1) recruiting C++ developers,
2) keeping the developers around for more than 1 year even with good pay and working hours (i.e, technology skill rot),
3) delivering enhancements at all instead of spending 100% of development budget on bug fixes
4) maintaining a functional build and deployment system - think C++ of various ages, some C, some Fortran compute libraries, Com objects, X different display libraries (plotting, screen graphing, printing, save to pdf, save to image), connection to home office, connect to different workflow databases, etc.
One doesn't replace a 20+ year old system originating on unix workstations, ported to windows 16, ported to win 32 nt, ported to xp, ported to 64 bit,... without paying a large cost.
Our rewrite effort was specifically blocked by management from adding open source libraries, third party libraries, custom controls, custom build/deploy steps, etc without a full business case justification process for each and every one to be included. The business owners did not want a repeat of the legacy system with each round of developers, managers, architects adding in their pet library, build tool, or special skill set.
What started this change was the company being burned by MS abandoning Silverlight after 3 years from the first usable version v2 to end of life with v5. They did not want the extra risk and went to reduce vendor forced end of life upgrading.
This, along with the well known historical X million dollar SAP failure, may be a dark spot that IT will have to live with for many years to come.
Uhmmm, no. There is a world of a difference between communism and socialism. GNU and other Free software is a kind of socialism. Most countries in Europe and also Canada in North America, are socialist.
FWIIW, communism is also a kind of socialism, but a particularly bad, demented, fascist socialism, which has now fortunately mostly died out.
.......FORTRAN
You lose, c fanboi asswipe!
If the license was copyleft license such as the GPL, any time a company made changes and distributed it, they would have to contribute back, encouraging them to pay the developers. The problem is they decided not to force companies to distribute the source to any binaries they distributed, so companies can just make proprietary versions of the software and not pay developers to work on the open source project.
After the RSA breach the cashcow that was SecurID took an existential hit which might have proved fatal for RSA had the EMC mothership not gone on a heavy acquisition spree with likes of startups like Netwitness, SilverTail and Aveksa.
The breach, together with a few, ahem.."other alleged security issues" not only severely tarnished their reputation but threatened the very survival of the division and its brand.
The point here, is that a global IT powerhouse, with thousands of employees, shareholders, partners and investors, all have a vested financial-trust relationship with the company. It wouldn't be a stretch to believe that contracts were probably lost, employees laid-off and SecurID competitors, like vultures, flocking to the smell of spilt blood (pun intended).
The risk a company assumes of not adequately ensuring the quality and security of its products has far reaching consequences.
Target's CEO just resigned because of the repercussions stemming from the store's recent breach.
At the end of the day who has more to lose if they screw up something like this? The company with employees, shareholders, partners etc., or two pro bono developers and an open source community (who all have their own jobs and priorities)
changing their product strategy with rent pitch of companies like Netwitness Target's CEO Gregg Steinhafel
It is possible for even free stuff to cost a lot more than it is worth !
When RSA was breached a few years back its flagship product, SecurID, took an existential hit that might have proved fatal for the division had the mothership not gone on a heavy acquisition spree of cutting edge security start ups.
The breach, together with a few other, ahem.., "alleged security quirks" not only severely tarnished the companies reputation but threatened the very survival of the division and its brand.
The point is that a global IT powerhouse, with thousands of employees, shareholders, partners and investors who are financially vested in its bottom line, can't afford to make mistakes of this magnitude. It wouldn't be a stretch to believe that contracts were probably lost, and employees laid-off. Not to mention competitors, like vultures, flocking to the smell of spilt blood (pun intended). At the end of the day the big companies can and do survive - the Microsofts the EMCs, the Adobes - but not without loosing a few appendages here and there. (Target's CEO for instance).
The risk a company assumes of not adequately ensuring the quality and security of its products has far reaching consequences.
At the end of the day who has more to lose if they screw up something like this? The company with employees, shareholders, partners etc., or two pro bono developers and an open source community (all who have jobs and priorities of their own)
In the long history of humankind (and animal kind, too) those who learned to collaborate and improvise most effectively have prevailed - Charles Darwin
The operative word here is "effective".
I'm a great believer in OpenSource, but when it comes to security, I'd rather use the code that was developed by someone who has something to lose.
The title was about openssl having a lack of resources. Two people only to support OpenSSL --wow.
The comments were how bad it was to only have two support people for OpenSSL.
Here is how I think that support dropped down to only two people.
a) In the beginning of the project there were many many eyes on the sources, and it evolved over time to be very bug free.
b) As the reported bugs began to diminish, one did not need 50 people to solve 10 bugs. Ergo, many of the active developers and analysts moved on.
c) Bugs finally started to not be reported, ergo, why should a legacy product, if it is working and it does not break need support, even if it does something useful.
d) Those who put the most into the development have an attachment to the product. They will continue support and provide enhancements.
e) PANIC A security flaw was detected. Does it take a cast of '00s to repair? No. Does it take a large cast of '00s to fix? No!
f) Now, lets do a postmortem and see what there is to see. Wow, old code that was commented out, left in the source just in case....
g) Cleanup completed, old unused code removed, its back to legacy status. How many people now required for support?
Second idea
Large software house, 1000 programmers. Team of 3 to 4 support product xyz. Major bug reported for xyz, Can company redeploy and retrain 25 programmers overnight to work to repair xyz? What about the ongoing projects.
Third Idea
Consider any product xyz (say Linux). Linux has it's detractors, but has a very very strong developer team for various subcomponents. If a particular subcomponent becomes legacy stable (no reported bugs in a 90 day window), how many individuals will remain to provide support for that component.
Conclusion
The article takes a jaundiced view of open source projects and support. Be it open source or commercial, the people rules are the same.
Leslie Satenstein Montreal Quebec Canada
Yes, I know that most programmers write internal software where it doesn't actually matter if it's "Free" or not, because it never leaves the company. (Does it even really have a license at all? I know I never have to agree to a license agreement to use software internal to my company.)
But for that software (like OS's and other back-end infrastructure) of a more universal nature it makes the most sense to NOT develop that internally. And writing that software requires a radically different skill set from database apps. How are programmers that write that software supposed to be paid? Answer (from this example, anyway): Not much. Shocker: There's very little money in support contracts for small-ish low-level libraries.
I have no problem whatsoever with the GPL. But I DO have a problem with RMS's insistence that NOT giving away your work to anybody who wants it free of charge is the only ethical means of programming. If you want to give your work away, that's great, and I'll support efforts to fight against anybody that tries to then charge for your efforts. But if I want to write some software and get somebody to pay for it, that should be my option too.
And he's actually quite horrible at predicting problems down the road... if he was better at it, the Hurd would have shipped or been canceled well over a decade ago.