Slashdot Mirror
Porting Open Source to Minor Platforms is Harmful
Tamerlan writes "Ulrich Drepper posted a blog entry titled "Dictatorship of Minorities". He argues that open source projects' attempts to support non-mainstream (read "non-Linux") operating systems slow down development and testing. While Ulrich may be biased (he is a RedHat employee) he has the point: if you ever read mailing list of any large open source project, you know that significant piece of traffic is about platform-specific bugs or a new release broken on some exotic platform."
There are many instances where OpenBSD developers indicated that a bug found in one port led to discovery of problems that affected several other platforms. It seems in this case that multiplatform support is beneficial, and the larger the number of platforms, the greater the likelihood that such bugs will be found and fixed.
I'm not sure I totally agree with this article - at least as far as Windows porting is concerned. Programs like OOo are gaining acceptance in the Windows world and that foothold has led my own organization to 'embrace and extend' that success. For instance, for the first time we will be purchasing Apples - running NeoOffice of course - and we already have a few Linux terminals here for public use.
I like to think of OSS/GPL stuff as a 'gateway drug' - to use an analogy. Using it may not automatically make people go to Linux, but it certainly makes it an increasing possibility.
"...Well, there's egg and bacon; egg sausage and bacon; egg and spam; egg bacon and spam; egg bacon sausage and spam..."
The development of netBSD to over 40 different platforms has brought a lot of good development to many different platforms that would have been dominated by mono-operating systems. A good instance is the handheld devices platforms (HPC,Palm PC, etc.), which would otherwise be dominated by Windows CE (except for the few Linux Palm PC's, but majority are WinCE). The development of netBSD for the majority of platforms has reached great maturity, and it is still developing well.
Overall the arguement is mostly bogus. For example many linux developers have trouble writing code that even compiles under any of the *bsds. That is just sloppy coding. If everyone got in the habbit of at least writing code that doesn't use system specific includes (linux developers seem the worst at this) and compiled with gcc -strict -Wall or something similar it wouldn't be much of any issue. While I can see that a request to make something work on OpenBSD VAX might be better ignored I fail to see how supporting at the very least linux/*bsd (Open, Net, Free) on ppc, sparc, sparc64, and x86 is supporting a minority. Overall OSS users/developers ARE a minority and to argue over which minority beats who is silly. Also, to only bother to support linux is no better than only bothering to support windows!
:(){
Let's just eradicate them once and for all. A homogenous Linux monoculture will be easier to maintain and be to the benefit of all of us.
The bugs due to platform bugs -- well, knowing about them helps improve the platform.
If you think fixing these bugs is a pain in the neck, fine. If you think it's a waste of time, however, think again.
Am I part of the core demographic for Swedish Fish?
Keeping your code portable helps eliminate stupid assumptions, which make your software useless when the dominant platform changes. Once, all the world was a VAX, and people did stupid things. Then, the world changed. They kept doing stupid things.
Think, for example, about 64 bit cleanliness. A piece of software which supported Alpha, UltraSPARC64 and SGI's MIPS64, and so on, wouold have been fairly trivial to port to IA64, and AMD64, and PPC64 when they started to become significant. OTOH, code which assumed it was running on a 386 would have been a pain in the ass to port to even just AMD64.
Also, by supporting a broad spectrum of compilers, you will probably be able to understand what is going wrong when you compiler of choice changes. Witness code breakage on gcc3. Devs who had already ported their software to a variety of compilers were better able to respond to any issues, and fix their code.
Many monoculturalists make stupid endian-ness assumptions. Now, Mac OS X is becoming a significant market. If you have stupid endian-ness assumptions, then you may wind up having to basically rewrite in order to gain access to those millions of potential customers/users.
Imagine if OpenGL only supported SGI and 386. Or libtiff only worked on i386. People just wouldn't use them. Things like that get used because they are ubiquitous, and you can build them anywhere.
Of course, you can overdo it. Take a look at InfoZip for example. No, seriously, take a look at it. It works on every platform you can think of, but the price is that the code is almost unreadable. The biggest problem is all the cruft needed to maintain 16-bit compatibility. It desperately needs updating to handle non-ASCII filenames intelligently, but the last thing that code needs is another layer of #ifdef's.
There comes a time when you just have to say "fuck the Amiga".
fish and pipes
Sometimes, we use hyperbole to make a point.
Unfortunately, I don't think that Ulrich is doing that.
AIX is not a minority platform. What The Fuck. Okay, so the AIX guys are asshats in the way they treat GCC, fine. But GCC's claim to fame is that it it is the cross compiling, multiplatform compiler du jour. I think Ulrich loses a lot of credibility to say that GCC needs to not support AIX because it's a minority platform.
*nix applications which run primarily in userspace should port to the various BSD's and Linux easily, and if they don't then 99% of the time it's a bug. And in many cases, it's a bug that will affect the working platforms eventually (relying on nonstandard behavior of system calls, linker oddities, assumptions about file placement, etc). And if a closed Unix platform has paid developers to assist in the porting, then it should run on that platform too. And if the paid devs are dickbrains, then a good project leader should say so. Behave, or fork and get your whining ass out of my tree.
These AIX GCC guys shouldn't be saying "This patch breaks AIX, kill it", they should be saying "This patch fixes *blank* on AIX", at least most of the time.
"IMO the most notorious case is how the gcc development is held hostage by Edelsohn and maybe IBM as a whole by requesting that everything always works perfectly well on AIX. How often has one seen "this patch breaks AIX, back it out". It cannot reasonably be expected that everybody tests on AIX. It is an proprietary OS running on proprietary and expensive hardware which not many people have access to. The overall development speed could be significantly improved by dropping the AIX requirement which, in some form or another, has been agreed upon by the steering committee. AIX is irrelevant in general today, i.e., the situation changed. And the people in the steering committee are too nice to just tell the very small minority causing the problem to take a hike."
GCC is the de facto standard because it runs on more platforms than anybody else.
If it ceases to run on all these platforms, it will either:
a) fork a project that will support them
b) another compiler will take its place as the de facto standard
c) people will be forced to use whatever the default cc is on their OS.
In any of these cases, the portability concerns will get an order of magnitude worse.
I rarely criticize things I don't care about.
Yeah - Sorry about that, our bad.
-- Sun Microsystems
Porting to other platforms/architectures often reveals bugs in your primary target platform. it is often worth the effort to port to other platforms on this basis alone.
also, if it takes you a lot of effort to keep architecture-nimble, there is something fundamentally wrong with your design. this in itself should be a warning.
But there is no benefit at all in supporting something like PA Risc, m68k, cris, etc as configurations of the generic code.
ulrich obviously has no clue whatsoever about embedded systems, and should therefore stfu on this point. one of the most popular embedded platforms is a 68k variant (coldfire) -- it's probably second behind ARM. by dumping support of 68k you castrate linux in the embedded marketplace. there's much more to 68k linux than sun3 and atari/amiga.
his rant against mingw as "undeserving" is stupid. mingw is an enabler -- it means people can develop for win32 without having to pay microsoft $$$$ for the privilege of doing so.
his 'dictatorship of the minorities' argument is actually self-defeating on this point because microsoft users are in the majority. by his own arguments, we should be concentrating on supporting win32 as the primary target for gcc and primary architecture for linux.
utterly ridiculous.
bug-eyed rants like his just serve to reinforce the stereotype that all open source advocates are completely unhinged. it is not helpful in the least.
Spoken like someone who's probably never tried running their JVM-based GUI application on a new platform. Java cross-platform compatibility is a nice idea in theory, but in practice you wind up having to test everywhere and tweak your code when you run into differences in GUI implementations, so it's definitely not write-once, run everywhere. A more complete API specification would help here, but if wishes were horses, there'd be a lot of poo on the road.
Obligatory Zawinski paraphrase:
Some people, when confronted with a problem, think "I know, I'll use Java." Now they have two problems.
If you write your code to be portable in the first place, fixing platform-specific issues should be quick and easy.
And, of course, you write your code to be portable because you make sure it runs on the big three: Windows, Mac OSX, and Linux.
Right?
Actually, I think a much larger problem is just that: Many OSS developers don't even try to support Windows. Yes, I know you hate the OS and don't want to support Microsoft, etc., etc.. But, how can you complain about major software not supporting Linux when you're writing your own software that doesn't support Windows? Isn't that entirely hypocritical?
My take: Port your software to every platform you can, especially Windows. This gives freedom of OS to your users. And if you're a Linux user yourself, you should understand just how valuable and important this freedom is.
It is rare that I can say someone is wrong on all counts, but I have not found one defensible statement in there. (Though I guess one could be hidden and I missed it)
His first mistake is thinking GNU is everything. Maybe for him it is, but for most people we use what works. When the boss sets me down on a AIX machine I want it to work - I'm not allowed to install Linux (though I'd install *BSD if I could wipe the OS), I'm supposed to get work done.
Minorities are useful despite the cost of working with them. Bugs that are 1 in a million may happen every time on AIX. 1 in a million bugs are very hard to find. I've spends days looking at a partial crash trace wondering why it broke, and if it will happen again. With no known way to duplicate the bug it is really had to fix, and hard to be sure the 'fix' works. When it fails every time the bug is easy to fix.
Good programmers should have no problem writing cross platform code. When your code breaks on AIX, it is a sign of bad code - even if the breakage is because AIX doesn't have a function you expect.
Cross platform compilers (gcc) are much easier for me to work with. Because gcc is cross platform I can compile my stuff at home and debug it, than bring it to work and compile it and assume it works. Particularly with gcc 2.95, the support for C++ was so bad that you could not count on code written for that to work on a better compiler.
Speaking of gcc 2.95, other vendors have had better compilers for years, while gcc is only arriving. Even today, gcc isn't a great c++ compiler. (though 4.x is much better) There is no point in throwing stones at other vendors - their compilers may have been expensive, but they at least worked close to right.
The upper/lower case differences with Windows are a non-factor. You should never have any word that differs by case only - it leads to many bugs if you do.
The API differences on Windows are mostly handled by Cygwin and mingw. Those areas that are different are places where you should have your code modular anyway. Mostly we are talking about device and networking code. IPv6 is on the way (has been for 10 years now...), you need some difference code to support that. There is no standard for device code - what works on OpenBSD won't work on linux, or FreeBSD.
True almost nobody cares are VAX - but it is interesting anyway. If you code is modular like it should be, then supporting those weird things isn't a big deal - you write you code, and let those are care about it test.
A short summary: There should be only one OS that anyone runs: RedHat Linux enterprize edition on x86. (not x86-64) Not Fedora core, much less gentoo or those other non-redhat distributions. You FreeBSD people can go to hell.
He wants to take his ball and go home, I don't care, we are better off without people like him in the open source world.
Porting software to different platforms has two distinct benefits:
1) identifying subtle bugs
2) preparing software for future platforms
- Subtle Bugs -
As stated in the parent post, porting software to various platforms help uncover bugs that may not surface during routine testing in a mono culture.
- Future Platforms -
Making software portable prepares software for the future. As computer technology advances, software that has been developed to be portable will be the first code running on the new hardware. I could go on and on about this, but I think the recent articles regarding Intel's Itanium already make this point loud and clear.
-rd
As you say, there are examples where porting has helped a project. I know that in porting one of my games to four platforms (Classic Mac OS -> Windows -> Linux -> OS X) has helped eliminate bugs that I never knew were there. Also, I learned things that have made my later projects easier to port since I more able to write them "correctly" to begin with. By avoiding platform specific libraries and techniques I write better code.
Lasers Controlled Games!
Hypocrite... (software should be free! So I'm holding mine hostage until you release yours!)
And let's imagine what would happen if the vendors of those proprietary OS's decided to play the same way...
Say goodbye to NFS, NIS, Rendezvous, Mono. Perhaps Sun would close Java (which would make the conspiracy theorists happy, but nobody else.) Goodbye to the largess of many large vendors (where do you think organizations like FSF get their funding?).
And that crack about "ideally to one". Linux is a nice OS and all, but believing that it is the one for all purposes from PDAs to supercomputer clusters, data warehousing to real-time control is silly.
Am I part of the core demographic for Swedish Fish?
Second, platforms are not stagnent. Code that only works on 386 linux may some day have to deal with a x64 only world. Who knows what may happen in the future. Making decisions because you reject portability means you reject the future for your code as well.
Third, different compilers are very useful for finding less obvious bugs. Ideally this means having a choice beyond gcc, if one is talking about C/C++, for example :). Using a single compiler means bugs your compiler doesn't itself know will likely be retained. Even using different versions of gcc can help. Different compilers often are good at finding completely different sets of bugs in source.
Finally, pointer/integer size and endian prejudices are evil in C/C++ code. You will find these things very quickly if you spend your whole life exclusivily on i386 and one day try to port to ppc.
As a GCC developer (bias: I work for IBM Research), the only time i've ever seen David Edelsohn complain about something not working on AIX, it was broken on other significant platforms as well (Cygwin, etc), or was latently buggy and just working by luck.
Judge for yourself. Go read the gcc list. Count the number of patches backed out in the past year because they broke AIX vs because they broke some other platform.
It sounds like an unnecessary personal snipe, which, for people who know Uli, well, i won't bother finishing that.
So if this is the most "notorious case" Ulrich's got, then he's wrong.
Particularly the "GCC would be developed much faster".
That is in fact, the funniest thing i've heard all day.
GCC would be developed faster if there was less sniping and fiefdom's and more collaboration. Which, except for a few people, has been what is generally happening. Our development process is accelerating, not slowing down.
And It certainly isn't slowed down because people need to bootstrap on AIX, which they don't.
Nobody has ever required patches be bootstrapped on AIX unless it is very likely to have some material affect on that platform.
This is just the same requirement we pose for any wide ranging change: Test it by compiling it for the architectures it is likely to break on.
Note i didn't say running. We don't require anyone have AIX boxen around. Cross compiles work fine.
Though if you break some architecture, you are expected to at least try to help the maintainer of that arch fix it.
And this is the logic which most Windoze software manufacturers use to throw at us Linux users. IT is why us Linux users are missing some great software out there.
So, tell me... In our minority, why on earth would we take that attitude?
Ulrich is off his rocker.
Wait a sec...
Porting Open Source to Minor Platforms is Harmful
What the fuck? Where'd you get that from? Read Ulrichs post. How about:
Delaying the development of features because of problems with minority platforms that can't be fixed by the bug reporters is Harmful
You may disagree, but unlike the title of this article, it does actually cover what Ulrich is talking about.
...adapting your application to architectures as diverse as x86, ppc, MIPS and Sparc at different word-widths is a great way to uncover subtle and long-standing bugs.
To be sure, robustness may be as optional for you as it was for Microsoft (and would still be, absent competition from Linux), but in the long run it seems to pay off.
Most of us Linux users would not regard, forex, The GIMP as particularly robust, but compared to the typical WIn32 app it's a paladin of reliability. My sister-in-law routinely leaves it open (and unsaved) for weeks on end, confident that it will still be there when she gets back (but a recent hard-disk failure of mine seems to have put the fear of God into her WRT reliability). She also happily browses everywhere fearlessly, knowing that she can't damage anything on her own machine, and nobody either I or her know of have ever been burnt by malware while browsing in Linux.
MS users just don't do that - not more than once or twice, anyway.
Got time? Spend some of it coding or testing
why even bother using such old hardware?
For the same reason Ulrich Drepper is wrong. An Ultra 1 is super cheap, now, and it gives a programmer the chance to test code on a big-endian 64-bit architecture. Are there lines of code with endian dependencies? Are there lines of code that assume 32-bit CPUs?
The same goes for testing on Linux, as well as NetBSD, Solaris, etc. Does the code really use POSIX intelligently? Is the program abstracted well from the kernel services?
This whole article is just a Red Hat employee tooting his company's horn. His advice is inappropriate, in that it promotes bad programming, just as Windows fanboys writing to Win32 can shoot themselves in their feet so often.
Of course the headline Slashdot reported is not what he said. Uli is abrupt, but he is practical, and not stupid. He's not always right, but when he's wrong he's interestingly wrong. If you think he's arguing for something stupid, you aren't paying attention.
What he said was *not* that Glibc, or Gcc, and whatnot shouldn't be ported to AIX, m68k, and whatever. What he said was that he does not care to *maintain* those ports, and should not be expected to. IBM (or IBM's customers) can certainly afford to maintain a port for AIX. Let them. Likewise, all those embedded-system houses dependent on m68k targets are welcome to step up and supply their own patches to keep their ports working.
If a patch to mainline breaks the AIX port, it's the job of the AIX maintainers to figure out how to fix the patch, not him, and not whoever contributed the patch (but has no access to AIX targets).
He's not even saying he would reject patches needed to support minority targets. Whoever's maintaining the m68k port doesn't need to maintain a fork. They are entirely welcome to send along whatever patches they need installed. They need only be sure their patches don't break any supported targets. This certainly makes more work for users of less popular targets, but it spreads the work around, instead of piling it up on those doing mainline development. The mainline maintainers have plenty else to worry about.
There are probably a few valid reasons, I think. Here are mine:
- Not everything requires the latest hardware. Keeping machines running, that are still fully functional (component wise), keeps them out of landfills.
- Sentimentality, I like the Sun SPARC hardware and want to keep using it. Even for trivial tasks like shell accounts for mates and some types of testing.
- Diversity is a useful thing of itself, an x86 monoculture can't be good for us long term.
I am deeply convinced that porting assures portability, and portability is one aspect of clean code where bugs and wrong assumpsons are noticed, resolved and corrected.
Surely porting to platforms such as the Alpha and UltraSparc was a very good basis for porting to platforms such as AMD-64. This is a crucial advantage for free software, where we can be sure that we will be able to support new platforms and make interesting platforms mainstream.
On the other hand, the premisis that the main maintainers can not be responsible for all the porting effeorts is reasonable. Debian is thinking along the same lines, and for good reasons.
I think it is wrong and bad to assume porting is a bad thing and avoid it. Even apparently futile projects such as porting free software for closed commercial platforms gives a large amount of flexibility in design and portability and helps projects such as embedded graphical environments.
Portability is just one facet of advantages of free software and as such is a precios thing that we have to cultivate. But it sould be just another part of the free and open collaboration development process, not an obligations for the main developers.
Just my 2 cents.
-Kvorg
glibc is one of the main reasons why Linux application deployment sucks in major (read: heterogenous) installations.
This is what Marc Espie, an OpenBSD developer said about Ulrich on O'Reilly's OnLamp (commenting the proactive measures OpenBSD takes in C programming vs. Ulrich's "Linux programmers are geniuses" view):
"We have had a lot of success explaining the issues and getting a lot of people to switch from strcpy/strcat to strlcpy/strlcat.
Weirdly enough, the Linux people are about the only major group of people that has constantly stayed deaf to these arguments. The chief opponent to strlcpy in glibc is most certainly Ulrich Drepper, who argues that good programmers don't need strlcpy, since they don't make mistakes while copying strings. This is a very mystifying point of view, since bugtraq daily proves that a lot of Linux and free software programmers are not that bright, and need all the help they can get.
(Considering the shining, flaming personality of Drepper, and the fact that he is always Right, this is not that surprising, though)."
Main difference between the BSD license and the GPL license: one is from California and the other is from Massachusetts