Slashdot Mirror
Porting Open Source to Minor Platforms is Harmful
Tamerlan writes "Ulrich Drepper posted a blog entry titled "Dictatorship of Minorities". He argues that open source projects' attempts to support non-mainstream (read "non-Linux") operating systems slow down development and testing. While Ulrich may be biased (he is a RedHat employee) he has the point: if you ever read mailing list of any large open source project, you know that significant piece of traffic is about platform-specific bugs or a new release broken on some exotic platform."
But I agree with the opinion if, as a developer and/or integrator, you are attempting to reach a mass market. Red Hat, then SuSE and others have abandoned the Sparc platform awhile ago. I bought an old Ultra 1 and am now limited to a few distros to keep my Sparc machine up to date.
But it was my understanding that the idea behind open source was to roll your own if your machine was not covered. If I want to keep the Sparc chugging along, I guess I'd better learn to port it myself.
"Rocky Rococo, at your cervix!"
The development of netBSD to over 40 different platforms has brought a lot of good development to many different platforms that would have been dominated by mono-operating systems. A good instance is the handheld devices platforms (HPC,Palm PC, etc.), which would otherwise be dominated by Windows CE (except for the few Linux Palm PC's, but majority are WinCE). The development of netBSD for the majority of platforms has reached great maturity, and it is still developing well.
If you write your code to be portable in the first place, fixing platform-specific issues should be quick and easy.
And, of course, you write your code to be portable because you make sure it runs on the big three: Windows, Mac OSX, and Linux.
Right?
Actually, I think a much larger problem is just that: Many OSS developers don't even try to support Windows. Yes, I know you hate the OS and don't want to support Microsoft, etc., etc.. But, how can you complain about major software not supporting Linux when you're writing your own software that doesn't support Windows? Isn't that entirely hypocritical?
My take: Port your software to every platform you can, especially Windows. This gives freedom of OS to your users. And if you're a Linux user yourself, you should understand just how valuable and important this freedom is.
Very good point. Supporting minority platforms is only troublesome if your code is riddled with i386isms and lazy coding practices. If you do it right the first time with always portability in mind, not only will you get cleaner and more maintainable code, you also get the ability to run just anywhere as a bonus. My prime example as usual is NetBSD.
/x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(85): warning: pointer casts may be troublesome [247] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(94): warning: pointer casts may be troublesome [247] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: bitwise operation on signed value possibly nonportable [117] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: possible pointer alignment problem [135] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: pointer casts may be troublesome [247] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: possible pointer alignment problem [135] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: pointer casts may be troublesome [247] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: possible pointer alignment problem [135] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: pointer casts may be troublesome [247] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: possible pointer alignment problem [135] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: pointer casts may be troublesome [247] /x/s.NetBSD.netbsd-3/crypto/dist/openssl/crypto/de s/fcrypt_b.c(111): warning: bitwise operation on signed value possibly nonportable [117]
It's sad that whenever you build some GNU sources or the latest Linux app du jour you get tons of warnings. Some projects even compile their files with 2>&1 >/dev/null. How sad is that?
It's not just the Linux folks. OpenSSL is actually worrying me:
etc. etc.
And this in a project that's driving quite an amount of sites, authentication mechanism and what not?
Most if not all of the sources that are native to NetBSD, i.e. not imported like OpenSSL and GCC compile without any warning. You automatically get a good feeling about using it.
Something needs to change in coding land.
</rant>
Of course it runs NetBSD. BTC: 1NT7QvbetmANwaMzhpVL6
primetest.cpp
primetest.java
primetest.pl (for fun, because I like perl)
Using Java Blackdown-1.4.2-01, gcc 3.3.5-20050130 and perl 5.8.5 on a 1.3 Ghz Pentium-M, I get these results:
neil@t40-n Documents $ javac primetest.java
neil@t40-n Documents $ time java primetest
real 0m7.809s ./primetest
user 0m7.700s
sys 0m0.033s
neil@t40-n Documents $ g++ primetest.cpp -o primetest
neil@t40-n Documents $ time
real 0m2.699s ./primetest.pl
user 0m2.676s
sys 0m0.007s
neil@t40-n Documents $ time
real 0m47.928s
user 0m46.207s
sys 0m0.138s
neil@t40-n Documents $
How about you? Sure it's a trivial benchmark, but it definitely shows Java way behind (by a factor of three!) C++ for number crunching. Of course we see Perl well behind both, but it's definitely not meant for number crunching, so no surprise really.
Game! - Where the stick is mightier than the sword!
I've actually done some fairly hefty optimizations (real-time array processing on an original Sparcstation, for a nuclear research facility) and know the sort of problem you are talking about. The best thing to do, once you have a working, solid, implementation is profile it and find the areas that - if accelerated - would offer the best improvement in performance.
The problem with having so little time - in your case a month - is that you can't optimize cleanly. You've got to hit the code in the best places first and dig your way through as far as you can in the time.
Typically, slow areas will involve moving lots of data, number-crunching and I/O. You can't do much with graphics I/O, unless you've some good specs on the hardware and know assembly. (That's one reason I learned assembly.) Number-crunching is tough, too - just do as little as possible, do what you can when the system is idling, try to merge operations, and stick to integer arithmetic as it is so much faster. Moving data is easy - don't, unless it'll absolutely kill you. Do everything in-situ, if you can. (I wrote a Mandelbrot generator that did ALL arithmetic inside the 80x87 coprocessor - nothing loaded, nothing saved, which meant no slow transfers. It was still slower than fractint, but it was a lot faster than any other floating-point fractal generator out there.)
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
I track development of The GIMP - it just compiles fine in all "strange minor platforms", and a recent chain of Bugs in Irix compiling was resolved overnight - a matter of the Irix user reporting the bugs, and the core developers commiting the fixes.
However, there is a non minor and weird platform which actually does generate a lot of trafic on the list, and is strange for most developers. Anyoen checking The GIMP bugizlla will find a lot of open bugs for Microsoft Windows Plataform. That however, doesn't slow the project either. It simply goes on, and the developers who work on Compiling and making the windows installer do what they can for the work arounds.
-><- no
With a project as big and important as GCC, you'd think they'd have a server for each platform set up for all their developers to play with. Gentoo has Sparc, MIPS, PPC, etc. boxes for their developers to use for porting software.
It seems to me that a smart idea would be to have some kind of system where a developer could submit a patch, which would then be sent out to a server farm, where each server would try to compile GCC with the patch, then run a test suite. Doesn't Mozilla do something like this nightly?
"I don't have access to a [foo] box" should never be a valid excuse with larger projects.
LOAD "SIG",8,1
People seem to be repeating a lot of "folk wisdom" about portability. Oh it's just bugs. Make your damn software clean you damn coder. Etc.
I can guarantee you that anybody who says this has never actually read the sources to glibc, binutils or gcc. Hell, they probably never even read the mailing lists! I have, and when Ulrich says an enormous amount of effort goes on supporting minority platforms he is totally correct. Hello, binutils isn't the GIMP people! Other platforms have totally different architectures and often need huge amounts of platform specific code from these projects. This isn't a case of sloppy coding, it's a case of massive amounts of work being done to support edge cases. Go read the sources to bfd sometimes. Adding support for one platform that uses different assumptions about basic things like memory layout can require huge reworkings of the code.
Essentially there are a lot of people spouting off here based on their experiences of compiling FooApp on FreeBSD or whatever. Have you written a C library? A threading library? No ... then you are probably not really qualified to judge how much work this generates for Ulrich.
Oh, and for the guy later on in this thread who says "AIX is not a minority platform, WTF?" - I say to him WTF. AIX most definitely is a minority platform. Maybe not in the world he lives in, but in the real world Windows is dominant, sometimes people think about Linux/Mac or even FreeBSD and everything else barely registers at all unless you administer high end servers or work on embedded software (most people do not).
I've been bitten by this mentality before. Back when exec-shield was first developed, it broke Wine (which I work on). So I set out developing a fix. Eventually, I wrote a GNU linker script that arranged virtual memory such that things would work correctly even when exec-shield was active. But it didn't work, because of a simple bug in the kernel. No problem, right? Just fix the bug, right? Well, actually, somebody did. A patch was written, submitted, got into Andrew Mortons tree ... and it didn't compile on Itanium. The original author didn't have access to such a machine, neither did I, and the person who reported the failure (who worked for Intel, IIRC) was overloaded with other work and couldn't fix it. So the patch was dropped.
In the end, a few months later, we had a different solution that was about a million times more complicated. Largely because a simple bugfix patch didn't compile for unspecified reasons on a platform nobody uses and this was grounds for it to be dropped. That mentality of "all computers are born equal" is why Debian has become a laughing stock and it cuts both ways.