Slashdot Mirror
Ryan Gordon Ends FatELF Universal Binary Effort
recoiledsnake writes "A few years after the Con Kolivas fiasco, the FatELF project to implement the 'universal binaries' feature for Linux that allows a single binary file to run on multiple hardware platforms has been grounded. Ryan C. Gordon, who has ported a number of popular games and game servers to Linux, has this to say: 'It looks like the Linux kernel maintainers are frowning on the FatELF patches. Some got the idea and disagreed, some didn't seem to hear what I was saying, and some showed up just to be rude.' The launch of the project was recently discussed here. The FatELF project page and FAQ are still up."
He needs thicker skin if he's going to deal with the LKML crowd. I wouldn't give up just because it's not merged into the official tree.
Cory Doctorow talking about cloud computing makes as much sense as George W Bush talking about electrical engineering.
The 32-bit vs. 64-bit split is handled pretty well on Linux (well, Debian drug its heels a bit on multiarch handling in packages, but even they seem to be getting with the programme).
Real multi-arch could be useful, but the number of arches on Linux is just too overwhelming. To get somewhat decent coverage for Linux binaries, they'd have to run on x86, ARM, and PPC. Plus possibly MIPS, SPARC, and Itanium. Most of those in 32-bit and 64-bit flavours. Those elves are going to be very fat indeed.
Finally! A year of moderation! Ready for 2019?
I don't get the point in bringing it up.
Things get rejected from the kernel all the time -- because not all things are good, useful, well coded, or solve a problem that needs solving. It's not new in any way.
This in particular seems like a solution in search of a problem to me. Especially since on a 64 bit distro pretty much everything, with very few exceptions is 64 bit. In fact I don't think 64 bit distributions contain any 32 bit software except for closed source that can't be ported, and compatibility libraries for any applications the user would like to install manually. So to me there doesn't seem to be a point to try to solve a problem that exists less and less as the time passes and proprietary vendors make 64 bit versions of their programs.
This idea is kind of broken for Linux. On MacOS, with 2 architectures, it makes some sense, since the actual executable code is not huge compared to data. On Linux, withe a couple of dozen architectures, executable code *is* going to start to take relevant amounts of space, and the effort involved in preparing them will be nontrivial. If this system were adopted, virtually no binaries would be made to support all available architectures, meaning that anyone not on x86 (32 bit) would need to check what archs a binary supported before downloading it, which is about as difficult as choosing which one to download would've been.
# cat
Damn, my RAM is full of llamas.
In the entire forked-up mess of the unix tree, there was only one thing that anybody & everybody cared about - source compatibilty. C99, POSIX, SuS v3, so many ways you could ensure that your code would compile everywhere, with whatever compiler was popular that week. For a good part of 4 years, I worked on portable.net, which had a support/ directory full of ifdefs and a configure script full of AC_DEFINEs. It worked nearly everywhere too.
Binary compatibility never took off because there is so little stuff that can be shared between binary platforms. Sure, the same file could run on multiple archs, but in reality that is no different from a zip file with six binaries in them. Indeed, it needed someone to build 'em all in one place to actually end up with one of these. Which is actually more effort than actually letting each distro arch-maintainer do a build whenever they please. OS X build tools ship with the right cross-compilers in XCode and they have more of a monoculture in library versions, looking backwards.
Attempting this in a world where even an x86 binary wouldn't work on all x86-linux-pc boxes (static linking, yeah...yeah), is somehow a solution with no real problem attached. Unless you can make the default build-package workflow do this automatically, this simple step means a hell of a lot of work for the guy doing the build.
And that's just the problems with getting a universal binary. Further problems await as you try to run the created binaries ... I like the idea and the fact that the guy is talking with his patches. But colour me uninterested in this particular problem he's trying to solve. If he manages to convince me that it's a real advantage over 4 binaries that I pick & choose to download, hell ... I'll change my opinion so quickly, it'll leave you spinning.
Quidquid latine dictum sit, altum videtur
On Linux, withe a couple of dozen architectures,
Kind of, but not really. No more than there are four architectures (PPC, ARM, X86, X86_64) for OS X. There's two architectures for Linux that actually matter, and they're the same two that run Snow Leopard. X86 and X86_64.
I can see why people are going to get up in arms about this. I've been as big a RISC booster as anyone, I think Apple gave up on PPC too soon, and I'm still bitter about Alpha, but that game's over. 32-bit and 64-bit Intel architectures are what matter, and those are the ones that almost all binaries will work for. I'm not running YDL any more, and neither are you. Game's over, instruction sets lost to marketing. The game's over, the fat lady's sung, picked up her paycheck, and gone home to watch House on her Tivo. Give it up and quit holding up the bus.
It just shows Ryan isn't used to contributing free software to someone else's project. I once had to wait months before I got my code accepted into a free software project and it wasn't the kernel. If the maintainers add every submission to a project, it will end up in an unstable, unmaintainable mess. Code can last a long time and someone will have to maintain the code even after he's lost interest in it. I am especially leery of code that touches a lot of difference places at the same time, as is undoubtedly the case here.
No because a package manager makes it easy to install software for the current arch. Even grandma doesn't benefit from having x86, AMD64 and arm binaries in a single package, much less from some random untrusted binary she downloaded from the internet.
So you're advocating Java?
Welcome to the Panopticon. Used to be a prison, now it's your home.
But the truth is the Linux movement needs every warm body it can to fight Microsoft.
</quote>
THAT is the problem. Stop trying to FIGHT Microsoft. Start making better software. Innovation, something so tremendously better they start copying YOU.
Vis-a-vis AMD and Intel and x86_64 and VT extensions.
Except software has zero marginal cost, so once you take the lead, it'll take a serious fuckup, and not just money to lose it.
Can you remind me again of the advantages of such fat binaries over a tar/deb/rpm file with multiple binaries? Thank you.
One really nice thing is you can install a single fat binary on a shared network drive and clients with different architectures can all run it without having to know what architecture they are on or without a client side script that needs to be installed, or a script that tries to identify the client's architecture. This is really useful in places where you want to offer software with limited licenses to users on site, when you don't know what they will be using.
With multiple binaries in a tar/deb/rpm you end up with multiple binaries and end users randomly trying them in the hopes that one will be the right one for their computer. A lot of users don't know their chip architecture or if it is 32 or 64 bit.
Another advantage comes from applications being run from flash drives, which has similar benefits. Being able to perform automated hardware upgrades is a nice advantage as well. For software in OSS repositories users can just grab them from the repositories when updating. For closed source software, however, being able to pull the applications directly from your old hardware to your new hardware (regardless of architecture) and have it work is really nice. Otherwise you have to find each and every commercial software package, re-download them, and then dig up all your serial numbers and re-register them. It's a huge pain, alleviated only by the lack of commercial software available on Linux these days. Ideally much of this could be mitigated by better package management that caters to commercial developers, but it certainly isn't there today and still does not handle software installed from optical disks.
The state of package management is atrocious, and so should not be looked to for solutions? I'd call that a pretty big one.
MOST packages need only the functionality of a dependency manager, everything else being a nice-to-have-when-you-need-it feature. This is why dependency management can be considered to be the central feature of a package manager- if you don't have dependency management, you'd be hard-pressed to find anyone who claims you have a working package manager.
And what do most package managers do? Utterly lazy dependency management. "Well, you need this package... so you should have the latest version of it. If you want another version, you should rename the package and depend on something else instead."
And that would be almost-excusable, except for the brain-dead "open source is king" approach for updates: "The whole-thing's free anyway, why not just re-send the whole thing?" binary patches are pretty-much unheard of. Of course, sending the whole thing is really just a work-around because-
Package managers generally do NOT bother to detect when they are about to clobber or alter "the wrong file". When they do, they don't bother to keep a record of what they /would/ consider to be "the right file", making "merging" impossible and difference examination a guessing game. That doesn't even matter, because the first step in an "Upgrade" is usually to just completely remove the existing package, which means...
Multiple versions of a single package co-existing on the same base install is generally impossible. Which really makes you wonder what the hell a package manager /does/ manage.
It's not third-party software, that's for sure. You want the bleeding-edge version of something? You just want to patch a broken package? That means you're not using the package manager, and that means you're on your own for everything. Either you build a /package/ for what you're doing on the side, or you don't get access to any of the supposed features. And anything that depends on what you're doing, you may as well just compile and track yourself- 'cause that's what you like doing, right?
The short of it is: Package managers seem so fundamentally broken that giving them another task seems like a waste of time. They'll just be replaced by a better system eventually anyway, right? And then you'll need to do it all again.
The closest to "right" I've seen is GoboLinux.
-- 'The' Lord and Master Bitman On High, Master Of All
You missed half the argument. It's hard, and it's pointless.
The grandma who has a netbook running ARM and a desktop running x86 will install software by going into Add/Remove Programs and picking "Fun Times Photo Album for Grandmas" out of a list. The package manager will figure out what needs to be installed for her, on both her ARM and her x86 computers.
She's not going to go to some random website and download a random installer file and use it on both her computers - her kids have told her over and over again that that's not safe, and she may lose her knitting patterns if she does it.
Seriously, the people who advocate this junk seem to be entirely unaware of the joys of package management. All FatELF does is re-solve a problem that package management has had licked for a couple of years now, and it solves the problem in a less efficient way.
It's hard, yes - but it's not worth doing just because it's hard.
Usability.
Your average desktop user does not want to go, 'Oh, well, I'm running on Processor X, with distribution Y, patch Z. I guess that means I need /this/ tarball (or this subdirectory of the big tarball).' Fat binaries solve this problem.
If I am a Mac OS X developer, fat binaries mean I don't have to make a separate Intel download, or separate PowerPC download. No worries about Joe User downloading the PowerPC version, then complaining about performance (not realizing they're running a PowerPC binary in Rosetta on an Intel machine), or so on. I can just have one download on my website, and the loader handles finding the correct binary.
Similarly, I can bundle 32-bit and 64-bit binaries for a given architecture into the same binary, rather than having separate 32 and 64-bit downloads (as is common on Windows). Tech-literate users may well know whether their system is 32-bit or 64-bit, but if I sat my father down in front of a brand-new Windows 7 machine from Best Buy, I doubt he would know whether to pick a '32-bit' or '64-bit' download for an antivirus program on a given website. He would, instead, call me.
Now, some software solves this problem by having a tiny installer you download, which then goes out and pulls down the correct packages from the Internet after examining your machine. This is one solution, though not entirely ideal (it means in order to do any install, you need to have internet access). Some installers include the entire set of binaries, and just install the correct one; this is fine, as long as you have an installer, but can break down if you try to transplant the hard drive into a new machine. For instance, Joe User picks up that nice Windows 7 Home Premium machine he saw at Best Buy, and plugs his Windows Vista drive in to copy over applications, unaware his old computer was running Vista x64, while his new Windows 7 machine is 32-bit. Joe has some Problems now, when he tries to run some of his old installed software that was 64-bit only.
At any rate, there are plenty of solutions to this problem; fat binaries are just one. None are perfect and all have their tradeoffs; in the case of fat binaries, the main problem is disk space. Package management tools have their own problems. (RPM dependency hell any time you want to go outside of your distribution's available packages, for instance, and the 'screw this, I'm installing PHP from source' result some sysadmins turn to.)
From a server standpoint, fat binaries aren't necessarily the most useful solution (unless you're dealing with clustered machines with variant processors or configurations, but a shared filesystem between them), but from a *desktop user standpoint*, fat binaries may be friendlier than other options.
At any rate, my *personal* opinion is that from a general desktop end-user standpoint (as opposed to a sysadmin/techy standpoint), disk space is cheap but usability is priceless. And my experience is that fat binaries require less work on the part of the end user (though, admittedly, more work on the part of the developer; building Universal Mac OS X binaries of software outside of Xcode can be a hair-pulling experience at times and inspire fond thoughts of Windows installers that just pick the right binary based on a system check).
So whether you feel Linux benefits from fat binaries may well boil down to whether you feel Linux needs to target general, non-techy desktop users more or not. Your own opinions may well differ from mine; not everyone's criteria and priorities are identical, which is probably a good thing. Otherwise we'd have a pretty homogenous software community out there!
--Rachel
Only the last phase of compilation, code generation, would occur on the users computer. One of the problems with source code is that it can take hours for it to compile, and getting it to compile right is never easy enough for granny. The purpose of a universal executable is that it should be easy enough for granny to use, which means download, double click, and it runs. None of this fiddling with a million dependancies and so on. Granted, the problem is parly due to the fact that each Linux distribution does something differently and puts things in a different place.
FatELF is a stupid implementation of a stupid idea. I.e., even if you want fat binaries, modifying the ELF format is the wrong way of doing it.
Yeah for the Linux kernel developers for keeping this kind of crap out of the kernel.
I've really appreciated just being able to pull down a single executable from a site and have it "just work".
Have you ever done that? Even once?
Absolutely yes. And if you're willing to forgive the word "executable" and allow a dmg with a single app in it that I can just drag and drop without picking which one to use or running any scripts, then I've done it quite often.
The universal binary system on osx was pretty sweet during transition. I went from PowerPC to Intel and very rarely had to think about it at all. If you think that's nothing special, fine, but to a lot of users that's a very nice feature.
Cheers.