Slashdot Mirror
Ryan Gordon Ends FatELF Universal Binary Effort
recoiledsnake writes "A few years after the Con Kolivas fiasco, the FatELF project to implement the 'universal binaries' feature for Linux that allows a single binary file to run on multiple hardware platforms has been grounded. Ryan C. Gordon, who has ported a number of popular games and game servers to Linux, has this to say: 'It looks like the Linux kernel maintainers are frowning on the FatELF patches. Some got the idea and disagreed, some didn't seem to hear what I was saying, and some showed up just to be rude.' The launch of the project was recently discussed here. The FatELF project page and FAQ are still up."
He needs thicker skin if he's going to deal with the LKML crowd. I wouldn't give up just because it's not merged into the official tree.
Cory Doctorow talking about cloud computing makes as much sense as George W Bush talking about electrical engineering.
I like my elves the way I like my tea: thin and exotic.. served while still hot.
and some showed up just to be rude.
I would never have believed that people in the Linux community would show up at an event just to be rude. I've always heard such glowing praise about the Linux community. They're always there to help the new guy, willing to mentor those learning the "So simple a caveman can do it" operating system and break the monopoly of Microsoft once and for all.
His comments can't be correct. Everyone knows what fine, upstanding individuals the Linux community is.~
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
The 32-bit vs. 64-bit split is handled pretty well on Linux (well, Debian drug its heels a bit on multiarch handling in packages, but even they seem to be getting with the programme).
Real multi-arch could be useful, but the number of arches on Linux is just too overwhelming. To get somewhat decent coverage for Linux binaries, they'd have to run on x86, ARM, and PPC. Plus possibly MIPS, SPARC, and Itanium. Most of those in 32-bit and 64-bit flavours. Those elves are going to be very fat indeed.
Finally! A year of moderation! Ready for 2019?
I don't get the point in bringing it up.
Things get rejected from the kernel all the time -- because not all things are good, useful, well coded, or solve a problem that needs solving. It's not new in any way.
This in particular seems like a solution in search of a problem to me. Especially since on a 64 bit distro pretty much everything, with very few exceptions is 64 bit. In fact I don't think 64 bit distributions contain any 32 bit software except for closed source that can't be ported, and compatibility libraries for any applications the user would like to install manually. So to me there doesn't seem to be a point to try to solve a problem that exists less and less as the time passes and proprietary vendors make 64 bit versions of their programs.
This idea is kind of broken for Linux. On MacOS, with 2 architectures, it makes some sense, since the actual executable code is not huge compared to data. On Linux, withe a couple of dozen architectures, executable code *is* going to start to take relevant amounts of space, and the effort involved in preparing them will be nontrivial. If this system were adopted, virtually no binaries would be made to support all available architectures, meaning that anyone not on x86 (32 bit) would need to check what archs a binary supported before downloading it, which is about as difficult as choosing which one to download would've been.
# cat
Damn, my RAM is full of llamas.
In the entire forked-up mess of the unix tree, there was only one thing that anybody & everybody cared about - source compatibilty. C99, POSIX, SuS v3, so many ways you could ensure that your code would compile everywhere, with whatever compiler was popular that week. For a good part of 4 years, I worked on portable.net, which had a support/ directory full of ifdefs and a configure script full of AC_DEFINEs. It worked nearly everywhere too.
Binary compatibility never took off because there is so little stuff that can be shared between binary platforms. Sure, the same file could run on multiple archs, but in reality that is no different from a zip file with six binaries in them. Indeed, it needed someone to build 'em all in one place to actually end up with one of these. Which is actually more effort than actually letting each distro arch-maintainer do a build whenever they please. OS X build tools ship with the right cross-compilers in XCode and they have more of a monoculture in library versions, looking backwards.
Attempting this in a world where even an x86 binary wouldn't work on all x86-linux-pc boxes (static linking, yeah...yeah), is somehow a solution with no real problem attached. Unless you can make the default build-package workflow do this automatically, this simple step means a hell of a lot of work for the guy doing the build.
And that's just the problems with getting a universal binary. Further problems await as you try to run the created binaries ... I like the idea and the fact that the guy is talking with his patches. But colour me uninterested in this particular problem he's trying to solve. If he manages to convince me that it's a real advantage over 4 binaries that I pick & choose to download, hell ... I'll change my opinion so quickly, it'll leave you spinning.
Quidquid latine dictum sit, altum videtur
My objection is that any such hierarchy of data could be stored as files.
Linux needs tools so that a directory can be manipulated as a file more easily. For instance cp/mv/etc should pretty much act like -r/-a is on all the time, and such recursive operations should be provided by libc and the kernel by default. Then programs are free to treat any point in the hierarchy as a "file". A fat binary would just be a bunch of binaries stuck in the same directory, and you would run it by exec of the directory itself. Also need filesystems designed for huge numbers of very small files and to make such manipulations efficient.
We need the tools to be advanced into the next century. Not use the workarounds of the previous ones as currently practiced on Unix and Windows.
Fatelf was never really a great idea in my opinion. Putting two binaries in a file is not a really good way to solve the problem as there are many more variations of CpU type including all of the x86 variation than one or two. it would be a better idea to do something similar to the AS/400, include, an intermediate form in the file, such as a syntax tree, convert it to native at runtime on the users system, and then store the native code inside the file next to the intermediate code. if the binary is moved to a new system, the native code can be regenerated again from the intermediate code. This does not even requite kernel support, the front of the file put shell code to call the code generator installed on the system, and generate the native code, and then run it. This way, things like various x86 extensions can also be supported and so on.
On Linux, withe a couple of dozen architectures,
Kind of, but not really. No more than there are four architectures (PPC, ARM, X86, X86_64) for OS X. There's two architectures for Linux that actually matter, and they're the same two that run Snow Leopard. X86 and X86_64.
I can see why people are going to get up in arms about this. I've been as big a RISC booster as anyone, I think Apple gave up on PPC too soon, and I'm still bitter about Alpha, but that game's over. 32-bit and 64-bit Intel architectures are what matter, and those are the ones that almost all binaries will work for. I'm not running YDL any more, and neither are you. Game's over, instruction sets lost to marketing. The game's over, the fat lady's sung, picked up her paycheck, and gone home to watch House on her Tivo. Give it up and quit holding up the bus.
This in particular seems like a solution in search of a problem to me. Especially since on a 64 bit distro pretty much everything, with very few exceptions is 64 bit. In fact I don't think 64 bit distributions contain any 32 bit software except for closed source that can't be ported, and compatibility libraries for any applications the user would like to install manually. So to me there doesn't seem to be a point to try to solve a problem that exists less and less as the time passes and proprietary vendors make 64 bit versions of their programs.
EXACTLY! We don't want choice, we want it to just work! Damnit, force people to do things the way they ought to do them, don't give them choice, they'll just screw it up.
Especially when that choice makes things EASY!
Quartz Extreme and Core Image. Are there any other real reasons to spend all that money on generic hardware?
It just shows Ryan isn't used to contributing free software to someone else's project. I once had to wait months before I got my code accepted into a free software project and it wasn't the kernel. If the maintainers add every submission to a project, it will end up in an unstable, unmaintainable mess. Code can last a long time and someone will have to maintain the code even after he's lost interest in it. I am especially leery of code that touches a lot of difference places at the same time, as is undoubtedly the case here.
The problem with fixating on something like ELF for fatness is that it's just the tip of the iceberg.
There's much more to the question of whether or not something will run on an arbitrary copy of Linux than the CPU arch.
Why do you think one of the discriminators in a fat binary can't be a distribution identifier, such that there are fat slices for supporting Debian, RedHat, Ubuntu, etc., all from the dame binary file?
Or that they can't have different slices in the fat binary for Gnome vs. KDE, or desktop vs. Android, and so on?
Also, the arguments about disk space are specious; at least in the Mac OS X world, there is a utility called "lipo" which will pull apart a fat bnary into only the pieces you need/want to install. Typically you only install the actually fat binary on server systems, where the software has to be able to run on multiple client machines, and otherwise you run scripts to reclaim disk space (or in the case of an embedded device, you run them over the install image before you install them).
Same goes for an Apple fat binary really...
Obligatory disclaimer: I am the person who maintains the fat binary loader in the Mac OS X kernel.
-- Terry
Commercial Games. That's who.
I want fat binaries for microcontrollers! Give me binaries that can run on PIC16F88, eZ80 and 68HC11!
There's nothing worst than having to replace a 0.50$ chip with another that cost 0.51$!
No because a package manager makes it easy to install software for the current arch. Even grandma doesn't benefit from having x86, AMD64 and arm binaries in a single package, much less from some random untrusted binary she downloaded from the internet.
But the truth is the Linux movement needs every warm body it can to fight Microsoft.
</quote>
THAT is the problem. Stop trying to FIGHT Microsoft. Start making better software. Innovation, something so tremendously better they start copying YOU.
Vis-a-vis AMD and Intel and x86_64 and VT extensions.
Except software has zero marginal cost, so once you take the lead, it'll take a serious fuckup, and not just money to lose it.
You missed half the argument. It's hard, and it's pointless.
The grandma who has a netbook running ARM and a desktop running x86 will install software by going into Add/Remove Programs and picking "Fun Times Photo Album for Grandmas" out of a list. The package manager will figure out what needs to be installed for her, on both her ARM and her x86 computers.
She's not going to go to some random website and download a random installer file and use it on both her computers - her kids have told her over and over again that that's not safe, and she may lose her knitting patterns if she does it.
Seriously, the people who advocate this junk seem to be entirely unaware of the joys of package management. All FatELF does is re-solve a problem that package management has had licked for a couple of years now, and it solves the problem in a less efficient way.
It's hard, yes - but it's not worth doing just because it's hard.
Regarding hard drive and network speed, in today's world of gigabit to the desktop and 10 gigabit backbones and 2 terabyte hard drives I don't know what you're talking about with "10 megabit" and "20 megabyte" cracks. You do realize that the primary expense in a networked workstation environment is administration, not hardware, right? The proper use for local hard drive in a networked workstation environment is for caching, not for software installation. We knew this truth about workstation management twenty years ago, but for some reason it has been forgotten in a world where Microsoft and their deranged horribly expensive and virtually impossible to manage workstation environment seems to be the model for how to do things. How many years of IT experience did you say you had, again? :).
Send mail here if you want to reach me.
Here's the log of the full conversation. We was hardly abused, and got persistent help over several hours which was patient and helpful, ultimately culminating in him realizing what he had done wrong and admitting to it.
Clearly, the myth IRC folk are at fault.
http://pastebin.com/m2cfd19dd
FatELF is a stupid implementation of a stupid idea. I.e., even if you want fat binaries, modifying the ELF format is the wrong way of doing it.
Yeah for the Linux kernel developers for keeping this kind of crap out of the kernel.
So, remind me again: why exactly is it not possible to implement all that in a package manager and we need to have a Really Fat ELF?
Because Linux distributions can not agree on a single GUI technology, let alone a package manager.
-- Terry
I've never used a package manager that forced you to upgrade all dependencies to the latest version to install a package. All of them allow not just required packages but required versions of packages, and only force upgrades of dependencies when you don't have a sufficiently recent version.
Some can do patches. I think RPM can. But unless you're using dialup, they're not really that much of advantage. And you also have the problem of having to provide patches from lots of versions to lots of versions. Or you can provide only patches from the last version to the current one, in which case they're useless for anyone who misses an upgrade.
I don't know any package manager that does this. For example, Pacman, the package manager of Arch (my current distro of choice), installs new versions of files with the suffix '.pacnew' if the old version was modified and doesn't clobber.
This is true on pretty much any OS. Multiple versions of the same package will install to the same paths, and your package manager would have to be pretty fucked up to do that. If you'd like to horribly violate widely adopted filesystem organization standards and patch your software a bunch to make it work properly with your new layout, you can do that, but there's no real gain.
They manage packages. Just because they don't implement every feature you'd like doesn't mean that's not true, though evidently they do implement many features you'd like, but you are too busy raging pointlessly to pay attention to the facts. In fact, the only feature they don't implement that you'd like is a major design decision that would require altering pretty much all software on the system for no benefit to the vast, vast majority of users.
Then you get it from elsewhere if the official repos don't provide it. You can even build your own package, something you certainly ought to be capable of if you're applying your own patches to software. You can even set up your own repos!
So basically what you're saying is, "you're not using the package manager except if you are". Gee, really?
Most of your post you've been toeing a fine line between being just wrong and bei