Slashdot Mirror


Ryan Gordon Wants To Bring Universal Binaries To Linux

wisesifu writes "One of the interesting features of Mac OS X is its 'universal binaries' feature that allows a single binary file to run natively on both PowerPC and Intel x86 platforms. While this comes at a cost of a larger binary file, it's convenient on the end-user and on software vendors for distributing their applications. While Linux has lacked such support for fat binaries, Ryan Gordon has decided this should be changed."

22 of 487 comments (clear)

  1. Gee, just 14 years by Anonymous Coward · · Score: 4, Funny

    after the diminse of NeXTStep!

    (c)Innovation!!(tm)(R)

    1. Re:Gee, just 14 years by TheRaven64 · · Score: 5, Informative

      GNUstep has supported cross-platform app bundles for a long time. You can include Linux binaries for various architectures, FreeBSD, Windows, and even OS X-with-Cocoa binaries in the same .app, then drag it to your platform of choice and have it work. The down side of this approach is that it consumes a bit more disk space because you have a copy of all of the data (not just the code) in every binary. The advantage is that the same bundle will work on platforms that use ELF (Linux, *BSD, Solaris), Mach-O (OS X) and PE (Windows) binaries. Given how cheap disk space is, and how trivial it is to thin a bundle like this (NeXT's ditto tool could do it, but all you really need is to delete the folders for targets other than the one you want from the bundle) it's not really a big disadvantage. Fat binaries on Linux would mean you could run the same binary on Linux/x86 and Linux/ARM, for example, but that's not exactly a massive advantage.

      --
      I am TheRaven on Soylent News
    2. Re:Gee, just 14 years by Hal_Porter · · Score: 5, Informative

      When did VMS take-over Windows? Which iteration? NT5 (2000/XP) or NT6 (Vista/Win7)? Or earlier?

      Dave Cutler, the architect of VMS developed Windows NT. Lots of Windows NT kernel mode terminology - working sets, paged pools, IRQLs, IRPS and so come from VMS and were not present in 16 bit Windows (which didn't really have any architecture).

      http://windowsitpro.com/Windows/Articles/ArticleID/4494/pg/2/2.html

      If you take the next letter after V you get W, M you get N and S you get T, so W(indows)NT is a successor to VMS. The Windows NT kernel run on Dec's preferred Mips architecture (and later the Dec Alpha) before it run on x86. Much of the development of 64 bit Windows was done on Alpha.

      Actually before Cutler worked on Windows NT at Microsoft he worked on a project to run Unix and VMS binaries on a single kernel in separate subsystems. Orignally Windows NT supported Win32, Posix, OS/2 and Win16+Dos subsystems, though Win32 obviously ended up being by far the most important. In fact Windows NT was originally so CPU agnostic that it run Win16 and Dos applications using a software emulator on Risc chips before it run them using V86 mode on x86.

      --
      echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
    3. Re:Gee, just 14 years by TheUser0x58 · · Score: 4, Insightful

      And I'll bet that Apple doesn't use this "universal binary" thing on their iPhone, either.

      You'd lose that bet. Apple provides complete support for universal binary on iPhone, allowing developers to compile for ARMv6 (compatible with every iDevice) and ARMv7 (newer ISA; works on iPhone 3GS + iPod Touch 3G).

      It makes sense for Apple, which only has to worry about 2 architectures on the desktop

      Actually, 4: PowerPC, PowerPC 64, x86, and x86 64. Though for the purposes of your argument its probably an immaterial difference.

      --
      -- listen to interesting music, support independent radio... WPRB
  2. Only useful for non-free applications by dingen · · Score: 5, Insightful

    If you have access to the source, you can always compile a version for your platform. The 'fat binary' principle is only useful for non-free applications, where the end-user can't compile the application himself and has to use the binary provided by the vendor.

    Since most apps for Linux are free and the source is available, this feature isn't as useful as it is on the Mac. Not that it shouldn't be created, but it makes sense to me why it took a while before someone started developing this for Linux.

    --
    Pretty good is actually pretty bad.
    1. Re:Only useful for non-free applications by betterunixthanunix · · Score: 4, Insightful

      Not everyone is skilled enough to compile the source on their own, especially for packages that must be patched to run on certain architectures. Personally, I would think this might be useful for distro maintainers who do not want to maintain separate packages across multiple architectures, although the benefits may not outweigh the costs.

      --
      Palm trees and 8
    2. Re:Only useful for non-free applications by eldavojohn · · Score: 5, Informative
      Well, that's an important point but the author of this defends himself:

      • Distributions no longer need to have separate downloads for various platforms. Given enough disc space, there's no reason you couldn't have one DVD .iso that installs an x86-64, x86, PowerPC, SPARC, and MIPS system, doing the right thing at boot time. You can remove all the confusing text from your website about "which installer is right for me?"
      • You no longer need to have separate /lib, /lib32, and /lib64 trees.
      • Third party packagers no longer have to publish multiple .deb/.rpm/etc for different architectures. Installers like MojoSetup benefit, too.
      • A download that is largely data and not executable code, such as a large video game, doesn't need to use disproportionate amounts of disk space and bandwidth to supply builds for multiple architectures. Just supply one, with a slightly larger binary with the otherwise unchanged hundreds of megabytes of data.
      • You no longer need to use shell scripts and flakey logic to pick the right binary and libraries to load. Just run it, the system chooses the best one to run.
      • The ELF OSABI for your system changes someday? You can still support your legacy users.
      • Ship a single shared library that provides bindings for a scripting language and not have to worry about whether the scripting language itself is built for the same architecture as your bindings.
      • Ship web browser plugins that work out of the box with multiple platforms.
      • Ship kernel drivers for multiple processors in one file.
      • Transition to a new architecture in incremental steps.
      • Support 64-bit and 32-bit compatibility binaries in one file.
      • No more ia32 compatibility libraries! Even if your distro doesn't make a complete set of FatELF binaries available, they can still provide it for the handful of packages you need for 99% of 32-bit apps you want to run on a 64-bit system.
      • Have a CPU that can handle different byte orders? Ship one binary that satisfies all configurations!
      • Ship one file that works across Linux and FreeBSD (without a platform compatibility layer on either of them).
      • One hard drive partition can be booted on different machines with different CPU architectures, for development and experimentation. Same root file system, different kernel and CPU architecture.
      • Prepare your app on a USB stick for sneakernet, know it'll work on whatever Linux box you are likely to plug it into.

      While you may be able to claim none of those points are overly compelling and target a very small part of the population, you have to recognize there's more than just satisfying non-free applications. Furthermore, I think you mean to say that it's "only useful for non-open source applications" as there are tons of free software applications out there that are not open source but are free (like Microsoft's Express editions of Visual Studio).

      --
      My work here is dung.
    3. Re:Only useful for non-free applications by turbidostato · · Score: 4, Insightful

      "Not everyone is skilled enough to compile the source on their own"

      By "end user" we can understand here "distribution maintainer" which already has the skills to compile the source (and that's not but a part in the lot of things that have to be done in order to integrate some software in a distribution).

      "I would think this might be useful for distro maintainers who do not want to maintain separate packages across multiple architectures"

      But they have to: they still must build and integrate for their supported platforms, then rebuild when bugs are found or the software is upgraded, then test... It's just the last step (producing the very binary packages) that changes so instead of multiple packages you'd end up with a single multplatform package. The distributor still need (almost) as much disk space and infrastructures as before, but then each and every user will end up with spending much more space in their hard disks (imagine the fat binary for, say, Debian, supporting eleven platforms).

      And then, please note that this will allow for single binaries for diferent hardware platforms but not for different version compilations (so it won't be useful to obtain binaries for, say, amd64 for Debian, Red Hat and SUSE).

      It seems it will only benefit to those that want to publish their software in an only binary form outside the framework of stablished distributions and that means closed source software. Of course they can look for their bussiness the way they feel better, it's only they don't get my simpathy so I don't give a damn about them.

    4. Re:Only useful for non-free applications by koiransuklaa · · Score: 4, Insightful

      The whole Linux distribution and installation system (such as, with apt-get) is great for setting up a server, but it's very awkward and unnatural for desktop apps. Apple is far ahead in that respect, and I see no reason why Linux shouldn't follow their lead.

      You've got to be kidding? Super-easy installation and automatic security updates for all applications is 'awkward'?

      If I understood you correctly, your suggestion is that desktop software should be hard to find, it should be installed from whatever website I happen to ultimately find and it shouldn't automatically get security updates. Sounds fabulous.

      Don't get me wrong, I agree that package management systems have their flaws (even inherent ones) but you just aren't making a good case against them... You could start with explaining what's unnatural about "Open 'Add applications', check what you want, click Install", and then continue with explaining what's awkward about totally automatic security updates.

    5. Re:Only useful for non-free applications by tyldis · · Score: 4, Insightful

      Please elaborate.

      I too agree that this is pointless for the end user in Linux, at least when it comes to free software. Only closed binary blobs will benefit, which IMHO is not something worth putting effort towards helping. They did their design choices and accepted the reality in doing so.

      As for the end user, she should just use the package manager of her distro and find whatever she needs. Not worrying about neither compiling nor platforms.
      For example, in Debian/Ubuntu you could more easilly package your installer to simply drop a file in /etc/apt/sources.d. Not only will the user be able to use the package manager to install your app like any other, she will also get security updates you publish.

      Let the package system handle these things, they do it well and does not bloat your boat.

    6. Re:Only useful for non-free applications by meringuoid · · Score: 5, Funny
      Free has meant 'no charge' for a lot longer than it has meant 'free (as in liberated) software'.

      Yes, that's right. That's why a 'freeman' was someone you didn't have to pay for his work, whereas a 'slave' was, er...

      --
      Real Daleks don't climb stairs - they level the building.
    7. Re:Only useful for non-free applications by Teckla · · Score: 5, Insightful

      But... "compiling for your platform" is just another way to install software. You could wrap this in a little application (call it "setup"), where you click "Next >" several times, and as a result you have a binary for your platform.

      Wow, the lack of grasp on reality around here really amazes me sometimes. But it looks like it worked for you. The open source fanatic fan boys shot your karma through the roof. Congratulations!

      Compiling non-trivial applications from source can take a long time. That fact alone can make precompiled binaries a big win for most users.

      I did the "compile from source" thing for a long time on FreeBSD before finally realizing the pointlessness of it all. Not only was I completely unnecessarily beating up on my hardware, but spending far too much time waiting for compiles to complete.

      These days, I grab precompiled packages whenever possible, and you know what? It's a hell of a lot better.

  3. Does Linux even need them? by GreatBunzinni · · Score: 5, Insightful

    Some people may claim that Linux may have some shortcomings but certainly the way that distributions handle support for multiple platforms and also the availability of binaries targeted at a certain platform surely isn't one of them. Linux already runs on a long list of platforms and software distributions already handle themselves quite nicely by building platform-specific packages, which also include all sorts of platform-specific binaries the applications will ever need. So, besides the empty "but Apple has them" rational, exactly what drives the need for universal binaries on linux?

    --
    Slashdot, fix your code or at least hire someone who is competent at it to do it for you.
  4. Re:Linking problems by dkf · · Score: 5, Insightful

    Could this technology also help binaries to link against multiple versions of standard libraries (glibc, libstdc++)?

    Probably not. Or not without getting headaches like you get with assemblies on Vista. Keying off the system architecture (32-bit x86 vs. 64-bit ia64) is much simpler than keying off library versions.

    The fix with standard libraries is for the makers of them to stop screwing around and stick with ABI compatibility for a good number of years. OK, this does tend to codify some poor decisions but is enormously more supportive of application programmers. Note that I differentiate from API compat.; rebuilding against a later version of the API can result in a different - later - part of the ABI being used, and it's definitely possible to extend the ABI if structure and offset versioning is done right. But overall, it takes a lot of discipline (i.e., commitment to being a foundational library) from the part of the authors of the standard libs, and some languages make that hard (it's easier in C than in C++, for example).

    --
    "Little does he know, but there is no 'I' in 'Idiot'!"
  5. Re:We need 1-file installs by John+Hasler · · Score: 5, Interesting

    > It's dead simple. We need something like this in Linux.

    "aptitude install " (or the pointy-clicky equivalent) works for me.

    --
    Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
  6. Re:Apple dropped it by PenguSven · · Score: 5, Informative

    Ask PPC owners that want to get the latest version of OS X.

    No, Apple didn't drop support for Universal Binaries. Most apps available for Mac today are universal binaries and work on PPC or Intel macs, and in some cases support PPC 32, PPC 64, Intel 32 and Intel 64. Just because a new OS doesn't support an older CPU architecture doesn't mean the functionality for Universal or "Fat" binaries is not supported.

    --
    What is...?
  7. oh boy, just pack all archs on a .deb by C0vardeAn0nim0 · · Score: 5, Interesting

    you know, just trick the good ol' .DEB package format to include several archs, then let to dpkg decide wich binaries to extract.

    is not that in linux the binaries are one big blob with binaries, libs, images, videos, heplfiles, etc. all ditributed in as a single "file" which is actualy a directory with metadata that the finder hides as being a "program file".

    being able to copy a binary ELF from one box to another doesn't guarantee it'll work, specially if it's GUI apps that may require other support files, so fat binaries in linux would be simply a useless gimmick. either distribute fat .DEBs, or just do the Right Thing(tm): distribute the source.

    --
    What ? Me, worry ?
  8. Not scalable by gdshaw · · Score: 5, Insightful

    To a first approximation, the size of the binary will increase in proportion to the number of architectures supported.

    This is something you might decide to ignore if you are only supporting two architectures. Debian Lenny supports twelve architectures, and I've lost count of how many the Linux kernel itself has been ported to. I really don't think this idea makes sense.

    (Besides, what's wrong with simply shipping two or more binaries in the same package or tarball?)

  9. Re:Apple Universal Binary is kinda of a joke. by TheRaven64 · · Score: 5, Informative

    You are confusing NeXT and Apple's approaches, I think. Apple puts both all of the different architectures in the same file. Your code is compiled twice, but it's only linked once. The PowerPC {32,64} and x86 {32,64} code all goes in different segments in the binary, but data is shared between all of them, so it takes less space than having 2-4 independent binary files. To support this on Linux would not require any changes to the kernel, only to the loader (which is a GNU project, and not actually part of Linux).

    --
    I am TheRaven on Soylent News
  10. Re:convenient for _closed source_ software vendors by turbidostato · · Score: 5, Insightful

    "Actually, having to maintain packages across several architectures can be tricky at times."

    Of course yes. But let's see if the single fat binary reduces complexity.

    "Some packages need to be patched to run correctly on different architectures"

    And they still will need that. Or do you thing that the ability to produce a single binary will magically make those incompatibilities to disapear?

    "the upstream maintainers can accidentally break those patches (e.g. if they are not personally testing on a given architecture)"

    That can happen too with a single binary exactly the same way.

    "It could even be the case that different architectures have different versions of the same packages, because the distro maintainers are busy trying to get everything to work."

    Probably with a reason (like new version needs to be patched to work on this or that platform). How do you think going with a single binary will avoid that problem? It's arguably that in this situation you would end up worse. At least with different binaries you can take the decision of staying with foo 1.1 on arm but promote foo 1.2 on amd64 in the meantime; with a single binary it would mean foo 1.1 for everybody.

    "I am not saying that this "universal binary" solution is the answer, but it might help streamline the build process at the distro level."

    Still you didn't produce any argument about *how* it could help.

  11. Unix (OSF) tried it with ANDF by Alain+Williams · · Score: 4, Interesting
    Architecture Neutral Distribution Format was tried some 20 years ago. The idea was to have a binary that could be installed on any machine. From what I can remember it involved compiling to some intermediate form and when installed compilation to the target machine code was done.

    It never really flew.

    If someone wants to do this then something like Java would be good enough for many types of software. There will always be some things for which a binary tied to the specific target is all that would work; I think that it would be better to adopt something that works for most software rather than trying to achieve 100%.

  12. Re:Use the source, Luke! by Thuktun · · Score: 4, Funny

    Now why would I want to do anything that fucktarded, when I can just use the source? And if I needed cross-platform that badly, I can always ship ONE java app with ONE instance of data. The '90s called, they want their obsolete fat and universal binaries back.

    The elusive (+1 Insightful, -1 Flamebait) post makes a brief appearance, flashing its brightly-colored plumage, before disappearing back into the brush.