Ryan Gordon Wants To Bring Universal Binaries To Linux

wisesifu writes "One of the interesting features of Mac OS X is its 'universal binaries' feature that allows a single binary file to run natively on both PowerPC and Intel x86 platforms. While this comes at a cost of a larger binary file, it's convenient on the end-user and on software vendors for distributing their applications. While Linux has lacked such support for fat binaries, Ryan Gordon has decided this should be changed."

Comments

Gee, just 14 years

[Anonymous Coward]

after the diminse of NeXTStep!

(c)Innovation!!(tm)(R)

Re:Gee, just 14 years

[TheRaven64]

GNUstep has supported cross-platform app bundles for a long time. You can include Linux binaries for various architectures, FreeBSD, Windows, and even OS X-with-Cocoa binaries in the same .app, then drag it to your platform of choice and have it work. The down side of this approach is that it consumes a bit more disk space because you have a copy of all of the data (not just the code) in every binary. The advantage is that the same bundle will work on platforms that use ELF (Linux, *BSD, Solaris), Mach-O (OS X) and PE (Windows) binaries. Given how cheap disk space is, and how trivial it is to thin a bundle like this (NeXT's ditto tool could do it, but all you really need is to delete the folders for targets other than the one you want from the bundle) it's not really a big disadvantage. Fat binaries on Linux would mean you could run the same binary on Linux/x86 and Linux/ARM, for example, but that's not exactly a massive advantage.

Re:Gee, just 14 years

[BuR4N]

NextStep isnt dead, it just got a new name when Next told Apple to buy them....

Re:Gee, just 14 years

[buchner.johannes]

Isn't that was http://autopackage.org/ is trying to do to?
I find it a benefit of Linux if there is only one instance of a XML library in my memory, though.

Re:Gee, just 14 years

[Hal_Porter]

Nextstep isn't really gone, it just possessed MacOS and now it walks around in its body, a bit like VMS did to Windows.

Re:Gee, just 14 years

[Amorya]

The 68k/PPC binaries were referred to as Fat Binaries. As a marketing term, Universal was only ever applied to PPC/x86.

When you say that MacOS doesn't use universal binaries any more, that's not strictly true. It doesn't ship with many that use both PPC and x86, because as you say OS 10.6 won't run on PPC Macs. But it still fully supports them and many software vendors are still producing them. These days, the same tech is used to store a 32 bit and a 64 bit version of the same x86 executable.

Re:Gee, just 14 years

[Hal_Porter]

When did VMS take-over Windows? Which iteration? NT5 (2000/XP) or NT6 (Vista/Win7)? Or earlier?

Dave Cutler, the architect of VMS developed Windows NT. Lots of Windows NT kernel mode terminology - working sets, paged pools, IRQLs, IRPS and so come from VMS and were not present in 16 bit Windows (which didn't really have any architecture).

http://windowsitpro.com/Windows/Articles/ArticleID/4494/pg/2/2.html

If you take the next letter after V you get W, M you get N and S you get T, so W(indows)NT is a successor to VMS. The Windows NT kernel run on Dec's preferred Mips architecture (and later the Dec Alpha) before it run on x86. Much of the development of 64 bit Windows was done on Alpha.

Actually before Cutler worked on Windows NT at Microsoft he worked on a project to run Unix and VMS binaries on a single kernel in separate subsystems. Orignally Windows NT supported Win32, Posix, OS/2 and Win16+Dos subsystems, though Win32 obviously ended up being by far the most important. In fact Windows NT was originally so CPU agnostic that it run Win16 and Dos applications using a software emulator on Risc chips before it run them using V86 mode on x86.

Re:Gee, just 14 years

[eugene2k]

No, autopackage is a package format much like rpm, only for portable (between various linux systems) packages

Re:Gee, just 14 years

[Grishnakh]

The problem is that disk space is NOT cheap at all, or plentiful. ARM-based Linux is used on a lot of embedded devices where there's only 16 or 32MB of flash space, total. This "fatELF" idea makes no sense, because adding in x86, x86_64, MIPS, Alpha, and SPARC binaries to your ARM binary will make everything take so much space that much more (expensive) flash memory would be needed.

This just isn't a very good idea. It makes sense for Apple, which only has to worry about 2 architectures on the desktop, and wants to make things easy for consumers, but that's it. It doesn't make sense for Linux. And I'll bet that Apple doesn't use this "universal binary" thing on their iPhone, either.

Re:Gee, just 14 years

[FooBarWidget]

As a former Autopackage developer, no, it isn't what Autopackage is about.

Autopackage attempts to do more than just packaging: it also tries to fight the binary compatibility problem. Probably the most widely known example is this one: compile a binary on distro A, run it on distro B, and get "symbol foobar@GLIBC_2.8 not found" errors. (There are a lot more binary compatibility issues than that though.)

However, making cross-architecture binaries is not one of Autopackage's goals.

Re:Gee, just 14 years

[Rubinstien]

Agreed. One additional advantage not covered in TFA that I remember fondly from OpenStep (which feature GNUStep shares when configured for non-flattened bundles) was the ability to have one network installation of an .app that would execute on any OpenStep machine that mounted the share, regardless of the architecture (m68k, Sparc, PA-RISC, Intel, in our case). This also meant you could move a hard disk between machines of different architectures and still boot from it. For those without these needs, and needing to conserve as much disk space as possible, there was always Lipo ( http://ss64.com/osx/lipo.html ).

--Rubinstien

Re:Gee, just 14 years

[TheUser0x58]

And I'll bet that Apple doesn't use this "universal binary" thing on their iPhone, either.

You'd lose that bet. Apple provides complete support for universal binary on iPhone, allowing developers to compile for ARMv6 (compatible with every iDevice) and ARMv7 (newer ISA; works on iPhone 3GS + iPod Touch 3G).

It makes sense for Apple, which only has to worry about 2 architectures on the desktop

Actually, 4: PowerPC, PowerPC 64, x86, and x86 64. Though for the purposes of your argument its probably an immaterial difference.

Re:Gee, just 14 years

[TheUser0x58]

The down side of this approach is that it consumes a bit more disk space because you have a copy of all of the data (not just the code) in every binary.

I'm not terribly familiar with GNUstep, but, in Mac OS X's implementation of application bundles, this is simply not true. Of course, architecture-dependent compiled executable code must necessarily be duplicated for each supported architecture, but the application data (which almost always is the most significant fraction of an application's size) is shared. The only reason data would have to be duplicated is if for some reason it is compiled into the binary. Though compiling data into the binary is common in many systems, application bundles render this practice unnecessary.

Re:Gee, just 14 years

[Darkness404]

Disk space may be cheap but I wouldn't call bandwidth that cheap. When you aren't using super-fast internet and are using Dial-up or cell phone internet, the difference between a 2 MB file and a 10 MB file is huge. Even if you have enough disk space for that, its still going to be painfully slow to download the larger it is.

Re:Gee, just 14 years

[Macman408]

There are several ways (both GUI and CLI) to strip out any architecture code that you don't want on the Mac, pretty much negating the occasional instance where disk space is not plentiful. For that matter, most of the application size isn't in the code itself, anyway. The biggest application I have, iWeb, weighs in at 518 MB. Of that, only 8.8 MB is the executable; it's pretty evenly split between x86 and PPC code.
Command line tools are a little different, since they don't contain other resources like the Application packages; the lipo tool I'm using to tell me the architecture size in the executable is about 150 KB, again about half x86 and half PPC. /usr/bin is 171 MB for me, so I could probably strip out a good 85 MB without the extra x86 code. (Well, OK, I just tried it; it dropped to 94 MB - so I could save 77 MB. But, more importantly, it's ridiculously easy - the command to get rid of everything but PPC code (eg x86, x86_64, and ppc64) is just "ditto -arch ppc /usr/bin ~/ppc-bin".)

Re:Gee, just 14 years

[MichaelSmith]

Fat binaries on Linux would mean you could run the same binary on Linux/x86 and Linux/ARM, for example, but that's not exactly a massive advantage.

On my intel based laptop I wouldn't mind the space for ARM binaries. But on my ARM based openmoko I want everything to be as light as possible.

Re:Gee, just 14 years

[sl149q]

Nobody is suggesting that we abandon support for dedicated single architecture formats. If you don't want a FAT binary on your embedded device then compile a normal one or strip the FAT one before or during installation.

For pretty much any other system there is more than enough storage for fat binaries. Even Smart phones start at 8GB these days. The cost (to end users) of NOT having FAT binaries is higher than the cost of extra FLASH or hard disk space.

Re:Gee, just 14 years

[MichaelSmith]

When did VMS take-over Windows? Which iteration? NT5 (2000/XP) or NT6 (Vista/Win7)? Or earlier?

All that happened was a few VMS developers went over to Microsoft to work on Windows NT. Thats as far as the relationship went. There is no VMS code in Windows.

Re:Gee, just 14 years

[Grishnakh]

I still don't see the benefit for most things. The one exception is proprietary software for Linux, where it'd probably be easier for the vendor to ship only one version which has all main binary formats included. But for open-source software, like that which you get from your distro's repositories, it just seems like it'll waste a lot of download time. If you're installing 10GB of software on your Linux box, an extra 5GB of unneeded binaries may not take up much HD space, but it adds a lot of download time, which is bad not just for you, but for whoever you're downloading from.

And even for the proprietary vendors, it doesn't seem like it's that hard for them to just ship all versions on a DVD, and have the install program figure out the right one to install. They probably already do it like this anyway.

Re:Gee, just 14 years

[MichaelSmith]

When did VMS take-over Windows? Which iteration? NT5 (2000/XP) or NT6 (Vista/Win7)? Or earlier?

Dave Cutler, the architect of VMS developed Windows NT. Lots of Windows NT kernel mode terminology - working sets, paged pools, IRQLs, IRPS and so come from VMS and were not present in 16 bit Windows (which didn't really have any architecture).

The way you express it, DEC would have a had a case against Microsoft for stealing their technology. Are you aware of any evidence that this happened?

Re:Gee, just 14 years

[synthespian]

You do realize your argument in based entirely on a conclusion? You take ARM and argue against fat binaries as if your argument would apply to all scenarios. Are you even a programmer? It doesn't look like you grok logic...

Re:Gee, just 14 years

[Foolhardy]

Here is a collection of references. See also Readers Write: How Microsoft got Windows NT, Everything2: The similarities between VMS and Windows NT and Windows NT and VMS: The Rest of the Story (use googlebot useragent to view full story).

DEC did sue Microsoft, but they settled for royalties.

Re:Gee, just 14 years

[j1m 5n0w]

I don't think you should dismiss the idea just because there are a few use cases where universal binaries wouldn't work. I've installed Linux on WRT54Gs before and am aware of their limitations, but I also can see how it might be convenient sometimes to distribute universal binaries (i.e. for desktop users). I'm not aware that anyone is advocating to remove standard ELF binary support from the Linux kernel, so I don't see how this would be a problem. If you don't want universal binaries, just don't use them.

Re:Gee, just 14 years

[Creepy]

From an application standpoint this may be meh, but I use them on mac mainly when I build libraries, and I'd expect the same thing on Linux. In the Apple case, lipo can be used to build and strip architectures from universal binaries - in fact, it often HAS to be used to build OSS projects because autotools uses macros and the -arch command is not valid when certain macros are used (because they depend on architecture and make the build ambiguous if I recall correctly). Basically build twice, once for each architecture, then lipo merge, and done. Use the lipo architecture removal command after an install and you can reclaim space. I've actually built with 3 architectures before - ppc, i386, and x86_64. There also is ppc64 (or something like that) but I never managed to get that to cross-compile.

In the case where BOTH architectures exist on the system you could build a library that supports both and anyone interfacing with that library would get the architecture of choice (for example x86 or x86_64) - it's win-win for a library developer - you maintain backwards compatibility with the old architecture and gain forward compatibility with the new architecture. Linux also has been ported to all sorts of architectures, so such a thing may be very important for a chip you may not have even heard of.

Re:Gee, just 14 years

[dryeo]

While you are right about NT sharing a lot in common with VMS, you're wrong about Windows NT first booting up on non-x86. I have a Byte magazine with the news article (probably an MS press release) around here somewhere about OS/2 NT ver 3 booting up to cmd.exe, forget the actual CPU but it wasn't a common one.
The name change probably happened not long after, still NT kept the OS/2 ver 1.x look and feel until ver 4 with the progman and fileman interfaces. Win 3.x also used the same interface and the interface was written by MS.
BTW, the kernel and loader names are also very similar to the OS/2 versions.

Re:Gee, just 14 years

[knorthern knight]

> The way you express it, DEC would have a had a case
> against Microsoft for stealing their technology. Are
> you aware of any evidence that this happened?

As a matter of fact, yes. See http://www.roughlydrafted.com/2009/07/30/readers-write-how-microsoft-got-windows-nt/#more-3661

| "So, Cutler walked down the street to Microsoft and offered them
| Mica which became NT. Later DEC sued MS and, in an out of court
| settlement, got royalties for the filched technology. Part of the
| deal included targeting NT (back) onto the Alpha platform."

Re:Gee, just 14 years

[swillden]

The way you express it, DEC would have a had a case against Microsoft for stealing their technology.

How so? Any non-disclosures DEC might have had with Cutler were long expired, no code was copied, so unless DEC had patents on specific elements of VMS which NT infringed and Microsoft didn't license, DEC could not have had any case.

Re:Gee, just 14 years

[MichaelSmith]

The way you express it, DEC would have a had a case against Microsoft for stealing their technology.

How so? Any non-disclosures DEC might have had with Cutler were long expired, no code was copied, so unless DEC had patents on specific elements of VMS which NT infringed and Microsoft didn't license, DEC could not have had any case.

Reminds me of arguments in the free software world about reimplementation and derived works. If you rewrite product A to make product B, and you have first seen the code in product A then its going to be hard to prove that B is not derived from A. Of course if B is MATLAB and A is VMS you should be okay. But when A and B are both operating systems and have eerily similar architectures...

Re:Gee, just 14 years

[shutdown -p now]

Orignally Windows NT supported Win32, Posix, OS/2 and Win16+Dos subsystems

It still supports 3 out of those 4. Win32 and Win16/DOS are fairly obvious (and yes, you can still run DOS software in Win7).

POSIX is often overlooked, mostly because it isn't really advertised much, but it is still there - and yes, this is actually a separate NT subsystem that works alongside Win32, not on top of it like Cygwin does - so it does have fast fork() and exec(), for example.

Re:Gee, just 14 years

[Lost Race]

Much of the development of 64 bit Windows was done on Alpha.

[citation needed]

Re:Gee, just 14 years

[Hal_Porter]

http://windowsitpro.com/article/articleid/7153/the-death-of-alpha-on-nt.html

The Future

Today, Dave Cutler’s team is using Alpha-based systems to develop 64-bit NT. At WinHEC (April 99), Microsoft was able to boot 64-bit NT on an Alpha-based computer. However, at the current pace of development, Intel might deliver its 64-bit chip (Merced) by the time 64-bit NT is ready. If this happens, the fact that Alpha was first would offer little competitive advantage. Microsoft will position the 64-bit NT Server as a high-end, low-volume solution for those applications that need maximum scalability—e.g., a large SQL Server database that needs gigabytes of RAM for caching. Would the 64-bit version of NT perform significantly better on the Alpha vs. Merced for this type of application? If not, then being first and fastest would NOT overcome Intel’s competitive advantages: compatibility and cost. In the past, Compaq would position VMS, True64, or a Himalaya system for a highly scalable database application. Will Compaq position 64-bit NT against VMS, Tru64, or Himalaya? Not likely. Could a 64-bit Alpha Windows 2000 Professional (Win2K Pro) workstation save Alpha on NT? No way. Therefore, Alpha on NT is dead.

Re:Gee, just 14 years

[IntlHarvester]

According to the book "Showstopper", early NT development work was done on an Intel i860 RISC system, not an x86.

http://books.google.com/books?id=ZWakyeSnG7AC&lpg=PA35&ots=ai8BFX45mz&dq=showstopper%20dave%20cutler%20intel&pg=PA67#v=onepage&q=&f=false

Re:Gee, just 14 years

[cgenman]

The problem is that disk space is NOT cheap at all, or plentiful. ARM-based Linux is used on a lot of embedded devices where there's only 16 or 32MB of flash space, total. This "fatELF" idea makes no sense, because adding in x86, x86_64, MIPS, Alpha, and SPARC binaries to your ARM binary will make everything take so much space that much more (expensive) flash memory would be needed.

I'm guessing that if you're running on an embedded device with 16 or 32MB of flash, you're probably never going to run Firefox, Open Office, Gimp, or any of a number of tools which could benefit from "run anywhere" binaries.

Personally, I'd love to re-write how applications interact with operating systems, and a specification for platform agnostic fat applications would go a long way towards that. Of course, I'd like to gut most of how applications are handled in modern operating systems, but that's a discussion for another time.

Re:Gee, just 14 years

[swillden]

If you rewrite product A to make product B, and you have first seen the code in product A then its going to be hard to prove that B is not derived from A.

You don't have to prove that B is not derived from A. The owner of A has to prove that B is derived from A. Further, the owner of A has to prove that B actually includes code copied from B. Merely taking the ideas and reimplementing them does not violate the copyrights on A.

There are a lot of misunderstandings about this floating around. Most of it derives from the "clean room" reverse engineering dreamed up by Compaq's lawyers when Compaq set out to duplicate IBM's BIOS. Because they needed to *exactly* duplicate the functionality -- not just the general ideas, they needed bug-for-bug compatibility -- there was a substantial risk that chunks of the code might end up being identical, because the requirements were so tightly-defined, and because the BIOS was machine code. No possibility of relying on different comments, code formatting, high-level structure, etc. to prove difference.

So, they had to come up with some way to PROVE that the code wasn't copied, even if it happened to be byte-for-byte identical in places. Their solution was the now-famous "clean room reverse engineering" approach, with every step of the process carefully documented, and with signed affidavits from the reverse-engineering engineers that they never provided any code to the Compaq BIOS and from the implementation engineers that they never saw any code from the IBM BIOS.

It was overkill, but the little startup Compaq was really afraid of the IBM behemoth and wanted to make sure there was absolutely no doubt. It worked. But because it became so famous, lots of people got the idea that it was necessary, that a developer who has seen the original code is "tainted". There's no such concept in copyright law.

Re:Gee, just 14 years

[mcgrew]

Yes, but what's the advantage to the user? I understand that it makes things easier for the developer, but all I see as a user is downsides -- more disk space used, more bandwidth needed to download, more time to load those fat binaries, possibly slower code execution.

What's in it for me?

Re:Gee, just 14 years

[TheRaven64]

Depends on what you're doing. Do you own more than once computer? For example, a handheld running ARM Linux, a laptop running OS X, and a desktop running Windows? I so, being able to just copy the application binaries for programs that you needed between them is useful. If you're a corporate user, then you can just put the fat binaries on an SMB / NFS share and everyone can run them. And, if you've downloaded the Windows version for yourself, you can give copies of it to your friends, even if they are running OS X or Linux when you want to interoperate with them (as long as it's not GPL'd or proprietary, of course).

The disk space is not a huge issue. Large programs tend to be large because they have a lot of non-code resources. Something like ls might be twice or four times the size, but it's tiny anyway. Something like OpenOffice would only be 5-10% bigger, if that.

Re:Gee, just 14 years

[commodore64_love]

Although you provided these citations, I'm not sure they are any more valid than the numerous WWW citations about a guy waking-up in a tub of water and finding a note about his missing kidneys. Every one of those links has a similar theme - "A friend of a friend of a former MS employee told my boss that....." That's urban legend.

I can believe that the same guy developed both VMS and NT. I don't accept that NT == VMS with a new skin.

Re:Gee, just 14 years

[Thuktun]

Too bad they didn't also port DCL to Windows NT. It was supremely verbose, but anything would have been better than having only the MS-DOS-compatible command prompt to choose from for a command shell. That probably would have been a little too much stealing, though.

Re:Gee, just 14 years

[Thuktun]

Otherwise, Windows would by now have a real command prompt and command-line tools.

Re:Gee, just 14 years

[Hal_Porter]

Well I dunno about that. Both DCL and the Dos batch file environment seem to be laughably crippled to me compared to a typical Unix shell enviroment.

Windows has made a few changes batch files though - you can write a which.bat file that searches you path for the first match. Useful if you have several compilers for different platforms and several copies of make and want to make sure the right ones are being run.

http://blogs.msdn.com/oldnewthing/archive/2005/01/20/357225.aspx

@for %%e in (%PATHEXT%) do @for %%i in (%1%%e) do @if NOT "%%~$PATH:i"=="" echo %%~$PATH:i

Of course it is still primitive.

Mind you in Windows 7 you apparently get the PowerShell and it will be backported to XP and Vista.

http://en.wikipedia.org/wiki/Windows_PowerShell

Still batch files have a killer advantage that if you can express what you want to do in one, you know it will run without the user needing to install Perl or Powershell. If you want to solve a problem like which.bat solves on someone else's machine, that's actually quite useful.

Re:Gee, just 14 years

[Thuktun]

Well I dunno about that. Both DCL and the Dos batch file environment seem to be laughably crippled to me compared to a typical Unix shell enviroment.

I don't disagree. I typically use Cygwin on Windows because I can't stand to be without a bash prompt.

Lumping DCL and DOS/CMD together is unfair. Over the years, I've built DCL scripts, Unix shell scripts, and DOS/CMD batch files, and that last is simply painful.

Re:Gee, just 14 years

[Hal_Porter]

> It's too bad that Windows uses UCS-2 (now UTF-16) internally though - I'd much prefer the simpler UTF-8.

Back when it was developed Unicode was a fixed width 16 bit encoding and UTF-8 hadn't been invented - UTF-8 was invented in 1993, the choice of character set in NT was made a few years earlier. One of the reasons for choosing UCS-2 was that all the cruft of variable length encodings could be avoided in the kernel.

Windows does now support UTF-8 as a code page, and the kernel internally supports UTF-16 as you say, but you have to wonder if supporting UTF-8 from the start would have been better.

Actually it has always slightly irritated me was that USB uses UCS-2 (or maybe UTF-16 now) for device names. That's ridiculous because every device I've seen uses US ASCII. UCS-2 or UTF-16 just doubles the size of the strings in the device ROM where space is really at a premium. Sure now it doesn't matter, but back when USB was invented and devices had tiny ROMs that decision had a price.

Re:Gee, just 14 years

[galanom]

It doesn't have to be mandatory for all binaries. Users can be allowed to still produce ISA-specific binaries.
Let Ubuntu adopt these universal binaries, but Gentoo will remain Gentoo.

Re:Gee, just 14 years

[7 digits]

Do you believe that everyone interview for that book didn't exist ? Urban legend too ?

Re:Gee, just 14 years

[OrangeTide]

And you are free to strip out the junk. on your ARM based openmoko you might want binaries for v5, v6 and v7 and maybe OABI and EABI variants.

Linking problems

[kdawgud]

Could this technology also help binaries to link against multiple versions of standard libraries (glibc, libstdc++)?

Re:Linking problems

[Anonymous Coward]

Could this technology also help binaries to link against multiple versions of standard libraries (glibc, libstdc++)?

Not really, it'll just make bigger binaries that'll run on a system you'll never use.

Re:Linking problems

[dkf]

Could this technology also help binaries to link against multiple versions of standard libraries (glibc, libstdc++)?

Probably not. Or not without getting headaches like you get with assemblies on Vista. Keying off the system architecture (32-bit x86 vs. 64-bit ia64) is much simpler than keying off library versions.

The fix with standard libraries is for the makers of them to stop screwing around and stick with ABI compatibility for a good number of years. OK, this does tend to codify some poor decisions but is enormously more supportive of application programmers. Note that I differentiate from API compat.; rebuilding against a later version of the API can result in a different - later - part of the ABI being used, and it's definitely possible to extend the ABI if structure and offset versioning is done right. But overall, it takes a lot of discipline (i.e., commitment to being a foundational library) from the part of the authors of the standard libs, and some languages make that hard (it's easier in C than in C++, for example).

Re:Linking problems

[martin-boundary]

Could this technology also help binaries to link against multiple versions of standard libraries (glibc, libstdc++)?

I think FatELF is too skinny for that. You want SantaELF, which links all those libraries statically in each binary...

Re:Linking problems

[WaywardGeek]

While your reply sounds a bit like flame-bait, I basically have to agree. The format isn't a universal binary that gets translated to each machine architecture when installed. Instead, it's basically an archive of pre-compiled binaries for each platform you support. So, for example, my stupid Qt application has to be compiled separately for Fedora and Ubuntu. This technology would in theory allow me to merge the binaries into a single FatELF binary. Personally, I'd rather just provide separate .deb and .rpm files.

However, the idea of a universal binary is cool. We could do something like the old p-Code, where we compile to a virtual architecture, and then translate it to the machine during installation. I liked the idea when they had it way back in the early 80's (late 70's?), and I was sad to see we didn't have the compute power back then to make it fly. I bet we do now.

Re:Linking problems

[Hal_Porter]

You could use Mono. Of course if you built C for CLR that would not hide differences in libraries as far as I can tell.

Or ANDF. Then again ANDF has never been used commercially and is owned by the OSF.

Re:Linking problems

[maxwell demon]

Why stop there? Just let each application carry with it a complete OS.

Re:Linking problems

[Bootarn]

Sounds a bit like exokernels.

Re:Linking problems

[TheLink]

Done already. You run those apps with vmware player.

Re:Linking problems

[BrokenHalo]

also help binaries to link against multiple versions of standard libraries

I suppose it should be possible to to this by using something like locate or ldconfig -v | grep foo.so to track down the appropriate libraries and create a mess of symlinks to make it work, but I wouldn't want anyone doing that on my systems, thank you very much.

Ryan Gordon appears to present a workable demonstration of how these fat binaries might be implemented without providing much in the way of insight into why they might be useful. After all, the idea is hardly original; there are countless examples of pieces of old Windows software that link their own copies of libraries rather than bothering to play nice with the rest of the system.

Re:Linking problems

[jthill]

I think Apple had the right idea with their PEF versioning, which allowed libraries to declare their own compatibility ranges.

The idea was each built library advertises the oldest version whose api it supports, and its current version. When you link against a library for your own build, the result is by declaration of the library implementor compatible with any version in that range. At runtime, so long as an available implementation's compatibility range overlaps the one you built against, it'll work — again, by declaration of the library implementor. You don't even have to know, which helps you and, more to the point, the local admin.

So the consequent rule was, when building a library reset its oldest-supported-version to the current version only when you remove or change the behavior of an existing api — adding a call or defining new selectors on an existing call was fine — and otherwise leave it alone. Once you understood it, it was simple.

Only useful for non-free applications

[dingen]

If you have access to the source, you can always compile a version for your platform. The 'fat binary' principle is only useful for non-free applications, where the end-user can't compile the application himself and has to use the binary provided by the vendor.

Since most apps for Linux are free and the source is available, this feature isn't as useful as it is on the Mac. Not that it shouldn't be created, but it makes sense to me why it took a while before someone started developing this for Linux.

Re:Only useful for non-free applications

[betterunixthanunix]

Not everyone is skilled enough to compile the source on their own, especially for packages that must be patched to run on certain architectures. Personally, I would think this might be useful for distro maintainers who do not want to maintain separate packages across multiple architectures, although the benefits may not outweigh the costs.

Re:Only useful for non-free applications

[Monkey-Man2000]

While this is true, of course a lot of free software can run on OS X as well. Compiling this is nearly as easy as Linux, but it's still quite useful just to download a universal binary of the full application if it's available. Smaller apps aren't a big problem, but for bigger ones it can become an unnecessary hassle. For example, I just had to compile Inkscape from scratch on Snow Leopard and I spent an afternoon tracking down and compiling all the dependencies because the universal binary doesn't currently run on 10.6. I really would have benefited from the universal binary if I wasn't so bleeding edge.

Re:Only useful for non-free applications

[eldavojohn]

Well, that's an important point but the author of this defends himself:

• Distributions no longer need to have separate downloads for various platforms. Given enough disc space, there's no reason you couldn't have one DVD .iso that installs an x86-64, x86, PowerPC, SPARC, and MIPS system, doing the right thing at boot time. You can remove all the confusing text from your website about "which installer is right for me?"
• You no longer need to have separate /lib, /lib32, and /lib64 trees.
• Third party packagers no longer have to publish multiple .deb/.rpm/etc for different architectures. Installers like MojoSetup benefit, too.
• A download that is largely data and not executable code, such as a large video game, doesn't need to use disproportionate amounts of disk space and bandwidth to supply builds for multiple architectures. Just supply one, with a slightly larger binary with the otherwise unchanged hundreds of megabytes of data.
• You no longer need to use shell scripts and flakey logic to pick the right binary and libraries to load. Just run it, the system chooses the best one to run.
• The ELF OSABI for your system changes someday? You can still support your legacy users.
• Ship a single shared library that provides bindings for a scripting language and not have to worry about whether the scripting language itself is built for the same architecture as your bindings.
• Ship web browser plugins that work out of the box with multiple platforms.
• Ship kernel drivers for multiple processors in one file.
• Transition to a new architecture in incremental steps.
• Support 64-bit and 32-bit compatibility binaries in one file.
• No more ia32 compatibility libraries! Even if your distro doesn't make a complete set of FatELF binaries available, they can still provide it for the handful of packages you need for 99% of 32-bit apps you want to run on a 64-bit system.
• Have a CPU that can handle different byte orders? Ship one binary that satisfies all configurations!
• Ship one file that works across Linux and FreeBSD (without a platform compatibility layer on either of them).
• One hard drive partition can be booted on different machines with different CPU architectures, for development and experimentation. Same root file system, different kernel and CPU architecture.
• Prepare your app on a USB stick for sneakernet, know it'll work on whatever Linux box you are likely to plug it into.

While you may be able to claim none of those points are overly compelling and target a very small part of the population, you have to recognize there's more than just satisfying non-free applications. Furthermore, I think you mean to say that it's "only useful for non-open source applications" as there are tons of free software applications out there that are not open source but are free (like Microsoft's Express editions of Visual Studio).

Re:Only useful for non-free applications

[dkf]

The 'fat binary' principle is only useful for non-free applications, where the end-user can't compile the application himself and has to use the binary provided by the vendor.

On the other hand, it's a poor platform if it is too hostile to third party software (some of which will be sufficiently specialist to be effectively commercial-only, because you're really paying for detailed support). The big benefit is being able to say "this is a Linux program" as opposed to "this is a 32-bit x86 Linux program"; for most end users this is just a much easier statement to handle because they don't (and won't ever want to) understand the technical parts. (There's a smaller benefit to people serving up applications over networked filesystems to an enterprise's heterogenous Linux systems, but that's a less common scenario.)

Marketing. It sucks, but sometimes you need it anyway.

Re:Only useful for non-free applications

[Sique]

But... "compiling for your platform" is just another way to install software. You could wrap this in a little application (call it "setup"), where you click "Next >" several times, and as a result you have a binary for your platform.
For those who know what they are doing, there is always the "expert configuration" button.

Re:Only useful for non-free applications

[should_be_linear]

Actually, this can be good even for Ubuntu. Think of having i7, Atom and Phenom versions of executable alongside classic 586 version. Also, having one repository instead of several can also streamline few things. Downside is more data transferred on software updates, unless some create really smart update which transfers only part of fat binary that is actually used on client.

Re:Only useful for non-free applications

[TheRaven64]

Actually, there's one case where I can see this being useful. I was talking a while ago to some of the OpenBSD developers about the planned Dell laptops that had both ARM and x86 chips. Their idea was to have a /home partition shared between the two and let users boot OpenBSD on either. If you had fat binaries, you could share everything.

The canonical use for fat binaries with NeXT was for applications on a file server. You would install the .app bundle on a central file server and then run it from workstations with different OpenStep implementations, such as NeXT cubes or other workstations running OPENSTEP, Windows NT with OSE, or Solaris machines. These would then run the correct version of the binary when you double clicked on the .app. If you wanted to copy it locally, you would use the ditto tool, which had the option of only copying the parts relevant to your architecture.

I'm not really a fan of how Apple implement multi-arch binaries, putting them in the same file rather than in different directories in the bundle. It saves a bit of disk space from not having to duplicate the data segments, but it removes the ability for you to trivially strip out the irrelevant bits. You can't, for example, download a universal binary and then remove the irrelevant architectures without a lot of effort. With NeXT, you could do this as part of the install process (just copy it to the install location with ditto) without the copy program needing to be able to parse the binary format.

Re:Only useful for non-free applications

[turbidostato]

"Not everyone is skilled enough to compile the source on their own"

By "end user" we can understand here "distribution maintainer" which already has the skills to compile the source (and that's not but a part in the lot of things that have to be done in order to integrate some software in a distribution).

"I would think this might be useful for distro maintainers who do not want to maintain separate packages across multiple architectures"

But they have to: they still must build and integrate for their supported platforms, then rebuild when bugs are found or the software is upgraded, then test... It's just the last step (producing the very binary packages) that changes so instead of multiple packages you'd end up with a single multplatform package. The distributor still need (almost) as much disk space and infrastructures as before, but then each and every user will end up with spending much more space in their hard disks (imagine the fat binary for, say, Debian, supporting eleven platforms).

And then, please note that this will allow for single binaries for diferent hardware platforms but not for different version compilations (so it won't be useful to obtain binaries for, say, amd64 for Debian, Red Hat and SUSE).

It seems it will only benefit to those that want to publish their software in an only binary form outside the framework of stablished distributions and that means closed source software. Of course they can look for their bussiness the way they feel better, it's only they don't get my simpathy so I don't give a damn about them.

Re:Only useful for non-free applications

[recoiledsnake]

Furthermore, I think you mean to say that it's "only useful for non-open source applications" as there are tons of free software applications out there that are not open source but are free (like Microsoft's Express editions of Visual Studio).

He means free as in Stallman-Free, not the free as in cost. That's what I don't like about this 'free' as in 'freedom' thing, it's needless confusing by trying to change the meaning that first comes to mind to people. They could've just gone with libre or something.

Re:Only useful for non-free applications

[Zobeid]

Maybe "most apps for Linux are free and the source is available" partly due to difficulties of distributing and installing binaries?

The whole Linux distribution and installation system (such as, with apt-get) is great for setting up a server, but it's very awkward and unnatural for desktop apps. Apple is far ahead in that respect, and I see no reason why Linux shouldn't follow their lead.

I read an opinion somewhere, and it made sense to me, that Linux treats all software as system software -- as part of the OS installation, effectively. System software and desktop apps ought to be handled differently.

Re:Only useful for non-free applications

[Yvan256]

But then you need a fat binary for your little installation program.

Re:Only useful for non-free applications

[Digana]

They clearly meant free, not gratis. Gratis is such a weak feature of software that I don't think it deserves to share a meanin with free.

Re:Only useful for non-free applications

[Bert64]

There are already smart update systems, there is a single source package which is compiled into multiple binary packages, your smart client only transfers the binaries which are appropriate for the architecture it uses. Because different architecture versions have different filenames, the packages themselves can already sit alongside each other inside a distribution repository... There is nothing currently stopping you creating a multi architecture install dvd. The reason it's not done is because it would be wasteful to download a 4gb dvd instead of a 700mb cd, lots of people have bandwidth caps these days and downloading gigs of alien architecture binaries would be a complete waste of your usage cap.

Re:Only useful for non-free applications

[dingen]

Furthermore, I think you mean to say that it's "only useful for non-open source applications" as there are tons of free software applications out there that are not open source but are free (like Microsoft's Express editions of Visual Studio).

I'm sorry, I should have been more clear. I mean free as in freedom. MS Visual Studio Express isn't free, it just doesn't cost any money to purchase.

Re:Only useful for non-free applications

[Bert64]

You could take that a step further actually...
Boot the core OS on the ARM cpu, and use that for all your day to day tasks, but power up the x86 on demand for heavy computing workloads. Think of the early PPC amiga addon cards, the core system was still 68k based but you could use the PPC chip for certain power hungry apps or games.

Not sure how hard it would be to engineer, at the very least you could boot the x86 system headless, and have a virtual network between the two so you could access it virtually remotely (X11, rdesktop etc), or have the capability to switch which of the two systems the screen/keyboard are connected to.

I would certainly buy such a system, i use a laptop for everything these days and the vast majority of what i do doesn't require much power... Infact, if the ARM side of things had hardware h.264 decoding i doubt i'd use the x86 side more than once or twice a week.

Re:Only useful for non-free applications

[koiransuklaa]

The whole Linux distribution and installation system (such as, with apt-get) is great for setting up a server, but it's very awkward and unnatural for desktop apps. Apple is far ahead in that respect, and I see no reason why Linux shouldn't follow their lead.

You've got to be kidding? Super-easy installation and automatic security updates for all applications is 'awkward'?

If I understood you correctly, your suggestion is that desktop software should be hard to find, it should be installed from whatever website I happen to ultimately find and it shouldn't automatically get security updates. Sounds fabulous.

Don't get me wrong, I agree that package management systems have their flaws (even inherent ones) but you just aren't making a good case against them... You could start with explaining what's unnatural about "Open 'Add applications', check what you want, click Install", and then continue with explaining what's awkward about totally automatic security updates.

Re:Only useful for non-free applications

[MartinSchou]

If you have access to the source, you can always compile a version for your platform.

. Yes ... you can.

Let's use OpenOffice.org as an example, if for no other reason than I was looking into building it optimized specifically for my computer (Windows).

Step 1) Getting the source

The source tarballs linked here contain a snapshot from SVN:
core source package
system source package
binfilter source package
l10n source package
extensions source package
testautomation source package

Okay, I probably don't need testautomation. Might be able to do without extensions. Wtf is l10n ? Okay, I probably don't need that either. Unless it can't build without it - after all, English is technically a localisation.

Screw it - I'll get them all.

2) Compile the damn thing
Build instructions. Gotcha, I'm on Windows, I have Microsoft Visual C++ 2005 Express Compiler (well better). Instructions:

This page is moved into the Building Guide. Please make sure to add new information there and make this page a redirect if it only contains duplicate information.

Well, at least the build instructions are completely updated and the wiki-editors have made sure that all the pages are up to date. I mean, it's not like that particular page has remained unchanged for a full quarter of a year.

Anyway, onwards and upwards. Build guide for Windows. Software requirements:

Cygwin, C/C++ Compiler (VC++ 2008 Express), Windows SDK Server 2008, GDI+ Redistributrable, unicows.dll from Microsoft Layer for Unicode, dbghelp.dll, instsmiw.exe and instmsia.exe, various extra dll files, Apache Ant, Mozilla binary distribution, and yet more dll files.

Now, once you have Cygwin installed, you need to configure that properly. Breaking links to executables, ensuring filemode is unix, installing yet more perl modules and possibly fixing a few issues you might come across.

So, yeah, sure, you could build it yourself, if you have the source. And it comes with build instructions for your platform. And you know what you're doing. Good luck getting Joe Sixpack to compile OpenOffice.org from scratch.

Now, I'm sure some people will argue, that OpenOffice.org isn't free in the proper sense, but it's licensed under LGPL v3.

No, my example wasn't a Linux one. Who cares. The main point is that it's not just that easy to build from source. Especially if you're talking about products, that didn't have that platform as its first intended target.

Besides, what is so horrible about having fat binaries on Linux? Wouldn't it be kinda cool, if you could simply pop your installation dvd of RAGE and support for a Linux installation out of the box, instead of requiring you to buy the game and then download the installer from their website? Or having to hunt down a limited Linux edition somewhere?

Or are you sure a frothing at the mouth fanatic, that the very idea of having ANY kind of proprietary software taint your harddrive makes you go into anaphylactic shock? If you are, you need to go find your epinephrine pen, because the hardware platform you're using is very likely to be closed off for mere mortals by the sheer number of patents covering it.

Some of us would like to use our computers as tools. If

Re:Only useful for non-free applications

[Anonymous Coward]

Not everyone is skilled enough to compile the source on their own, especially for packages that must be patched to run on certain architectures.

That's ok as not everyone needs to compile the source on their own. If a distro supports a certain architecture then the user only needs to install the package and he doesn't even need to know what architecture he is running. This isn't 1996.

Personally, I would think this might be useful for distro maintainers who do not want to maintain separate packages across multiple architectures, although the benefits may not outweigh the costs.

Is that even a problem? I don't remember seeing anyone complain about that problem. I've heard complains about how the packaging process being complicated for newbies to tackle it but not about multiple platform support.

Re:Only useful for non-free applications

[laederkeps]

But then you need a fat binary for your little installation program.

Or just have it use some already cross-platform scripting engine which can run the same script on all supported architectures. Python comes to mind, although the GUI portion would still require some care to be widely available.

Re:Only useful for non-free applications

[Anonymous Coward]

>>> which transfers only part of fat binary that is actually used on client

Brilliant. First people try hard to pack executables for different platforms together in a single file, then (other?) people try harder to separate them back, and everyone looks busy pushing the cart of progress...

Re:Only useful for non-free applications

[Sique]

... which is commonly known as "shell" and normally comes precompiled for your plattform.

Re:Only useful for non-free applications

[MartinSchou]

Well, kick whomever decided to call it "free software" instead of "liberated software" in the nuts.

Free has meant 'no charge' for a lot longer than it has meant 'free (as in liberated) software'.

Re:Only useful for non-free applications

[selven]

Non-open-source? That's a pretty convoluted way to say "closed-source".

Re:Only useful for non-free applications

[tagno25]

Because it is not small, and it is the kernel!

Re:Only useful for non-free applications

[selven]

Thank god English is free of that stupid distinction.

Re:Only useful for non-free applications

[mrmeval]

We already have all of this without the bloat blob that has irrelevant crap in it. Adobe set up a binary server for it's product for various flavors of linux and if they do the code right it works. All I had to do was add them into my repository list.

If they need some means of collecting money or locking it to only my PC that may be difficult but free is easy. Non-free would probably need a physical or internet dongle if they needed that level of paranoia.

Re:Only useful for non-free applications

[tyldis]

Please elaborate.

I too agree that this is pointless for the end user in Linux, at least when it comes to free software. Only closed binary blobs will benefit, which IMHO is not something worth putting effort towards helping. They did their design choices and accepted the reality in doing so.

As for the end user, she should just use the package manager of her distro and find whatever she needs. Not worrying about neither compiling nor platforms.
For example, in Debian/Ubuntu you could more easilly package your installer to simply drop a file in /etc/apt/sources.d. Not only will the user be able to use the package manager to install your app like any other, she will also get security updates you publish.

Let the package system handle these things, they do it well and does not bloat your boat.

Re:Only useful for non-free applications

[meringuoid]

Free has meant 'no charge' for a lot longer than it has meant 'free (as in liberated) software'.

Yes, that's right. That's why a 'freeman' was someone you didn't have to pay for his work, whereas a 'slave' was, er...

Re:Only useful for non-free applications

[slim]

I agree that fat binaries are not appropriate for applications in the distribution's archive. And I agree that the first port of call for any user should be apt-get / up2date / etc.

However there are many kinds of app that might not get into the distro archive, for all kinds or reasons. Maybe it's of really niche interest, maybe it's too new, maybe the distro-maintainer just interested in it. Or maybe it's proprietary. Some people are willing to compromise on freedom.

The last application I had trouble installing on Linux, due to glibc versioning problems, was a profiler for WebMethods Integration Server. Something like that is never going to get into the APT repository.

Re:Only useful for non-free applications

[dingen]

No, my example wasn't a Linux one. Who cares. The main point is that it's not just that easy to build from source.

Well, since TFA is about a fat binary system for Linux, it is kinda relevant to narrow your scope to just Linux. How stuff in Windows or any other operating system work has nothing to do with this new Linux-specific feature.

That said, Windows is probably the worst platform for consumers wanting to compile their own applications. It doesn't provide any tools to do so by itself and if the source you want to compile doesn't include something like a Visual Studio project file, you're in for a very hard time. Linux doesn't suffer this fate at all. Compiling an application is in most cases nothing more complex than typing "./configure && make" and you're good to go.

Besides, what is so horrible about having fat binaries on Linux?

Nothing. I'm not saying it is. In fact, I'm saying it isn't. It just doesn't surprise me that it took a long time before someone started to develop something like this, while other platforms had this feature for quite a few years, because the need for this on Linux isn't on the same level as it was for Mac OS X back when Universal Binaries made their entry.

Re:Only useful for non-free applications

[EsbenMoseHansen]

It seems it will only benefit to those that want to publish their software in an only binary form outside the framework of stablished distributions and that means closed source software.

Really? You cannot possibly imagine that someone making a tiny niche product outside of the purview of the established distributions would want to make binaries available to people?

Well, fuck you and your narrow minded obtuseness.

If I want to build in support for x86 64, i386, Power PC and a range of other platforms to make it easy for new users to get started, why the fuck would that mean PROPRIETARY CLOSED SOURCE SOFTWARE? Or did ease of use suddenly become a closed source model only?

Relax there, he probably just didn't think about it.However, I doubt that the this would help that niche product much, as a small developer is unlikely to have that many kinds of hardware to compile. A much better way to go about that would be to create a build service which can easily make packages for a large range of distributions and architectures.

Re:Only useful for non-free applications

[dingen]

Yeah, having one word with two different meanings is just awesome. Every language should eventually evolve into having only one word meaning everything.

Re:Only useful for non-free applications

[Thad Zurich]

I think Carlin showed that the f-word (no not "free", the other one) is very close to this already.

Re:Only useful for non-free applications

[turbidostato]

"Really? You cannot possibly imagine that someone making a tiny niche product outside of the purview of the established distributions would want to make binaries available to people?"

Yes I can, but I still don't see how the fat binary will help them anyway.

"Well, fuck you and your narrow minded obtuseness."

Oh, Jesus Christ! Now I see how fat binaries would benefit your example niche! Well, not.

"Or did ease of use suddenly become a closed source model only?"

So now looking at your web page, look for the download page, download by hand your product (which hopefully will be statically linked or at least bundled with a helluva lot of libraries), read for whatever the hell is the setup process your single product has and apply it, somehow becomes easier than opening the GUI for "installing programs" select the one I want to, double click and it's done? Or its, oh, the horror, command line counterpart, `aptitude install foo`, and it's done?

I'll tell you what "ease of use" means for the end user on linux: it means "your program is distributed along my distribution of choice so I can install it using the tools I'm already aware and acustomed to". If you produce, say, a database engine you will have a hard day to make it easier for me than `aptitude install postgresql` unless, of course, your install procedure is `aptitude install yourdatabaseengine`. It really can't come much easier than this. The fact your product is a tiny niche one doesn't preclude it from being at a distro repositories as is the case for the tens of thousands already there.

Unless, of course, you have some issue with your program being distributed from central repositories like, well I don't know... it being closed source?

Re:Only useful for non-free applications

[mftb]

Why didn't you just use MacPorts? It'd get all the deps for you.

Re:Only useful for non-free applications

[pmontra]

What's unnatural about clicking Applications, Add/Remove..., select the applications you want to install (they're split in several categories) and get them installed? It's way better than having to browse the web to look for the applications web sites, decide what's the right installer for you given your architecture and the versions of you system libraries, download each setup file manually and manually run them.

Basically apt/yum + the repositories are the universal binary thing this guy is proposing and using repositories your system only has to download and install the files you really need. People wanting to sell the software for money can just setup a repository and make their programs ask for a license code the first time they're run.

Universal binaries for Linux are a solution for a problem that doesn't exist.

Re:Only useful for non-free applications

[Stevecrox]

I have to disagree, this is useful for open source and is probably needed sooner rather than later. With windows I can take an install file from 1997 (Myst Masterpiece and Quicktime 6.5.2 in this example) and install it on my Win 7 machine, I don't have to care if its CentOS, Debian, etc... nor do I have to care if its version 7.4, 8.10, etc...

While the various package managers do provide pretty much everything, forcing key libraries to be backwards compatible and providing a file that will work out all the tech stuff for you would bring this aspect of Linux up to Windows/Mac OSX standard.

I program at work and I don't want to be recompiling binaries at home.

Re:Only useful for non-free applications

[dingen]

Who or what the free is Carlin?

Re:Only useful for non-free applications

[JackDW]

You're expecting the end user to have everything installed that he or she will need to build your application. End users probably don't have GCC and libc6-dev installed, not to mention the dozens of packages that are needed to compile a C++ program or a GUI program. Why should the end user have to install these things? He or she should just be able to install the libraries that are required, not the libraries and the headers and the C compiler and whatever else. And preferably, the library installation should be automated by the package manager.

Re:Only useful for non-free applications

[dvice_null]

Have you ever made a package for Debian or other distribution for an open source application? Did it require ANY extra effort for you to support all the architectures that are supported by the distribution? I only done Debian packages and that didn't require anything from me, even the compilation is done automatically for me. So I don't see any benefit from this for open source projects. The packages for other distributions are btw. done by other individuals from their own initiative and will.

Re:Only useful for non-free applications

[EsbenMoseHansen]

Free has meant 'no charge' for a lot longer than it has meant 'free (as in liberated) software'.

That does lend another meaning to "The land of the free" :P

Re:Only useful for non-free applications

[zach297]

But then you need a fat binary for your little installation program.

The installation program could be written as a cross-platform script interpreted by a native program. Better to maintain one cross-platform program if it means you don't have to for the others.

Re:Only useful for non-free applications

[Teckla]

But... "compiling for your platform" is just another way to install software. You could wrap this in a little application (call it "setup"), where you click "Next >" several times, and as a result you have a binary for your platform.

Wow, the lack of grasp on reality around here really amazes me sometimes. But it looks like it worked for you. The open source fanatic fan boys shot your karma through the roof. Congratulations!

Compiling non-trivial applications from source can take a long time. That fact alone can make precompiled binaries a big win for most users.

I did the "compile from source" thing for a long time on FreeBSD before finally realizing the pointlessness of it all. Not only was I completely unnecessarily beating up on my hardware, but spending far too much time waiting for compiles to complete.

These days, I grab precompiled packages whenever possible, and you know what? It's a hell of a lot better.

Re:Only useful for non-free applications

[selven]

It was... a joke. Thank god English is free of that stupid distinction?

Re:Only useful for non-free applications

[Nelson]

I disagree with this. I've run Linux on Alphas, PowerPCs, Itanium, and MIPS over the years and you're on an island, even with the opensource stuff. It's usually solvable but it can be a lot of work and the bottom line is that while many distributions supported alternative platforms with gusto at one point, all but PowerPC is pretty much dead ended and practically unsupported and PowerPC support isn't great. It would be kind of nice if build tools would just automagically add support for those platforms, it'd give them more relevance.

It's more of an argument that these kinds of binaries aren't needed. What is needed, if you ask me, is an easy way to do bundles so people can install and delete applications as easily as they can on Mac OSX. If that provides a mechanism for universal binaries then so be it. It's the libraries and all that versioning that is the headache.

Re:Only useful for non-free applications

[Anonymous Coward]

I'm a programmer, but I'd rather not have to compile everything myself. Experience has taught me that getting other people's projects to compile is a very frustrating and time consuming experience. The last time I made that mistake was when I wanted to make some trivial modification to the Chromium (aka Chrome, aka Iron) webbrowser; turned out it wasn't quite the same C++ as I had. After several days of fixing an error here, an erroneous cast there, and so forth, I finally gave up. I didn't even get it to compile, let alone run, let alone fix the problem.

Re:Only useful for non-free applications

[turbidostato]

"The last application I had trouble installing on Linux, due to glibc versioning problems, was a profiler for WebMethods Integration Server. Something like that is never going to get into the APT repository."

Why not? If it's free software it certainly can go in the "contrib" repo; if it's closed source but still redistributable it will go in "non-free". Being it "niche" doesn't preclude it from being in the repos: usually if it is license-compatible and there's at least one person willing to take the effort to package it, it will be accepted. Even if the packager is not a Debian developer, it will go with the support of the mentors list.

Of course it won't go in the repos if it's closed non-redistributable software but then we come back to square one since I already said that to be a "maybe" case for the fat binary case.

Re:Only useful for non-free applications

[Daimanta]

"Compiling an application is in most cases nothing more complex than typing "./configure && make" and you're good to go."

I personally know this is a lie. Pretty much every time I compiled software it was a bloody mess with me being required to edit config files and settings flags. Furthermore I hated the slowness of the compiling and the invisibility of the gains(speed).

The moment I met a decent packagemanager(aptitude) was the moment that I understood the advantages of linux software installation. No dependency hell, no nightmarish compiling and simple commands to install software. That's what the average user needs, not technical mumbojumbo.

Every time somebody suggests that compiling is easy, God kills a kitten. Think of all the kittens!

Re:Only useful for non-free applications

[Fred_A]

But... "compiling for your platform" is just another way to install software. You could wrap this in a little application (call it "setup"), where you click "Next >" several times, and as a result you have a binary for your platform.
For those who know what they are doing, there is always the "expert configuration" button.

There's a reason why some people chose not to run Gentoo.
Not to mention that nowadays, it's frequent for basic dev tools not to be installed on desktop machines by default (for non Gentoo users).

So while a proper fat binary supporting SPARC, SPARC64, Alpha, ARM, x86, ia64, m68k, PPC, s390 and MIPS (I'm sure I forgot a bunch) might be a bit overweight, I don't know if switching to Gentoo is the solution.

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

But... "compiling for your platform" is just another way to install software. You could wrap this in a little application (call it "setup"), where you click "Next >" several times, and as a result you have a binary for your platform.

While I agree adding a location for source in a .app bundle and letting package managers automatically compile them as part of the installation is a promising idea, I don't think it removes the need for multiple pre-compiled binaries since users won't necessarily have access to the best compiler for a given application and platform. Also, it is often useful to install from another system or run apps from a network drive or flash drive where you don't want to install it onto the system just run it without being locked to a single platform.

Further, if a user has to click "Next>" several times in order to perform a single action (installation) your UI design has failed. Click once or drag and drop should be the norm for application installation. Why would you want to pester the user uselessly?

Re:Only useful for non-free applications

[Tacvek]

Just the other day I tried to compile Berkley SPICE for under Linux. That was a real pain in the ass, since it apparently not only predates Linux, but it also predates ANSI C and POSIX being wide-spread enough to depend on them. By default it assumed that void* pointers where not available, so it used char* pointers, unless a specific #define was used, in which case some but not all of the erroneous char*'s were converted to void*. It made incorrect assumptions about what header files included a function (relying on header files implicitly including other header files when they are not required to), in some cases, bypassed including header files, and just added extern function declarations. Since this was K&R C the function declarations (prototypes) did not list the arguments, but they still managed to use return types different from those specified in POSIX. And in a few cases, the arguments passed were of the wrong type because apparently they were specified differently in early UNIX.

That is not counting the places where a function returned a local array, rather than a copy of the array. (In fairness, the author did comment this asking if it should have been returning a copy instead.

Some of the function names conflicted with those used in C99, which they obviously could not have predicted, but did mean I needed to compile with "-std=c89 -posix", which took me a little while to realize. Etc.

So despite the code being targeted at a Unix, it took me several hours to compile it for Linux. This just goes to show that porting can be a real pain, and end users should not be required to compile programs themselves. Now, locally compiling programs can be a valid install strategy, as seen in Gentoo, but a ported must have checked on each targeted platform that the code compiles as-is, or provided patches if needed. It also is a pain if the system you are installing on does not have compilation tools for some reason, such as being an embedded system with limited space.

Re:Only useful for non-free applications

[Rocketship Underpant]

Ah yes, that brings back memories of my failed attempt to use Linux for about a year before switching to OS X. As I recall, less than half my attempts to compile applications from source succeeded. In my final desperate days of trying to use Linux, I refused to download anything that didn't have a pre-compiled binary I could un-tar-zip-bleibenflaven and run.

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

They clearly meant free, not gratis.

"Free means both "gratis" and "libre". So if you're going to quibble, you should say he used "free" as meaning "libre" instead of free as meaning "gratis".

Re:Only useful for non-free applications

[icebraining]

Distributions no longer need to have separate downloads for various platforms. Given enough disc space, there's no reason you couldn't have one DVD .iso that installs an x86-64, x86, PowerPC, SPARC, and MIPS system, doing the right thing at boot time. You can remove all the confusing text from your website about "which installer is right for me?"

Yeah. On the other hand, the DVD will only have 1/7 of the apps it has right now, because the other 6/7 will be full with useless binaries.

Re:Only useful for non-free applications

[BitZtream]

Running 10.6 is hardly 'bleeding edge'. It would have been bleeding edge a year or so ago when the first developer seeds of it went out now.

Any app that hasn't patched for 10.6 at this point is a neglected app.

Do yourself a favor, use Sketsa instead. It actually produces standard SVG files. Using Inkscape to produce SVGs is like using Word to make HTML, you end up with a half assed, full of proprietary tags, mess chunk of an SVG that won't render right in just about any standard SVG display system if its more complex than a smiley face.

Yes, Sketsa is commercial, but the price is well worth the difference. Inkscape is crap.

Re:Only useful for non-free applications

[BitZtream]

Contrary to popular and extremely ignorant belief, slaves got paid. However, that 'pay' was generally only in the form of bad food and really crappy housing, with an occasional side does of ass beating towards the end of it.

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

The whole Linux distribution and installation system (such as, with apt-get) is great for setting up a server, but it's very awkward and unnatural for desktop apps. Apple is far ahead in that respect, and I see no reason why Linux shouldn't follow their lead.

You've got to be kidding? Super-easy installation and automatic security updates for all applications is 'awkward'?

Neither Linux on the desktop nor OS X is perfect when it comes to software installation and both should borrow from the other. Right now Ubuntu, probably the front runner for usability in desktop Linux, still has multiple programs used to manage packages and fails to handle installation from Web sites or disks well. It cannot run application off a flash drive easily and reliably, and frankly it sucks for installation of commercial software. A lot of commercial and noncommercial software is simply not in the repositories and I end up running a binary installer by hand or I have to resort to complex CLI copy and paste in order to get what I want running. But they're working on it and the new RC has a new package manager they eventually intend to solve some of these problems. Both OpenStep and multiple binaries would further the goal of having more usable application installation. For example, one could install an application on a flash drive then plug that drive into multiple computers with different architectures and run it without having to install it on the machines themselves (which is often not even an option).

If I understood you correctly, your suggestion is that desktop software should be hard to find, it should be installed from whatever website I happen to ultimately find and it shouldn't automatically get security updates. Sounds fabulous.

Like it or not, regardless of the platform I'm running, when I want new software I usually turn to Google. I read Web pages and reviews and comparisons and look at the developer's Web site. It follows then that an easier workflow is to directly install from a Web site by clicking a link in the Web browser. Ideally, this link should be a link to a software repository that will pull the application into my package manager and this is possible in some package managers but almost never used because there is not a single standard for package management on Linux. A less useful workflow is what I normally use. That is, I go through the process of deciding what software I want to use by looking at Web pages, then I open my package manager and copy and paste the name into the package manager. Then I sometimes find it and install from there with only on wasted step, but often I don't find it so I go back to my browser and install by hand using whatever method is necessary.

In short, there's a lot of room for improvement and multiple binaries are one way to make an improvement for desktop use, but which may well never happen because Linux development is still dominated by the server role.

Re:Only useful for non-free applications

[jedidiah]

Something else occurs to me... if you can't be bothered to create packages for
distinct platforms (in this case CPU architectures) then it seems that you
probably can't be bothered to go through the even greater trouble of actually
testing them.

If you support both x86 and PPC with any level of diligence then some sort of
fat binary format really doesn't buy you that much.

If you are any good, compiling source and building packages is a COMPLETELY AUTOMATED.

This is a very important fact lost on the clueless n00bs that don't understand
anything and see this as yet another opportunity to make lame pronouncements
regarding "why Linux hasn't taken off yet".

Packaging is the ver tail end of the process. It's the trivial sh*t that they
assign to college interns.

Re:Only useful for non-free applications

[BitZtream]

You've got to be kidding? Super-easy installation and automatic security updates for all applications is 'awkward'?

On my mac, I just download the app. Run it. If the app supports auto updating, it just hooks in on first run.

No package manager required. No dependency tracking, it just works. When I want to uninstall it, I just delete it and it cleans itself up on its own, sometimes not completely until next login.

A great example of this is CrossOver for Mac.

A package manager is nice for finding apps however, but trying to say that Apples system is bad in comparison is just silly. When you get a bunch of commercial vendor together, putting them all on the same 'repository' gets to be a bitch, they fight too much. This is why its rare for commercial software unless they can buy their way to the front of the display list.

No one is suggesting it be hard to find, not even Apple, which is why they have their own site with the common Mac software you can buy or download if its free or has a trial.

You can't compare Linux package managers which are practically designed to be anti-commercial to a commercial environment. Its just not the same ballgame.

Re:Only useful for non-free applications

[Bazer]

In my opinion, it's a solution in search of a problem. He's proposing a system where on each and every update every client has to download a binary version for all supported platforms. Let's calculate how that would affect the binary size of my /usr/bin and /usr/lib. For the sake of the argument, let's say that the binary size for all 32-bit architectures is half of the size of their 64-bit version and every distribution ships the same binaries:

• Fedora 11 - x86_64, 900MB - 100%
• Fedora 11 - x86_64, i386, ppc - 2.2 GB - 250%
• Debian - alpha, amd64, arm, armel, hppa, i386, ia64, mips, mipsel, powerpc, sparc, s390 - 7,2 GB - 800%

That's without debugging symbols for each arch. You do the math for other ditributions. Think of the cost of updates in terms of bandwidth for updates.

This problem has been already solved by package managers and those are far from the weak link he makes them out to be. Moving the architecture detection from the installation phase to the run phase will only add to the problem. Instead of relying on my system vendor's package manager, I have to rely on every application vendor to do the right thing.

Re:Only useful for non-free applications

[Anonymous Coward]

> Free has meant 'no charge' for a lot longer than it has meant 'free (as in liberated) software'.

Hardly. 'Free' meaning 'gratis' is actually short for 'free of charge.'

Re:Only useful for non-free applications

[jedidiah]

Bullshit.

Installing something like dvdstyler versus moviegate demonstrates just how bogus this is.

I install the former from the repositories and it grabs everything it needs. I am done in one command.

The latter will deal with NONE of it's dependencies. The app itself is nagware and it requires more nagware be installed to support it. There are also multiple supporting packages some of which overlap. You get to sort these out YOURSELF MANUALLY.

There is nothing "unnatural" about a DEB package or fetching something from your distros repositories. It might be a little different. Then again so is dragging an app icon to some predetermined folder.

OTOH, something like Oracle RDBMS has been doing the whole "shiny happy installer" thing for a long time across multiple versions of Unix. The same was true of Word Perfect and Loki had their variation.

It's not terribly hard to do installer scripts for a platform that has more script languages than most other platforms have apps.

Re:Only useful for non-free applications

[neiras]

I would think this might be useful for distro maintainers who do not want to maintain separate packages across multiple architectures, although the benefits may not outweigh the costs.

If I were a maintainer, I'd rather have builds for different architectures nicely separated from each other. Probably easier to manage ("Support another arch? Clone the build system and patch away!"), users only download binaries for their architecture (saves me bandwidth)...

With central repositories like Debian's, users don't have to care. For one-off package downloads from vendor sites it's pretty trivial to check the browser user agent and offer the correct package (my browser broadcasts "Linux X86_64" to the world).

If there were a later need for 'universal' binaries, I'm pretty sure they could be built in a post-compilation step. Some kind of ELF stub that checks machine architecture, then jumps to the appropriate binary internally. Still can't really see a need for it though.

I'd much rather see debdelta repositories gain widespread use. Over debtorrent! Fap.

Re:Only useful for non-free applications

[jedidiah]

Universal binaries might perhaps be the solution for companies that want to
sell payware for Linux. However, such companies don't need such a package
mechanism either. They can simply use the same sort of installer scripting
that has been in wide use pretty much everwhere forever. They don't need
a "universal" binary because they can interogate a system and determine
what needs to be installed.

The binary doesn't have to be fat. Perhaps the package is, but the binary
would just be the same normal thing as aways perhaps with some logic to
determine what gets installed.

FatPackage rather than FatBinary.

Re:Only useful for non-free applications

[jedidiah]

You will have to be a bit more precise when it comes to the "commercial applications" bit.

What little blob-ware I have run on Ubuntu has not required any shenangians.

OTOH, there is a lot less call for blob-ware on Ubuntu and Linux in general. Always has been.

Re:Only useful for non-free applications

[jedidiah]

...and a great counterexample is MovieGate.

It's right there on the Apple downloads page, and pretty prominent too.

It doesn't do any of this "automatic dependency management". It could sorely use it.

Re:Only useful for non-free applications

[butalearner]

You've got to be kidding? Super-easy installation and automatic security updates for all applications is 'awkward'?

If I understood you correctly, your suggestion is that desktop software should be hard to find, it should be installed from whatever website I happen to ultimately find and it shouldn't automatically get security updates. Sounds fabulous.

"Better is worse if it's different." - Linus Torvalds

Or if you prefer:

"The better is the enemy of the good." - Voltaire

Re:Only useful for non-free applications

[ghetto2ivy]

I see you've never gone looking for dependencies that aren't available on your platform.

I think its the common day equivalent to Sisyphus and the boulder. Each dependency has other dependencies that you need to compile before you can compile the app in 64 bit.

Re:Only useful for non-free applications

[BrokenHalo]

I'll tell you what "ease of use" means for the end user on linux...

As far as I'm concerned, one thing that might represent "ease of use" on Linux might be the release of source code that actually passes through gcc without falling in a heap. I've lost count of the number of times I've gone through the usual ./configure --prefix=/usr/local && mke && make install only to find stupid syntax errors have made it into the release, causing the build to fail.

Re:Only useful for non-free applications

[Burz]

what's awkward about totally automatic security updates

Wrong question. It should be: "What's awkward about managing apps that are lumped into one huge repository along with the OS?"

You see, application developers stay away from systems that don't clearly define what is and isn't included in the OS. Doubly so when the gatekeepers of that OS constantly demand or imply that apps must be surrendered to and repackaged by a clueless committee of distro managers. Compile-time and packaging decisions can be important for app developers; so why aren't those decisions being made by the developers?

Here's another question: Why are the Ubuntu LTS installs I did sitting there with no easy way for their owners to upgrade to current versions of Firefox and OpenOffice??

And a final one: What makes you think that inserting distro managers (OS people) in-between app developers and their end users will promote healthy relationships? Why is the operating system group the primary contact for people using scores of applications in the Linux realm?

Re:Only useful for non-free applications

[coryking]

If I understood you correctly, your suggestion is that desktop software should be hard to find, it should be installed from whatever website I happen to ultimately find and it shouldn't automatically get security update

So innovate dude. Create a stable API that developers can hook into so their applications can update themselves without re-inventing the wheel. Neither OSX nor Windows really provide a good way to do this, which is why every self-important installer puts in their own auto-update mechanism.

The idea of a central repository is one of those ideas that sound logical in theory, but in practice it simply will not scale at all. It might work okay for the iPhone which has a limited use or the current generation of linux distros who have a fairly small set of software available, but it will never work for a mainstream operating system that is expected to run *every* bit of software in existence.

Will Quicken be in your repository? TurboTax? SimCity? World of Warcraft? The software for my old TI-89 calculator? The software that came with my UPS? What about DB2--all the versions with all the varying patches? What about the specialized medical imaging software used by my doctor? What about the software used by the CNC Mill down at the local foundry? What about the POS software running at the pizza shop down the street? What about the software for my Harmony Advanced Remote? Will these be in your repository? If they aren't, how do you propose I install them?

Sorry, the idea of a central repository for every single bit of software a computer might run is, well, insane. Who maintains it? What happens when it goes down by mistake? In fact, it sounds a little cloud-like don't you think? Are you folk supposed to be anti-cloud?

So please tell me where I will be going to install the software for my old Harmony Advanced Remote?

Re:Only useful for non-free applications

[dingen]

I've compiled several tools for Mac OS X, Linux and FreeBSD using nothing but "./configure && make". I'm sure there are some exceptions and of course, sometimes you have to get some dependencies first (but you can compile those using exactly the same procedure)... but editting files or setting flags in order to get stuff to work has not been something I've encountered on a large scale.

I know my experience is not significant for the population as a whole, but I still feel no kittens should be killed when I or someone else says compiling sotware on a Unix-like box is pretty easy.

Re:Only useful for non-free applications

[nurb432]

Even better is to use something like python, it comes platform independent out of the box.

Re:Only useful for non-free applications

[koiransuklaa]

I meant exactly what I said. "Add/Remove Applications" is pretty usable UI that, as far as I can see, has none of the problems you refer to. This is on Debian so I'm fairly sure it's in Ubuntu as well.

My recollection is that this app has been available and well promoted for quite a while already (I imagine several Ubuntu versions) so I really don't understand what you are talking about.

Re:Only useful for non-free applications

[nurb432]

So have a GUI source tree client do it for you, or an 'install anywhere' installer on the actual application take care of your 'ports tree' in the background. Either way you get "click what you want to install, please wait a bit"

Both exist now.. Why not use what we have today instead of creating yet another wheel?

Re:Only useful for non-free applications

[jensend]

"Several tools," yes. Most major applications, no. Try building OO.O from scratch (if you succeed, you should consider submitting your resume to Sun Dresden). And you run into interesting version dependency issues and all kinds of joyous fun pretty quickly; the distribution maintenance stuff is a complicated problem. Further, there's no reason for every end-user machine to have all the development libraries/sources/headers installed which are required for building apps which have lots of dependencies as do most realistic GUI and multimedia programs. And even if the build system is perfect and foolproof and everyone you'd hope to distribute your app to will have all the necessary dependencies, the waste of time having everyone compile a larger project themselves can be a very serious detriment to seeing your project actually adopted in the real world.

For distributing a 10 KLOC command-line tool, having all your Linux end-users compile it themselves or hope it gets in their distribution is a perfectly viable option. For a lot of other things it simply isn't, and you need to distribute binaries. Those would be the targets of this project.

Re:Only useful for non-free applications

[dissy]

"Compiling an application is in most cases nothing more complex than typing "./configure && make" and you're good to go."

I personally know this is a lie. Pretty much every time I compiled software it was a bloody mess with me being required to edit config files and settings flags.

*snip*

The moment I met a decent packagemanager(aptitude) was the moment that I understood the advantages of linux software installation. No dependency hell, no nightmarish compiling and simple commands to install software. That's what the average user needs, not technical mumbojumbo.

Since you are already introduced to aptitude, one feature it makes nice is source compiling.

apt-get install package - gives you the binary install
apt-get source package - will download the source, which is garenteed to work with nothing more than ./configure && make
apt-get -b source package - Same as above step, but -b will build it for you, so even less typing.

Furthermore I hated the slowness of the compiling and the invisibility of the gains(speed).

That point you have very correct however, as the gentoo experiment discovered.
If a binary is available, it is much more preferable to use that.

With the system being talked about, nothing will change on that end, other than the executable binary being larger.

The difference will be you will only have one debian installer cd instead of one for each architecture.
The binaries it installs will run no matter what system you booted it on and install to.

As far as apt goes, this will be one more arch type, probably called 'fat' or something.
This means for the most popular distros, maintaining an x86 and ppc and mips CD along with your fat CD is still easier on the maintainers, and provides a method to get x86 only binaries for your PC still if the slightly higher disk space requirements are unacceptable.

So for some people, it does nothing. For some people, it helps a whole lot. I don't see anyone that this could be a downside for, since if it is not what you are looking for, you can still avoid using it.

Re:Only useful for non-free applications

[GofG]

A package manager is nice for finding apps however,

Huh? How do you use a package manager for finding new apps?

Also, my experience has been that in Mac OSX (admittedly, tiger), applications do not automatically clean up their files in Application\ Support. In any case, it's inefficient to have every program written for mac osx expected to do that on its own; much better to include that code in one binary (aptitude? apt-get? pacman? etc) which is guaranteed to get every trace of the program with purge.

Whenever I want new software to do a particular task, or just software that's better than my current, google is not my first stop. My first stop is #archlinux on freenode, explaining what I want to do with detailed specifications. Someone there without fail has already done all the research about that particular topic for me and can either tell me that no program exists or the name of the perfect program, which I then install via aptitude. If no one on archlinux has delt with that problem, I go out to other irc channels and ask the same question.

I have found that this method works far better than google. Google will land you software developers' websites where flaws and limitations are not easily admitted, even in a free/libre open source software community. Sometimes it will get threads in various gnu/linux forums where someone asked a question that might be the same question as yours (or maybe not), and you can look at the replies they got and make a judgement, which is almost the exact same thing as asking people in irc except it's some linuxquestions.com archive from 2004 and has been locked for years and the people are long gone.

The actual act of downloading and installing software, once you have determined what you want, is MUCH easier on a package manager than via the world wide web. If I get a list of applications that might work for what I want, I can just "sudo apt-get install rhythmbox exaile amarok quodlibet" and then those applications in their latest versions are downloading, with all of the libraries necessary automagically downloading too. When I determine which application I want to use, I can just "sudo apt-get purge rhythmbox exaile amarok" and every trace of those programs is removed from my disk.

Compare this to the mac osx method. You want a new audio player which can play flac (something there is a hilarious lack of on mac, I might add.) You do some googling and find one called "Cog" which is abandonware in alpha stages, but it might work well enough to play that new album you got off of demonoid. You try to download it from the developer's website, but he's over his bandwidth limit or something so it 404s. You google some more to try to find a binary and eventually find one on macupdate; it downloads as a .dmg which gets automatically mounted to its own volume in /Volumes. That dmg opens up in the finder with the binary and usually a softlink to the Applications folder for easy drag-and-drop. You install it, then unmount and delete the dmg file, then run the application. You find out just what a piece of shit Cog is, so you delete it out of your applications folder. However, it still has some files in ~/Library/Application\ Support which didn't get deleted. Also the binary didn't actually get deleted anyway; there's just a .trash file which contains a list of files that the finder isn't suppose to show, and which are queued for deletion next time the trash gets emptied.

Admittedly I blew it a little out of proportion, but there are still a significantly larger number of steps involved than just "sudo apt-get install cog" and "sudo apt-get purge cog". I suppose this is counterbalanced by the difficulty in compiling non-trivial applications like chromium (10 gigabytes of source?!) but you have to remember that on linux, we still have the option of doing it roughly the same way you do it. Download a .deb from a website, double click i

Re:Only useful for non-free applications

[FooBarWidget]

"Only useful for non-free applications"

Nonsense. As an example, take a look at wkhtmltopdf, an open source command line HTML to PDF converter. Extremely useful for automated PDF generation, but not many distributions package it. Compiling Webkit and QT4 and all dependencies is a huge pain so the wkhtmltopdf developers try to make peoples' lives easier by providing Linux binaries. And guess what? Cross-distro binary compatibility problems is a major issue.

Please, please people, get over this notion of binaries being only useful to non-free apps.

Re:Only useful for non-free applications

[Anonymous Coward]

Thats why we have distributors who compiles applications binaries from source. We get competition to markets between distributors and enduser gets better results.

And then we still remember always the open source (and free software) that source code IS the software as well, not just on executable format.

And you can always find someway help to compile the code yourself. If not freely, just give $5 to someone to do that for you. That's it.

Re:Only useful for non-free applications

[jipn4]

The term "free" originally meant "not in bondage" and "without obligations" (also "noble", "beloved"), going back to Old English and Germanic roots. That's the original sense, it's still the primary sense, and the FSF is using it in that sense.

"Free" in the sense of "free of cost" appeared in the 16th century, and that still is only a secondary sense of the word.

Re:Only useful for non-free applications

[jeremyp]

How long does it take to compile Firefox on your computer?

What if you don't have all the right dependencies?

Re:Only useful for non-free applications

[Blakey Rat]

OTOH, there is a lot less call for blob-ware on Ubuntu and Linux in general. Always has been.

And therefore it's perfectly acceptable that Ubuntu handles it like shit. Is that the logic you're going for?

Because it seems to me that you'd be better off making *everything* work well, no matter how much "call for it" there is.

Re:Only useful for non-free applications

[mark-t]

While you're right about it taking a lot of time for compile batches to finish, if you find you are spending a lot of time waiting for that, then you are probably trying to compile at the wrong times.

Re:Only useful for non-free applications

[slim]

Being it "niche" doesn't preclude it from being in the repos: usually if it is license-compatible and there's at least one person willing to take the effort to package it, it will be accepted. Even if the packager is not a Debian developer, it will go with the support of the mentors list.

I think the absence of "one person willing to take the effort to package it" is the problem. Especially if that effort involves seeking support from a mentor's list. Add to that the fact that it's an effort to be repeated for several distros.

Re:Only useful for non-free applications

[IntergalacticWalrus]

If I'm reading you correctly, what you're saying is that because the source is available, it's perfectly acceptable to expect the end user to compile everything? I can already hear you say "no, the end user should use his distro's package manager". But the hard reality is that your distro's package manager doesn't have everything. Eventually you always end up with some program that's not being maintained by your distro. Hence the forced compiling of stuff.

The end user should NEVER have to compile stuff to use it. EVER. (except for experimental code not ready for release) People need to get that silly idea out of their heads if they ever want Linux to become a relevant desktop operating system.

Fat binaries would be just as useful for open source apps as closed ones.

Re:Only useful for non-free applications

[dingen]

If I'm reading you correctly, what you're saying is that because the source is available, it's perfectly acceptable to expect the end user to compile everything?

No, I'm saying that because the source is available for most applications in Linux, there hasn't been a need for a 'fat binary' option, like there was when Mac OS X switched from PPC to Intel. So it makes sense it took some time before this feature was developed, since it's not exactly a priority.

People need to get that silly idea out of their heads if they ever want Linux to become a relevant desktop operating system.

I'm not an advocate of Linux on the desktop, I'm just stating my observations.

Re:Only useful for non-free applications

[knarf]

What is the big problem you see with installing software on Ubuntu? It uses the debian package format. Software which comes in that format can be installed by just clicking on a link to the package. Doing that launches the gdebi-gtk installer which checks whether the package and any dependencies. It also works when the package is somewhere on a local filesystem.

Running software from a flash drive is no problem whatsoever as long as the software is configured to either save all configuration data to that drive or to the user home directory or any other user-writable location.

Installing 'commercial software' is no different from installing any other software. The fact that some commercial software suppliers have chosen to ignore the debian package format and supplies the software in some other form (shell archive, zip or tar file, executable installer...) is a problem with those suppliers, not with Ubuntu. There is no reason whatsover why the deb (or rpm or whatever) package format can not be used for commercial software.

Re:Only useful for non-free applications

[dingen]

Be that as it may, I actually quite like the repository on my Linux box. And I also quite like the App Store on my iPhone. I think it would be very nice if something like this was also available for Windows and Mac OS X.

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

On my mac, I just download the app. Run it. If the app supports auto updating, it just hooks in on first run.

The problem with OS X's lack of package management is there is no central service for auto-updating third party apps, so many don't do auto updating and the others are connecting willy nilly at random times to random places, making security management and application level firewalling to prevent trojans and spyware much more difficult.

When I want to uninstall it, I just delete it and it cleans itself up on its own, sometimes not completely until next login.

Except this only works for applications that don't need to install kernel modules or which are CLI only or most Linux/BSD ports. Additionally, applications don't completely delete themselves on uninstallation. They often leave behind config files, which is useful in some instances, but without a package manager there is no simple "clean uninstall" option.

A package manager is nice for finding apps however, but trying to say that Apples system is bad in comparison is just silly.

The system in OS X is nice in many ways, but still lacking in some ways where Linux has an advantage. Both could be improved by adopting more from one another.

When you get a bunch of commercial vendor together, putting them all on the same 'repository' gets to be a bitch, they fight too much.

There are a lot of incentives for commercial software providers to not make use of repositories on Linux, but that does not mean they are inherently problematic for commercial software. The iPhone app store, for example, is a commercial software repository that not only works for vendors, but gives vendors incentive to develop for the platform.

You can't compare Linux package managers which are practically designed to be anti-commercial to a commercial environment.

But you can compare them to commercial repositories. For example, Ubuntu just introduced a new package manager in the latest RC and part of the roadmap is to include iPhone app store style commercial software, including a central mechanism for selling software. Ubuntu isn't there yet and can use a lot of improvement in that regard... but OS X can use the same. A lot of users would like a central app store application for OS X that plugs into a decentralized network of repositories, basically something that works like the iPhone store for end users, but offers the freedom of the current system for developers along with giving developers a place to advertise, sell, register, and update their applications. This is useful to end users as well for the ability to handle application specific tasks from a central location (like clean uninstall, updates, and security sandboxing).

Re:Only useful for non-free applications

[tepples]

Which means your setup package would have to include fat binaries of Perl, or fat binaries of Python, or fat binaries of whatever tools your shell script calls.

Re:Only useful for non-free applications

[tepples]

16 2.0 Ghz cores and compiling will only be marginally more time-intensive than binaries.

On what kind of battery?

Re:Only useful for non-free applications

[Hurricane78]

You have obviously no graps of software design principles.

Look at how it's done on pretty much all Linux distributions: You choose your architecture when you choose the install medium. From then on, the package manager pulls your packages for the right arch. There is no need to re-compile it for every user, if it's the same. Just offer specially optimized binaries for every arch right on the package repository servers. That's the basic principle: Never do something twice. It's like caching.

The other one is efficiency. And in the case of such a standard Linux package management, having the machine code for different architectures in that binary, despite you exactly knowing your architecture, is not only inefficient for no freakin' reason at all, but downright stupid. Yeah, disk space is cheap. But does that mean you should flood it for no reason at all? No. Of course not. Or else, why not just run a "dd if=/dev/zero of=/var/cache/$(date +%N).trash bs=1G" on every boot? It's cheap, ...no? And it's just as pointless.

What do you think? How likely is it at all, that a user copies a binary to a different architecture? I think zero point zero zero. There is just no freakin point to it. Except "because I can". So you lost all connections to reality, if you think it's a good idea.

Re:Only useful for non-free applications

[tepples]

if you find you are spending a lot of time waiting for [an application's source package to compile], then you are probably trying to compile at the wrong times.

Would "wrong times" include trying several similar programs from my distribution's repository, looking for the one that's the closest fit to my needs?

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

What is the big problem you see with installing software on Ubuntu?

I already covered that. A lot of software is not distributed in that format because a lot of Linux distros don't use .deb. Further, because repositories are not easy for commercial software developers with legal reasons to maintain their own distribution channels and because package managers can't handle sales and registration of commercial software, commercial developers have real incentive to go with prepackaged installers that do handle these, which mean they distribute using binary installers which means you end up having to manage application installs and updates in at least three different ways on your system, a huge usability problem. Current packages also don't allow for installation from another, existing computer while maintaining registration info, especially if it is from a different platform.

Running software from a flash drive is no problem whatsoever as long as the software is configured to either save all configuration data to that drive or to the user home directory or any other user-writable location.

The problem is, software is not so configured by default so end users try to use it and it fails. For more advanced users, you still have to install multiple copies of the software because you need a separate instance for each platform, one of the items the proposed changes would solve.

Installing 'commercial software' is no different from installing any other software.

Commercial, especially closed source app developers have different than OSS projects. That has been largely ignored in the creation of package managers and the result is predictable and bad for users.

The fact that some commercial software suppliers have chosen to ignore the debian package format and supplies the software in some other form (shell archive, zip or tar file, executable installer...) is a problem with those suppliers, not with Ubuntu.

This is why Apple crushes Linux desktop system providers. I, as a user, don't give a rat's ass if it is Canonical's fault or Adobe's. I just want someone to step up and make sure the end user experience is decent. Blaming the other guy doesn't help me. Developers will do what they think is best for them and if Canonical wants an OS that is useable and works they need to provide motivation to developers. They actually are stepping up in this regard and building a commercial software store into the new package manager to try to solve this problem, but it remains to be seen if they will do it well.

There is no reason whatsover why the deb (or rpm or whatever) package format can not be used for commercial software.

Sure there is, it is inconvenient and doesn't make developers the most amount of money. They have to run a separate server just to offer updates and another to do license validation. It's a lot cheaper and easier just to use one of the off the shelf binary installer packages and the service offerings that go with it. Current package managers combined with the .deb format, don't meet their needs.

Re:Only useful for non-free applications

[walshy007]

depends what you are doing really, for mainstream things the only thing I've found that does that are non-mainline kernel modules that haven't been updated because of interface changes. GCC is also a moving target.. the changes are always ever so slight, but an ever so slight change is all that is needed for some trivial break.

Re:Only useful for non-free applications

[oakgrove]

If you are targeting a particular platform, say Ubuntu x86/AMD64/ARM, I don't see how any of this can't be seamlessly automated. In much the same way that many programs you install in Windows automatically download and install .net or java or the VisualC redistributable. Just because it is installing compilers, etc. doesn't make it that big of a deal. Hell, in Debian, and it's derivatives, it could be a simple "apt-get install build-essential" somewhere in the fat-binary installation script to get most or all of everything the user would need.

Re:Only useful for non-free applications

[koiransuklaa]

That Apple downloads site is nice, thanks for the pointer. Very useful for a "mom" install in the same way that a simple package manager is.

You can't compare Linux package managers which are practically designed to be anti-commercial to a commercial environment. Its just not the same ballgame.

Oh, I can. I didn't do it in the previous post (I was just pointing out how stupid the GP comment was) but we certainly can and should compare them. IMO both approaches have room for user experience improvement and both have some inherent problems that must be 'worked around' (you touched on some of these already). The good and bad features of both approaches are closely related to other features of different OSes, like source licenses of course -- in other words Microsoft can't just start a comprehensive Windows software repository -- but that doesn't mean we can't compare the systems as wholes.

Re:Only useful for non-free applications

[tepples]

Of course [developers of non-redistributable software] can look for their bussiness the way they feel better, it's only they don't get my simpathy so I don't give a damn about them.

So you have sympathy only for those developers of redistributable software. How do you propose to recoup the cost of developing, say, a single-player or local multiplayer video game if it is redistributable?

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

Huh? How do you use a package manager for finding new apps?

You browse through or search the descriptions of the packages in the package manager.

Also, my experience has been that in Mac OSX (admittedly, tiger), applications do not automatically clean up their files in Application\ Support. In any case, it's inefficient to have every program written for mac osx expected to do that on its own;

Amen.

The actual act of downloading and installing software, once you have determined what you want, is MUCH easier on a package manager than via the world wide web.

Ideally a link of the Web should start your package manager downloading and installing a program. Having to go from a Web browser or IRC client, then enter the same package into a package manager to find it is an extra step that is not ideal.

Compare this to the mac osx method. You want a new audio player which can play flac (something there is a hilarious lack of on mac, I might add.) You do some googling and find one called "Cog" which is abandonware in alpha stages, but it might work well enough to play that new album you got off of demonoid.

That's funny. When I wanted to play a FLAC file I just did a Google search, found one of several Quicktime Components that could handle it, and dumped it into my Library and suddenly several of my music playing applications handled them just fine, including iTunes.

Admittedly I blew it a little out of proportion, but there are still a significantly larger number of steps involved than just "sudo apt-get install cog" and "sudo apt-get purge cog".

Yeah that's great and all, but it ignores all the software that apt can't handle, which ships as a binary installer for Linux or where you don't know the name of the package or what what repository it is in or which is not in any repository. Don't get me wrong; I think package managers are the way to go, but only when you can get them working well enough so that they can handle all your software and you're not juggling multiple install/uninstall methods from multiple interfaces. I installed some software under Ubuntu the other day and documented it is part of a usability study. 18 command line entries were required to install and run an OSS program, some dependencies of which were in repositories and some of which were not. In no one location did I find all the steps required for installing it and uninstalling it was performed by rolling back the VM because I had little hope of successfully doing so in any other way without crippling things it plugged into.

The fact that you alluded to package managers being a great way to "find" applications makes me think that you don't know what you're talking about. Perhaps you're used to the Add/Remove Programs thing on ubuntu; I would never use that as a way of seriously determining what application I want.

Ahh, but that and Google are the two places a normal user is likely to go. I'd note Ubuntu is replacing the package managers with a single unified one that is eventually intended to handle buying and installing commercial software as well.

Re:Only useful for non-free applications

[turbidostato]

"I think the absence of "one person willing to take the effort to package it" is the problem."

Of course it is. But then we are not talking about "easiness for the end user" anymore. We are talking about producer's percieved convenience.

"Especially if that effort involves seeking support from a mentor's list."

True. But then you still have the "second best" from the end user stand-point: produce you own repo and let the user get the software from it. It is not such a terribly difficult task and the only added nuisance for the user is adding a line to her sources list (and even then, it can be streamlined by providing a file to be dropped at /etc/apt/sources.d/).

" Add to that the fact that it's an effort to be repeated for several distros."

No, I don't want to add it since it's irrelevant; such a support is not affected by you using "thin" or "fat" binaries: you either support a given platform or you don't and that's the kind of choice the developer won't only have to take for Linux' distributions but for Windows and its multiple release, OS/X, etc. And still I'll point out that it's much easier to support "Debian and Red Hat" than it is to support "Windows and OS/X".

Re:Only useful for non-free applications

[turbidostato]

"Package managers are a necessary evil."

Your opinion. I've extensively managed software installations on Linux and Windows and even if package managers are a necessary evil, they are much better than the alternatives. By far.

"If I want software [...] I don't have to hope that the company I bought the operating system has put it in their database."

You forget that one thing is a package manager and a very different one a package source. Any company that cares can provide its own tigthly integrated package source for a distribution without permission or cooperation from the operating source vendor (yes: even closed source vendors can do that). And by using the package manager the end user gets for free centralized software inventory and upgrades without the need to go after each and every vendor's procedures as they reinvent the wheel.

"Package managers are only necessary because of the fragmented nature of the Linux universe."

Oh! that certainly explains why on the Windows side they have reinvented them (control panel's install/unistall app, install shield, windows update, msi files...) only worse.

Re:Only useful for non-free applications

[oakgrove]

fails to handle installation from Web sites or disks well.

That couldn't be further from the truth. When I go to a website, say frostwire.com, and click on the Ubuntu download button for an application, the app downloads and the installer automatically opens. I click "Install" and if there are any necessary dependencies, the installer (it's called gdebi-gtk for the curious) automatically fetches them for me and installs them too. It actually couldn't be easier. Even OSX requires an extra step or two beyond that. I perssonally find the installation process of $RANDOM_UBUNTU_PACKAGE from $RANDOM_WEBSITE easier than on OSX or Windows. In fact, it's scary easy as this could be a good vector for malware for a neophyte user.

It cannot run application off a flash drive easily and reliably, and frankly it sucks for installation of commercial software.

Just what exactly are you trying to run off of your flash drive? Obviously software in Windows that is designed to run without having to actually be installed is designed that way. If the author of a program for Ubuntu wants their program to run without being installed, all they have to do is, again, design it that way. You know what libraries and software come with Ubuntu out of the box. Design your flash program with that in mind and it'll work. You don't see this because the demand just isn't there. And as for commercial programs, what's hard to install? VMWare installs great, World of Goo installed great, Crossover installed great, Google Earth, etc. etc. I have not had a problem yet installing commercial programs.

Like it or not, regardless of the platform I'm running, when I want new software I usually turn to Google. I read Web pages and reviews and comparisons and look at the developer's Web site. It follows then that an easier workflow is to directly install from a Web site by clicking a link in the Web browser.

That's fine and that's your choice. However, like you said, this functionality is integrated into Ubuntu so there isn't really an argument there. Admittedly, package management could be friendlier. Canonical is taking this task very seriously, the first phase being the Software Store in version 9.10. Package management should be iPhone easy. I imagine it will be.

Re:Only useful for non-free applications

[turbidostato]

"So you have sympathy only for those developers of redistributable software."

Yes, that's true.

"How do you propose to recoup the cost of developing, say, a single-player or local multiplayer video game if it is redistributable?"

I do not propose anything. Why should I?

Re:Only useful for non-free applications

[tepples]

The 'fat binary' principle is only useful for non-free applications

And guess what the majority of (for example) games are.

Re:Only useful for non-free applications

[Jeremi]

Compiling non-trivial applications from source can take a long time. That fact alone can make precompiled binaries a big win for most users.

I agree that compiling from source (even in a semi-automated way) is too complex, error-prone, and time consuming to be convenient. I also agree that precompiled single-format binaries aren't as flexible as one might like.

Apple's "fat binary" approach to solving this problem is okay, but I think Linux could probably do better. How about something more clever/elegant, like distributing the binaries in a universal byte-code format (e.g. using LLVM), and then, either as part of the install process or during their first execution, they compile themselves into the appropriate native executable code for the machine they are running on?

If that was done (well), it would give you the more flexibility than fat binaries (since it would work for all platforms, not just those the developer thought worth including binaries for), and it wouldn't use any more disk space (or network bandwidth) than the current single-flavor Linux system.

Re:Only useful for non-free applications

[S.O.B.]

Mod yourself as flamebait!

You compile one puny kernel module and you extrapolate that to all applications. Try compiling something like OpenOffice or Evolution and get back to us.

...and it requires everyone to have a compiler on their system that may or may not be at the correct release.

...and it requires a powerful system unless you have days to wait for the compile. Good luck on a netbook.

Before you arbitrarily shoot down the idea maybe you could stop and think that not everyone has the same hardware, patience or needs that you do.

Re:Only useful for non-free applications

[eggnoglatte]

But "free" only has that meaning when applied to people. For inanimate objects "free" always means "no charge". Even for animals "free" tends to mean "free of charge". For example, most people would associate the term "free dog" with somebody giving away puppies from a litter, not with a dog that doesn't have an owner (there is a separate adjective for that: "wild").

I am sure there will be a number of people here on /. that will now say "its just semantics", but the fact is that terminology matters when you try to convince people to adopt a goal. So yes, the term "free software" is unfortunate, since the general public does not associate it with the same thing as its advocates. This is hurting the "free" software movement. It is also hurting the open source movement. Everybody else doesn't really give a shit.

Re:Only useful for non-free applications

[koiransuklaa]

Your whole post is based on a straw man. I am not advocating a single repository for the whole world, or even for a single user.

If Logitech wants to support e.g. Debian, then their best option would be open sourcing the software and letting Debian take care of packaging. If that's not possible they should tell their clients to add logitech.com as a software source -- in this option they'll need to do testing, building and packaging themselves.

This is not some crazy-ass idea, this is how responsible closed source companies that want to support Debian operate already. There is no need for a "stable API" for this...

Re:Only useful for non-free applications

[coryking]

If that's not possible they should tell their clients to add logitech.com as a software source

That doesn't solve anything but distributing the program itself, not any dependencies. Who distributes those? If you have 50 repositories on your list and all of them claim to offer "libapr-1.2", how do you know which repository to use? Keep in mind some of those repositories might have compiled the library different yet retained the same package name.

Plus you said "Debian". Does Logitech have to run a repository for every distro's proprietary repository system?

Yeah yeah, they should open source it. But guess what. Not happening for 95% of the software in the world. Thats reality and no amount of coercion will change it. If you dont address the concerns of that 95%, you are never gonna get linux in the mainstream.

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

fails to handle installation from Web sites or disks well.

That couldn't be further from the truth. When I go to a website, say frostwire.com, and click on the Ubuntu download button for an application, the app downloads and the installer automatically opens.

The problem being, that site is a rare exception. Very few sites tailor their downloads for each particular Linux distro and package manager and even those that do often have a confusing array of different download options that rely upon the user having a lot more information about their OS and hardware than is common among normal users. A lot more common situation is to be directed to execute a series of CLI commands or to be given a binary blob installer that must be downloaded and run. This is no worse than Windows, mind you, except that Windows users are accustomed to it as the only method, whereas under Ubuntu there are several different methods users must learn.

Just what exactly are you trying to run off of your flash drive?

All sorts of things, from specialized browsers to front ends for a remote ticket management service to a preferred word processor.

Obviously software in Windows that is designed to run without having to actually be installed is designed that way. If the author of a program for Ubuntu wants their program to run without being installed, all they have to do is, again, design it that way.

Right, so a developer for Ubuntu just needs to write a specific version of their application for removable media and network drives, or rely upon very expert users to configure their software to run that way, and install a different version for each platform and hope the user knows what the specific platform of every machine they use it on is and they have enough free space to install those multiple copies. You don't think that is more than a little cumbersome compared to simply being able to put any application on it and run it without having to know anything but the OS you're using?

You don't see this because the demand just isn't there.

Sure there is, mobilefirefox, for example. You just don't see it much because it's hard to do and because there's a significant barrier when you need a specialized version of the program just for a particular use.

And as for commercial programs, what's hard to install?

A lot of things are hard to install and/or easy to install but then bypass the repository and package manager entirely leading to user confusion, security messiness, and multiple services performing the same task but for different apps.

That's fine and that's your choice. However, like you said, this functionality is integrated into Ubuntu so there isn't really an argument there.

Except of course it's integrated into Ubuntu in a way that is incompatible with most other Linux distros let alone OS's, so it is simply not used. Further, it's build into Ubuntu in a way that currently is not satisfactory for most commercial software offerings and so is not used for them.

Canonical is taking this task very seriously, the first phase being the Software Store in version 9.10. Package management should be iPhone easy. I imagine it will be.

I hope so. My point was mostly that right now, it does not work very well for real world use cases. Further, because the subset of users of just Ubuntu Linux is so small, a lot more needs to be done to motivate developers to use it, including getting buy in for a single system that could work on all Linux distros and potentially OS X or Windows. Finally, developers and users need to pay attention and lend support when someone comes up with real improvements, like support for multiple platforms from the get go to improve all the use cases where currently Linux is failing in the real world.

Re:Only useful for non-free applications

[Teckla]

Apple's "fat binary" approach to solving this problem is okay, but I think Linux could probably do better. How about something more clever/elegant, like distributing the binaries in a universal byte-code format (e.g. using LLVM), and then, either as part of the install process or during their first execution, they compile themselves into the appropriate native executable code for the machine they are running on?

I think that's a great idea. That's what IBM's AS/400 does, and it works great. IBM was able to change the AS/400 from one CPU architecture to another using this approach.

Re:Only useful for non-free applications

[BikeHelmet]

This is totally correct. Installing stuff on my XP computer off the HDD takes an average of 4-12 seconds. Even most games don't take very long - once downloaded, 20-30 seconds to complete installation?

The first few compiles I did for linux software were fun, but I can't argue against the time issue. :P

@Requiem18th: Parent isn't flamebait.

Re:Only useful for non-free applications

[ucblockhead]

Strange, I have never had a setup application say:

"error: foobaz10.41.1.a not found"

Re:Only useful for non-free applications

[ucblockhead]

You can't tell me you've never run into the situation where installing a single open source package ends up taking you down a three hour maze of dependencies. Sure, sometimes you get lucky and everything just works, but many other types, you discover that application A needs libraries B and C installed, and B needs libraries D, F and G, and then the version of F you downloaded isn't compatible with package Y, so you try to upgrade Y only to discover that it doesn't work with package Z, until you say "fuck it" and just go try to find a binary.

Re:Only useful for non-free applications

[oakgrove]

Firstly, I think we're on the same team here for the most part in this particular context so look at this as more a friendly discussion rather than the typical point scoring flamefest. With that having been said...

That couldn't be further from the truth. When I go to a website, say frostwire.com, and click on the Ubuntu download button for an application, the app downloads and the installer automatically opens.

The problem being, that site is a rare exception. Very few sites tailor their downloads for each particular Linux distro and package manager and even those that do often have a confusing array of different download options that rely upon the user having a lot more information about their OS and hardware than is common among normal users. A lot more common situation is to be directed to execute a series of CLI commands or to be given a binary blob installer that must be downloaded and run. This is no worse than Windows, mind you, except that Windows users are accustomed to it as the only method, whereas under Ubuntu there are several different methods users must learn.

See, people are confused. "Linux" is not an operating system just like NT is not an operating system. Ubuntu is an OS, WinXP is an os, RedHat is an OS, Win 98 is an OS, just to give a few examples. So, problem one, is publishers publishing, and you (the hypothetical neophyte consumer), going and looking for the "Linux" version of the app when the operating system you are using is not "Linux" but rather Ubuntu. No website that publishes software targeted at OS's using the Linux kernel should have a link for "Linux" unless you are specifically targeting power users. It doesn't even make sense to do so. Of course, that is a perception and perspective problem. Hopefully as Ubuntu gains mindshare, that will change. If a publisher doesn't make an installer for the Ubuntu OS, then that is the publisher's fault. Again, hopefully, as Ubuntu gains mindshare, this will also take care of itself. The only thing that is really confusing about clicking on the Ubuntu link for an installer on a website is are you running 9.04? 8.10? Karmic? 64 bit? That is a problem that again could be easily solved by software publishers. An install script could easily distinguish between each of these versions and set up the software correctly so that all you would have to click on the download page would be "Ubuntu". As a side note, a .deb file and a .rpm file aren't really all that different. You could publish a file containing the binary and the config files necessary for .rpm, .deb, so on and so forth and some OS detection logic and call it a "Linux" binary if you just had to do that. And that link isn't any harder to click than the ones that say "Windows 2000/XP" and "Windows Vista/7" right underneath it. I could see the publisher not wanting to waste bandwidth by shipping 64 and 32 bit binaries in every package but that's a choice for them to make. And the vast majority of Free software that is distributed via a website link doesn't have the source automatically included so I'm not sure what your contention is with the binary blob thing. If a publisher really wants their software to be used by users of a particular OS, there are ways to make the install process from start to finish completely easy and intuitive. Hopefully, this will happen sooner rather than later. Frostwire is an excellent example of what can be done. If every publisher did as good a job with their website as they do, we wouldn't even be talking about this.

Right, so a developer for Ubuntu just needs to write a specific version of their application for removable media and network drives

Yep, you got it. Typically referred to as static binaries. I have several on my harddrive now that I just click and they run. No fuss, no muss. So, yes, this can be done just like it is in Windows. As a matter of fac

Re:Only useful for non-free applications

[synthespian]

So...I guess you must be one of those super-genius Linux whiz kids who, when porting Unix software:

1) never experienced incompatibilities among different library versions between Linux and other Unixen or between Linux distros themselves

2) never faced any problems with makefiles in your entire life

3) never experienced problems due to different compiler versions or the infamous GCC bugs

4) never ever used expensive professional proprietary engineering software on Linux, only to see it break in the next release (ABI hell) (and please take a moment to consider why Windows is dominant in engineering labs)

5) your linux distro has absolutely all the packages you could possibly wish for, and their release numbers are all synched with upstream all the time (please tell me which one that is - it's certainly not Ubuntu, Fedora, SuSE, Mandriva, or any other big one).

Yeah.. I can tell you've got years of experience.

Re:Only useful for non-free applications

[synthespian]

If you are any good, compiling source and building packages is a COMPLETELY AUTOMATED.

If that is like you say, then please explain why, e.g., Debian has a whole legion of people they call "developers" whose sole purpose in life is to take sources from other people and create deb packages?

Re:Only useful for non-free applications

[synthespian]

Big fucking deal. I remember the time when I used to walk around with 7 cds just of Debian stuff. So you carry around 7 DVDs. So what?

Re:Only useful for non-free applications

[synthespian]

You've got to be kidding? Super-easy installation and automatic security updates for all applications is 'awkward'?

You've got to be kidding. How many times was it the Debian repositories were hacked into? Remember that?

Re:Only useful for non-free applications

[ToasterMonkey]

They clearly meant free, not gratis. Gratis is such a weak feature of software that I don't think it deserves to share a meanin with free.

Gratis is THE reason "free software" is as popular as it it today. I'm deadly serious. It's the best marketing free software has because John Q Public does not give a rat's ass about the other meaning of that word unless it's used in the context of people. The marketing implications alone are more than enough.

Re:Only useful for non-free applications

[arndawg]

I was going to write a long rant about how i use gentoo and that it rocks. But my firefox haven't compiled yet and i can't be bothered with lynx anymore. I might come back tommorow and rant. kthx

Re:Only useful for non-free applications

[jrumney]

The distributor still need (almost) as much disk space and infrastructures as before

They'll need more, to provide the increased bandwidth required because everyone is now downloading binaries for all supported architectures from their servers instead of just the one architecture they need.

Re:Only useful for non-free applications

[turbidostato]

"So what if the user has a "centralized repository" that, for all I care, has outdated versions (and my options are doing the work for the package maintainer who's out in Mexico in the dope trip of his life) or doesn't have what I want *at all*?"

So what? Are you indeed trying to make a point? So what if your vendor provider doesn't feel like following procedures that will make its userbase happier and more fidel to their product? It certainly doesn't make its choice better than the alternatives.

"but free is free, right?"

I see. So all you want is trolling, right? So be it, Mr. synthespian and let's forget I already stated that providing it's own repo is an option even for your beloved closed source programs proudly produced by American labour.

Re:Only useful for non-free applications

[synthespian]

You will have to be a bit more precise when it comes to the "commercial applications" bit.

Oh, let me try to give you a clue:

Commercial applications: CAD/CAM software, image processing software, video processing software, professional music studio software, professional mathematics software, biomedical software, software for electronics, operations research software, software for trading stocks, spreadsheet software, software for guaranteeing your software is safe, hard real time software, software for the automotive industry, electronic document management and workflow software, video games, word processors, typesetting and professional publishing software, specialized proprietary IDEs, high-performance proprietary compilers for Fortran, Ada, Eiffel, etc. The whole gazillion software applications that are available for Windows (and to a lesser extent Macintoshes).

I'm pretty sure you weren't aware of them. Is it because your job is: 1) sysadmin; 2) PHP web guy; 3) Linux kernel hacker; 4) other job that means you're completely clueless as to what other human beings use computers for.

As long as Linux is something developed for and by people who are completely clueless as well as having this complete lack of critical thinking capacity, it will always remain a software for servers. In fact, due to the stupidity of the GPL, I have no hope anymore that this year, the next year, or the next decade will be "the year of Linux on the desktop." For a variety of reasons (economical, technical and - sadly - political), Linux is a complete failure in creating a healthy ecosystem of economically-viable software houses around the penguin OS (sysadmins selling support for the horrible headaches that is maintaining and installing Linux don't count - and dual-license scams don't either).

Stupid Linux fanboyism will destroy. Or, Linus' and Stallman's egos will inflate so much with dangerous gases, it will all go out in flames in a flash.

Fanboys, here's my advice (from someone who's used Linux for a decade now): you try and convince Joe Bloe, granny and mom and dad that your OS is coooler than Windows 7 and Mac OS. Good luck with Gnome and KDE. Happy C/C++ hacking. Yeah, that's the way forward.

Re:Only useful for non-free applications

[synthespian]

I think the way forward fot Linux distros is by paying up for distros so that they can make commercial deals with sellers of proprietary software we might need (codecs, compilers, engineering, virtualization, whatever).

But that's just me. I'm in the minority.

BTW, I think it's absolutely fabulous how people like Stallman were able to instill the mentality in kids that go to good universities across the world that they should pay LESS (or ZERO) for the tools of their trade than any other working category would pay. Linux fanboyism is such a mental disorder that a graduate from computer science wants to pay less to set up shop then a carpenter, an automechanic, a painter - not to speak of doctors and engineers. (You might want to look up how much a Zeiss optical microscope costs, or how much you'd have to chip in for an oscilloscope - and yet you refuse to pay for the people in the same work class as you for their work - IDEs, compilers, etc.)

How can that be? Well...That's what happens when you get dazzled by the likes of Mr. Stallman.

Re:Only useful for non-free applications

[marciot]

Well, the trouble with distributing code to end users, even if it is made as simple as a "rpm --rebuild" is that it comes with the assumption that the end-user has all the compilers and all the devel packages installed. For most tech-savvy geeky-type users, this may well be a reasonable assumption, but for non-savvy users, it might be that those things are not installed, especially on storage-limited devices like netbooks.

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

I think the way forward fot Linux distros is by paying up for distros so that they can make commercial deals with sellers of proprietary software we might need (codecs, compilers, engineering, virtualization, whatever).

Most Linux based business models use Linux as a free portion of a package sold for profit, whether that's an appliance, a service, or a cell phone. The interesting part of copyleft is it tends to creep and provide business advantages to partners who apply it more and more widely. Most of the specific items you cite, however, seem unlikely candidates in the mid and long term.

BTW, I think it's absolutely fabulous how people like Stallman were able to instill the mentality in kids that go to good universities across the world that they should pay LESS (or ZERO) for the tools of their trade than any other working category would pay.

The difference is the tools of the trade are primarily intellectual property rather than physical. I've never heard a commercial programmer claim they should get their hardware for free by law, but I've never heard a carpenter claim they should be paying for the right to imitate baroque era furniture. I have heard auto mechanics make a strong claim they should not have to pay fees in order to use the trouble codes coming out of automobiles.

But that discussion is a bit off topic.

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

See, people are confused. "Linux" is not an operating system just like NT is not an operating system. Ubuntu is an OS, WinXP is an os, RedHat is an OS, Win 98 is an OS, just to give a few examples.

That's funny, since I never see claims about the market share for Ubuntu versus Windows while I often see it for Linux versus Windows. I think the definition of operating system in those terms is very muddy so people are understandably confused, especially since almost all the software we're talking about will run on most any version of Linux, it just is difficult to install for normal users.

If a publisher doesn't make an installer for the Ubuntu OS, then that is the publisher's fault.

Why do I care? To me it doesn't matter whose fault it is, only if I can do what I want or not. That's where Apple is winning, by taking responsibility for broken parts of the user experience and filling in any holes rather than blaming others. Hopefully Canonical is doing the same to some extent and that includes making it easy and profitable and a good idea for developers to create software in a way that works well with Ubuntu.

You could publish a file containing the binary and the config files necessary for .rpm, .deb, so on and so forth and some OS detection logic and call it a "Linux" binary if you just had to do that.

Except you really can't today, because the package managers don't support it. What you can do is provide a link to a script or just a binary installer.

And the vast majority of Free software that is distributed via a website link doesn't have the source automatically included so I'm not sure what your contention is with the binary blob thing.

You're misunderstanding my term. When I refer to distribution as a binary, I mean distributing an executable that install the program, rather than just the program and letting the package manager execute and install it.

If a publisher really wants their software to be used by users of a particular OS, there are ways to make the install process from start to finish completely easy and intuitive.

Not really. Like if a publisher wants to have an app that can be run from the network on different Linux boxes from the same executable, I suppose they could replace it with a script that call the right executable for a given machine, somehow. I also think it's beside the point. A developer could write their own Linux variant to work with their software, but they aren't going to. They;re just going to pass on Linux as too difficult to develop for, and provide nothing or some less usable junk and let the nerds figure it out or fail as it pans out.

If every publisher did as good a job with their website as they do, we wouldn't even be talking about this.

If every Linux distro used the same standards and made the task easy for developers we wouldn't be talking about it either. Given Linux's tiny market share and the resultant lack of financial motivation for Website developers, which do you think is more likely to happen? Which can people interested in promoting Linux actually do?

Right, so a developer for Ubuntu just needs to write a specific version of their application for removable media and network drives

Yep, you got it. Typically referred to as static binaries.

That sucks as a strategy and is inferior to the solution in place on OS X. Why settle for an inferior solution?

Portablefirefox is a windows app, this only proves that there is demand for it on that platform.

They also specifically support Linux via WINE, so the demand is there on Linux if not the usability.

...if the demand were there, you'd see it.

That's a weak cop out for explaining away

Re:Only useful for non-free applications

[oakgrove]

Except you really can't today, because the package managers don't support it. What you can do is provide a link to a script or just a binary installer.

Sure you can. Build an installer that has the program binary, config scripts for deb, rpm, and whatever else, an OS detection routine and some logic that when it detects a deb, rpm, or whatever based OS, puts the package together and calls the native package installer like dpkg -i on Ubuntu to do the rest. Every bit of that is very doable and can be rolled up into one file. I'd throw something together just to show you but it's getting late. Maybe tomorrow.

I ain't mad at you but the rest of your post is mostly opinionated wool-gathering and I'm not really going to bother responding with more of the same.

Re:Only useful for non-free applications

[Volvogga]

Then you still need to say on your website, "Note that you must have Python installed to obtain this software."

Even after reading the author's defense, it still sounds like this is useful for providing support in niche application situations (one webserver for many client types, or amazing boot disks and installers), or to prevent DUMMY_MODE-ON from the less tech savvy users (then there are other reasons given, but they don't sound too impressive to me). In the latter case, you still have these users seeing the website and getting wide-eyed at the idea of having to find some mysterious language library. For that senario, I think the fatELF binary would be a decent idea.

Click, run, detect arch, download proper binaries/source, install.

It would seem to me this would save some decent bandwidth cost on large programs. The author says it would save on disk space for developers, but I kinda wonder if it would be negated by the bandwidth increase.

Re:Only useful for non-free applications

[oakgrove]

Caught a second wind.

That's funny, since I never see claims about the market share for Ubuntu versus Windows while I often see it for Linux versus Windows. I think the definition of operating system in those terms is very muddy so people are understandably confused, especially since almost all the software we're talking about will run on most any version of Linux, it just is difficult to install for normal users.

Yeah, and I see market share breakdowns for XP vs Vista. You see Linux vs Windows because many people see Linux as this big monolithic thing. Hint: IT'S NOT. Software in Ubuntu is NOT hard to install. Seriously, click on the .deb file and click "Install" on the little box that pops up. If the program is intended for Ubuntu yet is not distributed as a deb file then it is not finished. Start distributed Windows programs without a setup.exe or msi and watch the mayhem.

publisher doesn't make an installer for the Ubuntu OS, then that is the publisher's fault.

Why do I care? To me it doesn't matter whose fault it is, only if I can do what I want or not. That's where Apple is winning, by taking responsibility for broken parts of the user experience and filling in any holes rather than blaming others. Hopefully Canonical is doing the same to some extent and that includes making it easy and profitable and a good idea for developers to create software in a way that works well with Ubuntu.

What do you expect Canonical to do? Write anybody's app for them that wants to develop for Ubuntu? That's ridiculous. The pieces are there for a very smooth end user software installation experience. If publishers don't want to use them then what the hell do you expect Canonical or anybody else to do about it? Hold their hand?

If a publisher really wants their software to be used by users of a particular OS, there are ways to make the install process from start to finish completely easy and intuitive.

Not really. Like if a publisher wants to have an app that can be run from the network on different Linux boxes from the same executable, I suppose they could replace it with a script that call the right executable for a given machine, somehow. I also think it's beside the point. A developer could write their own Linux variant to work with their software, but they aren't going to. They;re just going to pass on Linux as too difficult to develop for, and provide nothing or some less usable junk and let the nerds figure it out or fail as it pans out.

Are you serious? Have you ever even used Linux? Hint... Make an executable script on the user's box that goes something like this:
#!/bin/sh
ssh -XC my.app.server.net $APP
Anybody with sense enough to set up an app server is going to be able to do this and it would work seamlessly. And that's just one way. *nix practically specializes in this.

If every publisher did as good a job with their website as they do, we wouldn't even be talking about this.

If every Linux distro used the same standards and made the task easy for developers we wouldn't be talking about it either. Given Linux's tiny market share and the resultant lack of financial motivation for Website developers, which do you think is more likely to happen? Which can people interested in promoting Linux actually do?

Yeah, and if Microsoft and Apple would hold hands and sing kumbaya, you could run Garageband on Windows 7. That makes about as much sense as what you said. The different Linux's are run by different people with different goals. Target Debian, Red Hat and Suse and you have practically cornered the market. Matter of fact, rpm's run on Debian as long as they are packaged to the standard and not the Red Hat specific superset. A competent developer would know this.

so a developer for Ubuntu just nee

Re:Only useful for non-free applications

[koiransuklaa]

Yes, I said "Debian". Debian is an Operating System, Linux isn't. Operating System is the smallest unit of supportable entity that Logitech has available. Trying to do anything else will lead to problems.

This is problematic of course if you imagined you could support "Linux" with a single binary package and achieve good quality. That's just not the reality (sorry). The situation isn't that bad though: I'm sure people like Opensuse will sell Logitech a license or a copy of OBS that they can use to build their app for 30 or so distribution versions in a jiffy.

Your question about libapr is ... a little contrived: It's fairly obvious that you can only use libapr 1.2 if it is available in the distribution you support. If it's not and you still need it, you probably want to include it in your app -- unless you really want to be in the libapr1.2 providing business from now on (you don't, that was rhetorical).

The above is just common sense, I would think. An "extra" repository (like logitech) should only carry packages for their software. Upgrading the operating system with new base libraries is definitely not appropriate. 3rd party libraries should probably just be included in the app. I don't want to sound confrontational, but... you seemed to understand the possible problems related to this: why did you then suggest a totally stupid solution when sane ones exist?

If you dont address the concerns of that 95%, you are never gonna get linux in the mainstream.

"Linux" as an entity will never be mainstream, imagining that it could by following some specific advice is delusional. Linux is already a thousand different things and it just spreads out more and more. Specific operating systems may well become mainstream -- personally I think they are already with Maemo in Nokia N900, but feel free to pick another metric.

I disagree with your opinion about "[open sourcing] not happening for 95% of the software" by the way. This is already happening with drivers: more drivers are being released as open source than ever have been. I imagine a lot of companies (like your example Logitech) will likewise realise that their real expertise is not in the crappy software they had to produce, and that allowing copying is just going to help their actual product... It just takes time.

Re:Only useful for non-free applications

[nemesisrocks]

It was... a joke. Thank god English is free of that stupid distinction?

but is it DFSG free?

Re:Only useful for non-free applications

[koiransuklaa]

Yes, of course I remember.

What is your point? Are you really saying that keeping desktop software updated is a bad idea?

Your delivery makes it hard to tell what you actually suggested as an alternative -- it almost seems as if you just saw something you could use to taunt and went for it without thinking how it relates to the conversation.

The issue here is Zobeids suggestion that package management should not be used for desktop applications. Please offer an alternative if you have one, and remember to explain how the updates (or the lack of) are inherently more secure than the ones in Debian.

Re:Only useful for non-free applications

[dissy]

It's just a waste of the user's disk space to have them keep code for other architectures.

But this plan would put that choice in the hands of the person who owns that disk space. It should be their choice to use this format of binary or to use the current one after all.

Your plan puts the choice in your own hands, which I would guess not many people but yourself care about :)

Everyone is always so quick to tout open source being the place a developer can scratch their own itch, yet so many slashdoters are so violently against anyone doing that.

Re:Only useful for non-free applications

[amplt1337]

Uhh... I feel your point is somewhat weakened by the fact you're talking about software that's over twenty years old.
You think porting it was hard? Just imagine if you'd gotten a "convenient" "runs-anywhere" binary instead...

Re:Only useful for non-free applications

[doug]

Actually, this can be good even for Ubuntu. Think of having i7, Atom and Phenom versions of executable alongside classic 586 version. Also, having one repository instead of several can also streamline few things. Downside is more data transferred on software updates, unless some create really smart update which transfers only part of fat binary that is actually used on client.

Hold on: isn't smart update which transfers only part of fat binary that is actually used on client the same as what we've got today. I mean, I just get the bits I use (x86_64) and nothing else. How often do you really have a single system that supports multiple architectures? Almost no one does at home, and at work folks usually have to support different rev levels for different situations. Either way, a central shared image where all the versions are updated lockstep isn't as useful as something that could bind to different library levels. From what little I've read, the fat binary isn't up to that task. It might be nice in a few situations, but it smells like a solution looking for a problem.

BTW: I could see this fat binary working in the installer to help newbies who don't really know what type of executable system to download, and I can see it being useful for 3rd party stuff, but not core Ubuntu. And even for that, I bet a bit JavaScript at download time could answer that question.

- doug

Re:Only useful for non-free applications

[99BottlesOfBeerInMyF]

Software in Ubuntu is NOT hard to install. Seriously, click on the .deb file and click "Install" on the little box that pops up. If the program is intended for Ubuntu yet is not distributed as a deb file then it is not finished.

Then most Linux software is not finished for Ubuntu and Ubuntu is suffering from an even greater lack of titles than anyone I've ever heard has asserted.

Start distributed Windows programs without a setup.exe or msi and watch the mayhem.

Your analogy is flawed because, in general, a .exe file will run on Vista or XP or Win7. They don't have a different package format for different versions the way RedHat and Ubuntu do. Both Windows and Linux have programs break as versions change. You need to compare Apples to Apples and version changes in Windows to version changes in Ubuntu.

What do you expect Canonical to do?

If they want to make their system more usable I think they should adapt their package manager in several ways. First, they should create a way to find and license software from within the package manager, a feature they've announced. Second, they should make sure those apps have the same functionality as other apps managed by the package manager (like auto updates). Adding customer reviews would be nice but is not essential. Adding the ability to automatically re-download already licensed apps or move those apps directly to a new machine and de-authenticate them on the exiting machine is also fairly critical. Finally, they need to make the method for developing and publishing these packages simple for users, by helping to tweak development tools to create the correct packages by default and by offering hosting by Canonical as well as an easy way for developers to run their own repositories that get added to the store.

If publishers don't want to use them then what the hell do you expect Canonical or anybody else to do about it? Hold their hand?

Absolutely. Helping companies clean up their software to work better is one of Canonical's big pushes right now. Remember the free usability testing they're offering? Ideally the available tools should make things easy without any hand holding, but if they are lacking or too unfamiliar to developers, Canonical needs to do whatever it takes to bridge the gap.

Are you serious? Have you ever even used Linux? Hint... Make an executable script on the user's box that goes something like this:

Way to miss the point. In order to use the solution you present you need to put it on every individual machine on the network. You think that's a reasonable solution? Most people just end up putting several executables on the server and letting people guess their platform until they hit the right one. You do know there are use cases where users bring their own machines, right?

Yeah, and if Microsoft and Apple would hold hands and sing kumbaya, you could run Garageband on Windows 7. That makes about as much sense as what you said.

So you expect all developers to magically reform their ways and ignore what is most profitable and you think that is more likely than a consensus and real standards being implemented across Linux distros? I'm talking about the more practical option. You're the one with a head in the clouds idea.

That sucks as a strategy and is inferior to the solution in place on OS X. Why settle for an inferior solution?

Yeah, so make a package and go for the superior solution that includes global security and functionality updates.

I'm not sure I understand your comment. What are you proposing? I'm proposing a merger of OS X's package styles (multiple architectures and portability by default) with Linux's package management style (central management and auto updates) to get the benefits of both on one platform.

Re:Only useful for non-free applications

[doug]

Then don't use apt-get. I usually use synaptic, which is presents a GUI with bunches of options. Click what you want, and it does all the apt-get stuff for you. It is about as painless as can be. Perhaps there are too many packages, but that is just a question of organisation of the repositories, or filtering at the client (synaptic) side. I can see the advantage of presenting "applications only" for "desktop application users", and leaving all the system stuff, the helpers, the libraries, and other minutia to those of us who care. There is already some of that the very coarse level, such as "I want a webserver", but I think you want an inbetween point.

- doug

Re:Only useful for non-free applications

[oakgrove]

Your analogy is flawed because, in general, a .exe file will run on Vista or XP or Win7. They don't have a different package format for different versions the way RedHat and Ubuntu do. Both Windows and Linux have programs break as versions change. You need to compare Apples to Apples and version changes in Windows to version changes in Ubuntu.

No, what I was getting at is that a setup.exe is a packaged installer. You run the exe and it runs a self contained script that copies the relevant files and settings to their intended directories and makes the appropriate registry entries for the program to work. If you just distributed the raw executables and expected the end user to put them in the right places themselves, it wouldn't work. This is akin to releasing a program for Ubuntu without packaging it in a deb file. And criticizing Canonical because third party software publishers aren't packaging their software in a deb file is like criticizing Microsoft if some developer didn't distribute their files as setup.exe's. I get that from your perspective this actually is Canonical's fault as they should somehow make it easier for developer's to make these deb files. Checkinstall is the program you are looking for. When the dev compiles their app they use checkinstall and it automatically compiles the app and packages it into a deb file. This tool is somewhat rudimentary and limited but it does work. Can the whole process be made even easier? Of course it can. Anything can be made easier. But with the plethora of good software that is packaged well and ready to go with just a click right now, difficulty of packaging software is not really the sticking point it's being made out to be by some people.

If they want to make their system more usable I think they should adapt their package manager in several ways. First, they should create a way to find and license software from within the package manager, a feature they've announced. Second, they should make sure those apps have the same functionality as other apps managed by the package manager (like auto updates). Adding customer reviews would be nice but is not essential. Adding the ability to automatically re-download already licensed apps or move those apps directly to a new machine and de-authenticate them on the exiting machine is also fairly critical. Finally, they need to make the method for developing and publishing these packages simple for users, by helping to tweak development tools to create the correct packages by default and by offering hosting by Canonical as well as an easy way for developers to run their own repositories that get added to the store.

I absolutely agree and have been preaching this exact thing to people any time the subject comes up. One of the greatest treasures of Linux that could really propel the platform is the package management system and sadly, it isn't being leveraged in an accessible user-friendly way. Hopefully the Software Store addresses this however, it is going to take a while as it's looking like it's going to be phased in over the next 4 releases. I understand that Canonical has to ration its limited resources but if I were Mark Shuttleworth, I would make this a top priority so as not find themselves in a position of too little too late as it's no secret that with the iPhones success and model, a lot of the industry is going to want to move in this direction. Even Microsoft is making some strides towards this model with the ease of driver downloads in Windows 7.

Helping companies clean up their software to work better is one of Canonical's big pushes right now. Remember the free usability testing they're offering? Ideally the available tools should make things easy without any hand holding, but if they are lacking or too unfamiliar to developers, Canonical needs to do whatever it takes to bridge the gap.

Your confusing my point. I'm not saying they shouldn't do this at all. I

Re:Only useful for non-free applications

[spitzak]

Free Range

Free Wheel

Free play (as applied to mechanical connections)

Free Choice

Free Enterprise

I think there are a lot counter examples to your claim. Of course I guess it is a Free Country and you can apply your own Free Interpretation.

Re:Only useful for non-free applications

[xiong.chiamiov]

Neither Linux on the desktop nor OS X is perfect when it comes to software installation and both should borrow from the other. Right now Ubuntu, probably the front runner for usability in desktop Linux, still has multiple programs used to manage packages

I go through the process of deciding what software I want to use by looking at Web pages, then I open my package manager and copy and paste the name into the package manager. Then I sometimes find it and install from there with only on wasted step, but often I don't find it so I go back to my browser and install by hand using whatever method is necessary.

Hold on, sonny! Don't blame Linux in general for Ubuntu's failings! As an Archer, I use one application (well, actually like four, but they're just wrappers around each other (yaourt -> powerpill -> pacman-color -> pacman)) to install applications, and only *once* have I wanted to install an application that was not in either the official repositories or the AUR. And guess what? It took me about 5 minutes to make a PKGBUILD for it, and now it's up on the AUR, so that anyone else who wants to install it can do so directly from yaourt. Yay!

Re:Only useful for non-free applications

[spitzak]

Although OSX might be better than Linux, I have not seen any examples of what you are talking about.

Download & install of an application on OSX seems to always consist of:

1. download the ".dmg" file
2. Double-click it and be a bit mystified until you find the "box" icon. Do not double-click anything that looks like the app or it might end up screwed up!
3. Double-click that and hit ok a bunch and install it (this step always works nicely)
4. For some software you will fuck up and it will not run unless you first double-click the "dmg" to open it. Not sure what that is, but an MP3 encoder did this and so did a version of Maya (but the new one is fixed).
5. Throw away the .dmg file so it does not clutter your desktop, after confirming that the software works.

No system does a "user friendly" thing. Windows users think it is "easy" to first "download" and then "install" is a sure sign of brainwashing. This is what I would consider user-friendly (though there are obvious security problems, but this is the most complicated I would consider to actually be user-friendly):

1. User clicks on the "run this software" button on the website
2. SOFTWARE RUNS NOW!
3. And it is installed, too, or asks user if they want it installed.

Re:Only useful for non-free applications

[coolsnowmen]

Never have I had a program I installed by hand depend on more than 1 other uncommon program/library I didn't already have installed. And, Package managers take care of that for all common libraries.

I haven't had that 'dependency hell' problem in 6 years. That would be like insulting windows vista by complaining about windows ME.

Re:Only useful for non-free applications

[badkarmadayaccount]

Thermonuclear?

Re:Only useful for non-free applications

[jedidiah]

Your response was TOTAL GIBBERISH.

Just spouting off a bunch of random nonsense is not an answer.

Give REAL EXAMPLES, not just random words strung together.

There's lots of software available for Linux that can be useful to a lot
of people and completely eliminates the need to "pay" anything for
"commercial software". So whining about it really makes no sense unless
you have a real axe to grind.

"Something being missing" wasn't the issue.

Although a proper example of that would be a refreshing change too.

Why should my next door neighbor be bothered if Windows vanishes in a puff of smoke at midnight?

Re:Only useful for non-free applications

[Tacvek]

No my reply was strictly a reply to the parent post. Even programmers find compiling to be a pain, especially if software is not pre-ported. I really don't see universal binaries as very valuable at all, given that they don't help with the porting. Once source is ported there is no reason for the porter or maintainer to ever compile for that platform again. Just create an auto-compilable source package. Gentoo is the ultimate example of that, but even the vast majority of binary packages for Debian architectures are auto-built from source.

Perhaps if LSB were working well enough that all x86 GNU/Linux distros could run the same version of an app, and all PPC GNU/Linux distros could run the same version of an app, a universal binary might be something a proprietary vendor for Linux might like, since it would ensure all users got a version that would work on their architecture (if theirs is supported at all) rather than accidentally downloading/ordering one for a different architecture.

Re:Only useful for non-free applications

[Tacvek]

Except that the last update to the software was more recent than 20 years old. It occurred at least a recently as 1993. Obviously a universal binary would have been worthless, but my reply was simply a reply to the parent, which was talking about how even for a programmer attempting to compile source code is not as easy as it it seems like it should be.

Does Linux even need them?

[GreatBunzinni]

Some people may claim that Linux may have some shortcomings but certainly the way that distributions handle support for multiple platforms and also the availability of binaries targeted at a certain platform surely isn't one of them. Linux already runs on a long list of platforms and software distributions already handle themselves quite nicely by building platform-specific packages, which also include all sorts of platform-specific binaries the applications will ever need. So, besides the empty "but Apple has them" rational, exactly what drives the need for universal binaries on linux?

Re:Does Linux even need them?

[buchner.johannes]

Well, the effort of packaging a application (a) to different platforms and (b) to different distributions is quite a duplicate one, involving a lot of people (and time).

Re:Does Linux even need them?

[Carewolf]

No Linux does not need them, even if you want to distribute multi-arch "binaries" linux already have a universal "binary" format called shell-scripts

Replace installed binaries with a symbolic link to a simple shell-script and you magically have a multiarch "binary".

#!/bin/bash
ARCH=`uname -m`
LD_LIBRARY_PATH=/usr/lib/$ARCH
`/usr/bin/$ARH/$0`

Re:Does Linux even need them?

[GreatBunzinni]

I believe the people behind autopackage will disagree with you. Nonetheless, I don't see how universal binaries would get rid of those non-issues. So, besides universal binaries being a solution looking for a problem, exactly what problems do they actually solve?

Re:Does Linux even need them?

[PeterBrett]

Well, the effort of packaging a application (a) to different platforms and (b) to different distributions is quite a duplicate one, involving a lot of people (and time).

Firstly, this proposal has absolutely no relevance to the difficulty of packaging to different distributions.

Secondly, packaging to different platforms has been solved. Most distributions now have compile farms where you submit a package specification (usually a very simple compilation script and a set of distribution-specific patches) and packages for all the various architectures get spat out automatically.

This proposal is a solution looking for a problem, as far as Free software is concerned. The only utility is where the application is closed-source and can't go through a Linux distribution's normal package compilation and distribution workflow.

Re:Does Linux even need them?

[Anonymous Coward]

That's true for free software that happens to be part of your distribution, yes. It's not true for non-free software, nor is it true for free software that isn't included in your distribution, or for free software that has updated versions that haven't been packaged yet.

Basically, there are three ways to handle an architecture transition.

First, you can use fat binaries. Apple have used this technique to transition between PowerPC and x86, and are using it again to transition between 32- and 64-bit. It works brilliantly.

Second, you can take Microsoft's approach. Assume that 64-bit applications know what they're doing, and that 32-bit applications do not. Keep separate copies of all of the system libraries, one for each. Apply lots of crazy path (and registry) redirection voodoo to any running 32-bit applications. Depending on which mode the current application runs in, it'll have a completely different view of the filesystem (and registry). This approach only really works for libraries included in the system. Third-party libraries have to include multiple copies, in different files, and use manifest files to work out which one to use. You still need separate versions for 32- and 64-bit systems, even though the 32-bit version will usually run on a 64-bit system. For some types of application, you need the 64-bit version on a 64-bit OS, because the 32-bit version doesn't work. For things like plug-ins or video codecs, you usually either need just the 32-bit version, or both the 32- and 64-bit versions installed at once.

Finally, you have the approach Linux currently uses. A 64-bit OS is 64-bit only. It can run statically linked 32-bit applications, but nothing else unless you install a separate set of 32-bit libraries. This approach is only possible on an OS where the vast majority of available software is both open source, and works when compiled as 64-bit. It would not be possible on Windows or Mac OS X. It works OK, until you try to install a single 32-bit binary application. Then you're screwed. You still have the problem of working out which version of something you actually need. Usually, you want a 64-bit version on a 64-bit system, but not always. Sometimes, you still need the 32-bit version (rare, because it often won't work), and some other times you would need both (very rarely - video codecs that need to be used in the native media player and a 32-bit web browser, for example).

There are only two ways to "fix" this problem. The first is to take Microsoft's approach, which doesn't solve the problem of having to choose which version of something to install. It also requires a lot of extra maintenance overhead, and considering that your average Linux distribution has much more software than Windows does, and virtually every library in the repository is treated as a system library, it's probably not possible to actually maintain such a beast. If it were, then someone would probably have already done it.

The other way is to take Apple's approach. You supply one copy of each library and executable, and let to OS work out which one to use. All libraries and plug-ins would all be available to both 32- and 64-bit applications. The OS installer will be able to install the 64-bit version automatically if you have a 64-bit machine, and install the 32-bit version otherwise. You could upgrade from 32-bit to 64-bit simply by installing a 64-bit kernel.

It becomes an even better idea if you throw ARM into the mix. While there would be no benefit in supplying x86/x86-64/ARM packages in a distribution, there would be a benefit in distributing stand-alone packages that way. It'd just cost a tiny bit of disk space for applications / libraries / plug-ins that were installed outside of the distribution's package manager. And you could easily write a tool to strip the unused parts of the binaries out if you don't need them. These exist for Mac OS X already. You could even do this as part of installing the package - if you install a x86/x86-64/ARM package on an ARM machine, just strip the x86 and x86-64 parts from the binary.

Re:Does Linux even need them?

[Jesus_666]

The fact that you can go to the website of your distro of choice and be presented with a list of one dozen different installation DVDs, two of which actually run on your system. Granted, unified installers could be done without fat binaries, but far binaries allow it to be done more elegantly.

I'm not even asking for the installed system to be fat. Just have the installer be fat and carry multiple package trees for the various architectures the disc supports. In the case of a net installer, there is really no excuse for more than one version of the CD as it doesn't even carry any platform-specific packages.

Re:Does Linux even need them?

[99BottlesOfBeerInMyF]

So, besides the empty "but Apple has them" rational, exactly what drives the need for universal binaries on linux?

I have two computers both running the same OS, but only one license for a vital closed source software application. I want to be able to run that application on both my computers and on the computer at school running the same OS, but where I don't have permissions to install software. With OS X, I can drop the .app bundle on a flash drive and run it directly from there on all three different architectures and I don't need a special "mobile" version of the software. With Linux, my best real bet is to install that same application in a VM, install the VM on the flash drive and try to get the same VM manager installed on all the machines. That's a serious lose for Linux.

On OS X a friend I was IM'ing asked what was the best software for a given task. He was running a different architecture, but also using OS X. I dragged the .app bundle into the chat, it transferred and he double clicked it. Problem solved. I might mention, this was closed source freeware, no longer being distributed by the developer. With Linux, well the installer for the software could theoretically still be on my machine, depending upon how it was installed and I could have zipped it up and IM'd that, but he'd have to know how to use it and it would not have been for the right architecture. That's another lose for Linux.

When I buy a new computer I usually use a program or built in system to automatically transfer all my files, settings, and applications from my old computer onto the new one. In OS X, this transfer happens over Firewire and is nicely polished. This includes all my closed source software programs as well as OSS. If the two machines are for different architectures, it doesn't matter, because the application bundles include versions for both. If I try the same thing on Linux, I have to hunt down new installers for all the closed source applications and then install them and then re-register the authentication codes from my little black book of codes. It's a whole lot more work and another "lose" for Linux.

Do you ever run software off of a network drive shared by multiple systems? Do you know the architecture of all the systems that will be accessing that application beforehand? On Linux, you need to install separate versions and the end user has to know what architecture the system they're using is running. Another lose for Linux.

When I'm running a laptop with limited hard disk space and I am running out of space, Linux uses less space because the actually binaries for applications are only for the platform I'm on. With OS X I have to go out of my way to run a program that will prune the unneeded binaries to conserve space. That is a win for Linux.

Hopefully you are now enlightened as to several of the ways multiple platform binaries used on OS X provide advantages over Linux which lacks that ability as well as the one case I can think of where it provides a drawback.

Re:Does Linux even need them?

[coryking]

The only utility is where the application is closed-source and can't go through a Linux distribution's normal package compilation and distribution workflow.

In other words, it would only be useful for about 95% of the software in existence? Yeah, fuck those assholes.

We need 1-file installs

[Jim Hall]

We don't need the universal binary, so much as we need the "1-file install" idea that MacOS has. This would greatly simplify installing a standalone application.

For those of you who don't know, if you download an app for MacOSX (say, Firefox) you are presented with one icon to drag into your "Applications" folder. This is really a payload, a "Firefox.app" directory that contains the program and its [static?] libraries. But to the user, you have dragged a single "file" or "app" into your "Applications" folder - thus, installing it.

It's dead simple. We need something like this in Linux.

Re:We need 1-file installs

[John Hasler]

> It's dead simple. We need something like this in Linux.

"aptitude install " (or the pointy-clicky equivalent) works for me.

Re:We need 1-file installs

[betterunixthanunix]

This is /., if it is not as pretty as Aqua, it does not count.

Re:We need 1-file installs

[dna_(c)(tm)(r)]

apt-get install firefox

or synaptic (click) firefox (check) apply (click)

A lot easier than finding/going to the website, clicking the download icon, waiting, draging an icon... Mozilla does this wonderfully (agent string used to present the download for your OS/language), most other downloads require you to look for the download you need.

Re:We need 1-file installs

[TheRaven64]

As I get tired of repeating, GNUstep has had this on Linux (and *BSD, and Solaris, and Windows) for many years. It supports NeXT-style bundles with different binaries (and, optionally, different resources) for different systems, so you can easily store Linux, Mac, FreeBSD, and Windows binaries in the same bundle.

Re:We need 1-file installs

[Anonymous Coward]

I think he meant a general solution to handle software that isn't in the repositories.

Re:We need 1-file installs

[RegularFry]

# aptitude install firefox

...seems to work for me. What distro are you on, and what (if any) error do you see?

Re:We need 1-file installs

[Peaker]

Which ancient OS are you using?

In Ubuntu at least the first command works fine (Haven't bothered to try the others)

Re:We need 1-file installs

[teg]

"aptitude install " (or the pointy-clicky equivalent) works for me.

No, it doesn't. "yum install firefox" and similar things like apt install something called firefox. In many cases this will be ok, but in many others it won't. E.g. when new releases of openoffice.org or firefox arrives. This often won't show up in normal repositories for a while, if it shows up at all before a later release of the distribution they are running.

Re:We need 1-file installs

[buchner.johannes]

Linux' way of saying "NO NEVER NEVER DO THIS! Also, it doesn't work." to installing stuff by downloading it yourself has beauty where Apples concept is malware-prone.

The only way I can see your suggestion implemented is by providing a file format that just contains what package you would like to install (for each possible distribution).

e.g. for Firefox.app:
Gentoo: ensure_installed(www-client/mozilla-firefox)
Debian: ensure_installed(firefox)
Fedora: ensure_installed(firefox) ... etc
Declarative, that is. Then the package manager takes care of the rest.

Re:We need 1-file installs

[Carewolf]

It's dead simple. We need something like this in Linux.

You may want to check out something called RPM and DEB files. They are single files packages used to install applications. The rest is just a matter of user-interface. Create an io-slave (KIO, GVFS) that shows installed packages, and executes install when files are dropped there and you have everything you ask for.

Re:We need 1-file installs

[jonbryce]

That is great for software supplied by your distro's repository, and most distros have lots of software available in their "contrib" or equivalent repository. Firefox of course usually comes installed out of the box, so it isn't an issue.

Where this could be beneficial is for software that isn't popular enough for the distros to package. At the moment, you have to publish different packages for each distro and for each architecture, and you probably won't bother about much beyond i386 and amd64.

Re:We need 1-file installs

[jonbryce]

jonathan@keisha:~> su
Password:
keisha:/home/jonathan # aptitude install firefox
bash: aptitude: command not found
keisha:/home/jonathan # zypper install firefox
Loading repository data...
Reading installed packages...
'MozillaFirefox' providing 'firefox' is already installed.
Resolving package dependencies...
Nothing to do.
keisha:/home/jonathan #

Re:We need 1-file installs

[jonbryce]

Except that you usually have to download and install a ton of dependencies to make it work. apt-get and similar make this easy for you.

Re:We need 1-file installs

[Anonymous Coward]

Yeah, and who actually uses GNUstep? NOBODY.

Re:We need 1-file installs

[Carewolf]

Yes, but that is an advantage of the Linux way, which makes the binaries slimmer. There is however nothing preventing a .deb files from containing all its dependencies and installing them to a package specific location or having them statically linked.

Re:We need 1-file installs

[Nimey]

By default Ubuntu associates .deb files with a simple graphical program (a front-end to aptitude) that will install them for you.

There's also Add/Remove Programs (Ubuntu until 9.10) and whatever the equivalent is now in 9.10.

Re:We need 1-file installs

[TheRaven64]

And who then complains about lacking features GNUstep has had for years? EVERYBODY.

Re:We need 1-file installs

[slim]

The Apple/NeXT .app approach is fairly different to RPM/DEB package files.

If you could execute the .deb or the .rpm directly (i.e. double click firefox.rpm, or type ./firefox.rpm, and have Firefox launch instantly), that would be more similar.

If you were aiming for this, on Linux, from scratch, you'd:

  - assign some sort of filesystem attribute that says "this is a directory that contains an application"
  - have a standard layout for where the executable is expected to reside within that directory.
  - application knows to find resources in paths relative to the app directory
  - package this as an iso or a tar.gz
  - to install, untar anywhere. Nothing else to it.
  - or, more like OSX, mount the ISO, run directly from the ISO, or copy the directory anywhere.
  - execute the directory.
 

Re:We need 1-file installs

[Anonymous Coward]

If you are talking about the delay e.g. Debian Unstable has, that is the price you pay for the packaging effort: Someone has to package the new version and make sure it co-exists nicely with other software. Sometimes it takes time (Firefox 3.5 is a good example: it brings in a ABI incompatible xulrunner which breaks other apps), but I'm glad someone does that job for me.

If you are talking about the delay until things show up in Ubuntu or Debian Stable repos... Well, maybe you shouldn't pick a release-based distribution if you want constant updates?

Re:We need 1-file installs

[pmontra]

It's simple: each non-free-software company sets up its own repository, instruct users about how to add it to the system sources list and that's all. By using a repository they gain multi architecture support (*) and system-managed updates of the applications.

(*) They have to compile their own software for all architectures and can't rely on distributions to do that, but there is something you to pay if you want to close your software.

Re:We need 1-file installs

[bk2204]

The problem with having one file to copy is that it defeats one of the most important benefits of Linux: shared libraries. Unlike Windows and Mac OS X, it's very common to have one copy of a library that's shared by all processes. If every one of the GTK-based programs on my system had a copy of GTK, Glib, ATK, Pango, and all the other libraries, the bloat would be huge. It would also be really bad for distribution mirrors, especially for a project like Debian, which is a non-profit and cannot pay for its own hosting.

Re:We need 1-file installs

[ZERO1ZERO]

But to the user, you have dragged a single "file" or "app" into your "Applications" folder - thus, installing it.

It's dead simple. We need something like this in Linux. You mean like this : http://0install.net/

wiki: http://en.wikipedia.org/wiki/Zero_Install

What do you mean you aren't already using it?

Re:We need 1-file installs

[grumbel]

"aptitude install " (or the pointy-clicky equivalent) works for me.

That only works for stuff you distribution provides, for third party stuff its already problematic and for stuff that should work across multiple distributions its completly useless.

Re:We need 1-file installs

[toby]

Linux package management, or Windows installer, is not the same thing at all.

If you've used a Mac you know that, from 1984 until today, to install an application you just physically copy it. The application is everything, there are no "other" hidden changes to the system, no registry changes, etc.

The user is in control, both in the installation and removal steps, and the operation is transparently obvious - in keeping with the Mac's original human interface philosophy.

Re:We need 1-file installs

[Burz]

The levels of denial in this discussion are epic: +5 for that ignorant one-liner?

Install this: Independent Software Vendors (ISVs). They make applications that are distributed independently and do not come with an OS attached.

Re:We need 1-file installs

[coryking]

Yes, but that is an advantage of the Linux way, which makes the binaries slimmer.

So I set up my own repository for customers to download my CNC milling software. My software, like any, has a lot of dependencies.

Do I host these dependencies on my repository or what? If I dont rely on my repository, which repository do I use? What if they nix support for the specific version of the dependency I depend on? Do I now have to host it myself? What if the dependency on the repository somehow gets changed so it is no longer compatible with mine but counts as the same version? What happens if the user has already added a third-party repository with installed the "same" dependency, only compiled in a way that makes it incompatible with my software? What then? I can't plow over the "same" dependency with my own or it would break other software. What do I do?

Worse, which type of repository do I host? YUM? Some RPM thing? What? Do I have to learn how to maintain a repository for every type in existence?

Help me out here. I just want my CNC milling software to work on Linux. It was so easy with my Windows and Mac customers. The Windows customers just ran "setup.exe" and it worked. The Mac folks did just about the same.

Re:We need 1-file installs

[Burz]

Firefox of course usually comes installed out of the box, so it isn't an issue.

Yeah, but how does a user get a current version of Firefox on an 18-month old distro like Ubuntu LTS?

I was going to say that people keep reaching for an example like Firefox because to use an ISV-distributed program like Photoshop would be odd: Photoshop and similar programs don't exist for FOSS distros. But then I remembered that even Firefox is an appropriate example of the problem.

Re:We need 1-file installs

[booyabazooka]

A 1-file software package... like a .deb??? Double-click in Nautilus to run gdebi-gtk, click "Install package", done. No "applications folder" necessary.

Re:We need 1-file installs

[jonbryce]

I would look in backports. I generally find what I'm looking for in there.

Re:We need 1-file installs

[Blakey Rat]

And I can take the resulting application file, put it on a USB key, then run it on a PPC Linux computer?

No?

Oh, well, then I guess it's not at all the same and you're full of shit.

Re:We need 1-file installs

[jeremyp]

Fat binaries in MacOS X/NextStep are not the same as bundles. Bundles are merely special directory structures containing several architecture specific binaries. Fat binaries are single flat files containing the executable code for two or more architectures. Fat binaries need support from the OS dynamic linker.

If you look inside an OS X application bundle you will see only one compiled executable file, but that executable will contain PPC and 386 executables and often 64 bit versions of both of those too.

Fat binaries can work without bundles. All of the command line executable programs supplied with OS X are fat binaries.

Re:We need 1-file installs

[crispytwo]

I agree.

I just bought (yes bought) two games over the last couple of weeks and although it was relatively painless to "install" (read "use"), the 1 file install would be welcome. Perhaps it would be a Gnome or KDE feature (or whatever DE you use) to be Application aware with tar files or zip files. By making these "detect" applications via some simple content, like a manifest.xml or whatever included into these packages.

1) different architecture binaries could easily be identified by the manifest file
2) Data doesn't have to be duplicated
3) Unpacking these would still work.
4) It's a relatively easy addition.
5) An application Icon could finally be specified. Imagine that!

To accomplish this doesn't have to be complicated, and it certainly doesn't have to be used for every application... I wouldn't want ssh to be 'universal', but Photoshop would be welcome.

Re:We need 1-file installs

[Burz]

I would look in backports. I generally find what I'm looking for in there.

Umm, that's not going to work for 70% of the techies and 100% of the non-techies that I know.

Re:We need 1-file installs

[oakgrove]

Provided you accept Android as a version of Linux, we have this.

Re:We need 1-file installs

[wolrahnaes]

Firefox is actually a great example of the problem. I'm an Ubuntu user and more than once in the last few years I have ended up stuck with an out-of-date version due to being stuck in "distro stability" hell. I shouldn't have to upgrade my entire operating system or even go through the trouble of finding backport repositories just so I'm not stuck using Firefox 2.0.x when 3.5.x is out. On Windows or Mac OS X, I can easily go to the Firefox web site and download a single file which will either install the application I want (Windows) or contain a single item I drag to my Applications folder (OS X). At that point I'm done. It doesn't matter if I'm using Windows XP, 2003, Vista, 7, 2008, etc. It doesn't matter if I'm running OS X 10.4, 10.5, or 10.6. The application just downloads and works.

That is what Linux needs to succeed on the desktop. Either drop the idea of distro stability for user-level software so we can get up-to-date versions within a reasonable amount of time or let the users easily install non-repo software.

I love the fact that most of my software is automatically installed and kept up to date by the repositories, but pretending they're the end-all solution is lying to yourself. I'm with Mr. Hall, Linux needs OS X style single file packages NOW.

Re:We need 1-file installs

[gbutler69]

I double-click a "DEB". How is this different or easier? Oh, yeah, I guess it is easier to just double-click than "Drag" a file. Dragging a file requires much motor coordination. Much more than double-clicking.

Re:We need 1-file installs

[Carewolf]

No, you ship the dependencies inside the binary package. Just one package. Or you statically link the libraries into your program so you don't have dynamic dependencies.

Re:We need 1-file installs

[koiransuklaa]

Help me out here. I just want my CNC milling software to work on Linux.

I already answered to you about the dependency problems in another thread (short version: no, you do not host 3rd party dependencies. You use the ones provided by the OS or include things in your app), but thought I'd commment on this.

The linux ecosystem is very, very different, that's a fact of life. Being bitter about "linux not being Windows" isn't going to help your bottom line. Expecting irrelevant details like universal binaries to solve your problem is also futile. It's another world, get used to it or stay out if it sounds like too much trouble.

Working the same way as you do with Windows and OS X may actually work for you but in that case you need to accept that you aren't giving your clients the best service possible (nothing wrong with that, just a decision to make).

If you want your software to be a "good citizen" in this environment, you will package it for specific distros. I mentioned Opensuse Build Service already and I really suggest you ask OpenSuse whether they can offer you the packaging/hosting service you need.

OBS can't do testing for you of course but that's another problem that needs to be solved in any case before you can advertise "Compatible with Ubuntu x, Debian y and Opensuse z" on your website.

Re:We need 1-file installs

[Ginger Unicorn]

Surely you mean GNOBODY.

Re:We need 1-file installs

[werfu]

One file install is dead simple, link all your apps statically, package them in one folder, than install them into /opt. All it lacks is a kind of autorun that would start the apps when you would click on the folder. Geez, it's already possible with installer : a bash script with a .tar.gz appended to it. Nothing would stop you from only tarring the directory and having that bash script from running the app.

Re:We need 1-file installs

[badkarmadayaccount]

You mean the RISC OS approach...

Re:We need 1-file installs

[slim]

Ah now, if all the good ideas in Risc OS were still used, we'd all be happier :)

convenient for _closed source_ software vendors

[koiransuklaa]

...it's convenient on the end-user and on software vendors for distributing their applications.

Sofwtare vendors? This only makes life easier for _closed source_ software makers. For everyone else this is a solution looking for a problem as package management and repositories don't really have a problem with different arches and versions.

I'm not saying this is useless (people do want to run closed source software), but the kernel, glibc and other patches better be good and non-invasive if this guy wants them to land...

Re:convenient for _closed source_ software vendors

[betterunixthanunix]

"For everyone else this is a solution looking for a problem as package management and repositories don't really have a problem with different arches and versions."

Actually, having to maintain packages across several architectures can be tricky at times. Some packages need to be patched to run correctly on different architectures, and the upstream maintainers can accidentally break those patches (e.g. if they are not personally testing on a given architecture). It could even be the case that different architectures have different versions of the same packages, because the distro maintainers are busy trying to get everything to work.

I am not saying that this "universal binary" solution is the answer, but it might help streamline the build process at the distro level. It might help.

Re:convenient for _closed source_ software vendors

[koiransuklaa]

Oh yes, maintaining packages for several archs is real work, I'm not claiming otherwise. I just don't see how universal binaries makes things easier. Coding, compiling, testing, patching -- all of those need to be done with all supported archs in mind in any case.

Re:convenient for _closed source_ software vendors

[turbidostato]

"Actually, having to maintain packages across several architectures can be tricky at times."

Of course yes. But let's see if the single fat binary reduces complexity.

"Some packages need to be patched to run correctly on different architectures"

And they still will need that. Or do you thing that the ability to produce a single binary will magically make those incompatibilities to disapear?

"the upstream maintainers can accidentally break those patches (e.g. if they are not personally testing on a given architecture)"

That can happen too with a single binary exactly the same way.

"It could even be the case that different architectures have different versions of the same packages, because the distro maintainers are busy trying to get everything to work."

Probably with a reason (like new version needs to be patched to work on this or that platform). How do you think going with a single binary will avoid that problem? It's arguably that in this situation you would end up worse. At least with different binaries you can take the decision of staying with foo 1.1 on arm but promote foo 1.2 on amd64 in the meantime; with a single binary it would mean foo 1.1 for everybody.

"I am not saying that this "universal binary" solution is the answer, but it might help streamline the build process at the distro level."

Still you didn't produce any argument about *how* it could help.

Re:convenient for _closed source_ software vendors

[Bert64]

Another option is for distributors to have compile farms, whereby they have an example of each architecture available to them, and it's a simple case of submitting a source package and it gets built automatically for each available architecture.
Also most patch breakage is changes which prevent the patch from applying, most build systems will apply arch specific patches even if you aren't building for that arch so breakage will be quickly noticed... A well written patch to fix a specific arch should not have detrimental effects on other architectures, and can quite easily be submitted upstream. I used to maintain a lot of Alpha and Sparc based linux systems, and would often submit patches upstream... A lot of those Alpha patches fixed up generic 64bit issues which meant that when amd64 came along those packages usually compiled cleanly right away.

Re:convenient for _closed source_ software vendors

[TorKlingberg]

It would be useful for specialty open source software that are not in the repositories (yet). Then users could install it without compiling from source or having to pick the right binary.

Is this really necessary? Or even advantageous?

[Tanuki64]

Most package manager can automatically create a binary package out of a source package. In many cases this resolves even problems with otherwise incompatible libraries. So for whom is such a fat binary advantageous? I'd assume mostly for closed source vendors. I have nothing against closed source in general, but if I pay for a software I expect at least a minimum of support. Such a fat binary does not look too userfriendly for me. Even if I can strip it down to my architecture. I suppose it does not solve the problems of incompatible libraries. I will follow the responses to this article, maybe I overlook someting and I will be convinced otherwise, but at the moment I would say: Superfluous.

Apple is still using it

[argent]

Apple is still using it for x86/x86_64 fat binaries in Snow Leopard.

Re:We really care

[argent]

Read the fine website:

Benefits:

[...]
        * You no longer need to have separate /lib, /lib32, and /lib64 trees.
        * Third party packagers no longer have to publish multiple .deb/.rpm/etc for different architectures. Installers like MojoSetup benefit, too.
[...]
        * Ship a single shared library that provides bindings for a scripting language and not have to worry about whether the scripting language itself is built for the same architecture as your bindings.
        * Ship web browser plugins that work out of the box with multiple platforms.
[...]
        * No more ia32 compatibility libraries! Even if your distro doesn't make a complete set of FatELF binaries available, they can still provide it for the handful of packages you need for 99% of 32-bit apps you want to run on a 64-bit system.
[...]

Re:Apple dropped it

[PenguSven]

Ask PPC owners that want to get the latest version of OS X.

No, Apple didn't drop support for Universal Binaries. Most apps available for Mac today are universal binaries and work on PPC or Intel macs, and in some cases support PPC 32, PPC 64, Intel 32 and Intel 64. Just because a new OS doesn't support an older CPU architecture doesn't mean the functionality for Universal or "Fat" binaries is not supported.

oh boy, just pack all archs on a .deb

[C0vardeAn0nim0]

you know, just trick the good ol' .DEB package format to include several archs, then let to dpkg decide wich binaries to extract.

is not that in linux the binaries are one big blob with binaries, libs, images, videos, heplfiles, etc. all ditributed in as a single "file" which is actualy a directory with metadata that the finder hides as being a "program file".

being able to copy a binary ELF from one box to another doesn't guarantee it'll work, specially if it's GUI apps that may require other support files, so fat binaries in linux would be simply a useless gimmick. either distribute fat .DEBs, or just do the Right Thing(tm): distribute the source.

Re:oh boy, just pack all archs on a .deb

[Nimey]

Instead of perpetuating a nasty hack, do a proper re-design of dpkg/.deb to support multi-arch.

Re:oh boy, just pack all archs on a .deb

[jipn4]

you know, just trick the good ol' .DEB package format to include several archs, then let to dpkg decide wich binaries to extract.

That's unnecessary, since the repository manager worries about architectures: it downloads the correct .deb file for your architecture. It's better than a "fat .deb" format.

Re:oh boy, just pack all archs on a .deb

[Rob Riggs]

That was my thought as well. Well, as a RedHat/Fedorda user, I thought you should be able to do this with RPM. But same idea. This is a problem best solved by a package manager and not a binary format.

However, after thinking about this further, it's a problem that doesn't need solving. I get all my binaries from repos that support all the architectures I use.

Not scalable

[gdshaw]

To a first approximation, the size of the binary will increase in proportion to the number of architectures supported.

This is something you might decide to ignore if you are only supporting two architectures. Debian Lenny supports twelve architectures, and I've lost count of how many the Linux kernel itself has been ported to. I really don't think this idea makes sense.

(Besides, what's wrong with simply shipping two or more binaries in the same package or tarball?)

Re:Not scalable

[mcfedr]

It doesn't exactly, because the data part of the binary is shared

Re:Not scalable

[evanbd]

To a first approximation, the size of the binary will increase in proportion to the number of architectures supported.

This is something you might decide to ignore if you are only supporting two architectures. Debian Lenny supports twelve architectures, and I've lost count of how many the Linux kernel itself has been ported to. I really don't think this idea makes sense.

(Besides, what's wrong with simply shipping two or more binaries in the same package or tarball?)

As mentioned by the other poster, data portions of the program are shared. In some cases, that means that data files are shared directly; multiple binaries, one data file. In other cases (libraries, etc) where the data is embedded in the binary, it simply means that the FatELF binary will compress to produce a combined file that's smaller than n * single architecture size. (The same is true for packing multiple binaries into one tarball, of course. Though in the case of several different files in one tarball, the FatELF version may have (slightly) better compression because it puts all the versions of one file right next to each other, rather than grouping all of one architecture together; that makes it more likely that the copies of the same data are within the same encoding block.)

The only thing wrong with shipping multiple binaries in one package or tarball is that, afaik, none of the major package managers support it. Sure, you could add support, but he decided this was a better approach.

In the case of something like Debian, it obviously doesn't make sense to have the package repositories use FatELF binaries, nor to include all possible architectures on one install CD / DVD. However, it might make sense to include a couple common architectures on a single iso that would work for most people, and have the obscure architectures get their own isos (or use jigdo) like they do now.

Re:Not scalable

[Hal_Porter]

Yup. I think the best way to do this would be to have a compiler that works in two passes - the first one would turn C into some intermediate format and would be run on the developer's machine, the second would turn that intermediate code into native code and then link it.

Of course for this to work there would need to be a NT 4.0 like similarity between the processor platforms. In the world of NT 4.0 all supported processors were 32 bit, all are little endian and all had the same alignment rules for structures. The same API existed across all platforms and all the types had the same sizes. Of course it is possible to force unaligned data, and in that case the Risc chips would fault, but NT 4.0 installed an alignment fixup handler to let the code run - albeit much more slowly - if they did this.

So Win16 code could be ported with a bit of work to Win32 and once ported it was trivial to cross compile. Still in practice no one bothered because 99.9% of the users had x86.

Still if the Risc chips had taken over, the NT 4.0 approach would probably have worked quite well. In fact now with x64 taking over this approach works quite well now. The current Windows API is cunningly defined to be portable between x64 and x86 you follow the rules you produce code which will build for x86 and x64. Your code will build for Itanium too - the only things that change size are pointer sized ones.

Now in the world of free software it's not like this - alignment, endianess structure packing rules and so on all vary. It's not standard on Unix to fixup alignment faults transparently. There are multiple library versions around on different distributions. So if you had an executable in some intermediate format there would be no way to know it would work once it was turned into native code on your system unless the developer had tested it. If he had tested it, he might was well give you the native binary for your system. If he hadn't you need to get the source code and port it.

Re:Not scalable

[Jesus_666]

Have the net-installer be fat. That way people can just download one ISO without having to worry about whether it's compatible with their system. The installer determines which architecture it runs on (optionally fine-tunable so you can install IA32 on an AMD64 system) and automagically fetches the correct packages from the server.

For regular installation CDs/DVDs this is a bit trickier but one could either include multiple package trees (space-intensive) or have all packages on the disc be fat and strip them down to the appropriate architecture(s) upon installation (less space-intensive but slower).

By all means, still offer architecture-specific installation media but do offer universal ISOs so people can just toss the OS on without having to worry about what architecture they have. Remember, desktop Linux is something considered worthwile and asking nontechnical users to decide whether they want installation media for IA32, AMD64 or maybe PPC64 is not beginner-friendly. Assuming that IA32 is correct might backfire, too, as ARM netbooks are beginning to appear on the market.

If Linux distros want to impress the casual user, "much easier to install than Windows 7" would be a good way to start.

Re:Not scalable

[gdshaw]

It doesn't exactly, because the data part of the binary is shared

Er, yes, that's why I said it was an approximation. However to address your point:

The data section might be shared (it can't be in the general case because of issues such as endianness). If it is then this might buy you 25% on the scale factor, but it doesn't change the fact that this technique is ultimately non-scalable.

Re:Not scalable

[99BottlesOfBeerInMyF]

The data section might be shared (it can't be in the general case because of issues such as endianness). If it is then this might buy you 25% on the scale factor, but it doesn't change the fact that this technique is ultimately non-scalable.

This varies greatly depending upon the application type. I just looked. For example, Firefox only shares about 4% of the application resources between binaries and the rest is compiled into each one and duplicated. Frontrow, on the other hand has about 90% of the application bundle made up of shared resources and only 10% is made up of the different binaries so you're only increasing the size of the application a few percent for every platform supported.

Given the fact that disk is so cheap and plentiful these days, I don't see that as much of an argument against a more flexible and robust system for cross platform software. In an absolute worse case scenario, you can always have a setting that prunes all the other binaries on install if you have a very disk constrained machine and all you've lost is a bit of bandwidth.

Re:Apple Universal Binary is kinda of a joke.

[TheRaven64]

You are confusing NeXT and Apple's approaches, I think. Apple puts both all of the different architectures in the same file. Your code is compiled twice, but it's only linked once. The PowerPC {32,64} and x86 {32,64} code all goes in different segments in the binary, but data is shared between all of them, so it takes less space than having 2-4 independent binary files. To support this on Linux would not require any changes to the kernel, only to the loader (which is a GNU project, and not actually part of Linux).

Re:Apple Universal Binary is kinda of a joke.

[Anonymous Coward]

OS X' universal applications are ONE SINGLE application, but the executable file itself inside the app - and there is only ONE executable, not two or more - contains code for all architectures. Let me repeat: ONE executable file with all architectures, not several executables. So you're wrong. It's a quite smart solution.

That said, OS X' universal files are pretty much on their way out, as Snow Leopard (10.6) doesn't play ball with PPC. As time goes on and people realize that x86 is a dead horse running, we might however see universal executables again, but then as ARM and x86.

Re:Apple dropped it

[Bert64]

It's also theoretically possible to include ARM support too, to make a binary that would also run on an iphone...

Not a particularly terrible idea...

[cfriedt]

This isn't a particularly bad idea and community-driven distros (or maybe the community of community-driven distros) like Ubuntu would probably benefit from it quite significantly. You can even strip unnecessary binary portions out of most programs (at least with the mach-o binary format), although it would really only effect disk usage. With the disk capacities today, it's really negligible disk-usage savings anyway.

In terms of the Linux kernel, this would mean a major overhaul for a large portion of the kernel and I can't see it being adopted very widely outside of the Desktop market.

Re:Apple Universal Binary is kinda of a joke.

[mcfedr]

That said, OS X' universal files are pretty much on their way out, as Snow Leopard (10.6) doesn't play ball with PPC. As time goes on and people realize that x86 is a dead horse running, we might however see universal executables again, but then as ARM and x86.

on mac it seems x86 is already a dead horse, snow leopard still uses universal binaries, they contain 32 and 64 bit code, rather than x86 and powerpc code that leopard had. some apps now contain up to 4 versions of the code, x86, x86_64, powerpc-32, powerpc-64

Universal Source ?

[obarthelemy]

I'm already amazed we have a universal x86 binary. With the architectural differences between an Atom and a Core7 or 9... I dare not think of all the inefficiencies this creates.

Wouldn't it be better to shoot for a Universal Source, with the install step integrating a compile+link step ? I know Gentoo does this, but Gentoo is marginal within the marginality that is Linux, on the desktop.

I'm amazed you can do real-time x86 emulation on non-x86 CPUs, but still can"t have a Universal Source.

Yet another unnecessary archive format

[bit01]

Hmmm, yet another unnecessary, incompatible, redundant archive format requiring yet more tools and libraries to deal with.

What's wrong with putting whatever flavor of binary you want in a tar.gz archive, zip archive or folder and have the system smart enough to pick the right one (using the existing, standard file id's and tools) when you execute the archive or folder? Yes, I realize the system needs to map pages while executing but for archives that is trivially dealt with by extracting before executing.

The amount of fuzzy, shallow, magical thinking that happens with software is just amazing. Please, if you insist on recreating the wheel at least have the good sense to think about what you are doing and stop assuming that giving something a new name necessitates creating an entire new software infrastructure that will unnecessarily create complexity and problems for large numbers of people.

---

For the copyright bargain to be valid all DRM'ed works should lose copyright.

Not a particularly useful one.

[WindBourne]

As somebody pointed out, this does not scale. In the end, if somebody is really wanting multiple systems, then they can simple create a dvd with multiple types on it. There is another solution though. Back in the late 80's/early 90's, the unix world was concerned about how a companies could create binaries for multiple systems. The best solution, though not implemented, was to compile to a universal binary, and then trans-compile this to a different arch. In light of all the work that has taken place on Java and now parrot, it may make far more sense to do that. With that approach, it would be possible to have various compilers go to a single arch, and then create a new back-end on compilers to take it to various archs.

Unix (OSF) tried it with ANDF

[Alain Williams]

Architecture Neutral Distribution Format was tried some 20 years ago. The idea was to have a binary that could be installed on any machine. From what I can remember it involved compiling to some intermediate form and when installed compilation to the target machine code was done.

It never really flew.

If someone wants to do this then something like Java would be good enough for many types of software. There will always be some things for which a binary tied to the specific target is all that would work; I think that it would be better to adopt something that works for most software rather than trying to achieve 100%.

Re:Unix (OSF) tried it with ANDF

[Abcd1234]

Uhh, how is a platform neutral bytecode format that's then compiled to native code at all similar to fat binaries? Oh right. It's not. It's a completely different solution to what is, admittedly, the same problem.

So, no they didn't "[try] it"... they tried something else entirely. OTOH, good job finding an excuse to show your geek creds by bringing up this little piece of esoterica.

Re:Unix (OSF) tried it with ANDF

[mikael]

It was called P-code

n the early 1980s, at least two operating systems achieved machine independence through extensive use of p-code. The Business Operating System (BOS) was a cross-platform operating system designed to run p-code programs exclusively. The UCSD p-System, developed at The University of California, San Diego, was a self-compiling and self-hosted operating system based on p-code optimized for generation by the Pascal programming language.

Re:Unix (OSF) tried it with ANDF

[Alain Williams]

Uhh, how is a platform neutral bytecode format that's then compiled to native code at all similar to fat binaries? Oh right. It's not. It's a completely different solution to what is, admittedly, the same problem.

One of the first things that I learned in IT is to try to provide the user with something that does what he wants to achieve, rather than what he asked for. You need to have an open mind, not just look at what is there before you.

Re:Unix (OSF) tried it with ANDF

[eggnoglatte]

That is not what the Mac does, though. On MacOS, you literally have the different platform binaries stored in a single file. For example:

# file /Applications/iMovie.app/Contents/MacOS/iMovie /Applications/iMovie.app/Contents/MacOS/iMovie: Mach-O universal binary with 2 architectures /Applications/iMovie.app/Contents/MacOS/iMovie (for architecture ppc): Mach-O executable ppc /Applications/iMovie.app/Contents/MacOS/iMovie (for architecture i386): Mach-O executable i386

Now, this may not sound super useful for a single machine, but it makes it so easy to share an Application folder on a common file server, for example. Just imagine - PowerPC, Intel 32, Intel 64, all sharing the same mount point and applications with the same path. Brilliant.

Re:Unix (OSF) tried it with ANDF

[FrankieBaby1986]

Dead on. Somebody please tag this article "JavaJIT"

Re:Unix (OSF) tried it with ANDF

[Abcd1234]

No, it wasn't. Unlike what was intended with ANDF binaries, USCD p-code was never compiled down to native machine code. Instead, you used a p-code interpreter to run it (the very wikipedia article you cite talks about p-code's speed penalty due to it being intepreted).

Re:Unix (OSF) tried it with ANDF

[lennier]

It almost seems like we could use LLVM for that nowadays. Hmm.

With open source code, no need for one binary.

[Cyberwasteland]

People have been asking for a universal binary or some sort of Universal Package Manager (UPM, that could work) for linux for ages. I think the main reason why there isn't one yet is the FLOSS nature of Linux. When the source code is open, there is no need for one single binary, it's up to the distro (or user) to make it into an fitting binary package. For that reason I think it wont change much if there was one single Binary. Also I've always thought that more applications should just use .bin files, compatible for all.

Less crazy than it looks

[Thad Zurich]

My initial thought was that this was insane -- Linux software should only be distributed as source and compiled on the target. Then I actually read the material and realized we were talking about distribution binary Linux installers. Since those pretty much have to be compiled before loading on the target, a multi-platform binary seems to make perfect sense in this context.

Re:Apple dropped it

[Nimey]

However, I noticed that Snow Leopard doesn't install Rosetta by default. At least this was so when I updated my work Macbook last week.

Seems a bit silly, since the installer claimed Rosetta took up only about 1.5MB of disk space.

Just what we need...

[Bloody Peasant]

While this comes at a cost of a larger binary file

Not to mention the greater ease for potential malware to work. Right now Linux is an extremely unfriendly and hostile environment for such malware. Why do we need to change that?

Re:Apple dropped it

[jimicus]

Wouldn't surprise me if this is to encourage users to demand a native x86 version of software - once every significant application exists as x86 binaries, Apple can drop support for Rosetta altogether and that's another developer or two freed up to work on furthering their products rather than backward compatibility.

He forgot the ARM, z10, m88k CPUs

[Colin Smith]

There may be more which should be included...

So... x86-64, x86, PowerPC, SPARC, MIPS, ARM, z10, m88k

How big are *your* binaries?

 

Re:He forgot the ARM, z10, m88k CPUs

[badkarmadayaccount]

Between 1.3x and 2x smaller than x86_32 binaries. LLVA FTW!

Re:We really care

[rhendershot]

* You no longer need to have separate /lib, /lib32, and /lib64 trees.

from what I gather he's not talking about system lib but application-specific trees. So for the majority of us using 64bit systems in a mixed world, only from those developers who chose to use FatELF would we derive any benefit. Even then the vast majority of dependant libs would still be required. And the compat libs. Skype anyone?

Over time it might prove beneficial. But over time the status quo will become 64bit. To me its a wash. Especially since I've experienced some times, admittedly fairly rare in the major repositories, but nonetheless times when a misbehaving application is pulled or broken in x86_64 but not i386 and I've been able to temporarily run the 32bit version. Since FatELF would, apparently, take that power away from me and defer it to systems-magic, well, as the shark's say; "for that reason, I'm out".

Less Fat, more Unreal

[iamspews]

This is cool and everything, but I'd rather Ryan spent the time on whatever it takes to get Linux Unreal 3 published.

Re:Apple dropped it

[peragrin]

Apple does that. when 10.3 came out apple stopped installing OS 9 classic by default as well. Support backwards compatibility for 2-3 generations and then phase it out. First phase is simply not installing it by default. Second phase is not to supply it. Snow leopard is the 3rd generation of OS after Rosetta came out. installed by default in tiger, and Leopard, they stopped installing it by default for 10.6.

personally I wish MSFT would do the same thing. I get really pissed when my "new application" requires the same installer that win95 had, and in order to run it I have to reboot into safe mode as my antivirus won't let it run. Seriously why does an Application built in 2009 still require the win16 subsystem to run? Why aren't the coders moveing onto new toolkits? Apple nudges and then pushes programers forward. MSFT let's them stay in the previous century and use bare metal knife switches to turn on the lights.

Re:Apple Universal Binary is kinda of a joke.

[agnosticnixie]

they contain 32 and 64 bit code

that's x86_32 and x86_64, although 32 bit x86 is apparently also soon to be a dead horse on mac (only the rev A intel macs can't handle it - they're 4 years old, so by 10.8-10.9 they will probably be dropped from the main support list, too), what AC means is that Apple will probably go with ARM more and more once they have a chip that compares to the current intel offerings and do 64 bit - for now I kinda doubt it, but some people like to predict things, like an apple tablet since the newton died.

Re:Apple dropped it

[MBCook]

They already have, to a degree. In Snow Leopard, you have to choose to specifically install Rosetta. If you don't, you can't run PPC programs.

If you try, OS X will prompt you to install Rosetta (which it will do at the press of a button), but it's not there any more by default.

Use the source, Luke!

[tomhudson]

You could use Mono. Of course if you built C for CLR that would not hide differences in libraries as far as I can tell.

Now why would I want to do anything that fucktarded, when I can just use the source?

And if I needed cross-platform that badly, I can always ship ONE java app with ONE instance of data.

The '90s called, they want their obsolete fat and universal binaries back.

Re:Use the source, Luke!

[Thuktun]

Now why would I want to do anything that fucktarded, when I can just use the source? And if I needed cross-platform that badly, I can always ship ONE java app with ONE instance of data. The '90s called, they want their obsolete fat and universal binaries back.

The elusive (+1 Insightful, -1 Flamebait) post makes a brief appearance, flashing its brightly-colored plumage, before disappearing back into the brush.

Re:Use the source, Luke!

[jedidiah]

Sometimes the insightful comment is necessarily flamebait.

You want "platform independence"? We already had that with source.

It just got FUD'ed to death because it seems kind of scary and there
were ever really any nice n00b friendly package mechanisms. Perhaps
Gentoo comes close, but I doubt it would fit the (n00b) bill.

Re:Use the source, Luke!

[jeremyp]

You want "platform independence"? We already had that with source.

No we don't. The Gnu autoconf system is testament to the fact that it is hard to write platform independent source code.

Re:Use the source, Luke!

[Blakey Rat]

You want "platform independence"? We already had that with source.\

Then what's that ./configure step for? None of that's in the source, right?

Re:Use the source, Luke!

[drsmithy]

You want "platform independence"? We already had that with source.

No, you didn't.

Re:Use the source, Luke!

[Anarchduke]

God forbid that anything involving linux would be friendly to n00bs. Why that might make linux popular or something.

I don't get it

[sjames]

What's the big benefit again? Instead of the package manager making the decision once at install time and all of the un-needed parts for platforms I'm not using stay on the install disk, now the decision is made each time I run the app and I get to clog my HD (or worse, my SSD) with all of them?

Now I can have the world's LARGEST hello world program with support for alpha, arm, avr32, blackfin, cris, frv, h8300, ia64, m32r, m68k, m68knommu, mips, parisc, powerpc, ppc, s390, sh, sh64, sparc, sparc64, um, v850, x86, and xtensa?

I'm guessing if this catches on, the most commonly used program will be 'diet' the program that slims down fat binaries by removing the architectures you will never encounter in a zillion years. (Just what are the odds that I will one day replace my workstation with an s390?)

If they want to do this, they should do it right and implement something like TIMI. Done well, it would mean that an app could run on a platform that didn't even exist when it was shipped (it worked for IBM).

Beyond the technical advantages of TIMI, it will provide us years of South Park references.

We need a new word to champion freedom for OSS

[pizzach]

It must sound as snazzy as Gratis. I vote for La Résistance as it will transform the image of us freedom fighters into that of cunning sophisticated individuals. The word is probably closer to the ideology of many of it's supporters, too.

*Runs away*

The x86 *is* a "virtual machine" these days

[Dogtanian]

We could do something like the old p-Code, where we compile to a virtual architecture, and then translate it to the machine during installation.

I assume you don't mean that p-Code itself was translated during installation (AFAIK it was translated on-the-fly during execution), just your modern equivalent.

I was sad to see we didn't have the compute power back then to make it fly. I bet we do now.

Possibly, but it would still likely give poorer performance. The benefits would have to outweigh that.

Also, in some respects, the x86 *is* a virtual machine that people are writing code for, albeit transparently for those who treat the CPU as a black box.

Reason being that all x86-compatible CPUs from the Pentium II/Pro onwards have been based around non-x86 cores (*). x86 code is translated on-the-fly by an outer layer before execution.

They probably had a good reason in that case. I assume that the x86 design- even by that stage- had acquired so much baggage that it was making it very difficult to proceed further with that architecture, and that even with the translation overheads, the non-x86 core made more sense in the long run.

In a perfect world, it would have been better still to be able to ditch x86 altogether, but for obvious reasons that wasn't going to happen here.

(*) I've heard them described as RISC-like, though someone else mentioned on Slashdot that in truth they weren't really *that* RISCy.

Re:Apple Universal Binary is kinda of a joke.

[RedK]

The binary is compiled twice. The way OS X packages its applications is the the Application that Icon that you click on isn't a File but a Folder with a predefined structure. So there there is a PPC and an intel port of the executable.

This is wrong. OS X has 1 binary file that contains both architectures in the App Bundle, not 2 executables :

$ ls -al firefox-bin
-rwxr-xr-x@ 1 user admin 42596 9 Sep 19:19 firefox-bin
$ file ./firefox-bin
./firefox-bin: Mach-O universal binary with 2 architectures
./firefox-bin (for architecture ppc): Mach-O executable ppc
./firefox-bin (for architecture i386): Mach-O executable i386
$ pwd
/Applications/Firefox.app/Contents/MacOS

Re:Apple Universal Binary is kinda of a joke.

[RedK]

That said, OS X' universal files are pretty much on their way out, as Snow Leopard (10.6) doesn't play ball with PPC. As time goes on and people realize that x86 is a dead horse running, we might however see universal executables again, but then as ARM and x86.

This is wrong also. Snow Leopard still makes good use of Universal binaries even without PPC being there. They can ship 32 bit and 64 bit in the same binary (i386 and x86_64). Universal binaries aren't going away.

$ ls -al iChat
-rwxr-xr-x 1 root wheel 5844848 29 Jul 01:28 iChat
$ file ./iChat
./iChat: Mach-O universal binary with 2 architectures
./iChat (for architecture x86_64): Mach-O 64-bit executable x86_64
./iChat (for architecture i386): Mach-O executable i386
$ pwd
/Applications/iChat.app/Contents/MacOS

Re:Apple dropped it

[BitZtream]

Considering how few PPC apps in use now days, it seems logical.

I've only been a mac user for a few months, but I've never seen a PPC binary, with the exception of the one 'hello world' universal binary I made just to see what would happen.

Distribute IR, compile dynamically

[Mike_K]

Why not write in Java or one of the Mono-supported languages and distribute something that will be dynamically compiled on the destination machine. It is fast, convenient and you don't have to have 10 OSs to target them....

m

Re:Distribute IR, compile dynamically

[badkarmadayaccount]

Register allocation isn't fun to do, even if you are a CPU. LLVM is a tad more efficient, I believe.

Why not make a universal archive and be done?

[Khyber]

How about we have a single chunk of data and executable code/libraries needed to run across any OS or any architecture? If you're going to call it universal you'd best damn-well make it universal! Take the idea of what OSX does for their apps and apply it to every OS possible? Even if the stuff has to be compressed to fit onto a single DVD you can have that decompress itself but still stay a single file (at least on the user end, the OS would still be able to read each file individually inside the archive.) For example, most games only need a different executable, the data (textures, maps, etc) stays the same.

You could even make it able to run on different distributions of the same kernel! No more stupid dependencies, no more RPM hell, and best of all there'd be no reason for it to just wipe out other installed software, such a beta versions of libraries and what not.

And then you could lock it down by making that file read-only. If there's an exploit, it won't destroy your software as it can't modify it, unless that exploit somehow allows write access.

Also...

[toby]

In OS X the Developer Tools that would be required are optional. (On Linux too, these days.)

Requiring apps to be built from source just wouldn't make sense for Apple's market.

Why not fat packages?

[Vellmont]

The underlying problem is that end users don't want to be bothered by having to know if they need the 64 bit version or the 32 bit version (or rarely some other platform).

We already have wildly successful packaging systems for linux that handle this class of problem rather well. So why not extend the packaging system to support multiple binaries in the same package? You'd certainly save on HD space. It also seems a bit cleaner.

please stop trying to turn Linux into OS X

[jipn4]

Linux doesn't need fat binaries because the package manager automatically installs the binaries that are appropriate for the machine.

OS X needs fat binaries because it doesn't have package management. I wish people would stop trying to bring OS X (mis)features over to Linux. If I wanted to use OS X, I'd already be using it.

Re:please stop trying to turn Linux into OS X

[Jeremy Erwin]

Linux doesn't need fat binaries because the package manager automatically installs the binaries that are appropriate for the machine.

So when you buy a game for Linux, or install a closed source binary, there's a magic "package manager" that works every time?

OS X needs fat binaries because it doesn't have package management.

Drag the application into the Applications Folder. To uninstall, reverse the process. Why must you complicate things?

Re:please stop trying to turn Linux into OS X

[99BottlesOfBeerInMyF]

Linux doesn't need fat binaries because the package manager automatically installs the binaries that are appropriate for the machine.

I wish. I installed a wikimedia server a few weeks ago. Gee it's OSS but not in the package manager. Well lets spend half an hour futzing with the CLI again. Oh, and commercial, closed source software needed to do my job, not in any repositories even if it is freeware because there's no easy way to host those repositories and download from a Web site across package managers. Again, I'm running binary installers and trying to pick from eight of them for different architectures. Gee it sure would be nice to install an application on a network drive and let the whole lab get to it from their laptops when they're in. Oh, but then I need to install multiple copies and hope users know what architecture they're using. Installation on a flash drive that I can stick into multiple machines, no luck there either. Want to upgrade my laptop to a new one with a new architecture, oh joy I get to re-download and dig up registration codes for all my applications that aren't in the repository because I can't transfer them because they don't support my new machine.

OS X needs fat binaries because it doesn't have package management.

Linux needs fat binaries both because their package management is insufficient and because they need it to support other use cases unrelated to package management.

I wish people would stop trying to bring OS X (mis)features over to Linux. If I wanted to use OS X, I'd already be using it.

I wish Linux devs would get over their Not Invented Here syndrome and start to copy the really cool stuff from OS X that I miss when I'm using my Linux machine. I wish Apple would copy some more of the stuff from Linux too, but at least they're steadily pulling in more and more and every release those deficiencies get smaller. A whole decade after OS X added system services and Linux still has nothing to fill that hole. That's just sad.

Re:please stop trying to turn Linux into OS X

[jipn4]

So when you buy a game for Linux, or install a closed source binary, there's a magic "package manager" that works every time?

Yes. The vendor's repository gets added to your repository list. Think App Store for the desktop, only better and without Apple's insidious control.

Drag the application into the Applications Folder. To uninstall, reverse the process. Why must you complicate things?

Why don't you ask all the companies (including Apple) that make you go through a wizard and install crap all over the system why they do it.

Re:please stop trying to turn Linux into OS X

[99BottlesOfBeerInMyF]

Yes. The vendor's repository gets added to your repository list. Think App Store for the desktop, only better and without Apple's insidious control.

First, last time I tried it, automatically adding a repository into a specific package manager was difficult or impossible depending upon the manager. Automatically adding a repository to an unknown Linux distro was too hard to try.

Second, aside from the defunct Loki, do any commercial games come distributed in a repository? (Even Loki only used it for demos, right?) I've only seen ones that ship with their own binary installers that bypass the package manager entirely.

Why don't you ask all the companies (including Apple) that make you go through a wizard and install crap all over the system why they do it.

For the most part, Apple doesn't use installation wizards, aside from on Windows where there are not really many other options. I'm a huge fan of package managers, but existing ones on Linux are so fragmented and lacking in features needed by closed source companies that they provide little of the advantage they could while adding complexity and making the interface less consistent.

Re:please stop trying to turn Linux into OS X

[linhux]

For the most part, Apple doesn't use installation wizards

While this used to be true, it isn't anymore (try installing iWorks, iLife, XCode ...). For complex software (say, you want to be able to run code post install, or install several packages at once), Apple recommends using PackageMaker to create pkg or mpkg bundles that are installed using the OS X Installer application. Also, OS X has package management that keeps track of what you have installed and what files belong to which application. See man pages pkgutil(1), installer(8) and packagemaker(1) for more information. Also, read Apple's Software Delivery Guide.

"application bundle" concept much older than OS X

[toby]

It has been part of the Mac System since the Mac was released in 1984; the existence of a resource fork made this possible in "Classic" MacOS. (But I can't remember exactly how Lisa OS does it.)

Somebody should tell this guy about ./configure

[dbc]

*sheesh* Just what we need. A way to distribute stale, broken, un-optimized binaries everywhere all at once.

Re:Somebody should tell this guy about ./configure

[Jeremy Erwin]

Calm down. Gentoo has almost finished building. In a few hours, you'll be able to use your machine again.

Re:Reversing decades of package management advance

[BitZtream]

Which OS does this happen on? Not Windows, Linux, OS X, *BSD ...

They ALL have DLL hell for closed source apps. If your solution is no closed source apps, then you live in a fantasy world.

Re:Apple dropped it

[Nimey]

That would be nice, but then someone might decide not to buy a newer version of Windows because their critical app only works on something ancient like Win98.

Plenty of those anyway, especially in low-volume markets like scientific equipment.

Rejoice though; with 64-bit Windows becoming more popular, crap old 16-bit, non-LFN installers from the mid-'90s will finally be going away.

Also, Macs apparently aren't immune to installer stupidity: Microsoft Office 2008 is fully Universal Binary, except for its installer, which is still PPC-only.

Re:We really care

[argent]

times when a misbehaving application is pulled or broken in x86_64 but not i386 and I've been able to temporarily run the 32bit version

And you can't build it from source because...?

Data Code

[benwaggoner]

And in fact, for large classes of interesting applications, installer and installed size is overwhelmingly data, not code. Games are going to be 95%+ data (check out how small the actual app is sometimes; often less than 1% the size of the data files). Microsoft Office has far, far more space allocated to fonts, clip art, all those multilingual spelling dictionaries, and templates than the actual *.exe files.

And even the self-contained .exe files in the above examples will also include a ton of bitmapped images for the GUI and such. Sure, command-line apps are going to have a lot more code, but even they will have the help text, tables, and other stuff that could be stored once in a platform-independent internal data structure.

Having behavior driven largely by data, not good is a good thing, of course. Data is a whole lot easier to debug than code, and bugs in data are generally much less catastrophic assuming the code itself does good validation.

GIM2, Webkit, Boost, KDE4 compiles?

[Ilgaz]

Compiling your own is always the best, especially when you get gcc etc. updates but here is the reality: Compiling some apps takes insane amount of time. Unless you really know what you are doing or your distro has support for special configurations, it has no advantage at all too. You just end up with the same binary.

I guess guy proposes it for "user level", general end user distros.

Re:Apple Universal Binary is kinda of a joke.

[IICV]

I don't understand why the OS would keep all those other files. The user's not very likely to take the hard drive out and stick it in a PowerPC box sometime in the future, so why not just delete all the files that aren't for your architecture?

In fact, why would you even download all those extra files in the first place? Wouldn't it be nice if you had some application that would selectively download just the files you need to install the software?

In fact, how is this at all better than the current package management solutions the various Linux distributions use?

Re:Linux is fine, but how about other platforms

[Jeremy Erwin]

Somebody didn't read the article...

Q: Do you have to read the entire FatELF file to load it?
A: Nope! Just a few bytes at the start, and then the specific ELF object we want is read directly. The other ELF objects in the file are ignored, so the disk bandwidth overhead is almost non-existent.

Q: So this...adds PowerPC support to my Intel box?
A: No. FatELF is not an emulator, it just glues ELF binaries together. If you have a FatELF binary with PowerPC and Intel records, then PowerPC and Intel boxes will pick the right one and do the right thing, and other platforms will refuse to load the binary, like they would anyway.

Q: Does this let me run 32-bit code on a 64-bit system or vice versa?
A: No. This doesn't let 32-bit and 64-bit code coexist, it just lets them both reside in the file so the platform can choose 32 or 64 bits as necessary.

Q: Do I need to have PowerPC (MIPS, ARM, whatever) support in my FatELF file?
A: No. Put whatever you want in there. The most popular scenario will probably be x86 plus x86_64, to aid in transition to 64-bit systems.

Fat packages or FatELF, Qemu and x86 on ARM netboo

[caseih]

One use case I can see for fat packages or FatELF (either one works) would be if we also had a smart boot loader that could take a single-architecture binary and run it through QEMU in conjunction with FatELF libraries on the system. If my ARM netbook ran Fedora, example, and installed FatELF libraries supporting both ARM and x86 (supposing cheap SSDs!) then if I really needed Adobe Reader for some reason which is only on x86, then I could download and install their Fedora x86 RPM and run it seamlessly.

This could also be done with plain fat packages too. A fat gtk-libs package, for example could dump files native to your architecture in /usr/lib or /usr/lib64 or both, while putting other architecture libraries in the Qemu system directory for that architecture.

Note that Qemu already allows this without fat libraries. But wouldn't expect a average netbook user to know how to download the rpm or tarball, extract everything and stick it somewhere, then populate the Qemu system directory with Fedora x86 libraries. The benefit of FatELF or fat packages here would be that the RPM would just install because it could see that the requisite FatELF libraries where installed or that the architecture-specific libs were installed in the Qemu system. This is all provided that RPM or DEB or whatever either knows about FatELF or implements fat packages.

I was curious

[ratboy666]

So I decided to have a look at my netbook -- an Acer Aspire One, running the Acer Linpus Linux system. It's the 512MB memory with 120GB disk model; not the SSD dress.

[user@ariel ~]$ find /usr /sbin /bin /lib -type f -print0 | xargs -0 file | grep ELF | sed -e "s/:.*$//" >list
[user@ariel ~]$ wc list
  12318 12318 455036 list
[user@ariel ~]$ ls -l `cat list` | awk -F " *" "{ sum += \$5 }; END { print sum }"
2090169634

For the "non-unix" types, this means -- traverse four directories (/usr /sbin /bin /lib) that contain all the binaries (well, I did forget /boot, but that's not very important for this discussion). We extract those files that mention ELF in the file-type, count them, and then sum their sizes.

Note that there are ~12,000 objects that would be expanded, taking up around 2GB of space. Which means that adding an alternate "fat" binary would roughly double this to 4GB. In other words, not very good for an SSD dress, but reasonable for a hard disk dress (my total data usage is 30GB on this netbook, so we would add 10% to the utilization).

On the other hand, this is storage resource that would not ever be reclaimed. There are other ways to achieve this result. For example -- simply provide multiple ELF objects in an archive, including a "bintester". Iterate each folder in the archive, test-running the bintester. If the bintester succeeds, use that. If none succeed, fail if its "closed object/source", or attempt to download and build the source. An example of this type of function is the VMware driver loader.

Or, use a neutral distribution -- JAVA, or Squeak, or the Microsoft CLR.

All of these provide similar results, without the need to modify the object loader. Which is something that I would really frown about -- After all, the loader is a very heavily used, very security sensitive component.

As to "fatness"; it turns out not to be as big an issue as I had originally feared (I did the measurement on the "worst-case" system I use, and started with the mind-set that it would matter a whole lot more than it turned out... my preconception was wrong).

Your mileage may, of course, vary.

PS. For full disclosure, you may note that I excluded /opt -- I have 3.9G of stuff there: OpenOffice.org3, Intel C 11.0, Adobe Reader, Microsoft MSVC 7, and some smaller stuff (tcl/tk dev, hua wei 220 support, and a virtual tape library . Since this may or may not impact a "standard" distribution, I decided to not include it (and, MSVC 7 is COFF format, used under WINE anyway).

Sounds unfortunate

[CarpetShark]

I find it a benefit of Linux if there is only one instance of a XML library in my memory, though.

Really? If using Linux lead to an XML binary in my memory, I'd want it out of there ASAP. Perhaps you meant on your disk ;)

Java

[CarpetShark]

However, the idea of a universal binary is cool. We could do something like the old p-Code, where we compile to a virtual architecture, and then translate it to the machine during installation.

Kind of like Java then.

repositories and app stores

[jipn4]

The way you should install non-free applications on Linux is to add the company's repository to your software sources. That way, you get automatic selection of the right binary and automatic updates. Some companies already support that.

Long term, I expect that SuSE, Ubuntu/Canonical, and RedHat will be offering app stores that make selling and buying commercial Linux apps as easy as iPhone apps.

Fat binaries are a throwback to the past; they aren't needed in the era of app stores and package management.

Re:repositories and app stores

[99BottlesOfBeerInMyF]

The way you should install non-free applications on Linux is to add the company's repository to your software sources. That way, you get automatic selection of the right binary and automatic updates. Some companies already support that.

Very few companies support that so far as I've seen and for good reason. First, different versions of Linux use different package managers, most with incompatible package formats and repository specifications. So if a developer wants to target "Linux" they need to host multiple repositories. Second, even if they put software in a repository for downloads and updates, none of the package managers to date support licensing and registration and sales, so they have to run a separate service for that task and most buy that service prepackaged from a company that also offers updates from within the app that will work on all Linux distros. Thirdly, since Linux distros do not yet offer a "store" for commercial apps, most users are discovering and buying from a Web page, and it's not easy to make a single link on a Web page that will add the right repository to the right package manager for a given architecture, so they end up providing a binary installer to figure it out on the machine and which also comes with the registration and update stuff from a third party.

Long term, I expect that SuSE, Ubuntu/Canonical, and RedHat will be offering app stores that make selling and buying commercial Linux apps as easy as iPhone apps.

Canonical has announced their intention to include such a store in their new package manager eventually, and it may relieve many of the problems listed if they do it well enough and make it attractive and cost effective for developers.

Fat binaries are a throwback to the past; they aren't needed in the era of app stores and package management.

I disagree. They're still useful for running apps from network and removable drives. They're still useful for installation from older computers when doing hardware upgrades. They're still useful for filling in the holes where package managers have not yet caught up. Until you can buy a new computer and automagically get all the same software installed from your old computer as well as registered and licenses transferred to the new machine, fat binaries will be useful... and maybe even thereafter for the use cases I described.

Also, I really like having the resources in a folder contained within an app, so it is easy to extract images or movies or audio files directly from a known location without any special tools, so separation of resources and binaries ala OS X is very attractive.

Re:repositories and app stores

[jipn4]

First, different versions of Linux use different package managers, most with incompatible package formats and repository specifications ...

Fat binaries address none of those problems.

I disagree. They're still useful for running apps from network and removable drives.

Well, and Linux supports them in numerous ways. You can pack any combination of binaries, shared libraries, and resources into any of a number of types of bundles. Unlike the Mac, you can even put all of that into a single file. There are tools available for doing that in numerous different ways.

All of that is rarely used because it's just not very useful.

Re:repositories and app stores

[99BottlesOfBeerInMyF]

First, different versions of Linux use different package managers, most with incompatible package formats and repository specifications ...

Fat binaries address none of those problems.

No, but that does speak to your assertion as to how developers of closed source software should be packaging their software for Linux. It explains why that is not a good choice for them and what needs to be fixed if that's the model that is to be used going forward.

I disagree. They're still useful for running apps from network and removable drives.

Well, and Linux supports them in numerous ways. You can pack any combination of binaries, shared libraries, and resources into any of a number of types of bundles.

What you mean is, you can create special versions of software that will work from a network drive or flash drive but no one does because it is hard. Since all the people who care about that functionality have moved to OS X anyway, Linux devs should just concede those users and give up on making applications as functional in their default form. Just because a function is not implemented in Linux apps does not mean the function is not in demand, it often just means the function is too hard to implement for the benefit it brings.

In the mean time users on OS X can and do IM full applications to other users on different platforms and the application "just works" while they shake their heads at those of us on Linux suffering while trying to accomplish the same task.

Re:repositories and app stores

[jipn4]

It explains why that is not a good choice for them and what needs to be fixed if that's the model that is to be used going forward.

Creating a standard package repository is a hell of a lot easier than trying to create software bundles that work across all Linux systems.

What you mean is, you can create special versions of software that will work from a network drive or flash drive but no one does because it is hard.

No, what I mean is that you can already do what a fat binary does; introducing a new format doesn't make it any easier or harder.

Just because a function is not implemented in Linux apps

OS X's simplistic approach to packaging is not a "function", it's a lack of a function. By restricting its world mostly to a handful of computer models and to simple desktop applications, Apple can get away with a simplistic approach to packaging... most of the time. Of course, many applications still need installers and then lack a good uninstaller.

In the mean time users on OS X can and do IM full applications to other users on different platforms and the application "just works" while they shake their heads at those of us on Linux suffering while trying to accomplish the same task.

The fact that you can do that is an illusion created by the Finder, which bundles up trees of files behind your back; in reality, OS X applications a complicated collection of files in a directory tree. The Gnome and KDE desktops could easily create the same illusion for Linux apps; that doesn't require a new format. I'm glad they don't because it's a misfeature that should be banished because it's more trouble than it's worth.

It's safer and easier to just say "You want program XYZ? Just use this URL."

VM

[shyisc]

Isn't this kind of thing what VMs are for? The kernel could have a VM, and run all binary executables that are compiled for the VM inside the VM. That way you can get real compile-once-run-everywhere, and with no bloating of the size of executables. It should then also be possible to port the VM to other OSs.

ELF does this already, doesn't it?

[MostAwesomeDude]

Talking to a couple guys learning ELF and Linux' SO loader, I got the impression that ELF already supports these multiple code segments, and that adding in headers to denote the arch is really simple. I'll see if I can get more evidence for this.

I'm not here for the lecture

[westlake]

Or maybe it's proprietary. Some people are willing to compromise on freedom.

I would define my freedom as a user as the freedom to install any app under any license without someone looking over my shoulder.

OSX makes this easy. Windows makes this easy. Linux makes this hard.

Dynamic binary translation is a better method.

[non-e-moose]

While fat binaries are one approach to run applications which are binary-only on Linux, a much better way is to use binary translation. A fat-binary approach would require application vendors to qualify both versions, which means approximately twice the cost. Translators can be developed by 3rd parties. There are a lot of commercial-grade binary translators and binary optimizers that have shipped over the years. Tandem, Digital, Transmeta, Transitive, etc. The messy parts are getting the OS conversion semantics correct when the source and target OS's are not very similar. Instruction decode can be a bit tricky, but it is not the development bottleneck.

Re:Apple dropped it

[jimicus]

I've only been a mac user for a few months, but I've never seen a PPC binary, with the exception of the one 'hello world' universal binary I made just to see what would happen.

That's largely because XCode was producing universal binaries several months before Apple actually announced what a universal binary was or why a developer might wish to produce one.

About the only PPC thing I have is the software for my Epson scanner - they haven't produced a Universal driver nor are they likely to seeing as it was discontinued a few years ago. The day I move to a version of OS X without Rosetta support I either need to use something other than Epson's software or buy a new scanner.

It's not a bad idea, actually.. or is it?

[tjstork]

The benefit would be that it would make data more transportable at first blush. The problem is clear enough : The whole process of installing an operating system and shipping data to it is a huge waste. Being able to take a drive and transplant it into a newer machine without having to re-install anything is an absolute time saver. I'm still loving Linux that it let me do that without too much fallout, but why couldn't I take a brick and put it on a faster processor, or a lower power processor, or in a friend's virtual machine, or anything... I don't need to be married to CPU architecture...

And that's really where it all falls apart, because you can't possibly ship a computer that has every CPU architecture in every binary.. But maybe you could have a bootstrapper / kernel that always gets all of the possible CPUs, just in case, for enough to be able to boot itself, mount its own file system and get to a network. Then, the operating system would replace the rest of the binaries with new versions, as part of your transplant process, and your computer would just work.

Re:Fat-Source Bundles

[IntergalacticWalrus]

Great... except having to compile an app to install it is a fucking waste of the user's time. And the performance gains from CPU specific compile flags are insignificant. Why is this a better idea than fat binaries again?

Are you a Gentoo user? I used to be a Gentoo user too. Then I actually became sane.

What's with the hate?

[IntergalacticWalrus]

Fat binaries are awesome, they're what makes Mac OS X the mainstream operating system with the most painless 64-bit transition. Whereas Windows and Linux use messy hacks to allow 32-bit apps to live in a 64-bit environment, in OS X it "just works" because all libraries are multi-architecture in a completely transparent way thanks to fat binaries. It also made the transition from PPC to x86 relatively easy, too.

I feel sorry for people who believe Linux doesn't need fat binaries. They don't understand all the advantages this system brings (and not just to non-free software).

Actually yes Linux needs a universal format

[Orion Blastar]

Linux needs to become more like Mac OSX than Windows.

What I would like to see in Linux in the near future:

Universal file format for X86, X64, and PowerPC executiables that replaces the ELF format (WIZARD format, ELF needs food badly!)

GNOME and KDE merged into one GUI that emulates both of them, GNIGHT or something.

Ability for Linux to use Windows based drivers when Linux based drivers do not exist, something better than that NDISwrapper but under a GPL license and built into Linux.

GNUStep being developed into something that resembles Aqua, Aero, and other GUIs and is backward compatible with the Mac OSX API calls to recompile OSX programs for Linux. Maybe even in the near future run OSX Universal binaries somewhat like WINE runs Windows programs.

Re:Actually yes Linux needs a universal format

[lordandmaker]

GNOME and KDE merged into one GUI that emulates both of them, GNIGHT or something. /blockquote? Why? And how would that work? They're far too different for anyone who particularly likes one to get along with the bastard lovechild of the two, so you'd just end up with two quite popular DEs, and something that tries to be both and mostly fails at it.

Why?

[Hurricane78]

Just because you can, doesn't mean you should.
I think for open software, that does not follow "traditional" models of software distribution, it's a pointless waste of resources.

Also, it won't affect me anyway, as I'm compiling everything from sources. (Gentoo)

Bandwidth is not unlimited

[tepples]

Given enough disc space, there's no reason you couldn't have one DVD .iso

I don't want a DVD image, which would burn through the vast majority of a 5 GB/month transfer allowance. I want a CD image, which burns through one-seventh of that.

the otherwise unchanged hundreds of megabytes of data.

How is byte-swapped data "otherwise unchanged"?

One hard drive partition can be booted on different machines with different CPU architectures, for development and experimentation.

Different machines likely have different layouts for the master boot record.

Re:We really care

[Captain Segfault]

And you can't build it from source because...?

The bug is in the source?

Exchange rate imbalances

[tepples]

Yes, Sketsa is commercial, but the price is well worth the difference.

Not to people who live in a country with an undervalued currency. Look at the blue countries on this map: an hour of labor is worth fewer U.S. dollars than it would be in yellow or orange countries like USA, Canada, Japan, and most countries of western Europe.

Forgot to link the map

[tepples]

Purchasing power parity, map at top of article

Not everything is in Add/Remove

[tepples]

I meant exactly what I said. "Add/Remove Applications" is pretty usable UI that, as far as I can see, has none of the problems you refer to.

There are plenty of applications that I would want to use that aren't in Add/Remove. The examples that I've seen fall into three categories:

• Free applications in the repository that show up in Synaptic but for some reason not in Add/Remove. These include IDLE, the standard editor for Python programs.
• Free applications not in the repository because they are less popular. These include CA65, a toolchain for 6502 assembly language that's popular on the NES homebrew forum at nesdev.com. (This particular one has a needs-packaging request lingering in Launchpad.)
• Non-free applications, especially those not licensed for redistribution. These will continue to exist until someone can figure out how to fund the development of higher quality video games than the major labels without restricting distribution.

Re:Apple dropped it

[drsmithy]

personally I wish MSFT would do the same thing.

They do. The difference is that a "generation" to Apple is on the order of 12-18 months, whereas to Microsoft it's more like 7-8 *years*.

MSFT let's them stay in the previous century and use bare metal knife switches to turn on the lights.

So do most enterprise-level vendors - but Apple is far more interested in forced upgrades and milking its customers.

Re:We really care

[argent]

The bug is in the source but not one of the only two binary packages (x86 and x86_64) you have?

Or you mean they've updated the x86_64 version to the newer source with the bug, but not the x86 version?

* You can't find an older version?

* You can't build from an older branch?

Or you mean they've got a 64-bit bug?

* You can't make a 32-bit build yourself?

This seems like a really obscure failure mode to me, and if this particular workaround was removed there are plenty of others available.

It appears you don't play video games

[tepples]

I do not propose anything. Why should I?

You propose no solution for distribution of non-free software. You propose no solution for funding development of free video game software. Therefore, you appear to propose the elimination of the video game industry. A lot of Slashdot users who like to play video games disagree with your proposal.

Re:It appears you don't play video games

[turbidostato]

"You propose no solution for distribution of non-free software."

No, I don't. Not at least here and now. I don't make bussiness on the distribution of non-free software so why I should? The day I need or care about distributing non-free software is the day I'll look for solutions to that problem. Meanwhile I have higher priorities to care about (like posting on Slashdot).

"You propose no solution for funding development of free video game software."

I neither develop nor use video game software, be it free or closed source. Therefore, why should I?

"Therefore, you appear to propose the elimination of the video game industry."

Unstated conclussion. On my previous message I didn't offer solutions to end famine in Africa either; am I proposing all those people should be anihilated? No. The fact I don't propose solutions about how should closed source bussiness be managed only states that: that I didn't offer such solutions. It specifically doesn't state I propose to eliminate them. One thing is that they don't get my simpathy nor my free effort looking for solutions for their problems, a different one that I try to eliminate them.

"A lot of Slashdot users who like to play video games disagree with your proposal."

Therefore is up to *them* to seek/offer your asked for solutions, not me. On the other hand I did *not* propose anything (see your own opening paragraph) so what's exactly those lot of Slashdot users disagree with?

Re:It appears you don't play video games

[swillden]

I neither develop nor use video game software

Satisfied. I'll just assume that "I don't care about non-free software" posts come with an unwritten "I play no video games."

Too strong an assumption. There are lots of Free video games. Some of them are quite good, even.

Good intentions, but a waste of time.

[keatonguy]

I don't think this is really relevant in an OSS-based platform. Most apps you get for Linux that aren't distributed through your package manager are in source, allowing you to run it through the compiler for whatever architecture you happen to be using, which makes having multi-architecture binaries a moot point. Which is not to say that multi-architecture support is a BAD thing, of course, just that putting it all in one binary is the wrong approach to take.

I don't know if the average repository stores packages in a particularly wide variety of architectures, but it seems logical to me that that's the place where you put in universal support if it isn't that way already.

I got the remedy for that

[jonaskoelker]

I got the remedy for that

aptitude install gnome-theme-aqua; gnome-terminal

Then, in gnome-terminal

aptitude install [program...]

Re:Apple Universal Binary is kinda of a joke.

[dkf]

I don't understand why the OS would keep all those other files. The user's not very likely to take the hard drive out and stick it in a PowerPC box sometime in the future, so why not just delete all the files that aren't for your architecture?

Turn it around. Why delete them? Disk space is really cheap and is getting cheaper all the time.

In fact, why would you even download all those extra files in the first place? Wouldn't it be nice if you had some application that would selectively download just the files you need to install the software?

A few points. Firstly, bandwidth is really cheap. Secondly, the space consumed by the binaries is a small proportion of the whole for most third-party apps. Thirdly, selectively downloading bits and pieces is far more likely to go wrong in hard to debug ways than giving everyone the exact same package.

In fact, how is this at all better than the current package management solutions the various Linux distributions use?

You only ever want software provided by your distribution maker? How... short-sighted. It would be far better if there was a thriving third-party vendor market, producing (and probably selling) things like specialist applications and games, and for people in that sort of situation, having the distro makers as sole gateways is wholly unacceptable; never ever going to happen.

Enabling third-party apps is an important step for being more than just a niche OS. Simplifying the process of delivering those apps is an important part of that enablement. Fat binaries makes that delivery simpler. QED.

Re:Apple Universal Binary is kinda of a joke.

[maccodemonkey]

"You are confusing NeXT and Apple's approaches, I think. Apple puts both all of the different architectures in the same file. Your code is compiled twice, but it's only linked once. The PowerPC {32,64} and x86 {32,64} code all goes in different segments in the binary, but data is shared between all of them, so it takes less space than having 2-4 independent binary files." Actually, this isn't true. The code is compiled four times (for a 32/64 bit unibin), linked four times, and then all four executables are stitched together into one executable file. With regards to disk space, Apple thought ahead, and the format supports stripping out of versions of the binary you don't want to keep around. For example, some Intel owners run tools on their machines that strip out the PPC versions of binaries to preserve disk space. Some PowerPC owners strip out the Intel versions of the binaries, but then usually run into trouble when they try to migrate their disk to an Intel machine. :) Stripping out a version of the binary will even keep a signed binary valid. FYI: Apple has a very similar sort of setup for language dependent resources.

Re:We really care

[rhendershot]

And you can't build it from source because...?

that's not the issue. The whole point of FatELF is to deliver binaries. I may have missed it in his site project description and FAQ but I was left thinking that I'd not be able to similarly substitute 32bit app as I now do.

SOHO User

[Vertana]

How many non technical people know the difference between 64 and 32-bit CPU's? Not a whole lot, if any. For those people, it's a bit daunting when their Linux friend/son/granddaughter/whatever tells them it'd be a great choice and the first thing they see is "Which download? 32 or 64?". Just put the .ISO with both 32-bit and 64-bit with a script that tests if the CPU is 64-bit capable at boot time. Or how about those people that don't know if their Mac has Intel or PPC chipset? Same thing, script at boot time to determine. If it's just two architectures the overhead could be well worth it.

Re:SOHO User

[Mystra_x64]

If you, say, write it down on boot everyone will know. Until then provide simple how-to's. Besides we are talking about Linux so it shouldn't be much of a problem.

This won't fix installation media. Also: LLVM?

[TD-Linux]

First off, I would like to point out that I don't support the idea of making binaries universal by packing them in the same .elf. Sure, it would be neat to have a DVD that can install to different architectures, it's currently not possible because the bootloader _has_ to be native. You'd probably end up with a bunch of boot floppies for a single installation media. In addition, is there any difference to having a package for each architecture on this DVD rather than universal packages? With separate packages the same amount of DVD space is consumed but far less is consumed on the target hard disk.

In addition, this isn't going to make companies support uncommon architectures any faster. Maintaining multiple packages is really easy, keeping the code working across different architectures is not so hard. I think it's mostly programmer laziness not bothering to compile other architecture packages - and there is no reason fatELF is going to decrease this laziness.

If you _really_ want architecture independence (which universal binaries don't really provide, they still only support architectures that the author had in mind), you'll need a recompiler of some sort. LLVM is designed to be suited to this task, so why not use it? Apple already does. Yes, I know it's a bit slower, but a small price to pay if you need your app to run on every system under the sun.

Of course, there is always the option of web apps...

Re:This won't fix installation media. Also: LLVM?

[Samurai Crow]

Mod parent up!

Re:Apple Universal Binary is kinda of a joke.

[YesIAmAScript]

file `which gcc` /usr/bin/gcc: Mach-O universal binary with 3 architectures /usr/bin/gcc (for architecture i386): Mach-O executable i386 /usr/bin/gcc (for architecture ppc7400): Mach-O executable ppc /usr/bin/gcc (for architecture x86_64): Mach-O 64-bit executable x86_64

And I wouldn't be shocked to see Apple package ARM Mach-Os into some files soon too, with this pseudo-netbook thing rumored.

Re:We really care

[argent]

that's not the issue. The whole point of FatELF is to deliver binaries.

True. It's to deliver binaries. It's not to work around a bug in some specific build of a binary. That's what building from source is for. However.

I may have missed it in his site project description and FAQ but I was left thinking that I'd not be able to similarly substitute 32bit app as I now do.

You must have. Actually, there's two different ways of doing it:

fatelf-split my_fatelf_binary

and:

fatelf-extract my_elf_binary my_fatelf_binary x86_64

Re:We really care

[rhendershot]

Actually, there's two different ways of doing it:

fair enough. technically possible.

A similar scenario is being able to select which I run, 32 or 64 bit. I did that for awhile with Firefox both to compare their general performance and to try things on one first as a kind of experimental environment. The reasons are really immaterial. I don't see a palatable way to do this with FatELF though. Another edge scenario, admittedly.

Linux, plural.

[Bent Spoke]

The vast majority of all desktop computers out there are x86. The vast majority of Linux desktops, ditto.

That said, currently we can't even easily produce binaries that run on multiple distributions for just the x86 .

Generally speaking, unlike Windows/OSX, you can not just get "the Linux Desktop Version" of a program. There is no Linux desktop. There is a plethora of them. For better or worse, the issues and complexity that arise from this plurality are due to the lack of central management and control.

It's called JAVA

[Anonymous Coward]

I thought this was the whole point of Java. Why are we going back and trying to solve a problem that was solved FOURTEEN YEARS AGO

Re:Apple dropped it

[Guy Harris]

It's also theoretically possible to include ARM support too, to make a binary that would also run on an iphone...

...as long as you don't use such minor unimportant frameworks as AppKit (no AppKit on iPhone OS; it's UIKit instead). You could make a fat command-line binary, I guess....

Re:Apple dropped it

[Guy Harris]

Considering how few PPC apps in use now days, it seems logical.

I've only been a mac user for a few months, but I've never seen a PPC binary, with the exception of the one 'hello world' universal binary I made just to see what would happen.

Never run Quicken for Mac, I guess (except perhaps for a pre-release of Quicken Life for Mac^W^W^W^WQuicken 2010 for Mac):

$ file /Applications/Quicken\ 2007/Quicken\ 2007/Contents/MacOS/Quicken\ 2007
/Applications/Quicken 2007/Quicken 2007/Contents/MacOS/Quicken 2007: header for PowerPC PEF executable

Not only is it not fat, it's not even Mach-O....

Re:Apple Universal Binary is kinda of a joke.

[Guy Harris]

You are confusing NeXT and Apple's approaches, I think.

You should think differently. NeXTStEP introduced a fat binary scheme to allow a single executable file to contain binaries for 68k NeXT boxes and x86 PC's running NeXTStEP. OS X picked up that scheme from NeXTStEP, so NeXT's and Apple's approaches are the same.

Your code is compiled twice, but it's only linked once.

No, it's compiled N times, and linked N times, once for each instruction set architecture.

The PowerPC {32,64} and x86 {32,64} code all goes in different segments in the binary, but data is shared between all of them, so it takes less space than having 2-4 independent binary files.

If by "data" you mean data in files under, say, Resources in the app bundle, yes, it can be shared. If by "data" you mean the data segments in the executable, no, it's not shared - a fat binary is just a bunch of Mach-O binaries with a special header wrapped around them, and each Mach-O binary has a full set of text/data/etc. segments.

To support this on Linux would not require any changes to the kernel, only to the loader (which is a GNU project, and not actually part of Linux).

If by "this" you mean something that looks like OS X fat binaries, you would have to change the kernel to understand fat binary files, running the appropriate executable within that file.

Re:Apple Universal Binary is kinda of a joke.

[Guy Harris]

If you know what you're doing, you can actually extract the Mach-O binaries from the fat binary and distribute them separately (I've done it with a hex editor, but I'm sure there are plenty of other ways to do it).

Yes, if you know what you're doing, you know how to use the lipo command to do it. :-)

Revolutionary suggestion

[petrus4]

In most cases, with Linux, we've got source code.

So if we want to run programs on a new architecture, maybe we could *gasp* compile from source?

No.

[dmsuperman]

Ryan Gordon needs to mind his own business. Keep your binaries and shit in OS X, we have source code to compile from which means that we can run it on any platform without requiring larger binaries, thanks!

Re:Apple Universal Binary is kinda of a joke.

[Captain Tripps]

f by "data" you mean data in files under, say, Resources in the app bundle, yes, it can be shared. If by "data" you mean the data segments in the executable, no, it's not shared - a fat binary is just a bunch of Mach-O binaries with a special header wrapped around them, and each Mach-O binary has a full set of text/data/etc. segments.

That's how I used to think it worked too, but then I had to modify a universal binary by hand one day. Turns out that the compiler tools are smarter than that.

My recollection is that the layout is something like this (for a PPC32/x86 binary):

• 1. Universal header
• 2. PPC32 Mach-O header
• 3. PowerPC code segment
• 4. x86 Mach-O header
• 5. x86 code segment
• 6. Data segments

Both architectures get positive offsets to refer their data, so they work just fine. Sure the PowerPC Mach-O "file" has a bunch of useless crap in the middle of it, but the Mach-O loader doesn't notice since those byte ranges aren't mentioned in the PowerPC header. Pretty clever.

You can confirm this by running strings(1) on a Universal binary and seeing that there's only one copy of the string constants.

Why?

[BumpyCarrot]

Binaries are only ever useful when compiled for the user's distro. Given that this usually happens at package level (at least for distros that even bother with package management), and that those packages are often platform-specific, I don't see what problem this would solve. Perhaps if there was a distro that was definitively "Linux" that had the userbase to support it. But then any candidate for such a role already has a package management system.

To clarify

[Jim Hall]

Okay, a bunch of you have responded with "Check out DEB and RPM". I already know about package management, thanks.

But if you look at an app package on MacOSX, you'll see it's presented as a single icon or "file" ... but use the command line, and you'll see it's just a directory. A directory with a special structure, so the system knows that it is really an application. Kind of cool. I think we need something like it on Linux, but I am not aware of such a thing. It would probably be some kind of virtual folder, implemented in GNOME or KDE. It's really just something like a tarball with a particular directory structure, maybe with a special extension, and GNOME/KDE knows it really should be presented to the user as a single object, using an icon located at a particular place within the tarball, with a special name.

Do I see it as the new way to release versions of Firefox, or GIMP? No. For that, use RPM or DEB for package management that's integrated with the rest of the operating system, tracking dependencies on shared libraries, etc.

However, I think this would be a great solution for third-party applications that aren't going to be part of a mainline distro - assume statically linked.

For example: commercial games. Download the latest version of "Sports Franchise 2009", put it in your "Games" directory or wherever you keep games, and just click it's icon in GNOME or KDE to launch it.

Mono?

[GWBasic]

Uhm, doesn't mono already give Linux this? All they need to do is finish gcc's support for CLR binaries and the problem is solved. (CLR / mono can call into the underlying platform's C API.)