Slashdot Mirror
Building Distributable Linux Binaries?
Grubby Games asks: "I make games for a living, and I want to ensure that my games will run on as many Linux distros as possible. However, since I distribute binary game executables, the programs often fail to run on certain distros because of missing dependencies, and so forth. So, how do the Slashdot Linux gurus handle this situation? I've heard a number of theories on the subject, but have yet to find one that results in 100% cross-distro compatibility. Is it even possible, short of distributing source code?"
How many comments until someone asks "Why not distribute the source?"?
Ah geeks..always finding a way to answer the question with a better(?) one.
Java and Python are obvious. You can always use things like JNI and the Python-C bridge if you feel you need better speed in some parts. As long as those parts don't link to any external code, you won't have any problem. They are both very fast these days, and by using things like PyOpenGL and Java3D you can get accelerated rendering and everything. Plus you would only need one codebase for both windows, linux, and OS X.
As for a static build, you may or may not want to do that. The size of the executable will be much larger, and it will take more memory to run, but you won't have the problem of everyone has a different version. It would solve your problem.
As an odd suggestion: why not build all your code into objects and distribute those with a little script to link them on the client machine into an executable. That would fix the problem right? The problem is you couldn't strip the binary (if you do that) because the linker needs the symbols (right?). But it would solve you problem. This is my understanding of how nVidia's kernel drivers work (they have a little glue source too, which you could include). This way they don't have to put out a version for every combination under the sun.
I hope this helps. This is one area where Linux is (rather far) behind Windows and OS X. It's not much of a problem if you've got the source, but if the application is closed source (like yours) then it can be a headache. This is something that the LSB was trying to solve, but I don't know how far they got (or how many distros follow their guidelines).
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
Not the only way, there is also statifier
From the page:
Statifier create from dynamically linked executables and all it's libraries one file. This file can be copied and run on another machine without need to drag all it's libraries.
If you're making games, it's usually a good solution to make the game engine open source, and charge for the data. The game engine can get ported to dog knows what platforms without any effort from you, and you can still get compensated by charging for the data files. The real value of a game tends to be in the data files, as they control the story line, the graphics, the sound effects, basically everything. The game engine is just like a library used to play the datafiles. Of course, not all games work like this.
Please correct me if I got my facts wrong.
If you absolutely can't have the source to your app open, you could do like the developers of binary kernel modules: ship your software in object format, with all the parts that interface with the rest of the system open source. That way, people can't read your actual application source, but they can compile and link the glue layer to make your code work on their system.
If you take this approach, be _very_ careful to have _everything_ that interfaces with the outside system in the open source part; I've worked with kernel modules where some OS calls were still made from inside the binary part, and they were hell.
Please correct me if I got my facts wrong.
I'd suggest doing a bootable CD. You can have total control, then. In my limited experience, gamers are typically not too disturbed by having to do a reboot to get to their game.
http://tinyurl.com/4ny52
``100% cross-distro compatibility. Is it even possible''
In a word, no. Binary compatibility just isn't. It happens to work in very select situations, usually on proprietary operating systems that are themselves binary-only, which means the developers of the OS would be dealing with the same headaches you would if they broke binary compatibility.
On Linux, where binary compatibility has very low priority because _everything_ on the system works fine without it, binary compatibility happens only by accident. It starts with the fact that Linux runs on many architectures, and people actually use it on most, if not all, of these architectures.
Then there are distributions that are still using libc5. Think they don't matter? What happens when libc7 comes out? And if you think that isn't going to happen for a long time, how about when X.org replaces the monolithic structure they inherited from XFree86 by their new, modular one? Or how about when Qt 4 replaces Qt 3, or GNOME 3 is released? These projects are not going to let themselves held back because of your app. Because this is what binary compatibility means: slowing progress, because some changes become impossible.
Please correct me if I got my facts wrong.
You might want to take a look at klik:
http://klik.atekon.de/
*shrug* Of course, any respectable Linux user won't use your software unless they can see the code anyway. How else do we know it's not chock full of security holes and spyware?
Ah. So that's why Loki went bust.
But you know what... I'm respectable, and I bought their games. And no, I didn't get any source code.
There are legitimate reasons not to release source code, and the original poster didn't give any details of his licence, so I think you're being excessively harsh to call it obnoxious.
The decision to distribute source code or not does not affect the quality of the software. It may make it harder for you to look inside, but remember - Sony didn't release their DRM source code either.
(Spudley Strikes Again!)
The entire point of having dynamically linked libraries is to have one central source to update your libraries at, and to share that functionality with other programs. If you are going to include the dynamic library with the program, it's easier to just make a statically linked executable...
But you can't statically link LGPL licensed libraries...
The AACS key is NOT 0xF606EEFD628B1CA427BEA93A9CA9773F
Mike
"Not an actor, but he plays one on TV."
You can do a few different options, each with their own drawbacks. Depending on the libraries you are currently using you will possibly have to use a combination of these items.
Static linking allows you to distribute a single binary that will run on any linux system so long as you have the correct minimum kernel version and CPU. Some of the problems with this will include the license on the libraries you are using may not allow this, and the application's code image changes from what you have debugged against.
Dynamic linking and putting all of the required dynamic libraries your application requires in their own personal library directory. This is the quickest solution, but not always the best solution. It does allow you to release periodic incremental patches to the binaries used by your applicaton.
Usage of a application such as Elf Statifier to take your debugged dynamic application, and bundling the applicable libraries into your application. This is a halfway point between a completely dynamic and static application that allows you to take your "GOLD" release and package it up without changing the generated code in your application.
Just release your application as a dynamic application and mark it with the correct dependancies in RPM format and follow the Linux Standards Base.
Most commercial applications will use either the 2nd method (dynamic and distribute all their own versions of various dynamic libraries) or the 4th method. In both cases they specify a message to the effect of "We support X distro version Y. It may work on your linux distro, but we only support X version Y."
As you are planning to make money off of this game, I wish you luck, and suggest you take a look into what the most popular linux distros for your target audience will be. Based on advertisements from Wal-Mart and Fry's selling Linux preloaded I would bet Linspire and Xandros is high on that list. As with all things, don't forget the testing across many distributions to make sure the solution you choose actually works.
The program isn't debugged until the last user is dead.
Distributors who want to make it hard for you to run your binaries are every bit as wise as the Unix vendors who tried to avoid standardization and the risk of mutual success...
--dave
davecb@spamcop.net
autopackage is probably the current best tool for this. It makes a single, easily installable (and removable) package while coercing my system's GCC and libs into versions that are suitable for other distributions.
:)
I use it currently for schism tracker (particularly the CVS builds that I do).
It works very well for me: Debian, Ubuntu, SUSE, Fedora, Gentoo, and Mandrake users have reported success with these binaries built with (gasp) a lone FC4 machine.
One day, I'll actually do a proper release and six years later it'll show up in the next Debian release. Then your fancy apt-get tools will work.
And to kneejerk jackasses that say "just release the source"- you must realize that the source is good for you and me (well me anyway, I cannot speak for you, obviously), the people who USE these programs have absolutely NO INTEREST in doing the work that I have just done for my own purposes.
Plus, having them use a single binary means that it's very easy to debug with nothing more than a core file.
Oh, and I suppose I am giving the submitter the benefit of the doubt that we are indeed talking about Free Software, aren't we?
As someone who is involved in supporting commerical closed-source applications on 3 Windozes, 5 Linuxes and 4 Solarises, I must confess - binary compatibility on Linux is too damn hard.
Solaris is way better than Linux in that regard, everything compiled on lower versions always works like a breeze on higher versions. Windows has a clear dividing line between retarded 95/98/ME and 2000/XP/2003/Vista, but we do not support 95/98/ME anyway, so binary compatibiity on Windows is wonderful, aside from some minor glitches with missing DLLs.
Now, binary compatibility on Linux is a total pain in the butt. Incompatibilities in glibc 2.2 vs 2.3, pthreads vs nptl, gcc C++ ABI is broken on every other gcc release, thread local storage, just to name a few hurdles.
Distributing statically linked executables is the most reliable way to go, if you want to support as many Linux variants as possible. However, there are few things to remember:
* If your application is security-critical, you link against static library and later security flaw is found in the library, OS security update leaves your application vulnerable.
* Never link statically with libdl. For example, application, statically linked on RH 7.3 witl libdl will misteriously crash on RH9
* Size of executables is big. Even worse if you have many executables using same libraries.
Oh, you said you are going to market games... Boy, you are better to build them for Windows anyway. Don't let me be misunderstood, I love Linux (except, for its binary (in)compatibility, of course), but it's just not the right market and business model for you anyway.
If you keep insisting on Linux, here is a hint. Most commercial vendors shipping products for Linux are oriented towards a limited number of distros, those used in enterprises. RedHat and SuSE that is. Mandriva? Ha-ha-ha. Oh, you say, Ubuntu and Debian are popular? Ouch, not among our cutomers.
my sstream of consciousness
Gamers shouldn't condescend so much to the casual gaming market, and certainly not their developers. These guys are living the dream, after all (and realizing the dream means coding down in the trenches). Power to you, Linux-supporting casual gamer guy.
Help poke pirates in the eyepatch, arr.
For regular binaries, a static build is pretty much guaranteed to work. Alternatively, all Linux distributions I know of do honor the LD_LIBRARY_PATH environment variable. Create a library directory on the target computer, then test the libraries already installed. If the ones you want are either not there or the wrong version, copy the ones you have on the CD into the library directory and have that directory searched BEFORE the default system library directories.
If using custom versions of standard libraries, use a chroot jail to guarantee that the program cannot see the regular library directories.
If you don't actually need standard functions at all, but use your own routines and maybe some system calls, then there's absolutely no problem - just remember that when you compile, you want to PREVENT GCC from linking in the C library. Rescue tools, installers and other components that have to run PRIOR to being able to check versions, etc, should either be statically linked OR not linked at all.
If you want to avoid including stuff that might not be necessary, then you can use yet another technique. Write a "loader" program and include that as source. All the loader does is dlopen() the application (which is now written as a shared library, not an executable) and start it up.
This technique works by linking to the installed libraries in the wrapper, and then when you dlopen() the application, the application can then access the linked libraries. The linking is entirely done in the wrapper, which would always be compiled for the local system.
This technique would only work if all the required libraries are present, and the parts of the API that are used are present and the same. If you write to the libraries which are damn-near universal, then this will work. If you are writing to libraries which are not universal, then you would still have to do the whole process of checking and installing your own libraries, then setting the LD_LIBRARY_PATH. In which case, don't bother with the wrapper and just do that for all libraries.
A final option is to provide TWO programs. The first program assumes nothing about the native system and doesn't do any checks at all. It just installs everything it needs in local directories, sets all the paths as needed, and runs in its own corner.
The second program would be a clean-up program. It has a list of applications it knows about, what versions of what libraries are needed, etc. It would pull libraries that are not system-wide into a standard set of directories, then use symlinks or LD_LIBRARY_PATH to repoint applications to where the libraries are. It would also need two additional capabilities - deinstalling libraries when NO application still installed uses it, and updating records (and shuffling links as needed) when an application is updated.
The reason for the second program is that many developers (especially commercial vendors) have the same problem. If a standardized mechanism existed for library tracking, with a well-known API, I believe it would become very popular very quickly. If such a program were developed, and was sufficiently widely adopted, you could abandon system-wide libraries altogether. Which would be no bad thing, as the existing system has problems with multiple versions and will also fail when libraries have the same filename. It's also slow when you've a lot of libraries installed.
By creating virtual directories, in which the ONLY libraries present are ALL that the application needs, you eliminate not only the application/library issue, but al
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Package your software into an LSB RPM. Anyone (using an LSB compliant distribution) can then install it.
:) And it works pretty well.
If you are using libraries that the LSB does not specify, build private copies and distribute them in your package.
Good lord! That's pretty much exactly how software distribution works on Windows!
nuff said. the reality is dynamic linking causes more problems than it solves. the argument about it saving space is 'blah' (i.e. true but irrelevant).