Slashdot Mirror
Petreley On Simplifying Software Installation for Linux
markcappel writes "RAM, bandwidth, and disk space are cheap while system administrator time is expensive. That's the basis for Nicholas Petreley's 3,250-word outline for making Linux software installation painless and cross-distro." The summary paragraph gives some hint as to why this isn't likely to happen anytime soon.
"That's the basis for Nicholas Petreley's 3,250-word outline for making Linux software installation painless and cross-distro."
Was it necessary to include the word count? It's hard enough to get slashdotters to read a small article and post intelligently, this cant help...
Autopackage comes to mind.
from the site:
* Build packages that will install on many different distros
* Packages can be interactive
* Multiple front ends: best is automatically chosen so GUI users get a graphical front end, and command line users get a text based interface
* Multiple language support (both in tools and for your own packages)
* Automatically verifies and resolves dependancies no matter how the software was installed. This means you don't have to use autopackage for all your software, or even any of it, for packages to succesfully install.
I just don't trust anything that bleeds for five days and doesn't die.
Static linking might be useful as a workaround for the more esoteric distros, but it has its problems. For one, if you statically link your application then anytime there's a security fix or change to the linked library you'll need to recompile the application, not just upgrade the library. This would probably cost more in administration time than upgrading a single library since multiple applications may be dependent on the one library.
emerge
Doesn't get any simpler than that. Come back in a minute to 12 hours (Depending on the package), and *poof* new software. Ditto BSD ports.
TODO: Something witty here...
Place user applications in their own directories
This single rule alone would eliminate most of the problems. It enables fallback to manual package management, it resolves library conflicts, it avoids stale files after uninstallation and it prevents damaging the system which can be caused by overwriting files during installation and subsequently removing files during uninstallation.
The first obstacle to overcome is the bad attitude many linux users have that if something is easy to install, or easy to use, it is therefore bad.
./configure
As I see it, many would like to keep the learning curve very, very steep and high to maintain their exclusivity and "leetness" if you will.
For instance, the post above mine displays the ignorant attitude that "easy to install" by definition equals "unstable software" and has only a jab at MS to cite as a reference.
That's truly sad (though that may just be a symptom of being a slashdot reader.)
As I see it, not everyone finds:
make
make install
to be intuitive, much less easy, never mind what happens if you get compiler errors, or your build environment isn't the one the package wants *cough*mplayer*cough*, or if you even have said development environment.
Nor does it mean the software is any more stable. Could be just as shitty. _That_ is a matter of the developer of the program, not the install process.
Thats the reason windows servers are more vulnerable to attacks, because they give you the idea that its easy to mantain them... Its the same thing saying that you dont need any pilot on an airplane (and that you can put there anyone) if you make a good autopilot engine... We need more knowledge in system administration, not more automatisms.
WARNING: Stupid Matrix reloaded spoiler in parent, in the middle of the 4th paragraph
Site slashdotted? Look here for mirrors, or even better: o
Did some of the suggestions remind anyone of the OpenStep frameworks idea?
Frameworks (very roughly) are self contained libraries containing different versions, headers, and documentation. Java jar libraries are somewhat similar.
The problem is that using frameworks requires major changes to the tool chain - autoconf et al, cc, ld etc.
Apple shipped zlib as a framework in OS X 10.0 (IIRC) but getting unix apps to use it was very difficult. Apple now only seem to use frameworks for things above the unix layer.
I suspect there are lessons to be learned from this. As another poster said, evolution rather than revolution is more likely to succeed.
Slashdot looked deep within my soul and assigned
me a number based on the order in which I joined
But installing gentoo is still hard.
/mnt/gentoo/ /mnt/gentoo
Insert cd.
login in from the command line
net-setup ethx
cfdisk
mkrieserfs
wget ftp.gentoo-mirror.net/pub/gentoo/stage1.tar
tar xzjdocmnaf stage1.tar
mkdir
chroot
scripts/bootstrap.sh
(10 hours later)
emerge ufed
edit use flags
emerge system
emerge gentoo-sources
configure kernel, having do lspci and googling obscure serial numbers to find out what modules to compile
install kernel
muck around with it's non standard bootloader
install cron and sysloggers
umount
reboot
spend two days sorting out the kernel panics
wait all week for kde to emerge.
processor dies of over work
huge nasty electricty bill arrives after running emerge for over a week 24/7
in other words, no
Use Debian and apt-get. No, seriously, could it be much easier?
--sdem
The complaints are, almost entirely, about libraries. But there's already a robust mechanism for determining that a library dependency is satisfied - the SONAME defines its binary compatibility. So if stuff is breaking, it's because library authors are changing binary compatibility without changing the SONAME. How about we just get library authors to stop breaking stuff?
First of all, RAM and disk space are NOT cheap. I spent 60 euros for 256 MB RAM, that's is not cheap (it's more than 120 Dutch guilders for goodness's sake!). A 60 GB harddisk still costs more than 200 euros. Again: not cheap. Until I can buy 256 MB RAM for 10 euros or less, and 60 GB harddisks for less than 90 euros, I call them everything but cheap.
What's even less cheap is bandwidth. Not everybody has broadband. Heck, many people can't get broadband. I have many friends who are still using 56k. It's just wrong to alienate them under the philosophy "bandwidth is cheap".
And just look at how expensive broadband is (at least here): 1 mbit downstream and 128 kbit upstream (cable), for 52 euros per month (more than 110 Dutch guilders!), that's just insane. And I even have a data limit.
There is no excuse for wasting resources. Resources are NOT cheap dispite what everbody claims.
Etc...
Who are you? The new #2 Who is #1? You are #617565. I am not a number, I am a free man! Muhahaha.
As I see it, the following things need to happen to really make application installation be very clean under any Unix like operating system:
Too damn many times I've tried to install FOO, only to be told by the packaging system "FOO needs BAR". But FOO doesn't *need* BAR, it just works "better" if BAR is present (e.g. the XFree packages from RedHat requiring kernel-drm to install, but working just fine (minus accelerated OpenGL) without it).
Were venders to do this, then a program install could be handled by a simple shell script - untar to
The system could provide a means to access the HTML (a simple, stupid server bound to a local port, maybe?) so that you could browse all installed apps' help files online.
As a final fanciness, you could have an automatic process to symlink apps into a
ls
and see them.
www.eFax.com are spammers
Nicholas Petreley writes:
Linking libthingy statically into application foo does not preclude the sharing. Each of the instances of application foo will still share all the code of that executable. So if libthingy takes up 5K, and you launch 10 instances, that does not mean the other 9 will take up separate memory. Even statically linked, as long as the executable is in a shared linking format like ELF, which generally will be the case, each process VM will be mapped from the same file. So we're still looking at around 5K of real memory occupancy for even 1000 instances of application foo. The exact details will depend on how many pages get hit by the run-time linker when it has to make some address relocations. With static linking there is less of that, anyway. Of course if libthingy has its own static buffers space it modified (bad programming practice in the best case, a disaster waiting to happen in multithreading) then the affected pages will be copied-on-write and no longer be shared (so don't do that when developing any library code).
Where a shared library gives an advantage is when there are many different applications all using the same library. So the "shared" part of "shared library" means sharing between completely different executable files. Sharing between multiple instances of the same executable file is already done by the virtual memory system (less any CoW).
The author's next point about sharing between other applications is where the size of libthingy becomes relevant. His point being that if libthingy is only 5K, you're only saving 45K by making it a shared (between different executables) library. So that's 45K more disk space used up and 45K more RAM used up when loading those 10 different applications in memory. The idea is the hassle savings trumps the disk and memory savings. The situation favors the author's position to use static linking for smaller less universal libraries even more than he realized (or at least wrote about).
For a desktop computer, you're going to see more applications, and fewer instances of each, loaded. So here, the issue really is sharing between applications. But the point remains valid regarding small specialty libraries that get used by only a few (such as 10) applications. However, on a server computer, there may well be hundreds of instances of the same application, and perhaps very few applications. It might be a mail server running 1000 instances of the SMTP daemon trying to sift through a spam attack. Even if the SMTP code is built statically, those 1000 instances still share unmodified memory mapped from the executable file.
now we need to go OSS in diesel cars
One of the greatest strengths of the UNIX platform is its diversity..
Package installation is a simple prospect on the Windows platform for the simple reason that the platform has little diversity.
Windows supports a very limited set of processors.. So there's one factor that windows packaging doesn't have to worry about.
Windows doesn't generally provide seperately compiled binaries for slightly different processors ("Fat binaries" are used instead, wasting space).. So the packaging system doesn't have to worry about that. On linux, on the other hand, you can get separate packages for an athlon-tbird version and an original athlon version.
On an MS system, the installers contain all the libraries the package needs that have the potential to not be on the system already. This could make the packages rather large, but ensures the user doesn't have to deal with dependencies. Personally, I'd rather deal with dependencies myself than super-size every installer that relies on a shared object..
Furthermore, on windows there arn't several different distributions to worry about, so the installers don't have to deal with that either.
All of these point confer more flexibility to the unix system but have the inevitable consequence that package management can get to be rather a complex art. We could simplify package management a great deal, but it'd mean giving up the above advantages.
This same problem occurs in the windows world as well, dll hell as it is often called. Here's how it works for windows. Say your program needs vbrun32.dll. You have a choice. You can put the dll in the same folder as the executable, in which case your program will find it and load the right dll. Or you can put it in the system or system32 dll in which case your program and others can find it and load it. However, if vbrun32.dll is already loaded into memory, your program will use that one. I remember we used to have problems with apps only working if loaded in the right order so the right dll would load.
As with Linux, if there's a bug in the library you have to update either one file or search through the computer and update all instances. But, as with linux, the update can mess up some programs, others might be poorly coded and not run with newer versions of the dll. I've seen this last problem in both windows and linux; it looks like the programmer did if version != 3.001 then fail instead of if version 3.001 then fail.
If everyone is forced to use the same library, you get these problems and benefits:
--1 easy point of update
--1 easy point of failure
--older software may not run with newer versions
--programmers may insist on a specific version number
--updates to the libraries can benefit all programs; if kde or windows gets a new file open dialog box, then all programs that link to the common library can have the newer look and feel by updating just one library.
On the other hand, if you let each program have its own, you get these problems and benefits:
--difficult to update libraries when bugs are found
--can run into problems if a different version of the library is already loaded into memory (does this happen with linux?)
--guarantee that libraries are compatible with your app
--compartmentalization; everything you need for an app is in it's directory. Want to uninstall? Just delete the directory. No need to worry that deleting the app will affect anything else.
--no weird dependencies. Why does app X need me to install app Y when they clearly aren't related at all. The answer is shared libraries. Which is why many people like Gentoo and building from source.
Microsoft has waffled back and forth on the issue. Under dos, everything just went into one directory and that was it. Windows brought in the system directory for shared dll's. Now the latest versions of windows are back to having each app and all of its dlls in one directory.
Personally, I think compartmentalization is the key, provided we get some intelligent updaters. If libthingy needs to be updated, the install procedure should do a search and find all instances of the library, back up existing versions and then update all of them. This wouldn't be that hard to do.
Important features of the way AmigaOS libraries worked:
* All libraries were versioned, but not on the file system level. Each library contained a version number in it's header.
* Versions of the same library were always backwards compatible. This was Law. Software using an old version of a library must continue to work on future versions. This also meant that developers had to think out thier library API beforehand (because you would have to maintain that API). Libraries could be extended though with extra functions.
* Application programs had to 'open' libraries before using them. When opening a library an application would specify the minimum version that it required of the library. (If no matching or later version was found then the program would have to exit gracefully).
* There tended to be few (compared to Linux anyway) libraries. Libraries tended to be biggish. A few big libraries instead of many tiny libraries. This made them manageable.
* The backwards compatibility rule/law for libraries meant that software could bring it's own version of a library and update the existing version of that library, but *only* if it was a more up-to-date version.
As a previous poster pointed out, a lot of the problem would disappear if people (library writers) maintained compatible interfaces for the same library soname. I'm pretty sure that this is the way it was designed to work.
anyway, a FYI.
--
Simon
Some very good points and ideas, but also IMHO some misguided assumptions and directions.
/usr/bin, /usr/lib, etc. ) Also the idea of having configuration files that resolve dependancies forces the application to use such configurations, which is also undesireable.
/opt or into /usr/bin or /usr/local if you don't want it there.
1) RAM & Disk space is not always cheap, or even readily available. There are many legacy systems where users would benefit from these advantages but the users are unable or unwilling to upgrade the system. What happens to old 486 and 586 systems where the motherboard doesn't support drives larger than X - there are work arounds, but the people who need easier install processes aren't going to tackle the complex system configuration issues to implement these. What happens when you can no longer obtain RAM in your community for your old machine, or it no longer has spare slots, etc. What happens if you have a second hand computer and simply don't have the available $$ to spend on upgrades, no matter how cheap they are. I don't like the idea of designing an easier-to-use system that excludes such people, no matter how small a portion of the market they may be. Hence redundant copies of libraries and staticly linked libraries are a very inelegant solution for these people.
2) We musn't impose requirements on application developers to use a given installer library, or code their apps to conform with particular standards that the installer requires - it is again unfeasible and undesireable in many circumstances. Developers have more than enough to worry about as it is without having to reimplement the way their app behaves to be installer friendly. The installer must exist at a level independant of the way the application has been coded, to a reasonable degree. I think that much of the problem that exists currently is that too much of the "packager" issues of making apps compatible to a hundred and one different unices has been getting dumped on developers and this both reduces their time for actual development and means that we have a hodge-podge of apps that are compatible to an unpredictable degree, because essentially developers don't want to be burdened with this.
3) Diversity is the spice of life, and it is the spice of unix. The community of unices is robust because it has adapted systems which are generally stable and reliable across a vast array of hardware and software. We want to capitalize on this tradition and expand and enhance it, not force anyone to use a particular layout for their apps & installations. This being said, I find the idea of local copies of libraries in the application directory unappealing, because it forces one to have a local directory ( rather than using
5) Aside from all these criticisms, there are many things I do agree with. Particularly that dependencies should be file specific, not package specific, that an integration of installer & linker is key to the organization of such a tool. I also agree that the installer should make use of auto-generated scripts wherever possible, and should provide detailed, useful messages to the end user that will help them to either resolve the conflicts in as friendly a way as possible, or to report the conflicts to their distribution. Also the installer should have advanced modes that allow for applications to be installed in accordance with a user or administrator prefered file system. That is one shouldn't be forced to install into
Given all this, is there any possible way to solve all of this in one consistent system? I think so - but it may require something that many will immediately wretch over. A registry. That's write, I used the foul windoze word registry. I propose a per-file database for libraries & applications that would record where given versions of given libraries are installed, under what names, in what directories, of what versions, providing what
There are a thousand forms of subversion, but few can equal the convenience and immediacy of a cream pie -Noel Godin
Linux is better than Windows because it's so complex and difficult to use?
/. can be. One day they are making out a bug in IE that crashes the browser to be a HUGE deal. Something that just about has to be purposely coded and would take 2 seconds out of your day to relaunch the window if you did run across a site with the code.
/. start focusing on their own hurdles instead of beating to death every little issue that pops up with Windows. Alternately I would like to see more than the handful of truly devoted Linux users admit, or even just realize, that Linux is far from perfect.
/., those vocal zealots have become the voice of the Linux community.
It's funny how hypocritical the crowd here on
Two seconds was a huge amount of time in the average Linux user's day yesterday, but today, hours and hours spent installing software is but a small price to pay.
I know I am oversimplifying this, I'm doing it on purpose to make a point. When the slightest problem is found with Windows, Linux users will type for hours telling how damaging it is and how it is just another reason not to use Windows. However, when a HUGE flaw in the very foundation of Linux is brought into discussion, it is trivialized to the point that the common concensus is that there is no problem at all.
Blind loyalty does no good for anyone but a dictator. Windows isn't the perfect OS, X isn't the perfect OS, and niether is Linux the perfect OS. Each has advantages and disadvantages. However, if you only focus on the advantages and do nothing to improve the disadvantages, or even admit they exist, they will always exist.
I'm gonna be honest here, I couldn't care less how long it takes to install something in Linux. I'm not posting this in hopes that the Linux Community will come together and work on solving the biggest problems facing Linux. I don't use Linux, probably never will, but I have tried it a couple times in the past. To me, Linux isn't even close to an alternative to XP and X (both of which I use daily). I feel Linux has years of catching up to do before I would even consider using it for anything but a web server.
I'm posting this for two reasons. Primarily, because I would like to see Linux users on
This might sound like flames to many slashdotters here, but that's only because I'm not slamming Windows as a creation of satan while singing the praises of Linux as a gift from the gods. If you really take the time to read what I've said with an open mind you will see I have said nothing defamatory about Linux, and in fact everything I have said would benefit Linux if people would take it to heart. Too many Linux users have become zealots and, at least here on
As proof, I offer this entire thread into evidence. The complexity of installing apps on Linux is quite probably the very largest single problem holding Linux back as a mainstream OS. As a computer user that started on the Commodore 64 and used UNIX in college as a CS major, I can testify that the main reason I wiped the Linux partition mere hours after installing both of the times that I tried it was the complexity of installing even the most basic of apps. However, if you read through this entire thread you will see that the majority of replies deny this problem even exists.
- Bandwidth. No-one wants to have to take 2-4x as long to download programs.
- Hard-drive space. Even if we all had 40GB hard-drives, no-one wants to waste it reproducing the same information a hundred times. People buy hard-drives to store data, not twenty copies of the same library.
- RAM.Loading two copies of the same library wastes RAM.
- Load-time.Having to load all of the libraries will increase load-time compared to cases where some were already opened (by other apps) and you don't have to load them.
- Consistency.Part of the benefit of having shared libraries is shared behavior. Destroyed if app X uses Qt 2.0 and app Y uses Qt 3.0.
- The Big 3S: Security, Stability, and Speed.Who knwos what insecure, unstable, and poorly performing version of a library each app comes with. And who knows what crappy options it was compiled with. Resolving these issues at one central point can be counted out. You want to deal with any of these issues, you'd have to do it for every application's version of a library. That means doing it many times separately.
The solution to dependency-hell is to design better dependency management. Reverse-dependency management -- so as to remove useless libraries when no-longern needed and avoid bloat -- would also be good. Gentoo is doing pretty well in these categories.On making install process' simple. I think that a graphical installation does not necessarily make things any easier. Anyone here played Descent 2? That installed by a good old-fashioned DOS-installation. And it was not particularly hard to install, even though it was not a GUI-install.
It is also not necessarily a good idea to abstract into oblivion the technical details behind an install. Part of the philosophy behind Gentoo, for example, is to take newbies and turn them into advanced users. I think that a clear well thought-out install guide is a useful thing. Gentoo's install guide is thorough and has virtually no noise. Compare that to the install-guides for Debian, which are affirmative nightmares, filled with irrelevant stuff. Furthermore, a helpful and friendly user-community is always a good way to help new users orient themselves. New users are going to ask questions on forums that advanced users find obvious. That should not be an invitation to say, "RTFM bitch" at the top of your lungs. All of us were newbies at one point, and just because we may have had to learn things the hard way doesn't mean that others should too.
social sciences can never use experience to verify their statemen
My specialty is software installation. I've written dozens of installers on a multitude of platforms. On the windoze platform, DLL happens for two reasons:
1. No backwards compatibility. All too often, new versions are released that break older programs. Even Microsoft has done this with major DLLs.
2. Stupid installer writers. You're supposed to check the version number of a file before overwriting it. All too often the file is overwritten without regard to the version numbers.
So to overcome these two problems, the smart installer coder would put all the DLLs in a private directory of the application (not in system/system32).
Of course, Microsoft came up with a new system that broke this simple fix. Registered libraries. Instead of using a specified path to get a DLL, you would ask the system to load the DLL (using registry information). The path was no longer considered. One, and only one, version of the DLL was allowed on the system, and there was no feasible way to get around this limitation. Someone came up with a fix. It would have been a major pain to implement and would require cooperation amongst the DLL coders, which isn't about to happen since the lack of cooperation was one of the core problems in the first place.
For a commercial level installer, missing libraries was absolutely unacceptable. My personal rule was to ALWAYS include dependencies in the installer package. This meant the installer was bigger and more complicated, but it guaranteed the application could be successfully installed without the user having to run off to find a missing library. Or did it? No - Microsoft decided that some libraries could not be independently distributed. The only legal means of getting the library was through the official Microsoft installer. And no suprise here, half the time the only official installer for a library was the latest version of Internet Explorer.
Requiring an upgrade to IE is a major problem for large companies. They standardize on specific software and don't allow the users to change it. Requiring a site-wide upgrade of something like IE (or the MDAC package) was not to be taken lightly. Especially when it was dicovered that the required upgrade would break other applications (back to DLL hell).
FYI, when a major customer pays your mid-sized company a couple of million dollars a year in license fees, they can definately tell you they won't upgrade IE. It's our job to come up with a work around. Too bad a measly few million paid to Microsoft wasn't enough to get them to change their ridiculous library polices.
-- Will program for bandwidth
My issue with Linux is that every time a new version of a library comes out, it breaks all prior apps. (usually)
The response is that compatibility slows progress by locking down the api. This is so short sighted that it is not even funny.
If programmers thought out how their libraries would be used it would be simple to add another call in a newer version. Instead they make short sighted decisions and ruin the use of a shared library.
IMHO any newer version of a library should work better than the previous version and be a 100% replacement.
This would fix a huge chunk of DLL hell and installer issues.
If tyranny and oppression come to this land, it will be in the guise of fighting a foreign enemy. - James Madison