Ubuntu Developing Its Own Package Format, Installer

An anonymous reader writes "While complementing Debian APT/DPKG, Canonical is now developing their own package format. The new package format has promised highlights of having no dependencies between applications, each package would install to its own directory, root support wouldn't always be required, and overall a more self-contained and easier approach for developers than it stands now for Debian/Ubuntu packages. The primary users of the new packaging system would be those distributing applications built on the Ubuntu Touch/Phone SDK. The initial proof-of-concept package management system is written in Python and uses JSON representation." This quote from the post by Canonical's Colin Watson bears repeating: "We'll continue to use dpkg and apt for building the Ubuntu operating system, syncing with Debian, and so on."

Comments

Good

[MightyMartian]

Good, another reason to avoid Ubuntu like the plague.

Re:Good

[Penguinisto]

Indeed... this sentence:

The new package format has promised highlights of having no dependencies between applications

...tells me there's gonna be a whole shitload of bloat, duplicate binaries, and a performance hit from Hell.

I could be wrong, but...

Re:Good

[Penguinisto]

The presence and use of /Library on my MBP says differently. ;)

Re:Good

[buchner.johannes]

Also, this might be the dawn of malware for Linux on the PC.

Re:Good

[Jeremiah Cornelius]

Go open a mac .app sometime. Libraries and resources galore can be found. The Systme libraries and frameworks can be over-ridden. like having ~/Library on a per-app basis.

Re:Good

[K. S. Kyosuke]

...tells me there's gonna be a whole shitload of bloat, duplicate binaries, and a performance hit from Hell.

Hmmm, wouldn't it be possible to solve the duplicate binaries problem by simply reference-counting identical files?

Re:Good

[Anonymous Coward]

This is going to be the best thing for linux. Most users will enjoy and maybe switch to linux instead of windows. Dependecies that link multiple times to other dependancies get very convoluted. Normal peoples linux finally.

Re:Good

[X0563511]

Sounds like the dynamic linker is going to need an overhaul to include data-deduplication. As well as the filesystem.

Re:Good

[Jeremiah Cornelius]

"Dunno if there's a way to specify that inside Xcode or not, but for our app we use a build script that includes some code like the following. The code uses Apple's install_name_tool utility to modify the application so that instead of pointing to /usr/lib/libsndfile.so, it points to a libsndfile.so path that is in the application's package instead.

Note this is just a cut-down script excerpt to give you an idea; it will probably require some tweaking before it works for you (and of course you'll need to modify it to operate on other libraries besides libsndfile if that is what you want):"

#!/bin/bash -e
 
BINARY="MyAppFolder/MyAppName"
FRAMEW_FOLDER="MyAppFolder/MyAppName/Contents/Frameworks/"
 
function DoInstallNameTool {
    xLIB="$1"
    xLIB_NAME="$2"
    xBINARY="$3"
    echo install_name_tool -change \"${xLIB}\" \"@executable_path/../Frameworks/${xLIB_NAME}\" \"${xBINARY}\"
    install_name_tool -change ${xLIB} "@executable_path/../Frameworks/${xLIB_NAME}" "${xBINARY}"
}
 
for LIB in $(otool -L "${BINARY}"|grep libsndfile|cut -d '(' -f -1)
do
    echo "Handling Lib: $LIB"
    LIB_NAME=$(basename "$LIB")
    echo " Adding ${LIB_NAME}"
    cp -Rf "${LIBSNDFILE_DIR}/src/.libs/${LIB_NAME}" "${FRAMEW_FOLDER}"
 
    DoInstallNameTool "$LIB" "$LIB_NAME" "$BINARY"
done

http://stackoverflow.com/questions/7470637/dynamic-library-in-application-bundle-mac-os-x

Re:Good

[king neckbeard]

Except for the fact that dependencies are already handled by package managers. It can already be done and is done transparently in Ubuntu Software Center.

Re:Good

[mr_shifty]

This. ^

Re:Good

[TheGoodNamesWereGone]

Flamebait??? C'mon people. Canonical seems to be hell-bent on digging their own grave.

Re:Good

[rroman]

Actually, the "shitload of bloat duplicate binaries" is quite good. Nobody gives a damn about 10 MB of their disk space because the program takes it's libraries with it. However, everyone gives ten tones of damn when they can't install new application because of "dependency problem". Solving dependency problems costs time and hence money. Disk space is cheap.
Disclaimer: I'm not saying, that new Ubuntu does that, I'm just arguing against the philosophy of bad duplicate binaries.

Re:Good

[Skrapion]

And that works reasonably well for popular open source projects, where volunteers can build new binaries to keep up with updated dependencies, but it's a real problem for closed-source projects.

Whether or not paving the way for closed-source projects on Linux is a good thing will depend on whom you ask, but I suspect that's one of the major motivations for this project.

Re:Good

[smash]

Which, given most people have upwards of half a terabyte of storage, and the OS consumes only 1% of that, storage space is no longer at a premium.

The benefits of totally self contained packages are huge. You'll probably find that the vast majority of your application is NOT duplicated data anyhow. It will be graphic resources, it's own code, etc.

Couple that with de-dup in modern filesystems (inline still a major overhead, but scheduled de-dup isn't a major problem) and the advantages to just linking to libraries everywhere are not so clear cut anymore.

Having lived with the OS X way of packaging for a few years, and dealt with both Linux, FreeBSD and Windows since the early 90s, I know which method I prefer.

Re:Good

[smash]

Yes. ZFS will do this, as will HAMMER. However, I suspect the gains REALLY aren't there. The core OS binaries (including all libraries) on a modern system account for what... 1-2% of the disk space? If that? The bulk of the code in your new application will be the new application. Even more disk will be consumed by the resources used only for that app (graphics, widget layouts, etc.).

Disk today is cheap. Having dealt with both linking (Windows, Unix) and self contained Applications (OS X), I much prefer the latter. Installation / un-installation is way, way easier. Compatibility issues are less common.

Re:Good

[TheRaven64]

No it doesn't. PC-BSD has had this model for application installs for ages. The installer hard links duplicate libraries and so on together. Hard links are already reference counted, and have been since the early days of UNIX, so you end up with one copy of each library. The logic in the installer is relatively complicated, but the uninstaller just has to delete the tree.

The way that the packages in the repository are built ensures that programs using the same library ship the same binary. If you upgrade just one program, then you'll have two copies of the library, until you upgrade all of them and then the old one's reference count will hit 0 and it will be gone from disk.

Re:Good

[tehcyder]

Also, this might be the dawn of malware for Linux on the PC.

Then it will truly be the year of Linux on the desktop, as Windows will lose its one remaining USP. Hold on a sec...

apt

[kthreadd]

Sounds like a complement to dpkg, not a replacement

Re:apt

[phantomfive]

Designing the capability to bundle libraries with an application install is a good idea, the problem is.....

Ubuntu is building their own system for doing it, instead of using APT, which gets them 90% of the way there. Most likely, it will be poorly done, which is the common fate of those who are too lazy to understand existing systems.

Re:apt

[Anonymous Coward]

Yeah, Colin Watson knows nothing about apt. Maybe he wouldn't be so lazy if he would start maintaining some packages in debian.

Re:apt

[exomondo]

Ubuntu is building their own system for doing it, instead of using APT, which gets them 90% of the way there. Most likely, it will be poorly done, which is the common fate of those who are too lazy to understand existing systems.

So you're saying Colin Watson is too lazy to understand existing systems? You've never actually looked at commits for APT have you.

Bloat

[paulej72]

If everything has no dependencies, then all of the dependencies must be included in the install. I wounder how many copies of each low level program would be on a given machine.

Re:Bloat

[fph il quozientatore]

If everything has no dependencies, then all of the dependencies must be included in the install. I wounder how many copies of each low level program would be on a given machine.

And, especially, good luck installing a security update on *all* your copies of a core library.

Re:Bloat

[Anonymous Coward]

Quite frankly, disk space is cheap. If this eats another 10 gigabytes of hard drive space over the traditional approach, I personally wouldn't even notice. If it made things even 50% less likely to break because of dependency problems I would be all for it.

Re:Bloat

[Anonymous Coward]

If it made things even 50% less likely to break because of dependency problems I would be all for it.

What if it makes things more likely to break because package A and package B are using incompatible versions of a library? What if you get compromised because a package uses an old version of a library with a zero-day exploit? Even though you already updated the same library for other applications.

Re:Bloat

[interval1066]

Quite frankly, disk space is cheap. If this eats another 10 gigabytes of hard drive space over the traditional approach, I personally wouldn't even notice.

Might as well use Windows with that philosophy. The big difference with Linux is its efficiency. Get rid of that, might as well get rid of linux.

Re:Bloat

[Kjella]

Might as well use Windows with that philosophy. The big difference with Linux is its efficiency. Get rid of that, might as well get rid of linux.

Fast, cheap, good - pick two? >:-) If you have a modern system, no OS should take a significant amount of resources on its own. Don't waste just to waste but as long as developer time is a limited resource, I'd rather have more features, higher quality and better performance than lower resource usage. The hardware is there to be used, that's what it's bought for. Hell, I regularly run an application that likes to chew through it's whole 32 bit memory space (4GB) and I let it because I still have 12 to spare and there's no direct replacement and it mostly works. What you care about is simply nothing that I care about at all, but then so I took your advice on my machine in favor of Windows 7. If Microsoft continues down their path I might have to migrate to Linux again, but if so I'd like a few other things than efficiency. Otherwise it might be a Mac, but I'd rather not.

Re:Bloat

[colinrichardday]

Quite frankly, disk space is cheap.

Not on a 64GB tablet.

Re:Bloat

[Tyler Eaves]

Seriously? Look at how bloated the average "modern" base install is. Look how many background daemons there are. Efficiency went out the window along time ago. The linux of Slackware, or even Gentoo, has largely ceased to be. It has become what it once reviled.

Re:Bloat

[ArsonSmith]

I know, my cluster base CentOS 6.3 is nearly 800Megs. What happened to the days when I could boot/root two floppies and have a full Unix OS.

Re:Bloat

[timbo234]

How about comparing like-with-like instead of new software with software from 10 years ago:
Ubuntu 12.04 (released 2012): 384MB minimum
Windows 7 (released 2009): 1GB minimum for 32-bit, 2GB for 64-bit
Windows 8 (released 2012): 1GB minimum for 32-bit, 2GB for 64-bit

Plus the minimum requirement for XP was 64MB, with 128MB recommended (http://support.microsoft.com/kb/314865), not 32MB.

https://wiki.ubuntu.com/PrecisePangolin/ReleaseNotes/UbuntuDesktop#System_Requirements
http://windows.microsoft.com/en-us/windows-8/system-requirements
http://windows.microsoft.com/en-us/windows7/products/system-requirements

troll bait headline

[Anonymous Coward]

A better headline:

Ubuntu Phone apps will use a different package format than debian/dpkg/apt

I guess that's not really as exciting though

Re:troll bait headline

[MrEricSir]

Ubuntu Phone apps

Let's be clear: Canonical's vision doesn't involve "phone apps." They want the same apps running on your phone and on your desktop.

Re:troll bait headline

[rsborg]

Ubuntu Phone apps

Let's be clear: Canonical's vision doesn't involve "phone apps." They want the same apps running on your phone and on your desktop.

Have they by chance been hiring Microsoft employees of late?

And what's with this OS unification frenzy that ignores how touch and pointers behave differently. I even hate it in OSX Lion with the "natural" scroll (though OSX is much less tablet-zombified than Win8). Scroll on a touchscreen is "natural" direction as your fingers track the object 1:1 in real time (hopefully). Scroll on a perpendicular surface separated from the screen (trackpad), and sensitivity could be greater than 1:1 - it's NOT natural to join the directions (it's somewhat workable for the laptop trackpad as you see your fingers and the screen in the same frame, but a magic trackpad just feels weird - I always "reverse" the scroll setting).

GoboLinux?

[Skinkie]

Wasn't this exactly what GoboLinux embraced?

The good old days

[thomasdz]

Remember the good old days when everything was just a single -r-xr-xr-x executable in /bin or /usr/bin

sigh, the good old days

Re:The good old days

[phantomfive]

Talk about simplicity. Here is an example 'Makefile' from Unix 6. Compare it to today's auto-tools:

chdir lib
cc -c -O *.c
ar r /lib/liby.a *.o
rm *.o
chdir ../source
cc -s -O y?.c
cmp a.out /usr/bin/yacc
cp a.out /usr/bin/yacc
rm a.out *.o

Re:The good old days

[Anonymous Coward]

You didn't build emacs back in the bad old days, did you?

Re:The good old days

[oodaloop]

Oh, I'm terribly sorry. I didn't realize I was standing on your lawn. I'll remove myself forthwith.

Re:The good old days

[MavEtJu]

Call me old-fashioned, but I prefer a proper Makefile which only compiles the files required and which separates between the build and install phase.

Re:The good old days

[phantomfive]

btw that's not old-fashioned, that's newfangled. You and your newfangled Makefiles, get off my lawn!!

More Flexibility?

[organgtool]

each package would install to its own directory

Would it allow users to install multiple versions of the same application from packages? One of my gripes with Linux is that it's not easy to test new or beta versions of software since there is no easy way to install from packages alongside the existing (stable) version. Yes, I know that I could build the app from source, but that can be quite a hassle sometimes.

Re:More Flexibility?

[X0563511]

Sure, if you call 'ldconfig' "a half dozen different methods"

Re:More Flexibility?

[interval1066]

Microsoft solved this (partially) using a centralized registry...

Um, the MS registry is a huge pain in the butt for developers and M$ knows it, but they can't get rid of it becuase its too ingrained. Getting rid of the registry was a huge selling point for Windows 8, as it was for Vista... and so on. I dare you to ask me why... if you don't realize its a huge honey pot for virii and hackers you have no business even asking. Linux DOES INDEED have a system for library control, its called pkg_config and it works very well. Its not my problem if developers are too lazy to use it. 90% of linux apps I've ever envountered use it, so don't come whining to me there's no soluton this lib hell of which you speak. I do quite well with Linux, thank you.

Re:More Flexibility?

[jedidiah]

> That's because Linux suffers from a similar problem that Windows 95/98, and XP to a lesser extent did: DLL hell.

"DLL hell" has squat to do with it. The package manager is going to want to replace one version of an app with another. That is the only real problem here. If you ignore the package manager, you can install what you want.

Linux has had versioned shared libraries for ever.

The registry is just crap and you're a moron for even bringing it up in this context.

Re:More Flexibility?

[julian67]

There is no such word as virii either in English or Latin. The plural of virus is viruses.

http://en.wikipedia.org/wiki/Plural_of_virus#Treating_v.C4.ABrus_as_2nd_declension_masculine

Re:More Flexibility?

[girlintraining]

Um, the MS registry is a huge pain in the butt for developers and M$ knows it, but they can't get rid of it becuase its too ingrained.

Hi. Developer here. No, no it is not. Pain in the ass is thunking, handlers that silently disconnect from message pumps, calls to win32 that are "unmanaged" and so often lead to memory leaks even when you diligently try to do garbage collection, undocumented exceptions that are thrown by system calls that even the debugger blanches at, and endless layers of class and interfaces nine levels deep, many of which only document the public methods and tell you nothing else. The registry though? The registry is stupid easy. If you think that's a pain in the ass, you fail as a programmer. Turn in your badge.

Getting rid of the registry was a huge selling point for Windows 8,

So was the 'Metro' interface. And I don't see many people around here talking about how great Windows 8 is going to be -- everyone says it's going to be a three coiled turd. And this is what you use to advance your argument?

I dare you to ask me why... if you don't realize its a huge honey pot for virii and hackers you have no business even asking.

There's no such thing as 'virii' and I don't think you know what a honey pot is. But perhaps you mean it's a rich repository of system and application settings that could be exploited by malicious people? Yes, that is correct. But then, scanning down the C: for a list of filenames can do that too. The registry has a rich set of access controls; Your lack of familiarity with other security concepts suggests you probably didn't know that.

Linux DOES INDEED have a system for library control, its called pkg_config and it works very well.

A tool used to assist in compiling is not a "library control" tool dude. It's main job is to tell whether a library is installed or not, what version it is, and where it is located. It doesn't do much else. It most certainly does not have any management functions.

Its not my problem if developers are too lazy to use it.

*facepalm* Who wrote the applications you're using right now? Are you sure their "lazyness" isn't your problem?

90% of linux apps I've ever envountered use it, so don't come whining to me there's no soluton this lib hell of which you speak.

That whooshing sound is the point flying over your head. 100% of windows applications have to go through the kernel to load dlls, and so it presents a standardized interface for doing so. Linux does not have this. Whether it's 90% of linux apps, or 99%, the lack of a standard means you, the developer, have to guess, pray, or assume, things about the environment. You can't rely on it.

I do quite well with Linux, thank you.

That's nice. I'm a neo-pagan who dances around poles naked, and it works quite well for me. But I'm not going to suggest it's for everyone, or that there aren't occasional problems in going "sky clad" in a public place. The police, for example, aren't as understanding as one might hope.

Re:More Flexibility?

[jedidiah]

> Does ldconfig allow for different versions of the same library to be requested by the application?

Yes.

> Does ldconfig warn you when a dependancy isn't met?

Better than that. You as a user can see what libraries are loaded from where and which ones are missing precisely.

The "superior alternative" is just a black box.

Re:More Flexibility?

[characterZer0]

You're just angry that I'm pointing out that linux lacks a central repository for application and kernel settings and you have to dig through /etc 's mass of files to do the same thing. Linux is still rocking the equivalent of ".ini" files, and yeah -- it is primitive. And I'm not a moron for bringing it up, you're a moron for not seeing that sometimes, your religion of choice, could benefit from looking outside of itself and seeing that other developers have done something better.

Text files for configuration are great. I can version control them, copy them from one system to another, see meaningful diffs between them, and invidual applications can choose formats that are sensible for them.

Why on earth would I want to cram everything into a central repository?

Re:More Flexibility?

[jedidiah]

> I want something that just works

As far as "just works" goes. Nothing that MacOS or WinDOS has can touch a Linux package manager.

That's why Apple copied this feature in their appliances.

Re:More Flexibility?

[Anonymous Coward]

Your ignorance of Linux is showing. Please refrain from commenting about things you do not understand.

Can you please point out which setting in the man page allows you to set this, oh great wizard of Linux? Because I think you're just being a contrarian right now, and haven't actually read the damn page.

You won't find that in the ldconfig man page because it's provided by the filesystem. This is a small snippet from the contents of my /usr/lib:

libboost_python-2.7-1_49.so
libboost_python-2.7-1_49.so.1.49.0
libboost_python-2.7-mt-1_49.so
libboost_python-2.7-mt-1_49.so.1.49.0
libboost_python-2.7-mt.so
libboost_python-2.7.so
libboost_python-3.2-1_49.so
libboost_python-3.2-1_49.so.1.49.0
libboost_python-3.2-mt-1_49.so
libboost_python-3.2-mt-1_49.so.1.49.0

Do you see what is happening there? Have you ever actually looked inside /lib or /usr/lib of a *nix system? Did you grasp what you saw? One application may need /usr/lib/libboost_python-2.7.so while another needs /usr/lib/libboost_python-3.2-1_49.so. Both get what they need.

In Linux the library's version is part of its filename. There is no "dll hell". DLL Hell in Windows was caused by a dll with a given filename being replaced by a different version with the *same filename* in the *same location*. I don't think you really understand the DLL Hell situation.

There's no man page for knowing what you're talking about.

Re:More Flexibility?

[Anonymous Coward]

/etc is the registry, and it is no more or less centralized than the Windows registry (which consists of multiple files).

The difference is that /etc is implemented using two of the most mature and well understood pieces of the system: a file system directory hierarchy and text files.

The windows registry, in contrast, is implemented using a limited and clunky database, which is difficult to manage and drives even experienced users to frustration.

And speaking of "rich access control". /etc has extremely robust access control via the filesystem. Access control that is well tested, well documented and well understood.

Re:More Flexibility?

[alcourt]

Query: ldd
Control: see the various environment variables that specify which lib dirs are used in what order for that environment you just created. (LD_LIBRARY_...)

Applications can specify the exact version number of a library (.so.1 vs .so.2).

Note, none of this is Linux specific. That family of operating systems is far from perfect in shared library handling, but I won't pretend that it doesn't inherit at least some tools that have at least some flexibility from older Unix tools.

Re:More Flexibility?

[girlintraining]

Oooh, your a pagan... is that the sign that I'm supposed to take you seriously? Shillin' for Microsoft... they're recruiting from you freaks now eh?

And yet they mod me as the troll in this discussion...

Re:More Flexibility?

[EvanED]

The difference is that /etc is implemented using two of the most mature and well understood pieces of the system: a file system directory hierarchy and text files.

You're simplifying.

You can't just say "text files", because I can't write "Hey Apache, I want you to put password protection on the stuff in /var/html/secret" in a file and have it work. Instead, Linux config files take many different formats, from INI-style to XML to shell scripts to custom formats like the X11 config.

I can very easily turn around the argument as follows:

"The difference is that /etc is implemented using a bunch of inconsistent formats. The Windows registry, in contrast, is implemented using a consistent, reasonably simple, typed key-value hierarchical database."

And speaking of "rich access control". /etc has extremely robust access control via the filesystem. Access control that is well tested, well documented and well understood.

The equivalent of registry-style access control would be more like separate ACLs for each line of a file. To do that, oh, you have to write a special program like crontab which runs with setuid (I guess that's what it does?) to edit the file but takes instructions from something else.

Useful? No clue, I don't do enterprise stuff. But from a pure objective level, the Windows registry offers a significantly more expressive set of security settings.

Re:More Flexibility?

[jedidiah]

> doing network administration for a Fortune 100 business with over 150,000 workstations. Every night, I push out a half-dozen new applications, patches, and OS installs.

You need to troll better. I'm not calling you a troll because you disagreed with me. I am calling you a troll because you are a very bad liar.

Some of us have actually worked in environments like that.

It's like you're bringing a knife to an NRA convention.

Re:More Flexibility?

[El_Oscuro]

It seems like every time I make a change to something in the Windows Registry, I have to reboot. On Linux, I usually just have to restart the app or service that uses it.

Re:More Flexibility?

[girlintraining]

Your ignorance of Linux is showing. Please refrain from commenting about things you do not understand.

Blame the idiot that said 'ldconfig' does that, you arrogant bastard, not me.

Re:More Flexibility?

[serviscope_minor]

Who are you and what have you done with girlintraining?

The dynamic loader on Linux is very flexible. There are actually several different systems of varying granularity availble.

The coarsest is using LD_LIBRARY_PATH to allow one program to use a different .so.

The next is versioning with libblah.so.?, which allows different .sos to be made available depending on how the program was compiled. This requires a little bit of care on the part of the library author to actually bother to increment the version numbers.

The next, more complex is symbol versioning within a .so. This allows a single .so to have multiple different versions within it, so a single.so can actually serve multiple different versions. This is really good for system libraries and allows a great deal of backwards compatibility without bloating the number of libraries, and while allowing the maximum amount of code sharing. It requires the most discipline, so is generally only done by the dedicated libc and libstdc++ people. It's particularly important for system libraries, since it allows libraries which depend on "different" versions of glibc or libstdc++ to be linked together without trouble.

This is why a modern libstdc++.so.6 can happily serve g++ compiled binaries from just after the last ABI change to now without trouble.

Keeping hundreds of linux boxes up to date and patched is a fucking nightmare.

What? With the popular, sane distros, you can install some version with some reasonable support term (DeadRat, CentOS, Ubuntu LTS, Suse) and just tell it to auto-update the packages at 3 AM. You can even point it at a local package mirror if you want to save on external bandwidth.

In fact one can easily create a PXE installer which will install a customised package list and set up that configuration for you. Once you've done that, you buy a new machine, mess with the BIOS just enough to PXE boot it, choose the version you want and hit go. 1 hour later, you'll have a working, freshly installed system which will keep itself up to date with security patches until the distro drops off support.

If you want to rely on 3rd party programs which sit in /opt, it is very easy to download the program, untar it and chuck the files into a dkpg or RPM file, and then just add that package to the list. In fact people like LibreOffice only make APTs and RPMs available making it a minor faff to install on a less popular distro.

Again, once you do that, it will automatically roll out to all machines. Of course there's a bit more interaction, but then it is up to you whether you want to upgrade to a new major version of LibreOffice or whatever. And this way, the users won't get exposed to those program's nasty auto-updater scripts hassling them for new versions or whatever.

Honestly, if you're having trouble keeping Linux boxes up to date, then you are doing something wrong.

Also known as application bundles.

[sidragon.net]

http://en.wikipedia.org/wiki/Application_bundle

Once again, racing as fast as we can towards where the puck was 20 years ago.

Because Apt-get is soooooo inferior.

[JeremyMorgan]

I think it's just a case of "because different" and "not developed here". I don't see how they could make any significant improvements over apt, but it doesn't surprise me from this group of hipsters.

Stop pissing in the pool Ubuntu.

Re:Because Apt-get is soooooo inferior.

[EvanED]

I don't want to get into whether you could use DPKG packages for what they want to do; that's beyond my scope of knowledge. However, what I can tell you is that for some tasks, apt-get just flat out doesn't work.

In particular, if you want to install programs to something other than /usr (for instance, you want to install to a network drive, or without root), then the mainstream package managers just flat out don't work.

Re:Because Apt-get is soooooo inferior.

[exomondo]

I don't see how they could make any significant improvements over apt, but it doesn't surprise me from this group of hipsters.

Right because Colin Watson never made any improvements to apt.

Stop pissing in the pool Ubuntu.

What is with this attitude of 'you're not allowed to create different things and disrupt the status quo'? Who cares if the build it?

Re:Because Apt-get is soooooo inferior.

[EvanED]

You can absolutely install programs in other directories than /usr, using the dpkg package system.

Yes and no. dpkg is sort of up to it, but the ecosystem needs a lot of support to make it work.

For instance, let's say that we have a library and a program, and Program depends on Library. (For the moment, these are separate packages.) The user installs Library to a different location. How is Program going to find it? The conventional strategy would be to compile Program with Library's path, and it gets added to the rpath field inside Program's actual executable. A modification of this conventional strategy which would work with a more directory-based package manager would be to use patchelf or something to rewrite the executable's rpath entry to include Library's actual install directory. But where's it going to get that information? I guess you could have the post-install script look up the installation directory from the dpkg database, but again, it's inelegant and would have to be done independently for every package, as opposed to having direct support in the package manager. And as I describe below, I suspect that wouldn't work.

The dpkg package specifies where the files are to be installed.
You can absolutely install programs in other directories than /usr, using the dpkg package system.

Like I said, I'm not very familiar with the dpkg-level of things. My understanding of your statements are that the package maintainer can choose the directory, but the only control over installation directory from the user's side that I see is --instdir.

And if this understanding is correct, --instdir is insufficient: dpkg will chroot to that directory to do the installation and run the configure scripts, but what if a dependency is installed outside of that root? The way that current packages are set up, I can't install both Program and Library to different directories.

If you want to use dpkg to install programs to a network drive, you can certainly do so. The filesystem makes no distinction between a network mounted directory and a local directory. It's all one big tree.

Oh, I was under no illusion that it would treat it differently; I was just mentioning the network drive situation to explain my use case a little bit more.

You will need to be root to install the dpkg package. Write access to the package database is only allowed as root.

Another place where dpkg falls short of my ideal package manager. Why is there a single centralized database when different users may have different software installed?

Goodbye Ubuntu

[FuzzNugget]

Hello Debian

I think this is for "apps" not applications

[jandrese]

I don't think we're going to see a completely separate install of Gnome for every single Gnome app on your system. I think this is intended for standalone apps like you see on phones and tablets.

NIH sydrome but already been done

[OrangeTide]

it seems like they suffer from Not Invented Here but at the same time, I seem to recall that Puppy Linux already installs self-contained package bundles into individual directories. Not unlike OSX/NeXT app bundles.

but is apt/dpkg perfect?

[OrangeTide]

Arch's pacman is has some significant improvements over apt/dpkg, especially when it comes to creating new packages. And gentoo portage/emerge has lots of useful features that don't exist in dpkg/apt either.

Debian is solid and has an excellent package system. But it isn't the pinnacle of achievement. We can take package management further still, I am certain.

Re:another broken feature

[Anonymous Coward]

Which is actually even worse, because it has to exist alongside dpkg/apt. Still re-building the wheel and adding useless bloat, no matter how you look at it.

Re:but... WHY?

[amorsen]

We need it because while current packaging systems are great for central control, they are bad for actually letting users contribute.

a) You cannot submit a bug to a developer, get a fixed beta release, and install that in the packaging system (unless you know how to handle spec files)
b) You cannot do parallel installations of newer (or older) versions for testing unless the package is built specifically for that
c) It is difficult to make distribution-independent packages, so users become dependent on the distribution for all software
d) Almost all packages require root, the packaging system cannot track software installed by users themselves

On the other hand, switching to a Mac-style packaging system has at least these problems:

1) Security updates to common code are unlikely to actually get applied to all packages
2) Some libraries will not be shared, costing extra memory and cache footprint
3) Centralized control over what software is installed suddenly becomes difficult
4) Without dependencies you need to define the minimal environment that software can depend on. LSB tried to do that and failed.

A crippled version of APT

[gringer]

From the email:

The proof of concept I wrote also isn't entirely new code. It's tiny due to using .deb as a container format (minus maintainer scripts, full dependencies, etc.), so I get to save effort by using dpkg to unpack things, which leaves us room to selectively use more of its features in future if we want to.

So they start off with what they think they need, then become more like APT as they need to add more features.

So the scope of what I've been considering is purely leaf apps built on a fixed "base system", which in the case of the initial target of the Ubuntu phone/tablet work would be the run-time part of the Ubuntu SDK.

In other words, this is something to be used in addition to APT (i.e. post-install), rather than instead of APT.

* no dependencies between apps; single implicit dependency on the base
      system by way of a Click-Base-System field

Just like Debian has an implicit dependency on the base system (except for base packages, which have more complicated rules). In other words, this system will only accept a single dependency, the Click-Base-System. I'm not quite sure why this is different from only accepting applications that only depend on Click-Base-System.

And note that the "each package will install to its own directory" bit is on the to-do list:

Obvious items I still need to work on:

• produce a strawman hooks implementation with some real worked examples
• integrate (or demonstrate how to integrate) the container isolation properties worked on elsewhere
• Click-Base-System field is skeletally simple right now, and may need to be expanded to at least leave open the possibility of multiple flavours of base system (see also GNOME's profiles idea)
• adjust unpack handling to avoid problems with project renames and name clashes, and to unpack each version into its own directory and flip symlinks to allow for multi-user independence
• integrate into the Ubuntu SDK, as well as providing examples of how it can be integrated into other build systems too

Re:but... WHY?

[EvanED]

We need it because while current packaging systems are great for central control, they are bad for actually letting users contribute.

One more:
e) Most current packaging systems don't let you pick an installation directory. (Many programs have directories set in stone when you compile them.)

This is one that has affected me, because in the environment I work in, we have standard stuff installed locally but a lot of more esoteric software as well as more current versions of software installed to network drives. (For instance, we run RHEL6 which comes packaged with Python 2.6, but I almost always use Python 2.7 off of the network. Or we have Clementine available for use only over the network.) But the way existing package managers work means that we can't really use them, which means that if I want to make something new available over the network, I have to go and chase down its dependencies myself and then compile everything.

3) Centralized control over what software is installed suddenly becomes difficult

I would argue that point for the most part. From some vantage point, it's already difficult to have centralized control over what's installed. After all, I can still do what I described above to install software, without the support of the package manager. It's just a pain. Or even if you switch to a better package manager that allows users to have control over what gets installed, you can still block that as an administrator by setting noexec on the drives.

Cutting Edge Debian

[tuppe666]

Hello Debian

Except Ubuntu users want cutting edge Debian, not tried and tested Debian...and unfortunately using Debian is not going to make it more cutting edge.

...Now if Debian decided to produce a (stable) cutting edge Desktop version (perhaps working with an existing Distribution team). To complete there ultra stable, you have me sold.

Re:Bad idea.

[EvanED]

Crap, shoulda previewed. Sorry for the formatting. Fixed version:

Then you would end up with a massive PATH variable...

First: that's only one solution. For example, I have (issues file system query) probably three or four thousand different packages available to me right now, all installed into their own directory. (This counts multiple versions separately. As a disclamer, some of those packages are very very old and won't even run on this system as the library dependencies aren't present or they are only built for another architecture, and most of the others are obsolete and there's little reason to use them. Point being: I have access to tons of packages.) My $PATH has 21 entries, including some redundancies.

The first step to reconciling these facts is basically what you suggest later in your post: that there are multiple directories with symlinks. For instance, ~/bin is in my $PATH, and it has a few dozen symlinks to some of those packages.) But doing this or what you said ("one possibility might involve putting symlinks in /usr/bin to program files into another directory where different versions of the program can be stored") still requires a package manager which can install to a different location, which is not really supported by the Apt toolchain.

Second: just because you have a package available doesn't mean you need it in your $PATH. I have a ton of different GCC versions available, and I use three or four with some regularity. (I use even more of them infrequently.) But I don't necessarily need, say, GCC 4.5 in my $PATH. If I want 4.5 explicitly, I can always run /path/to/gcc-4.5/bin/gcc.

Third: you have to remember who Ubuntu is aimed at. It's aimed at casual users. Casual users don't even care about $PATH in the first place, because they don't start things from the command line. They just hit the windows key or whatever they do to bring up the launcher, choose what program they want to run, and click it.

...and also does not follow the time honored Unix tradition which needs to be respected.

They need to be respected, even if it has significant problems in some environments and fatal problems in others (e.g. mine)?

Unix had a ton of really great ideas. But many of them have run their course. You can't progress if you say you can't change stuff.

Re:I never got "packaging systems"

[cas2000]

actually, it's something that Windows gets dead-wrong, because executable installer apps (setup.exe and the like) are just plain fucking stupid.

they're an unfortunate necessity because windows doesn't have, and never has had a decent package management system. and is unlikely to ever get one because the windows software market is primarily commercial and proprietary.

when you have 10 (or 100 or 500) packages to upgrade on a single system (and then multiply that by 100 or 1000 systems), executing hundreds of installer packages one after another is the worst possible way to do it.

i've never understood why Windows (or Apple) users tolerate that shit. it's a tedious chore that's ripe for automation - exactly the kind of thing that computers are good at doing and users are bad at doing (due to boredom, fatigue, loss of attention, ignorance, or stupidity)

which is precisely why linux distros (and other *nixes) don't do it that way. they have packaging systems because systems are consistent, predictable, and easily automated.

windows users and windows developers often have just the wrong way of looking at things, the wrong mental model of how things work and how they should work.

I ran across a program for Windows recently called Ninite. It's a multi-app installer and updater. it sounds like a good idea and is. it's a big improvement over the usual click-and-execute for each individual program.

except the way it works is weird and clumsy:

you go to their website and select which apps you want to install (from a bunch of internet-available apps, including free software and proprietary freeware like adobe flash), and then it builds you an installer app that you download and run, and it installs and/or upgrades the apps you selected off their website.

WTF?

nice starting idea, but the implementation is idiotic. Why not just have one Ninite app that fetches a list of available apps and installer URLs and whatever custom installer scripts ninite needs for them) and allow the user to select which apps whenever they run it?

i.e. instead of a moronic implementation, actually make a smart and useful implementation that copies good ideas from linux distro installers like apt-get and yum, and re-purposes them for the Windows environment.

(oh, and adobe are sending cease-and-desist letters and threatening to sue if the ninite developer doesn't remove the ability to install & update downloadable adobe products like flash, so his good ideas and good intentions are fucked by the corporate vermin mindset that dominates the windows software market)

another thing windows devs don't get is shared libraries (DLLs in windows terminology). Why does every single app have to install their own copy of the MS C++ libraries? or .net? or nvidia physx? and numerous other common library packages? these things are supposed to be shared resources provided and kept up-to-date by the operating system, not bundled with every app that uses them.

Look into Debian's CUT project

[neiras]

Except Ubuntu users want cutting edge Debian, not tried and tested Debian...and unfortunately using Debian is not going to make it more cutting edge.

...Now if Debian decided to produce a (stable) cutting edge Desktop version (perhaps working with an existing Distribution team). To complete there ultra stable, you have me sold.

You asked for it, Debian delivers. The Debian CUT Project aims to publish usable snapshots of Debian Testing on a monthly basis. They're pretty new but picking up steam.

Say what you will about Canonical...

[foreverdisillusioned]

I've definitely not been a fan of their track record these past few years, but they do occasionally hit on some really good ideas (if somehow managing to screw up their implementations: Upstart was a good in principle, and so was making a verical launcher/taskbar instead of making our widescreens even shorter.) This is another one. Well, sort of. Maybe not 'no dependencies' but at least 'no dependency conflicts, ever.' It's two thousand thirteen, why are we still tolerating this shit? There's absolutely no reason why the package manager can't chroot/symlink the little fuckers and make them see what they need to see so they can coexist with any other package. If the package manager is informed enough to recognize a conflict, it's also smart enough to resolve it without the user doing anything beyond confirming the installation of the extra required dependencies, and yet for some reason none of the current crop of package mangers do this.

I'd rather have a bad solution from Ubuntu inspiring people to do it the right way (see: Mir lighting a fire under GNOME's ass to help out with Wayland) than have no solution at all.

Re:I never got "packaging systems"

[EvanED]

another thing windows devs don't get is shared libraries (DLLs in windows terminology). Why does every single app have to install their own copy of the MS C++ libraries? or .net? or nvidia physx? and numerous other common library packages? these things are supposed to be shared resources provided and kept up-to-date by the operating system, not bundled with every app that uses them.

They are... sort of. Take the MSVC runtime. That's not maintained or installed necessarily by the OS for (some good, some mediocre) reasons which I'll explain in a second, but I'm 95% sure that it's shared between programs in the sense that when you see an program installer that spawns off the MSVC runtime installer, the latter will either be (1) installing a new version of the MSVC runtime or (2) discovering that it's already present and doing nothing. The runtime installs to a shared location. (Look at msvc*.dll in windows\system32.) So the runtime installer has to be bundled with individual programs for the upcoming reasons, but it's not wasting disk space by installing duplicate copies (subject to the caveat below) or wasting memory by loading the same thing from different hard drive locations at different time.

So why does it do this? Well, a couple reasons, I think. (I'm speculating here, but I consider them educated guesses.) First is the mediocre reason: there's no package manager. This leads to the "well, you have to ship the runtime installer with each program."

Second is the pretty-good reason: compatibility and correctness. You can definitely have multiple major versions (e.g. the runtime associated with VS2008 and 2010) installed simultaneously, and I think you might be able to have multiple patch versions of the same library installed simultaneously. I think the former is true in Linux too (libfoo.so.1.0.0 vs libfoo.so.2.0.0, though I thought those only show up for -devel packages... hmmm, not sure), but the latter isn't so much. It may well be that Program A installs version 1.0.0 and Program B installs version 1.0.1239, where on Linux the latter would likely be packaged to upgrade the former. (This is the caveat to my "no wasted space" claim above.)

Why is that good? Well, why did the library change? Bug fix? Well, MS usually will bend over backwards to maintain compatibility, which has often meant bug-for-bug compatibility with old programs, at least as a shim. If you take the Linux approach, then programs which rely on the old behavior of the buggy code will break. This is sometimes good (e.g. bad security-related fixes), but is often not. Or it doesn't have to be a bug fix, it could just be some behavior change within the specification. By keeping multiple versions around, the Windows approach keeps the individual programs happier.

How you weight these various advantages and disadvantages is up to you. I'm not really trying to argue that the Windows approach is better, just explain why it is as it is and give a fair description of what goes on.

Nope..

[Junta]

*If* everyone picked exactly the same lib version, yes.

In practice, people aren't going to standardize on the same library version.

Bonus problem: Now each app provider is responsible for addressing a hypothetical libcrytpo vulnerability rather than the distro patching it in one place.

Re:Nope..

[Burz]

A typical desktop distro is an example of a system that is very 'fat' in terms of available components, but poor in terms of what can be considered integral to the system. There is gobs of duplicated functionality, and the packaging system treats all these pieces pretty equally and a lot of it can get in the way when you have independantly installed apps. So I think the difficulty for upgrading that you speak of is more acute in desktop distros than it is for, say, OS X.

Ubuntu is starting to limit the role of the managed repository: It will continue to be used for the core OS, but apps will increasingly use the new format which primarily specifies the OS version as a dependency to be checked. If the app needs more than what the OS offers, then they are free to include the extra libraries in their own tree. This makes a lot of sense and I hope it catches on with app developers.

Re:grep: command not found

[Anonymous Coward]

Anyone on this site who is stumped by grep missing need leave. Now.

yeah and in MSDOS.....

[decora]

each program came with its own video card drivers, it's own printer drivers, it's own windowing system, it's own mouse reading code, and sometimes it's own memory management code.

Re:Scripts...

[BenJeremy]

SABNZBD is a great example of what I'm talkign about.

On Windows, it's... "click the installer" and it's pretty much done.

On Kubuntu Linux, I had to locate a solid tutorial, about 7 pages long, on installing it and getting it running correctly.

I have nothing against text files as config files, but geez... how about standardizing the damn things? ...and Where the f*ck are my logs? No, they don't get put in the same subfolders every time, and I NEED those logs a lot more in Linux than in Windows.

I'm running a LAMP server and I have Kubuntu running as a media server. Every time I plug in a new USB drive, I have to go through a bunch of extraneous steps just to permanently mount and share the drive on SAMBA - there just isn't a simple way to do it. I have to get the UUID of the drive, and edit FSTAB and the samba config files. Why does it have to be that painful?

I have init.d scripts that work fine on a couple of my python daemons, but not on a couple of other daemons (which manually start fine).

So I'm "holding it wrong?" - sorry, the OS should NOT have me so involved. I'm a software developer, my bread and butter is on Microsoft-based enterprise apps, but my background includes years working in embedded systems. Hell, I've written my own multi-tasking operating systems and built up entire IO libraries for devices that were custom built.

I'm wrong for wanting to drop an installer on my Linux box and have it install without issues. I should WANT to have to fire up the editor-de-jour for whatever flavor of Linux I'm using (let's not forget to fire it up with sudo!!), and sift through a bunch of options, hopefully all of them are documented, to configure it? Again, why? What's the point of having a GUI if not to make life easier for users? Why bother with VNC?

Do you think my parents would use a shell-based OS? Hell no. What I always get form Linux fanatics is the condescending "use the shell" answer, which is complete bullsh*t. Editing text config files isn't much different from using a shell either... Would it kill Linux developers to provide a GUI control panel? Would it kill them to provide a one-click install? ...and why don't they? Oh yeah... because who knows what desktop or distribution they will be installed in.... yes, it's all called Linux, but it's like everybody who gets upset takes theri ball and goes home... then branches off another distribution.

Sorry... venting again.

The point is, it shouldn't be so hard for people who are not autistic savants who dwell on TTY terminals, tapping away on monochrome 80x25 text displays, convinced of their superiority. I don't live and breathe Linux, so don't tell me I'm doing it wrong.

For the record, I've never had to do any "registry hacks" to install any damn Windows apps - not in 20 years of using Windows. Either there is an installer that works fine, or it's a portable app that can be dumped into a folder and run. I've written plenty of apps that can launch, and configure themselves for running as a service, or setting up as a startup app. I've also written a lot of software pretty "close to the metal" - apps running on over 400,000 PCs right now, where security and robustness is important. They work. They don't require any hacks. The user doesn't even get involved in the install process.

Re:I never got "packaging systems"

[cas2000]

Why is it SO hard for people who use linux to understand that there are multiple runtime libraries because [...]

it's not that we don't understand how and why windows DLLs are fucked up, it's just that since we're used to a versioned shared library system that actually works, we see the windows DLL mess as being fundamentally broken.

you don't even seem to understand what's being discussed here - after the second sentence, you digress into an irrelevant rant about kernel versions.

and, BTW, since linux can run even ancient a.out format binaries from the 1990s on modern kernels, the problem you are describing is actually a problem of shoddy practices by proprietary software companies.

Meh

[BrokenHalo]

I've got a better idea. Or rather, Pat Volkerding does. Never mind dependency checking, just let the user sort them out for himself. Slackware's package system is beautiful: a simple *.tgz or .txz file, with a note saying "and BTW, you'll need foo and bar as well". No need for separate directories full of statically linked packages, and it just works.

Of course, I have to admit that Debian packages work too, but if that nice Mr Shuttleworth thinks otherwise, then (sadly) there's another reason not to use Ubuntu.

While I can admire Shuttleworth's passion, he isn't doing anyone any favours by pursuing this "Not Invented Here" ideology. Everyone would benefit if he would just act like a part of the community, instead of trying to railroad it.