Package Management 101

I've been thinking about this problem for a while now. I spend a fair amount of time maintaining various systems across all three major platforms: Windows, OS X and Linux/BSD. I have conclusively decided that package managers in Open Source environments are bad, unhealthy and frustrating to use. They aren't just bad around the edges - they truly suck. Here's why, and how to fix them.

Package management is all about making maintaining a system trivial. It means that you can keep up-to-date with latest security and feature releases without having to spend a significant amount of time-bandwidth downloading, unarchiving, relearning the install process, and then configuring, making and finally installing -- that's 6 steps to upgrade something you already had there. Upgrade means to go from one release to another, more recent release. It does not usually mean 'reconfigure' but it certainly means 'alert me of new features / changes of behavior'. You don't want upgrades to be always silent.

Clearly installing-from-source every time is not a viable solution. It's too much work. Whilst you'll always usually get the best setup, you're not going to be actually able to use it since you're spending all this time making it work. There is one other alternative currently on the market. That is the job of a package manager. These range from Microsoft's Window's Update functionality to Apple's OS X Software Update, to more 'raw' managers such as Red Hat's up2date/rpm and Debian's apt-get/dpkg, and finally BSD's port family.

Using these isn't so easy either. If you are on the windows platform, you can either use windows update, or choose to remain unprotected. On OS X, you use Software update, or go it alone, with probable disastrous results. But with the Open Source family of packages, the story is a little different.

The user is presented with the awesome power provided by having absolute control of which packages to be put where. Actually, this isn't awesome control - this is the only option - the user really does have to have some kind of almost god-like knowledge of the system to be able to use it. It is possible to achieve a level of silence from the package managers, though, but to do so you have to accept in blind faith that the configuration options provided by the distributor are good for you. Sadly, too often they are not. There is no happy medium.

Take my recent subversion installation adventures. I'm trying to install subversion client, which has a number of dependencies. I don't want to have to spend time configuring from source: I did that on another box six months ago, and I don't have time to do it again. I just want to get svn installed without having to do any work, but with a reasonably sane configuration. I'm using RedHat enterprise, which is a reasonably average-yet-stable operating system to start from. First, i tried up2date, to learn that there are no packages called svn or subversion. Ok, it's fairly new and clearly no-one at redhat has built and qa'ed a spec yet. I'll bug them about that later. But other people have, so I resort to rpm. Of course, i have to deal with the dependencies, and, one of these is apr. APR is not installed as a package itself, but it's part of httpd. Here are my experiences:

% rpm -qa|grep apr
% rpm -qa|grep http
httpd-2.0.46-25.ent
redhat-config-httpd-1.1.0-4
% rpm -e `rpm -qa|grep http`

redhat-config-httpd-1.1.0-4&38;38;38;gt; ~error: Failed dependencies:
httpd-mmn = 20020903 is needed by (installed) mod_perl-1.99_09-10.ent
httpd-mmn = 20020903 is needed by (installed) mod_python-3.0.3-2.ent
httpd-mmn = 20020903 is needed by (installed) mod_ssl-2.0.46-25.ent
httpd-mmn = 20020903 is needed by (installed) php-4.3.2-8.ent
httpd-mmn = 20020903 is needed by (installed) mod_authz_ldap-0.22-3
webserver is needed by (installed) webalizer-2.01_10-15.ent
httpd &38;38;38;gt;= 2.0.40 is needed by (installed) mod_perl-1.99_09-10.ent
httpd &38;38;38;gt;= 2.0.40 is needed by (installed) mod_python-3.0.3-2.ent
httpd = 2.0.46-25.ent is needed by (installed) mod_ssl-2.0.46-25.ent
httpd is needed by (installed) squirrelmail-1.2.11-1

Yowzer! all this to remove a package which I know contains another package I need to remove because of conflicts? What is httpd-mmn? so on and so on. Fink is another great example. It's like apt but for OS X, and like apt, it has a habit of infuriating me whenever I come to use it. Recently trying to install subversion managed to make fink believe I had 64 other dependencies to install first. 64?? That's crazy. I don't need X to build subversion, nor do I need ruby (though i'm glad to have it)!

These are just two examples of 'bad practice' in package managers. They exemplify the horrific nature of the all-in-one approach that so many managers feel they must provide. Yes, you could answer '"read up on how the thing works, it can solve this for you", or, more pragmatically, "just man manager" (although, quite frankly, i'm getting tired of hearing that answer). I think these are just quick-escapes. I think we really do have a poor system, which is unfriendly to users old and new. I think that we should all do better.

Here therefore, is my manifesto for a better package manager:

1. Don't make it hard. Users don't want to have to jump through hoops to get a simple tool. These users may be developers who have a high level of Linux/BSD knowledge, but don't have time to waste screwing around with a package manager.
2. Don't assume stuff. I might not want to have mod_optional_hook in my apache, so why do you force me to install it and then go 'un-configure' it? This is loosely coupled with:
3. If in doubt, ask. There is no harm in collecting user preferences during install time. Just try to be intelligent about it and don't bother the user - keep it quick, simple and easy to use. (e.g. that means allow for backspace on user input so that if i typo, i don't get foor^? or similar).
4. Learn. It's really important to ask once, and remember. That's also not so hard. I don't mind spending a while configuring a package once, but i don't want to ever have to do it again, unless there are changes to the configuration options. That's easy enough to parse from ./configure --help -- it's usually a pretty standard format, at least until a database of configure options is built.
5. Any package manager MUST work cross platform. Package managers don't need to be distribution dependent. They should be bootstrap-able, i should be able to download a package (binary or source, but binary should be in case there isn't a developer chain on the target machine) and simple install the package, run it in bootstrap mode so that it will create initial configuration models (and, for bonus points, why not provide an ability to synchronize basic / local settings from other (base) servers so that i don't need to setup my preferences on new machines?
6. PMs should know where to look for packages. We all hate how sourceforge makes us navigate and find package download locations. It really is annoying to have to browse for it and get to a point where it downloads the package, when really you want to curl or wget it to your server.
7. Packages should auto resolve dependencies, or co-exist. I don't care if you need httpd or neon or whatever dependency - iI just want to install the thing.

I am sure there are more to add, however this is a good start. We should work hard to find a package manager solution which can effectively deal with all our requirements, and in such a way that we do it in a non-partisan and effective method. I look forward to your comments!

package-managing