Slashdot Mirror
Helping Perl Packagers Package Perl
jamie writes "chromatic has a great post today on the conflict between OS distributions and CPAN's installations of perl modules, along with some suggestions for how to start resolving this maddening problem: '[Though Debian has] made plenty of CPAN distributions available as .debs, I have to configure my CPAN client myself, and it does not work with the system package manager. There's no reason it couldn't. Imagine that the system Perl 5 included in the default package... had a CPAN client configured appropriately. It has selected an appropriate mirror (or uses the redirector). It knows about installation paths. It understands how to use LWP...' The idea of providing guidelines to distros for how to safely package modules is a great one. Could modules request (a modified?) test suite be run after distro-installation? Could Module::Build help module authors and distro maintainers establish the rules somehow?"
Fuck perl.
... is one of the coolest languages ever! There, fixed that for ya.
FTFY.
The World Wide Web is dying. Soon, we shall have only the Internet.
That's the kind of spirit that's made open source what it is today!
For your next trick, I'll bet you'll tell us that emacs > vi
Here's to the crazy ones
Fuck emacs.
Camping on quad since 1996.
A broken mess of modules distributed inconsistently is the quickest way to kill my interest in a platform...
Ruby gems? PHP Pears? Python pies?
The World Wide Web is dying. Soon, we shall have only the Internet.
Well, my next trick was actually going to be all about how I like vi. :)
"I like it when the red water comes out.."
Could a /. reader get laid? Could I go to bed before 4 AM? Could someone get me a real job?
No.
More seriously, not as CPAN works now. There is no "snapshotting" or "consistent distribution level" mechanism, so there is no mechanism to write consistent component compatibility lists, and no way for the existing CPAN to look back *out* into the operating systems available components. Even component naming is too inconsistent among distributions. And when different packagers fold specific packages into their basic Perl package, or components that are required for new modules move into the basic Perl package itself, you have dependency madness just waiting to bite you, very, very hard indeed if you let your local Perl integrator replace all the dependencies. This way lies Gentoo madness, and it's hideously unstable in the real world.
There are steps you can use, but they're dependent on Perl authors actually following packaging "best practices". Building RPM's from many perl components is fairly easy, especially with components like 'cpan2rpm' available, but then you have the crack-monkeys who prodoce component 1.00, 1.20, 1.201, 1.202, 1.2437, and then 1.30 and expect 1.30 to be considered "the most recent", and no way to flush the funky numbered ones from CPAN lest development continue and you get to 1.201 the hard way. And then there are idiots who can't be bothered to write actual Makefiles or Make::MakeMaker configurations, but each invent their own replacement for "make". And now that Perl 5.10 is out, too many "latest release" components form CPAN are going to simply demand updates to that new Perl release. What a wonderful crapshoot to upgrade your Perl, *on the fly* to a new major release simply because you want to print dates a certain way?
Then there's the continuing use of Apache 1.3 and the matching mod_perl by Debian setups, and the utter nuttiness of rolling *BACK* Perl components to be compatible with that. What a wonderful way to *completely* fuck up your existing deployed codebase as rolling back your Perl to perl-5.6 to revert the incompatible components for that one.
Oh, and don't forget the "site-lib" versus "vendor-lib" settings. God forbids most Perl authors be bothered to actually *CARE* about it, but it does make a different in whether a component will be loaded and actually used in place of a separately installed one deployed as part of the operating system's other dependencies or package management system.
At least the Perl crowd tries to solve this problem. The Python crowd has a terrible time coordinating distribution of third-party modules. That's why it's taking forever for Python 3.x to get deployed. Red Hat Enterprise 5, Red Hat's flagship product, still uses Python 2.4, released in 2004. (There's a Python 2.5 included, but it's not the one the system tools use.)
Perl has CPAN, which is reasonably well organized and well run. Python has the Python Package Index (formerly called Cheese Shop), but it's not well coordinated with Python releases. Things seem to be improving, but Python is 20 years old now and ought to have a mature distribution system.
This module installs a tool which installs modules directly from cpan but enforces the conventions of the native environment. So if there is a perl module from the debian repositories called kludge and the same module is available directly from cpan, the cpan module would understand that they were the same basic thing, and know how to relate the different versions.
http://michaelsmith.id.au
How much Perl packages would a Perl packager package if a Perl packager could package Perl?
FreeBSD already does this! Installing a package via cpan will create the metadata and register a FreeBSD package.
common-lisp-controller is an attempt to make Common Lisp package management play nicely with Debian package management, so, for example, trying to load a package within ASDF (Lisp) finds the appropriate Debian package if it's installed (like the linked article discusses doing for CPAN).
Strongly package-manager-based distributions like Debian usually don't put a lot of effort into making stuff outside the distribution work well, though. If you want some C library that isn't in Debian, you're going to have to build it yourself and stick it somewhere, and all the stuff in Debian isn't going to know about it. Similarly, if you want some CPAN module that isn't packaged in Debian (the Perl equivalent of a third-party shared library that isn't packaged in Debian), you're going to have to muck with it yourself, and it won't be in the usual dependency chain.
The usual Debian answer to these sorts of problems is: well, if it's maintained, useful, and under a DSFG license, let's just package it! Things get much less nice if the user is installing a bunch of custom stuff, and especially trying to get locally installed and package-managed stuff to play together nicely. CPAN having trouble with that is just one instance of nearly every other system that has its own custom package manager, from Python to GNU R to Scheme.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
The problem is module dependencies... any non-trivial module now has so many dependencies now that it is almost inevitable one will fail to install. And than you're kind of screwed (unless you go and build it b hand.) And then there's the insanity of auto-updating perl itself to get a module. CPAN's badly broken and needs to be replaced entirely, which is a lot of why I pretty much quit using perl for a long time. Recently, my job changed and I got a lot of perl code. Man, I miss Ruby now.
"He who would learn astronomy, and other recondite arts, let him go elsewhere. " -- John Calvin, commenting on Genesis 1
if you have a list of offending modules, then by all means don't keep it a secret
How much Perl should a Perl Packager package, if a Perl Packager could Package Perl?
Ask me about repetitive DNA
Just new, in fact.
CPAN gets its own directory tree, and the user can install modules to it in the usual way, but system packages can also depend on CPAN modules directly. A dependency on "CPAN:XML::Writer >= 0.604" from a system package will install XML::Writer in the CPAN tree, rather than having everything repackaged and duplicated or conflicting with packages from the distribution. The aim is to cover all the common domain-specific package managers (so also RubyGems, LuaRocks, PEAR, ...) on the same level.
I have a paper on it in the Distro Summit at linux.conf.au next month, and until then an overview of the system is in the linked blog post.
but "Helping Perl Packagers" or "Helping Package Perl" would be sufficient If you package Perl you're a Perl packager, and every Perl packager packages Perl at some point :)
Cpan works people. And it works well. It works on Windows(strawberry perl) it works on Debian, it works on Mac. When it fails, it's not that hard to figure out where and quickly get the scope/scale of the problem worked out.
Besides, it seems to me the whole packaging scheme is designed to be kind of loose on purpose. Criticizing it for a design feature is pointless.
If you really insist on blasting away at Perl, then you'll be blasting away at Python soon enough. Shocking but true, there are packaging issues with Python too.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
cpan on the other hand has only probably worked 50% of the time for me, with it frequently hitting an unresovlable problem with dependency's.
If you mod me down, I will become more powerful than you can imagine....
It sounds like homebrew provides a good solution. Perl (and Ruby and Python) already have mature packaging systems and they really don't need to interact with each other. So homebrew is a smart packaging system that plays nice with others.
The CPAN installer hasn't had the Perl auto-upgrade bug in several years...
Is the parent a bad post, or a good troll?
Some responses if I may...
Perl dependencies are specified by class name, not by distribution package name. So (theoretically) as long as there's a way to resolve a class to a file (which is standard) and thence to an operating system package ("which package contains file X" shouldn't be that hard) then there's no reason that Perl package dependencies can't be mapped down into distro package space.
As for the versions, 1.30 is more correctly more recent than 1.2437, because the CPAN turns multi-part versions 1.23.1 into decimals using an admittedly icky triplet system where each part of the multi-part is normalised into three digits.
1.2437 is a normalised version of 1.243.7. Downstream distos have implementations of this logic available to them in places like CPAN::Version. But yes, it is a bit weird for the newcomer. It's the price of 5-10 year back-compatibility, alas.
As for Perl 5.10, almost nothing on the CPAN will depend on it. The current recommended back-compatibility targets are Perl 5.6.2 for low-level or toolchainy stuff that needs a decade of back-compatibility, or Perl 5.8.5ish (around 5 years) for regular things (which is the first version where Unicode became bug free and universally usable).
So Perl 5.10 is having almost no impact on compatibility, and won't for at least another couple of years.
CPAN has had its problems, but it's been reliable for me (after learning its quirks). Even so, XS components which rely on shared libraries have added complexity to the distribution dependency resolution, configuration, and installation processes. Recent developments have improved this, but better integration with package managers can only help.
I did forget to mention BSDPAN (and a grant proposal to extend the reach of BSDPAN).
how to invest, a novice's guide
apt-get install dh-make-perl ; dh-make-perl --cpan Your::CPAN::Package --install
As for the CPAN client that asks too many questions - that's a matter of pre-configuring it for a distribution or for the installation options you chose when installing the distribution (it's not CPAN.pm's problem really, although it could indeed ask fewer questions).
"I love my job, but I hate talking to people like you" (Freddie Mercury)
Dependency's what?
This isn't just a Perl problem; there are several packages that I know of that different distros have problems with.
I think it's more the nature of F/OSS: anybody that can repackage things, will -- just like anyone that wants to, can cobble together their own distro.
I don't see the problem going away for Perl, or any other package -- or even for the various distributions. As long as there are True Believers in .deb vs .rpm vs. git, /usr vs /opt, Gnome vs KDE, and so on, I see it continuing to be a problem. Nobody wants to give up a little bit of their "freedom" to do-as-they-damn-well-please in order to establish some consistency and minimum standards so as to make life easier for mere users. I've previously suggested that the fragmentation of Linux (of which this particular situation is just an example) is what's REALLY keeping Linux off more desktops.
But, hey, what do I know? I'm just one of those folks that only wants to get actual productive work done, and remembers what it was like for me when I made the switch from Windows(tm) to Linux.
--- Asking inconvenient questions for over 30 years...
I recall the same issue with the Ubuntu and the Ruby community over RubyGems.
Ubuntu packagers would have preferred the use of APT instead of RubyGems for Gem installations, despite the fact that APT's lagged behind RubyGems.
It works on Windows(strawberry perl)
Right. Sure it does. Try to get the DBI to recognize Postgresql. I'll bet you a round of beer that it won't install. Ever.
CPAN was great. However, the sheer number of modules that don't even install is staggering and now CPAN is a bloody mess. I sincerely hope that with the (the time is near!) coming of Perl 6 it will be seen as a good opportunity to start all over with a fresh CPAN6.
There are tools that can do a decent job of packaging some modules. For example, using the CPAN2RPM tool you can create a SPEC file. The SPEC file can then contain a Requires and Provides section for generating dependency lists. These lists can contain version information such as a requirement that a package be within a particular version range. Frontend tools such as yum can then use these dependency lists to pull in appropriate modules contained in RPMs. But yeah, it won't (automatically) solve the bizarre and arbitrary versioning schemes of some modules, but it can be manually worked around by adjusting the RPM version versus the Perl module version.
Seriously, if Debian is 'breaking functionality' that makes part of their own distribution unusable, wouldn't it makes sense to file a bug report directly with Debian? While you are at it take all the Perl package maintainers and have them help elevate the priority of that bug report so they can't just ignore it. Having it officially declared a bug would be a logical first step.
OMG alliteration screw c++
I totally agree. Linux has Balkanized, shoving anything that wants to run on all distros into the same problem. Anyone who has done an "upgrade" knows how maddening (and rediculous) this can be. But who (or what) is going to "standardize" Linux? Is FreeBSD the answer?
In "Perl World" keeping Perl modules up to date is important. In "Distro World" maintaining a consistent and maintainable distro is important. These two desires are at odds with each other.
Does updating Perl Libraries ever result in some older Perl app or script breaking? My guess is that there is a strong possibility. Distro makers/maintainers are concerned that updates that occur without their knowledge will result in unpredictable chaos and difficult-to-track bug reports. By making the set up of CPAN deliberate, you are taking the destiny of your installation into your own hands -- and there is nothing wrong with that in the slightest, so long as you are aware and capable of managing your own operating environment.
Perl is an important, key-underdog player in the Linux ecosystem. I can't imagine Linux without Perl. However, the same goes for a few other components which are not updated by distros at the very same pace as the developers of those components release them. And there are many reasons for this -- good ones, bad ones and ugly ones.
Would it be nice if CPAN were configured right out of the box? Yeah... if you were a Perl developer. And there's nothing to stop a Perl developer from custom building his own Linux distro based around CPAN and all that. In fact, I would be really interested to see what a Perl centric Linux distro would look like. I believe an entire OS could be written in Perl... not saying it is a good idea, just that I believe it could be done ... from the memory management to the GUI interface, I think it could be done. But I am certain such a distro would be incredibly unique and speak of the differences of mind that Perl developers are known for. But it would be difficult and I think it would not take long before Perl fans begin to appreciate what distro-folk deal with... or it could be as perfect and ideal as Perlies say it would be.
Sorry guys, but I do a LOT of really large perl based projects. CPAN is fine. Hell, it works better than the supposedly wonderful rpm and apt based package managers. At least when you install from CPAN you know the stuff WORKS because it actually gets tested. The CPAN dependency system works fine too. All I can conclude is that people who have had problems with it have a whole lot of RPM installed packages that were guess what? BADLY PACKAGED by the distro.
I agree, it would be great if CPAN and package managers coordinated. There are things CPAN certainly lacks, like transactions and any real ability to uninstall. What it does do, it does well.
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
As usual, Gentoo is a step ahead of the competition in this regard (and has been for a long time):
jps@karma ~ $ eix g-cpan
* app-portage/g-cpan
Available versions: 0.13.01 0.13.02 0.14.0 ~0.14.1_rc1 ~0.15.0 0.15.0-r1
Homepage: http://www.gentoo.org/proj/en/perl/g-cpan.xml
Description: g-cpan: generate and install CPAN modules using portage
Since Portage is only a collection of installation instructions, any kind of vendor package is suitable for it; this is unlike the primitive package managers that come bundled with every other distribution that still have problems with vendor packages as well as software which they have no license to redistribute.
but what is the compelling reason that developers still stick with Perl, when there are more elegant languages out there? Python, Ruby and a few other scripting languages come to mind. I realize that Perl is a mature option, with lots of libraries/modules available. But at this point I can say the same about some of the others.
Perhaps because it would be too tough to totally revamp a project and rewrite it from the ground up? Even then the syntax and logic behind Python, Ruby, etc. are close enough that a decent developer could wrap their head around the nitty gritty details.
All I know is when I review some old Perl code I've written back around 1999 or 2000 and then place it alongside of similar work in Ruby I get cross-eyed over the asthetic ugliness of the Perl code. To each their own I suppose, so like I said it's not a troll. Just my US $0.02 :-)
For your next trick, I'll bet you'll tell us that emacs > vi
Fuck 'em both; ed is The One True Editor.
Remember, the first rule of Fuck Perl is that you do NOT TALK ABOUT Fuck Perl.
Fuck perl,
python is one of the coolest languages ever!
There, fixed that for ya.
Happy to be of service.
"Perl gem" sounds good!
Nae king! Nae laird! Nae yurrupiean pressedent! We willna be fooled again!
Fuck Python.
I usually download the tar.gz for a perl module, then it's a simple 'perl Makefile.pl && make && make test'. Install honors DESTDIR, so then I can package it myself, every time. If my distro ever does release an updated module, my package system should pick it up then.
Gems on the other hand, I haven't been able to package at all. Best solution I've seen is Debian, they set up a 'quarantine' under /var/lib/gems, with it's own bin directory and everything, to keep gems away from Debian packaged ruby libs. Then you get to fight with vendoring, config.gem, initializers, etc. I got to the point on one app where I just gave up and copied the libraries into RAILS_ROOT/lib. I sure hope rails 3.0 improves this.
chris@xanadu:~$ whatis /.
/.: nothing appropriate.
I am beginning to think emacs is, in fact, greater than vim (emacs is obviously > vi). But emacs key bindings are still far behind vim's
Climate Progress - Hell and High Water
Then I would say that you have no idea how to use CPAN. I have never had an "unresolvable problem" using CPAN.
Perl has problems, but there's a lot more irritating problems in the FOSS world.
Why, for example, does everyone have to develop their own versions of 'make'? Sometimes using two or three different varients in the same package. The Linux kernel uses it's own configuration routines, while many modules use somethings else, and libraries will use yet another method. Every time you need to add a module to drive some strange hardware, you have to spend several days just trying to figure out how you are supposed to compile it, just because the author of that package found a "better" way of handling the build process. It's really bad when some parts of the build use automake/autoconf, others use plain make, and others use some self-developed construction. Pick ONE package build process, don't pick every one.
Who would win this election: Andrew Weiner vs Andrew Weiner's weiner.
Please stop spreading FUD around. This bug (Perl updating itself through CPAN) has been fixed for several years.
At least once a month I need to setup a new Perl install with all the dependency-heavy modules and I have no issues whatsoever - if it's failing for you, you're clearly doing something wrong.
Have you considered asking for help instead of spreading FUD on Slashdot?
Close it when CPAN co-operates with the most common package manager on business linux systems - namely RPM.
Companies that make stuff that people actually need usually run SuSe or Red Hat, if they run any linux at all. Why not simply have a CPAN plugin that keeps the RPM db updated? Don't use it if you don't want it.
Ideally, Red Hat and Suse should be paying for this, since their customers would be the ones profiting by it.
Bit me on Ubuntu 8.04. Which is still the most recent LTS release, and readily available at VPS providers like Slicehost and Linode.
tasks(723) drafts(105) languages(484) examples(29106)
Apologies to Python/Ruby guys, but I have yet to encounter a very compelling reason to switch from Perl for the tasks I've been using it. Maybe it's just me.
Perl's object system blows but it has CPAN.
Ruby is slow and its community doesn't get coarse-grained dependencies.
Python is memory hungry and breaks compatibility frequently.
Perl 6 fixes the object system and is finally on a steady march to reality. I would have expected Ruby or Python to win by now but my money is on Perl 6 being the best option in the near future.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Is Rob Malda aware that you are trying to destroy his website?
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
distribution installed perl for distribution-delivered scripts
my own copy with using cpan and installing/updating what my perl scripts need
-> the only change is that in my own scripts the path to perl is different
>>It works on Windows(strawberry perl)
>Right. Sure it does. Try to get the DBI to recognize Postgresql. I'll bet you a round of beer that it won't install. Ever.
I haven't tried Strawberry perl, but it certainly worked out of the box for me with ActiveState. Did you install DBD::Pg?
Butterfly.
I know tobacco is bad for you, so I smoke weed with crack.
See post above, which I wholeheartedly second:
"Bit me on Ubuntu 8.04. Which is still the most recent LTS release, and readily available at VPS providers like Slicehost and Linode."
Both the ActiveState website, as the Postgresql website state that it isn't even available on Windows. Also, the Perlmonks website indicates it can't be done. So mad props to you if you have gotten it to work.
http://www.cs.cmu.edu/~blangner/perl/dbd_pg.html
http://archives.postgresql.org/pgsql-general/2005-05/msg01293.php
http://www.perlmonks.org/?node_id=568380
Fuck brainfuck.
Ah! The workaround is...
1. Use CPAN to upgrade itself.
2. Install everything else.