Slashdot Mirror

Look at the MP3::Info Perl module, you might recognize the author's handle. It reads (and writes) tag info. It's used by the "jukebox" module Apache::MP3 (sample site) to generate pages with track info.

Basically every web jukebox out there does something like this so I'm sure there's plenty of other code available to work from. The mod_perl way is to put SetHandler perl-script then PerlHandler [name of module] in your httpd.conf file so when a URL request falls within that Location or Directory, the perl module handles returning whatever you want it to return.

I like Apache::MP3 also. Namp! is the name of the project when all the bits are rolled together (apache, mod_perl, perl, Apache::MP3). Also CPAN is your friend.

There's a demo site so you can see the default interface and try some streams (Apache::MP3 includes a "demo" mode which stops the streams after 30 seconds).

You can block casual access with a simple .htaccess file. I'm pretty sure it *will* work on Windows, Apache, perl, & mod_perl are all available on the platform, it's just more work because all those components aren't already there.

I'll tell you two problems I've run into. If you use username/passwords in .htaccess to secure it, the username & password will be a part of the URL for each streamed track and may be clearly visible on the desktop, depending on which streaming client you're using. Also some older clients may not work with URLs that include the user:pass in it. It's been a while but I think Windows Media Player was the one that gave me the most trouble.

Embedded album art in a track may also cause trouble for some clients, specifically iTunes and RealOne (v9 at least, haven't tried the beta). In my testing the album art was added by MusicMatch and iTunes adds them another way (so each app can't see the other's album art) so how the art is added to the track may be a factor. Actually, I think it's more likely that some clients just can't handle streaming tracks with too many bytes of ID3 tag data but I haven't tried any experiments to prove it.

Whether or not you can fast forward or rewind *within* a track depends upon the client. WinAmp does it like a champ. I'm pretty sure Xmms does too. iTunes does not. Someone has told me RealOne Player can do it but it hasn't worked for me.

iTunes is a bad streaming client because it permanently adds each streamed track to your Library. You have to manually select and delete them to clean it up.

If you don't want to bother streaming your own music, I recommend the "Internet radio station" RadioParadise. 128Kbps (or lower in a variety of formats, eclectic, listener-supported, no ads.

SpeedLimit works by limiting the request rate of each IP address. If your web site consists of many small files (which sounds like the case), then curbing the request rate is enough to cure the most abusive bots. A determined adversary can still circumvent request rate limits with wget --random-wait, but it will be more frustrating for them, and a large percentage of clients can be expected to give up altogether.

If your web site has lots of large files, then (tooting my own horn here) your best choice is to use my mod_limitipconn module together with mod_bandwidth. The mod_bandwidth sets a total limit on traffic and the mod_limitipconn ensures that any single IP address gets only its fair share of that total traffic. I would also advise you in this case to use mod_bandwidth's built in ability to exempt small files from the bandwidth limits.

If your situation is such that the vast majority of visitors intend to download your whole web site, then your best option is to seed a bittorrent tarball of your whole web site.

That should just about cover all the bases. No single one of these proposals is ideal, but you have to realize that overcapacity has no elegant solution. The goal is to manage the situation as best you can using the available tools.

Bah. Better to take the Eagle Book and start making a module that returns 403 Sorry, try again after the media fuss is over error if the request Referer: header matches regex slashdot\.org... =)

For example, a site dedicated to a specific topic, such as Apache, or book reviews? Is there enough of a market, or interest, in any one of these categories? How many new stories are there every day about, e.g., Apache, that would interest readers without becoming ApacheWeek, an Apache mirror, or a site dedicated to programming Apache? (not that those are not all great ideas!) I'm definitely OK with being that technical, but the question is, would there be enough reader interest for it to be worthwhile? (i ran with the Apache topic here, but it could apply to a lot of other topics.)

darren

1. Php Pros:
   1. Easy to learn. Once Apache is configured with Php, it is seductively easy to write code.
   2. The Db connection pooling comes in handy.
   Cons:
   1. It is yet another language (I mean, I already know perl, right? Why do I have to learn another language?).
   2. Also, you tend to design page by page. it does not have a great library system to use. It has lots of code snippets you can copy.
   Notable sites: Sourceforge.net, and persiankitty.com (reputed to be the yahoo of porn))
2. mod_perl: This beast can be used in several ways. If you use it with Apache::Registry, y use Php,ou almost are doing a disservice to yourself. The three ways of using it are:
   1. embperl: Looks like Php style of embedding. Comes preconfigured with DBI and friends. But, too cumbersome to programs. It also does not encourage component programming. It provides the substrate for you to build other features you might like.
   2. mason: Has cool component features. Has neat features such as caching with intesting way of managing it, and autohandlers. Looks ideal for the publishing world, where it evolved. On the cons side, not too many components.
   3. Apache::ASP: Not used it much.
   Notable sites: dejanews uses embperl. techweb, stamps.com use mason.
3. Zope: Too different from others. The strongest suite is the built-in authorization systems and web based management system. You can program at embedded HTML level called DTML, or you can write python classes. There are several prebuild modules that run out of the box. As long as the prebuilt modules serve you well, you can go ahead and use zope. Otherwise, be prepared for a long learning curve. It almost looks black magic to me.

   As an aside, python has the weirdest variable scoping and declaration rules. I ought to know, I have a PhD in programming languages.

4. ACS: ACS is entirely different from others. It is not a language like Php, it is not a webserver like APache, it is not a app server like zope. What it is one man' use Php,s vision and experience about community web sites distilled into a toolkit. It is a datamodel, it is a set of practices, and a set of programs. It happens to use TCL, Oracle, and Unix. But that is almost besides the point.

   The biggest selling points of ACS for me are:

   1. The documentation. Quite possibly the best documented system. I can take an average joe out of the street and train him to use ACS in a systematic way in no time.
   2. The Data model. I would expect to pay in hundreds of thousands of dollars for such a datamodel. It constantly amazes me to find the little details that I needed in those datamodel.
   3. Best practices to run a website. How to harden a Unix system and set up the services so that you can sleep peacefully.
   The negative points are:
   1. Alas, it requires Oracle. I just learnt the existence of ACS/pg and I am rejoicing!
   2. Not too wide spread usage. I expect this situation would change. Look at this way. I tried building a community web site. It took me untold hours to gather all the snippets of information for a Php based site. It took me no time at all to build it using ACS. I would use ACS for a community based website over any other toolkit anyday.
   3. No good enough template mechanism. Fortunately, ADP in AOLServer is changing that.
   So, in conclusion, php and modperl and zope vs ACS is not a right question. You can implement ACS in php, and I frankly hope somebody does to save lot of human misery and suffering.

   Rama Kanneganti

But this isn't want you want to look at for the CGI performance issue. You'll get an order of magnitude (10-40x) by using Apache's mod_perl to pre-load the pre-compiled programs directly into your httpd daemons. The amount of support for Apache in Perl is phenomenal. In the Apache directory alone on CPAN, we have all these:

Apache-ASP Apache-Album Apache-Archive Apache-AuthCookie Apache-AuthLDAP Apache-AuthPerLDAP Apache-AuthenCache Apache-AuthenN2 Apache-AuthenNIS Apache-AuthenNISPlus Apache-AuthenPasswd Apache-AuthenPasswdSrv Apache-AuthenRadius Apache-AuthenSmb Apache-AuthenURL Apache-AuthzNIS Apache-AuthzPasswd Apache-AutoIndex Apache-CIPP Apache-Cookie Apache-DB Apache-DBILogConfig Apache-DBILogger Apache-DBILogin Apache-EmbperlChain Apache-Filter Apache-Gateway Apache-GzipChain Apache-HeavyCGI Apache-Htgroup Apache-Htpasswd Apache-Htpassword Apache-Icon Apache-Language Apache-Layer Apache-LogFile Apache-MimeXML Apache-Mmap Apache-Module Apache-OWA Apache-OutputChain Apache-PHLogin Apache-ParseLog Apache-Peek Apache-Perl-contrib Apache-PrettyText Apache-Proxy Apache-ProxyPass Apache-RandomLocation Apache-RedirectDBI Apache-RefererBlock Apache-ReverseProxy Apache-RewritingProxy Apache-Roaming Apache-SSI Apache-Sandwich Apache-Scoreboard Apache-Session Apache-SetWWWTheme Apache-Stage Apache-SubProcess Apache-Sybase-CTlib Apache-TempFile Apache-Throttle Apache-TimedRedirect Apache-Traffic Apache-TransLDAP Apache-UploadSvr Apache-Usertrack Apache-VMonitor Apache-correct_headers Apache-ePerl ApacheDBI ApacheMysql B-LexInfo HTML-EP HTML-Embperl HTML-Mason Taco eperl libapreq mod_perl

Of course, eventually even this breaks down. I don't think you want to handle 100,000 hits per second this way. For that kind of situation, you need to look into much more sophisticated systems of redundant daemons, sometimes with highly clever dispatch mechanisms way down low, such as with TCP splicing. See the latest Usenix `USITS' symposium proceedings for things in this realm.

• Writing Apache Modules with Perl and C by Lincoln Stein and Doug MacEachern.
• Damian Conway's Object-Oriented Perl.
• Elements of Programming With Perl by Andrew L. Johnson.

• Most Improved Kernel Module: FreeBSD's Linux emulation module.

• Unsung Hero: In descending order:
  • Kirk McKusick, for his more than two decades of tireless service and personal sacrifices for our community.
  • Gurusamy Sarathy, Perl project release manager, responsible for bringing fork(2) to Microsoft ports of Perl and a million other things to make Perl code truly robust and portable between Microsoft and Unix platforms, a true Godsend for those of us forced to co-exist on both.
  • Malcolm Beattie, for trailblazing the Perl-to-C compiler, the Perl external byte-code interpreter, the first Perl/Tk implementation,threading in Perl, and safe blackbox compartments for mobile agents in Perl.

• Best Newbie Helper: Mike Stok from comp.lang.perl.misc. He is patient and kind, never chiding nor arrogant. He has been doing this job for many years.

• Most Deserving Open Source Charity: The Usenix Association. They don't take sides. They promote technology and open standards while remaining vendor neutral. They promote all aspects of advanced technology, but are especially supportive of open source solutions. No organization has done more to legitimize us over the last twenty-five years.

• Best Open Source Advocate: Larry Wall. He doesn't rant against anyone, tries to help everyone, and gives his code away for use by anyone, even Microsoft users. He doesn't restrict his good works to things that only benefit his friends. He doesn't preach, but lives by example.

• Best Unix Desktop Eyecandy: The newest version of the randomizing X screensaver. It's really great in a room full of people on acid.

• Best Unix Desktop Earcandy: The following entry in one's .Xdefaults file:

  *visualBell: on

• Best Desktop Theme: ShinyMetal

• Best Open Source-Related Book: In order of highest to lowest, all worthy of the award:
  • Writing Apache Modules with Perl and C by Lincoln Stein and Doug MacEachern.
  • Damian Conway's Object-Oriented Perl.
  • Elements of Programming With Perl by Andrew L. Johnson.

• Best Perl Module: Damian Conway's Class::Multimethods module for traditional OO in Perl.

• Best Apache Module: mod_perl; how can there be any question?

• Best Open Source Text Editor: The vim editor (vi improved), complete with its gvim graphical incarnation and its perl and python plug-ins.

• Best Deserving of a $2,000 Award:
  1. The late, great Rich Stevens's children's college fund
  2. Larry Wall's children's college fund
  3. Dennis Ritchie's retirement fund.
  (Perhaps those count as charities :-)

• Best Designed Interface in a Graphical Application:
  1. The eesh shell for controlling Enlightenment.
  2. The ddd debugger
  3. MacOS X's environment.

• Best Designed Interface in a Non-Graphical Application:
  1. The {Free,Open}BSD ports collection: being able to just cd and type make and have everything happen is the best thing that ever happened to third-parts apps.
  2. The make menuconfig directive for building Linux kernels.
  3. The v4.0 trn newsreader, with scoring and plug-ins.

• Best Dressed: Larry Wall, whether he's wearing Hawaiian shirts, tie-dies, or best of all, his outlandish, pastel-coloured tuxedos.

• Favorite Slashdot Comment Poster:
  1. Guy Harris
  2. Tom Christiansen
  3. Enoch Root
  4. Jay Maynard

• Favorite Slashdot Author: David Brin wins this one hands down.

• Best Slashdot Story of 1999: Eric Raymond's story about viruses on Microsoft vs Unix.

• Big Dumb Patent Bully: Amazon, followed by Unisys.

• Big Dumb Domain Bully: NSI, followed by Etoys.

• Clue Stick Award for FUD in Journalism: Slashdot.

There are links to Amazon.com and O'Reilly.

Cheers,
-jwb

If people are so concerned about losing advertising revenue, then perhaps they should look into a way of fighting the "perpetrators" on their own terms--technologically. Put some anti-framing javascript in the page, or have the web server check the HTTP_REFERER variable to ensure that the pages are being snagged from their own links. Or--here's a novel idea--put the advertising in the HTML. A pretty simple ad serving CGI would handle rotation of ads very simply. That way, when people link to your internal page, they are getting one of your ads at the top of the page. Of course, this breaks down when people start running all their pages through an ad-f iltering proxy server or something similar... :)

I'm sick of people whining about how people are stealing their content. Get over it, or get into a different business. This is one of the prices of doing business on the web.

[PHP] was easier and faster than Perl for CGI stuff.

--tom