Slashdot Mirror

Roadmaster asks: "Usually, the most effective spam filtering techniques are somewhat resource intensive. Heuristic checkers like Spamassassin, or bayesian filters like spamprobe are processor and storage hungry. This is fine for small setups; I've been using spamprobe to filter spam for 3 users with great results. I'm now however faced with a big challenge: a mail server that will eventually be handling mail for over 50,000 users and needs to have some sort of anti-spam measures. What are some good and computationally cheap spam prevention measures?"

"Ideally, I'd prefer something that does reject the message if it's spam (SMTP result code 550 or something like that), unlike current Spamassassin or spamprobe setups that accept the message and only later decide whether it's spam. Solutions like MAPS RBL, ORBS are acceptable altough commentary on their accuracy would be welcome. Other possibilities I've thought of include checksumming (Vipul's razor or DCC) and simple header checks that could be implemented for instance in a sendmail milter.

Are several quick checks (DCC + RBL) accurate enough and still cheaper than one slow check (Spamassassin, bayesian filtering)? does stacking of similar techniques improve accuracy significantly? (DCC + Razor, RBL + ORBS). How can the good but expensive techniques be made cheaper? (Spamassassin's spamproxyd, hashed wordlists for bayesian filters, and so on). Discussion on all these aspects would yield some interesting conclusions on quick and efficient spam filtering."

Deagol asks: "After reading about it several times on Slashdot, I decided to start the Hacker's Diet this month. I've even lost 3lbs so far. I'm looking for software tools to make this thing easier. So far, I've been using Nut to track my calories and see what nutrients I need to balance. Though Nut has been invaluable, it lacks a clean recipe and menu interface (it has them, but it's light on features). I did the usual google/freshmeat/sourceforge search, but turned up very little. gnutrition looks promising, but it's 2 years unmaintained and uses an old version of the USDA database. My requirements are: that it use the current USDA database; have a flexible recipe and menu functions which tracks calories and nutrients; and finally that it runs under Linux (prefer OSS). Multiple-user support and easy of use would be nice, but not required."

PlaidG writes "An interesting article has been posted on Antigames.com about the community revolving around the yet-living Sega Dreamcast. It covers the reasons behind the continuing viability of the Dreamcast, and the thriving underground surrounding it." Quite apart from the cool stuff such as MP3 players or Dreamcast Linux you can hack around with, the array of great games now available so cheaply makes Sega's console very enticing, even past its prime.

uprightcitizen writes "Good news for the open source audio recording world! Ardour creator Paul Davis has announced a feature-freeze and has set a binary release date for the now-famous GPL multitrack audio recording application. Ardour has recently been featured in Sound on Sound and has been mentioned on Slashdot many times (here(1), here(2), etc..). The feature freeze is effective as of May 4 and the binary release date is set for sometime in July or August. Good Job Paul!"

Mortimer.CA writes "O'Reilly has a story about someone hacking Ultima VII so that it's multiplatform. Exult is replacing the the rendering engine so the game can be played on more than just DOS. A legal copy of Ultima VII is needed to play Exult. I have 'wasted' so many hours on the Ultima series that it's not funny: now I can waste them again on my Unix box." I might have to see if I still have Ultima VII kicking around. I haven't played it since my college days.

mbogosian asks: "Recently, my broadened responsibilities have me doing some database design and modeling, and I'm happy for the new knowledge and experience, but I'm a bit frustrated about the tool selection. I know most of us have had plenty of experience with at least a handful of all the wonderful Open Source development tools out there (like GCC, GNU Make, Subversion , and Perl to name a few). My question is this: where are OpenSource design tools? I've tried what I could find on SourceForge, but (as usual?) most of the projects that sounded promising were either still in the planning stages or seemed abandoned. Of course something which allowed be to create nifty class charts and output them to UML and/or SQL would be really cool, but I've yet to find something that works (especially in Linux). What are your favorite Open Source design tools and what do you like about them?"

digirave writes "Most multiplayer strategy/RPG 'door games' didn't survive the change from the BBS scene to the Internet. The few that did mostly stayed dialup and telnet only. Here are three BBS door games that were quite popular at the time, remade as open-source games for the Web using PHP and MySQL. First of all, there's the Tradewars 2002-inspired game Blacknova, and secondly the LORD (Legend Of The Red Dragon) remake, Legend Of The Green Dragon. Finally, there's a game similar to Solar Realms Elite, QM Promisance, which itself is a modified version of the original Promisance source."

digirave writes "Most multiplayer strategy/RPG 'door games' didn't survive the change from the BBS scene to the Internet. The few that did mostly stayed dialup and telnet only. Here are three BBS door games that were quite popular at the time, remade as open-source games for the Web using PHP and MySQL. First of all, there's the Tradewars 2002-inspired game Blacknova, and secondly the LORD (Legend Of The Red Dragon) remake, Legend Of The Green Dragon. Finally, there's a game similar to Solar Realms Elite, QM Promisance, which itself is a modified version of the original Promisance source."

G3ckoG33k writes "Cache optimization has now been made easy, ok, perhaps easier... The guys working with memory management tool Valgrind (see previous story at /.) are now up to version 1.9.5, and it's stable! Even more, there is now also an excellent GUI tool for using Valgrind for serious cache optimization; check out KCachegrind!!! Besides, who would have thought cache optimization would be not only intellectually but also visually beautiful?"

G3ckoG33k writes "Cache optimization has now been made easy, ok, perhaps easier... The guys working with memory management tool Valgrind (see previous story at /.) are now up to version 1.9.5, and it's stable! Even more, there is now also an excellent GUI tool for using Valgrind for serious cache optimization; check out KCachegrind!!! Besides, who would have thought cache optimization would be not only intellectually but also visually beautiful?"

parasew asks: "I am searching for Web-based Project Management Software, which should be (mod-)perl based, so I can enhance it or put it into an existing environment using MovableType, which is in a sort of alpha-state. I found a site about Call Center, Bug Tracking and Project Management Tools for Linux and also this short listing, but sadly they are just a bunch of projects which only come close to the kind of tool I am searching for. Gantt and Chronos, seem to be a very nice Web-Calendar packages written in Perl. I was just wondering why no one is using iCalendar (does anyone know of Perl-based Software using iCalendar), as most of the Agenda Software uses iCalendar, and even Mozilla Calendar is capable of subscribing to remote-Calendars. This looks very interesting to me. In general, I wanted to ask you Monks for the best way to do this. Should I create a new app from scratch or reusing existing stuff?"

"Here are the features I am looking for:

• The use of Calendars (multiple users) and iCalendar Support
• File-Pool for projects (CVS-based or similar)
• Progress-bar for showing the current state of a project
• A public calendar where users can publish events from their private calendars

I found a simple Calendar script from PerlMonks, and another easy one here. Of course, there are really lots of Calendar scripts out there, but I am asking you for hints. Which ones do you know of that comes close to my needs? There are several existing Modules for iCalendar on CPAN, and i thought about reusing stuff to maybe create a dirty hack of Chronos, Bricolage and add the CVS and iCalendar support on my own.

Please also see my topics on PerlMonks and MovableType

Thanks for any help, hints or suggestions."

javifs writes "Do you remember TAMU's security tools? If so you might remember a tool that was developed when COPS, SATAN, and ISS were (back in 1994): Tiger. You might think it was dead, well it's not. Tiger has resurrected at Savannah and even has a new webpage and logo! (cool, isn't it?) Tiger has some interesting features that merit its resurrection, including a modular design that is easy to expand, and its double edge: an audit tool and a host intrusion detection system tool. Free Software intrusion detection is currently going many ways, however, from network IDS (with Snort), to the kernel (LIDS, or SNARE for Linux and Systrace for OpenBSD, for example), not mentioning file integrity checkers (many of these: aide, integrit samhain, tripwire...) and logcheckers (even more of these, check Counterpane's Log Analysis pages). Also, free software Linux/*BSD distributions have a miriad of security tools to do local security checks: Mandrake's msec, OpenBSD's /etc/security, SUSE's Seccheck... maybe Tiger could substitute them at some point in the future. Do you think Tiger has a place in the toolkit of the security professional? (I might be biased, though, after all I'm the upstream developer for Tiger now :-) ) In any case, have you downloaded and tested the latest release candidate for Tiger version 3.2?"

anonymous writes "In this fascinating interview, Eric Laffoon and Andras Mantia give us a glimpse into the world of the Quanta Plus project. Read on for everything from tantalising references to Kommander, billed by Eric to be part of the foundations for the next generation desktop and user experience, to details of future plans for Quanta VPL (Visual Page Layout)."

anonymous writes "In this fascinating interview, Eric Laffoon and Andras Mantia give us a glimpse into the world of the Quanta Plus project. Read on for everything from tantalising references to Kommander, billed by Eric to be part of the foundations for the next generation desktop and user experience, to details of future plans for Quanta VPL (Visual Page Layout)."

Slashback tonight brings you a boatload of updates and amplification to previous Slashdot stories, including: the outcome of the RIAA-driven administrative crackdown on file trading at the U.S. Naval Academy, the legal status of ambiguously labeled Microsoft "gimme" software, more information on the insecurities of Blackboard's card-based payment system, and more. Read on for the details!

Every day, in every way, I am becoming a better and better Lt. Junior Grade. alanjstr writes "The Baltimore Sun reports 'The Naval Academy has disciplined 85 students who used a military Internet connection to illegally swap copyrighted music and movies, but it stopped short of carrying out its threat to impose the maximum penalties of expulsion or court-martial, an academy document shows.' It goes on to say that the raid was spurred less by the RIAA and more by the threat of losing the internet connection due to the enormous amount of bandwidth consumed. The academy had given students several warnings before raiding the dorm rooms. Some of the hard drives seized last November were found to contain one or two copyrighted files, while others ran into the hundreds or thousands."

I bet they could make a better agreement with Xiph.org Magnetic Confinement writes "In an effort to make life more difficult for civic-minded Mac users, NPR has decided to drop Quicktime from its available streams. Nothing specific on their webpage addresses it, just some suspicious vacancies remain. Their helpdesk response is officially:

'NPR.org had been offering some of its audio in the Apple QuickTime format under an arrangement with Apple QuickTime. We regret that we were unable to reach mutually acceptable terms for a new arrangement with Apple QuickTime. As a result, NPR is unable to continue offering its content in this format.

You can also contact Apple QuickTime directly at: quicktime@apple.com

Weston
NPR Online'"

A note that got lost in the bin for too long ... JulesVD writes "Microsoft has agreed to tweak its Windows XP operating system in response to recent feedback from the Justice Department over its antitrust settlement with the federal government. (See news on Yahoo!) Microsoft will give more prominent display to a button in Windows that allows computer users to remove the company's Internet Explorer browser, company spokesman Jim Desler said. The Justice Department is overseeing Microsoft's compliance with the settlement. Placement of the button in a hard-to-reach spot in Windows was one of several complaints Microsoft's rivals made to the department last year."

Proportionality isn't just for the personals. You may still be boggling (I am) at the recently announced RIAA suits alleging that colleges and college students are liable for billions of dollars in damages to the music industry for facilitating online file trading. Reader Derek Lomas writes in with another editorial indicating "growing support at Yale for legal alternatives".

Even biggerness. The Gathering is billed by some as the the world's largest computer party. MC68040, though, writes "I'd like to remind everyone to have a look at dreamhack, that 'also' is the largest LAN in Sweden twice a year ... Which had over 5000 participants in 2001 and even more in 2002.. *arhem* Biggest you say?"

If you want to fight about "LAN party" vs. "Computer party," leave me out of it!

How about calling it "900t"? An anonymous reader writes "As previously reported, mozilla.org's Phoenix browser has been renamed to Firebird. This hasn't pleased supporters of the Firebird relational database project. In an Australian LinuxWorld article, one of their administrators calls the name change "one of the dirtiest deeds I've seen in open source so far." In a MozillaZine article, the same person accused mozilla.org of "theft" and "corporate bullying". They don't explain how it was different when they picked a name that was already used by a BBS, financial software manufacturer, Fenix IDE and games company. Meanwhile, IBPhoenix, an organisation that supports the development of the Firebird database, has put up a protest page, encouraging people to spam the MozillaZine forums (even though MozillaZine had nothing to do with the decision) and send masses of email to many Mozilla developers (most of whom were not involved in selecting the new name). I find it rather hypocritical that the Firebird database people are accusing Mozilla of "the filthiest of dirty tricks" while at the same time advocating the harassment of many Mozilla developers."

Point of clarification. batkid writes "In response to the article 'Microsoft pirating their own software,' Seems like MS is taking it pretty seriously. I got the following response from Microsoft (I am a faculty member, but the response should be the same to students).

April 9, 2003

RE: Visual Studio .NET Professional Edition and Windows XP Professional software distributed during the Microsoft Faculty Seminars

Dear Faculty Member, Thank you for attending the recent Microsoft Faculty Seminar. The purpose of this letter is to clarify questions concerning the legal use of the Visual Studio .NET Professional and Windows XP Professional software distributed to faculty who attended the Seminar. The software received is governed by the electronic license embedded in the product set up that appears prior to installation and no additional documentation is required.

Notwithstanding language on the CD label for the copies of Visual Studio .NET Professional Edition and Windows XP Professional Edition that you received during your attendance at the Seminar, which appeared to indicate that a separate license document was required in order for you to legally use the software, this letter will confirm that use by you of the software received is governed by the electronic license embedded in the product setup that appears prior to installation.

You are required to agree to accept the terms and conditions of this license prior to proceeding with the products' installation. Acceptance by you of these "Click to Accept" licenses is the only license required for your use of the copies of Visual Studio.NET Professional Edition and Windows XP Professional Edition received. We recommend that you keep a copy of this letter in your personal files for future reference."

Thanks for passing that along.

What if Masterlock security was assured this way? Monday, you read that security researchers Billy Hoffman and Virgil Griffith (known as Vergil and Acidus) were were prevented from speaking at a security conference by means of a Cease and Desist order from Blackboard, Inc.. The two planned to talk about security flaws found in Blackboard's Transaction System.

In a mail posted at Declan McCullagh's Politech mailing list, David Yaskin of Blackboard responds to the criticism that the company's legal action has drawn. John R. Hall has posted a FAQ explaining some particulars of the Blackboard Transaction System which Virgil and Acidus aren't at liberty to discuss, as well as contradicting some claims that Yaskin makes in the posted email.

Slashback tonight brings you a boatload of updates and amplification to previous Slashdot stories, including: the outcome of the RIAA-driven administrative crackdown on file trading at the U.S. Naval Academy, the legal status of ambiguously labeled Microsoft "gimme" software, more information on the insecurities of Blackboard's card-based payment system, and more. Read on for the details!

Every day, in every way, I am becoming a better and better Lt. Junior Grade. alanjstr writes "The Baltimore Sun reports 'The Naval Academy has disciplined 85 students who used a military Internet connection to illegally swap copyrighted music and movies, but it stopped short of carrying out its threat to impose the maximum penalties of expulsion or court-martial, an academy document shows.' It goes on to say that the raid was spurred less by the RIAA and more by the threat of losing the internet connection due to the enormous amount of bandwidth consumed. The academy had given students several warnings before raiding the dorm rooms. Some of the hard drives seized last November were found to contain one or two copyrighted files, while others ran into the hundreds or thousands."

I bet they could make a better agreement with Xiph.org Magnetic Confinement writes "In an effort to make life more difficult for civic-minded Mac users, NPR has decided to drop Quicktime from its available streams. Nothing specific on their webpage addresses it, just some suspicious vacancies remain. Their helpdesk response is officially:

'NPR.org had been offering some of its audio in the Apple QuickTime format under an arrangement with Apple QuickTime. We regret that we were unable to reach mutually acceptable terms for a new arrangement with Apple QuickTime. As a result, NPR is unable to continue offering its content in this format.

You can also contact Apple QuickTime directly at: quicktime@apple.com

Weston
NPR Online'"

A note that got lost in the bin for too long ... JulesVD writes "Microsoft has agreed to tweak its Windows XP operating system in response to recent feedback from the Justice Department over its antitrust settlement with the federal government. (See news on Yahoo!) Microsoft will give more prominent display to a button in Windows that allows computer users to remove the company's Internet Explorer browser, company spokesman Jim Desler said. The Justice Department is overseeing Microsoft's compliance with the settlement. Placement of the button in a hard-to-reach spot in Windows was one of several complaints Microsoft's rivals made to the department last year."

Proportionality isn't just for the personals. You may still be boggling (I am) at the recently announced RIAA suits alleging that colleges and college students are liable for billions of dollars in damages to the music industry for facilitating online file trading. Reader Derek Lomas writes in with another editorial indicating "growing support at Yale for legal alternatives".

Even biggerness. The Gathering is billed by some as the the world's largest computer party. MC68040, though, writes "I'd like to remind everyone to have a look at dreamhack, that 'also' is the largest LAN in Sweden twice a year ... Which had over 5000 participants in 2001 and even more in 2002.. *arhem* Biggest you say?"

If you want to fight about "LAN party" vs. "Computer party," leave me out of it!

How about calling it "900t"? An anonymous reader writes "As previously reported, mozilla.org's Phoenix browser has been renamed to Firebird. This hasn't pleased supporters of the Firebird relational database project. In an Australian LinuxWorld article, one of their administrators calls the name change "one of the dirtiest deeds I've seen in open source so far." In a MozillaZine article, the same person accused mozilla.org of "theft" and "corporate bullying". They don't explain how it was different when they picked a name that was already used by a BBS, financial software manufacturer, Fenix IDE and games company. Meanwhile, IBPhoenix, an organisation that supports the development of the Firebird database, has put up a protest page, encouraging people to spam the MozillaZine forums (even though MozillaZine had nothing to do with the decision) and send masses of email to many Mozilla developers (most of whom were not involved in selecting the new name). I find it rather hypocritical that the Firebird database people are accusing Mozilla of "the filthiest of dirty tricks" while at the same time advocating the harassment of many Mozilla developers."

Point of clarification. batkid writes "In response to the article 'Microsoft pirating their own software,' Seems like MS is taking it pretty seriously. I got the following response from Microsoft (I am a faculty member, but the response should be the same to students).

April 9, 2003

RE: Visual Studio .NET Professional Edition and Windows XP Professional software distributed during the Microsoft Faculty Seminars

Dear Faculty Member, Thank you for attending the recent Microsoft Faculty Seminar. The purpose of this letter is to clarify questions concerning the legal use of the Visual Studio .NET Professional and Windows XP Professional software distributed to faculty who attended the Seminar. The software received is governed by the electronic license embedded in the product set up that appears prior to installation and no additional documentation is required.

Notwithstanding language on the CD label for the copies of Visual Studio .NET Professional Edition and Windows XP Professional Edition that you received during your attendance at the Seminar, which appeared to indicate that a separate license document was required in order for you to legally use the software, this letter will confirm that use by you of the software received is governed by the electronic license embedded in the product setup that appears prior to installation.

You are required to agree to accept the terms and conditions of this license prior to proceeding with the products' installation. Acceptance by you of these "Click to Accept" licenses is the only license required for your use of the copies of Visual Studio.NET Professional Edition and Windows XP Professional Edition received. We recommend that you keep a copy of this letter in your personal files for future reference."

Thanks for passing that along.

What if Masterlock security was assured this way? Monday, you read that security researchers Billy Hoffman and Virgil Griffith (known as Vergil and Acidus) were were prevented from speaking at a security conference by means of a Cease and Desist order from Blackboard, Inc.. The two planned to talk about security flaws found in Blackboard's Transaction System.

In a mail posted at Declan McCullagh's Politech mailing list, David Yaskin of Blackboard responds to the criticism that the company's legal action has drawn. John R. Hall has posted a FAQ explaining some particulars of the Blackboard Transaction System which Virgil and Acidus aren't at liberty to discuss, as well as contradicting some claims that Yaskin makes in the posted email.

droopus writes "iCommune finally has a new release available. It was released a few months back, but it was implemented as an iTunes hardware plugin, and Apple terminated the developer's license to use that interface. But it's back and with source code." It is now a standalone application.

droopus writes "iCommune finally has a new release available. It was released a few months back, but it was implemented as an iTunes hardware plugin, and Apple terminated the developer's license to use that interface. But it's back and with source code." It is now a standalone application.

tsetem writes "Over on ExtremeTech, they have a write-up on building your own Linux Media Jukebox for a little over $500 and a bit of elbow-grease. This is probably the PC we were hoping that the Lindows Media PC would've been." This particular project uses Freevo which has matured significantly since I last looked at it.

Gentu writes "There is a interesting interview over at OSNews with Fink's project leader, Max Horn. They discuss Fink's relationship with Apple, integration of their Unix/Linux ports to Mac OS X via Debian's packaging solution, ease of use on installation of the .deb packages, AltiVec optimizations and more."

johnmearns asks: "With hard drive prices so low I couldn't help but pick up a large drive and finally get around to setting up a file server in my house. I normally do all my home computing from my laptop and would like to play mp3s stored on the fileserver back through my stereo. I've found lots of nice streaming mp3 server packages like NetJuke, but I don't want to stream. I would like a player that offers controls that I could access via a web interface from any machine on the LAN. Many of the alternatives I've found seem to have a nasty interface and have been abandoned for years. I thought I'd check and see if other Slashdot readers were using for this. Preferably it would work easily with FreeBSD, but I'm not picky at this point!"

JulesVD writes "There is an article from Linux Journal about the latest plans for Linux audio functionality from the first developer's conference in Germany. Developers from more than a dozen countries attended this successful conference, representing organizations such as SuSE, Linux Audio Systems, Stanford University, IRCAM and Centro Tempo Reale. Topic discussions included in-depth presentations of the rapidly evolving Linux sound system, a look at the details of programming for professional audio standards and a survey of recent applications and audio-centric Linux distributions." Mmm...interesting reading (blantant plug for cool program), but I think the most important question is will it make Scrubby happy?

Clyde writes "The different worlds of DRM and Open Source have come together under OGG-S, a project that just recently went to beta with their Open Source DRM toolkit. The project license in GPL and uses OpenSSL for its encryption engine. It will be interesting to see if this project helps to spread the acceptance of Ogg Vorbis."

Red Hat Linux 9 is out, and as of today the ISOs are officially available to Red Hat Network subscribers ($60/yr). Or, as of right now, you can grab the same ISOs using BitTorrent. For those unfamiliar with this free/Free P2P download protocol, an introduction follows, written by ololiuhqui. Update: 03/31 23:45 GMT by J : After roughly four hours, BitTorrent has transferred over 500 full copies of all 3 ISOs, and a total of over 1.5 TB, at 170 Mbytes/sec. Thanks to the more than 3000 people who helped each other download the data, and especially to the more than 200 who got full copies and still have their clients open, to keep serving data to everyone else :)

Tectonic Rumblings

Every so often a new tool comes along that causes a shift from Bronze to Iron, that divides history into "before" and "after." The peer-to-peer world has certainly seen its share. Those who used 486s to encode and play MP3s remember it wasn't just abysmal modem speeds that kept people from casual trading, but the tiresome process of finding users and content; Napster freed us from that bondage, letting the computer do the heavy lifting and freeing people to do what they do best.

When the weaknesses began to show in Napster's overly centralized model, Gnutella stepped in with a distributed, decentralized network. Audiogalaxy gave us astounding variety (even the most obscure music could always be found sooner or later) and a rich sense of community that is still sorely missed. WinMX offered the ability to connect to multiple Napster-compatible networks; with the advent of multi-source downloading, Morpheus and similar programs allowed us to rise above the limitations of slow upstream (until it's hard now to find any P2P applications that don't use it); and EDonkey added the nice touch of being able to share files before they were done downloading.

So what's the next stage of P2P evolution?

Enter BitTorrent -- a "swarming, scatter and gather" file transfer protocol developed by Bram Cohen that's taking the net by storm. Even without a friendly, unified interface, BT's ability to scale in the face of overwhelming demand while minimizing the free rider problem ("leeching") has attracted a flood of new users. But as with any tool, understanding how and why it works will always make using it easier and more fun.

All technical references are taken from the BT server tutorial and the official documentation.

Let's Start with the Basics

BitTorrent is not a 'website' or a 'network', and strictly speaking is not even a program -- it's a protocol with a number of functional implementations.

Instead of jumping right into downloading, first we'll discuss how files are served. Most new BT users are familiar with going to a website and clicking on links to .torrent files, but this just provides a friendlier interface and isn't actually necessary. All you really need to serve is a public Internet machine. The "tracker" will "keep track" of who is connected and who has which pieces of the file(s) in question. Like any public Internet service, a static IP address and/or valid hostname will make it easier for people to connect to your tracker.

To start serving, you choose a file or directory to serve and run a program which generates a .torrent file. This contains a 'hash,' which serves as a checksum to ensure the file is the same on all systems, as well as the address of a tracker. A typical .torrent file is quite small, typically 5-50k in size.

The second step is to load the .torrent file into a BT client. The client asks you where to save the file, you point it at the existing and complete copy, it verifies that the file hash matches, says the download is done and sits there uploading when necessary until you cancel it.

Here's an animated graphic (.mng, currently viewable only in Mozilla) of a torrent transfer.

Getting Started

The official BT client is available for Win32, Mac OS X, as an unstable Debian package, and as Python source code.

Getting started is quite simple; the Windows installer asks no questions and provides no options, and the only behind-the-scenes addition is that Internet Explorer now launches BT when you click on links to .torrent files. (Mozilla users will need to edit Preferences, Navigator, Helper Applications and add the mime type "application/x-bittorrent", to be launched by the btdownloadprefetched executable.) You can also download .torrent files and load them locally without going through a website.

Once the .torrent has been invoked, the client will prompt you for a location to save the file to. The client then creates a file of the appropriate size containing all zeros, and connects to the tracker to get a starting list of some random subset of available peers (other users connected to the 'swarm'). BT then starts connecting to peers and downloading random chunks of the file, and begin uploading to other peers as soon as you have enough for it to bother.

Every time your client verifies another piece of the download, it tells the tracker it has a good copy of that piece. By directly utilizing each user's outgoing bandwidth, downloads can be generally be completed very quickly while minimizing the load on the original server, in effect turning the dreaded "Slashdot Effect" against itself -- the more who want to download, the more there are to upload. Sooner or later (usually sooner), the download is done, and the client continues to upload pieces to other users.

What's In It For Me?

Now your first instinct at this point might be to close the program, but you really ought to leave it open as long as possible afterward, to help seed the file into the network. But this is really a social and cultural issue which can't necessarily be addressed through technical measures; BT can enforce fairness during the transfer with its algorithms, but no software can force the user to keep the client open. Many tracker owners keep a close eye on such things, and will generally ban repeat offenders. In any event, "giving back" your bandwidth has never been easier, even for users behind firewalls or NAT (although as always, being able to avoid or go through these will make the transfers more efficient).

Alternative Clients and Other Tools

That said, there are perfectly valid reasons to want some control over the amount of bandwidth a P2P application uses, and an experimental, unofficial client (Win32, Python source) has been created to provide a friendly interface for this. BT will automatically adjust your download speed appropriately if you set a slower upload speed, but it's still an invaluable tool for some cable and DSL users whose downloads will choke and abort if they use too much upstream, or for anyone with limited upstream who wants to reserve some of it for other uses.

Currently, both the official and experimental GUI clients use a separate window for each transfer. BT++ (Win32, Python source) has made an initial attempt at combining all transfers into one window, as well as offering some other enhancements, but users report mixed results, with some saying "it works for me" and others that it's buggy to the point of unusable; still, it's one to keep an eye on. (Caveat: BT++ provides an option to automatically stop uploading when the download is completed. I believe this deliberately encourages people to do so even if there is no real need to do so, and would advise anyone using BT++ to refrain from using this option; it's unnecessary, detrimental to the BT networks, and may lead to your IP being banned as described above.)

TorrentSpy (Win32) is another useful tool that shows various statistics about your transfers, including which files of a multi-file torrent are complete. It's not meant to replace a downloading client, but to complement it.

I should add that the speed and time-to-completion numbers may not be wholly accurate, and will typically fluctuate wildly to some extent during a transfer. (After all, do you believe Windows when it tells you how long it will take to copy a file?) The "percentage completed" at least is accurate, and you may be able to get more accurate information using TorrentSpy. A new version of BT has just been released (3.2) and its reported changes include "more even and consistent download rates".

A Few Miscellaneous Points

It's quite possible to generate .torrents for files you want to serve and then advertise them on someone else's tracker. Since anyone can run a tracker, BT is more like IRC, Usenet or Direct Connect than something like Kazaa. Like Freenet, it works best if the content is highly in demand; it's also more effective on recently released stuff. One highly recommeded website is Bstark. It doesn't provide .torrents for anyone to download, but functions as a "metatracker", that is, a tracker that keeps track of trackers. If you're a statistics geek, the graphs are a lot of fun, and even for the average user it's a simple way to check what files are most in demand and most in need of someone to serve them. This is even more effective when you combine it with an alternate means of communication such as IRC or email, making it easy for users to check supply and meet demand. The .torrent file can also be distributed by any means, be it a website, IRC channel, email attachments or perhaps carrier pigeon.

Conclusion

With the 'entertainment industry' finally focusing their attention on IRC, the cantankerous and difficult granddaddy of Internet file sharing, BitTorrent has found a niche and filled it admirably. The author understandably wishes to focus upon using BT in a legal manner. As with any new invention, "the street finds its own use for technology," and BitTorrent will undoubtedly continue to be rapidly adopted for both licit and illicit use.

Given the decentralized nature of BT networks and the rapid development of new tools, it's only a matter of time before someone writes a GUI wrapper for an IRC client, web browser and all-in-one BitTorrent interface. After all, Napster did it, as do most other mainstream P2P apps like Kazaa. Like Direct Connect with its 'hubs,' there will always be multiple BT servers available, and a unified interface would not only make it easier for users to find and download content, but free them to focus on forming the social and cultural networks that are also needed. A website typically uses far too much CPU and bandwidth to handle popular traffic, but a BT tracker uses minimal bandwidth by itself. Perhaps the next-generation clients will try to automatically locate trackers, or help the user find and serve older content as well as new releases.

The late great Audiogalaxy had many strengths, but one of its most fundamental was the sense of community it encouraged. BitTorrent wisely fills a narrow set of technical requirements, leaving a great deal to human need and will. The ad hoc arrangements and customs that have so far sprouted as expressions of the will to fill these needs are often chaotic and messy -- but that's human action for you.

dubious9 asks: "Today I was refactoring a parser of mine to use a better implementation of a string searching algorithm. I went to the internet trying to find a good code repository where I might have a chance to find an implementation of a good algorithm, but a cursory glance turned out no clear winners. SourceForge was the best that I could find. Where is the best online repository/library of common and reusable code snippets?" We've tackled this subject a once or twice over the years, is SourceForge really the best answer or are there other options?

NoSun writes "Sun's latest project is to create a font library for XFree86, named Stsf, that would replace Fontconfig and Xft2. But the big question is: Does the world need yet another X font library that would create more incompatibility and fragmentation? Well known Gnome and GTK+ developers are against this (yet another) X font library which just re-invents the wheel one more time with the result of slowing down KDE and Gnome in the desktop race. "

jbuberel asks: "I just got myself a fancy new Canon Powershot S400 camera. One of it's nifty features is it's 'Panoramic Assist' mode that helps you line up a series of images that can later be stitched together to form one larger panoramic image. Of course the software that ships with the camera to do the stitching is Windows-only. After spending some time probing freshmeat.net and google.com, I came across one dead project, and another relatively academic project who's tools are largely undocumented. So are there any up-to-date tools for composing panoramic photos in Unix? With so many digital photo afficianados out there, I was surprised there wasn't an easy-to-use GIMP plugin for this."

An anonymous reader writes: "Carsten Haitzler (The Rasterman) has posted a "state of the union" for the enlightenment project on their mailing list. It has been over 2 years since the last major release of the Enlightenment window manager. It looks like 0.17 is a ways off but it's nice to see an update."

marXian writes "Opening in Norwich UK this week and subsequently visiting Cambridge is makb3th from theatre company pirateutopia.org. The show is very much Linux-powered using aalib, XDirectFB, VLC and more to set the piece (an adaptation of Shakespeare's Macbeth) on an off-shore data haven." Allright, pick your jaw up off the floor ;)

orangerobot writes "A new article on XML.com summarizes some of the response from the XML-DEV mailing list to Tim Bray's recent comments about his frustrations with XML. The overall feedback is mixed but several parsing packages are mentioned that satisfy some of Bray's complaints about the difficulty of using DOM and SAX-based APIs. The packages include Pyxie, XML::Filter::Dispatcher and XML::Essex."

Dan writes "Erik Reid has been working on adding DRI support for NetBSD. Direct Rendering Infrastructure, also known as the DRI Project, is a framework for allowing direct access to graphics hardware in a safe and efficient manner. Some of Erik's work has been imported into XFree86 4.3.0 which is now in xsrc tree. He has subsequently put together a fairly large patch which compiles and works on his NetBSD/i386 1.6P system with a matrox g450. Try out the patch and give him some feedback!"

Slashback this evening brings updates on social networks, Audioscrobbler, the Social Security-number security breach at the University of Texas at Austin, and more. Read on for the details.

Why meet people in real life? Roland Piquepaille writes "I wrote [Saturday] a column about social-network mapping tools mentioned by Slashdot. Slashdot readers sent me many comments and e-mails about other visualization tools. Here are these new tools, in no particular order: email constellations, Apache Agora, NetVis Module, EtherApe, inGridX, NameBase's Proximity Search, Surf3D Pro and the dazzling KartOO. Finally, a reader talked about another kind of tools, the Visual Thesaurus. This web tool is not about social mapping, but it shows graphical connections between words. In this previous column, "The Visual Thesaurus: What Does it Show About Thanksgiving?," I already explored this very funny tool. Check this new story for more the details about all these tools."

Update: 03/19 00:34 GMT by T : Directly related: Josh Tyler writes "Related to a recent Slashdot posting on social networks is this paper on automatically discovering communities based on email data, just published by our group at HP Labs. We find that simple communication data is enough to identify communities, both formal and informal, and possibly even to identify the leaders of these groups."

Speaking of online community ... TGK writes "Audioscrobbler (which many of us visited the first time it was posted here) has a new site up, and most importantly, new plugins for XMMS and Winamp 3."

From the site, a capsule description of what Audioscrobbler does: "It grows to know what music you like by monitoring what songs you play on your computer. From this information you can discover other users that share some or all of your taste in music."

Feedback is always cool. An anonymous reader writes: "Sudhakar Govindavajhala, co-author of the paper referenced by the Saturday Slashdot article 'Using Memory Errors to Attack a Virtual Machine,' has responded to many of your [Slashdot readers'] questions and comments. His commentary is located at his Princeton CS website."

Another reason that Social Security isn't. GregAllen writes "Remember the recent case of SSN data theft at The University of Texas? A student has turned himself in. In his confession he says that he acted alone, and had no intention to disseminate the information. Maybe this will convince them to stop using SSNs for student IDs." Bonker also points out that "Salon is carrying an AP article that's a followup to the story a few days ago about the mass of Social Security Numbers stolen from University of Texas. Christopher Andrew Phillips is described as a 'fine young man who has never before been in trouble with the law'. Apparently he wrote a program 'to access a university Web site that tracks employees who attend training classes'. Whether or not this was done for illegitimate purposes remains to be seen. As a former UTA student, I'm glad my SSN is no longer in danger!"

What's the state of the device? An anonymous reader writes "N-Philes.com did another State of the GBA Industry Article and Roundtable. Here is the Industry Article, and here is the Roundtable"

Update: 03/19 00:34 GMT by T : And one more presroi writes "Just one week after even slashdot has noticed the new 2.2.24 linux kernel, Alan Cox has announced a new version due to a security issue found in 2.2 as well as in the 2.4 branch. I hope that we all were to lazy to upgrade from 2.2.X to .24 until now :)"

Slashback this evening brings updates on social networks, Audioscrobbler, the Social Security-number security breach at the University of Texas at Austin, and more. Read on for the details.

Why meet people in real life? Roland Piquepaille writes "I wrote [Saturday] a column about social-network mapping tools mentioned by Slashdot. Slashdot readers sent me many comments and e-mails about other visualization tools. Here are these new tools, in no particular order: email constellations, Apache Agora, NetVis Module, EtherApe, inGridX, NameBase's Proximity Search, Surf3D Pro and the dazzling KartOO. Finally, a reader talked about another kind of tools, the Visual Thesaurus. This web tool is not about social mapping, but it shows graphical connections between words. In this previous column, "The Visual Thesaurus: What Does it Show About Thanksgiving?," I already explored this very funny tool. Check this new story for more the details about all these tools."

Update: 03/19 00:34 GMT by T : Directly related: Josh Tyler writes "Related to a recent Slashdot posting on social networks is this paper on automatically discovering communities based on email data, just published by our group at HP Labs. We find that simple communication data is enough to identify communities, both formal and informal, and possibly even to identify the leaders of these groups."

Speaking of online community ... TGK writes "Audioscrobbler (which many of us visited the first time it was posted here) has a new site up, and most importantly, new plugins for XMMS and Winamp 3."

From the site, a capsule description of what Audioscrobbler does: "It grows to know what music you like by monitoring what songs you play on your computer. From this information you can discover other users that share some or all of your taste in music."

Feedback is always cool. An anonymous reader writes: "Sudhakar Govindavajhala, co-author of the paper referenced by the Saturday Slashdot article 'Using Memory Errors to Attack a Virtual Machine,' has responded to many of your [Slashdot readers'] questions and comments. His commentary is located at his Princeton CS website."

Another reason that Social Security isn't. GregAllen writes "Remember the recent case of SSN data theft at The University of Texas? A student has turned himself in. In his confession he says that he acted alone, and had no intention to disseminate the information. Maybe this will convince them to stop using SSNs for student IDs." Bonker also points out that "Salon is carrying an AP article that's a followup to the story a few days ago about the mass of Social Security Numbers stolen from University of Texas. Christopher Andrew Phillips is described as a 'fine young man who has never before been in trouble with the law'. Apparently he wrote a program 'to access a university Web site that tracks employees who attend training classes'. Whether or not this was done for illegitimate purposes remains to be seen. As a former UTA student, I'm glad my SSN is no longer in danger!"

What's the state of the device? An anonymous reader writes "N-Philes.com did another State of the GBA Industry Article and Roundtable. Here is the Industry Article, and here is the Roundtable"

Update: 03/19 00:34 GMT by T : And one more presroi writes "Just one week after even slashdot has noticed the new 2.2.24 linux kernel, Alan Cox has announced a new version due to a security issue found in 2.2 as well as in the 2.4 branch. I hope that we all were to lazy to upgrade from 2.2.X to .24 until now :)"

Slashback with more on open code in government, Intel's new low-power mobile chips, the nature of the engineers, craftsmen or whatchamacallims who spend their days forging software, the CD price-fixing settlement, and more -- read on for the details.

Formalization schmormalization. kaisyain's review today of Software Craftsmanship raised a spirited conversation about the nature of software, software engineering, and related disciplines. cconnell conveniently submits a great companion piece: "I wrote this article a couple years ago but it has continued to get good readership within the software engineering community. Should provoke some interesting discussion..."

The bleeding edge costs money. JeffyVernon writes with an followup to CNET's early review of Centrino laptops: "AnandTech published two articles on Centrino today, an overview of the CPU architecture (including some interesting history behind the chip) and a roundup of four notebooks including the new Dell that wasn't in CNet's roundup. It looks like the 4.9lbs IBM T40p ended up winning the roundup, it lasted over 6 hours on battery!"

What scarcity was this exactly? RadBlock writes "Lawrence Lessig is addressing the issue of radio spectrum on CIO Insight... something that was talked about on Slashdot the other day. Lessig states that the spectrum has been defined too generally as if there can only be one message per frequency, when better equipment will vastly increase the amount of 'spectrum' that is usable."

I like that phrase "general welfare." We've mentioned eGovOS several times before -- now, here's a last-minute announcement that may be of interest: free registration is still open for next week's (March 17-19) eGovOS conference in Washington D.C., "Open Standards/Open Source for National and Local eGovernment Programs in the U.S. and EU." Perhaps some folks there ought to consider the question eugene ts wong raised the other day, namely, Which North American government offices won't move to Linux? Someone needs to set up a big map with different colored countries and states!

Who's laughing and where is his bank? deelowe writes "From ars. Back in September we reported on a class action suit leveled at a number of Music industry players that accused them of anti-competitive price-fixing. Back in January, we reported that victims of said price fixing could hit this website and sign up (too late now), and eventually receive up to $20 in the settlement, provided of course that you had actually purchased a CD between January 1 1995 and December 22, 2000. 3.5 million Americans made their way to the on-line form, and it appears that victims will receive $12.60 apiece, should a judge approve it."

They still have a while to go ... sp1nl0ck writes CNet News.com.com.com are reporting that The Neo Project guys have restarted the attempt to crack the 2048-bit XBox key following advice from their lawyers. CNet are citing a link to Operation Project X, but it was a bit temperamental in loading earlier. Maybe it's been CNetted..."

I'll still think of it as the GIMP for a few years ;) Agermain writes "CinePaint has just released its first Windows build. From their website: "CinePaint is an open source painting program used by motion picture studios to retouch images in 35mm films. It was formerly called Film Gimp. It has been used in a dozen feature films including Harry Potter, Scooby-Doo, and the Fast & the Furious... This first Windows beta release is mainly intended for developers and testers.""

Slashback with more on open code in government, Intel's new low-power mobile chips, the nature of the engineers, craftsmen or whatchamacallims who spend their days forging software, the CD price-fixing settlement, and more -- read on for the details.

Formalization schmormalization. kaisyain's review today of Software Craftsmanship raised a spirited conversation about the nature of software, software engineering, and related disciplines. cconnell conveniently submits a great companion piece: "I wrote this article a couple years ago but it has continued to get good readership within the software engineering community. Should provoke some interesting discussion..."

The bleeding edge costs money. JeffyVernon writes with an followup to CNET's early review of Centrino laptops: "AnandTech published two articles on Centrino today, an overview of the CPU architecture (including some interesting history behind the chip) and a roundup of four notebooks including the new Dell that wasn't in CNet's roundup. It looks like the 4.9lbs IBM T40p ended up winning the roundup, it lasted over 6 hours on battery!"

What scarcity was this exactly? RadBlock writes "Lawrence Lessig is addressing the issue of radio spectrum on CIO Insight... something that was talked about on Slashdot the other day. Lessig states that the spectrum has been defined too generally as if there can only be one message per frequency, when better equipment will vastly increase the amount of 'spectrum' that is usable."

I like that phrase "general welfare." We've mentioned eGovOS several times before -- now, here's a last-minute announcement that may be of interest: free registration is still open for next week's (March 17-19) eGovOS conference in Washington D.C., "Open Standards/Open Source for National and Local eGovernment Programs in the U.S. and EU." Perhaps some folks there ought to consider the question eugene ts wong raised the other day, namely, Which North American government offices won't move to Linux? Someone needs to set up a big map with different colored countries and states!

Who's laughing and where is his bank? deelowe writes "From ars. Back in September we reported on a class action suit leveled at a number of Music industry players that accused them of anti-competitive price-fixing. Back in January, we reported that victims of said price fixing could hit this website and sign up (too late now), and eventually receive up to $20 in the settlement, provided of course that you had actually purchased a CD between January 1 1995 and December 22, 2000. 3.5 million Americans made their way to the on-line form, and it appears that victims will receive $12.60 apiece, should a judge approve it."

They still have a while to go ... sp1nl0ck writes CNet News.com.com.com are reporting that The Neo Project guys have restarted the attempt to crack the 2048-bit XBox key following advice from their lawyers. CNet are citing a link to Operation Project X, but it was a bit temperamental in loading earlier. Maybe it's been CNetted..."

I'll still think of it as the GIMP for a few years ;) Agermain writes "CinePaint has just released its first Windows build. From their website: "CinePaint is an open source painting program used by motion picture studios to retouch images in 35mm films. It was formerly called Film Gimp. It has been used in a dozen feature films including Harry Potter, Scooby-Doo, and the Fast & the Furious... This first Windows beta release is mainly intended for developers and testers.""

paulbd writes "This weekend sees the first Linux audio developers conference at ZKM in Karlsruhe, Germany. Gathering together many members of the Linux Audio Developers mailing list and others, the conference will feature 2 days of in-depth technical presentations and demonstrations of many cutting edge Linux audio and MIDI applications." Desktoplinux.com has a related story about using Linux in a professional recording studio.

chromatic writes "It's easy to write code that works fine on single-processor systems but dies horribly on multi-processor boxes. Instead of spending thousands of dollars on a four- or eight-way system, you can use UML to emulate a multi-processor machine."

chipset writes "Found a way to hack the Belkin USB Wireless (802.11b) driver to allow other adapters to work. By using the Atmel Wan Driver page on SourceForge and these tips drivers can be modified to support other wireless adapters. This will get unsupported USB WLAN adapters working with the Mac."

Rien writes "I saw this BusinessWeek article referenced over at MacSlash. The author makes the case for Apple utilizing the Bochs Open Source IA-32 (x86) PC emulator to help counter Microsoft's recent purchase of Virtual PC from Connectix." I looked at Bochs, and maybe I R Dum, but I couldn't figure out how to install Windows on it.

Barry mentions his "sender pays" spamfighting plan more than once in his answers to your questions, and discuessed it at length in an InternetWeek.com article published on Feb. 20. Is Barry's plan workable? Do you have a better idea? Or should we all just get used to spam as part of the online experience, and learn to live with it and block it as best we can?

1) Back to the 90s
by gylz

If you had known back in the early 90s that spam was going to be the problem it is now, what steps would you have taken then to protect yourself and others from it?

For instance, what changes would you have advocated in the mail protocols and what standard procedures would you have told other ISPs to use to prevent spammers from getting a foothold in the first place?

Barry:

When The World began selling the first commercial dial-up internet accounts in 1989 one question we were frequently asked by the privileged few who had internet access was: How are you going to control them? To be honest, we never had a good answer other than developing what everyone thought was a pretty good AUP (Acceptable Use Policy) and promising to enforce it as best we could.

But even as the net developed, in the early-mid 90s, there were similar problems with system cracking and break-ins. Back then there were more open holes to just walk right through, get a privileged shell, or just cause mayhem. To a great extent spam can be viewed as a form of system compromise and similar to malicious cracking in many ways.

One of my pleas back then to other ISPs was to make some sincere effort to know to whom you were giving accounts. Many of the ISPs with big funding and marketing departments to match would just give out new accounts to anyone with a drink coaster and worry about it later, oftentimes much later only when the bill wasn't paid.

I think practices like these gave rise to the sense of anarchy and lawlessness on the net that came from the easy abuse of anonymity which persists today. At The World we were careful about not enabling new accounts until we were pretty sure we had valid information. Many ISPs did not do this and tracing problems back to an account on their service would lead to a dead end; the info they had on the account would turn out to be obviously fraudulent.

Also, and this isn't a regret but more of an observation, some early internet advocates wanted only end-to-end services which basically meant that every single computer on the net should be a mostly autonomous client and server. Dial-up made this impractical; you couldn't really run a web site or even a decent mail server over a part-time connection. But I think some of that ambivalence over goals contributed to inaction on issues which might have helped with problems we see today.

2) Acting Locally, Effecting Globally
by merlin_jim

Many posts talk about proposed changes to society, government, and technology to lessen the spam problem. However, an ISP has more insight into the problem than many others, and I thought I'd ask a question to tap that insight:

Given today's society, technology and infrastructure, what can an individual do that would be effective in reducing not only the personal strain of spam, but also lessen an ISP's burden.

What kind of strategies have you seen work. For instance, in particularly bad instances I'm prone to send an e-mail to spam@isp.net, abuse@isp.net, or admin@isp.net, but usually never even get a response. Is there a better thing to do? Are there things that are absolutely the wrong thing to do (such as replying to a spam)?

In short, what would you like to see users do in response to spam today?

Barry:

Pressure your legislators to enforce the laws already on the books! Hijacking others' systems, identity falsification, and fraud are already illegal. These aren't legitimate business people who send all this bulk mail, they're crooks.

Even if a spammer can sneak around the laws making it clear that the activity is illegal, this prevents a spammer from getting investors, incorporating, taking out bank loans, obtaining legal indemnification against liability, buying business insurance, registering with their state or owning intellectual property (e.g., trademarks), etc.

Something else everyone can do is install spam filters. And help others install spam filters. Ultimately, I believe it's an arms race between the filters and the spammers so other forces need to be put into play.

But my reasoning is that utilizing filters now will make the internet experience more pleasant and productive for many which is a good thing. Their wide-spread use will also serve as a wake-up call to those companies who are deluding themselves into thinking they're "white-hat" spammers so ought to be exempt. The filters throw their stuff away also.

The so-called legitimate advertisers need to get to the table with the ISPs and figure this thing out and stop thinking the status quo serves them.

At this point my thinking is that there isn't much difference, from the point of view of an ISP, between companies whose spam you don't hate and those whose spam you do hate.

When it's paper mail you have to put a stamp on a letter whether the intended recipient asked for the mail piece or not. I think we need to move in the same direction on the net with all bulk e-mailers. They need to start paying for the infrastructure they're exploiting.

The current situation is that people tend to define "spam" as e-mail which promotes products which they don't want others to think they want. We need to get beyond that because you're paying for any e-mail you receive, even if only indirectly.

3) why not whitelist?
by Aviancer

Why hasn't any large ISP or enterprise seriously considered whitelisting mail? The traditional blacklist idea -- when I see spammers I'll no longer accept their mail -- is so easily overcome that many spammers don't even wait one generation to change addresses. Instead, bounce all mail you don't recognize, with a note to the sender on how to inform the system that you are a real user. Nearly all spammers loose their incoming account immedately, so this seems the natural choice. There's some more detail on this method at the TMDA project.

Barry:

The easy answer is that the target moves too fast. How could we begin to keep up a whitelist at the ISP level on behalf of thousands or even millions of customers?

And how exactly do you propose to "inform the system that you are a real user"? Right there is the crux of the matter. What you're suggesting is one of those techniques which works pretty well for individuals but is unmanageable at the ISP level.

Something from the TMDA site I do agree with is:

Spam will not cease until it becomes prohibitively expensive for spammers to operate.

We just have slightly different approaches to making spam prohibitively expensive. Let a thousand flowers bloom!

4) Is there a reasonable solution?
by PincheGab

Given that junk mail in the regular mail is more acceptable (and I will mention that my wife (specially) does like to know when there's a sale on), and given that e-mail is the next big thing, what do you see as an acceptable solution/accord to spam?

I certainly am tired of deleting the penis enlargement and Nigerian bank deposit e-mails, but where is the balance and how do we attain it, if ever?

Barry:

I believe the only approach which will work is a "sender pays" model for bulk e-mail advertising. Such a model corrects the current situation on several levels:

a) Sender pays can provide an economy to enforce its own rules.

Most proposals I've seen to deal with spam are workable on paper but fail in this regard. If, when considering yet another spam proposal, you ask yourself who will pay for this or that solution, how will it be enforced (e.g., if it requires lawsuits who will pay the lawyers?) generally no answer comes to mind.

However, if we create a (bulk) sender pays model through some sort of trade association then that organization would have a revenue stream which can be tapped to enforce its revenue model, and a monied interest in defending that revenue model.

b) Sender pays creates a conduit of control between the sender and the ISPs.

Right now spammers can use an ISP's facilities to firehose any spam they want, to anyone and everyone they like, at almost zero cost. For example, kids' accounts are flooded with explicit pornographic come-ons. There's no ability to control that sort of thing.

What business allows its facilities to be used to offend its customers?

In a sender pays model one could also refuse to be paid and, hence, refuse the advertising. Spammers are trying to send their spam to the ISP's customers. I think the ISP has both a right and an interest in controlling that so as not to drive customers away. It's not reasonable that an ISP such as myself has no control over what sort of advertising is placed in my customers' mailboxes yet is left responsible for the quality of that experience.

c) Sender pays clarifies the legal situation without a need for new legislation.

Sending, and not paying, would become simple theft of service, wire fraud, etc.

5) ISP Tools
by feenberg

Do ISPs have the tools they need to prevent outgoing SPAM from their own customers? I look at Sendmail and don't see anything that would allow you to throttle mail volume, check outbound messages for SPAM, restrict new customers etc. There isn't even anything built in that would warn you about a customer sending a million messages. It would seem that a few tools like that would be a big help to an ISP too small to develop its own.

Barry:

I think the best tool is knowing who your customer is and having a clear and effective policy if a customer spams such as clean-up costs which should also include intangibles such as public relations costs.

But you're correct, better tools at that level might help if ISPs were inclined to use them. Many ISPs do use tools such as you describe, others obviously don't care.

6) RBL's
by sabri

One of the few measures that can be taken against spam is the use of blacklists (for instance via DNS). There are a lot of pro's and con's for the use of DNSBL's. How do you feel about these? Should DNSBL's be governmentally regulated? Do you use any DNSBL? Should an ISP enforce certain RBL's (let say, of open relay's) on its customers?

Barry:

I've always resisted using these blacklist services at the ISP level. There are several reasons why but the most important is control.

If the blacklist suddenly began blocking some site, such as a major university or corporation because it was the source of spam the night before, that might cause a big problem with our customers. Even if it could be worked around it'd be just another out of control detail which might send one into fire-fighting mode suddenly.

Another problem I've had with blacklists is that some have become rogue and gone power-mad, blacklisting addresses for reasons completely unrelated to their stated purpose such as personal politics.

Also, the blacklists I've looked into were volunteer efforts which meant the people involved often felt they could paper over any mistake or oversight or staff unresponsiveness with the excuse that they were unpaid volunteers so what do you expect? You can't have your ISP be dependent on organizations with that attitude. And what if I don't like a blacklist's policies or implementation of their policies? If I'm not paying them I can't vote with my wallet.

I suspect that anyone attempting to run a blacklist in a professional, paid manner would go broke; the service isn't worth what it'd have to charge to stay in business. The legal costs alone can be daunting. With legal issues even if you're right it can be expensive getting there. And customers of any service don't want to pay for your legal bills as the major cost of such a service. So we're back to problems with the economic models.

I don't think government regulation would help with blacklists, per se, except in very general ways (they can run the courts for the lawsuits!) The only analogy I can think of are credit bureaus but most of the government regulation in that area is to protect consumers. I don't think we want the government stepping in to protect spammers!

Finally, yes, just about all ISPs blacklist (block) offending sites. Doing it in-house gives them the control they need. It's not great to have to take this on but it's the only choice right now. Unfortunately it's becoming a major burden, and the results are not altogether predictable.

7) What would be the minimum actual cost?
by jamie

What would be your actual dollar cost of spam, if you didn't spend much time and effort fighting it?

Let me explain...

I sometimes hear that spam has significant costs in bandwidth and storage but I don't believe it. As far as I can tell, SMTP traffic is at most 2-5% of net traffic. And a quick calculation shows that an ISP's costs for storing its users' spam are fractions of pennies on the dollar. (*)

You've likened spam to a DDoS attack on your mail servers. Stories about being flooded with traffic sound impressive but computers are so fast now, it's hard to put anecdotes into context. So I'm looking for dollar amounts. For a customers paying b dollars per unit time, an ISP like yours has to spend c dollars per unit time on servers that can handle those customers' incoming SMTP traffic. If this is significant, I'm looking for c over a times b :)

Obviously admins to run the servers are an important cost. But for purposes of this question, suppose you wanted to do the bare minimum. Say you set up the SMTP servers to use just a few of the less-intrusive DNSBL lists, like sbl.spamhaus, relays.ordb, or list.dsbl, and then ignored them as much as possible.

The next most common argument I hear is that customers will abandon ISPs that don't fight spam. But every ISP has the same problem, so this is really a competitive advantage issue except for the small percentage of users who are actually driven off the internet by spam.

Then there's outgoing spam but I don't imagine that's too hard to recognize and stop quickly.

Let me know what I'm missing...

(*) Thumbnail calculations of spam storage follow. Let's say J. Average ISP Customer gets 20 spams a day at 10K each, and deletes them only every 30 days. That's an average of 20*10K*15 = 3 MB of storage. If the ISP replaces hard drives every two years on average and its total storage costs are ten times the actual medium costs (for labor, backup, redundancy, downtime), then at today's hard drive prices, that spam storage will cost the ISP 0.003 * 10 / 2 dollars, or about a penny and a half. Over that same year, J. Customer pays the ISP $100+.

Barry:

Your figures for the percentage of bandwidth which is spam are far too low. Others have put the numbers much higher. NewsFactor cites studies putting the figure somewhere between 17 and 38%. See http://www.ecommercetimes.com/perl/story/19803.html.

As to computers getting faster, that's not a primary issue in my mind. But addressing even that point, how rapidly should I have to amortize and replace my equipment just to accommodate spammers?

And what about the intangibles? They're becoming the major factor in all this. E-mail is the "killer app" on the net. Yet spam is fouling that e-mail experience.

People reading Slashdot might be sufficiently committed to e-mail that they'll wade through all the spam and tweak spam filters even if it takes hours per day and a clothes pin on their collective noses. But what about the many millions of people who aren't so committed to this technology?

As an ISP I can tell you they're giving up on the internet, to them the cost/benefit is just not worthwhile. That's not a good trend.

Another cost is that spam is undermining the standardization of protocols on the net, and thus introducing a pervasive chaos. Every ISP and many other sites are scrambling around implementing mostly different "solutions" to the spam problem. Some of these in-house solutions might be ok, others can be pretty bad.

One result is that e-mail is becoming less reliable as a communications tool. Your mail might get through, it might be kicked out or filtered as spam, you might be able to figure out why and get the message through on a slightly changed subsequent attempt, or maybe not.

Who needs this kind of craziness? How can this situation possibly be productive?

How productive is it to have millions of people installing and customizing spam filters? Or having really bright people writing spam filtering programs? And where is this all going?

In my opinion, if unchecked, I think the current trend is very destructive to the entire idea of a public network.

P.S. I realize in another answer I recommend installing spam filters, but I see that only as a temporary measure.

8) Collateral Damage
by aridhol

One of the greatest problems with spam-prevention techniques has to do with collateral damage. Can you see any solution to spam that either prevents or minimizes the damage to innocent bystanders, such as other users of a spammer's ISP?

Barry:

Yes, the solution I favor is going to a sender pays model aimed at bulk e-mailers.

Other approaches, in particular technical solutions, are prone to causing collateral damage. Inevitably as the arms race heats up, and spam filters have to take bigger and bigger risks to have any effect, collateral damage will become more common.

And it's already worse than you might imagine. Spam and similar are causing severe operational problems on the net and undermining standards as ISPs and others invent new ways to avoid the spew.

As one concrete example, right this minute there's a network provider who was just assigned most of the 69.0.0.0/8 IP address space. Unfortunately, this was formerly a spam and DOS (denial-of-service) cesspool so many sites out there just block the whole 69.* address space.

So the new owners are making appeals to firewall managers asking them to please remove their blocks in the 69.* space on the NANOG (North American Network Operators Group) list.

But NANOG is not a particularly big or influential mailing list. At best it's only aimed at North America while the blocking exists world-wide. But how do you communicate with so many sites and undo the problem? In a nutshell, you can't. I suspect their customers who get space in 69.* are going to find themselves blocked by many sites for many years to come.

See what a mess spam is causing? It's like asking how much can such a little tiny termite eat? And then the house falls down.

9) Spam Lawsuits
by ca1v1n

Do you think new laws that allow ISPs and end-users to collect damages from spammers on a per-message basis can be effective tools to reduce spam?

Barry:

Although it should be part of the picture I think this sort of litigation would be ineffective as a primary attack on the problem.

What we need to do first is stop the insanity!

To do that I say introduce sensible economics into e-mail advertising. You may find network TV commercials annoying, but imagine if just anyone could break into a station's signal at any time and insert advertising! That's what we have right now, and it's crazy.

If we were subjected to a few, well-paid and placed ads it might be annoying to some but others might even find it beneficial like the person in the previous message whose wife likes to know about the good sales. Or we could just pay a premium and not see another ad, analogous to premium cable TV. Or find ways to block them via our personal mail clients, analogous to what people do with PVRs. It'd just be a matter of economics and marketing and taste.

But right now it's complete anarchy, only the introduction of a viable economic model can tame the situation.

Also, I'm not optimistic about any legalistic approach so long as there's no scalable revenue stream associated with e-mail or its abuse.

Currently the general consensus on the net is that we don't even want sales taxes on e-commerce, which might be a reasonable point of view, but then we're going to ask that billions should be spent on courts and enforcement of new spam laws? Where is that money supposed to come from? Cut the fire dept? The schools? Not-growing corn subsidies? Without additional revenue something has to give.

Given a sender pays model money could be earmarked for private enforcement, such as investigation and litigation. And the case could be more realistically made as to the exact economic cost of spam. If an ISP was supposed to get paid for ads going through their system then anyone evading that is simply guilty of good old fashioned theft of service, no new laws needed. And legislators, who presumably would be getting their usual business tax cut of such revenue, could begin to see the logic in returning some tax money to defend these revenue streams.

There would still be challenges to be worked out internationally but it wouldn't be the first time a revenue model had to work on a global scale. Obviously international telephony and postal mail works well enough to combat fraud. But only with some sort of concomitant revenue stream attached to the activity could you possibly begin to tackle the problem, domestically or internationally.

10) Kill 'em all
by Lord_Slepnir

If you could meet a spammer, what would you say? What would you do? What caliber would you use? Would you want someone to do it for you? Is $10,000 a head too much?

Barry:

I would tell the spammer in no uncertain terms that spammers' days are numbered, just like junk faxers and other scam artists who exploited a brief window of vulnerability.

Situations like this don't last long.

Of course, then the spammer would laugh in my face because that's what sociopaths like to do when confronted. But, as the expression goes, we'll see who laughs last.

One thing is clear, however, spammers will not listen to reason. So any change in their behavior will have to be the result of force.

A reader writes:" Persistence for object-oriented systems is an incredibly cumbersome task to deal with when building many kinds of applications: mapping objects to tables, XML, flat files or use some other non-OO way to represent data destroys encapsulation completely, and is generally slow, both at development and at runtime. The Object Prevalence concept, developed by the Prevayler team, and implemented in Java, C#, Smalltalk, Python, Perl, PHP, Ruby and Delphi, can be a great a solution to this mess. The concept is pretty simple: keep all the objects in RAM and serialize the commands that change those objects, optionally saving the whole system to disk every now and then (late at night, for example). This architecture results in query speeds that many people won't believe until they see for themselves: some benchmarks point out that it's 9000 times faster than a fully-cached-in-RAM Oracle database, for example. Good thing is: they can see it for themselves. Here's an article about it, in case you want to learn more."

A reader writes:" Persistence for object-oriented systems is an incredibly cumbersome task to deal with when building many kinds of applications: mapping objects to tables, XML, flat files or use some other non-OO way to represent data destroys encapsulation completely, and is generally slow, both at development and at runtime. The Object Prevalence concept, developed by the Prevayler team, and implemented in Java, C#, Smalltalk, Python, Perl, PHP, Ruby and Delphi, can be a great a solution to this mess. The concept is pretty simple: keep all the objects in RAM and serialize the commands that change those objects, optionally saving the whole system to disk every now and then (late at night, for example). This architecture results in query speeds that many people won't believe until they see for themselves: some benchmarks point out that it's 9000 times faster than a fully-cached-in-RAM Oracle database, for example. Good thing is: they can see it for themselves. Here's an article about it, in case you want to learn more."

A reader writes:" Persistence for object-oriented systems is an incredibly cumbersome task to deal with when building many kinds of applications: mapping objects to tables, XML, flat files or use some other non-OO way to represent data destroys encapsulation completely, and is generally slow, both at development and at runtime. The Object Prevalence concept, developed by the Prevayler team, and implemented in Java, C#, Smalltalk, Python, Perl, PHP, Ruby and Delphi, can be a great a solution to this mess. The concept is pretty simple: keep all the objects in RAM and serialize the commands that change those objects, optionally saving the whole system to disk every now and then (late at night, for example). This architecture results in query speeds that many people won't believe until they see for themselves: some benchmarks point out that it's 9000 times faster than a fully-cached-in-RAM Oracle database, for example. Good thing is: they can see it for themselves. Here's an article about it, in case you want to learn more."

ubiquitin writes "To avoid confusion with the GIMP, the Film Gimp project has renamed itself to CinePaint. The project is essentially a legitimate fork of GIMP, and is focused on image manipulations for moving pictures." We've mentioned Film Gimp several times lately; it'll be even handier as programs like Cinelerra and Kino grow more polished.

mlamb writes "Statistical mail classifiers like PopFile save time on the part of their users, but don't do anything to actively combat spam. I just published an article that suggests a way to use classifier output against a spammer while they're connected to your SMTP server, and I'm launching a project called TarProxy to implement it."

ChrisDolan writes "The folks who benevolently dictate the creation of PCGen, a D20 character generator tool (e.g. for D&D), are going to start charging for downloads of data files. This comes after a long series of talks with Wizards of the Coast. The PCGen code will continue to be LGPL, but some of the data files (a separate download) will be more encumbered than just OGL (Open Gaming License). The specific data files that will cost are ones that were never released under OGL and have WotC IP in them. Details on the Code Monkeys site." PCGen is a nifty app, but all this stuff annoys me. I bought all the 3rd ed books already after all... it seems stingy to charge users twice.

Leeji writes "Here is an interesting read about one geek's project to build and launch a weather balloon. The flight recorder is a small $200 Soekris Engineering computer running Bering Linux. It also uses a Garmin GPS, HAM packet radio, an automated Aiptek Pencam Trio digital camera, army surplus batteries, and lots of geek duct tape."

Cap'n Grumpy writes "You know the score - you've just finished some coding, do a final cvs update before commiting, and all of a sudden all hell breaks loose. Your code now refuses to compile, or xunit starts flashing up red - test failures! One of the other members of your team has checked in something which breaks the build, and they just went out for lunch ... Argh! Did you know there is a solution to this problem? It is a system which makes it impossible for people to check in code which does not compile or test successfully. It allows coders to review others coding efforts code before it goes into the baseline, rather than after. It organises your checkins into logical change sets. It enforces continuous integration. It is linux based, and GPL'd. It's called Aegis."

fraser_joat asks: "The other day I finally took the time to watch Starship Exeter, previously reported on Slashdot. Coincidentally, I also revisited the BBC's excellent radio adaptation of The Lord Of The Rings, following the hype caused by the recent movies. The two of these got me thinking: while _Exeter_ was clearly a huge effort, it looks like they had a lot of fun making it. In many ways they are scratching the same sort of itch that generates free software. So what about audio drama? The technology needed to produce it is freely available, things like Ardour and Csound. So is it possible to produce an audio drama based on free texts such as those from Project Gutenberg in a distributed fashion, with contributers from all across the Net, just like with software? Would they even be useful as an introduction to classic fiction or just as pure entertainment?"

"While the technology exists to cut a play together, I see several possible problems:

• High-quality audio recording equipment is expensive, and homes are not ideal environments. Can source material of sufficiently good quality be generated without professional facilities?
• Since the actors could be widely separated, can they act in isolation in a sufficiently convincing manner that they can be cut together later, in the same way that film actors must pretend that the special effects exist during shooting?
• Are there good (royalty-)free sound effect libraries available?

I think the possibilities are interesting, if people can be gathered together to actually do it. Imagine the subtle horror of Poe's The Cask Of Amontillado, or the adventure of Stevenson's Treasure Island, all staying as faithful to the book as possible, without Hollywood's story-twisting and sensationalism spoiling it all.

It would need to be a real community effort - I fancy that I could produce a passable script adaptation of a book and help with the audio production and sound effects, but I'm no actor, nor do I have equipment at home that even approaches what would be required. What about it?"

LowneWulf writes "An InternetNews.com article mentions that the OASIS standards group today ratified the Extensible Access Control Markup Language 1.0 specification. But even better, Sun Microsystems Labs has backed this up with an open-source version in Java on Sourceforge."