Slashdot Mirror

Today we have Part 2 in our exciting 2 part series about the infrastructure that powers Slashdot. Last week Uriah told us all about the hardware powering the system. This week, Jamie McCarthy picks up the story and tells us about the software... from pound to memcached to mysql and more. Hit that link and read on.

The software side of Slashdot takes over at the point where our load balancers -- described in Friday's hardware story -- hand off your incoming HTTP request to our pound servers.

Pound is a reverse proxy, which means it doesn't service the request itself, it just chooses which web server to hand it off to. We run 6 pounds, one for HTTPS traffic and the other 5 for regular HTTP. (Didn't know we support HTTPS, did ya? It's one of the perks for subscribers: you get to read Slashdot on the same webhead that admins use, which is always going to be responsive even during a crush of traffic -- because if it isn't, Rob's going to breathe down our necks!)

The pounds send traffic to one of the 16 apaches on our 16 webheads -- 15 regular, and the 1 HTTPS. Now, pound itself is so undemanding that we run it side-by-side with the apaches. The HTTPS pound handles SSL itself, handing off a plaintext HTTP request to its machine's apache, so the apache it redirects traffic to doesn't need mod_ssl compiled in. One less headache! Of our other 15 webheads, 5 also run a pound, not to distribute load but just for redundancy.

(Trivia: pound normally adds an X-Forwarded-For header, which Slash::Apache substitutes for the (internal) IP of pound itself. But sometimes if you use a proxy on the internet to do something bad, it will send us an X-Forwarded-For header too, which we use to try to track abuse. So we patched pound to insert a special X-Forward-Pound header, so it doesn't overwrite what may come from an abuser's proxy.)

The other 15 webheads are segregated by type. This segregation is mostly what pound is for. We have 2 webheads for static (.shtml) requests, 4 for the dynamic homepage, 6 for dynamic comment-delivery pages (comments, article, pollBooth.pl), and 3 for all other dynamic scripts (ajax, tags, bookmarks, firehose). We segregate partly so that if there's a performance problem or a DDoS on a specific page, the rest of the site will remain functional. We're constantly changing the code and this sets up "performance firewalls" for when us silly coders decide to write infinite loops.

But we also segregate for efficiency reasons like httpd-level caching, and MaxClients tuning. Our webhead bottleneck is CPU, not RAM. We run MaxClients that might seem absurdly low (5-15 for dynamic webheads, 25 for static) but our philosophy is if we're not turning over requests quickly anyway, something's wrong, and stacking up more requests won't help the CPU chew through them any faster.

All the webheads run the same software, which they mount from a /usr/local exported by a read-only NFS machine. Everyone I've ever met outside of this company gives an involuntary shudder when NFS is mentioned, and yet we haven't had any problems since shortly after it was set up (2002-ish). I attribute this to a combination of our brilliant sysadmins and the fact that we only export read-only. The backend task that writes to /usr/local (to update index.shtml every minute, for example) runs on the NFS server itself.

The apaches are versions 1.3, because there's never been a reason for us to switch to 2.0. We compile in mod_perl, and lingerd to free up RAM during delivery, but the only other nonstandard module we use is mod_auth_useragent to keep unfriendly bots away. Slash does make extensive use of each phase of the request loop (largely so we can send our 403's to out-of-control bots using a minimum of resources, and so your page is fully on its way while we write to the logging DB).

Slash, of course, is the open-source perl code that runs Slashdot. If you're thinking of playing around with it, grab a recent copy from CVS: it's been years since we got around to a tarball release. The various scripts that handle web requests access the database through Slash's SQL API, implemented on top of DBD::mysql (now maintained, incidentally, by one of the original Slash 1.0 coders) and of course DBI.pm. The most interesting parts of this layer might be:

(a) We don't use Apache::DBI. We use connect_cached, but actually our main connection cache is the global objects that hold the connections. Some small chunks of data are so frequently used that we keep them around in those objects.

(b) We almost never use statement handles. We have eleven ways of doing a SELECT and the differences are mostly how we massage the results into the perl data structure they return.

(c) We don't use placeholders. Originally because DBD::mysql didn't take advantage of them, and now because we think any speed increase in a reasonably-optimized web app should be a trivial payoff for non-self-documenting argument order. Discuss!

(d) We built in replication support. A database object requested as a reader picks a random slave to read from for the duration of your HTTP request (or the backend task). We can weight them manually, and we have a task that reweights them automatically. (If we do something stupid and wedge a slave's replication thread, every Slash process, across 17 machines, starts throttling back its connections to that machine within 10 seconds. This was originally written to handle slave DBs getting bogged down by load, but with our new faster DBs, that just never happens, so if a slave falls behind, one of us probably typed something dumb at the mysql> prompt.)

(e) We bolted on memcached support. Why bolted-on? Because back when we first tried memcached, we got a huge performance boost by caching our three big data types (users, stories, comment text) and we're pretty sure additional caching would provide minimal benefit at this point. Memcached's main use is to get and set data objects, and Slash doesn't really bottleneck that way.

Slash 1.0 was written way back in early 2000 with decent support for get and set methods to abstract objects out of a database (getDescriptions, subclassed _wheresql) -- but over the years we've only used them a few times. Most data types that are candidates to be objectified either are processed in large numbers (like tags and comments), in ways that would be difficult to do efficiently by subclassing, or have complicated table structures and pre- and post-processing (like users) that would make any generic objectification code pretty complicated. So most data access is done through get and set methods written custom for each data type, or, just as often, through methods that perform one specific update or select.

Overall, we're pretty happy with the database side of things. Most tables are fairly well normalized, not fully but mostly, and we've found this improves performance in most cases. Even on a fairly large site like Slashdot, with modern hardware and a little thinking ahead, we're able to push code and schema changes live quickly. Thanks to running multiple-master replication, we can keep the site fully live even during blocking queries like ALTER TABLE. After changes go live, we can find performance problem spots and optimize (which usually means caching, caching, caching, and occasionally multi-pass log processing for things like detecting abuse and picking users out of a hat who get mod points).

In fact, I'll go further than "pretty happy." Writing a database-backed web site has changed dramatically over the past seven years. The database used to be the bottleneck: centralized, hard to expand, slow. Now even a cheap DB server can run a pretty big site if you code defensively, and thanks to Moore's Law, memcached, and improvements in open-source database software, that part of the scaling issue isn't really a problem until you're practically the size of eBay. It's an exciting time to be coding web applications.

Last week hundreds of you posted questions for Slashdot's CmdrTaco, AKA Rob Malda. Today we present his answers to 10 of the highest-moderated questions. CT: You can continue to sign up for 10 year anniversary parties but we're already working on shipping shirts so you won't be able to get a care package... but you can still try to run for the big grand prize by just taking videos of pictures or just doing something cool at your parties to prove that we should have been there.

1) Question: Trends (Score:5, Interesting) by vinn (4370)

You've probably followed more news stories and trends over the past decade than just about anyone else.

Based on that, what are your predictions for the next 10 years?

Some technology is obviously going to die a quick and painful death. Some of that technology will be good and some deservedly bad. What's going to catch on? What has staying power? Google has been a golden child the last few years, will that continue? Are there any big turnarounds coming? Who's got good stuff in the pipeline? Don't you dare tell me 2008 is the year of Linux (and I know you won't) - we've both been hearing that marketing crap for the past 10 years.

CmdrTaco:

I don't think I have a particularly unique perspective on these matters. We all read the same Slashdot. What we'll see is mostly obvious: Smaller, Faster, More Portable, More lawsuits, less individual rights. The year of Linux is long passed. Linux will have a strong position on the server for a long time, but as GNOME and KDE bickered with each other, Apple came along and gave the world a great desktop UNIX. It's sad, but true, and there's a huge lesson to learn there. It'll be interesting to see how long Google will be the golden boy- that just can't last forever, can it? I just hope that when the future gets here, we still have the right to copy our own data, and take apart and hack our own gadgets.

2) Have you any regrets? (Score:5, Interesting) by cOdEgUru (181536)

Have you ever regretted starting Slashdot, or investing so much of your time into this site? Did any actions by your peers, by the community or by your colleagues, as a result of a story posted on Slashdot or related to one, made you ever regret your decision to start Slashdot.

CmdrTaco:

Sure. Running Slashdot under the umbrella of a publicly traded company is a huge challenge. A company is a beast that must always eat more... and some people think that making a number for this quarter is so important that it means sacrificing ideals that might hurt you next quarter. Like if I put 15 ads on the page tomorrow, we'd make a lot of money for 3 days and then most of you would leave, and so we'd have great revenue for a week and then no revenue ever again.

So, much of my job is making decisions and fighting with other people at the company to make sure that there still is a Slashdot worth reading next year and the year after that. And advertisers would simply like to buy stories... now, contrary to what conspiracy theorists accuse us of, we don't sell stories. And it quite honestly hurts me when people accuse us of it. But it's scary to know that some folks in the company would be quite happy to do it, completely selling out the integrity of the site to get a bonus. I guess thats a big part of why I stay here: I think Slashdot matters and at least when I'm here I can try to keep it on the path.

3) In and out of Slashdot. (Score:5, Interesting) by pavon (30274)

These are probably pretty cliche questions, but I am interested in the answers.

What is a normal day at slashdot like? How much time do you spend improving slashcode vs picking stories vs the normal computer admin tasks vs other stuff. How are the workload/responsibilities split up among the different staff members? How has this changes over the years?

I also remember back in the old days, the work you did with Enlightenment, as well as the animated short you made (Duckpins?). I was wondering if you get the chance to do much programing outside of slashcode, or what other hobbies you spend your free time doing now (besides being married).

CmdrTaco:

I have a couple of different jobs. One is posting stories- on a day where that is my primary responsibility, I might get in at 7:30 a.m. and read submissions and post them until early afternoon. During this time I might reject a few hundred submissions, post a half dozen stories, and of course try to keep up on my email. Beyond that, I have a number of meetings (a monthly author meeting, a weekly coder meeting, and countless random other meetings for marketing/sales/etc.). I always have chat windows open with various members of the company discussing whatever projects are outstanding.

It's not that different from when we started, except that 10 years ago I would have a terminal window open with code, now I have a chat window open with coders, and 10 years ago I would post stories, and today I have a chat window opened to a group of people who can all post stories.

I read every story posted. I read discussions when the subject matter is particularly interesting. But after that, I make sure that everyone is working on the right stuff, and that things work the way I want them t o work.

These days my time for hobbies are limited, but when I have time I play video games or just goof around with software or hardware. Pretty much all of my free time is consumed by Zachary, the currently 12 lb. terror that exploded out of Kathleen last August. He's awesome.

4) Okay, I'll bite (Score:5, Interesting) by Skyshadow (508)

Something I've been sort of curious about for ages:

Can you talk a little about how you experienced some of the dotcom insanity, specifically as it unfolded here at Slashdot? For a while, it seemed like Slashdot was about to become wunderkind central -- the sale to VA, the infamous ESR post about uber-wealth, etc. I'd be interested to hear about how that experience translated from your side of the ball.

CmdrTaco:

I was seriously buffered from most of the dot com boom. I lived in Nowhere, Michigan so I only saw it when I went to SFO or NYC or Boston for a tradeshow or a meeting. It wasn't until it disappeared that I realized how big it was, and then only by absence: to go to an office building and see row after row of empty cubicles... it was sad.

Slashdot didn't change that much during that era. We added a few writers and a few coders. We bought a few new servers, but even today we run a very lean operation on the production side of the site. Basically 2-3 coders and 2-3 writers replaced me working 20 hour days.

As for the ESR post, I found it very embarassing. I'm of the Gen-X/Grunge era. I cling tightly to my flannel shirt and would never publicly make such a boastful post. Even today, I hate marketing Slashdot. I dislike doing press for Slashdot. I've always felt that if we do a good job, people will read, and there's no reason to hype the site. This is anathema to corporate life, which is why we do things like the 10-year anniversary thing. The only reason we're doing it is that I really felt that after 10 whole years it was worth a bit of reflection.

Personally, the bubble made it possible for me to own my own home at a time in my life when most people my age were living in 1-bedroom roach motels, or worse, with their parents. I'm thankful for that. But when the bubble burst, it took with it my dreams of having a private jet or something, and I was left with a job that pays really well doing something I like.

When the bubble burst I learned a lot and realized that I had made a number of mistakes a long the way. Lessons learned, I guess. It would have been nice to have zillions of dollars, but there are other things that are more important.

5) Silly Question (Score:5, Interesting) by LiquidCoooled (634315)

I assume that through the ether you have met Kevin Rose, but do you two get along or is it pistols at dawn?

CmdrTaco:

Yeah, I met him when I did an interview on some TV show I guess he was hosting. He seems like a sincere guy, and I have no problem with him.

People love to paint rivalries between Slashdot and whatever website they think we are battling at the moment, but I really resist the urge to compare Slashdot to sites like Digg. We do different things and serve different audiences. There's crossover to be sure, but to shoot a guy in low sunlight seems kinda silly.

6) What is this crazy tags thing? (Score:5, Interesting) by Reality Master 101 (179095)

Considering the FAQ hasn't been updated in almost a year, could you explain exactly what tags do these days? At one time, it seemed to be a vote-based system, now I have no idea how tags show up on articles. Frankly, since I didn't understand it and my tags didn't seem to affect anything, I gave up on using the feature.

Could we get a definitive answer to how tags work?

CmdrTaco:

I don't know exactly how 'Definitive' this is... but 'tags' is just an experimental system for us. We're using it for ideas on how we could improve moderation and firehose ratings. We're using it to see what ways people will try to screw with the system. Tags are very open-ended and are therefore used for many things. People use them for opinions, abuse, classification, and sometimes just as an attempt at wit. The system can be all of those things, but when we see abuse we definitely try to stop that.

Basically the way tags work is that you add words that you think are cool. If many people tag similarly, those tags appear on the articles. You can use tags to be informative (a tag like 'slashdot' on this story would make sense since I'm talking about Slashdot) or to provide helpful feedback to editors ('dupe' or 'typo' for example). It's very open ended, and as long as your tags are beneficial to others, we like seeing them.

I don't want to narrowly define tags, either: Sometimes a silly witty tag rises to the top. It may reflect an opinion or a joke, but thats ok as long as it's not mean. At the end of the day, we're learning a lot from how people use tags- knowledge that we're using to make firehose better, and ultimately to make moderation better.

Under the hood, we've actually ported moderation to tags... so now we can more easily expand moderation to incorporate aspects of tagging. The issue here is that we have 2 major differences between moderation and tags: moderation has a very limited domain of tags, and you are very limited in how much/how often you can moderate. So we can't simply flip a switch and use tagging instead of moderation, but many of the tools and rules overlap nicely. Personally I think it's probably the most interesting aspect of what we're playing with on the site. We're not doing tags like anyone else, and I think that's what is fun about it.

7) Most-visited sites.. (Score:5, Interesting) by B5_geek (638928)

What "Top-5" websites are in your daily/hourly must-read rotation? (Not counting RSS)

CmdrTaco:

I only really read the internet via the firehose and via RSS, so I guess I can't really answer this question. I think that if you read Slashdot's firehose, you don't really need to read any other tech news publications since it will contain the best of all those other websites. So the sites that I read beyond the direct Slashdot subject matter tend to be comedy websites or comics... these days Penny Arcade and XKCD are my favorite comics although my feed has a dozen more. Also things like Cute Overload or College Humor. If it's tech news, the firehose has it covered... but if it's funny, well I need to work to get that.

8) Thoughts of giving up? (Score:5, Interesting) by martyb (196687)

When were you most tempted to give up?

Dealing with a bunch of creative, resourceful, tenacious, stubborn, and sometimes outright hostile nerds, I'm sure there were MANY times when you were tempted to just give up on the whole thing. e.g. page-widening trolls; Church of Scientology; Microsoft source code, or even the release of slash code to the community and the barrage of insults.

I'm really glad you held on and persevered, but I'd like to know why.

CmdrTaco:

When the shitty days come, I really wonder why I do it. Hate mail in my inbox. Flame in the forums. DDoS attacks. Sales/Marketing pressuring us to do something stupid. Or the more standard stuff that goes along with being part of a company- paperwork and bureaucracy etc.

I can usually handle the user problems... I've come to understand that if you do anything successful you will create some percentage of fans... and as a subset of fans, you get anti-fans. On one level it's flattering: This is a person so passionate about your work that he will spend hours trashing you in any forum possible. It's crazy... it can really hurt if you let it, and sometimes it does.

I've pondered leaving many times over the years but I always come back to wanting to make this thing work. I really like Slashdot and think it's a better site with me here than away. I can't imagine what others would do to it if I left!

The thing is that every now and then we do something important. Like really important. We break a story, or house a discussion that changes a mind. I think that we serve an important role on-line. We're a pub where people gather to talk about the days events, and I think this has tremendous value. I think I still am here because there's a community here that I like. And besides, it beats flipping burgers.

9) Infrastructure (Score:5, Interesting) by blhack (921171)

Can you give us any insight into the hardware/platform that slashdot runs on? How many servers does it use? What did you code it in? (a half drunk, coked-up deaf guy screaming HTML into a tin can on a string?) How much bandwidth does it use?

I know this is more than one question, but my MAIN question is just: "What does it take to run slashdot, hardware/software/bandwidth wise?"

CmdrTaco:

We'll actually have a lengthy discussion of exactly this before the 10-year anniversary stuff is done. But in short, we're talking about a dozen dual CPU web-heads, 4 quad CPU mega database boxes. We share bandwidth with SourceForge, so we don't use much bandwidth... Slashdot doesn't host video or many pictures so we're fairly cheap. The code is all at www.slashcode.com so you can download it and play with it for yourself. It's all Perl/Apache.

10) What are the biggest threats to /. success? (Score:5, Interesting) by rjamestaylor (117847)

Slashdot is successful by any measure. You've certainly pioneered many things we now take for granted. Many "slashdot killers" have been attempted and failed or found a different niche. What are the biggest threats to /. success today and going forward?

CmdrTaco:

I think the single biggest threat to Slashdot is for us to try to be something we're not. We are NOT CNet. We are not Digg. We are not Wired. We are not Reddit.

Those sites have many things that define them... from the source of content to the method of content selection, to the sorts of business partnerships and types and quantity of advertising on each of those sites, each has a sort of place, and Slashdot isn't exactly any of those things.

The future success of Slashdot depends on us understanding what Slashdot was for the last 10 years and how to continue to be that in the future. The names change, but the fundamental underlying joy of technology shouldn't.

We need to know who you guys are, and what you want, and try to give you what you want in a website, but without selling out what Slashdot has been. We have a decade of legacy now... our single biggest threat is to ignore our past and try to be whatever is popular today... but that's not to say we can't change.

We need to incorporate many of these popular ajax/web2.0 technologies and ideas- our readers deserve the improved browsing experience. But it's a careful balance between taking what is good about what is available today, and blending it with what has worked about Slashdot throughout our history.

It's a mistake for us to want to be CNN or the Wall Street Journal or to spend our days chasing after Digg, or Reddit, or Kuro5hin, or whatever site follows them. We strike our own path. We'll never be the #1 traffic destination on the net, but we're still regularly a great website, and one that I'm proud to continue to be part of.

-- Pants are Optional

Discussion2 has been in beta for a few months now on Slashdot. Initially available only to subscribers, it now should be available to anyone willing to login and click the checkbox at the head of every discussion. It is far from complete: IE doesn't work (patches welcome, but since only a quarter of you use it, it's not a huge priority) and performance is lacking (you want a fast computer for larger discussions) but it's already an improvement for most users. Read on for some notes on what we have planned.

The primary reason for discussion2 was to get beyond the pain in the ass that is navigating large discussion threads on Slashdot. You know the problem: once threads get deep, you have to click repeatedly, waiting for tabs to load. Or even when you encounter a long comment, you have to wait to get the full comment text.

Cool Things D2 Does Now

1. Allows you to change your threshold, open and close threads, and expand long comments in place, without ever loading a new page.
2. Allows you to moderate a comment without clicking a save-button that loses your place in a thread.
3. A new, more intuitive user interface that more clearly displays the nature of comment thresholds.
4. Vastly Improved threaded view that allows you to see more of the discussion in less space, without clutter.

Some items on the TODO list (more or less in order of priority)

1. Make it Progressive - Right now D2 simply gets all the comments in a discussion. This sucks. We need to write a task to retrieve only appropriate comments. So if you are at Score:4 threshold, we don't bother retrieving the full text of all comments at Score:-1. And even better, if someone moderates or posts a comment, we need to update the page you are reading to reflect those changes. Again, the goal here is that once you load a page, you don't need to close it until you are done with the discussion. This actually has MANY subtle problems, like how do you notify a user when a thread 10 pages up has been replied to.
2. Make it Fast Actually I think solving #1 will mostly solve #2 at the same time. Since right now we get the full discussion, we are getting WAY to much data. We need to get say 50 comments at a time, not all 1000. This will give your browser time to catch up and make the whole thing "Feel" faster. Right now, on my machine a 200-300 comment page is very usable, but to much larger and it starts slowing down. This is all machine dependent. I'm sure there are good javascript tricks that would help improve performance.
3. In-Place Posting You should be able to post a comment without reloading a page. Right now you can just open a tab, but then you are looking at a stale discussion. This isn't that hard either- especially once we finish #1. Just need to open the reply page in a div, and when you save, make sure that the new comment is properly retrieved and inserted into the thread. But there's some subtle stuff here like how to handle previews. We need to change some of our error handling- the current system uses previews as an opportunity to warn readers about things that are "Wrong" about their comment. We need to figure out how to do that without launching new pages. It's not hard, but it'll take some time.
4. Compatibility ok so Opera's broken Javascript implementation won't work unless they fix their browser, but we'd like to make at least IE work for the trivial percentage of Slashdot readers forced to use IE by their corporate overloads. But since 2/3rds of you use Firefox, fixing IE is just not at the top of my priority list... I'd rather make it work better for the majority. And as every web developer knows, cross browser platform compatibility can be a real bitch. But before we are out of beta, it probably would be nice to get IE functional, if only for other websites using our source code that actually have IE as the dominant population.
5. Smooth out the UI there are a lot of parts to this problem. Right now the threshold change is buttons but it should actually be draggable, I'd like the widget to toggle from the top to the side, but need to build a horizontal version of the widget. The expansion/contraction of comments and threads have weird functionality that could be improved- for example there is a difference between expanding a comment and expanding a thread. And there's new concepts like expanding a child vs expanding an entire thread vs expanding "Siblings" vs expanding hidden children vs visible children. These are very interesting user interface questions that we'll start working out soon.
6. Rethink What Old Functionality By this I primarily mean discussion filters and ordering. By default D2 uses a thread ordered, chronological display. The old system had many other sort modes, but I'm not how sure how effective these are once threaded. So I may simply leave the old system in place for users who want to see a flat discussion ignorant of threads ordered by date or score. Since this is only a tiny percentage of users, I figure it can wait.

Conclusion

A lot of the stuff you see in D2 is just javascript you can easily play with yourself. We haven't mangled it or anything so you js haxx0rz are welcome to submit patches for interesting ideas. We don't have a backend for progressive rendering, but there are a LOT of features that we want to implement that wouldn't even require you to touch the perl. Of course if you're willing to hack perl, it's all up on the website not that anyone ever actually bothers to contribute anything more than ideas and complaints, but it sure never hurts to ask!

Already around 13,000 of you are using Discussion2. We're a ways off from flipping a switch to make it the default for everyone, but it's already substantially better for users with fast computers and Firefox. Hopefully in a few more weeks it will be good enough for everyone. Thanks for the help along the way. We hope you like the new system... I sure do. And mad props to Nate & Pudge for their work on this...

Recently we've added a new bookmark function that logged in users can use to maintain public lists of URLs for tagging and sharing with the world. You can read the bookmark faq to answer a few specific questions, check out the global bookmark lists that can be tagged, journaled over, or submitted as stories. Read on for notes on what we're planning for this.

The idea is that URLs are bookmarked, either using the javascript bookmarklett thingee or by directly visiting the bookmarks page. But when you are DONE bookmarking and tagging your URL, you can then write a journal or story submission about them. On a related note, journals are now directly submittable to Slashdot editors as well.

We plan to add RSS exporting as well as a few other features, but mostly what we're going to try to do is figure out ways to mine bookmarks for stories. You can best help by tagging bookmarks on the global bookmarks pages. This is all very experimental, so your help is appreciated.

Now as always on Slashdot we appreciate your feedback and bug reports. You can do this the slashdot SF project page. Or even better you could visit the site and consider submitting patches. Ideas are never in short supply here- time to code them all ALWAYS is. Join the mailing list! Or just start bookmarking and tagging stuff. Lastly, thanks to Tim Vroom who put all of this together.

update to clarify a point for the readers- this system is primarily an extension of our submission bin. You are welcome to bookmark & tag for your own pleasure, but our intent is that this system eventually be used to help us find content for the mainpage. So of course the functionality has overlap with other sites. No huge shock there. But our intentions is to use user bookmarks & tags as another input channel for editors picking stories for the audience.

riflemann writes "Another 26 episodes of Futurama will shortly go into production! This news comes from none other than Billy West (voice of Fry) himself, in a short post to his own message board. No further details are available, except that it's likely to be on TV, not straight to DVD." The best news is that means fresh quotes for slashteam to hide in the source code.

I've spent seven years working as a writer and editor for Slashdot's parent company. During this time I've been to at least a dozen mainstream journalists' and editors' conferences where the most-asked question was, "How do we adapt to the Internet?" You'd think, with all the smart people working for newspapers, that by now most of them would have figured out how to use the Internet effectively enough that it would produce a significant percentage of their profits. But they haven't. In this essay I will tell you why they've failed to adapt, and what they must do if they want to survive in a world where the Internet dominates the news business. I'm going to use the Bradenton Herald as an example, not because it's a bad newspaper but because I live in the middle of its circulation area. The Herald is a typical Knight Ridder small-city newspaper in every way except one: it serves Manatee County, an area with a fast-growing population where most new residents are old enough that they grew up reading newspapers every day. Despite these favorable factors, the Herald's circulation has declined by 3.5% in the last year. Of course, newspaper circulation declines are now normal rather than exceptional. Other newspapers have done far worse, with the San Francisco Chronicle recording a 16.4% drop in the last six months alone.

Readership vs. Circulation

Much of the Chron's circulation decrease was because it stopped giving away free papers. The Boston Globe also stopped a giveaway program and suffered a circulation decline as a result, although only about half as big a loss as the Chron's, but the Globe's marketing people have said that only half of the loss came from stopping the giveaways, and blamed the rest of it on the usual suspects, notably TV and the Internet.

These figures only measure paper newspaper circulation. They don't include Web readership, which generally seems to be trending (slowly) upwards on newspaper Web sites. Circulation figures can also be misleading because they only measure the total number of newspapers distributed, not the kind of people who read them. And readership quality can often be more important, in a business sense, than quantity. This is especially true for those newspapers (namely, just about all of them) that rely on advertising for the bulk of their income.

By definition, anyone who reads a newspaper online at home can afford a computer and an Internet connection, which means they aren't at the very bottom of the economic pile. Online readers are also likely to be more open to new experiences, products, and services than those who don't feel they need to use the Internet -- which by some estimates may be as many as half of all households within the Herald's circulation area, which has a higher percentage of retirees than all but a few other U.S. counties.

Journalism professor Douglas Fisher and media executive Alan Mutter have both talked about intentional circulation losses on their blogs. In his post, Fisher says, "The industry evolves to the point of small, expensive print publications and most of the 'mass' news on the Web somehow. Then, as we evolve toward paid content online will come issues such as whether a certain amount of 'base' information should be free for every person -- sort of like a public utility of information (perhaps presented as a social utility necessary in a functioning democratic society)."

Meanwhile, when newspapers talk about readership vs. circulation, they're typically trying to estimate how many people read each copy of their print product (pdf download) rather than come up with a total picture of their publication's readership, including its online presence. This is a mistake. Instead of treating their Web sites like unwelcome stepchildren, newspapers should turn them into their primary method of news delivery -- and teach their reporters, editors, and ad sales people how to work effectively with this new -- to them -- medium.

Slashdot Lessons

1. No matter how much I or any other reporter or editor may know about a subject, some of the readers know more. What's more, if you give those readers an easy way to contribute their knowledge to a story, they will.

Imagine a newspaper with a space for comments below each story on its Web site. This Slashdot story has comments directly attached to it, not tucked away from public view the way the Bradenton Herald's site hides reader comments on Bulletin Boards that aren't directly connected to any of the paper's articles or editorials. To make matters worse, the Herald's Bulletin Boards require a separate login to post. Even if you're a logged-in reader you must put in your username and password again to use them.

As a result of these posting barriers, you hardly see any reader comments on the Herald's site, and what few there are seem to come from a small group that posts over and over. Even the Herald's single (hard to find) blog, maintained by token hip-dude entertainment reporter Wade Tatangelo, draws so few daily comments that you could count them on the fingers of one hand -- and usually have four or five fingers left over.

By contrast, the Washington Post's Web site has two blogs, Achenblog and The Debate, prominently displayed on the Opinions page that almost always draw 100+ comments per post.

A truly Web-hip newspaper would not only allow but encourage reader comments on all of its stories, not just on a blog or two. With thousands of readers as fact-checkers, mistakes would rarely go uncorrected for long, and if there was any perceived bias in a controversial article, reader comments would make sure the other side got heard. Even better, a reader who witnessed an event the paper covered would be able to add his or her account of it to the reporter's, which would give other readers a richer and deeper view of it.

2. Not all readers know what they're talking about.

While some readers know more about any given topic than a professional journalist writing about it, most don't. Some, indeed, post anything about anything, including misleading or false information. This is why Slashdot has a moderation system, and why all newspaper Web sites need to have moderation systems in place before they allow reader posts attached directly to stories. Slashdot's, which is built into the code that runs the whole site, is probably too complicated for most newspapers, but everyone (including newspaper publishers) is free to download, use, and modify it. For those who don't want to use the code behind Slashdot, there are many other free (and proprietary) content management programs available that have similar -- and often simpler and less geeky -- moderation features built into them.

3. No matter what you do, some readers will post malicious and/or obscene comments

Slashdot removes posts only in response to Cease and Desist orders or legitimate copyright infringement complaints. We find that malicious or obscene posts are usually moderated into oblivion almost immediately, because our readers -- hundreds of whom have moderation power at any given moment -- have a sharp eye for stupid stuff.

A mainstream newspaper might choose to remove blatantly disgusting posts, which would take some staff time. There would also -- inevitably -- be second-guessing and complaints, including whines from readers who believed their posts were removed because they didn't follow the [fill in political party here] line, not because they used offensive language.

Moderation never makes everyone happy. Someone will always feel the rules are too loose, while someone else will believe they're too tight. And moderates -- I mean moderators -- will always get flak from ____-wingers who think they're biased. But these problems shouldn't stop grown-up newspaper people from soliciting and publishing readers' posts. They should already be accustomed to bias accusations.

4. What if readers post comments that advertisers don't like?

This is a problem, and one to which some newspapers are extremely sensitive --not just over readers' comments but sometimes over their own reporters' stories. A 1999 Washington Monthly article had some examples of how newspapers sometimes cater to advertisers instead of their readers. Allowing readers to comment on stories, and allowing them to post anything they want (other than obscenities, blatant hate speech, and personal attacks) increases readers' faith in the newspaper, which makes it a more effective advertising medium in the long run because some of that trust will rub off on advertisers that support it.

The Business Side of a Newspaper Web Site

Slashdot, like almost all other Web, broadcast, and print media outlets, depends on ad revenue for most of its income. For the first few years of its existence as a commercial entity, major advertisers were afraid to buy ads on Slashdot or other free-wheeling, community-driven sites. They worried that every time they touted a product, all the customers they'd ever irritated would post bad things about them. It's impossible to run a company of any scale without having at least a few dissatisfied customers, no matter how good your products and services are, so this was not an unjustified fear.

Luckily for Slashdot (and our parent company), many companies have learned that they are going to get criticized online whether they like it or not, so at the very worst, running ads on pages where they get slammed gives them a chance to tell their side of the story.

Keyword-based ad placement helps them do this. Imagine making software that's often knocked for its security vulnerabilities, while competing software is available that costs little or nothing and doesn't share your product's problems. You'd want to run a Get the FUD (Fear, Uncertainty, and Doubt) campaign on every Web page where the competing product was being discussed so that you could tell people who are (obviously) interested in the competing product how awful it is, and why they should buy yours instead.

On a local newspaper Web site, a developer intent on replacing pristine wilderness along a scenic river with ugly condominium towers in the face of opposition from local citizens' groups could run a keyword-targeted campaign explaining why their buildings would be better than a swampy, mosquito-ridden riverfront. They could stress the fact that they would reduce the population of turtles, spiders, alligators, shore birds, frogs, and other annoying wildlife, and that runoff from their chemically-fertilized landscaping would help keep local fish populations down by contributing to red tide, thereby reducing the number of smelly fishermen infesting the area.

Other, more sensible, businesses would use the same tactic -- keyword ad placement -- to sponsor discussions in a positive way. An obvious example here in Florida would be resort property owners linking ads to tourism-related stories and the discussions attached to them. With geotargeting becoming common on the Web, ads aimed at visitors could be visible to all of a Florida newspaper's online readers, while ads for a local business would only be shown to local residents -- unless the local advertiser was canny enough to realize that Florida has many thousands of seasonal residents, and that reaching these snowbirds through the local newspaper's Web site before they come South is a great way to get a leg up on competitors.

Some other ways to exploit the Web that newspapers don't seem to do well:

• Print-them-yourself coupons. This is lots cheaper than putting coupons in a print newspaper. Many newspapers boast that today's paper contains $___ worth of coupon savings. Why don't more papers make this boast about their online editions? TV stations could do this on their sites, too. This would be an entirely new source of revenue for them, since there is no way to put a coupon in a TV spot.
• Online ad circulars, similar to the paper ones that pack print newspapers on Sundays and holidays. The print ones are expensive to produce and deliver, especially in color. Online circulars would be far less costly.
• Selling sponsorships for community calendars and other "public interest" sections that should be on every newspaper's Web site -- but often aren't or are produced in too scattered a manner to be useful for readers. C'mon, newspaper (and local TV) people! A well-organized, database-driven events calendar is easy to produce. If you don't have one (and sponsors for it), you should.
• Sponsored, "free to individuals and small businesses," local classifieds. craigslist and eBay are busily taking the classified ad market away from newspapers, with Google getting ready to help them with this effort. The Poynter Institute's Steve Outing suggests that the best way to beat back this threat is to "Turn newspaper classifieds into an active and interactive community, instead of just static, dull listings. A cold-hearted newspaper classifieds database could well be smothered by Google classifieds. A local-focused interactive community may be less vulnerable."

The Local-Focused Interactive Community

I believe the future of not only classified ads but of local news gathering and distribution is the "local-focused interactive community." According to this article, craigslist founder Craig Newmark agrees with me. So do plenty of other Web entrepreneurs and venture capitalists who are busily building and financing "community" sites.

Local newspapers should have dominated all of this interactivity from the beginning. They had the name recognition and -- through their print editions -- the promotional muscle to make their Web sites into unassailable community hubs. But they didn't, and now they're reduced to playing catch-up.

If the Sarasota Herald-Tribune had followed through on its plans to incorporate reader-written blogs into its site, Suncoastblog.com probably wouldn't exist. This group blog is an admittedly lame effort, barely begun, put together by several people in this area (including me) who thought it would be nice to have a local site that might eventually cover events and places that don't make their way into the local papers. We know the Herald-Tribune, whose circulation area overlaps the Bradenton Herald's, had thought about hosting reader blogs at one point, because they asked readers to submit blog ideas several months ago. I submitted one and never heard back.

I also submitted a local computer business column concept to the Herald. I came up with it because the Herald has a Sunday business page it calls "Digital Manatee," on which I have never seen anything other than out-of-town wire service material even though there is more than enough local computer and Internet business activity to fill a weekly column, and enough local computer and computer service vendors to surround that column with profitable advertising.

The Herald's editor didn't respond to my proposal. I've written three computer-oriented books, and thousands of articles that have run online and in print all over the world, but I am apparently not worth even a polite turndown from my local paper's editor. No problem. A week later I was having lunch with a couple of local entrepreneur buddies. I told them what had happened. They suggested an online computer business magazine instead of a Herald column, and offered to finance it on the spot, out of their pockets.

I don't have time to start a new publication. But I am in a position to help someone else start one, and to write a story or two for it now and then. Financing's in place. So is a domain name. So at some point the Herald and Herald-Tribune may have (yet) another niche publication competing with them. It won't be a big competitor, but its ad revenue will come from lucrative business-to-business accounts you'd think a local newspaper would be eager to lock up with a weekly (or more frequent) column for local computer-using business people.

This doesn't mean the Herald has a bad editor or that another small paper would have reacted differently. I use this anecdote only to point out that it is now easier to start an online publication than for even a highly-qualified outsider to get his or her work into a local paper. Is it any wonder that local blogs and other online niche publications are springing up like mad? And as a corollary, is it any wonder that newspaper circulation and influence continues to decline?

Newspapers need to open up more to the communities around them. They need to stop confining their interaction with readers to advisory board meetings and questionnaires, and allow readers' stories, opinions, and thoughts to become an integral part of the newspaper itself. They should not allow readers to alter the newspaper's own words, as the Los Angeles Times did back in June with their laughable wikitorial experiment. Moderated comments are a much better way to give readers a voice. So are journals that allow (logged-in) readers the same level of freedom they'd have with their own blogs, but also give them the cachet of being published on a "major brand" Web site.

'Local' is the Key Word

The Herald, Herald-Tribune, and many other (if not most) local newspapers seem to think that they are still their readers' primary source of national and international news, just as they were 20 years ago. So that's what fills their front pages most of the time, with local and regional news stuck in a "B" or "C" section.

Welcome to the Internet age, local newspaper (and TV) people. I can and do get my national and international news from the New York Times, The Washington Post, BBC, Al Jazeera, Fox News, CNN, and other online media that cover faraway events better and faster than you ever will. I turn to you for local news. You tell me more about last week's home invasion robbery on 11th Street East than they ever will.

It's time for local newspapers to become truly local; to feature local news on the front pages of both their Web sites and print editions, with only a few out-of-the-area stories up front, augmented by an above-the-fold story list that tells readers where to find national and international news on their inside pages.

Add readers' stories and comments to the mix and you suddenly have a local online community, not just a newspaper. This will not take work away from professional reporters, photographers, and editors, who will still be the foundation of local news-gathering. In fact, increased interaction with local community members will probably give them more work than ever, because they will find themselves inundated with news tips and story suggestions they never would have found on their own. Some of these story ideas will be dreck and some will be invaluable. It will be up to the newspaper's editors to find the (rare) nuggets in the huge pile of dross they will need to sort through every day, and up to the newspaper's reporters to follow up on them.

One important thing a community-oriented, Web-based newspaper must do is credit readers for their story leads unless they specifically request anonymity. Another good idea is to pay readers who submit news stories that are written well enough that they can run with only routine editing and fact-checking. Those readers are, in effect, doing a reporter's work, and they should get some sort of compensation for it. Some may even turn into stringers capable of covering government meetings and other events when staff reporters aren't available, and a few of those stringers eventually ought to become staff members. After all, if a newspaper is going to be about, by, and for its local community, shouldn't that community be its primary recruiting ground?

Newspapers Will Not Die

Some newspapers (and newspaper chains) will probably not survive the shift from news-as-monologue to news-as-dialog. Most will, although those that wait too long to adjust will have much of their audience, influence, and ad revenue taken away by more agile competitors.

The smartest newspapers will follow my survival recipe or come up with their own way to become an integral part of their community instead of a building full of people who have been sprinkled with Secret Journalism Powder that makes them better and smarter than their readers. These newspapers will not only survive, but prosper. They may even become the prime outlets for bloggers in their communities, which will increase their readership and ad revenue. Extreme ____-wing bloggers won't want their words associated with the hated Mainstream Media, but most others will be happy to have a widely-read, influential outlet for their work.

Eventually, I expect print newspapers to become "snapshots" of their Web editions taken at 1 a.m. or another arbitrary time, poured into page templates and massaged a little by layout people, then sent to the printing presses, a pattern that has potential for significant production cost reductions if handled adroitly. From that point on, their paper editions will be distributed the same way newspapers are now.

Senior citizens and others who can't afford (or don't want) computers are and will continue to be a viable market. So will commuters who use public transportation. Then there are those -- a substantial part of the population -- who simply prefer reading words and looking at pictures on paper to seeing them on a screen. They will still want physical newspapers, even if they are not as up-to-date or as complete as what they'd get on the Web.

However it is delivered, text will not go away anytime soon. For a fast reader, it is the most efficient way to take in large quantities of information. Most people speak at a rate of between 130 and 200 words per minute. Most college students, according to a Virginia Tech student guide, can read non-technical material at 250 to 300 words per minute, and can increase that reading speed significantly with a little thought and practice. Listening to a city council meeting at 150 words per minute takes much longer than reading a meeting transcript at two, three, four or ten times that speed. Now have a skilled reporter -- whether a staff member, paid contributor or volunteer -- write an intelligent summary of that meeting, and even an average reader can learn what happened there in a few minutes instead of slogging through a two hour audio or video recording.

The Web version of that summary can be posted without waiting for the printing presses and delivery trucks to roll, and can have audio or video snippets embedded in it, but there is no reason not to make the text portion of it available on paper for those who prefer it in that form, unless the paper's editors decide so few people are interested in a city council meeting that it doesn't deserve a spot in the print version -- and tracking page readership on the Web version of the paper before the paper edition goes to press should give those editors a good idea of what they should and shouldn't put on paper.

Printed newspapers will have a significant following for many years to come. They may or may not become "expensive," as Professor Fisher predicts, but they will likely become smaller than they are now, and subscription sales efforts will probably be targeted more closely at groups unlikely to have Internet connections, especially senior citizens.

On the Web side, it's likely that newspapers will end up keeping most of their content free, with specialty sections (and posting privileges) reserved for logged-in users. Whether they'll be able to charge for some or all of their Web content is questionable. I paid $50 for a year's subscription to the NYT's Times Select program, and I don't think it's a good enough value that I'll renew my subscription when it runs out. I would be more likely to pay if I lived in New York and that subscription, in addition to what it gives me now, offered access to additional features like complete transcripts of government meetings. Indeed, I would happily pay at least $30 per year to the Bradenton Herald for a well-organized Web edition that gave me what I now get in the paper edition, plus government meeting transcripts and other useful subscriber-only features.

But if I paid for an online subscription to the Herald, I'd probably drop my subscription to the paper edition. I'd still be the same person, with the same interests, earning power and spending habits. The only thing that would change about me, from the newspaper's perspective, would be my news delivery preference.

The challenge for local newspapers that beef up their Web editions at the expense of their paper versions won't be to keep (or add) readers, but to teach advertisers that the Web, not paper, is the best way to reach their most lucrative potential customers.

This may not be easy, but it will be a lot easier than explaining to advertisers why they should keep spending money in a newspaper that has fewer readers, and less influence, every year.

Brian "Krow" Aker (Former Slashdot Coder now MySQL Employee) writes "I am pleased to announce the release candidate for MySQL 5.0. This version has been in development now for three years. We have worked to add update-able views, ansi stored procedures, and triggers. In addition we have added a number of fun features that we are experimenting with and resolved issues with bad data inserts (which personally annoyed the hell out of me when we rewrote Slashdot a couple of years back so I am happy to see this issue go away). We look forward to feedback on the candidate and will show some love for bug reports."

After almost 8 years, Slashdot's HTML is finally getting an overhaul. For now the changes are almost entirely under the hood, as we migrate the current skin to CSS. Slashdot itself will migrate in the next few weeks, but for now, we'd appreciate it if people who understand CSS could take a look at Slashcode. If you use a browser that lets you select a stylesheet, you can take a look at that site with the Slashdot CSS Skin. Keep in mind that Slashcode doesn't look exactly like Slashdot, so there will be some differences between that site, and the final version that will appear on Slashdot. We're mainly looking for feedback on compatibility issues and blatant bugs. You can use our our SF bug tracker to submit bug reports. Thanks for your help. Once we move Slashdot, work will begin on a new look & feel. If you have ideas, you could start playing with the CSS stylesheets now!

herrvinny asks: "What is the best weblog script to use on a personal web site? SourceForge and Google show plenty of weblogging systems available, but I just need a simple, powerful solution. Movable Type has been recommended to me, but I've heard of problems with spam, exploits, and comment flooding. I'd like to have a decently good comments section, where visitors can reply to my ramblings and have a fairly large toolset in which to do so, i.e. smilies, some limited HTML (bold, italic, etc). A small Polling plugin would be terrific as well. Which weblogging systems do Slashdot readers use and recommend? Some complexity isn't a problem; I can work in Perl, HTML, C (among other languages) if I need to. Also, what do people think of adapting Slashcode for such purposes?"

bcapps2012 writes "I just saw this on Slashcode.com and thought it would be of interest to many apple.slashdot.org readers. Pudge has gotten Slashcode installed on Mac OS X. As jwachter notes: 'For those of you who haven't been following the issue of how to get slash running on OS X, various Slashcode posters have been asking how to get it done for roughly 2 or 3 years now (essentially since the first preview of OS X was released).' Finally Mac OS X has joined the family."

Iorek writes "International Data Group/Sverige has a great interview with Tim O'Reilly, CEO of O'Reilly & Associates Inc. From predictions of eBay's purchase of Oracle to discussions of the failings of open source licenses, O'Reilly's certainly not reserved. I couldn't help but be reminded of the rise of this site and slashcode."

Anonymous Noder/LJ'er writes "Linux.com is running a story written by Slashdot's Krow, one of the authors of Slash comparing the LiveJournal site engine to the Everything2 engine. He went over the installs of the two engines and talks a bit about customizing both. I really like both sites so it is interesting to see someone talk about what makes them tick."

bcrowell asks: "I teach Physics at a community college, and a lot of the faculty are trying their hands at teaching internet courses. I'm going through the process of getting approval to offer some of our physics courses with online instruction plus lab, as an alternative to lecture plus lab. (My main motivation is to boost enrollment in some of our higher-level courses, which tend to get canceled if not enough people enroll.) The standard software for this kind of thing seems to be WebCT, but I get the impression that it's proprietary straightjacket-ware. I'd rather go with something open-source, especially since proprietary software seems to come and go, but the best open-source code is forever -- who wants to waste their time building a whole course around the flavor-of-the-month software? I'm particularly curious whether something like Slashcode might work. Most online courses include a requirement that people post a certain number of 'substantial' comments, where 'substantial' is a subjective term to be determined by the instructor. I know some teachers who say when they teach a large online course, they just don't have time to read all the posts, so they end up going by length a lot of the time. Wouldn't moderation by one's peers work better?"

WallyHartshorn asks: "I'm the web coordinator for an agency (1,200 employees) with a web site consisting of roughly 2,500 static HTML pages, plus a few hundred Acrobat files, a dozen CGI scripts, etc. Currently, updates are done manually by a staff of 2 full-time web developers (including me) and 5 non-IT employees who have web page development as about 25% of their job responsibilities. We have been considering purchasing some web content management software, probably something on the lines of RedDot, eMPower, or Microsoft Content Management Server. We've also been considering using Zope or building something ourselves from the ground up. We only have two Perl programmers and nobody knows Python. Given the current budget limitations, we might have more luck getting permission to spend a few months writing our own software than we would getting approval to spend thousands of dollars on a pre-built package. On the other hand, I could also see a "build from the ground up" project turning into a maintenance nightmare. What experiences have people who run web sites of a comparable size had with building their own web content management software versus purchasing one? (Please keep in mind that we are not running a blog, a news site, or a community site, so something like Slash would not work.) Our content consists primarily of reference material and services.)"

cpfeifer asks: "I'm joining a small startup that needs a knowledge management solution. I know that 'knowledge management' is seen an empty buzzword, but after working at a company where the communication is very poor, I see the value of allowing folks to post their own news instead of having it filtered through some sort of corporate newsletter. I've seen the commercial portals (Plumtree & others), but after seeing a couple of OSS publication systems (phpNuke, Slash), I think these would fit the bill quite nicely." Aside from some of the basic features found in weblogs (posting, archival, sorting and searching), what else is necessary for the proper maintenance and use of such a system? How hard would it be to adapt existing weblog-ware to this task?

swfnews guy writes: "swfnews.com (a slashcode based site) today published this article regarding how Overture's search term suggested tool can be used to see the desired piracy of a particular piece of software. I find it disturbing that more people searched for the crack for Flash Mx than for tutorials on how to use it."

Two Slashdot (and Slashcode) readers -- Alex McLintock of Openweb Analysts Ltd, and Dave Aiello of CTDATA -- have graciously contributed their impressions of the new O'Reilly book Running Weblogs With Slash. Since the Slash code is now used to run sites covering a huge number of topics (fly fishing, mountain biking, "news for nerds," etc.), it's time there was at least one printed reference to it. Running Weblogs With Slash author chromatic, Brian Aker, Dave Kreiger pages 262 publisher O'Reilly rating (see each) reviewer Alex McLintock ISBN 0-596-00200-2 summary It's the first book I know about slash: That means it's the best, and the worst.

dave_aiello's review:

I have been running two small-scale Slash sites since 1999. When I began developing web sites using Slash the only code available was Version 0.3, one of the first tarballs ever released by Rob Malda. That code was almost totally undocumented and not at all modular.

The Slash project has grown and the code has improved since then. Slash has now reached the point where it is a feature-laden, open source web publishing platform. However, documentation that can help new users has always been hard to come by.

I found a paragraph near the beginning of the book that I thought was important because it may help you to decide whether this book will be helpful to you:

This book is aimed at anyone interested in setting up and running a weblog with the Slash software. This includes system administrators and programmers, but attempts have been made to keep the discussion readable for people who have no desire to compile their own kernel or to rewrite the moderation system. It concentrates much more on how to accomplish things than how things work underneath, though it doesn't shy away from the greasy gears and wheels when appropriate.

The authors' choice of audience might also be why you have seen comments from some experienced Slash site maintainers indicating that they weren't entirely satisfied with the finished product. That's OK -- it wasn't written for them. But, if you have a desire to build a web site running on Slash and you don't want to spend the months that it took me to read the code line-by-line, "Running Weblogs with Slash" is a good place to start.

The main part of the book begins with an overview chapter. It starts with a brief history of the Slash project. The next ten pages are a review of the user interface of Slashdot, including a number of screenshots. This is followed by three important short sections: The Slash Author Interface (administration), The Slash Publishing Cycle, and The Slash Architecture. These sections do not contain screenshots, but author interface is discussed in detail later in the book, and the publishing cycle and architecture parts have diagrams to help you visualize the concepts being discussed.

The next chapter, Installing Slash, may be the most important one of the book. This chapter is surprisingly short (about 21 pages) considering the difficulty that some people have installing the software. Two of the most critical pages of the chapter cover getting help with the key underlying software: mySQL, Perl, Apache, mod_perl, and Slash itself. All of the important URLs and mailing lists are documented here. A short outline of the installation process, and a set of step-by-step installation instructions follow. There are also short sections on building sites that are distributed across multiple machines, and running multiple virtual Slash sites on the same server.

Although the installation chapter is clearly written, some users may have difficulty completing it without a little frustration. For instance, the book says "Although many UNIX-like operating systems come with Perl already installed, building Apache with mod_perl requires the Perl source." It would help if the detailed installation section of the chapter indicated how to uninstall a binary version of Perl from at least one of the major Linux distributions. The authors have been consistent, however, in focusing this chapter on critical, Slash-specific installation issues.

The next five chapters of the book focus on Slash site administration. The topics covered include editing and updating stories, reviewing and approving submissions, comment-related functionality, moderation, and managing sections and topics. These chapters are important because most people who want to implement a Slash site have never seen the administration interface and need some help understanding the tools that are available. These chapters are also helpful to people who have experience with other web publishing systems, because a quick comparison can be done between the Slash author interface and the system with which the reader is more familiar.

The main part of the book concludes with chapters on managing a Slash-based community, basic and advanced customization, and advanced administration. The customization chapters are the only places in the main part of the book where the Perl code that makes up Slash is discussed in any detail. Now that Slash implements much of the web site's look and feel through the Template Toolkit, modification of the application source is much less important than it used to be. The advanced administration chapter is primarily about the Slash daemon (slashd), the tasks it performs (dailyStuff, moderatord, portald), and how tasks can be added and modified.

The appendices are easily the most technical part of the book. They begin with a detailed discussion of the Slash architecture, which includes several process diagrams, and a discussion of the Slash directory structure. The second appendix discusses the Slash database schema. The appendix on the Slash Template Language looks quite useful to people trying to modify the appearance of their Slash site. The Slash API appendix covers some of the important high-level functions that would be useful in building your own Slash plugin or theme. The last appendix covers Slash configuration variables, which can be important if you have to modify the location of Slash content in your file system, or you need to change other fundamental aspects of your site's configuration.

I would strongly recommend this book to you if you plan to build a Slash-based site and you want to develop a basic understanding of the Slash software as quickly as possible. You will also need to find resources (either in print or on-line) to help you understand administration of the underlying operating system, Perl, Apache, and mySQL, if you have little or no background in these subjects.

I'm sure that many people will be able to successfully install and operate Slash by using this book as their primary reference. And, if they have difficulty, they will have sufficient understanding of the architecture and terminology to ask "good questions" on the mailing lists or the Slashcode web site.

Alex McLintock's review:

Executive Summary:

This review basically lists the flaws and missing features in the book. But that doesn't mean it is a bad book - just that it is easier to find fault than praise. If you need a book on running a weblog with slash then buy this book.

About the Reviewer:

My slash experience is in running a book reviews website http://news.diversebooks.com/ using slashcode. I did set up a site years ago called "mines-a-pint.com" using a pre 1.0 version of slashcode but the site fell over due to lack of spare time.

I am also a perl programmer and web developer so I can figure out a huge variety of problems with Apache, mod_perl, and MySQL. This is lucky because slashcode depends quite greatly on the administrator having those skills. Sadly this book doesn't look into them in great detail. Were this a Wrox book the publishers would have copied a few chapters from different books - or at least given more details on how to troubleshoot those systems. The O'Reilly strategy is that you can perfectly well buy their book on MySQL, their Apache book, and their mod_perl book(s).

Choosing Slash:

I was expecting more of a comparison of Slash to other weblogs. Why chose slash over others? It is in fact a hard question to answer unless you require a site really quite like http://slashdot.org The big benefit of being able to cope with a lot of traffic probably don't apply to most sites just starting out.

The book assumes you have greater than Slash 2.2.0, but presumably quite a few people such as myself have 2.0 installed and not upgraded yet. (I don't know how many people are running pre 1.0 slash sites). It doesn't mention how you upgrade. Lets face it - even if you installed the latest version today (2.2.5) you *will* have to upgrade at some point.

Setting Up Sites:

I expected this section to be a lot more detailed than the basic instructions - but it isn't so much better. When setting up slash sites there are at least three different types of user (slash, unix, and database) and these are not differentiated properly. The authors of the book fall too easily into using slashcode jargon without realising that someone coming to this for the first time wont know the slash-speak. I understand it because I've been reading the mailing list for nearly a year. However, someone picking up this book may not have.

Most jargon terms are introduced but they could be better explained - perhaps with more diagrams. As a typical example, it took me ages to understand the difference between topics and sections. It would have been great to see examples of how different slashcode sites decided what their topics and sections would be. Originally I just saw topics and sections as being a matrix of slots into which one would pigeonhole each article. However there are some things you can do with sections (the columns of the matrix) which you can't do with the topics (the rows). This seemed to me to be an arbitrary limitation of slashcode which an installer needs to be aware of when choosing topics and sections.

More screenshots of the default theme are needed. For instance it talks about the various slashboxes which are configurable but doesn't show them.

Virtual Slash Sites:

I pretended I was a newbie at this and tried to follow their instructions for installing virtual slash sites (ie multiple weblogs on the same machine). The instructions for setting up virtual users aren't complete. EG how do you *add* a new user to DBIx::Password I figured it out because I know perl but others may have significant problems....

Slightly Unusual Config:

I wanted to see if the book would tell me how to put all slash URLs inside a directory of a pre-existing site. (eg http://mysite.com/newsdir/slashpages ) but no found info on doing this at all....

This is a shame because it means that if you wish to combine slashcode with some other html you have to take the approach of installing slash and then adding the extra html content rather than the other way around. It should have considered the situation where someone already has a pre-existing website which they just want to enhance with some news capability.

Running a Slash Site:

There is not much description of the workflow of submitted articles through the system. Instead it is explained by describing what slashcode editors (aka "Authors") do. Looking at the article workflow is important because many other organisations will have different workflows and thus will need to change their behaviour to match Slashcode and not vice versa. This is generally a "bad thing".

Would I Buy This Book?

If I were installing slash for someone and I wasn't being paid to do all the maintenance I would have no hesitation in buying this book to help whoever is doing first line support. However I don't feel a desperate need to keep the book myself since I've already learned most of what it contains just by running a slash site. I know how to read the docs, look at the perl and database, and use the mailing list.

Ok - I can't do entirely without the book because it has some useful reference sections: a full third of the book is made from Appendices and index! One of the most useful of these is the chapter on Andy Wardley's Template Toolkit. I have a bit of an advantage here since I have used Andy's code before in a different web project. However most of my future slash site design will be done by a more junior web developer who doesn't have any perl experience and I expect this will be his most used chapter. (Check back to http://news.DiverseBooks.com in a few days for his additions to this review). The final appendix is useful too -- the list of configuration variables and their meanings.

You can purchase Running Weblogs with Slash from Fatbrain. Want to see your own review here? Just read the book review guidelines, then use Slashdot's handy submission form. (Psst! O'Reilly has also made available a 20% discount for Slashdot readers ordering by phone (800-294-4747), email (order@oreilly.com) or from the O'Reilly web site. Use discount code "#E1EW36."

lisam writes: "The Wiki plugin integrates nicely with Slash and has a lot of cool features, says Chromatic who introduces Wikis and gives a detailed explanation of how and why the Wiki plugin works in this OnLamp article. (chromatic is coauthor of O'Reilly's upcoming Running Weblogs with Slash.)" A lot of people just think of Slashdot style sites when thinking of the Slash codebase, but this article goes on about how to extend slash in cool ways. If you are interested in plugins, the repository is a good place to start.

lisam writes: "The Wiki plugin integrates nicely with Slash and has a lot of cool features, says Chromatic who introduces Wikis and gives a detailed explanation of how and why the Wiki plugin works in this OnLamp article. (chromatic is coauthor of O'Reilly's upcoming Running Weblogs with Slash.)" A lot of people just think of Slashdot style sites when thinking of the Slash codebase, but this article goes on about how to extend slash in cool ways. If you are interested in plugins, the repository is a good place to start.

If you meander over to Slashcode, you will notice that Slash 2.2.0 has been released. This is of course the website engine that runs Slashdot. The release has the message system, improved journal functions, new comment filters, and countless bug fixes. And of course a variety of optimizations that continue to make it possible to serve a quantity of pages that no other open source package like this can even touch :) Plus it's way easier to install. Now that we've got the Fry tree out of the way, its off to work on Zoidberg (which will include subscriptions, killfiles, and a few surprises)

On Tuesday, something terrible happened. The effects rippled through the world. And Slashdot was hit with more traffic than ever before as people grabbed at any open line of communication. When many news sites collapsed under the load, we managed to keep stumbling along. Countless people have asked me questions about how Slashdot handled the gigantic load spike. I'm going to try to answer a few of these questions now. Keep reading if you're interested.

I woke up and it seemed like a normal day. Around 8:30 I got to the office and made a pot of coffee. I hopped on IRC, started rummaging through the submissions bin, and of course, began reading my mail. Within minutes someone told me on IRC what had happened just moments after the impact of the first plane. Just a minute or 2 later, submissions started streaming into the bin. And at 9:12 a.m. Eastern Time, I made the decision to cancel Slashdot's normal daily coverage of "News for Nerds, Stuff that Matters," and instead focus on something more important then anything we had ever covered.

I couldn't get to CNN, and MSBNC loaded only enough to show me my first picture of the tragedy. I posted whatever facts we had: these were coming from random links over the net, and from Howard Stern who syndicates live from NY, even to my town. Over the next hour I updated the story as events happened. I updated when the towers collapsed. And the number of comments exploded as readers expressed their outrage, sadness, and confusion following the tragedy.

Not surprisingly, the load on Slashdot began to swell dramatically. Normally at 9:30 a.m., Slashdot is serving 18-20 pages a second. By 10 we were up to 30 and spiking to 40. This is when we started having problems.

At this point Jamie and Pudge were online and we started trying to sort out what we could do. The database crashed and Jamie went into action bringing it back up. I called Krow: he's on Western time, but he knows the DB best, and I had to wake him up. But worst of all, I had to tell him what had happened in New York. It was one of the strangest things I've ever done: it still hadn't settled in. I had seen a few grainy photos but I don't have a TV in my office and hadn't yet seen any of the footage. After I hung up the phone I almost broke down. It was the first time, but not the last.

The DB problem was a known bug and the decision was made to switch to the backup box. This machine was a replicated mirror of Slashdot, but running a newer version of MySQL. We hadn't switched the live box simply because it meant taking the site down for a few minutes. Well we were down anyway, and the box was a complete replica of the live DB, so we quickly moved.

At this point the DB stopped being a bottleneck, and we started to notice new rate limits on the performance of the 6 web servers themselves. Recently we fixed a glitch with Apache::SizeLimit: Functionally, it kills httpd processes that use more then a certain amount of memory, but the size limit was to low and processes were dying after serving just a few requests. This was complicated by the fact that the first story quickly swelled to more than a thousand comments ... we've tuned our caching to Slashdot's normal traffic: 5000-6000 comments a day, with stories having 200-500 comments. And this was definitely not the normal story. Our cache simply wasn't ready to handle this.

Our httpd processes cache a lot of data: this reduces hits to the database and just generally makes everything better. We turned down the number of httpd processes (From 60 on each machine, to 40) and increased the RAM that each process could use up (From 30 to 40 and later 45 megs) We also turned off reverse hostname lookups which we use for geotargetting ads: The time required to do the rdns is fine under normal load, but under huge loads we need that extra second to keep up with the primary job: spitting out pages as fast as possible.

This was around noon or so. I was keeping a close eye on the DB and we noticed a few queries that were taking a little too long. Jamie went in and switched our search from our own internal search, to hitting Google: Search is a somewhat expensive call on our end right now, and this was necessary just to make sure that we could keep up. We were serving 40-50 pages/second ... twice our usual peak loads of around "Just" 25 pages a second. I drove the 10 minutes to get home so I could watch CNN and keep up better with what was happening.

We trimmed a few minor functions out temporarily just to reduce the number of updates going to frequently read tables. But it was just not enough: The database was now beginning to be overworked and page views were slowing down. The homepage was full of discussions that were 3-4x the average size. The solution was to drop a few boxes from generating dynamic pages to serving static ones.

Let me explain: most people (around 60-70%) view the same content. They read the homepage and the 15 or so stories on the homepage. And they never mess with thresholds and filters and logins. In fact, when we have technical problems, we serve static pages. They don't require any database load, and the apache processes use very little memory. So for the next few hours, we ran with 4 of our boxes serving dynamic pages, and 2 serving static. This meant that 60-70% of people would never notice, and the others would only be affected when they tried to save something ... and then they would only notice if they hit a static box, which would happen only one in 3 times. It's not the ideal solution, but at this point we were serving 60-70 pages a second: 3x our usual traffic, and twice what we designed the system for. We got a lot of good data and found a lot of bottlenecks, so next time something that causes our traffic to triple, we'll be much more prepared.

At the end of the day we had served nearly 3 million pages -- almost twice our previous record of 1.6M, and far more then our daily average of 1.4M. During the peak hours, average page serving time slowed by just 2 seconds per page ... and over 8000 comments were posted in about 12 hours, and 15,000 in 48 hours.

On Wed. we started to put additional web servers into the pool, but that ended up not being necessary. We stayed dynamic and had no real problems on all 6 boxes all day. We peaked at around 35-40 pages/second. We served about 2 million pages. Thursday traffic loads were high, but relatively normal.

Summary So here is what we learned from the experience.

• We have great readers. I had only one single flame emailed to me in 24 hours, and countless notes of thanks and appreciation. We were all frazzled over here and your words of encouragement meant so much. You'll never know.
• Slashteam kicks butt. Jamie, Pudge, Krow, Yazz, Cliff, Michael, Jamie, Timothy, CowboyNeal, you guys all rocked. From collecting links to monitoring servers, to fixing bits of code in real time. It was good seeing the team function together so well ... I can't begin to describe the strangess of seeing 2 seperate discussions in our channel: one about keeping servers working, and another about bombs, terrorists, and war. But through it all these guys each did their part.
• Slash is getting really excellent. With tweaks that we learned from this, I think that our setup will soon be able to handle a quarter million pages an hour. In other words, it should handle 3x Slashdot's usual load, without any additional hardware. And with a more monstrous database, who knows how far it could scale.
• Watch out for Apache::SizeLimit if you are doing Caching.
• Writing and reading to the same innodb MySQL tables can be done since it does row-level locking. But as load increases, it can start being less then desirable.
• A layer of proxy is desirable so we could send static requests to a box tuned for static pages. For a long time now we've known that this was important, but its a tricky task. But it is super necessary for us to increase the size of caches in order to ease DB load and speed up page generation time ... but along with that we need to make sure that pages that don't use those caches don't hog precious apache forks that have them. Currently only images are served seperately, but anonymous homepages, xml, rdf, and many other pages could easily be handled by a stripped down process.

What happened on Tuesday was a terrible tragedy. I'm not a very emotional person but I still keep getting choked up when I see some new heart breaking photo, or a new camera angle, learn some new bit of heart breaking information, or read about something wonderful that somebody has done. This whole thing has shook me like nothing I can remember. But I'm proud of everyone involved with Slashdot for working together to keep a line of communication open for a lot of people during a crisis. I'm not kidding myself by thinking that what we did is as important as participating in the rescue effort, but I think our contribution was still important. And thanks to the countless readers who have written me over the last few days to thank us for providing them with what, for many, was their only source of news during this whole thing. And thanks to the whole team who made it happen. I'm proud of all of you.

Welcome to Slashdot, now running Slashcode 2.2. Since we never upgraded Slashdot to 2.0, this is a huge deal. The changes are numerous, but non-obvious unless you happen to be a fairly frequent user who actually hits most of the functionality on the site. Read on for an executive summary of major features and random props to folks that made it happen.

Ok, lets start with a minor vanity thing that I love. Krow set it up so you can hit unix style homepages to get fellow user info pages. Its just nicer to look at and easier to remember. I love it.

Messaging is in place. If you're logged in, hit your user preferences page and you'll see the option to configure it. You'll see notification on the homepage when you have messages. There are a variety of message types, like notification of replies to your comments, or moderations done to you. You can choose from a couple of different delivery mechanisms currently, right now email and web. Eventually we'll offer other options, but if people submit patches from CVS, "Eventually" will be sooner. Jabber is definitely planned. Inter-User messaging should happen soon too. Also note that the messaging replaces the old headline-mailer. I think you'll enjoy some of the other stuff you see there. All this messaging is the 0th child of Pudge.

Journals are a long awaited feature that people on Slashcode.com really like, so we brought it over here. Everyone contributed bits and pieces to this sucker, and its fairly complex so don't be surprised if there are some bugs. Users can enable or disable comments. Journals also features Friends Lists. This will be extended in the future, but for now you can use it to track a quick list of journals you want to scan regularly. Eventually friends will be bigger, but we'll worry about that after 2.2 settles down.

Discussions can now be created to house your own little chats without bugging us. This replaces the old hidden sids which were bugs to some, and features to others.

There are rss feeds scattered throughout the site now that will need to be documented so people can find them. The user help stuff will be updated soon.

There are several new user options for comment display. I'll let you poke around them yourself and see how they work. Some you'll like, some you won't. You'll notice that the old 'Fake Email Address' is now gone. We decided that it simply was to easy to abuse, so instead we offer auto spam proofing of your real email address. Of course this is purely optional for you paranoid spam-free types. This is Jamie's, as was the new domain linking stuff, and countless other usability features.

Meta Moderation received an overall at the maniacal hands of Cliff. Under previous versions of Slashcode, meta moderations were single autonomous events, and therefore fairly easy to manipulate by malicious users. Now M2's are tallied and when a consensus is reached, judgement is made (and karma is adjusted appropriately).

The 1% of submissions that are accepted will show up on User Info pages now. So now the 99% of you who have your submissions rejected will have something else to complain about to me an average of 15 times a day.

Forgotten passwords can be resent even if you don't remember your nickname. I can't fathom how so many people could forget their nicks, but you sure did. CowboyNeal is dancing in the streets with Mic Jagger and David Bowie knowing that he will have to respond to half as many people who can't figure out how to login. Also, plain text passwords aren't sent in emails any more, so while we're still not pretending Slashdot is secure, its more secure then it was.

A huge new deal is the fact that all the old articles have been re-imported back into the database. 2 million of them. Old discussions will continue to be closed out as they age, but users will still be able to adjust order and threshold when they view them. This was a really nasty task for Cliff since he had to import nearly 4 years of HTML back into the database... including a dozen different comment formats that evolved over that time. A huge task, but one that he managed to pull off by the deadline. And I think any users who read older stories will appreciate it.

CaptTofu fixed tons of security issues and greatly improved a lot of the security stuff that most of you will never notice. But believe me it makes dealing with bots and denial of service attacks a lot easier now (for example, if a robot is hitting us as they so often do, we can ban them without restart httpd). Tofu also had to "Manage" the team, which makes cat herding seem like a party.

We have some new hardware too. Replacing our 6 600mhz web servers are 5 shiny new dual ghz boxes. OSDNs Netops staff mercilessly beat these things into submission. Thanks to Yazz, Kurt, Toad and Trish. And also props go out to Uriah on the West Si-eed for helping with some raid bios problems that messed with our database stuff and really though us for a loop.

We also gotta thank the guys at SourceForge for having a functional bug tracking system that we (and the trolls *sigh*) could use. We love you all, and we shouldn't let little things like a language barrier stand between us. Lets both port to ML. Its like switzerland for web pages!

I also wanna thank Hemos for being our man on the inside, and Richard French, OSDN boss-man for giving us the room to put this all together.

I know I forgot to thank someone, and I apologize, but its past midnight and I've been going for 16 hours today.

Allright this is only scratching the surface. There are so many more little things that you'll notice, and either submit thinking they are bugs, or hate, or love, or whatever. But we're not fooling ourselves by thinking that we're out of the woods yet. There are still several dozen significant bugs that need fixing so we're gonna keep going. And then we have to update the FAQ with all new FAQs and corrections to the old FAQs! The real test will be when loads pick up. We expect that there will continue to be brief periods of trouble for the next couple days, but it shouldn't be significant since we did a lot of load testing, and managed to get the new setup to handle 2x Slashdot's normal peak load. We think things will be all right, but thanks to you all for having patience.

Now holy shit, I'm gonna sleep.

After far too long in development, Slashdot is ready to move to the latest version of Slashcode. To this end, we really need people to test our development box, aka Banjo. This is not a load test, so please don't beat the crap out of it (yet!). Please read comments, post comments, and test out wacky new features like Journaling and the assorted new messaging options. It is mostly identical but there are several cool surprises. We have many optimizations still to add, and only have a fraction of our "real" hardware on Banjo so don't worry about speed, instead look for things that don't work right (or at all!) and send those bug reports. (Known issues: imported comments are misattributed, and comment searching is disabled. Don't report these things please ;))

Rainstorm writes "MSNBC has a story on Slashdot, with a good bit of history on the site and its own interpretation of the goals of the site. It also references some possible outcomes of the recent VA Linux announcement." Is anyone besides me sick of reading about Slashdot? I feel like they write them just to get us to link them, and we almost have to link them because hundreds of readers submit them thinking that I care that yet another web site is blabbing about Slashdot. Anyway the story covers a lot, although I'm not at all worried about "The Future of Slashdot". OSDN continues to be very supportive, and the Slashteam is almost ready to move Slashdot to the new code tree with all sorts of wacky new stuff (bug fixes, messaging, journals, and more) Oh, and their poll doesn't work under Konqueror, so I reposted it here cuz darn it, I'm curious ;)

dam_ned writes: "O'reilly has put a nice introduction to mod_mp3 online. Mod_mp3 is an apache module that offers shoutcast/icecast like mp3 and Ogg Vorbis streaming, and allows for playlists via frontends like ampache. The project is only a few months old, but it's quite stable, and has far more features than shoutcast/icecast." Heh. You might recognize Brian Aker, aka "krow", from his work on Slash.

flikx sent in an issue that may affect several of us in the future, especially if you happen to work in a similar environment. It's a free speech issue, and one that puts a huge question mark on just how far U.S. First Amendment rights really extend. The short version: filkx ran a weblog called "SOS," for the laudable goal of providing an open discussion forum for student government issues. He had a steadfast rule of non-censorship, which soon landed him in political hot waters with the University of Utah. A new administration steps in, the site is shut down, and filkx now faces criminal charges, expulsion and a cute twist: the university claims that the site's content is now their own property. flikx notes, "I'd like to say up front that this is a fairly large and multifaceted issue, and would also like to draw attention to a related article up on Slashcode". You can read the details by clicking on the link.

The following is written by former-Slashsite admin, flikx:

Early last fall, I personally created a Slash site called SOS under student government for my University. I did everything myself with no outside help of any sort, and entirely volunteer. I have never been paid for my efforts in any way, and never expected monetary compensation. I did not actually work for the University, but was a student in Mechanical Engineering. I created said site with the ultimate goal of providing an open discussion forum and weblog for all students of the University.

During the course of operation of the site, I fielded numerous complaints about abuse to the site, and took them in a professional manner, though steadfastly refused to censor any content on the site. Remember that many in Utah are very conservative and dislike free speech on some levels...being fairly conservative myself, I never thougtht I'd run afoul of people. The problem is that I continuously ran afoul of politics as people threatened me repeatedly due to my failure to censor the site. The abuse was minimal, especially by Slashdot standards.

Six weeks ago, the administration censored the entire site due to the threat of legal action due to inapropriate content. The site was down for just over a week, and I was forced to implement strict posting guidelines and adopt a censorship policy for the site.

A new administration recently took over, and first on the list was to get rid of me, and the site. There's much more involved here in politics, plus scr1pt k1dd13 threats spoofed from my email and everything, but the bottom line is that the site was censored for good. The server was physically removed by the police, and the disks wiped after 'evidence' was removed. All known backups were destroyed, and they even obtained a protective order from me and banned me from the University property. I'm also suspended indefinately, and face immediate expulsion from the University. (BTW - I'm almost done with my Mechanical Engineering degree .. so this is not light by any means. If expelled, I'm forced to start over as a freshmen if I ever get into another school.)

So here's some of the problems with which I turn to the Slashdot crowd for a solution:

The administration threatened me, and had the legal team tell me that everything on the site is intellectual property of the University of Utah. Everything. That includes all stories, all comments, user accounts, even the graphic design I did. I have off-site backups of the site, and could easily redeploy the site elsewhere provided the time and hosting. I've already put 2000+ unpaid volunteer hours into the University, and they take away my work. It should be my right to operate an open discussion forum, but it seems that it's not.

What does the Slashdot crowd think about this issue? Should [or does] everything belong to my University? The only involvement the university had was hosting the site and buying the server, that's it. Obviously, the site could be moved elsewhere, and I still have a team together that could operate the site independant to the University. But as I am already being expelled and even facing criminal charges for 'computer crimes', this is far beyond your average Ask Slashdot."

Cliff: If you are interested in obtaining some context for this story, you can dig around the cached pages from SOS on Google.

flikx sent in an issue that may affect several of us in the future, especially if you happen to work in a similar environment. It's a free speech issue, and one that puts a huge question mark on just how far U.S. First Amendment rights really extend. The short version: filkx ran a weblog called "SOS," for the laudable goal of providing an open discussion forum for student government issues. He had a steadfast rule of non-censorship, which soon landed him in political hot waters with the University of Utah. A new administration steps in, the site is shut down, and filkx now faces criminal charges, expulsion and a cute twist: the university claims that the site's content is now their own property. flikx notes, "I'd like to say up front that this is a fairly large and multifaceted issue, and would also like to draw attention to a related article up on Slashcode". You can read the details by clicking on the link.

The following is written by former-Slashsite admin, flikx:

Early last fall, I personally created a Slash site called SOS under student government for my University. I did everything myself with no outside help of any sort, and entirely volunteer. I have never been paid for my efforts in any way, and never expected monetary compensation. I did not actually work for the University, but was a student in Mechanical Engineering. I created said site with the ultimate goal of providing an open discussion forum and weblog for all students of the University.

During the course of operation of the site, I fielded numerous complaints about abuse to the site, and took them in a professional manner, though steadfastly refused to censor any content on the site. Remember that many in Utah are very conservative and dislike free speech on some levels...being fairly conservative myself, I never thougtht I'd run afoul of people. The problem is that I continuously ran afoul of politics as people threatened me repeatedly due to my failure to censor the site. The abuse was minimal, especially by Slashdot standards.

Six weeks ago, the administration censored the entire site due to the threat of legal action due to inapropriate content. The site was down for just over a week, and I was forced to implement strict posting guidelines and adopt a censorship policy for the site.

A new administration recently took over, and first on the list was to get rid of me, and the site. There's much more involved here in politics, plus scr1pt k1dd13 threats spoofed from my email and everything, but the bottom line is that the site was censored for good. The server was physically removed by the police, and the disks wiped after 'evidence' was removed. All known backups were destroyed, and they even obtained a protective order from me and banned me from the University property. I'm also suspended indefinately, and face immediate expulsion from the University. (BTW - I'm almost done with my Mechanical Engineering degree .. so this is not light by any means. If expelled, I'm forced to start over as a freshmen if I ever get into another school.)

So here's some of the problems with which I turn to the Slashdot crowd for a solution:

The administration threatened me, and had the legal team tell me that everything on the site is intellectual property of the University of Utah. Everything. That includes all stories, all comments, user accounts, even the graphic design I did. I have off-site backups of the site, and could easily redeploy the site elsewhere provided the time and hosting. I've already put 2000+ unpaid volunteer hours into the University, and they take away my work. It should be my right to operate an open discussion forum, but it seems that it's not.

What does the Slashdot crowd think about this issue? Should [or does] everything belong to my University? The only involvement the university had was hosting the site and buying the server, that's it. Obviously, the site could be moved elsewhere, and I still have a team together that could operate the site independant to the University. But as I am already being expelled and even facing criminal charges for 'computer crimes', this is far beyond your average Ask Slashdot."

Cliff: If you are interested in obtaining some context for this story, you can dig around the cached pages from SOS on Google.

More updates and links below on HavenCo, robots that kick balls, and what can already be said to be one of the millenium's most anticipated movies thus far. Oh, and some nice auditory/textual backup to the recent story about RMS vs. Goliath.

Not asking, not telling. jeffthompson writes: "The Havenco web site says it is now fully operational and open for business." A lot of people seemed convinced that Havenco wouldn't even be around by this time, but sticking around is the best revenge. I'd like to go aboard one day, promise I'll wear a blindfold and not look at anything ...

GNU, Linux, GNU/Linux, Freedom and the American Way. bkuhn writes: "Now, both the electronic audio and transcript of RMS' NYU talk are available."

The audio is in Ogg format, and the transcription is in blessed plaintext. Thanks!

Sign #37 of the coming apocalypse: Speaking of Mundie, software Freedom (and free-ness), Simone Paddock of O'Reilly writes with news that might raise a few eyebrows:

"Microsoft Senior Vice President Craig Mundie set off a compelling debate recently when he discussed Microsoft's Shared Source Philosophy, which blends the sharing of source code with the preservation of intellectual property rights.

Tim O'Reilly invited him to attend the upcoming O'Reilly Open Source Convention (July 23-27, 2001 in San Diego). Mundie not only agreed to attend, he agreed to speak.

Mundie will discuss ways in which shared source differs from open source, and how the Shared Source Philosophy supports a strong software business case for commercial software use."

Michael Tiemann (CTO of Red Hat) will speak after Mundie, and a panel of IP law and software experts (including Tim O'Reilly) will discuss the issues raised. Sounds worth being in San Diego for. If you're interested, there's more information online.

In the Tolkein, not the endocrinological or Snow White sense. SomeoneYouDontKnow writes: "This is a follow-up on the recently-released LOTR trailer. It's now available for download. Two versions are available to suit your bandwidth and patience. Unfortunately, it's still only available in Real format, but I guess we can't have everything."

Semi-alive and kicking. IEEE Spectrum Associate Editor Stephen Cass writes:

IEEE Spectrum , the house magazine of the IEEE is launching an online forum devoted to the noble sport of robosoccer . Robosoccer is different from things like Battle Bots or Robot Wars in that the robots play in teams and the whole thing is completely autonomous once started. There are a number of competitions, the biggest of which is the annual Robocup tournament, which will be held in Seattle this year. Robosoccer is a great reasearch tool for exploring A.I., automous agent behaviour, computer vision, simulation and mechanical and electrical design. It attracts participants ranging from high school students to academic researchers.

Our website (which incidentally runs slashcode) will also be a clearing house for us to award sponsorship money for teams building robosoccer robots as well as a place to exchange information.

Hard to get enough of Robots playing soccer, and prize money means you can buy more marshmallows to roast at your victory bonfire.

After far to much time in development, Slash 2.0.0, previously known as Bender is now released unto the cruel dark world. This code is of course the source that runs Slashdot, however 2.0 is far more advanced then the code you see here, to say nothing of utterly embarassing any other weblog software available for free, and written by us, and codenamed after a cartoon. Plans are already well underway to move Slashdot to the new code base soon enough. The features and fixes are to numerous to mention here but besides scouring the codebase of the evils that I originally devised, it installs easier, is more flexible and customizable, and has a variety of improvements for users and administrators alike. Props have to go out to CaptTofu, Pudge, Krow, Cliff, Jamie and CowboyNeal who all contributed to this code, and also to OSDN who let us all do it.

Update

In response to several comments mentioned so far:

Yes, the abstraction layer is in place for supporting any DB. Only the MySQL has been really tested properly by us, but others say the Postgres version is good, and an Oracle version has been floating around for awhile. Adding other databases is relatively easy. Nobody has any idea which will be fastest.

No there is no built in spell check. It also does not automatically correct my habit of switching to/too then/than around (yes, I know the rules. I just don't care enough to backspace and fix it. Get spell checked news from CNN)

Codenames exist because they're fun. What fun is saying "I'm working on 1.2.7 when you can say "Bite my Shiny Metal Ass".

System requirements are still going to be reasonably high since this is a system designed for a million page views, not the 7 page views that most weblogs get. But if properly configured, it will run on very minimal hardware. Back in the day remember that this code served 800,000 page views a day on a single dual p2 w/ 500 megs of RAM. Today thats practically entry level for a real server.

Some minor moderation changes exist, but most of those will be going into Fry (2.2, the codename for the next release, the version that Slashdot itself will run). The 2.0 release was about giving us a stable platform upon which to build new features, not about significantly altering anything functionally. There are changes, but they aren't huge.

If someone wants a Slashdot Redesign, submit one. If I like it, I'll switch. But believe me, I'm picky. Slash itself is very flexible and can look like pretty much anything you want as many other slash sites on the net have demonstrated.

Slash will have more support for various XML DTDs as they come along, and depending on if we find them interesting or not.

Slash will do just fine on virtual hosts now thanks to clever work by Krow.

I've never cared for the term "blog," but gee, I can't really argue with the idea of a siteful of short newsy items updated several times a day. And everyone seems to agree Blogger is a great way to run your own such site, much better than the crappy alternatives. As Dan Bricklin says, "I didn't like the idea of Blogger being lost in the dotCom crash. Personal web sites were growing and important." So he got his company Trellix to license it. Go check out his account "How The Blogger Deal Happened" (and/or read the press release).

After several years of reader requests, we finally cornered CowboyNeal long enough to do a Slashdot interview. Questions were posted last week. Today we brush aside the mask (or at least the hat) and get a glimpse of the real Jon Pater (aka CowboyNeal).

1) Karma?
by glowingspleen

Since you're probably the only one with real access to all the user records, you're the one to ask:

Who among us is the current Karma Whore King, and what is their score? Is there a maximum amount of karma one can earn?brAnd finally, is there a cutoff level where you auto-post at 3 or above?

CowboyNeal:

Wow, a question about karma, what I consider to be the most boring subject in all of Slashdot. I dunno who is the Karma Whore King, since we instituted a karma cap at 50, and all karma levels that are above 50 are living on borrowed time. Getting a karma above 20 allows one to post with a +1 bonus, and from there on there are no rewards.

I said that karma bores me, and I don't understand why people get all hung up over it. It's not like having a high karma is gonna get you discounts in stores. If you're really worried about karma, go donate money to charity or donate your time to some place that can use it. I guarantee the reward will be better than anything that Slashdot karma can get you.

2) Give us the ups and downs!
by Dino

What's the best thing that happened to you since Slashdot started? Conversly, what's the worst?

CowboyNeal:

Easily, the best part was when I got a real job out of it. I had done a few odds and ends for BSI and Slashdot before coming on full time, but getting a check on a regular basis is something I still am thankful for. The worst part is probably the sheer volume of email that I get to deal with on a daily basis. That's not to say that I dislike the people that email me, but it gets troublesome when one has an inbox that can take up to half a day daily to get squared away.

3) A User Info slashbox?
by update()

Like a lot of people, I bet, I frequently look at my user info page to see if there have been responses to my posts, and what moderations I have received. Would it be possible to provide a Slashbox with that information on the main page?

CowboyNeal:

This kind of thing could be implemented, but with our current setup it just wouldn't scale. It would need to add another query to viewing the home page, for each person that has it chosen. There might be a few tricks to implement this that could save some overhead, but if I were to make such a box, it definitely wouldn't happen until after Slashdot is running bender, the new development branch of Slashcode.

4) Slashdot
by emmons

How did you become tangled up in this Slashdot thing with that CmdrTaco guy? How did you guys meet?

CowboyNeal:

I met both Rob and Jeff when all of us where freshmen in college. Rob was in the same Health Dynamics (read, Phys. Ed.) class I was, and Jeff and I shared a lab bench in Chemistry Lab. The CS department at Hope wasn't that large, and most CS majors were aware of Chips & Dips, and would read it regularly. When I graduated, I was approached by Blockstackers, who owned Slashdot at the time, and offered a job. About two months later, Slashdot was acquired by Andover.Net (now OSDN) and my employment was transferred to them.

5) Yes!
by OlympicSponsor

I've been following CowboyNeal's career for years, reading everything he writes, watching all of his movies and eating all of his cooking. Now I finally have a chance to ask him the question that's been burning me up inside: What's Taco really like?

Seriously, can you give us a breakdown of how much time each editor spends actually reading the site they nominally run? Like, time spent clicking on user comments?

CowboyNeal:

Hrm, it sounds like you might have me confused with someone else. I've never made any movies or had a restaurant *grin*. Honestly, I don't know how much time an editor spends doing those tasks. I would assume it varies from person to person, but I don't share office space with hardly any of them, nor do any of them ever have to report to me.

I know from our internal discussion lists and channels, that all of us are constantly reading the site, but as for a detailed breakdown, I'd have no idea where to start.

6) The future
by yamla

What will you be doing in five years? In ten years? If you expect/hope to still be with Slashdot, what sort of changes do you see happening to Slashdot in that time?

And yes, I know 5 - 10 years is an eternity.

CowboyNeal:

5 - 10 years is an eternity! I don't even know what I'm doing two weeks from now!

I wish I knew. I was the kid in school who when asked what he wanted to be when he grew up, didn't have the slightest clue. When I finally discovered computers, I knew that I wanted to work with computers in some capacity for a living. When I think about how much technology has advanced since I first started using a computer, it's truly amazing how far we've come. I'm hoping that someday we can put all this patent/copyright/intellectual property BS behind us, and by that time someone will have discovered the secret to putting infinite bandwidth into everyone's homes and with all of these new resources at our disposal, someone will discover the new killer app for all of it, repeat ad infinitum. That's how the system works. I just hope to be there and be a part of the process somehow. I think it's especially interesting how the internet and all of what it entails has made it easier for people to communicate and become exposed to experiences and views they normally wouldn't, so any new advances will hopefully only improve that.

I don't think Slashdot will do anything like try to take over all forms of media or any sort of grand scheme like that because there's already organizations into that sort of thing. I do think, however, that if a new way of delivering content that replaces or augments the web comes about, that we will most likely be there.

This is probably the best question I've been asked, because it was easily the toughest for me to answer. My job now is primarily the day-to-day maintenance of Slashdot, which means that I'm not usually concerned with looking to the future, but rather just making sure that our current system runs smoothly. In short, I'm not a leader but a follower.

7) What was your fav poll abuse?
by chabotc

What poll in which you were an 'option', did you find most amusing/entertaining?

What was the most depressing?

CowboyNeal:

The first few times I was a poll option I thought it was funny, but I have become rather immune to it now. Now it's to the point where I'm surprised when I am not an option. As for trying to pick out particular polls that amused or depressed me, I can't really remember any.

8) tell us the truth
by segmond

How often have you posted anonymously? do you have a pseudo handle? Have you ever trolled? Are you getting bored with slashdot? What is the biggest screw up that you did to the site that we never found out? Have you ever used "super moderating power" to mod down posts attacking slashdot editors...? What is your favorite pizza topping and sandwich?

CowboyNeal:

Nope. Nope. Nope. Nope.

That should bring me up to the "screw up" question, so I will now relate a story that showcases my stupidity. One day I was working on Slashdot's database, and cleaning up a lot of old database entries, when I suddenly realized I'd forgotten a "WHERE" clause in one of the SQL statements.

Unfortunately, this was after the statement had already run, and the blocks table was completely wiped out. If you know anything about Slashcode, you know that blocks holds everything from color schemes, to the slashboxes, the HTML that gives the site its look and feel, to all the headlines that go into the slashboxes on the right side of the homepage. Of course, this brought the site to a halt until we could restore from backup. I was able to berate myself before anyone else could, though, so anyone in the know about my mistake was fairly forgiving.

As for "super moderating power", I know I could go into the database and moderate like crazy, but my ethics won't allow me to do so. Also, I don't usually read comments attached to stories. I have banned IP addresses from which people have been hammering on the site with scripts at the rate of several requests a second, but I don't actively moderate any stories.

I don't know if I have a favorite pizza topping. I used to work at a pizza place when I was in high school, and learned to like almost every pizza topping there is, even anchovies.

My favorite sandwich is easily a veggie submarine. I'm not a vegetarian, but I find myself ordering veggie subs more than any other sandwich.

9) Anime
by spudwiser

Seeing as in most Geeks in Space episodes Anime quotes and quips spew forth from you, I think we the listeners (and deranged readers) should see exactly how your background in anime developed.

CowboyNeal:

Well, this answer is pretty boring. One day in college my friend and I decided we should watch some anime, because we had heard it was cool. So we took a week, and rented a movie or two per night, starting with Ghost In The Shell, then moving on to Akira, Fist Of The North Star, and Vampire Hunter D. I think those are the anime titles everyone starts with, because they're available at almost any Blockbuster.

But yeah, we discovered we liked it, and I just started watching more anime. I warned that this answer was boring.

10) Stories
by Fervent

Why don't you yourself ever post any stories, Cowboy Neal?

CowboyNeal:

Oh, but I do. On the rare occasion that we get to record an episode of Geeks In Space I'll post the announcement in the radio section. Sometimes a story falls through the cracks and I'll pick it up, also, but that happens much less often lately. Rob and Jeff like to say that the surefire way to not get a story posted is to email it directly to them instead of using the submissions bin, but if you want to absolutely sure your story doesn't get posted to the site, email me it to me instead. *grin*

An anonymous reader writes: "Tumbleweed, 'secure' email via http with SSL, so, not really end-to-end secure, just got the patent on private URLs; "Tumbleweed IME generates a private URL for each secure delivery. The private URL that IME creates is unique, tied to the sender of the package or transaction, to the content being sent, and to the intended recipient." I guess I can't tell my buds to surf to a non-public directory on my website to download stuff anymore." Many web applications generate these private URLs. Like the cheesy insecure bookmarkable login URL that Slash uses for example (which is just your name and password plain text in the URL which you should never use unless you're on acid, lazy, and/or realize that losing your slashdot user account will in no way affect your life because you are not a moron and use a unique password so bring on the packet sniffers ;)

We've been at LinuxWorld for the last couple days, and some interesting stuff has been going on: The SAMBA folks won the $25,000 IDG/Linus Torvalds award, and SGI announced the availability of RH7-based distro using XFS ^[?] . In other news, our BOF went well with many questions about Slashcode - and the Perl Monks booth has been doing great in donations. Update: 02/01 05:18 PM by CT : The highlight for me so far was judging the "Coveted" Golden Penguin Awards w/ Don. Actually, I seriously did covet the award, beautiful hand blown glass penguin made me wish I was a contestant. We judged that Linus got the definition of BogoMIPS wrong. Fortunately his still won, but it was truly joyous seeing the surprise on his face.

So, ever had a question you wanted to ask the various folks behind Slash and/or Slashdot? A healthy assortment of Slashdot coders and authors (like krow, cliff, roblimo, jellicle and timothy) will be on hand for a BOF session at the Linux World Expo on Wednesday, January 31, 2001, Room 1E11, 6:00 p.m.-7:30 p.m. So, if you happen to be around, feel free to join us and have your questions answered. And of course, the Slash and Slashdot folks will also be at random times in the OSDN booth on the show floor, so please stop by. Hope to see you there!Update: 01/15 12:55 PM by H : Rob and I won't be able to make itto the BOF -- outstanding plans -- but you can catch us on February 15th at the O'Reilly P2P conference in San Fransico. We speak at something like 11:30 AM or so, on a panel - but we'll be at LWCE as well.

Masem sent in a link to a NYTimes story (free blah blah required. Why is the Times so lame that they don't realize that hundreds of people are registered with my address?) on self adaptive websites. It talks about us, Everything2 (which IMHO is among the best examples of the genre out there, but since I helped create it, I'm biased ;) and of course the recently announced (and Slashcode Based) suck/feed Slashclone, Plastic. I found at least one mistake, but besides that, its not a bad piece, although it probably isn't saying anything that a regular Slashdot reader doesn't know already.

Slashcode 2.0 ("Bender") is officially in beta. We now have themes, plugins, an abstacted database layer (MySQL support is beta, PostreSQL is alpha, so finally the rivalry can be settled ;) a journal system for users, a spiffy template language, better mod_perl usage, ways now for other languages to talk to our authentication layer, and oh so much more. Best of all, the code has been massively scrubbed and de-Taco-spaghettified(TM). You can find a copy on the ftp server. Hopefully the beta phase will be fairly short, as once its over, we'll get to move Slashdot, and have a clean codebase to which we can add all those features we've been wanting for so long. Thanks to Tofu, Krow, CowboyNeal, and Pudge for the toil. And good luck to everyone out there interested in poking at it: this version will give you much more joy then the last one *grin*

grovertime asks: "As a screenwriter-cum-Web developer, I have a pet project to create a Slashdot for the entertainment industry. I have been playing around with the Venezuelan version of Slashcode, namely PHP-Nuke. As it is PHP-based rather than Perl-based, it cannot support frames within my Dreamweaver-built site. I would like to merge the two so as to extend the capability of the site to containing streaming audio and video, etc. Has anyone wrapped PHP (particularly Nuke - or even Slashcode) with a Dreamweaver constructed multi-media site?"

dfelznicasks: "Hello, I got the bad news from Covad to day: no DSL. I had plans on running a Slash site about political activism, anize.org. So now I am falling back to Web hosting services. I checked on Slashcode but HREF="http://www.slashosting.com">SlashHosting was a little pricey did not offer any mailing lists and was based on FreeBSD. I do not want to start a flamewar but I am comfortable in Linux and plan to move it to a dedicated Linux box within a year so I want to stay consistent. My question is does any one know of a good, reliable, cheap Linux based Webhosting service that is Slash code friendly?" I note that the submittor doesn't mention the SlashHost service which is listed above SlashHosting in the grand-mal list of Slash related sites, but it seems that page has gone missing (it's now a page about Free Academy Training). Are these services the only two players in town or are there other ISPs out there that are, if not Slash friendly, at least not Slash unfriendly.

dfelznicasks: "Hello, I got the bad news from Covad to day: no DSL. I had plans on running a Slash site about political activism, anize.org. So now I am falling back to Web hosting services. I checked on Slashcode but HREF="http://www.slashosting.com">SlashHosting was a little pricey did not offer any mailing lists and was based on FreeBSD. I do not want to start a flamewar but I am comfortable in Linux and plan to move it to a dedicated Linux box within a year so I want to stay consistent. My question is does any one know of a good, reliable, cheap Linux based Webhosting service that is Slash code friendly?" I note that the submittor doesn't mention the SlashHost service which is listed above SlashHosting in the grand-mal list of Slash related sites, but it seems that page has gone missing (it's now a page about Free Academy Training). Are these services the only two players in town or are there other ISPs out there that are, if not Slash friendly, at least not Slash unfriendly.

This has been submitted to slashcode a couple of times now. It has appeared on the Slash mailing lists and from talking to Bryan at ApacheToolbox he has noticed it too. When compiling mod_perl and PHP 4 together, you end up with a core file for an Apache process. Is there a solution for making the two of them play nicely together?

This has been submitted to slashcode a couple of times now. It has appeared on the Slash mailing lists and from talking to Bryan at ApacheToolbox he has noticed it too. When compiling mod_perl and PHP 4 together, you end up with a core file for an Apache process. Is there a solution for making the two of them play nicely together?

ffejbean noted that OSXTalk (hey, they run Slashcode!) has an article up noting that XFree86 and MacOS X are getting more and more friendly every day. Now you don't have to purchase a lame commercial binary, you can just install it yourself. If only those iCubes didn't cost twice what they should, this may just be a great platform yet. (BTW, I'm getting confused here, should I post this as Apple, X, or BSD? Ah well, close enough :)

At 9 pm eastern, Hemos & I are gonna be in #forum on irc.slashnet.org to answer questions about Slashdot. There's a lot of random stuff that's happened since the last time we did this, so this is a chance to ask questions about Story Selection, moderation, Slashcode, or whatever else is on your mind. If anyone wants to take our answers and send in updates and additions to the FAQ afterwards, that would make our lives easier, and our inboxes smaller ;)

Chris Nandor wrote in to tell us that Larry Wall has announced his vision for perl 6 as part of this keynote at the O'Reilly Open Source Convention. You can read an announcement at Perl.org or read Chris's summary of things (like information about the from-scratch rewrite being planned!)

The following was written by Chris "Pudge" Nandor... Perl Guru, Slashcode Guru, and all around swell guy.

Perl 6 To Be Complete Rewrite (But Not What You Think)

Larry Wall and other active Perl porters and Perl helpers met on Tuesday afternoon at Perl Conference 4.0 and mapped out a what is planned to become a complete rewrite of Perl that will become Perl 6 in 18 to 24 months, with a prerelease targeted for next year's conference.

Perl 5 will not be abandoned, but will primarily be concerned with bugfixes both major and minor.

The meeting for members of the perl5-porters mailing list was the result of an earlier, smaller meeting of Wall, Nathan Torkington, Chip Salzenberg, and others who basically decided that Perl needed to be fixed in certain ways, and that a rewrite was the best way to do it. Salzenberg started the Topaz project two years ago, to reimplement Perl in C++. Though Topaz itself will not be the basis for Perl 6, Salzenberg noted that the lessons learned in the experience will be very helpful to the new effort.

Torkington led the three-hour meeting, starting off by saying what was wrong with Perl. Much of the focus on the problems with Perl centered around how increasingly difficult it was to improve, extend, and embed Perl. A rewrite and redesign are needed, he said, and maybe it is time for a hard change. So while the focus of the effort seems to be on improving the Perl guts and API, the project will also be used as an opportunity to clean out some of the cruft, including bad and seldom-used features.

Some of the primary (and still vague) goals of the effort will be to reimplement the core so it is better, stronger, and faster; improve syntax; add new features where appropriate; have better version and installation management for perl and its modules; and have a clear and automated migration path, which may include a backward compatibility mode. Some old features may be removed, like typeglobs. Others will be improved.

The group hopes to re-shape Perl community, too. Instead of one all-encompassing perl5-porters list, tightly focused mailing lists with a terminal lifespan will be formed. To start off, the mailing list bootstrap at perl.org will be for discussion of the beginnings of the project. Like most, if not all, other new lists, when it has fulfilled its purpose, it will be closed.

Wall said, "Perl 5 was my rewrite of perl ... I think this should be the community's rewrite of perl, and the community's rewrite of the community."

Specifics have not yet been ironed out. A group has been formed to begin the work, which will primarily consist of planning the work to be done. No coding is to be done at this stage, only planning and support. Roles were determined for the group, and then they were filled. They now include the perl 5 maintainer (Jarkko Hietaniemi and Nick Ing-Simmons), the language design pumpking (Larry Wall), the internals design pumpking (Dan Sugalski), the documentation manager (Adam Turoff), the system administrator (Ask Bjoern Hansen), the quality assurance bloke (Michael Schwern), the spokesdroid (brian d foy), the customer relations person (Dick Hardt), and the project manager himself, Nathan Torkington. Other porters, such as Chip Salzenberg, volunteered to consult when needed.

Some of the short term goals of this group are to draft a language specification, start an RFC channel, get feedback, and set up mailing lists, a documentation repository, and a web site. Hansen will be setting up the mailing lists. The bootstrap list will be open to the public, while the list for the intial small group of people will be closed for posting by anyone not on the list. All lists will be readable by the public, through the web, and perhaps through email. Turoff is going to be preparing the notes for the morning meeting, and foy will be posting the notes and the mailing list links on the www.perl.org web site.

eries wrote a good summary of a major problem with the GPL, with regards to web applications. Essentially, since the GPL is about 'Binary Redistrubtion', it doesn't work with web pages where the output is distributed, but the software itself doesn't have to be. Anyway, hit the link to read his comment about it. Definitely an issue that I've thought a lot about wrt Slashcode, but never found a clear solution.

eries writes "We recently ran into some licensing problems for a web-based open-source project. Since our project is an OO package written in PHP (a scripting language) the GPL does not quite apply. The GPL is designed to handle "binary redistribution" for software packages, but ours cannot be distributed in binary form. That means that someone could make a website out of a derivitive work and never have to open or disclose their code - an apparent violation of the spirit of the GPL. Now, we think it might be possible to modify the GPL to construe instances of a website sending HTML to be "executed" on a browser as redistribution (more on this later), but we don't really know how to go about it. Consequently, we are forced to try and find another license. We have looked at the Artistic and the APSL, but these are both pretty dubious. Has anyone else run into problems like this? We want to write to both the OSI and FSF and bring this to their attention, so we'd like to find out of others have run into the same situation. You can also check out this related discussion on the Sourceforge forums page.

IMHO, we need a way of defining, in cases like these, what constitutes a derived work. Clearly, things like your db password and content like graphics, logos, layout, etc. are not derived works. But how can we draw the line in the case of added functionality? How is this handled in more traditional content-delivery systems?

Is the HTML produced by our scripts just output? The more I think about it, the more I am convinced that the application that a user is running when they use our software is not just a series of HTML files. Those files, by themselves, are worthless. Instead, their usefulness depends on an awful lot of "state" - the state of the browser (cookies, javascript), the web server, other scripts which the browser has to reconnect to, and the data in the database. The HTML files are just one component of the general state that our app produces. If you construe this more nebuloous "state" as the "binary" application that our software produces, then distributing it to a user via a web server is a non-sourced distirbution, and hence forbidden by the GPL. What do you think? I have to believe that someone running a mod_x application within an Apache server that gives access to it to tons of users is redistributing it without the source, unless you believe that clicking on "View Source" for this /. page you have access to the Slash source..."

nellardo asks: "I know that, these days, most web sites are running on Apache and many are running on Linux or some flavor of BSD as well. But what are the biggest sites that go beyond that? We've talked about open-source application servers and the like here before, but where are these being used? Why do I ask? Well, duh, I'm trying to convince management :-) Seriously, CMP uses Mason and Slashdot uses Slashcode, but what are some of the other large ones?"

Recently WAP ^[?] has come under serious criticism from a wide variety of places... Angus wrote a short piece saying that it'll be replaced. IcesTorm-I sent us an message on an IETF mailing list criticizing the format, and to suggesting that we use open formats like LEAP instead. Even Microsoft rejects the standard. Slashdot has supported WAP (well, kinda anyway) since I got bored a few months ago and slapped it together, and I'd tend to agree that its a crappy standard, but more due to the limitations of the devices that use it. (note: if anyone has a PDA format they're dying for on Slashdot, Send diffs -- not requests! We're working on some PDA formats, but there are only so many hours in the day, and we don't have devices that can do most of the formats users email me asking for). [Updated 7 July 18:25 GMT by timothy] Readers may also be interested in a WAP report prepared by Rohit Khare for 4K Associates, which is probably the most incisive (and one of the most critical) analyses on the topic to be had anywhere.

pudge writes "Yo. We released slash-1.0.3. Bug reports and CVS and file downloads are on SourceForge. Slashcode is now hosted at Exodus with Slashdot and Freshmeat. " The scary part is that now Slashdot and Slashcode are totally synched up... which means programmers can e-mail diffs instead of bug reports and feature suggestions (hint hint hint!)

At the request of countless users, we're happy to finally present a summary of the new setup over at Exodus. It's the result of over 6 months of work from a lot of people, so shadouts to Adam, Kurt, and Scoop, Team P:Pudge, PatG & Pater for the code, and Martin BSD-Pat and Liz for getting the hardware and co-loc taken care of. Now hit the link below and see what these guys did:

the original version of this document was written by Andover.Net Alpha Geek Kurt Grey. The funny jokes are his. The stupid jokes are mine.

The Backstory
We realized soon that our setup at Digital Nation was very flawed. We were having great difficulty administering the machines and making changes. But the real problem was that all the SQL traffic was flowing over the same switch. The decision was made to move to Exodus to solve these problems, as well as to go to a provider that would allow us to scatter multiple data centers around the world when we were ready to do so.

Meanwhile Slashcode kicked and screamed its way to v1.0 at the iron fists of Pudge (Chris Nandor) and CaptTofu (Patrick Galbraith). The list of bugfixes stretches many miles, and the world rejoiced, although Slashdot itself continued to run the old code until we made the move.

The Co-Loc
Slashdot's new co-location site is now at Andover.Net's own (pinky finger to the mouth) $1 million dedicated datacenter at the Exodus network facility in Waltham, Mass, which has the added advantage of being less than 30 minute drive for most of our network admins -- so they don't have to fly cross-country to install machines. We have some racks sitting at Exodus. All boxes are networked together through a Cisco 6509 w/ 2 MSFCs and a Cisco 3500 so we can rearrange our internal network topology just by reconfiguring the switch. Internet connectivity to/from the outside world all flows through an Arrowpoint CS-800 (which replaced the CS-100 that blew up last week) switch which acts as both a firewall load balancer for the front end Web servers. It also so happens that the Arrowpoint shares the same office building with Andover.Net in Acton so whenever we need Arrowpoint tech support we just walk upstairs and talk to the engineers. Like, say, last week when the 100 blew up ;)

The Hardware

• 5 load balanced Web servers dedicated to pages
• 3 load balanced Web servers dedicated to images
• 1 SQL server
• 1 NFS Server

All the boxes are VA Linux Systems FullOns running Debian (except for the SQL box). Each box (except for the SQL box) has LVD SCSI w/ 10,000 RPM drives. And they all have 2 Intel EtherExpress 100 LAN adapters.

The Software
Slashdot itself is finally running the latest release of Slashcode (it was pretty amusing being out of date with our own code: for nearly a year the code release lagged behind Slashdot, but my how the tables have turned).

Slashcode itself is based on Apache, mod_perl and MySQL. The MySQL and Apache configs are still being tweaked -- part of the trick is to keep the MaxClients setting in httpd.conf on each web server low enough to not overwhelm the connection limits of database, which in turn depends on the process limits of the kernel, which can all be tweaked until a state of perfect zen balance has been achieved ... this is one of the trickier parts. Run 'ab' (the apache bench tool) with a few different settings, then tweak SQL a bit. Repeat. Tweak httpd a bit. Repeat. Drink coffee. Repeat until dead. And every time you add or change hardware, you start over!

The Adfu ad system has been replaced with a small Apache module written in C for better performance, and that too will be open sourced When It's Ready (tm). This was done to make things consistant across all of Andover.Net (I personally prefer Adfu, but since I'm not the one who has to read the reports and maintain the list of ads, I don't really care what Slashdot runs).

Fault tolerance was a big issue. We've started by load balancing anything that could easily be balanced, but balancing MySQL is harder. We're funding development efforts with the MySQL team to add database replication and rollback capabilities to MySQL (these improvements will of course be rolled into the normal MySQL release as well).

We're also developing some in-house software (code named "Oddessey") that will keep each Slashdot box sychronized with a hot-spare box, so in case a box suddenly dies it will automatically be replaced with a hot-spare box -- kind of a RAID-for-servers solution (imagine... a Beuwolf cluster of these? *rimshot*) Yes, when it'll also be released as open source when its functional.

Security Measures
The Matrix sits behind a firewalling BSD box and an Arrowpoint Load balancer. Each filters certain kinds of attacks and frees up the httpd boxes to concentrate on just serving httpd and allows the dedicated hardware to do what it does best. All administrative access is made through a VPN (which is just another box).

Hardware Details

Type I (web server)
VA Full On 2x2 Debian Linux frozen
PIII/600 Mhz 512K cache
1 GB RAM
9.1GB LVD SCSI w/ hot swap backplane
Intel EtherExpress Pro (built-in on moboard)
Intel EtherExpress 100 adapter

Type II (kernel NFS w/ kernel locking)
VA Full On 2x2
Debian Linux frozen
Dual PIII/600 Mhz
2 GB RAM
(2) 9.1GB LVD SCSI w/ hot swap backplane
Intel EtherExpress Pro (built-in on moboard)
Intel EtherExpress 100 adapter

Type III (SQL)
VA Research 3500
Red Hat Linux 6.2 (final release + tweaks)
Quad Xeon 550 Mhz, 1MB cache
2 GB RAM
6 LVD disks, 10000 RPM (1 system disk, 5 disks for RAID5)
Mylex Extreme RAID controller 16 MB cache
Intel EtherExpress Pro (built-in on moboard)
Intel EtherExpress 100 adapter

At the request of countless users, we're happy to finally present a summary of the new setup over at Exodus. It's the result of over 6 months of work from a lot of people, so shadouts to Adam, Kurt, and Scoop, Team P:Pudge, PatG & Pater for the code, and Martin BSD-Pat and Liz for getting the hardware and co-loc taken care of. Now hit the link below and see what these guys did:

the original version of this document was written by Andover.Net Alpha Geek Kurt Grey. The funny jokes are his. The stupid jokes are mine.

The Backstory
We realized soon that our setup at Digital Nation was very flawed. We were having great difficulty administering the machines and making changes. But the real problem was that all the SQL traffic was flowing over the same switch. The decision was made to move to Exodus to solve these problems, as well as to go to a provider that would allow us to scatter multiple data centers around the world when we were ready to do so.

Meanwhile Slashcode kicked and screamed its way to v1.0 at the iron fists of Pudge (Chris Nandor) and CaptTofu (Patrick Galbraith). The list of bugfixes stretches many miles, and the world rejoiced, although Slashdot itself continued to run the old code until we made the move.

The Co-Loc
Slashdot's new co-location site is now at Andover.Net's own (pinky finger to the mouth) $1 million dedicated datacenter at the Exodus network facility in Waltham, Mass, which has the added advantage of being less than 30 minute drive for most of our network admins -- so they don't have to fly cross-country to install machines. We have some racks sitting at Exodus. All boxes are networked together through a Cisco 6509 w/ 2 MSFCs and a Cisco 3500 so we can rearrange our internal network topology just by reconfiguring the switch. Internet connectivity to/from the outside world all flows through an Arrowpoint CS-800 (which replaced the CS-100 that blew up last week) switch which acts as both a firewall load balancer for the front end Web servers. It also so happens that the Arrowpoint shares the same office building with Andover.Net in Acton so whenever we need Arrowpoint tech support we just walk upstairs and talk to the engineers. Like, say, last week when the 100 blew up ;)

The Hardware

• 5 load balanced Web servers dedicated to pages
• 3 load balanced Web servers dedicated to images
• 1 SQL server
• 1 NFS Server

All the boxes are VA Linux Systems FullOns running Debian (except for the SQL box). Each box (except for the SQL box) has LVD SCSI w/ 10,000 RPM drives. And they all have 2 Intel EtherExpress 100 LAN adapters.

The Software
Slashdot itself is finally running the latest release of Slashcode (it was pretty amusing being out of date with our own code: for nearly a year the code release lagged behind Slashdot, but my how the tables have turned).

Slashcode itself is based on Apache, mod_perl and MySQL. The MySQL and Apache configs are still being tweaked -- part of the trick is to keep the MaxClients setting in httpd.conf on each web server low enough to not overwhelm the connection limits of database, which in turn depends on the process limits of the kernel, which can all be tweaked until a state of perfect zen balance has been achieved ... this is one of the trickier parts. Run 'ab' (the apache bench tool) with a few different settings, then tweak SQL a bit. Repeat. Tweak httpd a bit. Repeat. Drink coffee. Repeat until dead. And every time you add or change hardware, you start over!

The Adfu ad system has been replaced with a small Apache module written in C for better performance, and that too will be open sourced When It's Ready (tm). This was done to make things consistant across all of Andover.Net (I personally prefer Adfu, but since I'm not the one who has to read the reports and maintain the list of ads, I don't really care what Slashdot runs).

Fault tolerance was a big issue. We've started by load balancing anything that could easily be balanced, but balancing MySQL is harder. We're funding development efforts with the MySQL team to add database replication and rollback capabilities to MySQL (these improvements will of course be rolled into the normal MySQL release as well).

We're also developing some in-house software (code named "Oddessey") that will keep each Slashdot box sychronized with a hot-spare box, so in case a box suddenly dies it will automatically be replaced with a hot-spare box -- kind of a RAID-for-servers solution (imagine... a Beuwolf cluster of these? *rimshot*) Yes, when it'll also be released as open source when its functional.

Security Measures
The Matrix sits behind a firewalling BSD box and an Arrowpoint Load balancer. Each filters certain kinds of attacks and frees up the httpd boxes to concentrate on just serving httpd and allows the dedicated hardware to do what it does best. All administrative access is made through a VPN (which is just another box).

Hardware Details

Type I (web server)
VA Full On 2x2 Debian Linux frozen
PIII/600 Mhz 512K cache
1 GB RAM
9.1GB LVD SCSI w/ hot swap backplane
Intel EtherExpress Pro (built-in on moboard)
Intel EtherExpress 100 adapter

Type II (kernel NFS w/ kernel locking)
VA Full On 2x2
Debian Linux frozen
Dual PIII/600 Mhz
2 GB RAM
(2) 9.1GB LVD SCSI w/ hot swap backplane
Intel EtherExpress Pro (built-in on moboard)
Intel EtherExpress 100 adapter

Type III (SQL)
VA Research 3500
Red Hat Linux 6.2 (final release + tweaks)
Quad Xeon 550 Mhz, 1MB cache
2 GB RAM
6 LVD disks, 10000 RPM (1 system disk, 5 disks for RAID5)
Mylex Extreme RAID controller 16 MB cache
Intel EtherExpress Pro (built-in on moboard)
Intel EtherExpress 100 adapter