Welcome to the New Server

Welcome to the new server guys. We made the switch late last night, and at this point, it appears that most of your DNS servers have caught up. (For those of you who haven't, hang in there, it should sort out before the day is out). We're sorting out the kinks as they show up, but for the most part, besides the DNS not being all here, we're pretty much set. As a minor system note, for those of you running scripts on Slashdot, please be careful. A couple of you are abusing Slashdot by running scripts that load the backend files (and please note that ultramode is deprecated: see the the code page for info on the new xml backends) excessively (2 people are loading the files more than 20 times a minute, when the files only update every half hour or so. I'm gonna ban IPs if you don't chill out! Anyway, hit the link below for some info on the new setup.

The first thing was to split the SQL off from the httpd. The mysql server now resides on a dedicated dual P2. It runs all the programs that handle keeping the HTML up to date, and NFS exports a nice file system to 3 other boxes (each is a single processor P2) that run httpd (and thats about it). Your hits are routed through an Alteon which divides the hits up amongst the 3 boxes.

The end result is that almost no code changes need to be made. There are tons of things we could do to make it faster still, but we'll look at that when we need it. During load testing this setup was able to handle 3x the load of the existing box.

The major remaining bottleneck is the banner ad frame load time. Browsers like to delay page rendering until they have the HTML for any included layers. For this reason we're going to work out a way to embed the ad HTML directly into the page and sidestep the need for layers. This ought to provide a nice improvement in page render time as well.

Anyway, thanks to all the guys who helped load test the new system before it went live (and if any of you are still running any scripts, you can stop now ;). Thanks to Andover for making this possible, and especially thanks to Peter and CowboyNeal for all your work.

Let us know if you notice anything funky.

Comments

NFS export? Why not Coda?

[Paul Crowley]

I'm surprised that you went for NFS rather than Coda - NFS is a bit suckful, and Linux's implementation doubly so. Coda would have given you a more secure and more efficient protocol for talking to the other servers. Get Andover to buy you a duplicate setup for testing new configurations, and benchmark the two against each other. Or get Mindcraft to do it :-)
--

How to handle address transitions with the DNS

[Anonymous Coward]

Great job! Congrats!

But next time you change the IP for your server, it might be a good idea to decrease the TTL for the IP of "slashdot.org" a few days before the change. That way, it won't take up to 24 hours for other sites to pick up the change after their DNS cache entry has expired :-)

How about an honorable mention do dN?

[ChiChiCuervo]

It would have been nice if Rob also mentioned the gurus from DigitalNation who put the servers together and provide the bandwidth to Andover (and now also /.).

I usually don't like to shamelessly plug my employer, but our tech dept is quite overworked and unsung.

I will, however, point out that I now get ZERO LAG!! Yay!

Re:You should use Solaris or FreeBSD

[pwe]

Both have "cachefs", and support NFS version 3. Linux' NFS is one of its weakest points - it is dead, bog slow compared to my Solaris servers.

If the filesystem you're mounting changes frequently, as I presume the /. filesystems do, then cachefs will probably slow things down. In my testing, using Solaris clients and Solaris servers, it worked best on read-only, rarely changed filesystems and then it worked very well.

That said, I'm not too happy with Linux's client NFS performance, but my problems seem to only occur when I try and use a Linux client with a Solaris server and I haven't actually figured out which side the problem is on. Hopefully, I'll have time over this weekend to try the NFSv3 patches.

PeeWee