On The Subject of Web Hosting

There's been quite a number of "incidents" over the last year with web hosting companies that haven't lived up to their end of the bargin, or have had other problems. The most recent was the CiHost Drama, which is now, happily, finished. One of many people affected by that outage wrote a short piece for us - but I'm interested in what everyone thinks about web hosting. What's the good places? What's the bad? What pointers can you offer to everyone else? Click below to add your thoughts.

Surviving Web Hosting
By Alan Cowderoy

Great, you've built your site. It's the best thing since Slashdot. The search engines rank it in the top ten hits for just about every keyword on the planet, even Yahoo lists it. The javascript rocks in any browser you care to name and the graphics go ascii for Lynx.

Then your hosting service takes you out because you've got too many hits, or toasts your email, or restores an old copy of the database or just plain simple goes down for days on end....

There's a problem and it's not one that's going to go away or at least not for the hundreds of thousands of sites who are necessarily dependant on hosting services.

Many in the newsgroups immediately reply that the answer is to rescue those linux cd's from under the bed and do it yourself.

For the majority of small sites doing it yourself is just not an option. It takes serious levels of expertise to run secure/resilient unix boxen not to mention the cost of the bandwidth. Given this, and assuming you don't have the cash or expertise to run your own, third party commercial hosting is the only real option. (Ok i would love somebody to come up with an alternative here. Community hosting? Special interest group hosting? Dunno you tell me.)

Meantimes, how should we build some sort of resilience into our sites and not get wiped off the net when the provider screws up?

The immediate reaction to this question tends to be "choose a good hosting package". Unfortunately that is easier said than done.

If the recent CiHost problems have taught us anything it's that choosing a hosting company recommended by the web hosting comparison sites is no guarantee that you are not going to have big problems. And if you can't trust them you've not got a lot of other criteria to fall back on.

Some warn against buying from 'resellers' rather than direct from hosting companies who have their own server farms. I may be missing something here but I admit that I can't see what is *necessarily* better about buying direct from the company owning the kit. What is so bad about reselling?

Apart from that other reliable criteria for choosing a hosting service are very thin on the ground.

Now there's free hosting packages, cheapo hosting packages and expensive hosting packages all touting for your traffic. In any case they are all commercial packages from commercial operations who have agendas that may or may not coincide with yours.

The fact of the matter is that keeping your site on the air depends on some third party not screwing up.

That means we need to look at damage limitation and what we can do to survive hosting problems. Like any damage limitation exercise its all about 'graceful degradation'. Lovely phrase. The idea is that when put under pressure you don't just die you sort of wither slowly. ie you hold out as long as you can not by accident but by design. You also hopefully hold out long enough to flower again somewhere else another day.

Now the bad news.

If the truth be known there's not a lot you can do.

Here's what I've been able to glean from the usenet and from my own experience.

The first bit is too basic to be true but has to be said... If you've not already done so (why ever not?) then the get your own domain name. Without your own domain you are wedded to your hosting service. In the worst case if you are not satisfied with your hosting package you will have to move. If you don't have your own domain all the work you have done to get listed on search engines and publicise your site will be wasted and you must start again. Once you have your own domain you simply have to move the files, change the domain details to point to the new host and after a few days the problem is solved. You may have suffered in the mean time but the damage is normally contained.

Even if you do have your own domain then make sure that you have backup copies of your site. This is easily said and done for static sites but much less obvious for sites running programmed backends and/or databases. That however is a whole other discussion. At any rate the comment stands.

Email.

I discovered this one by accident but it's fairly obvious, really.

If you put an email address on your site so people can contact you then make sure it's not on the same server as the website. Preferably make sure it's not even with the same hosting company.

Visitors to your site may well have noted your email and if your site is having problems will mail to find out what is going on. If the mail is in the same domain as the site then probably it's down as well. Similarly make sure that you have backup email addresses of your own you can use to contact people.

This sounds banal but it is in fact rather irritating. If you've just bought a domain name then presumably it's not just to have www.mydomain.com but me@mydomain.com as well. Now I'm telling you not to use the email address on your website. Sorry. Don't.

At the cheap end you can just use your normal ISP mail or one of the free addresses. For anything more serious you might want to invest in another domain and host it somewhere entirely different. This would allow you to generate another branded email name and get round the problem described in the last paragraph.

If you do keep your mail in a seperate domain then of course the opposite also works, your site can cover for you and keep people informed if your email suddenly explodes.

In my view this is also a very good reason for not using html forms for mail. If you do then when the website gets toasted so does your incoming mail.

Multiple domains/multiple hosts.

It may be a good idea to split your site up into multiple domains handling different sections of the site and hosted with different providers. Whether this is realistic or usefull obviousely depends on the structure of the site. The downside is that it will probably be more expensive. It may not be much more expensive though and it could be a good way of limiting exposure to server failure.

It might also be interesting to split mydomain.com, mydomain.net and mydomain.org domaines to different suppliers. Two of them could be extremely minimal 'limited service' packages designed normally to route to the main domain but to provide limited back up in case of problems. This leaves the problem of how people are supposed to know to go to .net when .com is up the shat. Even so a set up like that would give you some chance of maintaining some traffic.

Mirrors.

Mirror your site on another server - perhaps in another geographic region as well. Downside is cost again and if you're using databases you need serious technical expertise. Upside is that this has performance benefits as well. Probably only for the big boys.

DNS issues.

DNS contact on a different server. Your email contact address for your DNS details should *not* be in the same domain as your site. If you wish to change your DNS details the email request **must** come from the address registered for that domain. If your domain is down and the registered email contact is in that domain then you can find yourself in the situation of neither being able to access your site nor being able to move it!

It is also recommended that you not use your personal email address for this registration as this database gets trawled by spammers.

DNS on a different server.

Normally hosting packages propose DNS services 'in the bundle'. You are not obliged to do this though. You might want to have additional DNS servers hosted elsewhere. There is a free public DNS at http://soa.granitecanyon.com/. There may also be commercial DNS-only hosting services. In this case the aim would not just be to host DNS at another site but to use the extra servers as backups to the normal ones. This from a post by Kent W.England to alt.www.webmaster. "Here's a radical thought. Ask your ISP to set up their two name servers as secondaries to yours. Put them both in the zone file, along with your server as primary and then register your name server as primary and one of theirs as secondary in the domain records at Network Solutions. Then you'll have three name servers, but total control of the zone file through your primary. Another option is to list both their secondaries in the Network Solutions database and leave your primary in the zone file. Almost same effect."

Thou shalt know thy ip addresses.

DNS resolution converts between names and ip addresses. If the DNS servers take a dive you should still be able to get to your sites via ip addresses. The fact that the DNS servers are down does not necessarily mean that the http, ftp, mail and telnet servers are out. (indeed they definitely should not be). This is no help at all to your visitors but does at least allow you to get at your sites.

I hope other people have some ideas as well because this all looks pretty thin to me!

And remember web hosting companies are now big business. When they screw up there's a deathly hush.

Comments

How to do hosting right

[thogard]

The major points to web hosting is the DNS and http hosting side of it.

You must have several dns servers. This means you must have three working at all times on at least two diffeent networks so when one dies, you've still got at least one left. This should have different MX records to stash your mail on a nice mailq somewhere till you get back on line.

Telsta lost a major router that the main reverse dns for all of 203.*.*.* was on. Guess what that did to email all over oz-- telstra's bigpond internet service was deleteing messages because it couldn't reverse lookup addresses and assumed it was spam. Its nice when all your dns servers are on the same side of one router.

The next is the http hosts....
Web hosting compaines are everywhere but I run my own box. It may be on a slow modem link but I'm going to run my own server. If I need (or want) the speed, I'll mirror it on one or two other servers near MAE-(east|west). Its easy to build a cgi on a server to basicly webcopy an entire site so the main site only needs to very little hosting.

So you've got the worlds best site and need to up no matter what so what do you do?
1) set up your source site
2) set up your development/stage site (you don't work on the live one do you)
3) set up your dns servers (at least two plus your main site which is the "master" and you don't tell the nic about it)
4)get an account on one of the big players. Don't let them dns for you. Don't let them do email. They are only there for hosting.
5) copy your site to big site and wait for the hits.
6) have a plan in place to deal with your big site going away.

So what about databases?
Run them on the big server and have it send you changes every 15 minutes (or what ever you can afford to lose) to your source site. That way your "offical site" is someplace safe under your full control but a very good copy is off somewhere running fast with a big pipe.

Then there are all the security issues but I think thats a different /. discussion...

For thouse that care...
web.abnormal.com is a sparc 1 with 24m on a T1 somewhere in KCMO. www.abnormal.com usualy points to three servers including the main server and mirrors at jumpline.com and cqhost.com. Jumpline is running some sort of smp beast that has lots of bandwidth which I pay $15/mo for. I've been having problems with cqhost so they are out of the loop for now. Dns is done by the main server, an isp in Oklahoma, and one in the bay area. I've got current zone backups on two other servers ready to roll should I need it as well as a backup of the complete site here in oz. And this is only a personal site.