White Box, Or Big Names for Lower-End Servers?

LazloToth asks: "Those of us who manage small- to medium-size networks face the decision all the time: for the run-of-the-mill web, print, or storage server running on i386 architecture, should we buy HP or Dell, for example, or build it ourselves from commodity hardware and save some bucks up front? In my operation of fewer than 50 servers, one will see a mix of the two. For servers that take more abuse, I tend to buy the proprietary stuff. But not always. I wonder what experiences other admins and managers have had with do-it-yourself servers in a production environment, and whether they feel that white-box servers perform as well - - and last as long - - as anything else? What is the mix in your network of big-names to no-names?"

Comments

Two sides

[PunkOfLinux]

there are two sides the issue here

big name - warranty (saving your ass)

white box - if you build it yourself you know what's in there. It's cheaper. But you don't have a warranty.

Re:Two sides

[madstork2000]

Only if it is an on-site warranty, and the turn around is guaranteed in a short period of time. I have used White boxes, because I can usuall afford to buy 3 lowend boxes for about the price of a single Dell.

I run webserver and have about 25 boxes. I buy motherboards from only a couple of manufacturers that I trust. I run commodity Harddrives, and use rsync rather than fancy scsi. I look at the individual warranty on the parts.

I generally save enough $$ that I can buy the server and a "hot-spare" for well less than the price of a name brand box. I have had relatively few hardware issues, and even the ones I did have could be fixed quickly and cheaply. It is nice when no single component costs more than about $150.

I guess in essense I am warrantying it myself, warranties do no good if the server is going to be down for any length of time and you are dependent on a big companies whims.

-MS2k

Re:Two sides

[toddbu]

In addition to the money you save, you'll also save a lot of time. I've run both high-end Compaq machines and servers that I've build myself, and I've found the latter to be a lot easier to deal with. Here's why:

• No special drivers to load - Compaq has their own configuration tools and just keeping track of the CDs with the software was a pain. If something goes wrong, is it the driver, your OS, or something else?
• Inability to debug hardware - You can't drop a proprietary drive in another machine that you have in your office to see if it works. You have to have another proprietary machine to see where the problem is.
• Touchy hardware - Some might disagree with me, but I found Compaq hardware to be really touchy. When you spend $10-20K on a single box, you expect it to always run. I've found as good or better reliability in machines that you build yourself.
• Configured the way you want - It can be difficult to build out a proprietary machine just that way you want it. If the vendor is short on parts, you have to choose between getting it now in a different configuration or waiting until they have the part. When you build your own stuff, you buy what you want when you want it.

Don't get me wrong - there are times when proprietary systems make sense. I don't think I'd ever build my own laptop. But servers are better when you build them yourself.

We go proprietary

[mnmn]

I bought a noname 2U server at $600 CDN for the company which gave us alotta grief... modems would never work in its PCI slots. So we decided to always go proprietary. All our big servers are ibm xseries.. and we buy ibm xseries 206 ($600 USD) for cheaper stuff. We never go Dell on servers.

I know you can build a superior system thats whitebox. MDG sells machines for cheap with Intel motherboards. You can buy Tyan mobos for whitebox systems.

However keep support in mind. Everytime something breaks on IBM xseries servers, we call tech support. In 4 hours of calling the replacement part arrives, and the techie arrives the same or next day and replaces the part no questions asked. Sure we've had lots of trouble on our tape drives etc, but it gets replaced painlessly, no driver changes, and no financial hits.

Another benefit of name brands is that you can say youve worked on so and so servers in your resume. Smart employers wouldnt or shouldnt count that, but you do see people asking for MCSE and proliant servers, etc. Its even more specific when you get into UNIX... they'll only accept that brand of unix.

Proprietary is usually better

[duffbeer703]

Lights out management usually works better on IBM, HP or Dell systems. Also, building and fixing machines is a pain and gets time consuming & expensive, particularly if you get a bad batch of drives or motherboards that requires alot of fixing.

If you are running < 10-15 machines, I can see cost savings in going whitebox. But if you are tight on staff and runnings lots of machines, buying name-brand kit is cheaper in the long run.

White box all the way

[linuxwrangler]

The contents of the box are pretty generic for most purposes. Motherboard from Asus/Intel/..., BIOS from Phoenix/Award/..., Processor from Intel/AMD/Motorola/...

What really makes a difference is the vendor. I have a local guy who I can call and ask for recommendations and advice. If I tell him I want a Dual Opteron with 12 gig RAM, mirrored 74 GB hot-swap drives, dual hot-swap PS and a rack-mount case of my choosing he personally delivers it a couple days later.

Drive in my raid-array dies? He brings by a replacement the following day.

Oh, and the only number he gives me is his cell phone. And he answers it. Always.

With the exception of some specialized telephony equipment (actually a different white-box vendor specializing in that market - Dell et. al. wouldn't have a clue about this stuff), he is always my first call.

I've been using him for years. When the company he worked for ceased operations he started his own and service has remained outstanding.

I guarantee that nobody who uses the "name brand" machines can come anywhere close to the responsiveness and support that I get from my local vendor.

Pick a major vendor

[Frumious Wombat]

We went the white-box route on our first compute cluster, which were then converted to desktops later. Decent machines, but the power-supplies weren't up to the 24x7 operation and tended to eventually have the fans sieze up, causing the ps to overheat. Eventually other components showed that they could have been better, and we cannibalized some machines to keep others running. They were replaced by HP and IBM boxes under 3-year, next-day, service contracts.

The advantage of calling IBM, HP, or even Dell, is not simply the service contract (though your time is worth something), or the fact that their QC is superior to wherever you're getting your parts from, but that they have real engineers, who worry about such issues as optimizing air-flow, choosing proper fan-sizes, etc. Take apart an IBM xSeries 345 some time, then try to decide if you could actually buy parts to build a machine like that, for less than just calling IBM.

White-box systems may have once made sense, ( I remember a 386/40 AMD-based system that I wrote my thesis on that was still running when I came to visit years later), but with modern components, heat-loads, etc, it pays to invest in properly engineered hardware, backed up by a company willing to service it on short notice. WB hardware may still make sense for desktops, if your environment keeps the data in non-local storage, so that a new desktop can be dropped, booted, and put into production immediately. Never with servers.

We adopted an informal, simple, but effective policy: Do not buy any machine that doesn't come with a three-year warranty, or hard-drive that doesn't come with a five year warranty.

Go with the name brand

[alta]

With the low price of low end servers you aren't going to save a lot of money going with the low end box.

Consider this, you buy, or build a white box. You'll end up with very short warranties provided by different companies, very much a pain in the arse. You may save $200. Now think of how much a 50k/year sysadmin makes per hour (roughly $25 if I did calc right)

So, $200/25 = 8 hours...

Now you've got $200 which is equivelent to 8 hours. Are you going to spend more times on a whitebox than a dell? I would say so. ESPECIALLY if you're building yourself. Consider extra time spent finding parts. Extra time putting it together. Then when things fail you have to round up the warranties for individual parts. Probably your warranties won't be as good as what dell provides. And then repair time. I know as a sysadmin we tend to repair ourselves anyway, but consider it may be something you WOULD let an on-site tech repair because you're busy...

In a home situation, I'd say build your own. When you're off the clock, your time is free. but at work, when time IS money, buy the named stuff.

BTW, my numbers are BS. Play with your own, I think you'll draw the same conclusions.

Datacenter Experience

[binaryspiral]

I work in a fairly large datacenter, where I help support many of our colocated customer's equipment. Some of which we sell and maintain, some of which they purchase and colocate. I've seen a good mix of generic servers that were custom built because there was no pre-built options available. But when it comes to support - there are few options when things go down.

Motherboard company blames ram, ram company blames raid card manufacturer, raid card says it's a bad firmware version of the motherboard... two hours later, server is still down. Who's going to let us swap out a motherboard just to see if it works?

I don't see a price advantage to whitebox servers compared to modern server hardware from the big names. Anyone who's just looking at the price tag is fooling themselves.

Dell's hardware is unimpressive, I'll give a nod to the previous responder who mentioned that. And storage subsystems are still insane. Even with the evolution of SATA for slower mass storage, cost/MB is still too high with these subsystems.

Beware of a name brand's inexpensive servers. Some of the rock bottom units are cheap, but they lack some of the basics like raid, hot swap drives, expandability... On the bright side, even if you go with these cheap units, you'll still have service and support from a major player.

I have the exact opposite experience.

[khasim]

I have a server room filled with HP servers. We lease them for 3 years and then send them back. I have ZERO problems with the software and I get parts (usually drives) replaced in less than 24 hours.

These servers run 24/7 for the 3 years we have them and give us no problems at all.

white boxen for vanilla services

[dr_leviathan]

We buy hundreds of white 1U pizzaboxen from SiliconMechanics.com every month (not only are they white, but they are also blank -- we net boot debian GNU/linux onto them ourselves). SM has an excellent record for replacing broken parts, although we're never in an emergency when something breaks since we deploy backup hardware for everything. If something breaks we can switch to the current backup, start converting a spare machine to be the new backup, and then take our time getting the broken hardware fixed, its all under warranty.

All of our vanilla services: mail, web, and even database are on white boxen from SM. We have some black box stuff for heavy mass storage.

It's not the color of the box

[fm6]

You're not saying "Buy white box." You're saying, "buy from a good white box vendor." And how many of those are out there? From what I've seen, not that many.

Besides, by depending on this guy, you've created a one-man point of failure. What happens when this guy gets sick or goes on vacation? Where's your immediate response then?

Even if he never gets sick or takes time off, he's not going to be able to sustain this level of service. His own good reputation will work against him. He's obviously one of those people who has to do everything himself. He's probably not very good at delegating or training, so he's never going to be able to scale up his operation. So unless he starts turning away business and dropping customers when they get too big for him to handle, he's going to get in out of his depth.

If I were in your shoes, I'd want my hardware needs met by a solid organization, once I could count on not just now, but years from now. And that has to do with people, not with where the boxes are assembled.

Re:Opposite Experience Here

[phaze3000]

Wow, all I can say is you must live in some alternate universe to the one I live in.

Here getting Dell to come out a fix one of their servers (even with 'silver' 4 hour cover) is like getting blood from a stone. With the IBM auto-support I had one occasion where a disk failed and we had the replacement before anyone noticed the problem (incorrectly configured RAID monitoring was the culprit re the lack of notification).

Power Supplies in Generics

[BigBlockMopar]

Decent machines, but the power-supplies weren't up to the 24x7 operation and tended to eventually have the fans sieze up, causing the ps to overheat.

Oh yeah, big time Achilles Heel of the generic PC, assuming name brand mobo and stuff.

It's just impossible to get a good power supply in a generic PC. ("Good" means built with decent quality components, like the Astecs and Lambdas you'll find in proprietary systems. It does not mean "Comes with a ThermalTake Fan and is the choice of 14-year-olds and overclockers!".)

My best success was based on a simple formula: the power-to-weight ratio. Buy the heaviest supply marked with a given advertised wattage rating.

Then, for server use, step 2 is to open up the supply and replace all the made-in-Bangladesh-or-Taiwan-or-China electrolytic capacitors with Spragues or Nichicons rated AT LEAST 1.5x the voltage ratings of the capacitors which were in there. And then out comes the no-name 12V fan, only to be replaced with a (loud! expensive! moves a hell of a lot of air! lasts forever!) Comair Rotron 120V fan running directly off the power line. Also gives you a chance to fix the *many* cold solder joints you're likely to find in commodity power supplies. All told, usually under an hour per supply, with the new fan often costing more than the supply!

Since I started doing this, I haven't had a single failure of one of my white-box server supplies.