Slashdot Mirror
Going Deep Inside Xserve Apple Drive Modules
adamengst writes "If you've had an Xserve drive fail, you may have considered saving some money by putting a replacement drive inside its Apple Drive Module. That may be a false economy, though. TidBITS explains why, while pinning Apple down on exactly what goes into Apple Drive Modules and why they cost so much more than bare retail drives."
I was going to complain that this is not a very interesting story for 98% of Slashdot, who has never seen an XServe and is happier for it, but since the link is already slashdotted, I guess I should complain about that instead.
While I hate to copy it, the server being pummeled and reporting errors for 9/10 requests doesn't lead to ad revenue either, so here goes:
About a year ago, we bought an Intel-based Xserve with a pair of 80 GB SATA drives to act as our primary Web server. When the boot drive went flaky on us in October 2008, we were able to recover from the backup on the second drive and off-site backups, if a little shakily (see "TidBITS Outage Causes Editors Outrage", 2008-10-07). But although we were able to bring the machine back online, we didn't trust the drive that had failed. Since the Xserve has three drive bays, the obvious solution was to purchase another drive. Sounds simple, doesn't it? Not so much.
You cannot buy a bare hard drive and insert it into an Xserve, as you can with a Mac Pro (and having just added a drive to my new Mac Pro, I can say that Apple did a stunningly nice job in making it easy to add drives, especially in comparison to the awful approach they used in the Power Mac G5). Instead, Xserves require Apple Drive Modules, which are custom carriers containing drives.
For users accustomed to buying inexpensive hard drives, Apple's pricing on the Apple Drive Modules comes as a bit of a shock. An 80 GB SATA ADM costs $200 from Apple, and a 1 TB SATA ADM costs $450. In comparison, a bare 80 GB SATA drive can be purchased for a measly $35, and a 1 TB drive is only about $100. That would seem to point toward buying a new SATA drive and swapping it into the bad drive's ADM. However, when I started down that path, a number of problems arose, such that I bailed on a quick solution and simply purchased a new 80 GB SATA ADM to replace the bad one.
First, I wasn't sure whether my Xserve had SATA drives, as I thought, because System Profiler on the Xserve shows nothing on the SATA bus, instead including all drives on the SAS bus. (SAS stands for Serial Attached SCSI, and is a high-performance data transfer technology that supports fast SCSI drives and is downward compatible with SATA drives.) After some discussion with knowledgeable folks on the MacEnterprise list and careful reading of the drive details in the SAS section of System Profiler, it became clear that both SAS and SATA drives are shown in the SAS section, with SATA drives having "ATA" as the Manufacturer, and showing "Yes" in the SATA Device line.
Second, once I knew that I had SATA drives in my ADMs, I started investigating if there were any gotchas involved in replacing the drives. There turned out to be surprisingly little hard information about this, with some people having replaced an ADM's drive with no trouble and others experiencing performance or reliability issues. I did find a few discussions about how replacing drives isn't recommended, but giving no solid sources.
Confused, I contacted Apple to discuss why ADMs are so expensive in comparison to bare drives, exactly what an ADM does, what Apple recommends users do with failing ADMs, and whether or not replacing a drive in one is a good idea. That conversation revealed a great deal of interesting information about the ADM and shed some light on what people with flaky ADM drives should do.
Drive Selection -- The most important fact to know about ADMs is that Apple doesn't use just any drives. We've all benefited from the amazingly low cost of storage. But whenever manufacturers compete on price, they cut corners every way they can to reduce costs. Although drive reliability is generally good, everyone who buys bare drives regularly has a drive vendor they refuse to patronize due to bad experiences in the past. (As is often the case, these people all hate different vendors, depending on which one was having a bad run at any given time.)
Since the Xserve is designed to be in constant use - 24 hours a day, 7 days a week, for years at a time - Apple doesn't use the least expensive drives available, since those drives are designed for more normal duty cycles in desktop computers - 8 to 10 hours per day, with variable use during that time. Instead, Apple wor
The pixie dust is in the controller, not the platter.
Apple dude discovers that servers use, well, server class HDs and they cost more than normal ones.
Oh, and the 'sleds' that hold the HDs have some LEDs (cool!) and a controller board to work with the cooling system.
Like pretty much every other half decent server then.
I want a list of atrocities done in your name - Recoil
Yeah, this is real "pixie dust." A lot of it is speculative stuff that assumes that somehow NAS drives are magically different from normal HDDs. Also contains references to the "Bathtub Graph" which has been pretty much shown to be a load of bunk: http://www.neowin.net/index.php?act=view&id=38693
Sarcasm fail :(
Any life is made up of a single moment, the moment in which a man finds out, once and for all, who he is.
Wow, those Apple Disk Modules are cheap! A 1TB SATAII 7.2K RPM disk module for an EMC CX3 SAN runs about $1500. But I think they get to high grade the drive makers' inventory since they suggest only 1 hot spare per 30 disks.
The gist of the article is "We asked Apple why they're more expensive, and took their word for it." It's just regurgitated marketspeak about how Apple tweaks the firmware for the optimal performance, has special rubber on the grommets of the ADM that is specific to each drive to reduce vibrations, and how off-the-shelf drives are unreliable, slow, noisy, and hot.
They don't make an effort to verify this information at all. Because Xserves won't run with commodity drives, they can't do a proper comparison to determine how much is truth and how much is smoke-up-the-ass from Apple. This is such an astroturf article, it doesn't even pretend to be anything otherwise.
... nicely. Quoting TFA - "About a year ago, we bought an Intel-based Xserve with a pair of 80 GB SATA drives to act as our primary Web server. When the boot drive went flaky on us in October 2008, ... "
Welcome to pragmatism and reality - Drives fail all the times. So use cheaper drives in redundant mode, replace them with cheaper drives when they fail. You would have saved good amount of money even if the cheaper drives failed three time more than the costlier ones. (450$ for 1TB vs $100 for 1TB - from the same article.)
The summary makes it seems that there's no rhyme or reason why Apple charges more for their HDs and why can't the consumer simply replace it with a standard SATA HD. If you RTFA, it goes into a long list of reasons why. Whether you accept Apple's reasoning is another matter.
To begin, XServes use a HD module called ADM rather than simple HDs. On the new MacPros, they also use a module but those modules are designed to replace the HDs inside. For the XServe you apparently can't get a bare drive alone, you have to replace the whole module. The author begins to list the reasons:
And then the author concludes:
Well, there's spam egg sausage and spam, that's not got much spam in it.
If your OS could benefit from custom firmware...for example if you file system writes in certain sized blocks. I can see that being a case for specialized hard drives. But does that really account for the cost? If the drive's firmware is flashable, let the customer flash it to perform better with their OS choice.
The rubber grommet thing? Now that's some excellent bullshit. You are really telling me that someone spins up the drive, records the vibrating frequency, then selects the appropriate rubber grommets from the bin, then assembles the harddrive caddy? The bullshit flag is on the field, 10 yard penalty - roughing the truth. Again, even if that DID happen, does that justify the increased cost? I doubt it.
In the end, you still run a sluggish GUI on a server. fail. I bet if you ran your website on a stripped down *nix server, on a $1000 machine, your ass wouldn't be slashdotted right now.
THL phish sticks
Having worked in the disk mines of IBM many years ago, the SCSI disk controller is somewhat your pixie dust but the real reason is the disks, heads and other parts for the SCSI drives came from IBM's best manufacturing facilities. The deathstar ATA drive's parts came from the lesser manufacturing facilities. In theory a SCSI disk should not be much better than ATA but the reality is the best made, more reliable parts go to the high end more profitable products.
"Apple also told me that the rubber grommets that hold the drive to the ADM carrier are chosen specifically to match each drive's vibrational characteristics."
"Yes, sir. In order to reduce vibration we use the finest of synthetic compounds to minimize noise so that your cold room droogies won't have to suffer a higher level of acoustic trauma."
Get real.
Every major manufacturer re-labels drives for inventory and warranty purposes. They also use custom firmware to identify the drives for the same reasons. Special grommets? If you have worked with ADMs you know those grommets are extremely thin, shred to bits if you try to reuse them -more likely you lose them taking the thing apart, and are there more to keep the screws from coming loose than anything else. I have replaced drives in ADM modules before with RE drives -because the drives were mirrored- and haven't had a problem. If you are going to replace the drive in a server -a piece of mission-critical equipment- with the cheapest bulk OEM drive you can find, you will have problems.
Sig this!
Been happening for years.
Back in the late 80's, in addition to my dev job, I admin'ed a Motorola Delta 3600 box. We were looking for a little more space, manual said that it would take a Seagate ST-251N 40MB SCSI drive. So we bought on off-the-shelf.
It wouldn't work. It turns out that Motorola had custom firmware for those 251Ns.
So it's been going on for at least 20 years.
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
The special ingredient in XServe disk drives is... love. :-/
HP and EMC does the same thing. HP will charge you $500 for a 1TB SATA today and we just paid EMC $800 or so per 500GB drive for a bunch of drives.
one nice benefit is the support. HP has a proactive failure warranty. if it flashes and alert that it thinks the drive will fail you call them and you have a new drive arrive by UPS the next day. EMC will come out within 4 hours to replace it.
and they are guaranteed to work with the brand name RAID controller that is the same brand as your server. you're paying for the testing and special drivers knowing that everything you buy will work together and you don't waste time calling support and playing musical telephone
Go to any server manufacturer's website (or a retailer if they sell through the retail channel). Dell, HP, IBM, I don't care. Any of them.
Price up equipment sold specifically for servers. Note particularly the price they charge for a larger/faster hard disk.
Go on, I'll wait.
Right, now go onto your favourite cheap & cheerful parts supplier and look at how much they charge for a hard disk.
Is it really the exact same disk with that much price discrepancy? Well, I (along with a lot of sysadmins) would dearly love to believe that it isn't. Whether or not that's true I honestly couldn't say.
What I can say is that if you do go out and buy the cheapest disks you can to populate the server, warranty support from the OEM is going to suddenly become "Oh, you plugged some random disk in? Go away and come back when all the disks are from us". Which starts to look rather expensive rather quickly when the RAID's knackered and you need to resurrect the system as quickly as possible. If your job is on the line, it's soon looking even more expensive, and nobody wants to say "I was sacked from my last job because I cut one too many corners on a system that was critical to the business" in an interview.
It's not so much of a problem for the Googles of this world who write their own applications to live on huge clusters which have component systems being added and removed all the time. Most of us, however, don't have that luxury.
and the disk you put in grandma's desktop.
Yeah, the later is of vinyl.
This retarded fluff piece aside, the reason people buy (and pay a premium) for oem "blessed" hard drive replacements is because they JUST FUCKING WORK. If I save $100 on a hard drive, but spend two hours dicking with the raid controller to get it to play nice, or find out that it is in fact 2 mb smaller then the other drives, and now the raid won't rebuild, or has some firmware issue that I now need to rig up something to update, etc. I've lost money.
There is value in having everything already tested, and all your equipment in a "supported" configuration. When you have problems it makes it that much easier.
The fact that this article was apparently written by someone who does not know the difference between SAS and SATA makes it completely worthless. Clearly they are not qualified to admin the server they do have, much less write articles about the technical benefits of apple drives over other replacements.
Guys drives dies in Apple server hardware.
Guy looks into buying retail drive for replacement.
Guy asks forum members for advice and decides to call Apple
Guy calls Apple wanting to know why their drives ar 4x+ the cost of retail
Apple gives Guy song and dance about magical marketing BS
Guy falls for BS and tells everyone else to follow magical marketing BS
Honestly what did he expect to learn by calling Apple? Call any manufacturer, tell them you want to use cheaper 3rd part parts instead of their overpriced parts and be prepared for a load of horse shit to flow from the phone. I worked in the IT department of my college for an elective credit way back when. The head IT guy almost never bought OEM stuff if their was a cheaper retail part that would do the job. He insisted its just a 3rd party part with company logo stuck on it sold at a 3-5x mark up. He would rather used the money he saved for better things like new equipment or upgrades. Never had any problems.
TidBITS system guy here. Sorry for the troubles. We had a glitch in our Apache min/max/spare/etc settings that was triggered for the first time by Slashdot traffic. (A combination of a new method to zoom images and AJAX produced a very high set of spawned children for each new visitor.)
Freelance tech journalist for the Economist, MIT Technology Review, Macworld, and others
No, the first link is a DailyTech article about the study in the second link. Apparently they just misunderstood the text of the original study. So it's my fault for linking the damn DailyTech thing, and DailyTech's fault for not reading the whole study. The only similar competing study was by google, and their results were roughly the same.
The people who write these articles are stupid and contribute stupidity to IT in general.
1) Apple calls these beefier models "server-class" drives; you may also see terms like "RAID edition"
There is simply no data to back this up. The vendors themselves do NOT do sufficient testing to make these claims ergo Apple can not make these claims. This parallels the so-called better failure rates of SCSI/FC 'enterprise' drives and consumer SATA drives. In the FAST paper by Schroeder we see the following quote.
". . . we observe little difference in replacement rates between SCSI, FC and SATA drives, . . . ."
2) Firmware - the closest thing to an argument here is "may prevent Server Monitor from being able to report on the drive's health"
3) Carrier - "Apple also told me that the rubber grommets that hold the drive to the ADM carrier are chosen specifically to match each drive's vibrational characteristics."
This leaves out the most important thing. "So what" - ok if the drives vibrational characteristics are not matched what happens. Is it significant? Where is the data to say so?
4) Extensive testing - Essentially arguing that Apple does burn-in testing (which you could easily do yourself) however...again from the FAST paper:
"Contrary to common and proposed models, hard drive replacement rates do not enter steady state after the first year of operation. Instead replacement rates seem to steadily increase over time."
Drives act like mechanical devices NOT electronic devices.
In general - have you EVER read an article with so many "may"'s and "possibly"'s? There's very little here that could be risk assessed (giving some kind of probability of some consequence) - which means it USELESS as advice. The parts that actually IMPLY some kind of probability/consequence are not well supported by the studies with the largest sample sizes.
Parts of this may be true (it's impossible to say, since they don't specify exactly what they're comparing) -- but even if parts are true, it's misleading at best. In fact, some of it doesn't even seem to make sense. Let's look at real price lists from Apple and dell.
First we note that Dell doesn't seem to offer a 73 GB drive at all, so it's not entirely clear what they're comparing. The most likely possibility appears to be Apple's smallest option, a 73 GB SAS ADM ($300) to Dell's smallest, a 146 GB SAS ($349). While it's certainly true that the prices are comparable, it's also true that the Dell drive is twice as big.
For 300 GB SAS drives, the Apple site shows $650 while the Dell site shows $699. While Apple's price is lower, it's certainly not even close to $200 lower. To get a $200 price difference, it looks like they compared the full price of a 300 GB drive for the Dell to upgrade price for the Apple (i.e. the price difference for changing from the stock drive to the 300 GB drive).
For 1 TB hard drives, they have something of a point, but not a very good one. Apple's price for a 1 TB SATA drive is $450, while Dell's is $639. They fail to note, however, that Dell also lists a 1 TB SAS drive (an option not available for the XServe) for $679. Taking this into account, it looks a great deal as if Dell is simply doing their best to encourage their higher-end customers to use enterprise-class SAS drives by offering them at a purely nominal incremental cost over SATA drives.
The original article attempts to portray the situation as Apple offering prices that are at least as good as, and often better than the competition. In reality, there appears to be only one reasonable configuration where the Apple is likely to be competitive: the one using 300 GB SAS. At the low end, the Dell offers twice as big of a drive as the Apple for a purely nominal price difference. For lots of storage, the Apple offers only SATA drives where Dell offers SAS. If you're storing 1 TB of data (or more) the incremental cost of SAS is usually fully justified. There are undoubtedly exceptions, but they're not particularly common.
The universe is a figment of its own imagination.
And what would that be? Cases that don't require screw drivers to open? I'm seriously asking cause you've been able to replace drives on Macs as long as I've used them.
Most Dells and the like come on rails that let you slide them in and out, but you still generally have to open the case and disconnect the wires on desktop models. Do you eve know what he's talking about on the Mac Pro? How many PCs do YOU have that have drive bays in them as if they were a RAID controller where you just pop the drive out and it auto disconnects power and cable? I've yet to see one outside of the server room.
You claim you had this on your PC in 1989? Sounds more like you hadn't seen a PC in 1989, not then, or ever since whats being referred to here was most certainly considered high end back in the early 90s, and even earlier in this decade.
I've been able to replace drives on Macs for years. Low end models have more constraints on what you can put in them, but the 'professional' quality macs aren't really different than a 'PC', especially not now, but even for the last 20 years. They've got expansion slots. They've ALWAYS supported more drives than a typical desktop PC, remember they used SCSI to start with, 7 to 14 drives in a machine by default, not 2 (or 4 as soon as dual controllers became standard on PCs) as with IDE or 1 for some other technologies back in the day. Want an external disk? Every mac has a method for doing it. It wasn't till the last 10 years that we started having a method to just plugin drives come standard on PCs. And let me give you a hint, USB is a absolutely shitty method of transfering bulk data. SCSI, Firewire, and now SAS or SATA are all valid. How many PCs do you have that have any of those available externally? Mine has firewire, but up until just the last few years you had to by an addon card to get that. You're just ignorant if you think Macs are 'behind' PCs.
On low end 'grandmother' models it just requires more effort, but its certainly no worse doing it on an iMac than it is on something like the Gateway Profile units or the like which were basically the same thing as an iMac in Dell Black and Square instead of Apple White and Round. You're comparing a Pontiac and Cadillac and pretending they are different in some massive way. They aren't. Your Pontiac really isn't any different than the Mac users Cadillac as far as overall design. Its just the details and quality of components in the design that are different. They still work the same way and can share many many parts.
I remember installing a new drive into a Macintosh LE (I think, maybe it was something else) in highschool to go along with the external 1 gig drive we got. The only thing 'special' we needed was the case spreader, but you could use a flathead for that if you didn't mind nicking the case a little, the spreaders aren't mac specific so finding them wasn't hard. That was in 96. So thats only 13 years ago, but it wasn't exactly 'new' then either.
You're just another typical anti-mac fanboy. You don't know WHY you don't like Macs, you just hate them because ... I don't know why either. Stop spreading crap you heard from some other anti-mac zealot and get a clue on your own next time.
Mac users just don't need to upgrade so it doesn't happen often. They just want to use their PC, not fuck with it all the time like most slashdotters. One of the things most people here rarely get is that the rest of the world just wants to USE their computer, not spend 15 hours a day fucking with the innards or the OS just so they can play Minesweeper or view the photos of the grandkids at Disney World.
As for price, well, again, stop being a cluebie. Pretty much every discussion about Mac has this same crap. Someone spouts 'so expensive!' and someone else spouts back 'compared to what?' and shows a nearly identically priced compariable system from some one else. If you don't understand why Macs are generally 'more' expensive, then you probably also don't unde
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
I don't have to. Intel already has:
http://download.intel.com/support/motherboards/server/sb/enterprise_class_versus_desktop_class_hard_drives_.pdf
Intron: the portion of DNA which expresses nothing useful.
... how they use oxygen-free conductors in their wires to make the bits sound better.
That is all.
In this image : http://db.tidbits.com/tbthumbs/tn10166_System-Profiler-SAS-report.jpg
the drive model is listed as : st380815as n
2 seconds of googling shows this page : http://www.seagate.com/ww/v/index.jsp?vgnextoid=809a4d4b57cb0110VgnVCM100000f5ee0a0aRCRD
Uh... that doesn't look like a server / enterprise class disk to me. It looks like a normal old Seagate disk that Apple want to charge lots for cause it has an Apple sticker on it.