Slashdot Mirror
Vibration Killing Enterprise Disk Performance?
An anonymous reader writes "Is vibration killing disk performance? ZDnet reports on research that a carbon fiber anti-vibration rack increased random read performance by 56% to 246% and random write [performance] by 34% to 88%. Vibration is a known disk problem, but this is one of the few attempts to quantify its impact — which looks to be much greater than suspected."
So maybe sex in the server room IS a good idea! Not to mention all the puns you can make about racks.
Monstar L
Yes, already saw some crazed guy at Sun shout into an array of HDDs and it decreased performance.
http://www.youtube.com/watch?v=tDacjrSCeq4
If you want to have a laugh, check out Bryan Cantrill (Mr. DTrace) and Sun engineer Brendan Gregg shouting at disks and checking out latency heat graphs on YouTube:
http://www.youtube.com/watch?v=tDacjrSCeq4
I wonder how much this could affect the capacity of harddrives
Did anyone else think "how is Kirk going get out of this one"?
http://www.youtube.com/watch?v=tDacjrSCeq4
It's a lot easier to interfere with a moving head arm than it is to mess one up that's locked on a track, so this isn't surprising in the least for vibration to affect reads that require numerous long seeks. I'm surprised it's not worse than they've found.
Moving the head requires accelerated head stepping to top speed, stepping to close to the track, slowing down, stopping at the destination track, waiting for the head to settle, and reading an address block to find out where you managed to land. If you find you missed the track, you have to go through the whole seek process again. (usually only once more, those short adjustment hops are pretty reliable because they're lower speed) But that really hurts your single block read time.
Add to that the fact that the "high performance" drives are making more risky higher speed track changes, which increase the odds of missing your target and make the operation more sensitive to vibration. I've written direct HDD io code before, and sure, you can up the step speed to get very nice seek time boosts, but then you start missing your track and start getting reseeks. Usually you go with the fastest that's acceptably reliable, and that puts you on the bleeding edge of having problems, where things like vibration can run you off the deep end of the bell curve.
It wouldn't surprise me one bit if 50% of the "high performance drive" better speed is due to faster spindle speed, and the other half is faster (riskier) seek speed.
I work for the Department of Redundancy Department.
I'm guessing SSDs will help here.
I'm just curious why SSD is also prone to the problem.
..at SustainIT 2010, Turner had a really good analysis. Still some gaps - figuring out what frequencies hurt the most, and how individual drive types respond to what, is necessary followon. How various vendors' drive units transfer vibration from the rack into the unit, into the drive carrier, into the drive. That sort of thing. Now that the phenomena is identified, a lot more to do on it.
At the least, keep performance sensitive drives away from large sources of environmental vibration, such as your AC unit and so forth.
...a carbon fiber anti-vibration rack...
What is this? Pimp My Rack?; "Next up we'll add a tinted front door, lower the rack by an inch and fit it with diamond-encrusted rack nuts."
that CS disk drives are more sensitive to the vibration from physically coupled adjacent disk drives.
and
The problem is that most civilians don’t understand the problem and are not willing to pay to solve it.
Why should most people care about vibration caused by adjacent drives if most people only have one drive.
The other issue from TFA is that I can't believe a different rack can cause 250% performance improvement, unless you really stacked the deck against steel racks - loose screws, hard drives not properly mounted...? I assume this means that current server racks see I/O rates that are only 40% of what is advertised by manufacturers. Are we expected to believe that no one has noticed this? What about multiple drives in a server. There is no rack separating those drives. This reads like marketing, not real research.
http://www.greenplatformcorp.com/ is the site if you are interested and the "research" is several months old.
i guess the beach boys were wrong about those vibrations.
i have gone to great lengths to isolate my drives from the chassis of my systems to improve noise level (using rubber isolation mounts, shock cords, whatever)
the fact that the drive itself would not have as much mass holding it stationary never really entered into my decision making process... i just checked, and they do produce a substantial amount of vibration during seeks
perhaps i should have done some benchmarks
Seriously? Anyone who has developed airborne systems using COTS hardware has encountered this issue with spinning media.
I am very small, utmostly microscopic.
This is what you get when you drag a magnetic head across a surface. The sooner we get rid of mechanical storage the better. Solids are more robust, more energy efficient, quicker, denser, lighter. Cost and longevity issues are coming along. Yes, lets ditch the antiques already!
When our name is on the back of your car, we're behind you all the way!
I previously had a hard drive in my Mac Pro that was vibrating like crazy; it was making the entire machine vibrate, and all for only 80GB. It wasn't the boot drive, but rather a drive where I just had some random crap.
I decided getting rid of the vibration (and resulting buzzing sound) was more important than having a paltry 80GB more, so I copied the data off and yanked the drive. My machine seemed to boot and run quite a bit faster after wards. I was pleasantly surprised. My previous theory was that the drive was somehow causing a problem on the SATA bus that was slowing down the other drive. After reading this article now I realize the vibration was probably the reason.
That 80GB drive is now serving as a door stop (literally). :)
i'd actually be interested in seeing if performance decreases in a drive due to the magnetic field of a adjacent drive.
So maybe sex in the server room IS a good idea! Not to mention all the puns you can make about racks.
Also, don't shout at your disks since it increases latency, as the guys at Sun Storage learned:
http://www.youtube.com/watch?v=tDacjrSCeq4
Plus a lube job and, definitely, Window cleaning.
This is news we needed 20 years ago. SSD is going to replace mechanical HD over the next couple of years making the whole vibration issue irrelevant.
Ah well better late than never I guess.
Weaselmancer
rediculous.
I've always suspected this, but never really cared to address the issue in any meaningful way because wtf can you do about it? I've always hated metal racks, and all the "affordable" stabilization kits do not do much to transfer the vibration from fans and other moving parts from the servers. In fact, the metal racks make for a nice little conduit to help spread out vibration from server to server. The more full the rack, the more the impact.
Over the course of years, I've had plenty of pci cards shake lose for no apparent reason. People always wonder why parts fail, or come loose over time. I explain "vibration"... and then generally look at me like I have 2 heads. Just because you cant see it moving, doesnt mean it is not. And as far as servers grinding away 24/7, the vibration effect is cumulative, and measurable.
Now I have all the research that I need to run into work on Monday and demand we upgrade to a SSD SAN.... if only costs didnt matter.
20th century Marxism is not progress...
The disks all spin at roughly the same frequency (250Hz for a 15K RPM drive), so you could get some interesting resonance patterns in that frequency band as well as in its harmonics and frequencies that you get when you subtract rotational vibration spectrum of one drive from another. You can even hear these effects if you run two 7200 RPM drives in your desktop in a quiet room (assuming you don't have a dozen fans in the case that some people like to have for some reason).
The solutions is simple - dampen the drives to eliminate high frequency vibration transfer. Better yet, don't use screws to attach your drives at all. Use velcro.
It is going to matter only until price for SolidStateDisks becomes in line with what it should be
My dick is enterprising, and vibrations always increase its performance. Oh, dicks. Sorry. Forget I said anything.
Put identity in the browser.
Yeahhhh. Disks, not dicks. I even previewed.
Put identity in the browser.
I cannae stop the vibrations cap'n. The flux cores are fissioning at 150% beyond their rated nebular rate!
Paging Dr. Freud...
Sam ty sig.
...because the intertial dampeners should really be smoothing these vibrations out!
What about playing Stravinsky? Are HDs really plants in disguise?
That idiot has the loudest data center I have ever heard.
I drive a big rig ( International Pro Star Eagle ) and its a *LOT* quieter than his data center.
The data centers I have been in both have large fans ( low noise, high volume ), and sound dampining on the walls.
The individual racks were bolted to concrete. Duh!
Didnt Compaq and HP used to have rubber mounting brackets?
( Funny! My Capata is 'Idiocy'
Indeed it's not new that we know vibrations can impact disk performances - even a few years back Slashdot pointed out an article on Sun performance probes, and how even screaming at an array of disks under load could visibly impact their performance!
I believe vibration is one of the things that Xiotech has tried address with their ISE bricks.
http://www.xiotech.com/ise-technology.php
I don't work for Xiotech. I just think it's a cool idea.
About the last thing you would want is a light carbon fibre reinforced plastic rack if you want to damp vibrations. A well constrained steel rack with rubber washers would do a far better job because it is far stiffer for a start and you don't want to get much motion.
The answer is to learn from others. With high speed machine tools the trick for damping and stiffness is to use a very heavy base made from materials which damp the vibrations - such as grey cast iron where the damping effect from the graphite is from the portions that are at 90 degrees to what you get with fibre. The idea is to lose the energy of vibration between the poorly bonded graphite sheets and not to transmit it as you would get with a fibre.
..before I suspected something was not right and wrote a program continuously monitoring read speed. Then I started it, waited for the number to settle and put a disk into my CDROM. The reading speed decreased several times. After that I took the cdrom out, put it atop the case on a pile of foam rubber. That computer worked for 10 years more without any problems. The CDROM brand was Pioneer, if you care. HDDs were Maxtor and Western Digital. The story happen in 1997.
This shouldn't be dealt with at the rack level. It should be dealt with at the disk drive mounting bracket level, where it's far easier. There are brackets for that, from several suppliers. There are also rubber grommet kits for fans, to damp vibration from that source.
If you buy servers in bulk, it's something to take up with your supplier. It's the sort of thing that costs only a few dollars per unit at the factory.
Why can't we just query the drive firmware as to how many re-seeks it's doing? Yes I know this is probably not available in the standard command set but the drive must "know" that it's doing lots of re-seeks due to vibration??
Note: I'm not a server guy, i'm not a LAMP guy, I'm not a OS guy nor a DBA. I deal with two objects in the datacenter, storage devices and switches that move packets (either FC or FICON) to storage devices.
First when I heard the term 'enterprise disks' I figured they were talking not about the drives themselves, but rather of the highend EMC, IBM, HDS variety (HP/SUN don't count b/c they rebrand HDS). There's no dampening in those arrays, they're basically racks/cabinets on wheels with casters on them to lock them down. If I was reading from a raw disk, then I could definitely see how vibration would have an impact, but with Enterprise disk arrays, there's so much cache (in the array, not on the drive) and read-ahead algorithms in place that I could see how users wouldn't notice the difference. I'm not so sure that EMC/HDS/IBM would be willing to build their disk arrays out of carbon fiber. Especially with the price conscious consumers like myself that love nothing more than my yearly meeting with my storage vendor to discuss $/GB.
I know of some companies that put their highend (Superdome/p690) servers on earthquake pads, which in the event of an earthquake the server can stay put while the floor shifts underneath.
I've actually experienced this problem first hand. I used to work at one of the above mentioned storage companies and we manufactured a disk shelf that had 8 drives in the front and 8 in the back. There was a metal divider in both the front and back that separated the box into quadrants. We noticed one year that there was a significant drive loss in the field and upon further investigation, we noticed that one slot in particular had an abnormally high failure rate. So we flew to one such site that had these suspect drive shelves, and measured the vibration of each disk in their disk shelves (they had about 100) using a tool that look pretty much like an accelerometer at the end of a pencil. Turns out that the drive location that had the highest number of failures, was not abnormally vibrating, but that a drive 4 spots away was. It seems that if a the drive next to the divider had a "high vibrational drive" it would set up a standing wave which would eventually cause another drive, which was perfectly fine from a manufacturing standpoint to fail.
Sorbothane is a urethane developed specifically to attenuate mechanical vibrations.
It works.
No other kinds of rubber will attenuate vibration nearly as well, and some will even make vibration last longer than it would without the rubber present.
One use for it is tripod foot pads for amateur telescopes. You can go search amateur astronomy forums for user testimony about it. The only people who think that "any kind of rubber will work like Sorbothane" is the people who have not yet tried using Sorbothane.
There don't seem to be any distributors outside of the US, and that's a pity as it's nifty stuff. If you're elsewhere and you need to kill vibration then just pay the international shipping, because nothing else is going to work as well.
~
also i remember reading that on a comparative scale, the hdd read/write head is like a 747 flying at 25m above ground of something insane.
Duh. He'll divert auxiliary power to the inertial dampeners. There is nothing in Star Trek that can't be solved by redirecting power there or with a strategically placed redshirt.
Hmm, now I wonder which of the two do they use when a toilet's clogged. I'm betting it's one of the situations where a redshirt dies. But diverting power to the look has a certain appeal too ;)
A polar bear is a cartesian bear after a coordinate transform.
The actual location of the papers mentioned in the article is in the middle of the link string - trim the two ends.
That's why they upgraded to isolinear chips in later episodes!
... this is very old news.
In the course of every project, it will become necessary to shoot the scientists and begin production.
I'm kind of surprised that something as precision-oriented as (what are now) super-capacity hard drives don't have any accounting for this, when if that's the solution, they should have been right up there with fan controls as a reasonable tweak for overclockers.
So you want acceleration and braking controls for your HDD like you have on your car? Well, the best you are likely to get is the manufacturer's dumb tool that lets you config the HDD for quiet vs. fast operation.
i think you will find them being pcie cards, not 3.5" sata drives, as the drive shape limits chip numbers.
All of the SSD drives that I've seen are 2.5" drives. I've often wondered why no one uses some simple heat sinks and packs the chips as dense as can be into a 3.5" form factor.
I think the limiting factor is the SATA bus. Pack a 3.5-inch drive with NAND and a good controller and you will absolutely swamp a 3Gb SATA bus. When 6Gb SATA becomes mainstream, then you may start seeing 3.5-inch SSDs.
Luckily, all my drives are mounted through silicone vibration isolators. Gotta love Antec case design! :D http://media.giantpachinkomachineofdoom.com/2007-02/blog/images/u1_array_supersize.jpg
Friend: "The NIC is misconfigured..." Me: "No prob, I'll just telnet in and fix it." *Silence*
http://lesswatts.org/tips/disks.php
I'd like to buy homeland for our 10 million people. http://twitter.com/mahadiga
I work for a small ISP and we encountered this recently.
We bought a few SuperMicro small form factor chassis (http://www.supermicro.com/products/chassis/1U/512/SC512-260.cfm), and found that with the drives positioned directly next to a high speed prop, the performance of the disk went from a static 125Mb/sec to as low as a few kb.
The drives we initially bought were WD 1TB Green Drives, and we thought it was initially a "Green" feature. But with thorough testing (and after replacing the drives with Barracudas which suffered, but not as badly), we concluded the fault was singularly because of the vibration.
In the end we packed the prop with foam padding -- between the drive and prop, padding the drive's power cable, and between the prop and chassis (above and below).
Problem went away. But it took us a couple of months, a LOT of back and forth between our supplier, the distributor, and SuperMicro (the latter ignored it), and cost us a bunch more money and time than we had quoted our customer for.
The reason girls and Windows users don't understand UNIX is because all the documentation is in Man files.