Slashdot Mirror
Google Proposes New Hard Drive Format For Data Centers (thestack.com)
An anonymous reader writes: In a new research paper the VP of Infrastructure at Google argues for hard drive manufacturers and data center provisioners to consider revisions to the current 3.5" form-factor in favour of taller, multi-platter form factors — with the possibility of combining the new format with HDDs of smaller circumference which hold less data but have better seek times. Eric Brewer, also a professor at UC Berkeley, writes "The current 3.5" HDD geometry was adopted for historic reasons – its size inherited from the PC floppy disk. An alternative form factor should yield a better TCO overall. Changing the form factor is a long term process that requires a broad discussion, but we believe it should be considered."
Also, I thought the world was going SSD anyway, which is thinner, not thicker?
I just wonder if, by the time they agree on this (if they do) the price of SSDs will have dropped enough so that they can be used instead? Storage-wise they are already there, and then some.
But they need to convince someone that there is an economic case for making the fucking thing
I find 2.5" preferable for desktops/laptops. But for server farms — I doubt it matters at all what size is practical for PCs.
Multi-platter was always a good idea, I assume it stopped in a desperate attempt to cut costs.
8" hard drives often had 4 or even 8 double sided platters - and SCSI interfaces! Early 5.25" drives often had two, double sided platters. They desperately needed to access more data with less head movement because they had quite low areal bit density and used floppy-derived stepper motors for head positioning!
Sent from my ASR33 using ASCII
Bring back these removable 10 inch disk packs - 1970s tech
http://www.nf6x.net/2014/03/da...
Multiple heads on each side of the platter might be a better solution, one for the inner part and one for the outer.
I have a feeling that in a few years we'll be left with just expensive SSDs and even more expensive "datacenter" drives.
I apologize for the lack of a signature.
There are other form factors other than the typical low profile 3.5".
In particular there is the "half-size" thickness, witch is the thickness of 5.25" bays. It was a rather common form factor for 3.5" SCSI drives.
Realistically, there should be several size formats for different purposes and market segments. Google wants a new format for their purposes. That doesn't mean that it would necessarily be good for the needs of someone else.
I personally wouldn't mind seeing the return of the 5" HDD.
Given the more than doubling of area (capacity) in each platter by going from 3.5" to 5", I could live with higher seek times to have a 16TB HDD taking up one of the 5" bays in my PC case.
Surprised they haven't just gone with 2X or 4X height 2.5" drives. Same connectors, same platters, easy retrofit. You just need a different bracket.
apk has been saying this for years!!! Turns out this guy is brilliant, not a lunatic. -apk
Wow. That was harsh. Too bad you are incorrect. The cost of manufacturing a similar size SSD as a typical datacenter size drive is much much much more. Plus you don't need SSD to get LEED certification. Ridiculous.
We dont have any 3.5" disks in our data center, all smaller SAS disks.
Could be that Google has some inside information that leads them to believe that prices on SSD will not be dropping to acceptable levels any time soon, despite what SSD boosters would have us all believe. If they are proposing something like this, they must have some inkling that spinning platters have a great deal of life left.
Are we still talking about hard drives and seek time? In 1999 Jim Gray said "These calculations suggest the simple rule: Cache web pages if there is any chance they will be re-referenced within their lifetime." Rules of Thumb in Data Engineering
Taller, more heads, smaller platter, less seek distance -- the logical end point is the drum! I'm sure we can do better than the FH-1782 today.
Everything old is new again...
That's one of the weakest papers I have seen from Google.
1) new form factor - this will happen as soon as there are no more 3.5 inch bays outside the servers/data center (any day now, see 3TB 2.5 inch).
2) remove cache from drives - the cache is already getting smaller and smaller as a ratio of capacity, and as they point out it is very effective still. They want larger IOPs/blocks (#9), the drive will need to cache at least a few of those to keep the data path SAS/SATA/new free.
3) it seems they don't know the Seagate 8TB SMR, which already combines SMR with PMR (they call it CMR).
4) already solved by read racing in erasure coding.
5) host managed SMR are available.
6) see #4
7) see #4
8 ) no comment , NDA.
9) See #1
10) See #5
We are not in nineteen-sixties any more. Instead of rw head on a moving hand, we should have full random-access rw disk placed above the platter. We have wafers, we have photolitography in tens of nanometres of resolution, so why are we pretending that we need to do things like they were when disks were concieved? It is not like anyone is going to remove platters from the housing, so that heads need to retract out of the way. We can have a multitude of rw "faces" per each track, eliminating not only the seek time, but also shortening latency till sought sector begins (or conversely, slowing down angular speed for longer lifetime). Besides, there is possibility of not only random, but also non-blocking parallel access!
The more platters they put in the more expensive the hard drive is going to get. These things don't exactly have perfect yields, much like a CPU in which not every core on every processor necessarily makes it through manufacturing in a state that is usable, not every platter makes it through manufacturing. Now increase the number of perfect platters you need and all you are doing is driving up the price of one of the most expensive parts. Far better off going with SSD drives since the quantity will help increase production of the drives and bring down the price for everyone. They have already made it clear that storage density is not their concern but seek times are. Typical VP that doesn't understand how things are made just saying: "put in more of them, that'll make it better!"
+1 for repeating yourself almost exactly. +5 For not being correct. There is more involved in managing data center performance than simple access times or temps. Even with multi-ton cooling, a data center is *still* looking at a lower operating cost to spin hot drives than to use super $$$ high-capacity SSDs. SSDs do not solve any problems for data centers, because individual drive access time is not interesting to a data center. Rather, the performance in $ per KIOPS for the entire array is the real measure of performance.
I'm not sure where you learned all this stuff about data centre design ... but I wouldn't send your kids there either.
Hmm, Google's VP of Infrastructure vs some random on the internet. Wonder who knows more about the economics of data centres.
the cost of manufacturing an SSD is about 25% that of manufacturing a platter HDD
Really? I think if that were anywhere near true it would be reflected in the cost of SSDs. Do tell, where can I buy a 4TB SSD for $30!
The disk drive market is pretty competitive. I tend to think if SSDs cost 25% of an HDD to make, they'd be selling for a lot less than they are. And with Google's buying power probably even less for them.
He probably attended one that had economics as part of the curriculum. You know, SSDs are way too expensive for what he needs.So it doesn't make sense to buy millions of SSDs when spinning drives are much cheaper. He has fucktons of power available. Some of their data centers have hydro power right next to them.
It's true, Google are renowned for hiring only morons. I'm told that at the interview people are asked stuff like, is MongoDB web scale? And I'm sure they only promote the total chumps to VP. All he had to do was post an "Ask Slashdot" and you'd have no doubt politely schooled him. What a wankpuffin he must be!
Old 5.25" half and full height (and 3.5" in ye olde 1.6" form factor) did use a lot of power and generate a lot of heat. But besides modern power supplies being more efficient, he's asking for smaller platters stacked higher, not the large circumference drives of old. The smaller circumference is to reduce seek times, and that reduction of inertia is sure to address power and heat concerns at the same time. I'd be curious as to whether one of these tall, skinny drives would really generate more waste heat than the two or three smaller form factor drives (with two or three more motors and logic boards) it would take to provide the equivalent capacity.
Gamingmuseum.com: Give your 3D accelerator a rest.
I've worked for Google. They used to have a serious vetting process. Now it's more of who you know than what you know.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
" Plus you don't need SSD to get LEED certification."
No, but every single power saving and low-impact device you use brings you closer to attaining LEED certification. Hard Drives, with platinum (not an easy thing to mine and refine, speaking from personal experience owning two mines here in SoCal) you're ripping the shit out of the environment to get it in most cases.
"The cost of manufacturing a similar size SSD as a typical datacenter size drive is much much much more."
Uhh, what? Do you even know the materials used in an HDD vs an SSD (hint, the spinning rust drive uses expensive shit like platinum and helium.) The BOM alone is 1/4 the cost. It's only expensive because it's sold on performance and a 'new technology' versus HDDs.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
The current 3.5" HDD geometry was adopted for historic reasons --- its size inherited from the PC floppy disk.
The form factor of 3.5" floppy drives was decided during the early planning stage of the Great Data Railroad. You can place exactly 16 3.54" (90mm) bare floppy discs side by side within the standard railroad gauge of 4 feet 8.5 inches. For the original 1982 HP single sided format of ~280kB this yields roughly ~4.3mB along every 3.5" of railroad track, or 137 rows along the floor of a a standard 40-foot railroad boxcar without the use of stacking. Thus ~600MB was the capacity of a original single density data railroad car, though it was only only ~1mm in height.
While the floppy disc made data railroads possible, media stacking made them practical. A cylinder of bare floppy media ~10 feet high is roughly 3048 discs, so your standard railroad boxcar held ~1.8TB of floppy storage, in 1982! With an average rail speed of 18mph a single boxcar passes every ~1.5 seconds, which is ~1.2T terabytes or 9200 gigabits per second! By 1998 floppy media storage density had improved ~714-fold, yielding transfer rates of 6568800Gb/s or ~821 TB/s.
So why was floppy data railroad ultimately limited to this 'arbitrary' ~821 TB/s? Northern rail gauge of the US railway based on the English rail system which were based on tramways which used the same jigs used to build wagons whose wheel base was determined by ancient ruts that were left by Roman chariots which were sized to accommodate the width of two horses' asses. As not-quite debunked here.
So the short story is, any chain of decisions regarding technology leads back to some horse's ass.
<blink>down the rabbit hole</blink>
"SSDs do not solve any problems for data centers"
This is patently incorrect and the power bills alone prove it. Go ask any datacenter that switched over to SSD. There's a reason my box uses SSDs (lower requirement for cooling the TWELVE GPUS INSIDE.)
I design these kinds of systems for a living, among other things like owning a mine and mining gems and minerals, designing semiconductors, and much, much more.
"There is more involved in managing data center performance than simple access times or temps."
Most of that unnecessarily complex and useless. Feature creep in the name of "advancement" when nobody ever uses those features excepting maybe 0.000001% of the population.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
"Really? I think if that were anywhere near true it would be reflected in the cost of SSDs."
SSDs command a higher price premium because of A. Marketing B. actual performance and C. perceived 'new technology' to the general mass market, so the prices remain high. This is basic economics 101, man.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
The research paper is not available. Any pointers ?
Really? Go build a 8T HDD and a 8T SSD and compare the manufacturing cost. The manufacturing cost of the 8T HD is about $80.
something like this?
http://www.101101.io/storageen...
I realize this actual device had terrible data density by today's standards but I'm sure they could do something similar with modern platters.
I always believed that the Oval (tm) format had the best reliability and bit density.
CAP === 'excels'
Most of the data within these companies is "cool" meaning it's not actively being accessed. Take for example the massive amount of photos within FB. When was the last time you looked at a photo from 6months ago? If it needs to be accessed frequently, as in it becomes viral, then you move the photo from HDD to SSD. Sure there's 3-4TB SSD coming, but they're still much more expensive $/GB than HDD.
Also, Google's point isn't so much about $/GB but rather that they don't need as much reliability of the drive. Why buy enterprise grade drives when you're making 3+ copies of the data? Consumer grade drives can work for this. If your operational processes are smooth enough, the physical replacement process becomes trivial. Sure there's a lot of them to replace, but it's like replacing books on shelves. So if the cost of a drive can go down even further because it's 99.5 instead of 99.9, even better.
I would think that adding additional platters would greatly lower the mean time before failure on the drives?
The disk's spindle motor and actuator are shared across platters, but the media and read/write heads are per-platter. With many small platters your seek times would go down, but the odds of a head crash or media failure would be greatly increased.
After you've finished circlejerking over SSDs, you might realise that for businesses, they aren't all that over a HDD. The price and the performance are good enough that you can just pick up a box of disks and be reasonably happy for the next 6 years, by which point it's time to do a hardware refresh and you'll end up moving all your data to Azure.
I'm sure the same can be said for datacenters too, as this thing called money isn't unlimited.
Multi r/w heads aren't a new concept. Some of the really old drives had them, and in fact the very original magnetic recording "disks" had a r/w head per track. I think in the trade off of more heads versus faster spinning, faster spinning won out.
I seems that there should be a market for more platters, in a slightly different form factor.
If my maths is correct, eight 2.5" hard drives will fit into the 5.25" full height form factor.
According to Wikipedia, the hypothetical 5.25"FH hard drive can have 4TB x 8 = 32TB capacity. Maybe even 36TB as extra platters could be squeezed in.
Because it is internally 2.5" platters, it should have very quick seek times.
I guess the problem is the MTBF will drop from 1.2 million hours to only 150,000 hours.
If you have 12 GPUs in a box, i fail to see how using a SDD or HDD would make any noticeable difference in power and heat...
I apologize for the lack of a signature.
Any new solution would have to maintain backwards compatibility. The new standard would have to be ether 3.5" x 2, 3, or 4 bays; or 5.25" x 1, 2, 3, or 4 bays. The industry has 30 years behind existing bay standards, it would take them a long time to change their tooling.
Personally I thought the Sun Fire X4500 (a/k/a thumper) was a very efficient way to maximize storage density
they're using the cheapest consumer disks they can find, but now ask for a form-factor tailored for data centers. This does not compute. Why should such data center for factor disks get such consumer use that they can be had for the same low price?
Of course, consumers love everything that's called industrial quality, so maybe it does work out...
Assume a HDD uses 10 watts at $0.08 per kWh. That's $7.00 per year. Allow another $3.00 per year to cool the drive. If the drive needs to last 3 years, the HDD accounts for $30 of electricity over its lifetime. No large capacity SSD is within $30 of a hard drive.
Contribute to civilization: ari.aynrand.org/donate
Oh tell me vise one, how many years experience in HDD manufacturing do you have?
Calling one of the most advanced technologies existing in this world "spinning rust" tells me 0 knowledge which would make you a loud-mouthed know-it-all asshole. The lack of insight in that HDDs is a leading storage technology with several advantages (including economics) over SDDs just reinforce that.
But if your clueless rant makes you feel better that you haven't (and will not) reach a comparable position then I guess it is worth something. Just not for others.
Incorrect is the polite way to express it. Droolingly clueless is a better description of the level of ineptitude displayed.
A 1 Tbyte SDD sells for about $250. The flash IC's that are used in such a drive cost a total of $500 when bought in quantities of 1000. Even with huge quantity discounts, that doesn't leave much profit margin.
The competing hard drive sells for $50.
The idea that a 1 Tbyte SDD can be manufactured for 25% of $50 = $12.50 is beyond absurd.
Contribute to civilization: ari.aynrand.org/donate
So Google wants higher performance conventional hard drives? This should have been proposed around 1995. In today's world this has no chance.
You want performance? Get high performance 15,000 RPM drives from Hitachi or Fujitsu or IBM, or whoever makes them now. Just know that all really high performance drives are going SSD.
Current drives are made to optimize a combination of price, performance, reliability, and the standardized drive bays. Google attempts to revise the priorities so that performance ranks higher in the optimization criteria.
The thing is, does anyone care what Google wants? Are they big enough to get custom runs of HDDs made? Maybe, but I doubt it. Seriously, hard drives are a mature technology. And if performance is really the issue you need to abandon mechanical storage and go all-electronic.
I assign this issue to the That Ship Sailed Long Ago category.
This is not bounded by reality, but just some back of the napkin types stuff.
Let's say you have a 3" platter w/ 1TB capacity. And you can get up to 7 in a 1-inch high 3.5" drive.
That's 7TB.
The spindle is about 1" in diameter, but from looking at the IBM microdrive, it may be possible to reduce that to 0.33"
Next let's shrink the platter to 0.75". Because we're talking single speed, the amount of data is proportional to r (instead of r^2). So it's 0.42/2.5 = 0.168 TB.
The drive is 5.75" deep, Assuming 1.75" for stuff, that leaves 4", assuming there wasn't extra space in the first part, we can fit in 28 platters for a 4.7 TB capacity.
Because the platters are smaller, the access times should be faster. It may even be possible to stick another spindle into the system (more than gaining back lost capacity), and maybe more platters in a stack for higher capacity.
Practical issues: power, heat, complexity. MTBF is probably lower. Non-failure error rates may be higher. Heat and power may be higher. The 0.75" platters may be impractical because of physical reasons. There may be some fundamental problem with having a 4-inch high 0.33" diameter spindle.
He effected a bored affect.
I'm a people person dammit! What the hell is wrong with you guys???
OK lets start thinking about what this means. So if you start making more platters and more heads that raises the cost. It also increases the chance of a head crash. More and more platters and more and more head eventually lead to a platter that doesn't even have to turn and just heads that right and read to a stationary platter.
Hence you essentially have a solid state disk. Why should they spend money trying to improve a dying technology. Solid state disks will eventually take over there too. They should be looking at different forms of solid state disks.
Now they are just going to make it taller and when it fails, will they just dump the whole "tower" of platters? Or are they going to fix it (expensive)? Seems like a method for maximizing waste. This would have made sense if the platters were bigger in diameter, making the pack more reliable as well.
More platters yields more heads. More components to fail. This will increase failure rates for these drives at a given capacity over a similar capacity 3.5" drive with a lower platter count.
Spinning media is still hard to beat on price. Desktop 7200 RPM drives are at $.03/GB. "Enterprise" 7200 RPM SATA at volume is between $.03/GB and $0.05GB. Cheap SSD is around $0.60/GB to $1.20/GB.
A lot of data is still cold. At volume, this price difference matters a lot.
$10k for a ~12TB max of storage (if you used RAID0). I paid ~$800 for 6x3TB WD REDs (used in RAID6), getting the same 12TB. According to WD, the drives use 4.1W each when being accessed. That's ~25W total. Let's say the PSU is 80% efficient, so that's 32W from the wall. Electricity costs $0.12/kWh, so for the $9200 I saved by buying HDDs, I can get 76MWh of electricity, which would be enough to run these drives for 270 years.
Hell, for that money, I can pay all my electricity bills for 3 years.
270 years until SSDs pay off is a bit too long in my opinion. Of course, if you need the performance, then there is no way around it, but for me (infrequently accessed archival storage and backups of other servers), HDDs are enough.
Actually I recently built a ~100TB VOD server using a big RAID10 of hard drives with a couple of PCI-E SSDs (Intel P3700) as cache. The client would not have paid for 100TB of SSDs.
Since he keeps emphasizing how hard drives are made with expensive and environmentally-hostile platinum, perhaps he's just comparing the cost per kilo of platinum to the cost per kilo of sand. After all, that's what flash memory is made with, right?
I don't understand the logic of sacrificing storage capacity for seek time. In which case, you merely end up with an incompetent SSD, an defeat the whole purpose of having a HHD in the first place.
Wouldn't it make more sense to leverage the whole advantage of a HHD and go strictly for capacity, and use more intelligent caching or more hybrid technology to reduce seek time? You can already fit a lot of platters into the 3.5" format, and stuffing more hardware into a single enclosure will probably result in too many other trade-offs to be worth it, like reduced reliability. That's especially true if the new drives are made in such small quantities that economy of scale makes the 3.5" format a better choice in the end, anyway.
Seems like another attempt to force a new format on the market, whether it's needed or not.
Storage drum systems had many heads, arranged in a spiral around the drum so there was time at the end of a "ring" to select the next head. (Apparently nobody thought of making a straight line of heads and spiraling the data.) One of the later models of IBM multi-platter disk drives had 2 sets of head arms. All of these are mechanically complex, which is part of the reason for RAID (Redundant Array of Independent Disks) (redundancy of course being the other). Instead of trying to make one disk drive bigger and/or put more heads on it, make the disk SYSTEM bigger with modular increments of the then-current technology, and get more heads by spreading them across the modules.
There is a tradeoff between system complexity and speed. Highest performance has been attained on specialized systems with intelligent distribution of indices and data, particularly as differentiated by activity, across different levels of storage media; but there is benefit in maintaining a simple generic model of storage, particularly across replicated networked storage systems, so that multi-server and/or back systems need not be identical. One can envision managing active data in many gigabytes of RAM, backing it to SSD, further demoting it to HD as activity decreases; but this involves a lot of system complexity
Remember, though, that the 3330 10-platter drive held a whopping 200 MB, and a controller managing five of them could get all the way to 1 GB - when today I have a 64 GB microSD card in my camera that weighs nothing and runs at 50 MB/sec.
Or else it's just about the sheer quantity of data involved. Even a comparatively small additional cost-per-gigabyte adds up when you're storing all of Youtube plus caching every single website being indexed.
a proposal for something that decreases the reliability of mechanical disks even more. I don't want higher bit rates to deal with in addition to the stream of other unpredictable failure modes associated with these things.