Slashdot Mirror
Google Proposes New Hard Drive Format For Data Centers (thestack.com)
An anonymous reader writes: In a new research paper the VP of Infrastructure at Google argues for hard drive manufacturers and data center provisioners to consider revisions to the current 3.5" form-factor in favour of taller, multi-platter form factors — with the possibility of combining the new format with HDDs of smaller circumference which hold less data but have better seek times. Eric Brewer, also a professor at UC Berkeley, writes "The current 3.5" HDD geometry was adopted for historic reasons – its size inherited from the PC floppy disk. An alternative form factor should yield a better TCO overall. Changing the form factor is a long term process that requires a broad discussion, but we believe it should be considered."
Also, I thought the world was going SSD anyway, which is thinner, not thicker?
Multi-platter was always a good idea, I assume it stopped in a desperate attempt to cut costs.
8" hard drives often had 4 or even 8 double sided platters - and SCSI interfaces! Early 5.25" drives often had two, double sided platters. They desperately needed to access more data with less head movement because they had quite low areal bit density and used floppy-derived stepper motors for head positioning!
Sent from my ASR33 using ASCII
I just wonder if, by the time they agree on this (if they do) the price of SSDs will have dropped enough so that they can be used instead? Storage-wise they are already there, and then some.
The point is to keep spinning platters cost-competitive with SSDs - a taller, smaller form factor would increase performance and reduce TCO... I'm thinking they're looking at something like lots of 1.8" platters stacked 4" high, they can spin faster, have faster seek times, and package multiple TB per unit, and I think the longer single bearing should be a more favorable geometry than the ultra-thin notebook compatible drives that have been developed for the last 10 years. It will be slower than SSD, but the power performance (which is the key to TCO) should remain competitive with SSDs for a long time to come. Also, presumably, if this takes off it would be datacenter focused, so longevity (again, TCO focus) should also be "baked into" the design in favor of lower retail price.
Google$ - the economic case for making almost anything.
Multiple heads on each side of the platter might be a better solution, one for the inner part and one for the outer.
I have a feeling that in a few years we'll be left with just expensive SSDs and even more expensive "datacenter" drives.
I apologize for the lack of a signature.
There are other form factors other than the typical low profile 3.5".
In particular there is the "half-size" thickness, witch is the thickness of 5.25" bays. It was a rather common form factor for 3.5" SCSI drives.
Realistically, there should be several size formats for different purposes and market segments. Google wants a new format for their purposes. That doesn't mean that it would necessarily be good for the needs of someone else.
I personally wouldn't mind seeing the return of the 5" HDD.
Given the more than doubling of area (capacity) in each platter by going from 3.5" to 5", I could live with higher seek times to have a 16TB HDD taking up one of the 5" bays in my PC case.
Surprised they haven't just gone with 2X or 4X height 2.5" drives. Same connectors, same platters, easy retrofit. You just need a different bracket.
The power performance will NEVER COME CLOSE to being as low as an SSD.
Even my first-gen SSDs use far less power than a regular laptop drive. Taller drive geometry = more power to spin the spindle. You do know what an ?INDUCTIVE LOAD is, right? If not, protip for you: The amount of power you use to spin up those platters alone is all the power I need to find and transfer data from my SSD. And that's done before your drive heads even begin moving!
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
Wow. That was harsh. Too bad you are incorrect. The cost of manufacturing a similar size SSD as a typical datacenter size drive is much much much more. Plus you don't need SSD to get LEED certification. Ridiculous.
That was my thinking as well, we've already got HTTP4Google (a.k.a. "HTTP 2.0"), TLS4Google (a.k.a "TLS 1.3"), Google phones (Android), Google laptops (Chromebooks), they're proposing Google-optimised hard drives, why don't they get their own planet where they can dictate everything the way they want it.
Spin up? For data centers drives never stop spinning until they die. Data transfer is constant writes, a weak point of SSD.
Could be that Google has some inside information that leads them to believe that prices on SSD will not be dropping to acceptable levels any time soon, despite what SSD boosters would have us all believe. If they are proposing something like this, they must have some inkling that spinning platters have a great deal of life left.
Probably around the same as you could store in a 1U enclosure filled with 2.5" drives. Okay, a bit less, but at least one could seek in reasonable time.
CLI paste? paste.pr0.tips!
Taller, more heads, smaller platter, less seek distance -- the logical end point is the drum! I'm sure we can do better than the FH-1782 today.
Everything old is new again...
Rotational inertia is smaller on platters which have a smaller radius, by a factor of R^2/r^2. It's larger on more platters by a factor of N/n. The same constants of proportionality also apply to data storage density. So actually, taller thinner geometry ~= THE SAME rotational inertia so for the same RPM so it takes THE SAME energy to spin the spindle, given equal storage capacities. (Actually it's a *little* more byte for byte, because the spindle doesn't store data.)
BTW, do you know what an inductive load is? Because that has nothing to do with how much energy it takes to start a disc platter turning.
+1 for repeating yourself almost exactly. +5 For not being correct. There is more involved in managing data center performance than simple access times or temps. Even with multi-ton cooling, a data center is *still* looking at a lower operating cost to spin hot drives than to use super $$$ high-capacity SSDs. SSDs do not solve any problems for data centers, because individual drive access time is not interesting to a data center. Rather, the performance in $ per KIOPS for the entire array is the real measure of performance.
This can be doable, because this is intended for servers, so if drive companies could make the platters, server companies make the hot-pluggable bays, and third party makers make enclosures so the drives can be used as external drives.
I can see a drive using 2.5" platters (so less re-engineering would have to be done by the HDD makers, as opposed to a different diameter), stacking a good number of them in an enclosure. This would allow for some decent sustained I/O, since it could read/write from all those sides at once. Add HAMR and SMR, and these would be quite useful for low tier, read-mostly storage. It would be expected that there would be a very large I/O cache, or an array of SSDs to handle the random reads/writes, because of the nature of these drives.
MTBF, bit rot, and redundancy would be paramount, so hopefully these drives could be designed from the ground up to be more reliable, be it a larger bad sector relocation table, more ECC/parity, drive heads able to take more shock, or so on.
For the consumer, perhaps combine the physical drive with some SSD, so that the flash part works as a landing zone for data, minimizing the amount of random I/O the actual spinning platters have to deal with.
I'm not sure where you learned all this stuff about data centre design ... but I wouldn't send your kids there either.
Hmm, Google's VP of Infrastructure vs some random on the internet. Wonder who knows more about the economics of data centres.
the cost of manufacturing an SSD is about 25% that of manufacturing a platter HDD
Really? I think if that were anywhere near true it would be reflected in the cost of SSDs. Do tell, where can I buy a 4TB SSD for $30!
The disk drive market is pretty competitive. I tend to think if SSDs cost 25% of an HDD to make, they'd be selling for a lot less than they are. And with Google's buying power probably even less for them.
He probably attended one that had economics as part of the curriculum. You know, SSDs are way too expensive for what he needs.So it doesn't make sense to buy millions of SSDs when spinning drives are much cheaper. He has fucktons of power available. Some of their data centers have hydro power right next to them.
It's true, Google are renowned for hiring only morons. I'm told that at the interview people are asked stuff like, is MongoDB web scale? And I'm sure they only promote the total chumps to VP. All he had to do was post an "Ask Slashdot" and you'd have no doubt politely schooled him. What a wankpuffin he must be!
Old 5.25" half and full height (and 3.5" in ye olde 1.6" form factor) did use a lot of power and generate a lot of heat. But besides modern power supplies being more efficient, he's asking for smaller platters stacked higher, not the large circumference drives of old. The smaller circumference is to reduce seek times, and that reduction of inertia is sure to address power and heat concerns at the same time. I'd be curious as to whether one of these tall, skinny drives would really generate more waste heat than the two or three smaller form factor drives (with two or three more motors and logic boards) it would take to provide the equivalent capacity.
Gamingmuseum.com: Give your 3D accelerator a rest.
I've worked for Google. They used to have a serious vetting process. Now it's more of who you know than what you know.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
SSD doesn't hard fail by wear, only reduces size. The SSD could last 20 years and still only have half the sectors marked bad.
" Plus you don't need SSD to get LEED certification."
No, but every single power saving and low-impact device you use brings you closer to attaining LEED certification. Hard Drives, with platinum (not an easy thing to mine and refine, speaking from personal experience owning two mines here in SoCal) you're ripping the shit out of the environment to get it in most cases.
"The cost of manufacturing a similar size SSD as a typical datacenter size drive is much much much more."
Uhh, what? Do you even know the materials used in an HDD vs an SSD (hint, the spinning rust drive uses expensive shit like platinum and helium.) The BOM alone is 1/4 the cost. It's only expensive because it's sold on performance and a 'new technology' versus HDDs.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
The current 3.5" HDD geometry was adopted for historic reasons --- its size inherited from the PC floppy disk.
The form factor of 3.5" floppy drives was decided during the early planning stage of the Great Data Railroad. You can place exactly 16 3.54" (90mm) bare floppy discs side by side within the standard railroad gauge of 4 feet 8.5 inches. For the original 1982 HP single sided format of ~280kB this yields roughly ~4.3mB along every 3.5" of railroad track, or 137 rows along the floor of a a standard 40-foot railroad boxcar without the use of stacking. Thus ~600MB was the capacity of a original single density data railroad car, though it was only only ~1mm in height.
While the floppy disc made data railroads possible, media stacking made them practical. A cylinder of bare floppy media ~10 feet high is roughly 3048 discs, so your standard railroad boxcar held ~1.8TB of floppy storage, in 1982! With an average rail speed of 18mph a single boxcar passes every ~1.5 seconds, which is ~1.2T terabytes or 9200 gigabits per second! By 1998 floppy media storage density had improved ~714-fold, yielding transfer rates of 6568800Gb/s or ~821 TB/s.
So why was floppy data railroad ultimately limited to this 'arbitrary' ~821 TB/s? Northern rail gauge of the US railway based on the English rail system which were based on tramways which used the same jigs used to build wagons whose wheel base was determined by ancient ruts that were left by Roman chariots which were sized to accommodate the width of two horses' asses. As not-quite debunked here.
So the short story is, any chain of decisions regarding technology leads back to some horse's ass.
<blink>down the rabbit hole</blink>
"SSDs do not solve any problems for data centers"
This is patently incorrect and the power bills alone prove it. Go ask any datacenter that switched over to SSD. There's a reason my box uses SSDs (lower requirement for cooling the TWELVE GPUS INSIDE.)
I design these kinds of systems for a living, among other things like owning a mine and mining gems and minerals, designing semiconductors, and much, much more.
"There is more involved in managing data center performance than simple access times or temps."
Most of that unnecessarily complex and useless. Feature creep in the name of "advancement" when nobody ever uses those features excepting maybe 0.000001% of the population.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
"Really? I think if that were anywhere near true it would be reflected in the cost of SSDs."
SSDs command a higher price premium because of A. Marketing B. actual performance and C. perceived 'new technology' to the general mass market, so the prices remain high. This is basic economics 101, man.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
The research paper is not available. Any pointers ?
They could call it "Google Earth"... oh wait...
Really? Go build a 8T HDD and a 8T SSD and compare the manufacturing cost. The manufacturing cost of the 8T HD is about $80.
something like this?
http://www.101101.io/storageen...
I realize this actual device had terrible data density by today's standards but I'm sure they could do something similar with modern platters.
Most of the data within these companies is "cool" meaning it's not actively being accessed. Take for example the massive amount of photos within FB. When was the last time you looked at a photo from 6months ago? If it needs to be accessed frequently, as in it becomes viral, then you move the photo from HDD to SSD. Sure there's 3-4TB SSD coming, but they're still much more expensive $/GB than HDD.
Also, Google's point isn't so much about $/GB but rather that they don't need as much reliability of the drive. Why buy enterprise grade drives when you're making 3+ copies of the data? Consumer grade drives can work for this. If your operational processes are smooth enough, the physical replacement process becomes trivial. Sure there's a lot of them to replace, but it's like replacing books on shelves. So if the cost of a drive can go down even further because it's 99.5 instead of 99.9, even better.
I would think that adding additional platters would greatly lower the mean time before failure on the drives?
The disk's spindle motor and actuator are shared across platters, but the media and read/write heads are per-platter. With many small platters your seek times would go down, but the odds of a head crash or media failure would be greatly increased.
Some drive arrays power down hot spares to save energy (and periodically powering them on to run a drive check). Even though it isn't much energy, keeping platters spinning does take some, and then the HVAC system has to deal with the heat from the platters.
The thing about SSD is that it can store data using zero watts. For dense data centers where power management is essential, this ability is critical, even if SSDs cost more, for saving wattage and showing off a "green" data center. Since SSDs are denser than HDDs, are more shock resistant, and -tend- to have a better MTBF, once cost comes down, there may be no need to bother with a new HDD form factor at all.
Of course, this doesn't say that SSDs are perfect. When they lose data, it tends to be gone for good, as opposed to HDDs where the magnetic domains are present, and can be accessed somehow, barring physical damage from head crashes.
Multi r/w heads aren't a new concept. Some of the really old drives had them, and in fact the very original magnetic recording "disks" had a r/w head per track. I think in the trade off of more heads versus faster spinning, faster spinning won out.
I seems that there should be a market for more platters, in a slightly different form factor.
Someone needs to invent a RW 'bar' that is long enough to go from the spindle to the outmost track. Any point on the bar would be capable of reading or writing. Instead of moving a narrow head on an actuator arm, you just 'activate' (electronically) the part of the bar that is currently over the track you want to access. If cheap enough, you could install a dozen of these bars all around the disk so that you would never need to rotate more than 30 degrees of a circle before a sector you want is under one of the bars. Rotational latency would be very low even on a 2000 rpm drive. Even better would be the ability to activate multiple points on the bar simultaneously so that you could read or write multiple tracks at the same time.
HTTP protocol improvement was on a slow track until Google introduced SPDY, and then that was used as the framework for HTTP/2. Note that Google didn't get everything it wanted--SPDY requires encryption, while HTTP/2 technically does not (though no major client is implementing it without requiring encryption). It also took input from many industry sources; Google may have been the motivator, but was not remotely the final decider.
I'm not sure about the claim behind TLSv1.3, but so what if Google got it started? The independent analyses I've seen suggest that it has substantial improvements (DH required, shorter handshake, some other things) over TLSv1.2, so why not go with it, especially since it's being done as an open standard?
At this point, we have so much legacy protocol cruft with everyone afraid to move forward for fear of breaking things that I'm happy to have Google push the boundaries. They don't win on everything (look at how many projects they've shut down because they don't get the hoped-for uptake), but they're willing to do things that improve the Internet as a whole.
Now, if they could just start working on a replacement for SMTP, maybe we could get some real improvements.
You can never go home again... but I guess you can shop there.
I thought it was a HOSTS file with a built-in ad server.
If you have 12 GPUs in a box, i fail to see how using a SDD or HDD would make any noticeable difference in power and heat...
I apologize for the lack of a signature.
Yet 2 out of my 5 SSDs have failed, and those are reputable brands. The first one developed memory hole that the controller does not see, it is just completely unable to read several sectors (a 64kb piece) and responds with seek error, and has made no attempts at remapping them, the other lost its capacity and is showing like 20mb drive (some kind of firmware update mode), which also has all of it's sectors unreadable.
Any new solution would have to maintain backwards compatibility. The new standard would have to be ether 3.5" x 2, 3, or 4 bays; or 5.25" x 1, 2, 3, or 4 bays. The industry has 30 years behind existing bay standards, it would take them a long time to change their tooling.
Personally I thought the Sun Fire X4500 (a/k/a thumper) was a very efficient way to maximize storage density
Assume a HDD uses 10 watts at $0.08 per kWh. That's $7.00 per year. Allow another $3.00 per year to cool the drive. If the drive needs to last 3 years, the HDD accounts for $30 of electricity over its lifetime. No large capacity SSD is within $30 of a hard drive.
Contribute to civilization: ari.aynrand.org/donate
Oh tell me vise one, how many years experience in HDD manufacturing do you have?
Calling one of the most advanced technologies existing in this world "spinning rust" tells me 0 knowledge which would make you a loud-mouthed know-it-all asshole. The lack of insight in that HDDs is a leading storage technology with several advantages (including economics) over SDDs just reinforce that.
But if your clueless rant makes you feel better that you haven't (and will not) reach a comparable position then I guess it is worth something. Just not for others.
Incorrect is the polite way to express it. Droolingly clueless is a better description of the level of ineptitude displayed.
A 1 Tbyte SDD sells for about $250. The flash IC's that are used in such a drive cost a total of $500 when bought in quantities of 1000. Even with huge quantity discounts, that doesn't leave much profit margin.
The competing hard drive sells for $50.
The idea that a 1 Tbyte SDD can be manufactured for 25% of $50 = $12.50 is beyond absurd.
Contribute to civilization: ari.aynrand.org/donate
This is not bounded by reality, but just some back of the napkin types stuff.
Let's say you have a 3" platter w/ 1TB capacity. And you can get up to 7 in a 1-inch high 3.5" drive.
That's 7TB.
The spindle is about 1" in diameter, but from looking at the IBM microdrive, it may be possible to reduce that to 0.33"
Next let's shrink the platter to 0.75". Because we're talking single speed, the amount of data is proportional to r (instead of r^2). So it's 0.42/2.5 = 0.168 TB.
The drive is 5.75" deep, Assuming 1.75" for stuff, that leaves 4", assuming there wasn't extra space in the first part, we can fit in 28 platters for a 4.7 TB capacity.
Because the platters are smaller, the access times should be faster. It may even be possible to stick another spindle into the system (more than gaining back lost capacity), and maybe more platters in a stack for higher capacity.
Practical issues: power, heat, complexity. MTBF is probably lower. Non-failure error rates may be higher. Heat and power may be higher. The 0.75" platters may be impractical because of physical reasons. There may be some fundamental problem with having a 4-inch high 0.33" diameter spindle.
He effected a bored affect.
I'm a people person dammit! What the hell is wrong with you guys???
Yet 2 out of my 5 SSDs have failed, and those are reputable brands. The first one developed memory hole that the controller does not see, it is just completely unable to read several sectors (a 64kb piece) and responds with seek error, and has made no attempts at remapping them, the other lost its capacity and is showing like 20mb drive (some kind of firmware update mode), which also has all of it's sectors unreadable.
Some of the 1st generation had issues. But spindle drives still continue to fail far more often than spindle
More platters yields more heads. More components to fail. This will increase failure rates for these drives at a given capacity over a similar capacity 3.5" drive with a lower platter count.
Spinning media is still hard to beat on price. Desktop 7200 RPM drives are at $.03/GB. "Enterprise" 7200 RPM SATA at volume is between $.03/GB and $0.05GB. Cheap SSD is around $0.60/GB to $1.20/GB.
A lot of data is still cold. At volume, this price difference matters a lot.
$10k for a ~12TB max of storage (if you used RAID0). I paid ~$800 for 6x3TB WD REDs (used in RAID6), getting the same 12TB. According to WD, the drives use 4.1W each when being accessed. That's ~25W total. Let's say the PSU is 80% efficient, so that's 32W from the wall. Electricity costs $0.12/kWh, so for the $9200 I saved by buying HDDs, I can get 76MWh of electricity, which would be enough to run these drives for 270 years.
Hell, for that money, I can pay all my electricity bills for 3 years.
270 years until SSDs pay off is a bit too long in my opinion. Of course, if you need the performance, then there is no way around it, but for me (infrequently accessed archival storage and backups of other servers), HDDs are enough.
Actually I recently built a ~100TB VOD server using a big RAID10 of hard drives with a couple of PCI-E SSDs (Intel P3700) as cache. The client would not have paid for 100TB of SSDs.
Since he keeps emphasizing how hard drives are made with expensive and environmentally-hostile platinum, perhaps he's just comparing the cost per kilo of platinum to the cost per kilo of sand. After all, that's what flash memory is made with, right?
I don't understand the logic of sacrificing storage capacity for seek time. In which case, you merely end up with an incompetent SSD, an defeat the whole purpose of having a HHD in the first place.
Wouldn't it make more sense to leverage the whole advantage of a HHD and go strictly for capacity, and use more intelligent caching or more hybrid technology to reduce seek time? You can already fit a lot of platters into the 3.5" format, and stuffing more hardware into a single enclosure will probably result in too many other trade-offs to be worth it, like reduced reliability. That's especially true if the new drives are made in such small quantities that economy of scale makes the 3.5" format a better choice in the end, anyway.
Seems like another attempt to force a new format on the market, whether it's needed or not.
Storage drum systems had many heads, arranged in a spiral around the drum so there was time at the end of a "ring" to select the next head. (Apparently nobody thought of making a straight line of heads and spiraling the data.) One of the later models of IBM multi-platter disk drives had 2 sets of head arms. All of these are mechanically complex, which is part of the reason for RAID (Redundant Array of Independent Disks) (redundancy of course being the other). Instead of trying to make one disk drive bigger and/or put more heads on it, make the disk SYSTEM bigger with modular increments of the then-current technology, and get more heads by spreading them across the modules.
There is a tradeoff between system complexity and speed. Highest performance has been attained on specialized systems with intelligent distribution of indices and data, particularly as differentiated by activity, across different levels of storage media; but there is benefit in maintaining a simple generic model of storage, particularly across replicated networked storage systems, so that multi-server and/or back systems need not be identical. One can envision managing active data in many gigabytes of RAM, backing it to SSD, further demoting it to HD as activity decreases; but this involves a lot of system complexity
Remember, though, that the 3330 10-platter drive held a whopping 200 MB, and a controller managing five of them could get all the way to 1 GB - when today I have a 64 GB microSD card in my camera that weighs nothing and runs at 50 MB/sec.
Or else it's just about the sheer quantity of data involved. Even a comparatively small additional cost-per-gigabyte adds up when you're storing all of Youtube plus caching every single website being indexed.
Umm . . . a taller drive geometry in a single package with a single motor (rather than multiple packages) should take less *net* power to spin for the same amount of storage. Make the radius smaller (fewer tracks, less seek time) should lead to less rotational momentum and even less net power. If we're making special data-center drives in vibration mounts, maybe we can talk about magnetic frictionless bearings or other mechanical enhancements. OF COURSE an SSD will always be more efficient - but the cost will still be higher.
a proposal for something that decreases the reliability of mechanical disks even more. I don't want higher bit rates to deal with in addition to the stream of other unpredictable failure modes associated with these things.
My first SSD, which I bought in 2009 as an early adopter, started stuttering horribly within a year. I ended up ripping it out and went back to a HDD (at a significant performance increase over the SSD at that point), which I only replaced this year. I ran some secure wipe on the SSD that was supposed to restore performance. I let it sit on a shelf for a year anyway, and when I tried to finally use it for something also it only lasted a few days before corrupting everything on it. I'd secure wipe it, and it would be fine again for a bit, then shit itself all over again. It finally stopped responding. Other SSDs were not nearly as interesting, as they would just suddenly stop responding. These were all reputable brands too (Samsung, Intel, Crucial). The only brand that hasn't given me trouble is Sandisk.