The Next Decade In Storage

Esther Schindler writes: In this article, Robin Harris predicts what storage will be like in 2025. And, he says, the next 10 years will be the most exciting and explosive in the history of data storage. For instance: "There are several forms of [Resistive RAM], but they all store data by changing the resistance of a memory site, instead of placing electrons in a quantum trap, as flash does. RRAM promises better scaling, fast byte-addressable writes, much greater power efficiency, and thousands of times flash's endurance. RRAM's properties should enable significant architectural leverage, even if it is more costly per bit than flash is today. For example, a fast and high endurance RRAM cache would simplify metadata management while reducing write latency."

Comments

Maybe

[jandrese]

There are a dozen different memory technologies that "in 10 years time" will revolutionize everything. I'll believe it when I see it. Until this, this gets filed away with Bubble RAM and whatnot in the "will be nice if it ever pans out" file.

Re:Maybe

[sabri]

There are a dozen different memory technologies that "in 10 years time" will revolutionize everything.

It doesn't have to be in 10 years time. But did you expect the rise of the All Flash Array 10 years ago?

Legacy spinning disks will be as dead in 10 years as tape is today.

Re:Maybe

[Anonymous Coward]

What planet do you post from that didn't make strides in memory technology over the last 10 years?

Re:Maybe

[The Grim Reefer]

Legacy spinning disks will be as dead in 10 years as tape is today.

I can't tell if you are being sarcastic or serious. But tape is no where near dead as backup media for business. Very few people used tape for home backup. Spinning disks are still in most homes. I don't think spinning disks will be anywhere close to to being as "scarce" as tape is today. For cheap massive online storage, it's still pretty hard to beat. For streaming video, it's a fantastic solution. And with the way resolution for video is going, spinning disks are perfect.

I have a mix of SSD and spinning disks. I probably have around 50TB of spinning disks for video, music, pictures and documents. I also have 7 external spinning disks for backup. I'd be very surprised if this will change much in the next ten years. Unless some new terabyte plus writable bluray replacement comes along, or prices on SSD take a huge nosedive.

Re:Maybe

[Harlequin80]

And I will go on to add that tape is a ridiculously overlooked solution for home backup. You can pick up a second hand LTO3 drive off ebay for under $100 and it will often come with a handful of tapes. Instant easy safe backup solution.

Got photos and camcorder you just cant replace - stick it on a tape and drop a copy at a relatives place. House burns down, full HDD failure / corruption. The tape is sitting there. God I picked up a box of 20 unopened lto3 tapes for $10. I give 1 to the MIL every month and she takes it home. When I go over there I collect all the older tapes that have built up and recycle them.

People will think nothing of spending $100+ on an external Hdd or similar. Or a recurring fee to an online backup solution. But seem to think tape is out of the question...

Re:Maybe

[tlhIngan]

Legacy spinning disks will be as dead in 10 years as tape is today.

Unlikely, because spinning rust still has a cost advantage over semiconductor storage for the same size. I mean, given $500 or so for 8TB, and a 1TB SSD is around $400 or so, that's 3 process nodes worth of improvement (or about 5 years).

Remember, storage capacity of semiconductor memory follows Moore's law. Tricks like multi-bit cells do increase capacity, but it's hard to make it reliable. 3 bits means 8 voltage levels and that is getting to the point where noise can change a bit easily.

And that's the problem with SSDs because hard drives aren't following Moore's law.

Re:Maybe

[boristdog]

Too many of us have "gone to the backup tapes" and found them to be corrupted.

I'm not saying "don't use tape" but I've been burned too many times with only having tape backups (and these were expensive enterprise systems) in the past. If you use tape, also use something else. Belt & suspenders.

Re:Maybe

[Guspaz]

10 years ago? Sure, that's in the ballpark of when consumer SSDs started to become a thing. Intel getting into the game 7 years ago blew it open, but they weren't the first. So ten years ago, you wouldn't have been crazy for following Moore's Law and making a prediction that flash-based storage arrays would eventually make sense.

In terms of legacy disks being as dead in 10 years as tape is today, there are a few problems with that. First is that tape isn't dead, it's still in widespread use in enterprise (it's still the best medium for corporate backups), and second is that there are enough new developments in the works for keep magnetic discs competitive with flash for years.

I'm not saying that it'll be the case forever, but saying that legacy HDDs will be dead in 10 years sounds like a radical prediction.

Re:Maybe

[mlts]

Storage is in tiers, and each tier is different. From the stuff in registers to what is stashed on Amazon Glacier, and everything in between (RAM, SSD, HDD, etc.) A revolution at one strata will have a completely different impact than a revolution at another level.

Take RRAM, MRAM, or some random access memory technology which is up to speed with DRAM, except cheaper and doesn't need refreshed. This would end up not just supplanting RAM, but also making inroads on SSD, depending how inexpensive it is. Will this fundamentally change computing? Somewhat, although I doubt that RRAM would ever drop near the price of HDD or even SSD.

Or, take WAN bandwidth. If the average home had terabytes of bandwidth, a phone had the same, this would change things fundamentally. Cloud storage could go from stashing occasional files to being a tier 2 NAS, especially with proper client security and encryption. However, this is extremely unlikely as well.

Perhaps a tape drive company is able to make reliable media with the bit density of hard disk platters, and is able to fit 100 TB on a cartridge for $10, with drives costing $500. Far-fetched, but if this happens, it would have a different impact to computing than memory costing 1/100 of what it does... but it would be significant.

Improvements in the middle tiers may or may not help things. Bigger hard drives will have to deal with currently small I/O pipes, making array rebuild times longer, and forcing businesses to go past RAID 6 to ensure the drives have protection when things get degraded. Already, some arrays can take 24 hours to rebuild from one lost HDD, and if capacity increases without I/O coming with it, we might have to have RAID levels that factor in not just two levels of parity, but three or four, perhaps with another level just for bit rot checking.

So, when someone says that there are storage breakthroughs... it really depends on the tier that the breakthrough happens at.

Re:Maybe

[Harlequin80]

I do a read back on each tape before I wipe and reuse - and it works out roughly that they are on a 6 month cycle.

So far I haven't had a single read back fail. I don't compress the data and it is only roughly 100gb that I backup but so far I haven't had any issues.

My theory is that if the backup system is too hard it wont get used. I have everything on spinning media any way and usually I have 2-3 taps sitting at the MIL. So a worst case scenario is the most recent one is dead and I lose an extra month. My process is on the first of every month a script runs, wipes the tape in the drive, copies the data across and then ejects it. At some point over the next month I will pull that tape out, stick another in the drive and run my check script which copies it all back to my nas and if it has no errors deletes the directory off stuff its copied and exits. If it errors it emails me and freezes. Simple and fairly idiot proof.

Sure it's not perfect, but it is better than most systems people have.

Re:Maybe

[Anonymous Coward]

If you are a working for ANY organization in the storage role and you have EVER been caught without a backup to ANY file at ANY time.
you fucking suck at your job. And so does your boss. Just put the other tape in from your DR site. Or fucking shoot yourself you incompetant SOB.
On the other hand, if your a slashdotter whose only experience is fucking up a $100,000 tape library for some hick town in west virginia... on weekends...I feel your pain.
anyway, none of this matters, 10 years is too long to make predictions. For the next 5 years however, I predict we will gorge ourselves on whatever 1 year warranty shit Thailand can muster - which won't involve RRAM, or anything else close.

Re:Maybe

[mcrbids]

Get off my lawn, blah blah...

Meanwhile, flash has revolutionized storage. We saw at least a 95% reduction in query times on our DB servers when we switched from RAID5 15K SAS drives to RAID1 flash SSDs. Floppies are history, and 32 GB thumb drives cost $5. SSDs have been catching up to their HDD brethren, now just 2-4 years behind the cost/capacity curve, and spinning rust has just about reached EOL, with Shingled Hard drives that make you choose between write speeds and write capacity being a necessary compromise for increased capacity.

I have no idea why you'd be so dismissive.

Re:Maybe

[lgw]

Too many of us have "gone to the backup tapes" and found them to be corrupted.

1 - That tape was bad when written - verify at least some after write and you're fine.

2 - Old-school home backup drives were total crap: QIC, crap; DAT, crap. But LTO is solid. Not perfect -- do that verify -- but worlds apart from the low-end crap tape that has all vanished.

Re: Maybe

[robi5]

You are the second guy who thinks the gp wrote tape is dead. He wrote hdd will be AS dead AS tape is today. No more, no less. I.e. as a backup media if that (ssd follows moore's law but hdd?).

Re:Maybe

[jedidiah]

The media loves to push this narrative. It's almost as bad as other ideas they feel compelled to shove down everyone's throats.

I'm not convinced. Storage requirements only seem to be escalating while storage technology for the most part doesn't seem to be keeping up. It doesn't matter how much the pundits wish their fantasy were true.

An all flash array is as impractical now as a single SSD drive was 10 years ago (or 15). It's an application limited by it's pricetag.

You can build all sorts of crazy things if you can spend someone else's money freely.

In 10 years, spinning rust won't be nearly as obscure as tape is now.

Re:Maybe

[sabri]

Meanwhile, flash has revolutionized storage. We saw at least a 95% reduction in query times on our DB servers when we switched from RAID5 15K SAS drives to RAID1 flash SSDs.

This, exactly this. HDD will work just fine for your grandparents, while everyone who appreciates performance has moved on to flash.

The increased low latency read speeds combined with data deduplication, compression and instant off-site replication simply can't be matched by legacy spinning drives. It that is technology that is available today. Assuming that RRAM, as mentioned in TFA, becomes a generic technology that replaces flash, you'll have all the advantages of flash without the (very few) disadvantages such as wear.

Re:Maybe

[sabri]

I can't tell if you are being sarcastic or serious. But tape is no where near dead as backup media for business.

I'm serious, but you are right. In the near future, spinning disks will be used for the same applications and seen as the dinosaur of technology: backup and low-performance works.

The truth of the matter is that spinning disks are simply to slow for modern day technology. Compare your laptop when using a 7200rpm disk or an SSD. Compare your Oracle database query times when using a legacy storage vendor or an all flash array that can do 1 million IOPS . It is the performance aspect that matters in modern day computing. The bottleneck is storage, not your CPU, not your memory, storage.

It makes all the difference, especially in transaction-driven enterprises. But sure, for backups you can use spinning disks or tape. Just as our modern cars run on dinosaurs, for every legacy technology, there is still a usecase. I like to run my MSX emulator once in a while :)

Re:Maybe

[sexconker]

Legacy spinning disks will be as dead in 10 years as tape is today.

So, not at all dead? Tape is a live and well, and it is the best medium we have for backups.

Re:Maybe

[Tough Love]

An all flash array is as impractical now as a single SSD drive was 10 years ago (or 15)

Flash drives have been on the market for 35 years. Those drives have always been practical for someone, and flash arrays that are shipping right now are practical for someone. The cost equation for flash arrays is the same, including variables like how much money your application earns or saves by running faster and cooler, and whether your application does not actually need a whole lot of space.

Re:Maybe

[Anonymous Coward]

An all flash array is as impractical now as a single SSD drive was 10 years ago (or 15). It's an application limited by it's pricetag.

All flash is extremely practical for many use cases. My laptop is all flash. At the Enterprise, thanks to dropping prices, smarter algorithms, inline deduplication and compression, it’s a no brainer for lots of cases. Especially VDI.

Re:Maybe

[Anonymous Coward]

I agree with all the flash comments, but why are people linking to Violin? They aren’t where the market is going, they are going to bankruptcy court if anywhere. No links, but companies like EMC (XtremeIO), Pure Storage, UCS Invicta, SolidFire are eating their lunch.

Re:Maybe

[Dutch Gun]

You don't buy spinning disks for their speed. You buy them for their capacity relative to cost. In other words, they're really the only practical solution for online bulk storage on a budget. Different technology - different tradeoffs.

Obligatory car analogy to follow: It's sort of like declaring that semi-trucks are too bulky, slow, and inefficient to be used as modern vehicles. Well, that's true, but only if you're considering the sole use cases to be "commuting" or "family outings", and completely overlooking "bulk commercial transport".

It seems likely that solid-state technology will someday surpass magnetic disks in terms of capacity and cost per GB, but we're nowhere near that point yet. In ten years time? I think it's a reasonable bet that we'll be close to parity by then, at which point we'll probably see the end of disk drives.

Re: Maybe

[PlusFiveTroll]

At one point you spent huge sums of money on memory, or a smaller large pile of money on lots of drives if you were in the moderate sized database world. With SSD you get excellent performance at a cost that ends up being far cheaper than disk per IOP. There are many applications where flash is replacing both memory and disk.

Re:Maybe

[Anonymous Coward]

but why are people linking to Violin? They arenâ(TM)t where the market is going, they are going to bankruptcy court if anywhere. No links, but companies like EMC (XtremeIO), Pure Storage, UCS Invicta, SolidFire are eating their lunch.

I guess Violin ends up top in a google search. From what I can see their new management have done a great job cleaning up the mess left by the previous CEO and have tons of investors backing them up... New products are coming out (like the Concerto) and they beat EMC, Pure and most of their competitors alive when it comes to both performance and price.

Re:Maybe

[Dutch Gun]

Flash is definitely the future, but is still about eight times as expensive per GB. That means if you want to store a lot of data, spinning disks is still the most economical route by far. There's another use-case for disks as well, when a disk is require to do a huge amount of writes - flash is actually very slow to write, and of course, has limited write capacity even with wear-leveling algorithms.

Eventually, of course, flash will reach price parity, and it's unlikely anyone will continue to manufacture spinning disks after that, since the benefits obviously outweigh any drawbacks. The only real question is the time frame, and I'm not sure I'd even hazard a guess.

Re: Maybe

[Dutch Gun]

IOP is not the only metric that must be considered. Spinning disks are still at least eight time cheaper than SSDs *right now*, and in some use cases the speed at which that data is retrieved is not as important as the cost per GB. Of course that's going to change eventually, but until it does, if you want 6TB of local storage you can either buy one 6TB spinning disks at $270 or six 1TB SSDs at about $475 each for $2850 total.

SSDs are simply not as economical for bulk storage (not speed-critical databases) at the moment. Sure, solid state will eventually catch up, but until it does, spinning disks will still be with us.

Re:Maybe

[WMD_88]

I agree with all the flash comments, but why are people linking to Violin?

It's actually one guy linking to Violin. Probably works there.

Re:Maybe

[hairyfeet]

Every time we get an article like this I always remember holodiscs...remember those? A TB or more of storage on a disc the size of a DVD and a buck or less a pop. It was supposed to "revolutionize storage" and was always "right around the corner".....don't hear anything about it anymore, do ya?

Holodiscs, PRAM, phase change memory...all are "in 10 years" or "right around the corner" but until I see an actual affordable unit in my hot little hands I'll consider this like every other pie in the sky storage solution, just a bunch of hot air and empty promises.

Re:Maybe

[hairyfeet]

Nah what you'll have is what I've started doing with my customers, the SSD for speed and the HDD for storage. You get the best of both worlds and as a nice bonus the HDD can have an image of the SSD so when it shits the bed you can be back up and running in less than 20 minutes.

The problem with SSD is that as the wafers shrink? The errors start shooting WAY up. And so far we haven't seen any sign this is gonna change which makes HDDs frankly the safer choice for storage. I'm sure I'll get some anecdotes from the few guys that had a HDD just shit without warning but at the shop I can tell ya that is REALLY rare, most give you either SMART warning or "delayed write failure" errors long before they die, and while SSDs have been getting better they do so through having a lot of spare cells to cover for the dying ones. As they keep shrinking the process this will get nothing but worse.

BTW just FYI but here is a little tidbit I bet most don't think of, ya know how many SSDs manufacturers will brag about how low their return rate is, like that is an example of how long they last? Yeah...the numbers are total bullshit. You see when they die you get ZERO warning and therefor can't wipe your data and since they demand you send your drive back, which means your pictures, documents, CC info, everything on that PC, will end up in the hands of some Chinese refurb center? Well every shop you talk to will advise you to just shitcan the drive rather than risk ID theft so into the trash they go. So you can't really go by the return numbers when it comes to SSDs as most aren't gonna be foolish enough to send it back and risk ID theft for an RMA.

I know I've personally RMAed several HDDs which I managed to keep working long enough to boot&nuke, have yet to send a single SSD back as when they die? "Its dead Jim"

Re:Maybe

[Anonymous Coward]

"I have no idea why you'd be so dismissive."

Cost per GB. Flash is still orders of magnitude more expensive than hard disks. People are pack rats and 3TB drives are $100. I can get a 3TB drive for $100 or I can get ~500GB for 200-300 bucks. That's still 2-3x the cost for 6 times less space.

Re:Maybe

[Kjella]

I'm serious, but you are right. In the near future, spinning disks will be used for the same applications and seen as the dinosaur of technology: backup and low-performance works. The truth of the matter is that spinning disks are simply to slow for modern day technology. Compare your laptop when using a 7200rpm disk or an SSD. Compare your Oracle database query times when using a legacy storage vendor or an all flash array that can do 1 million IOPS . It is the performance aspect that matters in modern day computing. The bottleneck is storage, not your CPU, not your memory, storage.

Except that using a computer to store your family photos is a very valid use case, it doesn't have to be about "computing". When people play Angry Birds they're battery bound, when people stream Netflix they're bandwidth bound, when people play FPS games they're GPU bound, when old people want big screens with big text they're interface bound. And I'm sure there's people doing weather simulations that'll say they're CPU/memory bound. Yes, if you measured the time the average desktop user spent waiting for IO before SSDs it was significant, but I got my first one 5+ years ago so it's getting history.

Today a modern SSD can do 10000 IOPS at QD=1 with a average response time of <0.05 ms. Or in layman's speak, near instant. For an enterprise database sure 1 million IOPS is nice, but would the average user notice the difference if you could put one in his laptop? I think not. Even in the enterprise it's getting to be a replacement for poor optimization and running bad SQL, sometimes I really feel like replacing our dev environment's disks with a single 5400 RPM HDD so maybe we'd get rid of the worst offenses. A lot of people are simply incapable of designing and querying large databases efficiently because table scans worked so well on the small stuff.

Re:Maybe

[drinkypoo]

The truth of the matter is that spinning disks are simply to slow for modern day technology.

What? Who told you that? People who need that much speed can afford multiple disks, and then the speeds go up again. As long as SSD is more than twice as expensive as HDD, there will be a strong market for HDD.

The bottleneck is storage, not your CPU, not your memory, storage.

It doesn't take that many SATA3 HDDs' sustained throughput before you saturate a typical workstation bus. For some few workloads, you are absolutely correct. For most others, you are absolutely incorrect. If we were limited to one storage device in scientific computing, you would be correct, but arrays are not a new technology.

Re: Maybe

[drinkypoo]

At one point you spent huge sums of money on memory,

But now RAM is cheap, so you can have huge amounts of memory without spending huge sums. Big memory caches are still used in many cases.

With SSD you get excellent performance at a cost that ends up being far cheaper than disk per IOP.

It's still a hell of a lot slower than DRAM.

Re:Maybe

[trparky]

I'd have to call bullshit on the "most give you either SMART warning or "delayed write failure" errors long before they die" part. I've had many a drive that was working fine one day, monitoring software showed no signs of pending drive failure, and then... dead the next day. *click* *click* *click* *click* *cry*

You say the problem with SSDs is when wafer shrink, well... it seems that manufacturers have thought about that and have gone back to larger lithograph processes. In fact, Samsung has done just that with their 850 Series SSDs by going to 40nm and 3D-NAND. Not only has Samsung done this but Intel has also been considering SSDs of their own to have 3D-NAND.

Re:Maybe

[trparky]

You're talking about sequential reads. Yes, multiple drives can help in sequential read speeds but 4K Random Read Speeds is what spinning hard drives absolutely suck at. And before you mention that I'm just talking about benchmark numbers, yes... I am talking about benchmark numbers but 4K Random Read benchmark tests very closely mirror real world activity.

You can see this in how the average OS boot-up is slow as shit on an HDD. This is because OS boot-up is pulling seemingly random (at least to the HDD) bits of data from all over the drive which results in having to find the data (seek times) and that's what HDDs absolutely suck at.

Re:Maybe

[trparky]

I beg to differ on the "average user" comment.

Take my brother's notebook with a slow 5400 RPM hard drive and boot-up times of more than three minutes. I put an SSD into it and it took off like a rocket with sub one minute boot-ups. Same thing happened with my desktop. Even launching a simple desktop program such as Microsoft Word can benefit from an SSD. You double-click the icon instead of waiting as the HDD retrieves several different DLLs from all over the drive to load into RAM, the SSD can load it all in one shot and have the program on the screen in less than two seconds.

SSDs have absolutely stunning performance numbers when it comes to retrieving random bits of data from all over the drive and that's what counts, random read speeds. HDDs absolutely suck at random read speeds and it all comes down to seek times, which absolutely suck.

Re:Maybe

[poet]

Yeah... although I agree that SSD is going to slaughter a spindle, part of your problem was the extreme ignorance of running RAID5 for a DB.

Re:Maybe

[drinkypoo]

Yes, multiple drives can help in sequential read speeds but 4K Random Read Speeds is what spinning hard drives absolutely suck at.

Stripes improve seek times, too.

Re:Maybe

[Phishcast]

That's some pretty old-school thinking there. For modern storage arrays with decent caching algorithms the level of RAID protection often has little to do with IO response time and throughput. I've dealt with DBAs who have insisted their archive and redo logs live on RAID-1 or RAID-10 storage because that's what they were taught oh-so-long-ago. They want me to carve out and dedicate four spindles for them to do RAID-1 on a storage array with nearly a thousand spinning disks in it. I've got a storage array with half a terabyte of cache or more, 100% write cache hits and 95+% read cache hits. 4 spindles vs 1000 servicing IO with some extra RAID overhead? Not a difficult call.

Re:Maybe

[hairyfeet]

And I have to call bullshit on your bullshit as the ONLY drives I have seen what you are describing, and frankly I go through more HDDs in a week on average than most folks will see in a couple years, if the frankly notorious "post Maxtor" Seagate 1TB and 1.5TB...let me guess, you got one of those?

Its too bad you don't hang out where the parts hounds do friend, we could have warned ya, just FYI I'd be avoiding any Barracuda 1TB and 1.5TB as the failure rates with those are just nuts. BTW the reason you didn't get a warning? Blame Seagate the skinny is they used a KNOWN BAD controller chip (hence no SMART) on those they got from the Maxtor deal, you see Maxtor got their hands on this dirt cheap ARM controller, we're talking super cheap to manufacture, but the problem is they get flaky as fuck when they get hot and tend to go past the bounds and forget the geometry when their little brains get too warm. Maxtor dealt with this by limiting them to their cheap 5400 RPM drives (you may remember them, DiamondMax? The channel was flooded with them for awhile) but when Seagate bought Maxtor their bean counters looked only at the BOM and how cheap they were to crank out so WHAM! Suddenly they started slapping them in every cheapo drive with predictable results. BTW this is why some were able to be "fixed" with firmware and a few got perfectly working examples, the firmware added checks and some lucky sods got drives with the enterprise ARM chip when they ran low of the cheapos.

But none of that changes the fact that at least with a HDD you can always get your data off (if no other way you can throw together a cheap clean box and swap platters) and as long as you avoid the sucky Seagate models you should always get plenty of heads up. In order of best to worst I would rank 'em Samsung (especially the Ecogreen, in tests I found the 5400 RPM Ecogreen to often match or exceed the Seagate 7200 RPM of same size) followed by Toshiba, Hitachi (despite the buy out they are still making their own drives and they are damned good) WD Green followed by Blue and finally dead last Seagate. If you don't mind a bit of advice? Avoid Seagate above 500Gb and below 2TB, there is a reason why you see all the Tiger kits with either 500Gb Seagates or 1TB WDs and Hitachis,its because they don't want to get their kit builders pissed off by shipping shitty drives. But none of that changes the fact that with HDDs you always have options on getting the data back, if that SSD shits the bed? Its over, its done, you can't even do a controller swap to save the data, its gone forever.

BTW if the drive was clicking? Stick in a ziplock in the freezer for a couple hours, you'd be surprised how much data you can save from even a drive suffering from click death. Just use a sector level copy software that has the ability to ignore errors (I use Paragon but there is plenty of software out there that does the same) and you'd be amazed how much you can get back, I've gotten as much as 85% off drives ticking like a bomb....just FYI but I have yet to recover so much as a single .jpg from a failed SSD, despite their claims that "an SSD will fail into read only mode" in the wild I have yet to see this occur.

Re:Maybe

[trparky]

I was able to recover 95% of the data from the drive after letting the drive cool in a refrigerator (not freezer) so I all that a win for me. But, I'll never trust a Seagate as long as they exist. And yes, a lot of my drives were Seagate drives. Thanks for that bit of info.

Oh, and let me guess.... those SSDs were OCZ SSDs? Right?

Re:Maybe

[Agripa]

1 - That tape was bad when written - verify at least some after write and you're fine.

This is not a solution unless you verify the tape in a different drive and maybe not even then. The second to the last time I used tape backups, the tapes verified fine yet none of them could be read later and when that happened, the drive STILL verified newly written tapes as good even though they were not.

The drive I used after that worked great for years and then became unsupported on the software side. The 4mm and 8mm drives I considered after that were demonstrably junk. I started using optical and hard drive backup after that.

In other news, NSA funds storage technology

[Anonymous Coward]

For example, a fast and high endurance RRAM cache would simplify metadata management while reducing write latency."

NSA guy sees "metadata management" and has a wet dream.

Re:In other news, NSA funds storage technology

[sabri]

NSA guy sees "metadata management" and has a wet dream.

That's not metadata as you think of it. It's the metadata associated with storage.

Re:In other news, NSA funds storage technology

[Em Adespoton]

NSA guy sees "metadata management" and has a wet dream.

That's not metadata as you think of it. It's the metadata associated with storage.

But... it's just metadata, isn't it?

Re:In other news, NSA funds storage technology

[DougOtto]

This isn't a new idea. Sun's QFS did this (as do several other multi reader/writer filesystems) it's mostly locking data.

Re:In other news, NSA funds storage technology

[Anonymous Coward]

Just the tip?

Re:In other news, NSA funds storage technology

[Anonymous Coward]

NSA guy sees "metadata management" and has a wet dream.

That's not metadata as you think of it. It's the metadata associated with storage.

Whoosh.

It's called a joke.

Re:In other news, NSA funds storage technology

[Em Adespoton]

I think you missed my attempt at humour.

umm

[Anonymous Coward]

No.

Oh wait, there wasn't a question in there.
In ten years 1980 storage increased thousand-fold, doubt they gonna do it again. Changing from a HD I can't lift to one that slips in my (coat) pocket is tough to match.

Bubble Memory

[io333]

I want my Bubble Memory. I have been waiting 35 years for it.

Re:Bubble Memory

[Paul Fernhout]

Yeah, I remember the hype back then. I used a Hitachi/GE A4 robot with bubble memory in the 1980s, and like you, I waited for bubble memory to go widespread, but it just seemed to fade away. According to this article, it seems like bubble memory found more industrial & military applications than consumer ones because of the price and power issues:
http://www.dvorak.org/blog/wha...

Re:Bubble Memory

[Tablizer]

I want my Bubble Memory. I have been waiting 35 years for it.

They misread the request and gave us a mortgage bubble instead.

Re:Bubble Memory

[peragrin]

I see your bubble memory and raise you holographic memory.

Re:Bubble Memory

[drinkypoo]

Well, now we have MRAM, which is kinda-similarish and which does the same job exactly. And while we don't have it in our PCs yet, it is being used in consumer-level devices in very tiny quantities. If MRAM ever gets cheap enough to replace storage, then we'll really see a shift.

Technologically maybe...

[Kjella]

Practically I don't feel it's very significantly different anymore. Sure a little faster CPU, a little faster GPU, a little more RAM, bigger and cheaper SSDs but it's mostly the same. To feel that big a difference what you had before must have been rather crap, I still remember how adding a Gravis Ultrasound turned PC sound from shit to excellent. Or adding a new graphics card so you could have transparent, splashing water in Morrowind. Getting a floppy drive for my C64 so I didn't have to wait ages for the tape player. I hereby predict this will be the least exciting decade for storage, except the ones that follow it.

Re:Technologically maybe...

[Harlequin80]

God I remembered that in Morrowind. LOVED that game.

I will go one further though and how you really wanted a 3dfx card in order to play Mechwarrior in all its glory. And then with the C64 an Action Replay card that meant you could instantly resume your game just like a console.

Re:Technologically maybe...

[houghi]

For me it was the CD player that was available for the PC. DUAL SPEED. That and the 56K modem coming from 28.8. I was able to use it at 50K.

Re: Technologically maybe...

[newsdee]

I was the first kid in school to get a CD drive... mono speed... which I brought from a trip to the US. Six months later everybody had 2X because importers skipped directly to the faster version :-D

Re:Technologically maybe...

[sound vision]

Remember when burning a CD took more time than playing it (and maybe took more than 1 disc)?

Re:Technologically maybe...

[drinkypoo]

I will go one further though and how you really wanted a 3dfx card in order to play Mechwarrior in all its glory.

I presume you mean Mechwarrior 2, which is probably the game engine with the most rendering engines EVAR. They had a version of Ghost Bear's Legacy for pretty much every 3d accelerator on the market at the time, and came bundled with most of those, too.

Re:Technologically maybe...

[karnal]

BUFFER UNDERRUN

Had a fairly high end machine at work at the time; pentium pro 200 + all SCSI inside. Seagate 1.2GB drives x2 and a cd burner. Oh how buffer underruns used to piss me off.

Re:Technologically maybe...

[Anonymous Coward]

Yeah, and I was just thinking about how ripping mp3s used to take about twice the length of the track - the other day I was doing a FLAC to mp3 conversion & it was something like 160x on my decidedly mid-range desktop.

Re:Technologically maybe...

[phoenix_rizzen]

Going from an IBM PC-compatible system with a 4 MHz CPU and a Hercules Monochrome graphics chipset (16 shades of amber FTW!) over to a friend's house where he had a dual-speed external CD-ROM playing Wing Commander 3 with FMV was a quantum leap in computing power (I think it was a 486?).

Going from that IBM PC-comptabile system to a Compaq Presario all-in-one with a 486sx2 66 Mhz CPU, VGA graphics, onboard SB16-compatible sound, and a 19.2K modem was the next quantum leap. Using the computer to browse BBSes and talk with people over FIDOnet around the world blew my teenage mind.

Going from a SoundBlaster 16-compatible sound chipset to a Gravis Ultrasound ACE (and all the extra cables that required) in my own 486dx4 133 MHz system was another quantum leap in computing power. Playing MOD trackers and MIDI files off the Internet just blew my mind. A sub-512 KB file that sounded like a full symphony of real instruments? Mind ... blown!

Going from a 19.2 K modem to a K56Flex modem (the non-standard 56.6 Kbps setup) and connecting to a K56Flex modem pool at the local college and hearing those extra beeps at the end, and actually connecting at 53.3 Kbps was mind-boggling. Under 10 minutes to download 1 MB (or something like that)! Web browsing was now a thing!

But storage hasn't really blown me away. Sure, going from dual 5.25" floppies (under a MB of storage) to single 3.5" floppies (over a MB of storage) to CD-R/RW to DVD-R/RW to USB flash stick was interesting, but not mind-boggling. Going from a 40 MB HD to a 20 GB HD to multi-TB HDs is awesome, but not "mind ... blown" territory. Progress has been steady over the past 20 years without any real giant leaps.

About the only thing in storage that has really amazed me is ZFS and how easy it makes managing storage systems in the 10-100 TB range with disks spread across multiple JBOD chassis. But even that was done in a steady progression over the past 7 years or so, without any real giant leaps.

Maybe if MRAM, RRAM, memristors, and all that other non-volatile RAM stuff actually appears, then storage will be existing again. Otherwise, it'll just continue to plod along, slow and steady, with capacities increasing each year, and prices slowly coming down, and speeds increasing slowly. Storage is actually one of the least exciting areas of technology right now.

Re: Technologically maybe...

[dfsmith]

When I got my first CD-ROM drive, it was half speed (75kB/s). Oh yeah! It was awesome—I wish I still had some of those early CD-ROM demos. (The Intel monk/book of the dead one sticks in my mind: it talked to you!)

Re:Technologically maybe...

[Harlequin80]

Probably. Going back a little now in the memory banks. All I can remember is I didn't have a 3dfx card and I really really wanted one! I think I had something like a Matrox 3D card that could almost do it.

storage? for 10years?

[turkeydance]

everything will be out-of-date and not shiny by then. sometimes i don't read so well.

WOM

[Anonymous Coward]

I'm still waiting for the WOM https://en.wikipedia.org/wiki/Write-only_memory_(joke)/ chips I ordered years ago. They were going to solve everything.

yeah, but in 10 years ...

[Anonymous Coward]

Won't we have fusion?

Won't be as big an impact as predicted

[Guspaz]

Two points to make:

1) per-byte accessing doesn't matter for secondary storage, because your filesystem is still going to want to write things in blocks. You'll still want to have logical chunks of data to have checksums for and such.

2) Modern SSDs already do the whole hybrid approach, mixing SLC and MLC/TLC. And I'm not talking enterprise drives, I'm talking the cheapest budget drives. Samsung, for example, calls this "TurboWrite", and they include it in their "EVO" drives, which are some of the lowest cost-per-gig drives on the market. They allocate a small portion of each TLC drive as SLC, and all the writes hit the SLC first. This provides both a nice speed boost (since SLC erases so much faster), and a nice reduction in write amplification (since SLC has an effectively unlimited lifespan from a practical standpoint).

So, replacing that SLC with RRAM would certainly provide a performance improvement, but it wouldn't be a huge difference.

Re:Won't be as big an impact as predicted

[gl4ss]

sure it matters. ...that is if they get it scaled to the promised level of replacing ram and hd with this for cheaper.

surely you would't mind having just 1 tb of what was essentially ram that didn't get wiped if you lost power? but of course, that has to wait until they can produce the thing.. even if that was supposed to be 4 years ago.

Re:Won't be as big an impact as predicted

[Guspaz]

RAM that doesn't get wiped when I lose power? Well, modern operating systems basically simulate that anyhow. Who actually turns off a laptop instead of just closing the lid to sleep it? And even when you do turn something like Windows off, these days it actually just goes to sleep or hibernates in the background. There are also diminishing returns for throwing more RAM at problems, so going from the current, say, 16GB to 1TB isn't going to change much. Loading games, for example, still would take time, because the system still has to decompress stuff. Going from SSDs that can do 100MB/s to SSDs that can do 500MB/s didn't reduce load times by 80%.

Re:Won't be as big an impact as predicted

[drinkypoo]

per-byte accessing doesn't matter for secondary storage, because your filesystem is still going to want to write things in blocks

if we had fast online storage we could write in bytes, someone would surely come up with a densely packed filesystem which made use of that functionality.

There is one last revolution for storage

[Headw1nd]

I feel the time is fast approaching when there is no difference between RAM and storage, and when that happens it will set the stage for a quantum leap in programming. Not only will it eliminate the ever-present need to shuttle back and forth to some slow long term storage media to retrieve this or that, but it will change the assumption that a program being run is inherently different than one that is stored. I believe the fusion of the two will bring about some revolutionary concepts.

Re:There is one last revolution for storage

[PRMan]

This is already the case on most phone apps.

Re:There is one last revolution for storage

[Anonymous Coward]

it will change the assumption that a program being run is inherently different than one that is stored.
...

  I believe the fusion of the two will bring about some revolutionary concepts.

I have been to that future, I have written self modifying code in Perl by seeking back before the __END__. Revolutionary isn't exactly the word I'd use. Additionally, VMS is a thing.

Re:There is one last revolution for storage

[Tailhook]

HP is marketing these ideas as "The Machine." The basic concept is using Re-RAM (ions) for all storage, fiber optics (photons) for all communications and electronics (electrons) for all processing. Ions, photons and electrons in a flattened crossbar matrix. Look up Martin Fink's recent presentations if you need a Buck Rogers fix.

The incredibly small, simple and easy to fabricate cell structure that Re-RAM seems to offer is just too compelling to ignore. Crossbar (the company) appears to be solving the Re-RAM problem. All we're trying to do is move ions around with current. There is a long list of possible materials and designs yet to be investigated. Eventually a sweet spot will be found. When that happens non-volatile storage density and speed will leap forward an order of magnitude, and the whole storage stack from the CPU cache to the tape drive will get flatter.

Or not. It's not like we need this to make the future exciting. Humanoid robots alone will provide more than enough excitement for the rest of my life.

Re:There is one last revolution for storage

[Anonymous Coward]

You've obviously never used a 16-core machine with 200GB of DDR3 RAM with an 8-disk RAID array.

That happens to be my personal development machine.

I can create a VM with 64GB of storage in a RAM disk and still run it with 32GB of actual "RAM". No swapfile anywhere...

It is a bit odd executing 'make -j' and not worrying about how many processes start up...

Turboencabulator.

[toonces33]

Or the Turboencabulator. It sounds neat - whatever it does.

Wake me up when...

[Anonymous Coward]

It has a failure mode other then "the controller barfed and blew away all your data".

I'm still using magnetic media on all my important systems. I can take the performance hit with RAID 5. I have seen far too many SSDs blow themselves away due to a controller issue. I have never seen a mechanical drive fail like that, the data is usually partially readable if something really bad happens (or you can pay someone to swap the platters and read the data if need be- try doing that with flash, where the levelling algorithms are all proprietary).

If any of these technologies are directly accessible or at least partially accessible in the event that the controller fails, count me in. Otherwise, I don't need a way to store 2TB of data that could literally evaporate at any time.

Where's my USB3 LTO6 drive?

[swb]

I like this idea, but:

LTO3 isn't big enough to be worth the headache of 2-3 tapes per TB. LTO5 would be very usable, but requires an annoying SAS card which are just expensive enough to make the whole solution a little spendy.

I'm mostly kidding about USB3, but I sort of wonder why you couldn't have a USB3 interface for a connection medium.

Re:Where's my USB3 LTO6 drive?

[Harlequin80]

400gb uncompressed 800gb compressed has been more than enough for me so far. I don't try to backup everything. I have a huge tv and dvd collection but I'm not worried about that. Those are infrequently synced with friends so if I lost the lot I could get a lot from them and the interwebs will provide the rest.

The bit I can't replace are photos and camcorders and I haven't cracked 100gb yet.

As for the USB - LTO driver are actually pretty hard to feed with enough data fullstop. You heard the tape spin up and spin down if you can't get the data to it fast enough. I don't know what the impacts of lots of idling are but I can't imagine it is good so I think USB will still be a ways off.

Re:Where's my USB3 LTO6 drive?

[swb]

On paper at least, USB3 is supposed to support 625 MBytes/sec while the LTO6 is only capable of 160 MBytes/second, so in theory as a data bus, USB3 should be able to handle it.

IIRC, USB2 was polling driven and run at full rate would cause noticeable spikes in CPU utilization. My guess is that changed in some way for USB3 to support faster data rates, but I don't know how.

I use a USB3 gigabit NIC on my Surface Pro and I don't notice any network performance issues at all.

Isn't there a diminishing return?

[swb]

Most storage vendors make a big deal out of the fact (sometimes with actual data) that a lot of data isn't accessed often enough to warrant spending a premium on the storage medium it sits on and sell products that automatically track and tier blocks based on access frequency and writes.

As long as there is a big price gap between large 10k spinning disks and solid state, won't hybrid arrays still make economic sense? Unless you do something weird, you'll mostly have an all-flash experience but at $/TB prices closer to all disk. In this space, going all flash is just a waste -- 1,000,000 IOPs is great if you need 1,000,000 IOPS, but if you can only do 10k IOPS, isn't it just wasted IOPS?

If reliable SSD comes far enough down in price, dumping all spinning disk might make economic sense even if it isn't 1:1 parity in price because you can then ditch the complex block tracking and tiering systems which add a bunch of cost.

Re:Isn't there a diminishing return?

[Rob_Bryerton]

Your servers may not be able to drive a flash array to 100,000 (or 1,000,000) IOPs, but the array itself can use a good bit of this; most of these all-flash arrays have inline global block deduplication and compression. In a scenario where you have a multi-TB database and Prod/QA/Dev/Test environments, you need a copy of the DB for each environment. With an all-flash array, you need 1 copy of the DB and then leverage snapshots w/no apparent performance hit (all those extra IOPs at work).

Real example: Where I work, we have an environment similar to that described above. On a traditional array, I needed to use ~150 15k FC spindles to meet the space (~50TB) and performance requirements. We're just now replacing that w/an all-flash array which takes all of 6U rack space and has 15TB usable space; we're seeing ~6:1 dedupe/compression across the environments...

Re:Isn't there a diminishing return?

[swb]

I'm not sure dedupe and compression are arguments for all-flash. Both of those things are heavily CPU bound, dedupe requires a block checksum computation and a lookup to find a match and compression takes computation.

My experience with dedupe is limited, but inline dedupe writes take a serious performance hit without relying on massive RAM cache to hold writes. Post-write dedupe makes for faster writes and doesn't really cost I/O because it can be done on a deferred basis when there is low I/O loads, which also helps with CPU consumption. Post-write dedupe can then be done to whatever disk with little penalty since it's background I/O.

I think deduped reads benefit from flash because the nature of deduped blocks is that they're not aligned with any file(s) and require a lot of random I/O -- the file request, a lookup for the deduped blocks that belong to file and then the random I/O of reading the non-aligned deduped blocks from dedupe store and the non-deduped blocks from the actual file.

This obviously would thrash spinning disk, but if you go back to the logic behind most tiered systems with flash disk it is unlikely to be a penalty for frequent reads since that data will already be on flash. Obviously flash/disk sizing ratios matter here, but the hybrid systems I've worked with usually have 1:5 ratios.

I don't see where all flash specifically aids snapshots -- most snapshots just collect write deltas, not complete duplicates of the LUN. Working with a handful of snapshots of a LUN isn't any more performance hit unless you regularly promote a snapshot to a complete volume.

And none of this is an argument against flash -- obviously it adds huge performance gains, my thought is just that in most cases the performance of hybrid flash gives nearly the same experience as all flash, provided that adequate flash tiers are provided and that the controller tiers blocks and not entire files.

Obviously there will be corner cases -- levels of I/O where tiering activity can't keep up, situations where nearly all data is read at about the same frequency where it can't be tiered efficiently, or massive flood writes that overwhelm flash landing zones and force writes to spinning disk. Tiering fails in these cases or is marginal in its effectiveness.

Seconded.

[aussersterne]

For a very long time, tape drives and media gave tape drives and media a bad name.

Consumer QIC — about 1% of tapes actually held any data, total snake oil that took 10 days to "store" 10 megs (immediately unreadable in all cases)
4mm — Tapes good for one pass thru drive; drive good for about 10 tape passes
8mm —Tapes good for maybe 10 passes thru drive; drive good for about 100 tape passes before it starts eating tapes

For all three of the above: Don't bother trying to read a tape on any drive other than the one that wrote it; you won't find data there.

Real QIC —Somewhat more reliable but vulnerable to dust, magnetic fields; drive mechanisms not robust, finicky about door closings

Basically, the only tapes that have ever been any damned good are 1/2 inch or wider and single-reel for storage. Problem is that none of these have ever been particularly affordable at contemporary capacities and they still aren't. Any non-enterprise business should just buy multiple hard drives for their rotating backups and replace the lot of them once a year.

Re:Seconded.

[lgw]

Not quite: AIT was also good - that was Sony's take on 8mm. It was still helical scan with the problems that brings, but it was much cheaper than DLT at the time and just as reliable (since DLT went far downhill before the monopoly broke and LTO happened).

I used DAT professionally too - it wasn't terrible, but you definitely wanted to verify and reading on other drives was somewhat iffy (but worlds better than QIC), and we could re-use a tape a few times. Still, it beat 9-track reel-to-reel.

Metadata management?

[Moskit]

RRAM cache would simplify metadata management

This rings a bell....

NSA stores metadata of millions of web users for up to a year

Now you know who is behind the developement, and why such memory has a chance on the market.

History in perspective

[Anonymous Coward]

History, and an understanding of the context surrounding past discoveries, is very important when you start claiming an upcoming "thing" will be the most significant change in history.

What is the "leap" between 2015 technology and potential 2025 technology? Is it anywhere near the magnitude of change there was between vacuum tubes and transistors? Punch cards/tape to magtape to hard drives?
The "new tech" of today all seems geared toward simply making things happen faster or doing more of it. The earlier storage breakthroughs actually created new abilities. It was simply impractical to store vast quantities of data on paper tape: it was low density, low resolution, slow and fragile. You could not, for instance, store a movie or even a song on paper tape. Magtape enabled storage of new media types.

Today's computing equipment already effectively models and displays in real-time visual representations of the physical world and any other form of data we care to work with. A typical hard drive can store thousands of hours of high-def video, probably every page of text ever printed and it can all be accessed in a fraction of a second.

So let's go with the author's premise: in 2025 storage will be 100 times as fast and 100 times as dense. What "...exciting and explosive..." new abilities will we have with those upgrades? What in 2025 will we be able to do that we simply can't do now with current storage tech? Will weather prediction be 100 times better? Will my movies look 100x as colorful? Will business accounting take 1/100th the time to run payroll?

Explosive data storage

[Anonymous Coward]

Explosive data storage? I'm not sure I'm going to like the future. Better have good backups!

100 GB RAM

[NewYork]

Where can I buy a 100 GB RAM machine?