Automated Tiered Storage Coming to Desktops?

roj3 writes "Tiered storage has been the scourge of administrators because the vendors tell us to hold meetings with all departments and then classify data to storage tier based on its type or relative importance. eWeek has a story about a new approach to tiered storage — sorting it all by usage patterns. Regularly used data goes on high-performance storage, idle data goes on slower/cheaper storage. Volumes and files even span several types of drives or RAID levels. Is automated tiered storage headed to desktops?"

Comments

Networks, sure.

[celardore]

I can see the usefulness of this technology over a busy network with multiple users and masses of files and storage... I just can't see needing anything more than a mirror&stripe RAID array on a PC with only one user. Even that could be considered excessive.

Re:Networks, sure.

[0racle]

That only makes sense if the people in the household wish to learn how to use what you've mentioned. Since current evidence points to the fact that most people look at computers as a magical box that can not be understood, the chances of them learning how to do a fraction of what you suggest is about as likely as you winning the lottery.

The XP file sharing wizard is too much for a lot of people and you think a raid array sharing up OS images over a network via PXE makes sense?

Re:Networks, sure.

[jwjcmw]

"Life is changing to the digital a bit more evey day. And just as we have cardboard boxes in our attic holding the things we dont use, file cabinets in our office alphabetized, firesafes for important documents, and Safe Deposit boxes for wills. The average home user will need to know and use the digital equivalents."

Or, if you are like many people, you have documents on your desk and in piles on the floor that you will never use, your kids birth certificate is in a stack of papers from when you had to take it to school for registration, your file cabinets have partially labeled folders that are in chronological order...as in the order that you stuffed them in the filing cabinet, your will is in the "to be filed" folder in the bottom of said filing cabinet and you could fill the bathtub with your old phone and electric bills.

Hopefully the digital equivalents will be better for the organizationally challenged.

Great Idea

[Jazz-Masta]

This is exactly what everyone is looking for. People defrag their hard drives in the hopes to increase performance. There is no reason why storage that is accessed more shouldn't be on the high performance drives. Or at least some sort of class rating that defines what storage may need high performance. For example, automatically installing and saving 3D Max to a RAID 0 media, and saving word documents to the lesser-performing drives.

I try to follow this idea all the time with my system. Fast stuff goes on RAID 0, slow stuff, and backup stuff goes on the ole' 200 GB backup drive.

Re:Great Idea

[pla]

This is exactly what everyone is looking for.

No.

You (and a number of other posters on this topic) have described what we look for - Geeks who want to get the most out of their systems with the least expense. If I could get killer performance with a RAID0 of tiny but fast drives (think Raptors, or even Cheetahs if you don't mind dealing with SCSI), while still having the capacity of a cheap 400GB IDE drive - Of course I'd have such a setup (and in fact, many of us already do, we just manually transfer things to/from the big-n'-slow).

Most people, however, do not want this. For starters, most people don't even need the huge drives they already have - If you gave them just the pair of RAID0 36GBs, they'd never use even half that the capacity, so no need for ever moving files to the slow storage. Then failing that, the members of the Sixpack family that manage to store hundreds of GB only fill it with downloaded porn, music, and movies - Uses that really don't need fast drives, just tons of space.


So while it sounds useful in theory - in practice, such a setup would just add cost and complexity without providing any tangible benefit to most users. I suspect even most Geek users would rarely notice the difference (aside from OS load times), and would only make such a setup for bragging rights.

Already have teirs...

[Kaenneth]

Registers, CPU cache, on-chip cache, RAM, local disk, Network/Removable Media, Paper/Human memory...

It's all about feeding that data hungry CPU, as quickly as possible.

Certainly could be done in a desktop

[Don853]

Put two 10k Raptors in Raid 0 for your games and other stuff you need REALLY FAST, and then have a big 250GB 7200RPM drive for everything else. People are doing that already.

All you would need is some software for automatically moving it around. Though most people with desktop rigs like that probably would rather control what is on which drives themselves.

Oh....good..

[JerBear0]

"idle data goes on slower/cheaper storage"

So that special little something that you need once a year, but when you need it, you need it RIGHT NOW is tied to the foot of a pigeon fluttering around the warehouse somewhere. Frequency of use does NOT denote importance.

Re:Oh....good..

[Kadin2048]

Frequency of use doesn't denote importance, but it might denote how quickly you need to be able to recall it. Similarly, importance doesn't imply that quick recall is necessary. If you don't use something frequently, it might be okay to store it somewhere that takes a while to recall from, even if it is "important," as long as you know where it is so that you can get it back.

As an example, financial records for past years might be very important, but you don't need to be able to access them in a tenth of a second. As long as you can get to them if you really want to (sacrificing a few seconds), then it's all right.

The way I see this translating to reality is that you'd keep all your old documents in slow-speed storage, but then keep an index in high-speed storage, so that you could easily search (both by name and by content) and decide when to pull stuff out of your archives.

This is no different than what people have been doing for centuries with paper. Just because the card catalog is located in the center of the library doesn't mean its contents are inherently more valuable than the actual books (which might be in the basement, back shelves, wherever); it just means that the catalog gets accessed much more often.

Actually, in the physical world, people often exchange speed of recall for certainty of recall. You put important documents in a safe-deposit box, rather than your kitchen counter, because even though it'll take you longer to get them out of the box, they're guaranteed to be there when you need them. Likewise, a system which traded off speed for redundancy would probably be appropriate for "important" but infrequently-accessed electronic documents.

No but it is correlated

[davidwr]

Apply "frequency of use = urgency" to BIGNUM pieces of data and you will have a very useful albeit sub-optimal algorithm.

Yes, there are exceptional cases, like the President's access to the Nuclear Briefcase. It hasn't been used for real in a long time if ever but when he needs it it had better be close at hand. However, these special cases can be treated as the special cases they are.

Re:No but it is correlated

[cperciva]

Yes, there are exceptional cases, like the President's access to the Nuclear Briefcase. It hasn't been used for real in a long time if ever but when he needs it it had better be close at hand.

Oddly enough, I think most people in the world would prefer that it wasn't close at hand when Bush decides he wants it.

A better example is fire extinguishers -- most of them will literally never be used, but there's a very good reason to ensure that they are readily available.

This is "new"?

[Medievalist]

IBM mainframes that literally pumped water were doing this decades ago.

What, you say water cooling is coming back too?

It already is

[malraid]

That's why you have HDD with cache. That's the whole concept of "virtual memory". The next step might be hybrid hdds (solid state / mag platters). But I don't think it will go much farther than that. Multiple raids is overkill for the average desktop.

Just read TFA:

[Ant P.]

$50k for a 6TB fileserver? What's that extra $40000 paying for that a normal fileserver loaded with RAM can't do just as fast?

Re:Not so new...

[hotrodman]

  No kidding. So they find a way to put less-used data on slower disks, that still COST NEARLY AS MUCH. The entry price is still listed as $50,000. Big fuckin' deal. Let me know when you take a bunch of garden-variety servers, and do this, with the super cheap clone raid server with 40 terabytes of SATA as the 'last tier' for slowest files, where I can build 100 terabytes for $50,000.

    And yet, managers will get a woody over this buzzword compliance and want to give these guys millions to have the 'latest and greatest'.

    And have it still work with tape, too, and not tied up in some cumbersome, proprietary protocol owned by one little company that could go out business.

I'd love to see it...

[drinkypoo]

...but we can't seem to even get a fucking trashcan right.

I should never have to empty my recycle bin manually, except where I want to perform a security erase - which should be a function delivered with my operating system. This is the height of stupidity.

It's not even a hard problem! There's functions which programs use to check for free space. Lie to them. Don't count files in the recycle bin against the available free space. If you're about to run out of space, delete the least recently used file. Perhaps you might also base things based on total number of accesses, or other criteria, but I believe (perhaps naively) that making the trash can an automatic FIFO from which files are automatically deleted when disk space is low would be about a hundred times better than what we have now.

Also, I want this functionality on all operating systems. Unless I explicitly request deletion, no file should ever be unlinked, deleted, or whatever you call it when I delete it, whether through the command line or the GUI.

This is not hard and it would make everyone a lot happier.