Slashdot Mirror
Automated Tiered Storage Coming to Desktops?
roj3 writes "Tiered storage has been the scourge of administrators because the vendors tell us to hold meetings with all departments and then classify data to storage tier based on its type or relative importance. eWeek has a story about a new approach to tiered storage — sorting it all by usage patterns. Regularly used data goes on high-performance storage, idle data goes on slower/cheaper storage. Volumes and files even span several types of drives or RAID levels. Is automated tiered storage headed to desktops?"
I was using systems that did this 10 years ago. Granted, back then it was disk+tape not different speed disks, but it's the exact same thing.
Looks to me like an excuse to charge 8-10x what you should be paying for storage of that size.
- Adam L. Beberg - The Cosm Project - http://www.mithral.com/
I could see a use for something like this. Personally, I've stopped throwing stuff away. With the exception of temporary and cache files, storage is cheap enough that I just don't delete anything on the off chance that I might want it again. Every email, every instant message, every dictated note (I use a little Olympus digital recorder), every digital photo, it's all saved. By the time I fill up my main hard drive with stuff, I can just buy another one that's probably between two and five times the size, dump everything onto it, and keep the old one as a historical backup. (I keep online backups as well, but I won't bore you with it here.)
I don't think I'm that atypical in this regard. GMail brought the idea of saving all your email, forever, to the masses; Flickr gives you an unlimited amount of photo storage; and technologies like Apple's Spotlight make it relatively easy to search through gigabytes of saved information and pull up related items. What we haven't seen yet is a lot of popular interest in redundant backup systems: that'll come later, once people start realizing how much of their lives they're stored away on the crummy OEM drive in their Dell. (Probably after a lot of them fail and we hear some real horror stories.)
It's not hard to imagine a near future where people just get used to not throwing anything away. In that situation, tiering storage -- allocating the fastest media to the most frequently accessed information -- could have big performance gains. And assuming that you have a relatively static amount of frequently-accessed information, and basically only add information to the "infrequenly accessed" category, a tiered system means that you only really have to add storage to the bottom tier. It's a pyramid where the base gets larger and larger, but the upper part remains basically the same size.
So for example, as you save more and more emails (infrequently accessed information), they automatically get saved onto inexpensive, slower drives, which are then mirrored to each other for redundancy. A single, fast drive could hold the system -- maybe solid state storage? -- and more frequently-accessed data. A smart system would know what information needs to be moved up to faster storage to be very useful (uncompressed digital video, for example, wouldn't be much fun to work with off of a slow drive), and what can be left there as it's accessed (MP3s and compressed video could be played directly from slower media).
I think it's an interesting technology with a lot of possible applications, but as with a lot of other things, it'll be the home user who arrives last to the party, because their storage is the least centralized. Unless there's a move away from storage on individual desktop PCs and towards storage on per-home servers, it'll be a while before most people require or see the benefit in such a thing.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
I think we'll actually see the opposite:
With multiple PCs per household, it makes sense to get rid of the hard drives at the PC level and put them in a RAID enclose that is secured into a wall.
This, however, is a threat to Microsoft because you'll be able to PXE-boot any image of your choice (just think that perhaps your employer or bank supplies their own secure image in order to connect to their resources). Someone needs to get Windows to PXE boot at the hardware level (emulate IDE or something).
This will be huge but we've got to squeeze Microsoft into it, first. Then, everyone will be free to try linux and see what we've all been jabbering about.
More
Apples and pomegranates you compare;
Channels of Fiber come not cheap.
Terabytes 6 with connection of light for less than $50k you will not find.
Terabytes 6 with connections of wire you may.
SATA drives, untested are delivered.
SATA drives with fewer bearings.
SATA drives with short life.
Enterprise storage is not easy.
Also, I want this functionality on all operating systems. Unless I explicitly request deletion, no file should ever be unlinked, deleted, or whatever you call it when I delete it, whether through the command line or the GUI.
The problem with this is, is that it causes a significant reduction in performance.
Ideally, the operating system chose the best possible spot for that file when it got written. Once that file is deleted, that spot will once again be the fastest best possible spot- for at least something. If the operating system skips that spot for a new file, then this new file isn't going to be accessed quite as quickly.
Truly automatic tiered storage solves this problem by splitting the directory services from the storage system- that is, the file's _name_ is no longer tied to the volume that the file happens to live on (and no, this isn't the same thing as symlinks or shortcuts). This allows the decision as to what the best spot for a file is to be deferred until later- and even spanned across multiple volumes!
Unfortunately, such a beast is very difficult- if we make a reduction in our requirements- say that performance isn't very important- or perhaps that we can stop using our computer for a few hours each evening, then it's probably possible. What we need is a new kind of file system that supports either atomic moves between disks, or a filesystem that splits the names from the storage.
A few research projects have been focused on these kinds of changes- but they all tend to break UNIX semantics (Amoeba immediately springs to mind)- and those UNIX semantics are, in-fact, the most widely used and recognized semantics for filesystems anywhere (Even Windows uses them!)-- people who develop a filesystem incapable of supporting them, really need to have a real good reason for breaking everyone's hard work.
While they often do, it hasn't yet been seen as good enough for general purpose stuff.
Back in Mac OS 8 days, I use to use DiskExpress Pro. I had configured it to put the most used files at the outer cylinder (i.e. fasted part) of the drive, and the less used files on the inner cylinders.
The software would analyze file usage, and move them around every day. The anecdotal evidence I have that it worked on such small scale was that my girlfriend later asked me how I got the computer to start responding faster.
I don't know how well this technology would help on newer systems. I suspect at least a little. Perhaps it would really show gains for people who are video editing. Alas, Alsoft never updated their software to work on OS X.
Essentially, these guys are doing what DiskExpress did, but on a larger scale. I have to wonder if they are stepping on any of Alsoft's patents.
Call me a power user, but I do think that people should be at least mirroring their drives. I have heard too many people complain about losing something important because of hard drive failure.
It's not a server, it's a SAN. You connect a server via HBA to the SAN unit. The cost differece is in the performance of the drives you're getting (8 15k 146gb fiber channels and 8 500gb 10k fiber channels), these aren't the same Maxtor 250 GB SATA drives you picked up at Best Buy last week. (then there's the enclosures, controllers, io cards, etc....)