Data Storage Leaders Introduce New Wares

louismg writes "Data storage giant EMC announced upgrades to their storage hardware family this morning, and claimed performance increases of 25% to 100%, with increased capacity and disk speeds. This comes two weeks after competitor BlueArc announced Titan, the world's biggest ever NAS box, which claims throughput of 5 Gbps and 256 terabytes in a single hardware file system. How much is enough, and as IT administrators, what is the answer to today's issues - improved hardware, or software?"

Comments

I have some predictions too...

[ivan256]

I predict that the storage industry will continue to produce boring incremental improvements on archaic paradigms untill somebody comes out with something revolutionary. Yes, that was vague and truly deep. Since you probably didn't read the article, here's the spoiler: it's esentially the same thing the author of the story said. Given the history of the industry, you can bet you'll get old and go grey before something revolutionary comes from one of the established players.

Something revolutionary is coming soon though.

Re:I have some predictions too...

[zuzulo]

Adding the dimension of time to data storage as in the link you provide is hardly revolutionary (cf cvs and other version control systems). On the other hand, there are some very interesting developments in distributed file and archival systems.

Some of this work is happening in the academic community (OceanStore, et al) and some is happening in the commercial sector (Avamar, Connected, etc etc).

It seems to me that the storage industry is advancing on two main fronts.

First, hardware is getting better and better at a fairly rapid rate. Storage densities, I/O speeds, hardware based data protection are all improving. This area is generally characterized by incremental improvements like you discuss and is where established players like EMC and other hardware players dominate.

Second, the community is in the process of developing software that attempts to handle (index, search, backup, restore, distribute, etc) the exponential growth in amount of data stored.

The difficult problem between the two is the algorithmic one at this time. This is where revolutionary approaches are needed. And, in fact, there are quite a few folks working in this area. More interesting, perhaps, is the number of efforts that have tried and failed to make significant headway.

I agree that there are likely to be revolutionary changes in the software that we use to interact with data, and sooner rather than later.

Re:I have some predictions too...

[ivan256]

Adding the dimension of time to data storage as in the link you provide is hardly revolutionary (cf cvs and other version control systems).

There have long been snapshoting solutions too, the key diference here is that you can go back to any point in time, and that is truly new. With other version control systems you can only go back to where you manually told it to checkpoint.

As for revolutions in indexing and searching storage, I have yet to see something that's not a new take on an old concept. There are lots of companies with cluster file systems and database filesystems out there. When somebody comes up with one that's more than incrementally better than what we had 20 years ago, and they can make it work, I'd be interested in hearing about it.

Seagate, too!

[morcheeba]

Also today, Seagate launched a family of server-class 2.5" drives sporting 10k rpm and an Ultra320 SCSI or Fibre Channel interface. No details on Seagate's web site yet, though.

Improved backups..

[Sri Ramkrishna]

What they need is improved backups. I don't give a fig about space if I can't back it up. So maybe someone should be looking at how we're supposed to be backing this stuff or archive this stuff. Or are we supposed to keep a warehouse of EMCs around? I can lay a bit that we are going to need serious backup infrastructure than what we have today to keep up.

sri

The price

[dtfinch]

BlueArc appears to charge about $100/gb for storage solutions, and claims that its price is less than its competitors. At first, this looks to me like an insanely high price because my last hard disk cost $0.88/gb. But after some thought to the other hardware involved, I figure I could build an almost equally capable solution for $8-$20/gb, not counting software development costs. But adding the cost of the room to hold it all, plus the insane electrical and air conditioning costs, $100/mb is starting to look fairly reasonable for those who really need what they offer, and need it soon.

Re:BS

[PurpleFloyd]

Your problem is because of Windows (or DOS, if you're even more of a masochist). It will tend to move the file in small chunks, so it goes something like this: read a little bit (maybe a few K) from disk, copy it to memory, seek head to new location, write that tiny amount back to the new location, then go back to the previous location and start over again with a new tiny chunk. As a result, your hard drive's heads are in transit more often than they're reading data, and speeds really suffer. Remember that old versions of DOS and Windows were designed to run on systems with very little memory; this strategy, while slow, also uses very little scratch space.

If you're using Linux and want to copy a lot of stuff from one place to another, you can use dd ('disk dump', designed for moving large files) and specify a blocksize of a few megs; this means that you will be moving data a few megs at a time, rather than a few K at a time - of course, this means that you have to use that much more memory. Also, I would imagine that Cygwin would allow you to use dd under Windows; another option is NTFS, where transfers from one directory to another on a single drive are nearly instantaneous. Of course, then you lose compatability; while FAT variants are understood by almost all OSes, you will have an unpleasant time trying to mount and use an NTFS volume from anything other than Windows. It's all about tradeoffs, but hopefully something here will help.