Slashdot Mirror


Big (and Small) Developments In Storage

louismg writes "On the same day that BlueArc released the Titan 2000 family, with performance more than three times higher than rivals EMC and NetApp, including a global namespace and scalability to 512 terabytes, EMC took a low-end approach by unveiling a line for the SMB market, dubbed Insignia. Red Herring claims that BlueArc's announcement changes the storage game, while The Register says that small means beautiful. What makes sense for today's IT infrastructures, with data growth showing no sign of slowing?"

27 comments

  1. 512 terabytes in capacity by wwmedia · · Score: 0, Troll

    The Titan 2000 gives customers the option of consolidating multiple applications with various requirements on the same system and allows a single storage pool of 512 terabytes in capacity.

    thats alot of pRon :)

  2. EMC's product is software, not hardware by TallMatthew · · Score: 2, Insightful
    It sounded like EMC came out with a product in a small footprint, as it was compared to a high capacity product, but not so much. From TFA:

    The Insignia line consists of a basic, low-cost version of the VisualSRM management package, an SMB edition of the eRoom collaboration software, an SMB edition of the RepliStor replication package, and an SMB edition of Storage Administrator for Exchange.

    How can we get more from our customers who already shelled out top dollar for our storage? With a variety of cleverly-named software packages, of course. Ugh, I *despise* EMC.

  3. Commodity Mass Storage by Philip+K+Dickhead · · Score: 2, Funny

    As both SAN and NAS envrionments will proliferate both in density, and diversity. I forsee that, as these become more and more commonplace, we will have at least one locally attached to each computing device in the near future - giving birth to LA-SAN/NAS.

    --
    "Speaking the Truth in times of universal deceit is a revolutionary act." -- George Orwell
    1. Re:Commodity Mass Storage by mwvdlee · · Score: 1

      One step ahead of you: I've invented i-SAN/NAS!
      Integrated SAN/NAS, installed inside the same box as the computer for a zero-external-space formfactor, connected by a low-level abstracted, storage specialized communication protocol over a high-speed wire-electric transmission medium.

      --
      Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
    2. Re:Commodity Mass Storage by kevin.fowler · · Score: 2, Funny

      LA-SAN/NAS

      Is that near Mexico city? I think I've partied there before.

      --
      Bury me in mashed potatoes.
  4. Better organization! by Myself · · Score: 4, Interesting

    More bulk storage only solves half the problem. As the volume of email and document archives grows, organizing and searching them becomes more interesting than just storing them.

    Personally, I'm facing a minor storage crisis with a 15-gig music tree, a 20-gig photos tree, and dozens of gigs of other useful stuff that I only need on an occasional basis.

    When I offload the camera, the files go to the laptop. To avoid duplicating files, the most recent few weeks worth of photos should always be on the laptop. But anything older should move off, to free up space on the drive. I'd like to keep as much of my music in both places as possible, but I need a way to replicate file moves, metadata changes, and deletions so the same bad rips and inaccurate tags don't persist after I fix them once.

    There are a set of notes files, mileage logs, and other small files which I'd like to synchronize between the desktop and laptop. However, there's a possibility of both copies changing simultaneously (or at least between syncs), so I'd need a way to reconcile changes.

    These little dilemmas have me wasting a lot of space until I resolve them. Stale data and useless files are all over the place, and I don't have the notes I need when I need them. It sounds like I might be able to do most of this with rsync, but the notes files, CVS maybe?

    Has anyone tackled these issues before in a useful way, perhaps a sensible-storage-organization HOWTO? More space is not the issue right now, I need more sense.

    1. Re:Better organization! by lipilee · · Score: 1

      closely tying into this is redundancy of data - you might have all the HDD on the world if you don't have a usable distributed fs to use on 'em.

    2. Re:Better organization! by The-Bus · · Score: 1

      There's some data which makes sense to have categorized. Say, work by client or project, music by genre/album/artist, photo by subject or date. But at some point you get to documents which you don't need to update often or plain just don't have their own archiving system. I stick all that stuff into a "Miscellaneous" folder which is then organized by month. It's sort of a giant junk drawer. The difference is that I remember when I worked on something and I can find it pretty easily. Anything else I can just use the OS's search or an indexed-search program like Copernic Desktop Search. It's not ideal, but it works for the time being for anything that can't be categorized.

      As far as a "storage crisis" that can be resolved pretty easily. With $100 you can get 200+ GB hard drives if you know where to look. By the time you fill one or two of those up, 500 GB hard drives will be that much cheaper. Or, just get in the habit of searching for files over 20MB and deleting them if you don't think you'll need them again.

      --

      Small potatoes make the steak look bigger.

    3. Re:Better organization! by jube_fl · · Score: 0, Interesting

      We are trying to over at http://www.storagerevolution.com/. It is a new open source project created by a bunch of people with the same problems. Unlike the SNIA based aperi, it is not vendor controlled, it is totally controlled by users.

      Data management is the largest problem we face, and the only tools to even start managing this problem are for the enterprise market. They are totally out of reach to the small to medium business, in both complexity and price.

      We would appriciate everyones feedback as to what is important to them. This is the most valuable content anyone could deliver at this stage in the process.

      Thanks

    4. Re:Better organization! by AntiDragon · · Score: 1

      Off topic so be warned!

      I feel that half the problem in file management (i.e. search and retreival) lies in metadata. With the right metadata attached to each file, classification, organisation and search becomes far, far easier. The problem lies in who the metadata is generated and stored.

      For example, I'm building a specific CRM/Helpdesk hybrid thingy (technical term!) to replace our current system and massively bloated spreadsheets. Now because it's work based there's no issue in forcing some minor metadata to be supplied each time one of us submits or updates a document. But such an intrusion would not be welcome everytime you upload your photos or save a letter would it?

      Existing stuff like Google Desktop Search, Beagle and Spotlight use file attributes and in certain cases an understanding of a particular file type (PDFs, DOC etc) to build up a metadata database.

      Love 'em or hate 'em (easy choice!) but I was quite excited by MS's WinFS concept when they first mentioned it. That it did away with traditional folder structures and was based purely on metadata. It was a fantastic idea - not exactly new but certainly and idea ready for the desktop. Of course that's flopped spectacurlarly since.

      So where to next? Native metadata support will need new software, new filesystems and new user memes to be successful. But I believe you're right - what good is 20TB of data if you can't find that photo you're looking for?

      --
      "...So I hung back and lurked. For 18 months. Can't beat a good old-fashioned lurking."
    5. Re:Better organization! by Anonymous Coward · · Score: 4, Informative

      You may find FolderShare of some use. It's an ad-hoc peer-to-peer directory synchronisation application.

      You set up one (or more) directories on at least two machines, tell it to keep those directories in sync - and magic happens. There's a central internet server that your machines connect read and store only the file checksums. And if more than one computer is online (either on the same LAN - or anywhere on the internet), files will be copied and the directories syncronised. It's a brilliant idea.

      I personally have several FolderShare shares, including a large folder off my home directory that I keep organised and know that it will be automatically replicated to my other machines for distributed backup. *And* it works well for my MP3 directory and my digital photos.

      It used to be a pay service, but Microsoft bought it and immediately opened the service for free to everyone. I believe their primary intent is for it to be a part of Windows Live when Windows Vista launches. FYI, there are Windows and Mac clients.

    6. Re:Better organization! by zerocool^ · · Score: 1


      How about a document management system like moodle or Microsoft Sharepoint?

      --
      sig?
    7. Re:Better organization! by Jack+Tanner · · Score: 2, Informative

      You are looking for Unison, at least as far as your notes and non-audio/video data.

    8. Re:Better organization! by Anonymous Coward · · Score: 1, Funny
      and dozens of gigs of other useful stuff that I only need on an occasional basis.
      Pr0n, eh?
    9. Re:Better organization! by CastrTroy · · Score: 1

      I think that for personal use, Metadata is a lost cause. Unless you can find someway for the computer to figure out the metadata on it's own (as in CDDB for mp3 tags), Then files that people create aren't going to have very good or any metadata. I think a much better approach is just to keep your stuff organized, but making a good tree of folders. Type of document, project, date, lots of other things can be used to make these folders. Then I find that it's easy to find the stuff you need. You can even use symlinks if you want to put something in 2 folders.

      --

      Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
    10. Re:Better organization! by NoMoreNicksLeft · · Score: 1

      Haha. I'm just going to have a few thousand of my closest friends to manually key in all the metadata, in exchange sharing the content. Of course, only works for specific kinds of content...

    11. Re:Better organization! by CastrTroy · · Score: 1

      Exactly It works with things like mp3s because everyone has the same songs. You can't do that for things like word documents, digital photos, or anything else you are actually creating yourself. If you are the only one or one of few with the document, then it's not very likely that anybody other than you is going to be generating the metadata.

      --

      Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
    12. Re:Better organization! by juan2074 · · Score: 1
      If any user spends the up-front time to create metadata, s/he can take advantage of such a search and storage mechanism.

      For the rest of us, we'll still have the old, slow 'guess the filename, hope you find it somewhere' approach.

    13. Re:Better organization! by Anonymous Coward · · Score: 0

      you can find someway for the computer

      "some way".

      on it's own

      "its".

    14. Re:Better organization! by AntiDragon · · Score: 1

      My fault - I was rambling on.

      Some metadata is transparent. After all MIME/File type is a piece of metadata present in every file. Even if some OSes have to rely on file extensions. Yes, I'm talking to you my monolithic friend....

      I don't think metadata is a lost cause. I think it's the future. The point I was arguing is how to hide it from the user. You're right - the system needs to be better able to supply extra metadata itself. Having MP3s automatically embellished with CDDB info is a fantastic idea. There also needs to be better ways of applying this information. Even the GUI hand holding of OS X seperates out "browsing" from "searching"

      It will be a long time and a dramatic change before manual folder filing and hierachial filesystems get superseded but I still beleive it's worth persuing these other options. Of course, flipside is *we* still need to be in control, not the system.

      --
      "...So I hung back and lurked. For 18 months. Can't beat a good old-fashioned lurking."
  5. HP by Anonymous Coward · · Score: 0

    HP have had products like the MSA20 for some time that compete directly with and are industry tested.

  6. iDisk? by simong · · Score: 1

    As slow as it can be, it seems you want something like iDisk so that you can have anything you want available to you anywhere. I think the issue is at the presentation layer rather than the storage layer, although I wonder what it would take to have roving volumes on a distributed SAN...

  7. how big is that market by Anonymous Coward · · Score: 0

    Look,whether you use a NetApp and are limited to say 192Tb or a Titan at 512Tb is not importatnt for MOST installations. The huge issues is, as it always has been and always will be, BACKUP -- or more properly backup and the retention policy that drives the back up scheme. Half a petabyte of spinning disk doesn't mean much without some data protection.

  8. Who modded that down? by Myself · · Score: 1

    Sounds perfectly on-topic to me... I'm reading some of the documents now and it seems like they're working on solving exactly my problem. It'll take some more research about the group to be sure, but so far I like what these folks are up to. I just hope the downmodder gets metamoderated into extinction...

  9. Shell Scripting by Short+Circuit · · Score: 1

    I wrote a script on my computers to facilitate the transfer of data from remote sources to local trees. Basically, the script accesses a specific directory common to all my media, and moves the data to a timestamp directory somewhere in my homedirectory. I originally wrote it to make working with flash drives easier, as I was frequently downloading stuff at work to use at home.

    I'd have a /transfer directory on a CDROM or flash drive, and all the data in it would be moved to $HOME/transfer/$timestamp whenever I ran the script.

    Why is this relevant? About a year ago, I expanded the script to access $HOME/transfer on my laptop through scp. My only grip was that the throughput was low...ssh encryption makes my laptop CPU a significant bottleneck. If I need to use the script again, I'll probably rewrite it to use rsync.

    If you run Windows on your laptop, you can modify the concept to use a mounted SMB filesystem. All that remains is a script on the laptop to examine the timestamps of all your data and remove anything that's too old.

  10. Both by NorbrookC · · Score: 1

    "What makes sense for today's IT infrastructures, with data growth showing no sign of slowing?"

    Both really. Alright, it's really cool that they can now store up to 512 terabytes. But for many businesses, that storage capacity isn't needed - and won't be for a while. By the time they do need it, they'd be at the point of needing to replace the old system anyways.