Slashdot Mirror

← Back to Stories (view on slashdot.org)

Auditing Large Unix File Systems?

Posted by Cliff on from the getting-the-whole-picture dept.

jstockdale asks: "The recent article on perpendicular recording hard drive technology brought me, as a unix(tm) admin, to reflect on the management of data systems and file servers of capacities >1TB (which exist today and tomorrow will become commonplace). Since Google for once seems useless, what suggestions does the Slashdot crowd have on methods and software to audit changes, visualize file system usage, and in general to determine the qualitative and quantitative nature of the content of large unix file systems?"

3 of 21 comments (clear)

  1. I like treemaps by Krellan · · Score: 5, Informative

    I like the idea of treemaps.


    http://www.cs.umd.edu/hcil/treemap-history/index .s html


    Hehe, it was originally made to see what was taking up all the room on an 80MB hard disk :)

    There's various software available based on this concept, most working like "du", except that you get the results graphically. You typically see a large picture on screen of what directories and files are taking up the most space. It looks like a piece of Mondrian artwork, with the size of rectangles corresponding to the size of space taken, so it is easy at a glance to see what is hogging all of the disk space. It can be drilled down, of course, by clicking to zoom in.

    A quick Google search revealed SequoiaView:


    http://www.win.tue.nl/sequoiaview/


    Unfortunately this only runs on Windows, but I'm sure there are similar Linux programs available.

    --

    Dr. Demento On The 'Net!

  2. Tivoli by keesh · · Score: 2, Informative

    produce a load of products that do all that kinda thing for you. Expensive, but then you could always employ a bunch of monkeys, erm, summer students to do the same stuff manually...

  3. I am facing the same problem... by Desmoden · · Score: 4, Informative


    I have a 8.8TB (raw) EMC Symm here, and another one in Austin TX. Then I have another 600GB or so in Sun jbod. Most hosts are connected to the EMC over a 2Gb SAN using brocades.

    I just got this environment a couple weeks ago, and there was NO documentation. So figuring out disk usage over 15 systems has been a nightmare.

    As much as I hate to admit it, EMC's ECC and Storage Scope have been a huge help. I could have done it using Veritas as well, but the EMC tools are nice.

    And I'm soon going to add another 2TB SATA array over iSCSI so then we'll see if ECC can really manage "other peoples disk" :)

    But welcome to the new exciting field of Storage/SAN architect/admin. With arrays from HDS and EMC coming in the 46TB flavor and more, resorce management is a big job.

    For instance my counterpart (he handles windows, I handle *nix) found that we had 1.2TB of unused BCV's (it's a 3rd mirror, weird EMC'izm) that had never been used!

    So it's all about documentation. And do yourself a huge favor, and come up with a clean scalable disk/lun/volume/mountpoint naming convention. This can be so critical. Not sure what other people do, but I'll have something like:

    host 1: dg001-dg005
    host 2: dg006-dg010
    host 3: dg011-dg015

    the disks are enclosure_name_id_lun#

    volumes are v{dg}{v#} so first volume for disk group dg006 would be v1601.

    Then mount points are based on either oracle{SID} or port, or app name or something logical.

    Keep everything unique! So then you can move luns form host to host without having issues. This also allows you to generate usage reports and know that if v3206 is at 68% exactly what host, disk group, volume, and app are involved.

    Ug, sorry, went off, but I'm in this mess right now myself and so I have some very strong feelings about it right now =)