Auditing Large Unix File Systems?

jstockdale asks: "The recent article on perpendicular recording hard drive technology brought me, as a unix(tm) admin, to reflect on the management of data systems and file servers of capacities >1TB (which exist today and tomorrow will become commonplace). Since Google for once seems useless, what suggestions does the Slashdot crowd have on methods and software to audit changes, visualize file system usage, and in general to determine the qualitative and quantitative nature of the content of large unix file systems?"

Comments

Re:I like treemaps

[cicadia]

A similar program for linux is FSV. In one mode, it gives you a 3D (OpenGL) view of just such a map of your filesystem.

Screenshots

Re:I like treemaps

[hellgate]

> Unfortunately this only runs on Windows, but I'm sure there
> are similar Linux programs available.

kdirstat.
The currently available devel version comes with treemap.

With TB sized installations you will probably want some
additional tools (or at least have kdirstat import a database
built by a daily cronjob [1]), running a complete scan will
take forever.

[1] Some coding involved :-)

Granularity and documentation

[Chagatai]

In the Unix environments in which I've worked (predominantly AIX and Lnux), large disk/file system management comes down to two things: granularity and documentation. First, you have to consider how specific you want to be when laying out your data.

For example, on one recent project on which I worked, a PeopleSoft/Oracle environment was built on a pSeries system. There were instances for every conceivable piece of architecture which led to 50+ file systems. And we're not talking about 50+ file systems off of /, but file systems within file systems within file systems. This was good for separating data but made df an ugly mess.

Conversely, I worked on another project for with a homebrew app designed to track tickets. Rather than using a database the genius who designed the methodology stored every ticket as a 1024-byte file. This caused the system to eat up inodes even though the NBPI was set to 1024, and it caused an additional fun feature: the ls command could not work. With over 200,000 files per file system (all in one directory), ls could not parse in all of the files. The homebrew guy actually had to write an app to crack open the inode table to list the files. When you set up your environment, first consider what degree of granularity you need.

Next, document everything. Consider this situation: an HP Virtual Array with 100 LUNs each cabled to two brocade switches for redundancy going to ten different systems. Would you know just by popping a cable what effects would happen? Documentation is key for managing large disk/file system environments. This also applies to naming file systems, logical volumes, volume groups, and any other pat of your system.