Slashdot Mirror

← Back to Stories (view on slashdot.org)

Ask Slashdot: Smarter Disk Space Monitoring In the Age of Cheap Storage?

Posted by timothy on from the percentages-matter-too-though dept.

relliker writes In the olden days, when monitoring a file system of a few 100 MB, we would be alerted when it topped 90% or more, with 95% a lot of times considered quite critical. Today, however, with a lot of file systems in the Terabyte range, a 90-95% full file system can still have a considerable amount of free space but we still mostly get bugged by the same alerts as in the days of yore when there really isn't a cause for immediate concern. Apart from increasing thresholds and/or starting to monitor actual free space left instead of a percentage, should it be time for monitoring systems to become a bit more intelligent by taking space usage trends and heuristics into account too and only warn about critical usage when projected thresholds are exceeded? I'd like my system to warn me with something like, 'Hey!, you'll be running out of space in a couple of months if you go on like this!' Or is this already the norm and I'm still living in a digital cave? What do you use, on what operating system?

27 of 170 comments (clear)

  1. I delete things when I'm done using them by Anonymous Coward · · Score: 5, Funny

    I never run out of disk space.

    1. Re:I delete things when I'm done using them by bobbied · · Score: 4, Interesting

      I'll bet that's not true...

      Seems to me that the stuff I work on keeps getting bigger and bigger, as does my collection of digital pictures and videos. Where I attempt to pare down what I keep, some of it stays around...

      I expect that most users do the same things and thus data keeps piling up. I don't think it matters how well you are at deleting stuff you don't need anymore.

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    2. Re:I delete things when I'm done using them by dissy · · Score: 4, Interesting

      I delete things when I'm done using them

      1) Many of my things I either desire to use for many years to come (a video download I paid for), or am required to keep to cover my ass (taxes, logs, most data at work due to policies, etc)

      2a) The cost of more storage space is almost always less than the cost of the time to clean up files that could be deleted. In the context of work this does depend heavily on exactly who made the data and their rate of pay / work load - but I've noted the higher up execs and managers tend to be the worst hoarders as well as of course the highest rates of pay. Most of the lower techs on the shop floor don't even have access above read-only to the network storage here, though that is far from universal everywhere.

      2b) Yes there are other people whos time is not as expensive, but no one other than the datas owner/creator can know 100% what needs to stay vs what can go (and sometimes even the owner/creator chooses wrong.)

      3) After deleting/archiving data, the chances of you needing it in the future are typically higher to much higher than the chances you are really done with it.

      4) For the small number of times you really are done with it (like, totally and fur sure), the amount of data that gets deleted is generally such a small percentage of the whole that, while still a good thing to do, doesn't really help much with the problem at hand - freeing up a lot of space for future needs.

      I never run out of disk space.

      You either have too much free storage space, not enough data, or possibly both :P

  2. Performance issues? by brausch · · Score: 3, Insightful

    How does performance change as the big disks approach full? That was always one reason for the rule of thumb about keeping at least 10% free space on UNIX.

    --
    "Almost every wise saying has an opposite one, no less wise, to balance it." - George Santayana
    1. Re:Performance issues? by Anonymous Coward · · Score: 5, Informative

      Well, ext4 strives to scatter files around disk to avoid fragmentation. Once the disk begins to approach full, it has to use even smaller and smaller holes to place data into, which causes some fragmentation.

    2. Re:Performance issues? by __aaclcg7560 · · Score: 2

      You want to keep the hard drive at 50% or less to maximize performance. If the hard drive is more than 50% full, the read/write head takes longer to reach the data. If the hard drive is 90% full, most OSes will have performance issues.

    3. Re:Performance issues? by gnasher719 · · Score: 4, Insightful

      ou want to keep the hard drive at 50% or less to maximize performance. If the hard drive is more than 50% full, the read/write head takes longer to reach the data. If the hard drive is 90% full, most OSes will have performance issues.

      Actually, any OS will have performance issues, because the transfer rate (MB/sec) drops from the outside tracks to the inside tracks. That's why for home use, you just buy the biggest hard drive that you can easily afford (if you need 1TB, you buy 3TB), because that way you use only the parts of the drive with the highest transfer speed, and the average head movement time is also a lot less.

    4. Re:Performance issues? by RenderSeven · · Score: 3, Interesting

      I typically partition the drive into two logical drives. The inner partitions with awful performance are where my media goes (movies, music, photos). The performance falloff is non-linear. Also, performance degradation over time is worse for the inner tracks, so inner tracks are where you put data that is more or less static, or at least written sequentially.

    5. Re:Performance issues? by kuzb · · Score: 4, Insightful

      That's an interesting idea for the budget-minded, but personally I think if performance is actually an issue I'd use SSDs for things that need to be performant, and store everything else on regular drives.

      --
      BeauHD. Worst editor since kdawson.
    6. Re:Performance issues? by NoNonAlphaCharsHere · · Score: 2

      Given a spherical cow of uniform density...

      That isn't how First Fit works. Ever.

    7. Re:Performance issues? by afidel · · Score: 4, Insightful

      Inner tracks have better seek times, which is why high performance applications often "short stroke" drives (ie artificially restrict the percentage of the drive used so that only the inner tracks are utilized, though with modern drives and transparent sector remapping it's unlikely this practices actually works), outer tracks have better streaming performance because more sectors move under the head in a given timeframe.

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    8. Re:Performance issues? by gstoddart · · Score: 2

      Hmmm ... if the goal is to keep all of my disks under 50% to maximize performance ... don't I effectively need twice as much disk? And if it's under RAID I'd need at least 4x as much disk?

      Which kind of defeats the purpose of both having cheaper disk, as well as having monitoring to let me know when it's filling.

      Sorry, but who has the luxury of buying twice as much disk so we can keep them all under 50%??

      What you say might get you a performance boost, but otherwise it doesn't make a lot of sense to me.

      --
      Lost at C:>. Found at C.
    9. Re:Performance issues? by RabidReindeer · · Score: 3, Insightful

      If you use Unix on a server, you should have multiple partitions.

      I use LVM, you insensitive clod!

      Juggling physical partitions is a royal pain.

  3. We have more but we USE more. by pla · · Score: 5, Insightful

    Today, however, with a lot of file systems in the Terabyte range, a 90-95% full file system can still have a considerable amount of free space but we still mostly get bugged by the same alerts as in the days of yore when there really isn't a cause for immediate concern.

    When we had drives in the 100s of MB range, we used a few MB at a time. Now that we have drives in the multi-TB range, we tend to use tens of GB at a time. In my experiences, a 90 percent full drive has as much time left before running out as it did a decade ago.

    Perhaps more importantly, running at 90% of capacity kills your performance if you still use spinning glass platters as your primary storage medium (not so much when talking about a SAN of SSDs). In general, when you hit 90% full, you have problems other than just how long you can last before reaching 100%.

    1. Re:We have more but we USE more. by Vellmont · · Score: 4, Informative

      Exactly. The question is strange (and the attitude of the poster is odd too... 20 years ago is "days of yore", and "olden days"?) Methinks dusting off the word "whippersnapper" might be appropriate here.

      Oddly enough, a similar question fell through a wormhole in the space time continuum from Usenet, circa 1994. "Now that we have massive HDs of 100s of megabytes, and not the dinky little ones of several megabytes from the Reagan era, do we still have to worry about having 95% usage alarms?"

      The truth being, if you got to 95% usage somehow, what makes you think that you're not going to get to 100% sometime soon? Maybe you won't, but you can't know unless you understand how and why your usage increases. That's not going to be solved by a magic algorithm alone, it involves understanding where your data comes from, and who or what is adding to it. This isn't new. The heuristics and usage question, and estimating when action needs to be taken is just as relevant now as it was 20 years ago.

      --
      AccountKiller
    2. Re:We have more but we USE more. by vux984 · · Score: 2

      But you are four years past the safe lifespan of your disk, and when needed, it could fail.

      Hence... backups.

      Hoarding capacity for a decade is as foolish as running out of space tomorrow.

      Hoarding capacity? I don't even really know what that is supposed to mean.

    3. Re:We have more but we USE more. by afidel · · Score: 2

      YOU don't use 10's of GB at a time, but I bet your organization does. My company has expanded their storage by 50% per year compounded for at least the last 10 years (I've been here 8 and I have 2 years of backup reports from before I started), and I don't think we're that unusual if you look at the industry reports for GB shipped per year.

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
  4. Recommend: Hard Drive Sentinel by Bomarc · · Score: 4, Informative

    I install the shareware version of Hard Drive Sentinel on all my Windows systems. It not only will warn you about hard drive usage (%); it will also warn you about errors on the drive -- and in my case I was able to predict that two drives were going to fail (saving data) before they actually failed.

    Their support has been very responsive and courteous, their product can work through (see drives behind) most RAID controllers.

    And no, I don't have any affiliation with HDS.

  5. Whatever is measured is optimized. by QuietLagoon · · Score: 4, Insightful

    ...when there really isn't a cause for immediate concern.

    It all depends what one is concerned about. Is maximizing disk space down to the last possible byte important to you? Or is performance in accessing random data important to you? Or is wanting to keep artificial limits imposed by monitoring systems important to you?

    .
    Once you determine what is actually important to you, then you monitor for that parameter.

    Whatever is measured is optimized.

  6. Re:Bigger question by Bigbutt · · Score: 5, Informative

    It's a configuration option when you newfs a file system. Man newfs or mkfs.

    [John]

    --
    Shit better not happen!
  7. It's all about the data prouction rate by aglider · · Score: 3, Insightful

    You insensitive clod! In the age of MBs, we were producing KBs of data. In the age of GBs we were producing MBs of data. And in the age of TBs we are producing GBs of data. And so on. Thus a 90% full filesystem is as bad as 10 year ago. Unless you are still producing KBs of data.

    --
    Sent as ripples into the electromagnetic field. No single photon has been harmed in the process.
  8. Re:Bigger question by DarkOx · · Score: 2

    I don't know; the default 5% might be excessive for really big volumes but keeping at least %1 free seems 'smart' pretty much no matter how many orders of magnitude the typical volume grows to be. The typical file size has grown with volume size. We now have all kinds of large media files we keep on online storage now that previously would have run off to some other sort of media in short order.

    The entire port of the reservation is so in the event of calamity the super user retains a little free space to work in; if (s)he is going to be able to be able to shuffle things about they might well need what we nominally think of as quite a bit of space. Those things today might be a 100GB VM image or something on 20TB SAN volume for example.

    --
    Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
  9. Synology by krray · · Score: 2, Interesting

    You're living in a digital cave IMHO.
    Don't worry, I was too until recently...

    Always mucked with fast external storage as the "main" solution -- firewire, thunderbolt, etc. This system is the main and had a few externals hooked up, that system had another, another over there for something else. It was a mess all around. How to back it all up??

    Gave them all away -- bought a Synology

    Then bought another (back it up :).

    180-200M/sec throughput is the norm. On the network. Beats out most external drives I've ever come across. Everything ties into / backs up to the array. Home and work now too.

    I use everything but Microsoft products. They're shit.

    My filesystem is 60T w/ under 10T used today. I'll consider plugging in more drives or changing them out in the Synology somewhere between 2017 and 2020...

    1. Re:Synology by nabsltd · · Score: 2

      180-200M/sec throughput is the norm. On the network.

      You have a 10 gigabit network? I ask because a 1 gigabit network can only provide 125MB/sec throughput. I know that some of the Synology units offer link aggregation support, but that also usually requires support in the switch and multiple network cards in each client.

      That said, even 200MB/sec isn't particularly good if you can only provide that total to one client at a time, especially for the cost of a Synology enclosure that can hold enough drives for 60TB of storage.

  10. Check_MK by tweak13 · · Score: 3, Informative

    We switched to Check_MK for monitoring. It's basically a collection of software that sits on top of Nagios.

    The default disk monitoring allows alerting based on trends (full in 24hours, etc.) or thresholds based on a "magic factor." Basically it scales the thresholds so that larger disks alert at a higher percentage, adjustable in quite a few different ways to suit your tastes.

  11. Re:Bigger question by jader3rd · · Score: 2

    Create a large file, that the super user then deletes when the super user needs to fix issues.

  12. Re:They've reset that date from 2005? by dbIII · · Score: 2

    ZFS raidz2 is pretty well RAID6 with an awareness of what is going on with the files in the array giving a variety of improvements (eg. resilver time normally being vastly shorter than a RAID6 rebuild time). A few years of seeing RAID6 in action was ultimately what drove me to ZFS on hardware that's perfectly capable of doing RAID6.
    Anyway, the "raid only has five more years" article keeps on getting warmed up, and keeps getting disproved by the very reasons given for the RAID use by date. Increasing capacity has only been possible by increasing the data density on the disks which means the heads pick up more information - thus faster read and write speeds. Better controllers also made a massive difference. Now dedicating lots of cycles to many cores of fast CPUs (instead of the processors in the controllers) is once again making a massive difference. It's only three hours to do a scrub on a 12 x 1TB 7200rpm drive system here with an i5 CPU and it would take close to the same to resilver a new drive. That is six mirrors so faster than raidz or raidz2, but still, it's not a huge amount of time to replace drives now even though that's bigger than the 500GB or so that was supposed to take forever to rebuild.