Slashdot Mirror
Microsoft Redesigns chkdsk For Windows 8, Improves NTFS Health Model
MojoKid writes "Microsoft can't do anything to magically make hard drives stop failing when parts go bad, but Redmond is rolling out a new NTFS health model for Windows 8 with a redesigned chkdsk tool for disk corruption detection and fixing. In past versions of the chkdsk and NTFS health model, the file system volume was either deemed healthy or not healthy. In Windows 8, Microsoft is changing things up. Rather than hours of downtime, Windows 8 splits the process into phases that include 'Detect Corruption,' 'Online Self-Healing,' 'Online Verification,' 'Online Identification & Logging,' and 'Precise & Rapid Correction.'"
1. Your data has disappeared (detect corruption).
2. Hit self on head with brick because your data wasn't backed up (online self-healing)
3. Hit self on head again to see if your data has reappeared (online verification)
4. Identify brick by matching to lumps on head (online identification and logging)
5. Give brick to neighbor's kid to hit you on head with again (precise and rapid correction)
It seems like Microsoft is really going out of it's way to innovate in Windows 8. I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems. Yes, you can use RAID or something, but that will bring costs significantly up. It's better to see these things before failure actually happens.
Why is Online Self Healing different and from "Make the damn FS work properly"?
WTF FS is that has problems that can be fixed online?
Rethinking email
Considering that the current chkdsk is actually capable of causing massive logical damage , Microsoft has a LONG way to go to make it function as intended.
sane precautions with your data such as RAID and/or backing up your information
RAID and backing up should never be considered an "OR".
.. "RAID is not a backup strategy".
Repeat after me
I am Slashdot. Are you Slashdot as well?
It seems like Microsoft is really going out of it's way to innovate in Windows 8. I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems. Yes, you can use RAID or something, but that will bring costs significantly up. It's better to see these things before failure actually happens.
Well, it's nice to see NTFS catching up to features in ZFS. Which is available for Solaris (and OpenSolaris, Illumos, OpenIndiana), FreeBSD, and NetBSD. Not sure of ZFS's status with Linux via ZFS-FUSE or that third-party kernel module that is in development. Linux also has Btrfs in development.
For the Mac OS, Apple was originally looking into ZFS, but dropped it supposedly due to licensing issues. There is also http://tenscomplement.com/our-products for ZFS storage solutions for the Mac.
Apple does have http://www.nobius.org/~dbg/ Dominic Giampaolo, the Be File System rockstar, and there have been a lot of rumors about future filesystem development to meet the goals they originally intended to have with ZFS.
here the highlight.
if disk.mbr.has_grub
for part in disc.partitions
if part.type.not_ours
chair.throw() # dammit... let's do something about it
part.raw_write(offset=random(1,part.size),data=random(1,255)) # voila'
end if
end for
end if
---- MISSING MISCELLANEOUS DATA SEGMENT --- [sigdash] trolololol
I'm glad that this does not mean over the internet.
Given the other phase names, I surprised the marketing department didn't call this "Detect Awesomeness!".
sometime in the future, some of your storage device will fail
SSD? Sure. NTFS has such a vibrant disk activity life, it is amazing. I mean, how is it even possible to constantly write something to the drive, like every second, all the time, even during idle. There is a huge gap for improvement. The current NTFS behaviour can only be called a calamity.
chkdsk is a standalone app. Can I use v8 on my v7 OS?
You must spend more time actually working with non-Windows systems. Multiple filesystems, most free, some commercial have been doing these sorts of things, and more for YEARS.
Pain is merely failure leaving the body
I wonder if that is really NTFS's problem. There is no reason for a filesystem (even a poorly designed one) to be writing sectors to disk all the time unless running programs request the OS to do so. The most likely culprits are poorly-designed Windows applications that write to disk all the time.
Score: i, Imaginary
I was curious as to why MS is continuing on with NTFS, surely there must be something newer coming out of their R&D labs. So a quick google turned up this from the same blog, but earlier this year: building the next generation file system for windows refs
I am Slashdot. Are you Slashdot as well?
Ah, finally, all those Online-something goodies for chkdsk. I've always wanted to have Windows Activation Wizard popping up before my chkdsk session, just in case I was in doubt was my copy legitimate. (It is btw)
Ah, glad to see some of the spirit and philosophy of DiskDoctor has made it over to Windows. I wonder if that 'repair' utility actually ever worked for anyone ever.
Why OpalCalc is the best Windows calc
A FAT USB drive (or Android phone) doesn't need to be 'safely removed'. You can just yank the thing and it's fine (as long as it's finished its r/w operations).
These two statements are mutually exclusive. The translation of your post, once only the facts remain, is "I hate windows." Why didnt you just say that you hate windows?
"His name was James Damore."
Right up until your primary gets some corruption and proceeds to mirror it to the other.
It still hangs at 100% CPU for no apparent reason even after 20 years.
It seems like Microsoft is really going out of it's way to innovate in Windows 8. I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems. Yes, you can use RAID or something, but that will bring costs significantly up. It's better to see these things before failure actually happens.
Rather than take sane precautions with your data such as RAID and/or backing up your information, you want a warning 1 minute before your drive fails?
1 minute should be more than enough for anybody.
I just hope both Apple and Linux developers would add something similar
You mean like btrfs? (which has many additional advantages that Microsoft can't simply "add" to NTFS without replacing it entirely; it's like how ext4 is a good improvement on the old filesystem design, but overall it's limited in very fundamental ways. NTFS is similar; it needs to be thrown out entirely)
Just delete an important file or directory on a RAID1 and see how much that "backup" protected you. Or install a virus. Or have data corruption on a disk.
thegodmovie.com - watch it
I don't think it is. I think it's another of those far-too-obvious ones like "googlewatch" and "apple-fan" which I think have just been set up to troll.
If it's being paid for (as we;re meant to believe has been going on for ages) then the subtlety level has dropped an order of magnitude.
Also, the names of the accounts are all highly suspect - this one just so happens to be named after the programming language used in OS X and iOS? Come on!
No.
What happens if, say, you get a virus, or your app goes haywire, and The Critical Irreplaceable File gets corrupted or deleted?
RAID protects from hardware failures. It does nothing against software or human errors. And given my history with hardware, software and people, I'd say the first is generally the most reliable.
This is good news! I still swear by running chkdsk /f after power loss or hard reset, even with Windows 7...I've had a few cases where there was some corruption and Windows didn't give a warning about it.
Also, it's about time NTFS got upgraded. Extents, checksums, etc. are no longer new and unnecessary.
Not really:
a) RAID stores everything and only one version, backup is usually selective and versioned
b) backup is independent from the main storage. Its storage rests most of the time and it should be in several places - in case of fire or flood, for example. RAID is always spending its working resource and, more true for some schemes than other, has similar usage patterns across disks. If one sector on one disk in RAID1 failed, it might mean that this sector is often accessed and will soon fail on other disk.
RAID is tactical, backup is strategical. RAID1 gives you added read performance with negligible write performance loss and a chance to continue operations without breaking for repairs. Backup lets you repair everything when shit hits the fan.
It seems like Microsoft is really going out of it's way to innovate in Windows 8.
No Windows 8 is Microsoft implementing a lot of things that Linux/BSD already have. This would include an attempt to force a first gen "Duel OS" onto its users.
Yeah, like zfs or btrfs with checksumming of all [meta]data and built-in RAID.
thegodmovie.com - watch it
The Professional Edition will include a DVD of Daily Affirmation with Stuart Smalley, in case the user blames himself for the computer's failure during stressful times.
WARNING: Smartphones have side effects--most of them undocumented.
RAID protects against hardware malfunctions, but not user malfunctions. Backups protects against both.
Ever wonder why it's *always* turned on?
I just set up 2 servers that had 128GB RAM, running MS Server 2008 Enterprise. Guess what the boot drives had? Yep, a 128GB swap file.
There are two types of people in the world: Those who crave closure
Defrag and indexing is automatically disabled on a SSD
int main() { while(1) fork(); }
you can back up a virus with a non RAID 1 back up system.
En garde!!!
... ... ...
I don't know if this is the real APK or not. Wow.
I wonder what entries he adds or removes from his hosts file when his car doesn't start.
Non impediti ratione cogitationus.
I thought making sure that all read/write operations have finished was the point of "safely remove".
The Tao of math: The numbers you can count are not the real numbers.
Delete an important file. Then ask that question again.
Do you even lift?
These aren't the 'roids you're looking for.
Considering that the current chkdsk is actually capable of causing massive logical damage , Microsoft has a LONG way to go to make it function as intended.
You mean it's suspected of causing additional damage in a couple of comments.
It's very possible that there are long standing bugs. It's also possible that it just tried very hard on a hopelessly borked drive and failed.
I looked in to disabling it, and it
See, truncated data. That's what happens when you remove without unmounting.
Seriously. Your strategy puts at risk all the writes during that rebuild process.
Can I have both? Can I have a drive that, on that one minute warning, immediately flushes it's cache and goes offline?
It would make recovery after replacement much smoother. Clone and go, no worries about incomplete writes etc.
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
That only "backs up" against drive failure. What happens if something gets deleted? What happens if a process goes mad and scribbles all over something important? What if someone breaks in?
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
And if you are smart and have previous backups and/or incrementals, you can get what you need without bringing the virus along for the fun.
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
It's called fsck, not chkdsk.
And the fact that ext4 doesn't support online fsck is a major annoyance for a lot of sysadmins.
It's not about running well or not, it's about the system being shut down in a middle of a write operation.
btrfs can do online fsck, and I'm looking forward to it just for that.
Windows, since XP, defragments the filesystem when idle.
You can turn this off via TweakUI for XP, I know this. Not sure how to do so on Vista/7.
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
Yeah. e4fsck is a much more modern name.
"Speedup" features that waste time writing stuff to disk are very poorly named....
Can you be Even More Awesome?!
You can just yank the thing and it's fine
This is true if you've set the drive's caching policy to "optimize for quick removal" rather than "optimize for performance". (Names are from memory and may not be exact.) "Optimize for quick removal", which Windows turns on automatically for removable media drives, syncs the file system continuously.
(as long as it's finished its r/w operations).
The trouble is figuring out when this has happened, as a lot of USB mass storage devices don't have a blinking access light. Only one of my USB flash drives has one, and my Android device does not. As maxwell demon pointed, out, the "safely remove" on a removable drive is useful for making sure that all writes have completed.
I've always wondered about this, and I can see it for RAID 5 (also, I've seen RAID 5 setups fail irrecoverably), but what about RAID 1, where you're just mirroring. Wouldn't that essentially be the same as backing up to a different hard drive?
Drives don't need to fail for a backup to be needed. You might think you are formatting an SD card and end up formatting an essential partition. You could do like I did once while setting up an OS on a VM and told it format the wrong drive. Say what you will about Linux, but it continued to run even though the root partition had been reformatted out from underneath it. Or, it might be something as simple as someone goes in and deletes your LDF files trying to save hard drive space. It might even be a software issue that trashes your necessary files or partitions. It could be a virus...
No matter what causes the problem, in a RAID setup, the error or mistake will be copied to all of your "backup" drives. If you don't have an offline backup, you are screwed.
There is no "I disagree" mod for a reason. Flamebait, Troll, and Overrated are not substitutes.
Kindly have your own reading comprehension checked.
Non-Windows does NOT automatically and exclusively mean Linux.
Also, once that reactionary rash of yours stops flaring up, go investigate what ZFS and its tools, such as scrub, can do for people who care about data integrity.
Also, in your detailed reading of the article, did you note, for example, "NTFS detects", "NTFS attempts", "NTFS validates", "healing feature built into NTFS",
"introduced a new file system (emphasis added) ReFS"
Here, have another look: http://blogs.msdn.com/b/b8/archive/2012/05/09/redesigning-chkdsk-and-the-new-ntfs-health-model.aspx
It seems the lighting under your troll bridge is, well, a bit dim.
Pain is merely failure leaving the body
I remember being told not to run chkdsk on my ext2 volumes because a) there was no need and b) more harm than good. I never once ran it on a Linux desktop. And forget about defrag. Is there some reason NTFS can't do this?
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
If there is one thing you can count on in the world, it's someone screaming "RAID is not a backup!" at the top of their lungs in any conversation dealing with RAID.
Yes, thank you. We get it. RAID does not protect against deleted files, etc. You can go back to shouting other contrarian favorites in other threads.
In the mean time, if and when one of the drives in my RAID-1 mirror fails, I'll be sure to throw its working partner straight into the garbage can. I certainly wouldn't use it to restore my entire filesystem that would have otherwise been obliterated.
I don't know about you, but I'm constantly deleting files by accident, and getting personal data destroying viruses (via a time machine from the 90s) where as my drives never, ever fail.
"I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems."
It's called SMART and has been around for a while already.
Warns you when your hard drive starts acting outside normal parameters or throwing soft errors (which it usually does before it actually fails).
How can chkdisk fix corrupted hardware? Yes, "disk corruption" is about hardware, not about a filesystem. It would need low level interfaces to the actual controllers (these days inside the drive) and do things most vendor support software doesn't even do. So the summary is blatantly wrong, you can't fix a broken disk with chkdsk, not even the new version.
What they are actually doing is classifying 18 different forms of filesystem corruptions and are building the OS and filesystem drivers in such a way that they can fix a few of them online and do analysis of the ones they can't fix while the filesystem is online, so they can fix them later without having to scan the entire filesystem. That means that only the actual repair might require the filesystem to be put offline for a limited time and in general does not need a reboot. How they plan to do this without accidental overwriting of lost clusters is a mystery to me, but they must have thought about this.
I was promised a flying car. Where is my flying car?
Wake me up when there's pervasive Reed-Solomon error correction everywhere.
Well i'd say it depends on WHAT data is being backed up via RAID 1. For example if its something that can be replaced but would be a PITA, like say media files? Then as long as the odds of them doing something really stupid are low then i'd say the risks are low.
The big thing I've found is it is VERY important to have offline discs images of the OS because that is the area one will have to worry about malware, bad patches, etc and that will not be protected in any way with RAID since whatever went wrong with the first will be simply copied before you know what is happening. for my customers i recommend a USB HDD for OS images and any important files because 1.-it is easy to take offsite and replace with another. 2.-It is cheap enough that having more than one isn't cost prohibitive, and 3.-if one has more than one and rotate them even if a bug were to somehow get in and infect both the system and the USB drive attached it would still leave them the offsite backup to restore from.
But as long as one has offline backups as well for critical files then i see no harm in having RAID 1 as simply another layer of defense. Of course i would recommend something like HDDTune to check the SMART to keep an eye on the drives since a good SMART tool will often let you know long before it is noticeable that there is something wrong with a drive. i personally like to install Kel's CPL Bonus Pack on a system as it gives me all the tools I could need right from the control panel, like HDDTune, CPU and GPU-Z,HW monitor, etc. oh a bit of advice, if one wishes to use kel's CPL Bonus in Vista or 7 remember to run as admin when installing.
So while I wouldn't recommend RAID 1 as any kind of backup strategy by itself i see no problem in using it as part of a defense in depth strategy along with offline and offsite backups.
ACs don't waste your time replying, your posts are never seen by me.
Are you out of memory or something? because i have 8gb on my netbook and desktop and neither hardly hits the HDD at all unless i'm actually launching a program or searching for files and both are NTFS (Win 7 HP X64) so I just don't see where you are getting that. Looking down at the HDD indicator for the little 1.8GHz Sempron I have in the shop running XP it isn't seeing much disk activity either unless I am launching something, and its XP Home with only 1.5Gb of RAM.
So all i can think of is maybe your system is RAM starved or you are running something in the tray that is constantly hitting the drive for some reason. I would look at the tray apps as the only time i've seen that type of constant activity in a non RAM starved machine it turned out the tray app for a drawing pad he rarely used kept writing to the disc. When i killed that it was right as rain, so you might want to look into that.
ACs don't waste your time replying, your posts are never seen by me.
If anyone wants an example of why RAID should always have a backup solution and not just and/or solution. Please check http://dslreports.com/ , as they just recovered from a powerloss at nac.net which took their entire array system with it, and fudged 2 years worth of data, which had to be sent off for recovery. That was on April16th, the site is just starting to come back up in the last two days.
Some info here: https://docs.google.com/document/d/1kll86bDn_MgWoo6Ja7oHo_yvI0SCqggEvNWwPWIcrHY/edit?pli=1
Om, nomnomnom...
*Volume doesn't need to dismount to run CHKDSK; it's modus operandi is now very volume shadow copy-esque.
*Windows will run an online "disk health check service".
*If a the NTFS Driver finds an issue with the disk, It reports it to the "disk health check service".
*The "disk health check service" then runs chkdsk on just the defective file, or of needed, the entire drive, fixing the problem upfront before it can corrode and make things worse.
*The Disk health check service will give you a nice bar graph, pie chart, or health bar to see how healthy the disk is. (Read: The market-speak is brutally thick in this article.)
Long story short: This is handy for maintaining fileservers and RAID 1 arrays; for the average single-disk machine, basically it covers up hardware failure. Oh, and it'll improve the security of the OS since most viruses that do low-level editing of a disk don't re-write the CRC32 at the end of the cluster, thus the OS will detect the bad cluster and replace it from backup (Windows File Protection Services).
Set up event viewer on the domain to report all disk errors to a single system and report anything on Disk 0. If a machine ever gets a disk 0 error, replace the disk. It WILL fail a surface scan 100% of the time. End of story.
That's what removable media and rotation schemes are for.
thegodmovie.com - watch it
Yeah, kinda like the *NIXs dropped fsck so long ago.
Vintage computer games and RPG books available. Email me if you're interested.
Why would indexing be disabled?
Google released a study that showed SMART to be almost useless.
It wasn't a big surprise to me, as among the hundreds of drive failures I've dealt with in my 13 years experience supporting desktops and laptops, a SMART alert was involved only once or twice.
I don't always use unix-like operating systems; but when I do, I prefer FreeBSD.
happily wearing my SSD in the process.
You don't have to worry about SSD wear any more than of a HDD's. It is only an issue if you use a memory card as your SSD.
Please stop spreading false information. Flash is flash. It has a limited number of write cycles. Straight from Intel:
Are you really buying into these marketing names? Are you really implying that Linux and OSX dont have something similar to a journaling filesystem?
We have no details on what this means, whether there is a background filesystem check going on at all times, whether it might increase wear and tear on the drive, or what; its a little early to start calling this a great development.
Not if you use ZFS. Corrupted files get fixed in a mirror.
Unless Im mistaken, there is no "primary" in RAID 1, nor does one drive "mirror to the other". Data writes sent to the controller are mirrored to both drives simultaneously. If for some reason on-disk data gets corrupted, that will not be "mirrored over" to the other in any scenario I can think of. If you got a bad sector on one disk for example, any errors induced by it would not appear on the second drive (how the array would behave in that scenario I dont know, but it likely depends on the raid controller).
What happens if something gets deleted?
Not that its a good idea to go without a backup....
But windows has since at least XP had a concept of "shadow copies", whereby you can view the state of the drive at various checkpoints, and recover data. Right click your C: volume, properties, shadow copies-- click one of the checkpoints and click to view the data. You can restore individual files quite quickly doing this.
Open up task manager, go to performance, click on resource monitor. Use this to track down what is running on your system that generates so much disk activity. On my machine, the disk housing the C partition is at this moment at 0.00% activity, and has averaged about that for quite some time. Average IO for C: over the past few seconds was around 800 bytes/sec.
If what you posted is accurate for your system, you need to get it fixed.
Because there is basically no point to it. The point of indexing is that rotational media has substantial access time penalties for fetching various info from random locations on the drive. By indexing it, you place all that metadata in one place so you can get a sequential read.
But with an SSD, there is not a substantial difference between random and sequential reads, and no significant penalty, so the indexing is not terribly useful. Additionally, indexing means more writes as well as increased background reads, which not only impacts performance (for little gain) but also wears out the drive.
Ditto for defragmenting; in fact defragging and indexing attack the same core problem, but from different angles, which is why neither is necessary on an SSD.
This "logging" you speak of is a feature of basically every modern filesystem used by basically any OS you might stick on a computer. HFS+, ext3+, btrfs, NTFS, all of them journal, and hence are slightly slower and slightly more active than non-journaling filesystems.
But most people with a clue recognize that those slight disadvantages are more than made up for by having a FS that remains consistent even after an unclean shutdown from ie a power outage.
As for the amount of activity you talk about, you are grossly overstating it. Open up resource monitor and you can watch how active your drive is. I promise you, except for the occasional recovery checkpoint (daily or less frequent), the OS is generally not to blame for any excess IO activity you see.
Only Vista and above do that. XP did not allow you to schedule a defrag, IIRC.
Sure, but aside disabling unnecessary features (such as defragmentation), you really don't have to specifically avoid wearing a SSD. They can take a lot of R/W, possibly even more than HDDs.
You are essentially saying "yeah, if you don't write very much to it, the SSD will not wear out", which is exactly the obvious corollary to what I pointed out. It just is not so that the user doesn't have to worry about wearing out his SSD. It's a failure mode that has to be considered. You can't just wave it away.
1) You can write an HDD CONTINUOUSLY for its entire expected mechanical and electronic lifetime without ANY wearout due to the writing - it essentially has an infinite design number of cycles with respect to writing. It is, inherently by design, free from wearout due to this particular cause. The SSD is emphatically not so.
2) "Unnecessary" features are not the only source of write data. A lot of us like to, you know, WRITE DATA to a drive because that is its PURPOSE. Some of us have the need to write more data than others.
Dude, if you are doing a chkdsk on a failing hard drive, you are asking for trouble. Also, it has been said time and time again that Linux prevents its filesystems from becoming unstable by having journals and saving the system from power outages and the like. I believe NTFS is an inferior filesystem, but that is just my opinion. Take a look at EXT3, XFS, JFS, ZFS and others. Yeah "going out of it's way". Why don't they come up with something a least as good, huh. BTW most computers have a hard drive scan in hardware that can run outside of the operating system, eliminating the need to rely on Windows to tell us whether our hard drives are going bad.
Society use your Sciences
*EXT4
Society use your Sciences
The new Windows 8 chkdsk program detects corruption... if you try to use it in Washington D.C., for example, it goes crazy and shits itself, curls up in a ball and cries while sucking its thumb.
Task manager can be tweaked to display the disk writes (I/O) of individual processes.
Processes --> View --> Select Columns
You can do something similar with Process Explorer,
which was an awesome program that got bought by Microsoft,
but has avoided the meddling that usually follows being aquired.
Process Explorer will show you the programs attached to processes like svchost.exe
so you can pinpoint exactly what's kicking up problems.
[Fuck Beta]
o0t!
It seems like Microsoft is really going out of it's way to innovate in Windows 8. I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems. Yes, you can use RAID or something, but that will bring costs significantly up. It's better to see these things before failure actually happens.
How can blatant astroturf posts be +5 intredasting ?
Seriously, it read like a lazy marketing blurb...
This OR people never heard aboot S.M.A.R.T. reporting...
Raid is also not a full proof protection against disk corruption. The classic RAID model assumes only one disk in an array will fail at a time. With today's disk sizes, this assumption simply cannot be relied on. There are various, newer, RAID configurations that are more resilient than that, but they are rather expensive for casual use (you, obviously, don't).
The chances of one disk having latent errors that only show up during the intense operation of a disk sync are not insignificant. Such a problem took all of the data on a server a month ago for me.
Since you rarely back up EVERYTHING (i.e. - file system, cache directories and all), a sane configuration is to back up everything important AND ALSO to install RAID to decrease the chances that you'd need to rely on said backup.
The GP was not saying that you shouldn't RAID. He was saying that, in addition to installing RAID, you should also back up.
Shachar
The number 1 feature I want in current filesystems is block-level checksums.
I've had to perform data recovery for a number of people recently (yes, backups help, but sometimes having them just 24 hours out of date means there are advantages to attempting to recover the data off the failed or failing drive or array)
Now, using a combination of tools I've been able to get the faulty drive to give me back data, but I've got no way whatsoever of knowing if the data it's given back to me is actually the data that was stored on it in the first place.
Having end-to-end checksums would easily allow me to assign a confidence level to data recovery procedures, letting me know that the data I have retrieved is what was stored - it would also allow better control over operations like fsck or chkdsk if the blocks that hold metadata are also checksummed, that way it would be possible to tell if a block has been randomly corrupted somehow, or if it's stored as intended.
Specialist Mac support for creative pros, Melbourne
I didn't say schedule.
To turn it off (try it in an XP VM, if you refuse to believe truth):
1. Run TweakUI
2. Click on "General" (the second item in the left hand pane)
3. Scroll the "Settings" list in the right hand pane.
4. The next to last item is "Optimize hard disk when idle".
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
Certainly doesn't replace a backup, but that's good to know about.
I assume these shadow files are counted as free space? Eg, when you actually do need the room they "make way" for the new data?
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
I have had a legal secretary delete about 10,000 files at the click of a mouse. Sometimes, backups are very handy things.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
Even so, RAID 1 is not a valid backup strategy because of things like trojans/virus, user accidentally deleting files, software-induced database corruption, etc.
Even worse: with Windows the hibernation file is different from the swap file: hiberfil.sys vs pagefile.sys.
NTFS has been a journaled file system since the release of Windows 2000 in late 1999, which was before the first journaled filesystem support was merged into the Linux kernel.
I don't always use unix-like operating systems; but when I do, I prefer FreeBSD.
This might reduce some writes (not all): http://technet.microsoft.com/en-us/library/cc959914.aspx
Warning: it could break apps that rely on having correct "last access times". Windows does not have Linux's relatime stuff yet.
But many Linux systems would also constantly write something to the drive. Especially if they have services that log stuff to a local syslog.
By the time you've got enough fragmented files to cause problems it's usually time to move everything to a bigger disk anyway. If it isn't, the easy (but time consuming) way is to copy everything off and then back again. Since there's no hidden system files and no requirement to just dump the files immediately all over the disk in fragements it just works - no tool more complex than cp, tar or rsync required.
Windows fanboys shouldn't take it as an insult - different priorities and different approaches result in different behaviour. NTFS dumps those files down almost ASAP so fragmentation is very common - other filesystems buffer more and are prepared to wait longer for that disk to spin around again so fragmentation is very rare, especially is there is something like NFS in the middle. The fragmentation is a throwback to MSDOS and the need for quick removal of floppy disks versus the *nix approach where the thing won't let you unmount anything until it's really sure everything is done and the user just has to put up with the wait.
Hardware dies if nothing else.
Cutting the power on a modern file system also results in it having to finish what it was up to when the lights went out.
Ah, I agree with that. If you have workloads that actually write enormous amounts of data, then wearing the drive is indeed becomes a concern. But for general use you don't have to tiptoe around features like "NTFS indexing/logging" and lose your sleep over the flash wearing out.
Online defrag is planned for btrfs, too.
I thought it had online fsck, but it seems it's only 'planned', not available yet either.
Does this really deserve a whole article on Slashdot and why are you talking in fake rapper slang, like off The Wire?
"In Windows 8, Microsoft is changing things up"
AccountKiller
"It seems like Microsoft is really going out of it's way to innovate in Windows 8 ...
List of File Systems
AccountKiller
It seems like Microsoft is really going out of it's way to innovate in Windows 8
Naah, this ReFS stuff is nothing. Once they ship Object File System^H^H^H^HStorage+^H^H^H^HRelational File System^H^H^H^HWinFS, then we'll really be cooking with plasma!
fsck with ext4 takes literally seconds for 100s of GB. It is so fast, I am using it every time I mount my backup medias. For example 100GB encrypted partition: [~] # time fsck /dev/mapper/read-snap
fsck from util-linux 2.20.1
e2fsck 1.41.14 (22-Dec-2010)
read: clean, 880109/6406144 files, 22357360/25599488 blocks
real 0m0.184s
user 0m0.022s
sys 0m0.010s
So it just took 0.2 Sec for the check, whereas the Windows chkdsk takes a few minutes. I don't know what chkdsk does, why it takes so long. But as long as chkdsk takes minutes, it is just garbage. How about to make it perform in seconds (or fraction of seconds) just like fsck with ext4 (or ext2, ext3 it was).
http://www.mueller-public.de - My site http://www.anr-institute.com/ - Advanced Natural Research Institute
Sure, the filesystem itself does journalling, which translates into additional writes. But if no application writes anything to disk, the state of the filesystem does not change; thus, pretty much by definition, there is nothing for NTFS to journal during those periods. On the other hand, if you are going to write something anyway because an application requests so, the additional cost of journalling (either in terms of I/O performance or flash wear) is probably not that much. Hence my reasoning that poorly designed applications are the real culprit for poor I/O performance.
Regarding defragmentation, you might have a point; I'm not aware of how NTFS defragmentation works.
Score: i, Imaginary
It seems like Microsoft is really going out of it's way to innovate in Windows 8. I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems. Yes, you can use RAID or something, but that will bring costs significantly up. It's better to see these things before failure actually happens.
Rather than take sane precautions with your data such as RAID and/or backing up your information, you want a warning 1 minute before your drive fails?
1 minute should be more than enough for anybody.
Your wife begs to differ.
No they do not, they consume substantial space in System Volume Information. They also may not always be enabled.
They are most useful on network shares, where a user may end up deleting a file. You can allow them to restore their own file without having to mount recovery media.
For the NTFS equivalent (if any) of the Unix atime, if an application reads a file from the disk, its access time might need to be written to the disk, causing disk writes.
Bingo Dictionary - Pragmatist, n. A myopic idealist.
The problem with RAID 1 is that when something happens and the drives no longer "agree", then which is correct? You've got even odds of mirroring the corruption onto the "good" drive. I've got personal experience with this, and personal experience with failed RAID 5 setups. The sad thing is that I've not done much RAID in my life, and yet, a disturbing amount of it led to bad experiences.
Like the man said, "RAID is not a backup strategy".
Karma: Poor (Mostly affected by lame karma-joke sigs)
Whether or not the corruption is "mirrored over" is irrelevant. What is relevant is that one of the drives contains corrupted data, and therefore, reads are going to, in some cases, return corrupted data. The really frightening thing is that the array and the system is incapable of knowing which drive is corrupted.
I've been there, done that, and I don't use RAID 1 anymore.
Karma: Poor (Mostly affected by lame karma-joke sigs)
No. It relies on System Restore, which annexes a percentage of the total drive space for its exclusive use. Usually about 10% - 20%.
For a site about things like basic rights, Slashdot users sure do like to censor "dissent".
That is because in the event of a STOP error, Windows needs a page file at least as large as the memory in order to generate a crash dump (or core dump in Linux parlance).
For a site about things like basic rights, Slashdot users sure do like to censor "dissent".
Most raid controllers run a consistency check... and there is an ongoing double write/double read-verify. In case of bad sectors such a read delay will usually fail the drive... but suppose you can find a raid controller that will do anything...
120 characters ought to be enough for anyone
Unused I/O and CPU cycles are wasted unless they are used for something. Take for example the pre-paging case... if you allocate and page stuff out when the system is not doing anything, when the system needs memory it does not need to wait on memory IO being used to transfer the contents to swap. The concern should only be if your inteded use of CPU cycles and IO are being pre-empted on unused efficiency... which I doubt is the case most of the time.
120 characters ought to be enough for anyone
I think he means the indexing service, for search... which gives you instant search results.
120 characters ought to be enough for anyone
I put an SSD in a spam filter... it died in 3 months. Then, it's replacement died in 3 months. It made a huge difference in performance... but replacing the disk every 3 months and loosing mail was unacceptable. I put a WD velociraptor in it, and I think it's still running to this day. The 3rd replacement SSD went into a laptop and died in 2 years. I have had 2 other SSDs fail on me and so far, in laptops, I have had more SSDs fail then regular hard drives. If the real-time/speedieness wasn't so addictive, I'd have given up on SSDs already.
120 characters ought to be enough for anyone
I might have some sympathy with your position if I hadn't seen a technically savvy person like yourself bitten just a few months ago by the belief that RAID is a backup.
Did he get a virus? No.
Did he delete some file he had no other backup of? Nope, guess again.
Did a hard drive fail? No again. All four drives in the RAID-5 were A-OK.
What happened is a power outage in a thunderstorm. The RAID subsystem disappeared while the OS was writing something to the disk. When power came back, THE FILESYSTEM WAS GONE. Not corrupted, gone. MIA. Poof.
The RAID monitor saw all the disks, but the OS wouldn't even acknowledge that there was a mountable filesystem there any more. The victim had to resort to one of those data recovery programs that just scans the disk for file-like objects and saves them off one by one without metadata to an external hard drive.
He had no backup because he thought this 4-disk external RAID was expensive as-is. To back it up, he would have had to double his hardware costs. Now he's looking at the cost of recreating the home movies he had stored on that RAID and realizing that a thousand dollars or so for an offline backup RAID is actually cheap.
ext3fs was added to the 2.4.16 kernel in January 1993. you're welcome.
Typo. 1999.