MS06-049 Causing Silent Data Corruption

Uncle Mike writes "It looks like there is a problem with the recently released MS06-049 / KB920958 patch. If you have compression activated on any folder, then the compressed data is at risk from corruption. New files that are close to a multiple of 4K in size will have their last 4,000 bytes or so overwritten with 0xDF. Although this problem has been reported to Microsoft, as yet there appears to have been no official announcement. "

Comments

interesting

[Intangion]

its interesting how when they make a patch that corrupts your data you dont hear anything from them.. but when someone makes a program to allow fair use by opening DRM on their movies they come up with a CRITICAL patch within ours to prevent it. i think that speaks to their priorities, protecting their drm IMPORTANT protecting your data hmm.. not so important

Re:interesting

[exclusive_lock]

Well, I would think the liability of messing up data on any number of companies and households should be enough to trigger a prompt reaction from Microsoft.

I would assume they are working on fixing it right away, with the PR team working on a statement for damage control.

Re:interesting

[erroneus]

Have you read the EULA? Well, neither have I actually, but you don't have to be a partiualrly educated guesser to know that there is a provision in the EULA regarding the loss or corruption of data. You agree to endemnify Microsoft against any such loss. Further, they make no guarantee of suitability of the OS for any particular purpose and make no claim that the product is reliable in any way.

You know, if I were to create a series of advertisements, I would make it similar to the "Truth" campaign against smoking and cite the Microsoft EULAs to indicate what it is the public is buying. Comparing that to what would be acceptable in other products would quickly make Microsoft seem rather ridiculous. No one reads the EULA and in many instances has been ruled legally non-binding. People pay more attention to speed limit signs than to EULAs.

Re:interesting

[teflaime]

a partiualrly educated guesser to know that there is a provision in the EULA regarding the loss or corruption of data. You agree to endemnify Microsoft against any such loss. Further, they make no guarantee of suitability of the OS for any particular purpose and make no claim that the product is reliable in any way.

Actually, only home and small business users suffer from this problem. Most large companies can leverage Microsoft into contracts that force Microsoft to cover various loss of business expenses and recovery expenses on things like this. The company I consult with not only has their contract set up to force Microsoft to take responsibility for these things, but has a monetary value on these issues set in excess of $20M a day that Microsoft would be responsible for if their product ever caused data loss or business interuption.

Re:interesting

[deadlinegrunt]

"...Have you read the EULA? Well, neither have I actually..."

Are you this person by chance?

Re:interesting

[exclusive_lock]

As the late Steve Irwin would say: "CRIKEY!".
You're right, I should've known that venomous EULA would turn right back and bite me (and all Microsoft customers) in the rear.

"Satisfaction Guaranteed!"*



* The term "Satisfaction" and "Guaranteed" are used only for illustration purpouses in a figurative, subliminal manner.
Enlarged to show texture. Serving suggestion.
As a matter of fact, no satisfaction guaranteed whatsoever, by any means.
Reading the words "satisfaction" and "guaranteed" above certifies you accept this disclaimer, its terminology, grammar and syntax errors as the single source of truth given to us by the Flying Spaghetti Monster.
By the way, why are you reading down here? Our legal department wants to know who reads this stuff (and sue them).
Don't bother to ask "Who is it?" the next time someone knocks at your door, it's them.

Re:interesting

[coolgeek]

I am absolutely astonished. That must be a gag. Jeez, I really hope it's a gag. Sadly though, it's probably not a gag.

Re:interesting

[RxScram]

I have GOT to get me one of those Badonkadonk tanks!

Re:interesting

[X0563511]

Well, if you look closely you find that this patch is for Windows 2000 SP4 only, and all other versions of windows are not affected.

That does make a big difference, win2k is not MS' top priority.

Not that I condone their delay or lack of forsight, however.

Re:interesting

[mr_death]

Jesus H. Christ! Do these f'ing idiots test anything?

And doesn't this just give you warm fuzzies about the reliability of Vista and its 50 million lines of code?

Re:interesting

[funfail]

Then you have no gain from compression.

Re:interesting

[geminidomino]

It looks like a skiff from ROTJ...

Re:interesting

[hackwrench]

http://support.microsoft.com/?kbid=920958

APPLIES TO Microsoft Windows 2000 Service Pack 4 Microsoft Small Business Server 2000 Standard Edition

I use XP, so I'm safe. Whoo, I was scared there for a minute.

Re:interesting

[PantsWearer]

This might help: Jon Swift

How does something like this happen

[guruevi]

What type of programmer puts such possibilities or leaks in a program? I have been programming for a long time and I never had stuff like this happening. Data integrity is one of the primary things you want to maintain and you should be extra careful when handling and altering files not your own.

Re:How does something like this happen

[cmowire]

Well, you assume that whoever made the MS06-049 patch actually took the time to understand the code he was patching and/or contacted the person who origionally wrote the code in question.

Re:How does something like this happen

[avalys]

If you really have been programming for a long time, you must only be writing very simple programs if you've never had something like this happen, and you think that being "extra careful" is all you need to do to avoid it. What type of programmer does this? Every type of programmer - it's unavoidable.

The programmer is not to blame here. The real question you should be asking is "What type of QA department fails to catch a bug like this?"

Re:How does something like this happen

[something_wicked_thi]

Oh, please.

MS bashing is fun and all, but do you have any idea how a kernel works? Anything can step on anything else. An off-by-one error in a kernel can be catastrophic to any number of things. This one does sound suspicious, but keep in mind that the code that is failing is probably only peripherally related to the code that was patched. They say they patched a buffer overflow. Maybe the buffer was already being overflowed by the compression code and patching it caused the compression to break. That might explain why it's the last 4000 bytes or so in a file that's almost a multiple of 4K.

The real question is why they didn't catch it in testing, especially with MS's extra-long patch process where they spend so much time testing (that is the current excuse for the months that pass between reports and patches, right?). Being "extra careful" does not save you from these types of bugs and being a programmer for as long as you have, you ought to know that being careful just doesn't cut it.

Re:How does something like this happen

[CosmeticLobotamy]

What type of programmer puts such possibilities or leaks in a program?

Every programmer that's ever worked on something longer than 6 or 7 lines of code? Except you, of course. I've been in the bathroom after you and am always impressed by the way it smells just like roses.

Re:How does something like this happen

[dhasenan]

If data compression is being handled in the kernel, there are serious design issues here.

And in this case, being 'extra careful' consists of writing sufficient tests.

Re:How does something like this happen

[something_wicked_thi]

If data compression is being handled in the kernel, there are serious design issues here.

Really?

And in this case, being 'extra careful' consists of writing sufficient tests.

And tests are good only when you run them.

Re:How does something like this happen

[theshowmecanuck]

I agree and disagree with you. As long as the programmer properly unit tested his/her work, then you can shift blame to QA. I have seen developers not properly unit test their code too many times, relying on the QA department to do their work for them. But yes, unless it happens in very rare circumstances (is this the case?) someone should have caught this in testing somewhere... but not necessarily just QA.

IANAQAT (I am not a QA tester).

Re:How does something like this happen

[LearnToSpell]

if (len >= ELF_PRARGSZ)

It happens. The only interesting thing here is how long they'll take to deal with it.

Re:How does something like this happen

[theshowmecanuck]

Made me think of Grannies Perls of Wisdom I read on Java Ranch (I first found it about 6 or 7 years ago...): "Testing can show the presence of bugs, but not their absence."

Re:How does something like this happen

[Joe The Dragon]

the type who is working there ass off to get visat out on time and only has vear little time to work on xp updates.

Re:How does something like this happen

[Maxo-Texas]

I understand where you are coming from-- but I suspect if all your code was put in front of 1 million hackers, they would probably find more stuff than you would think possible.

I've been coding for 20 years and the IT-QA department catches bugs in my code (after I unit tested them- and another developer cross unit tested them) all the time. That's why they are IT-QA, it's what they do.

Re:How does something like this happen

[Rashkae]

Maybe you should ask Linus... I seem to remember a released stable kernel that neglected to sync file systems before shutting down.....

I love Linux, hate Windows, but point it, sh!t happens.

Re:How does something like this happen

[kalirion]

Some software bugs manifest in rare cases, and can only be found by code inspection or luck. Unless you work with languages that allow 100% guaranteed mathematical proofs of correctness.

Re:How does something like this happen

[coolgeek]

BS it's not a QA issue. When you're writing a file system, you must develop a test program that serves as a proof that the file system works. At least that's what they taught me back in the CP/M days. Anyway, that's not QA's job, it's the developers job. Either they failed to develop such a program, they under-developed it, or the guy coding the patch failed to run it. PERIOD.

Re:How does something like this happen

[darthlurker]

Sounds like something done because the programmer was being "extra careful". char buf[ BUF_LEN ]; while( strlen( src ) > BUF_LEN ) { memcpy( buf, src, BUF_LEN ) src += BUF_LEN; } memcpy( buf, src, strlen( src )); Then someone comes along and feels it would be safer if the buffer was initialized for the first pass: char buf[ BUF_LEN ]; memset( buf, 0xDF, BUF_LEN ); // Better safe than sorry(?) while( strlen( src ) > BUF_LEN ) { memcpy( buf, src, BUF_LEN ) src += BUF_LEN; } memcpy( buf, src, strlen( src )); // Whoops! buf is now 4k with "garbage" in end.

Re:How does something like this happen

[edmudama]

Baloney. Don't blame the testers if they can't find all the bugs written by a poor programmer. It's the [good] programmer's job to test their own code first, as they have the most intimate knowledge of all the ways it could fail.

Re:How does something like this happen

[aaronl]

Not in this case... the issue is that the last so much of a file is *overwritten* with 0xDF. Had the erroneous data been appended, padding it out to a 4kb boundary, then you would be correct.

Plus, why would you pad with 0xDF instead of null? (There might be a reason, but I don't know of it.)

Re:How does something like this happen

[LinuxDon]

At least in the case of Linux it would have been patched before the article reached slashdot! ;)

Re:How does something like this happen

[DragonWriter]

It's not the programmer's job if the programmer's employer has passed that job to someone else. Now, if the programmer is the best person to do that and the programmer's employer has directed that they not do it and that someone else do it, then its the employer's fault for adopting a flawed process.

Re:How does something like this happen

[Rob Kaper]

Plus, why would you pad with 0xDF instead of null? (There might be a reason, but I don't know of it.)

So this is how Microsoft claims support for ODF. Clever.

Re:How does something like this happen

[danpsmith]

You care about integrity? No wonder you don't work for MS.

Re:How does something like this happen

[homesteader]

This is the farmer sowing his corn,
That kept the cock that crowed in the morn,
That waked the priest all shaven and shorn,
That married the man all tattered and torn,
That kissed the maiden all forlorn,
That milked the cow with the crumpled horn,
That tossed the dog,
That worried the cat,
That killed the rat,
That ate the malt
That lay in the house that Jack built.

Re:How does something like this happen

[IllForgetMyNickSoonA]

Why?

File system is handled by a kernel. File system compresses files before writing them to the disk, respectively decompresses them during read operations. Therefore, the compression is handled in kernel. Where would you handle it?

Data compression is not like black magic. As the matter of fact, the most data compression algorithms out there are mind boggingly simple and very well understood.

Of course you could move the file system into the user space, but that would introduce some bad performance penalty.

Re:How does something like this happen

[abigor]

No, that's exactly where it gets handled. Using Linux as an example, different filesystems, compressed or not, are kernel modules accessed via the VFS. cramfs is a (rather lame) compressed filesystem built right into the kernel. Same with squashfs. Linux also has strong encryption (the CryptoApi) built right into the kernel for use with encrypted file systems.

Also, you may remember the file corruption bug from an older version of the 2.6 kernel - was it 2.6.10? It was much worse than this one from MS, which only affects compressed files on Windows 2000 SP4.

Re:How does something like this happen

[abigor]

"the same kind of programmer that lets buffers overflow."

You mean the authors of sendmail? Or maybe you were referring to the various Linux kernel root exploits? Oh wait, you meant the authors of PHP.

Re:How does something like this happen

[sm62704]

Well, now, I dodn't say "only microsoft" now did I? As they say, keft shift happens and that last guy who was perfect got nailed to a tree.

I screw up too. But yes, a bigger screw up would be to not correct it immediately.

A Paradox...

[__aaclcg7560]

If data is being silently corrupted, is there a problem if no one can hear it? That could explain Microsoft's silence.

Re:A Paradox...

[danceswithtrees]

is there a problem if no one can hear it?

Do you mean a write-only hard drive? Sort of like write-only memory http://en.wikipedia.org/wiki/Write_Only_Memory?

Re:A Paradox...

[TheGratefulNet]

"if bits fall off the platter but there's no one there to notice, are they still lost?"

How to avoid

[neonprimetime]

assuming you're using Windows

It has been confirmed that either turning off the compression attribute (disk space permitting) OR uninstalling KB920958 will prevent further loss of data.

Re:How to avoid

[PFI_Optix]

"assuming you're using Windows " ...if you're using Linux, the process is far more complex. Got a Mac? You're screwed.

Re:How to avoid

[SheeEttin]

Well, if you're installing Windows patches on Linux or a Mac, you're screwed already...

Re:How to avoid

[CosmeticLobotamy]

I wish I had one of those cute ASCII graphics of a circle going over a tiny guy's head handy. I'll try to make my own, but I'm probably gonna screw this up.

0
----
| <-You
/\

o <-Joke

... Crap.

Re:How to avoid

[sm62704]

Well, if you're installing Windows patches on Linux or a Mac, you're screwed already...

Linux and OSX are Windows patches!

If the RIAA et al subpoena you

[maartynp]

be sure to place your music files in compressed folders and let the 'evidence' self destruct at the hands of thine OS...

Re:If the RIAA et al subpoena you

[godefroi]

Hopefully that's a joke. Pretty much nobody would put music on a compressed drive, as nearly ALL of the music formats in common use today are compressed. Rather heavily. Those music formats that aren't don't compress very well anyway.

Additionally, the thought that MS would release a patch that intentionally corrupts data is unthinkable, for ANY corporation. The civil (and possibly criminal, who knows) liabilities would be ENORMOUS.

Re:If the RIAA et al subpoena you

[Jorgensen]

Liabilities? I'm not so sure about that - the "EULA" goes a loooong way to covering their backside all the way down.... I agree that it's unlikely to be intentional though.

Re:If the RIAA et al subpoena you

[abigor]

And here, ladies and gentlemen, we have in the parent post the Stupidest Comment of the Year. Yes, the only thing compressed disks contain is music, so MS intentionally corrupts them to satisfy the whims of the music oligarchy.

How old are you?

Re:If the RIAA et al subpoena you

[cheshire_cqx]

Especially since mp3s will not compress anyway.

Re:If the RIAA et al subpoena you

[EndlessNameless]

The EULA would not absolve them from liability in the event of *intentional* malfeasance.

RAID

[Karma Farmer]

As is often pointed out on slashdot, this is why it's so important to have a good backup plan. Like most slashdotters, I recommend RAID.

Re:RAID

[phoenix.bam!]

RAID is not data backup. It is hardware backup. In this situation the RAID would just have multiple copies of the same file. Data backup is done with tape. Tape you can go back to and get an older version of the file, RAID offers no such solution.

Re:RAID

[khang]

wrong, RAID would just mirror the data corruption

Re:RAID

[Lehk228]

RAID will do absolutely nothing to protect data written incorrectly by the OS

RAID is not a backup

Re:RAID

[TangoCharlie]

Yeah! Why have 1 copy of a corrupted file? Two corruted copies is much better!!

Re:RAID

[sm62704]

Like most slashdotters, I recommend RAID.

I prefer Black Flag myself.

Re:RAID

[kj_in_ottawa]

Pehaps MS should use RAID... I hear it kills bugs dead

Re:RAID

[isolationism]

I can't believe there were > 0 people who replied to Karma Farmer's comment thinking it was anything but an attempt at humour/troll, much less that any such poster would get their manties in a knot over it either.

Re:RAID

[jonnythan]

That was classic. Just the right mix of replies.

Kudos to you, my friend.

Re:RAID

[clickclickdrone]

Why is it that slashdotters by and large are unable to spot jokes? Time and time again I see someone post something funny which then generates hoards of posts by people taking it seriously "oh no, I really think you do need oxygen tanks and scuba gear underwater, just chewing oxygum isn't enough".

Close?

[Aladrin]

"close to a multiple of 4K in size"

How close is close? Is 162k close to 164k? Sounds like it is to me. From the examples in the discussion cited, it seems that anything over 4k is at risk, not just things 'near' a 4k boundary.

I would even hazzard to guess that the size matters not at all, but rather the contents of the files. If the contents match a certain pattern, the compression goes awry and adds the garbage to the end. (Accidentally overwriting the real data.)

Who wants 'encryption'??

[KlomDark]

Tagline's wrong, compression and encryption are two very different things!

And BTW, I got this same story rejected last week. Fuckers.

what i think

[robpoe]

Well, it's interesting that 0xDF0xDF0xDF0xDF0xDF0xDF0xDF0xDF0xDF0xDF0xDF0xDF0x DF0xDF0xDF0xDF0xDF0xDF0xDF

Re:what i think

[Stavr0]

I agree, some other people have meßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßß ßßßßßßßßßßßßßßßßßßßß

Re:what i think

[HTH NE1]

I agree, some other people have meßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßß ßßßßßßßßßßßßßßßßßßßß

Oh, that explains it: it's a beta patch.

Re:what i think

[Stavr0]

I agree, some other people have meßßßß . . .

Oh, that explains it: it's a beta patch.

ß / 0xDF is &szlig ; or Esset. So the article is incorrect, the last bytes are overwritten with random data in the form of white noise. "ßßßßßßßßßßßß" is pronounced "ssssssssssssssssssssssssss". OMFG!11! SNAKES ON A PLATTER!

Re:what i think

[Jerf]

Just FYI, that's actually the German "Eszett" or "scharfes S".

Just 'cause it's interesting, not intended as a correction.

MS06-049 ...

[Anonymous Coward]

nicknamed (0x)DF aka Data F**ker

Re:MS06-049 ...

[drkfdr]

Destroy Files! DF is a nice value.

What type of programmer?

[Marbleless]

Human ones.

Have you never had a bug in software you've released?

If yes, you are human and fallible just like the poor sods who should get their bums kicked for this stuff up.

If no, then either you aren't telling the whole truth or I have to welcome you as our new alien overlord etc etc etc ;)

Re:What type of programmer?

[ifrag]

Have you never had a bug in software you've released?

Nope, that's just high energy neutrons impacting your PC causing soft errors during execution. Oh don't worry, when the sun finally consumes itself that code will run fine :)

0xDF

[DirtyFly]

0xDF :)

this reminds me of a virus for the Amiga computer that replaced all the files content with the word LAMMER :)

Jorge

http://www.retroreview.com

Re:0xDF

[DirtyFly]

http://en.wikipedia.org/wiki/Lamer_Exterminator

Strange

[A beautiful mind]

I've never heard Windows called MS06-049 before...

Re:Strange

[Duhavid]

Cause only THX1138 referes to it that way.

Heh

[3.5 stripes]

Good troll.

More background please...

[Chris Pimlott]

The summary blurb is rather cryptic. MS06-049 is a patch to... what? Just Windows 2000 or XP too? And this was a patch for some vulnerability, assumedly? Which?

After a bit of research, here's what should have been included: MS06-049 was an elevation of privledge issue discovered in the kernel of Windows 2000 SP4 only. The patch for the issue, KB920958, appears to have a bug resulting in corruption of compressed folder.

The title is misleading as well. MS06-649 is the issue and KB920958 is the patch; the patch is what's causing the corruption, not the original issue.

You can stop now

[rbarreira]

I have been programming for a long time and I never had stuff like this happening.

If other replies to you have accurately guessed the truth and, as they say, you've never worked on a complicated piece of code, I will save you from spending any more "long time" working on your programming. Here is the program you're trying to make:

 

#include <stdio.h>
 
int main (void)
{
    printf ("Hello world!\n");
    return 0;
}

Re:You can stop now

[0xABADC0DA]

I hate to burst your bubble, but you did not check the return code from printf. What if stdout is closed, as in "./a.out >&-"?

Original troll never writes any bugs, so his hello world is more like this:

int main(int czArgCount, LPSZ *lpszArgv[]) {
    if (-1 == printf("Hello world!\n")) {
        if (errno == EBADF) {
            if (-1 == fprintf(stderr, "Error stdout closed!\n")) {
                int fdTty = open("/dev/tty", O_WRONLY, 0666);
                if (fdTty != -1)
                    write(fdTty, "Hey dumbass dont close my streams\n", 34);
            }
        }
        exit(1);
    }
    exit(0);
}

Re:You can stop now

[Frank T. Lofaro Jr.]

You failed to check the return value of printf!

Re:You can stop now

[joe_bruin]

I hate to burst your bubble, but you did not check the return code from printf. What if stdout is closed ...

Your program fails to take into account the case that printf(), fprintf(), and write() printed less characters than those that you provided. It further does not handle getting an EINTR on write().

RETURN VALUE
              On success, the number of bytes written are returned

Re:You can stop now

[jjon]

Undefined symbols errno, LPSZ, EBADF, and O_WRONLY.  Please include some header files.

Also the wrong type for argv:
LPSZ *lpszArgv[]  - Wrong
LPSZ lpszArgv[]   - OK
LPSZ *lpszArgv    - OK
char **lpszArgv   - Better (Why use Windows typedefs for no reason?)

Re:You can stop now

[0xABADC0DA]

The code was supposed to be funny on multiple levels, including the more than several errors I included on purpose. You obviously missed the subjoke about hungarian notation causing more errors than it prevents. ;-(

Why even bother with compression anymore?

[jonwil]

In the modern age where hard disk space can be had for so cheap, why would you even want to bother with disk based data compression?

Re:Why even bother with compression anymore?

[Quila]

Logs? If you have many servers all writing gigabytes of detailed logs, and they have to be kept for a long time, you can save a lot of $$$ worth of space by running compression.

Re:Why even bother with compression anymore?

[jZnat]

But logrotate will take care of that for you using a user-space program (gzip, or any program you specify). You don't just make /var/log a compressed filesystem...

Re:Why even bother with compression anymore?

[coolgeek]

How can you calculate the cost of CPU cycles? If a $150/hr programmer is sitting there waiting whilst CPU cycles are being burned, those cycles cost much much more than when your $15/hr customer service rep is putting a customer on hold.

Re:Why even bother with compression anymore?

[doesnothingwell]

Remember Stacker, Drivespace, Doublespace and maybe a couple of others? I watched others lose data and I took that as a sign. "But its like free disk space..." they said. Its like watching a gratuitious shot to the groin, sure glad it wasn't me. My tin foil hat is starting to itch, maybe M$ is stirring up the older OS sediment to spur upgrades?

Re:Why even bother with compression anymore?

[241comp]

Because processing power is even cheaper. Especially when you are doing archival operations such as backups - write once, read once (maybe).

Re:Why even bother with compression anymore?

[Lagged2Death]

You don't just make /var/log a compressed filesystem...

I'm no MS fanboy, but... suppose the OS in question had some sort of directory-compression scheme that had a seven-year track record of impressive stability and effectiveness? Why not use it?

Disk compression earned a terrible rep back in the 90s, when DOS/Windows and Windows 95 themselves were so unstable there was no chance that it could work properly. But MS finally got it right when they swiped tech from Stacker and included directory compression in NTFS. I've never heard of anyone having a problem with it until now.

Back when I up my home Windows 2000 box, disk space was less cheap and I was more poor, so I've got some compressed folders to un-compress. Curse you, Microsoft! Stop screwing up the few things you've done well!

Re:Why even bother with compression anymore?

[tylernt]

Think bigger.

Hundreds of VMWare virtual disks is one place compression comes in handy. Without compression you'd need a multi-terabyte array. With compression, you only need half a terabyte.

Another use is for backup servers, where storing hundreds of GB of data is important but speed is not (and you hate tapes).

Another use is when your server fills up and you don't have a budget for new hardware until the next quarter.

It all adds up to big hardware savings, and indeed is the only reason some of my servers -- sadly -- are Windows instead of Linux (which does not have a reliable on-the-fly-compression filesystem yet).

Re:Why even bother with compression anymore?

[dbIII]

are Windows instead of Linux (which does not have a reliable on-the-fly-compression filesystem yet)

It's called tape and hardware compression :). There always is the option to use gzip, bzip2, jpeg, png, mp3, mpeg4 or a variety of other methods to compress data. You really should choose the correct one for the job instead of having a one size fits all method - with tapes I often get far better results turning off hardware compression if the data is in a format that is hard to compress by it's text focussed algorithm, but I'm dealing with geophysical data (think of sampled audio with a lot of tracks).

In a few linux distributions the manual pages are compressed with gzip and are unzipped into memory by the manual reading program. You don't have to do things at the filesystem level.

Re:Why even bother with compression anymore?

[tylernt]

I'm using snapshot-style rsync backups, so gzip is not an option.

http://www.mikerubel.org/computers/rsync_snapshots /

We can combine rsync and cp -al to create what appear to be multiple full backups of a filesystem without taking multiple disks' worth of space. Here's how, in a nutshell:

rm -rf backup.3
mv backup.2 backup.3
mv backup.1 backup.2
cp -al backup.0 backup.1
rsync -a --delete source_directory/ backup.0/

If the above commands are run once every day, then backup.0, backup.1, backup.2, and backup.3 will appear to each be a full backup of source_directory/ as it appeared today, yesterday, two days ago, and three days ago, respectively--complete, except that permissions and ownerships in old snapshots will get their most recent values. In reality, the extra storage will be equal to the current size of source_directory/ plus the total size of the changes over the last three days--exactly the same space that a full plus daily incremental backup with dump or tar would have taken.

Re:Why even bother with compression anymore?

[dbIII]

Good point - a compressed filesystem would help you, and this sort of mirror is handy (I do it too for a few volumes all under 80GB so I don't have to care about compression). A lot of people use the word "backup" for this sort of mirror, I have too by accident - but I have real backups on media I can keep for a long time and store in another building as well. If it is important enough it is on more than one tape.

I really wouldn't trust any experimental compressed filesystem for anything that you can't get back another way without a lot of work - so I think you are correct that there is no disk option in linux that will suit. Anyone have any examples that prove this wrong? - preferably some text and not just a picture of someone pretending to be a dancing tree.

Those files were important.

[Anonymous Coward]

Those files were important! Sheißßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßßß

When you have a monopoly

[Colin Smith]

What're your customers going to do?

Re:When you have a monopoly

[HTH NE1]

When you have a monopoly what're your customers going to do?"

Well I believe I'll invest in a second-party operating system!

Re:When you have a monopoly

[Tackhead]

> When you have a monopoly
>
> What're your customers going to do?

The guy at the keyboard of a Windows Vista box, using Microsoft Office at work, and Windows Media Player at home is not the customer, he is the product. The customers are Dell, AOL, media licensing conglomerates, and so on.

Re:When you have a monopoly

[IHateAllofYou]

Hopefully they will get wise to the sham that are the majority of your products and either switch or complain endlessly.

I bought the XP Pro license (like a dumbass I know) after pirating it for a couple years in an attempt to do the right thing and it really just made things worse somehow. Maybe thats because I don't have an easy way to take off SP2 with my legit license.

Of my 8 computers though it's 1 OpenBSD, 3 Slackware, 3 Solaris and 1 Windows. Hopefully by the time XP is at its EOL Transgaming Cedega will have seriously matured.

If you don't like they're priorities switch. Its not an option for some but Microsoft won't listen until they lose serious market share.

Re:When you have a monopoly

[theCoder]

That may be accurate for televion broadcasts, but it isn't so for Microsoft. Customers are people who pay for services. AOL and the media companies aren't paying MS anything, other than licensing fees for the services they use from Microsoft (i.e., their Windows PCs). Microsoft is paid by the guy at the keyboard of the Windows box (or his employer).

Microsoft may be able to leverage all those customers into a product for another customer (such as advertising or licensing DRM solutions), just like the movie theater leverages their movie watching customers into a product for advertising. Until Windows is free (as in beer), the guy using Windows is a still a customer.

Re:When you have a monopoly

[zecg]

Best comment I've read in a long time.

Re:When you have a monopoly

[Citizen of Earth]

Well I believe I'll invest in a second-party operating system!

I think most of them will just say "Please use more lubricant next time."

Re:When you have a monopoly

[noidentity]

Like the parent says, the only data they care about is that which can just be re-downloaded if it becomes corrupt (but you'll have to re-buy it too).

Re:When you have a monopoly

[dennisparrott]

well, i still choose to see myself as "the customer"... and since i am the one shelling out the $$$$ for that crap, in the future i am going to choose not to play.

ubuntu linux is easier to (re-)install than windows xp and you don't have to spend hours on windowsupdate to get it "patched". when flash 9 or better plays on linux, i will be making my exit stage right, thankuverydamnmuch.

they can keep their drm-jail, their tcpa crapola, their data corruption, their high prices, ...

yep. it will be nice being free.

Re:When you have a monopoly

[hunterkll]

An enterprise with only dozens of workstations? Man, upsize your enterprise for synergistic effect or downsize your words! :D

Or if you put down the tinfoil hat

[jbellis]

maybe one patch was just easier to write.

--
Carnage Blender : Meet interesting people. Kill them.

Re:Or if you put down the tinfoil hat

[Alien Being]

Even if this is a tricky problem to fix, MS could at least warn their customers about the problem.

After 25 years of dirty tricks from Redmond, you have the gall to call their critics paranoid?

Re:Or if you put down the tinfoil hat

[JimDaGeek]

Or maybe just keep apologizing for MS? Even if this patch is somehow more complex, why not announce it? Why not throw some more brains and eyeballs at it? It would be a really big coincidence if the DRM patch was "just easier to write". I somehow doubt that. And looking at the history of Microsoft over the past decade, I wouldn't bet on the fact that this patch is being pushed to get finished just as hard as the DRM patch was pushed. Nope, this is just another example of Microsoft's priorities. Big media companies and their licensing fees are more important for MS. Microsoft wants their DRM crud to be used as the #1 DRM "solution". So MS will bend over backwards for the industries that want DRM....until they have the market that is. Then it will be another low priority task on the list of things to do. Microsoft, bleh.

Quick!

[jZnat]

Someone figure out how to apply this problem to Windows Media DRM and we'll get a fix in no time!

Possibly some weird M$-esque operator

[gatkinso]

...similar to their (in)famous debug version of the new operator (IIRC generates guard bytes set to 0xCDCDCDCD).

While they are doubtlessly not releasing images with debug info, they might be using an overriden new operator that does something similar (for a variety of reasons).

It is hard to say, but this type of error - while *not* acceptable, *is* understandable,

Still no response from Microsoft

[astaines]

As of two minutes ago a search on http://search.microsoft.com/results.aspx?q=KB92095 8&l=2&mkt=en-US&FORM=QBME2 showed no reference to data corruption. Any tech journalists reading this?

Re:Still no response from Microsoft

[cyberbian]

It appears the original discussion comes out of the TechNet Discussion Groups:

TechNet Discussion on Weekend Update

Re:And this is NEWS?

[Joe U]

MS products have been corrupting data esp. with respect to compression for a LONG TIME.

I've never heard of this, how about some proof?

You made a potentially libelous allegation, please back it up with some facts.

2000 only.

[antdude]

It seems like the problem is only for 2000 according to this reply.

Re:Has anyone seen this problem?

[tttonyyy]

I use compression on folders in XP Pro. and Home SP2. I have not seen this problem on my systems at home and work. I always get the newest patches on their first release dates. I even defragged (PerfectDisk v6.0 with its patches) over the weekend. I haven't seen anything odd. I am usiDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFD FDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFDF

You might want to double check. ;)

Re:Has anyone seen this problem?

[phasm42]

The patch only applies to Windows 2000 SP4. Also, it happens when you create files -- it does not proactively seek out and corrupt existing files.

Welcome to the world of sarcasm

[Inominate]

nt

Re:Welcome to the world of sarcasm

[Lehk228]

unfortunately people misunderstanding RAID is so common it's hard to tell sometimes

How about using a Beowulf Cluster?

[JoshDM]

Here I am losing my geek license twice today; first by not knowing ^H and now by not knowing what the heck a Beowulf Cluster is and yet referencing it.

Forget it, it's a bug

[dtfinch]

As a matter of policy, Microsoft generally doesn't fix bugs in already released software, with the exceptions of publicly known security flaws (and then only once a monthg), service packs (notice WinME has had zero service packs), and $50 hotfixes. Since Microsoft now depends on returning customers more than new customers, and their customers have little chance of switching vendors, they have every motivation to make older versions as unpalatable as possible.

I personally haven't seen any files corrupted though. We'd see much more than a few newsgroup postings if this was a widespread problem.

Re:Forget it, it's a bug

[dtfinch]

After a bit of checking, we have a 2000 server with the patch, and it has a number of compressed files, but none of the compressed files that were updated since the patch are roughly a multiple of 4k and mostly uncompressible, the two other requirements for corruption.

Re:Forget it, it's a bug

[plague3106]

As a matter of policy, Microsoft generally doesn't fix bugs in already released software, with the exceptions of publicly known security flaws (and then only once a monthg)

Not true; updates are released every tuesday.

service packs (notice WinME has had zero service packs)

While Win2k and XP have at least two each. ME was a crumby OS which was meant to have a new OS for people that had to have a new OS every couple of years.

and $50 hotfixes

I've never had to pay for a hotfix, nor has anyone that has gotten the hotfix and had it solve the exact problem for which the patch was intended.

Since Microsoft now depends on returning customers more than new customers, and their customers have little chance of switching vendors, they have every motivation to make older versions as unpalatable as possible.

Most don't buy Windows; Windows comes with their new PC. Its 2006. Why is someone running a six year old operating system anyway? Certainly they can if they want, but to expect support from the manufacter is silly. Hell, cars have warranties which expire sooner than that.

Re:Forget it, it's a bug

[DragonWriter]

Most don't buy Windows; Windows comes with their new PC. Its 2006. Why is someone running a six year old operating system anyway?

Uh, well, WinXP was released in 2001, so its only about a year and a half newer than Win2K. I really don't see a huge difference between running a 6.5 year old OS and running a 5 year old OS.

Certainly they can if they want, but to expect support from the manufacter is silly. Hell, cars have warranties which expire sooner than that.

So? Cars aren't very much like computer operating systems.

Re:Forget it, it's a bug

[plague3106]

Uh, well, WinXP was released in 2001, so its only about a year and a half newer than Win2K. I really don't see a huge difference between running a 6.5 year old OS and running a 5 year old OS.

I dunno perhaps because there are a lot of changes in XP; namely that SP2 adds a bunch of stuff and fixes a huge amount of features. At the very least because they actually still sell XP.

So? Cars aren't very much like computer operating systems.

Yes. For some reason computer users expect their computer to be supported until the user dies of old age. They also expect free fixes to boot. Just because they expect it though does not mean its reasonable.

Re:And this is NEWS?

[cyberbian]

As early as MS-DOS 6... and/or earlier?

Can Cause Corruption

Please don't make me pull out my service logs to show the number of cases where doublespace accidentally corrupted an entire C: drive... I might have to get medieval on ya...

Clean up in aisle five!

Re:How does something like this happen?

[140Mandak262Jamuna]

My pure guess based on /. comments: How can this happen? Loop counting error. Probably from integer division of the file size/ 4K chunks. Allocator did it right, loop missed the last chunk. Very common. Typically a novice error. But can blindside even experienced ones. QA should have nailed it.

Re:Has anyone seen this problem?

[Opie812]

That made me lol.

If only I had mod points. (and knew what the hell that meant)

Re:scandisk

[ocelotbob]

Yeah, because disk corruption issues are unheard of on the mac.

Compressed files, are you kidding me?!

[dave562]

This is a bit of a tangent, but a somewhat relevant one none the less. But first of all, bad Microsoft! You freaking imbilices (probably misspelled to show how dumb I am too.)

Is anyone out there seriously using disk compression in a production environment? Didn't anyone teach you guys that disk compression is a crutch and not a solution? For as long as I've been working with servers, all of my mentors have led me to believe that it is pretty much generally accepted practice not to use disk compression due to the potential for data corruption and the performance hit your servers take. If you need to compress files to save space, throw them onto some LTO or DLT media and pull them completely offline.

If you're working for a company that can't come up with more money for disk space, maybe you need to click on the Dice.com adds that are all over /. here.

Re:Compressed files, are you kidding me?!

[Anonymous Coward]

Actually with sufficient processing power, it can be a performance win, since you're reading less from the disk.

Re:Compressed files, are you kidding me?!

[dave562]

Actually with sufficient processing power, it can be a performance win, since you're reading less from the disk.

I could believe that. But whether or not it's a performance hit, I'm still going to stick to the school of thought that says your asking for trouble if you're compressing your files. That might show that I'm stuck in the neolithic days of the mid-1990s, but so be it. =)

Re:Compressed files, are you kidding me?!

[MrP-(at work)]

By default windows compresses all windowsupdate/service pack uninstall directories (i.e. c:\winnt\$NtUninstallKB123456$), it also compresses the dllcache directory (which keeps backups of system dlls and drivers)

Re:Compressed files, are you kidding me?!

[dave562]

In both instances, it makes sense to compress that data on a case by case basis. I'd draw the analogy to using .zip compression to compress a few files. The kind of compression that I'm saying is a bad idea to use is the compression that compresses EVERYTHING on the entire volume / partition. That's the kind of compression that will come back and bite you in the ass. I'm not sure if you remember the days of DOS 6.2 and DoubleSpace, but after experiencing that shoddy product first hand, I'm wary of trusting anything Microsoft puts out that proports to safely compress your data.

Re:Compressed files, are you kidding me?!

[mibus]

I'm still going to stick to the school of thought that says your asking for trouble if you're compressing your files.

Sure, you do that. In the meantime, I'll continue using JPEGs, MP3s, ODF, ... :)

Re:Compressed files, are you kidding me?!

[ZorbaTHut]

I have some compressed files. They're sparse files generated by a program that doesn't actually handle sparse files well. Conveniently, the program is smart enough to write all 0's to them, so I just compress the directory and be done with it. As I write this, I'm saving about 80% of the disk space they would otherwise use.

They're not performance-critical, so in this case, Windows' compression is incredibly handy.

Similarly, I have a few disc images that I keep around solely so I don't have to put DVDs in the drive when I want to play games. These tend to be "CD hack only" images, so the only parts of them that aren't solid 0's are the parts necessary to fool copy protection. One of them is 1gb uncompressed and 700kb compressed. My disc image mounter still mounts it flawlessly.

Compression can be very, very useful sometimes.

Re:Compressed files, are you kidding me?!

[dave562]

Sure, you do that. In the meantime, I'll continue using JPEGs, MP3s, ODF, ... :)

Those aren't volume wide compression algorithms. Apples and oranges man.

Not necessarily....

[King_TJ]

I just installed a *bunch* of Windows XP patches on my Mac Pro last night. Hey, just 'cuz it's a Mac doesn't mean I want the XP installation on a second hard drive in it to be unpatched!

But yeah, yeah... I got the joke.

Good for Microsoft

[VonSkippy]

With 300G drives costing less then $100, what type of cheap bastard uses file compression? //in my day we had 10M hard drives - and we LIKED IT!

Re:Good for Microsoft

[GotenXiao]

Freaking masochist.

They should force them to mention in advertising!

[LinuxDon]

The EU should force them to mention it in their advertising when they shout it out for having such a reliable OS.
Since the advertising is in great conflict with the EULA.
Just like the "Smoking can cause cancer":
"Using this or other Microsoft products can cause critical data loss, system instability and significant loss of profit for any reason, Microsoft accepts absolutely no responsibility."

Re:Has anyone seen this problem?

[crawling_chaos]

Also, it happens when you create files -- it does not proactively seek out and corrupt existing files.

Yep. That one's still in QA.

three letters

[Morphine007]

DOD

In Soviet Microsoftia...

[Shadyman]

...Hard drive compresses you!

Re:Has anyone seen this problem?

[antdude]

Ha ha, very funny. :P Maybe you should check yours because of tttonyyy. [grin]

Re:And this is NEWS?

[Joe U]

"A client must enter the Windows Disk Critical Section before calling the direct call entry point of a MRCI server. This protects the MRCI server from being reentered. Failure to do so will cause data corruption and data loss in a multitasking environment."

So basically, program according to spec or you can mess up your data. That's not a flaw, and not the MS program corrupting data, so try again.

As for your service logs where doublespace 'accidentally' corrupted a C drive, do you have a log of what the user was doing before? I'll give you a scenario.

User who has been using DOS for years always turns computer off via power switch.

Enter Doublespace and Smartdrive with write caching.

User saves large quantity of data and turns off computer via power switch.
Open files are not saved, files in process of being written are corrupted.

Is the problem with Microsoft or the User?

Re:And this is NEWS?

[kb0hae]

Hi. Well, there was Doublespace. later there was Drivespace. There were also some non-MS programs. The basic way that they worked was that everything on your hard drive except a few files needed to boot the computer and manage the compression were made into 1 huge compressed file. The problem was that if ONE BIT of data in the compressed file was corupted, you most likley lost everything.

I tried several of these types of programs and never got nearly the compression that was promised. Good thing I had a tape backup at the time!

I have to agree with another poster here...with large hard drives going for low prices these days who needs to compress folders? Files maybe if they are transfered over the internet (or via a media with limited space).

Ubuntu n. Ancient African word meaning can't install Debian!

Re:Has anyone seen this problem?

[tttonyyy]

Ha ha, very funny. :P Maybe you should check yours because of tttonyyy. [grin]

A fair point. :D

MS06-049 = Lotto649 ?

[kj_in_ottawa]

Just did a re-read of this and it seems the significant digits are the same as one of the prominent Canadian Lotteries. Is this coincidence or does it say something of the likelihood of you getting your data back. .-.-. -.- .--- ...-.-

The LAMER Exterminator !!!

[Schraegstrichpunkt]

This is completely unacceptaLAMER!LAMER!LAMER!LAMER!...

What kind of idiot ...

[Schraegstrichpunkt]

... makes such massive changes to the VM of a stable kernel that allows this sort of thing to happen in the first place?

Oh wait...

Re:How does something like this happen?

[marko123]

Loop counting errors can be known as a "heuristic fencepost error," one of my favourite error names, next to "bad magic number."

In essence, "Count a fence's posts, then loop using the number of joins between the posts as a counter. You will miss one fence post."

classic excuse

[fusion9290991]

Windows ate my homework.