Slashdot Mirror
Vista Runs Out of Memory While Copying Files
ta bu shi da yu writes "It appears that, incredibly, Vista can run out of memory while copying files. ZDNet is reporting that not only does it run out of memory after copying 16,400+ files, but that 'often there is little indication that file copy operations haven't completed correctly.' Apparently a fix was scheduled for SP1 but didn't make it; there is a hotfix that you must request."
the box I "make use of" has just 15,000 mp3s...
They who would give up an essential liberty for temporary security, deserve neither liberty or security - Ben Franklin
16k files should be enough for everybody.
At the end, there will be free therapy. And Cake!
Physics is nothing like religion. If it was, we'd have an easier time trying to raise money!
And that is one of many reasons we are all still running XP
Copyright 2010. All rights reserved. This comment may not be copied in any way including, but not limited to caching.
Isn't it a little odd that if you strip off the first and last digits of the number "16,400", it's 640, as in 'no one needs more than 640k"?
Start a happiness pandemic
M$ is scared that people will try to copy their documents to another computer before reverting back to XP. Smart, very smart Micro$oft! On a tech note, what kind of number is 14,600? I would have thought 16,384 would be better.
void r() { printf("recursion is "); r(); }
I don't want to start a holy war here, but what is the deal with you Vista fanatics? I've been sitting here at my freelance gig in front of a Vista PC (an Intel Core 2 Duo w/4 gigs of RAM) for about 20 minutes now while it attempts to copy a 17 Meg file from one folder on the hard drive to another folder. 20 minutes. At home, on my ancient Mac running OS 9, which by all standards should be a lot slower than this Vista PC, the same operation would take about 2 minutes. If that.
In addition, during this file transfer, Firefox will not work. And everything else has ground to a halt. Even Notepad is straining to keep up as I type this.
I won't bore you with the laundry list of other problems that I've encountered while working on various Vista PCs, but suffice it to say there have been many, not the least of which is I've never seen a Vista PC that has run faster than its Mac OSX counterpart, despite the Vista PC's same chip architecture. My 286/12 with 2 megs of ram runs faster than this 2.4ghz mhz machine at times. From a productivity standpoint, I don't get how people can claim that Vista is a superior operating system.
Vista lovers, flame me if you'd like, but I'd rather hear some intelligent reasons why anyone would choose to use Vista over other faster, cheaper, more stable systems.
they can only send 16,000 files to the RIAA and MPAA to check, at once.
If you are using Vista as a server, you pretty much deserve what ever happens to you.
Well.. maybe. Or Maybe not. But Definitely not sort of.
Oh, the fools! If only they'd built it to let you copy 16,401 files!
I have 13K+ music tracks on a backup disk. If I try to copy them with the Explorer UI, it does nothing - No error message or anything. I reverted to Robocopy, which works fine. You must be doing the same thing. Doesn't anyone at Microsoft have a big music collection to copy, or do they just use their Macs and iPods for that?
Imagine how much harder physics would be if electrons had feelings! -Feynman, maybe
Maybe you're backing up to an external hard drive?
The underlying problem is a Windows OLE component memory leak. Microsoft has a hotfix for the issue at http://support.microsoft.com/kb/942435/en-us
-- "I never gave these stories much credence." - HAL 9000
.. when your GUI is using 2 gigs of RAM.
The reason being is I've setup a Vista system and copied about 100,000 files (totaling about 60GB) drive to drive in a single operation, without error. So while I'm not saying this isn't a Vista error, I'm wondering what else has to be done to trigger it. The persisting across reboots, even if you break it down smaller really makes it sound like another program is somehow interfering with the copy. I'll have to mess around with it at work, we have Vista test machines and Cadence installs north of 250,000 files when you install its libraries. I know it installs fine, though that isn't a copy strictly speaking as it is files being extracted from archives.
I'm just wondering if perhaps there isn't more to this than just "OMG Vista runs out of memory!" If it is a memory issue, why then haven't I encountered it, doing far larger amounts of files?
How the F%$^ can this be a problem? A file copy is a simple operation. There's simply no excuse for this... This should have never been a problem in the first place. What pisses me off is that I need to buy a new laptop, Vista is now forced down my throat, and I have no option to get XP pre-installed.
Actually, the bug is in the shell, not the kernel and only files with altnerate data streams trigger the leak. The KB article that Adrian links to states that very clearly, but he's been on an anti-Windows rampage lately that's blinded him to the facts.
Very few files have data streams, so the vast majority of users won't ever see a problem. Kaspersky choses to pollute every single file with a stream, however, which is why systems with it installed exhibit the problem.
FTA: The "Out of Memory" error (which is affectionately known at the PC Doc HQ as the "Out of Cheese" error
I don't think there has ever been a version of Windows that could deal with large numbers of files. Particularly if you are using the GUI interface. The whole thing is a toy operating system, really.
A few years ago, while investigating a similar problem with a production server (a SERVER not a client machine) the machine would gradually grind to a halt doing the copy, while still responding (but slowly) to other operations.
I found that the "copy" command did much better than a drag and drop operation, but still would have a problem eventually. Finally, I found that this was a known problem, and that to solve it, a dedicated MS employee had written a utility called "robocopy" the "robo" not being for "robot", but for "robust" (really, it said that!).
Using that usually got the job done, much more slowly than it should have, but at least I didn't have to re-boot the machine daily to clear things up.
Now that Gates is too busy with other things to take tours of the data center, really, Microsoft should do itself a favor and ditch the VMS underpinnings of Windows (some of which they have probably forgotten how to maintain) and build your nice GUI on top of BSD or something similar. That way you won't break your budget (in manpower and electricity) trying to match the Google server farms.
Once that's done you will have the experience needed to do the same on the desktop. You will be doing the world, and yourselves a favor. Thanks in advance!
and then you take the 640, divide by 2, get 320, remove the last 0, and get 32, the reverse of 23.
Then you multiply 23 by 30, get 690, take 42, reverse it to 24 and subtruct this from 690. You get 666.
What does that tell you? Ha? Ha?
You can't handle the truth.
Very few files have data streams, so the vast majority of users won't ever see a problem. Kaspersky choses to pollute every single file with a stream, however, which is why systems with it installed exhibit the problem.
So it's Kaspersky's fault that alternate data streams are apparently no longer supported by Vista, despite being a basic part of NTFS?
The original rant may be found here.
"How the F%$^ can this be a problem?"
To ensure backward comparabillity. I'm a techno luddite. I got my first DVD drive this year and was slow to get CD drives. All my systems have floppy drives.
98 can be made to puke copying a big file from a floppy. If the floppy is bad you may as well reboot. Delete a few gigs from a hard drive and it goes awy for ages and will more often then not kill the gui task. This is very repeatable. Again, if the CD is bad, reboot.
I can make XP croak as well copying huge files from a CD or floppy which is handles very very badly (see 98). And my biggest drive is 20G (albeit a damn fast one). It seems to do ok copying big files from hard disk to hard disk but even with SCSI RAID with huge caches and the correct drivers you can't expect much left of your CPU when its doing this. Do two at once and you may as well go rebuild your transmission while you're waiting. Apparantly DMA and interrupts are unknown concepts at Redmond; PDP-11's did this just fine (unless you turned off DMA and interrupts in which case it was no faster than a 4Mhz Z-80 CP/M system)
There's really no excuse for this. In the days of 8 bit microprocessor systems we still went out and got the biggest pre-production drives we could to see if they'd copy ok. They may have filled a room but the Navy did indeed have 100 megs online pumping its data through an 8085. Eventually. We knew it'd work cause we tried it. This was 1981.
This is why they use real (IBM, SUN) computers to serve up say, the root or com zone. The root zone isn't big but the com zone is. Copying it isn't a problem on any unix system I've tried, just don't try to load it into BIND on anything but a massive computer or it'll just hang. And not gracefully either.
Windows is for games and sometimes works well enough to run some office tools. As long as you don't need accuracy.
Big files or LOTS of small files are a problem for computers. This isn't news folks. It's just sloppy carelessness.
Need Mercedes parts ?
What does that tell you? Ha? Ha?
You've got too much time on your hands?
"We returned the General to El Salvador, or maybe Guatemala, it's difficult to tell from 10,000 feet"
mod parent as stupid.
Likely, they're allocating memory to store file attributes or some such that are not being free'd when done with. Hence running out of memory. If you had coded a day in your life you'd see that.
Tom
Someday, I'll have a real sig.
only files with altnerate data streams trigger the leak
Well that's what you get for crossing the streams. Egon warned us. Kaspersky's risking total protonic reversal. I guess they were fuzzy on the whole good/bad thing.
Developers: We can use your help.
This is a minor problem, absolutely rare event, occurs with next to no regu...
**OUT OF MEMORY ERROR, SYSTEM HALT**
Karma Whoring for Fun and Profit.
Actually, a lot of the problems I've noticed with XP is related to the stupid fucking way that Windows handles it's file cache. It will literally swap out PROGRAMS YOU ARE ACTIVELY USING to expand the file cache during a large copy/read operation.
Anybody that has ever tried to alt-tab while copying huge files knows about this.... then you sit and wait for the pages to be swapped back into memory. And you might as well get some coffee, cuz with the hard drive already being pegged for the copy operation, it's gonna take awhile. Oh, and once it's finally done and you need to alt-tab back to the original program.... well, hope you need more coffee.
Lately I've been playing with a program called CachemanXP. Google it. It seems to give you more control over the memory and process management functions of Windows. It also lets you do a 'kill -9' equiv, which (as far as I'm aware) even Task Manager won't do, as it insists on trying to do a graceful shutdown first.
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
We have found the first slashdot user that has no porn.
That sort of depends if you are talking average home user or average work user. The average home user may actually have this kind of problem - since downloads to the tmp directory are then copied to the correct folder once downloads are complete. Update EQII, WOW & FFXI & you've gone a long ways towards 16K files. Add in patch Tuesday, and your average user is probably going to hit real close to 16K files if they try to keep the PC up for a month.
I probably come reasonably close to 16K files copied in a week on my work PC, so a crash like that would hit me every other week or so - not something I would consider 'Enterprise Ready'.
MS has a habit of programming for the home environment & pushing it into the Business environment.
Actually, it's more like 90 / 90000, but you're only a few orders of magnitude off.
Not necessarily his fault, maybe he used Excel to calculate it...
It's forced down your throat because you couldn't be bothered to source a laptop from anywhere but CC?
Give me a break.
"It does not do to leave a live dragon out of your calculations, if you live near him." - Tolkien
It's not a simple operation. Think about how a file copy works within Vista:
1. The file is opened.
2. The file is scanned for viruses.
3. The file is scanned for adware.
4. The file is scanned for DRM violations.
5. The user is asked if they're really sure they want to copy the file.
6. The user is asked again if they're sure they want to copy it.
7. The OS makes a judgement on how long it will take to copy so it can update the pretty stats in the gui.
8. Lots of flashy graphics and widgets are loaded to show you a pretty animation while you wait.
9. The file is copied.
10. The destination file is verified that it is intact.
11. The destination file is scanned for viruses.
12. The destination file is scanned for adware.
13. The destination file is scanned for DRM violations.
14. The file is successfully copied.
Hell - I'm surprised their OS can even handle copying 1,600 files, let alone 16,000.
Just disrupt the deflector shield with a tachyon burst.
"Very few files have data streams, so the vast majority of users won't ever see a problem. Kaspersky choses to pollute every single file with a stream, however, which is why systems with it installed exhibit the problem."
Yeah, that's the typical Windows world attitude.
The operating system is specified to do certain things. It doesn't do them. Well, if not many people use this feature, so what? One of the way we make the feature list long is by including lots of features that don't work, but we figure nobody will use them and nobody will find out...
"Waiter, there's a fly in my soup."
"What kind of soup?"
"The orange scented celery puree.
"Oh, hardly anyone orders that. You should expect flies in it. It's your own fault for being foolish enough to order it."
"How to Do Nothing," kids activities, back in print!
"Vista is now forced down my throat"
I got news for you, that is Vista, but it isn't your throat it being forced into.
There are no loopholes. It's either legal or it's not.
Microsoft employees buy another copy of their 13K+ music collections if they want another copy.
Insert self-referential sig here.
Shouldn't this be "divide by 8.3" in the true DOS spirit of things? ;-)
Ever wondered whats wrong with the world? http://www.ishmael.org/
First of all, the issue is how Explorer handles extended attributes (EAs), which are distinct from alternate data streams (ADSes). The kernel and NTFS have always provided full support for EAs and ADSes (since NT 3.1). Explorer (and for that matter Win32) has never had very good support for ADSes, and almost nonexistent support for EAs. EAs were implemented in support of the OS/2 subsystem. ADSes are the 'official' way to attach metadata to a file, and scale better than EAs. The only Win32 functions that have ever provided access to EAs are the BackupRead and BackupWrite functions which are designed to handle all metadata on a file transparently. Looking at the imports from shell32.dll to ntdll.dll on Vista, it looks like the shell bypasses Win32 when dealing with EAs, invoking the syscalls NtQueryEaFile and NtSetEaFile directly (bypassing API layers like this is something Microsoft tells ISVs is a big no-no).
This is just Yet Another Windows 95 shell bug (yes Vista uses the same shell architecture ported through each version from Win95). It is not the end of support for EAs or ADSes. If anything, it's a belated attempt at better support, done poorly. The shell has always been, IMO, one of the lower quality windows components, especially when it comes to properly interfacing with lower layers. This bug does not surprise me. I've been using robocopy for nontrivial file transfer for a while now.
For those situations...
/s
Run -> "cmd" -> del %dir\*.*
It will clear most stuff and you will see error messages fly by... redirect output to a file for later examination if desired.
I use the good old 'del' whenever I know I will be deleting something like 20k files and do not wish to waste time waiting for windows to prepare for that operation... why the heck does Windows need to scan directories to be deleted before deleting them is beyond me, just delete them and be done with it. Same thing for copying, Windows wastes time scanning the source directory for no apparent reason since it won't tell you you have insufficient disk space to complete the operation until the target drive runs out of disk space... or any other errors for that matter, until it runs into them while carrying out the actual operation.
Linux has quirks, so does Windows. Linux has the excuse of being an relatively immature desktop OS but on the Windows side, it can only be written off as the result of half-ass design decisions.
Yes, but everyone knows one-in-a-million chances come up nine-times out of ten.
Not a sentence!
find -name "*base*" -exec chown us {} \; ; ln -s