Slashdot Mirror
Vista Runs Out of Memory While Copying Files
ta bu shi da yu writes "It appears that, incredibly, Vista can run out of memory while copying files. ZDNet is reporting that not only does it run out of memory after copying 16,400+ files, but that 'often there is little indication that file copy operations haven't completed correctly.' Apparently a fix was scheduled for SP1 but didn't make it; there is a hotfix that you must request."
The Excel bug.
"Slapping lipstick on a pig does NOT make it Natalie Portman. Paris Hilton, maybe, but not Portman." - UncleTogie
the box I "make use of" has just 15,000 mp3s...
They who would give up an essential liberty for temporary security, deserve neither liberty or security - Ben Franklin
This is probably why Bill Gates had Vista kicked out of Nigeria!
16k files should be enough for everybody.
At the end, there will be free therapy. And Cake!
Physics is nothing like religion. If it was, we'd have an easier time trying to raise money!
And that is one of many reasons we are all still running XP
Copyright 2010. All rights reserved. This comment may not be copied in any way including, but not limited to caching.
...occurs when a Vista user (running Kaspersky Anti Virus 6 or 7) tries to copy a large number of files (~16,400). So if you're like most people in the world, and have never touched Kaspersky AV (or Vista, for that matter), then this is a non-issue.When I copy a bunch of files from one directory to another, I get 'Explorer has stopped working and must restart'. I've resorted to using DOS to copy the files. I wish I had stuck with 2000 Server :)
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
M$ is scared that people will try to copy their documents to another computer before reverting back to XP. Smart, very smart Micro$oft! On a tech note, what kind of number is 14,600? I would have thought 16,384 would be better.
void r() { printf("recursion is "); r(); }
I was thinking "big deal", who copies that many files at once?
Then I read it's cumulative between reboots! I can imagine this will hit many servers that have any kind of auto-copy job they do on a schedule.
I don't want to start a holy war here, but what is the deal with you Vista fanatics? I've been sitting here at my freelance gig in front of a Vista PC (an Intel Core 2 Duo w/4 gigs of RAM) for about 20 minutes now while it attempts to copy a 17 Meg file from one folder on the hard drive to another folder. 20 minutes. At home, on my ancient Mac running OS 9, which by all standards should be a lot slower than this Vista PC, the same operation would take about 2 minutes. If that.
In addition, during this file transfer, Firefox will not work. And everything else has ground to a halt. Even Notepad is straining to keep up as I type this.
I won't bore you with the laundry list of other problems that I've encountered while working on various Vista PCs, but suffice it to say there have been many, not the least of which is I've never seen a Vista PC that has run faster than its Mac OSX counterpart, despite the Vista PC's same chip architecture. My 286/12 with 2 megs of ram runs faster than this 2.4ghz mhz machine at times. From a productivity standpoint, I don't get how people can claim that Vista is a superior operating system.
Vista lovers, flame me if you'd like, but I'd rather hear some intelligent reasons why anyone would choose to use Vista over other faster, cheaper, more stable systems.
they can only send 16,000 files to the RIAA and MPAA to check, at once.
... the exact number is 16384 ?
How much Ram does Vista POS DRM System need??
Why did you switch from a server class version to a desktop version of an operating system? Sounds like a bad choice in the first place.
If you want news from today, you have to come back tomorrow.
Oh, the fools! If only they'd built it to let you copy 16,401 files!
It's a feature!
Nothing to see here people, move along.
Maybe you're backing up to an external hard drive?
I don't think your average user would do this.
I have learned that copying files using the UI in Vista is a very painful thing to do - even if you don't have 16K+ files.
Heaven forbid you ever extract files from a cab you found on the internet - it will ask you for confirmation for every single file, without the option of 'yes for all'.
Just about everyone is.
Running with Linux for over 20 years!
Before the flamethrowers fly, people should note these lines from TFA:
"occurs when a Vista user (running Kaspersky Anti Virus 6 or 7)"
"Although the problem occurs where users are running Kaspersky security products, it's a kernel leak that lies at the root of problem (the problem's not confined to systems running Kaspersky software, that just that this application seems to exacerbate the issue)."
Yes It will. It is called 'do the same for all files' or something similar. Apparently the old style confirmation was too simple.
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
I just recently moved over 20,000 files from one drive to another, both drives are SATA drives (using Vista Enterprise, Japanese version) and I didn't have a problem...I assume that this bug was found on English Vista.....anyone else using another region's Vista have this problem?
I want an OS that lets me re-organize my pr0n anytime I want. I *need* to be able to select 50K-100K files at a time and move them from place to place without slowdowns. Ever try, in Windows, to search your network for all the *.jpg files, select a few hundred thousand of them in the search window, and drag them to the new firewire disk you just plugged in? It's painful, lemme tell ya.
Anybody want to suggest an OS that would work for me? I'm serious.
-- "I never gave these stories much credence." - HAL 9000
Is this related to the playing music and network file copy slow down bug as well?
.. when your GUI is using 2 gigs of RAM.
I call sensationalist bullshit. I just moved 20000+ files across a network connection to my Vista laptop about 2 or 3 hours ago without the hotfix applied. Memory usage does not appear to have increased at all from the typical baseline, and all files are present and accounted for.
Even though I plan to slap Ubuntu on this laptop the moment I hear linux has perfect power management support for it, I still have given Vista a fair shake. Methinks this has little to do with Vista itself and more to do with antivirus products sucking, as they always have. I've got no love for an industry that can only keep itself afloat by never perfectly solving the problem it exists to solve.
The reason being is I've setup a Vista system and copied about 100,000 files (totaling about 60GB) drive to drive in a single operation, without error. So while I'm not saying this isn't a Vista error, I'm wondering what else has to be done to trigger it. The persisting across reboots, even if you break it down smaller really makes it sound like another program is somehow interfering with the copy. I'll have to mess around with it at work, we have Vista test machines and Cadence installs north of 250,000 files when you install its libraries. I know it installs fine, though that isn't a copy strictly speaking as it is files being extracted from archives.
I'm just wondering if perhaps there isn't more to this than just "OMG Vista runs out of memory!" If it is a memory issue, why then haven't I encountered it, doing far larger amounts of files?
When I discovered a similar bug in Windows NT eight years ago (incomplete copying a large directory tree, silent), I installed FAR and haven't bothered with using Windows Explorer for any important stuff ever since. It makes me glad skills learned years ago are still useful: I'm using FAR in Vista.
17779 eligible voters in a district, 17779 'vote' as one. This is Russia.
Amazing that Microsoft are so short-sighted that they don't consider this important enough to include the fix in SP1.
Think of the potential loss of important files just because this thing doesn't report when it fails.
OK 16400 is a lot of files but its not unrealistic number. Just my windows directory alone has about 15800 files (not that I would want to copy it).
I just hope this bug directly adversely affects enough managers that make purchasing decisions to drive a few more to adopt Linux as a company-wide platform instead of windows.
2^14 is 16384, so Vista uses 14 bits in its for loop for copy. The question is why? even signed integers are 15 bits in old systems so are they using 1 bit for other purpose?
easy...kdawson modded it..
Very probably it's 16,384, as in 2^14. I'm sure it was a hardcoded limit. So typical, Microsoft... so typical.
16400 is supiciously close to 16,384 or 2^14.
signature pending slashdot approval
Why is this flamebait? This is a good question.
Linux. For all of your pr0n handling needs.
Did it make it 10-fold worse? 100-fold? 1000-fold? Did they just not have enough files to find out what the limit for just the kernel leak without KAV is?
FTA: The "Out of Memory" error (which is affectionately known at the PC Doc HQ as the "Out of Cheese" error
I don't think there has ever been a version of Windows that could deal with large numbers of files. Particularly if you are using the GUI interface. The whole thing is a toy operating system, really.
A few years ago, while investigating a similar problem with a production server (a SERVER not a client machine) the machine would gradually grind to a halt doing the copy, while still responding (but slowly) to other operations.
I found that the "copy" command did much better than a drag and drop operation, but still would have a problem eventually. Finally, I found that this was a known problem, and that to solve it, a dedicated MS employee had written a utility called "robocopy" the "robo" not being for "robot", but for "robust" (really, it said that!).
Using that usually got the job done, much more slowly than it should have, but at least I didn't have to re-boot the machine daily to clear things up.
Now that Gates is too busy with other things to take tours of the data center, really, Microsoft should do itself a favor and ditch the VMS underpinnings of Windows (some of which they have probably forgotten how to maintain) and build your nice GUI on top of BSD or something similar. That way you won't break your budget (in manpower and electricity) trying to match the Google server farms.
Once that's done you will have the experience needed to do the same on the desktop. You will be doing the world, and yourselves a favor. Thanks in advance!
Cat got your tongue? (something important seems to be missing from your comment ... like the body or the subject!)
I believe the poster is referring to working within 1 cab file, not for all cab files...
It's either none or all it seems.
Who is general failure, and why is he reading my hard drive?
I seem to remember in 2004 copying 1600+ trip pictures that were 2MB each from an XP laptop to my XP desktop and it failing to complete and displaying no errors. That's the only time I've ever seen Windows do that though. It used to happen a lot with Mac OS 7.x and 8.x. To the point I would copy one folder at a time and check the contents as I went. OS X fails to recalculate folder sizes in the List view.
Oh, speaking of calculating folder sizes in list view, does Vista do this? There is a handy add-on for XP called Folder Size but it doesn't work with Vista. The author claims he went through some hassle trying to get it to work with Vista but he doesn't say WHEN he went through the hassle, so I figured maybe Microsoft would have added that feature.
Everyone should use Folder Size if they're using XP. It's really nice. Especially if you use Macs on a regular basis as well.
No sig for you. YOU GET NO SIG!
I was raring a file up to email and I got the out of memory error. Vista blows.
Enjoy Every Sandwich
My XP slowed to a crawl and seem to loose almost all but major functionality when WinZip copied about 20k files to XP's desktop. It was like it had a virus.
It took some time to hunt down what the actual culprit was. Then it took hours to delete those files. Once deleted, it worked normally again.
I know,,, they want to start doing it like vista is a printer, you can transfer a fixed amount of files then you have to pay more and more and forever more.
:)
that's not a glitch they want to fix, it's a lawsuit they want to prevent
The original rant may be found here.
I wonder if the out of memory is for real. I wouldn't be at all surprised if it was out of some other resource, but couldn't come up with anything more meaningful to say - like an infamous version of Microsoft Word many moons ago that said it was out of memory when you didn't have a default printer defined in Windows.
What is this XP you speak of? Is it some sort of DOS shell?
...laura
And why is this marked as OffTopic? It's just an answer to the question posed in the parent.
Moderation retards.
Plus many more....
The reason this shows up most with kaspersky is because it adds an alternate data stream to every file and you're then copying thousands of files with ADSs which exacerbates the bug.
...and you've eaten your pen. simply stunning.
That actually makes me worry even more.
Now I'm not working at MS, so no way to _know_ it, but I don't think they'd move file copying into the _kernel_. (Then again, they did move stuff to kernel space before to gain 1% more speed, so it's not outright impossible.) But seriously, the kernel in any OS is there to provide the essential stuff, not file copying. You might get file open, file read, file write, and file close in there, but not the graphical shell's implementation of reading from one and writing in the others.
Additionally, _if_ they had file copy implemented in the kernel, then there would be no way for Kaspersky to get in the way there and make it any worse. So they probably don't have a kernel function that copies files.
What I'm getting to with this big tangent is: probably some other kernel function is leaking, and it's probably called from other programs too. I.e., you might get some side-effects even if you _don't_ copy sixteen thousand files in one go. If it's a memory leak in the file-related functions, any other program opening and closing files lots would have the exact same effects... eventually. Even if it doesn't fully run out of memory, well, wth, I have better uses for my RAM than to have 100 MB wasted to such leaks.
E.g., if I brute-force search for some text in all files on my hard drive (I do exactly that now and then), even with some other program (e.g., I use Total Commander lots), how do I know it doesn't happen to use the same functions?
Plus, what interests me more is this: well, copying 16k files in a burst is a good way to cause a leak to run out of RAM. But what happens if you just leave your computer on long enough and copy lots of small files in smaller batches? I mean, wth, nowadays a lot of people leave their computers on. It isn't just for linux uptime brag-fests any more. Does the same effect happen if I leave my computer on for a month and copy, say, 600 files a day?
A polar bear is a cartesian bear after a coordinate transform.
It's an old troll, just updated for Vista, gods people.
This thing is almost ten years old.
The rock, the vulture, and the chain
...why would anyone want to copy 16400+ files? What if a person were to upgrade their hard drive.Linux Zealots: Smarter than Mac Zealots, but still zealots.
mod parent as stupid.
Likely, they're allocating memory to store file attributes or some such that are not being free'd when done with. Hence running out of memory. If you had coded a day in your life you'd see that.
Tom
Someday, I'll have a real sig.
Not that the old style confirmation was any good. XP has no 'no to all' or 'rename,' so if you don't want your data overwritten, you have to stop the operation.
And "Yes to all" doesn't seem to mean that. It only means "Yes to all to this particular question in this directory."
I just used Explorer to try to move an 8gig drive to a folder in another. It took a couple dozen tries.
and/or Mac...in essence it's the same thing though....just a kick ass GUI on it :)
wiki.osx86project.org
^For those of use who don't have two grand^
But unlike you guys, I don't need any specific reason to hate it. The entire picture of Vista [and Windows in general] is just sick. Aside from gaming [which Linux distros are technically capable of] there is really no advantage over the properly selected Linux distro. And being that I do more than game, it's an easy choice to use pretty much any Linux distro over XP or Vista.
Tom
Someday, I'll have a real sig.
Try holding Shift key while clicking. It works as "No to all" for file replace dialog which pops up while copying files. It might do the trick.
With Great Power Comes No Love Life! - Samit Basu
Errh - why does Kaspersky add an alternate data stream to each file? There's probably a reason, but I'm too dumb to work it out.
Dude, you are friggin' hardcore if on a recurring basis you're copying around 50K-100K images of pr0n.
That's almost unfathomable.
Reminds me of something an old prof used to say about stacks of floppies full of porn that used to get traded around
Cheers
Lost at C:>. Found at C.
This is a minor problem, absolutely rare event, occurs with next to no regu...
**OUT OF MEMORY ERROR, SYSTEM HALT**
Karma Whoring for Fun and Profit.
Quote>I don't think your average user would do this.
I have learned that copying files using the UI in Vista is a very painful thing to do - even if you don't have 16K+ files.
You've kind of hit on exactly why the average user would do this... backing up data, moving their music collection to another drive or device or computer, etc... the list of possible reasons an average user (who wouldn't select something like RoboCopy or Total Commander or whatever) WOULD choose to use Windows Exploder to do this.
(I think) you are confusing your (correct) knowledge of using the right job for the right task - with an average user's "knowledge" of "Explorer is the only tool I know that does this task"
Very similar to way too many "average users" who click on the "Internet" button/icon on their Windows machine. We all know (well most of us here do) that IE is NOT the Internet. MANY "average users" dont realize there are other/better tools than Explorer to use for mass file transfer.
Regardless, it's not like Explorer is doing (or rather, SHOULD be doing) any massive, unseen mumbo-jumbo that should cause such problems (slow downs or memory leaks when copying)... but simply, that just isnt the case. Even if there are better tools (which I think we all are in agreement there are), the average user (rightly) has the expectation that copying a file is simply that... copying a file. It just isn't... so the average user is wrong, due to poor design (the thumbnailing routines and routines that handle music/video metadata, etc) and bugs just being patched (the OLE component), and who knows what else that will pop up later.
Now, in MS's defense, copying through the UI (especially if it is handling the background tasks it should be - like verifying/fixing shortcuts, thumbnailing, etc) will have some more overhead then similar "Just copy/move the files" programs... that's to be expected. The only problem is MS hasnt fixed the routines that SHOULD BE handling those transparent, background tasks either.
StarTrekPhase2 - The Five Year Mission Continues!
Except that Kaspersky most certainly does install itself into the kernel. After all, that is where those types of drivers need to be: living in a ring 0 world to truly protect the computer from malware, which is hopefully confined to ring 3. Also, the symptoms reported don't necessarily have to be kernel leaks. UI components not drawing correctly can occur if GDI resources are leaked for a long enough time, since there is only a finite amount of them available, even when you have 4GB of RAM installed. One application can't take all of them, but a leak in one along with several others taking some would make the symptoms occur.
The real problem is the handling of alternate data streams in a component used by Explorer. Read the KB article. Kaspersky stores information about each file in an alternate data stream, so it is little surprise that it exacerbates the problem. Alternate data streams are a highly dubious feature; I wouldn't be surprised if they become deprecated next Windows release. The idea of maintaining a parallel, but hidden filesystem is pretty bizarre, but more importantly, ripe for abuse by malware. Then again, this may be how MS implements file revision history in 2003 and Vista.
I can't say I'm surprised, really. Vista was rushed out the door and not ready for prime time. Unintended consequences of interactions like this were bound to occur. Maybe that is why I dislike running resident AV software on my machine. Stay out of my kernel, y'know?
Coworker calls me up a few weeks back. She has a three year old Dell and wanted to "update" it so went to Best Buy and came back with a "memory stickboard" and a steaming fresh pile of Vista Home edition. After getting her to explain what she was wanting to do, I told her to just leave everything sit and bring her computer and original software to work. I figured she just needed to have a virus/spyware cleaning done and maybe have a few things pointed out to her. I told her to make sure that the Vista remained unopened so she could return it and not touch the RAM, either. Odds were she probably didn't even have the right type for her computer. I also told her to make sure her precocious 12-year old knew to not touch anything.
So, what happens? She goes to bed and he goes to town. He installs the RAM, does a full install of Vista and wipes all her shit off the computer, and installs Vista on his laptop for good measure. Oy. She finally brought the machine in today. I was very surprised that Vista was clickable. You could navigate about in it without lag. I think that might in part be due to not having half the required drivers so we don't have sound, network, modem, etc.
I think she said she paid like $200 for that copy of Vista. She could have returned it if the little shit had just fucking listened to what he was told. I told her it would be funny to tell him that's his Christmas. Bet the little shit will listen the next time.
Kwisatz Haderach
Sell the spice to CHOAM
This Mahdi took Shaddam's Throne
And if you have indeed read TFA, you'd have read also that this is only that Kaspersky makes this problem stand out. You don't need Kaspersky to trigger that bug:
> Although the problem occurs where users are running Kaspersky security products, it's a
> kernel leak that lies at the root of problem (the problem's not confined to systems
> running Kaspersky software, that just that this application seems to exacerbate the
> issue).
A comment above already mentions that.
And they can't get an operating system out the door that copies files properly? I'm saying this not as someone seeking a flame war or anything but someone who shelled out the $$$ for a copy of Home Premium. I also say this as someone who when copying my entire documents directory to an external drive may have experienced this bug because I had to restart the system to unfreeze stuff at the time.
Hey MS, less features and more SOLID OS please or the penguin will really be biting away at you more then it is now.
~~ Behold the flying cow with a rail gun! ~~
Last week my work station had some sort of update happening (later determined to be both a Windows XP and semantics..)that was supposed to happen at 3am not 4:30pm
I left work around 6:30 realizing my drive 6+ gigs space was vanishing..
Next day I had a warning of low disk space of 135megs
After much digging I was able to determine that some of the 11.5+ gigs of drive space I should have was the offline cache that I was able to remove using disk clean and turning off the cache.
But the other 6+ gigs took longer to figure out.
looking at log files and the results of a current drive defrag analysis as I did a defrag the day before this all happened...Apparently after the updates happened my work station decided to offline cache my local drive and in the process stopped doing so when it hit the 135meg low drive trigger. In addition to this it made this part of the offline cache unavailable. In other words it was doing nothing but taking up drive space.
Searching for a solution, as I know where the files were but unsure of how to remove them safely...
In the offline files option you hold down ctrl+shift and mouse click on "delete" and this effectively reinitializes the cache. Reboot required (duh)
after rebooting I turn off cache after setting it to 0 size (as MS might decide to change it next update..... regardless of my now having set updates to notify ONLY....
So where do I send the bill to MS for the company time I spent on this mess?
Clue time for the Barney generation:
16,400 is suspiciously close to 16384, a power of 2. Someone probably declared a static array of 16K worth of strings. Arbitrary fixed length arrays are bad. Bad, Microsoft programmer!
Now, in Barney speak:
Duh-ahuh! Whats probably happening here is that somewhere in Vista, or Explorer, there's a fixed limit to the number of files you can copy or move in any one operation. More than likely, there's some sort of clipboard mechanism at work that only allows for 16,386 entries to be placed in it. Thats also why Vista is complaining about being out of memory. I love yooou.
My notebook is running Vista Home Premium. Core2Duo, 1GB ram, blah blah blah. One of my hobbies is photography, and I have about 18,000 digital photos that I've taken over the years. I keep this collection on a couple of different computers for safety reasons. I also keep them backed up on an external USB drive, and use that drive as a transport mechanism between the other computers (I'm too cheap for a gigabit LAN). When I bought the notebook a few weeks ago, I had no problem copying all of those photos from the external drive to the notebook. While I have not spot checked every single photo, the byte counts match and it certainly appears that they are all there.
While it is unpopular around here to say so, Vista seems to work pretty well for me. YM, of course, MV.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
Likely, they're allocating memory to store file attributes or some such that are not being free'd when done with. Hence running out of memory. If you had coded a day in your life you'd see that.
I have *been* coding for well over 25 years and you don't allocate memory for each file in a copy process, you allocate a buffer ONCE and use it for each file. Then you make sure it is freed.
If you like calling people stupid, or criticizing people of whom you have no knowledge, perhaps you need to look in the mirror.
I noticed that when you would copy large amounts of files everything would be swapped out to do so. Including the explorer menu. It was rather pathetic to have to wait to get things out of swap for something as trivial as hitting the start button.
The phrase "more better" is acceptable English. suck it grammar Nazis
Antivirus softwares need to scan each bit passing to the disk which slows up file copy on any Windows system. Before XP culprit was the default of disabled DMA for HDD. Yes linux does it a bit faster but it's not a huge difference in practice.
With Great Power Comes No Love Life! - Samit Basu
Make an image of it?
Bitter, not morose.
It is 16384, but the missing bytes (1784) is the hidden DRM count you are not allowed to see or measure. They even fscked that up (using Excel, I suspect) - it should have been 1984 Orwellian bytes.
User (looking entirely like an idiot):
"An OS that can't copy files? Yeah right! Whattay think I am, some kind of idiot?"
"If you're not passionate about your operating system, you're married to the wrong one."
Does HAVE to be limited to "extended attrib" files?
I wonder if users of say, AutoCAD, are affected. Anyone using HUGE files and using XREFS to link to hundreds of drawings with thousands and thousands of objects (maybe hundreds and hundreds of thousands in a single drawing) might stop and wonder what parts of the drawing might not be showing up. I wonder if AutoDesk will generate a checksum tool (if there isn't one) to daily allay fears users may have.
When I copy and paste in some drawings, TENS of thousands of objects get copied or moved, sucking up a huge chunk of the 512 MB my machine has, and it bogs it down.
Probably other apps like backup applications theoretically could have a problem, no?
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
Remember that the memory consumed by this bug is not released until you reboot. That means you must reboot after syncing your portable music player. This is a serious flaw and unacceptable in modern computing. What's more shocking is such a flaw cropping up in a legacy system like Winblows. Chances are that this is some kind of digital restriction that has blown up in their faces.
Based on the very funny article at
/. this site once and for all, it was the subject of a previous article today!]
http://www.meangene.com/essays/microsoft_interview.html
it's probably:
CopyAllFiles(HideouslyLongListOfFiles) {
[ pop file name off HideouslyLongListOfFiles and copy it ]
[ call CopyAllFiles(HideouslyLongListOfFiles) ]
}
Which of course leaves the door open for
CopyAFile (ListOfAllBytesInTheFile) {
[ pop a byte off ListOfAllBytesInTheFile and copy it ]
[ CopyAFile (ListOfAllBytesInTheFile) ]
}
[Might as well
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
x86: http://thehotfixshare.net/board/index.php?automodule=downloads&showfile=3594/ x64: http://thehotfixshare.net/board/index.php?automodule=downloads&showfile=3610/
If moving files with ADS in them causes problems on wista, what on earth will the rootkit writers do?
-q
... proceeded to copy the rest of them in my batch over over 27 million files I copied once. I don't do that very often. But hey, I had to fill up my new 500 GB disk somehow :-)
now we need to go OSS in diesel cars
"An idle Excel spreadsheet is the Devil's workshop."
I have 47000+ files in my home directory on my Linux box at work (lots of source code, physics simulation stuff etc).
I guess if I were running Vista, backing up all these files to another hard disc wouldn't work too well!
I've got 10,000 files just in my photos directory. If I want to move them to another computer, I am 2/3 of the way to loosing my family photos. I'm sure that there are lots of people with more pictures than me.
...that it didn't even lock the files away from the user's own will without any way to access it.. That happened on a XP Home Compaq OEM computer once when I copied around 2000 files to a folder, like some anti-piracy countermeasure.
now we need to go OSS in diesel cars
Principally a programmers differencing tool, but also great for file copies, website deployment over FTP, all sorts of things.
I'm not ruling out the possibility that they might be Evil, mind you. There's ample evidence for that as well. It's just that I've seen their API designs and the product of their OS design and I'd have trouble believing that they could be checking what you're doing on your computer without being caught at it. Hell I'd have trouble believing they're capable of consistently putting their underwear on UNDER their pants! (Oh, I'm SO not going to get a job offer there NOW!)
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
I'd choose Vista over faster, cheaper and more stable systems because it helps fight piracy. :)
Just disrupt the deflector shield with a tachyon burst.
Sorry, but this has always been a problem with Windows. I'm sure if you have top-spec machines you might never see it, I'm sure if you shut down every hour because of the way you use your computer, you might never see it...
but I've always had this problem with every Windows past 3.1 (I didn't use 3.1 with enough files, so I wouldn't know if it was earlier too).
File copy has ALWAYS taken an extortionately large portion of system resources when you do it in the GUI. Windows 95/98 was terrible and once it had hit the magic mark, it would just churn and not get any further (the estimated time would just go mad). Eventually it would run out of "resources" (which had nothing to do with free memory, but more things like file handles etc.) and start erroring, losing icons, blue-screening, etc. and the only fix was a reboot.
I can reproduce the same problem in XP in a snap - just get a truck-load of files and start copying them WHILE DOING SOMETHING ELSE, e.g. renaming, filing, categorising, or just working in the background. It'll take a while but eventually everything will slow to a crawl (and, yes, I've done this many a time on systems without any form of file read interception like antivirus etc.). It tends not to crash quite so bad but it will swap like mad and slow everything to a crawl.
And again, doing it via command-line copy won't reproduce it anywhere near as easily. A few years ago I was categorising and filing approximately 50Gb of VB programs, website source, emulator roms, millions of tiny files etc. and it was a pain to do precisely because of this. I reverted to a command-line about halfway through which sped things up a lot.
It's DEFINITELY more to do with the NUMBER of files, though, and not the individual size. A million tiny files copies a lot slower than one massive one of the same size. I've always just put it down to "something" in the copy GUI routinges not releasing file handles of already copied files in time to read in new ones.
And, sorry, but this is something that I've never been able to reproduce in KDE. Even so, on both OS, I hate the fact that the copy GUI takes so many resources - often the window can't even redraw itself properly until the copy has finished.
And don't even get me started on "Estimated times" in file dialogs...
Guess not. Use Kubuntu/Mac/PCBSD instead. They can copy files fine. Cheap too.
Sure, you could. Hey, you could also write your own copy program! Or, you could boot into Linux and do it from there!
Seriously though, why are we making excuses? Copying files is a pretty basic file system operation, and just because there's some other way to do what you're trying to do doesn't mean that it's ok that the system runs out of memory.
The average user is probably not going to make an image, or boot up into Linux, or write his own copy program. He is going to assume — and rightfully so — that he can back up his hard drive (or whatever) by dragging and dropping in Explorer. When Vista wigs out, and leaves his system in an inconsistent state, he is going to be very upset — again rightfully so — and telling him "You should have followed nomessages' advice and made an image instead" is not going to make him feel any better.
There is no reasonable explanation for it except that THEY are checking what YOU are doing with YOUR computer.
That explanation is even less reasonable, if only because it should no more run out of memory reporting on your activities than performing the file copy itself.
So you would reuse a buffer for copy files, but not for phoning home, yet claim 25 years as a programmer?
Your bias is burning so bright it's blinding you.
It's official. Most of you are morons.
It's not a simple operation. Think about how a file copy works within Vista:
1. The file is opened.
2. The file is scanned for viruses.
3. The file is scanned for adware.
4. The file is scanned for DRM violations.
5. The user is asked if they're really sure they want to copy the file.
6. The user is asked again if they're sure they want to copy it.
7. The OS makes a judgement on how long it will take to copy so it can update the pretty stats in the gui.
8. Lots of flashy graphics and widgets are loaded to show you a pretty animation while you wait.
9. The file is copied.
10. The destination file is verified that it is intact.
11. The destination file is scanned for viruses.
12. The destination file is scanned for adware.
13. The destination file is scanned for DRM violations.
14. The file is successfully copied.
Hell - I'm surprised their OS can even handle copying 1,600 files, let alone 16,000.
Just disrupt the deflector shield with a tachyon burst.
Is this somehow related to the stupid inability of pretty much every Windows version ever to NOT start copying files until it can assure there is enough room on the target disk? I've lost track of how many times something has started copying, gone for a few minutes, then just stopped because 75% of the way in Windows realized there wasn't enough disk space to finish.
I will not use a Microsoft Product until the SP2 comes out.
Chas - The one, the only.
THANK GOD!!!
I hope that soon this story will be covered on his blog. Previously he described other weird things happening during file copy procedures, as well as the Vista network performance issue. Hopefully, his story will provide enough low-level details for the hungry minds.
The saddest poem
"Very few files have data streams, so the vast majority of users won't ever see a problem. Kaspersky choses to pollute every single file with a stream, however, which is why systems with it installed exhibit the problem."
Yeah, that's the typical Windows world attitude.
The operating system is specified to do certain things. It doesn't do them. Well, if not many people use this feature, so what? One of the way we make the feature list long is by including lots of features that don't work, but we figure nobody will use them and nobody will find out...
"Waiter, there's a fly in my soup."
"What kind of soup?"
"The orange scented celery puree.
"Oh, hardly anyone orders that. You should expect flies in it. It's your own fault for being foolish enough to order it."
"How to Do Nothing," kids activities, back in print!
On my computer (granted its a mac, but bear with me), I nightly mirror my home directory to an external drive. My home directory has well over a quarter million files in it. 16,400 files at a time is completely unacceptable from a backup standpoint.
Requesting a hotfix from MSFT support is pretty easy these days. Find the KB articles, go to this address, enter your email, the KB number, platform information, and they email you the hotfix.
It's a lot better than the old days where you had to get a support ticket opened, find a human, convince them there was a hotfix, and get them to provide you the bits somehow.
Nope, that's a Styx song.
Certain gifted/cursed types can see those numerical interactions instantly, thus wasting no time.
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
Just a peeve of mine..."and all files are present and accounted for." As a former military man, I have to correct this misused cliche. It is "All present OR accounted for", not both. Something is either present, or if it is not, but it is accounted for. Thanks for letting me rant.
that the error message is lying and it is in fact file descriptors that are leaking? I can imagine people writing the info window saying "we'll never explain to the John Q. User what a file descriptor is, whereas memory is something they have a chance to understand; come to think of it, there's a ready answer for the helpdesk techs whenever somebody calls about this - buy more RAM!"
I can assure you, the best way to get rid of dragons is to have one of your own.
Redefining "pathetic" is usually quite a chore for most people, but you seem to handle it with aplomb.
That explanation is even less reasonable, if only because it should no more run out of memory reporting on your activities than performing the file copy itself.
So you would reuse a buffer for copy files, but not for phoning home, yet claim 25 years as a programmer?
There are plenty of reasons why you would use a single buffer for the copy, but a small buffer for each file. Once scenario could be that since a copy is going to be faster than internet, you would queue up file information packets structures to be sent out, the programmer probably just forgot to delete structs. If you wanted stealth, you would do this slowly or when the computer is not being used.
We know as a fact that (1) Windows machines phone home periodically. (2) That Vista performs "rights management" on files copied to and from the hard disk(s). Is my assertion so far off base?
Once you get past 16,400 mp3's, the RIAA is notified.
Gentoo (a linux distro that compiles from source) has all its packages in a simple directory structure with files for each package. That is a LOT of files. The intresting thing to see is just how much speed up you can get using the various filesystems on offer. Reiserfs really makes a HUGE difference to the speed. Same HD, same kernel, same drivers, just a different filesystem and you can easily get a tenfold increase over other journalled filesystems.
Same with compiling, I know mount /tmp on tmpfs (ram disk) and the speed increase is enormous.
HOWEVER, I now have a ext for boot, jfs for / (no real reason, I just like how it handles itself reiserfs for /usr/portage tmpfs for /tmp and /var/tmp (and I really should choose a new one for / and pick jfs or xfs for my movie collection). Somehow I think that this is just a bit harder to support then NTFS for everything.
Windows has to do LOTS of things and has to do them reasonably, that is actually a lot harder then do a single thing and do it really well. I think the problem here is that windows does more then just copy the file but actually tries to process the file and then doesn't properly release the memory. MS LOVES doing this, it is the MS way, it can't leaves files alone, it must read them so you can see how many minutes a song has, the dimensions of the image or the creator of that movie. This case makes it clear because anti-virus software also LOVES to read files while they are being moved around. That is its job, but it means that this is FAR from the straight copy your old systems did.
MS has always had troubles with file operations, just try to copy a large amount of files, it will takes ages to get ready to even start copying it, way more then is reasonable. Its system just ain't designed for it. Take the undo option, if you think about it, that is a memory hog waiting to happen, does it really have to remember ALL those thousands of file that a single command can generate? Ouch.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
you just keep on trying till you run out of cake.
Reminds of a problem I never solved using ntbackup on windows server 2003.
Because ntbackup uses one large file, and a file copy uses 2k (somtimes 4k) of paged kernel memory per 1Meg when copying a file (In our case another SATA HDD), and Kernel memory seems limited, our backup would always fail at around 50-60GB. The only solution we found was to create multiple smaller backups of around 30GB.
46137
Anyone try using Firefox lately? Sitting here, doing ABSOLUTELY NOTHING, with 3 tabs open, it's at 69MB right now. I have seen it hit 300MB before, and the longer you have it open, the worse it gets.
Memory leaks happen. They're called bugs. Bugs eventually get fixed.
No, this has nothing to do with a bug in Vista. This has to do with the Slashdot editors throwing more red meat to the dogs. Just like the broadband stories that pop up every few weeks (how many times do we need to be told that the US isn't #1 in broadband?), and just about every other Vista story.
You may notice something though - what hasn't been popping up. There are fewer vulnerabilities for Vista: Secunia reports just 15 vulnerabilities ever, compared with 24 for XP in 2007 and 45 for XP in 2006.
People are complaining that Vista requires too much memory and doesn't have 100% hardware support. Well, 2GB of DDR2 now runs about $60, and pretty much any new hardware sold today is Vista-compatible (not to mention lots of older hardware, including everything in my 3 Vista PCs, all of which predate Vista).
People won't care that Vista has higher hardware requirements. They will care that it's more secure and more robust. Ask anyone who has overclocked their GPU too much:
It may not seem like a big thing, but display drivers are very complex and they shouldn't be able to bring down the computer.
How can you expect consistent moderation? Because of the way the system works, the first mod can effectively hide a post from other modders (yes they advice you to mod at -1 but who does that?) so wether a post gets seen or not depends on who gets to see it first. Mod it down, and it is gone. Mod it up, and more people see it, this includes downward modders but also upward modders, so it totally depends on the first mod.
Then there is the fact that moderation is totally random. So random that sometimes I get them constantly while othertimes I go for weeks without. Meta moderation is even worse.
Then there is mods themselves, not the moderators, but the way you can mod. There is no simple "FALSE" mod. or "you are an idiot" mod. Or a "True" mod. Flamebait and troll are way to often used to mean "I don't agree with you." Insightfull and intresting far too often for someone just stating the facts.
So don't expect sense from the moderation system, just accept that some of the best posts will disappear forever because a mod didn't like it, and that some crap will be modded up.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
Microsoft employees buy another copy of their 13K+ music collections if they want another copy.
Insert self-referential sig here.
It's just the latest in a long list of poorly-implemented file operations that includes these gems:
1. Shift-drag, which moves directories, but if there's not enough room, it stops halfway through, leaving two half-complete, completely now un-re-mergeable directory half-trees.
2. The famed (File) Explorer bug where Explorer starts doing things such that, once it starts, you can't delete a folder without a reboot because it gives you a "file is in use" error -- along with the highly offensive suggestion to "please close the application in order to delete the file", which you can't do because it's Explorer. Ok, you can kill Explorer and (from the Task Manager window, do Run->explorer, but that's hardly obvious.)
3. The attempts to keep idiot users from deleting files in use prevents people from being able to delete virus files and the like. In the olden days, you could do option-drag the file to the trash and clobber it even if it was open, but MS changed that to "help" the situation.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
Maybe they're implementing filesystem transactions in memory ? See what happens when you issue three or four copying commands at the same time with a third/fourth that size ?
Religion is what happens when nature strikes and groupthink goes wrong.
I know the standard for gifted has slipped some but come on "and subtruct this from 690. You get 666." is basic math (666 + x) - x = 666.
1*2*3*4*5*6*7*8 + 666 = 363546
So take 1 times 2 times 3 times 4 times 5 times 6 times 7 times 8 and subtract this from 363546 and you get 666... wow.
PS: You can play with the numbers to make it sound cool like add 49 to 1/2 of 1234 and you get 666.
Of course, this kind of thing never happens on a Linux Desktop.
DCMonkey
I expected this very problem to happen and have taken corrective measures. Simply, I use one large file for everything.
Think about the innovation that is being created here. I can access all the spreadsheets I have ever created whilst updating my current webpage project, search for an e-mail archive and read the latest TPS Report Coversheet without changing files. It means I don't have to partition my hard drive. And I only have *one* file ever to backup.
This has simplified my daily work to the point I fired all my IT staff. Thanks Microsoft!
So you're saying you are too god damn stupid to consider multi-threading file copies?
You may have been coding for 25 years but that doesn't imply that you are any good at it.
I think, therefore I am an Atheist.
Well, some of them must have huge files collections. That's probably why Robocopy is made by Microsoft.
http://www.microsoft.com/downloads/details.aspx?familyid=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en
dude do you search for posts regarding yourself?
Well, perhaps if you'd coded a day in your life in a backward, non-garbage-collected language like C...
The cake is a LIE.
If you can read this, I forgot to post anonymously.
+1, WHOOSH!
If you can read this, I forgot to post anonymously.
hahaha
Jesus.
Can people PLEASE give up on vista already? Let it die.
Hey, I hear OSX Leopard is due to be released in 10 days.
(Insightful? Or flamebait? I vote insightful...)
This reminds me of the memory leaks which FOSSie zealots continue chanting are not there... until the bugfix report mentions how one of the memleaks was fixed. But then they can continue denying the security holes, and that should keep them until the next bugfix report where they can then switch back to denying memory holes.
Ah, it's so fun being a FOSSie, and pretending your polished turds don't smell.
16,400 files?! Why, that's more files than a person could use in a LIFETIME!
Why would you do that? The bottleneck in file copy is I/O, not CPU. A multi-thread file copy will just make more CPUs wait for the hard drive(s).
Use Linux
/autofs -iname "*.jpg" | cpio -pv /media/SEAGATE/ ... assuming you use autofs to auto-mount visible shares on the network and your firewire drive is mounted as /media/SEAGATE of course, you shouldn't have any problems, besides griding the network to a halt perhaps.
find
- Michael T. Babcock (Yes, I blog)
Just as 640K of RAM is sufficient.
Yes; I have verified on many machines and gave up on copying very large video files on Wiin2K. Always corrupted NTFS after more than ten 2GB or larger files were copied. Not sure if it happens on XP. Can anyone confirm if you can copy hundreds of multi GB files on XP?
The simple file operations in Windows (and indeed in other OSs) still drive me insane. If I want to move/copy/delete a bunch of files (say several thousand) stored in a deep hierarchy of several folders, I've yet to see a drag-and-drop solution which will handle this (in any OS). Specifically, if a single file fails, the entire operation is aborted, leading to tons of wasted time as one figures out what was/wasn't moved/copied/deleted. I mean it's been decades... why hasn't the technology matured? The closest I've come to a solution which allows pausing/resuming/skipping/bandwidth control/etc. of copy/move operations is a handy little program for Windows called TotalCopy (http://www.ranvik.net/totalcopy/). Why this function (a few KB) isn't built into Windows by now is beyond me.
... Real Men Of Genius.
This latest news about Vista is just confirmation that the issue has been ignored again, and eye-candy that any sane user would instantly disable because of the system hit has been implemented at the expense of system usability: form above function. Here's a little ditty I wrote when Vista previews were coming out. As pertinent now as it was then:
-
-
Bud Light presents
[Real Men Of Genius.]
Today, we salute you, Mr Impatient For Windows Vista Guy.
[Mr Impatient For Windows Vista Guy.]
While others marvel at an operating system whose primary repair
tradition is a complete wipe, you just can't wait for more of the
same.
[I just love my Long Horn!]
Yes, it lacks security, efficiency, speed, heck, just about
everything. But ever since 1985, when you first jammed your floppies
into that curvaceous 186, you've been enraptured with Windows.
[It was five and a quarter inches!]
Despite the fact that it requires an array of Crays to run already
invented technologies at sub-optimum speeds, you will beat the rush
and see Notepad and Clock run in CPU-crippling GPU-hogging
translucency.
[It turns on all my pixels!]
So crack open an ice cold Bud Lite, oh Chevalier of the Control Panel,
because whilst the rest of us wonder what Vista will bring, you
already know.
[Mr Impatient For Windows Vista Guy!]
Bud Light beer. Anheuser Busch, St. Louis, Missouri.
I can just imagine the warteam conversation:
Test: File copy fails after 16K files! We can't ship like this!
Dev: The fix is easy and I can have it in tomorrow.
PM: 16K files? Only music pirates are gonna hit this. If anyone else sees this problem it goes away with a reboot, right?
Release Mgr: Won't fix!
Well there is a old legend that if you copy more than 666 files on WindowsME and open a linux related site at once Steve Balmer will apear from the monitor, throw a chair at you and you will be desintegrated
that's what they WANT you to think!
Windows has never been able to copy a large number of files. I've been trying to copy millions of files on windows boxes since 1997 and it never worked. Explorer usually just hangs, I guess OOM is an improvement. Install cygin. Use rsync. Done. Also should point out that you have to move folders like 5 times in vista before they finally move. No errors, source and destination remain partially copied over. Probably the same bug.
With as slow as Vista is at copying files, it would take months to copy 16,400 of them.
If you could reason with religious people, there would be no religious people
what are you doing? please put me down!
See?
You're gifted, too!
Microsoft can't do math, remember? Just like excel2k7, When window counts the number of files and come up with 16400, it miscalculate to 1,000,000,000...no wonder the leak.
The only possible interpretation of any research whatever in the 'social sciences' is: some do, some don't
I've noticed a bug in XP where the machine rebooted itself (no errors reporting, no blue screen) every time the user was moving, copying or deleting many little (or some big) files at once. We eventually found that some of those PCs had hardware problems, but others were OK, and the defect happened too many times to be a simple coincidence.
It's possible that the folks at MS have created a new bug while rewriting some of those routines.
Any help on that XP bug is really appreciated: I've still some of those machines here. They all have this problem under XP but work perfectly under Linux and MemTest86 doesn't show any problems.
Billy confirmed it.
Well, consider the following situation.
I will admit that if you're copying extremely large files from disk, a multi-threaded file copy probably won't help much. However, if you are copying many hundreds of files of varying sizes, it makes a large difference. There will be more IO requests being processed by the OS. Now, modern OSes optimize IO requests by various criteria, so essentially, it can order the hard drive to seek in a more intelligent manner and therefore improve throughput.
Additionally, you could have many threads copying the same file, just in different chunks. Since a large file is extremely unlikely to be stored on the HDD in a completely non-fragmented and sequential fashion, the above argument applies.
You are still limited by the IO speed of the HDD, but you can substantially cut down on seek times with multithreaded file copying and intelligent IO requests. And considering that in many cases it takes longer for an HDD to seek than to read a block of data, the time savings add up in a very large way.
I think, therefore I am an Atheist.
Bingo. This is what separates the "coders" from the "developers." And for the record I've only been professionally writing software for about 6 years now. Though I started "coding" when I was a kid [e.g. 20 years prior].
/. hehehe.
Very likely it's just some extra data allocated per file when there is a hook [like AV] involved that isn't getting freed. As others pointed out copying files sans-antivirus seems to work just fine.
This is what debugging skills are about. Diagnose, differentiate, and reason. Don't just guess and then post flamebait on
Tom
Someday, I'll have a real sig.
So THAT'S what the problem is?! Jesus Christ.
You know what makes that worse? There's no easy way in Windows to tell what has the damn file locked, without Process Explorer.
probably redundant, didn't bother to read comments to find out
I had this problem and it was driving me nuts. It is not a Vista problem. It is a Kaspersky problem that Kaspersky doesn't want to deal with (all other AV products work 'fine'), so quit whining about how horrible Vista is. It apparently has something to do with the way KAV keeps files in memory to scan before allowing them to be written to disc.
The hotfix should be open to everyone, but oh well. It takes two seconds to submit the request and they got it to me in about 1 hour. Now I can tell when I hit the 16,4---whatever limit. There is a kind of balk to the copy process, and sometimes an error which requires clicking "Try Again" once or twice. IT happens only once in the copy process, even if you multiple of that magic 16,blahblahbah number.
Well... once you've done that, don't forget to free all the metadata objects in all your threads, check whether the Unix "cp" is faster... agree with that, remove all your crap to use a single thread and ask kernel architect to optimize read/write accesses !
Btw, I never understood why scanning disks is so fast once filesystem is in cache with Unix, while the stupid dog is still waiting for the same amount of time the second time under XP:
Perform the test:
find . -name '*.txt'
find . -name '*.mp3'
ie: takes 3 minutes the first time, while the second is instantaneous with my Linux Box, same system with XP... wait, coffee time !
It's a shame that Open Source chooses names that turn most people off.
If you want news from today, you have to come back tomorrow.
What?! This news is saying some idiot still use Vista??
Honestly, how can [Cancel or Allow] anyone co[Cancel or Allow]py that m[Cancel or Allow]any files anyway?
Surely no one would ever need to copy more than ~16,400 files.
I remember not even Win95 could copy lots of files... Win98: The same. Win2k: The same. WinXP: The same. 12 years and still the same problem? I really don't remember about Win3.1 or Win286... If ever the Windows code gets open-sourced, lots of programming hackers are going to laugh for years upon seeing the buggy code, I am sure. How come people now have alternative choices and still use a buggy closed-source OS is beyond me. Poor Windows users... (happily posting from Debian GNU/Linux 4.0 amd64 etch + some lenny).
So you're saying you are too god damn stupid to consider multi-threading file copies?
You may have been coding for 25 years but that doesn't imply that you are any good at it.
Let me think, ummm, the RAM and CPU are much faster than the disk. No matter what you do, the disk will be the bottleneck. So, Do I create threads and more OS context switches for something that isn't going to be any faster, and may, in fact, be slower and cause other programs to be slower? I don't think so.
Bingo. This is what separates the "coders" from the "developers." And for the record I've only been professionally writing software for about 6 years now. Though I started "coding" when I was a kid [e.g. 20 years prior].
/. hehehe.
The Ad hominem level here is ridiculous. I'm sure glad you feel comfortable impugning someone of whom you have no knowledge.
I like this the best:
Very likely it's just some extra data allocated per file when there is a hook [like AV] involved that isn't getting freed. As others pointed out copying files sans-antivirus seems to work just fine.
A guess about what it might be, then:
This is what debugging skills are about. Diagnose, differentiate, and reason. Don't just guess and then post flamebait on
And you see no irony?
You're conjecture, while interesting doesn't really work on systems with file system caching that can do "scatter gather" at the file system and kernel level. The optimization happens behind the scenes.
At least for the last couple decades, the disk will ALWAYS be slower than the CPU and RAM, so no matter what you do, it will spend time waiting, and it doesn't make sense to introduce more context switches for the multiple threads coping to the same I/O bottleneck.
As for seek times on hard disks, that is so unpredictable. Modern hard disks only present cylinders and heads to the controller, most of the time there is bad sector remapping, internal sector redundancy like RAID, and other things that make modern disks "storage subsystems" almost completely divorced from the physical characteristics of a hard disk.
This is what makes this an amateur level mistake. Sure, OS,shell and tool creators make mistakes. But what is absolutely critical for a professional product, is reliable, visible and meaningful error reports. Without them you can corrupt your data and notice only a long time later. This is completely unacceptable.
What mystifies me is that MS gets away with this stuff. They are producing a toy and for a toy, Winwos is pretty usable. But for anything that needs to work reliable, Windows is unfit. Seems to me too meny people do not undertand that it does not have to be like this and that there are alternatives ou there that meet professional standards.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I call B.S. "Vista is crap" and "Vista sucks".. yeah yeah... but there's something else wrong here. I read mention of a possible virus scanner above, which may be the case. But I KNOW this doesn't affect a plain ol' install of Vista. I copy _tremendous_ amounts of data on a regular basis. I copy (Just using explorer, file by file) entire drives from different systems every few days onto my NAS box through my Windows Vista 'Business' PC. I've NEVER had an issue running out of memory (2GB RAM). In fact, just recently, I copied 29,912 files (70GB) from my girlfriend's XP computer using Windows Explorer in Vista. It copied just fine to my PC. Then I fired up her new Vista Home Premium laptop and copied those files over the network using her Windows Explorer. Everyone wants to rag on Vista - and yeah, it might be 'too little, too late' - but it works and plays well. It looks elegant and polished and it works almost no differently. Sure, there are applications that don't work well or at all in Vista, but who the hell expects _every_ software title to work with the successor to a SIX YEAR OLD OS? Give us (especially me) a break! ;)
Zero hang ups. Zero issues. I rest my case.
Fact: Everything I say is fiction.
sarchasm ('sär-"ka-z&m) : The giant gulf between what is said and the person who doesn't get it.
;)
Mod GP +1 Funny, he's lost my mod now. Incidentally, the first three digits of my uid are 640...what are the chances?
ZDNet is reporting that not only does it run out of memory after copying 16,400+ files [...]
...
Okay, here is the solution that should work:
sh -c "ulimit -v 500000; explorer.exe"
Oh, wait
Oh ffs Gates never said that!
"Yeah Tommy, before Zee Germans get here
Nah, it is not a legitimate bug, it is a known bug.
It is still present in Solaris as of this writing (at least in the version we got here).
GPL Solaris might fix it.
Patents Drive Free Software as Hurricanes Drive Construction Industry
Read TFA completely, then:
This explains why does the business market is refusing to adopt Windows Vista. "Where are those TPS reports???? Did you copy them?"
BTW I recall the old days when I had a Disk Operating System instead of windows I'd took for granted that vista had to be an improvement over that but I was wrong.
It's impressive how many serious bugs are there in Vista. And Microsoft doesn't care to really issue a fix for it. How the hell didn't this go into the SP1? It seems they have ridiculously strict freeze periods before releasing anything. This thing is probably a one-liner. Anyway SP1 is supposed to include somewhat more complicated changes than a explorer file copy bugfix.
I can't copy that much files!
[cancel] [allow]
Just put the think in the box and return it to the store.
I think you need to get your "think" back from the store and unpack it again.
>...why would anyone want to copy 16400+ files?
Clearly a man who never backs up his pr0n collection.
I want a list of atrocities done in your name - Recoil
Well we know that it's running out of memory. So it's a memory leak. Diagnose. Differentiate. It happens when there are hooks into the file system (e.g. in this case an AV is present). Reason. Likely since we're running out of memory when an AV is present, it has something to do with how the OS passes data to the AV and does not free it.
True, I'm just speculating, but it's a lot better reasoned than what the other poster came up with. Since the problem doesn't occur when an AV isn't present, it likely isn't a "multi-threaded" issue. If that were the case it would appear with or without the AV.
And my "attack" is based solely on experience. There are far more coders out in the wild than developers. If you can't tell me the difference between testing and verification, you're not a developer. If you can't describe a very modest design process, you're not a developer. etc. If you look at the quality of the vast majority of software, then say "why on Earth did they write it this was?" It was likely a coder who wrote it.
Not saying coders are stupid, but they solve problems in a very different way than developers. A developer solves a problem by analyzing the problem, gathering requirements, devising methods to test and verify the solution then implements it. A coder solves a problem by "writing just enough code" so the problem goes away. Coders for example, tend to write less modular code (e.g. reusable) than developers because they don't plan ahead to think about what could be re-factored and broken up. Coders typically don't write libraries for example. Writing a good library means you have to think about how other developers/coders will use it. It requires you to follow a design, do things sensibly without a vast re-invention of the wheel all the time, etc.
That being said, there is a need for coders in business. A good coder isn't stupid, they have experience and know how to turn a design document into software. But as I said, there are a lot of mediocre coders out there as well who are basically one step removed from being script kiddies.
Someday, I'll have a real sig.
Not only that but many new disks introduce variants of NCQ which allows the controller to optimize requests based on locality. So likely the majority of seek operations are being optimized out at that level.
Tom
Someday, I'll have a real sig.
...I've put in two Vista boxes (Dell preloads, but *clean* preloads) and both have exhibited similar symptoms. One one, unzipping a 12mb zip file took *twenty seven minutes" to complete, which I'd expect to take around 5 seconds on XP. During this time whilst I was sitting there tapping my fingers the progress bar stuck on "estimating time to copy" for the majority of it.
Copying files from SMB network shares is similarly slow: incredibly slow to mount, and glacial to copy. There's an MS hotfix available if you call PSS that purports to fix this, but I didn't notice an improvement. Turning off automatically generate file previews might speed it a little, but there's something very broken in the world of Vista file copying...
..he'd have put the video & ROM at the bottom of memory and let general purpose memory start from wherever this finished. That way general memory could just continue on up no matter how much you put in instead of having to dick about with a load of page switching nonsense to avoid the top 384K if you had more than 1Meg...
"I'm sailing, away
I talk about stuff.
It was an OLE memory leak...which creates a great problem with a new O/S, potentially costing thousands of dollars or even millions if exploited by hackers.
The OLE mechanism is written in C/C++, isn't it? well, many thanks, again, for these wonderful languages that have made our life an adventure.
(when humanity will get to grips, we will perhaps use a more sane programming language for our O/Ses and services).
Unlike humans used to computers begin counting at 0 hence there were 21 address buses giving 1 088k in all.
Still 640k was 10 times more than what Steve Jobs said people needed!
With all the Vista bashing in this thread, I'd like to say something nice about it: It looks pretty.
It may not be absolutely correct (note that there is humour there), however, it hits the nail on the head.
Exactly why does Explorer need to 'verify' before deleting?
You have a sick, twisted mind. Please subscribe me to your newsletter.
Microsoft's self service hotfix page @ https://support.microsoft.com/contactus2/emailcontact.aspx?scid=sw;en;1410&WS=hotfix
True, I'm just speculating, but it's a lot better reasoned than what the other poster came up with. Since the problem doesn't occur when an AV isn't present, it likely isn't a "multi-threaded" issue. If that were the case it would appear with or without the AV.
I have not seen that it documented that does not happen without AV
And my "attack" is based solely on experience. There are far more coders out in the wild than developers. If you can't tell me the difference between testing and verification, you're not a developer.
This is the pot calling the kettle black. We have neither code nor internal documentation, without in-depth analysis of the problem with a good debugger and tracing the execution, everything else is speculation.
I have no issue with your description of "coder" vs "developer" per se' but I think you miss something bigger. "Coders" and "developers" are a method of pushing computer science into the realm of blue collar work. Perhaps in this day with most of the important work behind us, at least is seems that way, programming is nothing more than a trade, certainly the current state of the industry shows that, but there was a time when "programmers" understood things like hash tables, trees, recursion, clock cycles, CPU caching, instruction execution time, I/O, etc.
Now, guys with a couple years working in the field with almost no education (not necessarily school mind you, but education such as books and basic curiosity and research), feel completely comfortable impugning people in place of proper argument.
In my book, if you have to insult, then you've already lost the argument, you just want smear the person who beat you. Smarter people see through this tactic, stupid people buy into it.
Just put the think in the box and return it to the store.
I think you need to get your "think" back from the store and unpack it again.
Oh look! someone who thinks it is witty to pun a typo!! I bet you'll commemorate this day in finger paint.
Get a life.
Is when Internet Explorer downloads a large file, it forces you to stick it first in a temporary directory, then it slowly COPIES it wherever you actually said you wanted it. Fucking ridiculous.
Long answer: see "Short answer"
What's purple and commutes? An Abelian grape.
It's a memory leak related to large NTFS alternate data streams, not any fundamental limit. If you're not running Kaspersky (which attaches data streams to every file) and you don't specifically use alternate data streams, it shouldn't be a problem.
What's purple and commutes? An Abelian grape.
What's purple and commutes? An Abelian grape.
How is parent post a troll? The entire topic was Vista not being able to copy more than N files and how bad that was. Pointing out that Unix has had the same problem for ~30 years is on topic and relevant.
I don't think there's really any possibility for trolling for responses with parent post. Are there people out there that actually think a number of files limit in Unix is a good thing?? I doubt it.
Whereas you seem to get off on belittling other people for daring to make a joke. Get over it, troll.
Whereas you seem to get off on belittling other people for daring to make a joke. Get over it, troll.
A joke? Seriously? If one can not fight back after an attempt was made to insult, then insults should be stopped. And you have the nerve to call me a troll.
You are running AutoCAD with TENS of thousands of objects and you only have 512MB of ram? WOW, I find working with dozens of objects with 1gb ram to be slow. I can't imagine what you're going through.
Does that mean Microsoft is excused from shitty coding practices? Yes - it often seems that way - but why when "If you had coded a day in your life you'd see that." do we continue to trust these continuing incompetents with our businesses and government? Why do we continue to trust what is the corporate equivalent to a convicted felon to house our data?
~just curious
You underestimate the amount of turnover at a company like MSFT. I suspect the true "in the blood" comp.sci nerds at MSFT are not working on explorer.exe or kernel32.dll, etc.
Which is why a lot of the seemingly simpler things are routinely getting done wrong.
Someday, I'll have a real sig.
C0cKLeSs????? (W7F?!?!"?!!1111!)
Now, that was useful. Thanks.