Slashdot Mirror
Which OS Performs Best With SSDs?
Lucas123 writes "Linux, Vista and Mac OS perform differently with solid state disks. While all of them work well with SSDs, as they write data more efficiently or run fewer applications in the background than XP, surprisingly Windows 2000 appears to be the winner when it comes to performance. However, no OS has yet been optimized to work with SSDs. This lost opportunity is one Microsoft plans to address with Windows 7; Apple, too, is likely to upgrade its platform soon for better SSD performance."
Geniuses often don't play well with others.
They didn't compare anything to Linux, so I just have one question:
How easy would it be to modify Windows 2000 to be even better? Replace the file systems, alter the way the kernel writes to the drive, etc?
If I have nothing to hide, don't search me
It conflates the results of several independent tests to form the view that XP is somehow best. It also bandies about meaningless numbers like one OS being x% faster than another without giving any hint of the metric.
Avoid.
Nick
Filesystems are fundamentally engineered to cope with the high latency of hard drives, so I'd imagine there are a lot of assumptions to unlearn. But what other implications are there for the OS? Since the tradeoffs between RAM and persistent storage are smaller with SSD, maybe the changes should go beyond the filesystem into the virutal memory system?
...a lost opportunity... Since the market has already hit its peak and it's too late now. And they'll never be able to sell to the 20 people that are using SSDs.
Whale
What is your criteria for a different file system? NTFS is already a very good one for most of the important criteria.
from TFA: "Linux is "always faster" than Vista or Mac OS X"
FTA:
"Linux is "always faster" than Vista or Mac OS X -- to the tune of 1% to 2% -- because like Windows 2000, "it never runs anything in the background."
I'm sorry , what? Have these people never heard of daemon processes? What the hell are they talking about?
If this is their level of expertise I think I'd take any tests they do with a whole cellar full of salt.
Uhm, what are they talking about? What are the things "running in the background" that are presumably hitting the disk all the time? Why do they believe that Windows 98 somehow magically bypass the wear-levelling built in to the SSDs? Or are they talking about raw flash, not sitting behind a controller?
I find this a little scary too: Microsoft also plans a certification program for SSDs
The conspiracy theorist in me see a future with proprietary extensions and requirement for getting certification is to make windows-drivers only... :P
c++;
Have they tuned Linux for SSD? Like setting no-op IO scheduler (which gives about 20% speedup on some workloads)?
I suspect that Win2000 and Win98 win because they have the most simple (and stupid) IO schedulers. That's a problem for conventional HDDs, but it's an advantage for SSDs.
Also, they are talking about "Win98 doesn't support wear-levelling technology". But that's incredibly stupid since modern 'disk-like' SSDs do wear-leveling in hardware.
NTFS is terrible. It's ability to fragment itself into tiny little pieces that cripple server performance and stability is proof of that.
Since they didn't test Solaris, the test is meaningless. It's the only OS in existence right now with caching and data management features designed specifically to take advantage of flash to improve real-world performance. The submitter's assertion to the contrary is a deliberate lie, an assumption that until Microsoft does something it hasn't been done, or at best sheer ignorance.
Read up on the ZFS L2ARC and the use of supercap/DRAM/flash subsystems for separate intent logs that make up the hybrid storage pool. There are plenty of white papers and other material out there, and of course you can also read the source code.
They meant relevent things. For example, XP will defrag your disk without you asking it to as soon as you're marginally idle... Vista the same, but will also index your disk continually...
According to Far, Mac OS X runs "a little faster than Vista" with an SSD drive, but Linux is "always faster" than Vista or Mac OS X -- to the tune of 1% to 2% -- because like Windows 2000, "it never runs anything in the background."
Ok, so Linux and Windows 2000 never run anything in the background. My head exploded so I stopped reading.
The article is all over the map, discussing in vague terms everything from boot-up speed to I/O speed to some sort of generic "runs a little faster" that I assume (?) means overall system or app benchmark performance.
When actual numbers are quoted, they sound somewhere between questionable and boring. The article quotes all sorts of differences in the range of 1% and 2%. Leaving aside the question of what this is a 2% difference in, and whether a difference that small is even consistently measurable outside of sampling error and quirks of their particular setup, does it actually matter? I'm certainly not going to choose an OS based on a 2% difference in SSD performance.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
A lot of the really small low power laptops use SSD hard drives. Dell's mini's use them. There is a market for these little laptops. Granted these are not for gamers, or engineers. But for email, web surfing, taking notes in class, these things work fine. I have seen a lot of those Dell mini's in the hands of college students. This fall I may see a lot more.
for when you need to partition your wife
"
Linux is "always faster" than Vista or Mac OS X -- to the tune of 1% to 2% -- because like Windows 2000, "it never runs anything in the background.
"
WTF? Have they never heard of "daemons" or "services" ?!?
WTF!
Actually recent benchmarks have shown that defragging doesn't make *that* much of a difference - http://www.maximumpc.com/article/the_disk_defrag_difference?page=0%2C2 I've never heard of a fragmented drive affecting machine stability. That's like saying having a 5400 RPM drive instead of a 10,000 RPM drive in a server will make it crash... It makes no sense. Fragmentation has nothing to do with data integrity which is the only thing that would affect stability.
Also, it's "its" not "it's".
You are using English. Please learn the difference between loose and lose; they're, there, and their; your and you're.
Even if their methodology was clear.
Even if their methodology was valid.
Even if every percentage point was accurate.
Even if all of their arguments were valid.
(And none of these are true.)
Why would I care enough about 5% to let that pick my OS?
On most distris slocate also indexes your disk in the background by a cronjob not asking the user to do so... I also think the test is trash judging on the niveau of this articel.
It's not only 'bad' it's weirdly bad.
According to Far, Mac OS X runs "a little faster than Vista" with an SSD drive, but Linux is "always faster" than Vista or Mac OS X -- to the tune of 1% to 2% -- because like Windows 2000, "it never runs anything in the background."
Never runs anything in the background? What in the world does that mean? Am I missing something, here, or is this just the wrong terminology to use? Even at the most basic use level, you can "background" an application in Linux with the ampersand ... I'm confused.
Not to mention that they didn't test Linux... or, presumably, any Unix OS. They basically only tested Apple and Microsoft operating systems. Hm. I actually like Windows on occasion, and I find that pretty stupid. (I also like Linux :) )
It did have some interesting things to say though, like the block alignment.
Pretty damn hard, since step 1 of the project is "disassemble Windows and make sense of it." There are reasons people like working with Open Source stuff, and having the source is one of them.
"Believe me!" -- Donald Trump
Obviously, SSD's are in their infancy. NO OS has been even remotely optimized for them yet, I'm sure (except maybe the big hitters, like Solaris). I'd be willing to be my left leg that the next version of every commercial OS (OSX, Windows, Linux*) is optimized for them. This article is irrelevant.
Wasn't a massive test. Used Swingbench against 170g database. Quad core, 2gig. Disks were 300g 15k fiber, dual pathed. Swingbench seemed to be like 50 users running VERY poor queries, so this was almost 98% reads. Ms response time was between 3-4, cache hits minimal. Added EMC (1) 146g SSD into the disk group. Response time hit .4 to 1 ms response, identical tests.
XP will not. OS X will. In fact, OS X will defrag your thumb drive without your permission. It was bad enough that it was wearing my disk for no benefit, but it also made my thumb drive remarkably non-resiliant to power outages.
Your ad here. Ask me how!
Stop posting on Slashdot and go back to jail, Hans.
the source of windows 2000 was leaked a few years ago. http://www.kuro5hin.org/story/2004/2/15/71552/7795
Overall, I would state that this article is useless beyond "cocktail gossip." And really, SSD should have a specific FS written to its peculiarities, which would, of course, render the "OS" questions moot.
Ruby Neural Evolution of Augmenting Topologies
I developed the distinct impression that the author of this article didn't really understand much of what he was writing about, or else dumbed it down so badly that it lost any semblance of accuracy. I'll admit, I'm no expert (not even close), but to me it seemed very inaccurate.
For example, there's some talk about how Windows boots compared to OSX, and he goes on to claim that the "BIOS does lots of stuff" while Windows is waiting for the hard drive to spin up. Um, does me mean while the BIOS is POSTing? As in, before Windows has anything to do with the boot procedure? As in, this has nothing whatsoever to do with Windows as he seems to be suggesting? Maybe he's just taking the direct quote without qualification here, but it seems like something that would be important to qualify if your pretense is writing an article about operating systems and not motherboard/BIOS architectures.
Really, it was just riddled with moments like that were one end of a sentence didn't seem to jive with the other, but it could just be my ignorance on the subject.
SSDs may change before they become very popular. Right now they are cool, but still too expensive and too small to be used in mainstream computers. We've got a few more years before they start to become the sort of thing you see in normal systems. Well in that time they certainly could change. What is true about their performance today may not be then.
Will we still be doing benchmarks in Windows 2000?
If we will be, why dont we run benchmarks on Apple 7.x or Windows 3.1 ?
I knew it! I'm a genius!
Fragmentation causes more needle shifts than regular burst read/writes. If a hard-drive is most likely to die from needle shifts, fragmentation could wear on the drive more than a nice and tidy system.
Of course this is all speculation and moving the needle could have absolutely nothing to do with death rates, who knows.
Bye!
LOL.
I had in past run a Windows server for about 2 years. And it was terribly slow, despite the fact that in the beginning it was blazingly fast. I believe it was O&O Defrag which actually returned the server to life: downtime on Sunday with boot time defragmentation did the miracle.
Same thing in the company I work for right now: IT recently took off net two file servers and exchange server to defragment file systems, because FS performance went considerably down. Folks have said that they "lose to fragmentation 30% of FS performance," meaning that system works about twice faster after defragmentation. That's why they schedule at least one down time for every windows server in company.
Whatever synthetic benchmarks people perform - it is irrelevant.
Long term real life experience tells otherwise.
All hope abandon ye who enter here.
Can anyone weigh in on how Andriod or another full-featured mobile OS performs with flash, considering that flash is the standard storage type for the platforms they were developed for?
From TFA . . .
Now that is what I would call a ringing endorsement!
I'll see your senator, and I'll raise you two judges.
I've never heard of a fragmented drive affecting machine stability.
Hear it now. I use to work at a digital invoicing company which used ext2 on linux. We were seeing extremely slow reads and writes to the disk. Now, in this environment, we were writing upwards of 10 gigs of data a day(spread amongst thousands of files). We would also delete 9 gigs of that daily. Compound this behavior over a couple of years and we were left with a heavily fragmented disk. The solution was simple... we re-wrote all the data.
Fragmentation has nothing to do with data integrity which is the only thing that would affect stability.
Wrong again, fragmentation makes the disk have to work harder. Think about it... the disk could read 10 bytes incrementing a byte at a time or it could be required to skip 20 gigs to read each byte(this is an over-simplification). This will increase wear and tear on the moving parts as well as extra heat. So, extreme fragmentation will likely decrease the life of your disks.
depends on your definition of crash. Having a slower HD will cause disk IO to stack under heavy load. What is the real difference between a server that has crashed, and one that doesn't respond? Fragmented disks will have the possibility of a similar effect. The linked article measures the difference between a disk that is 7.5% fragmented and after its been defragged. 7.5% isn't much. It really depends upon which files have been fragmented. If they are rarely accessed data files, then obviously there isn't going to be much of a performance difference. If they are frequently accessed, then the difference would be huge. I've seen desktops that topped out at 30% fragmentation. The performance after dragging is night and day. Its much more important for me to have consistent performance ( due to automatic reallocation of files when they would be fragmented) than a gradual slowdown that requires downtime for defragmentation.
Note, haven't run windows in production servers for the past ten years. But I don't think NTFS has changed that much since then.
Well.. maybe. Or Maybe not. But Definitely not sort of.
... using fewer applications on a given time, disable the virtual memory.
As the currenlty available UMPC (like the newest eeepc) has 1 or even 2 Gigabytes of RAM, one can choose to sacrifice some addressing space in exchange to avoiding swapping memory to SSD.
It works, I tried it on a eeepc from a friend, and the little thing started to fly. However, it was unable to open as many webpages as before.
It's an exchange. You trade (virtual) memory size for speed.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
That's because Mac users are non technical and open their files by name rather than cluster number. If you do that then defragmenting doesn't break anything.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
as in - are there any circumstance where reducing the number of needless writes makes the machine run slower. All I can imagine there being a drawback would be virtual memory - but does that still exist in modern OSs?
Not to rain on your anecdote, but 10GB of changes a day isn't a whole lot. I have some systems that will read and write that in 8 seconds. We don't worry too much about drive fragmentation, but our file tend to be stored in gigantic chunks anyway and we don't do lots of little read/write accesses that would tend to fragment the system. In my experiance, the best way to avoid fragmentation is to leave lots of free space on the disk. If you let the disk get above some threshold (I'm not exactly sure where it is, but I'm guessing somewhere around 75%) then it will start fragmenting a lot, especially if you update large files repeatedly (Databases).
I read the internet for the articles.
XP will not. OS X will. In fact, OS X will defrag your thumb drive without your permission. It was bad enough that it was wearing my disk for no benefit, but it also made my thumb drive remarkably non-resiliant to power outages.
Actually, no, they both do for varying definitiosn of defrag. What the parent was talking about is almost certainly advapi32.dll's ProcessIdleTasks() routine which does hot-band bootup files described in layout.ini to the head of the disk. (giggity)
Then again, saying Linux doesn't "do stuff in the background" like this is misleading too -- cron jobs, Tracker/Beagle/Strigi on various distributions, and so on.
How is it surprising that a decade-old operating system runs faster on modern hardware than modern operating systems bloated with extra features? They should have tested MS-DOS as well, I'd bet they would have a new 'winner'.
Support Right To Repair Legislation.
i alwasy got a kick out of the NT4 documentation for how to defragment a partition.. the "best pratice" was to back the data up to tape then format the partition and restore from the tape..
alwasy gave me a good laugh..
'...if only "Jumping to a Conclusion" was an event in the Olympics.'
Who chooses an OS around a disk FFS! I'd have thought most people would chose an OS that suits there needs based around usability, security, app availability or whatever and then (maybe) select a disk that works well with that OS. Is it really that much of an issue anyway?
It is not immoral to create the human species - with or without ceremony, Samuel Clemens.
15 percent of the source of Windows 2k leaked. Probably not enough to be useful.
Actually thru extremely slow hard disk drives you are able to discover a lot of race-conditions that would otherwise go unnoticed.
You know those 15% might as well be all that's been changed. *snicker*
WIN2K. I resemble that remark. I am still running Win 2K and Office 97, and they still WFM! WooT!
In the land of the blind, the one-eyed man is king.
Filesystems is the biggest deal.
Apparently nilfs2 is EXTREMELY good for flash based memories.
However it's still in development and not main line. It has the added advantage of being log based with continuous snapshots and the filesystem at any time can be mounted (read only) at any previous snapshot time.
For a spinning disk this filesystem is bad, causing lots of fragmentation.
nope you're a coward :P
I can't call that English
Even if fragmentation occurred in NTFS as much as the parent post claims, SSD's are uneffected since SSD's have no seek time.
Windoze of course! Bring om em viriiiii!!!!11
How ridiculous. Of course win2k is faster, it is an older operating system, it performs faster on normal hard drives! Is there a shortage of news stories today?
only 15% of the source code:
OS X defragging your thumb drive made it less resistant to power outages? What are you talking about?
10GB's of small files several of which get deleted was the source of the fragmentation. The quantity and the distance between files was the point, but your point is also valid. The condition this place was in was largely due to the fact that data was not removed per any normalize bahavior(no FIFO or LIFO). It was largely dependent on a several hundred humans hitting an accept button. Some humans were really good about it, others would add a month delay.
Linux will have good support SSD support, you can be sure of it as Linus uses it himself, he mentions it in his blog. (he is using an intel SSD drive) http://torvalds-family.blogspot.com/2008/10/so-i-got-one-of-new-intel-ssds.html
the rest can go to hell. I only need 4GB space total anyway.
I am the genius !
I'm positive, don't belive me look at my karma
This is how you defrag ReiserFS, except you don't need to;) Inb4 hehehe he killed his wife it's so funny guys.
93rd rule of Slashdot: No matter how obvious my sarcasm is, my comment will be taken seriously by someone.
Presumably, increases the probability of a write operation happening when the power outage occurs; the probability of data corruption/ bricking da stick is therefore higher at that point too.
Okay, I'll buy that, but unless power outages (undefended by UPS) are a constant worry, it doesn't seem like it would be a real issue.
It's like not using the radio in your car because you're concerned it might affect the life of your car battery. You're probably not wrong, but it's a rare person who cares.
>> Microsoft plans to address with Windows 7 and that
>> Apple is likely to soon upgrade its platform for as well.
Apple will likely release TWO versions of Mac OS X, before Windows 7 even comes out (and by "comes out" I mean _really_ comes out - SP1, not the beta that they call their final release). Given that Apple ships SSD laptops already, I'll be STUNNED if Snow Leopard doesn't contain SSD optimizations.
>I'm suspicious of the suggestion that a log-based
>filesystem will cure all the ills of the limited flash-
>controller based wear leveling.
Yeah. Total bull.
Anybody who thinks the filesystem can do really well has
bought into the crud from most existing vendors about how
you have to use those things differently. If you really
do believe that, you shouldn't touch an SSD with a ten-foot
pole.
If the flash vendor talks about "limits" in the wear
levelling, and how you have to write certain ways, just
start running away. Don't walk. Run away as fast as you
can.
>A question keeps coming up in my mind about what happens
>when you split an SSD into multiple partitions, and what
>*you want to happen*. I use separate partitions for root,
>boot, and var, because I tend to make root and boot
>read-only.
Again, if your SSD vendor says "align to 64kB boundaries"
or anything like that, you really should tell them to go
away, and you should do what Val said - just get a real
disk instead. Let them peddle their crap to people who are
stupider than you, but don't buy their SSD.
So what you want to happen if you split an SSD into multiple
partitions is exactly nothing. It shouldn't matter
one whit. If it does, the SSD is not worth buying. If it is
so sensitive to access patterns that you can't reasonably
write your data where you want to, just say "No, thank you".
Anyway, I have a good SSD now, so I can actually
give some data:
- Most flash-based SSD's currently suck.
I don't have these ones myself, but last week we had the
yearly kernel summit here in Portland, and a flash
company that shall remain nameless (but is one of the
absolute biggest and most recognizable names in flash)
was selling their snake-oil about how you need to write
in certain patterns.
So I called them on it, and called them idiots. Probably
one reason why I didn't get one of the drives they were
handing out, but one of the people who did get a drive
was the Linux block system maintainer. So he ran some
benchmarks.
Those things suck. You will never get any decent
performance of anything but a very specialized filesystem
out of them, unless you use them as essentially read-only
devices.
For a basic 4kB blocksize random write test, the SSD got
around 10 IOps. That's ten, as in "How many fingers do
you have?" or as in "That's really pathetic". It means
that you cannot actually use it as a disk at all, and
you need some special filesystem to make it worthwhile,
and certainly means that wear levelling is probably not
working right.
(For the math-challenged, 10 IOps at a 4kB blocksize
means 40kB/s throughput and 100ms+ latencies for those
things. It also means that even if some operations are
fast, you can never trust the drive)
- In contrast, the Intel SSD's are performing exactly as
advertised.
I did get one of these, with warnings about how
if I want to get low-power operation etc I need to make
sure that disk-initiated power management is enabled etc.
Whatever. The important thing is that the Intel SSD does
not care one whit where you write stuff, or how you do
it. With the same 4kB random write b
He said, "You'll be able to tell your grandchildren that you helped assemble the first NT supercomputer," and I cringed.
Heh. That's why I said "presume". I've never considered it before that comment. Rare probability * rare probability = similar probability to that of all of these obscure diseases presenting to one hospital in New Jersey; the practice of one Gregory House, MD.
Agreed, a scheduled quarterly (or monthly for mail servers) (re)boot defrag of servers does wonders. Drives tend to fragment more during install of software, more so than any other time. I always do a defrag after installing software.
My work issued laptop (it just stays docked all the time) is set to defrag nightly... I know this is overkill, but I check in/out several large projects from time to time, so I like it tidy. Paid for defragmentation utilities like Diskeeper etc, work very well for auto-defrag environments.
Michael J. Ryan - tracker1.info
Why would you install it? We're assuming it's not on an old system still in use because it can't take XP, or your business can't justify the upgrade costs. Windows 98 is not supported anymore, so Microsoft have nobody patching the new holes, so that gets less secure by the day. Windows 2000 is next on the list to be abandoned. Would you install it now?
If Microsoft open sourced their older OS's the story would be different, but cutting off support is all part of the big stick prodding you to buy the new stuff. They don't sell all their supported versions of their software, if they did Vista would REALLY have bombed. If you're installing an OS, it's for one of two purposes...either to use (present to future) or to test (present). If it's for using you don't want to have to drop it when the support cuts off.
Having Windows 98 or 2000 to install on it assumes you have the CD's because you have no chance of buying them. As interesting (and flawed) as this article is, it'd be better to compare supported OS's you can actually download / buy. At least then it may have some merit.
Running an OS from a SSD is still relatively new, so this will get better. I can't see Apple doing much innovation here since they like charging a fortune for "return to base, non consumer replaceable, custom parts"; which would include a worn out SSD. It's in their financial interests to keep things wearing out. Microsoft got caught with their pants down on the whole netbooks phenomenon, just as they did with multi-user computing and the internet. I'd imagine Vista taught some lessons inside Microsoft (even if they try to keep the public "Vista is great" front) but they've long been riding the "just buy a new PC with another Windows licence when it's slow and worn down" train for a long time now; I can't see them wanting to step off it. They use new versions of Windows to let the OEMs sell new hardware.
Linux / BSD by comparison has always been about getting better performance, as much as doing stuff "because you can". With the code being open, there is no commercial vendor thinking ahead for hardware sales to steer the projects off. I see Linux / BSD being the logical innovators here. Having said that, as the recession gets worse both Apple and Microsoft know their customers will be tightening their belts so who knows? Free software will really bite into previously solid Microsoft territory as companies look for ways to cut costs and still compete. The recession may be the best thing to happen to Linux / BSD to help catapult the user base.
OS X does NOT defrag your disks, ever. Maybe you are talking about Spotlight indexing your filesystems?
Obviously, SSD's are in their infancy. NO OS has been even remotely optimized for them yet
The Wii console has a 512 MB SSD, formatted in an SSD-specific file system called SFFS.
I'm still using Windows 2000, though I have a (legal) Windows XP SP2 CD on the shelf. Who's your daddy?
In fact, OS X will defrag your thumb drive without your permission.
That's true. I run Defrag on my Mac thumb drives all the time to improve performance, and it works even without Administrator permissions. It makes a HUGE difference in performance. And since it is a compressing file system, it can double the storage capacity of the thumb drive. Too bad they don't work with Vista out of the box, but you can download a hacked driver from OS X Panther that is Vista compliant.
Linux is not a "very major OS" [...] that has less than a few percent of the desktop market
Who said anything about the desktop market? There are plenty of subnotebooks, handhelds, and embedded devices that boot from flash into a Linux-based environment. I would imagine that even servers could benefit from the faster seeks and lower heat dissipation of SSDs for some workloads.
he goes on to claim that the "BIOS does lots of stuff" while Windows is waiting for the hard drive to spin up. Um, does me mean while the BIOS is POSTing? As in, before Windows has anything to do with the boot procedure?
Perhaps he really meant to say this: If the BIOS didn't need to wait for the boot drive to spin up, it could probably self-test less of the system before handing control to the bootloader and kernel. The kernel would do further self-tests, prefetch modules and other files needed early in the boot process, and start up other hardware, and doing them all at once would save time. I imagine that LinuxBIOS (coreboot set to load Linux) does something like this.
I am at the Ubuntu Developer Summit at Google and listened to a talk given by tytso a few days ago. He mentioned that both ext4 and btrfs will support a new ATA command to tell the drive that a particular sector in no longer in use so that it can reuse it for better wear leveling. So it appears Linux will have better support for SSDs in the near future.
It really depends on your definition (or use) of (the word) stability.
Fragmentation may not make the machine crash in the normal sense, but on a server that is near it's capacity already (capacity as measured by timeouts of various server daemons), it can impact stability - for instance when a database connection is dropped because of a timeout in reading the data and so on. A drive that is not fragmented (ie: a file system that does not fragment as horrendously as NTFS, or one kept defragmented) will not run into those issues nearly as soon.
In today's world of SATA and IDE servers, the issue is more critical as well, as there is more CPU overhead during disk access by the file system and disk drivers.
So, you are probably correct when it comes to crashes and such of the hardware or OS... but fragmentation does play a decent role when it comes to a high use server (or similar client applications) where "tons" of concurrent disk reads occur.
And of course, the difference in performance on such scenarios can be staggering. In a web server or SQL server environment, on a "big" server where there can often be "tons" of files for "tons" of sites/clients, the impact is easily measurable. Our server has over a million files on one partition - for which we cannot determine what will be accessed when (ie: no "smart" or dedicated caching of 99% of them), and because of the sheer volume/size of the partition, it is also "impossible" to cache them all - or even a large percentage (ever try to set up a half terabyte disk cache? I know I couldnt even begin to afford to - even if I found boards that supported that much) - which means that data in the cache will expire or be pushed out by other requests long before the cache becomes fully beneficial for 99% of the reads. While a read-ahead cache helps (ie: disk reads are still faster than our WAN bandwidth), it only helps so much because the individual files on the server are often many times larger than the machine's full installed memory amount.
These are areas where I have seen marked differences in NTFS, JFS and HPFS386. HPFS386 handles the load (a few hundred to a thousand concurrent web/ftp connections) far better because it barely ever fragments. NTFS is abysmal in this area. JFS is barely better, but handles writing quicker (as it does not pre-allocate storage space based off size - which is helpful for a lot of concurrent write operations as overhead is reduced somewhat).
Currently, most of our web serving is done via an HPFS386 partition as it is quicker and less resource intensive - and the partition still shows no noticeable fragmentation... while much of our FTP data resides on a JFS partition because, unlike HPFS386, it can handle files larger than 2GB and partitions faaaar larger than 64GB.
The particular file system limitations aside, I can definitely tell you, JFS is far less happy trying to read a bunch of large fragmented files at the same time as HPFS386 is at reading a bunch of large non-fragmented files.
Of course, I tend to have more CPU power than I need, and span certain services on different drives and file systems as appropriate... but then again, I am not an Amazon or Slashdot or whatever where ya never know what the peak traffic tomorrow may be. Even the best planned server implementations for companies that cannot always judge "Gee, this is the max traffic I ever expect to see, lets make the server 4 times as powerful as need be" isn't always good enough, as something can - and probably will happen to push the server beyond even reasonable expectations times four.
We've seen that happen here quite often - "The Slashdot Effect" - and while a file system that does not fragment isn't the sole answer, it does increase a server's ability to handle high loads without timeout issues while waiting for data from a fragmented drive.
So, to summarize, reading a few fragmented files at any given time may not show any serious or even noticeable drops in performance - but, reading a LOT of fragmented files at the same time will be quite noticeable (compared to defragmented files on a better file system).
StarTrekPhase2 - The Five Year Mission Continues!
Well, I'm the hamfucker!
Oh, come on now! *throws chair* Stop ruining my minions^H^H^H^H^H^H^Hteam's astroturfing with your 'facts' and 'logic'! *throws another chair* I'm gonna fscking KILL that Anonymous Coward guy!!
-- Steve B.
My blog
Please rinse it off when you are done.
Horns are really just a broken halo.
I believe XP does have the "Disk Layout task" which runs when the System is Idle. This process is not a defrag but might be confused with defragging.
I couldn't find a good explanation/reference but this article from MSFT gives an overview:
Windows XP does not enter standby after the exact period that is configured in the Power Options profile
Also:
Benchmarking on Windows XP
(look in the section entitled "General Concepts: The Dynamic Nature of Windows XP")
>Well, no, actually -- remember how Firefox used to leak memory? A lot of that was actually memory fragmentation. And RAM has exactly as much seek time as an SSD -- that is, none at all.
Memory fragmentation's a different issue. Individual memory allocations need to be contiguous, so if you're memory's fragmented into lots of small chunks, each large allocation will need to grab a new chunk.
Files don't need to be contiguous, so that can't happen. And, since SSD solves the seek time problem, fragmenting them isn't much of a performance hit either.
In fact, maybe apps like Firefox could use memory-mapped files for large allocations on an SSD-based system to eliminate the need for contiguous memory allocations in the first place.
Posted from my Android phone. Oh, I can change this? There, that's better...
Oddly enough, that's one issue that wouldn't matter with solid state discs.
Once Solaris can talk to half a dozen USB mass storage devices that I have which every OS can talk to (Linux, Windows, the finicky BSD) except Solaris, I might care about how fast it is at doing so.
Solaris is dead, get over it. Sun killed it, as they kill all their products, by sitting on laurels, insulting users and competitors, ignoring suggestions, and rambling on about theoretical abstractions while blithely ignoring weird limitations and leakiness littered through their product.
Oh, and you're talking about OpenSolaris. That's an entirely different OS, which will be used even less than Solaris's declining market share. The only reason people stick with Solaris is stability. Stability of the kernel and of APIs. OpenSolaris is all sorts of backward incompatible and if you look at some of the bug reports on sun.com you'll see that Sun just doesn't care when they break things any more.
Not an issue? So, where's your laptop with its infinite battery?
Nick
Do you often run your laptop down to a dead battery with a thumb drive plugged in?
Not to rain on your parade, but he doesn't specify when this was. 10GB might've been a lot of changes.
You claim that disk fragmentation decreased stability, but you didn't actually cite an example of decreased stability. All you said was that the data was fragmented and that after defragmenting it was no longer fragmented.
I am the walrus! Goo goo ga joob!
"But this one goes to 11!"
Really? Puppy Linux...
...And this adds what exactly to the conversation?
Why is that funny? That method will work on virtually all file systems.
"But this one goes to 11!"
NTFS is so "good" it needs to be defragged. How primitive.
Hans? Is that you?
What an awful article. I hope for these companies' sake that their comments come from clueless PHBs with no real influence on the companies' direction. We have one representative claiming that Linux and Windows 2000 don't run any "background processes". We have the same guy claim that their SSDs will wear out faster on Win98 because it doesn't support load-leveling, which is absurd because this function is performed on the hardware itself and is transparent to the OS. Then a guy from Micron says that XP is slow with SSDs because it doesn't "align" the data in 4K blocks, which is dependent on how a disk is formatted and generally false, because the default cluster size for NTFS in XP is 4K. It would likely only be otherwise if an NTFS partition was converted from FAT, which usually results in 512 byte clusters.
Gamingmuseum.com: Give your 3D accelerator a rest.
I've installed Linux (with ext3 mounted as ext2 and the noatime option) on a bloody slow Asus 701 (the original Eee PC) more than once and have to say that even the slow SSD in this thing makes me not to want any HD again ever.
Reason? Well, the really fast seek-times make up for the slow data transfer. In all day use you and your apps and your OS are reading constantly tiny files strewn all over the drive and a SSD feels just so much faster here. Slow writing means saving a file takes a second or two, but everything else just flies. Together with the utter silence of the thing this feels just *right*. Going back to a computer with actual spinning platters and fluttering heads feels like driving a steam-driven car now.
Don't believe in numbers. Trust your own experiences.
Not often, but maybe twice a year I misjudge how much power I have in my laptop. I don't use OS X but as someone who often has flash memory of some form plugged in I could see how that could be annoying.
Nick
just because something works doesn't* mean you should do it that way
*unles you live in perl - then you just do it and it is done
'...if only "Jumping to a Conclusion" was an event in the Olympics.'
Well, considering NT didn't have a defragmenter built-in to the OS, and a lot of NT servers had tape backup drives, it seems like a reasonable solution to me. The other option was buying third party defragmenting software. Why not use what you already have without having to buy anything else? Besides with the length of time some of the defrag programs would take to run, re-writing from tape could potentially be quicker.
"But this one goes to 11!"
You have little knowledge of Windows or the NTFS. Your comment regarding its tendency to fragment is proof of that.
Gamingmuseum.com: Give your 3D accelerator a rest.
I think you mean "head", not "needle", unless we're talking about 33 RPM vinyl discs here.
Gamingmuseum.com: Give your 3D accelerator a rest.
So the REAL solution would have been to put a defrag tool in NT. Microsoft here promotes a crappy way of doing things (or, at least, very annoying) only because they fail to provide a tool which was necessary with their system. You should'nt have to buy software to compensate defaults of your system and you should'nt have to be obliged to hardware (tapes) for the same reason.
"Freedom can only be the whole of freedom; a piece of freedom is not freedom." Max Stirner
You claim that disk fragmentation decreased stability, but you didn't actually cite an example of decreased stability.
Read my full post and you will see where I note that fragmentation makes the disk work harder and increases heat which can shorten the life of the drive. Yes, I don't have a case study of fragmentation causing disk failure, but in the case of moving parts, it is pretty common knowledge that more movement and more heat are causes of failures.
Nuke it from orbit, its the only way to be sure!
NTFS has always had a problem with fragmentation, the more fragmented, the longer it takes to access files. We lost an Exchange server when trying to defrag the drives using the Microsoft tools, the drives were like 90% capacity and the defrag corrupted the drives. Luckily I wasn't the one who did that defrag...
NTFS also framents the MFT section, which isn't always defragged using the MS tools. The best way to fix MFT fragmentation is to reformat and restore.
Sleep: A completely inadequate substitution for Caffeine.
Kernel developer Dave Jones reckons that using noop with SSD may not be the best because with other schedulers the ioscheduler will do "merging of adjacent read requests".
You forgot opensolaris and zfs there are components in zfs that directly address ssd for write intent logs and read cache as part of the file system. opensolaris using gnome is becoming more of a desktop everyday.
This was also the supported way on other OSes, including most *nix all the way up to the 1990s, and OpenVMS. This is simply because they didn't include a defragger. Defrag existed, but since third-party software is generally out of scope for OS documentation it is usually excluded.
Gamingmuseum.com: Give your 3D accelerator a rest.
Why do people always forget OSes like BeOS, Haiku, AROS... ? ;) :D
BFS might be specially SSD optimized, but it uses extents (does ext3 finally support them ? oh wait, no that'll be in ext4!) (maybe the allocator can be tuned to allocate aligned extents for SSD), xattrs in the inode when they fit, logging, and 64bit right away of course
And Haiku doesn't run that many useless daemons also
You still run web services on OS/2? Nifty :-)
Gamingmuseum.com: Give your 3D accelerator a rest.
But this is NEW JERSEY... we're talking a lot of Superfund sites there.*
* former NJ resident-- send your haughty retorts elsewhere
Gamingmuseum.com: Give your 3D accelerator a rest.
I run a RAIV-5 array, you insensitive clod!
Sam ty sig.
Or they could just run enterprise Linux or Solaris servers, with filesystems that don't crap their pants and need mommy to clean up after them. It must be humiliating to have to schedule regular downtime just to amortise the cost of Windows' sheer stupidity.
Sam ty sig.
Yeah, something like that I guess is what I meant :) I didn't look at it in very much detail... I was mostly refering to whatever XP is doing that trash the hell out of my HDD while the computer is idle. I assumed it was defragging, because even after several year of usage of my machines as developer workstation and for gaming, my disk fragmentation was always very low. I guess it may have just been a coincidence.
Whoops :)
ext3 and a lot of modern filesystems do not need defraging.
http://www.heise-online.co.uk/open/Tuning-the-Linux-file-system-Ext3--/features/110398/3 This article explains how an Ext3 filesystem can be less fragmented than say NTFS but still need defraging under extreme conditions.
The background process using your thumbdrive was probably Spotlight indexing. You can set up exclusions in the Spotlight section of System Preferences.
"Anonymous could not immediately be reached for further comment." - International Business Times
Oh, i'm an idiot... I just noticed that I have scheduled defragging every night on wednesday... never noticed that before, so I guess its a default configuration. That would do it. Learn something new everyday!
I think you replied and debated the wrong parent thread. The man agrees with you, it was silly to leave OpenSolaris out of this test since it is a viable desktop as well as server OS and it is optimized for SSDs.
Don't expect me to disagree, I'm running OpenSolaris on a cute "toy weight" Toshiba laptop with 80Gb SSB. It boots in less than 30 seconds, much faster than my Toshiba Windows HD laptop wakes from sleep or hibernation, but it would be nice to see a like-with-like comparison. I'd be surprised if Linux, OSX or Windows beat Solaris on most SSD benchmarks but if they do, Solaris definitely has the observability to be able to get to the root of any problem and fix it.
XP will not. OS X will. In fact, OS X will defrag your thumb drive without your permission. It was bad enough that it was wearing my disk for no benefit, but it also made my thumb drive remarkably non-resiliant to power outages.
XP *will* in fact. Just check the task manager when idle, you'll notice the defrag program running in the background with HDD activity. That's why I can leave my Windows XP workstations on 24/7 and then when I decide to defrag once a month, there isn't much to defrag at all. I never knew why until I caught my system doing a "hidden" defrag via task manager one day when I noticed HDD activity and nothing was going on with the desktop.
slocate is a once-daily cron job. And that's not the same level of indexing that Vista does - slocate builds a list of filenames. Vista tries to go through each and every file and build a searchable content database. Does a great job of bringing a system to its knees.
The point of the article is that solid state drives have different performance characteristics than hard disks. Since SSDs are random-access devices, fragmentation does not increase seek time. Therefore, it makes sense that a filesystem that doesn't try to worry about fragmentation could perform better on SSDs.
The major overhead for traditional disk access is seek time, i.e. the head moves to the right cylinder and sector. For many decades filesystems have been designed to minimize this seek time.
SSDs changed the game. This kind of optimization is no longer necessary. Now you can blast random IOs to the disk without worrying about extra overhead. This means the filesystem design will be much simpler. A lot of the code could be removed or simplified, and thus making them more robust and faster. They don't need to be as "advanced" as what we have today.
I am not surprised that w2k has a better benchmark - it probably wasn't optimized for traditional disk as much. Simpler is really better.
So, it doesn't fragment?
This is big news. You should call up Bill Gates and let him know he doesn't have to defrag anything at Microsoft any more.
Baaaaaah.
Well, Windows doesnt cut the load we need to handle, and since the latest AMP stack is virtually always available for OS/2 (and I've got a bunch of IBM servers that OS/2 is fine tuned for), I can get the benefit of AMP, the unbeatable (in the PC server world) performance of Domino GoWebserver on OS/2 (for a subset of those servers that arent running Apache), take advantage of one of the few fully featured REXX implementations (including WPS, OS, and networking integration) and fully utilize my hardware with an OS tuned to use all the hardware's added features...
Like any other tool, it just happens to be the best for my job, and easiest (and compared to some OS's like Windows, the cheapest) to maintain. Heck, my "ancient" Netfinity 7000 M10 still does the vast majority of my hosting...
StarTrekPhase2 - The Five Year Mission Continues!
When ext2 was still common, 10GB was often half of an entire hard drive, so yeah, it was a lot of data.
--guru
Also, it's "its" not "it's".
Now you're just trying to confuse him.
If you're talking about a file server that routinely serves many users simultaneously, fragmentation has actually been shown to speed up throughput. What's that? you say.
Assume you have many large files that take several seconds to read being accessed simultaneously by several users. If they are all defragmented, on average the standard deviation from the center of the disk is higher than if they are highly fragmented. This would be irrelevant, but for the fact that a preemptive multitasking OS will periodically swap the current task out and switch to service another user...who's in the middle of reading a different file. The effect is that the head jumps around anyway, despite all of the files being totally defragged.
So in this scenario, how to increase total throughput for all users? It turns out that the more you fragment a given file, and the more a given file is dispersed over the entire disk, the lower the average seek time between context switches as the machine jumps from one task to another. (Well, to a point—if a file is so heavily fragged that the head often has to jump even during a single context switch, that's not optimal either.)
Anyway, this information is literally decades old at this point...I wonder if it's still relevant to modern OSes / file systems? Multiprocessor systems and caching might have solved this during the years in between...can anyone comment?
but have you considered the following argument: shut up.
You must have missed the change. It was a major feature for the Linux 2.6 kernel series. All linux processes now run in the foreground. Running a process in the foreground saves on overhead, thus resulting in a significant improvement in speed. Easily more than the 1-2% that the article mentions. I suspect the article downplayed the speed advantage, to cast Windows in a better light.
Linux achieves this ability by forking the process from the shell, and redirecting the inputs and outputs away, thus giving the illusion of a backgrounded process. The beauty of this is that it's all completely transparent to the end user. The system behaves just as before, apart from the speed increase.
Everyone knows that processes in the foreground run faster. Myself, I used to background a process only out of absolute necessity. These days though, a background process is actually a foreground process, so I don't worry at all about running multiple jobs at the one shell now. Linux is truly a state of the art operating system!
Did this very thing on the microVAX we had back in 1988. No other other tools existed. So, perform complete backup. Format disk, and restore. But only needed to do this very occasionally.
While all of them work well with SSDs as they write data more efficiently or run fewer applications in the background than XP
They also didn't really do a good job of comparing Vista to XP. Vista is by no means an OS that "runs fewer applications in the background". It also thrashes the hard drive a lot more than XP due to SuperFetch and the search indexer.
Well, Win2000 was (admittedly a couple of generations removed from) an offspring of OpenVMS. OpenVMS defragmentation (according to Digital) was best performed by doing a backup of the drive to tape... and then restoring the image back onto the drive.
Silly.
Executive Software made a bundle off of their defrag software for OpenVMS.
I'm an individual! Just like everyone else!
Hear it now. I use to work at a digital invoicing company which used ext2 on linux. We were seeing extremely slow reads and writes to the disk. Now, in this environment, we were writing upwards of 10 gigs of data a day(spread amongst thousands of files). We would also delete 9 gigs of that daily. Compound this behavior over a couple of years and we were left with a heavily fragmented disk. The solution was simple... we re-wrote all the data.
Your anecdote in no way supports your claim that fragmentation has an effect on data integrity. All you showed with your story is that it may have an effect on performance in some scenarios.
Wrong again, fragmentation makes the disk have to work harder. Think about it... the disk could read 10 bytes incrementing a byte at a time or it could be required to skip 20 gigs to read each byte(this is an over-simplification). This will increase wear and tear on the moving parts as well as extra heat. So, extreme fragmentation will likely decrease the life of your disks.
As above, your hypothetical in no way supports your claim that fragmentation has an effect on data integrity. All you showed with your hypothetical is that if your unstated assumptions about load and usage patterns impacting hardware failure rates are true, then fragmentation may have an effect on hardware failure rates.
The bottom line is that fragmentation has nothing to do with data integrity. The two issues are completely orthoganal.
So much better than the ext3 version of defrag
Actually flash based SSD's do have a seek time though they are faster than harddisks, eg. OCZ Core SSD has random seek time of 0.4ms where as WD Raptor harddisk has about 7ms.
- Raynet --> .
Sun is ahead of the curve (yet again frankly) and is introducing optimizations that allow Solaris/ZFS to take decisions to optimize the use of SSDs
IANAL but write like a drunk one.
I will assume you are under 25 years old and that your computing teachers are rubbish.
Those of use slightly older remember that SunOS became popular as a desktop operating system, very popular in the academic circles and in certain industries that required serious processing power in a compact package.
Later Sun machines like the Sparc Classic, Sparc LC, Sparc Station 10 and the Ultra 10 became classics of the computer industry that you would spot in trading desks in banks, as workstations in oil companies and as CAD stations in Formula 1 teams, amongst multitude of many other roles.
Sun developed a graphical environment called OpenView and was the first UNIX commercial company to put Gnome as the default on offering for their desktop machines.
There are many more points that disprove your silly assertion, but I'll leave it at that.
IANAL but write like a drunk one.
People solving problems that cost lots of money don't care about how many USB devices you can connect to a machine.
IANAL but write like a drunk one.
Agreed; synthetic benchmarks don't tell the whole story. It's become fashionable to dismiss defrag as archaic and a waste of time.
I am only a low level trainee worker bee so I don't deal directly with production servers, but according to the Big Bees they were able to cut down backup times by about 25-40% on some servers after installing Diskeeper Enterprise Server and defragging the disks. That software is not cheap(!) but the big Bees seem to love it for what it does.
Defrag is not dead. Not just yet.
Fragmentation causes more needle shifts than regular burst read/writes.
Of course most people don't think "stable" means "kills your drive slower" for an OS. Especially not in a discussion about SSD's.
and that's the only official way to defrag HFS+. Of course there are 3rd party tools to do it (iDefrag) even though the Mac users are under the impression that HFS+ does not need be de-fragmented.
Of course anyone who knows how file systems work will know that is not true.
OS X has some built in strategies to defrag files on the fly, but they don't apply to files larger than 20 MB or writable files or files that have less than 8 fragments, so obviously it is not sufficient.
As the island of our knowledge grows, so does the shore of our ignorance.
I've been using SSD's since Windows 2000 came out,
Via the CENATEK "RocketDrive" PCI 2.2 133mb/sec bus, 4gb of PC-133 SDRAM!
----
FOR "END-USER" TYPE USE PATTERNS ON FILES THAT ARE BOTH OF READ/WRITE NATURE & CONSTANTLY ONGOING TASKS:
1gb Partition #1 = pagefile.sys placement...
1gb Partition #2 has folders on it for:
1.) %temp% & %tmp% ops to take place on it, via the environment "in memory .ini file" every app gets
2.) Webbrowser caches
3.) Logging by the OS (event logs, easily moveable via registry edits/reg file merges) + apps' logs
----
Does it make a diff., even for "end-user use patterns" of that nature?
Sure, those things which go on, ALL THE TIME mind you, of BOTH read/write nature no less (where FLASH based SSD's take a beating is writes, wear levelling notwithstanding, that's just for longevity more than performance) get F A S T E R...
Yes - you notice it.
Smaller, but, "long-term" performance gains (not huge, but present nevertheless), of less fragmentation on your main OS & Programs bearing HDD (1000x slower than SSD) from fragmentation webpage caches, logs, & paging files cause in themselves AND other files...
Larger noticeable improvements to overall system performance, by it NOT being burdened w/ constant ongoing head movements that impede program + data loads mind you (that are incurred from paging, logging, & temporary operations by the OS & apps!)
Fast on my type of system for that kind of data (most of which I could care less if I lose or not mind you, even though it has a UPS backing it), because it's smallish files I move for webpage caching & SSD's are good for this, but also pagefiles, & %temp% ops, all of which are READ/WRITE I-O... because I don't use a FLASH based SSD (weaker on writes typically/historically @ least).
Especially on a rig like mine, w/ only 512mb DDR-400 RAM, now running Windows Server 2003 SP#2 as a workstation (default install), tuned & trimmed "to-the-max" has been running setup like that since 2003, solid as a rock stable, & fast.
(Thus - I use SSD's to speed-up the SLOWEST part of any system - it's HDDs!)
Bottom-line here? Well:
WANT MORE SPEED? SPEED UP THE SLOWEST THING ANY PC CONTENDS WITH - DISKBOUND I/O.."
----
FOR COMMERCIAL/ENTERPRISE-CLASS/MULTIUSER/TRANSACTION-BASED ENVIRONS (take your pick)??
Take a read:
http://techreport.com/articles.x/9312/7
PERTINENT EXCERPT:
"Wow. Seriously.
The i-RAM is in another league in IOMeter, offering transaction rates that are an order of magnitude higher than its closest competition. It doesn't take long for the i-RAM to get revved up, either. The card hits its peak transaction rate with just two simultaneous I/O requests."
APK
P.S.=> Nuff said... apk
Nope. Why do you ask?
Git off my lawn! I run wax cylinder backed punch card RAID6, you insensitive clod!
Only an order of magnitude? Uhhh, I prefer true SSDs - a RAMdisk. Thanks anyway.
I know tobacco is bad for you, so I smoke weed with crack.
It must be humiliating to have to schedule regular downtime just to amortise the cost of Windows' sheer stupidity.
You seem never had to deal with usual corporate IT.
They take special pride for their own incompetence. After all, bigger impact their actions have, more important they feel themselves in company.
And management compatible "sweet talk" seems to be first thing they are taught at the "IT inCompetence Courses" so they always have ready nice plain explanation to management, sometimes with PowerPoint slides.
It's kind of funny, because we also have bunch of Unix and hordes of Linux boxes. All Windows servers go down periodically, mostly scheduled downtimes, sometimes unscheduled. Unix servers mostly rebooted due to heavy abuse of resources and fact that our IT still doesn't know how to kill a process. Linux servers ... in past year they rebooted two of them to replace faulty RAM modules.
All hope abandon ye who enter here.
It only has to happen once to make an impression, at least if your backup on your laptop (a different machine) gets lost when the power surge that preceeded the power outage fries the HD.
Your ad here. Ask me how!