Grandparent, and the whole chain, are *way* off base here. SSD LBA remap table fragmentation has absolutely nothing to do with file system fragmentation. ext3 will cause just as much of a slowdown as NTFS would. I share the same appreciation for Linux as everyone else around here, but in this case it is in no way the magic bullet we might like it to be.
Intel has solved theirs about 95%, but they are helped by their write speeds being limited to 80 MB/sec. With the new firmware, it is *very* hard to get an X25-E to drop below its rated write speed.
OCZ has not yet solved it. They currently rely on TRIM, and in my testing that alone is not sufficient to correct the fragmentation buildup. IOPS falls off in this condition as well.
20% is too little. I've seen drives, even SLC drives, drop by more than 50%. Only some drives bounce back properly. Others rely on TRIM to clean up their fragmentation mess.
All flash drives *do not* have the issue. What really happens is your small write IOPS will be on the low side and your sequential writes will always be full speed *unless* you implement some form of write combining. The write combining cheats a bit by taking small random writes and writing them in a more sequential fashion to the flash itself.
The catch is that when you come past that now fragmented area, the controller has to play musical chairs with the data while trying to service the write originally requested by the OS. End result - slower write speed.
Some well behaved controllers (Intel, Samsung) will take a little extra time to defragment the block while it's servicing the sequential write. Optimized controllers (Intel M series) will now rarely fall below their advertised write speed of 80 MB/sec.
Other more immature controllers leave the data fragmented and simply move the whole block elsewhere. This results in a compounded fragmentation, which can eventually drop write speed to 1/3 to 1/5 of it's write speed when new.
They learned the hard way that you should not build a reactor so small that it requires *manual* withdrawal of control rods. By manual I mean a guy hunkered over the core with his hands on the rod itself. End result: said man impaled by said rod - to the ceiling.
For write combining SSD's, remapping occurs *within* blocks. You don't necessarily need to write huge sequential files to an X25 to correct this, writes only have to match the block size.
OCZ is getting there, but they are trying to keep up with the IOPS of Intel's 10 channel controller with their own 4 channel controller. Something has to give. In this case it is their Vertex fragments fairly quickly and won't come back on its own. It *requires* a TRIM utility to be run on it to restore full write speed.
It's a tradeoff. With the new firmware, the X25 goes *slightly* slower with random writes of random sizes, but this is because it is more aggressively defragmenting itself on the fly. It does this to keep closer to its full write speed. The Vertex does not have the luxury of that overhead with only 4 channels.
With 4k writes it was likely writing at a lower speed right off the bat and kept that lower value. Try it at higher queue depths and you will get increased parallelism within the drive. Then you should see higher initial speed that will fall off as the drive fragments.
It's not how much you write to it, it's how you write to it. Write a bunch of small files to random locations and it will fragment, dropping subsequent write speed. There is a pic of the effect on the last page of my article:
What you're thinking of is essentially the response time of the flash itself. Most drives appear to be engineered to assume the flash is at its end of life and keep their timings to that level. No drive I have tested has slowed for this particular reason.
The problem Intel fixed is not the same thing you're thinking of. Anand's methodology was flawed, in that he was writing the OS back to the drive in sector-by-sector mode, which is effectively a large sequential write. This acts to heal drives that write combine and is not in line with how that OS would have got there in reality. The subsequent writes he did accomplished nothing more than seeing how far that particular drive could fill the 'holes' in the partition (i.e. how fast it can perform small random writes).
d) When their datasheet also says, "Should the host system attempt to exceed 20 GB writes per day by a large margin for an extended period, the drive will enable the endurance management feature to adjust write performance, this feature enables the device to have, at a minimum, a five year useful life"...
You make many good points, but I should point out that the quoted feature never made it into the retail product. When conducting the testing for my article, I wrote several TB per day to my X25-M and experienced no drop in write speeds - provided those writes were more sequential than random.
Constantly hitting an X25-M with small writes will net you at most an average 50% drop in sequential write speeds. The drive will eventually reach an equilibrium based on the mix of write sizes you hit it with. The M has larger flash blocks and has to track a relatively higher level of write combining, and it is possible for it to get 'stuck' at some very low write speeds (see the article for more detail). This is a unique condition that Intel is currently looking into.
Getting back to the quoted section: The write speed slow downs seen in my testing resulted only from the ratio of small/large files written and had nothing to do with the rate / volume of data written over any particular time period.
In response to the post, I would recommend either an MLC SSD with very high IOPS (Intel), one of the newer MLC SSD's with on-board SRAM cache (OCZ Vertex / 3rd gen Samsung), or for the highest overall read/write throughput, a pair of SLC SSD's in RAID-0. For SLC, the Intel drive is very good, but there are much cheaper alternatives out there (i.e. G.Skill rebranded Samsung SLC). Note that the X25-E uses write combining, and will take that same 50% worst-case sequential write hit. Other SLC units are not as fast at small writes (no combining), but their performance remains rock steady regardless of what you hit them with.
There was no difference in how long it took to fragment. If we wrote a nasty enough mix of smaller file sizes to the drive, performance would drop right at the point where all flash was written to at least once (i.e. just over the 80GB mark).
After running HDDErase on the drive, it went the same *exact* 80 MB/sec write speed each and every time. Additionally, running successive software secure erasures (writing 0's across all 80GB) showed 0 drop in speed even after 10 passes.
In testing several different SSD brands / types, I have yet to see a slowdown that would suggest block erasures take longer over time. I suspect the block erase timing is based on flash that is at or near its end of life.
Nothing is lost, it just goes slower. Due to the fragmentation caused by write combining, the drive has to shuffle more data around when you write to those same areas later on. The flash is still being written at full speed as it copies the data internally, but the drive can only accept new data at the reduced speeds seen.
We're working with Intel to help them reproduce the more significant issue we saw.
You are incorrectly assuming that the unsprung weight is responsible for the power generation. The generation happens as the surface changes are leveraged against the body of the car, meaning your weight figure would need to be at least the vehicle body mass being supported by that wheel. In this case power would be generated even with an ideal 0 lbs of unsprung weight.
I've bought a total of 10 1TB WD GP drives. 6 when they first came out, 2 a month later, and 2 a month ago. None have been DOA, and not so much as a hiccup so far.
Just a data point for ya.
Note: the same cannot be said for the 400GBish 7200 RPM drives of theirs from a few years back. Those things were a head crash waiting to happen.
I'd venture a guess that WD gets a bad rap from a bad start into the 'RPM race'. I built an array when the 400GB drives came out. I had even sprung for the RE2 server class drives. Over the course of 2 years I had to RMA 5 out of 6 of them. The problem was they ran extremely hot even with above average cooling. Their lineup around that period must have been built on a bad thermal design.
I've since switched to the new GP (Green Power) drives and not had a single problem in a full year of use, using 10 of them. They run *significantly* cooler (hottest drive is 100F and that is with weak airflow) and appear to be much more reliable as a result of it.
Also on WD: I've raid-0 striped a pair of nearly every generation of Raptor they made and never had so much as a hiccup.
So yeah, they had a rough spot, but I'd say they would be higher on the list if you go on their current product lines.
Say you were ripping 2 DVD's in parallel at max drive speed, and had 2x DV camcorders connected and ripping from those as well. You will still only be ripping at a fraction of what the MLC SSD is capable of.
If you were doing that same thing on a HDD, you would get nowhere near max throughput as the heads would be seeking across 4 different file writes constantly. A velociraptor could likely handle that sort of load but you would be treading the line.
Any sort of heavy read at that time would thrash the crap out of the HDD, slowing writes significantly, and no doubt dropping frames on the DV streams, while the SSD would just laugh and carry along with no issue. The SSD does not have to seek all over the place, so you get closer to the throughput figures regardless of how many parallel streams are being written to the device (to a point obviously, but it would still smoke the HDD).
The review is slashdotted at the moment so I can't RTFM, but...
If a velociraptor beat an SSD in boot time, well, something is wrong with their test, or perhaps the bios was waiting on the SSD to initialize (entirely possible based on the added intelligence on their controller chipset). I just went from an SLC SSD to a velociraptor and the difference is painful. Boot time is slower. The system is just 'laggier'.
You can't judge the differences between SSD and HDD from charts and graphs on review sites. Reserve judgement until you have actually sat down at an SSD driven system. It is on par with the difficulty we all used to have explaining the difference in 'feel' between a single and dual cpu system back before they were mainstream.
Seek time dropping to 0.1 msec changes the entire equation. Events that would usually thrash your hard drive for several seconds happen *instantly* on an SSD. When you boot into the windows desktop, everything acts as if it was already cached, and does so even if other drive intensive tasks are already running.
Remember the reason everyone puts their swapfile on a second hard drive? SSD's nullify that reasoning.
A velociraptor beating it on write speed is irrelevant - a typical windows system will be reading from the drive occasionally during the writes. An HDD will drop to significantly below its max 'straight line' speed when you throw in a bit of random access (or fragmentation). End result: the SSD will still roast it in practical use.
I only switched to the raptor as a stopgap so I can sell off my current SSD in preparation to get this Intel unit. After seeing the change in speed / responsiveness in practical usage, the new SSD can't get here fast enough...
- They 'cheated' on ATTO, only configuring it to start at 8k. Last I checked, default sector size is 512b. Regular day-to-day apps, such as Outlook, perform random sector-level access to the PST when downloading mail.
- If you're going to do an SSD roundup, how about at lest grabbing a few drives off of the SSD top 10. Specifically, Memoright (#1 on that list) makes an SLC drive that competes with the other SLC drives on price, yet outperforms them all: http://www.storagesearch.com/ssd-top10.html
- Disclaimer: I own a Memoright drive. I don't claim to be a fanboy, I just did my research beforehand (along with trying out a few other drives), and found the best thing going at the time.
- The Intel drives, expected to come out this month, are likely to bury everything on that review.
Slashdot Mirror
Grandparent, and the whole chain, are *way* off base here. SSD LBA remap table fragmentation has absolutely nothing to do with file system fragmentation. ext3 will cause just as much of a slowdown as NTFS would. I share the same appreciation for Linux as everyone else around here, but in this case it is in no way the magic bullet we might like it to be.
Allyn Malventano
Storage Editor, PC Perspective
Intel has solved theirs about 95%, but they are helped by their write speeds being limited to 80 MB/sec. With the new firmware, it is *very* hard to get an X25-E to drop below its rated write speed.
http://www.pcper.com/article.php?aid=691&type=expert&pid=5
OCZ has not yet solved it. They currently rely on TRIM, and in my testing that alone is not sufficient to correct the fragmentation buildup. IOPS falls off in this condition as well.
Allyn Malventano
Storage Editor, PC Perspective
20% is too little. I've seen drives, even SLC drives, drop by more than 50%. Only some drives bounce back properly. Others rely on TRIM to clean up their fragmentation mess.
A more important note is that some initial TRIM implementations have been poorly implemented, resulting in severe data corruption and loss:
http://www.ocztechnologyforum.com/forum/showthread.php?t=54770
I posted elsewhere regarding the fragmentation issue here:
http://hardware.slashdot.org/comments.pl?sid=1227271&cid=27883769
Allyn Malventano
Storage Editor, PC Perspective
All flash drives *do not* have the issue. What really happens is your small write IOPS will be on the low side and your sequential writes will always be full speed *unless* you implement some form of write combining. The write combining cheats a bit by taking small random writes and writing them in a more sequential fashion to the flash itself.
The catch is that when you come past that now fragmented area, the controller has to play musical chairs with the data while trying to service the write originally requested by the OS. End result - slower write speed.
Some well behaved controllers (Intel, Samsung) will take a little extra time to defragment the block while it's servicing the sequential write. Optimized controllers (Intel M series) will now rarely fall below their advertised write speed of 80 MB/sec.
Other more immature controllers leave the data fragmented and simply move the whole block elsewhere. This results in a compounded fragmentation, which can eventually drop write speed to 1/3 to 1/5 of it's write speed when new.
I authored the original articles on the matter:
http://www.pcper.com/article.php?aid=669
http://www.pcper.com/article.php?aid=691
Allyn Malventano
Storage Editor, PC Perspective
There was SL-1:
http://www.radiationworks.com/sl1reactor.htm
They learned the hard way that you should not build a reactor so small that it requires *manual* withdrawal of control rods. By manual I mean a guy hunkered over the core with his hands on the rod itself. End result: said man impaled by said rod - to the ceiling.
For write combining SSD's, remapping occurs *within* blocks. You don't necessarily need to write huge sequential files to an X25 to correct this, writes only have to match the block size.
I touched on this at the end of this page:
http://www.pcper.com/article.php?aid=691&type=expert&pid=8
OCZ is getting there, but they are trying to keep up with the IOPS of Intel's 10 channel controller with their own 4 channel controller. Something has to give. In this case it is their Vertex fragments fairly quickly and won't come back on its own. It *requires* a TRIM utility to be run on it to restore full write speed.
It's a tradeoff. With the new firmware, the X25 goes *slightly* slower with random writes of random sizes, but this is because it is more aggressively defragmenting itself on the fly. It does this to keep closer to its full write speed. The Vertex does not have the luxury of that overhead with only 4 channels.
With 4k writes it was likely writing at a lower speed right off the bat and kept that lower value. Try it at higher queue depths and you will get increased parallelism within the drive. Then you should see higher initial speed that will fall off as the drive fragments.
It's not how much you write to it, it's how you write to it. Write a bunch of small files to random locations and it will fragment, dropping subsequent write speed. There is a pic of the effect on the last page of my article:
http://www.pcper.com/article.php?aid=691&type=expert&pid=10
What you're thinking of is essentially the response time of the flash itself. Most drives appear to be engineered to assume the flash is at its end of life and keep their timings to that level. No drive I have tested has slowed for this particular reason.
The problem Intel fixed is not the same thing you're thinking of. Anand's methodology was flawed, in that he was writing the OS back to the drive in sector-by-sector mode, which is effectively a large sequential write. This acts to heal drives that write combine and is not in line with how that OS would have got there in reality. The subsequent writes he did accomplished nothing more than seeing how far that particular drive could fill the 'holes' in the partition (i.e. how fast it can perform small random writes).
Guys,
You're welcome :).
Kidding aside, it was great to have a manufacturer as large as Intel work with us and have something good come from it.
Allyn Malventano
Storage Editor, PC Perspective
d) When their datasheet also says, "Should the host system attempt to exceed 20 GB writes per day by a large margin for an extended period, the drive will enable the endurance management feature to adjust write performance, this feature enables the device to have, at a minimum, a five year useful life"...
You make many good points, but I should point out that the quoted feature never made it into the retail product. When conducting the testing for my article, I wrote several TB per day to my X25-M and experienced no drop in write speeds - provided those writes were more sequential than random.
Constantly hitting an X25-M with small writes will net you at most an average 50% drop in sequential write speeds. The drive will eventually reach an equilibrium based on the mix of write sizes you hit it with. The M has larger flash blocks and has to track a relatively higher level of write combining, and it is possible for it to get 'stuck' at some very low write speeds (see the article for more detail). This is a unique condition that Intel is currently looking into.
Getting back to the quoted section: The write speed slow downs seen in my testing resulted only from the ratio of small/large files written and had nothing to do with the rate / volume of data written over any particular time period.
Article in question:
http://hardware.slashdot.org/article.pl?sid=09/02/13/2337258
http://www.pcper.com/article.php?aid=669
In response to the post, I would recommend either an MLC SSD with very high IOPS (Intel), one of the newer MLC SSD's with on-board SRAM cache (OCZ Vertex / 3rd gen Samsung), or for the highest overall read/write throughput, a pair of SLC SSD's in RAID-0. For SLC, the Intel drive is very good, but there are much cheaper alternatives out there (i.e. G.Skill rebranded Samsung SLC). Note that the X25-E uses write combining, and will take that same 50% worst-case sequential write hit. Other SLC units are not as fast at small writes (no combining), but their performance remains rock steady regardless of what you hit them with.
The G.Skill SLC drive I mentioned:
http://www.newegg.com/Product/Product.aspx?Item=N82E16820231186
Regards,
Allyn Malventano
Storage Editor, PCPer.com
There was no difference in how long it took to fragment. If we wrote a nasty enough mix of smaller file sizes to the drive, performance would drop right at the point where all flash was written to at least once (i.e. just over the 80GB mark).
After running HDDErase on the drive, it went the same *exact* 80 MB/sec write speed each and every time. Additionally, running successive software secure erasures (writing 0's across all 80GB) showed 0 drop in speed even after 10 passes.
In testing several different SSD brands / types, I have yet to see a slowdown that would suggest block erasures take longer over time. I suspect the block erase timing is based on flash that is at or near its end of life.
Al Malventano
PCPer Editor
Nothing is lost, it just goes slower. Due to the fragmentation caused by write combining, the drive has to shuffle more data around when you write to those same areas later on. The flash is still being written at full speed as it copies the data internally, but the drive can only accept new data at the reduced speeds seen.
We're working with Intel to help them reproduce the more significant issue we saw.
Allyn Malventano
PCPer Editor
And small random jiggles are hard to capture as electricity.
Since this system is inductive, those small random jiggles are the *easiest* to capture as electricity.
You are incorrectly assuming that the unsprung weight is responsible for the power generation. The generation happens as the surface changes are leveraged against the body of the car, meaning your weight figure would need to be at least the vehicle body mass being supported by that wheel. In this case power would be generated even with an ideal 0 lbs of unsprung weight.
It is the *sprung* weight that matters here.
None of the online review sites ever mention input lag and on some monitors, it's a huge problem.
http://www.behardware.com/articles/632-1/lcds-images-delayed-compared-to-crts-yes.html
http://www.behardware.com/articles/689-2/22-inch-lcd-monitors-the-3rd-wave.html
A review of the ION platform with a dual core Atom 330 is here:
http://www.pcper.com/article.php?aid=663
Lets just hope they don't bring in these guys to start the things up...
http://www.radiationworks.com/sl1reactor.htm
I've bought a total of 10 1TB WD GP drives. 6 when they first came out, 2 a month later, and 2 a month ago. None have been DOA, and not so much as a hiccup so far.
Just a data point for ya.
Note: the same cannot be said for the 400GBish 7200 RPM drives of theirs from a few years back. Those things were a head crash waiting to happen.
I'd venture a guess that WD gets a bad rap from a bad start into the 'RPM race'. I built an array when the 400GB drives came out. I had even sprung for the RE2 server class drives. Over the course of 2 years I had to RMA 5 out of 6 of them. The problem was they ran extremely hot even with above average cooling. Their lineup around that period must have been built on a bad thermal design.
I've since switched to the new GP (Green Power) drives and not had a single problem in a full year of use, using 10 of them. They run *significantly* cooler (hottest drive is 100F and that is with weak airflow) and appear to be much more reliable as a result of it.
Also on WD: I've raid-0 striped a pair of nearly every generation of Raptor they made and never had so much as a hiccup.
So yeah, they had a rough spot, but I'd say they would be higher on the list if you go on their current product lines.
Say you were ripping 2 DVD's in parallel at max drive speed, and had 2x DV camcorders connected and ripping from those as well. You will still only be ripping at a fraction of what the MLC SSD is capable of.
If you were doing that same thing on a HDD, you would get nowhere near max throughput as the heads would be seeking across 4 different file writes constantly. A velociraptor could likely handle that sort of load but you would be treading the line.
Any sort of heavy read at that time would thrash the crap out of the HDD, slowing writes significantly, and no doubt dropping frames on the DV streams, while the SSD would just laugh and carry along with no issue. The SSD does not have to seek all over the place, so you get closer to the throughput figures regardless of how many parallel streams are being written to the device (to a point obviously, but it would still smoke the HDD).
How many times did they reboot it?
[/thewebsiteisdown]
The review is slashdotted at the moment so I can't RTFM, but...
If a velociraptor beat an SSD in boot time, well, something is wrong with their test, or perhaps the bios was waiting on the SSD to initialize (entirely possible based on the added intelligence on their controller chipset). I just went from an SLC SSD to a velociraptor and the difference is painful. Boot time is slower. The system is just 'laggier'.
You can't judge the differences between SSD and HDD from charts and graphs on review sites. Reserve judgement until you have actually sat down at an SSD driven system. It is on par with the difficulty we all used to have explaining the difference in 'feel' between a single and dual cpu system back before they were mainstream.
Seek time dropping to 0.1 msec changes the entire equation. Events that would usually thrash your hard drive for several seconds happen *instantly* on an SSD. When you boot into the windows desktop, everything acts as if it was already cached, and does so even if other drive intensive tasks are already running.
Remember the reason everyone puts their swapfile on a second hard drive? SSD's nullify that reasoning.
A velociraptor beating it on write speed is irrelevant - a typical windows system will be reading from the drive occasionally during the writes. An HDD will drop to significantly below its max 'straight line' speed when you throw in a bit of random access (or fragmentation). End result: the SSD will still roast it in practical use.
I only switched to the raptor as a stopgap so I can sell off my current SSD in preparation to get this Intel unit. After seeing the change in speed / responsiveness in practical usage, the new SSD can't get here fast enough...
These guys are idiots. A few points:
- They 'cheated' on ATTO, only configuring it to start at 8k. Last I checked, default sector size is 512b. Regular day-to-day apps, such as Outlook, perform random sector-level access to the PST when downloading mail.
- If you're going to do an SSD roundup, how about at lest grabbing a few drives off of the SSD top 10. Specifically, Memoright (#1 on that list) makes an SLC drive that competes with the other SLC drives on price, yet outperforms them all: http://www.storagesearch.com/ssd-top10.html
- Disclaimer: I own a Memoright drive. I don't claim to be a fanboy, I just did my research beforehand (along with trying out a few other drives), and found the best thing going at the time.
- The Intel drives, expected to come out this month, are likely to bury everything on that review.