Slashdot Mirror
Garbage Collection Algorithms Coming For SSDs
MojoKid writes "A common concern with the current crop of Solid State Drives is the performance penalty associated with block-rewriting. Flash memory is comprised
of cells that usually contain 4KB pages that are arranged in blocks of 512KB. When a cell is unused, data can be written to it relatively quickly. But if a cell already contains some data, even if it fills only a single page in the block, the entire block must be re-written. This means that whatever data is already present in the block must be read, then it must be combined or replaced, and the entire block is then re-written. This process takes much longer than simply writing data straight to an empty block. This isn't a concern on fresh, new SSDs, but over time, as files are written, moved, deleted, or replaced, many blocks are a left holding what is essentially orphaned or garbage data, and their long-term performance degrades because of it. To mitigate this problem, virtually all SSD manufacturers have incorporated, or soon will incorporate, garbage collection schemes into their SSD firmware which actively seek out and remove the garbage data. OCZ, in combination with Indilinx, is poised to release new firmware for their entire line-up of Vertex Series SSDs that performs active garbage collection while the drives are idle, in order to restore performance to like-new condition, even on a severely 'dirtied' drive."
FP? ....
A weakness was found in first generation drives, the second generation drives fixed it.
Film at 11.
No sig today...
It seems like this function should be performed in the operating system. The firmware should just make available the info and commands an OS needs to do the right thing.
"Garbage collection" has already quite different usage in CS. And while what has to be done to those SSDs isn't technically the same as defragmentation on HDDs, it is still "performing drive maintenance to combat performance-degrading results of prolonged usage, deletion of files".
One that hath name thou can not otter
First cleanup result will be C:\windows
- -
Attention all personnel: Funny leak detected.
If you have an app that needs SSD, when will the drives ever be idle ?
I want to delete my account but Slashdot doesn't allow it.
So what does this do when forensics are being done on one of these drives? Is the firmware just doing a better job of marking a dirty block available or do the dirty blocks have to be zeroed at some point. Even if the blocks are just marked will they output zeros if 'dd'ed by an OS?
This will significantly increase writes. I'm sure it's still worth it, but we ought to know what kind of effect this will have on the time before one hits max writes on the flash device.
I don't want my porn garbage collected thank you very much. Who died and made you king of deciding what's garbage.
These posts express my own personal views, not those of my employer
Wouldn't the drive benefit from a real understanding of the filesystem for this sort of thing? If it knew a sector was unallocated on a filesystem level, it would know that sectors were empty/unneeded, even if they had been written to nicely. Or should computers now have a way of tagging a sector as "empty" on the drive?
Either way, it looks like an OS interaction would be very helpful here.
Or are modern systems already doing this, and I'm just behind the times?
The Garbage collector restores performance of the drive. Nothing comes free, so a question - at what cost?
In theory there is no difference between theory and practice. In practice there is. - Yogi Berra
The new drive deleted all my %INSERT_POPSTAR% songs!
Captcha is "tragedy", how fitting...
Comment removed based on user account deletion
Isn't this just going to degrade the life of the drive more by moving data around to clean up cells/blocks?
void clean(std::vector<Jew*>& jews)
{
for(std::vector<Jew*>::iterator i = jews.begin(); i != jews.end(); i++)
{
if((*i) != NULL)
{
(*i)->PutInOven();
delete (*i);
(*i) = NULL;
}
}
jews.clear();
}
Garbage collection should collect itself and throw itself into the trash.
Then again, we can pair up a slow write media, with a slow programming language, a perfect pair.
Close the blinds. Put your dresses on. Apply lipstick.
Read about Java gc. You'll find the section on generational gc interesting.
Not on your life! Unless I've got Aslan to help me, of course.
No, not MILFs.
http://www.linux-mag.com/cache/7345/1.html
This GC stuff is only needed as long as the FS uses small blocks. File systems should be able to use arbitrarily sized blocks.
Hopefully btrfs will be able to use large blocks efficiently too.
thegodmovie.com - watch it
Is this what dreams are for ?
I feel like my brain goes 'garbage collecting' every night...
I see OCZ already released some sort of garbage collection tool, but it only works on Windows. Kind of annoying since I bought their "Mac Edition" drive for my MacBook. Hopefully they'll put this in a firmware update, too, and hopefully I won't have to boot DOS on my Mac to update the firmware with a utility that blows over my partition table this time. That was a lot of fun going from version 1.10 to 1.30 firmware.
So when is my DOS Norton Speedisk going to support defragging SSDs?
In the article it says
But if a cell already contains some data--no matter how little, even if it fills only a single page in the block--the entire block must be re-written
Is this correct?
From whatever I read in AnandTech, it looked like we need not rewrite the entire block unless the available data is less than total - (obsolete + valid) data.
Also, the article is light in details. How are they doing the GC? Do they wait till the overall performance of the disk is less than a threshold and rewrite everything in a single stretch or do they rewrite based on local optima? If the former, what sort of algorithms are used (and are the best for it) ?
rajmohan_h@yahoo.com
No, OCZ released wiper, which is a trim tool. Trim and GC are different; in particular, GC requires no tools or OS support.
Next time you write into that block, this operation will be performed anyway. This is why some SSDs have huge delays on writes, because they delay your write until the data merging is done. Also, not every block needs merging anyway, an area of a file that spans 512K (128 pages) is written in one chunk anyway and never needs re-merging.
To be honest, the data retention time on NAND (where the data just drains out like DRAM) is becoming as big a factor as write wear anyway. You're going to have to move the data around a little to make sure it stays valid.
Honestly, I'm pretty spooked about SSDs right now, the only reason I use one is a friend gave it to me for free to test.
http://lkml.org/lkml/2005/8/20/95
That's funny I always thought scuzzy drives needed garbage collection. Ta-dit-boom. Thanks, try the veal.
from the take-out-the-tash dept.
Is "tash" a play on words regarding SSD's, or does the taking-out-the-TRASH department have a job opening for a grammar nazi?
Slashdot 'BSD is [gay-sex-access.com]? decentralized 1nfluence, the 'superior' machine. (7000+1400+700)*4 these rules will SLING you can have left in
Oh Wow, SSDs still have issues. Hey it's new technology, it's still very expensive relative to the technology it replaces, we haven't yet seen how well it holds up long term, and everyone who jumped to it early because it was the new bright and shiny thing should consider being a bit more cautious next time.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
How would this affect camcorders that record to flash memory? I'm interested in getting a camcorder that uses flash memory for its inherent ruggedness and low power consumption. If the memory ages with time is it better not to get the ones with built-in memory that are not easily replaced by simply inserting a new memory stick? Does it mean that the memory at the beginning of the stick will age more, given that it will be overwritten more?
....defrag, anyone?
I also wonder, does this impact battery life in a laptop with these SSDs?
no comment
I have been working closely with OCZ on this new firmware and wanted to clear things up a bit. This new firmware *does not*, *in any way at all*, remove or eliminate orphaned data, deleted files, or anything of the like. It does not reach into the partition $bitmap and figure out what clusters are unused (like newer Samsung firmwares). It does not even use Windows 7 TRIM to purge unused LBA remap table entries upon file deletions.
What it *does* do is re-arrange in-place data that was previously write-combined (i.e. by earlier small random writes taking place). If data was written to every LBA of the drive, then all files were subsequently deleted, all data would remain associated with those LBAs. This actually puts OCZ above most of the pack, because their algorithm restores performance without needing to reclaim unused flash blocks, and does so completely independent of the data / partition type used. This is particularly useful for those concerned with data recovery of deleted files, since the data is never purged or TRIMmed.
Slashdot-specific Translation: This firmware will enable an OCZ Vertex to maintain full speed (~160 MB/sec) sequential writes and good IOPS performance when used under Mac and Linux.
Hardware-nut Translation: This firmware will enable OCZ Vertex to maintain full performance when used in RAID configurations.
I'll have my full evaluation of this firmware up at PC Perspective later today. Once available, it will appear at this link:
http://www.pcper.com/article.php?aid=760
Regards,
Allyn Malventano
Storage Editor, PC Perspective
this sig was brought to you by the letter
I'm afraid you fucked up twice.
One, you bought an OCZ at all. Bad mistake. Avoid OCZ at all costs. They don't seem to care about using end users as alpha and beta testers for their SSDs.
Two, you bought OCZ's 'Mac Edition' product, which by all appearances is a way to try to milk Mac users out of extra money for the same damn product as their normal SSD. It's an extremely cynical ploy. Macs do not need different drives.
I think it ends up being like NCQ. The drive's processor can be much more specialized and can do the processing much more efficiently.
Except that the next generation of file systems will probably be COW (e.g., ZFS, btrfs, BFS's LogFS), so the stuff being written down will be going to a new LBA / sector. In which case it'd be good if there was some kind of SATA / SAS comment to tell the drive to act in a linear fashion, since the OS will be handling the layout anyway.
There's certainly the TRIM command, but if file systems are going to layout data from LBA 0 to 'x' and then back again when they hit the end (almost like a tape), there's no sense doing extra wipes of blocks and shortening the life of the drive if you don't need to.
This is BS ... and has been handled by flash drivers for YEARS!
Why not just use 512 blocks instead of 4K ones?!?! ... Should solve the problem!
"My Name is Ozymandias: Look upon my works, ye mighty, AND DESPAIR..."
http://tech.slashdot.org/comments.pl?sid=1327945&cid=28981391
See my subject-line above, & the url I just posted, where I await your reply... that ought to be good for another large laugh, lol!
"Surely you're suggesting this in jest? - by ion.simon.c (1183967) on Saturday August 08, @12:27AM (#28993767)
No, but YOU must have, what with this statement from you there:
"2) You're talking to APK. He exists to write wall-of-text comments. His depth of knowledge is *really* shallow, so don't expect a good conversation out of him." - by ion.simon.c (1183967) on Thursday August 06, @08:09PM (#28980845)
In regards to that statement from you I just quoted... well, again: See my subject-line above, & the url I put into this reply... see you there, coward.
LOL, ever see the film "The Watchmen"? IF NOT?? Then, do see it...
I simply state that, because... Well - Since you make me laugh, often quite hugely, in your trolling of myself here?? ALL I can think of, is @ the start of that film, when "the Comedian" (you, in this case, since you make me laugh so much), gets his behind handed to him, by Ozymandias...
Because the list I produce there in that url above? Clearly shows I have such works + that I have done fairly well in this science of computing, whereas by way of comparison?? You have nothing like that list to YOUR credit...
(SO - Despite your stating I have a "shallow level of know-how" in this art & science... WELL - just based on the partial list of things I have been fortunate enough to have been graced with, by OTHERS?? Opinions, clearly vary, and, outnumber yours (as well as being from RESPECTED publications, books/magazines/newspapers/forums articles & more, that are FAR MORE CREDIBLE than yourself))
BOTTOM-LINE: Most of all? It's no "crime" to have done well, & to be able to put it out, vs. attackers like yourself, trolls, that have never done anything they can evidence to that effect, that helps others - I do so, to merely defend myself vs. your false allegations directed MY way (& that one above is NOT a "1st" from you either...).
(One would think that someone who has done well, & can evidence that easily, has a deeper level of knowledge in a science than one who clearly, does not... such as yourself. You never seem to learn, so, it's time for "another lesson" to be taught to you, & as usual, you ONLY BRING THIS ON YOURSELF, in attacking myself!)
APK
P.S.=> You came into the url I posted above, implying things about me that are negative & untrue (per your own words quoted above NO LESS)... now, it's "time to face the music" there, & "eat your words", ONCE AGAIN (what is this now? The 2nd or 3rd time you have tried this on me, always to your own dismay no less?)
Have some "intestinal fortitude", & back up your b.s. there then - so, once more, per my subject-line above?
Why are you running from that reply of mine to you, complete with the quote of your own lame accusations/implications you stated in regards to myself?? Perhaps because my reply will show you for what you are???
I think, no rather KNOW, so... &, so will anyone else reading there! apk
An entire article and posts about garbage files and not one person mentions Ellingston Mineral and it's Gibson computers....I am dissapoint.
Nothing like a crazy, off-topic, poorly written wall-of-text comment with scores of unnecessary bold and caps to get your point across.
especially when the point is that you don't exist to write pointless wall-of-text-comments
bravo, sir
:x
You'd need a separate memory to keep a table of pointers, just like an object table in some OO language implementations. This way, you wouldn't need Garbage Collection so much as just plain old compaction. The extra level of indirection wouldn't be so bad, because it's Flash -- random low-latency access is what it's good at.
(The separate memory would have a granularity of just 64 bits, so rewriting just one pointer will have less overhead.)
"Nothing like a crazy, off-topic, poorly written wall-of-text comment with scores of unnecessary bold and caps to get your point across." - by Random Destruction (866027) on Saturday August 08, @10:51AM (#28995887) Homepage
Ahem: "Beg to Differ" - There is nothing "crazy" about facts that disprove b.s. in trolling attacks on myself (Especially facts that easily & clearly disprove the quote of Ion.SIMIAN.c's own frothing b.s. that came spewing out of his pie-hole - which strangely resemble your reply here (thus, you are nothing more than another of his multiple "sock puppet" accounts ion.SIMIAN.c uses to bother others in his trolling others on this website)).
By the by: GOOD CHOICE OF A NAME FOR THIS "SOCK PUPPET" ACCOUNT OF YOURS, ion.SIMIAN.c - as you always bring your own destruction upon yourself, so it makes sense!
And, no - it's NOT "ok" for ion.SIMIAN.c to attack & troll others, without some "remuneration"... so, let him, defend himself (not via sock puppet either, as we all know YOUR tactics on this account, ion.SIMIAN.c ...)
----
"especially when the point is that you don't exist to write pointless wall-of-text-comments" - by Random Destruction (866027) on Saturday August 08, @10:51AM (#28995887) Homepage
Anyone is free to read my last reply, inclusive of ion.SIMIAN.c's trolling attack on myself, & my rebuttal reply which is loaded with easily verified concrete visible facts, which clearly disprove his trolling b.s., easily... "TOO easy" - & I will allow others (no, not "sock puppets", like you) to judge for themselves.
(Plus, I think it is BEST that ion.SIMIAN.c defend himself... instead of "you", sock-puppet, and, in the very thread in question where he attacked me!)
APK
P.S.=>
"bravo, sir" - by Random Destruction (866027) on Saturday August 08, @10:51AM (#28995887) Homepage
Thank you, too bad I cannot say the same to you "Random Destruction" (another one of Ion.SIMIAN.c's multiple 'sock puppet' accounts he uses to troll others, clearly)... apk
If I expect the drive to last 5 years I can do 5 rewrites a day and still come in under 10,000. Or another way of thinking of that is I can rewrite 1% of the drive every 5 minutes and be fine. So as long as the drive is rotation materials around (which it didn't look they were) caching should be fine.
Why assume, when you can measure? ;)
With smartmontools, I can read two values: The Average Erase Count and the Maximum Erase Count. http://www.transcend.de/support/dlcenter/datasheet/SSD25-JMI%20Datasheet%20v1.06.pdf (page 23).
According to the numbers, my MLC SSD would have lasted ca. two years with my usage pattern. But I replaced with a conventional hard disk, because I needed the space.