Slashdot Mirror
Ubuntu Wants To Enable SSD TRIM By Default
jones_supa writes "During the first day of the latest virtual Ubuntu Developer Summit, Canonical developers finally plotted out the enabling of TRIM/DISCARD support by default for solid-state drives on Ubuntu 14.04. Ubuntu developers aren't looking to enable discard at the file-system level since it can slow down delete operations, so instead they're wanting to have their own cron job that routinely runs fstrim for TRIMing the system. In the past there has been talk about the TRIM implementation being unoptimized in the kernel. Around when Linux 3.0 was released, OpenSUSE noted that the kernel performs TRIM to a single range, instead of vectorized list of TRIM ranges, which is what the specification calls for. In some scenarios this results in lowered performance."
Well it's about time.
I'm still waiting for for firefox or chrome to make themselves SSD friendly. I know we all have RAMdisk, but I swear, after the OS, web browsers seem to generate the next highest number of 'writes'.
TRIM makes your new flash toys go weeeeeeeee, instead of them going only wee
Windows 7 incorporated TRIM support for SSDs back in 2009. I know the Linux kernel can do it with the right mount options and has been able to for some time, but after a while you just assume distros are setting things automatically as expected (there's very few situations where TRIM is a bad idea, particularly on a desktop-focused distro like Ubuntu).
There's a reason I still feel like using a poor-man's system when using Linux on the desktop. They just don't think hard enough about automating stuff. Heck, Ubuntu (and no other distro I believe) doesn't enable Wake-on-lan when you shutdown, whereas Windows 7 and onwards does. This is something you have to script in yourself. Why the fuck aren't distros doing things you can reliably expect in commercials operating systems!?!
I swear, after the OS, web browsers seem to generate the next highest number of 'writes'.
I'd bet a lot of these writes are for caching received HTTP response bodies to disk. Otherwise, desktop browsers in low-memory environments would have to act like Firefox for Android and Chrome for Android. When I open multiple pages in tabs in these mobile browsers, they tend to discard entire pages as soon as I switch to another tab and reload them when I switch back. This interferes with my common use case of opening multiple pages in tabs and then reading them while offline and riding transit. Firefox for X11/Linux can keep pages open on a Dell Inspiron mini 1012 laptop with 1 GB of RAM, but Firefox for Android can't on a first-generation ASUS Nexus 7 tablet with the same amount of RAM. I guess the difference comes from two differences in the environment: swapping is more acceptable on X11/Linux than on Android, and desktop browsers are more likely to keep things in disk cache than mobile browsers.
More like Linux doesn't follow the best practice recommendations of the standard when it wouldn't be all that hard to do so.
Solid-state drives (SSDs) are an alternative to hard disk drives using flash memory instead of spinning platters. This greatly improves read speeds but doesn't do quite as much for write speeds. One reason is that each sector on a solid-state drive can only be erased a finite number of times before it starts failing. For this reason, the microcontroller in an SSD perform wear leveling to spread writes across more physical sectors. TRIM is a feature that an operating system can use to notify a drive that a range of sectors has become unused, which helps wear leveling run more efficiently. A cron job is a program that runs periodically in the background, and Canonical (the publisher of Ubuntu, a distribution of the GNU/Linux operating system) wants to add a cron job that scans attached drives for unused sectors and sends TRIM commands for these sectors. It's possible for an operating system kernel to send a TRIM command for multiple ranges of sectors, but the current version of Linux doesn't know this and instead sends one range at a time. This slows down deleting files because the kernel has to notify the drive of each sector range as the file is deleted. To work around this missing feature of Linux, the cron job will TRIM when a drive isn't busy doing something else.
A filesystem just notes which blocks are erased, it doesn't actually erase them. A flash based disk would normally keep maintaining the contents of erased blocks, since it doesn't know which blocks are still in use and which are not. Due to the way flash memory based disks work, keeping the contents of erased blocks causes significant overhead. Flash memory is erased and written in big blocks, so to write just a small sector, an SSD has to read a big block, modify it and write it back. This read-modify-write cycle causes so-called write-amplification, where writing a small amount of data actually causes much more data to be (read, erased and then) written to the flash memory. This is the reason why some disks are fast initially but become much slower once the entire capacity has been used once. With TRIM, the OS can tell the disk which blocks are no longer needed, so that the disk can treat them like empty blocks and not copy them in the read-modify-write cycle. (It's actually more complicated, but that's the idea.) The criticism is that the Linux kernel uses TRIM inefficiently (it uses many individual calls instead of combining several discontiguous erased blocks into one TRIM call.)
I imagine that discussing the suggestions on Slashdot first is a way to avoid presenting half-baked suggestions to busy developers.
"As long as that SSD doesn't stall trying to pull blocks off the top of that queue, it really doesn't matter how deep it is. So if you have 10GB of free space on your partition, you only need to call wiper.sh / fstrim once every 10GB worth of file deletions."
This isn't necessarily true. Earlier Trim will improve the performance of the SSD drive because the drive knows more free space -- more free space allows the drive to 1) pre-emptively erase flash 2) coalesce fragmented blocks 3) more efficiently combine write blocks 4) perform wear levelling operations with less overhead.
Early trimming can have a similar effect to the manufacturer increasing slack space which increases performances on nearly all SSD's.
Quick terminology note: Flash storage is divided into large blocks, commonly called pages (to avoid confusion with disk blocks). Each page contains many disk blocks.
Flash storage has an interesting property in that you can change individual bits in only a single direction (either from 0 to 1 or 1 to 0, depending on the flash type). To change it in the other direction, you must wipe an entire flash page, which means rewriting the contents of a large number of blocks. To avoid a high risk of a power failure causing the loss of data that wasn't even changing at the time, the flash controller does not do the erase and rewrite in place. Instead, it rewrites the entire page in a different physical location (with an updated copy of the changed block or blocks), and then atomically changes the block or page mapping so that the blocks are now associated with the new physical page. It then erases the original page so that it can be reused during a subsequent write operation.
This need to erase and rewrite has a side effect, however. As the flash drive gets more and more full, it eventually runs low on pages that can be erased ahead of time, because eventually every block on the disk has had something written to it at some point in the past, even if that block is no longer actively being used by any actual file. The disk does keep some spare pages around, but that only goes so far towards fixing this problem. This means erasing pages during the write operation itself, which is a much slower operation than writing to a pre-erased page. Many of those pages, however, may contain only data that is no longer relevant—blocks from files that were deleted a long time ago. Therefore, if the flash controller could somehow know that it is safe to pre-erase those pages ahead of time, they could be ready to go when you need to write data to them.
Unfortunately, it isn't practical for a flash controller to understand every possible file system, which makes that somewhat difficult. To solve this problem, they added a new ATA command, called TRIM. The operating system sends a TRIM command to tell the flash controller that the blocks within a certain range are no longer in use by the filesystem, which means that the flash pages that contain those blocks can be pre-erased for fast reuse.
Check out my sci-fi/humor trilogy at PatriotsBooks.
This is not so simple.
The original TRIM command is non-queued. It can kill drive performance on servers, so enterprise drives are designed to work well without TRIM. If you want better, and more importantly consistent performance then you should overprovision the drive. Overprovisioning means that you do not partition 20-40% of a new drive (or a used drive, after a secure erase). Those blocks will never be used, therefore the drive always have plenty of free space, so there is no need for trim.
Queued TRIM command appeared only in the SATA 3.1 specification, so only new drives support it.
Of source it doesn't implement the standard because it's a OS kernel, not a hard drive.
The drives implement TRIM, Linux just doesn't take full advantage of its capabilities.
The drive does shit (shit that you don't get to know the details about) when issued a TRIM command.
The OS is responsible for sending that TRIM command.
TRIM tells the drive when data is deleted, allowing the drive to do whatever it thinks is best when writing pages of data or erasing blocks of data.
Without TRIM, the drives considers all previously written data to be valid because it doesn't know about deletions (they're done at the logical level within the file system).
TRIM enables your drive to have much more flexibility when writing (and overwriting) data, and when load balancing and garbage collecting. It also reduces the need for load balancing and garbage collecting.
All decent modern SSDs support TRIM for good reason. All decent modern OSs should as well.
Now if I could just get Intel to enable TRIM on RAID 0 for my chipset (1 generation behind the cutoff), I'd be set.
Linux fully supports TRIM and failure to enable it will not damage the device in any way. What will happen is the device will slow down and spend more time freeing blocks as-needed if the drive is increasingly full.
Of course, if your SSD is your boot drive and you have /home elsewhere, chances are you aren't going to suffer and current drives are significantly faster than older ones (and at their worst, still significantly faster than rotating media.)
I'm aware of that. But what should I do if I want the browser to write the page to flash memory, such as if I'm about to go offline for an hour?
WTF? SSDs are important Linus. What is the major malfunction here?
There was a fully functioning implementation on its way into the kernel, but Linus lost it when his only copy of the code was lost because his SSD failed.
Why go to the trouble of implementing a command implicitly when you can implement it explicitly and avoid unintended side effects?
Because the explicit command causes unintended side effects in drives manufactured prior to the command's introduction.
What in the fuck, you seriously consider some dude's admitted speculation as proof that this is a real risk?
Here's some vastly more likely semi-informed speculation: Like most modern standards, SATA has many optional features, and standardized discovery methods to inform software what each device can and cannot do. If a drive says it doesn't support TRIM (or, to be more precise, lacks some new capability tag which says it can do TRIM), the OS simply never issues a TRIM command.
It's like you (and that Immerman dude) have no concept of how sane people design protocols to safely accommodate future extensions.
Not to mention operating systems would still need to change the way they handle the disk to support the 0xFF method
Any file system supporting "secure" deletion should be filling deleted files' sectors in the background anyway.
Which operating system would that be? I'm not aware of any mainstream OS which even tries to implement secure deletion at the kernel level. I know of exactly one (OS X) which implements it in a user-visible way at all, and there it's just an option to "secure empty" Trash (read: it overwrites file contents in place). This is really not very secure since it is 100% userland code which has no way of tracking all the blocks ever occupied by that file and scrubbing them all -- it can only scrub blocks currently occupied by the file. It can defeat casual attempts to recover file contents, but there aren't any guarantees.
But it's worse than that. Let's say you tried to write an OS For Paranoids which, at the kernel level, scrubbed all blocks as the file was deleted or as they ceased to be part of a file. This still would not be truly secure. ATA (and other abstract block device protocols like SCSI) implements semantics somewhat like this: "If you read block N, you'll get back whatever was last written to block N, or undefined data if N was TRIMmed and not subsequently written to". There is absolutely no guarantee that the device did not make invisible, unaddressable extra copies of the contents of block N behind the operating system's back. As a matter of fact, SSDs frequently do just that -- it's required if you're going to implement effective wear leveling. Even HDDs frequently leave invisible copies of old data behind whenever they have to spare out a bad sector. Merely overwriting blocks is not, and never has been, a true guarantee that the contents are actually gone from storage media.
This is why good SSDs implement the ATA Secure Erase command, which (if implemented according to spec) truly erases 100% of the media, including all overprovisioned space, any invisible copies of old data which it may contain, and so on. But wait, oh mah gawd that's another new command- HOLY SHIT IT WILL NEVER WORK BECAUSE NOBODY CAN EVER ADD NEW THINGS TO A STANDARD!!! why did I not see this before?!
Tepples, please, for the sake of my sanity, please consider no longer arguing endlessly with posters who are explaining how shit works in the real world. Next time, instead of succumbing to the temptation of raising objection after objection based on your ideas about how things ought to work, consider the possibility that those ideas are naive and wrong.
Those blocks will never be used, therefore the drive always have plenty of free space, so there is no need for trim.
It's not quite that simple either.
SSDs write in pages, but erase in blocks of pages. When a page is changed it gets rewritten to another block. The original page is marked as free, but it can't be erased until the whole block is free. Therefore the SSD performs garbage collection of free pages, re-packing them into complete blocks.
On its own the SSD only knows which pages it freed during rewrites - it doesn't know about pages that COULD be freed because they're deleted. Overprovisioning prevents blocking when there are no free pages (that's a huge win), but the drive still wastes lots of time and wear-life moving deleted data around during GC. TRIM provides the necessary hint to prevent that waste.
This isn't about Linux kernel not being smart enough, it's about crap SSDs that have horrible performance when TRIM is used during normal operations. So Linux can't use it during normal operations.
New things are always on the horizon
It makes you wonder what other optimizations are not being done but are done in Windows.
I'll toss in one: in Windows 8, an USB memory stick is automatically powered down if it is mounted but has not been in use for a while.
With multiple file systems per drive, a given file system doesn't necessarily know the drive is idle so some other process would need to do the delayed TRIM which is what Canonical is suggesting.
Why would a filesystem need to know? On FreeBSD, the filesystem just spits a BIO_DELETE command into the GEOM layer, and it is up to GEOM to schedule when to dispatch it - it's free to reorder it, as long as it doesn't move it after a BIO_WRITE with an overlapping range. If the filesystem needs to know about the status of other filesystems then that's a serious layering problem. The FS should not be making the decision about whether to send the BIO_DELETE, because it's the responsibility of something lower down the stack to decide what to do with it. For example, a RAM disk can use it to free the underlying memory. You don't want every filesystem to have to know about every possible kind of device.
I am TheRaven on Soylent News
The word you are looking for is API. An implementation is on the side of the thing that exposes the API. The OS uses the API, it does not implement the standard. Implement would mean providing the API. Not making use of an API doesn't mean you're not standards compliant. Eg. None of my OSs make use of (U)EFI (because coreboot is amazing). My OSs are following the boot standards, They just aren't using the API that was implemented for (U)EFI.
You "execute" TRIM through the API of its implementation. Calling it on delete is an optional part of the protocol. Not calling it on deletes is not deviation from the standard.