Slashdot Mirror

TRIM automatically zeros the blocks of deleted files and they are GONE aside from vague sci-fi and probably nonexistent NSA-type forensics.

The DATA SET MANAGEMENT (TRIM) ATA command *does not* necessarily zero the blocks (you mean NAND pages). This is a common misconception, and I'm still not sure where it stems from. What the DSM command chooses to do depends on what the SSD manufacturer chooses. When this matters is when the next read operation happens (whether it be NCQ or non-NCQ, doesn't matter) against the same NAND pages that were previously TRIM'd (I can't say LBA or LBA range because of how the FTL map works -- LBA 12345 before and after TRIM may actually refer to two completely different NAND blocks, based upon wear levelling and several other technologies).

It's possible to determine what the behaviour is based on IDENTIFY DEVICE data, which allows an OS to essentially know what "model" the vendor chose to implement. Utilities like FreeBSD's camcontrol(8) will show you this setting in a user-friendly fashion (Linux might have something similar, not sure; Windows I doubt, but it could be something smartmontools/smartctl could implement with ease). Rather than explain it all here, I'll simply point you to the ACS-3 specification PDF and page and you can read it yourself. You want Section 7.5.3.2 (TRIM bit), which is under Section 7 (Command Descriptions), subsection 7.5 (DATA SET MANAGEMENT / command 0x06): http://www.t13.org/Documents/UploadedDocuments/docs2013/d2161r5-ATAATAPI_Command_Set_-_3.pdf

Once you read that, you'll understand how it's not necessarily guaranteed that the underlying NAND content has been zeroed. Essentially, the DSM command removes a list of LBAs from the FTL map -- it doesn't necessarily zero out those NAND pages in the process. Is this a security problem? No, because the only way you'd be able to access the physical NAND pages would be to bypass the FTL (which AFAIK you cannot do aside from desoldering the actual NAND flash chips).

The subsequent individual's reply stating that the options are "zeros, ones, or random" is not entirely accurate. The options are: "non-deterministic" (i.e. it could be the same data as what was previously there prior to DSM, it could be random, it could be the letter "Q" repeated, it could be an ASCII picture of a snake), "deterministic with static content/value" (what the content/value is isn't specified), or zero. In other words: 2 of the 3 choices are more or less up to the SSD vendor to decide for themselves. Aren't specifications fun when they don't really define behaviour? :-)

Hope this was informative!

There are plenty of comments in the thread that mention devices other than storage. The ATA command set is designed primarily for storage. It is probably "complete enough" to connect a custom video chipset through it, but you need to rewrite all the drivers. Thunderbolt will allow you to just plug the card, or two, or three as you need them. You can build a pretty impressive video machine with that if you want to. Apple always went for extra features - they need to justify the price, after all.

So don't focus on storage - there are indeed lots of solutions for storage already, the NAS being the cheapest and probably sufficient for majority of the users. Think of *what else* you can do with a notebook if suddenly you can insert additional PCIe cards into it, and the bus bandwidth is not a concern. You think connecting several powerful GPUs to your notebook is not important? That gives you plenty of performance. If you are a videographer you can run a lot of filters on full resolution, uncompressed video this way, or do whatever else you need. If you are a business you can now develop and sell external PCIe devices that connect to the host via the Thunderbolt interface.

That method works, but for the day or more it took them to do that, using the Secure Erase ATA command on that drive would have been more secure and taken only an hour or two. The Secure Erase command is part of the ATA standard and present on every ATA drive larger than 15GB. The command "dd" cannot access and erase every sector as ATA drives do not allow access to certain sectors, like reallocated sectors. Even though SCSI drives do not have this limitation, I still wouldn't erase one with "dd", there are probably better open source tools. An even better and faster option for even more secure erasure on ATA drives, is to use the drive in encrypted mode. When done with the drive, toss the encryption key. This makes any data on the drive practically unusable. Reuse of the drive is still possible with a standard reformat after unlocking the drive.

More reading:
Hard Drive data erasure methods are described on page 27 of the PDF or page 19 as printed on the document:
http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf

Describes different methods of data sanitization on magnetic hard drives. Discusses hard drives exclusively, unlike the NIST paper above.
http://cmrr.ucsd.edu/people/Hughes/DataSanitizationTutorial.pdf

Page from the author of the above paper with a DOS program that can send a Secure Erase ATA command to a drive, no source though:
http://cmrr.ucsd.edu/people/Hughes/SecureErase.shtml

However, t13.org the website of the ATA standards body is here, and has the last drafts of standards available here (nearly as good as the actual standards, which cost money):
http://www.t13.org/Documents/MinutesDefault.aspx?DocumentType=4&DocumentStage=2

Start here though for the Secure Erase Command:
http://www.t13.org/Documents/UploadedDocuments/docs2009/d2015r1a-ATAATAPI_Command_Set_-_2_ACS-2.pdf

That method works, but for the day or more it took them to do that, using the Secure Erase ATA command on that drive would have been more secure and taken only an hour or two. The Secure Erase command is part of the ATA standard and present on every ATA drive larger than 15GB. The command "dd" cannot access and erase every sector as ATA drives do not allow access to certain sectors, like reallocated sectors. Even though SCSI drives do not have this limitation, I still wouldn't erase one with "dd", there are probably better open source tools. An even better and faster option for even more secure erasure on ATA drives, is to use the drive in encrypted mode. When done with the drive, toss the encryption key. This makes any data on the drive practically unusable. Reuse of the drive is still possible with a standard reformat after unlocking the drive.

More reading:
Hard Drive data erasure methods are described on page 27 of the PDF or page 19 as printed on the document:
http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf

Describes different methods of data sanitization on magnetic hard drives. Discusses hard drives exclusively, unlike the NIST paper above.
http://cmrr.ucsd.edu/people/Hughes/DataSanitizationTutorial.pdf

Page from the author of the above paper with a DOS program that can send a Secure Erase ATA command to a drive, no source though:
http://cmrr.ucsd.edu/people/Hughes/SecureErase.shtml

However, t13.org the website of the ATA standards body is here, and has the last drafts of standards available here (nearly as good as the actual standards, which cost money):
http://www.t13.org/Documents/MinutesDefault.aspx?DocumentType=4&DocumentStage=2

Start here though for the Secure Erase Command:
http://www.t13.org/Documents/UploadedDocuments/docs2009/d2015r1a-ATAATAPI_Command_Set_-_2_ACS-2.pdf

The ideas behind this are applicable to any O.S. and there are proposed standard ATA commands to manipulate the Non-Volatile cache, see http://www.t13.org/Documents/UploadedDocuments/docs2007/D1699r4b-ATA8-ACS.pdf. I hope Linux and Mac hackers are working on it.

I'm not sure if the drive takes advantage of NV cache without specific O.S. support. Even without O.S. support, the drive could decide "You keep reading blocks X Y and Z, so I'll store them in NV cache" (drives already do this with their conventional RAM cache, typically 8MB) and "I'll keep your pending data writes in NV cache while waiting for the disk to spin up".

Windows Vista's ReadyDrive takes specific advantage of this feature: "During shutdown or hibernate all the disk sectors needed to boot or resume are pinned into the NV cache... Offsets within files and/or specific LBAs can be specified by the PC OEM in registry for pinning in the NV Cache". I converted the MS PowerPoint presentation http://docs.google.com/Present?docid=dcvvrqtp_13gj635t&fs=true (yay Google Docs, die PowerPoint DIE!).

These drives will be available with PATA and SATA interfaces.
A new revision of the standards is being worked upon to support hybrid drives.
Take a look at http://www.t13.org/docs2005/e05113r0-Hybrid_Disk_T 13_Presntation.pdf

the reason you can replace them with newer ones that have large drive support is because:

There's more profit in selling you a new card than letting you update the old ones! Ta-Daa!

Anyway, you can just plain skip the logic anyway. I've written ATA drivers for controllers before. I know how it's done, and I personally have done it and know that there is no restriction to what controllers can have LBA48 software written for them.

Go get the spec from t13.org ourself. All you have to do is write 4 registers twice. The drive remembers both the most recently written and the previously written value. The controller does nothing new at all.

As to bridge boards, they could be out of code space, they might have limits on the size of the data to be stored (can't store >28 bits of sector or >8 bits of sector count), or they just might want to sell new chips too.

Oxford might have made a "smart controller" and don't feel like switching over and going to "dumb mode" to get it to work. Or perhaps their controller is "too smart" and thus cannot be made to work in LBA48 mode. I know that the >8 bits of sector count (which allows transfers larger than 512KB at once) means your DMA equipment in your bridge might have to be able to do transfers larger than 128KiB. This was a problem on a device I worked on. It might be a probem on special bridge chips that DMA straight from the drive to the bus (like the Cypress USB chips).

Or perhaps they just want to sell new chips too. It doesn't matter, I never said all bridge chips can be updated to do LBA48, in fact, I believe I gave reasons you perhaps couldn't.

I can assure you this isn't the case with any PC interface, since they all have to work with the standard Windows driver in order to boot Windows, and thus have a "dumb mode" where you can write any registers in any order you'd like. You then write the 4 registers in question twice and run the DMA engine a couple times in a row (actually, Windows doesn't do transfers larger then 128KiB anyway with the default drivers) and you're there.

As I said, any PC controller can do it. Bridge chips can be different, due to various factors.

As to your last paragraph, well, again there is no requirement for LBA48 hardware. And the software is clearly running in LBA48 mode (despite being on what you call "non-LBA48 hardware"), since without it, you can only address 2^28 sectors, and thus you simply have no way to access those sectors above 128GiB.

I do appreciate applying logic to a situation. But in this case, you're wrong, and you should do some research about it before looking even more foolish.

As perhaps something to spur you to this, let me give you a link about how series 1 Tivos (which were released in 1997 or 1998) can use LBA48, with a new kernel.

http://www.courtesan.com/tivo/bigdisk.html

I know this isn't a "knockout blow" to your theory, but perhaps it is enough to spur you to look further and realize your mistake.

Finally, here's the link to the ATA spec site.

http://t13.org/#Project_drafts

Their specs aren't exactly transparent, but maybe you can learn something from those too.

You can't. You still need to pass the password with the SECURITY ERASE UNIT command.

See the end of this document.

• The eye-searing white background
• The hard-to-read multi-column format (great on paper, sucks on a screen)
• The hideous Type-1 fonts
• Putting everything into a PDF just to include one or two gratuitous images
• Converting what were once plain text files into PDF (why???)
• "Unable to extract the embedded font 'ZDCCZL+Times-Roman'. Some characters may not display or print correctly."
• Grow old and die waiting for GhostView to render a PDF.
• Huge PDFs that contain scans of paper documents.

Well, they shouldn't be expecting undocumented, nonstandard behavior then. As I said in my original post, the ATAPI FLUSH CACHE command flushes the write cache. See section 8.12 of the ATAPI specs. I don't think there's any standard ATAPI command to flush the read cache.

Anyway, a standard command doesn't mean that every OS is required to send that command; it means that if an OS sends that command, it can expect a certain result.

Karma whore. But I'll also add T10, the SCSI and Serial-SCSI guys and T13, the ATA and S-ATA guys Draft standards and other documents galore!

That's actually not a bad idea, and has at least some potential for Good, IMHO. With storage as cheap as $1 per gigabyte, an OEM moving their 650MB Recovery CD into this area seems like a great idea (not great if you want to swap in a new hard drive yourself, but they're not targeting power users)

However, Phoenix seems to be under the impression that just because they have this space, they should use it. A web browser and TCP/IP stack? Someone explain to me how this is good, and how the target "non power user" will be able to make sense of this technology without RMA'ing their computer if it gets that hosed.

Ok, after having done more research than I thought would be necessary...

Most "ATA/100" systems aren't implementing ATAPI-6. They're implementing ATAPI-5 with an extention that includes UltraDMA Mode 5. ATAPI-6 does have 48-bit addressing, and Maxtor has implemented an extention that adds UltraDMA Mode 6 (aka ATA/133).

Note that ATAPI-5 is the current official standard. ATAPI-6 is _not_ yet official. See the Technical Committee T13 website for details. Another good reference is ATA-ATAPI.com, along with PC Guide ATA standards.

The net effect here is don't confuse the physical interface (ATAPI) with the network interface (UltraDMA). Yes, nitpick at the terms, but that's what it boils down to. Your "ATA/100" motherboard does not support 48-bit addresing.

I agree, however, on the crappy design, the marketing blurbishness, the projection of HD speeds, and your recommendation about not running out and buying a 133 adaptor.

Meanwhile, back in the present, I've probably spent more than a thousand bucks on software and PC documentation over the last 10 years without ever getting anything that resembles a complete description of the hardware.

A complete description of a modern computer in all its complexity would likely fill a bookshelf. Just the latest draft of the ATA specification is about the same thickness, when printed double-sided, as the Apple IIe Technical Reference Manual I got back in 1987, which provided complete schematics, specifications, and even source code for the ROMs (except for the BASIC interpreter, and if Apple hadn't gotten that from M$, it probably would've been published too).

In any case, it's not like joining, say, ICANN, to be done from the comfort of your keyboard.

So, I suspect we're not all going to run out and do it. But, we can support some folks we trust to do so. My first thoughts are to ask Mr. Gilmore and/or Bruce Perens, if IBM's left hand would let its right oppose these doings. Noise won't help here, but a combine in the form of that supporting Damian Conway's Perl work should be possible. Can one of our existing organizations (YAS or SPI [if there's still anyone home at at the latter]) pick up the banking effort?

For myself, I pledge to donate USD100 to such an organization for this purpose. Are there seven others willing to step up to the plate? If so, we've got a membership in hand.

Well, I definitely need to join the T13, since I am a "materially impacted individual." Moreover, I will be most materially impacted if I don't make it to the October "Plenary and working group" in Maui.

Sometimes I worry that I'll develop Alzheimer's disease,
but no one will notice.

Well, I definitely need to join the T13, since I am a "materially impacted individual." Moreover, I will be most materially impacted if I don't make it to the October "Plenary and working group" in Maui.

Sometimes I worry that I'll develop Alzheimer's disease,
but no one will notice.

EFF joined the committee and attended their meeting. If you care about whether your disk drives will conspire behind your back to murder fair use, you should too. See www.t13.org for how to join. Get a "voting" membership, you may need that vote.

--

All the Phoenix proposal does, is provide a somewhat standardized way for doing vendor-specific extensions to the command set. Drive vendors *already* have the ability to do this - "vendor specific" command codes have been part of the ATA standard for years. Up until now, though, there was no way for a disk drive to report *which* vendor-specific command codes it implements, or what interpretation it puts on them.

The Phoenix proposal reserves a range of 8 command codes (previously listed as "obsolete") for the new "generic command" feature. It allows a drive to use these commands to implement *any* sort of vendor-specific features that the manufacturer and customer might want. The drive can make available a 128-bit code intended to identify the command set. This 128-bit GUID (Globally Unique ID) is *not* an encryption key - it's simply a way of identifying which command set the drive implements.

Vendors could implement all sorts of useful capabilities using this mechanism. It might be a way to support drive-based data compression, or to "tune" drive performance in customer-specific ways, or to retrieve information about the drive's health which goes beyond what's available in the normal drive error and performance logs. Heck, it could be used to implement a "play a tune by chattering the drive head back and forth" feature if you wanted to, or to issue "perk" and "pour" and "clean filter" commands to a robotic coffeemaker attached to an ATA interface if you wanted to do something that silly.

The Phoenix proposal is not a "similar copy-prevention plan". It does *not* enshrine any sort of copy protection algorithms, rules, keys, key management strategies, or licensing rules into ATA-6 in the way that the CPRM proposal would have. It says nothing about copy protection at all. I spoke with the gentleman from Phoenix who wrote the proposal, and he told me that it was an idea that he'd been thinking about for some years because he could see that it could have a lot of useful applications. The CPRM hoorah was what prodded him into actually writing it down and proposing it.

The *only* relationship between the Phoenix proposal, and any sort of copy control, is that the generic commands _could_ be used as the interface to a copy management scheme if a vendor chose to use them in that way. But, as I said above, vendors could ALREADY be doing this if they wanted to, through the "vendor specific" command codes that have been in ATA for years.

The Phoenix proposal also gives the drive user a way to shut off whatever generic command set is supported by the drive. This can be done either via the SET FEATURES command (disabling the generic features until the drive is physically powered down) or via DEVICE CONFIGURATION OVERLAY (disabling the features even across powerdown). The proposal states that these disable features "shall" be supported - which means that the disable is *mandatory*. A drive which was sold with a generic command set and GUID (of *any* sort) that could not be disabled, would be in violation of the ATA-6 spec - this would cause the drive to fail qualification at a *lot* of customers.

The T.13 committee is not going to be "registering" generic command sets, their 128-bit GUIDs, or their interpretation. That is *strictly* up to the drive manufacturers and their customers (assuming that the Phoenix proposal passes, that is). That's the biggest reason why the GUID was spec'ed with so many bits in it... a drive vendor can simply pick a GUID for their feature set at random, and not have to worry about the chance that another vendor will choose the same GUID.

As I said, folks, I think we won one this time around. The committee members were well aware of how much public upset there was over CPRM. If IBM hadn't withdrawn the CPRM proposal, I believe it would have been voted down. I can't predict the future, but I personally doubt that they'll ever try to reintroduce it.

The bigger battle to come will almost certainly lie elsewhere - in the courts, and in Congress.

The T13 web/FTP sites seem to be going through some changes at the moment (this was mentioned at the meeting), and I don't see a copy of the Phoenix proposal on-line yet. Keep your eye on the technical-documents directory. The proposal will be document e01112r1 when it's posted there.

Read through it for yourself and see what it says. And, if you still have concerns, come to the next T.13 meeting, in April - they're open to the public, they're free, and no preregistration of any sort is necessary.

there is a ton of information at T13.org. these links require a PDF reader:

• e00148r2 Content protection of recordable media (CPRM) 12/6/00 anderson
• e00152r0 CPRM presentation 10/13/00 anderson

Note the following contact information: Jeffrey Lotspiech <lotspiech@almaden.ibm.com>. Is this the lead project engineer? If you feel like emailing him for information, BE POLITE. Also the presentation mentions someone named David Goldschlag, who may be able to shed more light.

The second link especially has lots of nice diagrams and information about data structures - possibly useful for constructing workarounds and educating people about what this is exactly. Highly worthwhile reading.

still no mention anywhere about SCSI... seems like a "safe" alternative? I'm ready to ditch IDE forever.

Well, no company has yet adopted the spec, and t hasnt even been approved by the standards body. IBM seems as if they will be the first. You can contact IBM's storage dept. here, and you can find t13's website here, where you can email the members, or even join by yourself, if you wish. The docs on CPRM arehere, here, and here, and you can see all the documents from the future spec here. Who says we have no say?

Well, no company has yet adopted the spec, and t hasnt even been approved by the standards body. IBM seems as if they will be the first. You can contact IBM's storage dept. here, and you can find t13's website here, where you can email the members, or even join by yourself, if you wish. The docs on CPRM arehere, here, and here, and you can see all the documents from the future spec here. Who says we have no say?