Slashdot Mirror
Write Bits Directly Onto a Hard Drive Platter?
kidcharles writes "I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion. I need to be able to control exactly where 1s and 0s will appear physically on the platter. Normally when data is written to a drive the actual bits that get written are determined by the file system being used, as modified by whatever kind of error handling the drive itself is using (e.g. Reed-Solomon). All of the modern innovations in file systems and error handling are great for reliable and efficient data storage, but they are making my particular task quite daunting. My question for Slashdot: is there a way to get down to the 'bare metal' and write these bits? Any good utilities out there to do this? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."
Just make your own controller chip for the drive.
No, there aren't.
You would need to completely bypass the drive firmware in a way that would make the drive quite unusable afterwards; this is generally not possible in hardware. It's not like they're going to let you control the voice coil circuitry directly as permanent damage to the drive would likely result! This is likely going to be a highly custom solution and you will need special equipment to do it. Contact the drive manufacturers or a very big data recovery specialist.
They (the bits you write to your typical /dev/sd* device) would still be wrapped up in sectors, the sectors be enhanced with error correction, then encoded (NRZ, Manchester or whatever is appropriate to the magnetic storage) and then written out. What OP wanted is to manipulate the bit that comes out of encoding. Given the usual stack of SCSI-like commands (and not direct access to the write head signal as early HDD controllers had) this does not seem feasible.
Besides the fact that Flash doesn't have a hard disk platter (his problem obviously isn't just to save data), Flash chips also contain internal logic to map out defective blocks, and in addition they remap in order to avoid writing single blocks too often in order to prevent them from failing early. So even if the HDD platter isn't really needed, he won't get exact control on Flash either.
If you had at least suggested a floppy. There you can indeed control exactly where your bits go, and it's even a magnetic medium.
The Tao of math: The numbers you can count are not the real numbers.
dd only bypasses the filesystem. It doesn't override the HDD firmware, which has to avoid bad sectors and write ECC information to the platter as well.
I would love to hear why the submitter thinks he needs to write directly to the platter.
RTFS
Knowing everything there is to know about dd isn't going to tell you where on a physical disk your data is stored. dd only sees a long string of zeros and ones that are a far abstruction from what physically happens on the drive.
There doesn't seem to be any way to be sure exactly where the data is written except by a deep understanding and maybe a replacement of the drive firmware.
Old MFM hard drive should do the work. The entire device was controlled via software in dos. I have one lying around. A whopping 5 MB!
http://en.wikipedia.org/wiki/Modified_Frequency_Modulation
Now all you need is a 8 bit ISA slot and you are set.
RES PUBLICA NON DOMINETUR
Hahaha. 300-400MB isn't "very old". It's only 20 years since I bought a 80MB hard disk for £150 and that was considered massive as a couple of years earlier 20MB was cutting edge and cost a few hundred pounds.
It can not be done with off-the shelf equipment and software for many reasons, some of which are:
Line Codes
Error Correcting Codes
PRML
SMART
LBA
etc.
Specialist Mac support for creative pros, Melbourne
No it doesn't.
If I read it correctly (it was a couple of years ago) all Spinrite does is move the head to different tracks before trying to read the data. The theory is that a seek from different distances might align the drive head slightly differently to the data.
As far as I know there's no way to position drive heads directly via IDE/SATA, much less write individual bits. IDE/SATA are high level protocols, all the low-level logic is inside the drive controller.
No sig today...
NOR flash doesn't require or use wear leveling, error correction, or bad sector management (as long as your wear pattern isn't ridiculously unbalanced) - use that if you want true "write raw bits" style functionality. Most microcontrollers use this type of flash, and chips are readily available, though they are expensive compared to NAND flash and only exist in smaller sizes.
NAND flash requires you do your own wear leveling, error correction, and bad sector management. You can still control all the bits, but be prepared to have the chip eat some of them.
Only "cooked" NAND flash devices such as SSDs and SD cards actually do all that on a built-in controller.
Get yourself an old, totally unintelligent S412/506 MFM controller from out of an IBM PC or PC/XT. These were fairly dumb devices (g=c800:5 in debug, anyone?) for which you were meant to enter a "bad sectors list" printed on the front of the drive after performing the low-level format yourself. The earlier the drive, the closer to bare metal, so you might want to look for any of the 2, 5, or 10MB (yes, MB) full-height (2 x 5.25" drive bays stacked on top of one another) drives that were floating around then. You'll also want to get yourself a set of ribbon cables.
You should be able to use a drive/controller combination like this with any machine with ISA bus slots up through about the 386/486 era, and that would let you also go back and grab an early Linux distro (say, kernel 1.2.13 days, like Slackware 3 or so) that included drivers for such a controller that were actually in use and known to work at the time, giving you a base on which to build more code.
If 10MB is too small, you might just have luck going up to the largest of the MFM (80MB) or even RLL drives (160-200MB, just get an RLL controller instead) drives. I don't remember whether there were any ESDI drives back in the day that didn't remap their own sectors, but if there were, these controllers were 16-bit ISA and somewhat smarter (also with Linux drivers from the period available) and went up to 680MB or so.
But if you're looking for the best chance of success for your purposes and don't need tons of storage, my educated guess would be that the MFM controller out of an IBM 5150 PC plugged into a 5MB ST506 hard drive and connected to a SIMM-based 80386DX mainboard with 8 SIMM slots (for 8MB ram) might be the easiest combo to find and get working in practical terms that has a chance of doing what you want.
STOP . AMERICA . NOW
I think anything > 120MB may be unsuitable for this purpose, as they are likely to use ZBR, i.e. different sector densities depending on the circumference of the track. I'd also recommend a really old MFM or RLL disk, where the head movement is controlled with one set of wires and the analog signals from the read/write heads use another set of wires. MFM and RLL drives use the ST506 interface, which should be "easy" to control with microcontroller and a little bit of glue logic.
*abstraction*, not *abstruction*
"Place me in the company of those who seek Truth, but deliver me from those who believe to have found it."
I don't think what you want to do is possible, because I don't think modern drives actually write the 1s and 0s in the way that earlier ones did. They use Viterbi coding, which means that the exact change to the magnetic field to encode a 1 or 0 depends on what the preceding pattern of 1s and 0s is. They literally can only write sectors at a time.
Then again, what is meant by a sector has changed with time with different servo technologies. With the embedded servos in current drives, you would need to know a lot about the particular geometry of the particular drive you are using.
And to do any of this, you would have to bypass the controller completely - which, I am pretty certain, cannot be done without serious hardware modifications
Consciousness is an illusion caused by an excess of self consciousness.
You know how data restoration companies do it?
They take out the spindle with the platters, and put it in their own reading device with its own controller. And with that you can read and write the exact bits (as long as quantum physics allow it). But the head has to be compatible (e.g. perpendicular recording needs entirely different heads).
I bet those devices can be bought, and I bet their controller is actually just software on the computer (for flexibility). I also bet they come with different head configurations.
But they are definitely not going to be cheap.
Hey, at least it is a real solution. :)
Any sufficiently advanced intelligence is indistinguishable from stupidity.
No less then *7* posts to the SAME XKCD COMIC?!? Christ, bad enough people don't RTFA, at LEAST read the fine comments!
"Gratuitous complexity is akin to chaos" - True Vox
It's been posted and modded to oblivion multiple times.
$_ = "wftedskaebjgdpjgidbsmnjgcdwatb"; tr/a-z/oh, turtleneck Phrase Jar!/; print
You have got a lot of catching up to do. Drives just don't work like that any more.
The GP is right, Spinrite does not and can not do most of what it claims on a modern HD. The only potentially useful thing it does is re-read sectors up to hundreds of times, but there is safer software that does the same thing.
More importantly, Spinrite can be downright dangerous to your data. Spinrite writes any data it recovers TO THE SAME DRIVE! It can very easily overwrite other data you want to keep and/or lose the recovered data again because the drive isn't stable enough to hold it reliably. Not writing to the damaged media is data recovery 101 and Spinrite fails badly.
For the benefit of the poster, who doesn't seem the type to know what a servo track is:
A servo track is how the hard drive knows where the heads are over the disk. Older hardware could just read the angle of the arm, but with vast increases in density it became necessary to put position data on the platters themselves where more precision is possible.
Spinrite may "write" 00000000 to a specific location on the disk, but that 00000000 is actually encoded as something else when it is written to the disk to maintain a minimum number of bit transitions. In the Telecom world, B8ZS is commonly used for encoding. AFAIK, the HDD manufacturer's algorithms are proprietary.
It is also worth mentioning that a hard drive head does not read a binary signal from the platter. It reads an analog waveform and converts it to digital via the encoding and ECC algorithms the OP is trying to bypass.
-ellie
Scott Moulton of http://myharddrivedied.com/ is the smartest HDD guy I know. If you absolutely have to have a solution, talk to him.
First, ones and zeros on a harddisk platter are not magnetic north and south, they are different frequencies of an FM sine wave. Second, the formatted tracks on a harddisk don't line up physically. The start of the track is wherever the head was when the controller was told to format that track. Third, Sectors aren't written in precise places inside their formatting. The low-level formatting process writes headers across the track that are used later to find the data. There's enough extra space between those headers that a sector of data will fit there, based on the ability of the controller to read where the sector starts and be able to write it. If you really need to write magnetic north & south to specific spots, you should look at Floppies. Floppy disks have a hole to mark one spot on the disk, so the start of track comes much closer to lining up. You can even write ones and zeroes instead of FM patterns if you have an old Apple II, which didn't use FM encoding.
Quick! Go read his FAQ! Right there under 'Can I low-level format my drive' it says 'not possible on any modern hardware'. Randomly poking bits at specific PHYSICAL locations (which is what the poster wants to do) is low-level formatting.
No, they don't. Not anymore. Take a typical WD drive for example. Clusters are composed of areas of data, separated by areas of correction code (ECC) data. The latter are written by the drive hardware, or firmware at least, and depend on what is written to the data area. (In other words, it's meta-data, for detection of bad writes or corrupted areas of the disk.)
So even aside from boot sectors and so on, the data areas of the drive are separated by areas written by the drive's control firmware, so the latter are not directly writable by the user. Unless there is some kind of low-level command that can be sent to the drive, telling it to not write ECC data at all (and I don't see any reason for the existence of such), then the answer is NO: you just can't do that with a newer drive.
The difference between the way you learned hard disks worked long ago and the way we use them now is due to IDE technology. Internally, the drive is still commanding cylinder, head and sector. Externally (to the PC), the interface is very different.
Way back when we used to have the computer telling the drive controller board to operate the physical drive mechanisms, with the controller telling the drive which cylinder, head and sector to retrieve. This was called Cylinder-Head-Sector (CHS) addressing. If you ever worked on an original IBM PC-AT, you might remember having to set the "heads and cylinders" in the BIOS settings. You, the human, had to select from a list in the BIOS how many heads and cylinders the disk drive had. (And the older the PC, the fewer the choices. That meant every drive you could buy already had to be supported in the BIOS, but since there were so few originally, it didn't matter too much. As I recall, the max size drive supported by the original AT BIOS was 33MB, even though the largest drive IBM offered at the time was something like 20MB, if you had all the money.) The protocol was called ST-506, named after the first commercially available 5-1/4" hard drive.
Integrated Drive Electronics (IDE) changed that. The controller board was moved from the PC to the drive itself. So now the controller board was set up by the drive manufacturer to know exactly how many heads and cylinders the drive had. To talk to an IDE drive, ATA was developed as a protocol. The computer would no longer send commands for a specific physical drive geometry, but would instead ask for a "logical block number", and the IDE would do a translation (looking it up in a map) to command the hardware to move to the correct physical cylinder, head and sector to retrieve the data. This is called Logical Block Addressing (LBA).
So unless the original story author knows exactly what drive he's dealing with, and can download the lookup table from the drive, what he's asking for is no longer realistic. Each individual drive has its own unique mapping from LBA to CHS, because each drive maintains its own map of bad sectors. A computer asking for logical block #52 might physically get it from head#0, cylinder#3, sector#21 on one drive, but due to bad block mapping on a different drive it might get it from head#0, cylinder#4, sector#35.
SCSI works on a similar principle, and Serial ATA has made further improvements on the scheme for performance and size changes. But all modern drive interfaces use some form of logical block addressing to access the data, and none use direct cylinder-head-sector addressing.
John
If you want an exact image of 1's and 0's that you determine, forget it. Bits these days are encoded as transitions between magnetic phase boundaries. They also encode the clock data needed to recover the data along with the data. As a result, what's put down on the disk has to guarantee a certain number of these transitions or you loose sync. In other words, you may think you're writing '0000000' but on the disk it's a bunch of magnetic transitions, not a constant stream of the same magnetic polarity. It's not even that a single phase transition translates into either a 1 or a 0; groups of these transitions translate into a group of the actual data bits. Some patterns of phase transitions can simply never be written onto the media by the write electronics in the first place because they would not result in reliable data recovery by the read electronics.