Slashdot Mirror
Write Bits Directly Onto a Hard Drive Platter?
kidcharles writes "I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion. I need to be able to control exactly where 1s and 0s will appear physically on the platter. Normally when data is written to a drive the actual bits that get written are determined by the file system being used, as modified by whatever kind of error handling the drive itself is using (e.g. Reed-Solomon). All of the modern innovations in file systems and error handling are great for reliable and efficient data storage, but they are making my particular task quite daunting. My question for Slashdot: is there a way to get down to the 'bare metal' and write these bits? Any good utilities out there to do this? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."
It works for me. http://www.staples.com/Staples-Dry-Erase-Markers-Chisel-Tip-Black-4-Pack/product_607101?cmArea=FEATURED:SC1:CG11:DP1101 Now that someone has said it no one else can be a smart ass and they have to be informative.
May I suggest a magnetic needle and a steady hand? http://xkcd.com/378/
Regards, Boyan
Just make your own controller chip for the drive.
I'm fairly sure you would have to go fairly deep and directly hack/modify hard disk firmware to do what you describe. Such hack would be non-trivial and most likely model-specific.
(nt)
http://xkcd.com/378/
No, there aren't.
You would need to completely bypass the drive firmware in a way that would make the drive quite unusable afterwards; this is generally not possible in hardware. It's not like they're going to let you control the voice coil circuitry directly as permanent damage to the drive would likely result! This is likely going to be a highly custom solution and you will need special equipment to do it. Contact the drive manufacturers or a very big data recovery specialist.
You would need to replace the firmware inside the drive or use an undocumented manufacturer mode. Whatever they use to write the servo tracks would be interesting to you. You will be in the situation of the firmware writer: There will be problems all the way. Be prepared to find a way to position the heads (ever tried to find a servo track?). Most likely you also need to at least parametrize the amplifiers in the DSP part of the firmware that does the analog-to-something-to-digital so you can have direct influence on the "bits". Good luck
First thought was - WoW! Somebody wants to defeat disk caching, ECC and make life bloody difficult to boot. The best way, if you really want to do direct disk RMW operations - is to use Flash. Its fast, reasonably portable, and won't fail you if you use it properly.
I'm not 100% sure, but on a *nix system you might be able to do this by writing to /dev/, where device represents the HDD.
If you don't need lots of bytes, you should use an old disk. I mean very old, maybe going back to those 300-400MB disks we had on 486 computers or even older.
Recent disks have embedded error-handling mechanisms which will prevent you from writing what you want, where you want (physically speaking of course). The disk controller translates your "physical" disk addresses into its own internal mappings. The only way to bypass this would be to remove the disk controller's chip and put your own in place.
Maybe your question would be: "how hard is it to create my own disk controller's chipset?"
What possible reason would need to do what your suggesting
may be its my lack of imagination but why
The submitter appears to know enough about what he's asking to know that it's also impossible / completely impractical. Recording ones and zeros directly isn't done for a reason, submitter appears to understand this.
Presumably there is a way of at least reading specific sectors. Surely data recovery people use something like this to get raw data from a damaged drive bypassing anything the drive might do to try and correct errors. Maybe a data recovery specialist could help you.
How are you planning to read the data back? Solve that, then think about writing. As others have said, you'll probably have to build your own drive controller.
Your requirements are wrong, sorry.
Yes, I don't what they are. Still wrong.
You could try this, or ask yourself again: is this what you really want to do?
Seriously, you are out of luck. Unless you come across a sympathetic hard drive manufacturer there is very little you can do.
I can't help directly, but can give one important advice - careful how you distribute the bits! If too many ones get on the same side of the platter this will destabilize it, causing it to wobble due to the weight difference ( a one weighs quite a bit more than a zero, you know!) and potentially tearing the platter in two!
This kind of copy protection has been outdated for quite some while and should have died with floppy disks ...
These kinds of questions are stupid: "I need to do XYZ for a project, how do I do XYZ?", where XYZ is one or more of complicated, ridiculous, vague, nonsensical, etc. Try telling us what your project is, and then we might be able to suggest a useful solution, possibly not involving XYZ at all, or involving a very particular/practical version of XYZ.
"I'm working on a project to build a nuclear powerplant. Is there documentation on how to do it? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."
It seems to me that if you're involved in such a project and have to ask how to do it, it's doomed.
Also, the whole idea of it seems rather impossible. Why would you want to do this in the first place? Have in mind that hard disks don't write bits as is, not just because of ECC. They use an encoding to ensure that there are no long strings of 1s or 0s. If you just wrote some data without regards to that you'd run into a sequence you'd be unable to read later, due to not having a clock signal to figure out where a bit ends and the next begins.
Modern hard disks require using special encodings and servo data in order to be readable at those densities. It just makes little sense to me to want to bypass it. Unless you're working for a disk manufacturer this just seems very odd, but in that case you'd have access to the required equipment and information.
I'm actually more interested in why the fuck he wants to do this, much less how it would be achieved.
The person who would probably know for sure if it is even possible to achieve that level of control is Steve Gibson from grc.com, the author of Spinrite http://www.grc.com/sr/spinrite.htm. Spinrite has some amazing low level hard drive access, that it uses for data recovery and drive maintenance.
dd only bypasses the filesystem. It doesn't override the HDD firmware, which has to avoid bad sectors and write ECC information to the platter as well.
I would love to hear why the submitter thinks he needs to write directly to the platter.
Can you tell us what it's for, or is it secret? I assume whatever you're up to involves the destruction of the HD in question?
If you can find a working *very old* IDE drive (back from the era when they had bad blocks printed on the label that you had to map manually) I'd guess those use purely physical sector / block mapping.
somehow I doubt whoever is running this project hasn't heard of something called ROI. however if you have money and time to waste some options include: 1) find a better solution to meet your requirement - i.e. think "what is the end result I need to achieve" before thinking of the technical solution "how am I going to achieve this" 2) partnership with hard disk controller manufacturer 3) reverse engineer firwmare on a hard disk controller and remove any error correction code/etc (and take the data unreliability that comes with that) + create your code to actually write and read the disk. In addition this will lock your product to a particular model of a particular brand of hard disk. anyway if you ever succeed let us know how it was done.
The stuff Spinrite does stopped working 15 years ago. On modern drives you're pretty much doomed. Considering Gibson has been known to rip off SYN cookies and spout such nonsense as "Windows RAW sockets are going to destroy the internet", I wouldn't go anywhere near him for advice.
No you don't. Rethink what it is that you're actually trying to achieve and try again.
were you expecting to see a sig here? perhaps you'd rather see the inside of an ambulance!
Got myself a disk editor, and wrote something on the very first sector.
After that I can't use the HD anymore.
The computer can't even recognize it anymore.
Muchas Gracias, Señor Edward Snowden !
Short answer: no, it's not possible to do that. It's not impossible, simply incredibly hard to do. Unless you happen to be very handy with a soldering iron and go dumpster-diving in the backyard of an HDD company (as many others have pointed out).
Longer answer: how about creating a 'virtual' hard drive? There are utilities out there that probably let you create (a) a virtual interface (let's say IDE) and (b) a virtual hard drive, attached to said interface. The next step, of course, is to hack these (VirtualBox?) in order to make them do whatever it is you want them to do. This being said, I strongly suspect it involves non-trivial virtual machine hacking and that it probably does not respond to whatever your needs are.
Other than this (very twisted) idea, sorry, bare metal writing has been disabled a long time ago, and for good reasons, too.
The right to offend is far more important than the right not to be offended. (Rowan Atkinson)
Use a linear tape drive, you'll be able to control exactly where the ones and zeroes are on the tape. ...Oh, platter? Fuck that.
Old MFM hard drive should do the work. The entire device was controlled via software in dos. I have one lying around. A whopping 5 MB!
http://en.wikipedia.org/wiki/Modified_Frequency_Modulation
Now all you need is a 8 bit ISA slot and you are set.
RES PUBLICA NON DOMINETUR
The cynic in me wonders if he is working for some hack start-up company trying to develop a DRM scheme for a hard drive, similar to the crap the studios do to DVD's to make them difficult to rip. Typically this involves creating some sort of currupt/invalid area on the disk that your particular device will know to ignore, but will lock up anyone else trying to read the data off the drive.
The optimist in me wonders if he is trying to defeat such a scheme.
It can not be done with off-the shelf equipment and software for many reasons, some of which are:
Line Codes
Error Correcting Codes
PRML
SMART
LBA
etc.
Specialist Mac support for creative pros, Melbourne
No it doesn't.
If I read it correctly (it was a couple of years ago) all Spinrite does is move the head to different tracks before trying to read the data. The theory is that a seek from different distances might align the drive head slightly differently to the data.
As far as I know there's no way to position drive heads directly via IDE/SATA, much less write individual bits. IDE/SATA are high level protocols, all the low-level logic is inside the drive controller.
No sig today...
>Any good utilities out there to do this?
Yes. My solution involves a screwdriver and a sharpie. And you can write bits of information to the bare metal, like "this drive is broken, dont waste your time trying to fix it".
My colleague was not happy finding the drive on his desk and told me its a low level joke. Well, yes.
Remove the control hardware. Build your own.
If that's beyond your skill then you're attempting to do a project beyond your skillset.
Use an old drive and take clue from
http://en.wikipedia.org/wiki/Cylinder-head-sector
...but I tell you only when you tell us what do you want to achieve.
I guess you want to use modern drive technology to write fine magnetic structures to a plate. Probably producing the equivalent of a magnetic image or a hologram. What you need to do is keep the disk's platter, motor, head, etc. and dump all the electronics. Draw a line on the disk, radial to produce an index and use an optical sensor to read it. Drive the disk at a reasonable speed and then draw your patterns one scan line at a time, using the optical as a trigger to your circuit. You can step inwards from the edge after you are sure that each line is drawn correctly, perhaps with a second head linked to an o.s.
Have you considered the difference between previous tech and perpendicular recording? I suggest perp. is better for you.
We don't need to work like that any more.
I guess the dd utility is not enough for your needs, otherwise you would have already tried it.
Right?
I was just thinking that the submitter might be an idiot. Then I read your post. You're definitely an idiot.
start with a drive less than 528 mb
Get yourself an old, totally unintelligent S412/506 MFM controller from out of an IBM PC or PC/XT. These were fairly dumb devices (g=c800:5 in debug, anyone?) for which you were meant to enter a "bad sectors list" printed on the front of the drive after performing the low-level format yourself. The earlier the drive, the closer to bare metal, so you might want to look for any of the 2, 5, or 10MB (yes, MB) full-height (2 x 5.25" drive bays stacked on top of one another) drives that were floating around then. You'll also want to get yourself a set of ribbon cables.
You should be able to use a drive/controller combination like this with any machine with ISA bus slots up through about the 386/486 era, and that would let you also go back and grab an early Linux distro (say, kernel 1.2.13 days, like Slackware 3 or so) that included drivers for such a controller that were actually in use and known to work at the time, giving you a base on which to build more code.
If 10MB is too small, you might just have luck going up to the largest of the MFM (80MB) or even RLL drives (160-200MB, just get an RLL controller instead) drives. I don't remember whether there were any ESDI drives back in the day that didn't remap their own sectors, but if there were, these controllers were 16-bit ISA and somewhat smarter (also with Linux drivers from the period available) and went up to 680MB or so.
But if you're looking for the best chance of success for your purposes and don't need tons of storage, my educated guess would be that the MFM controller out of an IBM 5150 PC plugged into a 5MB ST506 hard drive and connected to a SIMM-based 80386DX mainboard with 8 SIMM slots (for 8MB ram) might be the easiest combo to find and get working in practical terms that has a chance of doing what you want.
STOP . AMERICA . NOW
yup
how about taking the hard drive apart and then using a pen to write 1s or 0s exactly where you want them on the platter. To control exactly where these are written you might need some kind of clamp and robot arm but you should be able to get these fairly simply and use open source software to control it
I don't think what you want to do is possible, because I don't think modern drives actually write the 1s and 0s in the way that earlier ones did. They use Viterbi coding, which means that the exact change to the magnetic field to encode a 1 or 0 depends on what the preceding pattern of 1s and 0s is. They literally can only write sectors at a time.
Then again, what is meant by a sector has changed with time with different servo technologies. With the embedded servos in current drives, you would need to know a lot about the particular geometry of the particular drive you are using.
And to do any of this, you would have to bypass the controller completely - which, I am pretty certain, cannot be done without serious hardware modifications
Consciousness is an illusion caused by an excess of self consciousness.
I'm working on a project that requires writing protons to an atom nucleus in a completely controlled fashion. I need to be able to control exactly where protons will appear physically on the nucleus. Normally when atoms are created during fusion the actual atoms that get created are determined by the energy input of the fusion reaction as modified by whatever kind of atoms are being fused (e.g. hydrogen into helium). All of the modern innovations in nucleosynthesis and alchemy are great for particle colliders and crackpots from the Middle Ages, but they are making my particular task quite daunting. My question for Slashdot: is there a way to get down to the 'bare metal' and transmute lead into gold in a cost-effective manner? Any good utilities out there to do this? Obviously a free and open source solution would be preferable, but I'm open to anything at this point.
Finally had enough. Come see us over at https://soylentnews.org/
Hmm, I was thinking of how Mebroot installs itself in the MBR of the drive. Maybe looking at that code might help you. :)
However, I don't think one has to deal with firmware. I think writing a simple filesystem that writes or seeks sequentially only might be a better way of dealing with this issue. I think there are a couple such file systems for NAND flash that pretty much work this way. Then, you can put all your directory structures at one end and all files on the other end of the drives.
There are several abstraction layers below HDD sectors.
You know how data restoration companies do it?
They take out the spindle with the platters, and put it in their own reading device with its own controller. And with that you can read and write the exact bits (as long as quantum physics allow it). But the head has to be compatible (e.g. perpendicular recording needs entirely different heads).
I bet those devices can be bought, and I bet their controller is actually just software on the computer (for flexibility). I also bet they come with different head configurations.
But they are definitely not going to be cheap.
Hey, at least it is a real solution. :)
Any sufficiently advanced intelligence is indistinguishable from stupidity.
"I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion."
Are you sure?
The reason I ask is I'm working on a project that requires me to move data faster than light. At least that's what I spent last Monday working up the math to prove that data replication between our different data centers has an upper bound enforced by the fabric of the universe and that it was impossible for me to achieve the project's stated goals without essentially inventing warp drive. As it turns out after a meeting it was determined that the goal was just a stated guideline. It also turns out the price of faster data transfer rates is prohibitive and after a further meeting the stated project goal was total baloney. Yes. Baloney. We had sandwiches. It was a nice meeting.
[signature]
hard disk art :)
Maybe a 1/32 pic of Fatty Arbuckle?
The man can write bits on any hard drive platter just by glancing at it menacingly.
This was easy to do on the C64. Commodore is still waiting for the mainstream to catch up.
Don't you hate it when people refuse to accept the premise of a technical question and write long monologues why the submitter is working with false assumptions even though they don't know what exactly they are dealing with? Yeah, me too. Makes them look arrogant, ignorant and smug. I'm going to go ahead though and reject the submitter's premise: there is no chance in hell that you're on the right track with whatever project you're attempting to do. But instead of merely dissing you for incompetence, I'll lay out a few scenarios (might as well, since you didn't supply any of your own).
If the actual physical bits matter to you...
you're either a hard drive manufacturer or a clueless person who should read up on how drives actually work. And we both know you're not working for a manufacturer. What you need to know is that there are several layers of indirection between the write call from within an OS down to the actual magnetic platter. These layers are there for a reason. At the very least, the onboard controller of the drive abstracts away the physical block allocation, and the drive won't work without the controller at all. Since the intricacies of the drive's physical address space are not accessible from the outside, there will never never never be a reason to try and fiddle with it directly. Because you can't.
If you are looking for disk I/O without a filesystem...
we're finally in saner territory. There are valid reasons to do this, e.g. speed and overhead considerations. Some database vendors actually have features like these. In this scenario, you're using the entire drive as one big addressable blob. A good starting point would be to have a look at the source code of a simple filesystem, such as ext2. Strip away all the actual file handling stuff and learn what you can from the disk I/O routines. On the other hand, if you didn't arrive at this conclusion yourself, that's not a very encouraging sign.
If you simply want a drive without error correction...
you're not developing software that will run on any modern system. If you accept this caveat, you can buy an ancient drive off ebay and use that. However, keep the first scenario firmly in mind: there is simply no reason to control the exact placement of every single byte if you don't plan on literally putting the drive under a microscope afterwards. Otherwise, this has no practical implications and, again, you are on the wrong track.
If you're a DRM/malware/virus developer...
I will sleep very comfortably tonight, because you had to ask about this on Slashdot, signaling once more that you're doing it wrong.
The stuff Spinrite does stopped working 15 years ago.
What, bullshit? That still works just fine, most of the time.
No less then *7* posts to the SAME XKCD COMIC?!? Christ, bad enough people don't RTFA, at LEAST read the fine comments!
"Gratuitous complexity is akin to chaos" - True Vox
They won't understand what you're asking and won't be able to help in the slightest, but please record the call for us.
No, not in the slightest.
Some operating systems have volume management tools that do let you get a great deal more specific about where data is written. Normally, I'm a Linux or Wintel guy, but I have managed some AIX boxes a few times and from setting up a large one once I recall that it's logical volume manager allowed you to create volumes on the SCSI drive systems that were very specific about how and what part of the drive was used. You could specific, for example, that a particular volume use only the outer tracks (or inner or middle I suppose) of the drives -- in addition to a great many options for raid striping and transaction logs and so on.
The idea is that the outer edges of the platters travel faster under the read/write heads than the inner and so performed differently. Also, you could keep the head from having to jump around as much by keeping all the data that tended to be used at the same time on the same tracks -- reducing your average seek time when reading randomly from that pool of data.
In practice, I think this mattered a great deal more when us old guys were dealing with 80+ millisecond random seek times on 5 1/4" wide full height (what would now be considered two bays) drives -- or larger disk-pack based drives (aka washing machines) with the massive physical movement necessary for those read/write heads.
Today, I think the admin is better focused on distributing the data load better across different drives/arrays to even the load out and also focused on reducing overall disk i/o in creating database schema and applications. Focusing on very fine details like this is likely to have only marginal benefit next to those key areas --- but I can't presume to know for sure what this project has in mind.
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
I suppose it MIGHT be possible to jigger some drives to format a cylinder in some specific weird format, so it contains information you can only get at if you use a 666 byte sector size, but even this kind of thing hasn't really been possible on hard drives any time... I don't think even ST506 MFM and RLL controllers let you do much more than jigger some encoding of the sector and the sector size, otherwise you'd have been able to upgrade an MFM controller to RLL in software.
Heck, the ability of the Apple-II and Amiga *floppy* controllers to play clever games at the bit/nybble/byte level was unusual.
And given that the most common use of that kind of shenanigans was for copy protection, the speculation this fellow is looking for some help in coming up with a hard DRM scheme seems spot on.
Hopefully it's some socially useful purpose like a magnetic scaffolding for nano-engineering such as DNA/drug synthesis, building fuzzy/nano structures (batteries, drugs, whatever) at the areal densities of modern drives. I guess you'd figure out how to distribute the bits, drop the medium onto the platter, and go from there. Probably easier technologies to futz with, but if this is the goal you could probably accomplish a testbed with a hardware/firmware hack, worst case an FPGA etc., but they'd be better off sharing the wealth by partnering with an entity that does this for a living, like a drive manufacturer, or maybe a data recovery house.
My first hard drive was a 40 MB Seagate ST-251 MFM hard drive. I almost got the ST-225 when I was buying the parts for my first IBM-compat from JDR Microdevices.
I'm still very thankful that I didn't go with my alternate plan of getting the Lt. Kernel 20 MB hard drive for my Commodore 128. I imagine I wouldn't quite have the career I have today had I gone in that direction.
I'm a big tall mofo.
So, do you need to write to a specific sector? Or do you have to position the data in a specific *place* on the disk (eg 1 cm from center, 20 rads)? If it's the first, you only need to bypass the filesystem, so just fopen "/dev/sdX" under Linux, or use CreateFile "\\.\PhysicalDriveX" on Windows, seek to the position you want, and write away. For the latter, you need to bypass the drive's logic, so you will have to either re-write the firmware, or implement your own controller board.
What about doing it the other way around? You could study how the filesystem works with one specific set of hardware, and write specific data in specific milliseconds in specific places to force the hard disk to write the 1s and 0s exactly were you want them. You would be very limited, but it could work.
4 - A robot may not masturbate, except where such action would conflict with the Second Law.
Randall Munroe
http://outcampaign.org/
The only reasonable.... NOT.... way to do it in software would be to implement the inverse (you know, ^{-1}) of the disk driver in sw and then just trick the hardware: generating "proper" data such that, after the plethora of bit crunching done by the hard disk driver, you get what you wanted in the first place.
In short, sounds like big trouble. You're going to need plenty of legal advice before this thing is over. As your attorney, I advise you to rent a very fast car with no top. And you'll need the cocaine. Tape recorder for special music. Acapulco shirts.
I'm working on a project that will require me to hire someone experienced in hard disk drive physical systems, encoding and controller firmware. I'm posting a question to Slashdot that resembles the kind of idiotic question that typically comes from a pointy-haired boss so I can offer the job to the person who posts the most rational, knowledgeable and nonconfrontational answer.
It is possible that he wants to achieve something similar to Burning visible images onto CD-Rs with data (beta)
You will need a reasonably sized HD, like this one where ones and zeros occupy a space of about postage stamp size.
There's a reason for things like error correction that you're trying to omit. The primary benefit of omitting them is that you will never be able to retrieve your data.
But disk manufactures won't expose it to any one (no without the proper leverage)
I know that cause use use those api's to simulate bad sectors for testing our embedded system video capability.
go ahead and talk with some of the manufactures, you'll get the best answer from them.
Yes, there are ways to do this. I'd start by taking the SANS data recovery class taught by Scott Moulton.
His website has a lot of info, too:
http://www.myharddrivedied.com/
If you'd like to write 1s and 0s will physically on the platter, just borrow Mr. Slate's dictation bird.
The first XKCD reply (the second reply to the story, and the first of many) was more than 2 hours ago. Congratulations, you fail at internets
I'm surprised I haven't see this yet: http://xkcd.com/378/
First of all, you don't write "ones" and "zeroes" on a surface. The only thing you can write are flux transitions, changes from north to south magnetic polarity and back. In order to be able to write and then read them back you also have to add clocking transitions and DC-restoration transitions. It takes a fair amount of sophisticated analog and digital electronics to do the writing and reading. The digital parts are semi-programmable, but the analog parts are not-- you're stuck with writing and reading the type of flux transitions that the analog electronics is optimized for.
I can't imagine why you think you *have to* be able to write a particular pattern. Unless you're going to use the platter to test out some kind of magnetic imaging camera.
I would love to hear why the submitter thinks he needs to write directly to the platter.
I bet he wants access to the magical free space he hears is on every drive but is invisible because of the evil error correction schemes.
if you describe your requirements so abstractly? It sounds to me like you've come up with a solution ("Let's write the bits directly to the hard disk") and now you want pointers on how to make it work without explaining why.
If you just want to get around the file system, then use raw block I/O to the device. However it *sounds* to me like for some reason you've decided the *actual physical* layout of bits on the drive is important, in which case you are going to have to write your own disk driver -- if the drive electronics themselves don't defeat your attempt to know where bit is physically written.
It's inconceivable to me that you'd actually need to know this. Since you only *read* the data through an interface, it should make no difference if you *write* though the same interface, as long as it's consistent. In other words, unless you are going to disassemble the drive and examine it with an atomic force microscope, you have no way of telling the difference between a physical layout and an equivalent *model* of a "physical layout".
If you can't say *why* you need to do this, at least explain the parameters (how much storage, how fast, what kind of retreival etc.). My first reaction was that you should not use a magnetic disk at all, but an MTD flash device. Even so, you're dealing with an abstraction. You have no idea whether the device itself has mapped a bad bit to a different location at the hardware level. However short of tearing the device apart and putting the flash chips in a special circuit, you'd have no way of telling.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
1. Screw open harddisk, remove parts until platters are visible
2. Take felt tipped pen
3. Write 1's and 0's.
Sound good?
Why is there so much ill-feeling towards the poster? Technically, what he's asking *must* be doable--though probably very hard--so why not admit that you either don't know how to help him (which is probably true for most people), or point him in the right direction, if you can? Or, if you know for a fact that it can't be done, just calmly explain why.
There are two ways;
I would advise you to pick up a copy of FreeDos. Dos gives you less trouble accessing the hardware than Windows, Linux, MacOS X or fill in nearly every other big modern OS.
Then you can simply use C++ to access BIOS calls to access the harddrive, or write some code that accesses your diskcontroller directly. I do think you probably need to code yourself, or get someone to do it for you.
This would give you the most flexibility.
An other option is to use linux and a simple utility called dd, you can simply use that to write a binary-file directly to a certain sector or number of sectors. For example; dd if=sample.bin of=/dev/hdb would write a file called sample.bin to the second harddisk starting from the very first sector.
I think I've actually done something like this.
Back in the day, I worked on a Commodore 128, and an external 1581 disk drive.
It was really cool, the disk drive had it's own operating system, different from the computer.
you would form special packets on a floppy disk, which the drive would load and execute as code in the disk drive's buffers.
So you could actually "add" commands to the operating system of the disk drive.
Was a common copy protection scheme back then.
To save something to the disk at a certain spot, you could determine the track and sector you want to write to,
then read that sector, modify the bits you want to change,
then write that sector to one of the disk drive's buffers, and send a command to write sector to disk, giving it the
track, sector, and the number of the buffer to write from.
You had to write your own software to write to the disk drive, I think i used assembly language for that.
Anyway, a program to do this now might be difficult, but if you step back to DOS operating system and an old drive, need not be mfm,
you can prove it works by using good old DEBUG. With DEBUG you can read a specific track and sector from disk into a memory location,
then modify it, write it back to disk, one sector at a time. But it is tedious to do for a big project.
Or get a DOS track and sector editor and do your tests in that to get it working.
There are millions of tracks and sectors on a single disk--this would be too slow for a real project,
but you can get a proof that it works going, and take it to someone who now can see what you are doing and write a
small executable that can be scripted, or write a new program that will do the whole thing.
As an aside, because of the nature of the disk drive, you are still limited in writing to existing tracks and sectors, there is not a
way to write to a specific bit at a polar coordinate. Firmware won't take that command. you have to have some sort of translation
table in your program too, as the the tracks are larger at the periphery of the disk, and have more sectors per circle (track) than they do
closer to the middle.
You have to know about the physical layout of the platter to convert this to a translation table of tracks and sectors that your disc drive
can write to.
Consider storing the data instead in a disk file as polar coordinates: value, angle, distance.
That is information you have to have anyway to decide where to write it on the disk...
If you can tell us more about it, we can help better.
First - I should say I'm not an expert on everything that can be done with the newer logical volume management tools in the Linux distros...I used them rarely.
That said, this isn't quite the same thing as most people can accomplish with Linux on a SATA drive even with the LVM. You can, so far as I know, specify which blocks, numerically, to use for a partition. There is not (again, to the best of my knowledge) a direct mapping from that block order to the actual tracks and sectors on the physical platter. There is mapping that happens between the drive and the controller (which is on the drive on all modern sata and pata drives) and there is mapping between the system bios and the controller (to account for limitations in the original head/cylinder counts available on the original AT drive specs). I don't know how much effort is made by the drive manufacturers to match what they report as sector order to physical location given that seek times have gotten so low over the years.
I don't think specifying the block choice used on an sata drive does anything even close to what I was describing.
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
Since the absolute addressing of physical mapping is handled by the drive controller I doubt a manufacturer is going to allow anyone to get direct control of the hard disk.
The issue is if a user space application could do so they could write persistent viruses into the sector reallocation reserve space and no OS could remove that data unless the OS also had access to that space and\or a sector gets reallocated into that space wiping the content.
In addition due to rotational issues you would need to first pre-determine an arbitarary starting point on each platter, write a pattern to identify that and build a custom sector mapping. You have to remember that due to sector remapping sequentially mapped sectors may not in fact be "next" to one another on the hard disk.
You are trying to pin the tail on a doney travelling at over 7200 RPM which is no small feat.
You could try building your own custom firmware for the disk controller but I am certain that disk manufactures have specialized firmware and disk options for, shall we say, less then normal buyers.
-=[ Who Is John Galt? ]=-
of course this way you can only work with bytes instead of bits, but that's just a mathematical problem, you can solve this with some bit-shifting.
or something like that...
The MAFIAA is a bunch of mindless jerks who will be the first up against the wall when the revolution comes
Why is it every time I read an article that is either A: Horribly Biased or B: Idiotic in Premise it is always posted by KDAWSON?
I can understand trolls and trolling in comments, but seriously Taco, from your editors?
-=[ Who Is John Galt? ]=-
If you wrote your own filesystem or other record delimiting scheme and you could interface the drive normally at the block layer. Sure the controller might map blocks elsewhere but anytime you read the drive its going to reply with the same block from the same "logical" location, even if you move it to another machine.
So for all practical use, what difference does it make? Unless you were planning to actually remove the platters from the drive and install them in a different drive later, then you have a problem but if that is the case I think you should take a few steps back and reconsider your process..
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
1. Just get any disk. ...
2. Format with a FS that allows filesizes as big as the drive.
3. Create a contiguous flat file on the drive.
4.
5. Realize you were trying to reinvent a wheel that took >40 years to make in the first place?
Sound like an interesting way to build a cryptographic system.
With the technologies currently used for self-timing data writes on hard drives, it's likely that no hard drive spindle motor will maintain a perfect rotation rate, making your job even more difficult.
If it doesn't have to be a hard disk, I might start with a floppy disk, even just for practice. Really what you need to do is control a stepper motor from code in memory. It doesn't even need to be on a separate disk controller. Once you can control the motor, writing the bits is easy. It sounds like a project we did for assembly language class back in the day. First learn how to program on the 68000, then control a stepper motor, screen, and other peripherals, writing/reading bits here and there, prove it works. After reading other comments, it sounds like most of the schools out there don't do projects like that anymore. At the time, for fun, I even controlled it at home without the PIC and assembly language. It is quite easy to control motors through the parallel port since there are so many data ports. It just a matter of using C to get that done. You can then control the steps of the motor as well as the head with keys on the keyboard. Manually at first making individual movements to prove it works. Then automated/programmed as much as you want later. --you didn't say it had to be marketable or fast or work with current hardware... Really get started with something small and work up from there. We don't know what you are capable of with the limited information you provided us.
without using very old hardware or building your own hard drive controller.
Have fun looking for a new job.
Just use programmed I/O and LBA addressing.
Modern hard drives once "blanked", which can be as simple as cat /dev/zero > /dev/sda || /dev/hda ||...etc, contain nothing but an embedded servo track.
SO, if I write to LBA sector "0", physically, the bits will be on LBA sector zero, which is a deterministic physical location on the actual platter, determined by the servo track.
Disabling features like automatic relocation is also easy, all modern drives have that functionality.
This is the lowest level you can go. It disables automatic relocation and defect remapping, it has no file system overhead, no OS overhead, nothing. Nothing but pure and simple 512 byte sectors [ for drives lower than 1TB], and the drives automatic ECC.
You can arbitrarily write/read any physical LBA sector and the drive will stupidly do what you ask of it.
What kind of "Project" would require this kind of "Bit-Control"? It least his post got kidcharles' name in the spotlight and "15 seconds of fame" :-)
I am 111010 years old this year!
Remember, 1 + 1 = 10.
Ok, Maybe I missed something (not unusual) but is not the question about cooked space (file system i.e. ext2, fat, ntfs.....so on and so forth) versus raw space (many databases write their own proprietary format)? Oracle does it, so do other open source databases I believe. Or did my train for thought derail?
http://www.google.com/#hl=en&q=cooked+space+vs+raw+space+database
No hour on a horse is ever wasted. Winston Churchill
Not only does the ST506 interface separate control and data into separate systems (two cables, with actual wires), but the ST506 drive, servo, control, etc. is really easy to work with: very slow circuitry, socketed DIPs, big pins, actual holes through board, lots of wires leading between and to things like the stepper and spindle motors, and you can get at everything (including entire drive disassembly down to magnets and platters) with little more than a computer screwdriver set with a flat head, a philips head, some small torx bits, etc.
So once you've got a working OS + software + hardware platform up, you also have the freedom with that generation hardware to "edit" the drive in any number of ways by adding your own electronics, etc., with little more than a soldering iron and a leftover box of 74-series logic, if you want to get that basic, and since mainboards of that era have at least two serial ports built in, well...
Probably given your project you didn't need me to say all this extra, but in case you're not old enough to have ever *seen* an ST506 era hard drive or controller, I thought I'd mention it.
STOP . AMERICA . NOW
http://sites.google.com/site/seagatefix/
http://forum.hddguru.com/seagate-terminal-commands-t6411.html
Some seagate drives have a serial port that you can mess with the drive controller directly. If you can do what you want, I dunno.
The LightScribe feature on some CD & DVD disks/drives appears to be close to what you want to achieve, i.e. it writes 'bits' that create a visible graphics design ("bits anywhere & everywhere"), without regard to what the bits mean as data. So if you can find the tricks behind that, you might be able to find someone who can implement a similar scheme in a HD.
I'm not saying that you're right, but it's definitely the case that the tools his company offers, and the explanations about how they work, are unbelievably amazing. Either they're truly brilliant, or the man's a fraud. I don't have the insight to tell which is the case.
So I don't use his software.
"Good news, everyone!"
I'm working on a project that need to deliver last year. Does anyone know of a free time machine or at least a open api whereupon I can build one. I don't mind if it takes a long time to complete as long as success is guaranteed.
Thanks in advance.
<quote><p>Hahaha how cute. This high school student thinks he knows something. And smug as we</p></quote>
Got nothing better to do than start a flame?
I tried to give an answer, maybe as stupid as it could be, but better than your for sure.
Why don't all of the morons who answer with such stupid sentences cut their veins and RIP?
The world would be a better place, for sure.
The GP is right, Spinrite does not and can not do most of what it claims on a modern HD. The only potentially useful thing it does is re-read sectors up to hundreds of times, but there is safer software that does the same thing.
More importantly, Spinrite can be downright dangerous to your data. Spinrite writes any data it recovers TO THE SAME DRIVE! It can very easily overwrite other data you want to keep and/or lose the recovered data again because the drive isn't stable enough to hold it reliably. Not writing to the damaged media is data recovery 101 and Spinrite fails badly.
Hahaha how cute. This college student knows even less than the high school student he is making fun of and so resorts to mudslinging. And smug as we
On a related note, anyone know of a utility that will let us visualize what the disk is doing? (Other than those disk drives with little windows over the read heads, that is :P )
I'm sort of looking for something like one of the old DOS/Win3.11 defragger (maybe it came with Stacker?) that showed the layout of used blocks and which ones the defragger was reading and writing to put everything back in order again. I suppose the current Windows defrag utility still sort of works like this, but it's not quite as interesting to watch.
I'd love to be able to tap into the Linux kernel and just watch the read and write patterns as the system is working. Might also be marginally useful for seeing where different types of commonly-accessed files sit on the platter, maybe it could help rearrange them to optimize readahead buffer hits. I'm thinking of hacking together my own thing using strace and fs block lookups, but just wanted to check if there was something that already did most of this.
http://www.trademe.co.nz/Electronics-photography/Other/auction-36725938.htm
If you send a million dollars to the Swiss bank account I name I will guarantee your success. I will want to remain anonymous since I am you from the future and I can't interact with you too much or I risk altering my own time line. Simply place the million dollars in the account and you will thank me later.
It's not a possibility: your primary limitation isn't the filesystem, it's the drive itself.
Modern drives are, for all intents and purposes, black boxes. They do much the same kind of thing internally as an SSD does, but present the data (and drive) to the OS in the more traditional fashions through geometry and the like. The 'beginning' of the drive may not be, as indicated by tests on read/right speed across the platters being reasonably consistent (and many drives being composed of multiple platters). LBA errors result in the internal remapping of 'secret' good sectors to those previously-good but now-damaged sectors. There's precious little way to know exactly what is where on a disk.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
Haha, so I was right, but then again, smug high school students who recently installed their first Linux distribution on the family computer (mom made me keep Windoze though... hate her so much! M$!) probably encapsulate a good 75% of all readers of Slashdot. Tell me, do you often give advice when you have no idea what you are talking about? I bet you are THAT guy who sits at the front in each lecture and lets the teacher know of any (no matter how minor or obvious) error he or she does.
This site would be 10x better if people just you stopped posting pointless comments, maybe then browsing the comment sections would be reasonably enjoyable. Fortunately, I made sure you got your deserved Flamebait moderation, but probably not for long. So many basement dwelling smug idiots here... so few moderation points.
You are also obviously an American (I'm sorry), so therefore very likely fat. I bet you also watch Anime (sorry, Animu!). Instead of trying to memorize UNIX commands, how about hitting the treadmill once in a while? Just a friendly tip from this AC.
... just do it in software. Meaning, code up a drive interface in C/C++ ... make it look like system calls to a physical drive, but behind the scenes said code would just write to a big pre-allocated 'drive file' sitting in your comp's filesystem.
I'm sure there's a bunch of ways to implement this, and problems to solve for sure. But it's probably less effort then overriding the error correction, etc, of modern HD's.
Huh?
Why would you need to do that?
As others have said, your choices are few:
A) find ancient hardware that will actually let you do it
B) create your own hardware
C) write to a disk image instead and emulate the entire thing in software
Obviously C is going to be the easiest, most portable, but least efficient of the three. Without knowing why you need to do this, though, or what exactly you need to do, I’m thinking C is the best.
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
When someone says they want complete control over the locations of bits on a drive that could be taken to mean the actual magnetic domains (which aren't simply digital values anyway) or it could just mean the logical digital values that are seen by the OS and filesystem drivers. I wonder if by using phrases like 'magnetic hard drive platter' and mentioning Reed-Solomon encoding, kidcharles has made people think of the lowest level meaning of 'location of bits', whereas in actual fact he just wants something like /dev/sda.
'A free and open source solution' is mentioned.. but theres no mention of what form this solution might take. In the case of a custom HDD controller (which people have suggested as a solution) this would mean something like a VHDL or Verilog implementation (but it would be highly vendor and model specific, plus there'd probably be IP issues). But when people say 'a free and open source solution' they tend to mean software running on a common desktop OS. Oh and I guess there's the question about 'any good utilities' which suggests that kidcharles isn't after an entirely programmatic approach.
So, if I'm correct, any decent disk editor will supply the required level of control. I can't be bothered to look any up (Google will help out here) but I know there are plenty (open source, free or proprietry) for Linux, DOS and Windows. Hell, you could possibly do it all with shell scripting under Linux if you so desired :-)
Alternatively, maybe kidcharles wants to make spirograph like artwork for creatures that can see in the magnetic spectrum.
Would it be possible to create an emulated disk layer which would handle the request?
III.IIVIVIXIIVIVIIIVVIIIIXVIIIXIIIIIIIIVIIIIVVIII
Not using commonly available hardware. The disks sold today contain small computers that take your requests for specific sectors and translate them for maximum reliability and efficiency. This frees your main processor from doing that work and reserves your I/O bandwidth for data; it's more efficient.
I did what you are talking about in the early 1980s using a PDP-11/34 computer and RL01 and RL02 disk drives. That system is still running today in an aerospace application, but I would never write something like that today. It's no longer an optimal use of resources.
I don't know of any utilities specifically designed to re-write the firmware on hard drives with custom code, but the firmwares are definitely writable, so you should be able to do it if you are motivated enough. I'd start by googling it, personally.
You might also want to look at the various linux bios projects out there; those guys are re-writing the firmware on their motherboards, which is a somewhat similar process.
Maybe the author want to make "art" using a light scribe like method... I guess that his guess is that a bunch of "1" could be visible somehow.
The stuff Spinrite does stopped working 15 years ago.
Don't tell that to the drive I used Spinrite to recover last night...
He's trying to hide his child pornography.
Every floppy and disk drive I can recall (even back to 8-inch floppy days) uses some type of controller. The controller translates your track and sector requests into stepping motions of the write arm(s) on the drive. Once the controller indicates that the head is positioned over your requested track, there is a latency while you wait for the "index" mark to appear which indicates the start of the track data. At that point you supply special codes to tell the controller that you are starting a new sector, the sector number, sector size, etc, after which you send the data. All of the sector info and data are encoded automatically by the controller and combined (at least in MFM encoding) into a bit stream. My belief is that attempting to work out all the combinations needed to get exactly the series of 1's and 0's you want is probably not a practical solution.
About the only solution I can think of that would allow you to do this is to actually hack into the signal supplied to the recording head at the end of the write arm. That means that you will need an oscilliscope to locate the signal and determine the voltage and duration required to write 1's and 0's. You will also need to understand whatever mechanism the drive uses to detect the index mark (the start of a track). If the drive does not use index marks, you won't be able to align the bits on one track in relationship to the bits on any other track. Assuming you manage to obtain all this info, you will then need to cut the drive line from the controller to the head and splice in your own drive line that will supply the raw write signals.
My guess is that this will not be a practical method for achieving whatever it is you are aiming for, in which case I agree with the previous poster that recommended the use of a XY table driving a custom write head over a custom magnetic surface.
On the other hand, you didn't specify that you need to be able to *read* any of the bits you want to lay down, so why not just hand in any old drive and tell them it was a piece of cake and let them worry about how to verify your work :-)
On a more practical note, if what you want is to lay down an exact pattern of bits onto a magnetic surface, the easiest path might be to use a cheap off-the-shelf laser printer. Simply create a 300 dpi image of the bit pattern and print it. You can them pick up the pattern off the printer drum.
The tools are fine. The descriptions are overblown.
Why not use a virtual drive? Write some software that emulates a hard drive in a flat file, and code it to allow you to write to specified locations on the "disk".
No idea what your trying to do... art, fast file system, error recovery, special new virus etc... but I'd start in one of two places. If you are trying to write your own filesystem because everyone else is "doing it wrong" then check out Oracles "unbreakable" installation.... a bit dated and I don't know if they still do it this way, but the gist is you mount the drives with "rawfs" and they have their own file system stack that runs on top. Pretty effective way to trim down the layers of stuff before reads and writes and speed up raw access. .... again, depends on your implementation if this would be useful to you.
If you actually want to write ones and zeros I'd check the source for fdisk and see what its doing.... might be a gem there since it clearly is aware of geometry and has some access to writing in places you normally wouldn't.
Good luck!
Good old Steve Gibson of Gibson Research (http://www.grc.com) used to know how to do this trick. He has a utility called SpinRite that allows the software to write and read in places on the disk where the disk intelligence will not allow.
Even the very oldest hard drives didn't have physical control of exactly where data was written. Oh, they laid out x sectors on y tracks, and you could maybe measure where track 0 was and where track y was on the platter itself (by marking the actual head position), and determine the physical location of those tracks.
But the starting sector? Purely arbitrary: CPU or disk controller said "now", there was sector 0, and all the other sectors followed (the length of sectors being determined by the number of sectors).
Now, you _can_ possibly locate where sector 0 begins (I visualize a tiny spray gun electronically controlled to fire when track 0, sector 0, bit 0 is read). But I don't envy you that job. (Ever try physically modifying the read/write heads of a hard drive? No, I didn't think so.)
Again, with the older drives (especially the washing machine sized drives back in The Day), you could do physical stuff. Hell, I had an old 25MB hard drive that wouldn't start sometimes. But it had a flywheel (really) accessible between the control board and the drive case, and I could reach in with the eraser end of a pencil and "spin it up". I called it "kickstarting the hard drive", used to amuse hell out of visitors).
None of the new highly compact hard drives will let you do that sort of physical stuff. And they certainly aren't accessible to external CPU control: way too much (as said above) is done on the controller board.
So good luck to you. I'd give it up if I were you.
Shouldn't this be possible with a plain old floppy drive? Sure, that's just 2 MB of storage (raw bits), but hey, there is a linux utility (superformat) with "example" code to get you started...
I'm surprised by the number of people who seem hostile to this question, mostly because they can't imagine any circumstances where they would want to do this. I can think of several. In fact, I was thinking about something similar with optical disks just earlier today. (I'm curious whether small holograms could be created by writing an interference pattern directly onto a CD or DVD, but I would need exact control over how the tracks lined up to achieve this.)
Here's some concepts just off the top of my head:
* He's come up with a new disk encoding scheme, and wants to try it out.
* He's doing research into how long disks retain data, and is questioning basic assumptions like whether surface bits permanently magnetize the platter underneath, affecting later bits in the same location. Or how far the domains spread.
* He's working with self-assembling molecules and needs to give them a patterned magnetic surface to build on.
* He wants to 'print' a 2-D picture onto a small portion of the hard drive, and bounce a laser off it. (The magnetic alignment of the surface domains would polarize the photons slightly.)
* He's making a high-resolution rotational position encoder, so by reading across 32 tracks knows the rotation of the platter down to a few nanoarcminutes.
* He wants to totally destroy the contents of a disk. (I assume this is what most of the hostile people think is the intention, presumably as the payload of some virus)
Those are just the ones off the top of my head.
However, the ranting people do have a point that without knowing WHY you want to do this, we can't really suggest the best solutions. A lot of people have recommended going back to super-old MFM hard drives that allowed this, but we don't know if you require the density of modern hard drives.
To do this with a modern drive, you're basically going to have to rebuild the controller. Either totally remove the controller board (leaving handy raw connections to the stepper motors and drive heads) or cut the connections between the microcontroller and the low-level electrical functions of the drive, and substitute your own. Here's where knowing your accuracy requirements could have help, because if you want relatively large bits, you can get away with fairly low-frequency components. A 20Mhz microcontroller can, with say an external high-speed shift register, push out an 80mbit/s serial stream, which equates to >120,000 bits around the track of a 6,000RPM drive. Not quite the same density as the manufacturer, but better than the old MFM drives.
Your next problem is going to be this: It's really, really hard to tell where the head is on a platter. I've no idea how modern drives do it, but it used to be done with 'marker' bits either in the track, or on a nearby 'index' track, plus a little timing. Preserving these location marker bits is actually the most important job of the drive controller.
It's not impossible, merely very difficult. I could probably make one with only several weeks of hard work.
Alternately, you could try getting into the firmware and re-writing it to your specifications, but that might take longer. You would have to reverse-engineer a lot of stuff that is specifically hardened against this, but at least the hardware would be stable.
Jeremy Lee | Orinoco
that wants to paint electromagnetic pictures of the Mona Lisa.
Back in the good old days, we had MFM drives. These things were huge stupid spindles that required a separate controller board. If you had one of those, you could design your own minimalist controller chip (as an FPGA or fast PIC), and access the MFM platters directly. That was, oh, 15-20 years ago. Today the controller is built right into the drive, that's that little board on the underside. Today's controllers are quite a bit more complex than the old ones, with much tighter tolerances to support modern bit densities. The short version is that if you're on Ask Slashdot, you do not have anywhere near the level of skill and topical expertise to design your own controller. Either find yourself a hard drive engineer, or hide your porn in the Recycle Bin like everyone else.
-Billco, Fnarg.com
This site seems to have the kind of information that the submitter needs.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
Find a drive that support the old ATA command "WRITE LONG" and "READ LONG". These commands write out the entire sector including the ECC PAD so a sector ends up being 512 bytes (arrange the bits in any order you want) and 8 bytes of ECC (again with Write long just write out 520 bytes of data any order you want). This won't actually put the bits and bytes on the drive exactly because the drive has randomize the data on the platter to ensure reliability, but it will give you more physical control of where the actual data is located on the drive.
In other words if you put the bits on the drive yourself in something remotely human readable you will have a high failure rate in reading them back correctly.
The only way around this is to follow the other suggestions and use a really old drive that can literally write the bits to the drive directly without any modern ECC, PRML or any other obstructive acronym.
use a felt pen (the permanent marker kind). but make sure you write legibly. you can then control exactly where the ones and zeros are written, and even control their shape...
Drives today have the ability to do that, problem is it's buried in code used by engineering and the manufacturing test process that isn't accessible to customers. I do it all the time. You wont be able to read it back, encoding and any error correction functions are gone, but the capability is there. Find an independent consultant in the drive industry for help.
Have you considered writing to a virtual disk file of some kind? You could create your own "disk" file and write to it any way you want. You just need to create your own virtual controller and disk format - though you could probably come up with some open source tools from something like Xen or Bochs to help you.
I don't know that this would be easier but it may not be harder and it could be free. If you do manage to get it to work this way you'll have the advantage of not being locked down to any specific hardware.
long ago programmed first adaptec products: harddisk controller chips
which looked to be pre-programmed microcontrollers. the datasheets
of these chips had the info about extra bits written on disk for the pll
sync. etc, see also the fd-1791 datasheets.
maybe analyse such old adaptec controller like this one:
Adaptec ABC4000 SASI->ST506, (2x Wren CDC 9415-86)
never seen opensource harddisk controller firmware despite rms.
http://xkcd.com/378/
Use emacs
hence your troubles.
There are a lot of comments to shift through, so I imagine someone else has already put it out there that the long and short of it is: not without custom hardware. Additionally, if you have the hardware to do something useful with the bits once they've been written, then you should have the means to perform the writes. If you're wanting to get exactly the bits that the drive controller thinks are in particular places without regard for OS caching and so on, then doing direct SCSI passthrough will get you as close to what you want as the drive will allow, though this still leaves ECC and you'll have to deal with sector boundaries (512b for the majority of drives, 520b seems to be mildly popular as well). As has been pointed out by many, knowing why you want to do this may lead to a better solution.
Unfortunately, modern hard-drives are more closed than the iPhone. It's practically impossible to get anything past the built-in controller, unless you're willing to open up your HD and do some hard hacking.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
I'll give the direct/short answer to the question you're asking.
No.
The reason is that hard drives only write data in precise locations so they can find it later. You can't write anywhere other than those locations because the drive won't do it... not even with new firmware. The read/write heads may not even be capable of addressing the locations you want to write. The only way to write in an arbitrary location is to remove the disks from the drive in a clean room, and use a very precise CNC read/write head to address the locations you want. Disk manufacturers have machines that do this.
If you confine your request to only writing arbitrary locations within the physically addressable areas of the disk, then you can do what you want if and only if you write new firmware for the drive... it used to be firmware was in EPROM and couldn't be altered, but you can flash it nowadays. So you A) Buy a drive then B) Re-flash the eprom with code to do what you want (custom developed after reverse engineering the original firmware) and then you can write wherever, without worrying about niceties like the end of sector marks the drive uses to keep track of data locations. You'll have to do that sort of thing yourself.
So, confining our discussion to disk areas that the default drive firmware will write to, If you're looking for a way to ignore/override the OS I/O code, the disk controller firmware and the disk firmware in order to "talk to the bare metal" it may be possible depending on the particular combination of OS, controller, and disk, and assuming you have the right privileges in the OS. But there's no standard way to do it, nor API. You have to know exactly how the hardware in question works, down to the chip level and in some cases below. If the computer you're interested in has a different drive, controller or OS I/O code than you wrote your program for, then you have to re-write it.
I'm sure a lot of us here would be curious to know what you're trying to do... accessing non data sectors on a disk hasn't been done commonly for years, and when it was it was used for some awful copy protection methods (awful as in they created compatibility issues, even with "standard" PC hardware and also they could still be broken).
Erik
It is actually quite easy, you need to redefine "bit" though to be whatever is the minimum chunk of data your controller will write to the disk. You can then pad out your data to be all 1's or 0's to fill up that minimum chuck. You can then invent your own parity/error system at whatever level you like. Sure you are not turning on/off individual atoms/whatever but this should achieve the granularity you are seeking, at the expense of capacity.
I bet he wants to draw a picture on the platter and sprinkle some magnetic filings on the platter to see it.
Wow I really hope that's not true, I just made myself puke.
Just use a butterfly.
The very first logical sector of a hard drive should hold the master boot record and partition table. If your hard drive was never partitioned by you, it would hold a default partition table that shouldn't be hard to guess. If you destroyed the partition table, your OS would not recognize the file systems on it, but you should see the device in the disk management utility which would offer you to partition and reformat the drive. If you guessed the partition table right and write it back, you'll get all your filesystems back.
However, logical sector is different from physical sector. Logical sectors are the numbering accessible by software. Physical sector is only meaningful on the platter. Hard drives have been known to hold reserves of physical sectors so it can map out bad sectors to good ones. The drive could theoretically elect to store such mapping in a physical sector, where the first physical sector holds a root mapping.
I just want to make sure, the data recovery company is talking about firmware in the first logical sector, or the first physical sector?
I once had a signature.
He can't have been too far wrong about RAW sockets. Microsoft quietly took them out of Windows XP three years later.
http://seclists.org/nmap-hackers/2005/4
--
The internet is the greatest source of biased information in the history of mankind.
Floppy disks had 40 tracks, numbered 0 through 39. But most floppy disks do have enough surface for one or two extra tracks, numbered track 40 and on. If you format it, you could store extra data on it outside of the file system. This is how some DOS games did copy protection because diskcopy ignored the extra tracks and assumed they did not exist.
I once had a signature.
Is there any reason why you couldn't use dd with an offset to accomplish this?
Ah dude... you do realize theres like a metal cover over the hard disk platters and even if you were to succeed the 1's and 0's would optically look no different.
There is no reason to do what your asking. If you just want to be able
to read and write from a disk as a gigantic linear array you can do so with the drive logic in place.
If you need performance get a faster spindle and use a write buffer.
Direct access is not going to buy you a damned thing.
Seems like useless requirements.
Remember this next time you are struggling getting requirements out of a non-technical manager or user. The submitter obviously has the technical background, but is making a common mistake.
What is it you want to do? "I want to write bits directly..."
Really? That's your final goal? Just to write bits? No, there's some other task you want to complete, and you've determined the best way to complete that task is to write bits directly to the platter.
But if you can't write bits directly to the platter, or you don't know what additional issues may arise when you do so, how can you determine that is the best course to take?
So slow down, back up a step, let us know what your real goal is. You want X, and you think the best way to X is to write bits to the platter.
Because that gives you absolutely no control over the physical location...?
maybe get a new "empty" (possible?) harddrive and do some "test" writing on it. ....?
i'm sure there's a way to acctually see where the data has been writen to on the
disk (defrag seem to know?).
erase the disk to "empty" again (possible?) and repeat.
i'm sure this way you can figure out what the black-box inbetween the OS and the physical
magnetic disk is doing. once you figure this out you'll just have to "cheat" your way thru the
not-so-black-anymore-box to get those ONeS and ZeROeS in the order you want on the disk
(50% tongue in cheek)
It isn't worth trying.
It can be done, but pretty much requires a couple of EEs and possibly a semiconductor fab. At a minimum it requires documentation from the manufacturer about how they map the virtual cylinder/head/track/block IDs used by SATA/IDE into the physical locations on the media. Once you have this information, you could create your own disk device driver to force data into the specific locations. This still doesn't get you beyond the structure imposed by the low level format of the drive, though.
Alternatively, you could rip the controller hardware off the disk drive and reverse-engineer it to figure out the hardware location mappings on your own. You'd then have to build your own controller hardware (hence the fab) to gain the required control and impose your own low level format on the drive.
As a practical matter, your best bet is to write your own file system driver. That at least is plausible, even if it's reinventing the wheel.
We are the 198 proof..
It would be nice to know why the poster thinks this feature is important
It would be nice to know, but it's their research project and that's how they want to proceed. So assume the requirements are set in stone and go from there.
This kind of copy protection has been outdated for quite some while and should have died with floppy disks ...
It's not copy protection at all - he just wants to find a way to write images directly to the platter. I would have to suggest that using lightscribe on a cd would be easier and more cost effective. ;)
Kidcharles: You can do it but you'll need: A) lots of experience with ASM B) Controller's Hardware Tech. Manual (you control the controller), your BIOS Tech. Manual, your OS Tech. Manual C) A debugging hardware board and/or a couple of machines to work on. D) You should at least have some experience using BIOS INT 13h calls You may be able to accomplish what you want with just the BIOS INT 13h calls (INT stands for interrupt) which allows some pretty specific disk access. Generally how raw write/reads are done. You can also expect quite a few low level formats from making mistakes. Hope that helps some
The more complex the task, the simpler the steps need to be.
On modern hard drives its very hard to get down to the platter level. Thats one reason why you can't low level format a hd. Can it be done? Yes. Is it easy? No.
What it would require is for you to write some assembly code to allow you to talk to the hd's controller chip. It has to be in assembler and not a high level language for the simple reason is that most of them do not allow you to access the hardware directly. The other reason why you would have to use assembly is because Windows will prevent it for the most part as well.
If you google low level disk access you might find some that might help you out and even give you some code. Any drive you do this on might not work anymore so try it first on a drive you don't care about.
if there is any way to do the impossible with hard drives, he's got the notes on it.
there is no good earthly reason to bit-poke a drive, and few in this universe, which is why the electronics for controlling drive writes/reads are all in the drive now, massively chipped, with sensors to adjust the servos for temperature, pressure and the like to put the bits smack in the middle of the track.
if this is supposed to be a new economy, how come they still want my old fashioned money?
Look up software called victoria or mmhd. Start reading the forums. What you are trying to do is possible but most are right in that you need to bypass the onboard controller using nested int calls. You will need a fair amount of specialized hardware as well and the ability to put the drive in factory test mode. Might want to look into fujitsu as they are a little more loose lipped that lets say, seagate.
I believe the technical term for this is deep magic - or I would, if I thought the poster understood what the words 'write', 'bit' and 'platter' mean in the context of hard disk drive tech. As it is, the question is so nonsensical it's bizarre.
I was going to make some joke about a ISA to USB converter.... but at the last moment before submit I decided to search.
It exists:
http://www.arstech.com/item--usb2isax7.html
Due to the nature of ISA I can't imagine this works well for all ISA hardware.
Used to be able to read and write arbitrary sectors with assembly code using BIOS ISR "int 13h". You can only read and write sectors. If you want to read/write bits, you need to get into the firmware loaded on the hard-drive themselves. Contact the hard-drive manufacturers. Some of the obsoleted firmware's source may be available to you if you execute NDA with them. Better yet, go through an University if you have acadamic contacts and the process may be smoother. Even wit firmware changes, I still believe that you can only read/write sectors at a time due to hardware design and not any individual bit within a sector. In any case, hard drive companies will have the final answer for you.
If you are not able to obtain off-the-shelf-hard drive, use the platters, and find/make a high-resolution CNC machine that you can control using the hard drive components. Use the hard-drive head on CNC machine. At the end of the day, hard drives are all controlled by motors.
Unless, as others have suggested, you have nefarious motives. IO subsystems can be more complex than the OS itself, if you indeed know anyting about either. And you would be trying to re-invent the wheel by trying to do what the poster says he wants. And I can guarantee your "new wheel" whould not be very round or useful. People who write competent IO subsystems tend to have genius level IQs.
Can you do it in floppy disk?
If you can use a SCSI hard drive (including SAS, FC, and parallel transports), most of them support the 'write long' command. This allows direct access to the bits on the disk, though the implementation is vendor specific and can vary. I've used this command to write intentional 'bad blocks' on a drive so I can test error recovery. Here is a tiny blurb on this command:
http://en.wikipedia.org/wiki/SCSI_Write_Commands#Write_Long
Much more detail in the SCSI block command set:
http://www.t10.org/drafts.htm#SBC_Family
In the past I've used the 'scu' utility to perform these operations, as well as vendor utilities of various kinds (I forget names).
http://home.comcast.net/~SCSIguy/SCSI_FAQ/RMiller_Tools/scu.html
Here is one reason the OP may want to do this: http://cua.mit.edu/ketterle_group/Projects_2007/Pubs_07/Boyd2007%20PRA.pdf (trapping ultracold atoms using the magnetic field from a hard disk platter).
Almost every time someone asks a question where they obviously have made an implementation decision that depends on "doing it differently than everyone else on the planet" while providing no information about what they are trying to actually accomplish, the problem can usually be solved in a much simpler way. While it is possible you are doing something exotic like trying to turn a hard drive platter into a meta-material by patterning high density magnetic patterns on it and so you really do need to be able to control the bits at the hardware level, odds are low.
You provided no information about what you are trying to do. There are pretty good odds that if you provide information about what you are trying to do instead of trying to get people to come up with a way for you to do it 'the hard way' you, will get an answer that will work for you.
If you really need a physical drive, you'll have to make your own controller or use really old hardware. Why not try a virtual drive? I bet qemu has quite a bit of helpful source code. Real filesystem source code might also be helpful, like ext2/ufs.
Geez. This is not the first thread with the majority of wasted posts in the "wah wah why won't you tell us why you need this, you're a big dummy plllbbbb" category, but it's getting more and more aggravating over time. Slashdot seems to be slipping into the idiocracy with this stuff.
The guy can't tell you why because he's under NDA. Or his higher-ups gave him unreasonable requirements that he tried and failed to dissuade them from, but he wants to be diplomatic instead of making a public stink over it. (That was my case last time I got this treatment here.)
The fact that the guy's disciplined enough to not respond to the flamebait indicates to me that he probably knows exactly what he's doing.
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
Or is a terrorist.
Just because it CAN be done, doesn't mean it should!
When I first saw the request, I wondered if it was for a copy protection scheme in which an entire (USB/FireWire) disk is loaded with the application and distributed with it.
Disks are amazingly cheap right now.
myke
Mimetics Inc. Twitter
Can't you just specify the offset for what you want to write in DD? Don't need a filesystem, don't need a partition, etc. Am I missing something, here?
Yeah, I was going to mention PRML as the main problem. There ARE no 1s and 0s on the disk surface anymore. Instead, the read head receives an analog signal and tries to statistically determine what the LIKELY values are supposed to be.
Software sucks. Open Source sucks less.
First, ones and zeros on a harddisk platter are not magnetic north and south, they are different frequencies of an FM sine wave. Second, the formatted tracks on a harddisk don't line up physically. The start of the track is wherever the head was when the controller was told to format that track. Third, Sectors aren't written in precise places inside their formatting. The low-level formatting process writes headers across the track that are used later to find the data. There's enough extra space between those headers that a sector of data will fit there, based on the ability of the controller to read where the sector starts and be able to write it. If you really need to write magnetic north & south to specific spots, you should look at Floppies. Floppy disks have a hole to mark one spot on the disk, so the start of track comes much closer to lining up. You can even write ones and zeroes instead of FM patterns if you have an old Apple II, which didn't use FM encoding.
Steve Gibson can get it done for you. http://www.grc.com/
There is nothing wrong with yr Internet. Do not attempt to adjust the picture. We are controlling the transmission - NSA
Talk to these folks, they'll get you on the right track.
Plan 9:
http://plan9.bell-labs.com/plan9/
http://plan9.bell-labs.com/wiki/plan9/mailing_lists/
If there's a way - they'll set you on the right track.
~hylas
Careful there. I know you're addressing a typical user of the hard drives, but there ARE ways of -- and reasons for -- getting at the "bits themselves". One of my colleagues, as a fresh student at one of the National Labs in the US, was tasked with recovering data from a crashed (literally crashed, so that the controller was destroyed and platters bent, with lots of data loss) disk. Of course there are ways and means of doing so, usually involving fantastically specialized equipment and lots of expense (and, often, the complicity or orders at the Federal level).
He ended up building a sweet little system which could scan the surface (in a low-level clean-room, of course), albeit slowly, and interact directly with the bits.
I'll have to ask him how they figured out the filesystem, the error-correction, and so on, without any of the usual partition tables and so forth.
How much of the "recovered" data do you think he didn't have to fake?
Can you give us some Linux or Windows tools that do what Spinrite does correctly (i.e. tries many times to re-read corruptly read sectors) but fixes what it does poorly (write recovered data back out to another hard drive)?
No, it is not. You build a cryptographic system with math, not with specialized hardware.
Rethinking email
Have you ever heard of the Unix command dd? I think that may be what you want. It writes directly to a disk, regardless of filesystem and/or format.
http://en.wikipedia.org/wiki/Dd_(Unix)
e.g. a 20MB MFM became a 30MB RLL?
If so, RLLs would be out, because the compression system would get in the way of direct bit placement on the platter.
I still have a couple of those old Seagates somewhere (including the full-length ISA RLL controller card), along with a (Western Digital?) 150MB ESDI drive. Ah, the good old days.
Nothing is inexplicable; only unexplained -Tom Baker, Doctor Who
Roadkil's unstoppable copier. Works on both Windows and Linux
http://www.roadkil.net/program.php?ProgramID=29
dd_rhelp works with dd_rescue to try to recover from disks with bad sectors.
http://www.kalysto.org/utilities/dd_rhelp/index.en.html
Without the purpose of the original question, we're all left running around like headless chickens.
I can only assume that the unrevealed purpose is kept hidden for a reason. I'll make a punt and suggest that anything legitimate could be done by implementing a file system along with diagnostic tools, or even just with flat files.
My response would be that the target technology is all proprietary, and the whole chain is carefully-engineered to convert the material analogues into digitals. There's even a fair chance that the near-physical stuff abstracts away the bit-level materials, so you'd have to re-engineer right down to the platter. You'd need a very expensive and shiny lab to do that.
It is optional, but was well supported by drives in the 90s, and may still be. It was typically used by vendors like EMC to be able to predict disk failure before it occurs (i.e., by increasing need for ECC codes indicates a bad sector may be forming)
So grab a SCSI card and whatever SCSI drives you or someone else has laying around and see if this is supported...
In the Linux Kernel, and probably every UNIX, it is possible to write directly to the raw device using system calls.
If you need to write data that doesn't fit in the regular 512 bits block, you will need to read the full block, modify it and write back the full block. If you need to write bits one by one, it will be terribly slow (maybe 100k/s), but it will works and be very cheap. And the amount of needed code is very small and perhaps there is already some library for that.
Sure, technically there will be ECC code transparently written to the physical disk, but personnaly I consider this a great feature. If you really need to to access or modify even the ECC bits, or get ride of them (which I seriously doubt), then there is special hardware available that are mostly used for data recovery or forensic analysis.
1) You want logical locations, not physical locations. I promise.
2) On Unix-like systems, you can open the device node, and seek directly to the locations of your choice.
int const fd = open("/dev/sda", O_RDWR);
3) If you have a predetermined image you want to slam onto the disk, use dd. Depending on what else you want to do, other command line programs may also be useful. For example:
~$ uname /dev/disk0
Darwin
~$ sudo hexdump -n 600
0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
00001b0 00 00 00 00 00 00 00 00 38 2d 00 00 00 00 00 fe
00001c0 ff ff ee fe ff ff 01 00 00 00 2f 60 38 3a 00 00
00001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
00001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa
0000200 45 46 49 20 50 41 52 54 00 00 01 00 5c 00 00 00
0000210 ab 81 40 18 00 00 00 00 01 00 00 00 00 00 00 00
0000220 2f 60 38 3a 00 00 00 00 22 00 00 00 00 00 00 00
0000230 0e 60 38 3a 00 00 00 00 48 75 00 00 de 3e 00 00
0000240 fa 0c 00 00 7d 31 00 00 02 00 00 00 00 00 00 00
0000250 80 00 00 00 80 00 00 00
0000258
~$
Good luck.
Memo to Senator Stevens: the posters above are joking. You do not need to introduce legislation to prevent these disasters. Kthxbai.
Just use a piece of old technology that's still available to buy.
Use a floppy disc instead of a hard disc.
Yes, except Steve Gibson would develop this über-leet super seekret method for doing it, would call it by some funky name (Project Y?) and then boast about how it's hand-crafted by a virgin in 100% assembly language - all the while it just reproduces the functionality of something that already exists (dd)...
Specialist Mac support for creative pros, Melbourne
In fact, if I asked this to a 100 engineers, I'd really expect that at least 99 of them will immediately ask "why the fuck would you want to do that!?". And the one who doesn't probably works at a harddisk manufacturer.
And yet the who doesn't is PRECISELY who you want to be talking to - the other 99 have NOTHING to offer you.
.
A decade or so ago, you could do this.. more or less. Your "low-level" format and all that - nowadays, there is logic on the drive unit itself you have to deal with.... there are more levels of abstraction... so even the lowest-level access I can think of off-hand from a normal modern OS won't actually come anywhere near controlling the drive itself. You'd need to actually mess with drive firmware to do that.... I think.
And yeah - check out Steve Gibson... Spinrite did stuff like that - not sure how valid that is with current media though (the principles probably still apply, but even he may be dealing with some level of abstraction now)
Back in the day one could use bios commands to write track sector platter 3d addresses - I know all of the geometry has changed, but the basic technique may be the same with a translation algorithm.
When I left Quantum a decade ago, they had already begun developing the technology for hard drives to write their own servo tracks.
If you want an exact image of 1's and 0's that you determine, forget it. Bits these days are encoded as transitions between magnetic phase boundaries. They also encode the clock data needed to recover the data along with the data. As a result, what's put down on the disk has to guarantee a certain number of these transitions or you loose sync. In other words, you may think you're writing '0000000' but on the disk it's a bunch of magnetic transitions, not a constant stream of the same magnetic polarity. It's not even that a single phase transition translates into either a 1 or a 0; groups of these transitions translate into a group of the actual data bits. Some patterns of phase transitions can simply never be written onto the media by the write electronics in the first place because they would not result in reliable data recovery by the read electronics.
No, that son of a bitch drank all my beer and shit in my bathroom sink. We are most assuredly NOT friends.
Get one of those sharks with lasers
Disable the laser.
Strap a frickin' big magnet to the shark's body,
Strap some beefsteak ro your hard disk.
- oh yeah- make sure you're in freshwater, not saltwater!
Saltwater will play havoc with the electronics on board the drive!
Trust me- I KNOW from experience.
.
- aqk
F U
Your quandary does not speak of the precision required in locating specific disk areas. In the "OLD DAYS", servo tracks were used- read-only, permanent. Holes in the media were also used, but difficult to retrofit. Temperature and voltage will cause inaccuracies. One would need to construct a controller in order to have "bare metal" write-ability. It seems doable. Good fortune, berk
First off, without a more detailed description of what the user really wants to do, it is very hard to say whether it is doable. If he wants to write say, an isolated 1 in a stream of 0s at a certain physical location on the media, that will not happen on any modern drive. Many responses have pointed that out. If he wants to write a 512 byte block of 'something' at a physical location, that is quite doable. If he wants to write a given frequency pattern to a given spot, that is also doable, but very drive specific. All drives have a manufacturing 'special mode' which allows for physical positioning of the actuator. All drives have commands that allow special patterns to be written. Modern disk drives are very complex. The amount of effort required to reverse engineer the controller hardware and firmware of a given drive would be huge. I have worked in the disk drive industry for over 20 years. Don't listen to folks that tell you 'Steve Gibson can do it'. He can't. There are people in San Jose, Colorado, Rochester, etc that do this stuff every day. It is not easy.
I would do it LIKE THIS ...
Build an app that creates a file on a flash drive and then write directly to that file, and use that file INSTEAD of a hard drive.
IF flash is too slow then you should be using RAM to set your bits instead of a hard-drive MUCH FASTER ! You say you need to set persistent bits, then write to PRAM, but then that is SLOW again, or create an app that launches at startup to reset the previous RAM bits.
IF you MUST do it on to THE hard drive then it sounds like you are a hacker, so the HD companies are just not going to allow you to do this.
Now what if you created a disk in RAM, a perfect virtualized image, then twiddled the bits in the OS to full it into believing the RAM image is THE HD, that might work but I am SURE firewalls in the OS will be guarding against this but that does not mean it can't be done.
Tracking reply
.... tell me you are not working for a HDD manufacturer....?
I think what this guy really wants to do is get a lot of Slashdotters to post speculative, mostly-useless, and silly replies.
He did pretty well.
op:all u need is a very small magnet.
next question, move on people nothing to see here