Slashdot Mirror


Write Bits Directly Onto a Hard Drive Platter?

kidcharles writes "I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion. I need to be able to control exactly where 1s and 0s will appear physically on the platter. Normally when data is written to a drive the actual bits that get written are determined by the file system being used, as modified by whatever kind of error handling the drive itself is using (e.g. Reed-Solomon). All of the modern innovations in file systems and error handling are great for reliable and efficient data storage, but they are making my particular task quite daunting. My question for Slashdot: is there a way to get down to the 'bare metal' and write these bits? Any good utilities out there to do this? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."

113 of 578 comments (clear)

  1. magic marker by Rivalz · · Score: 5, Funny

    It works for me. http://www.staples.com/Staples-Dry-Erase-Markers-Chisel-Tip-Black-4-Pack/product_607101?cmArea=FEATURED:SC1:CG11:DP1101 Now that someone has said it no one else can be a smart ass and they have to be informative.

  2. May I suggest by Blazarov · · Score: 2, Funny

    May I suggest a magnetic needle and a steady hand? http://xkcd.com/378/

    --
    Regards, Boyan
  3. Sure by Jurily · · Score: 5, Informative

    Just make your own controller chip for the drive.

    1. Re:Sure by Anonymous Coward · · Score: 2, Interesting

      Agreed. I would find is difficult to believe there is a practical use for this, but if there is an actual requirement to control zero and ones in specific tracks/sectors, you might as well build your own chip for a drive. :)

      If the solution sucks that maybe they're trying to solve the wrong problem. :)

    2. Re:Sure by amn108 · · Score: 4, Informative

      Modern drives come with their own on-drive controllers, as far as I know. Chaining another controller on top will not give you any power to override the on-drive controller behavior. If the on-drive controller makes its own decisions on where and how to write data, your custom controller will not be able to override these decisions, I am afraid.

    3. Re:Sure by adolf · · Score: 4, Insightful

      A whole chip? Really? That sounds hard.

      Just Ebay an old 20MB Seagate ST-225 MFM drive, and write whatever bits you want.

      It doesn't know any better.

    4. Re:Sure by Smidge204 · · Score: 5, Insightful

      The custom controller would REPLACE the the on-drive controller, not sit on top of it.

      =Smidge=

    5. Re:Sure by Jim+Hall · · Score: 5, Interesting

      A whole chip? Really? That sounds hard. Just Ebay an old 20MB Seagate ST-225 MFM drive, and write whatever bits you want. It doesn't know any better.

      Or, the submitter could contact Seagate or another drive manufacturer and ask what it would take to get a drive with special firmware that let him write 1's and 0's directly to the drive wherever he wanted. Basically, remove the intelligence on the drive.

      It's not that impossible for drive manufacturers to do things like this for you, if you have the $$ to pay for it. I don't know what the cost would be, especially since he's really only [probably] looking for a few drives for this project. If it's grant-funded research, the grant would pay for it. If it's an independent project of some kind, he's in for a surprise.

      I took a behind-doors tour of a major drive manufacturer a few years ago. During our visit, we were able to visit with engineers - one of whom was head of the firmware engineering team. He told us lots of stories about the firmware requests they've fulfilled. One example was a customer who supported lots of old PBX systems. These PBX systems ran software from a hard drive, but due to the age the system only supported drives up to (around) 200MB. Nobody made drives that small anymore, so this drive manufacturer re-wrote the firmware for them ($$) so a 120GB drive (the smallest they made at the time) would only recognize & address the first 200MB.

      So yeah, I'm sure a drive manufacturer like Seagate could write custom firmware for him that would meet his project needs.

    6. Re:Sure by vtcodger · · Score: 2, Interesting

      Does is have to be a hard drive? The Apple IIe was a wretched computer with awful software, but it had a floppy controller that was possibly the cutest electronic device ever made. It's been an awfully long time, but my recollection is that it was built from a handful -- six or eight -- of TTL chips and that not only COULD one control exactly what was written where in software, one HAD to control what was written where in software.

      --
      You can't see ANYTHING from a car, You've got to get out of the goddamned contraption and walk...Edward Abbey
    7. Re:Sure by CharlieG · · Score: 4, Insightful

      That was my first idea too "Find an old MFM/RLL drive, and have at it" - then I realized, "You know, I'll bet that more than 1/2 of the readers of /. don't even remember them"

      --
      -- 73 de KG2V For the Children - RKBA! "You are what you do when it counts" - the Masso
    8. Re:Sure by Gordonjcp · · Score: 2, Insightful

      but if there is an actual requirement to control zero and ones in specific tracks/sectors, you might as well build your own chip for a drive.

      Not to mention that disks haven't actually written "zeros and ones" as such for at least 15 years, which is why a single pass of /dev/zero will wipe a recentish disk beyond recovery.

    9. Re:Sure by lavaboy · · Score: 2, Funny

      damn. I was going to say "Call Woz."

      --
      Steve -- If you have to call it a system, you don't know what it is.
    10. Re:Sure by TheLink · · Score: 3, Informative

      But you couldn't control exactly what was written with the Apple II drives.

      IIRC the spec was you could not expect to _reliably_ write a bit stream that has more than two zeroes in a row. In practice you could usually get away with three zeroes in a row. You had to do stuff like write sync streams e.g. 11111111001111111100111111110011111111001111111100. Which are basically 5 x 10 bits of 1111111100.

      You could control the drive head too. You could even move the head in sub-track increments - e.g. half track. However, tracks have to be at least a full track apart to be reliably read/written.

      All these is why I find the submitter's request rather strange. When you get down to the low level details, there is no such thing as exact.

      Even if you are writing one bit on the entire platter with a huge magnet :).

      So I'm curious on what he's really trying to achieve.

      --
    11. Re:Sure by idontgno · · Score: 2, Informative

      You have a 30 MB HDD until you realize that your MFM HDD wasn't being sold as an RLL HDD because it failed RLL density checks at the factory, and your bits start decaying within hours or days of writing them.

      Yes, this is the bitter voice of experience. An ST-255 wasn't always the equivalent ST-238.

      --
      Welcome to the Panopticon. Used to be a prison, now it's your home.
    12. Re:Sure by operagost · · Score: 2, Insightful

      The Apple IIe was a wretched computer with awful software

      You know, I'm far from an Apple fanboy, but I used the Apple IIe and by the standards of the day, it was clearly not a "wretched computer with awful software". It had good graphics, a built-in ML monitor and BASIC, few quirks, and a low enough price that it dominated the USA public school system. The C64 was a better value, but it had its own problems (like the glacially slow floppy drive).

      --

      Gamingmuseum.com: Give your 3D accelerator a rest.
    13. Re:Sure by marcosdumay · · Score: 2, Insightful

      Some people think that is ettiquete for posting on Slashdot. This site is inherently linked to free software.

  4. That is gonna be hard by PaulIsTheName · · Score: 5, Insightful

    You would need to replace the firmware inside the drive or use an undocumented manufacturer mode. Whatever they use to write the servo tracks would be interesting to you. You will be in the situation of the firmware writer: There will be problems all the way. Be prepared to find a way to position the heads (ever tried to find a servo track?). Most likely you also need to at least parametrize the amplifiers in the DSP part of the firmware that does the analog-to-something-to-digital so you can have direct influence on the "bits". Good luck

    1. Re:That is gonna be hard by omglolbah · · Score: 2, Insightful

      Yet to someone who is not a troll and has technical knowledge it makes perfect sense! :-p

    2. Re:That is gonna be hard by lobsterturd · · Score: 2, Informative

      For the benefit of the poster, who doesn't seem the type to know what a servo track is:

      A servo track is how the hard drive knows where the heads are over the disk. Older hardware could just read the angle of the arm, but with vast increases in density it became necessary to put position data on the platters themselves where more precision is possible.

  5. Your own Disk controller chipset? by Anonymous Coward · · Score: 3, Insightful

    If you don't need lots of bytes, you should use an old disk. I mean very old, maybe going back to those 300-400MB disks we had on 486 computers or even older.

    Recent disks have embedded error-handling mechanisms which will prevent you from writing what you want, where you want (physically speaking of course). The disk controller translates your "physical" disk addresses into its own internal mappings. The only way to bypass this would be to remove the disk controller's chip and put your own in place.

    Maybe your question would be: "how hard is it to create my own disk controller's chipset?"

    1. Re:Your own Disk controller chipset? by ranulf · · Score: 2, Informative
      > I mean very old, maybe going back to those 300-400MB disks we had on 486 computers or even older.

      Hahaha. 300-400MB isn't "very old". It's only 20 years since I bought a 80MB hard disk for £150 and that was considered massive as a couple of years earlier 20MB was cutting edge and cost a few hundred pounds.

    2. Re:Your own Disk controller chipset? by Anonymous Coward · · Score: 5, Informative

      I think anything > 120MB may be unsuitable for this purpose, as they are likely to use ZBR, i.e. different sector densities depending on the circumference of the track. I'd also recommend a really old MFM or RLL disk, where the head movement is controlled with one set of wires and the analog signals from the read/write heads use another set of wires. MFM and RLL drives use the ST506 interface, which should be "easy" to control with microcontroller and a little bit of glue logic.

  6. Re:How about by PaulIsTheName · · Score: 3, Informative

    They (the bits you write to your typical /dev/sd* device) would still be wrapped up in sectors, the sectors be enhanced with error correction, then encoded (NRZ, Manchester or whatever is appropriate to the magnetic storage) and then written out. What OP wanted is to manipulate the bit that comes out of encoding. Given the usual stack of SCSI-like commands (and not direct access to the write head signal as early HDD controllers had) this does not seem feasible.

  7. Slashdot trolled by dotgain · · Score: 5, Insightful

    The submitter appears to know enough about what he's asking to know that it's also impossible / completely impractical. Recording ones and zeros directly isn't done for a reason, submitter appears to understand this.

    1. Re:Slashdot trolled by bertok · · Score: 5, Interesting

      The submitter appears to know enough about what he's asking to know that it's also impossible / completely impractical. Recording ones and zeros directly isn't done for a reason, submitter appears to understand this.

      Keep in mind that the question is not just why the submitter wants to write directly to the magnetic domains of a hard disk, but why the submitter asked the question of how to write directly to a hard disk.

      I can think of a number of reasons for the latter:

      - The submitter could be an author writing some cyberpunk techno-thriller, and wanted some technical feedback from the 'experts' on Slashdot.

      - The submitter could be a CIA drone trying to write a report on how it's impossible for the terrorists to somehow 'hide' data on hard drives that even 'dd' can't read.

      - The submitter could be a troll.

      That's just off the top of my head.

      On the other hand, I seriously can't think of a good reason why anyone would want to actually damage a hard drive that way. If all you wanted to do is very precisely create a surface with a complex pattern of specifically oriented magnetic domains, there's easier ways. For example, there are equipment suppliers that can provide high-precision two axis steppers with ready to use computer control interfaces. It's not hard to get equipment that'll go down to atomic precision.

    2. Re:Slashdot trolled by AHuxley · · Score: 2, Interesting

      The magnetic domains of a floppy disks and removable storage devices did spook the FBI in the Phillip Hanssen case.
      http://slashdot.org/article.pl?sid=01/02/22/0622249&mode=thread
      Did any of the old black box recorder like devices write in a loop directly?

      --
      Domestic spying is now "Benign Information Gathering"
    3. Re:Slashdot trolled by mbone · · Score: 3, Interesting

      The submitter doesn't appear to know enough to say why he wants to do this. If this were a consulting job, I would say, what are you really trying to do ? My guess is that, whatever that is, there is a better way to do it.

    4. Re:Slashdot trolled by Anonymous Coward · · Score: 2, Insightful

      Or maybe the submitter is a developer who has been told to do something impossible/completely impractical by someone in management. Management doesn't believe that what they are asking for is impossible/completely impractical so this is the submitters attempt to get an informed body of people to say the task is impossible/completely impractical so management will believe him.

    5. Re:Slashdot trolled by SharpFang · · Score: 4, Interesting

      - Submitter wants (for whatever weird project of theirs) a surface with magnetic image on it (a'la LightScribe, only magnetically).
      - Submitter missed the actual point, and wants to write raw data to drive (some embedded project) circumventing partitioning and filesystem overhead - quite frequent.
      - Submitter wants to record a lot of analog/unreliable data - skipping error checking and using the extra space to store more data (errors = acceptable noise)
      - Submitter doesn't care about data storage and wants to use disk heads as a very fast magnetic field sensor/generator

      This is quite doable with ANCIENT pre-ATA drives. The computer controlled everything then, the head position, the moment of writing and so on.
      It is doable with old ATA drives that run PIO modes, probably with a bit of electronics tinkering.
      It is about impossible with modern drives. You'd have to replace the whole electronics with your own and drive the head and the motors with your own.

      --
      45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
    6. Re:Slashdot trolled by dotgain · · Score: 3, Informative

      You have got a lot of catching up to do. Drives just don't work like that any more.

    7. Re:Slashdot trolled by iamhassi · · Score: 2, Funny

      "I hope he get his $100, and share with us if possible."

      gives new meaning to "my 2 cents worth"

      --
      my karma will be here long after I'm gone
    8. Re:Slashdot trolled by vadeskoc · · Score: 5, Insightful

      If I had to guess, I'd say this is a physics grad student who is looking to use this hard drive for something other than data storage. Where I work (in academia) we repurpose hardware all the time for purposes other than it's intended one, in order to do science, since commercial stuff is often engineered way beyond our capability to build it from scratch. That is, provided we can overcome details of its implementation that arise from its original (true) purpose. This may be one of those cases where the thing is engineered beyond hope of hacking, but I'm kind of disappointed at the "you're-an-idiot-because-hard-drives-aren't-meant-to-do-that" attitude in these posts. Is this really indicative of the level of imagination and curiosity on Slashdot?

    9. Re:Slashdot trolled by Crudely_Indecent · · Score: 2, Insightful

      The submitter doesn't appear to know enough....

      Or the submitter is a consultant who knows enough about his non-disclosure agreement to reveal only the bare minimum about his goals without revealing the purpose of the project and violating his NDA.

      --


      "Lame" - Galaxar
    10. Re:Slashdot trolled by c++0xFF · · Score: 3, Interesting

      I was stumped as to why one would want direct access to the drive like that, until I read your post. Thank you.

      But I also agree that this is just about impossible, depending on what he's trying to do. There's no way he can control the heads of the disk: too many layers of abstraction exist between the user and the hardware. But it should be possible to control where data is placed on the disk: this is what disk defragmentation does, after all.

      But even then I'm not sure that what the device reports is necessarily what happens on the platters. Flash memory drives, for example, relocate information to balance the number of writes and extend the life of the drive.

      I would recommend contacting the drive manufacturer and then making my own block device driver for Linux based on their recommendations. But even then I'd question the core reasoning: is this really, really necessary?

    11. Re:Slashdot trolled by ElizabethGreene · · Score: 3, Informative

      Spinrite may "write" 00000000 to a specific location on the disk, but that 00000000 is actually encoded as something else when it is written to the disk to maintain a minimum number of bit transitions. In the Telecom world, B8ZS is commonly used for encoding. AFAIK, the HDD manufacturer's algorithms are proprietary.

      It is also worth mentioning that a hard drive head does not read a binary signal from the platter. It reads an analog waveform and converts it to digital via the encoding and ECC algorithms the OP is trying to bypass.

      -ellie

      Scott Moulton of http://myharddrivedied.com/ is the smartest HDD guy I know. If you absolutely have to have a solution, talk to him.

    12. Re:Slashdot trolled by SEWilco · · Score: 4, Funny

      It is about impossible with modern drives. You'd have to replace the whole electronics with your own and drive the head and the motors with your own.

      Find a drive with characteristics similar to what you want, and contract with the manufacturer to modify the drive to do what you want. The manufacturer has the tools and engineers to alter the existing firmware. It won't be cheap. If the manufacturer refuses the proposal, buy the company.

    13. Re:Slashdot trolled by Hognoxious · · Score: 4, Interesting

      Where I work (in academia) we repurpose hardware all the time for purposes other than it's intended one

      What you say is all well and good, but you don't think that knowledge of that purpose might help with the solutions?

      I'm kind of disappointed at the "you're-an-idiot-because-hard-drives-aren't-meant-to-do-that" attitude in these posts.

      It's nothing like that at all. It's more like that-won't-work-so-what-are-you-trying-to-achieve-maybe-something-else-will-work, which is fair enough.

      Is this really indicative of the level of imagination and curiosity on Slashdot?

      A lot are asking "Why do you want to do that?", which sure looks like curiosity to me.

      --
      Confucius say, "Find worm in apple - bad. Find half a worm - worse."
    14. Re:Slashdot trolled by Crudely_Indecent · · Score: 2, Insightful

      He was smart enough to consult with his peers.

      The problem with doing something that hasn't been done before is that you have to figure out how to do it.

      The first step is research. Slashdot seems a likely place to find that research starting point or a glimmer of wisdom that might point in the right direction.

      --


      "Lame" - Galaxar
    15. Re:Slashdot trolled by bws111 · · Score: 2, Informative

      Quick! Go read his FAQ! Right there under 'Can I low-level format my drive' it says 'not possible on any modern hardware'. Randomly poking bits at specific PHYSICAL locations (which is what the poster wants to do) is low-level formatting.

    16. Re:Slashdot trolled by Jane+Q.+Public · · Score: 2, Informative

      No, they don't. Not anymore. Take a typical WD drive for example. Clusters are composed of areas of data, separated by areas of correction code (ECC) data. The latter are written by the drive hardware, or firmware at least, and depend on what is written to the data area. (In other words, it's meta-data, for detection of bad writes or corrupted areas of the disk.)

      So even aside from boot sectors and so on, the data areas of the drive are separated by areas written by the drive's control firmware, so the latter are not directly writable by the user. Unless there is some kind of low-level command that can be sent to the drive, telling it to not write ECC data at all (and I don't see any reason for the existence of such), then the answer is NO: you just can't do that with a newer drive.

    17. Re:Slashdot trolled by plover · · Score: 4, Informative

      The difference between the way you learned hard disks worked long ago and the way we use them now is due to IDE technology. Internally, the drive is still commanding cylinder, head and sector. Externally (to the PC), the interface is very different.

      Way back when we used to have the computer telling the drive controller board to operate the physical drive mechanisms, with the controller telling the drive which cylinder, head and sector to retrieve. This was called Cylinder-Head-Sector (CHS) addressing. If you ever worked on an original IBM PC-AT, you might remember having to set the "heads and cylinders" in the BIOS settings. You, the human, had to select from a list in the BIOS how many heads and cylinders the disk drive had. (And the older the PC, the fewer the choices. That meant every drive you could buy already had to be supported in the BIOS, but since there were so few originally, it didn't matter too much. As I recall, the max size drive supported by the original AT BIOS was 33MB, even though the largest drive IBM offered at the time was something like 20MB, if you had all the money.) The protocol was called ST-506, named after the first commercially available 5-1/4" hard drive.

      Integrated Drive Electronics (IDE) changed that. The controller board was moved from the PC to the drive itself. So now the controller board was set up by the drive manufacturer to know exactly how many heads and cylinders the drive had. To talk to an IDE drive, ATA was developed as a protocol. The computer would no longer send commands for a specific physical drive geometry, but would instead ask for a "logical block number", and the IDE would do a translation (looking it up in a map) to command the hardware to move to the correct physical cylinder, head and sector to retrieve the data. This is called Logical Block Addressing (LBA).

      So unless the original story author knows exactly what drive he's dealing with, and can download the lookup table from the drive, what he's asking for is no longer realistic. Each individual drive has its own unique mapping from LBA to CHS, because each drive maintains its own map of bad sectors. A computer asking for logical block #52 might physically get it from head#0, cylinder#3, sector#21 on one drive, but due to bad block mapping on a different drive it might get it from head#0, cylinder#4, sector#35.

      SCSI works on a similar principle, and Serial ATA has made further improvements on the scheme for performance and size changes. But all modern drive interfaces use some form of logical block addressing to access the data, and none use direct cylinder-head-sector addressing.

      --
      John
  8. Re:You Da Man!!! by maxwell+demon · · Score: 3, Informative

    Besides the fact that Flash doesn't have a hard disk platter (his problem obviously isn't just to save data), Flash chips also contain internal logic to map out defective blocks, and in addition they remap in order to avoid writing single blocks too often in order to prevent them from failing early. So even if the HDD platter isn't really needed, he won't get exact control on Flash either.

    If you had at least suggested a floppy. There you can indeed control exactly where your bits go, and it's even a magnetic medium.

    --
    The Tao of math: The numbers you can count are not the real numbers.
  9. Wrong. by feepness · · Score: 5, Insightful

    Your requirements are wrong, sorry.

    Yes, I don't what they are. Still wrong.

    1. Re:Wrong. by xtracto · · Score: 2, Funny

      No, its only that his writing algorithm a word.

      --
      Ubuntu is an African word meaning 'I can't configure Debian'
    2. Re:Wrong. by weirdcrashingnoises · · Score: 2, Funny

      But how can be sure?

      --
      sigs... don't talk to me about sigs....
    3. Re:Wrong. by jellomizer · · Score: 3, Interesting

      It would be nice to know why the poster thinks this feature is important. The only things I can think of is the following...
      He is using parts from a hard drive to make a robot. So he wants to control a Robot using the hard logic board, to control the mechanisms.
      He is using the high speed nature of the drive and the noses it makes to make a speaker out of it by moving writing bits in the right spot.
      Trying to make a full real-time application.
      Researching better algorithms for drive storage.
      Attempting to make some sort of digital watermark on the drive.

      The first two are just silly hacks which would get geek creds but not overly useful.

      The Real Time System You probably need more then off the shelf and free software to do the trick

      And the Research you could probably program a simulator to simulate the results.

      The Watermark idea will work until someone fills up the drive.

      So I would agree with the poster unless you come up with a good reason where there isn't an other approach I don't see the need to do this.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
  10. Be careful! by Anonymous Coward · · Score: 5, Funny

    I can't help directly, but can give one important advice - careful how you distribute the bits! If too many ones get on the same side of the platter this will destabilize it, causing it to wobble due to the weight difference ( a one weighs quite a bit more than a zero, you know!) and potentially tearing the platter in two!

    1. Re:Be careful! by Anonymous Coward · · Score: 5, Funny

      I can attest that improper bit stacking is a very real danger - one that almost cost me my life.

      It was in 1999 in Sierra Leone, and my last assignment before I left for Sydney was to punt three hundred reams of bit-printed A4 four kilometres up the Moa to a nearby monastery.

      It had been a hectic week and the crew the boss had assigned to load the raft had already worked a double shift. Subsequently, they failed to read the job sheet properly and loaded the raft with the all the paper with the zero-bit set on one side and the one-bit set on the other.

      I got about three hundred meters from the dock when I was startled by a large black snake swimming past. I quickly moved from where I was sitting on the zero-bit stack to the one-bit stack, and of course the raft immediately capsized.

      I was told when I reached the shore that the snake had most probably been a mamba - one of the deadliest snakes in Africa. If I had been bitten I would have been dead within minutes.

      I hope this serves as a warning to anyone involved in any kind of data stacking. In short, always properly disperse your bits - your life may depend on it!

    2. Re:Be careful! by gardyloo · · Score: 3, Funny

      Oh, hell yeah. Sometimes on a slow day, when there's little wind and a crowd outside the building, my colleagues will send around a bunch of emails full of ones. The cables outside twist and swing, and people look around for the source of wind.

          Those recent communication satellite crashes? Too many ones got in their relay buffers, weighing them down and destabilizing their orbits.

           

  11. Hm ... by garry_g · · Score: 5, Insightful

    This kind of copy protection has been outdated for quite some while and should have died with floppy disks ...

  12. Tell us your project? by marcansoft · · Score: 5, Insightful

    These kinds of questions are stupid: "I need to do XYZ for a project, how do I do XYZ?", where XYZ is one or more of complicated, ridiculous, vague, nonsensical, etc. Try telling us what your project is, and then we might be able to suggest a useful solution, possibly not involving XYZ at all, or involving a very particular/practical version of XYZ.

    1. Re:Tell us your project? by Scutter · · Score: 4, Insightful

      These kinds of questions are stupid: "I need to do XYZ for a project, how do I do XYZ?", where XYZ is one or more of complicated, ridiculous, vague, nonsensical, etc. Try telling us what your project is, and then we might be able to suggest a useful solution, possibly not involving XYZ at all, or involving a very particular/practical version of XYZ.

      Except he doesn't want your alternate solution. He wants the solution he requested. If you don't know of a way to do it, then move on.

      --

      "Tell me doctor, with all of your defenses, are there any provisions for an attack by killer bees?"
    2. Re:Tell us your project? by mikael_j · · Score: 2, Insightful

      I bet you're one of those people who force the rest of us to post extremely long and verbose questions on forums, IRC and mailing lists because you always come up with completely irrelevant answers if you don't get a wall of text explaining in detail why someone is looking for a specific type of solution (and even then you probably try to sneak in your irrelevant suggestions).

      An example of what I meant by the above: Let's say you have a motherboard where you need to update the BIOS, the computer has no optical drive, no floppy drive and you have no USB memory sticks available, the machine is using grub2 and you're looking for a way to flash the BIOS anyway to get around some silly bug without having to wait until monday morning so you can buy extra hardware just to flash the BIOS. The correct answer would then be something like "Check out flashrom, you should be able to flash the BIOS directly from Linux without even rebooting then". The answers that are likely to show up on any random *nix forum or IRC channel are:

      1. Get a new motherboard
      2. Get a CD-RW drive
      3. Get a USB memory stick
      4. Get a floppy drive
      5. Edit /boot/grub/menu.lst (no can do since it's grub2) and make a FreeDOS boot floppy (right after you replace your boot manager just so you can do a one off BIOS flashing, yay)
      6. Install Windows
      7. Create a FreeDOS boot floppy (see: No floppy drive)
      8. etc ad nauseum...

      In fact, even if you explained all the above chances are some people would still give stupid advice and sometimes it's even beneficial to not explain yourself because you know from experience that 90% of those who read your question don't know enough about the subject matter to understand it and will instead give irrelevant answers.

      /Mikael

      --
      Greylisting is to SMTP as NAT is to IPv4
    3. Re:Tell us your project? by Lehk228 · · Score: 5, Insightful

      nobody has the solution he requested, because the solution he requested is stupid.

      --
      Snowden and Manning are heroes.
    4. Re:Tell us your project? by Hognoxious · · Score: 2, Interesting

      Except he doesn't want your alternate solution. He wants the solution he requested.

      And I want a pony. Point is, nobody at all can understand a good reason for doing it, so that tends to make people think he's jumped prematurely to a diagnosis - fussing about "how" before he's really sure "what".

      He's stating a half-assed solution, rather than expressing what the problem is - for which the best solution might be entirely different.

      http://catb.org/~esr/faqs/smart-questions.html#goal

      --
      Confucius say, "Find worm in apple - bad. Find half a worm - worse."
    5. Re:Tell us your project? by IICV · · Score: 3, Insightful

      This bash quote is, I think, appropriate:

      <glyph> For example - if you came in here asking "how do I use a jackhammer" we might ask "why do you need to use a jackhammer"
      <glyph> If the answer to the latter question is "to knock my grandmother's head off to let out the evil spirits that gave her cancer", then maybe the problem is actually unrelated to jackhammers

      When someone comes in with a question like this, it is entirely worth asking what he's actually trying to accomplish - because writing ones and zeros directly to a hard disk platter is the deepest sort of black magic, and there's absolutely no reason why this is the only way to accomplish what he wants to do.

  13. If you have to ask, it's hopeless by vadim_t · · Score: 4, Insightful

    "I'm working on a project to build a nuclear powerplant. Is there documentation on how to do it? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."

    It seems to me that if you're involved in such a project and have to ask how to do it, it's doomed.

    Also, the whole idea of it seems rather impossible. Why would you want to do this in the first place? Have in mind that hard disks don't write bits as is, not just because of ECC. They use an encoding to ensure that there are no long strings of 1s or 0s. If you just wrote some data without regards to that you'd run into a sequence you'd be unable to read later, due to not having a clock signal to figure out where a bit ends and the next begins.

    Modern hard disks require using special encodings and servo data in order to be readable at those densities. It just makes little sense to me to want to bypass it. Unless you're working for a disk manufacturer this just seems very odd, but in that case you'd have access to the required equipment and information.

    1. Re:If you have to ask, it's hopeless by vlm · · Score: 4, Insightful

      It seems to me that if you're involved in such a project and have to ask how to do it, it's doomed.

      Agreed, except that he may only need that initial kick to find terms to google.

      For example, I know that to do what he wants, he needs to learn about the MFM vs RLL interfaces from the mid 80s, not because he will emulate either, but if he doesn't understand both and how they worked and relate to each other, he's has no chance. He is more or less trying to make a hard drive version of the catweasel floppy controller. If he never heard of a catweasel he would have a hard time figuring out he needs to learn about it.

      Image scans from the very detailed technical manuals for DEC's minicomputer hard drives are available online. This is from the era of individual TTL chips, he's basically going to replicate / emulate / reverse engineer that hardware into his microcontroller in order to write individual 1/0, after all they had a solution that worked 40-50 years ago. If he didn't already know that, he would have a hard time figuring out he needed to know that.

      Now if he already knows that stuff, and is really asking how to get MPLAB working so he can program his PIC, well yeah then he's well and truely lost.

      But given those first steps/hints, I think a reasonably experienced EE/CE type could probably figure out the rest of it.

      Adding to your nuclear analogy, if someone went back in time and told folks after 1920 to play with neutrons and U-235, they would get quite the head start on the rest of the world that didn't get that idea to try that until much later. They had all the prerequisites by 1920 but it took many years to get the right combination of ideas...

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
  14. Re:DD by dotgain · · Score: 5, Insightful
    Read the summary - there's a lot more going on with the data after 'dd' has done with it. Even if I write a continuous stream of zeroes to the disk, that's not what will wind up on the disk. For some reason we've yet to establish, he wants direct control over exactly what bits are recorded where. Unfortunately, doing this will mean it'll be damn near impossible to read the data back, as there will now be no reliable way to determine what track / sector we're located at. There'll be no means to detect errors, let alone correct them. He probably won't even be able to verify what he's just written.

    I'm actually more interested in why the fuck he wants to do this, much less how it would be achieved.

  15. Re:DD by Lord+Byron+Eee+PC · · Score: 2, Informative

    dd only bypasses the filesystem. It doesn't override the HDD firmware, which has to avoid bad sectors and write ECC information to the platter as well.

    I would love to hear why the submitter thinks he needs to write directly to the platter.

  16. Re:man dd by 1s44c · · Score: 2, Informative

    RTFS

    Knowing everything there is to know about dd isn't going to tell you where on a physical disk your data is stored. dd only sees a long string of zeros and ones that are a far abstruction from what physically happens on the drive.

    There doesn't seem to be any way to be sure exactly where the data is written except by a deep understanding and maybe a replacement of the drive firmware.

  17. Re:You Da Man!!! by 1s44c · · Score: 2, Interesting

    First thought was - WoW! Somebody wants to defeat disk caching, ECC and make life bloody difficult to boot. The best way, if you really want to do direct disk RMW operations - is to use Flash. Its fast, reasonably portable, and won't fail you if you use it properly.

    You still won't know where the data is actually written. Flash does wear leveling.

  18. Re:You Da Man!!! by dotgain · · Score: 3, Insightful

    If you had at least suggested a floppy.

    Actually that's the best suggestion yet. After he's spent a week banging bits into the 82072A, he'll either have gone insane, or will have realised he's well and truly barking up the wrong tree. Once again, OP: I'd love to know what you're up to, if it's not trolling Ask Slashdot.

  19. Re:Talk to Steve Gibson author of Spinrite by marcansoft · · Score: 4, Interesting

    The stuff Spinrite does stopped working 15 years ago. On modern drives you're pretty much doomed. Considering Gibson has been known to rip off SYN cookies and spout such nonsense as "Windows RAW sockets are going to destroy the internet", I wouldn't go anywhere near him for advice.

  20. You're doing it wrong! by sfraggle · · Score: 2, Insightful

    I need to be able to control exactly where 1s and 0s will appear physically on the platter.

    No you don't. Rethink what it is that you're actually trying to achieve and try again.

    --
    were you expecting to see a sig here? perhaps you'd rather see the inside of an ambulance!
    1. Re:You're doing it wrong! by Anonymous Coward · · Score: 2, Interesting

      You can calculate this by finding out C/H/S parameters (and the physical dimensions of the platter(s)), calculating the LBA, reading the sector where the bit should 'appear' and rewriting it with the 'one' bit changed (ATA controllers read/write 512 bytes at a time (sector size)).
      Simple, LBA .

    2. Re:You're doing it wrong! by Goaway · · Score: 5, Insightful

      And if he's trying to see if he can read data patterns off platters by hand from a dismantled drive, and needs a known test pattern to calibrate his equipment with?

  21. I tried it once by Taco+Cowboy · · Score: 4, Funny

    Got myself a disk editor, and wrote something on the very first sector.

    After that I can't use the HD anymore.

    The computer can't even recognize it anymore.

    --
    Muchas Gracias, Señor Edward Snowden !
    1. Re:I tried it once by lightversusdark · · Score: 2, Interesting

      I had a drive fail last year with some personally valuable information on (during the 3 hour window of time I had repurposed its backup drive for a data transfer - sods law, should have just gone and bought another drive).
      I ended up sending it to a data recovery company who told me that the first sectors were corrupt, which is where the "softest" part of the firmware was stored and consequently there was no possibility of recovering the data, even with a controller swap or extracting the platters.
      A friend of mine who works for the UK police data forensics team (paedo-investigator) said that other options that exist were mind-numbingly expensive.
      I guess this is what you did.

      --
      "There is nothing nice about Steve Jobs and nothing evil about Bill Gates." - Chuck Peddle
    2. Re:I tried it once by lightversusdark · · Score: 2, Interesting

      It's a Hitachi 2.5" 160GB manufactured in APR-07 labelled as having "Apple HDD Firmware 2006". It was from a white MacBook.

      The guys that looked at it (ABC Data Recovery in Sheffield) gave me the explanation above.

      My friend said that if that was the case, if the data could be extracted, it would have to be reconstituted by hand which would be expensive.
      He recommended MJM Data Recovery in Somerset to do this.

      --
      "There is nothing nice about Steve Jobs and nothing evil about Bill Gates." - Chuck Peddle
  22. Hmmm. The twisted, it is strong in this one. by Noryungi · · Score: 4, Insightful

    Short answer: no, it's not possible to do that. It's not impossible, simply incredibly hard to do. Unless you happen to be very handy with a soldering iron and go dumpster-diving in the backyard of an HDD company (as many others have pointed out).

    Longer answer: how about creating a 'virtual' hard drive? There are utilities out there that probably let you create (a) a virtual interface (let's say IDE) and (b) a virtual hard drive, attached to said interface. The next step, of course, is to hack these (VirtualBox?) in order to make them do whatever it is you want them to do. This being said, I strongly suspect it involves non-trivial virtual machine hacking and that it probably does not respond to whatever your needs are.

    Other than this (very twisted) idea, sorry, bare metal writing has been disabled a long time ago, and for good reasons, too.

    --
    The right to offend is far more important than the right not to be offended. (Rowan Atkinson)
  23. Solution: by VulpesFoxnik · · Score: 5, Informative

    Old MFM hard drive should do the work. The entire device was controlled via software in dos. I have one lying around. A whopping 5 MB!

    http://en.wikipedia.org/wiki/Modified_Frequency_Modulation

    Now all you need is a 8 bit ISA slot and you are set.

    --
    RES PUBLICA NON DOMINETUR
  24. DRM? by ilo.v · · Score: 5, Interesting

    The cynic in me wonders if he is working for some hack start-up company trying to develop a DRM scheme for a hard drive, similar to the crap the studios do to DVD's to make them difficult to rip. Typically this involves creating some sort of currupt/invalid area on the disk that your particular device will know to ignore, but will lock up anyone else trying to read the data off the drive.

    The optimist in me wonders if he is trying to defeat such a scheme.

    1. Re:DRM? by halcyon1234 · · Score: 3, Funny

      The cynic in me wonders if he is ... trying to develop a DRM scheme for a hard drive. The optimist in me wonders if he is trying to defeat such a scheme.

      The romantic in me wonders if he is trying to create a piece of art. Next week's front-page headlines are like "Hard-Drive Hacker Recreates 'Starry Night' Directly On Platter".

  25. You can't do it. by PhunkySchtuff · · Score: 4, Informative

    It can not be done with off-the shelf equipment and software for many reasons, some of which are:
    Line Codes
    Error Correcting Codes
    PRML
    SMART
    LBA
    etc.

  26. Re:Talk to Steve Gibson author of Spinrite by Joce640k · · Score: 3, Informative

    No it doesn't.

    If I read it correctly (it was a couple of years ago) all Spinrite does is move the head to different tracks before trying to read the data. The theory is that a seek from different distances might align the drive head slightly differently to the data.

    As far as I know there's no way to position drive heads directly via IDE/SATA, much less write individual bits. IDE/SATA are high level protocols, all the low-level logic is inside the drive controller.

    --
    No sig today...
  27. Re:You Da Man!!! by marcansoft · · Score: 2, Informative

    NOR flash doesn't require or use wear leveling, error correction, or bad sector management (as long as your wear pattern isn't ridiculously unbalanced) - use that if you want true "write raw bits" style functionality. Most microcontrollers use this type of flash, and chips are readily available, though they are expensive compared to NAND flash and only exist in smaller sizes.

    NAND flash requires you do your own wear leveling, error correction, and bad sector management. You can still control all the bits, but be prepared to have the chip eat some of them.

    Only "cooked" NAND flash devices such as SSDs and SD cards actually do all that on a built-in controller.

  28. There is a real possibility to do this. by Anonymous Coward · · Score: 2, Funny

    >Any good utilities out there to do this?

    Yes. My solution involves a screwdriver and a sharpie. And you can write bits of information to the bare metal, like "this drive is broken, dont waste your time trying to fix it".

    My colleague was not happy finding the drive on his desk and told me its a low level joke. Well, yes.

  29. Mel, is that you? by evilmatt · · Score: 5, Interesting
  30. 8-bit ST412/506 MFM + Linux circa 1994-5 by aussersterne · · Score: 5, Informative

    Get yourself an old, totally unintelligent S412/506 MFM controller from out of an IBM PC or PC/XT. These were fairly dumb devices (g=c800:5 in debug, anyone?) for which you were meant to enter a "bad sectors list" printed on the front of the drive after performing the low-level format yourself. The earlier the drive, the closer to bare metal, so you might want to look for any of the 2, 5, or 10MB (yes, MB) full-height (2 x 5.25" drive bays stacked on top of one another) drives that were floating around then. You'll also want to get yourself a set of ribbon cables.

    You should be able to use a drive/controller combination like this with any machine with ISA bus slots up through about the 386/486 era, and that would let you also go back and grab an early Linux distro (say, kernel 1.2.13 days, like Slackware 3 or so) that included drivers for such a controller that were actually in use and known to work at the time, giving you a base on which to build more code.

    If 10MB is too small, you might just have luck going up to the largest of the MFM (80MB) or even RLL drives (160-200MB, just get an RLL controller instead) drives. I don't remember whether there were any ESDI drives back in the day that didn't remap their own sectors, but if there were, these controllers were 16-bit ISA and somewhat smarter (also with Linux drivers from the period available) and went up to 680MB or so.

    But if you're looking for the best chance of success for your purposes and don't need tons of storage, my educated guess would be that the MFM controller out of an IBM 5150 PC plugged into a 5MB ST506 hard drive and connected to a SIMM-based 80386DX mainboard with 8 SIMM slots (for 8MB ram) might be the easiest combo to find and get working in practical terms that has a chance of doing what you want.

    --
    STOP . AMERICA . NOW
    1. Re:8-bit ST412/506 MFM + Linux circa 1994-5 by aussersterne · · Score: 2, Informative

      Depending on where you had set the base address of the BIOS in memory (many controllers had jumpers that let you shift it around).

      Some controllers also had multiple routines in addition to the lowlevel format: various kinds of analytics, verify passes, some had self-tests that you had to be careful with because you could tell it to seek to a cylinder way beyond the drive boundary and knock off the heads with a few hundred rapid bangs, which was actually sort of okay because you could literally replace head assemblies and platters between drives sitting on your workbench. Not safe in theory, but actually in practice sort of okay. Once we even forgot to reinsert the air filter (yes, the air filter) in a drive and it started losing sectors much faster than we thought was normal. When we realized, we popped it back out (we used to use the drives in a variety of old minis and a couple embedded systems but manage them on PCs) and put the air filter back in and mapped a bunch more sectors out, and the drive continued in service for another 2-3 years.

      On some you could also use BIOS routines adjust the stepper timing in ranges, from dozens of milliseconds to just a few microseconds, which could rapidly speed up your drive in use, but of course would rapidly lose your data if the stepper couldn't step reliably at that speed, which was often impossible to know if they had spot-sourced the stepper. :-)

      --
      STOP . AMERICA . NOW
  31. Re:Data Recovery by paulatz · · Score: 2, Funny

    Once you can read, you can also write. I've learned it at schools some 20 years ago

    --
    this post contain no useful information, no need to mod it down
  32. Re:Data Recovery by obarthelemy · · Score: 2, Funny

    Good. Go write "Paulatz wuz here" next to the HOLLYWOOD sign.

    --
    The Cloud - because you don't care if your apps and data are up in the air.
  33. Not on modern hardware by AlecC · · Score: 2, Informative

    I don't think what you want to do is possible, because I don't think modern drives actually write the 1s and 0s in the way that earlier ones did. They use Viterbi coding, which means that the exact change to the magnetic field to encode a 1 or 0 depends on what the preceding pattern of 1s and 0s is. They literally can only write sectors at a time.

    Then again, what is meant by a sector has changed with time with different servo technologies. With the embedded servos in current drives, you would need to know a lot about the particular geometry of the particular drive you are using.

    And to do any of this, you would have to bypass the controller completely - which, I am pretty certain, cannot be done without serious hardware modifications

    --
    Consciousness is an illusion caused by an excess of self consciousness.
  34. That's nothing! by L4t3r4lu5 · · Score: 5, Funny

    I'm working on a project that requires writing protons to an atom nucleus in a completely controlled fashion. I need to be able to control exactly where protons will appear physically on the nucleus. Normally when atoms are created during fusion the actual atoms that get created are determined by the energy input of the fusion reaction as modified by whatever kind of atoms are being fused (e.g. hydrogen into helium). All of the modern innovations in nucleosynthesis and alchemy are great for particle colliders and crackpots from the Middle Ages, but they are making my particular task quite daunting. My question for Slashdot: is there a way to get down to the 'bare metal' and transmute lead into gold in a cost-effective manner? Any good utilities out there to do this? Obviously a free and open source solution would be preferable, but I'm open to anything at this point.

    --
    Finally had enough. Come see us over at https://soylentnews.org/
    1. Re:That's nothing! by drewhk · · Score: 5, Funny

      "transmute lead into gold in a cost-effective manner"

      Rob a bank with a gun?

  35. A real (but expensive) solution: by Hurricane78 · · Score: 5, Informative

    You know how data restoration companies do it?
    They take out the spindle with the platters, and put it in their own reading device with its own controller. And with that you can read and write the exact bits (as long as quantum physics allow it). But the head has to be compatible (e.g. perpendicular recording needs entirely different heads).

    I bet those devices can be bought, and I bet their controller is actually just software on the computer (for flexibility). I also bet they come with different head configurations.
    But they are definitely not going to be cheap.

    Hey, at least it is a real solution. :)

    --
    Any sufficiently advanced intelligence is indistinguishable from stupidity.
    1. Re:A real (but expensive) solution: by xtracto · · Score: 3, Insightful

      I like your reply because it is one of the only to suggest a real sollution.

      In a way, the provided answers show the current technically "watered down" slashdot community. I guess 10 years ago this questions would have given really interesting answers.

      I myself had a similar question about 2 or 3 years ago. What I wanted to do was to use the LCD from an HP laptop for some DIY project. The problem was how to interface VGA RGB output with the propietary input of the LCD (made by samsung IIRC). I did a lot of research, got schematics/specs of the particular LCD I was working for and got into the point where I knew I had to create a RGB-to-something decoder/controller (I just let it go so now I dont remember). I even realized that creating the decoder/controller was a really challenging task (mostly time consuming, time I didn't had) or I could buy a 3rd party controller which was quite expensive.

      Although I have very little knowledge of hard disks, I would suggest the OP to first focus on one specific type of drive, second, get the schematics of the drive controller and then depart from there.

      I specially hate these post crying DRM or whatnot. Even though I "am new here" (see my /. ID) I really wish I could find a slashdot like page frequented by real tech geeks /nerds.

      --
      Ubuntu is an African word meaning 'I can't configure Debian'
  36. Are you sure that's your requirement? by Zarf · · Score: 5, Funny

    "I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion."

    Are you sure?

    The reason I ask is I'm working on a project that requires me to move data faster than light. At least that's what I spent last Monday working up the math to prove that data replication between our different data centers has an upper bound enforced by the fabric of the universe and that it was impossible for me to achieve the project's stated goals without essentially inventing warp drive. As it turns out after a meeting it was determined that the goal was just a stated guideline. It also turns out the price of faster data transfer rates is prohibitive and after a further meeting the stated project goal was total baloney. Yes. Baloney. We had sandwiches. It was a nice meeting.

    --
    [signature]
    1. Re:Are you sure that's your requirement? by TubeSteak · · Score: 3, Funny

      The reason I ask is I'm working on a project that requires me to move data faster than light. At least that's what I spent last Monday working up the math to prove that data replication between our different data centers has an upper bound enforced by the fabric of the universe and that it was impossible for me to achieve the project's stated goals without essentially inventing warp drive.

      I have a solution for your problem:
      1. Declare your main data center King. This requires a coronation ceremony and a crown.
      2. Declare your backup data center Crown Prince. This does not require a ceremony.
      3. Push the big red button that kills your King (the main data center).

      Thus, by the laws of royal succession, your backup data center will instantly become King with all the knowledge of its predecessor. And that is how you to move data faster than light without violating the laws of physics.

      --
      [Fuck Beta]
      o0t!
  37. Headlines are superfluous by thasmudyan · · Score: 5, Insightful

    Don't you hate it when people refuse to accept the premise of a technical question and write long monologues why the submitter is working with false assumptions even though they don't know what exactly they are dealing with? Yeah, me too. Makes them look arrogant, ignorant and smug. I'm going to go ahead though and reject the submitter's premise: there is no chance in hell that you're on the right track with whatever project you're attempting to do. But instead of merely dissing you for incompetence, I'll lay out a few scenarios (might as well, since you didn't supply any of your own).

    If the actual physical bits matter to you...
    you're either a hard drive manufacturer or a clueless person who should read up on how drives actually work. And we both know you're not working for a manufacturer. What you need to know is that there are several layers of indirection between the write call from within an OS down to the actual magnetic platter. These layers are there for a reason. At the very least, the onboard controller of the drive abstracts away the physical block allocation, and the drive won't work without the controller at all. Since the intricacies of the drive's physical address space are not accessible from the outside, there will never never never be a reason to try and fiddle with it directly. Because you can't.

    If you are looking for disk I/O without a filesystem...
    we're finally in saner territory. There are valid reasons to do this, e.g. speed and overhead considerations. Some database vendors actually have features like these. In this scenario, you're using the entire drive as one big addressable blob. A good starting point would be to have a look at the source code of a simple filesystem, such as ext2. Strip away all the actual file handling stuff and learn what you can from the disk I/O routines. On the other hand, if you didn't arrive at this conclusion yourself, that's not a very encouraging sign.

    If you simply want a drive without error correction...
    you're not developing software that will run on any modern system. If you accept this caveat, you can buy an ancient drive off ebay and use that. However, keep the first scenario firmly in mind: there is simply no reason to control the exact placement of every single byte if you don't plan on literally putting the drive under a microscope afterwards. Otherwise, this has no practical implications and, again, you are on the wrong track.

    If you're a DRM/malware/virus developer...
    I will sleep very comfortably tonight, because you had to ask about this on Slashdot, signaling once more that you're doing it wrong.

    1. Re:Headlines are superfluous by Anonymous Coward · · Score: 2, Interesting

      except for "If you are looking for disk I/O without a filesystem..." i agree to all your arguments

      in linux though, you have the /dev/(sd[abcde...|012345...]|hd[abcde..|012345...]) block device files, which you can directly access through normal filedescriptors like a normal file.

      These give you linear access to the whole harddisk, accessable through open, read, write, seek and ioctl. Windows surely has something similar, else partitioning programs like fdisk or partition magic would have a real hard life...

      I guess though, that this kid really meant the physical bits, and without building his very own harddrive with its very own controller he is totally out of luck, not even the firmware is capable of that! (rle(yes i am old) and other encodings are done in hardware from what i remember, while the firmware takes over the logical structure of the data positioning etc)

      The bits you send to the harddisk are not what is written to the platter, but a special encoding representing them.

      You might have luck with some 1950's style harddisks, which might not havent gotten to those features yet.. but you will have to find them in a museum

      No kind of harddisk (controller) on any ibm compatible ever gave you access to that part.

      btw harddisk recovery from broken drives is seldom done with special controllers, but through replacing the broken controller board on the harddisks themselves through a similar model

      now back to the part about why he is wanting to do this.... while it is possible this would be used for drm, this would never work realiable without taking care of what the used filesystem wants to do... you need to allocate the used blocks as a file, else they will be overwritten at random times.....

      Even the good old protection style to make use of unused bytes in filesystem blocks at the end of files wont work on most filesystems today anymore, as they can put more than one (small) file into single blocks nowadays to save space

    2. Re:Headlines are superfluous by Idaho · · Score: 5, Insightful

      Don't you hate it when people refuse to accept the premise of a technical question and write long monologues why the submitter is working with false assumptions even though they don't know what exactly they are dealing with?

      In fact no, I hate it more when people do not state their actual purpose, especially in cases where what they are asking combined with the fact that they *need* to ask, cannot prevent one from wondering whether they really understand what they're doing, or whether they are addressing the right problem from the right angle (which, in this case, appears extremely unlikely).

      The problem with this topic is *exactly* that we don't know what we're dealing with, and the first thing any decent engineer would do is to try and figure that out (in fact, you started out to do just that). Surely the stated goal doesn't stop at writing bits in exact locations just for the sake of it, right?

      In fact, if I asked this to a 100 engineers, I'd really expect that at least 99 of them will immediately ask "why the fuck would you want to do that!?". And the one who doesn't probably works at a harddisk manufacturer.

      --
      Every expression is true, for a given value of 'true'
    3. Re:Headlines are superfluous by gardyloo · · Score: 4, Interesting

      Since the intricacies of the drive's physical address space are not accessible from the outside, there will never never never be a reason to try and fiddle with it directly. Because you can't.

      Careful there. I know you're addressing a typical user of the hard drives, but there ARE ways of -- and reasons for -- getting at the "bits themselves". One of my colleagues, as a fresh student at one of the National Labs in the US, was tasked with recovering data from a crashed (literally crashed, so that the controller was destroyed and platters bent, with lots of data loss) disk. Of course there are ways and means of doing so, usually involving fantastically specialized equipment and lots of expense (and, often, the complicity or orders at the Federal level).

            He ended up building a sweet little system which could scan the surface (in a low-level clean-room, of course), albeit slowly, and interact directly with the bits.

            I'll have to ask him how they figured out the filesystem, the error-correction, and so on, without any of the usual partition tables and so forth.

    4. Re:Headlines are superfluous by your_mother_sews_soc · · Score: 2

      I think the original question is valid and shouldn't be dismissed, as well. During the 1980's in the peak of mainframe development, a few of us were writing channel programs, small but really cool pieces of "machine code" that would be executed on a disk controller. We did this to add a big performance gain to our software. Remember, back then memory was expensive and a resource shared by possibly hundred of concurrent jobs (programs).

      With a channel program one could directly control the reading and writing of consecutive blocks, cylinders, and tracks. If you could get cylinder allocations to meet your needs, then all the read/write heads are aligned and you could write to (read from) as many platters the disk had simultaneously. If a disk had 11 platters, you would chain up the commands to point to the data and do a DMA 'blast" of data out to 11 tracks in parallel. Then, you would step out to the next consecutive track and blast another string of data, and so on.

      The neat thing of channel programming is that these systems would also allow you to query the drives capabilities and you could write them dynamically. So no matter what, your software could adapt to the connected hardware. And, since the disk controllers were small computers themselves, once the channel program was set up (and they were just sets of descriptors - commands and addresses, not unlike any other machine code) the work was done asynchronously with your application code. Or you could wait if you couldn't figure out how to do that.

      I think the inability to control disk I/O programatically is a serious deficiency in modern PC's, whether they are desktop or rack-mounted. Remember, the operating systems of today are not meant for people who read IBM's Principle of Operations and Supervisor Services and Macros. They are designed for the least common denominator.

      --
      My user name was a mistake. Input wasn't restricted, my bad.
  38. Really? Do NONE of us read? by True+Vox · · Score: 3, Informative

    No less then *7* posts to the SAME XKCD COMIC?!? Christ, bad enough people don't RTFA, at LEAST read the fine comments!

    --
    "Gratuitous complexity is akin to chaos" - True Vox
  39. He may mean something a bit different. by CFD339 · · Score: 2, Interesting

    Some operating systems have volume management tools that do let you get a great deal more specific about where data is written. Normally, I'm a Linux or Wintel guy, but I have managed some AIX boxes a few times and from setting up a large one once I recall that it's logical volume manager allowed you to create volumes on the SCSI drive systems that were very specific about how and what part of the drive was used. You could specific, for example, that a particular volume use only the outer tracks (or inner or middle I suppose) of the drives -- in addition to a great many options for raid striping and transaction logs and so on.

    The idea is that the outer edges of the platters travel faster under the read/write heads than the inner and so performed differently. Also, you could keep the head from having to jump around as much by keeping all the data that tended to be used at the same time on the same tracks -- reducing your average seek time when reading randomly from that pool of data.

    In practice, I think this mattered a great deal more when us old guys were dealing with 80+ millisecond random seek times on 5 1/4" wide full height (what would now be considered two bays) drives -- or larger disk-pack based drives (aka washing machines) with the massive physical movement necessary for those read/write heads.

    Today, I think the admin is better focused on distributing the data load better across different drives/arrays to even the load out and also focused on reducing overall disk i/o in creating database schema and applications. Focusing on very fine details like this is likely to have only marginal benefit next to those key areas --- but I can't presume to know for sure what this project has in mind.

    --
    The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
  40. One more possibility... by Schraegstrichpunkt · · Score: 2, Funny

    Randall Munroe

    1. Re:One more possibility... by bsDaemon · · Score: 2, Funny

      Yeah, but he's a CIA troll who's writing a cyberpunk thriller one panel at a time, so isn't he the 'all of the above' option?

  41. It CAN be done! by symes · · Score: 3, Funny

    You will need a reasonably sized HD, like this one where ones and zeros occupy a space of about postage stamp size.

  42. Re:Data Recovery by RivieraKid · · Score: 2, Insightful

    Wow, nice contribution. You totally miss the point, which is that writing arbitrary magnetic fields to an arbitrary physical location on the platter is the very last thing you want to do during data recovery. Now, if you just want to write arbitrary data at arbitrary logical locations on the disk, then you can already do that with the disk in it's current state.

    Neither of these scenarios are remotely data recovery, since by definition, you are overwriting data.

    For what it's worth, I'd disagree that once you can read a human language, that you can also write it since there are visual and linguistic cues in written language that you have to provide when writing. In most people, the two are closely linked, but I'd be willing to bet that while learning to read/write, your reading aptitude probably has a slight lead over your writing aptitude.

    --
    "Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves
  43. What kind of answer do you expect by hey! · · Score: 2, Insightful

    if you describe your requirements so abstractly? It sounds to me like you've come up with a solution ("Let's write the bits directly to the hard disk") and now you want pointers on how to make it work without explaining why.

    If you just want to get around the file system, then use raw block I/O to the device. However it *sounds* to me like for some reason you've decided the *actual physical* layout of bits on the drive is important, in which case you are going to have to write your own disk driver -- if the drive electronics themselves don't defeat your attempt to know where bit is physically written.

    It's inconceivable to me that you'd actually need to know this. Since you only *read* the data through an interface, it should make no difference if you *write* though the same interface, as long as it's consistent. In other words, unless you are going to disassemble the drive and examine it with an atomic force microscope, you have no way of telling the difference between a physical layout and an equivalent *model* of a "physical layout".

    If you can't say *why* you need to do this, at least explain the parameters (how much storage, how fast, what kind of retreival etc.). My first reaction was that you should not use a magnetic disk at all, but an MTD flash device. Even so, you're dealing with an abstraction. You have no idea whether the device itself has mapped a bad bit to a different location at the hardware level. However short of tearing the device apart and putting the flash chips in a special circuit, you'd have no way of telling.

    --
    Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
  44. Re:ob XKCD reference by ShadowRangerRIT · · Score: 2, Informative

    It's been posted and modded to oblivion multiple times.

    --
    $_ = "wftedskaebjgdpjgidbsmnjgcdwatb"; tr/a-z/oh, turtleneck Phrase Jar!/; print
  45. Re:ob XKCD reference by laejoh · · Score: 3, Funny

    Nice, 'course there's an emacs command to do that, good old C-x M-c M-modintooblivion...

  46. Re:You Da Man!!! by hedwards · · Score: 2, Insightful

    Considering he's asking this question to begin with, what makes you think he hasn't gone insane already? This isn't something that most people technically inclined enough to program would be silly enough to even consider asking.

  47. Re:Talk to Steve Gibson author of Spinrite by washu_k · · Score: 2, Informative

    The GP is right, Spinrite does not and can not do most of what it claims on a modern HD. The only potentially useful thing it does is re-read sectors up to hundreds of times, but there is safer software that does the same thing.

    More importantly, Spinrite can be downright dangerous to your data. Spinrite writes any data it recovers TO THE SAME DRIVE! It can very easily overwrite other data you want to keep and/or lose the recovered data again because the drive isn't stable enough to hold it reliably. Not writing to the damaged media is data recovery 101 and Spinrite fails badly.

  48. Not a Stupid Question - And a suggestion by Jeremy+Lee · · Score: 4, Interesting

    I'm surprised by the number of people who seem hostile to this question, mostly because they can't imagine any circumstances where they would want to do this. I can think of several. In fact, I was thinking about something similar with optical disks just earlier today. (I'm curious whether small holograms could be created by writing an interference pattern directly onto a CD or DVD, but I would need exact control over how the tracks lined up to achieve this.)

    Here's some concepts just off the top of my head:
    * He's come up with a new disk encoding scheme, and wants to try it out.
    * He's doing research into how long disks retain data, and is questioning basic assumptions like whether surface bits permanently magnetize the platter underneath, affecting later bits in the same location. Or how far the domains spread.
    * He's working with self-assembling molecules and needs to give them a patterned magnetic surface to build on.
    * He wants to 'print' a 2-D picture onto a small portion of the hard drive, and bounce a laser off it. (The magnetic alignment of the surface domains would polarize the photons slightly.)
    * He's making a high-resolution rotational position encoder, so by reading across 32 tracks knows the rotation of the platter down to a few nanoarcminutes.
    * He wants to totally destroy the contents of a disk. (I assume this is what most of the hostile people think is the intention, presumably as the payload of some virus)

    Those are just the ones off the top of my head.

    However, the ranting people do have a point that without knowing WHY you want to do this, we can't really suggest the best solutions. A lot of people have recommended going back to super-old MFM hard drives that allowed this, but we don't know if you require the density of modern hard drives.

    To do this with a modern drive, you're basically going to have to rebuild the controller. Either totally remove the controller board (leaving handy raw connections to the stepper motors and drive heads) or cut the connections between the microcontroller and the low-level electrical functions of the drive, and substitute your own. Here's where knowing your accuracy requirements could have help, because if you want relatively large bits, you can get away with fairly low-frequency components. A 20Mhz microcontroller can, with say an external high-speed shift register, push out an 80mbit/s serial stream, which equates to >120,000 bits around the track of a 6,000RPM drive. Not quite the same density as the manufacturer, but better than the old MFM drives.

    Your next problem is going to be this: It's really, really hard to tell where the head is on a platter. I've no idea how modern drives do it, but it used to be done with 'marker' bits either in the track, or on a nearby 'index' track, plus a little timing. Preserving these location marker bits is actually the most important job of the drive controller.

    It's not impossible, merely very difficult. I could probably make one with only several weeks of hard work.

    Alternately, you could try getting into the firmware and re-writing it to your specifications, but that might take longer. You would have to reverse-engineer a lot of stuff that is specifically hardened against this, but at least the hardware would be stable.

    --
    Jeremy Lee | Orinoco
  49. Well, since we're dancing around the short answer by EriktheGreen · · Score: 2, Insightful

    I'll give the direct/short answer to the question you're asking.

    No.

    The reason is that hard drives only write data in precise locations so they can find it later. You can't write anywhere other than those locations because the drive won't do it... not even with new firmware. The read/write heads may not even be capable of addressing the locations you want to write. The only way to write in an arbitrary location is to remove the disks from the drive in a clean room, and use a very precise CNC read/write head to address the locations you want. Disk manufacturers have machines that do this.

    If you confine your request to only writing arbitrary locations within the physically addressable areas of the disk, then you can do what you want if and only if you write new firmware for the drive... it used to be firmware was in EPROM and couldn't be altered, but you can flash it nowadays. So you A) Buy a drive then B) Re-flash the eprom with code to do what you want (custom developed after reverse engineering the original firmware) and then you can write wherever, without worrying about niceties like the end of sector marks the drive uses to keep track of data locations. You'll have to do that sort of thing yourself.

    So, confining our discussion to disk areas that the default drive firmware will write to, If you're looking for a way to ignore/override the OS I/O code, the disk controller firmware and the disk firmware in order to "talk to the bare metal" it may be possible depending on the particular combination of OS, controller, and disk, and assuming you have the right privileges in the OS. But there's no standard way to do it, nor API. You have to know exactly how the hardware in question works, down to the chip level and in some cases below. If the computer you're interested in has a different drive, controller or OS I/O code than you wrote your program for, then you have to re-write it.

    I'm sure a lot of us here would be curious to know what you're trying to do... accessing non data sectors on a disk hasn't been done commonly for years, and when it was it was used for some awful copy protection methods (awful as in they created compatibility issues, even with "standard" PC hardware and also they could still be broken).

    Erik

  50. Why ask us if you already know the answer? by mcmonkey · · Score: 2, Insightful

    Remember this next time you are struggling getting requirements out of a non-technical manager or user. The submitter obviously has the technical background, but is making a common mistake.

    What is it you want to do? "I want to write bits directly..."

    Really? That's your final goal? Just to write bits? No, there's some other task you want to complete, and you've determined the best way to complete that task is to write bits directly to the platter.

    But if you can't write bits directly to the platter, or you don't know what additional issues may arise when you do so, how can you determine that is the best course to take?

    So slow down, back up a step, let us know what your real goal is. You want X, and you think the best way to X is to write bits to the platter.

  51. Re:No. by number11 · · Score: 2, Insightful

    You would need to completely bypass the drive firmware in a way that would make the drive quite unusable afterwards

    "Afterwards" wasn't part of the spec. He just wants to write the 1s and 0s. Reading them wasn't part of the spec that he gave us.

    Seriously, if he just wanted to use a HD for data storage and retrieval, he already knows how to do that. Maybe he just wants a magnetic platter decorated with his logo printed in magnetic domains, a la lightscribe. Or something entirely different. Since he didn't tell us, but yet wants to do something so unusual, we can't make assumptions.

  52. Tell us *what* you are trying to do, not *how* by Snowhare · · Score: 2, Insightful

    Almost every time someone asks a question where they obviously have made an implementation decision that depends on "doing it differently than everyone else on the planet" while providing no information about what they are trying to actually accomplish, the problem can usually be solved in a much simpler way. While it is possible you are doing something exotic like trying to turn a hard drive platter into a meta-material by patterning high density magnetic patterns on it and so you really do need to be able to control the bits at the hardware level, odds are low.

    You provided no information about what you are trying to do. There are pretty good odds that if you provide information about what you are trying to do instead of trying to get people to come up with a way for you to do it 'the hard way' you, will get an answer that will work for you.

  53. Several things you need to know. by Measure+Twice · · Score: 2, Informative

    First, ones and zeros on a harddisk platter are not magnetic north and south, they are different frequencies of an FM sine wave. Second, the formatted tracks on a harddisk don't line up physically. The start of the track is wherever the head was when the controller was told to format that track. Third, Sectors aren't written in precise places inside their formatting. The low-level formatting process writes headers across the track that are used later to find the data. There's enough extra space between those headers that a sector of data will fit there, based on the ability of the controller to read where the sector starts and be able to write it. If you really need to write magnetic north & south to specific spots, you should look at Floppies. Floppy disks have a hole to mark one spot on the disk, so the start of track comes much closer to lining up. You can even write ones and zeroes instead of FM patterns if you have an old Apple II, which didn't use FM encoding.

  54. All drives encode the bits by mscritsm · · Score: 2, Informative

    If you want an exact image of 1's and 0's that you determine, forget it. Bits these days are encoded as transitions between magnetic phase boundaries. They also encode the clock data needed to recover the data along with the data. As a result, what's put down on the disk has to guarantee a certain number of these transitions or you loose sync. In other words, you may think you're writing '0000000' but on the disk it's a bunch of magnetic transitions, not a constant stream of the same magnetic polarity. It's not even that a single phase transition translates into either a 1 or a 0; groups of these transitions translate into a group of the actual data bits. Some patterns of phase transitions can simply never be written onto the media by the write electronics in the first place because they would not result in reliable data recovery by the read electronics.