Ask Slashdot: How Does One Verify Hard Drive Firmware?
An anonymous reader writes: In light of recent revelations from Kaspersky Labs about the Equation Group and persistent hard drive malware, I was curious about how easy it might be to verify my own system's drives to see if they were infected. I have no real reason to think they would be, but I was dismayed by the total lack of tools to independently verify such a thing. For instance, Seagate's firmware download pages provide files with no external hash, something Linux distributions do for all of their packages. Neither do they seem to provide a utility to read off the current firmware from a drive and verify its integrity.
Are there any utilities to do such a thing? Why don't these companies provide verification software to users? Has anyone compiled and posted a public list of known-good firmware hashes for the major hard drive vendors and models? This seems to be a critical hole in PC security. I did contact Seagate support asking for hashes of their latest firmware; I got a response stating, "...If you download the firmware directly from our website there is no risk on the file be tampered with." (Their phrasing, not mine.) Methinks somebody hasn't been keeping up with world events lately.
Are there any utilities to do such a thing? Why don't these companies provide verification software to users? Has anyone compiled and posted a public list of known-good firmware hashes for the major hard drive vendors and models? This seems to be a critical hole in PC security. I did contact Seagate support asking for hashes of their latest firmware; I got a response stating, "...If you download the firmware directly from our website there is no risk on the file be tampered with." (Their phrasing, not mine.) Methinks somebody hasn't been keeping up with world events lately.
This is pointless without JTAG hardware to directly access the flash memory.
Normal users would read/update the firmware through the existing firmware, so if that's been tampered with there's no way you can be sure.
Firmware is usually not signed. Furthermore, I am not even sure that most drives support reading the firmware. Overwriting with a "fresh" firmware might also be impossible, since I assume it happens through vendor extensions of said firmware. A malicious version could be able to thus protect itself.
In the end, such an elaborate scheme is probably directed towards very high value targets. I don't think this is the kind of trojan that runs out in the wild. I could be mistaken, though.
Like you, I do wish it becomes more secure in the future. If anyone has a list of vulnerable targets (brands, models etc), I would be interested to know.
Best solution is to wrap your hard drive in tin foil to keep the harmful mind-rays out.
Most likely there are no such tools as no-one thought it could be a vector of infection. Just like the BIOS; which used to be a non-reprogrammable ROM chip. I for one didn't know current hard drives even had firmware that can be replaced by the user, let alone that it may be a potential attack vector for malware.
Depending on how hard it is to read the installed firmware from a hard drive (is this even possible in the first place?) it shouldn't be too hard to write a tool that can read the firmware, and calculate a checksum for verification. The hard part is going to be, how do you know that your software gets the actually installed firmware - or just a known good but inactive piece of code provided by a compromised firmware, pretending that this is the software that's installed? The moment a firmware is installed, you probably need to call onto that very firmware to get a copy of it from the drive. Unless this read-firmware routine is provided by a special, hard coded circuit.
Seagate is correct. Putting a hash on the website doesn't improve security at all because anyone who can change the download can also change the web page containing the hash.
The fact that this practice is widespread in the Linux world originates from the usage of insecure FTP mirrors run by volunteer admins. There it's possible for a mirror to get hacked independently of the origin web page. A company like Seagate doesn't rely on volunteers at universities to distribute their binaries so the technique is pointless.
A tool to verify the firmware is poetically impossible to write. What code on the drive would provide the firmware in response to a tool query? Oh right ..... the firmware itself. To make it work you need an unflashable boot loader that acts as a root of trust and was designed to do this from the start. But such a thing is basically pointless unless you're trying to detect firmware reflashing malware and that's something that only cropped up as a threat very recently. So I doubt any hard disk has it.
BTW call a spade a spade. Equation Group == NSA TAO
Kid, take your meds and fuck off.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
I'm still waiting for the first CEO to go to jail for refusing this.
Either it's easy to say "No", or nobody bothers, because "war against terror etc.".
Windows 2000 - from the guys who brought us edlin
If you ask the drive to read out the whole flash.
The maybe the firmware would have to go to the platter to get the real image.
This might have a different power consumption or sound than a real drive going to the flash?
Might not work for a solid state drive.
JTAG seems safer.
WTF? No stories by him have been posted here in weeks. Find a new anti-meme or GTFO.
We need jumpers or physical switches that prevent firmware stored in flash (whether it be GPU firmware, BIOS, HDD firmware, network card firmware or whatever) from being overwritten unless you specifically flip that switch.
Professional? No contractor could possibly take the spyware, as Edward Snowden did of an enormous number of records?
News stories seem to assume that the NSA is well-managed, but secret agencies have secret management. Managers of secret agencies can keep their mistakes and bad management secret.
Jumping to page 3 of the article in SpritesMods.com: Parts on the [hard disk] PCB: "My target was to try and compromise the security of a system by using hard disk firmware mods."
Actually, writing a verification firmware is possible (assuming, that it was written after attacking code was written and writer was not NSLed)
Simply because attack code doesn't know what output verification code must produce. It either must execute new code (will be busted) or not (will be busted). Putting a full blown interpreter (or some trap mechanism on flash access) will screw timing - again it will be busted.
should be that firmware is firmware. Please test rudimentary blocks of computing devices before you produce 100 of millions of a series.
I expect the manufacturer actually does something like a read/write test for typical conditions.
I may even accept or wish to get HDs which are one year behind SOTA, if they were not pushed out of the door in whatever shape the SW is due to a marketing deadline.
For such device i happily would pay more, if the "programmable" fuses are set/burnt.
Look at the Dells that were bootjacked by the NSA. Dell had likely provided the source code to the bios to the NSA for approval for government work, and as a result, Dells BIOS got some unwanted extra NSA code which let NSA take control of Dell Servers across the world.
http://www.zdnet.com/article/nsa-hacked-dell-poweredge-server-bios/
So you would likely flick the switch trusting it was Dell BIOS update yet its a BIOS update carrying spyware.
Then of course the other problem, NSA gets Fedex/UPS/TNT or others to divert a laptop to a local blacksite where the spyware is installed. Did that kit you bought online take a detour? Mine did.
http://news.yahoo.com/nsa-intercepts-laptop-deliveries-install-spyware-143011341.html
Of course any physical switch would simply be flipped for the malware install.
What is the process of writing a new HDD firmware? Does the drive listen for some specific ATA command followed by the program data?
How much CPU power is in HDD controllers and how big is the flash storage on the controller?
I'm mostly just curious, but I wonder how much "elbow room" there is to do something nefarious like blocking updates or protecting boot sectors without compromising drive performance significantly.
Is there a mechanism for running software on the drive controller -- passing input, getting output, etc?
Actually, the much hated Secure Boot (with the shim loader, MOK, and GRUB2), combined with full disk encryption (for example using LUKS), and in filesystem compression (btrfs2) can quite nicely protect you from anything that a malicious firmware in a harddrive could do. The firmware will only ever see encrypted data passing through it, except for when loading the bootloader and the kernel, which will both be cryptographically verified by UEFI. The in-filesystem compression is there to compensate for the compression SSD drives normally do themselves to gain additional speed that will be impossible to do that on encrypted data.
Sure, this basically converts the problem to trusting the main BIOS (UEFI), but that's something you have to solve in any case.
Have a read of this: http://spritesmods.com/?art=hd...
There is a decent chunk of compute power in the controllers.
SSDs
Learning HOW to think is more important than learning WHAT to think.
Well, if you're the drive controller firmware, storage space is a non-consideration for the most part. You can store stuff on the drive platters. A certain percentage of those are reserved for sector reallocation anyway, so you could use those without anyone really every being the wiser.
That was fascinating, thanks for the link (and the lost 45 minutes reading about the guy's other genius hacks).
I have long believed that if it was as hard to maintain a car as it is to administer a computer, the world would stay home and read books.
It is the manufacturers' responsibility to ensure that hardware does what it is supposed to, does it correctly, and does ONLY what it is supposed to do. In the coming age of self-driving cars, personal care robotics and so forth, it is inexcusable for the builder to make defective stuff. I suspect we will have to go back to first principles instead of relying on software recipes that were invented back in the 1960's.
We will run into another level of complexity when our machines start modifying themselves for something called "better performance."
This does not answer your question, but this is a legitimate area for concern and I thank you for bringing it up.
"The mind works quicker than you think!"
Because it's never been an issue before.
If it's about seagate hds you can take a look at seaget.With this you can dump the buffer and memory of your harddrive. Here is an explanation https://blacklotus89.wordpress... and here is the code https://github.com/BlackLotus/... Maybe this can be used to dump the firmware as well (somehow)
Boot from a randomly chosen Linux rescue disk, and check the various proms. You've used the boot rom to boot a CD/DVD, but what you've booted is wildly different from the Windows systems that are the common target, so the attackers will have great difficulty in hiding what they've done from an unfamiliar system.
It's actually easier to hide evil stuff in disk proms, as your only access to them is via routines *in* the disk prom, as one of the other commentators pointed out,
davecb@spamcop.net
Are these root vectors playing the odds and assuming they'll be installed on an x86 machine running Windows7, so they put that payload in the firmware?
It's not like the firmware has an IP stack.
It doesn't take very many bytes to make one. And your hard drive is communicating over a bus. You'd be surprised what types of communication protocols are recognized over various internal data paths... How do you think those old Ethernet-over-SCSI adapters worked?
Hire a Linux system administrator, systems engineer,
I doubt you need much, really.
All the malware part has to do is to read the rest of the software from disk upon boot, then hide that part of the drive from the OS. This way you could hide a pretty big piece of software on the disk, and with today 500 GB kind of capacities being the norm, the user won't notice unless they look really really carefully at the numbers.
Dear Mr. Low-ID, here's your Interwebs 101: Don't feed the trolls. That is, unless you're in for quality counter trolling.
CLI paste? paste.pr0.tips!
Here is the problem:
Manufacturers guard their intellectual property fiercely, and they guard their proprietary firmware fiercest of all. Thus the API for uploading drive firmware is Write Only (WO). Thus within the existing API and interface there is by design no way to validate the firmware. What that means is that, if you are able to build your own firmware (because you have a copy of the source, obtained deviously) then you can alter it to your own ends and even make it so that the (WO) overwrite API does nothing.
Outside of the existing interfaces though you can with sufficient skill get some knowledge. If the firmware is stored on a flash chip separate from the drive CPU you can get a copy of the microcode by probing the chip directly either during read cycles with the drive active or by controlling the chip fully with the drive off. Unfortunately you cannot do this so easily if the firmware is stored in flash within a drive micro-controller. As to JTAG, that may or may not work because in production a manufacturer may choose to disable that interface to prevent competitors doing exactly what you are wanting to do.
In Summary, you are SOL unless manufacturers rewrite their firmwares to add a secure means of proving firmware validity, and don't ask me how.
MOD PARENT UP
So, you're saying that it's so simple to get SHA256 collisions that thousands of people getting sued for torrenting can fuck these copyright companies right over?
I don't think I quite believe you and last time I checked I needed quite a server farm to (reliably) produce one collision in a meaningful amount of time.
Which is why I always laugh my ass off at all these people who use PGP to sign things and put a hash on the same website you download it from ... look you can verify this file you downloaded from the website hasn't changed because theres no way anyone would be smart enough to update the hash as well!
That's why you SIGN the hash. Then only the public key needs to be published by a different route.
And it doesn't HURT to publish it on the web site as well: Then someone tampering by substituting a different public key sets off alarm bells when that differs from the public key obtained from another site or by another path. Blocking that makes man-in-the-middle more complex: The attacker has to have essentially total control of the path to the victim and be able to recognize and substitute the public key whenever it shows up. One slip-up and somebody may raise the alarm.
Meanwhile: Even if publishing hashes on the same site may not provide additional security against MITM, it DOES let you check the download wasnt corrupted in transit (in ways other than malicious substitution). With modern protocols that's less of a problem these days than it used to be, but a check would be comforting.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
If you ask the drive to read out the whole flash.
The maybe the firmware would have to go to the platter to get the real image.
Or the malware could regenerate the un-attacked version.
For instance: If it's a patch that loads into an otherwise cleared-to-known-vallue region it can detect that region while reporting flash content and report the cleared value, instead. Add a couple other tiny regions where it saved (or alread knew) the previous contents where it "sank it's hooks" and you can't tell it's there from its replies to dump requests.
JTAG seems safer.
Yep. JTAG, in principle, could be corrupted. But it would require substantial hardware support that almost certainly isn't there (yet!)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
Do you think an infection would be more likely to come from hackers that somehow modified the drive after the manufacturer shipped it, or do you think it is more likely that the NSA (or whoever) already work with the manufacturer?
If the latter, chances are the option with least security risk is to not update it at all, especially if its an older drive.
We need the actual sources and currently there are very few companies who have done this for even some chips. I can literally count on one hand the number chips with whoms code has been released. AR9271, AR7010+AR9280, AR9170. There is one other I believe for which I'm leaving out as I forget what it is. There are sources for at least one or two reverse engineered chips too I believe.
Sadly the responses you'll get if you inquire are “what do you need that for” and “there are so many other places with proprietary code its utterly pointless”.
The problem with this thinking is its not pointless. We simply have to work toward getting companies to release more code than they currently do. Getting sources released for a stage-one bootloader might not be significant in and of itself, but it is a start. Right now we can't really design one good system where all the sources are available because there are components where there aren't any options where the complete set of sources is available. Short of leaving those components out you probably won't be able to do it right now. You can essentially say that no individual or state is secure. Period.
What people are forgetting is that there are numerous state adversaries whom are actively attacking our security. Even if you aren't concerned with the US government you should be concerned with security in general. There are near 200 states all of which could be significant adversaries not to mention other non-state actors. There are numerous criminal organizations, corporations, resistance groups, and drug kingpins all of whom would love such vulnerabilities and/or backdoors.
With some performance compromises we're not that far off from having secure computing options. Give enough interest and in a few years we could do it.
Nuke it from orbit. It's the only way to be sure.
Get free satoshi (Bitcoin) and Dogecoins
And those proms are read by the firmware in those proms. Why does the firmware have to respond with what is actually running?
BTW call a spade a spade. Equation Group == NSA TAO
Not really, it may have once been NSA but once the code was discovered it could be anyone using it. It just becomes another "tool" in the "public" toolbox.
Just because you don't understand something doesn't mean it doesn't happen. It means you don't understand. The mere fact you didn't know firmware was upgradable means you are unqualified to tell us things aren't a big deal.
Depends on the checksum - I understand the better ones like SHA and maybe even MD5 are relatively irreversible, making them extremely difficult to spoof. Yes, you could do it if you had enough resources to throw at the problem, but I'm betting even the NSA probably has better things to do with their computation centers that week (month? year?). Sure, SHA-1 is only 160 bits, but that's still 10^48 possibilities, and you'll have to recompute the hash with a sizable fraction of that many different fillers to get a false-match.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
The best we can hope for is systems offering assurances they can be restored to their initial state. The only way to do this is for hardware to physically lack any means of remembering.
If you don't have proper reading comprehension skills, you are unqualified to reply.
No-where did I try to say it's not a big deal.
Only on-disk, non-addressable controller proms are "read" by the software in the proms.
The boot prom has to boot stuff or the product can't be sold, and in this case is used to boot a program that runs on the hardware that continuously reads the prom. That HW can verify it, and all the other proms which are reachable from the CPU, including all sorts of stuff plugged into the various busses. That includes some disks, the ones we were worried about viruses wiping.
For some specific disks, you may have to pull the drive and clamp directly to the prom's pins.Those are the ones a spy would want to subvert.
davecb@spamcop.net
We need open source hard drive firmware.
We also need open source integrated circuits.
Even so, having a physical switch is already helpful. I don't need to worry about the firmware malware if I don't ever have the intention to flip the switch. I only need to take measures to secure the computer against the malware if I plan to flash the firmware. A physical switch is a very powerful countermeasure to thwart remote attackers. U2F tokens also use it to secure two-factor authentication.
I once had a signature.
I'm thinking this might be similar to what some of the video card manufacturers have done (such as with the R9280X cards), where a physical DIP switch on the card selects between firmware flash A or B. If you suspected corruption, you could flip the switch to use the alternate, which presumably would be loaded from the factory with good, working firmware of whatever version was most recent at the time the product was manufactured.
I suppose this would technically only give you "one shot" at recovering from a firmware hack ... but better than nothing, right? And in the meantime, it would give protection to people from such things as a corrupt flash update or a way to do an easy A/B comparison between 2 firmware revisions.
If you cannot audit the source code of the hard drive firmware, you must keep hard drives outside of your circle of trust. That means that all hard drive traffic should be encrypted with keys not available to the hard drive. Digital signatures and time stamps can also be employed to ensure that the drive isn't utilizing replay attacks or swapping blocks around. As a bonus, this protects against failures in the transmission path, in even stronger ways than ZFS uses checksums. And remember, once you're out, you're out. There's no coming back.
When the code executing the CPU resides on the hard drive, compromising the hard drive gives you everything. In addition, hard drive controllers and network controllers could be compromised to provide direct leak paths without involving the CPU using DMA.
There is enough flash and ram to run Linux on the controller. I've seen it done at Ruxcon/Breakpoint where the hard drive booted up to the point where it couldn't find a root disk to mount.
It is trivial to make firmware that watches for things like /etc/shadow files and returns something else. You can have this code activate by searching for data that would be logged and hunting for the magic key and that is trivial since every system logs to disk.
Re "Basically the keys to the castle without presenting any evidence that your doors are all open. as useful if not more so than UEFI takeover."
Ready to go after any wipe or AV or heuristics, behavioral analysis. Collecting data and waiting for a network or usb stick out.
Domestic spying is now "Benign Information Gathering"
For downloads of updates, yes, checksums and the like can and probably should be widely published. As others have said, having the checksums ONLY on the vendor's web site probably isn't any good but if they were "all over the web" they would essentially be tamper-proof.
Better than a checksum would be a cryptographicly signed by a public key that was issued by a major company that you trust already.
As far as the firmware that is on the drive:
Unless you have a way of directly reading the firmware memory without using the firmware itself, forget about it. Any attempt to ask compromised firmware to give you a data dump of itself would likely just get it to lie to you. Yes, there is probably equipment out there that can read the chips but you probably don't have it and you probably can't afford it unless you are doing it as a business or as part of a larger business (such as computer manufacturing, where you may want to validate that OEM drives contain the firmware that should be on them and not the ones that $SPYING_GOVERNMENT_AGENT installed).
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Just checked, apparently mine is version
WD-3.02.6-NSA-1
This used to be easy because the write protect switch could operate either through the high voltage programming supply or the write strobe. Internal charge pumps have obviated the need for an external high voltage programming supply and embedded Flash has no write strobe to access.