Hidden Debug Mode Found In AMD Processors

An anonymous reader writes "A hidden (and hardware password protected, by means of required special values in processor registers) debug mode has been found in AMD processors, and documented by a reverse engineer called Czernobyl on the RCE Forums community today. It enables powerful hardware debugging features long longed for by reverse engineers, such as hardware data-aware conditional breakpoints, and direct hardware 'page guard'-style breakpoints. And the best part is, it's sitting right there in your processor already, just read the details and off you go with the debugging ninja powers!"

Comments

Just a matter of time...

[TrisexualPuppy]

One of my pals at NVIDIA was talking about this in a generic sense. Evidently, all of the big design houses have reverse engineering departments where they scrape down to the silicon and get things running. They never make any public info, but it's crazy what kind of logic blocks they find on silicon.

These exist on "all processors" as ways to test the processors and increase yield cheaply. The moment that the engineering samples go out, competitors get their hands on them, and it's only days or weeks before they figure out what's really going on. Kind of cat-and-mouse.

Re:Just a matter of time...

[TrisexualPuppy]

By the way, here's a guy who does this in his spare time. He may not have the $10+ million budget that the big boys have, but it should give you a little context as to what really happens in industry.

(As the original article was instantly slashdotted, I can only guess that the AMD exploit was found through software avenues.)

Re:Just a matter of time...

[GWBasic]

They never make any public info, but it's crazy what kind of logic blocks they find on silicon.

Sometimes scraping can tell simpler things, like an accurate estimate about how much profit a company is making on a chip, and thus how much money the company will have to invest in its next generation of chips.

Oops, slashdotted!

[ThreeGigs]

Apparently the debugging process doesn't help to combat the effects of a thorough slashdotting.

Re:Oops, slashdotted!

[Danathar]

yea, why stories continue to be posted with direct links instead of using things like coral cache is beyond me. If you KNOW the site you are going to link to can't handle a slashdot load then DON'T LINK DIRECTLY TO IT.

Of course this does not include sadistic evil people who enjoy watching websites crash and burn (probably a sizable but not large percentage portion of the slashdot community)

Security?

[JSBiff]

I wondered the same thing - if these debug features are useful to developers debugging their own software, why not market this as a feature? The only thing that occurred to me, is that, maybe there is some sort of security problem with this debug functionality? Does anyone know - could these debug features be used to do something like break Operating System security models, leading to privilege escalation issues, or for other nefarious purposes?

Re:Security?

[Anonymous Coward]

It is possible that the debug features are for their internal use and they don't quite work as intended. They may be useful as such, but if there are implementation bugs that require cumbersome work-arounds on the software side, it may be that they are waiting for a non-buggy implementation before publically documenting the features.

It is also possible that they don't want to put the resources to documenting and supporting the debug features. After all, AMD is a small company compared to Intel and not that profitable. They even had some layoffs during the worst recession - what if they had to lay off the guys responsible for the debug mode?

Re:Security?

[slashqwerty]

Does anyone know - could these debug features be used to do something like break Operating System security models, leading to privilege escalation issues, or for other nefarious purposes?

If there is some way to enable privileged instructions without using a privileged instruction it would completely circumvent operating system security.

Perhaps the slashdotted site answers this but I have to wonder why not just have a separate opcode to turn the debugging on?

Re:Security?

[fuzzyfuzzyfungus]

Any CPU debug mode worthy of the name should be able to violate OS security six ways from Sunday, and silently at that, without any difficulty. By the same token, though, any CPU debug mode worthy of shipping in commercial silicon really ough to be possible for the firmware and/or kernel to lock for the duration of operation. If userspace can kick it off, a brave and exciting new world of AMD-specific malware is about to begin...

Re:Security?

[TrisexualPuppy]

It's probably that AMD doesn't want to claim that they ever marketed the feature as such. If they did, it would put Intel up to create and release a debugging interface for their silicon. Then both would be forced into competing to produce a better debugging interface. This drives production costs up for a component that may be used by less than 1/100 of a percent of the users when they should have been putting their efforts elsewhere.

Re:Security?

[Anonymous Coward]

Also, if it's triggered in userspace, the OS can block it.

Not if your OS is rooted. Or root-kitted. Or both.

Re:Security?

[Anonymous Coward]

if it's triggered in userspace, the OS can block it.

Not necessarily. Memory access can be blocked because the MMU controls what and where an application can write, transferring control to known code (the OS) on violations. Interrupts can be blocked because invoking them gives control to the OS. Priviliged instructions can be blocked because non-ring0 execution gives control to the OS.

The OS can't choose to block for example the "xor" instructions in any reasonable way. It's possible by basically single-stepping through instructions at 100x the overhead, but it's not feasible.

Re:Security?

[camperdave]

it would put Intel up to create and release a debugging interface for their silicon.

Maybe Intel already has a debugging interface on their silicon. This AMD interface has remained hidden for who knows how many years, why couldn't the same thing happen with Intel? After all, it's not as if just anyone can reverse engineer a CPU.

Re:Security?

[jimicus]

IIRC, microcode updates are typically packaged as part of BIOS updates.

Tell me, how many PCs - as a percentage of those sold - get their BIOS regularly updated?

Re:Security?

[TheRaven64]

Intel has provided debug registers for ages. You can have up to four hardware watchpoints in pretty much any Intel - and AMD - chip. TFA is Slashdotted at the moment, but 'hardware data-aware conditional breakpoints, and direct hardware 'page guard'-style breakpoints' can both be implemented on any chip since the 386.

Re:Security?

[TheRaven64]

Okay, the site's still down but I found a cache:

In fact, this is a fairly small incremental improvement over the existing hardware debugging support in x86 chips. It provides some extra control codes allowing the address in DR0 (one of the four registers i386 provides for hardware watchpoints) to do some slightly more clever things. For example, a watchpoint can be triggered on a partial match, rather than an exact match, to the address - this is really nice because it lets you put a watchpoint on the whole of any data structure that fits within a page. With the i386 watchpoints, you can only watch a single word with each register (4 words in total), while this means you can watch anything smaller than a page (and you can watch things bigger than a page by marking the page as no access, trapping the access, then unprotecting, single-stepping through the load / store, and continuing the process, which is how you implement watchpoints when you run out of debug registers).

Re:Security?

[makomk]

Perhaps the slashdotted site answers this but I have to wonder why not just have a separate opcode to turn the debugging on?

Because there's already a whole bunch of privileged MSRs that normal user code mustn't have access to - many of which are undocumented and processor-specific - so adding a few more is no big deal. Adding new opcodes, on the other hand, requires more work and risks them clashing with Intel's opcode choices at a later date.

Re:Security?

[Joe U]

Also, if it's triggered in userspace, the OS can block it.

Not if your OS is rooted. Or root-kitted. Or both.

That's already game over. If you own the OS, why would flipping the processor into a new mode help?
  You already own the security system.

Re:Security?

[tibit]

AFAIK they are packaged with every major linux distro out there, and I can't but presume that Windows ships with microcode patches as well.

Re:Security?

[TheLink]

Say you own the OS but it is in a virtual machine.

If flipping the processor into a new mode allows you to get out of the virtual machine and pwn the Host too, then yes it makes a difference.

Re:Security?

[anotheryak]

Exactly, it's probably a bit of a kludge, and making it into a stable, documented, supported feature is going to be expensive with a lot of support and a small user base.

I have modes like this in some of my own products, and sometimes I'm leery of even having some other people on my own team have access to the debug modes, because of the potential for disaster and a WHOLE lot of handholding from me.

It's not worth the time it would take for me to set it up for broader use, and if I did, they would break things and then come running to me.

Re:Security?

[wbo]

AFAIK they are packaged with every major linux distro out there, and I can't but presume that Windows ships with microcode patches as well.

Microcode updates for Windows machines are distributed through Microsoft Update and are downloaded and installed automatically if automatic updates is enabled (and it is enabled by default). No BIOS update required.

An example of such an update can be found by looking at Microsoft KB936357

Re:Security?

[Anonymous Coward]

I used to work for a processor emulator tools company called Applied Microsystems Corp, Redmond WA. now defunct.
Up thru processors type 68040 emulations tools could be mounted external to the processor chip and performed the functions mentioned ( hardware breakpoints, memory maps, all register shadowing, soft and hard breakpoints, etc, all the things that you need to perform basic computer system development. As the complexity of the systems increased beyong those early 8/16/32 bit cpu, all those hardware functions became embedded into the processor itself and are accesed by usually an I2c serial link. Emulation tools went from hardware and software complex and external to the system to hardware simple internal and software complex and external. The bottom dropped out of the eumlator tools market ( from 50k$ to 5K$ ) and that whole industry went away and got smaller and fewer in number. But the hardware tools remain inside the cpu's now, accessible by hardware licensed developers in NDA contracts. But for a few years even a few technicians had an understanding of the fine details of cpu internals. Now all that is buried again.

Re:Security?

[LordNimon]

It is possible that the debug features are for their internal use and they don't quite work as intended.

Ding ding ding ding ... we have a winner!

I work for a processor design company. If this feature is kept secret, it's because the company does not want to put in the resources to make sure it works completely on every chip. It probably uses lots of hacks and violates the architecture in some obscure way. AMD does not want customers depending on this feature and then insisting that it works for future design wins.

Re:Security?

[Anonymous Coward]

If an OS running on real hardware can block this call coming from user-mode then a hypervisor can block it coming from a VM. And if it can't be blocked you're p0wned either way. A virtual machine makes no difference.

Re:Security?

[dch24]

Here's the site's text, since it's still down.

Listed in categories X86 Internals Articles
Most recent version 0.1
Most recent release date November 11, 2010
Description Super-secret debug capabilities of Atlon XP and better AMD processors

Here unveiled by Czernobyl aka Czerno - Mail This

PAGE IN CONSTRUCTION
You can help Yes, YOU - Please DO correct grammar and other English language mistakes - Please add formatting, bold, italics... as needed for better accessibility - Please contact me over any errors inaccuracies complement to the technical contents - If you own a 64-bit AMD processor, please DO check and report any differences. Contents are intended to be released at a later time under liberal copyright copyleft options. Please do NOT make the contents available elsewhere BEFORE it is READY to be unleashed I retain the 'moral' ownership rights.

TODO mucho...

Summary AMD processors Athlon XP and better have included firmware-based debugging features that expand greatly over standard, architecturally defined capabilities of X86. For some reason though, AMD has been tightly secretive about these features hint of their existence was gained by glancing at CBID's page URL below . Herein will be uncovered what was found through direct experimentation, in the hope it may be useful to software developers, possibly included in future debuggers or debugger plug-ins. I'll use the term expanded for the capabilities covered here, since the term debug extensions was already taken. The item is WORK IN PROGRESS, but USABLE hence released AS IS Author can be contacted email, or PM, or public thread on the reversing board Note All addresses values are hexadecimal unless noted otherwise.

The following four new machine specific registers MSR are involved in the expanded debug facilities. All those MSRs are password protected against casual access read write access RDMSR WRMSR to the registers is granted only if EDI holds the correct password value, viz. EDI 9C5A203A. Otherwise, a GPF exception occurs.

All these registers default reset values are zero.

• Control C001_1024 width 8 bits Data_Match
• C001_1025 width 32 AMD64 64 Data_Mask
• C001_1026 - ditto - Address_Mask
• C001_1027 width 12 bits

Let's start by looking at the Control MSR DbgCtrlMSR2, C001_1024 since it is the key to the new features. The low 8 bits can be set reset - only bit 0 and bit 1 will be examined here. The other bits had no effect in my preliminary, limited experimentation and will be left at zero. Readers are invited to further investigate report A - When Control 0 default , the AMD processor's debug firmware operates as defined by X86 architectural specs. In order to switch on the expanded debug capabilities, we must set Control MSR's bit 1. IOW, set Control 2 or 3, more on this later .

B - When Control 2 binary 10 , the operation of any breakpoint defined in DR0 is modified as will be described. DR1 to DR3 breakpoints aren't affected at all . DR0-based breakpoint is now controlled by the new registers Addres_Mask, Data_Match and Data_Mask, in addition to DR7. General notions about the Mask MSRs both in Data_Mask and Address_Mask, a 1 bit means don't care .

Formally, when a comparison of two values addresses needs to occur under mask, the Mask is ANDed to both and then the compare is done. A match occurs if the masked values are equal. A mask value of zero thus is equivalent to no mask. A mask of all ones makes match occur on just one value considering compare length . Note for Address_Masks, those only make sense that are binary zeroes followed by zero or more binary ones. Not sure if that is enforced by the AMD firmware Now let's examine expanded DR0-breakpoint operation Instruction breakpoints DR7 type 0 break occurs at any address which matches the BP address in DR0 under Address_Mask. Recall Address_Mask is significant to 12 bits only. Thus the widest possibl

Re:Security?

[ajlitt]

Except the UPS company makes that feature available for money. If AMD isn't releasing this info to anyone, then it isn't golden screwdrivering.

Re:Security?

[mysidia]

Not that I'm that knowledgeable about virtualization software, but I can't imagine that they would run priviledged code in the virtual machine as priviledged code on the host CPU, so it doesn't matter anyways.

In a VM running under hardware-based virtualization (AMD-V / VT-x) , privileged code in the guest generally does run at privilege level 0, also referred to as (privileged), kernel mode, or ultimate privilege. This is required to implement a protected mode operating system; a modern guest OS needs to be able to implement its kernel mode, user mode, and have the hardware implement restrictions on its user processes, typically by reducing its user processes to RING 3, or RING1 / 2, which are permission levels the OS can configure to specify the allowed privileges. However, code in the VM is running as a guest OS.

It means the guest OS must run with ultimate privilege. To address this, CPU manufacturers created a new privilege level called "ring -1". When the virtualization feature is active in hardware, the virtual machine monitor runs at ring -1, and runs guest OS code at ring 0.

As Intel puts it: VT-x and VT-i allow guest software to run at its intended privilege level. Guest software is constrained, not by privilege level, but because for VT-x it runs in VMX non-root operation

In a software-emulated VM, instructions are translated and rewritten by the hypervisors anyways, and the hypervisor itself generally runs in Ring 3 user mode not kernel mode in that case, It is up to the hypervisor to protect itself.

The ultimate security disaster?

[pyalot]

Since TFA is down by now, and I can't get the exact details... does this mean that any program running and setting the right bits in the right registers can get "processor root" access to everything the processor does, irrespective of any security constraint the OS may place on that process? Oh dear

Re:The ultimate security disaster?

[pyalot]

It's not that I'm surprised. But you need to recall that AMD chips a goodly chunk of data/hosting-center cores, which run many clients on the same machine... AMD will need a very good indemnification clause to wind their way out of that dammage responsibility.

Re:The ultimate security disaster?

[Eil]

Not a security hole. This debug mode is not some kind of eleet hax0r backdoor. It's for debugging the processor and microcode.

It's the OS responsibility to ensure that normal applications can't simply do whatever they like directly to the hardware, including the CPU.

Re:The ultimate security disaster?

[Smallpond]

Since TFA is down by now, and I can't get the exact details... does this mean that any program running and setting the right bits in the right registers can get "processor root" access to everything the processor does, irrespective of any security constraint the OS may place on that process?

Oh dear

Any program that can read and write to any processor register already has complete access to everything on your computer. The reason this is secret is not to protect your data, its to protect AMD's secrets.

Re:The ultimate security disaster?

[maxwell demon]

Ultimately it's the CPU which implements the security at the instruction and memory level. The OS only tells the CPU what code should run under which security context.

hidden data-aware conditional breakpoints

[digitaldc]

Hidden for a reason? Do they not want Nvidia to see this?

So it's a software JTAG

[leptechie]

I'm actually surprised to find out that everyone's surprised. I've been hacking routers and now work for a telco surrounded by disassembled set-top boxes, and both have serial and JTAG interfaces abundant. Many require soldering, so in that respect it's "hidden" from customers. Maybe: - It's often more expensive to engineer these things out of the test systems to ready for production - and just maybe it's still actually useful especially as you peer deeper into the GHz to get more performance from an existing design.

Re:Hidden?

[neokushan]

I can think of many reasons why it might be hidden. For example, it may be hidden because the cost of supporting it would outweigh the benefits of admitting the "feature" is there. I don't just mean in terms of documenting it and releasing that info for developers, I mean in termins of testing it for security reasons. Plus, let us say that a theoretical bug is found that creates a hole someone can exploit - is it patchable? It's a whole can of worms AMD may be right to avoid opening.

Re:Coral Cache link

[Trevelyan]

Won't work at all, because you lost the '!' on the end..

Try this

Re:Hidden?

[icebraining]

And how do you know some top black hats don't already know about this for years and have already exploits for it? It's a classic example of security through obscurity.

If it's not safe (and if it's baldly tested, it is), I'd expect AMD to disable it on a physical level, not leave it there "hidden" for someone with poor intentions to find out.

google cache of the article

[avgapon]

http://webcache.googleusercontent.com/search?q=cache:EzsEFcoAZDAJ:www.woodmann.com/forum/archive/index.php/t-13891.html+Czernobyl+AMD&cd=1&hl=en&ct=clnk

Re:google cache of the article

[eulernet]

And the documentation about the new opcode is probably this one:

http://cbid.softnology.biz/html/undocmsrs.html

Just an extension of existing debug facilities

[Menacer]

Based solely on the Google cache of the forum post describing this (linked above), there's no need to go into hysterics. For hardware and systems geeks, this is very cool. It's an extension of the existing x86 debug registers (DR0-7) that allows you to set a debug watchpoint that only fires when specific data is loaded in.

There are a lot of researchers and tool builders that would love to have this because it would allow them to take a watchpoint fault whenever they only when they have a specific value from a specific location. For instance, let's say that every so often you get a null pointer exception at a specific address. However, if you current go into gdb and set 'watch 0x{address}', you're going to take a breakpoint every single time that pointer is accessed.. Wouldn't it be great to do something like 'watch 0x{address} NULL' and only stop your debugger whenever 0 gets written into that address?

That's what the forum posts imply, at least. "Guys, I've reversed this in part... breakpoints defined in DR0 can be made to fire only on data match (under optional mask), plus masking of any or all of 12 low address bits ! Works also for I/O break points, provided CR4_DE is set, of course !"

I would wager that this is not a large security concern. Access to DR7 is restricted to ring 0, and therefore enabling debug breakpoints must be done by the operating system. While extremely interesting (I wish I could read more!), Czernobyl appears to be describing a modification to debug breakpoints that are already enabled.

Re:Just an extension of existing debug facilities

[Menacer]

Oh, and the summary's description, "hardware data-aware conditional breakpoints, and direct hardware 'page guard'-style breakpoints", matches up with the line I copied & pasted from the forum post. I previously described the "hardware data-aware conditional breakpoints"where you can make hardware take a fault if an address of a memory operation is matched && the value of the memory operation matches. Looking through my notes, embedded Power ISA (Book III-E) processors also let you set value-dependent watchpoints using the Data Address Compare (DAC) Registers. I'm not sure about other ISAs.

The second party of the summary's statement refers to to 'page guard'-style breakpoints. This is referenced by Czernobyl's "masking of any or all of 12 low address bits". Again, this is a very interesting extension of the x86 debug registesr, which only allow debug watchpoints of size 1, 2, 4, or 8 bytes (and the latter only in certain microarchitectures & modes) However, by masking out the low 1--12 bits of the address into don't-cares, it's possible to set watchpoints anywhere from 1-4096 bytes, limited to powers-of-two and size-alignment. This is cool from an x86 standpoint, but ARM, MIPS, and Itanium (off the top of my head) already do this.

Suffice it to say, the stuff that Czernobyl found is very cool in relation to x86, especially if these facilities were officially released to the public at any point in the future. However, it's very unlikely to cause any kind of AMD-only viruses or other scary security concerns. These features exist on other ISAs without any kind of world-shattering problems. :)

Re:Just an extension of existing debug facilities

[Menacer]

Sure, but it's much faster to do it in hardware. This is the whole reason data watchpoints exist (See, for instance, the paper "Some Requirements for Architectural Support of Software Debugging" by Mark Scoctt Johnson from ASPLOS-I), as you could technically have your debugger put address & data checks around every memory access, but that leads to completely unacceptable overheads. It's faster to let the hardware check the addresses in parallel with regular execution and take a fault only if you touch the watchpoint.

Similarly, if the hardware will check the value before taking a debug interrupt to the kernel and subsequently signaling/scheduling of the debugger, it will be much, much faster than performing all that and then have the debugger check the address & throw this particular interrupt away before continuing execution. That constant interrupt cycle can cause 10,000x or more slowdowns if you're constantly accessing a value & taking bad watchpoints on it.

Re:Just an extension of existing debug facilities

[deKernel]

If you are an application developers, I would agree with you. Any decent debugger should allow you to set a conditional breakpoint, but I am not sure if you can say that for kernel debuggers which are very different animals typically.

Re:Hidden?

[arivanov]

That is not always the case.

For example they did not properly document and release the docs on the hardware RNG in their first chipsets when it came out. As a result it ended up supported only on Linux on a "friend-of-mine" basis and MSFT (on whatever basis). The other OS developers did not know about it for a while (more than half a year). I remember personally telling Theo De Raadt on BUGTRAQ at the time to stop talking rubbish that AMD does not have a hardware RNG and he was genuinely shocked. However the fact is a fact - it was not open.

This is just an example, i can think of quite a few others.