Attack Steals Crypto Key From Co-Located Virtual Machines

Gunkerty Jeb writes "Side-channel attacks against cryptography keys have, until now, been limited to physical machines. Researchers have long made accurate determinations about crypto keys by studying anything from variations in power consumption to measuring how long it takes for a computation to complete. A team of researchers from the University of North Carolina, University of Wisconsin, and RSA Security has ramped up the stakes, having proved in controlled conditions (PDF) that it's possible to steal a crypto key from a virtual machine. The implications for sensitive transactions carried out on public cloud infrastructures could be severe should an attacker land his malicious virtual machine on the same physical host as the victim. Research has already been conducted on how to map a cloud infrastructure and identify where a target virtual machine is likely to be."

Comments

Not just 'a' crypto key

[ChristW]

The published paper is an interesting read. Obtaining the crypto key to libgcrypt is only one application. In general, the authors say, it is possible to construct a side-channel attack on other, unrelated, processes in the attacked VM.

Re:Not just 'a' crypto key

[cbhacking]

"See" from one VM into another, on the same physical hardware. It's a cache-based attack; the attacker fills the cache lines of the CPU, then yields and hopes the hypervisor schedules the other VM on the same CPU core next, and that the other VM is executing a particular set of cryptographic operations. These operations have a very specific behavior with regard to the processor cache. When the attacker's process runs again, it determines which of its cache lines were flushed, and this information tells it something about the operations the other VM is doing.

Obviously, this is a very slow and error-prone operation, not something you can do in real-time, or even within a few seconds (which, on modern CPUs, means a few billion cycles). Rather, the attack requires hours - tens of trillions of CPU clock cycles - to get enough samples of the CPU behavior to determine which samples are meaningless, and to have enough information to piece together the cryptographic key. Obviously, this assumes that the other VM on the same host is frequently engaging in cryptographic operations using this key.

I don't think I'd go so far as to call this attack truly practical, yet, but it's possible in at least some situations. That fact, by itself, is quite scary for two reasons. First, stealing a key is the kind of thing that you don't need to do Right Now in most cases; if you can do it within a reasonable time frame (which may be months), that's enough. Second, these types of attack are almost always slow and/or difficult when first demonstrated. The problem is, now that the attack is known to be possible, it has garnered a lot of attention and many people will be examining it to see if they can improve on it somehow.

Don't trust hardware you don't own.

[Anonymous Coward]

"public cloud infrastructure". The very thought of that makes me cringe, then laugh at the absurdity of it.

We can't even code bug free operating systems. What makes anyone think we can code a bug free hypervisor? I'm still confused as to why people believe that VMs are inherently secure- are they secure because VMware/Xen/Oracle says they are? Or are they secure because they've been tried and tested in the fires of time? All I ever see about hypervisors these days is some inflated marketing terms or new "cloud" interoperability features or some other random junk that solves an imaginary problem someone first had to go out of their way to create. I've never seen anyone actually come out and say "This version of our hypervisor is even more secure then the last because of XYZ!".

The company I work for makes extensive use of "cloud influenced" features in-house. It's awesome to be able to two-click a LAMP stack into existence through a nice web portal or do the same for a couple of Win2K8 instances. Some idiot was preaching about outsourcing our hardware to someone else and putting everything "in the cloud". Luckily management saw it for the farce it was and put that guy in his place pretty quickly.

So again, I'm really curious as to why people explicitly trust: A) Their services/platforms to someone other then themselves, and B) expect that VM hypervisors are bullet proof.

Re:Don't trust hardware you don't own.

[DarkOx]

Well a hypervisior is in theory a simpler beast than a operating system. Depending on your prespective it has less attack surface. I think thoes are good reasons to think we could get it right. The real source of trouble is the x86 world just does not provide the hardware isolation features needed.

Re:Don't trust hardware you don't own.

[photon317]

I don't think reasonable people expect hypervisors to be bulletproof. Security is a sliding scale though, and for many purposes the security level offered by a responsible cloud provider is good enough for what they're hosting there. If my bank hosted their critical system in AWS, I'd freak out. If Pandora hosts systems there to stream music to me? I could care less. If Pandora puts their billing system there that has my credit card number? Ok, I start to care a little more, but the risk is manageable if they're being careful about the design, and ultimately if someone rips their whole CC database, my CC company or I will notice the fraud activity quickly and issue me a new card. Life goes on.

Why do companies want to use virtualized infrastructure in the first place? Because it offloads work that's not directly relevant to their business. Let me quote directly from Bruce Perens' recent Ask Slashdot responses:

There is no point in having your own programmers write anything that is not a customer-visible business differentiator for your company if you can get it from the Open Source community. A “business differentiator” in this case means something that makes your company look better than a competitor, to the customer directly. Too much “glue code”, and “infrastructure” is written by organizations that have no real need to do so if they would adopt Open Source. The message that is driving them to do so is the huge stack of cash being made by the companies that do use us.

He was talking about it making sense for companies to build on top of OSS lower-layers. The same applies to the cloud infrastructure stuff. For most businesses, infrastructure is not a differentiator anymore. Why have company employees concerned with managing network switches, racks, cooling systems, datacenter fire protection codes and systems, insurance, servers? Or calling vendors and leading them in the building to replace failed drives and RAM modules, or even giving a crap about hardware at all?

If my company's purpose in life is to deliver, e.g., some social iPhone app and a backend network service that supports it, I have no differentiating interest in that level of infrastructure. I still need an IT department, but it can be a small one focused on using that cloud infrastructure correctly (e.g. security, configuration management, etc). When you can shift off that whole layer of complexity to a large-scale specialist, you've reduced the total complexity your company has to manage directly. Focus on the areas that matter, not the common ground. Did your company design, engineer, and build its own kitchen appliances for the company breakroom? Didn't think so...

Re:Don't trust hardware you don't own.

[mrbluze]

Better still, don't trust hardware any more than you trust software. A virtual environment has the potential to bring the worst of both. I am against flying in clouds, lest they be connected to cumulo granite. I am all in favor of research that reveals how centralization of information is bad for the vast majority, and this goes to demonstrate the problem quite well.

Re:Don't trust hardware you don't own.

[Anonymous Coward]

I'm still confused as to why people believe that VMs are inherently secure

You're missing the point. No system is secure. What's nice about VMs is ... as many as you need, same price. So just chuck them often, don't even worry about checking them to see if they've been compromised. How awesome would it be if we could just destroy and replace physical desktops and servers 100 times a day? That would get expensive... but not with VMs, which allows you to move the security layer back to the image, and screw securing the actual running system.

Re:Don't trust hardware you don't own.

[bob')DROP TABLE user]

I'm still confused as to why people believe that VMs are inherently secure

You're missing the point. No system is secure. What's nice about VMs is ... as many as you need, same price. So just chuck them often, don't even worry about checking them to see if they've been compromised. How awesome would it be if we could just destroy and replace physical desktops and servers 100 times a day? That would get expensive... but not with VMs, which allows you to move the security layer back to the image, and screw securing the actual running system.

Yes... That way they can re-build the vulnerable system. I hear it takes a long time to steal credit card information...

Re:Don't trust hardware you don't own.

[Anonymous Coward]

I think the details of what's actually been done are relevant here.

This is a side-channel attack against a specific piece of code which has a very clear operational signature.

It's a brute-force implementation of exponentiation mod p using repeated squaring, such that bits from the key directly result in jump operations, and one side of the jump is very cheap and one side is very expensive. Modern implementations of the same exponentiation process (e.g. in OpenSSL) have optimizations which prevent this from being the case.

It is amazing that it's been done at all, but the number of assumptions it rests on regarding precisely what the other VM is running do seem to make it an attack of little practical value. This is more a piece of interesting math than it is an indictment of cross-VM security. And the appropriate response is probably some more neat math to make an algorithm for the same problem which is provably not attackable via this methodology.

Re:Don't trust hardware you don't own.

[Anonymous Coward]

I'm still confused as to why people believe that VMs are inherently secure

You're missing the point. No system is secure. What's nice about VMs is ... as many as you need, same price. So just chuck them often, don't even worry about checking them to see if they've been compromised. How awesome would it be if we could just destroy and replace physical desktops and servers 100 times a day? That would get expensive... but not with VMs, which allows you to move the security layer back to the image, and screw securing the actual running system.

Yes... That way they can re-build the vulnerable system. I hear it takes a long time to steal credit card information...

You're not seeing it. The system isn't vulnerable until the attack gains control of image construction. The time between VM destruction and replacement can be quite small... every minute if you wish, the hardware and software can handle this with no interruption of service. Let's see an attacker infiltrate a VM, and then successfully perform a side attack on another VM, and get what they're after, in under a minute. Likely, however, security checking is what occurs every minute or less (hash comparisons or whathave you), and the VM purging happens using a slightly longer period, say every 5 or 10 or 15 minutes. In this scenario an attacker must accomplish their goals nearly instantaneously to be successful, which means they have no time to "hack" away at a system before it no longer exists. This is not a bad idea considering the alternative, which is to thicken security and put more locks on the doors. It doesn't matter if you have a vault for a front door, it's only a matter of time before a determined theif gets in your house, and you can stand there with a shotgun, but you can't shoot all the thieves. But if you blow up your house, there is nothing there to steal, nothing to break into...

Re:Don't trust hardware you don't own.

[indeterminator]

I'm really curious as to why people explicitly trust: A) Their services/platforms to someone other then themselves

The hosting providers have a financial interest in being trustworthy. If they lose the trust, they lose their business. Doing it yourself has its own failure modes too.

Also, for many new companies running their own datacenter would be cost-prohibitive, so trusting may be the only choice they have.

Re:Don't trust hardware you don't own.

[mpeskett]

When you can shift off that whole layer of complexity to a large-scale specialist, you've reduced the total complexity your company has to manage directly. Focus on the areas that matter, not the common ground. Did your company design, engineer, and build its own kitchen appliances for the company breakroom? Didn't think so...

Surely in handing over responsibility for managing that complexity, you also hand over control of what could be intensely critical components of your business. They may do a perfectly good job at a lower cost, but in the (hopefully infrequent) event that the shit hits the fan, the job of fixing it is out of your hands and out of your control and that ought to be scary.

I don't know. Maybe it makes enough sense in the bulk of cases to be a good plan, but the risk of having your entire infrastructure yanked out from under you because of a black swan event or just a regular-grade fuck-up at an unrelated company sounds like something best avoided.

Re:Hypervisor leaks cached data

[gnasher719]

It appears that the hypervisor leaks data from one VM to another by not clearing a cache.

What is leaked is not actually the data in the cache; another virtual machine running on the same computer cannot access that data. What is leaked is some information about cache usage, which may then allow an attacker to find out what the other VM has been doing. The attacker fills the cache with data, switches to another VM, and when it gets control again, the attacker measures how long it takes to access the data that it put into the cache itself. If it's fast, then the attacker knows that the other VM hasn't touched that part of the cache. If it's slow, the attacker knows that the other VM touched this part of the cache.

Detailed blog post

[Anonymous Coward]

You can find a more detailed blog post about this here:

http://blog.cryptographyengineering.com/2012/10/attack-of-week-cross-vm-timing-attacks.html

Re:Hypervisor leaks cached data

[Anonymous Coward]

It appears that the hypervisor leaks data from one VM to another by not clearing a cache. If that is all, this leak can be fixed by explicitly clearing the cache when switching to another VM. This will probably cost a few CPU cycles (and cause a few extra cache misses when a VM is resumed).

The problem isn't data leaking but the change in latency to access memory when on the same cpu where a crypto algorithm is running. The keys can be reverse engineered if the crypto algorithm uses a well known table. There is no direct data leakage across VMs required. This is not a joke it is effective, but you have to get you VM onto the same server as the VM you are attacking. You can avoid the issue by using a dedicated server in the Amazon cloud case, or an Extra Large VM in Azure.

Summary of the attack

[dachshund]

This post gives a high-level summary of the attack:

http://blog.cryptographyengineering.com/2012/10/attack-of-week-cross-vm-timing-attacks.html

(I previously posted this as AC, but it vanished.)

Re:Who would have thought......

[Anonymous Coward]

No, it isn't since modern operating systems tend to isolate programs from each other, and in the case of this article the programs are even running in disparate virtual machines, which should put a wall between the two. It is only through exploiting the processor cache that the key could be extracted. The attacker monitors how the victim fills the instruction cache. Since the victim's crypto algorithm follows different code paths depending on the key, the researchers were able to determine key.
This kind of side-channel attack was not universally thought practical so this is news and would be good to think about how to mitigate this problem.

Not Likely Reproducible in Production Environment

[Traiano]

Before anyone gets carried away, here are a few important quotes from TFA:

• "We assume the attacker knows the software running on the victim VM and has access to a copy of it"
• "We demonstrate how to use interprocess interrupts (IPIs) to abuse the Xen credit scheduler in order to arrange for frequent interruptions of the victim’s execution by a spy process running from within the attacker’s VM...[then much later]...we leverage the tendency of the Xen credit scheduler to give the highest run priority to a VCPU that receives an interrupt."
• "We will only be able to spy on the victim when assigned to the same PCPU, which may coincide with only some fraction of the victim’s execution."

In other words, this exploit requires: knowing what cryptographic software is being run, the presence of Xen and an apparent security hole therein, and lucky core colocation of the VMs in an environment that could easily have dozens of VMs running against more than a dozen cores "over the course of a few hours".

In short, all of this is unlikely to be reproducible outside of a lab.

Re:Hypervisor leaks cached data

[foniksonik]

Problem is that a leak of any PII data for customers of a business is a PR nightmare and potentially a largish lawsuit costing millions. Millions may be more than was saved by virtualizing.

Re:Hypervisor leaks cached data

[CastrTroy]

Millions aren't often saved by virtualizing your hardware. Almost all numbers I've seen show that the cost of running on virtual hardware is actually more costly than running on your own servers after you amortize the price of the servers over their lifespan. Often buying your own hardware pays for itself within a year. Hosting in the cloud makes sense in a small number of instances where you have wildly varying amounts of traffic and need to be able to scale up and scale down very quickly to big load changes. It also allows you to get some nice servers on day one without much capital investment. But that's not being very business smart. If the servers can pay for themselves in the first year, you should really be buying the servers. It can also be very cheap if you are utilizing almost no resources, but that is something I would consider more of a home project, and not something that is really something that business would be looking at.

change the algorithm

[Chirs]

If necessary you could simply do unnecessary work on one of the code paths so that they end up doing the same amount of work on each path.