Microsoft's Larry Osterman On Threat Modeling

Schneier has pointed out an excellent series of blog posts about threat modeling by Microsoft's Larry Osterman. The series focuses on the PlaySound API as an example. "As you go about filling in the threat model threat list, it's important to consider the consequences of entering threats and mitigations. While it can be easy to find threats, it is important to realize that all threats have real-world consequences for the development team. At the end of the day, this process is about ensuring that our customer's machines aren't compromised. When we're deciding which threats need mitigation, we concentrate our efforts on those where the attacker can cause real damage."

Comments

Standard Microsoft Threat Modeling Dialog

[eldavojohn]

Consumer: My company doesn't need Vista, we're using Linux which has about the same amount of bumps and hiccups.
Microsoft: You mean you're using an operating system that validates over 450 of our patents?
Consumer: Well, I know that isn't true but ...
Microsoft: But it'd be a shame if your company was ever engaged with our world class legal team instead of being a 'partner' with the largest software maker ever?
Consumer: But we only have 20 employees.
Microsoft: We know--perhaps you'd be interested in purchasing a copy of our lap dog here, Novell's SUSE?
Consumer: But we already use Red Hat ...
Microsoft: We heavily suggest you re-evaluate SUSE and when you do your trade study please do note that it's the only Microsoft Certified Genuine Linux. Also, it would be a shame if we had to exercise our patent portfolio on Red Hat and subsequently ... well, no reason to get into details. Have a nice day!

Re:Standard Microsoft Threat Modeling Dialog

[Anonymous Coward]

Given that the article talks about the audio api, it probably went more like this:

Threat: User may play a song without paying for it.
Mitigation: Render the internet useless while playing music.

Threat: User may complain about the network being crippled while playing music.
Mitigation: Blame hardware, blame drivers, then make up some excuse that playing audio requires super-low latency priority for the audio playing app and the network is sacrificed to ensure smooth playback.

Threat: User may notice that the network is not crippled when playing video with audio and complain that the argument about priority is bogus, etc.
Mitigation: Ignore any users that claim this while rolling out a patch to cripple the network when playing video, then return to blaming hardware and/or drivers.

That's got to be a hell of a job

[Skyshadow]

Try to imagine this guy's work day: He gets to wake up in the morning, hug his kids and then go into work and spend all day trying to figure out the right combination of security defaults that will (a) let people go out and do stuff while (b) protecting them from their own "I'm a average Windows user" level of abject stupidity.

Put another way, imagine that instead of just setting up a computer for your parents, you had to set one up for *everybody's* parents. All at once.

As much as it's fun to give MS shit for their products, I think I'd last about two hours in that position before I went into the executive washroom and slashed my wrists.

Re:That's got to be a hell of a job

[ScytheBlade1]

I've got his RSS feed in my RSS reader (http://blogs.msdn.com/larryosterman/rss.xml). I enjoy reading about the details of what goes on inside of MS, and I really do enjoy getting the story straight from the horse's mouth. For example, the whole "playing a video kills my network performance" thing. Slashdot is, well, Slashdot. It'll spin it how it wants to.

Larry started doing this threat modeling bit a while back, as the first article is dated some time ago. He's taken his time, and demonstrated what to do and how to do it in great detail. It's perfectly clear that he actually knows how to program things, correctly. And securely. This series of posts on threat modeling was wonderfully insightful into how things should be done.

Just because it's MS doesn't mean that it needs to be senselessly bashed. This would be one of the reasons as to why it shouldn't be. This guy knows what he's doing, and he does it well. Gasp, he works for MS.

Re:That's got to be a hell of a job

[beckerist]

Meh, I'm a newb to advanced security. For the rest of you (like me:) http://en.wikipedia.org/wiki/Threat_model

Basically, in a nutshell, a "threat model" is a designers / programmers way of saying "all potential flaws in our application." An example: a 1 meter hole in the side of the Death Star.
Just thought I'd pass that along as I learned it...eh...3 minutes ago.

I prefer Attack Trees.

[khasim]

http://en.wikipedia.org/wiki/Attack_tree
By Bruce Schneier.

Face it, no matter how secure your little bit of code is, if the SYSTEM is vulnerable, your little bit of code is vulnerable.

Which is where Larry goes wrong in TFA.

You can put all the locks you want on your front door. But if you don't fix the huge hole in the wall next to it, you aren't improving your security at all. No matter what you claim.

Re:I prefer Attack Trees.

[swillden]

A threat model is about admitting we have a bad product, saying that fixing it properly is too hard /expensive so we will try and work out what the largest holes are and fix those first.

Nonsense.

Threat modeling is a crucial exercise for any system that wants to be secure. Note that "system" is more than just "software", so just testing your software against all possible inputs is insufficient, even if it were actually possible.

For example, let's suppose the system under consideration is the Windows access control system, responsible for ensuring that only authorized users can read/write files. What are the attack vectors? How many of them can be addressed with input validation testing? How can your approach to security analysis/testing address an attacker who reads/writes data from the disk directly, bypassing your software (assuming you're writing Windows) entirely? Is this a threat that you need to mitigate? If so, what can you do to address it? Encrypt the data? Fine, where do you put the keys? Or, if the user enters the decryption key, how do you ensure that an attacker can't get that key? Can the attacker spoof the key entry screen? Why or why not? Can the attacker get it by shoulder surfing? Is that a threat you need to mitigate? What other ways might the attacker get the key?

Obviously, filesystem access control is a complex system and there are multiple software, hardware and social engineering attacks against it. Some can be mitigated by properly-implemented software, some can be mitigated (or, more accurately, shifted) by appropriate application of cryptography, some might require hardware modification and some might simply be out of scope. Threat modeling allows you to clearly define and document all of the analysis you have done, and all of the decisions you have made. It provides the basis for design and implementation -- which must both then be done properly or the other analysis was all irrelevant. The sort of testing you described is just one piece of proper implementation.

Security is much harder than you seem to think it is, and threat models are a key component of doing it right. I find it rather nice that Microsoft seems to be learning this lesson, even if it is something they should have done 10+ years ago.

Re:Double-plus ungood

[Mongoose Disciple]

Funny that "threat mitigation" doesn't exist in the aerospace industry...

What would you call passing through airport security to fly on a passenger aircraft?

Airplanes typically don't stand up to serious attacks. I'm not sure where you're trying to go with this analogy.

This is a poor security model

[Cassini2]

For high reliability code, you write code on the assumption that other code may have problems. You write code defensively. For any kind of complex system, people will make mistakes. Thus you have to continually verify program integrity and security in a multiply redundant manner. You don't wait until a trust barrier is crossed.

For example, if I have an application controlling a power plant, even if the computer is already running "foreign" code at my privilege level, the control application may still be up. It isn't until the foreign code impacts on the control application that a major security problem exists. Unfortunately, Microsoft makes it easy for the foreign code to impact the control application. The code could consume all available CPU resources, it can consume almost all available disk bandwidth, it can run Windows out of Virtual Memory or Handles. None of these attacks are easily blocked by existing Windows security. Of course, when the ability to inject code into the control system is considered, rogue program security gets worse.

Microsoft's threat model does not even consider the effect of "friendly" code, to impact on critical code. If you can't detect adverse interactions with friendly code, then I really question the ability to survive malicious code impacts.

At least two levels of redundancy are required. Malicious code almost always starts as an unexpected consequence of a friendly application, and the first level of redundancy is that the authors write the friendly application in a secure manner. In practice, relying solely on this level of redundancy is fraught with peril. As such, additional levels of redundancy are required. A key second level of redundancy is that even if a friendly application runs amok, the control system should keep running. This implies isolation between applications running at the same privilege level.

The intent of multi-tasking operating systems is to prevent applications at the same privilege level from affecting each other both directly and indirectly. Applications should not be able to affect each other either directly (application to application) or indirectly (via slowing the computer to a crawl.) By only looking at trust escalation issues, Microsoft has a key areas of redundancy and security.

Re:Look at how YOU would do it.

[toadlife]

If it's so idiotic to "integrate the browser into the OS", then why does Apple do it with OSX and why does the KDE team do it with their desktop environment?

Re:Look at how YOU would do it.

[toadlife]

When you open "My Computer" or "Windows Explorer" or even "Control Panel" on a Windows box you're opening IE. No, you are not. Explorer is explorer, not internet explorer. It uses some of the same dlls that Internet explorer uses, but that kind of library sharing is standard practice in any large desktop enviroment, weather it be OSX or KDE or Gnome.

Microsoft *could* reinvent the wheel 20 times in order to make sire every single app has their own libraries tpo use, but that would be stupid.

Re:Look at how YOU would do it.

[toadlife]

Yeah, you could "remove" Safari but the libraries that provide all of Safari's functionality would remain. You could also remove IE from Windows, but most of it's functionality would remain as IE most just calls external dlls - dlls that other parts of the system share.

If you really wanted to remove Safari from OSX, you would have to remove the entire webkit framework that it and many other OSX applications rely on, and I really don't think you would want to do that.

Law #1 is a lie.

[zestyping]

I see they're still using the old, tired, "Immutable Law #1" that Scott Culp made up many years ago.

Law #1: If a bad guy can persuade you to run his program on your computer, it's not your computer anymore
[...]
When you choose to run a program, you are making a decision to turn over control of your computer to it. Once a program is running, it can do anything, up to the limits of what you yourself can do on the computer.

It's simply wrong, and it's deceptively named.

One of the important jobs of any operating system is to isolate and protect applications from one another. To assume the so-called "Immutable Law #1" is to pretend that this responsibility doesn't exist.

I'm sure they've heard of encapsulation, right? Defense in depth? How about the principle of least privilege? The thinking behind Law #1 -- that when a user runs a program, that program automatically gets full rights to pull all the user's privileges out of thin air and exercise them in whatever way it wants -- runs counter to all of these fundamental security concepts.

A revision of the Law that's somewhat closer to the truth would be something like: "If a bad guy can persuade you to run his program on your computer, and your operating system allows that program to damage the system, other programs, or your data, it's not your computer anymore." The operating system does not get a free pass.

Re:Look at how YOU would do it.

[drsmithy]

Which ones? I just tried unsuccessfully to access the web through Finder, and when I try to access the local file system through Safari it was smart enough to call Finder, but doesn't access the file system itself.

Explorer doesn't "access the web", either, it just loads up the IE components inside the Explorer window (in the same way you can embed an Excel spreadsheet into a Word document and it fires up Excel from within Word).

I mentioned KDE in my original post, which does integrate the browser and UI tools, but since that's a desktop, not an OS, it's a different matter.

No, it's exactly the same "matter". You are creating a false dichotomy.

I suppose it's arguable that "Explorer" is only a UI level system in Windows, but that seems disingenuous since unlike in Linux Windows has only one supported UI, and the UI is coded and provided by the OS maker as part of the OS.

You are conflating a marketing issue (Windows only comes with one shell) with a technical issue (how the various components run and interact).

It's also true that unlike in KDE where nothing can execute with elevated privileges without a password being entered (either the user's password through sudo or the root password through an su), Explorer can and has been known to execute code as 'system' without checking first.

That's a pretty serious bug. Evidence ?

I'm also not clear on how two different executables that access the exact same dlls, and perform in an identical fashion are not the same thing. The fact that Internet Explorer loads the "bookmark" module, and the "Google Search Bar" module, and Windows Explorer loads the "jump around the file system" module doesn't make them different software. They are functionally identical except for a few modules that are loaded differently depending on how they are called.

Try thinking about how many different programs do exactly the same thing using glibc (because, well, that's the point of having a shared library to do it).