Slashdot Mirror

← Back to Stories (view on slashdot.org)

Famed Security Researcher 'Mudge' Creates New Algorithm For Measuring Code Security (theintercept.com)

Posted by EditorDavid on from the Underwriters-Laboratory-for-code dept.

Peiter "Mudge" Zatko and his wife, Sarah, a former NSA mathematician, have started a nonprofit in the basement of their home "for testing and scoring the security of software... He says vendors are going to hate it." Slashdot reader mspohr shares an article from The Intercept: "Things like address space layout randomization [ASLR] and having a nonexecutable stack and heap and stuff like that, those are all determined by how you compiled [the source code]," says Sarah. "Those are the technologies that are really the equivalent of airbags or anti-lock brakes [in cars]..." The lab's initial research has found that Microsoft's Office suite for OS X, for example, is missing fundamental security settings because the company is using a decade-old development environment to build it, despite using a modern and secure one to build its own operating system, Mudge says. Industrial control system software, used in critical infrastructure environments like power plants and water treatment facilities, is also primarily compiled on "ancient compilers" that either don't have modern protective measures or don't have them turned on by default...

The process they use to evaluate software allows them to easily compare and contrast similar programs. Looking at three browsers, for example -- Chrome, Safari, and Firefox -- Chrome came out on top, with Firefox on the bottom. Google's Chrome developers not only used a modern build environment and enabled all the default security settings they could, Mudge says, they went "above and beyond in making things even more robust." Firefox, by contrast, "had turned off [ASLR], one of the fundamental safety features in their compilation."
The nonprofit was funded with $600,000 in funding from DARPA, the Ford Foundation, and Consumers Union, and also looks at the number of external libraries called, the number of branches in a program and the presence of high-complexity algorithms.

41 of 77 comments (clear)

  1. Re:Nonprofit In Basement by Megol · · Score: 1

    Too little space in the attic?

  2. Re:Yep by Anonymous Coward · · Score: 1

    Exactly. $600K, and two people in a basement doing simple software assessments does not equal a non-profit. It amounts to a very lucrative and profitable government contract. I would even go so far as to venture the idea of laundering.

    Former NSA, DARPA funding. "nonprofit" ... Yeah, and the moon is made of cheese.

    I bet their house is a fucking mansion.

  3. Re:Yep by Anonymous Coward · · Score: 3, Insightful

    They've evaluated 12,000 programs, and they have to purchase the ones that don't have fully functional trial versions available. That isn't going to be cheap. I presume they've set up a decent lab, that could be $50K-$100K just in hardware. Then there's developer time, lawyers, the technical review board that looks at their static analysis methods...

    If this effort improves the state of application security (or at least steers users away from products that aren't improved), I'd say $600,000 is a pittance to pay. Preventing just one SCADA compromise could save many times that amount of cash. I'm OK with my tax money going here.

  4. Re:Only tested OSX? by Desler · · Score: 1

    Who cares about security on OS X? 10% of all desktop users I would imagine. And where did you get the idea they were only testing OS X applications. You seem to have invented that out of whole cloth.

  5. Re: The usual reason for old tools by Anonymous Coward · · Score: 1

    Who hasn't?

  6. Modern compiler protective measures by khz6955 · · Score: 2, Insightful

    "Microsoft's Office suite for OS X, for example, is missing fundamental security settings .. despite using a modern and secure one to build its own operating system, Mudge says. Industrial control system software, used in critical infrastructure environments like power plants and water treatment facilities, is also primarily compiled on "ancient compilers" that either don't have modern protective measures or don't have them turned on by default"...

    Security is only as good as the underlying Operating System and Memory Management Unit , which is to say in the case of Microsoft Windows running on Intel hardware is non-existent . And nobody in his/her right mind would connect industrial control systems directly to the Internet.

    1. Re:Modern compiler protective measures by Desler · · Score: 3, Informative

      Security is only as good as the underlying Operating System and Memory Management Unit , which is to say in the case of Microsoft Windows running on Intel hardware is non-existent

      The hell are you talking about? Intel chips have had MMUs for 30 years now.

    2. Re:Modern compiler protective measures by Anonymous Coward · · Score: 5, Informative

      > And nobody in his/her right mind would connect industrial control systems directly to the Internet.

      I used to work in the oil & gas industry (I'm retired now).

      We used to deal with a lot of eccentric PLCs and other control systems.

      A lot of the earlier equipment would just work. Sure, you had to program it using some ancient software package running under pure DOS mode with an equally antiquated laptop, but once you'd done that all you had to do was feed them power and away they'd go.

      Then they started including protection systems in the PLCs. I could never figure out why, it just made them all a huge pain in the ass to deal with. I guess it had to do with regulations (since some of that equipment could, conceivably, be used for very nefarious purposes if it landed up on the black market), but it always seemed to me like it had more to do with eliminating the second hand market and ensuring vendor lock-in.

      Sometimes it was just a hardware FOB located somewhere on the controller in a proprietary port. Sometimes it was a literal 3.5" floppy drive built straight into the unit itself, sometimes it was a floppy drive that you had to connect temporarily to load up the licensing information off a disk. Sometimes you could "activate" the unit over whatever port you were using to program the thing (sometimes RS-232, sometimes RS-485, sometimes 10 base T ethernet, etc). For the most part, it was all offline, while there were a few systems that required online connectivity you really just had to download a bunch of files to a computer somewhere, then hook that computer up to the PLC and let the software work it's magic.

      Then, someone (I'm looking at you, Allen Bradley) decided it'd be a great idea to make the PLCs require a live fucking internet connection to activate with some remote server.

      I'll never forget the day I was doing field work up in Northern Alberta at a huge oil production facility, and someone forgot to pre-activate the PLCs we were working with at the time. Of course everyone was on a tight deadline and the hardware had to be operational NOW, not tomorrow or the day after, and the PLCs were already installed and wired up in the control cabinets, so we couldn't just yank them out and take them up to the control office and plug them into the internet. We landed up stringing together god knows how many spare CAT5 cables, couplers, and hubs to form a temporary 200m line that ran all the way across the facility floor, through several doorways, up and down at least three stairwells, and into the office where they actually had internet. And even then, the fucking PLC wouldn't activate because the firewall rules were setup for default-deny-all, and nobody could figure out what the hell the thing wanted before it'd activate, so we found someone fairly high up that was desperate enough to basically say "turn around, you don't wanna see this" and plug the thing straight into the modem for a few minutes.

      Of course, the likelihood of that system getting pwned at that exact moment was pretty much a statistical impossibility, but still. From what I've heard, there's a lot of control systems out there that now need persistent connections to the internet, and if that connection fails then your licenses will invalidate and everything will grind to a halt.

      But... yeah. That's one way critical systems can land up connected to the internet.

    3. Re:Modern compiler protective measures by phantomfive · · Score: 2

      Then, someone (I'm looking at you, Allen Bradley) decided it'd be a great idea to make the PLCs require a live fucking internet connection to activate with some remote server.

      They should be publicly shamed and plastered against the wall.

      --
      "First they came for the slanderers and i said nothing."
    4. Re:Modern compiler protective measures by Tom · · Score: 2

      And nobody in his/her right mind would connect industrial control systems directly to the Internet.

      aka "someone is sure to do exactly that"

      --
      Assorted stuff I do sometimes: Lemuria.org
    5. Re:Modern compiler protective measures by Antique+Geekmeister · · Score: 1

      > And nobody in his/her right mind would connect industrial control systems directly to the Internet.

      The designer of an industrial system usually has _no_ control over how remote sites configure their local networks. None.

      Many admins, and their supervisors, insist on dynamic monitoring of equipment to report its status. The investment in time, energy, and even network hardware to provide better protected network access to that equipment is a real expense which they often choose not to pay. If they think about in a conscious way, they think "my need to monitor or control this equipment at need is more important than maintaining a fragile and resource costly secured access to this and the other equipment we have to deal with".

      I must also admit that this is a reason the Internet Of Things is dangerous, and I'm grateful that NAT based access has been used so effectively to extend the lifespan of IPv4. The forced switch to NAT for IPv4 users has enforced a basic defense for most environments against casual network scanning.

    6. Re:Modern compiler protective measures by steveb3210 · · Score: 1

      A firewall would do just as well as NAT without the overhead of NAT

    7. Re:Modern compiler protective measures by khz6955 · · Score: 3, Informative

      @Desler: "The hell are you talking about? Intel chips have had MMUs for 30 years now."

      Yea, and for 30 years now the Intel MMU has been unable to reliably isolate user processes or at least tell the difference between code and date.

    8. Re:Modern compiler protective measures by Antique+Geekmeister · · Score: 2

      I must say, from long experience, that maintaining a pure firewall does _not_ do as well as NAT. The network overhead of NAT is unnoticeable with even the most modest household modems and routers of the last few decades. Maintaining even a modest firewall is often fragile, vulnerable to profound configuration errors, and likely to cut off expected services at the most inoportune moments. This is compounded by the genuinely awful interfaces and management tools for many firewalls. Simply activating NAT is so vastly simpler and reduces the attack surface so profoundly that it leaves time and money to do more effective internal firwalls, to configure as desired. that I find myself alarmed at _any_ environment that insists on putting all its devices on publicly routable IP addresses and relying, on correct and consistent configuration of firewalls to protect those systems.

    9. Re:Modern compiler protective measures by Jeremi · · Score: 1

      A firewall would do just as well as NAT without the overhead of NAT

      ... but only if it was actually installed, which I believe is the parent poster's point. Because NAT has become largely necessary for IPv4 access, people have a motivation to install and use it. (People should be motivated by security concerns as well, of course, but all too often they are not, because good security isn't necessary to get the system working ASAP, and sometimes gets in the way of getting the system working)

      --


      I don't care if it's 90,000 hectares. That lake was not my doing.
    10. Re:Modern compiler protective measures by steveb3210 · · Score: 1

      People didn't choose NAT - it just came with whatever home router they bought and thats just how the world works for them. If NAT hadn't been needed, I think the world would have evolved perfectly fine with home routers that came with a proper default firewall without the need for NAT.

  7. Re: Only tested OSX? by Desler · · Score: 1

    But that's hard and stuff.

  8. Re:The usual reason for old tools by Anonymous Coward · · Score: 1

    This a thousand times over.

    I've had to change over to new toolchains quite a few times, and although most code keeps working, there's always the code written by That Guy and that code always breaks in a million different little ways at every perturbation. And a toolchain change tends to be the big perturbation that breaks all fragile code.

    And that wouldn't be so bad, perhaps, if you just got a list of compiler errors so you could tell management that there's a hundred problem sites, so it's going to take nine months. No, when you fix one thing, another problem appears, you have no way to get a bound on it. And when you made it compile it turns out that some things have started to work differently in subtle ways, sometimes making the program crash, sometimes just making it do the wrong thing.

    And as if that wasn't bad enough there's often politics involved. I've been in a situation where we were ordered my higher management to use a new tool, but That Guy wasn't on board and of course That Guy has a lot of clout in the organisation, otherwise he'd have been fired ages ago. So predictably, the new tool found lots of serious errors in code he was responsible for, some of which actually explained mysterious, hard-to-diagnose problems our clients had been having. But he wouldn't allow any fixes to ‘his’ code because of ‘muh code stability’. We talked to management but they, after letting the situation drag on and fester, eventually decided to... abandon the new tool! (And not fix any of the issues we found.)

  9. Mudge - famed hacker by phantomfive · · Score: 1

    Best part of Mudge's wikipedia page is where it describes L0pht as a "hacker think tank." rotfl way to sell yourself.

    --
    "First they came for the slanderers and i said nothing."
  10. Re:Yep by MadX · · Score: 1

    Yes, I agree that the costs are going to be huge - ESPECIALLY the lawyers. We all know who gets the lions share

  11. Re:Yep by arglebargle_xiv · · Score: 2

    I bet their house is a fucking mansion.

    It's beyond that, it's practically a palace, room for over 300 people.

  12. Re:Nonprofit In Basement by arglebargle_xiv · · Score: 1

    Why in the basement? Seems a bit suspicious to me.

    His mom lives upstairs?

  13. helpful perhaps, but doesn't solve the problem by Anonymous Coward · · Score: 1

    This is all well and good, but it doesn't really solve the problem. The most severe problem is that conventional C/C++ programming is inherently prone to critical memory access vulnerabilities. And while code analysis tools can help, they don't (and cannot) correctly identify all such memory access bugs. But since the advent of C++11, it is actually possible and practical to substitute C++'s unsafe elements (i.e. pointers, references, arrays, ...) with compatible, (memory) safe replacements, thus eliminating any possibility of invalid memory access. One (nascent) implementation of such replacements is the SaferCPlusPlus library (http://duneroadrunner.github.io/SaferCPlusPlus/). (Note, shameless plug.) The performance cost is quite modest and anyway, imo well worth it given the security costs we're paying now.

    1. Re:helpful perhaps, but doesn't solve the problem by Anonymous Coward · · Score: 1

      > The most severe problem is that programming is inherently prone to access vulnerabilities.

      Fixed That For You

      > But since the advent of C++11, it is actually possible and practical to substitute C++'s well known and stable elements (i.e. pointers, references, arrays, ...) with complex, untestable abstractions vulnerable to compiler specific and destabilizing re-interpretation of what the source code actually said..

      Fixed That For You, Too.

  14. Firefox ASLR by ameen.ross · · Score: 3, Informative

    Really? Firefox has no ASLR? So when did that regression happen, because it used to have it, even for binary components (addons), see: http://blog.kylehuey.com/post/...

    Or am I misunderstanding somehow?

    --
    $(echo cm0gLXJmIC8= | base64 --decode)
    1. Re:Firefox ASLR by Anonymous Coward · · Score: 2, Informative

      I can confirm that at least on Win7 Firefox uses ASLR. For example, Firefox.exe has an image base of 40 0000h but it's loaded at 10D 0000h. Similar story for some other modules I've checked.

      Maybe it's different on OS X though, because that is apparently the only platform this ‘famed hacker’ tested on. His main claim to fame, by the way, is boldly boasting he could bring down the entire internet in 30 minutes. Turns out that was an erm... slight exaggeration.

  15. Test your own code for these features. by Anonymous Coward · · Score: 1

    There's a shell script that'll tell you what is and isn't compiled with these options on your own system.

    http://tk-blog.blogspot.com/2009/12/new-version-of-checksecsh.html

    Despite what the summary says, you actually have to explicitly tell the compile to enable these security features, otherwise you don't get any of them.

    Compile with these options: -fPIE -D_FORTIFY_SOURCE=2 -fstack-protector-all

    Link with these options: -Wl,-z,relro,-z,now,-z,noexecstack -pie

    When compiling shared libraries, change "PIE" to "PIC" for some reason.

  16. CyberInsecurity: The Cost of Monopoly by khz6955 · · Score: 2

    How the Dominance of Microsoft's Products Poses a Risk to Security

  17. Fairly dismal community here by mspohr · · Score: 2

    I submitted this story since it looked like an interesting approach to a thorny problem of measuring and reporting on software security.
    I was hoping that someone would comment on this approach to grading software for potential vulnerabilities.

    At 50 comments now, nobody has posted a comment which addresses the topic of the article.
    Instead, we have a lot of people who are apparently jealous that someone is getting paid for providing a service.
    Other people have taken the opportunity to trash talk various operating systems, languages, hardware and software.
    Most benign (but still irrelevant) are war stories how some courageous, smart iconoclast overcame co-worker and institutional stupidity to save the day.
    And, of course, personal attacks.
    I guess this is a sign of the times. We have no discussion of substance, just flame wars.
    Make Slashdot great again!
    I really must find something (anything) better to do with my time.

    --
    I don't read your sig. Why are you reading mine?
    1. Re:Fairly dismal community here by gweihir · · Score: 1

      The metric is pretty worthless. It can identify extremely bad code, but that it is. It will however con people into thinking it is much better than that and as such do a disservice to software security. Yet another worthless metric that delivers a mostly meaningless number.

      The only way at this time to get a good assessment of code-quality is still having an experienced and capable expert look at it manually. Unless we get strong AI at some time (highly doubtful) that will very likely remain the only way.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    2. Re:Fairly dismal community here by mspohr · · Score: 1

      It seems to me that it identifies code which uses unsafe programming practices (such as compilers without security settings set) and code which uses unsafe libraries. The code itself might be OK but the environment might be dangerous.
      For instance, they mentioned that the Windows version of MS Office uses the latest version of the compiler with security settings and is therefore scored highly but the OSX version uses an old "unsafe" compiler and scores poorly.
      I agree that the methodology has its limits but don't think it's "worthless". If you have a program which scores low on their scale, it's probably best to avoid it if possible. If you have a program which scores high then it's probably more secure (but not guaranteed secure). This metric has some value.
      There is no guarantee of bulletproof code but there are good programming practices which can lead to better code.
      I agree that the only current way to really be certain about code is to look at it manually but it seems that they are starting to put together an AI engine which over time could become more capable.

      --
      I don't read your sig. Why are you reading mine?
    3. Re:Fairly dismal community here by gweihir · · Score: 1

      Well, this is a judgment call. I personally find metrics that may give people a false sense of security a very bad thing. The thing here is that this is a decidedly "experts-only" metric because most people cannot interpret it. Metrics are however routinely used by non-experts (a.k.a. "managers") and that makes any kind of expert-only metric dangerous.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    4. Re: Fairly dismal community here by mspohr · · Score: 1

      I agree that many people will misunderstand the limitations of the metric (managers are a good example).
      However, it may help push good programming practice.

      --
      I don't read your sig. Why are you reading mine?
    5. Re: Fairly dismal community here by gweihir · · Score: 1

      I doubt it. Those that use good programming practices use them because they realize their worth. The others are a lost cause IMO.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  18. Gentoo by omkhar · · Score: 1

    An actual security reason to keep using Gentoo!

  19. WTF? by gweihir · · Score: 1

    First, I have never heard of this "famed" person and I have been in computer security research for quite a while. Second, what they describe is basically worthless: They can identify really, really, really bad code, but if it is better than that their metric is unusable.

    Sounds like a con to get attention and funding to me, nothing more, and they do harm by promoting yet another useless metric.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    1. Re:WTF? by ebvwfbw · · Score: 1

      What is quite a while? 5 years? That's really nothing, I go back 30 years. Maybe you're in the windows world? Here, let me google that for you - http://lmgtfy.com/?q=mudge . If you don't know him, don't take it too hard. We can't know everything and everyone after all.

      Check out l0pht, etc. Stuff that was done about 20 years ago. I've met him, he's a really smart guy. Just look at his work with l0pht crack. If he's coming up with it I bet it's good.

  20. Re:Airbags? Anti-lock brakes? by gweihir · · Score: 1

    Does not sound like a security expert to me. More like somebody that want so con people out of their money.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  21. Office for Mac by JesseEnjaian · · Score: 1

    The lab's initial research has found that Microsoft's Office suite for OS X, for example, is missing fundamental security settings because the company is using a decade-old development environment to build it, despite using a modern and secure one to build its own operating system, Mudge says

    hahah I went to undergrad with one of the developers. Good to know he hasn't been asked to update it since then. Seriously though, that's kind of the problem with the corporate form and fiduciary duty: companies will nicely box and sell a turd as a wholesome source of fiber if it's the only way they can figure out how to increase profit.

  22. Re:Tor browser by AlphaBro · · Score: 1

    More expensive? Sure. Almost impossible? No.

  23. Re:Yep by Beeftopia · · Score: 1

    I was always puzzled about the outrageous rates at which companies billed out software engineers. But when I got into consulting, I found out the hard way how important lawyers are. And then the larger the company gets, the more specialized people are needed. Contracting officers, accountants, site security, hardware, health insurance, unemployment insurance, taxes. All of those costs have to be covered by the revenue from products, services and billed-to-the-client staff. That made the hourly rates suddenly seem much less dazzling, and let me understand why I got such a small slice of them.