Slashdot Mirror


Why "Designed For Security" Is a Dubious Designation

itwbennett writes The list of products designed to be security enhanced that turned out to be anything but seems to get longer by the day. In just the latest instance, reported by Wired last week, the crowd-funded privacy-enhancing home router Anonabox had to be recalled after an independent researcher discovered serious security flaws in the product. But security experts caution that the real problem may be bigger than vulnerabilities hidden in application code: "Designed for security products don't just have to be good. They have to be beyond reproach," explains John Dickson, a Principal at the Denim Group. "All it takes is one guy with a grudge to undo you."

58 comments

  1. OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 3, Informative

    OpenBSD proves the claim to be wrong.

    1. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 0

      OpenBSD proves the claim to be wrong.

      OpenBSD is as flawed as anything else: http://www.cvedetails.com/product/163/Openbsd-Openbsd.html?vendor_id=97

    2. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 0

      Every single OS and implementation which ever claimed to be "secure" proves the claim to be wrong.

    3. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 0

      In 2014 it had one DoS vulnerability, and there's no data for 2015...

    4. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 0

      In 2014 it had one DoS vulnerability, and there's no data for 2015...

      And a DoS attack isn't particularly a "security" vulnerability. A vulnerability, yes, it'll take you down, but it's not going to compromise the securit of the machine.

    5. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 0

      So OpenBSD had 147 vulnerabilities between 1999 and 2015.

      Debian, the best of the Linux distros, had 234 vulnerabilities between 1999 and 2015.

      Firefox, which is arguably simpler than both OpenBSD and Debian, managed a whopping 1173 vulnerabilities between just 2003 and 2015.

      OpenBSD is clearly doing something right. Mozilla is clearly doing something totally wrong.

    6. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 3, Insightful

      Probably because OpenBSD isn't designed for security. They have their priorities straight: portability, standardization, correctness, proactive security and integrated cryptography.

      Most "designed for security" products reverse the order of things. Start with a set of cryptographic solutions, sprinkle on some magic security dust, hack on it until it appears to work (i.e. "correctness"), and toss standards and portability concerns out the window. Even though those latter two things give you a fixed point of reference to shoot for when it comes to correctness (whether or not the reference itself is flawed).

      Security is a process, not a product. I would note that "proactive security" is another way of saying bug mitigation--sandboxing, address randomization, stack smashing detection, etc.

    7. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 0

      You're comparing apples and cantaloupes there though. Any browser is going to be susceptible to more vulnerabilities than say an operating system kernel which is much more tightly controlled. Another thing to consider is for all that they fixed what has yet to be discovered? OpenBSD is a good distribution. So is Debian. But Linux has more eyes on it than OpenBSD so there may be more scrutiny there.

    8. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 0

      Interesting theory.

      I have another: Firefox is becoming less and less interested in stability (it's been particularly bad this year) and more and more interested in chasing fancy new features. I expect version 93 (due any moment) to include a kitchen sink.

    9. Re:OpenBSD proves the claim to be wrong. by Anonymous Coward · · Score: 0

      That website is more than useless. It's dangerous.

      For example, Red Hat had 186 code execution CVEs compared to 31 for OpenBSD; and 93 privilege escalations compared to 14. Score for OpenBSD? But look at the CVEs! They're almost all for third-party software. Many of those which count against Red Hat should count against OpenBSD, and vice-versa.

      There's no rhyme or reason behind why those vulnerabilities are attributed to one or the other system.

      If you want to compare apples-to-apples, you need to narrow your focus: kernel and libc. Linux has more kernel vulnerabilities than OpenBSD. And glibc is a fscking nightmare of problems--memory corruption, thread races, etc. Most of this is because Linux and glibc try to be more featureful, and go through great lengths to improve performance. Simplicity is one of the key goals of OpenBSD. For OpenBSD, no performance optimization is worth the cost of difficult to understand code. Something like RCU would never became as pervasive in the OpenBSD kernel as it has in the Linux kernel, presuming it ever made it in at all.

      Want proof? Let's look at the Linux kernel vulnerabilities, none of which (AFAICT) were included in the Red Hat vendor results:

      http://www.cvedetails.com/product/47/Linux-Linux-Kernel.html?vendor_id=33

      That's 8 kernel execution exploits in 2014, and 1 for 2015. How many in OpenBSD? Well, going by this DB it's 0 and 0 (in fact 0 all the way back to 2011). Score for OpenBSD again? But that's not entirely fair, because most OpenBSD fixes don't get a CVE, even though they may be exploitable. So just for good measure, let's look at OpenBSD kernel bug fixes covering the same time period:

      http://www.openbsd.org/errata54.html
      http://www.openbsd.org/errata55.html
      http://www.openbsd.org/errata56.html

      I count only a few kernel panics. No code execution.

      I'm sorry, but there's no equivocation here. Based on my professional experience as a security engineer _and_ what limited data we have, OpenBSD is far and away the more secure product. Of course, I don't use OpenBSD for everything. Performance and hardware support matters. And writing secure application code should be your #1 priority. But I'd never run a complex, internet facing service (e.g. a web site with complex server-side request processing) using regular software on Linux as long as it was my decision. For example, the only box I'd ever trust to run a vanilla PHP or Node.js webapp would be from a chrooted OpenBSD web server. If I had to use Linux, it'd be on throw-away nodes where valuable data was kept on separate boxes (preferably running OpenBSD), with Linux simply hosting the front-end software. And I'd want to make sure that those Linux nodes got automated security updates. Any intervention required simply allows too much risk to enter the equation.

    10. Re:OpenBSD proves the claim to be wrong. by TheRaven64 · · Score: 1

      Let's take a web browser as an example. Chromium has a number of sandboxing strategies that provide different levels of protection. On Linux it can use chroot (which can't restrict network access), SELinux or seccomp-bpf (which aren't great because they don't differentiate between different instances of the same program, so one sandbox can access anything that any sandbox can access). On OS X, it uses the TrustedBSD-based sandboxing framework. On Windows it uses kernel ACLs. On FreeBSD, it can use Capsicum, which provides the best isolation (though this support isn't fully upstreamed, it's in progress because Google is porting Capsicum to Linux for the ChromeBook). On OpenBSD, it can only use chroot, because the OpenBSD developers believe that complexity is the enemy of security and don't implement security features for userspace software to use (they did have systrace, but it was shown to be trivially vulnerable to timing attacks and never fixed).

      Bottom line: the features that a kernel provides can have a big impact on overall system security, and OpenBSD lacks a lot of the security features that you'd expect from a modern system.

      --
      I am TheRaven on Soylent News
    11. Re:OpenBSD proves the claim to be wrong. by Burz · · Score: 1

      You forgot AppArmor, which is one of the most widely-used on Linux. Of course, I'd much rather have Qubes' isolation mechanisms so that my banking, work and leisure activities (and even my NICs) don't even share the same virtual machine -- My data sets are kept separate and the interfaces between those domains are simple and very strong.

  2. just a marketing phrase by turkeydance · · Score: 1

    Just Do It

  3. OpenBSD by Anonymous Coward · · Score: 1

    Despite a few discovered problems over the years, for the most part, OpenBSD's security reputation holds up quite well, especially in comparison with other projects and products, open and proprietary.

    1. Re:OpenBSD by Anonymous Coward · · Score: 0

      Security by simplicity is an acceptable design.

  4. All it takes is you not doing what you claim by Anonymous Coward · · Score: 0

    You promise something, you don't deliver, what do you expect to happen?

    At least Linus says he's not going to warranty his shit fit for anything.

  5. meaning by Anonymous Coward · · Score: 0

    security is the possibility to not do something.

  6. "Designed For" by jklovanc · · Score: 1

    The phrase "designed for" is irrelevant. It does not matter what something is "designed for". The important part is the implementation. "Designed for security" is not the same as "secure".

    1. Re:"Designed For" by gstoddart · · Score: 1

      Moreover, "designed for security" is just a meaningless marketing term. It's a catch phrase, but it doesn't actually mean much, apparently.

      You can't just say "I'm making the most secure thing evar" and have that mean anything unless you've spent a LOT of time and effort making it so. You can't just throw something together and think you've made something secure.

      And if you make this big bold claim, and then trip on your own dicks, you look like idiots.

      My general rule would be to treat a claim like "designed for security" as at best puffery, and at worst a dangerous lie designed to make your product look good. But I sure as heck would't treat it as an indication of actual security.

      --
      Lost at C:>. Found at C.
    2. Re:"Designed For" by rtb61 · · Score: 2

      Designed for security needs to follow the KISS principle, keep it simple stupid. Only allow it to do what you need it to do, zero flexibility. Keep the device as simple as possible, not one feature beyond what is absolutely necessary to it's designed function. This is the most important rule in designing for security. Your lock should only open able one way, the way you choose. The whole idea of bios/os/application is the first thing that has to go, great for general computing but for secure computing to many ways of doing things you didn't intend for it to do.

      Break down the overall function into their broad elements and separate and secure each element and provide fixed encrypted communications pathways between each element. At least with completely separate hardware and operating applications between each element, parallel processing will be far more efficient and far less processing capability will be required.

      The more complex, the harder to secure, the simpler the easy it is to secure.

      --
      Chaos - everything, everywhere, everywhen
    3. Re:"Designed For" by Anonymous Coward · · Score: 0

      I'm "designed for being handsome". Unfortunately the hackers have been very active...

    4. Re:"Designed For" by jklovanc · · Score: 1

      Agreed

    5. Re:"Designed For" by jklovanc · · Score: 1

      That is your definition of what "designed for security" means. There is no authority to confirm or deny the correctness. There could be other "definitions". See how useless the phrase is? It is just market speak.

    6. Re:"Designed For" by Anonymous Coward · · Score: 0

      Designed for security needs to follow the KISS principle, keep it simple stupid.

      This post designed for redundancy design.

    7. Re:"Designed For" by rtb61 · · Score: 1

      What definition? I was just formulating an element of designed for security and not the totality of it, just one rule. Design is not a definition, design is a continual process (now that is a definition, well, only in part ;) ).

      --
      Chaos - everything, everywhere, everywhen
    8. Re:"Designed For" by jklovanc · · Score: 1

      Designed for security needs to follow the KISS principle, keep it simple stupid.

      In your opinion. Other people may have other opinions. Since there is no authority different people may use the same phrase and have different opinions about it.

  7. Re:The Republicans don't think us peons... by Coren22 · · Score: 1

    So, you are the Republicans hate us troll. Thank you for outing yourself, now you can receive justified scorn.

    --
    APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
  8. Re:The Republicans don't think us peons... by Coren22 · · Score: 1

    Did I argue anything? I am confused.

    No I don't believe anyone hates "us", but projecting hatred specifically on Republicans when the Democrats are just as guilty is very naive. It just shows your partisan blindness on the issue.

    http://en.wikipedia.org/wiki/P...

    Take a look at everyone who voted for it in 2001 and renewed it in 2011. Are they all Republicans? Than it isn't just Republicans that are against privacy.

    --
    APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
  9. Re:The Republicans don't think us peons... by Fwipp · · Score: 1

    Greenwow posts as themself fairly regularly.

  10. on designed for "security" by secPM_MS · · Score: 1

    Systems submitted for evaluation under TCSEC B2 and better had to be designed for security - layering, TCB minimization, ... were all mandated in addition to support for trusted MAC functionality. When I am designing for "SECURITY" I want to simplify the critical protocols so that they can be described by a state machine and then implement them in silicon.

    1. Re:on designed for "security" by Anonymous Coward · · Score: 1

      This!

      I'm not an electrical engineer and wouldn't know the first thing about moving an algorithm to silicon. But I always code things like parsers and protocol stacks using explicit state machines. It's the only way you'll have a chance at being able to proof the correctness of each step. Of course, explicit state machines can be buggy too, and it takes practice to write them in a manner that makes them easily reviewable. But you really have no chance in hell of shaking out complex bugs from a mess of functions and/or deeply nested conditionals processing data in arbitrary chunks.

      For example, a parser which processes data octet-by-octet means each step has only 256 possible values to deal with. That means you can easily verify each step, either by manually or by automated provers--e.g. KLEE. And it makes it easier to verify (though not perfectly) the different dispositions your data structures can end up in.

      Most of my software is simply a collection of blackbox components with explicit state machines--usually hand-rolled, but sometimes using tools like Ragel or LPeg--at their heart. I know there's a lot of other software out there that automates the writing of state machines, but they tend to introduce their own layer of obfuscation. And automated state machines generally make integration difficult. For example, they may not be restartable, making them difficult to integrate with non-blocking I/O. The only big exception in this regard is Ragel, but for all its elegance Ragel has an initially steep learning curve, and so isn't suitable where your code will need to be hacked on by many others.

      And nobody should be allowed to use regular expressions unless they can write their own regular expression implementation.

    2. Re:on designed for "security" by Burz · · Score: 1

      When I am designing for "SECURITY" I want to simplify the critical protocols so that they can be described by a state machine and then implement them in silicon.

      It would be interesting to see a Xen hypervisor implemented in silicon, as that is what Amazon EC2 and Qubes OS base their security on. Qubes doesn't even use kernel-based permissions for its single-user desktop model; It gives you the means to control dom0 and everything else resides in VMs.

  11. Rule #1, don't taunt happy fun hackers by GoodNewsJimDotCom · · Score: 3, Insightful

    I've found that the more you tout that you have good security, the more recreational hackers come out of the wood work who would otherwise have no interest in your product other than you make it sound like a challenge. If you want good security, do your encryption, do your trip wires, keep important stuff server side, etc etc, but don't brag about it. Bragging about security on the Internet is like putting on a white karate outfit with a black belt and strutting all around the low income parts of town. Maybe you are secure in your components or your not, but don't go looking for people to try and break you.

    1. Re:Rule #1, don't taunt happy fun hackers by Anonymous Coward · · Score: 1

      If you're afraid of peer review, you have no goddamn business desigining secure devices or software.

      It's better to be hacked to pieces by someone who will tell you how you've been hacked than to be hacked by someone who will keep everything a secret, to be sold to the highest bidder.

    2. Re:Rule #1, don't taunt happy fun hackers by Kardos · · Score: 1

      It sounds like your approach to security is to risk manage it, like the car companies in Fight Club. Doing "some security stuff" and then keeping quiet and crossing your fingers hoping that nobody takes an interest does not inspire confidence. If a recreational hacker can defeat your security on a whim just to show off, you don't stand a chance against actual criminals who will quietly break your security and then proceed to exploit you for everything they can and for as long as they can.

      > Maybe you are secure in your components or your not, but don't go looking for people to try and break you.

      Actually that's exactly what you do. Look up "bug bounty".

  12. This.... by Whiteox · · Score: 1

    SRW Iron https://www.srware.net/en/soft... is touted to be a secure browser [Warning: Demands Java after install]. I don't think it is.
    In fact, playing around with FF shows that the problem isn't the browser, but the reliance on 3rd party cookies as 1 example of the way websites are constructed.
    If you load FF's Lightbeam and check all the 3rd party sites, block access to them, they often stop the parent website from operating properly or at all. Typically, Google and most banking sites won't work without 3rd party links or cookies.
    Then there are routers that claim security but are still running buggy old firmware. AV software like Bitdefender also have issues. AV software still refuses to scan for pup, browser addons and other malware that the UAC allows! I mean if you download an app, UAC asks for permission which you give for that instance, but it automatically gives permissions for all the other installs that come with the package. Why?
    I reckon half of the security issues can be fixed if some clever plug-ins, better AV database and a trusted installer with UAC can be done. EG Spoofing 3rd party links and cookies within the browser.
    I went here http://alcpu.com/CoreTemp/ and downloaded the app a few days ago. It installed on Vista and a Win 7 machine (with MS Defender) I was building. The payload installed as well (Trovi) - I wasn't paying attention btw but the 2nd time I installed it on a Vista machine I had an option of opting out. As a test, I Installed it on another Win 7 box with an updated Kaspersky. It installed (without the payload or opt out!) and when I checked the reports, there was no log or trace that there was any payload at all. Weird, but my respect for Kaspersky has increased and/or the UAC was working properly.
    We all talk about security but there are fundamental, easily fixable things we can do right now. I don't think that this has to do with the OS as most of these issues are external.

    --
    Don't be apathetic. Procrastinate!
  13. As always by Anonymous Coward · · Score: 0

    The biggest security hole lies between chair and the keyboard.

  14. Anonabox was a fraud we pointed out from the start by Anonymous Coward · · Score: 0

    It's not that it was based on a pre-existing product that was the problem- even though that was one of the major things pointed out and they were implying they designed the device itself (for no apparent reason; ie they weren't claiming that they were giving it a new name because of some ethical reasons, like promoting another company, for which was otherwise producing poorly designed products, or, unethical products).

    The product was never going to be secure because the devices code was not available. It could never be totally free software friendly as the hardware (the chips it used) was dependent on non-free software. Without those pieces the device doesn't work.

    Before you even begin talking about security you need the ability to be able to fix the security-related bugs. That's not happening when your not even in control of it. It's the same reason anti-virus software is fraudulent. Anti-virus companies can't fix the bugs because they don't have the code to fix it. All anti-virus does is camouflage the fact you can't do anything about the poor-security of the software you use.

  15. Challenge Accepted!!!!! by perry64 · · Score: 1

    is how I assume all hackers (regardless of hat color) would read "Designed for Security."

  16. So how do you develop relatively secure software? by hlee · · Score: 2

    Here's what works in most practical systems with a little effort:
    - Threat model. Sequence diagram of all external communication between all servers and clients. Apply STRIDE analysis. May be take a step back to see if you can simplify the workflow.
    - Assurance model. State diagram of system. Capture success and error states. Unit tests for each case.

    Add to that third party oversight:
    - Static analysis tools.
    - Third party verification.

    I assume you're not developing mission critical systems that control functions in a nuclear power station, or even a car breaking system. Rather you're looking at consumer or enterprise level systems that involve some confidential, and possibly credit card information. Short deadlines and budget constraints mean you can't spend forever coming up with a solid specification or even do extensive analysis.

  17. Hubris by Rick+Zeman · · Score: 1

    All it takes is one guy with a grudge to undo you.

    Exactly. Just like when Larry Ellison said that there weren't any holes in Oracle....

    1. Re:Hubris by Celarent+Darii · · Score: 1

      The largest hole with Oracle is the hole in your wallet....

  18. Re:So how do you develop relatively secure softwar by Anonymous Coward · · Score: 0

    All the buzzwords in the world won't help you if the code sucks.

    Time and time again experience has proven that security starts with correctly implemented, well-written, reuseable software. Excessive focus on modeling the overall architecture, or even the components, is part of the _cause_ of insecure software.

    When need to learn to get our priorities straight.

    Your examples of nuclear control systems and car breaking systems only show how horrible the system is. Toyota's breaking system suffered from a stack overflow. And industrial control software is hardly the epitome of secure. Showing reliability and consistency in a controlled environment gives absolutely no assurances whatsoever when hackers are able to tickle bugs.

    STRIDE and similar systems are merely work products for project managers. And just because you have unit tests doesn't mean the software works. Well written software will be easy to unit test, but poorly written software can also be unit tested--it's just that poorly written software is better at hiding bugs from the tests.

  19. Secure software by Anonymous Coward · · Score: 0

    To the best of my knowledge, there is one single "software" developed to be secure today, and that is OpenBSD.

    Everything else, and I do mean everything literally, has gotten "security" tucked on later in the process (yes, obviously Windows is there too).

  20. Why? by wisnoskij · · Score: 1

    "Designed for security products don't just have to be good. They have to be beyond reproach" Bullshit. The term is used to designate that at least some minimal amount of effort was used to make the program secure, but not only is a perfect unhackable program impossible but you also get what you pay for. No one expects the $29.99 piece of software with that label to magically have better security than the DoD.

    --
    Troll is not a replacement for I disagree.
  21. duh, marketing is regulated....not by Anonymous Coward · · Score: 0

    Hey I've got some "organic", "natural flavoring", ... oh, and don't forget your "diet" soda with aspartame to keep you an addicted junkie...

    OH, and for bonus I bet you'll love some "internet freedom" laws... don't forget the "right to work"!

  22. Important distinction by Anonymous Coward · · Score: 0

    Security product (firewall, anti-malware software, anonymizing router) != secure product
    it all depends on the quality of the implementation: code and configuration

  23. Finding a Security Flaw Grudge by fsterman · · Score: 1

    I was just embroiled in a dispute with someone who is selling security related software that refuses to address key issues with their security model. I think the situation is probably similar here, software engineers that have the best of intentions but simply lack the expertise to properly execute. Most programmers are engineers who are perfectly capable of building out a working system. However, when it comes to security related software, it's not good enough for something to just work, you have to be able to have a deep understanding of how every component interacts with the larger system.

    --
    Is there anything better than clicking through Microsoft ads on Slashdot?
  24. Torturous sentences by wonkey_monkey · · Score: 1

    The list of products designed to be security enhanced that turned out to be anything but seems to get longer by the day.

    It took me about four tries to parse that one properly.

    Designed for security products don't just have to be good

    That one could've used some quote marks too.

    Captain Pedantic, away!

    That is to say: I am Captain Pedantic (metaphorically speaking) and I will now go away.

    --
    systemd is Roko's Basilisk.
  25. Re:Finding a Security Flaw Grudge by Anonymous Coward · · Score: 0

    As an engineer doing security related products, it is never about our ability. It is unrealistic timelines and priorities.

    Have true stories. It is all about fooling the customer (imagine them as a white fat middle aged PHB). They never have the time to do true evaluation. If there is some encryption somewhere somewhat relevant, they are ok with that. And their engineers usually dont care either. I can hack into our devices without problem (and as an example, pull out a whole nations biometric database of its citizens).

    Well our boss got a Lamborghini (really!) for that project, we engineers got a months salary as bonus. Life goes on. Next project. Next PHB.

  26. Re:So how do you develop relatively secure softwar by Anonymous Coward · · Score: 0

    or even a car breaking system.

    Can we now also blame car burglaries on security experts? Awesome.

  27. Re:So how do you develop relatively secure softwar by Anonymous Coward · · Score: 0

    None of this means anything if you don't know what you are doing though.

    I have worked 10 years developing secure software that actually are secure and I have also worked as a security auditor. Most people just know shit about writing secure code, creating a threat model and so on. Most companies that do security audits are horrible as well, they more or less grep through your code after strcpy and such and that's it. Or they just note what libraries you use and google for vulnerabilities for those and say that you are vulnerable.

    Doing an audit is more than looking at the source, about 50 % of the security problems lies in the design rather than errors in the code. Those are really hard to find, because you really do have to understand the design and why it was designed as it was in the first place.

    The security industry is full of people trying to make a quick buck due to the NSA scare.

    Also, designed to be secure is such a broad statement. An application can be designed to solve one security problem, but it becomes useless because it ignored several other aspects of security because the creator didn't understand the scope of the problem, the attack surface or even what problem s/he was actually trying to solve.

    I have yet to audit the software where no security problems exist.

  28. Windws 8.1 login not designed for security by dog77 · · Score: 1

    The default login for Windows 8.1 is your Microsoft email / cloud account and password. So anyone watching you type in your password has access to your cloud account.

    I don't understand why Microsoft is not given more flack for this decision.

    Your alternatives is to not use their cloud, or use 4 digit PIN or a series of screen swipes, but they don't support a local password. If you understand how to set it up, you can supplement the PIN/swipe with a USB key, but it is not a visable user option and you have to understand what your doing.

    1. Re:Windws 8.1 login not designed for security by Anonymous Coward · · Score: 0

      It's not just Windows - Mac/OSX has that as the first option, too.

  29. Unrealistic expectation by msobkow · · Score: 1

    Expecting any software to be "beyond reproach" is completely unrealistic. All software has bugs; some of them will be exploitable.

    --
    I do not fail; I succeed at finding out what does not work.
  30. Re:So how do you develop relatively secure softwar by Anonymous Coward · · Score: 0

    You need both.

    About 50 % of security issues stem from bugs in the code and the other 50 % are design flaws. The issues from bugs might be found during a code review, but most reviews do not review the design of the software.

    And you don't have unit tests to prove that the code is correct, for all you know the tests might actually be buggy, tests are software as well. You have the tests to prove that changes doesn't change anything that worked before.