Ask Slashdot: Reviewing 3rd Party Libraries?

Carcass666 writes "It is usually good to use existing libraries, rather than reinventing the wheel, especially with open source. Unfortunately, sometimes we have to work with closed source implementations. Recently, we were diagnosing a .NET assembly and, after getting nowhere with the vendor, ran it through a decompiler. The code was a morass of SQL concatenation, sloppy type conversions, and various things that are generally thought of as insecure.

My question is: What are Slashdot readers' preferred tools for analyzing .NET and Java compiled libraries (not source code) for potential security vulnerabilities? Ideally, I would like to know if a library is a security liability before I code against it. For example, Microsoft used to have something called FxCop, but it hasn't been updated for current versions of the .NET framework."

Comments

Open source

[Anonymous Coward]

Easy: use open source libraries.

Re:Open source

[arglebargle_xiv]

Easy: use open source libraries.

Yep, like GnuTLS, or Apple's SSL implementation. You know there won't be any bugs in those, or if there are they'll be very quickly fixed and not sit there unnoticed for years.

Re:Open source

[grcumb]

Easy: use open source libraries.

Yep, like GnuTLS, or Apple's SSL implementation. You know there won't be any bugs in those, or if there are they'll be very quickly fixed and not sit there unnoticed for years.

I remember back in 2008, when the Debian OpenSSL package was found to have a gaping hole in it. I was fascinated at the fact that it had been able to lie their, dormant, until it was discovered and immediately fixed. By rights, the damage should have been widespread.

Back then, I wrote:

My hypothesis – sorry, my speculation is this: People at every stage of the production process and everywhere else in the system trusted that the others were doing their job competently. This includes crackers and others with a vested interest in compromising the code.

So, perversely, yeah: The fact that the GnuTLS hole remained unnoticed for yonks is -weirdly- an argument for using open source libraries. Notwithstanding the fact that the vulnerability remained unpatched for years, it appears to have remained pretty much unexploited for the same period of time.

When processes are perceived to be robust, by black hat and white hat alike, then the mere presence of trust in the system makes them more trust-able. (I won't say trustworthy, because hindsight shows us that they're not.)

Do you want your code to work all the time

[i kan reed]

If yes:
Only use language defined standard libraries and build everything else you need yourself, and even then trust but verify. The third party libraries are always going to be cobbled together.

If no:
Fuck it.

Re:Do you want your code to work all the time

[cheater512]

And make sure you don't do a even worse job.

Re:Do you want your code to work all the time

[i kan reed]

No, they're all shit. The open source ones, if they're really popular and get a lot of eyeballs, are okay.

Re:Do you want your code to work all the time

[Anonymous Coward]

Agreed.

Libraries always have bugs - just like any other code - and pretty much always have crap documentation - just like any other code -. The more obscure, the worst. Open or closed source is irrelevant, but open source ones tend to be more popular, ergo...
They also frequently enough do 95% of what you want, except for that key small bit the authors didn't think about that you only discover for a new feature, 6 months and 2 releases in when your pretty much wed to it.

At least with open source you can:
a) figure out what it does;
b) fix the bugs, or at least find out why it doesn't work and work around it;
c) change it so it does what you need.

Re:Do you want your code to work all the time

[cyber-vandal]

Cool I'll just get started on reimplementing OpenSSL.

Short answer: I don't

[msobkow]

I don't check libraries for security vulnerabilities. I check websites for information about that, and to see how often the provider is refreshing the library with patches and fixes.

If I don't get the feeling that they take their security seriously, I don't use the library. I'm not about to start testing every library of the OS that I build against, nor the Java stack itself. To do so is asinine unless you're in an extremely high security arena -- you have to start with a certain level of trust, and if you don't trust your vendor, don't use them.

Besides, not one of the binary analysis tools I've ever heard of did a really good job. Even source code analysis can miss bugs. If it were possible to fully automate testing in such a fashion, testers wouldn't have jobs.

Re:Short answer: I don't

[bill_mcgonigle]

I'm not about to start testing every library of the OS that I build against, nor the Java stack itself.

Which is sane, but if you use an open source OS and an open source Java stack, there are other people doing that kind of testing, and even more importantly, social pressure for the developers, because they know other people will be looking.

If you're using a secret-source library, then you're completely at the developer's mercy (though your OS can detect certain atrocious behaviors). Like a sibling post mentioned, make sure you can sue them if they screw up and you get hit because of it. Liability is another way to place social pressure on developers.

If it's secret source *AND* you sign away your right to sue, then you better tell your insurance agent, and be prepared to pay dearly for the coverage. That library has got to be essential and/or irreplacable to put yourself in that kind of risk stance.

Re:Short answer: I don't

[plover]

About the only way to deal with third party libraries is through the terms of the contract. If you agree to license it, you're going to hold them responsible for security violations. Perhaps you stipulate they must run their code through a designated scanner like Fortify or Klocwork and they must agree to fix all critical or severe errors, or that they undergo an annual independent code review.

If all that seems like it's too heavy handed for a simple library, just wait till you get hacked. That's a lot more expensive.

Re:Short answer: I don't

[Desler]

Which is sane, but if you use an open source OS and an open source Java stack, there are other people doing that kind of testing, and even more importantly, social pressure for the developers, because they know other people will be looking.

That's funny since there was plenty of "social pressure" on GnuTLS about its crappy code and yet it had unfixed security flaws for most of a decade.

Re:Short answer: I don't

[drolli]

That is an illusion. I have seen Open source code which looks like shit if you take 10 Minutes to look into it. I have seen worse close-source, though, but i have also seen great closed source code.

If you really need to have a look at a close source lib, you can sign a license agreement with the compnay in question.

Re:Short answer: I don't

[mr3038]

I don't check (I prefer word "review" or "audit" here) the libraries for security vulnerabilities before I start using them. However, I only accept libraries than come with the source and I do cursory review of the code with a question in mind:

"Would I be willing to fix a bug in this library if the original author were not willing to fix it?"

Only if the code looks sane enough that I can answer "yes" I even start using the library. And the security is only a small part of the picture here! If the library does anything important within the product, any major bug in it's behavior will cause major issues for my product, too. If I cannot (at least in theory) fix the library, I'm not going to use it.

I might use a closed source library for some totally optional feature in the product but even in that case I'd keep looking for another solution with the source. And with "optional feature" I mean something that can be disabled or removed if any evidence comes up for the library having a security issue.

Re:Short answer: I don't

[bill_mcgonigle]

That's funny since there was plenty of "social pressure" on GnuTLS about its crappy code and yet it had unfixed security flaws for most of a decade.

You'll notice that FileZilla is the only major app that uses it for online work, and that's mainly used by Windows people.

The 'society' knew to use openssl.

Re:Short answer: I don't

[bill_mcgonigle]

. I have seen Open source code which looks like shit if you take 10 Minutes to look into it.

Of course you have - anybody can open source anything.

Have you seen major, mature, popular projects with code that looks like shit?

Re:Short answer: I don't

[drolli]

Major and popular: yes, one or two. And dont expect me to go into the details. Look for yourself.

Typical sicknesses to look out in open source code:
-bad glue code
-missing or ill-defined tests
-lack of documentation
-code which the current maintainer imagined to start a ne coding style (and misestimated the work associated with this)

Still, i have to admit, the worst open source i have seen comes from commercial projects open sourced at some point.

open source only

[gbjbaanb]

IIRC FxCop is a source-code analyser.

There are others, Fortify 360 is one I used at a security-conscious company. But in all cases, they require the source code

Though, to be fair, if you're using a 3rd party closed-source library, then you're at the provider's mercy and should go for other avenues of protection - if you can't see security updates coming regularly, then after the fact protection works: you sue them if it fails. Generally, you don't need to know the source for a closed-source library, its a black box and should be treated as such.

Re:open source only

[Anonymous Coward]

IIRC FxCop is a source-code analyser.

No it is not.

FxCop is a free static code analysis tool from Microsoft that checks .NET managed code assemblies

http://en.wikipedia.org/wiki/FxCop

FxCop actually works pretty well for what was asked for. Microsoft hasn't failed to update FxCop instead they integrated it more into Visual Studio. You can also write your own rules if you have to keep it updated. I would follow the ASP.NET source code project on codeplex, their build scripts integrate code analysis using nuget and a few other msbuild tricks but i do not think they have it setup to scan another assembly. It can't be hard to configure it too however.

All that said i think everyone else's advice is better.

Limit your use of 3rd party libraries as much as possible for many reasons let alone upgrade paths can be difficult.
Use a tool like nuget to make sure they are updated properly.
Follow the projects closely for changes.

OWASP has tools

[Anonymous Coward]

Try OWASP's DependencyCheck tool. https://www.owasp.org/index.php/OWASP_Dependency_Check
It was originally built for Java, but they are starting to move towards .NET vulns as well. Good quality project and developers. The tool essentially tells you if there are publicly known vulns (CVE's).

Apart from that, various static analysis tools work on binaries, both commercial and open source. For Java open source, try PMD or FindBugs. For .NET, FxCop was what I would have recommended.

Write your own code and use FOSS

[turgid]

The more I see of other people's code, the more I am inclined to write and test my own. That way I know it works and when it doesn't, I only have myself to blame. This isn't always possible because most tasks are way to big for a single person, so stick to well-used, well-understood, well-tested (in the real world) FOSS solutions. In general, closed-source vendor-proprietary code is dreadful.

This sucks

[ChrisMaple]

Beta is worthless. I'm out of here, and it will be a long time until I even look here again.

Re:This sucks

[satuon]

Is it still a thing? I get Slashdot classic right now, and I thought they had given up on this thing. See how it looks for me - http://imgur.com/k8JEJsU

Adoption

[Dan East]

Committing to a 3rd party library is a lot like adopting a child. It's a long term commitment that's not easily broken, and you can't ever have a thorough understanding of what the relationship will be like ahead of time. I started a long post about the 3 main reasons for going with a 3rd party library, but decided to delete my long-winded rant. I'll just say that for the most part it comes down to saving time (and thus possibly money). You're rolling the dice and hoping at the end of the day (whenever that is - 5 years from now, 10 years from now?) simply utilizing a 3rd party library will have saved you time and money.

I think I have a hard time with commitment (as in platforms, OSs, and 3rd party libraries), and that's probably to do with the number of platforms I've been involved with over the years, and the number that are now dead and gone. If you are the type to embrace and commit (like "I love Microsoft and I love C# and I'm going to jump in with both feet and that will be my universe") then sure, go ahead and use as many 3rd party libraries as you can. If you hope to have any kind of future portability of your code (as in compiling versions for Windows, iOS, OSX, Android, Linux) then you are entirely at the mercy of those 3rd parry libraries and what they will or won't support down the road. I mainly write code for myself (my own products I market), thus I consider the code I write as an investment. That is why I primarily use C++, because it is the only language I can create native applications in for all the platforms I just named (and more), and also why I look for public domain code or libraries with licensing and source code availability so I will know my future using that library is assured (I can build for other platforms, even if that means doing some work porting the code a bit).

I know that's not really answering you question ("How can I know if I can trust a 3rd party library"), and is more an answer to the question "Should I be using 3rd party libraries in the first place?"

Re:Adoption

[Anonymous Coward]

I'm pretty sure you can create native applications in C# for all the platforms you named.

Java tools

[Guus.der.Kinderen]

Two tools that I use regularly to check Java artifacts: FindBugs: http://findbugs.sourceforge.ne... OWASP Dependency Check: https://www.owasp.org/index.ph...

Defensive in depth

[somedude69]

You don't use only one tool. Look at id software for example (when carmack worked there), they used three (3!) different static code analysis tools on their code, besides the compiler itself. That's quality, and that's something which attracts customers which is looking for quality.

Re:Defensive in depth

[somedude69]

Although, I would be happy if more than maybe 10% of all programmers cared about compiler errors and warnings at all. Static code analysis tools are the next step:
1. Make the code compile (what!!!, I'm not allowed to commit uncompilable code to my team mates???).
2. Make the code compile, without warnings (when it does, enable 'treat warnings as errors').
3. Make the code compile, without any analysis warnings.
4. Make all unit tests pass.
5. Make all system tests pass (in the test/staging environment).

Yes, using CI is of course something any developer *always* use year 2014. Even when coding your own 'hello world'. Else your doing it wrong(tm).

FYI: FxCop

[MobyDisk]

For example, Microsoft used to have something called FxCop, but it hasn't been updated for current versions of the .NET framework

FxCop is still under active development and ships with Visual Studio 2010, 2012, and 2013. They merely changed the name to "Code Analysis"
http://blogs.msdn.com/b/visual...

Fortify SCA

[dougTheRug]

in my day job I work for Fortify. You can contact the developers of this library and request that they use static analysis product on their software, or request a security review from a 3rd party. We would for sure catch those SQL injections and more. But we would need the original source code. You can probe for things from the binary, but the results don't come back in a way that is very actionable for the developers. As for your predicament: I think you would be better off writing your own library, rather than putting the insecure one to work.

Source code can come with proprietary libs ...

[perpenso]

Easy: use open source libraries.

Yes, having the source code to a library is very important. However open source is not the only way to go. Source code can come with proprietary libraries. Some commercial vendors of proprietary libraries offer binary-only licenses and more expensive source code licenses. I've had more than one reluctant manager come to realize that the extra expense of the source license was absolutely worth it. We fixed some bugs that affected our project, gave the fixes back to the vendor and found our fixes incorporated into their source code.

Re:Source code can come with proprietary libs ...

[retchdog]

What a great feeling it must be to pay extra for the privilege of fixing their bugs for them. Did they at least give you a discount? (Rhetorical question; I know they didn't.)

Re:Source code can come with proprietary libs ...

[Carcass666]

In this particular case, the library is a component of a deployed system (put into place before I got here) for inventory management. The library is the "documented" way to be able for our website to be able to query the system and to be able to perform operations on the inventory (take some out of stock, put some back in, etc.). I could deal with the database directly, but then I don't have any guarantee that I am implementing the same business logic as the library. This particular vendor is very touchy about their proprietary code (and, after seeing a bit how it is built, I can certainly see why).

So, I guess the point I'm trying to make is sometimes, you're screwed, and you have to deal with the mess somebody else made. At least, that's where I'm at right now...

Re:Source code can come with proprietary libs ...

[perpenso]

What a great feeling it must be to pay extra for the privilege of fixing their bugs for them. Did they at least give you a discount? (Rhetorical question; I know they didn't.)

We paid for the privilege of not being dependent upon then, of controlling our fate. Our barely on time project only lost two days. It was a win-win.

Sloppy type conversions

[wiredlogic]

Don't expect .NET decompilers to faithfully reproduce statements as they were in the original code. What you get is functionally equivalent to the original source but it will have been munged by two tools along the way. It isn't necessarily indicative of bad coding practices. Higher level deficiencies like the SQL processing cited will still be obvious and the tools won't impact poor design decisions.

Re:Sloppy type conversions

[Anonymous Coward]

This should be modded higher. Judging the code quality based on the disassembled code is almost impossible.

The compiler can make optimizations and reorder code based on what it knows is possible / impossible with the code paths. This can end up looking like very poorly written code even if what was originally written was very well structured and conforming to best practices.

Even String concatenation for SQL calls is not necessarily horrible depending on the types and / or validation performed. Assembling a sql query to fetch a record by numeric id using string concatenation will not generally lead to a security vulnerability (assuming the numeric id being concatenated is an int object). Not best practices by any means, but also not the end of the world.

Re:Sloppy type conversions

[coolsnowmen]

HAHAHA, if that is how you really feel, how did you even get this far on /. ?

Oh, praytell, what kind of mainstream porn would you find acceptable for me to wank to.

That's an easy one

[Minwee]

In situations like this I usually recommend arson.

Re:Don't do it

[wiredlogic]

Reverse engineering is legal in the US. If you only gain knowledge of internal workings and don't reproduce copyrighted or patented code then there are no damages a vendor can pin on you even if it violates a restrictive license. The worse they can do is revoke the license and force your management to select a better vendor.

I don't have a favorite

[JMZero]

And haven't found any that are terribly impressive in their abilities. They'll catch certain kinds of problems, but tend to lose their way pretty fast in more complicated code. Anyway, this list might help start you out in the right direction:

http://en.wikipedia.org/wiki/L...

Many Eyes

[Jaime2]

Good security comes from a lot of people's testing and input. If you look investigate a product, you will only be able to categorize it into two categories: "utterly craptastic" and "probably utterly craptastic". The only way to be assured of good quality is to use libraries that a lot of people use and have had success with. Don't bother looking at the binary, look at the reputation.

Re:Many Eyes

[FormOfActionBanana]

That's utterly crap advice. Since a lot of softwares in popular, active use have critical vulnerabilities.

The example quoted just above (http://ask.slashdot.org/comments.pl?sid=4862577&cid=46414687) in which nobody got the sarcasm... says:

You know there won't be any bugs in those, or if there are they'll be very quickly fixed and not sit there unnoticed for years.

He was referring to https://www.gitorious.org/gnut... and https://www.imperialviolet.org..., not to mention http://bsd.slashdot.org/story/... which also sat unnoticed for years.

ILSpy

[Dwedit]

This is not a security analyzer, but ILSpy is the .NET decompiler that I currently use. Red Gate .NET Reflector went commercial only, so this is what's left.

Re:ILSpy

[hermitdev]

Another free option for .Net is jetBeans dotPeak. It's worked fairly well for me.

Do not use FXcop as any promise of quality.

[TiggertheMad]

One thing that the OP said that I found to be kind of disconcerting: FXcop was a pretty crappy tool. it could spot some odd code patterns in syntax, but it cannot detect 'good' code. I could implement a bubble sort function that FXcop would give a giant gold star to. Weird syntax might be worth looking at to see if there is underlying problems, but that is about it.

not a bot

[globaljustin]

i'm an idiot...not a bot...just an idiot

gotta be an all-time "did not read past the headline" situation for me...TFA is about...um software libraries

gah...sorry

Reflector is the way to go

[Xoc-S]

FXCop (now incorporated as Code Analysis) is not a security tool. It looks for bad coding practices not malicious software. This might catch some stuff in the process, but it is not the main purpose.

On the other hand, Red Gate's Reflector decompiles the code into C#, VB.NET, F#, IL, or MC++. You can then look for malicious code. I mainly look for code accessing classes in the System.IO namespace, System.Web, System.Net, or similar namespaces, because these are the ones that are likely to either mess with existing files or connect to the Internet.

You can use the ILDASM (Intermediate Language Dis-assembler) program that comes with the .NET Framework, but it only decompiles into intermediate language (IL). This is enough to find the calls, but most people are not adept at reading IL.

Reflector is worth every penny. Besides looking for security problems, I use it all the time to figure out what the Framework is really doing, fix bugs in other people's libraries, sign code that wasn't signed originally, translate VB.NET code to C#, etc. (To translate code, compile it in one language and decompile it with Reflector into the other.)

Re:Reflector is the way to go

[Kalriath]

Or if you don't want to pay for Reflector, you can use ILSpy (a free and open source .NET Decompiler), dotPeek (free, from the markers of Resharper), or JustDecompile (free, from Telerik).

rm - just rm

[SpaceLifeForm]

Don't waste your time, just delete them. The probability that they don't have security problems is nearly nil.

One user helping another also happen in non-FOSS

[perpenso]

The point being that you paid extra to get the source and *then* not give you anything back for improving their product. I'm hoping they weren't that ungrateful.

We paid for a library that was useful and saved us time. We paid extra to not be dependent. We contributed back our fixes to help other users of the library and to simplify things when we got an update. In those updates, some bugs were fixed by the developer of the library, others were fixed by other customers.

Did you think that one user can only help another user in the FOSS world? One user can help another as long as they have access to source, and such access does not require FOSS.

.Nyet !

[BlazingATrail]

.NET, closed source library.. (munching popcorn).. you poor thing. what did you expect was going to happen ?

Re:.Nyet !

[whitedsepdivine]

You do know that isn't true. Microsoft has open sourced a lot of their primary libraries.

Re:.Nyet !

[BlazingATrail]

Are linux versions available? Can I get Windows for free? sorry but Micro$oft PR doesn't work here

And?

[Tablizer]

Recently, we were diagnosing a .NET assembly and, after getting nowhere with the vendor, ran it through a decompiler. The code was a morass of SQL concatenation, sloppy type conversions, and various things that are generally thought of as insecure.

Okay, but what were the unexpected things you found?

code analysis

[chentiangemalc]

For .NET libraries I use .NET reflector., and can integrate into real time debugging with Visual Studio. FxCop is no longer needed as its built into Visual Studio, run Code Analysis option. For analyzing Java Libraries I like free tool jd2gui. For native libraries on windows I like IDA Pro/OllyDbg/WinDbg/AppVerifier and the Windows App Certification kit in Windows 8.1 SDK.

Hp Fortify

[whitedsepdivine]

http://www8.hp.com/in/en/softw... This product scans yours and third party libraries for security problems. It doesn't scan for standards or performance. For performance you can use red gate ants, but there isn't anything for standards.

Guidelines not Software Tools

[cyberhooligan77]

You mention "Tools", did you mean "Software Tools". And, you have to use a decompiler.

I think, "Reviewing", in this case, means more like guidelines & I.T. (rules) policies.

Things, you may want to consider:

* Does a library provides source code ? (No Source Code, preferably, avoided) If you have to pay a extra, for the source code, then choose no library, or pay for the source code, but, dont buy propertary libraries without source code or support.

* Does a library is for a particular programming language or programming enviroment, or several ?

If it's for several programming languages, make sure can be compiled, loaded, or integrated to other languages.

* Does a library provides documentation, both, as files, and web ?

I have deal with several libraries whom lack one or another. Don't trust a library that doesn't have files independent of web. There are many great software libraries, where it only has web documentation. When a winter storm arrives,say "good bye" Internet, and say "good bye" to documentation. The same goes if a hacker or virus attack the documentation server, or a technical malfunction at the network or server.

Web documentation is good, because sometimes its updated regularly. But, don't count to have available all time. Some developers just take the comments of the source code & generate both, we pages, & local files. And, sometimes, this is helpful.

* Does a third party library provides support ?

The answer is similar to the previous answer. Open & Free Software can be great, but, sometimes, lacks this, because, many developers cannot provide full time support. (Unless paid by companies or groups like Apache or Google ).

* Does a third party library provides an A.P.I., or its structured. ?

A good software can be difficult to integrate with other software if its not structured, by classes, modules, or functions.

Just my [spartan] 2 silver cents [coins].