Report: Aging Java Components To Blame For Massively Buggy Open-Source Software

itwbennett writes: The problem isn't new, but a report released Tuesday by Sonatype, the company that manages one of the largest repositories of open-source Java components, sheds some light on poor inventory practices that are all-too-common in software development. To wit: 'Sonatype has determined that over 6 percent of the download requests from the Central Repository in 2014 were for component versions that included known vulnerabilities and the company's review of over 1,500 applications showed that by the time they were developed and released each of them had an average of 24 severe or critical flaws inherited from their components.'

Comments

The root cause : poor unit testing

[Dr_Barnowl]

Why?

Because if you don't test your code, you don't know if changes to it break it.

Changing the components your code is composed of is a big change.

Therefore : people get nervous about changing the components they have used (even changing the version).

What should be happening : when you're planning a new release, raise the component versions to the latest and run your test suite. If it passes, good job, release it.

What is actually happening : the version numbers never get edited, because that version worked, and if you change it, OMG, it might stop working.

Re:The root cause : poor unit testing

[david.emery]

Testing is no cure for bad design or bad coding, which are the -root cause-. The specific design and code techniques to prevent vulnerabilities need to be better communicated and enforced (by open source code reviewers, as well as commercial developers).

That's not to argue testing is unimportant. But it's not the root cause of vulnerabilities, and it's not clear to me that we know how to test for a lot of vulnerabilities.

Re:The root cause : poor unit testing

[Dr_Barnowl]

Oh, yes, testing doesn't fix bad design. But it helps to avoid the problem mentioned - which is that projects use versions of components with known problems that are known - and thus have been fixed in newer versions.

Re:The root cause : poor unit testing

[Anonymous Coward]

What is actually happening : the version numbers never get edited, because that version worked, and if you change it, OMG, it might stop working.

Components provided by Apache and Google (especially guava), have a habit of tend to break backward compatibility.

Apache components tend to get much more complicated to use, Axis-Axis2. And Guava components are deprecated in one release then removed in the next ;so if you have a component that's been changes/removed and you miss a version you have to go back and hunt for the release notes.

Re:The root cause : poor unit testing

[david.emery]

ONLY if there are tests to catch the problems that exist in the earlier version!

Either (a) there was a test contemporaneous with the faulty component that wasn't run; or (b) a subsequent fault was discovered, a test for that fault is developed, and that test is subsequently associated with that component (version).

Re:The root cause : poor unit testing

[oh_my_080980980]

He's not talking about vulnerabilities in components. He's talking about testing the application that uses the component to see if if the application functions properly. The test on the application is to see if the latest version of the component will break the application. That was his point - people tend not to use the latest version of the component for fear it will break the application that uses the component. You are talking about security testing the component. That's a different test.

Re:The root cause : poor unit testing

[DarkOx]

Root cause or not tests are what let you 'fix' the vulnerabilities, re-factor to correct design issues, etc.

I have to agree with the parent. Having good test coverage is the difference

between: We are going to be exposed for weeks while I'll 'try' to understand all the impacts of this change and hope QA spotts any potentially disastrous bugs before we go to production.

and:
Cool fix is in, tests are passing. Lets yet QA run the build for a day or so and we can get this out the door before it hits Slashdot.

 

Re:The root cause : poor unit testing

[Anonymous Coward]

Passing tests is not proof for correctness. Even if your tests pass, you can't be sure that you didn't break something. Even with 100% code coverage

Re:The root cause : poor unit testing

[DickBreath]

Well tested code is best. So the few unit tests you have should be run many times to ensure code is well tested.

Re:The root cause : poor unit testing

[DickBreath]

The root cause: poor management (in most cases)

The root cause is not poor unit testing. Not bad developers. It is managers who won't allow the change to be made. It ultimately will always come down to money. They are unwilling to spend on having a reasonable staging environment that closely mimics the production system such that making these changes could be done safely and receive proper testing. And people to do that work also cost money.

In short: management doesn't care, due to money. So the product can just self-destruct. (like SourceForge)

Re:The root cause : poor unit testing

[Yaztromo]

What should be happening : when you're planning a new release, raise the component versions to the latest and run your test suite. If it passes, good job, release it.

What is actually happening : the version numbers never get edited, because that version worked, and if you change it, OMG, it might stop working.

Part of the problem I run into with this is that sometimes projects stick with old dependencies because at some point, some major version came along that significantly changed the organization of the API in such a way that the latest component version an't just be dropped in, but requires significant resources refactoring your code to use it. Getting management buy-in for that when there aren't any big customers breathing down their neck to get a flaw fixed can be neigh on impossible.

I ran into this recently myself. During internal testing, I discovered a flaw in our product when accessing any of our web resources using an IPv6 destination IP in the URL (i.e.: http://18080./ A quick bit of debugging showed that an external library we had been using for several years was doing some brain-dead parsing of the URL to pull out the port number; it was just doing a string split after the first colon it found, and presumed the rest was the port number.

Modifying the Maven POM to use a newer version of the API in question was initially difficult because the project had since reorganized their own library structure, breaking things into multiple smaller JARs. Except that some of the functionality was actually _removed_, and isn't available at the latest API revision (functionality we had been using, naturally). Classes had moved around to different packages than where they were previously, and various interfaces appear to have been completely rewritten.

Upgrading to a version of the library that actually fixed the flaw was going to be akin to opening Pandora's Box. Unfortunately, our former architect (from whom I inherited this code) was the type of guy who just liked to throw external libraries at every problem. In the end we had to document the fault for all current versions of the product, and now I'm trying to get management buy-in to do the work necessary to upgrade the library in question for the next version of our product. And this is for just one library out of over 100 that need similar attention.

Suffice to say, I'm not happy about this state of affairs. Unlike the previous architect, I push against using third-party libraries as our solution to everything. If I were allowed to rewrite everything from scratch, we could avoid these problems. Things are unfortunately messy out here in the real world, and when libraries decide to significantly change their interfaces your program uses to access their functionality, no amount of unit tests is going to make upgrading those libraries any easier.

Yaz

Re:The root cause : poor unit testing

[lgw]

These are flawed Java components, not complete systems. What kind of component-level testing is generally useful for avoiding security issues? Most of the issues I've seen have either been from each component assuming the other was checking for something, or from anti-patterns like depending on "string cleaning" to avoid injection attacks - implementation choices that are bad practice, but have no flaw you can point to at the time the component is written.

Plus in general insecure code tends to be the result of implementation choices, where the component would be fine with any normal unit test of its interface, but is vulnerable to very specifically crafted malformed input.

For more complete systems, fuzz-testing and a variety of common injection attacks are great, but that makes little sense to me at the component level.

Re:The root cause : poor unit testing

[Anonymous Coward]

I can't second this enough. I've run into this problem countless times managing java packages on Fedora. It's made worse by the open source java community seeming to be allergic to backwards compatibility and using version numbers willy-nilly. I have package 1.2.3, and 1.2.4 was released. Should just be a bug fix and drop in replacement right? Nooooo...Not in java. The projects will break the api in between any release. The numbers have no meaning other than version 1.2.3 != 1.2.4. Other than that, all bets are off.

Re:The root cause : poor unit testing

[Paradise Pete]

if you don't test your code, you don't know if changes to it break it.

And if you do test your code, you still don't know. But you'll catch some of them.

Re:The root cause : poor unit testing

[Anonymous Coward]

For a lot of systems that are N-tier and have complex data pass through (I used to work in cell network AAA and policy enforcement software as one example) that sometimes isn't even clear what the data is (if you are tunneling data through your system to another).... sometimes those data entities and the way they are packaged can change systemic behaviours.

Now, you could argue that this means there are design and implementation issues. Most of these N-tier systems have a bunch of implementers that differ by layer and sometimes within a layer (sometimes not). They may have different technologies at different layers. And many have interconnectivity concerns that no engineer would endorse but were critical to meeting a customer wish.

The end result is there IS no regression test or unit test that can hope to generate all the cases that might, under some strange external control, bork your component in use and possibly larger parts of the system.

All you tend to be able to say is 'for the main use cases, for the known inputs we can model, nothing goes kablam!'. You know that anytime you change a library version, it's possible something that might be a bug for nobody else using that library might be a bug for you. The odds are worse in lesser used or esoteric libraries that these sorts of large N-tier frameworks tend to need for some function they provide.

Even if an API doesn't change, even if the thing works with simple test data, somewhere along the line, somebody is going to inject some data that the testers have never even seen. The older component version just quietly chugged along, but that new, safer and more secure version the developers upgraded to and found no flaws with in limited coverage tests will blow up.

Teams are averse to this and so are corporations and customers. A security issue is a potential threat. A library borking the system is a threat. One is a 'might happen sometime' and the other is 'surprisingly common'.

Guess which the biz types who fund these projects choose? Almost always stick with the tried and not yet proved to be a dagger in the heart.....

Re:The root cause : poor unit testing

[Anonymous Coward]

Yes and sometimes you hit the situation where a component continues evolving but it is referenced by another component you need that does something special but stopped being maintained and there is no replacement. So you can't upgrade component A because component B might break and there is no upgrade for component B.

In the glorious world of imagined designs, there are none of these sorts of interdependencies. In the real world, they happen far too often.

And sometimes, when you expect a minor change, you get some major re-org that they don't think justifies a major version number change. That's hell of a different sort.

The entire layered architecture for software pretty much guarantees this will happen.

Re:The root cause : poor unit testing

[Anonymous Coward]

IME, testing is absolutely a cure for bad design. Why? Design is all about encapsulation and minimizing internal and external dependencies so that over the long haul the code is easier to hack on. Testing is easiest when, guess what, your components are well encapsulated with minimal dependencies. Testing is also easier when each component is a complete functional unit. That is, your APIs are functional, without leaky abstractions or a bunch of knobs and buttons.

In other words, targeting ease of testing means that you're simultaneously improving the design. Robust testing and robust design are exceptionally difficult problems. But if you target both simultaneously, and aim for where they overlap, they're a little bit easier to do well.

I'm not an advocate of any particular methodology or toolkit. Usually I just write my components so that each one can compile to a command-line utility. The beauty of that is that while I'm designing and crafting an API, I'm simultaneously _using_ that API to implement the command-line utility. That means I get feedback not only on whether the code works as expected, but also whether my functional API is sufficient, too simple, too complex, etc. I also code mostly in C, and my "IDE" is the shell, thus command-line utilities fit perfectly into my build. My regression "framework" is just a shell script that builds and runs the various command-line utilities with various test inputs.

I write server software, and specifically heavily multiplexed, non-blocking server software. I was writing NGINX before NGINX existed; before libevent existed. Targeting simple command-line utilities, in addition to the operational environment, means that I have to think long and hard about design. I can't add a feature for production and call it day. I have to add it in such a way that I can wrap it up in a small routine called from a small utility, lacking most (or all) of the rest of the framework. Keeping that discipline is hard but definitely pays off. The fact that the operational environment and the testing environment are drastically different is, I think, the _key_ for me. If I tried to implement a fancy unit testing framework to make it easier for me to write unit tests, that would kind of defeat the purpose.

And, no, it's not like every single line of my code is testable. I'm a practical person, and trying to test it all would probably make me want to implement a fancy framework. I don't care about unit test coverage stats. I don't even keep them. I just like how the discipline of unit testing keeps me disciplined and my code lean and well designed.

As a bonus, because all of my code is exceptionally well encapsulated, with minimal cross dependencies, I can reuse source code from different projects. Rather than maintain a toolbox of technical routines (e.g. strlcpy, etc), my toolbox is made up of dozens of individual _functional_ components that I can mix-and-match. That means that, yes, each component may have it's own copy of strlcpy or a strerror_r wrapper. But I've learned long ago that code reuse is about functional reuse. By duplicating the inconsequential, highly technical stuff that will rarely, if ever, change, I make it easier to reuse the code that matters and which will be hacked on the most.

Re:The root cause : poor unit testing

[RabidReindeer]

This is somewhat deceptive. Sonatype supports Maven component archives.

One of Maven's chief claims to fame is that when you build a project, it doesn't grab "the latest" versions of dependencies, it grabs the selected versions of dependencies. On the grounds that "If it ain't broke, don't fix it".

This ensures a predictable product because everyone who does a build, no matter when, no matter where, will be pulling in the same resources to build with.

The problem arises when one (or more) of those selected component versions turns out to have issues. The build ensures that the product will be consistent, and thus will pass its own tests, but as the old observation goes, testing cannot prove the absence of bugs, only their presence. So if there was a vulnerability, an old project's tests wouldn't see it. And because you're asking for a specific library release version, later fixes don't get automatically included (of course, neither do later breakages, but they ignored that aspect).

In theory, then, this is simple to fix. Just update the project (POM) to pull in newer, better dependencies.

And the NEXT version of Windows will fix all your problems, and I've got a very nice bridge in NYC for sale cheap.

If you're working on a project, you generally have all you can do to keep up with issues in your own code, let alone some supposedly trustworthy third-party libraries. You cannot afford to be constantly updating the dependency versions and even if you could, there's the issue of "dependency Hell", where changing the version of Hibernate can conflict with the version of slf4j which can conflict with junit, which can conflict with... I usually like to budget 2 or 3 DAYS when I'm ready to start upgrading dependencies.

Sonatype doesn't get a pass here, though. If they/Maven supported a mechanism that could flag builds that have known weak dependencies, it would help a lot. Management, of course, would promptly command it to be turned off to ensure "productivity", but at least we'd have some help short of periodically manually auditing every library in a complex project (like that's ever going to happen).

Re:The root cause : poor unit testing

[ConceptJunkie]

Testing won't fix bad design, but it might give you a handle on how bad the problem is.

Re:The root cause : poor unit testing

[UnknownSoldier]

Hit the nail on the head with the problems of external libs.

> Unlike the previous architect, I push against using third-party libraries as our solution to everything.

You're not alone, my boss and I are of the same mind set, and I've noticed the same pattern amongst coders:

* The more better programmers minimize the amount of external libs.

* The inexperienced / junior programmers are so gung-ho to include every library under the sun that it is almost dizzying. And then they complain why their project takes 5 minutes to start up or so slow to navigate into their IDE (because code completion is usually to blame.)

If a problem _requires_ a library then, sure, we'll use that, but we would rather write almost everything ourselves so we fully understand the context of the problem.

Re:The root cause : poor unit testing

[Kjella]

No, the problem is that you got no knowledge or control over the quality of work done on the component. There are lots of ways to subtly break it through failing on a particular set of inputs or a particular sequence of events that is combinatoric in nature and essentially impossible to predict. For example imagine you have functions a() through z() and it'll crash if you call c() after g() unless you've called v() in between. How can you unit test for that? You can't. What if the new component has a threading bug that'll only show up one in a hundred runs under heavy load?

Of course, if they have proper unit tests it shouldn't start regressing and breaking in surprising ways. That's a pretty big if. And it really only needs to happen once before you and your manager get chewed out for breaking the system just to keep up with the bleeding edge. It's pretty hard to argue against code that has withstood the test of time, as long as none of the flaws and limitations seem to matter to you. I know that happens to some code I run into, it's ugly as sin but it actually works and I just pray I don't ever have to debug it.

Re:The root cause : poor unit testing

[Darinbob]

People don't like to test third party products, much less looking inside them. The whole point is that it's like going to the store and buying a box of something from the shelf. To do any more than that is to admit that the adage of "never reinvent the wheel" is inappropriate to software.

Partially it's a junior programmer mentality too. They do not understand that third party components frequently break; they've never encountered hardware that has bugs, compilers that have bugs, libraries that have bugs, etc. I suspect many of them when presented with a test case that fails on a third party component will first suspect that the test must have been written incorrectly.

On the other hand it can become extremely difficult to update versions of third party components. Ie, we've made several in-house bug fixes to the library (*all* of them have bugs!), now the next release has decided to layout all the code differently, rename variables, rewrite comments in Linear B, etc. So the job then is to integrate the two together which can be a very lengthy task.

Re:The root cause : poor unit testing

[SwashbucklingCowboy]

Sorry, but no, it's not that simple. Lots of vulnerabilities come into a project because of dependencies that are poorly managed. Project A depends upon project B which in turn depends upon project C and C has the vuln. All the unit testing of A in the world will not turn up that vuln. That requires system testing and that's a lot more involved.

Re:The root cause : poor unit testing

[Anonymous Coward]

* The more better programmers minimize the amount of external libs.

Yes. Good programmers know when to use an external library. Great programmers know when not to use an external library.

Re:The root cause : poor unit testing

[Bite The Pillow]

Wait, what?

If you write code, part of the documentation before you start should be a "risks" statement, where you state that a dependency on X external, third party library, exists, and that any vulnerability could cause issues in your application. Also, that substantial upgrades to the library interface will affect maintainability if any interfaces are changed, or are deprecated.

When someone throws a pile of libraries at a problem, that risk statement gets lengthy.

Rewriting from scratch is not the best solution when you use a robust, mature library like zlib. They had vulnerabilities, fixed them, and upgrading was no chore.

Management loves risk statements, because they hate risk and want to know where they are exposed. Written correctly, the risk will analyze the maturity of the library and the guarantee that it makes.

Kernel calls in Linux are not guaranteed to break ABI, but some things in Linux do have a level of guarantee. And as much as I hate to say it, COM/ATL interfaces where you can choose which version of the interface to use, but it can be patched behind the scenes - which turned into the idea of a "service contract" that can't be broken.

Your problem isn't with using external libraries, it's using ones without service contracts, or immature ones. And you're reacting by throwing out the baby, bathwater, bathtub, house, plumbing infrastructure, and electrical grid.

You can put a stop to that shit by explaining what a contract is (management should be familiar) and what happens when it breaks. And things diverge, since the answer isn't "lawsuit them to death." Now you research the maturity level of each library, starting with the ones you hate, and make a case for removing some of them.

Re:The root cause : poor unit testing

[Yaztromo]

Wait, what?

If you write code, part of the documentation before you start should be a "risks" statement, where you state that a dependency on X external, third party library, exists, and that any vulnerability could cause issues in your application. Also, that substantial upgrades to the library interface will affect maintainability if any interfaces are changed, or are deprecated.

When someone throws a pile of libraries at a problem, that risk statement gets lengthy.

Which is all well and good if you're doing greenfield development. It's not so good when you're inherited a codebase where none of this was done in the first place, and you're tasked to keep it going. As I said, the real world can be a messy place. In my case, the previous lead architect just threw immature libraries at every problem willy-nilly, at a time before I worked for the company. I get to inherit the problems this lack of foresight caused, and don't have the benefit of going backwards to fix it.

Your problem isn't with using external libraries, it's using ones without service contracts, or immature ones. And you're reacting by throwing out the baby, bathwater, bathtub, house, plumbing infrastructure, and electrical grid.

I don't disagree -- I'm hardly anti-library (you won't get too far in development without them, unless you're doing low-level embedded work). I like a good, stable, well-designed library that has been around for a long time, and where breaking changes are rare. Unfortunately, when the previous architect would throw a library at every problem that may have required eight lines of code, without care for anything other than ensuring the library license permits us to ship it.

And FWIW, I have brought these issues up with management. Their stance is they don't want to see any backend changes of any sort (where the bulk of our library woes lie) -- their focus is on the front end only. They're more than happy to defer the risks to some unspecified future.

As I said, the real world of development can be messy. I'm stuck with a codebase that is full of immature libraries, libraries that are no longer maintained, and libraries with bugs where getting new libraries introduces major changes requiring major refactors. If I were permitted to start fresh, I'd be doing things the way you described, and we wouldn't rely so much on "randomLibrary.jar" that was someones pet project seven years ago with 'LGPL' attached to it, but which has since been rewritten or abandoned. It's a willy-nilly application of such libraries to every problem that crops up without any analysis that I'm against, and not the use of libraries in any and all situations.

Yaz

Re:The root cause : poor unit testing

[strikethree]

I wonder why there are so many articles busting a programmer's balls (only guys are coders apparently(wtf?!)) for not participating in code reuse.

Hell, if I had to go through all of those hoops during maintenance phase, which is most of the time spent on the project, I would reinvent the wheel every damned time.

But, whatever.

Re:The root cause : poor unit testing

[Anonymous Coward]

> Sonatype doesn't get a pass here, though. If they/Maven supported a mechanism that could flag builds that have known weak
> dependencies, it would help a lot. Management, of course, would promptly command it to be turned off to ensure "productivity",
> but at least we'd have some help short of periodically manually auditing every library in a complex project (like that's ever going to happen).

Some managers would. But it would plant the responsibility/blame where it should be ... so when a project is in review, (keep/enhance, scrap, replace), there would be clear visibility that the project chose to disable security awareness. I contract at banks, and for example there's a lot of appetite for internal secure interfaces - no plain sockets or HTTP - and at my current place, you get a black audit mark if you're non-compliant which affects your access to funding and hardware. The underlying message is: if you are cooking problems for the future, we'll give your funding to someone who isn't.

This could be similar. In fact, I'm rather thinking that there's a fair amount of meta-data regarding upgrade releases, from user review ("1 star, API had totally changed") to developer notes ("you really want this upgrade") and bugs fixed ("major security hole fixed") that could be supplied automatically allowing you to rate your projects. You could even get a Sonar-style summary of the estimated costs to stay current. My place would pay money for that - it's a big black hole at the moment.

Re:The root cause : poor unit testing

[goose-incarnated]

What should be happening : when you're planning a new release, raise the component versions to the latest and run your test suite. If it passes, good job, release it.

What is actually happening : the version numbers never get edited, because that version worked, and if you change it, OMG, it might stop working.

Part of the problem I run into with this is that sometimes projects stick with old dependencies because at some point, some major version came along that significantly changed the organization of the API in such a way that the latest component version an't just be dropped in, but requires significant resources refactoring your code to use it. Getting management buy-in for that when there aren't any big customers breathing down their neck to get a flaw fixed can be neigh on impossible.

I ran into this recently myself. During internal testing, I discovered a flaw in our product when accessing any of our web resources using an IPv6 destination IP in the URL (i.e.: http://18080./ A quick bit of debugging showed that an external library we had been using for several years was doing some brain-dead parsing of the URL to pull out the port number; it was just doing a string split after the first colon it found, and presumed the rest was the port number.

Modifying the Maven POM to use a newer version of the API in question was initially difficult because the project had since reorganized their own library structure, breaking things into multiple smaller JARs. Except that some of the functionality was actually _removed_, and isn't available at the latest API revision (functionality we had been using, naturally). Classes had moved around to different packages than where they were previously, and various interfaces appear to have been completely rewritten.

Upgrading to a version of the library that actually fixed the flaw was going to be akin to opening Pandora's Box. Unfortunately, our former architect (from whom I inherited this code) was the type of guy who just liked to throw external libraries at every problem. In the end we had to document the fault for all current versions of the product, and now I'm trying to get management buy-in to do the work necessary to upgrade the library in question for the next version of our product.

It might not be worth it - this scenario will no doubt play out again. You'll move to version X, and make thousands of code changes to make the move. In about five years your successor will find some bug in version X that is fixed in version X+5. Unfortunately, architectural changes would be once again required, much like now. It's a vicious cycle.

Your best bet is to continue using the current library and simply not use it's URL parsing functionality - write your own small function to do just that one thing. Every change to code raises the probability of fresh bugs appearing, regardless of unit-tests.

Re:The root cause : poor unit testing

[hattig]

My previous employer was of this mindset. Even with in-house dependencies. Nothing was other updated, out of fear.

A horrible environment to work in, of course. Every few months a component upgrade would be required, and because it was 100 releases out of date the upgrade was a horrible horrible experience.

And they had generally reasonable test suites too. It was pure fear of downtime because of the monolithic architecture of the application.

Re:The root cause : poor unit testing

[hattig]

Just maintain a visible Risk document with all the issues. Document the estimated fix time. Let it have visibility.

Then when the shit hits the fan, you have your arse covered. Not that this will protect you against particularly nasty management scum...

Using Apache libraries, or ${largeCompany} libraries is one thing. But random crap found on Github?

All you can do is overestimate work, and use the time to kill off the libraries one by one.

Re:The root cause : poor unit testing

[hattig]

Ah, the good old Junit / Hamcrest / Mockito Maven Pom upgrade joy!

Re:The root cause : poor unit testing

[UnknownSoldier]

That's a perfect summary and great quote!

Re:The root cause : poor unit testing

[Anonymous Coward]

> Sonatype doesn't get a pass here, though. If they/Maven supported a mechanism that could flag builds that have known weak dependencies

That is the point of the article.

Sonatype do have this system built into their Nexus Professional repository manager.

Is this unique to Java?

[gstoddart]

I'm betting if you have a large enough pool of open source things, which depend on other open source things, then the bugs in the dependencies will trickle up to the projects which rely on them.

Though, admittedly, Java has also made this more annoying -- a decade or so ago when I was actively working on a Java project, it always amazed me how a new version of Java could completely break everything and then you'd have to re-test and re-certify everything.

It got to the point we put in very large bold characters in our release notes ... we work on this version of Java, if you get clever and introduce your own version of Java, we won't talk to you until you confirm the bug in the version we support.

A surprising number of clients were willing to blaze trail with whatever version of Java came along, and then kept expecting we'd be supporting custom versions from vendors or features which didn't exist when our version was built.

Eventually we learned to dread a new release of Java. Because invariably things went to hell and stopped working.

Re:Is this unique to Java?

[Anonymous Coward]

Eventually we learned to dread a new release of Java. Because invariably things went to hell and stopped working.

Ditto for Python.

Re:Is this unique to Java?

[Anonymous Coward]

The last major release that "broke" things for me was the 1.4.2 -> 5 transition in 2004. Since then (5 -> 6, 6 -> 7, and 7 -> 8) have been relatively painless, If you were relying on an undocumented feature, or compiling against com.sun.* or sun.* classes you did so at your own risk. If you stuck to the documented JDK, you were usually ok.

Re:Is this unique to Java?

[Dr_Barnowl]

I've only ever seen instances where new versions of Java broke things by removing deprecated components like JINI, but I generally tend to stick to OpenJDK for everything - as the "official" Java (the benchmark for certification) maybe it has less "clever" in it than the others.

Clients may be keen to move onto newer versions of Java because of the immense litany of security defects that get listed by Oracle when they release a new version, and because of their apparent enthusiasm for end-of-lining support (alas, Java 7, we knew you well, but no more public updates after April 2015).

Re:Is this unique to Java?

[TheCarp]

And others. I had a hell of a time back when my job involved compiling all third party software we used that wasn't part of the OS distribution. Several really fun PHP updates. My particular favorite was when a minor point release actually moved entire modules out of the core and into a seperate download..... talk about incompatable versions!

Re:Is this unique to Java?

[Daniel Hoffmann]

Yeah this is a common problem in pretty much all platforms, what makes Java stand out is too many Java things are actually specifications, not implementations. It kinda mixes all the headaches of conventional development (dll hell, outdated libraries, testing against multiple hardware/OSs) with the headaches that developing for browsers have (multiple implementations of the same specifications). One of the things that make people like the Spring is that, unlike J2EE, Spring is an implementation, not a specification so it usually works is you change your application server for example. Well some parts of Spring relies on Servlets, which is a specification, but Servlets implementations are ok (although Java6 does not support Servlets 3.0 which is a pain in the ass if your client is on Java6 and refuses to update).

Re:Is this unique to Java?

[TheCarp]

> It got to the point we put in very large bold characters in our release notes ... we work on this version of Java, if
> you get clever and introduce your own version of Java, we won't talk to you until you confirm the bug in the
> version we support.

It gets really fun when open source folks do this, I actually had this conversation recently:

"Have you tried the latest version? That module has been updated since the version you are using"
"No, but I am looking at your code on github now, line X would be where there should be logic to handle this case, but it isn't there."

I then did verify it with the latest version, but it failed as expected. :)

Re:Is this unique to Java?

[houstonbofh]

It got to the point we put in very large bold characters in our release notes ... we work on this version of Java, if you get clever and introduce your own version of Java, we won't talk to you until you confirm the bug in the version we support.

Which is how we ended up with the management nightmare of different hardware requiring different and incompatible versions of JAVA for the "Web Client" to manage it. So, one workstation to manage Cisco. One workstation to manage EMC. One for HP. One for the phone system and a different one for the voicemail... And hope to God no one clicks "Update" on the popup before reading it!

Re:Is this unique to Java?

[drinkypoo]

It got to the point we put in very large bold characters in our release notes ... we work on this version of Java, if you get clever and introduce your own version of Java, we won't talk to you until you confirm the bug in the version we support.

It's really not just you. Every enterprise-class piece of Java software I've ever installed came with its own copy of a specific version of the JVM redistributable, and required that you install it.

Re:Is this unique to Java?

[gstoddart]

I haven't directly touched Java in years ... but one of the things which struck me was that it just seemed too damned brittle.

What should have been core APIs for published interfaces would suddenly change in the number of parameters between versions, or not be there at all, or return something new.

They'd "fix" something by simply deprecating it/removing it.

It felt very much like a young language which was constantly shifting under your feet, constantly calling for a do-over, and often breaking backwards compatibility.

Flash forward a decade, and I see the same kinds of things you mention -- software which ships with exactly one version it supports, and woe to anybody who ever updates it or has security people who say "you can't run that version".

It feels kind of half-assed and not very maintainable if you need to spend that much effort not breaking it. All the while the platform itself constantly nags you to update, but if you do that you're probably screwed.

And then Oracle started putting the Ask.com bullshit in, and made it essentially a platform you can't really trust because it's constantly trying to subvert your machine.

Re:Is this unique to Java?

[DickBreath]

> (1) Is this unique to Java?
> (2) I'm betting if you have a large enough pool of open source things, which depend on other open source things . . .

You answer your own question. It may be unique to Java because Java has an absolute embarrassment of open source riches. Some of them have been around for a long time. Bad management leads to big projects not getting upgraded. Not just libraries, but even the Java runtimes that they run on. Just look at how many developers on reddit complain about big systems still running on decrept ancient dinosaur versions of Java, way before even Generics.

Re:Is this unique to Java?

[Gr8Apes]

The litany of security defects are largely edge cases in portions of the libraries most don't use or browser based (ie, applets) which don't concern 99% of java devs (I wish I could say 100%, but somewhere, some idiot is still writing applets) The core has been relatively stable since J5.

Re:Is this unique to Java?

[psyclone]

I agree that 4 -> 5 was difficult, 5 -> 6 and 6 -> 7 was easy, but 7 -> 8 is difficult again. Mostly due to app server containers like Tomcat and JBoss -- specifically the JSP compiling part needs a lot of love for Java 8 in servlet containers.

Re:Is this unique to Java?

[Anonymous Coward]

"Eventually we learned to dread a new release of Java. Because invariably things went to hell and stopped working."

Any it's the same for Python, C++, Perl... What's interesting is that's why markup languages were created.

Re:Is this unique to Java?

[Anonymous Coward]

" this is a common problem in pretty much all platforms "

And here I am using Windows 7 running software written for Windows 3.1 and Windows 95 (among more recent software of course).

Backwards compatibility can be done, it just takes time and resources. And if experience with Windows has taught us developers anything, it's that most of that time is spent dealing with bugs in the application software, e.g. where it relied on undocumented behaviour, implementation details, and so on.

So when I read this:

" Eventually we learned to dread a new release of Java. Because invariably things went to hell and stopped working. "

I'm pretty sure that the bug-infested code is actually the application code itself. I am not necessarily blaming the developers, it's probably just management that decided there was no budget for proper (unit) testing and code review, nor enough budget to actually write the code to begin with, really. But the point stands.

Re:Is this unique to Java?

[KGIII]

I have done some work in Java. I am proud to say that I am not in the 1%! I have never made a Java applet. I can not think of any reason to do so. It *might* have been viable in the past as there may have been little choice otherwise (Flash? What was that one from Microsoft, ActiveX?) but I can not think of any reason to do so now and I am having a hard time making a case for having done so in the past.

Re:Is this unique to Java?

[hattig]

It was viable in 1998! That was when I last wrote an applet (a tetris game to run on a 40MHz ARM-based Set Top Box from Acorn that never saw the light of day).

No, no, wait, I did write a rotating 3D globe applet in 2006 for a laugh.

Applets were a bad idea, are a bad idea, and should be dead. Luckily browser security policies are making maintaining in-house applets non-viable so they can move to better installation mechanisms now.

Re:Is this unique to Java?

[Gr8Apes]

I wrote a couple as a test and quickly abandoned those, as well as some desktop apps painful as they were, way back somewhere between 99 through the early 2000s. The 1% (hopefully much much less) is in reference to today's devs. No dev today should be writing an applet. In fact, I'd be perfectly happy if Oracle removed the applet code and browser integration "capability" (used very loosely) completely.

Probably not only Java components affected

[QuietLagoon]

It wouldn't surprise me if similar audits found the same level of vulns in component libraries for other development environments.

.
Java developers are no different than other developers.

Re:Probably not only Java components affected

[Anonymous Coward]

No they are.

They just get maven to download millions of random fucking jars fr them.

Re:Probably not only Java components affected

[Anonymous Coward]

No way would I have more than one MS C library installed, or .NET version. That would be ludicrous.

Monkey See, Monkey Do

[Anonymous Coward]

That's the real reason there are problems: Most software isn't so much written as blindly copied; most programmers have no idea what they're doing, and this is particularly true of the Big IDE users at which Java has always been targeted.

What about JavaScript code?

[Anonymous Coward]

It's accepted in industry that JavaScript is an inferior language compared to Java. Popular only for its ubiquity in web browsers, JavaScript lacks most of the language features needed for writing robust, reliable software at a large scale.

Java does offer far more of the features needed for large-scale software development, while also attaining a reasonable level of quality. Java has a working type system, while JavaScript does not. Java supports real modularity, while JavaScript only supports hacks. Java supports classes, while JavaScript only supports crippled and near-useless prototypes. The list goes on and on.

Additionally, Java is typically used by trained professionals who have at least some formal education under their belt. JavaScript, on the other hand, is a language embraced by beginners who have no experience.

So if we're having problems with software developed using Java, and Java is better suited to software development than JavaScript is, then this means that the problems we're having with JavaScript are astronomically bigger than what we're experiencing with Java.

While we need to address these concerns involving Java, obviously, we can't neglect to focus on the major problems that JavaScript is causing, as well.

Re:What about JavaScript code?

[Anonymous Coward]

WTF. JavaScript and Java have basically nothing to do with each other except for the name, so why are you comparing them?

Re:What about JavaScript code?

[Anonymous Coward]

It's accepted in industry that MOO is an inferior language compared to Java. MOO lacks most of the language features needed for writing robust, reliable software at a large scale.

Java does offer far more of the features needed for large-scale software development, while also attaining a reasonable level of quality. Java has a working type system, while MOO does not. Java supports real modularity, while MOO only supports hacks. Java supports classes, while MOO only supports crippled and near-useless prototypes. The list goes on and on.

Additionally, Java is typically used by trained professionals who have at least some formal education under their belt. MOO , on the other hand, is a language embraced by beginners who have no experience.

So if we're having problems with software developed using Java, and Java is better suited to software development than MOO is, then this means that the problems we're having with MOO are astronomically bigger than what we're experiencing with Java.

While we need to address these concerns involving Java, obviously, we can't neglect to focus on the major problems that MOO is causing, as well.

Re:What about JavaScript code?

[Dr_Barnowl]

He's saying that Java, because of it's nature, and the type of programmer that uses it, could reasonably be expected to be more rigorously programmed than Javascript, so if there are horrific problems with Java, then the problems with Javascript are like the Elder Gods descending upon the web.

Possibly not so bad at the client side, but all this Node.js stuff that's popular....

Re:What about JavaScript code?

[oh_my_080980980]

Which still has nothing to do with the original topic.

Re:What about JavaScript code?

[Anonymous Coward]

That's the whole point: the original topic and its "problem" is totally irrelevant compared to the many real problems out there.

The problems affecting Java are equivalent to a bad haircut, while the problems affecting JavaScript are like a simultaneous combination of brain cancer, gangrene, explosive diarrhea, liver failure, a torn aorta, herpes, a split scrotum and autism.

Intelligent people don't focus on the minor problem until the major problems have been dealt with sufficiently.

We can talk about Java later. First we need to deal with JavaScript.

Re:What about JavaScript code?

[DickBreath]

> JavaScript lacks most of the language features needed for writing robust, reliable software at a large scale.

There is no reason that it should not be possible to write huge enterprise applications in JavaScript. (or assembly language).

It's just that it becomes a huge unmaintainable mess.

The reason for higher and higher level languages, including type checking, is to make the compiler to more and more of the bookkeeping of writing software. You could do it in assembly language if you were willing to do enough bookkeeping.

What JavaScript (and similar dynamic languages) bring is that you can write smaller projects very quickly. See Ruby on Rails for example. But then Twitter started with Ruby, and in 2012 switched to Java for scalability. They said that Ruby was the right language when they started.

Re:What about JavaScript code?

[UnknownSoldier]

He's saying that if a better language (Java) has all these problems with API version control then how does a worse language (Javascript) have any hope of doing a better job avoiding .lib hell

The problem isn't unique to Java -- this tends to happen in C the least while Java the most from what I've noticed. When you rely on ANY third party library you are eventually going to run into this problem where the API changes. Now multiply that by each external library you are using. Some days we spend more "busy-work" just updating the codebase to work with the external libs then doing any actual real work. :-(

There is no easy solution to when the architecture changes. :-/

Re:What about JavaScript code?

[Anonymous Coward]

Javascript (or really, Javascript used within Node.js) is the language of choice for writing web apps because that's what it was designed to do, and it fulfills that promise.

Java wasn't designed to do anything well, and it fulfills that promise

As far as your claim that Java programmers can be presumed better educated - nonsense. Most Java programmers can't do anything more than wire up libraries of libraries

Re:What about JavaScript code?

[KGIII]

You expect people to remain on-topic? You have been a member longer than I have. (I may have read longer, that is irrelevant.) You should know better than that.

Re:What about JavaScript code?

[Anonymous Coward]

> Intelligent people don't focus on the minor problem until the major problems have been dealt with sufficiently.

That doesn't match real-world observation.

And in any case is wrong. Focus on the major problems or minor problems you can solve.

Re:What about JavaScript code?

[KGIII]

Are you a werecow? Moo moo moo... I'm a werecow.

The Doctor is in!

Re:What about JavaScript code?

[Rakarra]

Intelligent people don't focus on the minor problem until the major problems have been dealt with sufficiently.

"Sorry, I'm not going to triage your arterial bleeding. Why should I? Nothing matters compared to a cure for cancer."

FUD

[Anonymous Coward]

This is nothing more than fearmongering FUD. Many eyes garuntees such bugs are impossible.

Go back to your caves Microsoft shills!

Re:FUD

[DickBreath]

It's not FUD. It's clickbait. It will make up for the revenue shortfall from SourceForge.

Re:FUD

[Anonymous Coward]

It's not FUD. It's clickbait. It will make up for the revenue shortfall from SourceForge.

Sonatype has no association with SourceForge, this comment makes no sense.

Hello Xerces

[Anonymous Coward]

They mean you. Get off the web!

Just block them

[sunderland56]

One basic problem seems to be that repositories are providing downloads of known vulnerable components.

Once a bit of software has a known vulnerability, it should *immediately* be deleted from all repositories. Responsible developers will post a fix in a timely manner; hacks will wait weeks/months/years to update, Eventually people will move away from the badly written bits of software - because they aren't available. Problem solved.

Re:Just block them

[houstonbofh]

Delete stuff from the Internet... Hmmm... Sounds like a wonderful idea. How?

Actually it is a terrible idea, even if it could work, because looking at how the code progressed is how you learn. Not to mention that I can patch and old version to fix the vulnerability, but not have to move to the new and incompatible version.

Re:Just block them

[Anonymous Coward]

You should apply for a governmental position with North Korea. Your draconian enforcement ideas would be right at home.

Re:Just block them

[ERJ]

You don't know how the software is being used. Maybe it is Apache's commons-net which has a vulnerability in the FTP client while my software only uses the SMB client. Maybe the next revision up has API changes that break compatibility.
In the ideal world everything would be kept up to date but time is a finite resource and if there is not a compelling reason to update it seems silly to waste time on it.

Re:Just block them

[Anonymous Coward]

You don't know how the software is being used. Maybe it is Apache's commons-net which has a vulnerability in the FTP client while my software only uses the SMB client. Maybe the next revision up has API changes that break compatibility.

+1.

The OP is advocating for the modern equivalent of book burning.

Re:Just block them

[Anonymous Coward]

I wouldn't necessarily say book burning, but it's not difficult to make an outdated package hard to obtain, specifically remove or move the old versions of your software, the point is to break URLs to tell people it's not the latest version. You can still make the old version available (by maintaining a branch for it in git).

With that said, I'm not so sure it's useful, some distributions do keep up to date, an maintain scripts that import the patches themselves, this could break those scripts, and would probably cause more trouble for end users doing the building (think Gentoo users).

This is not surprising

[ErichTheRed]

This basically defines some of the problems of "enterprisey" software:
- It's composed of a million glued-together libraries.
- It's written by chronically understaffed/overworked IT department employees.
- Rigorous testing either (a) doesn't exist, (b) is so onerous that most developers try to avoid it, or (c) is outsourced/offshored to the lowest bidder, and therefore isn't completed without the staff basically doing the tests for the outsourcer.
- Anything that breaks it is avoided at all costs because of all of the above.

By extension, this is why some companies are stuck running IE 6 for key applications, or Office 97 because rewriting the scary mess of macros that runs a process isn't something anyone wants to do. I do systems integration work, and new versions of Java, web browsers, etc. are miserable. They introduce bugs small enough to be annoyances (rendering problems, etc.) and big enough to break the entire system.

The key to fixing this is for the software architects to require that developers move up to at least a semi-modern release of their key libraries, test everything against them, and remove the old outdated ones once all the bugs are fixed. The problem is that this is never done.

Re:This is not surprising

[JustNiz]

>> The problem is that this is never done.

The reason is that many Software Director positions are now filled with technically clueless people that are basically salesmen rather than engineers.
They have no comprehension of the concept of technical debt, or the need to spend time on activities that don't directly translate into new features.
The net result is that you're always just piling more crap onto the top of a steaming turd pile so making it worse, instead of working to replace the shit.

Re:This is not surprising

[Anonymous Coward]

Not only understaffed but underSKILLED. When you hire monkeys, you get bananas.

Re:This is not surprising

[tyme]

>> The problem is that this is never done.

> The reason is that many Software Director positions are now filled with technically clueless people

Now?

Re:This is not surprising

[ahabswhale]

Actually, this is a problem with ALLsoftware. Most programmers are not experts on security or how to write software that is secure. Libraries just exacerbate the problem because even if the code is 100% unit tested, it doesn't mean it's safe and a lot of these libraries are huge. There's simply no way to know how secure they are. That said, the companies I have worked for run scanners against apps to test them for vulnerabilities. Just because a library has a vulnerability, it doesn't mean that vulnerability is exposed in such a way that it can be exploited externally.

Quit using Java then!

[Anonymous Coward]

I'm sorry, but please stop using Java if you need third-party libraries, because those third party libraries are rarely updated precisely because of this. If you fix a bug in a third party library and drop it in, there goes all the security signatures and crap.

If you need third party libraries. Develop in C.
If you need third party libraries to never change, statically compile them (except GPL licensed software, don't use LGPL/GPL software if you intend to never open source your software.)

Java software should rely on only the JRE, because otherwise it's inherently non-portable (eg 32bit and 64bit runtimes, much less Operating systems.)

The most irritating part of developing on Linux is all the "dll hell", where as on Windows, the DLL hell is fixed by just installing hunderds of versions of the otherwise same library and never relying on the system library.

And this is the problem with all software development. You can not rely on the system libraries to continue to work for any period of time. But if you ship your own libraries, you can't ever replace them.

If only there was a way to actually have software dynamically link to the "most recent version that will work."
eg have the runtime interrogate the system library and other installed libraries for the functions and return types to make sure that the library hasn't changed (or has been replaced with "cracked/compromised" versions.)

On Windows (see many "plugin" systems) software can be version independent as long as the API doesn't change or deprecate anything (see OpenGL) without breaking forward compatibility.

Accept the fact

[cloud.pt]

It's about time everyone stops whining. There are things in life you're better safe than sorry, but then there are things in life you just can't change: not every single entity can keep maintaining what they create. Human beings are limited, and so are human organizations - they lack money, workforce or simply the patience to put up with some "critical flaws" that are just too rooted in bad design to be solved without a restructuring.

THAT IS THE REAL FLAW.

There are good ways and bad ways to create reusable components. Black boxing (containing) everything for starters (sans the closed-sourced part) is something people tend to limit the scope to testing and/or to services outside a fully-fledged system's component border. Technologies like SOA are just one of many ways to plug&play every new piece of technology that performs a very specific task in a different way of a previously flawed one. Think project Ara. It's not only fun to develop like this (although some have problem conceptualizing it), but it's also more robust in the long run. Using such paradigms is what we, as the "clients" of such "aging and flawed" components can do push better development of individual components.

Now, each and every component developer has to find ways to keep their work atomic, so as to not conflict with the principles of technologies they are developed to work for. This might all seem like an utopian way of what to expect of the coding community, but then again we are also still looking for the best ways to apply near-perfect political views designed hundreds of years ago, which are yet to achieve full potential. I keep my hopes up for both issues, but my expectations low.

Re:Accept the fact

[Bite The Pillow]

What's the flaw? That an organization is not earning enough cash flow or pathos to fix critical flaws in their product?

Or that people choose reusable components poorly?

Or that humans are humans? I suppose that kind of is a flaw, unless you assume from the start that that's kind of true, but if you don't then isn't the flaw on the ignorant?

Or that you are mixing some sort of political statement with asking people to stop whining? Because no matter your politics, someone is either going to whine or feel so unsafe doing so that, while unhappy, they don't.

Why don't you collect your thoughts and try again?

Re:Accept the fact

[Anonymous Coward]

Why don't you stop deflecting?

It's a good point - organisations are not perfect and it is wilfully stupid to expect a random collection of software providers to keep everything up to date at all times and to a level of quality for running business-critical production functions.

Re:Accept the fact

[cloud.pt]

Neither of your options, so I guess you missed the point and decided to look cool with a very subtle...

Why don't you collect your thoughts and try again?

The "REAL FLAW" is the conjuncture. The lack of determination flaw is intrinsic to society and nobody can change it. It applies to both humans and their collective forms. I subliminally mention a lot other flaws. One of them is most dev in the community has this general idea that everything should be future-proof, even stuff they don't code, yet stoically and lightheartedly include in their code as if it was custom-made to their spec (-.-). You want custom featuresets? Do custom components. That's what determined companies (those that attempt not to have the first flaw) do, and they grow. They go the extra mile even if it costs them a leg, because they have the goal of having robust, future-proof software.

Things like free code, freely distributed (closed) components or even freely available supporting cloud services (i.e. everything you might use in your code, developed by someone else that is not related to your your business model, and that their business isn't improved directly by the fact that you use that component [keyword directly], or they don't have a business model to begin with, so there really is no long-term incentive [end of big parenthesis]) are NOT to be used unless you accept their flaws. Some are open so you can explore their flaws. Some have a proper license that allows fixing the flaws if it better suits you, this way becoming a bit more like the aforementioned "determined" entities that go the extra mile.

But that is the current state of the union about component usage. The article mentions a more "meta" problem. It is obvious the 2 flaws I directly mentioned in this comment, the "society lacking determination" one and the "dev community has great expectations" one, are pretty much unsolvable. So what I proposed in my first comment was a good middle-ground:

1. - "user-type" devs (the ones who include a component in their work) should change their work modularity to have plug&play components when needed (with varying degrees of plug-ability, but the core notion is they should do it more often and more profoundly) , and...
2. - "component-supplier-type" devs should adjust their work to be even more modular, closer to a perfect black-box, requiring no external changes (even underlying system ones such as e.g. an upgrade in Linux kernel to make use of new binary and/or feature FOO that is less prone to critical flaw X).

These aren't 'bugs'

[Anonymous Coward]

They are 'features', installed by you-know-who, NSA, CIA, DEA, FBI....

Dependency Vulnerability != Product Vulnerability

[dark.nebulae]

It is important to note the following:

Sonatype has determined that over 6 percent of the download requests from the Central Repository in 2014 were for component versions that included known vulnerabilities.

That means that when building a project the devs are using an older version of a dependency than a newer, fixed version. You don't pull your own artifacts from Sonatype, just dependencies.

Yes this can mean there's a bug that might be exposed to end users, but frequently a dependency is just a dependency used by the developer's code. Sure there could be a transient vulnerability, but I don't think vulnerabilities will be that transparent but that depends upon the nature of the vulnerability and the use of the component in the dev's code.

This is more of a "DLL Hell" situation. Vulnerabilities can be solved in one component, but due to interdependencies it may or may not be possible to use the fixed version. If you're choosing to use an older version of a dependency, well then that's a bad choice. But sometimes those older versions you can just be stuck with due to interdependencies, business direction (only version X has been approved by corp even if Y has critical fixes you may still be stuck with X), etc.

Why?

[Anonymous Coward]

I've been told in tens of thousands of Slashdot comments(all modded up) that Open Source software is more secure. No wonder this site is dying.

Re:Why?

[houstonbofh]

I've been told in tens of thousands of Slashdot comments(all modded up) that Open Source software is more secure. No wonder this site is dying.

It is not more secure due to magic. It is more secure due to patches and updates. If you do not apply them, you do not get that security. It is like having locks and a car alarm, but leaving your keys on the dash.

Re:Java is the Devil

[oh_my_080980980]

Because .NET and C# are better....

Nah....PHB

[Zecheus]

It doesn't matter what language is used. Developers don't upgrade because a PHB doesn't like the risk-benefit ratio.

No mention of Sonatype's business?

[Captain Damnit]

It should be noted that the company releasing this report, Sonatype, markets a product called Insight Application Health Check that scans your binaries for libraries with known vulnerabilities.

I have never used their service, and can offer no comments on its utility or value. However, it is a bit unseemly that TFA doesn't mention that the source of their information about this very real problem also sells a service that solves it. This is a knock on IT World, not Sonatype.

Re:No mention of Sonatype's business?

[Anonymous Coward]

Sonatype, markets a product called Insight Application Health Check

Sonatype also makes a product called Nexus which manages dependency repositories using maven. That pretty much puts them in a key position to actually know WTF they are talking about.

Re:No mention of Sonatype's business?

[Anonymous Coward]

You mean what problems they caused.

.NET to blame for buggy desktop software.

[Anonymous Coward]

.NET to blame for buggy desktop software. Shocker.

K.I.S.S. - Ugly Lasts

[Tablizer]

If you want software to last more than one season, then rely on old-school "plain jane" HTML for most of your UI with a little JavaScript only where absolutely needed.

Plane-jain HTML is not glamorous or fancy, but it has been supported and will be supported for a good while. If your org wants fancy, it has to pay the Fancy Tax.

If you have to repaint the entire screen for an activity, so be it. If it's a relatively small-user-base app, the overall bandwidth overhead is not really bigger than downloading fat JS libraries. If you code your HTML to mostly rely on CSS, then HTML redraws will be small packets.

Yeah, because..

[Anonymous Coward]

java is the only language in which code rot occurs..

What sort of shit ass submission is this?

Re:Yeah, because..

[KGIII]

Who insinuated this?

Oh, nobody. It is easy to rant about things that are only in your head. Unfortunately, for the rest of us, your rant made it past the submit button which means we are subjected to your mindless inanities. Seriously, how did you manage to reach that conclusion?

Over simplified

[Anonymous Coward]

FOSS is much more prone to borking backwards compatibility than COTS. The best run FOSS will deprecate in release X+1 then delete in X+2, but at the end of the day you're still forced to change your application code so that the FOSS maintenance team can keep themselves pure and no amount of unit or functional testing is going to prevent that.

Plenty of application development teams make rational decisions not to install FOSS security fixes when the FOSS team has chosen to bundle those fixes with incompatibilities.

Re:Over simplified

[Anonymous Coward]

Plenty of application development teams make rational decisions not to install FOSS security fixes when the FOSS team has chosen to bundle those fixes with incompatibilities.

For critical applications (the kind on the "don't use our product for X without special license" list) I've gone as far as forking the operating system and even forking router software so I can apply security fixes without taking the functional changes they were bundled with. I've got very valid reasons for downloading older software versions.

Re:Over simplified

[chipschap]

FOSS is much more prone to borking backwards compatibility than COTS.

Do you have any backup for this statement?

The only possible thing I can think of is that COTS can potentially have greater control over a closed ecosystem. But even COTS today often relies on various third-party libraries.

Java ecosystem has not leadership

[Anonymous Coward]

That I am aware of. I know, Oracle is not doing much to help this problem. In the non-Java, non-PHP side ecosystems, some companies pipe money into open source projects and lead their direction so developers deal with less problems like this. They do happen, just not nearly as often.

Re:Java is the Devil

[Anonymous Coward]

At least it doesn't try to shove the Ask toolbar down your throat.

Re:Java is the Devil

[Anonymous Coward]

Because .NET and C# are better....

Having stepped into the Java world after having done both C++ and C# for many years, there are quite a number of things about both that are better than Java.

Blame Maven

[_xeno_]

This is a problem that Maven has created, mostly.

What the summary doesn't mention is that "large repository of open source software" is a Maven repository. Maven allows you to specify dependencies for your Java project.

The problem is that you have to specify a specific version of whatever you use. So let's say you use OpenFoo 1.1 and that at the time you write your code, the latest version of OpenFoo is 1.1.3.

Now assume a horrible vulnerability is discovered in OpenFoo 1.1.3, so they release OpenFoo 1.1.4 to fix it. Well, your Maven POM says you require OpenFoo 1.1.3, so until you go in and manually change that, you will only ever use 1.1.3. There is - by design - no way to say "I want the latest 1.1 version." You can only describe a single, specific version.

So it's no surprise that Sonatype will see a ton of old Maven projects continuing to download outdated Maven artifacts. There's no way to say "I want the latest version of a specific branch" you can only specify a single version. Which means that a project that hasn't changed in years will still pull in the old versions of the libraries, even if it would work with the later versions.

Re:Blame Maven

[Anonymous Coward]

If only there always was an OpenFoo 1.1.4. Often the main creator decides to rewrite the thing, break backward compatibility, change the API entirely, call it OpenFoo 2.0 and forget that the ugly, cludgy, badly designed OpenFoo 1 ever existed. Then it's up to you to fix that bug. At least it's open source.
This "move fast, break things" isn't good for enterprises. Enterprises want dependability and solid minimal effort upgrade paths, and as little radical re-writing as possible. It's hard to get customers to pay for things that "work" well enough in most cases. So, on we go, with buggy old crap crap, forever.

Re:Blame Maven

[Anonymous Coward]

Oops, I meant that it's hard to get customers to pay to change things that already "work" well enough, I mean.

Re:Blame Maven

[Anonymous Coward]

There's no way to say "I want the latest version of a specific branch" you can only specify a single version. Which means that a project that hasn't changed in years will still pull in the old versions of the libraries, even if it would work with the later versions.

No, Maven does support version ranges: You can say stuff like this: <version>[1.0.0,2.0.0)</version>. Here's a pretty good thread on the subject from StackOverflow.

To be fair, I don't think that most projects do this, but at least it's supported. Also, I'd guess that an analysis of a lot of the other artifact repositories like PyPi, Bower, or npm would produce similar results.

Re:Blame Maven

[Mars Saxman]

That sounds like the most useful way of doing things. If I haven't built and tested my codebase against a specific library version, how can I assert that my codebase works properly with that specific library version?

The counterargument holds that this should never happen as long as people use semantic versioning properly, but that's no more realistic than expecting people to release bug-free libraries that never need to be upgraded.

Re:Blame Maven

[davidleelambert]

And the comments on that answer say that the "LATEST/RELEASE features" are deprecated or no longer supported, although the links they give to back that statement up are broken.

Re:Blame Maven

[Anonymous Coward]

You can only describe a single, specific version.

That is by design so you can get repeatable builds. Think about the problems it would cause if your application changed depending on when you built it. Yes, it would be convenient, but as someone that has built large Java apps with >= version dependencies, it is a nightmare.

The real problem with upgrading a Java dependency are the transitive dependencies. I can upgrade a single library and test it well, but many Maven dependencies have dozen, if not hundreds, of transitive dependencies. The last time I tried to upgrade a library, we had more than 250 libraries that were also upgraded. When you do the math, it’s easy to understand why it happens. If your one dependency depends on five things then those five things depend on five things then those five things depend on five things, you’re already up to 125 dependencies you have to test. There’s no silver bullet to solve that problem.

Re:Blame Maven

[Anonymous Coward]

depends on five things then those five things depend on five things then those five things depend on five things...125

Your math is wrong. That's actually 3,125 new versions. IIRC the most new .jars I've seen from a single version upgrade is just over 1,200. That wrecked our development for days. CI servers started timing out due to the time it took to download all of those .jar files, and our developers hammered our Internet connection for more than a week. We've since added a web proxy, but even with that the first build will take hours after a simple pom.xml change. Our remote devs in Ukraine and South America have it even worse. The last time I upgraded our Spring version, it took all night to download the dependencies while in a hotel. Modern libraries are just too interconnected now. Also, open source libraries just update too often.

As an example, the last version upgrade we did was to Spring and Apache Commons. It took us six weeks of work to stamp-out all of the bugs. Our unit tests and integration tests caught nearly all of them, so we're doing everything we can right, but the transitive dependency problem is just intractable.

Poor Testing is NOT the Problem

[HannethCom]

I am not going to argue that having better automated test is not a good thing. It is a good thing.

The problem is more about how these third party components are maintained. The majority of third party components I have worked with, upgrading to a newer version of the component meant rewriting large sections of code just to get the project to compile. The interface to the component changed. The tests would cover the happy paths and some bad paths, but a lot of manual testing and mitigation of new bugs would need to still be done. This doesn't even cover if the newer version of the component will even work for what you used it for. Yes, sometimes newer versions of components remove features, or change a feature you relied on in such a way that it is no longer usable.

You say don't use components from third parties that keep making these breaking changes? Guess what, that means using no third party components because basically everyone makes these same mistakes. The problem is usually worse with commercial components. Sometimes the wheel is so complex that you just can't re-invent it, but you never know when the third party is going change their axle size, or wheel diameter.

Re:Two Words

[Anonymous Coward]

'OMG screwdrivers suck! I tried to hammer in this nail with one and it didn't work!'. Christ you're an idiot. I don't come down to where you work and slap the whatever it is you use out of your mouth do I? You know why? Because every job has a right tool, and the minute you think all jobs are alike or any tool is a panacea you expose yourself as a shit engineer.

And what job did you use Java for to make this highly educated declaration? Let me guess - you think 'Java' means the web plugin that nobody has used in 10 years and you made a shitty applet?

There are jobs (real ones, that matter) where Java is absolutely clutch. If you ever saw the sheer magnitude of what is required to build enterprise systems for coordinating the scheduling / generation / distribution / remuneration for a 10 state region of the US power grid you would shit your little boy pants. Now imagine you need to deliver the common core of that same system to 5 more multi-state regions. Now imagine you need to sell it in Europe, Asia and the Middle East, each time adapting it to new operating protocols, languages, ergonomics. Ever done that? No? Then shut the fuck up.

Funny thing? We have EXTREMELY meticulous security audits, static code analysis, penetration testing, etc. You know where all the SEVERE issues are found? Surprise - in the C applications. Why? No not because C 'sucks', but because there is literally no way to write bad Java code that causes a buffer overrun and allows arbitrary malicious code execution. If ever there is, this is a JVM flaw not a code flaw and is promptly fixed. Once. For everybody.

I can tell you what these 'vulnerabilities' in TFA are - I see them all the time in static code analysis. They are things like 'log forging', potential (usually spurious) null references, SQL injections, etc. These problems exist in any language. The fact that they can be picked up by SCA and put in a report like this is actually a good thing (oh and 24 of them ain't shit if you are including all severity levels). And they are easily addressed by staying on top of your dependency stack.

How about this: You and me both get started on an application to store, monitor and analyize 25hz data from every Phasor device in an entire, very populous country of Asia. I'll use my wealth of common components and frameworks, you go ahead and roll it yourself out of whatever the fuck you use. We'll meet back in a year, and let's see which one is more complete, bug-free and secure. Deal?

Re:Java is the Devil

[Anonymous Coward]

.NET, C# and Microsoft in general are usually very good about backwards compatibility. Some notable exceptions are VB.NET and Vista.

Java Versioning is Crap

[Anonymous Coward]

Seriously, this is one of the big problems with the Java ecosystem. Java releases that encourage applications to version check (because of feature instability and repeated backwards compatibility problems). Then applications that do that, and thus often get stuck in the past.

Everyone involved in this needs to get a slap. You do not chain an app to a library/environment/JVM unless absolutely necessary. And preemptive environment checking "just in case" of compatibility problems deserves a double slap. Yeah, I know there are times when this stuff seems necessary. Whenever that happens, maybe give your dependency design a re-think. Unless you enjoy being the problem child of software providers?

Sonatype FUDs Open Source ..

[nickweller]

April 2013: "Sonatype's annual survey of 3,500 software developers and shows struggle in setting corporate policy on open source and enforcing it" ref

April 2013: "Control and security of corporate open source projects proves difficult | New Sonatype survey finds 80 percent of most Java applications comes from open source" ref

Nov 2014: "Software developers use a large number of open-source components, often oblivious to the security risks they introduce or the vulnerabilities that are later discovered in them." ref

April 2015: "open-source also represents a vast, unpatched quagmire of cyber-risk that’s putting public safety at grave risk. That’s the assessment of Joshua Corman, CTO at Sonatype" ref

Re:Sonatype FUDs Open Source ..

[Bite The Pillow]

Are you refuting any of these claims?

#1: Corporate policy on open source is extremely difficult, mostly based on the legal teams who have to approve not being familiar with the idea, terms, language, or really any part of it. "This opens us to risk," they say, and the initiative is killed.

#2:

When asked about how well their organizations control which open-source components are used in software development projects, 24 percent did say, "We're completely locked down: We can only use approved components." However, 44 percent answered, "Yes, we have some corporate standards, but they aren't enforced," and 32 percent said, "There are no standards. Each developer of team chooses the components that are the best for their project."

When asked about whether their company's open-source policy addressed security vulnerabilities, 24 percent answered, "We must prove that we are not using components with known vulnerabilities." But the remainder of the respondents indicated a weaker effort on security, saying they simply had a policy to avoid known vulnerabilities or their policy does not address security vulnerabilities.

#3 and #4, pretty much what this article is about.

Wouldn't you want people to be aware of these potential issues and avoid them? You can't avoid problems that you don't know about. And finally, you have 4 examples. I'd like to know how many articles they published in the interim, and why you didn't include any information on them.

Re:Sonatype FUDs Open Source ..

[KGIII]

Their business model pretty much relies on open source. Why, pray tell, do you think that this is FUD and what would be their motive for doing so? Why would they want to harm their bottom line?

It's Better and Worse Than This...

[SwashbucklingCowboy]

It's better in that just because a component has a vuln doesn't mean that vuln is exploitable in all situations. Unfortunately, people are TERRIBLE at determining if a vulnerability is potentially exploitable or not.

It's worse in that the data in the NVD is often wrong and has lots of missing versions. For example, CVE-2013-5960 says "The ... in the OWASP Enterprise Security API (ESAPI) for Java 2.x before 2.1.1 " and it lists the affected versions only as 2.0.1. The description is wrong (the issue was fixed in 2.1.0) and the list of versions is incomplete as there are more versions that are affected. Another example, CVE-2014-3604 says "Certificates.java in Not Yet Commons SSL before 0.3.15 ..." and then lists the affected versions as 0.3.15 - which is the version it was fixed in and it doesn't list the versions that were actually affected.

Re:Two Words

[KGIII]

How about this: You and me both get started on an application to store, monitor and analyize 25hz data from every Phasor device in an entire, very populous country of Asia. I'll use my wealth of common components and frameworks, you go ahead and roll it yourself out of whatever the fuck you use. We'll meet back in a year, and let's see which one is more complete, bug-free and secure. Deal?

Sounds like a fun idea. I am game. You go with Java (an excellent choice for this) and I will go with C++. Should we begin today? You have experience in the field so you have an advantage. I accept that challenge.

a href="http://kilcadirci.com/" title="kl çad

[Anonymous Coward]

kl çadr

This is very typical

[FithisUX]

and this why you use the latest and greatest and update your interface to updating dependencies. It has nothing to do with FOSS.

Why does Sonatype serve insecure lib versions?

[presidenteloco]

Why not replace the download with an error message stating that you need to upgrade to version whatever.

Since they already know which versions of what have critical security flaws.

Too obvious?

Not as good for the bottom line as selling a library flagging tool?