5,198 Software Flaws Found in 2005

An anonymous reader writes "Security researchers uncovered nearly 5,200 software vulnerabilities in 2005, almost 40 percent more than the number discovered in 2004, according to Washingtonpost.com. From the article: 'According to US-CERT...researchers found 812 flaws in the Windows operating system, 2,328 problems in various versions of the Unix/Linux operating systems (Mac included). An additional 2,058 flaws affected multiple operating systems.'"

Comments

Axe Grinding

[alanw]

Brian Krebs is clearly either extremely stupid, or has an axe to grind. If you look at the Cert Cyber Security Bulletin 2005 Summary, you can see that many of the lines in it end in "(Updated)" A simple count of lines gives the results that Brian quotes, however there are far more "(Updated)" entries in the Unix/ Linux Operating Systems section. Removing these lines gives the following results:
including excluding
"(Updated)" "(Updated)"
Windows 813 671
U/L 2328 891
Multiple 2057 1512

(sorry about the spacing - can't find any way of doing it)

greatly reducing the proportion of Unix/Linux vulnerabilities

Re:Axe Grinding

[ginotech]

That is messed up. You're right, simply updating a vulnerability doesn't make it a new one. You know why Linux and co. have more updated ones, though? Because people can actually see the bugs in the code!

Re:Axe Grinding

[someone300]

Also, isn't this more of a survey of the security flaws of the software running on the operating systems, rather than the operating systems themselves anyway? The summary linked article seems to imply that it's an OS flaw.

7-Zip isn't an OS vulnerability, nor is 4d web star.

Couldn't this be tilted against linux/unix/whatever due to the larger amount of crappy server/networking software available for it?

Re:Axe Grinding

[camcorder]

Also from security perspective I would like to know ratio of remote vulnerabilities on these platforms and how much of them DoS vulnerabilities and more critical compromise vulnerabilites.

It's correct that a DoS vulnerability might be actually more critical as it was thought (as in recent IE bug). I think numbers as such very deceptive. From an user perspective I can say this year brought me lots of stupid worm mails which mostly targeted from Windows platforms.

Re:Axe Grinding

[click2005]

Where is the mention of seriousness of the flaws? How many allow root access or something else serious/critical instead of "clicking this button makes the tool tab disappear" or something.

They also fail to mention that a lot of these flaws are not in the OS itself (or essential components) but in 3rd party software.

A lot of the software isnt even included in a standard installation.

Re:Axe Grinding

[twiddlingbits]

As someone pointed out some of these "flaws" are not OS flaws but issues with application software, and the Severity level are not indicated. So, until the list is sorted accurately it's hard to tell if Win of *NIX was better.

The way I read the results, *NIX list cover the whole set of OSes of this type. There are at least three major versions of UNIX (Solaris, AIX, HP-UX) and multiple releases/versions of each in production. I know that Solaris 8,9 and 10 are all still supported by Sun in 2005 and that is a very big base of installed servers. There are about a dozen LINUX distros, some with serveral releases/versions in production. The Windows numbers cover XP, Win2K, Win2K server, Win2003server. If you count desktops, the Windows installed base is bigger meaning a flaw may affect more users.

However, until someone publishes a more detailed study,with the methodology described, we are ALL just speculating.

Re:Axe Grinding

[jc42]

Hey, you missed the even bigger method of increasing the unix/linux score: counting each distro separately.

Thus, if you go to distrowatch.com, you find 100 distros for linux alone. So for most actual kernel bugs, you can count each one at least 100 times. And for apps that run on all unix releases, the multiplier can be a lot higher.

Of course, there are several distros of Windows, too. But not nearly as many, and the people adding up the bug counts somehow always seem to miss this trick with Windows.

Anyone else got a favorite way of producing misleading bug scores?

Re:Axe Grinding

[pintomp3]

TFA keeps talking about vulnerabilities and flaws interchangabily. a flaw doesn't mean a vulnerability. although i believe updates should be included in the tally, the tally is trivial. few of the unix/linux flaws make your computer vulnerable compared to the windows ones. that is more a design issue though.

Re:Axe Grinding

[cbiltcliffe]

Anyone else got a favorite way of producing misleading bug scores?

Not so much a misleading bug score, but misleading about bugs nonetheless:

"All the bad guys know about all the bugs in Linux, because they can see the code. But only Microsoft knows about bugs in Windows, and they fix them before anybody finds out."

Paraphrased, of course, but pretty much what every Microsoftie analyst says on a daily basis.

The state of security

[Ckwop]

There's two ways to look at this. I would say that it is quite unlikely that the quality of software with respect to security went down in 2005. Computer Security now has such high profile that software houses across the world are spending many dollars trying to provide better security.

If you accept that security quality has not gone down, then you must conclude our ability to detect vulnerabilites is getting better. This is universally a good thing. Every vulnerability the "good guys" find before the "bad guys" is one we can have fix for before the bad guys take over our system.

Then there's the other side of these figures. That's alot of vulnerabilities. Now, fair enough not all vulnerabilities are created equally but I'd bet at least 10% are serious enough to get your system taken over if you're not careful. That's a lot of ways to break in to my system and it's a lot of work to make sure you're not vulnerable.

We have such a long way to go. For example, in PHP if they'd just follow Microsoft's example and put a SQL injection and XSS attack filter on information passed to web-pages we could close a serious hole in many web-applications. I've not looked at Ruby on Rails but I bet it fails this test too.

For gods sake, if you're not writing an operating system you have no business using C. Read me lips: YOU CAN'T WRITE SECURE C. Not now, not after 20 thousand hours of training, not ever. Sure, it's possible to write secure C in theory but the difference between theory and practice is that in theory they're the same and in practice they are not. In practice, you have deadlines, in practice you have people on the team who have less security training than others, in practice you have developers who have just had children and don't get a lot of sleep. In practice, people make mistakes. Code reviews may help but they wont remove everything. If you write your software in C you're doomed to having silly security bugs. If you want to remove most of the worry about overflows, use a language that rules them out.

Another thing, why should code we execute on our computers run at the maxmium privellege set of the user who's running it? Suppose my program checks a HTTP page against an MD5 hash periodically and sends an SMS through an internet based SMS gateway. Why should that program, if it wants to, be allowed to access the disk? I don't know about Java but C# has got a set of attributes that can control this type of behaviour. Really, we should be forcing declarations at the language level about what permissions each method of the program needs - the default being none of course.

Simon.

Re:The state of security

[Decaff]

because C and ASM are still the only choices for coding performance sensitive applications

No. If you look at an area where performance and reliability is critical, you will find that Ada is the dominant language (with Java having increasing use)

Re:The state of security

[canuck57]

For gods sake, if you're not writing an operating system you have no business using C. Read me lips: YOU CAN'T WRITE SECURE C.

I beg to differ, C can be real secure if written that way. The problem comes in that most people do not know how C works inside yet they code something. Then of course to your next point:

Code reviews may help but they wont remove everything.

This would solve alot of issues. How many environments routinely run bounds checking and code reviews for functionality AND security? How many people who really understand C reviewed the code?

And security problems are not just C problems, any language like Java, .NET, PHP, C# can also have their issues. CERT and others concentraight on the operating systems that we all use but generally skirt applications security which can be very bad. Job schedulers written in Java that allow root access, data warehouses that give up encoded (but not encrypted) UIDs/passwords ovr the net, the list is long. And how many people use unencrypted telnet/ftp/imap/pop3 even though secure options exist? I know senior NT and UNIX admins that don't know what a key pair is let alone what a certificate chain is. But they have a half dozen certifications.

But secure code begins with it's priority, in design and takes more time to code no mater what language you use. Having knowledgable coders helps alot. But we are in a day and age where we only want cheap coders. And here is a hint, cheap coders are never good coders or they would not be cheap. There in is the issue, more time is something people do not want to do either in training, coding or review.

Re:The state of security

[Decaff]

I was right with you until you mentioned Java, thanks for the laugh.

http://mae.pennnet.com/Articles/Article_Display.cf m?Section=Articles&ARTICLE_ID=234337&VERSION_NUM=2 &p=32

"Aonix engineers have demonstrated hard-real-time Java that reaches the run-time efficiency of C, which makes it able to meet the needs of command-and-control applications such as network-centric warfare, Future Combat Systems, and low-level telecommunications control-plane software, Aonix officials say."

"The Navy Open Architecture guidelines also state that all new development will be done in Java and C++, he adds. "

Laughing now? Or perhaps feeling a little foolish?

Re:The state of security

[dsanfte]

I've never seen "concentrate" spelled quite like that. +2 points for originality.

Re:The state of security

[Decaff]

And security problems are not just C problems, any language like Java, .NET, PHP, C# can also have their issues.

Apart from the fact that .NET isn't a language, I would be interested to know what issues you think Java and C# have. Almost all the problems with C (and almost all problems with security) are due to bounds checking. Java and C# have automatic built-in bounds checking.

PHP can have issues because it is interpreted, and so you can get code injection. Java and C# aren't interpreted.

Re:The state of security

[lancejjj]

What? C? On Linux? Can you say S-L-O-W!? Just wait until you get some load on that server!

The fact is that C -or- Java running on top of ANY operating system is a recipe for a performance disaster. Folks with the need for speed know this already.

The only way to program for speed and performance is to:

  1. snip off any pins on the CPU that could induce interrupts

  2. write your program

  3. Make sure your program only uses the highest performing registers on the CPU

  4. Make sure your program performs no memory I/O once loaded, and performs no loops (a common error!)

  5. Convert your program into opcodes and poke them into memory byte by byte (Opcode mnemonics are for candy-assed wimps.)

  6. Execute your program!

If your program & its data is too big and bloated to fit onto the CPUs internal registers, then your program will be TOO DAMN SLOW. Then it'll be time to build a custom circuit: keep it small, keep it cold, and keep it massively parallel.

Language choice?

[Anonymous Coward]

I would like to see some data showing the correlation between applications written in unmanaged languages and those with buffer overflow and similar exploits.

Modern unmanaged C++ is fine (STL containers instead of arrays, RAII, etc.), but I often wonder why people still write in C at all, particularly when it comes to Open Source software. We are not the bearded heroes of the 70s - it's time to write in a modern language. If you don't want to sacrifice speed and system level programming for a managed environment, write in modern C++.

Re:Language choice?

[penguin-collective]

Modern unmanaged C++ is fine (STL containers instead of arrays, RAII, etc.),

Modern unmanaged C++ is NOT fine; STL permits many kinds of bugs that are analogous to buffer overflows. Furthermore, modern software systems are composed of many different modules, and just because you happen to be careful in your modules doesn't mean others are careful in theirs. Finally, without full garbage collection, you cannot have full runtime safety.

but I often wonder why people still write in C at all, particularly when it comes to Open Source software.

People prefer C to C++ because for the small increase in safety that C++ gives, it's far too complicated and complex a language. People don't use languages other than C/C++ because those languages interoperate poorly with existing C/C++-based libraries (this is C/C++'s fault), tend to have bloated runtimes, and have only a tiny user community. And, yes, many people don't even realize that there is a problem.

We are not the bearded heroes of the 70s - it's time to write in a modern language.

The bearded heroes of the 70s actually knew better. Back in the 1970's and 1980's, C was of no significance. When people were using HLLs back then, those languages were generally a lot safer than C. The rise of C was a historical accident, related to the rise of BSD UNIX and microcomputers.

But, yes, I share your sentiment: it would be good to see security bugs by language choice. And I'll give you this much: C++ is an improvement over C, but it's not a solution.

Re:Language choice?

[jc42]

I often wonder why people still write in C at all, ...

Well, my last big project was written almost entirely in C for the simple reason that that's what the client wanted. We did a lot of prototyping in perl and python, but that code wasn't acceptable for delivery; we had to rewrite all the production code in C. If not, it wouldn't be accepted.

Much of the explanation was that the client had accepted C++ and java in earlier projects, and they were disasters for all the familiar reasons. They were determined that this wouldn't happen again, so they went with a "proven" language with a track record of use in major successful systems.

Similarly, I have a couple of friends who recently did a project in Cobol. They hated it, but they wanted to get paid, and that's what the client would accept.

In the Real World[TM], the decision about which language to use is very often made by managers who aren't programmers and don't have a clue about the real issues. So they make decisions based on things that they can understand and measure.

Re:Language choice?

[Anonymous Brave Guy]

Modern unmanaged C++ is NOT fine; STL permits many kinds of bugs that are analogous to buffer overflows.

Huh? Granted there are some silly design decisions in the C++ standard library, like making the unchecked indexing use operator[] and the safer, checked version use at() on a std::vector. Still, it's much harder to get things like overruns using the STL, where much code is iterator-based, and harder still to do it in a way that won't be obvious to any remotely competent code reviewer (who will ask why you're not using the safer indexing if your loop does count instead). What sort of thing did you have in mind?

Finally, without full garbage collection, you cannot have full runtime safety.

Garbage collection is an answer to one category of programmer error, and it's far from the most serious. As I often mention in these discussions, I haven't caused a memory leak writing in C++ in years, and I've got automated tools running on the projects concerned just in case to prove it. It's actually quite hard to get a memory leak in C++ if you use basic good programming practices.

On the other hand, in a language with GC you still have problems with messing up thread safety to cause race conditions, messing up thread synchronisation to cause deadlocks, greedy acquisition/lazy release of resources crippling the performance of your process (or, more commonly, other processes running on the same machine but not under the same run-time framework), SQL injection, using naive encryption and/or insecure transport protocols. These are all common flaws seen in real programs, and can be just as damaging as any buffer overflow, if not more so.

And I make this case without, until now, mentioning the IME very real problem that a lot of cheaposoft programmers who grow up relying on GC don't have the same appreciation of low level mechanics as those who don't, which causes them to write unnecessarily naff code with problems of its own.

At the end of the day, GC is a useful tool for many programming jobs, but it's only a tool, not a silver bullet. It's no substitute for a good programmer who knows what he's doing.

Re:Language choice?

[aaronl]

And here is the problem with that:

PERL - not installed on some UNIXes
Python - not installed on most UNIXes
Ruby - not installed on any UNIXes

If your app won't run in the default environment of your target platform, you create a lot more work to change the environment. Or you could write the app in a way so that it *will* run in the default environment, which means using C or shell. Usually, PERL will work, but there are several places that it isn't installed by default, even today.

Re:Language choice?

[YU Nicks NE Way]

In the Real World[TM], the decision about which language to use is very often made by managers who aren't programmers and don't have a clue about the real issues.

No. The decision is made be a manager who needs to balance the business issues of long-term supportability and cost within the current infrastructure against other technical benefits. That decision is not as simple as, say, vi versus Emacs, much less C versus Java versus C# versus Python.

Re:Language choice?

[Decaff]

At the end of the day, GC is a useful tool for many programming jobs, but it's only a tool, not a silver bullet. It's no substitute for a good programmer who knows what he's doing.

You write well on this matter, but I think the evidence really is to the contrary. Hundreds of millions (if not more) lines of code have now been written in languages that use garbage collection. Some of these languages are high-performance and some are used for real-time work, and they all work fine.

Garbage collection is now routinely used even for multi-threaded languages (perhaps the best example is high-performance Java application servers like Tomcat).

So I am afraid I do think it is a silver bullet, and manual memory management will soon look like part of history except for the most specialised uses.

OSOS

[Konster]

812 flaws in the Windows operating system? When did they start counting flaws? December 28th?

Explorer vs Firefox

[dynamo52]

Firefox: 1
Explorer: 45
Explorer wins!

This count must be wrong!

[corvair2k1]

I've released more than that by myself this year!

Re:Hooray \o/

[spike1]

So, where did you read that windows is more secure all of a sudden?

You didn't take those figures at face value did you?
Those figures said they were for linux AND other univx variants like OSX...

So, 2500 between OSX, openBSD, netBSD, freeBSD, Linux, Solaris, etc... (not to mention all the flaws listed for the dfifferent linux distributions probably got duplicated across several distros)

versus 900 for windows
(I'm rounding up)
Was this 900 split between 95/98/98SE/ME/2000/XP/Vista?
or just for XP?

There're lies, damned lies, and statistics

shocking numbers

[CDPatten]

"researchers found 812 flaws in the Windows operating system, 2,328 problems in various versions of the Unix/Linux operating systems (Mac included). "

If we listened to just the media you would have thought Windows has thousands and the others only had a few dozen. I promise I'm not trolling, but do those numbers stop and make anyone on the site re-think stances? We all saw the numbers that put Firefox with more holes then IE earlier this year too. Could MS be doing a better job, but just getting hammered in the press (who are mostly Apple users by the way)? MS holes get lots of press while other operating systems get a free pass.

"I know Microsoft issued a lot of patches this year that fixed quite a few vulnerabilities ... but 812? My suspicion has always been that Microsoft sometimes fixes multiple flaws with a single patch, even though its advisories may make it appear as though the patch addressed a singular issue. "

MS always has an attached KB article that details everything their path does. I don't think that statement is denial.

I find myself defending MS allot on this site, and it's nice to have some numbers from a respected neutral organization to debate some of you guys with. I'm sure after this piece they will be re-classified as MS zealots, but what can ya do.

Re:shocking numbers

[penguin-collective]

If we listened to just the media you would have thought Windows has thousands and the others only had a few dozen. I promise I'm not trolling, but do those numbers stop and make anyone on the site re-think stances?

No, it doesn't. First of all, there are a dozen different versions of UNIX and Linux, each with their own set of flaws. MacOS is an almost entirely different system except for a kernel compatibility module and a bunch of command line utilities. Second, the number of bugs discovered or number of bugs fixed tells you little about the security of an operating system. Individual bugs have very different consequences for system security.

I find myself defending MS allot on this site, and it's nice to have some numbers from a respected neutral organization to debate some of you guys with. I'm sure after this piece they will be re-classified as MS zealots, but what can ya do.

If you are using numbers like these to make an argument that MS products are "more secure", the zealot is you. But, then, you already admitted as much.

Re:shocking numbers

[Liam Slider]

"researchers found 812 flaws in the Windows operating system, 2,328 problems in various versions of the Unix/Linux operating systems (Mac included). "

If we listened to just the media you would have thought Windows has thousands and the others only had a few dozen. I promise I'm not trolling, but do those numbers stop and make anyone on the site re-think stances? We all saw the numbers that put Firefox with more holes then IE earlier this year too. Could MS be doing a better job, but just getting hammered in the press (who are mostly Apple users by the way)? MS holes get lots of press while other operating systems get a free pass.

The situation is, as usual, more complicated. As typical, they are lumping together pretty much every flaw in pretty much every little piece of software that actually runs on Linux/Unix/Mac and is typically included in Linux/Unix distributions and/or Apple's software....vs the flaws found in Microsoft Windows, and Microsoft software... Gee, is it any surprise they are finding more flaws outside the Windows camp? Several different operating systems and all their software vs one. Furthermore, it does not address which flaws were critical and which weren't, which were fixed and which weren't, how quickly they were fixed....and which side does better in those regards. Also it doesn't address the fact that it's easier to find bugs to fix in an open source environment than it is in a closed source one.

These numbers are meaningless.

[master_p]

Ok, I've made a 'hello world' program in C++...I had 0 bugs in it, do I win?

Seriously now, these numbers are useless without mentioning lines of code and programming languages. Suse Linux 9.3, for example, has over 7,000 RPMs, which is an enormous amount of software.

Absolute bug numbers are meaningless.

Umm am I stupid or something?

[bogie]

Because I know I just woke up but that CERT page is listing APPLICATIONS FLAWS and NOT OS flaws.

Is a flaw in "Gold FTP explorer" or Photoshop a Windows OS flaw?

Am I the only one seeing this?

Re:Umm am I stupid or something?

[ceoyoyo]

Well, for the purposes of security, if the runtime library is distributed with the OS then it should be counted as part of the OS. So installed-on-every-copy-of-windows-because-every-p rogram-needs-to-use-me.dll is counted as part of the OS but funky-library-for-doing-bad-stuff-from-Claria.dll isn't.

Not so shocking ...

[lasindi]

"researchers found 812 flaws in the Windows operating system, 2,328 problems in various versions of the Unix/Linux operating systems (Mac included). "

If we listened to just the media you would have thought Windows has thousands and the others only had a few dozen. I promise I'm not trolling, but do those numbers stop and make anyone on the site re-think stances? We all saw the numbers that put Firefox with more holes then IE earlier this year too. Could MS be doing a better job, but just getting hammered in the press (who are mostly Apple users by the way)? MS holes get lots of press while other operating systems get a free pass.

If you look at the first post, you'll see that the real count of vulnerabilities isn't so shocking after all:

Windows 671
UNIX/Linux 891
Multiple 1512

Also, when you consider the fact that "UNIX/Linux" includes many different operating systems (e.g., GNU/Linux, *BSD, OS X, etc.), you can't give any one Unix operating system the blame. Remember that although some code is shared between projects, GNU/Linux and the *BSD are more or less completely different code bases. In any case, the simple counts of vulnerabilities don't take into account the severity of each, so the real winner is even more ambiguous.

I find myself defending MS allot on this site, and it's nice to have some numbers from a respected neutral organization to debate some of you guys with. I'm sure after this piece they will be re-classified as MS zealots, but what can ya do.

While Brian Krebs might be tainted by his misrepresentation (see the post I got the numbers from), I can't imagine anyone here claiming that US-CERT is somehow a bunch of MS zealots. In fairness to Microsoft, they've definitely come a long way with SP2, and I don't feel nearly as vulnerable when using an SP2 machine as I did with previous Windows versions (though the recent WMF hole makes me a bit more worried). without considering the severity of each vulnerability. But they're still no where near the point where I would switch from Linux.

Re:a nugget of wisdom

[wnknisely]

Counter nugget:

Count the number of IIs exploits vs Apache and correlate to the number of installations. If your logic held, there should be many many more exploits out there for Apache.

Re:Other issues

[symbolic]

I've noticed that on some of the 'nix-based alerts, the initial "discovery" was made in 2004, but not reported by various distros until after the beginning of 2005. I also noticed that with some of them, ALL of the distros listed reported the problem in 2004, but then, someone else chimes in right after the beginning of 2005 (Avaya Security Advisory), basically restating what has already been announced by several other parties prior to 2005.

Re:a nugget of wisdom

[imroy]

So why don't web servers count when 'entire operating systems' do? Web servers are always connected to some sort of network, if not the Internet. They wouldn't be much use otherwise. They often have all sorts of modules/plugins loaded, some third-party. They often have to run all sorts of interpreted languages (Perl, Python, PHP, ASP, etc) with scripts written by all sorts of people. They can also run other executables on the host system. They often have to access a database, either on the same machine or over the network. They often send email and even receive it (e.g confirmation emails).

Most importantly, they're often very public machines (not including intranets). And they can be holding (or have access to) very valuable data e.g banking details, email addresses, passwords. Web servers may be out-numbered by desktop machines, but they're still very attractive targets.

So, would you like to have another try at explaining why Apache HTTP server has been the most used web server for almost ten years now, but is not the most attacked?

Unfair

[Kaelthun]

"The ignorant define themselves" why is there even a discussion going on about the essence of the word "flaw"? Fact is that this research has not been fair because all Linux distro's, UNIX variants (such as BSD) and Mac are counted as one, and MS Windows as another. You cannot compare the multitude of Linux distro's to the one-man platform of MS Windows. If there would have been a tally between, say, Redhat, Ubuntu, FreeBSD, NetBSD, OpenBSD, Mac OS (I dunno what version it is in atm) and MS Windows, and all stats would have been listed seperately ... that would have been fair and clear. Now it's just a mash of all these stats with just one simple query on it SELECT bugs FROM stats WHERE os = Windows. THey just mashed the rest together and called it "the rest".

Re:Unfair

[ComputerizedYoga]

thing is, all of those distros use the same software base. Redhat, Ubuntu, *BSD, they're all host to apache, samba, bind, openssh, php, gcc ... they're all essentially the same, once you get past package management, the kernel and the c libraries.

If you want to count "OS" flaws, you need to remove ALL the third-party apps. That means in linux, you'd JUST be counting the flaws in the kernel and glibc, and in BSD only the core system as well. And those aren't even going to be distro-specific.

While you're right that it's probably not fair to shove os-x vulns in with the unix/linux category (os-x is its own unique animal and has a lot of things that no other *nix has) I think it is fair to mash together the F/OSS nixes. Or at least to mash together their non-os-specific parts.

Of course, these comparisons are inherently unfair, if they're used as a metric for "which OS is more secure". That's become something of a moot point. No matter how someone calculates their metrics, someone or another is going to be displeased with their methodology. What's more interesting, and more to the point, is the sheer number of vulns found across the board, and that's the whole point of the story.

Re:Bugs != volnerability

[TeknoHog]

Looks like your spell checker has a volnerability...

From Bob the Bot

[Tablizer]

The problem is obviously humans. If we kill the humans the problems wouldn't happen.

(hard real-time Java) != Java

[lordcorusa]

I fully agree with you that the Java language is capable of efficient real-time use. However, I think that everyone needs to be perfectly crystal clear that the Java environment to which most Java developers are accustomed is not.

Hard real-time Java programming is vastly different from normal Java programming. Most of the standard Java class libraries are gone. Exception handling is gone. Automatic garbage collection is gone. Almost all third party class libraries are gone. Coding hard real-time apps in Java feels very much like coding C apps from scratch, even if you don't have to manually allocate and deallocate blocks of memory. From the article:

JRTK, a hard-real-time mission-critical subset of the Real-Time Specification for Java (RTSJ) as defined by the Java Community Process, includes many efficiencies over standard Java offerings. No garbage collection is used on objects in the real-time heap. A standard subset of Java libraries is restricted with each library's time and memory resources clearly defined. Partitioning clearly separates soft real-time components from hard-real-time components to ensure hard-real-time schedules as well as program reliability and robustness.

I guess my point is this: hard real-time Java is not the Java with which 99.9% of so-called Java developers are familiar. Choosing Java over C or Ada for a hard real-time system will not enable you to hire lesser programmers, nor will it significantly increase your pool of eligible employees. No matter which language you use, to do hard real-time systems correctly and effiently you must hire only top-tier programmers. Top-tier programmers can make use of any relevant language. Hire any lesser programmers and they will screw up, regardless of language choice.

Re:(hard real-time Java) != Java

[lordcorusa]

Note: When I refer to language, I mean the syntax and semantics and primitive types. When I refer to environment, I refer to the language plus any standard libraries; I am not refering to a particular VM. I think my definitions may be the source of some of the confusion, as Sun appears to want everyone to think that the java.* and javax.* libraries are part of the language. As far as I am concerned, they are just part of the standard environment. (I bend my own definition a little to include packages under the java.lang library as being part of the language, because they are object-oriented wrappers for the primitive types.)

Your original post mentioned work by Aonix on hard real-time Java. My post referred to Java features that are not able to be used in hard real-time applications. Here is my reference; granted it's a year old, but I haven't found anything newer to contradict it. The author is (or was at the time of writing) in charge of the Aonix real-time JVM.

http://www.stsc.hill.af.mil/crosstalk/2004/12/0412 Nilsen.html

Note the table of differences between traditional Java, soft real-time Java, hard real-time Java, and safety-critical Java. As you can see, hard real-time and safety-critical Java are highly restricted compared to traditional Java. Safety-critical is a subset of hard real-time that is even more strict; it requires formal proofs of safety and therefore throws out most Java features and standard libraries, leaving only the core syntax and single-threaded semantics of the language. (Well technically, safety-critical allows multiple threads but they cannot overlap, so its like single threaded programming for purposes of proofs.) Safety critical requirements are defined by FAA specification DO-178B and several JVM suppliers are working on DO-178B-compliant JVMs.

Soft real-time applications can use all (or almost all) of the Java standard libraries, whereas hard real-time applications can use only a restricted subset of the standard libraries and safety-critical applications are restricted to an even smaller subset of the standard libraries. Therefore, one great advantage of the Java environment (remember my definition of environment), its extensive standard library, is neutralized with respect to hard real-time and safety-critical applications.

Furthermore, almost all third party libraries depend on standard libraries that are forbidden under hard real-time or safety critical constraints. Therefore, these libraries are also forbidden and another great advantage of the Java environment (remember my definition of environment), the extensive field of third-party libraries, is lost to hard real-time and safety-critical applications.

The Real-Time Specification for Java defines a set of library calls and semantics which, when implemented within a general-purpose Java virtual machine.... [snip] In fact, Sun demonstrated a real-time Java application running alongside non-real-time applications on the same VM. > Note the emphasis on the word "within". Applications that implement real-time features get the JVM+RTS. Applications that do not implement real-time features fall through to use the traditional JVM. However, non-real-time applications do not automatically become real-time applications simply by being run on Java RTS. Nothing here contradicts my original assertions.

I'd like to know more about the real-time application they demonstrated. What kind of real-time application was it: soft or hard? Did you see the source code of the real-time application? If so, what libraries did it use? The link you provided didn't provide much concrete information. However, it did provide a link to Sun's official Java RTS page:

http://java.sun.com/j2se/realtime/

I am not sure how they can claim conformance for soft real-time, because they do not yet provide a real-ti

what's your problem?

[penguin-collective]

At the end of the day, GC is a useful tool for many programming jobs, but it's only a tool, not a silver bullet. It's no substitute for a good programmer who knows what he's doing.

Perhaps your problem is that you don't understand what a "safe language" is. A safe language is a language that makes guarantees about type errors, error detection, and fault isolation. A language with dynamic memory allocation needs to have a GC in order to be safe. A safe language does not make guarantees about security or parallelism or race conditions, it doesn't necessarily make programming any easier, and it doesn't necessarily help the programmer avoid errors.

And I make this case without, until now, mentioning the IME very real problem that a lot of cheaposoft programmers who grow up relying on GC don't have the same appreciation of low level mechanics as those who don't,

No, the problem is that there are too many people like you in this industry, people who don't even understand what a basic concept like a "safe language" means.

Re:Hard real-time != fast

[Decaff]

People repeatedly demonstrate that Java is as fast as C. They do this for the same reason that members of religious groups keep having to tell themselves that their prefered creator of the universe is better than anyone else's: because the moment they stop reminding themselves they'll realise it's in direct contradiction to reality.

I think it is the other way around. Some C programmers maintain a stubborn faith that their way of working is essential for high performance in the face of increasing evidence to the contrary.