C# Memory Leak Torpedoed Princeton's DARPA Chances

nil0lab writes "In a case of 20/20 hindsight, Princeton DARPA Grand Challenge team member Bryan Cattle reflects on how their code failed to forget obstacles it had passed. It was written in Microsoft's C#, which isn't supposed to let you have memory leaks. 'We kept noticing that the computer would begin to bog down after extended periods of driving. This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles. The computer performance would just gradually slow down until the car just simply stopped responding, usually with the gas pedal down, and would just drive off into the bush until we pulled the plug. We looked through the code on paper, literally line by line, and just couldn't for the life of us imagine what the problem was.'"

Comments

Stupid Slashdot headline

[RzUpAnmsCwrds]

This is a stupid, stupid article headline. Of course you can have a memory leak in a managed language! Any Java programmer who's decent understands that.

It's not C#'s fault. The team had references to the obstacle list (event handlers), which prevented garbage collection. The .NET CLR did its job, just like it was supposed to.

Re:Stupid Slashdot headline

[cnettel]

There is no explicit delete in C#. They may have added something in their framework, or they're just saying that they called delete on the collection where they thought they stored all objects. The GC isn't psychic. (It could be a good thing to ask the GC to delete an object explicitly and get an exception if it's indeed still referenced elsewhere, but that's not possible in C#, or Java for that matter.)

Re:Stupid Slashdot headline

[Mr. Shiny And New]

You might not be able to use weak references since they introduce (at least in Java) a layer of indirection. For example, an addListener method usually takes an interface of some kind of Listener, not a WeakReference (to a listener).

Now, if you have control of the implementation of the object who accepts Listeners you can store them internally in a weak collection, which allows them to be garbage-collected. This would work but may not be what the programmer intends. Actually in a language like Java I'd hazard that usually the programmer wouldn't want that at all: consider an application that listens to UI events. As a programmer I want to be able to stick listeners wherever they are needed and leave them there permanently. If I don't need a pointer to the object, I don't want to keep it around, and thus may not have a reference to the listener EXCEPT in the event-management collection. That's the advantage of GC languages: as soon as the object which creates those events (say, a dialog box) goes away, the objects it refers to have one fewer pointer and may be eligible for GC.

Anyway, lots of code has issues like this: we had a problem at my work where an Apache taglib was caching some compilation in a cache that would grow for ever. It was a simple code fix to solve that problem, but there was no way for us to even SEE the problem until we ran our application under load in a profiler. Fun fun fun.

Re:Slashvertisement

[JanusFury]

The sad thing is that Microsoft offers a perfectly servicable profiler for free that can be used on any C# application and is better than most commercial native Win32 profilers...

only 10KLOC?

[basiles]

What surprises me most is the small size of their software, only 10 thousands lines of source code (I think that the average car processor already have these for today's cars -ignition & braking systems-). Given a team of a dozen programmers working for a year, I was expecting at least 50KLOC, or maybe 200KLOC (for example, the GCC compiler is 3MLOC, and the linux kernel has comparable size.)

Of course memory leaks can happen with garbage collected languages, but these leaks are a little easier to find....

Maybe they should have coded in a higher level language like Ocaml, Haskell.

And yes, I'm sure most of an autonomous vehicle software is not low-level drivers, but in the planification & perception tasks. On such tasks, higher-level languages definitely make sense.

I also did not understood what kind of libraries these teams are using.

I'm also surprised that it is apparently so easy to get funded to have only 10KLOC inside a car!

Hard/weak references for event handlers

[djinnn]

I see quite a few comments from C/C++ coders who wonder whether managed memory people know how event handling works. If they knew a little more about managed memory languages, they'd know a reference does not have to be "hard": you can have a reference to an object that does *not* prevent garbage collection.

So I guess the real question here is whether event handlers should be hard-referenced (as they are here), or just soft/weak referenced...
From a developer perspective it's quite natural to think that, as long as his code doesn't hold any reference to an object, it should be garbage collectable. If registerEvent() shall hard-reference handlers, documentation should be *very* explicit about it (and the need to unregister a handler for GC to work on it).
On the other hand, if handlers are not hard-referenced you can no longer register anonymous class event handlers...

Re:Hard/weak references for event handlers

[tgd]

Weak references also incur the overhead of a check on every call to ensure the object hasn't been cleaned up. This was sloppy, poorly tested code. The engineers on it made a mistake and caught it too late. It happens.

The poster of the article was trolling, and not only trolled with the post, managed to get a troll posted to a slashvertisement which was not even trolling.

Impressive on the part of the person who submitted it, but disappointing considering Taco's comments a few weeks back about articles that are truly nothing but advertisements.

don't these kids learn anything anymore?

[m2943]

(1) You are supposed to test your software.

(2) You are particularly supposed to test your software if you send $200k and 1 ton of hardware careening through the street on autonomous real-time control.

(3) Garbage collectors do not prevent memory leaks.

(4) Garbage collected systems can be good for building real-time systems, but you need a real-time garbage collector or you need to treat the system as if it didn't have a garbage collector at all.

What "ruined their chances" was not that they overlooked a memory leak, what ruined their chances was that they didn't know what they were doing.

Re:I'll show you mine if you..

[TheRaven64]

Mod parent up, and anyone doing any AJAX coding pay attention. This is one of the easiest bugs to create in JavaScript. JS is a very powerful language with a Self-like object model which a lot of people seem to treat as a basic variant. This means that they create closures when they intend to pass function pointers and end up creating large numbers of object references (the closure itself is an object and it retains a lot of references) preventing the garbage collector from doing anything useful. This is particularly common with AJAX where closures are often used for handling asynchronous events (which would be a gorgeous coding style if the syntax were slightly less ugly).

Re:Well, there's your problem!

[DaveAtFraud]

When writing parallel software, you need to treat the interactions between different threads of execution the same way you treat interactions between different code elements. This is a *DESIGN* issue. That's why debuggers aren't usually useful for finding such problems. Debuggers let you find sequential logic errors but rarely give you any insight into state coupling errors that only occur during real-time execution of the program. This is true whether the error occurs between different threads of the same code or different processes of the overall program. For some reason, people are particullarly unable to see such problems between different threads.

I first ran into this sort of problem in 1983 when working on a CDC mainframe. The only way to find the bug was the line by line analysis method since even compiling the code with debug caused it to run slower and the nature of the problem changed. That's as much detail as I remember.

I expect to see a lot more of these kinds of errors pop up as multi-core CPUs become more prevalent (true parallel execution) and people continue to assume that they can just crank out code without taking the time to understand the design. I'd also expect the prevalence of multi-core processors to create a demand for more parallelism. If you don't take advantage of the additional cores, your program will only be as fast as if it were on a single core system. If the competition can create a program that uses the additional cores, your program will seem slow.

Cheers,
Dave

Re:I'll show you mine if you..

[Blakey Rat]

Ditto, I submitted an article about an article that was about a survey done of corporations which concluded that corporations would not quickly adopt Vista. (This was before it was released, which frankly is a 'duh'... do corporations quickly do anything IT related?) But anyway... I spent a lot of time and had someone else read over it, attempting to get the *perfect* Slashdot submissions. There were no grammar errors, no spelling errors, the link was directly over the right keywords, etc.

The Slashdot editor who posted it moved the link so it looked like I was linking to the original study, not the article about the study. It's like they felt compelled to make a change, so they made one even if the change didn't improve the quality of the article.

I will say that the rest of the text remained unchanged, and really the only problem with the submission is that people who thought they were going to a study were actually going to a newspaper article about a study, but the point is Slashdot editors *do* make changes all the time.

Slashdot not a news source? Agree!

[KWTm]

Slashdot is a blog not a news source. I wish more people would realize that.

Agree. It's time to get this out in the open.

People complain that Slashdot sucks: the headlines are sensationalistic, the editors get commissions based on the number of dupes they post, and articles about 6-month-old events get posted as "news".

So why do I even bother visiting Slashdot? The answer is two things: the community of posters, and Slashcode moderation.

The value of Slashdot is in its community. You and I, dear Slashdotters. Our collective mind will pick through the various articles, point out their flaws, expose sensationalist FUD for what it is (and, surprisingly, will do this equally for anti-Linux and anti-MS FUD), debate various trends, and provide a signficantly international (though heavily USA-centric) perspective.

This value is enhanced by Slashdot's moderating system, so that information and insight can bubble to the top among the mass of inane posts. Metamoderation limits the amount of crack that the moderators can be on.

So, Slashdot editors, take note! *WE* are the reason we are here. *YOU* are not. Many of us don't even bother to read the articles any more, preferring to soak up the collective wisdom of techies from varying age groups and fields. If you piss us off, and the collective community of Slashdot deteriorates, then there's no reason for me (or others) to keep coming back.

Think about it.

Slashdot *does* have editors. When I submitted...

[nil0lab]

Slashdot has editors. I know this, because the stuff below "nil0lab writes..." is heavily editted from what I actually submitted! In fact, I started my actual submission with something like "in a shameless plug for some code analysis product..."