Slashdot Mirror
C# Memory Leak Torpedoed Princeton's DARPA Chances
nil0lab writes "In a case of 20/20 hindsight, Princeton DARPA Grand Challenge team member Bryan Cattle
reflects on how their code failed to forget obstacles it had passed. It was written in Microsoft's C#, which isn't supposed to let you have memory leaks. 'We kept noticing that the computer would begin to bog down after extended periods of driving. This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles. The computer performance would just gradually slow down until the car just simply stopped responding, usually with the gas pedal down, and would just drive off into the bush until we pulled the plug. We looked through the code on paper, literally line by line, and just couldn't for the life of us imagine what the problem was.'"
I'll show you my perpetual motion machines if you show me your perfect autonomous garbage collector. You go first.
"There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy."
The linked "article" is just a "sponsored review" for a C# profiler...
This is a stupid, stupid article headline. Of course you can have a memory leak in a managed language! Any Java programmer who's decent understands that.
.NET CLR did its job, just like it was supposed to.
It's not C#'s fault. The team had references to the obstacle list (event handlers), which prevented garbage collection. The
Just because a language is garbage collected doesn't mean you can't "leak" memory (in the more standard definition of "waste memory over time"), it only means you can't completely lose track of references to objects (which is often used as a more technical definition of "leak"). It is quite common for people coding in such languages to accidentally generate live object structures that are mostly made up of garbage that they should have released their references to. Put another way: these people's program was legitimately claiming memory and never releasing it due to their limited understanding of how event handlers work.
This is a programming error, plain and simple. From TFA:
Though we thought we had cleared all references to old entries in the list, because the objects were still registered as subscribers to an event, they were never getting deleted.
So references were held to the objects in two places - the list of encountered obstacles, and the list of event subscribers. They were being removed from the list of encountered obstacles, but not being unsubscribed from the event.
How do you think event subscription works? Something has to hold a reference to the objects that are subscribed to the event! That thing is going to hold a reference until you unsubscribe the object - it neither knows nor cares about any other list of references you may be maintaining separately, how could it?
This is a coding error. A subtle, non-obvious one perhaps, but a bug nevertheless. It is not an error in the CLR, and in fact the article never paints it as such. That particular bit of spin is wholly down to the submitter.
It's official. Most of you are morons.
I've RTFA, is wasn't a memory leak caused by C#, is was caused by bad programming. After that,the whole article starts to advertise some obscure profiling tool. Maybe they should should have written the whole thing in C++ and use valgrind instead. Just an ideea...
This section totals 15 points.
Background:
There are more types of resource leaks than just memory leaks. A memory leak is when your program keeps hold of memory it's not using. An object leak is when your program keeps hold of objects it's not using. A file descriptor leak is when your program fails to reuse the descriptors for files it has closed and will not reopen. Many other types of leaks could be considered.
Exercises:
1. Determine which issue this scenario describes.
2. Figure out which issue can be handled by automatic memory management.
3. Discuss whether, and if so why, the answers to Exercises 1 and 2 mean there is some conceptual discord between the wording of the scenario and the use of the term "memory leak".
I don't see why they just didn't write it in C.
They were using massive cooling systems and having very thorough code reviews, sounds like a perfect reason to use C over C#.
What surprises me most is the small size of their software, only 10 thousands lines of source code (I think that the average car processor already have these for today's cars -ignition & braking systems-). Given a team of a dozen programmers working for a year, I was expecting at least 50KLOC, or maybe 200KLOC (for example, the GCC compiler is 3MLOC, and the linux kernel has comparable size.)
Of course memory leaks can happen with garbage collected languages, but these leaks are a little easier to find....
Maybe they should have coded in a higher level language like Ocaml, Haskell.
And yes, I'm sure most of an autonomous vehicle software is not low-level drivers, but in the planification & perception tasks. On such tasks, higher-level languages definitely make sense.
I also did not understood what kind of libraries these teams are using.
I'm also surprised that it is apparently so easy to get funded to have only 10KLOC inside a car!
I see quite a few comments from C/C++ coders who wonder whether managed memory people know how event handling works. If they knew a little more about managed memory languages, they'd know a reference does not have to be "hard": you can have a reference to an object that does *not* prevent garbage collection.
So I guess the real question here is whether event handlers should be hard-referenced (as they are here), or just soft/weak referenced...
From a developer perspective it's quite natural to think that, as long as his code doesn't hold any reference to an object, it should be garbage collectable. If registerEvent() shall hard-reference handlers, documentation should be *very* explicit about it (and the need to unregister a handler for GC to work on it).
On the other hand, if handlers are not hard-referenced you can no longer register anonymous class event handlers...
:(){
That's why there are no memory leaks in C/C++ code [/sarcasm]
(1) You are supposed to test your software.
(2) You are particularly supposed to test your software if you send $200k and 1 ton of hardware careening through the street on autonomous real-time control.
(3) Garbage collectors do not prevent memory leaks.
(4) Garbage collected systems can be good for building real-time systems, but you need a real-time garbage collector or you need to treat the system as if it didn't have a garbage collector at all.
What "ruined their chances" was not that they overlooked a memory leak, what ruined their chances was that they didn't know what they were doing.
As is the case with Microsoft's GC, Java's won't delete things that are still being referenced by other things, because it quite reasonably assumes that an object which is referenced by another object that hasn't itself been marked for collection isn't garbage.
The main problem with garbage collectors (I like GCs, so this isn't a diatribe against them) is that far too many mediocre programmers assume they have a magical ability to know precisely what they want their code to do. The reality of course is that they use algorithms to decide what should be collected, when it should be collected, and how it should be collected, and those who are unfamiliar with the particular strategies that their GC uses can therefore not only write code with more than a few memory leaks, but also code that results in the GC being used so inefficiently that it does vastly more work than would be necessary if the same functionality was implemented in a slightly different way.
There are plenty of articles about Java memory leaks that can be found by Googling "java memory leaks". Googling "java GC tuning" will produce some useful links to articles containing tips on ensuring that it's not used inefficiently.
I'm not going to change your sheets again, Mr. Hastings.
Well the Event Subscribed 'problem' is well known and makes sense if you think about it. I mean subscribing to an Event means placing a pointer to a delegate of a method in a event subscriber list.. when someone raises that event then each delegate in the list is invoked... so basically it is an implicit reference and hence can prevent the it from being marked for garbage collection.
;) :)
However, i had another memory 'leak' problem where the Garbage Collector simply didn't collect in time which caused my application to use more and more memory until it reached the system limit and crashed... i found that simply calling
GC.Collect();
GC.GetTotalMemory(true);// (the true 'forces' collection
once would fix this problem... i though i needed to call it every minute or so... but when calling just once it did SOMETHING that prevented this problem from occurring again.. no idea exactly what.. but it works
They can't have had anyone on the team with experience of coding for Swing in Java then - you get these all the time, sometimes hanging tens of megabytes of unwanted GUI objects off a single listener registration, and learn how to spot and fix them.
They also didn't pick a very good hack because it didn't leave the car in a safe state when the software broke.
Lack of practical experience I'd say. A few more events like that and they'll make decent devs one day.
A funny thing happened with during my co-op this summer:
I was working at a coal-fired power plant which needed a new pollution control device before 2010. There, I would dig through the literature, and try to find suitable products and operating conditions for this device. Anyway, this involved a lot of meetings, conference calls, and business lunches with the suppliers in question.
Then there was Joe.
Joe was our Alstom sales rep: portly, humorless, slow to speak and slower to understand. He was also a devote Utahnian.
Well, one day, we were killing time while waiting on a conference call, my supervisor left the room, and we started talking about universities. Then he dropped the bomb:
"In my Senior year, I worked on developing perpetual motion machines."
My supervisor then reentered the room, and we got back to work. I felt like I'd just seen a dancing frog.
Criticisms of the team aside, I would like to say that neither Java nor C# have made any steps to remedy problems like this with seem to be all too common with inexperienced developers. Both Java and C# need to support attaching to event handles with "weak" handlers. That is, the handler will not hold onto the object which defines the handler (and will automatically deregister itself sometime after the object has been collected). In many cases, there is a need for an object to listen and handle an event from another object, but only whilst the object that is listening is still referenced (with the exception of the reference held by the object firing the event).
In C#, the (admittedly ugly) way to implement this is to use an anonymous method and a weak reference: The "closure" that is created for the anonymous method does not hold a reference to "this" as it does not access any of "this"'s fields or methods unless it's through the weakreference.
The code has a flaw where the event handler code (only a few bytes to hold the closure) will never deregistered be collected unless the event is fired sometime after the owner object has been collected. This can be fixed by using a NotifyingWeakReference (a weak reference that raises an event when it has been collected).
these don't have any problems with circular reference structures - if it can't be reached from a root and marked, it'll get collected.
still just a blunder, as you say.
this article should be binned - misleading title and nothing but a puff-piece for a profiler. i much prefer YourKit, incidentally:-)
Actually, C# doesn't reference count at all, it 'Reference Traces' :)
Please, let me explain; it's quite sad how often people don't get this ...
.Net has its block of managed memory, called the Managed heap. It's separated into 3 'generations'. This heap has 2 areas, free space and reserved space, from top to bottom.
When you allocate and object to the heap, by using the new command (object o = new object();) there is a set of rules? that have to be enforced:
The GC manages Reference tracing, and this doesn't occur when the object goes out of scope, it actually happens when the Heap is full and you attempt to allocate a new object.
In something called 'the sweep', the GC goes through each object in the heap to see if it's reachable. To do this it starts with so-called 'roots'. It then traces to see which objects are referenced by these roots.
A root identifys a storage location, which referes to objects on the managed heap, or objects that are set to null. For example, all of an applications global and static objects are considered to be it's roots. (hence the reason that all C# apps have a static void main).
When the sweep starts, it assumes that all objects are garbage. So for each root object, it builds up a graph of the objects that root references, and marks them as being live.
However, if it finds an object that's already in the graph, it stops traversing that path. This is two (massively) increase performance by not scanning the same object twice, and more importantly, it stops you getting into an infinite loop by scanning a circular list.
The pinch is, it prevent the circumstance that you mentioned! :)
Because the strong reference to a linked circular list is gone, the circular list isn't attached to a root object, so it gets disposed. If you don't want it to get dropped, unless it theres a memory shortage, the C# GC also supports something called Weak References, but I'm not going to go into those here as it's headhurting
So once all the roots have been checked and we've got a nice graph of all the objects that are referenced by the live parts of the application somehow, the second stage of GC happens.
Any objects that haven't been touched by the walk are of course still marked as Garbage. The GC now walks up the heap linearly, looking for contigious groups of garbage which are now considered to be free space. The GC looks for the next live object and moves it to the start of this free space with a good old memcpy :)
This ofcourse invalidates all the root pointers, so the GC then updates the points in the root objects.
:)
So now, we've got rid of all the garbage and our heap is pleasantly compacted; Take that Heap Fragmentation, Kerpow!!
But, that's not all she wrote of course
Now we're free'd and compacted, the 'nextObjPtr' is moved to the top of the heap. At this point the new object creation that triggered the collection is performed and the new object appears at the top of the heap.
This is a dramatic over-simplification and I've not attempted to explain finalization or weak references, but it's still good to know this stuff, it helps us as .Net programmers to consider how to write our code properly :)
The other thing I've not explained is how the Generations work:
When we remember we are all mad, the mysteries disappear and life stands explained.
Slashdot editors are even more pathetic than I thought they were. It's bad enough that they didn't skim through the article, but they apparently didn't even take a look at the URL. Look at this thing:
http://www.codeproject.com/showcase/IfOnlyWedUsedANTSProfiler.asp
"IfOnlyWedUsedANTSProfiler"? That didn't raise any flags?
Of course, I'm trying to assume good faith and not just conclude that the editors knew this was an advertisement, but they sure are making that difficult.
It's the programmer and the language. Give the world's best carpenter a ball-peen hammer and ask him to build you a beautiful armoire, see what happens.
You can say now that they'll be much further next year, but until then "Which means that the language did the job very nicely" should be "Which would mean that the language did the job very nicely." If you put in a reminder of some sort to come back and say I told you so, I'd be more than happy to eat my words if they continue using C# and place in the top 33%. Hell, I'd even concede that you might be right if they manage the top 50%.
I say, however, that there is a right language for the job. Sure, there's overlap, but you don't implement your FFT in Perl when the problem is that you need the fastest FFT possible, you don't write a word-processor in assembly, and you don't write anything in Brainfuck even though they're all Turing-complete. Anyone who says you can do anything in any language is trying to justify using their favorite language for absolutely everything.
<xml><I><am><so><damn>Web 2.0</damn></so></am></I></xml>
I usually like /. articles, even the ones against MS, but I cannot just skip over this one:
if the moderator read the article he would have noticed that the article was an advertisement for the profiler product, not just a review of it (it was written directly by Red Gate).
Second, the article itself says that they found that the error was in how they coded the application, because they left some reference so the garbage collector didn't trow away the objects.
This is a really bad article and bad information.
Come on. Really. What kind of idiot marketer sends in stories like this to Slashdot? We know what happens. First, you get derided mercilessly for trying to sway us with your ridiculously transparent attempt at marketing. Then, the real experts come out and poke holes in everything you've said. Then everyone else chimes in with better (and often free) alternatives. You and your company end up looking like buffoons, and your product ends up looking like utter garbage.
You may think you're pulling one over on the editors, and maybe you are. But you aren't pulling one over on us, and I think after all these years, the editors know this. So, just don't. Unless your product or service is absolutely bulletproof people here are more likely to shoot it full of holes than rush out and buy it.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
People complain that Slashdot sucks: the headlines are sensationalistic, the editors get commissions based on the number of dupes they post, and articles about 6-month-old events get posted as "news".
So why do I even bother visiting Slashdot? The answer is two things: the community of posters, and Slashcode moderation.
The value of Slashdot is in its community. You and I, dear Slashdotters. Our collective mind will pick through the various articles, point out their flaws, expose sensationalist FUD for what it is (and, surprisingly, will do this equally for anti-Linux and anti-MS FUD), debate various trends, and provide a signficantly international (though heavily USA-centric) perspective.
This value is enhanced by Slashdot's moderating system, so that information and insight can bubble to the top among the mass of inane posts. Metamoderation limits the amount of crack that the moderators can be on.
So, Slashdot editors, take note! *WE* are the reason we are here. *YOU* are not. Many of us don't even bother to read the articles any more, preferring to soak up the collective wisdom of techies from varying age groups and fields. If you piss us off, and the collective community of Slashdot deteriorates, then there's no reason for me (or others) to keep coming back.
Think about it.
404555974007725459910684486621289147856453481154 in hex is "You sank my Battleship?"
[GPG key in journal]
Hey, here's a wacky idea that's just crazy enough to work - DON"T USE DYNAMIC MEMORY ALLOCATION! Why in holy hell would someone construct what amount to an embedded real-time system using dynamic memory. Define fixed memory allocations for everything. Run tests. If the memory is insufficient, the program crashes. Then you can see where the program crashes and why. Then you can fix it.
Just because you *can* do something doesn't mean you should.
Brett
Slashdot has editors. I know this, because the stuff below "nil0lab writes..." is heavily editted from what I actually submitted! In fact, I started my actual submission with something like "in a shameless plug for some code analysis product..."