Slashdot Mirror

← Back to Stories (view on slashdot.org)

C# Memory Leak Torpedoed Princeton's DARPA Chances

Posted by ryuzaki0 on from the omg-a-tumbleweed dept.

nil0lab writes "In a case of 20/20 hindsight, Princeton DARPA Grand Challenge team member Bryan Cattle reflects on how their code failed to forget obstacles it had passed. It was written in Microsoft's C#, which isn't supposed to let you have memory leaks. 'We kept noticing that the computer would begin to bog down after extended periods of driving. This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles. The computer performance would just gradually slow down until the car just simply stopped responding, usually with the gas pedal down, and would just drive off into the bush until we pulled the plug. We looked through the code on paper, literally line by line, and just couldn't for the life of us imagine what the problem was.'"

7 of 560 comments (clear)

  1. Slashvertisement by shartte · · Score: 5, Informative

    The linked "article" is just a "sponsored review" for a C# profiler...

  2. This is not a C# memory leak! by Tim+C · · Score: 5, Informative

    This is a programming error, plain and simple. From TFA:

    Though we thought we had cleared all references to old entries in the list, because the objects were still registered as subscribers to an event, they were never getting deleted.

    So references were held to the objects in two places - the list of encountered obstacles, and the list of event subscribers. They were being removed from the list of encountered obstacles, but not being unsubscribed from the event.

    How do you think event subscription works? Something has to hold a reference to the objects that are subscribed to the event! That thing is going to hold a reference until you unsubscribe the object - it neither knows nor cares about any other list of references you may be maintaining separately, how could it?

    This is a coding error. A subtle, non-obvious one perhaps, but a bug nevertheless. It is not an error in the CLR, and in fact the article never paints it as such. That particular bit of spin is wholly down to the submitter.

    --
    It's official. Most of you are morons.
  3. Re:I'll show you mine if you.. by Anonymous Coward · · Score: 5, Informative

    It's not the garbage collector's fault. If an object is still in use, it can't be collected and destroyed. Managed memory only prevents the kind of memory leak where the programmer "loses" all references to the memory and thus never frees it. It also prevents the kind of bug where memory which is still in use is freed. Programs usually crash when that happens (either the OS terminates them due to a memory protection violation or they overwrite their own data and crash later on). That is also what would likely have happened in this case if it weren't for managed memory, because obviously the programmers mistakenly thought that these objects were no longer in use, so they would have freed them when they were still handling events.

  4. Re:Well, there's your problem! by __aasmho4525 · · Score: 5, Informative

    just to be clear, THE BUG WAS NOT IN THE RUNTIME, not by any stretch.

    there are very clear constructs in place in the language/runtime to allow any object to unregister itself from event registrations it initiated.

    this was VERY MUCH a bug in the end-user software, not the runtime (i've written code almost IDENTICALLY to this and blew lots of time having made this same mistake).

    the only thing the runtime could do to protect the idiot developer (myself included) is automagically make all event references WEAK references, but that has plenty of undesirable side-effects too... in clr, you can do this yourself if you're so inclined... (just like in a JVM)

    cheers.

    Peter

  5. Re:Reference counting by Etrigoth · · Score: 5, Informative

    Actually, C# doesn't reference count at all, it 'Reference Traces' :)

    Please, let me explain; it's quite sad how often people don't get this ...
    .Net has its block of managed memory, called the Managed heap. It's separated into 3 'generations'. This heap has 2 areas, free space and reserved space, from top to bottom.

    When you allocate and object to the heap, by using the new command (object o = new object();) there is a set of rules? that have to be enforced:

    • Allocation occures in a contigious range of the freespace, that's as big as the size the clr determines the type to be.
    • The order of objects must be in the order of creation
    • There must be no gaps between objects
    • The oldest objects are in the lowest address space

    The GC manages Reference tracing, and this doesn't occur when the object goes out of scope, it actually happens when the Heap is full and you attempt to allocate a new object.

    In something called 'the sweep', the GC goes through each object in the heap to see if it's reachable. To do this it starts with so-called 'roots'. It then traces to see which objects are referenced by these roots.

    A root identifys a storage location, which referes to objects on the managed heap, or objects that are set to null. For example, all of an applications global and static objects are considered to be it's roots. (hence the reason that all C# apps have a static void main).

    When the sweep starts, it assumes that all objects are garbage. So for each root object, it builds up a graph of the objects that root references, and marks them as being live.
    However, if it finds an object that's already in the graph, it stops traversing that path. This is two (massively) increase performance by not scanning the same object twice, and more importantly, it stops you getting into an infinite loop by scanning a circular list.

    The pinch is, it prevent the circumstance that you mentioned!
    Because the strong reference to a linked circular list is gone, the circular list isn't attached to a root object, so it gets disposed. If you don't want it to get dropped, unless it theres a memory shortage, the C# GC also supports something called Weak References, but I'm not going to go into those here as it's headhurting :)

    So once all the roots have been checked and we've got a nice graph of all the objects that are referenced by the live parts of the application somehow, the second stage of GC happens.

    Any objects that haven't been touched by the walk are of course still marked as Garbage. The GC now walks up the heap linearly, looking for contigious groups of garbage which are now considered to be free space. The GC looks for the next live object and moves it to the start of this free space with a good old memcpy :) This ofcourse invalidates all the root pointers, so the GC then updates the points in the root objects.

    So now, we've got rid of all the garbage and our heap is pleasantly compacted; Take that Heap Fragmentation, Kerpow!!
    But, that's not all she wrote of course :)

    Now we're free'd and compacted, the 'nextObjPtr' is moved to the top of the heap. At this point the new object creation that triggered the collection is performed and the new object appears at the top of the heap.

    This is a dramatic over-simplification and I've not attempted to explain finalization or weak references, but it's still good to know this stuff, it helps us as .Net programmers to consider how to write our code properly :)
    The other thing I've not explained is how the Generations work:

    • We create a new object and the CLR realises that Gen0 is full and a sweep occurs.
    • Firstly, before the sweep and trash, Gen0 has the youngest objects that the GC has never seen.
    • Now that the sweep has finished, all the compacted obje
    --
    When we remember we are all mad, the mysteries disappear and life stands explained.
  6. Re:Well, there's your problem! by FreeGamer · · Score: 5, Informative

    It obviously doesn't work in situations like this where the bug is in the runtime and not the application. RTFA. FTFA:

    Though we thought we had cleared all references to old entries in the list, because the objects were still registered as subscribers to an event, they were never getting deleted. We added one line of code to remove the event subscription and, over the next three days, we successfully ran the car for 300 miles through the Mojave desert. As another poster points out, this is just an advert for a profiler, which helps people who use coding practises that they did not initially fully understand. As much as I wanted it to be a bug in the C# runtime, it's just another PEBKAC issue. The /. article introduction was wonderfully ambiguous on this point, if anything it was inflammatory ("C# memory leak"). Poor article selection if you ask me, but it's been many years since the /. editors genuinely cared about the content on this site rather than the number of hits/adclicks.
    --
    Free Software games list and commentary
  7. Re:I'll show you mine if you.. by fitten · · Score: 5, Informative

    There's also the issue where you need to explicitly remove your event listeners when you no longer need the object. The listener keeps a reference to the object (via the interface) so even if it goes out of scope or what-have-you, YOU may think you don't have any references to the object but it implicitly does, through the listener you handed to the system. So... if you're using event listeners, make sure you explicitly remove them in your object's destructure... or else you'll end up with a memory 'leak'.