Memory Leaks

G3ck0G33k writes: "Is there any free software version/clone of Rational's programs PureCoverage and/or Purify? I have worked with both of them on fairly large projects (>150,000 lines of code) and they were great to work with. When the first runs of Purify found nearly fifty instances of minor memory leaks, I was deeply frustrated/impressed. A free (perhaps GPLd) clone would be so interesting; Rational's licensing is killing my current budget. Of course, the more kinds of leaks it may detect, the better. GeckoGeek" We had a similar question last year but there's no harm in seeing what the current answers are.

Comments

Bounded pointers, etc.

[Lumpish Scholar]

For the other kinds of stuff Purify does (aside from memory leaks), look at Greg McGary's bounded pointer work.

Bad news: You'll have to build your own gcc (Greg's changes haven't yet been accepted in to the gcc trunk), and all your libraries (just as Purify re-writes all your libraries).

Good news: The resulting code is much faster than Purify'ed code, and finds some problems Purify doesn't. I know of a major software development effort (hundreds of developers, millions of lines of code; sorry, can't give details) that uses bounded pointers to great advantage.

Other tools: GNU Checker, dbmalloc, Bruce Perens' Electric Fence, MemProf, mpatrol, and Mprof; Google searches will turn them all up.

Memory leak detection

[pthisis]

The Boehm-Weiser garbage-collecting malloc() can be built in a leak-detection mode. Every time an object is leaked, it prints out the address of the memory in question. Do that. Then it's 15 lines of python to correlate that back with the malloc() calls; I wrapped malloc/realloc to print out the line number and filename, e.g.

void *our_malloc(size_t howbig, int line, char * file)
{
void *p;

p=GC_malloc(howbig);
fprintf(stderr, "Line %d of %s/%s(): %p\n", line, file, p);
return p;
}
#define malloc(x) our_malloc(x, __LINE__, __FILE__)

with similar for realloc (and make free do GC_free).

Then run the proggy, redirecting stderr through a simple python script: (leading spaces have been replaced with underscores since slashdot doesn't do PRE)

import sys

a={}

for line in sys.stdin.readlines():
__line=line.strip()
__num=line[line.find("0x"):]
__try:
____num=num[0: num.index(" ")]
__except:
____pass

__if line[1]=="i":
____a[num]=line
__else:
____print "Leaked object: "+a[num]

When I run my program this way I get the following output:

Leaked object: Line 43 of leak_stuff.c/(): 0x806efe0
Leaked object: Line 43 of leak_stuff.c/(): 0x806eff0
Leaked object: Line 55 of leak_stuff.c/(): 0x806dfd8

Which tells me which lines to look for the initial allocations of leaked objects at.

The garbage-collecting malloc is really cool; it's at:

http://www.hpl.hp.com/personal/Hans_Boehm/gc/

for now, but rumor has it that gcc will become the official source for it at some point (it's needed for the Java compiler).

Sumner

Re:Memory leak detection

[pthisis]

Be careful using Boehm in production code; the web pages has the caveat:

C compilers may not hide pointers in the generated object code. In our experience, standard commercial compilers obey this restriction in unoptimized code. Most aggressive optimizing compilers do not obey this restriction for all optimized code. For details and examples see papers/pldi96.ps.gz. However, it is difficult to construct examples for which they violate it, especially for single-threaded code. In our experience, the only examples we have found of a failure with the current collector, even in multi-threaded code, were contrived.

However, the gcc developers claim the gcc does in fact violate this constraint. So using Boehm gc with gcc may not be safe in production code. The gcc mailing list has had a couple of threads on how to make gcc garbage-collector friendly in the future (once again, Java is one impetus for this). Until then, I'd stick to manual mm and use the gc only to help find leaks.

Sumner

Re:Memory leak detection

[PD]

I was going to write the same thing about the Boehm GC, but yours was the first message I saw. Seriously, this collector is an excellent thing. A person would have to be crazy to do manual memory management unless they had a solid technical reason not to.

Re:Memory leak detection

[d^2b]

dmalloc (www.dmalloc.com) seems to work pretty well for finding memory leaks. It is distributed under a BSDish
license.

Compiles and runs out of the box on an alpha
running Linux.

GUI? uh no. It has a nifty command line utility to control logging etc...

Re:Roll your own

[tyoud1]

Nod, he's a drunken master. :)

Your kung fu is no good, Anonymous Coward.

Here's a quick and easy solution

[spacewhale]

Write a malloc wrapper and #define it in place of the real thing. With #define you can easily log the location in the code, amount of RAM, and location in memory to a file, then write a script in the language of your choice to see which locations in RAM weren't dealloced, and match them with the appropriate malloc call, which also contains the location in code. It took me about an hour to implement this in a multi-thousand line program and it works very well. The only thing it doesn't catch is when a library call mallocs something and expects you to dealloc it, but i solved this by including a fake malloc call that just logs but doesn't actually malloc, so you'd call it right after the library call that actually does the malloc.

dmalloc for memory debugging

[epperly]

I like dmalloc for memory debugging. It even found a memory bug for a program that purify choked on. It doesn't have a GUI.

use C++

[mj6798]

In C, this is a never ending battle. Even with Purify, you are going to spend lots of time introducing bugs, then tracking them down. If you must stick with C, consider using one of the C interpreters (EiC, cint, etc.). Machines have gotten fast enough that you can use them for debugging your code. Or stop worrying about it and just use the Boehm garbage collector as a garbage collector.

I switched from C to C++ basically because I couldn't get Purify for Linux. C++ has allowed me to adopt clear, well-defined memory management strategies and automate various pointer checks. I hardly ever get memory leaks or pointer errors in my C++ code anymore.

But no matter what you do in your own code, if you are using C or C++, you will always be exposed to numerous pointer bugs and leaks in library code. Most real-world C++ code commits the same memory allocation sins and has the same pointer bugs as real-world C code--people aren't taking sufficient advantage of C++'s smart pointer facilities (even STL is flawed in that way). Therefore, for multiprogrammer projects, I wouldn't use anything but Java or another safe language anymore.

Re:I asked REdhat once..

[szomb]

ElectricFence detects overruns of malloc()d buffers (hence its name). Unless this changed recently I am fairly sure it has nothing to do with leak detection?

Probably a bit late for existing projects but...

[MadAndy]

One of the places I'm involved with doesn't use the standard malloc calls. Instead we use something more like:
get_mem(ptr, size, "widget hash table")
When debugging, get_mem keeps track of all allocs. At the end, just before the program shuts down the heap dump routine is called which lists all outstanding memory blocks along with the debug string so you can see where they were allocated.

It's also often practical to call the dump routine at various points within the program and give the output a quick look-over or diff - it's amusing how often you can nip these problems in the bud this way.

Also, if you get really desparate, change the get_mem routine to increment a global counter and tag that to the end of each allocation info block. If you keep a program debug log and log each allocation it makes it easy to see where a loose block was allocated - grab the unique ID from the dump and search the log file for it.

A handy feature about this trick is that you use #define to define get_mem, so when you go to production you simply define it to malloc and throw the debug string away - no speed or size cost in the running program. In addition, it basically costs nothing except an hour or so to set it up in the first place. The catch is you have to use it religiously from the start of your project.

A really simple trick, but it has saved me so much work!

Roll your own

[Ratbert42]

In college, I rolled my own wrapper for malloc(), free(), and array/pointer dereferences. A couple hours of coding that wrapper caught most of my memory leaks and seg faults. If I could do it when I was half-drunk and didn't know what I was doing, you've probably got a developer on staff who can handle it.

Free Beer?

[Ratbert42]

A free (perhaps GPLd) clone would be so interesting; Rational's licensing is killing my current budget.

Maybe you should put a developer or two on that project and see how long it takes them to build something similar. I think Purify runs about $1,500 now (could be wrong). That's what, two Aeron chairs? That shouldn't kill any real company's budget. Numega's Boundschecker is a viable cheaper alternative though. Or just rip off the free trial versions.

When I've seen Purify bought, a developer downloaded the trial and built a list of all the problems he found and fixed using it. When he showed his manager how much pain and suffering the product could save it was an easy sell. (The hardest part was countering the "so everything's fixed already?" mentality.)

MEMPROF

[kijiki]

Its by one of the RHAD labs kids. Its basically just a GUI around bohem's garbage collector in leak-detector mode.

Its not purify (it really aims for leak detection, not all the other errors purify finds), but the efence + memprof combination gets you about 85% of purify's functionality.

It seems to handle threaded apps reasonably well, and C++ doesn't faze it. The only down side is that its hard to get running on non-x86 platforms.

Checker

[anaymouse]

Try Checker I think AX.25 pointed to some relevant information, but was moded has redundant for some odd reason.

mpatrol

[brianmed]

mpatrol is another tool to help with this.

It can:
- log your memory usage
- report on improper memory usage
- profile your memory usage
- work with your applications *without* re-linking (assuming your OS allows this)

The web page is at:

http://www.cbmamiga.demon.co.uk/mpatrol/

In addition, the author has excellent documentation. The pdf manual actually has a section that lists competing products and what they do.

http://www.cbmamiga.demon.co.uk/mpatrol/files/mp at rol.pdf