Tools for Analyzing C++ Class Code Generation?

Milo_Mindbender submits this query: "I've got a midsize Linux project which uses a lot of STL and other C++ template code. Even considering this, I end up with a lot bigger text (generated code) segment than expected. I know the information about the amount of code generated for each class is in the objdump, but prying it out by hand is a problem when you get five line long template invocations and hundreds of methods to wade through. Can anyone can recommend some tools that analyze binary or objdump output and summarize the amount of code generated for each class, including each unique template or STL class?"

Comments

This article should have been called

[GigsVT]

"Why you shouldn't use OOP for anything important."

Re:This article should have been called

[be-fan]

Too bad that C++ templates have nothing to do with OOP, but rather procedural programming. And while they have lots of problems (like code bloat) they are also phenomenally powerful for certain things (generic data structures). Even if you write nothing about procedural code, you could benifet from templates.

Re:This article should have been called

[be-fan]

Of course, that second "procedural" should be "generic" :)

Re:This article should have been called

[mattgreen]

It'd be interesting to hear what you'd use, given that you fail to grasp that a template has nothing to do with OOP.

cat got my tongue

[sanity_slipping]

I want to write an insanely clever and informative post and answer Milo's question in a way which would cause the calculating hordes of Slashdot moderaters to moderate my post to Score:5, Astounding!

Unfortunately, I've never used C++ before in my life. (except for a "Hello World!" back in High School)

Me Too!

[kzadot]

Our large corporation also has a project, a client has requested that we implement BubbleSort, QuickSort, ShellSort, and one other sort that we learnt in cl^H^H^H^H^H^H^H^H^H err, discussed in a business meeting. The err, client also requests a 1 page analysis comparing and contrasting the differences between the searches.

Can code generation do this? It could really increase my chances of a good mar^H^H^H err profit yeah.

Re:Me Too!

[Monkelectric]

Did you goto UC Riverside to? :) I swear I was given that *exact* assignment. But thats why its funny eh? :)

Very cheap visualization tool

[Mr. Darl McBride]

There's a very cheap STL project visualization tool, made by a company called "Prince." Their fine visualization tools are available in the pasta aisle of your favorite grocer. Simply bring water to boiling, stir in the Visualization Tool, and soon you'll have a representation of the calling methods employed by the average STL application.

For each additional programmer involved in the project, double the contents by stirring in a random different kind of Visualization Tool.

If you're targeting multiple architectures with different STL implementations, the Visualization Tool can be enhanced by having the lead of each platform boil their own pot of Visualization Tool, and then throwing the still-boiling contents at the other chefs^W developers.

If you aren't clear as to where I'm headed, I'll next recommend a more painful exercise: Go ahead and try debugging somebody else's template-heavy code.

Uhhh, perl or python?

[ComputerSlicer23]

Uhhh, use your favorite scripting language. Doing the demangle might be a bit tricky, but nm or obj dump should do either of those for you. Then just parse the class names.

Use a map or a hash. Oh, and lastly, move all the code in headers out into the .cxx/.cpp file, your code will compile faster, and probably run nearly as fast, with the exception of a handful of performance critical classes (finding them is important).

Kirby

Re:Uhhh, perl or python?

[p3d0]

To demangle, use c++filt.

Re:Uhhh, perl or python?

[Chris_Jefferson]

How much do you use C++? :) g++ is quite intellegant about what is / isn't worth inlining. Also template code MUST go in headers, because it has to be included in every file that makes use of it, because templates are retarded... :(

Re:Uhhh, perl or python?

[ComputerSlicer23]

I've written C++ for a living for about 5 years, and I'm reasonable well read about the subject.... I didn't consider that it was all templated code (I was thinking about cutting down the binary size, I rarely use templates). I use g++ for a living, every day of the week. It might be pretty smart about it, but I all I know is that it takes about 5 times as long to compile the code using 2.96 as when you leave things in the header. The binaries are twice the size, and if you do it while leaving optimization on, you'll really, really regret it. Compiling will become a get up and go get lunch on a rebuild of a decent sized project. I optimize my code for development time, not for run time. I'm pretty expensive, computers are cheap.... :-) I found that my reference counting base class was the only thing that was important enough to leave in the headers. Everything else was a waste of compile time.

As a general rule, I've found that if you are using that many templates, you are doing it "wrong". I very rarely use templates for anything but storage classes (which means I almost always use the STL ones). Templates are pretty useful, but in a lot of cases, you can design templates as inheritance, and make your stuff more OO like (Bjarne even mentions that most people don't use inheritance with their templates, as a tool they work well hand in hand). Heck, I could even implement most of the STL as something you have to inherit from and no templates (it'd be awkward to use, but it'd work).

Oh, I'd love to upgrade compilers, but I've got several problems. One of which is, I'll have to rework a lot of my code base (the std namespace gave me fits last time I tried). Oh, and my release platform is RH6.2, so until I upgrade my production machine, I'm using a 2.91 compiler to deploy. So it's not terrible source compatible with g++ 3.2.

Kirby

no tools but...

[gears5665]

For others learning the language I found these books helpful for C++:

Effective C++ by Meyers
Effective STL by Meyers
and STL C++ Standard Library by Josuttis

Nope

[joto]

I've got a midsize Linux project which uses a lot of STL and other C++ template code. Even considering this, I end up with a lot bigger text (generated code) segment than expected.

Huh? What did you expect? Templates lead to bloat per definition. That's what they do: a lot of binary code is generated by a small amount of typing. If bloat becomes a problem: redesign.

I know the information about the amount of code generated for each class is in the objdump, but prying it out by hand is a problem when you get five line long template invocations and hundreds of methods to wade through. Can anyone can recommend some tools that analyze binary or objdump output and summarize the amount of code generated for each class, including each unique template or STL class?

I wish I did. But most likely, I think you are fucked. Without an understanding of the code, it's hard to debug template problems. Find out where the bloat is by analysing the source code, not the binary.

OpenC++

[angel'o'sphere]

Search at source forge for Open C++ or OpenCPP, not sure how it is written.

That is a programmable source to source transformer for C++.

It spits out one singel file of c++ for every compiled C++ unit, removing the includes, by expanding them into the output.

That way yo can source level debug the processed c++ code, not exactly what you want but likely more powerfull in the long run anyway.

Your request varies far to heavy from compiler to compiler.

angel'o'sphere

John Lakos

[sohp]

If you do not already have a copy of Large-Scale C++ Software Design by John Lakos, get it. I realize this isn't a tool per se, but he has a terrific section in chapter 10 on Using C++ Templates in Large Projects.

Re:John Lakos

[p3d0]

I second that. Great book. I don't use templates much so I can't speak to that particular chapter, but his other advice is sound.

Turn off inlining

[Anonymous Coward]

This is more of a suggestion than a debugging tool but...

You didn't say which compiler you were using but, for example, with a recent g++, try turning off some of the inlining and turn on -frepo. This will make sure that, for example, there is only ever one copy of map::find.

I don't know; maybe the latest g++ does a good job of this already.

Re:Turn off inlining

[Anonymous Coward]

hmm, that should have said map, less than, string, int, greater than :: find. How do you prevent it from looking for tags ?

Re:Turn off inlining

[VE3MTM]

Also, try using the compiler flag -Os (Optimize for size) with GCC/G++

Reducing template bloat

[mattgreen]

Enough uninformed comments already. Most STL code bloat can be controlled pretty easily. What you do is make a wrapper over an STL class. For example if you're using std::vector you'd make a Vector class:

template<typename T>
class Vector
{
public:
//ctors as necessary
Vector();

T get(size_t i)
{
return static_cast<T>(vec[i]);
}

void insert(T item)
{
vec.push_back(static_cast<void*>(item));
}

pri vate:
std::vector<void*> vec;
};

You may wish to provide the exact functionality of the container you're wrapping, or you may wish to change the semantics. It's up to you.

It is kinda ugly for what it does but it works. Only one instantiation of std::vector is made, it is void*. Putting the member function definitions in the header file ensures they will be inlined if necessary.

I think Scott Meyers came up with this tip first.

Re:Reducing template bloat

[saurik]

If your data type really can be static_cast'd to a void * then there's probably a much better solution to this problem in almost 90% of the cases: it's called a "better compiler". I don't know what compiler you are using, but Visual Studio .NET finds functions whose assembly code generated the same and merges them (or whatever you want to call it: links all usages to the same implementation) at link time.

So if you have a vector of int * and a vector of long * and a vector of MyObject * and a vector of unsigned long they should all generate the same assembly code and you should only end up with a single set of these in your resulting binary.

Re:Reducing template bloat

[mattgreen]

News to me. The Effective C++ books are somewhat dated, but I hadn't heard of compilers being able to do this.

Re:Reducing template bloat

[angel'o'sphere]

All modern C++ compilers do this, but at LINK time not at compile time. So the *.obj files are still big.
Also, probably gcc doesn't do it, no idea.
angel'o'sphere

Re:Reducing template bloat

[Daleks]

I think you mean

T get(size_t i)
{
return *static_cast<T*>(vec[i]);
}

void insert(T item)
{
vec.push_back(static_cast<void*>(new T(item)));
}

You also need a destructor that walks the vector and calls delete() on each pointer-to-void.

Re:Reducing template bloat

[uid8472]

If your data type really can be static_cast'd to a void * then there's probably a much better solution to this problem in almost 90% of the cases: it's called a "better compiler".

Alternately, it's called a vector implementation that has a partial specialization for pointers, like the example in Stroustrup.

Re:Reducing template bloat

[Ardias]

You also need a destructor that walks the vector and calls delete() on each pointer-to-void.

Calling delete or delete[] on a void pointer is not safe. A void pointer is opaque, and so how would the compiler know which delete function to call? Each class can have its own delete and delete [] operators. You can get around this problem by casting the void pointer to the original type before deleting it.

If you don't have delete and delete [] operators for your classes, but always use the global delete and delete [] operators, then this is less of a problem. But, you still have the problem of calling the correct destructor - which is not available through a void pointer.

The STL Standard requires that each container destruct its elements when the container is destroyed, or the element is removed. If the container elements are objects, they are safely destructed. If they are primitives or pointers, then they have no destructors anyway.

If you need a container of pointers, then how about having a container of smart pointers instead? The smart pointers are objects and get destructed automatically. When it gets destructed, it can delete the object it points to, decrement a ref-count, or do whatever else it supposed to. A nice side effect is that you no longer need the unsafe cast to a void pointer.

Re:Reducing template bloat

[Ed Avis]

Huh? Any half-decent implementation of the standard library will already have a specialization of vector for pointer types so that all pointer types use the same object code. And things that aren't pointers cannot safely (and portably) be cast to void* and back again.

Re:Reducing template bloat

[Gauchito]

He can cast each void * to T before deleting. Ala:

delete static_cast <T *> (vec[i]);

Re:Reducing template bloat

[lkaos]

Are you retarded?

You'll just be creating multiple instances of Vector for each type. You save nothing. The STL classes are not that large at all. Most of the vector functions are really small and by the time you wrap them all you'll have the same size wrapper class.

Re:Reducing template bloat [Mod Parent Up Please]

[mhotas]

Yes it was Scott Meyers and it works great!

Get a better linker.

[beerman2k]

...

{} - The empty sig

radical idea and shameless plug :-)

[bmac]

Having used C++ templates extensively, I can say that compiler-generated code is a bad idea. A compiler should generate assembly, not other code. That said, I really like the *idea* of C++ templates, yet they fail in a number of subtle ways, not the least of which is that if the implementer of the STL construct screws up, you're screwed. Personally, I rather like keeping my own classes (containers and all) under my own control, so, when (not if) there's a bug, I can put in my own debug statements to track it down.

Note that C++ is not the only language that screws up the manifestation of so-called generics. Java and C# don't even have them (yet), and my experience with software of any kind of complexity that has major features added after the initial design and implementation is that they are bug-ridden POSs (and that ain't point-of-sale :-)

The problem (IMO) is that there should be two phases between the translation of the program description to the final executable. The first phase should be a code generation phase where meta-level programming constructs are expanded into their final program description, and then, two, your standard compilation phase. C++ already jumbles the two and what you get is - guess what - jumbled. If the first phase produced specific source *files* in your project then you would have greater control.

Now for the blatant self-promotion. I am about to release a nifty piece of shareware that will generate such source files for you. While based on relatively simple text replacement, it gives the user the ability to generate collection classes for C#, Java, C, and C++, and (arg!) VB in a way more powerful than C++ templates, which it is loosely based upon.

The purpose for my writing the software was to automatically generate type-safe containers in C#, but with the ability to extend each container instance with its own functionality (not dissimilar in the end result from subclassing). Basically, it works as a parameterized code definition mechanism and the proof that it works is that I am using six auto-generated collection classes in the software itself (who else here likes the work of M.C.Escher?).

Anyhow, in answer to the initial question, your problem will never be gotten under control until you write your own containers (thereby permitting testing & debugging like any other code you write).

For any /.'ers who would like such a tool (I haven't even named it yet), keep a look out in my journal for the big announcement.

Thanks slashdot, I now feel like a dirty whore. But just like a *good* whore, I *am* providing a much-needed service :-)

Peace & blessings,
bmac

STL: that bad?

[cronie]

Just wanted to bring your kind attention to this amazing fact: everyone tries to handle STL code bloat, is looking for a (inexistent) STL debugging tool, suggests to completely re-design compiling principles just to make STL compile a bit better, writes books about good STL coding (how to make your binary 3 times smaller), rends the air by saying "do you know what 'S' means in STL????" (one of slashdot postings)...

So why use it at all? Just because it is standard? Then go program with Fortran or Ada, they are pretty standardized, yet almost completely useless/impractical. And so STL.

(Ah... forgot this: IMO, IMO, IMHO, IMVHO, IMVVHO)

Huh? No?

Re:STL: that bad?

[mattgreen]

I'll take STL containers over anyone's home-brewed containers any day. Do your containers work with the functional include file? Do they support forward and reverse iteration? Do they even support iterators? Probably not. STL has consistant semantics. If you're the type of person that doesn't want to learn anything new then you don't have to use them.

Re:STL: that bad?

[HuguesT]

Maybe aerospace software engineers have something to say about Ada not being practical. Probably similar to what the (real) engineers would say about your comment about FORTRAN. The fact is most engineering/scientific software (think PDE solvers, etc) is available and maintained in FORTRAN. There is no real alternative to BLAS (Basic Linear Algebra System) for example. It's often free too.

STL is incredibly useful, but like many powerful tools it is not 100% intuitive to use effectively at first go. It is easy to overuse templates (Hey, I can support ALL TYPES in my library, yay!) and not all compilers are optimized to remove redundant code, but such compilers are becoming available.

Like you say though, if you don't like it you don't have to use it, or you can use it more sparingly.

Re:STL: that bad?

[cronie]

I'm definitely not that kind of person :)

I'm afraid STL is responsible for discrediting the C++ language and for the fact that C is still in wide use novadays, whereas it should have been replaced with its successor, C++. In C vs C++ wars people usually argue just around efficiency. It is obvious (to me at least) that the C++ language itself with good OO design can produce sound code. Only when it comes to STL and templates in general you easily loose control over code generation and also source code portability.

So my point is that even if STL is an elegant and consistent tool, still the outcome falls short of today's requirements. Nobody would allow you to write OS kernels, DB servers or other time/size critical stuff with STL. Likewise, you would never write such things with Perl or Python.

Re:STL: that bad?

[cronie]

...and not all compilers are optimized to remove redundant code, but such compilers are becoming available.

This reminds me the situation with Java: Sun promised to its army of Java fans that there would be hardware Java VM by 2000. Sun failed to implement Java processor but the trick worked and Java is still popular.

So where are those compilers?

Re:STL: that bad?

[bmac]

Yup, STL is *partly* responsible for C++ being mostly worthless, but let's not forget the other big nail in C++'s coffin: multiple-inheritance. The ol' diamond of death means compiler-determined (and not in any kind of spec, that I know of, tho I may be wrong) implementation, which is an auto-no-go in my book. Then again, I've *never* seen a practical reason to use multiple inheritance. Most books just use some assinine example like animals, mammals, and cows. WTF?! I use C because it's deterministic, and the implementation offers no surprises, except for bugs in the linked libs, of course :-)

Even in C# (which I use for rad development as a transition to straight C + win32/bsd), the Hashtable collection is totally fscked. After a certain number of elements it just goes to the weeds. Totally ridiculous. Some of the bugs in the *$2000* .NET IDE are so absurd that you wonder if Micro$oft isn't try to screw developers from being able to write sophisticated programs at all.

Oh well, crappy software means I still have an market :-)

Peace & Blessings,
bmac

Templates and OOP

[Ardias]

Yes, you can use simple templates with procedural programming. In that sense, knowledge of OOP is not needed to use templates. But, there are ways in which templates make OOP easier.

Templates can be combined nicely with several design patterns, such as observer, visitor, factory, and iterator. Template base classes for each of these design patterns can provide the common behaviors and implementations of these design patterns.

In addition to augmenting traditional OOP, templated classes can still have a lot of OOP-like properties. The classic examples here are in the STL. If we list the basic features of OOP, our list would include: Abstraction, Polymorphism, Inheritance, and Encapsulation.

The STL classes provide abstract interfaces into containers, algorithms, and iterators. The functors do provide a type of polymorphism through both templates. Several containers provide polymorphism through operator overloading. You can write classes to inherit from C++ containers and iterators, but that has various problems and issues with it. And they encapsulate the implementation details quite well. (e.g. - You don't have to worry about the mechanics of balancing red-black trees when you use std::map and std::set.)

Specialization

[unsinged int]

Many of the STL classes do this already...it's called pointer specialization.

Switch to Visual Studio

[Anonymous Coward]

I think Visual Studio .NET includes a tool that does exactly what you want. Of course, once you've got it, you'll probably be too busy porting your program to C# to take advantage of the full power of the .NET framework, but its there if you need it.

Radical idea: Implementation versus specification

[Frans Faase]

I totally agree with the poster. C++ (and most other programming languages) are implementation languages. Most programmers are not aware of the fact that the main reason why we use implementation languages lies in the fact that we have to produce highly optimized code to make computers perform. Specification languages are hardly used, because we lack the proper tools to transform (formal) specification into implementation code. The reason behind this is that much of the engineering in "software engineering" lies in choosing the right implementation, e.g. it is not (yet) possible to automatically convert high-level specifications into executable code. I think it is really sad that after more than twenty years of software engineering, we are still working like craftsman, and not like engineers. In this sense software engineering hardly can be called an engineering science. For some more ideas, read The Art of Programming.

Where are those compilers?

[Anonymous Coward]

A lot of them are popping up in the embedded world (A must if you are using C++ in small memory spaces).

These guys are doing a bit more than just template reduction here

You're right!!!

[Nicolay77]

These is the point where teory and practice met. And in theory STL does everything right and good once for all. But in practice there is MFC, Qt, wxWin, and so on for real programs, because STL is still a teoric tool unless you are the writer of the STL itself and know how to debug templates.

I believe that mastering templates is more complex and long than mastering everything else in the C++ language. And that's their main pitfall, despite being absolutely powerfull and even turing complete (just see some entries in ioccc).

Yes, but...

[Nicolay77]

I've seen template bloat in a lot of places, but MI bloat?

Just because something can be done in the language doesn't mean everybody just goes and uses it. You're right that MI is too prone to mistakes, and it can be replaced most of the time by java-like interfaces.

I use C++ because is deterministic, can be used to really big proyects, when it runs is fast as hell (can be faster than C when using inline functions), and because I don't use every shitty feature of the language just because it's there. The right tool for the job can be a subset of C++.

Re:Yes, but...

[bmac]

Ah, grasshoppa, you understand :-)

I'd still like a clean C++ compiler that
doesn't include all the cruft necessary
to implement templates & MI, tho. I think
that would streamline things even more.
And yes, I do desire the inline directive
for my C code. I mean, c'mon, it has to
be pretty easy, right, especially if they
can do all the MI and template crap, what's
a little parse tree substitution and one
more keyword :-)

Peace & Blessings,
bmac
www.mihr.com -- True peace & happiness
are just a wish away...

Re:Yes, but...

[cronie]

Actually I'd leave MI but without virtual bases, which cause this "MI bloat". I do use MI sometimes and I can't convince myself that I could have avoided it.

I'd also leave templates, they are useful and potentially powerful, but should be used with great care - that's it.

He-he, so your circumcised compiler gets back to life, almost.

Just don't use STL, and leave the compiler alone :)