Visualising Code Structure in Large Projects?

TheMaccLads asks: "I've recently joined a new C++ project, and it's in a terrible state. There are 100-odd source directories, dozens of libraries, and a couple of dozen executables and DLLs. Some executables pull in (i.e. compile themselves) the occasional source file from a library, instead of using the libraries. My job is to port a subset to unix, but I need a tool to visualise all the relationships between directories, projects, libraries, and so on, because my brain will overheat soon otherwise. Preferably a tool that will do it by parsing the MS Dev studio projects and workspaces, but if I have to write it myself in Perl, I will! Anyone know of any tools? Or suggest an approach?"

Comments

Pen and Paper.

[spt]

Really! Just start drawing lines and boxes as you delve through manually. If you get something to do it automatically, you still won't have a good visualisation in your head.

Hmmm

[Lord Hugh Toppingham]

Software Emancipation's Discover product is pretty good at that, as is Takefive software Sniff++.

Theres other products out there too, All are expensive.

Doxygen

[brenfern]

I have used this, it is fantastic; it will work with your old C++ code straightoff, & also accepts javadoc-style comments. Handles the worst code elegantly. Draws pretty graphs for you. Does the bits of a programmer's job that really ought to be automated.

Re:Doxygen

[bitMonster]

I agree. It works wonders on 3rd party sources too, even when they did not use any of the supported commenting conventions.

Get it.

I wish it supported python, which is the other OO language I routinely use.

McCabe Visualization tools

[Jack Tanner]

McCabe & Associates makes some software that will automatically create a graph for visualizing a large software project, even one using mixed languages. Their marketing department renames the software every other week, but that's what they do. Arthur Watson (PhD, Computer Science, Princeton) did their most interesting research a few years ago, but has since left the company.

Their software will also help re-engineering and testing efforts: it'll tell you how complex your code is (and thus what parts of it are most likely to break), and it'll instrument your source code and show you what logic paths you've hit in your program during testing, and what code remains untested. It used to be pretty solid stuff (and pretty pricey!); I'd love to see some free/open software that does stuff like this.

Disclaimer: I'm a former intern.

CodeWright for mapping

[andaru]

If you are going to map out the project by hand, CodeWright from StarBase (formerly from Premia) is excellent.

It is the best editor I have seen yet (multiple language support, totally configurable, excellent tech support), and it is great for navigating large projects. It parses all of the files in your project as text (so you can browse code that does not compile), and is a good complement to Dev Studio's build-based browsing.

It integrates with Dev Studio, so the two editors and environments will update each other when you switch between them.

I probably shouldn't admit this on /., but at one job, I had a Windows box dedicated to running CodeWright, editing QNX code over Samba. It was actually worth the extra box. At another job, I was using it to edit files for Sun and DEC UNIX (no Samba this time), and it was still worth it to ftp the files back and forth, rather than use what I could find in the UNIX world at the time.

I know also that there is some really good code mapping software out there, but I can't give you any names off of the top of my head. Large sheets of paper really do the trick, though. I have seen people spend a lot of time with visio getting very little done, but I haven't used it myself, so don't listen to me and I'll shut up now.

My story

[Kz]

A few years ago, I found myself at a similar situation; I had to port a Bridge game from DOS to Mac. I had the full source code, but it was so old I couldn't make it compile. Even worse, I had never played Bridge!

I started playing with the executable, and learned the cards game (very boring! never played it again).

Then I wrote some utilites using .BAT files, the 'Find' command from DOS, and QBasic. Those utilities allowed me to type any function or global variable (there were lots of globals!!) and it would tell me in what file was defined, declared, and what functions used it (with a tree of callers)

It took me a couple of days to write the utilities, and proved really lifesaving

after that, I used paper. If there was a long function with lots of nested loops and goto's (it was C, but looked like bad basic); I would print it and tape the paper to the wall. Then just spend hours looking at it, pencil at hand, drawing arrows and notes over the code

Very slowly, I separated the engine from the UI and IO code.

Then I just wrote a new application on the mac, that called the old engine code. Of course the game gained a lot of mouse manipulability, sound effects and even a video intro (without QuickTime, of course)

In short, first craft your tools, then use them. Spend lots of time just looking at the code. Slowly the hate will melt into pity for it, then your job will be to liberate it!

Some other solution

[fm6]

I remember seeing some other solution somewhere that might be applicable. Can't remember what it was called. No wait, I'm thinking of a Thai cookbook. Never mind.

Am I missing something?

[fm6]

Doxygen seem to be a sort of Javadoc clone. The main difference is that it's not limited to Java and it supports more output formats. Nice if you have an API you want to document with a minimum of work, but that's hardly a "visualization tool".

Unless I overlooked something, the only diagrams Doxygen draws are class hierarchies. Unless you have a lot of classes, I can't see bothering with a whole big app just for that.

Doxygen sounds like it's handy for generating API docs with a minimum of effort, but pretty useless for analyzing mysterious legacy code.

Re:Am I missing something?

[bitMonster]

Yes.

The class hierarchies are very sophisticated. For example, you can see a page with a summary of all the methods for a derived class showing where all of these methods are inherited from. The same view shows all the attributes on each method, such as virtual, inline, static, etc.

There is also a whole section generated on include file hierarchies. You can see quickly where everything is declared without doing a bunch of greps to try to see how things fit together.

Finally, the hyperlinking scheme is simply inspired. For example, it will even take you to the exact line of a header file that corresponds to a particular method (complete with syntax highlighting).

Go look at it. It's not going to reverse engineer design documentation from the intent of the code, but it gives you a huge advantage over a big pile of C++ files. (If you think you are going to find a tool that does the former, you're smoking something.)

Post-It notes and a DryErase Board, Bake at 350

[QuietRiot]

5 People
3 Afternoons
1 Huge Pile of Code
2 Large White-boards
3 small little cubes of those MultiColored post-ITs
2 Handfuls of assorted colors DryErase markers

Start by pouring all the ingredents into a medium sized classroom-type-room with lots of chairs and a small assortment of refreshments. (Be sure to wash off the white board.) Tell all the people what you are trying to do and tell them they will have to help you out for at least one afternoon over the next couple of days. [Whatever. Intimate time with code. They'll learn something. You'll get to talk to eachother.] Tell some they have to stay, others they'll have to help you tomorrow, etc. Look over the code and decide which portion you'd like to work with today. Isn't it pretty? Now - by applying the markers and the PostITs to the White-board, carefully extract the useful parts of the code, leaving the nasty, hairy choke behind. Go for the structure. Go for the connections. Dispose what is leftover. Take a high resolution picture. Go home and get some sleep.

Repeat the above for each piece of the program you'd like to work on for each particular day. After you have extracted all the good, and none of the bad, combine the extract with your wonderful programming skills, the sarcastic cheers from your friends in nearby cubicles, and big high-resolution printouts of your photography work in a CPU Unit Processor, blending until firm. Chill, Serve and Enjoy!

Re:Post-It notes and a DryErase Board, Bake at 350

[flacco]

You left a couple things out:

First, you forgot to include the disgruntled Evil Janitor in your list of people, for a total of 6.

Insert the following step before "Repeat the above...":

"Evil Janitor swaps around pretty colored paper squares in the middle of the night."

What I want...

[ptomblin]

...is something like etags, but which will show who calls the function/method I'm looking at now. Etags in vim is great for tunneling down to the definition of the functions that I'm calling, but not so good for going "up". So far I mostly end up using "find . -type f -name \*.[Chc] -print0 | xargs -0 grep foo", which is time consuming and annoying.

Re:What I want...

[MemRaven]

Well, I've been using Visual SlickEdit (mentioned elsewhere in the thread) for a couple of years now as virtually my exclusive editor, and it runs on almost every platform I care about (Intel Linux, Windows, Sparc Solaris, FreeBSD, etc.). Its tagging goes way beyond what etags does, and has built-in reference analysis. Once you've built your tag database (which automatically updates when you edit a file in slickedit), you can choose to view references. You type in a tag (such as a method or variable name), it'll pop up a list of potentially matching tags (so if you have a lot of things named getName(), for example, it'll let you choose which one), and then it will display all code which references that tag. You can then double click on a code region to go straight to the file and area where that exists (although it shows you a few lines at a time normally). I think there's actually a keyboard shortcut to do that for a particular tag (just like you can automatically go to the declaration of a tag)

Although this seems like a convoluted feature, it's extremely useful for doing any kind of refactoring work where you want immediate feedback analysis. You get to see immediately where everything is used, and it's got other tools which help with that as well (like the class browser and the like). (It also has Visual Basic-style autocomplete, which I absolutely love, because on large projects I never remember exactly what everything's named).

Admittedly, you might not want to change editors just for this feature, but you might want to have it in your arsenal of tools for when you do need/want it.

no real shortcuts

[markj02]

If you really need to understand the code, you will just have to spend the time. Diagrams and tools may help a little, but ultimately, there are only so many bits you can get into your head per hour.

Source Insight... (and 30 day free eval)

[MeerCat]

You say "DLLs" so I'm guessing PC rather than Linux... Source Dynamics produce Source Insight which is a groovy C++ editor that scans all your code, holds it all in a metabase, and while you're perusing it runs a lazy thread in a seperate window to show you more details of whatever is under the cursor. It's not a C++ visualisation tools (it has a speedbar outline of the current file) but is very good at helping you jump navigate thru large amounts of source code.

The bonus is that it runs very fast, is useful as an editor, and knows enough about all your code to do smart things like smart-gloabl-rename, find all references to symbols, etc.

They do a 30 day free eval too...

T