Slashdot Mirror

- automatic build process, generating a makefile, or at list a part of it (dependencies) if you want flexibility. And don't tell me you like telling which .h you need for every single .c / .cpp file you got... it's stupid, repetitive and SHOULD MUST BE automatized.

Which only works well if you're using a non-recursive build system. See Recursive Make Considered Harmful. Basically, in a traditional recursive make build, make doesn't really know all of the dependencies, so it can't find much parallelism at all and you wind up not really spawning that many additional threads. I didn't reread the article, so I don't remember at what point the parallelism stops working, but I seem to recall that it broke down at pretty low levels.

However, it's probably still pretty cool if your build is non-recursive. I haven't found many large builds that are, though so I don't really know.

For all non-Java development, unless mandated by a contract, I use GNU Emacs as my primary environment, with the GCC tool chain and GNU Make. This works amazingly well on all stages of development, but as others have said, keep things simple: try to avoid mazes of recursive makefiles, for example (read Peter Miller's Recursive Make Considered Harmful, then read it again.)

Your choice of tools will depend a bit on how cross-platform you need things to be. If you are developing now, or think you will be developing in the future, for multiple platforms and compilers, take that into account from the start and save yourself a lot of headache. Don't bother with AutoGen and friends unless you are distributing this to third parties: it's a PITA. It's great if you have absolutely no control over future systems, but otherwise there are better ways.

For Java development Eclipse is going to be your very good friend. I'm not a big fan of IDEs, but Eclipse is really nice insofar as it integrates with the Java language very well and can help you a lot. It's refactoring capabilities are worth the learning curve. Unfortunately I have not had a lot of luck getting ANT integration working, so I maintain a separate ant build file that I use for release builds in my projects.

For documentation purposes Doxygen is the thing to use. It has Emacs integration as well (written by yours truly). We use this to generate reference documentation, and then have Python scripts to massage the HTML output as we need. With its XML output you can use XSLT instead, though I haven't tried that. We need to produce multilingual documentation, and after many different attempts with tools have settled on LaTeX with a customized version of the Python macros. It works really well, if you can find doc writers who are comfortable writing with it. We are experimenting with Lyx to see how it integrates with our hand-coded documents. We use JavaDoc for Java reference documentation, though we could (should) migrate to Doxygen.

Other tools that are essential: Valgrind is a must use. I prefer it to Purify. Use Bugzilla for bug tracking: easy to set up and maintain. I recommend Perforce for your SCM. I'd avoid CVS. Give Subversion a look, but we've been happy with Perforce. Depends on how much you want to spend. We use AutoGen a lot to generate sources: very useful. And pick a scripting language you are all comfortable with and use it: Python is what we use, Perl works. It doesn't matter which one you use, as long as you're all comfortable with it. We have Python scripts that produce hourly summaries of Perforce activity, for example. We've tied our Bugzilla into Perforce... lots of things can be done.

SCons remembers the command line used to compile/build a given file, so it automatically figures out that it should rebuild that file if the command line arguments change. With Make it is very difficult to do that, so "make clean" is used much more often than it should be needed.

There are a couple other reasons "make clean" is used more often than it should:

• Generating proper dependencies from source is a pain. SCons does this automatically for all of the built-in builders (notably including C/C++), and it has infrastructure for making your user-supplied builders do the same.
• Most makefile setups are recursive. They should not be (see Recursive Make Considered Harmful for a good explanation). It's a pain to make non-recursive make, because the tools aren't set up for it. SCons is.

SCons is written in Python, and the SConstruct files it uses analagously to Makefiles are fundamentally Python scripts

...which is wonderful. The thing about make is that you have to be familiar with so many different syntaxes and APIs to do the simplest thing. There's the shell + make + m4 + autoconf's libraries (workarounds for non-portable shell utilities) + automake's libraries. It's a huge pain, because there's not a single place to look things up. You have to find the appropriate shell utility...then check in the autoconf/automake documentation to see if there are portability wrappers. scons is simpler; you can find most stuff with just their manpage, and possibly Python's documentation when you need to do actual programming in the build system. One (very simple) syntax. Two sources of API documentation.

The autoconf/automake system is nice in that it makes no assumptions about the user's system beyond make - everything is just generated to plain shell scripts and makefiles. But it's such a pain for the developer that it's not worth it. SCons assumes the user has a working Python installation, which makes everything more pleasant. Python provides the same level of functionality with the same interface on every platform.

This sounds like a specific application of the cutting stock problem.

Cutting stock is an interesting integer programming problem.

The airlines particularly, have famously huge integer programming problems.

From what I've heard from friends who are researching the airline problem, there are approximately 800 constraints, and 9 trillion variables.

Another famous integer programming problem is the the travelling salesman problem.

Integer programming is a pretty interesting topic. There's a good deal of theory behind it, but if you just want to model some systems there are plenty of software packages that let you do that too. The most accessible one is probably solverwhich comes with Microsoft Excel.

Unfortunately, the makefile creator most people use, automake, creates only recursive makefiles. Maybe a replacement like unsermake will get automake developers thinking about radical changes. I wouldn't mind seeing M4 go away, for one.

The single-Makefile approach is pretty well-defined in Peter Miller's classic paper Recursive Make Considered Harmful. The technique works well and is very reliable for simple build situations, but it has some scalability problems when you try to apply it to large, complex projects with a lot of variants to build.

The basic issue is that any solution that uses a single Makefile with a lot of included subsidiaries relies on a lot of ad-hoc macro conventions to pass values back and forth between the files. Again, this works fine, but the macro manipulation gets really tricky and not easily extensible when you have to add new build variations. You end up having to pull apart your macro conventions and gluing them back together over and over and over, adding more complexity each time.

You'll end up saving a lot of Makefile-hacking time if you go from the beginning with one of the many more modern build tools that are designed from the ground up with complex builds in mind.

Considering that the Travelling Salesman Problem is NP complete and affects almost any problem that involves delivering something to several destinations in an optimal fashion. A solution to NP problems ould have widespread ramifications in improving many aspects of businesses that involve deliveries (including the airline business now that I think about it).

Salesmen will certainly be happy when such an algorithm is found. For other classes of NP-complete problems, check this book.

From the interview:

...kbuild 2.5 builds a global Makefile from Makefile.in fragments in each directory then compiles and links only the sections of code that absolutely need to be recompiled

This is excellent, and I hope more open source projects start to go this way. It's been known for a while that recursive make is a bad idea because it's inaccurate. Naive recursive makefile structures tend to miss stuff that needs to be built/installed and fixing that problem (usually with ugly hacks like make dep) generally results in building stuff that doesn't need to be built.

What Keith describes is a nice solution that provides the benefits of recursive make without the problems: Use per-directory makefile fragments which can be maintained locally, but automatically generate a complete, tree-wide makefile that is actually used for the build.

There are tools other than make that provide more elegant solutions, but given that they never seem to catch on, I'm happy to see that someone is applying the tool we have (make) correctly, for once.

I'm looking forward to this one.

The Travelling Salesman Problem is very computationally intensive. Just picture hundreds (if not thousands) of destinations. Getting everyone to their destination in the most efficient manner is VERY computationally intensive. Take a look at the algorithms in the link above.

You're thinking of the issue of source file dependencies, which are something separate and are not addressed by CML2. There is a separate project to replace the current recursive make and kluged dependency analysis with something that works properly.

The Dish was based upon the events of 20th July 1969.

The true story involved not the Parkes tracking station but the little known Honeysuckle Creek tracking station. Honeysuckle was responsible transmitting the bulk of NASA's southern hemisphere communications during this period.

Complete details can be found here.

Note also that the Honeysuckle Creek station, dishes and all, was dismantled some years later (IIRC it's now farm paddock) while the Parkes station is still operational, hence the film set was based in Parkes.

Aegis seams as the prefect tool to use with XP.
But have anybody used it for XP?
I would like to know !

Knud

I think the RSA alogrithm is available for free for non-commercial use. Cryptext uses RSA, and is non-commercial.

The patent expiring means that any big company can use the alogrithm in their product.

Which means that, theoretically speaking, Microsoft could build in 'Privacy enhancements' using a (kerberos'd?) version of RSA. I'm not MS-bashing, just food for thought.

Take for example Aegis. Find me a tool like that for Windows which costs less then 1000$ per seat. Or compare the editing power of UNIX editors to the pitiful editor built into most IDEs - not to mention their brain dead build process (BTW, both Emacs and VI can and are commonly used as a UNIX "IDE" to great effect). It goes without saying that you can't replace your IDE's editor or build process, even if you are willing to pay.

Sure, UNIX developement tools are harder to learn (they are not harder to use). But one can hardly claim to be a serious professional and dismiss the UNIX tools as too hard to use. These tools simply assume a power user - one willing to invesrt the extra effort to get the (large) additional benefits. That's what being a professional programmer is all about.

• 4.1. A Single Makefile Is Too Big
  
• 4.2. A Single Makefile Is Unmaintainable
  
• 4.3. It's Too Hard To Write The Rules
  
• 4.4. I Only Want To Build My Little Bit
  
• 4.5. The Build Will Take Too Long
  
• 4.6. You'll Run Out Of Memory

cook home page

The same is true for all the programs they want to replace. At best, this competition will give some developer experience they can use for enhancing the standard tools. At worst, it will divert some free software talents towards enhancing and maintaining a little used set of alternative tools, rather than enhancing the tools used by the rest of the community. Most likely, someone will have wasted US$200.000.

• Tools listing: http://www.eccentrica.org/Mammon/toolkit.html
• For Java: http://www.pcug.org.au/~mayon/classcracker/

--

According to the article, the filtering software favored by the Ausie censors bans things with "names such as Pamela". Are they completely out of their head?!?

I guess they don't want to hear about the international space station project (one of the Astronauts is Lt. Col. Pamela Ann Melroy).

There goes Australian Women's Lib history, where Pamela Denoon was apparently a major player.

I wonder if it will also filter out info on the PAMELA Magnetic Spectrometer, scheduled for launch two years from now.

Do these censors have any idea how stupid they look when they suggest things like this.