ICFP 2004 Programming Contest Results

jnagra writes "The results of the 2004 ICFP Programming Contest have been announced. First place is to a program in Haskell, second place is to a program in C++, and the judges' prize is to a program in OCaml. The ICFP contest is an annual contest to promote functional languages (although programs in any language are accepted) and bestows on the winners unlimited bragging rights."

Comments

Visualizing the winners

[TheRealFoxFire]

For those of you who want to see the winners in action, the Formicidae simulator from Ant Wars can be used. It comes with a converter from the ICFP language to Ant Wars bytecode. To do so:

1. Grab a copy of Formicidae from the download page
2. Use the included convert.py on an ICFP .ant file:
convert.py <filename.ant>
3. Run formicidae.[sh|bat], and provide a world (ICFP maps included in the worlds/ subdirectory) and two .antc files
4. Check ICFP Mode
5. Enjoy the match

Interestingly, this year's winning ant is already beaten by some of the competitors on Ant Wars, due to the fact that some Ant Wars ants have more aggressive defensive tactics that wind up decimating Dunkosmiloolump's ants that too brazenly approach their ant hill.

Re:"Already?"

[TheRealFoxFire]

My point was rather that some Ant Wars ants, which have been created in the last week, not three months, defeat the winner which they had never seen.

I'm not trying to take away from the winning teams accomplishment, which is stupendous, but rather to point out that its interesting that there were no ants in competition which aggressively defended their bases.

Lisp results not very impressive

[GCP]

I'm not quite sure what to make of the underwhelming results of teams that use Lisp in these ICFP contests every year. Of course I can see that there are many ways in which the contest isn't a "fair" test of language against language. If one team has a dozen Inria guys whose full time job is OCaml development against another team with a single Lisp hobbyist, it isn't much of a fair fight.

It appears that winning depends more on choosing a good strategy than a good language and then implementing that strategy quickly and accurately. Choosing a winning strategy should be just as easy for a Lisp team as for anyone else, and helping you "discover" a good strategy is supposedly a strong point of Lisp. And as for implementing quickly and accurately, Lisp is said to have all sorts of advantages in that regard.

Even so, the number of teams that choose Lisp each year and the relatively poor showing of those teams implies to me that the amount of advantage Lisp provides is not as great as some (e.g., Paul Graham) would have us believe.

These contest problems are the sort of non-mainstream challenges that Lisp is supposed to be particularly good at, so I would expect more teams to choose Lisp to help them explore the problem and discover a winning strategy. Instead, Lisp appears to have, at best, average popularity among these programming language fans. I understand the overweighting in Haskell and OCaml given the name of the contest, but Lisp is roughly as functional as OCaml, so its lack of popularity puzzles me.

And of those who choose it, the results don't seem to imply that it gave them any advantage.

Yes, there are all sorts of ways in which the contest isn't a level playing field, but I'm still a bit puzzled at why the purported advantages of Lisp aren't showing up. Maybe they're real, but they don't appear to be very significant.

Re:Lisp results not very impressive

[brucehoult]

Even so, the number of teams that choose Lisp each year and the relatively poor showing of those teams implies to me that the amount of advantage Lisp provides is not as great as some (e.g., Paul Graham) would have us believe.

Yeah, that puzzles me as well.

Dylan is in essence a version of Lisp that happens to be written in a C/Pascal-like syntax instead of S-expressions. My friends and I have been doing pretty well using it, winning ICFP prizes in two of the last four years (2nd place in 2001 and Judge's Prize in 2003).

That's a single team's record.

There were 12 teams using Common Lisp and 9 teams using Scheme this year. I don't think we're any smarter than the other Lisp teams. In fact 7 of those 21 beat us this year, which is pretty much the same as our overall position in the field. So if our Dylan team is picking up prizes from time to time I don't know why other Lisp dialects aren't. It's a mystery.

Re:Lisp results not very impressive

[GCP]

I (think I) understand the theory behind the stronger typing in OCaml & Haskell vs. Lisp. It helps the compiler catch more bugs automatically.

It seems to me as though this would slow down the initial process of exploring the problem, as Paul Graham claims. Proponents claim that it makes up for that by making larger systems more reliable.

But these contest entries don't ever become large systems, and the extra slowdown initially wouldn't have time to pay off in the long run if the long run is only 24hrs (or 72hrs).

Likewise with Haskell's isolation of side effects into the monad ghetto. Again, the theory of this is that the inconvenience pays off over the long run in reliability, if I understand correctly.

I can understand the value of such features when creating avionics, nuclear power plant controls, or automated securities trading, but these contests aren't about reliability. They're about quick & dirty, do only what you have to do to score a short-term win. If a team ends up with a bug that doesn't impede their victory, then they still win. If the bug does impede their victory, then they lose, but it's hardly a costly loss, so they have little to lose by being more aggressive.

Since they have so little to lose, I'm surprised that the majority of winners aren't primarily highly-dynamic/questionable-safety languages like Perl and Lisp. And I'm surprised that C++ ever wins, except where speed matters (as was the case last year, I think.)

For the circumstances of this contest at least, it seems to imply that if you are good at figuring out smart approaches to a problem, it doesn't matter what language you use as long as you are VERY quick and reliable in expressing your intentions in that language, meaning so much experience with the language and its dev tools that you can write your code as fast as you can type.

Re:What happened to Dylan?

[brucehoult]

We (DylAntz) were there, we just seem to have had an "off" year, managing only 25th out of 87 in the lightning (24 hour) contest and 137th out of 361 in the main (72 hour) contest.

It would be nice to win every year, but actually I'm pretty happy with a record that goes:

2001: 2nd
2002: 35th
2003: Judge's Prize
2004: 137th

I think this year's contest results had very little to do with how good your chosen programming language is. Instead it was all about how well you could get around the limitations of the ant instruction set and devise strategies for your ants to follow.

Sure, you could make a compiler that gave you a language with loops and subroutines and (very limited) variables and turned that into ant code -- and we did that -- but in the end you had to have something smart you wanted to get your ants to do. We spent a lot of time on making out ants explore the world and gather food efficiently (including raiding the opponent's nest), but our strategy for defening our own nest turned out to be inadequate.

For some reason the results say we used Perl, not Dylan. I don't know why. It's true that we had a quick&dirty ant assembler written in Perl that some of us used to play around with strategies while we waited for the proper compiler (written in Dylan) to be ready, but that wasn't used for the final submission.

We submitted the following programs all written in Dylan:

- world simulator library
- very fast simulator (2 sec for 100,000 rounds)
- slow simulator with OpenGL interface for visualization
- high level compiler for ant brains

Congratulations to the winners and we're looking forward to next year's contest!