Optimizing Development For Fun

chromatic writes "Geoff Broadwell has written an analysis of optimizing an open source project for fun, specifically the Pugs project. Broadwell argues that making development fun and easy leads to higher quality code and a faster velocity of development, even when implementing a frivolous project (a toy Perl 6 interpreter) in an uncommon language (Haskell). The Pugs leader, Autrijus Tang, will speak about both Pugs and Haskell at EuroOSCON."

Comments

Want to make dev fun?

[Anonymous Coward]

Use Python!

Seriously - we use it except in a few places where C/C++ is a better fit for interfacing with DirectX. The results? People are having real fun and getting a ton done. We couldn't be happier.

Re:Want to make dev fun?

[Anonymous Coward]

Except that this article is about people developing in Haskell - a language that's every bit as fun to use as Python (it even has optional syntactic whitespace), but compiles to native code (so it's faster) and infers types at compile-time (so it's robuster).

Think Pugs is cool? Look at this. These guys have implemented a complete transactional file system - looks like a Unix FS, but provides snapshots, unlimited undo, true copy-on-write handling of links (including link cycles), and perfect concurrency without requiring any OS-level threading. In just 540 lines of Haskell.

Give it a try. The learning curve's steep at first - it's got some concepts you simply won't have encountered anywhere else. But if you're smart enough to have dropped C++ in favour of Python, you're smart enough to handle Haskell.

Re:Want to make dev fun?

[jonastullus]

i agree that python has a tendency (at least for me) to cause relaxation and make even otherwise boring tasks fun due to its extremely readable nature and good balance of write-once vs. read-often.

and also, c++ is not such a hard competition when "having fun" is on your agenda. every single time that i sit down to program c++ i end up heavily frustrated at some point due to wrong "#include" order, some whacko unreadable compiler error or simply a hard to find segfault (most often due to forgotten pointer initialization, *will i ever learn*)...

yet, python tends to have its own set of problems, mainly stemming from the fact that NO type checking is done at startup time and so you have to start your program again and again, everytime only finding/fixing one small typo! certainly most of this problem can be eliminated by good unit testing practices, but a little more help from the interpreter/compiler would still be nice at times.

my current craze is for languages with type inference (standard ml, ocaml, haskell), which for me combine the cozy compiler assistance of typed languages with many/most of the liberties of dynamically types languages like ruby, python, perl, lisp, scheme...

well, except for restrictive languages like c++ and java (i don't know cobol personally ;-), programming can be fun in any of the above languages (well, maybe not perl, but that's a question of personal taste).

Re:Want to make dev fun?

[jonastullus]

## I think you underestimate the ability of people to write bad code

but i don't.

good python code is extremely readable, and medium python code is a pleasure to WRITE, because many things just seem to solve themselves.

BUT, writing big software or maintaining medium-to-crappy python code is far worse than many of the alternatives (except perl, of course)... this is exactly why i have become such a big fan of type inference, not to speak of garbage collection which i had my doubts about in the past, but fully embrace nowadays.

python's shady object-orientation makes many things harder than they need be (compare ruby).

• lacking private/public flags for methods
  
• ugly "__something__" pseudo-access-control
  
• implicit creation of attributes on write-access (*hello typos*)
  
• not so beautiful handling of "method not implemented" functionality
  
• incessant and visually displeasing explicit "self." everywhere
  
• lacking abstract/virtual classes, interfaces
  
• too frequent "necessity" of metaclass-hacks (although the possibility is very nice)
  
• verbose attribute initialization in constructors
  
• the evil "param = []" default value pitfall
  

and other defects with python's object system have led me to believe that it is not an ideal language for big-scale OO development.

yet, for rapid prototyping i know few languages that allow the programmer such unhindered experimentation and the *possibility* of resulting readable code. also for applications of a few thousand lines, written by a few people it can actually create a certain fun factor!

Re:Want to make dev fun?

[Fahrenheit 450]

Sure, if you want an uglier, single paradigm language that's harder to maintain, it's great.

Uglier is in the eye of the beholder. Hell all I hear about Python is how clear and clean it is, but every time I look at it I can't help but think how ugly it is, with all of it's __some_crap_or_another__ and whatnot. Sure, it's nicer than, say, C++, but that's a bit like saying someone is cuter than Karl Malden... Bottom line, saying something is uglier should mean little to anyone who isn't you -- we all like and dislike different things for different reasons.

As for harder to maintain, I suppose you have some evidence to back up this assertion? If you do, I'd love to see it...

The "lines of code" argument is tired, old and bullshit, frankly. Anyone can write a minimal line version of practically anything, but that says nothing for maintainability, readinability or anything else.

I pretty much agree with you on the first point. I won't say it's BS, but I do believe that it's overhyped. The part you may or may not be missing is that it's not the ability to squeeze tokens out of an algorithm that's important. It's the ability to say a lot, and clearly, with just a few words that's important (a subtle but real distinction). This was touched on a bit by Simon Booth and Simon Jones in their paper Are Ours Really Smaller Than Theirs? It's the succinctness that they talk about that is important, as it should lead to fewer errors, and actually aid maintainability and readability as it allows a person to keep more of the program in their head. Of course I don't know if it actually does this -- I'd love to see some follow on research...

Someone's written a 3 line RSA utility in perl, but have you taken a look

Taken a look at it? Hell, I own the t-shirt!

Avoid Recursion

[Dante Shamest]

Things to Avoid: The Haskell Wiki

Mod story +5 Insightful

[Spy Hunter]

Most insightful article about software development I've ever read. Every open source project could learn a thing or two here, and closed-source commercial products could learn a bit too.

IMHO this philosophy could go a *lot* farther too. We should be building these types of concepts into our software development tools (not just source control but IDEs and compilers and even languages). It should be as easy as possible for users to get the source, build it, modify it, and submit their changes. Ideally as easy as editing a Wiki. Though the inherent complexity of software means that Wiki simplicity will probably never be reached, we could certainly do a *lot* better than we do now.

In an open source project the ease of the process of getting, compiling, modifying, and submitting changes to the source is directly related to the number of new contributors joining the project, which is directly related to the rate of improvement. Traditional software development tools have far too many pitfalls and require far too much know-how for casual users. The process of contributing to open-source projects could and should be a lot more automatic and foolproof, because attracting contributors is the single most important thing an open source project can do to improve itself.

Re:Mod story +5 Insightful

[Spy Hunter]

People depend on software to get their work done. Broken software can mess up your data. Malicious software can do bad things, like give your credit card number to Russian gangsters.

People shouldn't depend on software under active development; only released software. Obviously before releasing a piece of software some quality assurance work will have to be done. This needs a different process than normal development work; it can be done on a branch of the code. This problem is largely solved by existing large OSS projects. By adding more contributors more errors will be introduced but more errors will be fixed, and overall more useful code will be produced.

If you take the total number of people in the world who are interested in and capable of doing OSS programming, and divide by the number of OSS projects, the result is a number close to 1.

And why is that? Because OSS programming is hard to do. It's very hard to jump in and immediately contribute to a project that you use. If we make it easier, more people will do it. You could have made a similar argument against Wikipedia before it started. Who wants to write an encyclopedia article? It turns out that everybody does, if it's easy enough. Making it easy is the key.

Re:Mod story +5 Insightful

[l00k]

If you take the total number of people in the world who are interested in and capable of doing OSS programming, and divide by the number of OSS projects, the result is a number close to 1. This is why most OSS projects have a single author. Imagining that "only a couple of people" have the privileges to fix a bug is actually optimistic -- the most likely case is that only one person is interested.

If you took the total number of people who contribute to the Wikipedia, and divide by the total number of Wikipedia articles, you get a result much less than 1--0.6. So, how does 0.6 of a person write an article? They don't. Groups of people work on articles not as individuals, but as a community. And when they have added their bit to that article, they move to another one.

Good points

[Anonymous Coward]

Form the article:

* Embrace anarchy. One of the key realizations of modern Internet projects (the oft-quoted Web 2.0) is that on the whole, your users can be trusted.
* Avoid deadlocks. There should be nothing blocking a programmer from committing his code.
* Cast committer rights far and wide. A central core committer group is necessarily slower than allowing every developer to commit as desired.

This all boils down to one thing: Open the CVS access for everybody. And that is exactly what I have come to hate about the GNOME project: Development is essentially closed because every piece of code needs to be reviewed and there are not enough reviewers. I often wondered whether these guys really believe in Open Source. Is KDE development equally restrictive and boring?

Let's run with this idea a little

[wyoung76]

If we use wikis as a front-end to the cvs, and by default mark all modules with a simple security level (no disk access, etc), and wrap each thing with unit tests for basic error capturing, and had incremental compilation, then we might have an easier and more collaborative way of coding.

To further the idea a little more, the wiki also helps in documenting the structure of the project, and like other tools would be able to show any/all references to given code.

Certainly, if you're a hardcore coder, you'll have a preference for direct access to the cvs. For more casual coders, or those who just want to wander around looking for interesting things to help on, the wiki frontend could help.

That being said, I have no idea how well this idea could work, but for smaller prototyping projects it might just work.

Re:Let's run with this idea a little

[Ramses0]

I did something similar to this at work... some greasemonkey scripts wired to a wiki as a backend datastore with some extra popup helper dialogs.

You would then run it via Python's great runtime eval stuff.  Since I'm pretty tired, maybe some code would help demonstrate:

def Import( wikiword ):
    print ">>> Importing: " + wikiword
    text = getTextFromWikiWord( wikiword )
    code = getCodeFromText( text )
    if len( code ) == 0:
        raise "No Code Found For: " + wikiword
    exec code in globals()
    return

def getTextFromWikiWord():
    # you can do some sort of HTTP fetch or file.open here

def getCodeFromText():
    # this basically grep's for [code]...[/code] blocks
    # and smushes them together

def Execute( wikiword, **params ):
    print ">>> Executing: " + wikiword
    Import( wikiword )
    str = wikiword +"( **params )"
    eval( str )
    return

if __name__ == '__main__':
    Execute( sys.argv[1] )

...the basic premise being that each function gets stuck onto it's own wiki page (def foo()...), wiki provides the equivalent of CVS, and with Python I basically wrote a new "import" function that imports code on the fly in about 20 lines of code.

Before going farther with this, read up on Knuth's "Literate Programming" ideas.  Also, take off your "programmer's goggles" and  put on your "not an expert programmer's" goggles.  With a sufficient level of abstraction, and good whip-cracking on the architecture to keep most complicated stuff "out of sight", non-programmers can pretty effectively build software out of componentized pieces.  They can even build some component pieces themselves if they're motivated and can understand what's going on.  And once it's in a wiki hidden between [code] blocks, lean on the wiki to provide gooood documentation (think wikipedia, but with little blocks of code).

--Robert

Re:refactoring dirty prototypes

[Pseudonym]

[I] am in awe how these two radically different communities (haskell and perl) managed to find each other ;-)

It's not such a shock, IMO. Ever since it's been a "real" language (about Perl 5 or so), Perl has always been inspired by Lisp-like languages with its support for programming with lists, map/filter-like programming, higher-order programming and so on. And Perl is still a prerequisite for building the most common Haskell compiler, GHC.

Haskell and Perl represent flip-sides of the programming coin. I find it no surprise that they've found a home together.

This makes sense

[doc modulo]

Because there are 2 kinds of fun/joy systems in the brain.

The kind triggered by dopamine, the short high that doesn't last, the Mtv, nicotine, "I want to buy this" kind of fun.

And the relaxed kind of contented joy that works with the serotonin system in the brain. Which does last and is an indicator of good relationships with friends, experiencing nature and knowing that "all is right in my domain, I'm ok for exploration into new things" kind of fun.

That last kind of joy is an indicator of efficiency and "everything is as it should be" and if you feel that while coding, you must be on the right track.