Paul Graham: Hackers and Painters

larsberg writes "Another wonderful article from Paul Graham on hackers, their lifestyle, and their tools. It's entitled "Hackers and Painters", and provides a great description of how the great hackers write code. The article is definitely worth a read, especially for those who have an inkling that any field that has to place the word "Science" in its name probably isn't really a science after all."

Comments

Wait...

[Anonymous Coward]

So, he's saying Political Science isn't Science?! No wonder everyone looks at me funny when I wear a lab coat around the office...

Re:Wait...

[xtermz]

You laugh, but in a networking class I had back in high school, the instructor wore a freakin lab coat. I never did understand that....

Re:Wait...

[Anonymous Coward]

You've obviously never had to empty the bit bucket.

Re:Wait...

[stratjakt]

Where are your observable phenomena?

The data streams in and out of the processor dont count? The output on your screen doesn't count?

Your testable hypotheses?

I think I can make an O(log(n)) algorithm to do blah blah. I think that malloc algorithm A will improve throughput 30% over algorithm B.

Your experimental data?

No data in computers, that's for sure.

Your replicable results?

Nope. No way to copy a computer program. Good point.

Re:Wait...

[Gabrill]

So I guess anything with Engineering in the title really means they just use crayons and legos more often?

Blurring the lines

[John Paul Jones]

Safecracking is an "art" too.

I guess it depends on your inital reaction to the term hacker. It should be someone who hacks code, vs. a cracker that willfully circumvents security measures and breaks into a network. Unfortunately, you need to consider the source of the quote to get at the real meaning.

Re:Blurring the lines

[gazbo]

RE: "nowadays"

If you find a copy of the Jargon File before ES-"raving lunatic"-R took it over, you'll see that it defines hacker to mean "one who breaks into systems" without any sort of warning about incorrectness. It is only when ESR took over that suddenly this became "strongly deprecated".

I can't remember whether that version even had an entry for cracker in it. But don't take that as an implied assertion that it didn't.

Re:Blurring the lines

[dsplat]

go read a newspaper and you'll see that I'm right

Every time I read a newspaper on any topic I know something about, I find more errors than facts. I don't expect them to understand technical areas that reporters never learn. But I catch them misquoting sources, misspelling names and getting people's jobs titles wrong when all the correct information is available on a company web site within easy cut-and-paste range. I wouldn't take a newspaper as the final answer on any subject.

Re:Blurring the lines

[(trb001)]

Eh, I think the author's main point was that hacking, much like painting, is the creation of something that has been imbibed with the author's creativity. Safecracking doesn't yield any new creations to the world, you just open a safe. It may be an art form to crack a safe quickly and precisely, but I think you're really stretching the definition of art form and blurring it with common slang.

--trb

Re:Blurring the lines

[orac2]

Actually, even the AP stylebook (ISBN 0-7832-0308-4) recognises the ambiguity. The entry for hacker (page 127) is

Hacker: a highly skilled computer enthusiast. In common usage, the term has evolved to mean one who uses those computer skills to unlawfully penetrate proprietary computer systems.

In case of Slashdotting (it's already a bit slow)

[Anonymous Coward]

May 2003

(This essay is derived from a guest lecture at Harvard, which incorporated an earlier talk at Northeastern.)

When I finished grad school in computer science I went to art school to study painting. A lot of people seemed surprised that someone interested in computers would also be interested in painting. They seemed to think that hacking and painting were very different kinds of work-- that hacking was cold, precise, and methodical, and that painting was the frenzied expression of some primal urge.

Both of these images are wrong. Hacking and painting have a lot in common. In fact, of all the different types of people I've known, hackers and painters are among the most alike.

What hackers and painters have in common is that they're both makers. Along with composers, architects, and writers, what hackers and painters are trying to do is make good things. They're not doing research per se, though if in the course of trying to make good things they discover some new technique, so much the better.

I've never liked the term "computer science." The main reason I don't like it is that there's no such thing. Computer science is a grab bag of tenuously related areas thrown together by an accident of history, like Yugoslavia. At one end you have people who are really mathematicians, but call what they're doing computer science so they can get DARPA grants. In the middle you have people working on something like the natural history of computers-- studying the behavior of algorithms for routing data through networks, for example. And then at the other extreme you have the hackers, who are trying to write interesting software, and for whom computers are just a medium of expression, as concrete is for architects or paint for painters. It's as if mathematicians, physicists, and architects all had to be in the same department.

Sometimes what the hackers do is called "software engineering," but this term is just as misleading. Good software designers are no more engineers than architects are. The border between architecture and engineering is not sharply defined, but it's there. It falls between what and how: architects decide what to do, and engineers figure out how to do it.

What and how should not be kept too separate. You're asking for trouble if you try to decide what to do without understanding how to do it. But hacking can certainly be more than just deciding how to implement some spec. At its best, it's creating the spec-- though it turns out the best way to do that is to implement it.

Perhaps one day "computer science" will, like Yugoslavia, get broken up into its component parts. That might be a good thing. Especially if it meant independence for my native land, hacking.

Bundling all these different types of work together in one department may be convenient administratively, but it's confusing intellectually. That's the other reason I don't like the name "computer science." Arguably the people in the middle are doing something like an experimental science. But the people at either end, the hackers and the mathematicians, are not actually doing science.

The mathematicians don't seem bothered by this. They happily set to work proving theorems like the other mathematicians over in the math department, and probably soon stop noticing that the building they work in says ``computer science'' on the outside. But for the hackers this label is a problem. If what they're doing is called science, it makes them feel they ought to be acting scientific. So instead of doing what they really want to do, which is to design beautiful software, hackers in universities and research labs feel they ought to be writing research papers.

In the best case, the papers are just a formality. Hackers write cool software, and then write a paper about it, and the paper becomes a proxy for the achievement represented by the software. But often this mismatch causes problems. It's easy to drift away from building beautiful things toward building ugly things that make m

hackers and painters?

[grub]

I highly doubt Picasso, Rembrandt or Cézanne strutted around saying "H3y d00dz, ch3ck 0u7 7h3 l337 p41n71ng5 1 h4x0r3d t0g37h3r l457 n1gh7! ph34r m3!

Re:hackers and painters?

[cruff]

I don't know any real hackers that speak like that either, just the wannabes.

Re:hackers and painters?

[NecrosisLabs]

Actually, Art History is filled with that (without the 1337 speak, of course). Read Vasari's Lives of the Artists, for instance.

Re:hackers and painters?

[BrokenHalo]

Maybe I'm just too old (OK, yesterday was my 40th birthday, so call me a curmudgeon if you want, I've been called worse) but when I started out as a systems programmer, it was fashionable to speak English. Leet-speak doesn't exactly do much to promote ease of communication, does it?

Re:hackers and painters?

[_ska]

the people who love to code *are* the real hackers.

just because the media coopted the term to mean script-kiddies and crackers doesn't mean the original meaning is lost to the old school.

PG, whatever else you can say about him, is an old school lisp hacker. This can be confusing to people who are too young or uninformed, but I can see why P.G. holds onto the original meaning.

So you hate it when people use the term hacker in its original meaning? How do you think they feel about people using the new bastardized version?

Is it just me?

[dreamchaser]

Is it just me or did I just waste 5 minutes of my life reading an overly verbose article based on a flawed analogy? Painters do not create something functional; hackers (read: programmers) do. Sure, code can be 'beautiful' to those who can appreciate it, but is it more art than science? Given the deterministic nature of digital computers, I think not. Are there creative elements to coding? Sure. That doesn't mean hacker==painter.

Just my ten cents. Your milage may vary.

Re:Is it just me?

[dreamchaser]

You are either trolling or you are confused between what a hacker does and what a cracker does. My guess is the former, but I'll give you the benefit of the doubt.

Hacker: writes code, likes to write neat code, usually of a functional nature

Cracker: breaks into systems.

I know, I know...the media long ago hijacked the term 'hacker' to refer to both sets of activities. The article in question uses the correct definition of the term though.

Re:Is it just me?

[akadruid]

When you meet someone now who is writing a compiler or hacking a Unix kernel, at least you know they're not just doing it to pick up chicks.

At least he finished with a well thought out and carefully researched conclusion.
That changes my perceptions completely

Re:Is it just me?

[moogla]

He (the article author) underestimates the scientific mind and determinism of some programmers.

Many of the best programmers I know are wont to draw up proofs and diagrams on paper before sitting down to code. Then as they evolve their code they do tests, draw more conclusions, and figure out what needs to change next.

If they went as far as to document that whole cycle, they would be 80% of the way to a research paper. ::shrugs::

It's an art, but so is writing elegant, easy-to-understand proofs. It's an art, as much as designing a new car is to engineers.

Re:Is it just me?

[Anonymous Coward]

Architects produce something functional, yet architecture is considered high art. The same goes for gardeners, interior decorators, portrait painters (originally) and most other art forms, although their original use may now be defunct.

Would you consider any of these people to be scientists or engineers? Does the fact that their art is useful have any bearing on that question? Does the deterministic nature of gravity and paint make them empirical investigators? The average architect probably knows a lot more science than the average hacker.

Re:Is it just me?

[hibiki_r]

He's not referring to all programming as "hacking". In fact, the article mentions how most of the programmers in the corporate world do jobs that have absolutely nothing to do with his concept of hacking. Hacking has a heavy dose of design on it. In fact, Graham argues that design happens to be more important than the actual implementation, it just happens that the implementation gets done as you design.

On most cases, coding to a design, instead of coding to user specs, restricts the task so much that there is little chance of creativity. That's not Hacking at all. On the other hand, desigining a +100,000 LOC complex, flexible system, designing it's data structures and object relationships, is so much more complex than "grunt coding" that I think it is way closer to architecture than it is to building a brick wall.

If you belive that all design/code is more like building a house than paiting it's just because you've not seen a truly brilliant, innovative design yet. I hope you're lucky enough to see one, or better yet, create one.

Re:Is it just me?

[Lurch00]

No, it's not just you. I think a lot of self-labeled hackers have a romantic notion of being artists, probably in part because that's an acceptable excuse for eccentricity. You hit it spot on when stating software was functional versus paintings. I think that coding is mostly creative (though not really artistic) in reality. There is significant math behind it, but I think that math is just a language to communicate certain ideas. The people who do actual computer science typically aren't the people doing the implementation. Likewise, in traditional math its not the mathematicians doing the actual implementation work, it's engineers and tradespeople.

If you really want to pick an analogy, I'd pick carpentry. Not finish carpentry, as that's too artistic on the scale, but rather functional carpentry. For instance, a carpenter with no real experience and no tools can build a table out of what they find lying around. Some tables have four different legs of four different sizes, one's attached by childrens paste, and another with a high strength steel bolt. See most OSS packages for the coding analogue. A better carpenter has a few tools and makes a better structure, giving a more functional table. Better programs stand up well provided you embrace their rigid design criteria. A master carpenter might the best looking table and design it such that he could come back and add drawers later or replace it with a bigger top or taller legs.

Carpentry, like software, is mostly about figuring out what pieces you'll need and how to piece them together. That's the essence of engineering. Those who study traditional engineering disciplines often scoff at the idea of software engineering because it violates the traditional way of doing the engineering before construction. That hasn't always been the case, and even today some software is developed by traditional engineering techniques. That's not an invalid way to do it, although it is more expensive. Software engineers are embracing the engineer-on-the-fly paradigm, and that's a new idea that nobody really knows how to teach.

In carpentry as in software, the engineering and fabrication are distinct and separable. I wouldn't consider it unlikely that in the future software architecture (engineering) will be done by a different group of people than the coding (fabrication). This is starting to show up now with many companies doing their architecture in UML and then sending it overseas for actual implementation.

This has run way longer than I wanted, so I'll stop here.. Just a few thoughts on a slow morning.

Re:Is it just me?

[TopShelf]

That doesn't mean hacker==painter.

Does this mean I should stop printing reports on canvas?

Re:Is it just me?

[Tim Fraser]

I think you are right in pointing out that there are limits on how far you can take this hacker==painter analogy. However, I felt that the article contained some amusing insights, nonetheless.

I've spent the majority of my (admittedly short) career turning DARPA grants into math-envious papers, just as the author described.

A couple of years ago, I did step back and wonder if what I was doing was really science. I didn't seem to fit the idealized notion of a scientist that I learned about in high school: I didn't seem to be forming hypotheses about some already-existing aspect of the universe and testing them in a methodical way. And I didn't have a labcoat.

I spent most of my time hacking around, following my intuition, trying to make the computer do interesting things. I also experienced the sometimes-on, sometimes-off rollercoaster of inspiration that the author described.

So, some of the aspects of the author's painter-analogy seemed apt, at least in my case. Your mileage may vary, of course.

Anyways, now I've got to go convince my boss that it would be good for our project if I went and spent a day at the art musem! ;^)

- Tim

Re:Is it just me?

[pileated]

Though I'm a painter who's gone to programming and done it for last 5 years or so I've always thought programming closer to carpentry than painting. But the reason for that is that carpentry is a craft and painting no longer is. Both can be art. But painting has divorced itself from it's utilitarian roots and thus is no longer a craft, much to its detriment. And actually to the greater number of people who might enjoy it if it was.

But my point is that progamming is a craft just like carpentry is and painting was. All crafts can rise above the mundane but serviceable to the artistic. In carpentry that means understanding the nature of your materials, all of the possible joints and why you might use them. In painting this was once understanding your materials (when they were a bit more traditional and fewer than they are now) and also learning about drawing the human figure, lighting, perspective, etc. In programming it's learning some basic algorithms etc. But all of these crafts allow artistry and it's artists who move the fields forward. Artists do love to create beautiful things, just as the article points out. And those that do almost always inspire the most talented younger members of that parcicular field. That's because it's beauty that is compelling, not knowing how to make a dovetail or an infinite loop.

Okay, I'm getting off topic. So let me just summarize. Programming really is like a craft but all crafts are capable of being art and it's the art of it that makes it compelling. Painting was once such a craft but has gotten away from its utilitarian roots so looks just too artsy to some people. So they choose carpentry as a better analogy. In the end though, like the article says, it's the art that counts.

And that's why the best programmers, carpenters, painters are in it.

I am like a painter

[archeopterix]

A grumpy guy in blue overalls carrying a big brush, a bucket full of paint and painting a gazillion walls a day. At least that's how I feel at work. Yes, I am definitely a painter.

Re:Computer programming is NOT an art. Get over it

[Anonymous Coward]

Is architecture an art? At what point does a structural engineer become an architect, or a civil engineer become a landscaper?

Yep

[MagPulse]

I liked reading this article because lately I've started to work on my drawing skills. It's very humbling to do something I'm skilled at (coding) and then move to something where people can barely tell what I'm trying to draw.

One commonality I've seen so far is that you just have to jump in and do what you can before anyone can help you. Posting questions like "how do I write an OS?" or "how do I draw such-and-such?" will yield theory but not get you far. On your first tries it's going to look like a bunch of scribbles (or spaghetti code that is far from compilable), but you have to put something down for others to critique. And of course coding and art both take tons of practice time. This goes along with just trying and not worrying about the results. If I didn't code unless I was sure each line was perfectly bug free.. well, that's impossible.

I've been working on realistic and anime-style people. Humans are the most rewarding subjects and also one of the hardest to draw, but I wouldn't want to draw anything else. For anyone else wanting to start in this direction, I recommend the PolyKarbon BBS. There are some amazingly talented people there that are very helpful. This site with anatomy books is a good reference. If you have more helpful links, like a newsgroup for new artists (I haven't found any that are good), please post them.

The Art of Computer Programming

[Xiver]

Programming / Hacking is neither and art or a science and yet it is both. If you don't program you probably would not understand, but if you've ever implemented your own b-tree in an application, you'll probably agree. Most likely, whether or not you agree depends on what type of software you have written in the past.

Art and science are probably closer than most people believe. Leonardo da Vinci painted some of the most astounding scenes ever painted; yet, he also studied science, literature, and the Christian bible. Many mathematicians would say that math is an art, heck there are probably some artists that believe art is a science.

Knuth says that computer programming is an art, but I dare you to read his books and claim they are devoid of science.

In short... It's all depends on the application.

Re:The Art of Computer Programming

[DrCode]

Yes, I do understand. I've been programming for years, but never did anything 'artistic'. A few years ago, I managed to write a small novel. And it struck me that writing fiction was really a similar activity to writing software. The main plot of a novel is like the overall design of a program; the subplots are like subroutines; and the classes are like the characters. Both activities take a good amount of planning, attention to detail, consistent style,... and lots of typing:-)

I used to be a "software engineer" ...

[cushty]

... whenever I first met people but I found their reply was "Oh" followed by a long pause. Now, whenever I introduce myself and have to say what I do, I tell people I'm an "artist": I take bits and bytes and create a masterpiece, or I colour-by-numbers.

Re:Computer programming is NOT an art. Get over it

[zakezuke]

If it evokes an emotional reaction, it's ART!

Using microsoft products evokes a raticaly diffrent reaction then using let's say an Apple product, or a Linux product.

Why would it be insulting for art and science to be the same thing. A good deal of science goes into art, and art into science.

Re:Computer programming is NOT an art. Get over it

[djtrippin]

Engineering isnt an art huh? So the things we design for the masses dont play on emotions when we sell them? Tell me that cars dont have an emotional role in most peoples lives, or that the old majestic buildings dont have a symbolic artistic beauty to them.

Re:Computer programming is NOT an art. Get over it

[MartinG]

To make an analogy between programming and painting is a mistake, because "computer programming" is more like "putting paint on a canvas"

Neither is an art in itself, but a programmer or painter _can_ use their tools to create art.

In other words, saying "computer programming is art" is wrong just as saying "paint and brushes are art" but a painting or a program can both be art.

This is a pointless exercise

[paiute]

This is nothing more than one of those compare and contrast essays which one has cranked out hundreds by the end of high school. Given any two professions, one could derive the same relationships: e.g.,

"When I finished grad school in blank I went to blank school to study blanking. A lot of people seemed surprised that someone interested in blanks would also be interested in blanking. They seemed to think that blanking and blanking were very different kinds of work-- that blanking was cold, precise, and methodical, and that blanking was the frenzied expression of some primal urge.

"Both of these images are wrong. Blanking and blanking have a lot in common. In fact, of all the different types of people I've known, blankers and blankers are among the most alike."

Re:This is a pointless exercise

[no_opinion]

I disagree. It seems like there is a stigma against the "hacking method" of development in the large software engineering companies. I totally related to the hacking mentality he described, and I have felt (since college) that this is not the "right way" to do things. I learned to hack in jr. high and high school, while college taught me rigorous design and implementation methods. These methods are anti-hacking because they focus on a lot of up-front paper design and process, whereas the hacking mentality involves working out the problem in code on-the-fly and refining over time. It's not unlike roughing out a sculpture in clay and progressively adding detail and changing things here and there as you go rather than working out the sculpture ahead of time.

So the point of this is that he is trying to say that hacking is a legitimate technique that has advantages over the traditional, slower and possibly less flexible, software engineering moethods and he's doing it by drawing parallels to how artists work. I think the truth is that you need a little of both.

There is place for everybody

[dorfsmay]

"Perhaps one day "computer science" will, like Yugoslavia, get broken up into its component parts"

This is already the case. There are people specializing in writing comp-sci software (compiler etc...), there all those double-majors who write software for their other degree specialization (software for given domains like geophysical etc...), there are people who specialise on the ergonomic of the GUI, etc....

This isn't that different from people designing a car, a thing that is both functional and can be beautiful. It takes engineer and designer to make a car.

Now don't tell me "I am the creative type", I am writting software, science doesn't apply to me. I once work in a software lab where the lab had been producing software for several decades. One day the person in charge of QA convinced the person in charge of development to bring some quality type tools like cyclomatic analysis (McCabe etc...). A few engineer came up with that argument that they were producing art, that a software couldn't / shouldn't judge them.

Well, they did two things: They ran McCabe against a lot of their software, some that have been in the market for decades, and there was a direct correlation between the "level" the tool was finding and the number of patches applied to the piece of software. Then they analysed code per current programers: The artsy types were writting complex code that the QA dept. kept sending back !!

Conclusion, there is a place for "out there" artsy type to inovated in a small shop, there is a place in ergonomic to write "beautifull software", but a serious piece of software does need some science.

Think about this, how beautifull would a car that looks good but keeps breaking down be ? Doesn't this remind you a lot of the software out there ?

Re:There is place for everybody

[Nursie]

Doesn't this remind you a lot of the software out there ?

Reminds me of quite a few cars too...

mathematics is an art too...

[oliverthered]

anything that requires creativity, and not just observational descovery is an art.

You can do computer science, and you will get a bullet proof system, but it takes a hell of a long time. People arn't that good at science, that why we created computers. code-breaking and missile targeting and the nuke.

Re:mathematics is an art too...

[BrokenHalo]

And a lot of real science is an art, too, in the sense that it takes creativity and imagination to bring it to life.

I spent over 10 years as a sysprog, and never questioned the term "Computer Science" for a moment until 2 years ago when I changed focus and went back to school to study biotechnology. Suddenly, I was faced with the "Scientific Method (tm)" that every other snotty-nosed nerd has had to cope with for the last x^y years, and it was quite a comedown.

The scientific method, however, is a beautiful means of giving substance to a construct of the imagination.

Great Quote

[gregmac]

So hackers start original, and get good, and scientists start good, and get original.

What a great quote. This is so very true.

Dynamic typing is a win here because you don't have to commit to specific data representations up front. But the key to flexibility, I think, is to make the language very abstract.

From what I've seen, very few people - espessially those with degress in computer science - share my views on programming. This article takes a different approach to it, but it's the same view I have, when it gets down to it. I usually say that when programming, you shouldn't be bothering with types, memory locations, pointers, and other nonsense that has nothing to do with how the program works. Or in other words, the formal 'scientific' aspects of programming.

Most people will disagree with me here, and I've been involved in many arguments over it. My programing language of choice right now? PHP. Why? Because it sucks less than the other choices. It still boggles my mind that C is used to do any high-level programming (ie, anything besides api's to system calls, and writing drivers and kernels). "But it's so much faster" I hear all the B.Sc's saying. And they're right, it does run faster. It also takes ten times as long to code. And ten times as long to find all the strange bugs and buffer overflows that eventually show up as exploits.

Paul Graham hints at it in his article, but there is no good language right now for writing applications in. PHP in itself is a nice language to write, although it's an interpreted language, not compilied. Perl is a bit too messy for my liking (Paul also mentions this when he says he knows people who wrote perl programs and came back and couldn't understand how they worked), although it is quite powerful. Java is nice in theory, but implemented a bit slowly, and it's a bit too scientific, really -- you spend so much time handling exceptions and making sure all your code is very formal.

So what's the answer? I don't know. But it doesn't exist yet, as far as I know. Until then I'll continue running my slow PHP programs on modern "slow" computers. That run at a mere 1.5 GHz.

Re:Great Quote

[Warped1]

The exceptions and formality of Java are supposed to aid development by making sure you've crossed all your t's and dotted your i's when it comes to error handling and type checking. This too helps prevent bugs, and makes your code safer.

When utilized properly they're nice things. =) But that's just my humble opinion ...

Re:Great Quote

[rjh]

I wrote an essay about Language Holy Wars a while ago which, perhaps, you might find interesting. It's over here.

Personally, I find it nothing short of terrifying that C is still used as much as it is. When you're dealing with hard realtime constraints, or an embedded system which barely has a bit of memory to spare, etc., then I can see it... but otherwise, for God's sake, use something appropriate to the problem set.

Personally, I love LISP. If only there were good UNIX API bindings for it, and a good graphical toolkit...

Re:Great Quote

[Jerf]

The exceptions and formality of Java are supposed to aid development by making sure you've crossed all your t's and dotted your i's when it comes to error handling and type checking. (emphasis mine)

You carefully qualified your statement so I won't lay into you ;-) That is indeed the "strong, static type checking" party line.

An increasing number of smart people are starting to ask questions about that party line though. They'll try out a dynamic language like Python, and the disaster promised by the static typing advocates conspicuously fails to materialize. For two examples of this, see Are Dynamic Languages Going to Replace Static Languages? and Strong Typing vs. Strong Testing. A lot of other people are leaning this way too in newsgroups and on personal weblogs, and of course a lot of people still believe the party line.

Personally, I'm suspicious of "received wisdom" that's much older then 10 or 20 years, as the static typing claims are; the world has changed a lot since then in a lot of ways, not least of which is our improved understanding of how to build things (i.e., even in non-technological ways).

Re:Great Quote

[RealAlaskan]

Paul Graham hints at it in his article, but there is no good language right now for writing applications in.

Paul Graham is a well-known Lisp programmer. He didn't beat us over the head with it in his article, but I'm pretty sure that he considers Lisp (Common Lisp, in particular) to be that good language, for yesterday and today at least.

I suppose that it's an acquired taste, but I'm convinced that it's a taste well worth acquiring. Here is a little screed I wrote to explain why I hold that conviction. Graham wrote several articles which tell his reasons. Some which pop to mind are: Beating the averages, Lisp in web based applications and What made Lisp different

By the way, Lisp doesn't have to be very slow. Here is a pointer to a paper which might get you started.

Re:Great Quote

[Anonymous Coward]

> Personally, I love LISP.

Amen to that.

> If only there were good UNIX API bindings for it,
> and a good graphical toolkit...

Lisp has good UNIX API and good GUI toolkits. [PUBLIC DOMAIN LICENSE, Native compiler with kickass code-generator.]

Lisp has good UNIX API and good GUI toolkits. [Commercial, free personal use, great IDE and GUI editor, kickass code generator, database, object system, etc, etc. This is the stuff dreams are made of, best development environment I have ever had the pleasure to use.]

Lisp has good UNIX API and good GUI toolkist. [Commercial, free for personal use, relatively affordable, best GUI toolkit (native look and feel on all supported platforms.)

If that isn't enough for you. GNU Clisp runs on more platforms than a flaming wazooka. Don't be bothered by the interpreter (most Lisps are compiled btw, and NOT interpreted as legend has it.) Clisp is FASTER than the stuff you find in your /usr/bin (Python, Perl, TCL or whatnot.) Clisp has a C interface (like most other Lisps, including the ones above) which lets you link to any shared/static library which has a C interface (almost all Unix software.)

There is also Corman Lisp which is strictly Windows for now. This beast has direct access to Win32 API and COM objects.

- Kumade Khawi

Re:The converse

[cmmike]

> Would you like Van Gogh to work on your teeth?

he's more of a plastic surgeon, actually.

leave your teeth to Ecsher.

Re:really?

[Junks Jerzey]

Admitedly I haven't RTFA but ...

And when you RTFA, you'll realize that your comment has nothing whatsoever to do with the article.

Re:Computer programming is NOT an art. Get over it

[Kadagan AU]

Art deals with human emotions, feeling, states of mind[...]

Excuse me, I've seen some pieces of code that are VERY beautiful and emotional. I'm not saying that a programmer and a painter should be grouped together, but there can be some extreme beauty in a piece of code. That's like saying that minimalists aren't artists because I think that a yellow dot on a red piece of canvas doesn't draw any emotions. Other people might look at that dot and see their entire life summed up, and get all teary eyed. Different people get emotional reactions to different things. I can still remember to this day the moment I first saw and understood recursion. I nearly creamed my pants when I read through my first recursive function. That's just one of many emotional reactions I've had to beautiful code.

remember, not everyone thinks just how you do.

This is part of a larger problem

[under_score]

that is being addressed partly by the research and practice of agile methodologies. The idea is to change the focus of software development to a new set of principles (for example favoring running software over documentation).

There is also a problem with the way that many managers and business people view software creation as a construction or engineering process. I wrote a paper about this: "The Software Construction Analogy is Broken". The summary is that software has so many attributes that are unlike physical things that its creation cannot be accurately mapped to the creation of buildings. For example, the economics of distribution are completely different: a building cannot generally be moved after it is constructed, yet software can not only be moved, but also can be duplicated for almost zero cost.

Ultimately, I think that software creation is actually the creation of completely new media of communication. Every program created defines a new set of communication interactions that didn't exist before. We don't really have any "science" for that.

Paul Graham isn't the typical hacker

[Junks Jerzey]

Paul Graham is a smart guy. He made millions selling his company to Yahoo. He's written several books on Lisp. He regularly has speaking engagements. And he does practice what he preaches, actually using high level languages rather just bashing away at C++, but still using it for everything, like most people do. He also manages to completely stay away from the usual topics, like Linux vs. Windows. Oh, all right, one more "and": And he has some unpopular opinions, like that of OOP being overrated smoke and mirrors.

That said, his view of what it means to hack is certainly different than what it usually means in geek circles. Actually, I should go further than that: Paul Graham isn't even a geek. Nobody would call Feynman or Dyson a geek, would they? Paul Graham is someone of high intelligence who happens to be applying that intelligence to computer programming (and writing, and speaking, and painting). This is much different from the typical hacker who pounds out C code because he has nothing better to do and revels in the geek traditions of arguing about Linux distributions, Star Trek movies, and yes, posting to Slashdot. In short, Paul Graham is a geek by association, because of what he decided he likes to do, whereas most hackers revel in their own geekiness, pointless and inbred though it may be.

Re:Paul Graham isn't the typical hacker

[crgrace]

Nobody would call Feynman or Dyson a geek, would they?

Have you read any of Feynman's books? That guy was an uber-geek. Any guy who would ask every girl at a party "Would you sleep with me?" based on a theory that 0.1% would say yes is pretty much a poster child for geekness. And don't get me started on his little geek-jokes in his technical writings.

Of course, being a geek doesn't make him any less a genius.

Re:Paul Graham isn't the typical hacker

[Junks Jerzey]

Have you read any of Feynman's books? That guy was an uber-geek

Yes, I've read them. The meaning of "geek" has changed. Feynman lived his own life; he did was he wanted. "Geek" these days is a lot more of a word for branding someone with a certain set of limited interests. If someone goes home and writes fantastic software in the evenings, he or she isn't a geek. But if he writes essays about Star Wars, endlessly posts to discussion forums about the evils of Microsoft, and spends more time installing Linux and patching kernels than actually doing anything with it...that's what "geek" now means. And it's deservedly derogatory.

Art, Programming and design skill

[ralphart]

My degrees (bachelors and masters) are in painting, and I've worked for the last 8 years writing code. I'd say my art background has definately been a benefit.

Art (can) be about seeing overall patterns and relationships between seemingly dissimilar things, as well as about organizing and processing information in a way that gives meaning to it.

Sounds like the skills I use everyday as a programmer.

Re:really?

[NickFitz]

Admitedly I haven't RTFA but ...

This whole "hacker lifestyle genre" thingy is a bit too dramatizied by the geek wannabes. Go out and actually try to write software that must be supported and updated.

Shame you didn't RTFA, as your point is explicitly addressed:

Source code, too, should explain itself. If I could get people to remember just one quote about programming, it would be the one at the beginning of Structure and Interpretation of Computer Programs.

Programs should be written for people to read, and only incidentally for machines to execute.

You need to have empathy not just for your users, but for your readers. It's in your interest, because you'll be one of them. Many a hacker has written a program only to find on returning to it six months later that he has no idea how it works.

I felt it was important to the worth of the article that he actually covered important points like this. Overall, whether or not one agrees with the author's use of the word "hacker" in this context, I as a programmer recognised so many things in this article that I have come to understand and appreciate in the past 30 years of mucking about with these wondrous machines. I was able to think of specific examples from my own experience of every important point made.

YMMV... :-)

Re:"a language that lets us scribble and smudge"..

[BrokenHalo]

No, just an 029 card-punch :-)

Computer Science is Science

[Dr. Bent]

Computer Science is as much a science as Physics or Biology. The problem is that "Computer Scientists" rarely, if ever, do actual Computer Science after they get thier degrees.

How many people out there with C.S. degrees have gotten jobs that require them to develop, for example, a new compression algorithm? When's the last time you wrote your own language and a complier to go with it? I'm not saying it never happens, but the reality is that most Computer Scientists wind up being Software Engineers.

I mean, It's important to have the scientific background when being an engineer. How many civil engineers do you know that never learned static-state physics? But developing software systems is no more a science than designing cars or buildings. It's applied science, which is a different thing.

Hacking, Painting and Musicians

[SuperGrut]

I am the only Hacker/Painter at my company. All of the other Hackers are Hacker/Musicians.

They act like I am really unusual. Util I read this article I was starting to think I was the only one.

Why are Musicians so common but Painter's not in the Hacker community?

Re:Computer programming is NOT an art. Get over it

[skurk]

Programming is as an art as bridge buidling is.

What about this?

Art or no?

Too man over-generalizations

[prgrmr]

I've never liked the term "computer science." The main reason I don't like it is that there's no such thing.

Sure there is. Just because a term is overused doesn't mean it does not have legit application. Just because he doesn't like the term because it doesn't fit in with his vision does not provide a basis from which to dismiss the term.

Good software designers are no more engineers than architects are.

How many people today only design software, and never code or test it? How many people design software for software's sake, as opposed to people who design software that is supposed to do something and thereby provide a means to an end?

And he couldn't even 'splain the Desi-Lucy relationship correctly.

Rose tinted shades

[coyote4til7]

Sometimes it's fun to read the slashdot reactions before RTFA. One poster header "painters" and assumes Graham is talking about the Bubba who shows up to cuss and slap cheap paint on apartment walls. Other people assumes he's talking about the code monkeys in the world. The "fun" thing about today is no one listens anymore. Maybe it's just me, but he's not comparing the techniques. He's comparing what makes these people tick. Einstein is one of the great Physics hackers, but he was also supposed to play a mean violin. The wiring that allows people to pull visions out of thin air works in both worlds. Graham isn't saying anything terribly new (at least to me). I was screwing with creating code in Junior High and writing short stories by High School. By the time I was done with college I had a degree in Physics, work in the humanities and published poems and paintings that had been in shows. I've been asked the same question and I always said Physics (or hacking) and the arts back up to each. More correctly the one that can create in Physics/thru hacking is often the same mind that could envision great art. Same mind, different techniques: creation ex nihilo.

Re:Just dawned on me

[Molt]

Odd, I find the Slashdot topics to be remarkably connected..

"Let's boycott people and not buy any DVDs!"
"New sci-fi DVD out this week, buy it!"
"All big computer companies are simply profit-oriented"
"Apple releases new thing"
"Most of the world's population are sheep driven by obvious marketing"
"Isn't this new thing cool and shiny? Buy one! Hell, buy two! Shiny shiny!"
"Microsoft does thing, isn't it evil?"
"Linux does thing, rejoicing in the streets"

...oh, and..

"Ask Slashdot: Finding your rear-end with both hands"

people always look at me funny

[kraksmoka]

when i tell them that i went to school for music, singing in fact, before i got into photography, which led me into web design, which got me into web application development.

i always felt that the most important issue, was not just being a maker, or constructivist, but being professional about my business conduct. if there was one and only one thing i learned it was that a proper presentation of one's self would decide if someone else will pay you to make things for them. after that, its all wine and roses, and blooshot eyes and doing what you love.

The supreme accomplishment is to blur the line between work and play. Arnold Toynbee (1889 - 1975)

He's Got Some Very Valid Points

[Greyfox]

If programming is a science, it is still a poorly understood one. Part of the huge missing factor could very well be the creativity involved in building something completely from scratch.

Introspection is not unwarranted in our field. Many things he said in his article were true. I view hacking and programming as two very different things. The way I see it, a Programmer is a code monkey who implements management's vision of how the software should be 40 hours a week and then has nothing to do with computers the rest of the time. A hacker can build something truly unique and will no doubt be working with computers at other times as well.

The IT industry does not recognize the difference. They have a very narrow slot that you get put in if you can program. For the hacker, work in the IT industry can be very unfulfilling. It is the very rare hacker who is given the ability to create what he wants (It is the very rare painter who is given the ability to create what he wants, either.)

I think this is why the Open Source movement has taken off like it has. There has been a lot of pent up demand to create, but programming is a collaborative field. One hacker might create something nifty, but when you get the synergy of him bouncing off thousands of other like-minded people, some truly amazing things can be made.

If you look at the indignation that arises here when someone goes on about the DMCA or software patents, it is very close to the anger and frustration a painter would feel if you told him he couldn't use a given color in his painting or paint a certain subject. I don't think cold and calculating science would become so enranged.

So I think that the entire process is much more about creativity than you give it credit for. Most large programming projects are extremely chaotic systems which seem to have a life of their own. It is far more fuzzy than you'd expect something that was made only of 1s and 0s to be.

Amen, brother!!!

[master_p]

Programming languages should be abstract. We should not be dealing with types, memory management, file I/O and any other implementation details. We should be coding abstract algorithms / architectures, which are later applied to data.

I have also battled it out here on Slashdot. C sucks!!! at first, it seems the all-out powerful language that you can do anything with it. And indeed you can!!! but it takes a lot of time, because the programmer has to define all details by hand.

I've caught myself thinking much more clearly when I did not have to consider implementation details. How come there is no real abstract compiled language around ?

Keeping programs as short as possible

[MarkWatson]

One of the reasons I love to code in Lisp so much is that it is a concise language - Paul Graham has it right when he says that short programs are easier to change.

When I have to code in Java (most of the time), I try to keep my applications as short as possible by first developing and testing low level class libraries to support a project - once these are tested, I find it easier to write much shorter application programs that I can tweak easily.

Still, I find Common Lisp to be my most productive language (Smalltalk is pretty good also :-).

-Mark

a great related article about . . .

[kraksmoka]

http://www.salon.com/tech/feature/1999/11/30/lions /

pg mentions john lions and his supressed Unix tome. this is a great feature article about it, and some (possibly not so anonymous) linux kernel hacker with a really kewl girlfriend.

Photography, hacking and painting

[arsinmsn]

Back in the 19th century there was a long argument about whether photography could be a "real" art, like painting. It was argued that because the film simply received an image, that it was far to mechanical to be art. Despite the fact that people know how much manipulation goes into photography, many folks still feel that it can't be a real art because it isn't like painting. The hand of the artist may be less apparent in some paintings, but we can usually see it and understand in a very physical way the motions and choices the artist went through.

Sadly, one of the crafts of programming is to conceal the hand of the programmer. Like photography many programmers aim to make their products as transparent as possible to facilitate user interaction. So even a truly magnificent program shyly, slyly evades the user's direct gaze by hiding in the maze of code, and covering its tracks with easy interface.

Perhaps games or computer generated amination like Matrix aspire to art in their very best moments, but let me say one more thing about the history of photography; photographers started out imitating other art forms like painting and drawing. Many people now feel that though this was a natural starting point, photography couldn't become an art until photographers understood the real strength of their medium and exploited it.

I suspect that programmers are only starting to discover the real potential of their medium, and that the most complex, expressive uses of uses of computing are not yet finished imitating other media. True computer will not only be different from what we might imagine, and different from what we are capable of imagining right now; however, it is never-the-less what we WILL imagine.

I suspect that, as in all arts, it will not be the general public who decide what is great programming, but specialists, some of whom probably participate here. These experts have the insight to tell which programming is so elegant, so provocative, and expressive that it stands head and shoulders above the usual thing.

Keep an eye out.

Empathy

[mahler3]

If Mr. Graham had empathy for his readers, one way to show it would be to avoid the use of footnotes that required manually scrolling to the bottom of a long web page.

Otherwise, a fine and insightful article.

I gotta agree with this guy...

[ynohoo]

I came to programming after years of painting (this guy did it the other way around), and have to agree with many of the analogies he draws, i.e. learning by experience rather than intensive studying; designing by code rather than specification; empathising with the user as part of the design process; &c.

Some folks ask me why I don't paint anymore, and I tell them I get my creative kicks writing software. Nice to know I'm not the only one who thinks this way (because you know the management won't understand!).

Not just you...

[mooman]

For me, I thought the article meandered a bit and even outright contradicted itself in places.

To wit:
He espouses how important it is to keep the design fluid and to change it midstream as necessary, even being fast and loose with data types. While that may indulge the hacker in being creative, it also wreaks havoc on one conveniently omitted aspect of software: maintainability. Rare indeed is the code that is written and never touched ageain. I'm sure someone can toss in statistics about what percent of time or effort is invested in the creation of code, versus all the maintenance required in it later...

To ensure you are not shooting yourself in the foot with this method, the hacker/developer/maker needs to devote a certain amount of time in thought and design before beginning to code. If a stumbling block is detected, legitimate thought must be put into how much code must be *undone* and started over to ensure clean code at the finish. Merely exploiting the fact that you have sloppy data types that can be used in multiple ways will only lead to more convoluted (and certainly less intuitive) code.

Another issue that Paul just sort of tosses out as fact is his claim of how most mediums were at their peak "early on", as in "The paintings made between 1430 and 1500 are still unsurpassed.". Now, I'm no art expert, but doesn't painting predate this by several millennia? Wouldn't the use of perspective by the Greeks seem like a monumental echelon above the flat art of the Egyptians? And aren't the Eqyptians to be praised for their realistic portraits relative to cave art? I'm pretty sure that 15th century art only represents generation after generation of gradual improvements. The fact that the artists following that era began to explore more abstract approaches like impressionism, expressionism and whatnot (http://www.quizbowlonline.com/artmovements.html) does not automatically elevate the "realists" to the pinnacle of the medium.

In any case, it was oversimplifications like this that made the premise a little harder for me to swallow. Very valid points were made, but the analogies do break down in a few places...

But I still marvel at how effective Popfile is. Paul has his moments of genius, there's no doubt.

Re:"a language that lets us scribble and smudge"..

[stephenb]

If hackers are painters, then text editors are the "bowl of fruit" painting. Check freshmeat if you don't believe me. :)

It's _one_ point of view

[ites]

Having been doing both software and art (painting, drawing, photography) for over 20 years, I agree with Graham only in a few aspects.

Software is obviously not a single subject: it is like language, and can be applied in many different ways.

Let me take a simple example. There are two main ways that people look at the world of problems: they search for the commonality, or they search for the particularity. Now, I would argue that good software is built by finding the common patterns and solving those, but that is just the way I make software. Other people I've worked with do the opposite: they focus on hundreds or thousands of individual cases. Clearly we have different types of mind - what I consider mindless and irrelevant detail is to my colleagues the only part worth solving.

Painters, artists obviously work the same way. So do writers, archietcts, and all creative people, including scientists. There are hacker chemists, and there are the synthesizing chemists.

Personally, I find that the best art comes from using standard patterns in new way: for instance, good photgraphy relies on excellent understanding of light and subject, but every image must be unique. The best science takes the other extreme: determined effort to find the core patterns and understand those. Ten experiments that are each unique are worthless. Ten experiments that give the same results each time are perfect.

Two more dimensions to this, in my experience. First, men and women tend to different approaches when it comes to creating and problem solving. More men like to hack, not because we're taught that but because our brains work that way. Second, with age our minds also seem to change: older brains can understand a depth of asbtraction that escapes younger minds. Software is all about abstraction, unlike science or art, but very much like language. Young geniuses may be able to hold thousands of details in their heads, but they cannot (generally) use them to build large-scale abstractions that work well. Again, the same for writers.

No accident that some of the best programmers ever are also linguists and writers.

Re:So why is Lisp fading?

[MarkWatson]

Good question!

One answer is that it is not fading (too much :-)

There are several fine commercial Lisp development systems and many fine/good free Common Lisp compilers.

You are right though - relatively few programmers seriously use Lisp. This does not bother me in the least. Personally, I always like to use the right tool for a job. Choosing the right tool also involves issues like "can I find a programmer to maintain this code", etc.

I used Common Lisp recently on a commercial product that I developed - I was only planning on selling "shrink wrapped" executables and the algorythms that I used were tricky (for me at least) to figure out - so using Lisp seemed like a good choice.

On the other hand, I make most of my living writing (or getting customers started with) Java server side stuff - using Java and web standards helps insure the value of the resulting project (with source code) over the long haul.

A few months ago, I had to do some tricky coding - I used Smalltalk to get the algorithm right - then rewriting in Java was trivial.

BTW, I enjoy using different programming languages.

-Mark

Re:So why is Lisp fading?

[Raffaello]

Because lisp, like all tools for highly skilled workers, takes time to master. People who have taken this time are fewer, and so, more expensive to hire.

Most companies will foolishly opt for 3 code monkeys (i.e., inexperienced, unproductive programmers) rather than one lisp master at 3 times the salary. This overlooks the fact that the lisp master is often 5 or 10 times more productive.

It's the same reason businesses choose cheap everything. Bean counters rarely look past initial cost.

Paul Graham's success with viaweb (now Yahoo Store), and ITA Software's success with Orbitz suggests that the increased productivity of a handful of lisp hackers can easily beat whole cube farms of code monkeys when time to market counts, which is, essentially, always.

To summarize, what you're missing in this picture is that management decisions such as implementation language are often made by managers, not master programmers who actually know about such things.

Re:So why is Lisp fading?

[mrdlinux]

I've been using Common Lisp for 3 years, and this is how I see it:

Common Lisp is actually a relatively new language. It draws on older languages such as MACLISP, ZetaLisp, Scheme, and ALGOL, certainly, but it breaks away from each of them. Unlike every Lisp before, it takes lexical scoping (by default) from Scheme. But it retains a separate namespace for variables, functions, etc, unlike Scheme. It uses function names drawn from MACLISP but hammers down the semantics of interpretation and compilation (in MACLISP these could be very different).

The language design was originally coordinated by Guy Steele, and from that the ANSI standard was created. This took the better part of a decade. The intent of the language was to be practical and to resolve problems that existed, so implementations did arise and follow the ongoing standardization. But it still took a long time for them to become mature and settle down. I cannot speak from personal experience, but it does seem that in the mid-90s the quality/quantity of the available compilers was not at all near what we have today. Particularly the free ones.

Nowadays there are four high-quality commercial implementations that I can think of right away (Allegro, LispWorks, Scieneer, MCL), three high-quality free implementations (CMUCL, SBCL, OpenMCL) and three up-and-coming or partially-compliant (CLISP, GCL, ECLS).

Every day it seems that there is more and more progress. I think that what happened is there was a lull over the past ten years and now things are starting to speed up once again.

But why was there a lull at all? It's hard to say exactly, and I wasn't personally involved in it, but what I have read has led me to see the problem as one of poor business decisions and a drop in confidence. Many eggs were placed in the Lisp Machine basket (many CL ones too) and when those failed in the marketplace (in the late 80s/early 90s) it took a lot with it. The LispM failure was probably due to two factors: (1) Overblown expectations of AI falling over, and (2) Too far ahead of its time to be financially feasible. The LispMs were very interesting machines and environments but they were far too expensive to be able to compete against Unixes, even if they had more features and better design. (The Unix Hater's Handbook enshrines the feelings of those people forced to move from LispMs and Genera, among other systems, to Unix).

In the meantime, there was the "Free Unix"es which grew up in the early 90s and helped to push out most of the remaining non-Unix systems. Only Microsoft, with its budget, managed to survive; and Macintosh on its hardware platform. Perl and Python grew up in the Unix environment, and given the current lock Unix has, it is no surprise that these languages are widely used. They also marketed themselves as "scripting languages" to fit in under the radar of C and C++ programmers, whereas Common Lisp was built to be an industrial-strength application (or even system) programming language. Anyone who knows Lisp also knows that it makes for it's own "scripting language" very easily, though.

Common Lisp didn't grow up in the Unix environment (not solely, anyway), and Lisp users had different ideas of what an OS should be. So there was an conceptual mismatch between the two that really hasn't been fully resolved. These days, there are many people (like me) who learned on Unix/C and then picked up Lisp. So there is more and more a trend to play well with Unix. The commercial Lisps are ahead in this game because they need to be in order to survive.

I wouldn't say that CL is so much difficult as it is different. This is probably another reason why it didn't catch on in ths Unix world as much. Universities did, in many cases, chug out students who were exposed to Lisp (or very often Scheme, which is different). The trouble is, many inculcated their students with the concept that Lisp is a language for theoretical AI work rather than practical programming. Actually, many high-level langu