Slashdot Mirror
Using Lisp to beat your Competition.
kovi writes "Paul Graham, the guy who developed what finally became Yahoo!Stores (and made him $50 million richer) wrote
an article that explains how he used Lisp (the infamous programming language) as a competitive advantage against the competition. As a bonus: thoughts on startup experience." Its in pdf, but its actually worth a read. Very nifty.
In a funny sort of way, while Scheme isn't used for many large projects, it's nearly as alive as Common Lisp -- which is to say, not very alive. Scheme is more popular in Academia at this point, in large part due to the book Structure and Interpretation of Computer Programs, by Abelson and Sussman, which is a great book for learning to think differently about programming. It's the standard introductory text for programming at MIT.
Common Lisp is far more practical than Scheme. It has every feature you would want to use, and many features that you would never want to use (e.g., dynamic scoping). It was designed by committee by smushing a bunch of previous Lisps together, and it shows. But it actually has a lot of useful code for it, and a lot of useful features.
Scheme is designed by a committee of mathematicians posing as computer scientists, and that shows too. Common Lisp has an ANSI standard, Scheme has a strong standard that is only endorsed by convention and the academic credentials of the committe. It's an interesting difference.
Common Lisp is also interesting in part because of its rather novel object system, CLOS. It implements what's called a Meta-Object Protocol, and is supposed to be very Deep (though I haven't used it myself). It uses a style of generic programming, as opposed to object methods -- it looks reminiscent of C++ function overloading, but is somewhat more general (arguably more general than methods). There are comparible object systems (e.g., TinyCLOS) available for Scheme, but not generally built into the language.
Common Lisp has been used more in AI, where Scheme is a more important foundation for language design.
Why wait for Google to find it, just so you can read the essay in plain text? Don't! Here it is already. And since the Web server seemed a bit sluggish, perhaps in anticipation of the /. Effect, here's a mirror of the PDF original. Enjoy.
Scheme is much prettier than CL; it's breathtaking in its conceptual purity. And it's the foundation for the best book on programming ever written.
One of the strengths of CL is that it has a large and comprehensive standard library, and tons of clever bits that make a working programmer's life easier.
In other words, learn them both!
Peace,
(jfb)
To spur "enterprise Linux," Big Bang, the distributed two-phase commit.
I've been a big fan of Lisp since I first learned it, but I've always had trouble articulating why it is so useful. Now that XML is around and the world is saying it's the greatest thing since sliced bread - I have an analogy.
/. Say
Lisp and XML both support
- Arbitrarily complex data can easily represented in text.
- Parsing of data in an easy fashion.
However, I think LISP has some advantages over XML:
- XML standard is becoming too complex, LISP data format is about as simple as you can get. You can write a LISP parser in 100 lines of code. This makes LISP ideal for tiny "fun" applications, to the larger enterprise applications.
- XML is much more verbose to write - making it easier to read by humans, but also making it something you don't want to write by hand - witness all the tools that have been writing to assist XML writing.
- LISP allows for execution and interface with code. Sometimes data can't be stored and loaded in a 100% static format. It's very useful to be able to embed calls to your program to generate data on the fly. And it's very easy to mix and match data and code.
To see LISP in work in the real world, download some of my programs:
Abuse - a side scroller action game published by Electronic Arts. Almost all the data loaded by the game is specified externally in LISP files. All but the main character's AI functions are written externally in LISP. Source code for this is available.
EZIP - This program shows my new HTML-like dialog layout library. I encourage you to look at the LISP code that generates the dialog boxes. 3 very nice dialog boxes specified in 100 lines of code. LISP is easier to write than HTML because with an editor that does paren matching you can see opening and closing of rows, tables, and columns. Here is an excerpt of the LISP code from ezip:
(well
"Lameness filter encountered. Post aborted.
Reason: Junk character post.") when I post the code... oh well
(This looks better in an editor). What is cool is that changes to variable names such as "filename" and "outname" are automatically applied . If this were an Active-X control with IE showing HTML then I'd have to have a "Submit" button which sent the results of the form, and then parsed the changes I wanted by hand.
This dialog software uses the same algorithm as IE to layout dialog boxes so when I open I dialog I don't need to specify any sizes in the code. The layout algorithm automatically determines optimal size for tables, cols, and rows. This allows it to calculate a size for the tab control, and a size for the window itself. Language translations just work regardless of how long the German word is for "duck" It's a huge advantage over conventional dialog layout methods like GUI editors.
Golgotha -A 3d action game that used external LISP files to specify data for the game. This allowed artist to add new models and textures to the game. They may not know lisp but they can copy a line of code and change file names like there is no tomorrow. Source for this is available.
I'm not going to say that people should go out and programmer entire apps in LISP - but I think it's an excellent way to represent data extern to your program - perhaps better than XML because of the flexibility it can allow you.
-- Virtual Windows Project
Actually having used LISP and C++ (along with many other languages) in my career, I can speak with *some* authority on the differences. I'm frequently astonished with people who express opinions about some language with out first learning something about it. People seem to learn only some small corner of a language and assume the rest of the language is like that chunk, or worse, latch on to some half formed opinion expressed by some other person, and claim that to be the final word on the language.
The saddest reality is that there is a 100:1 productivity ratio among programmers, and there are many more at the low end than at the high end. I have noticed that the best programmers all know LISP. They grok LISP. As a result, they *really* understand computation deeply enough for that understanding to translate to any other language, though it still takes time to grok the core idea of that other language (C++ comes to mind as a devilishly complex language to truely master).
Every language has a core quality that the programmer must grasp to use the language effectively. If you can't learn that core idea, the game is over.
So -- when you say that the code is being rewritten in C++ because your shop can't maintain it, what you are saying is that you can't understand the elegant design, so you will replace it with something simpler (you hope), and will ultimately find that the resulting program is far more brittle, and harder to maintain.
You are also saying that you don't have any of those programmers who truely grok computation.
As support for my position: I developed software for engineering automation in the 80's and 90's, then moved to control systems when my company was sold and destroyed by poor management. Technically, the product (written in CommonLISP) was a screaming success. We just could not make good sales/management/investment decisions. I now write control system software in C++. LISP would work better, but I needed to be compatable with an existing code base. I consider myself to be expert or at least competent in both languages.
LISP is not that hard to learn, by the way. The syntax is *really* simple, the language uniform. What *really* stalls people is not the parens or the effeciency, or loose typing (as is often reported), the core problem people choke on is the abstraction. If your mind is not well enough trained to understand the idea of mapping a function over a domain and expressing that directly, rather than say as a looping procedure, or if you can't grasp recursion, you can't really pick up speed, and when you read code that uses those ideas, you will be lost entirely.
We could all stand to learn much more. Study LISP -- it will improve your programming skills hugely, I promise you.
Things I love about lisp.
- Macros. Because the syntax is so regular and the whole lisp system
is available during macro expansion, you can extend the language in
very powerful ways with macros. *THIS* is a killer feature in lisp.
- Generic Functions. This is a generalization of virtual functions
where a function can dispatch based on more than one parameter. Anyone who
has used the visitor pattern in C++ knows implementing multi-dispatch in
C++ comes with tradeoffs.
- Method Combination. Before/After, Around methods.
- Higher Order Functions
- Metaobject Protocol
- Efficient Compilers. Checkout CMULisp for a free advanced compiler
Things I hate about lisp.- Not many good libraries. I would especially like to find a good gui library. I understand there are people working on this (Free CLiM).
- Language seems to be dying. Not many employers are interested in lisp people.
- Typing Issues. I prefer stronger typing than lisp has. I think
C++'s type system is just right. However, I understand there is no
right answer to a type system. It is a tradeoff. Lisp allows some
elegant tricks. For example, a pipe in lisp is built from a cons. The
second element of the cons can either be another cons (the next
element of the list), or a function to call to create the next element
of the list. After the function is called, it (the cdr of the cons
cells) is replaced with the new element. I can do the same thing in
C++, but both the new cons cell and the function have to share a base
class.
Just some random thoughts on lisp. BTW, I highly recommend Paul Graham's "OnLisp" if you're interested in seeing what the language can do. Other good books are: Norvig's "Paradigms of Artificial Intelligence Programming" and if you're interested in Lisp's object model try "The Art of the Metaobject Protocol" by Kiczales, Rivieres, and Bobrow.To a Lisp hacker, XML is S-expressions in drag.
... since on-line store building is no longer rocket science. After Graham showed how it ought to be done, on-line store building became a solved problem, and hence boring. Thus, it needed to be redone in languages more suited to use and maintenance by less brilliant developers. Let's face it, 99% of IT work has to be doable by the average programmer.
... the first time.
I would argue that everything should be done in Lisp
To a Lisp hacker, XML is S-expressions in drag.
Emacs is definetly difficult enough to qualify as a life long challenge. Emacs is like 42 in Hitchikers Guide to the Galaxy. Don't get me wrong, xemacs is my portal to the world. But, my configurations and modules are as cryptic as the Knights Templar. I've forgotten how half of them work anymore. Emacs is like Darth Vader. One day you realize that you are mostly Lisp/Emacs, but your not sure how you got there. You know its somehow evil, but the power of the darkside is just too tempting to ever go back.
Someone you trust is one of us.
"Scheme is a simple, yet powerful, programming language. As a member of the Lisp family of languages, it is dynamically-typed and mostly functional. Since it is much smaller than Lisp, it can also be used as an embedded language, a scripting language or an extension language. The Hotdog Scheme compiler currently compiles most of the Scheme language. It has been extended to support development within the .NET framework, allowing integration of Scheme and other languages targeting the runtime (Visual Basic, C++, C#, etc.).
I'm a little scared of the flame I'll get for bringing up .NET, but take a look at the site. It's some interesting stuff.
A speech...
> I know a bit of Lisp -- I love Emacs and wanted
> to be able to customize it more, so I played
> around with Lisp. I learned enough to do some
> pretty cool things, and learning Lisp has taught
> me a lot about programming in general...
Emacs Lisp is not a modern Lisp. It is an archaic and *very* poorly designed implementation of a Lisp developed at MIT in the 1960s and 1970s called MACLisp. Do not confuse Emacs Lisp (which even its maintainers think is a piece of shit -- just ask them) with a *real* Lisp like Common Lisp or Scheme.
Emacs Lisp is a bane to the Lisp community. An embarrasment. It carries on the awfulness of dynamic scoping (modern Lisps use lexical scoping), horrible garbage collection (GCPRO isn't just a hack, it's a crock -- of shit), and poor support for language extension (no easily constructed user-defined types, no object system, etc).
> But I'm not convinced that Lisp is the "highest
> level language of all high-level languages" or
> "the most powerful of all high-level languages".
There is no such thing as the 'highest level HLL'. Someone can always make a more abstract interface atop the language in question. So this statement is meaningless.
There is no 'most powerful' language in any field. Power is a completely relative term. It's relative to the immediate situation at hand. Flexibility on the other hand is measurable by the number of different uses a particular technique can be put to.
VB is not any higher level than Lisp is. Both remove you from the actual workings of the system. Lisp is *much* more abstract in many ways that VB will ever be. Common Lisp provides a generic and OS-independent interface to filesystems of all sorts, something that cannot be said of VB, which is locked into Microsoft's OS products. Common Lisp also provides a generic I/O system that is OS-independent. VB is again not OS-independent.
Someone mentioned that Common Lisp had poor I/O facilities. I must disagree -- in Guy L. Steele's book _Common Lisp: The Language_ 2nd ed. there are over 100 pages in chapter 22 on I/O alone. That's not including the following chapter which focuses on filesystem interfaces and accounts for another 50 pages, and the chapter on prettyprinting code which takes up another 20 pages. Common Lisp certainly has a lot of I/O support, in many ways much better than C. And if you want someone else's I/O then you can use a foreign function interface to some other language library to call out to it.
> Lisp code is hard to read for most programmers
You just need a better tool. Turn on parenthesis highlighting in your Emacs. Use the sexp-movement functions. Or, quite simply, learn to expand your stack. I see plenty of piles of matched parens and braces and brackets in C. And they're *much* harder to sort out than parens. And if you really don't like parens then you can glue your own syntax onto Lisp. That's something that isn't easy to do with other languages.
> This story isn't about Lisp being used for a
> kick-ass user interface or a 3d engine because
> (IMHO) Lisp isn't as well suited to those
> things.
Lisp was used for one of the first production 3D development environments, Symbolics Inc.'s S-Products. This was later sold to Nichimen Graphics who still uses Common Lisp as their base system for extensive 3D products. Nichimen's systems were used for a number of Nintendo 64 games, for instance.
Kick ass user interfaces can be written in literally *hours* rather than months as in C or C++. And I write user interfaces in C and C++ so I know what I'm talking about. I could do my job in Lisp in one tenth the time, but unfortunately it's too late in the game to rewrite the existing code base.
Don't open your mouth or your foot may fall into it. Always do your research before you post uninformed opinions.
Yes it is the same Robert Morris, but for Pete's sake, don't mod me up so everyone knows!
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
I'm a big fan of Common Lisp as well as Paul Graham (having read his book ANSI Common Lisp, I can see that he possesses both in-depth technical knowledge and a sense of humor). This article seems to match what I've read of him in quality, and I look forward to reading it in more depth when I get home.
I'm not a Lisp guru by any means, but the one thing that I always get a kick out of is how easy it is to become a low-level guru. After a few weeks of playing, I was doing things I wouldn't have dreamed of doing in C or Java. And it changes one's perspective. It all translates to machine code at the lowest level, of course, but after learning Lisp I can say without a doubt that I'm a better C programmer.
Lisp is an interesting language, because in some ways it's ridiculously high level (closures, generic functions, garbage collection, et cetera), but you're also able to get down and dirty with the cons cells with no trouble. I think this quote expresses it best:
Lisp is extremely versatile. While it was originally used in AI, it's honestly the best tool for most situations I come across. (You can see one thing I've done here, and I've done some other stuff that I haven't had a chance to post yet.) Whenever I need to do more in the shell than loop through a few files, I write it in Lisp (I've written 5-line programs to leech an entire Web page's MP3 archive). Lisp is great at processing logs, the output of various subprocesses, and other such things. It's also got a wonderful OO system.Graham's "Blub" example holds true for everyone I've met who has a disdain for Lisp. The advanced features it provides really do go over their heads, which is sad, because these are often intelligent people. Also, they don't look beyond the syntax differences, and often have a lot of misinformation (Lisp is slow, you can't do iteration in Lisp, Lisp is for lists and AI) fed to them by CS professors or whoever. I also often see a lot of posts from newbies who want to write "C-Lisp" and give up when that doesn't work. Lisp is a different paradigm, and needs to be treated as such.
If you're interested in Lisp, I would recommend reading The Evolution of Lisp (don't be angry at the poor fonts in the PDF; they didn't use scalable TeX fonts, the weenies :), Paul Graham's ANSI Common Lisp, and Winston and Horn's LISP, 3rd Edition (but ignore them when they disparage car and cdr), in that order.
Also, don't be confused by the various Lisps out there. First, ignore Emacs Lisp. Among its quirkyisms, it's dynamically scoped, which means that if you declare a variable, every function you call will also have access to that variable. Secondly, Scheme and Common Lisp are vastly different. Scheme is much leaner, and has 1 namespace, which means you can't name a variable and a function the same thing (I dislike this, but it's a hotly contested issue). Common Lisp has a huge set of standard APIs and much more versatility prebuilt into its core, while Scheme tries to stay small so as to provide an easily implementable standard. I'm a Common Lisp man, myself, but try things out for yourself.
One final thing is that if you hang out with them, you'll realize that most of the long-time posters are extremely knowledgable and have a great sense of heritage. I've learned a great deal by simply lurking through their flamewars, since I find out a lot about what issues may crop up for an individual programmer.
If you want a bit more advocacy, see my recent posts on the subject.
--
TO BUY A NEW CAR WOULD MAKE YOU SEXUALLY ATTRACTIVE.
I'm not sure how many people out there will remember, but many moons ago, back around post-Doom Year 1 or so, a non-id software company called crack-dot-com released a nifty side-scroller called Abuse. Not only did Abuse feature a cool mouse-keyboard combo for aiming/moving and some cool level designs, it also featured a complete set of game editing utilities and an honest-to-God LISP interpreter that let you completely reprogram the game. I can't think of too many games that feature their own LISP interpreter, unless you think emacs is fun and difficult enough to qualify as a game. J
Have you actually used a Lisp environment in the past twenty years or so? Every major Common Lisp implementation has a compiler, and at least one doesn't even have an interpreter. What specifically about the condition system do you find inadequate compared to other languages? It definitely offers support at a level at least as high as something like C++ or Java.
As far as doing real work in functional languages, perhaps you ought to ask the Erlang people about that. Or check out this link.
Allegro CL, like implemented by Franz , is _extremely fast_. It is compiled, you can "live on top of the compiler", ie. interactively compile a function and disassemble it.
You can either ignore types, or at the end of a project, optimize it with more specific types (ie define vars/members as floats etc). The hard part is third party libraries, but CL does have a hell of a lot of libraries SPECIFIED IN THE STANDARD. Yes, it has a standard. It also has OO, CLOS, which is a very flexible OO-implementation.
And if you didn't know, check out this:
the control software of the Deep Space 1 (DS1) spacecraft, which is written in Common Lisp, has been chosen together with another system as NASA's 1999 Software of the Year.