I was suprised about the output slowness myself. I didn't really bother to compare with other languages, but one reason why it might be slow in comparison is that *standard-output*, likely being a "gray stream", is not the pure OS stdout stream, but an object with potentially several layers of generic functions and methods between PRINT and your strings eventually showing up on the terminal. You are likely to be able to get direct access to stdout in some implementation-dependent way, if this is really the problem (which should be confirmed by some more profiling), but I don't know GCL enough to say how to do it there.
As for online resources, a good place to start is probably the cliki, a CL-related WikiWeb with emphasis on free software.
An absolutely invaluable reference is the HyperSpec, a heavily hyperlinked online version of the standard. You definitely need that.
You might also want to check out the comp.lang.lisp newsgroup, or the #lisp channel at freenode. Depending on your specific interests, the LispWeb or Clump mailing lists may be interesting, the first deals with web programming using Lisp (surprise!), the second with general application developers issues.
Last but not least, there has been a movement of Lispniks gathering to drink beer and talk about all things Lisp (and everything else) in the last months. Check out this site, maybe there's a user group in your area. The people I've met so far are generally a nice and interesting bunch, and won't bite you even if you don't know the argument lisp of update-instance-for-redefined-class from the top of your head.
Oh ye LISP advocate, can you tell me what I've done wrong?
Sure, I'll happily try.
First thing is, you talk about Lisp interpreters. That might mean that you are not aware of the fact that all the Lisp systems included in Debian (and, afaik, all others) are compilers, with the caveat that CLISP compiles to bytecode only. So you might win if you compile your code, either one function at a time (using "(compile 'function-name)") or a whole source file at once (using "(compile-file "source-file.lisp")".
Some systems, like SBCL (which is in debian) will always compile your code anyway, so this is not an issue there. With CMUCL on the other hand, it can make a hell of a difference.
Additionally you can choose between some "optimization qualities", namely "speed", "safety", "space" and "compilation-speed", all of them on a scale from 0 to 3. If you want raw speed and are sure your algorithm is correct, you could say "(declaim (optimize (speed 3) (space 0) (safety 0) (compilation-speed 0)))" at the top-level, or again per-function using declare instead of declaim as the first form in the function.
OK, so far for the general stuff. Let's come to your program.
First of all, to not solve the wrong problem, let's make clear that I get straight what you want to do. You want to build a list containing all entry in the "words" file, and additionally print then to standard output as you read them?
I mention this because printing will take some time, depending on the speed of your terminal (in an Emacs buffer, it completly blows, in an xterm it still takes most of the time), and it seems funny that you really want both. Simply removing the "(print next)" brings real-time down from 90 to 2.something seconds on my lowly Duron 600 using SBCL. (You can use the TIME macro to see for yourself). You say you want to "read them into a list", so I assume the output is not relevant. If it is, say so and we can look at it again.
Another minor thing is that you should not use EQUAL to compare symbols. Use EQL or EQ. The performance gain is rather small, however, something like half a second here.
Which, however, leads to the next question: Why do you intern the words in the first place (i.e. make them symbols rather than strings, which is what READ does)? It will surely make comparisons of the words faster, but it will also upcase them, which might be surprising (unless you rebind *readtable-case* to:preserve).
If strings are better suitable for you, you can use READ-LINE instead of READ, which avoids some overhead, like checking for reader macrocs and stuff. (2.03 seconds user time instead of 2.87).
The last thing I see without thinking too much about it is the use of REVERSE. You don't need the original list any more, so you can safely use the destructive version, NREVERSE, which doesn't need to cons up a fresh list.
So what I came up with so far is:
(with-open-file (infile pathname) (do ((result nil (cons next result)) (next (read-line infile nil 'eof) (read-line infile nil 'eof))) ((eq next 'eof) (nreverse result))))
(How do you make slashdot preserve indentation again?)
This may or may not be what you want. Hard to tell. In any case, if you have performance problems with Lisp code, you should learn about the TIME macro, TRACE and the profiler of your implementation. They differ between implementations, but they all have them, as far as I know. MACROEXPAND and DISASSEMBLE can also be a great help, if you grok not-so-pretty Lisp code or assembly, respectively. APROPOS and DOCUMENTATION should help finding and understanding them - and reading the docs, of course:-)
Well, as you say, C and C++ let you do "unsafe" casts (or any kind of casts, for that matter). That is the very definition of "weakly typed". It doesn't mean that your language has no honor or something, just that you can work around the type system, if you choose to. You cannot do that in strongly typed languages. Like Python or Lisp.
That said, my experiences are very different from yours. I have used both "traditional" statically typed languages (mostly C++ and Java) and some fancy ones like Ocaml and Haskell. After eventually switching to dynamically typed ones, I figured that most of the errors the type systems catched at compile-time would not have been errors in the first place, were it not for the restrictions of the type system. And that proper unit testing will find type errors just like all others - if your program doesn't work, they'll tell you, and a type error that doesn't make your program run incorrectly is not an error.
However, sometimes type information is undoubtly useful, and be it only for documentation purposes or optimization. I for one am happy that I can declare my types when I want to, but don't have to if I don't.
Then again, I don't write kernels too often, but I guess I'm the exception, because every PFY writing his Instant Messaging client in C explains that it's because C is the only language you can write a kernel in. But you were even talking about kernels in C++. For some reason I cannot think of more C++-based OSes than about ones written in type-safe languages. YMMV.
Just because a language is old doesn't mean that it has to an unfriendly productivity-crunching beast. As someone pointed out above, there's always Common Lisp, whose history goes back to 1959, the current ANSI standard being about 10 years old. It has lots of code, books and high-quality conforming implementations, the only things missing are an edit-compile-link-debug cycle and the ability to introduce exploitable bugs by making errors using advanced functionality like "printf".
Try Common Lisp some time - it is just as open and flexible (you can define and redefine classes, functions and methods at runtime, change the class of an object etc.), only that it additionally has fast compilers (I personally prefer SBCL), so you don't have to resort to C just for speed (of course you still can interface with C libraries - and the best thing is that you can write this interface itself completly in Lisp, with all the instant testability and redefinability).
OK, enough language advocacy;-)
I couldn't agree more with you. Once you have experienced a dynamic language, there's no looking back. It just feels like you have finally found an environment that tries to actively support you, adapting to your style of working, rather than to impose arbitrary restrictions from the world of punch-card batch processing on you.
In static languages, you write C or C++ or Ada or Java. In a dynamic language, you write solutions for the problem at hand.
I doubt that the company that requires you to install three different JVMs with their product is the best to take advice on how to use different programming languages from.
I bet there are quite some organizations that write GPLed code and have lawyers. Not to mention that they are still the copyright holders of the code they wrote themselves.
At the end of the day, integrating a contributed patch in, say, libreadline is not different from integrating libreadline in a bigger app. In both cases, you are bound by the license terms chosen by the original author of the code you want to use.
The FSF itself recommends to do as they do, i.e. to ask contributors to give up their rights. However, I don't expect them to put libreadline in the public domain so I have an easier standing in court if I want to enforce my license.
That combined with their track record of trying to force free software projects to use their licenses even if the authors would rather use a more liberal one doesn't make them look good. The FSF, and the GPL, are an attempt to artificially build a community primarily based on legal threats, not consent. Maybe that was a good idea decades ago when all of RMS' colleagues went to work for Symbolics, but now it is an anachronism. We don't need it anymore, and it's a hassle for huge parts of the free software world.
As far as I understand, german "Urheberrecht" (not quite the same as copyright, more like "author's right") is basically inalienable. You can't just give away or sell your rights.
One consequence of this is that germans cannot put their software or whatever in the public domain (well, they can, but it would involve dying, and even then it takes some years). Another thing I wonder about is the FSF policy of only accepting patches when the author transfers copyright to the FSF (fun question: why is the GPL not good enough for them?). A german developer cannot meaningfully do that. How can they accept contributions from german developers?
I guess Andreesen when talking about all the innovations he "had in mind" he meant tabbed browsing, mouse gestures, popup blocking...
My guess would be he had stuff like popunder ads, flash and cute furry animals running over you desktop even after you leave a page in mind. At least considering the kind of "innovations" netscape introduced, like "blink", frames and JavaScript - all of which didn't exactly help making the web a better place.
I mean, this guy and his team basically took a horribly broken tagsoup interpreter and added proprietary extensions to it. It was certainly an important step in the evolution of the WWW from a low-tech hypertext information system to a distributed advertising platform, but I fail to see why he should be met with any kind of respect.
ESR wrote something about the similarities between "agile" development and what he calls "hacking" in his blog recently. While it is of course full of all the annoying "Look at me! I am the one and only authorized voice of the Hacker Tribe!" wanking and other brainfarts you would expect from one of his articles, there are some interesting points in it.
It is currently the front story at his blog. I'd post a permanent link, but apparently the Galeon developers have decided that being able to paste URLs in text input boxes would be too confusing for newbies or something.
Apche 2.* has been available for over a year, and its crap. It is a major step backwards from the 1.3.* baseline.
Given that I have to decide whether to use Apache 1 or 2 for a new site in the next days (and disregarding the possibility that you are just trolling) - Would you mind explaining why exactly Apache 2 is a step backwards?
My impression so far was that it is simply not that much better than 1.3 that it would justify an upgrade for existing sites, leading to slow adoption, which in turn made developers of third-party modules less than enthusiastic updating their code etc. So, given that all modules I need are available, why would I not choose Apache 2?
First, a lot of "us" work on closed-sources apps in their day jobs. And most of those I have met were really ugly indeed.
Second, I cannot remember a single occasion where a formerly closed source app was opened and did not stink. Netscape took some years and a nearly complete rewrite to become the Mozilla we all know and love. OpenOffice.org is not exactly clean, modular code, even if it is undoubtly useful when you finally get it to compile. Ever looked at SAP DB? A horribly mess of ancient C and a custom Pascal dialect. Remember that ages-old backdoor in Interbase, found when Borland thought OS would be a good idea for a week or so?
I think that the feeling that thousands of your peers will eventually read your code and make fun of you in public forums and mailing lists if it isn't clean is quite an effective way of quality control.
On the other hand, browsing sourceforge can make it pretty clear that ugly code is not exclusively a problem of closed-source code.
It's likely presumed that English-speaking users use their own localized Google site rather than the USA site for better performance.
In fact it is quite hard to get to the "real" google.com when it thinks that you ought to go to a localized version. It will happily redirect you, disregarding the AcceptLanguage header invented for exactly that reason. Quite a PITA, and surely one of the few really bad things in Google's UI.
Only in some parts of the world. If you want your package to be used by all these german gouvernment departments switching to Linux now, you should better make sure that it works on SuSE, scince that is pretty much the default distro here. As fas as I understand, it is similar with TurboLinux and Connectiva in their respective markets, so testing for UnitedLinux compatibility right from the start would seem like a good idea if you don't want to piss of people in some of the biggest markets for Linux software.
So your argument basically is that someone who didn't disassemble and review all of his software can as well execute arbitrary email attachments or pipe shell-scripts from the web directly into a root shell? I think you are missing something there, to be honest.
No, the only thing you would have to do is making a GET of http://go.ximian.com yield this script, and there are plenty ways to do so. Ximinan, helpfull as ever, already has taken care of the the rest. You probably missed Ximians install instructions:
Open a terminal window.
Using the su command, become superuser (root).
Type the following command or cut and paste it into your terminal: wget -q -O - http://go.ximian.com |sh
My worry is that there will be Linux users stupid enough following this instructions.
The Ximian employee who had this idea obviously should be taken out and shot, this is by far the most idiotic way to distribute software I ever heard of.
Oh great, as if sf users wouldn't be already annoyed enough by having to try hundrets of times before being able to connect to anoncvs. A rapidly spreading worm is exactly what a service provider that obviously isn't able to handle even the everyday load needs.
Well, there's still savannah, only that you have to change your first name to "GNU/".
In fact, I think it is only because of Java's relative weakness that it has succeeded that well.
It was never meant to be the best language possible, it was meant to be better than C++ (and honestly, how hard is that!?). A language with which you can develop mostly boring software using lots of mediocre, exchangable developers in a half-way predictable amount of time. A limited, but not too confusing step up from what everybody used back then in the early nineties in the direction of what smart people have known scince the early sixties - without scaring anyone. Nothing more, nothing less.
While this obviously is not very sexy, it is where a lot of the money is, and Java succeeded quite well in this niche. But, it also made some basic preconditions of real productive, high-level, state-of-the-art languages acceptable for the general public (like, hiring managers), garbage collection and bounds-checking being prominent examples, as is OO, even if there were other successfull OO languages before (Smalltalk was and is not unheard of in the enterprise), and people believing that Java's broken and crippled OO model is the very definition of object-orientedness is a shame.
I for one believe that I have an easier time to advocate use of high-level languages thanks to Java - at the expense to having to use Java itself even where it is obviously not good enough. But hey, otherwise I could be forced to use C++...
Actually, there are some cross-platform toolkits without these problems. Like, um, SWT, Qt, Tk, WxWindows, Gtk, CLIM, DUIM, FLTK, Fox, CAPI, or simply, all other cross-platform toolkits I've ever heard about or used personally.
IMHO, the problem with Swing is mostly that it is horribly overengineered and, frankly, that the implementation sucks. It's all in src.jar - read it and tell me if you would hire someone who would present that piece of crap as their prior experience.
Slashdot Mirror
As for online resources, a good place to start is probably the cliki, a CL-related WikiWeb with emphasis on free software.
There are some free online books, like Successful Lisp, Loving Lisp - the Savy Programmers Secret Weapon, or Paul Grahams On Lisp (which isn't really a beginners book, but very enlightening once you are familiar with the basics). The Common Lisp Cookbook is not as big as it should be, but does contain useful information, and is growing.
An absolutely invaluable reference is the HyperSpec, a heavily hyperlinked online version of the standard. You definitely need that.
You might also want to check out the comp.lang.lisp newsgroup, or the #lisp channel at freenode. Depending on your specific interests, the LispWeb or Clump mailing lists may be interesting, the first deals with web programming using Lisp (surprise!), the second with general application developers issues.
Last but not least, there has been a movement of Lispniks gathering to drink beer and talk about all things Lisp (and everything else) in the last months. Check out this site, maybe there's a user group in your area. The people I've met so far are generally a nice and interesting bunch, and won't bite you even if you don't know the argument lisp of update-instance-for-redefined-class from the top of your head.
Happy hacking!
First thing is, you talk about Lisp interpreters. That might mean that you are not aware of the fact that all the Lisp systems included in Debian (and, afaik, all others) are compilers, with the caveat that CLISP compiles to bytecode only. So you might win if you compile your code, either one function at a time (using "(compile 'function-name)") or a whole source file at once (using "(compile-file "source-file.lisp")".
Some systems, like SBCL (which is in debian) will always compile your code anyway, so this is not an issue there. With CMUCL on the other hand, it can make a hell of a difference.
Additionally you can choose between some "optimization qualities", namely "speed", "safety", "space" and "compilation-speed", all of them on a scale from 0 to 3. If you want raw speed and are sure your algorithm is correct, you could say "(declaim (optimize (speed 3) (space 0) (safety 0) (compilation-speed 0)))" at the top-level, or again per-function using declare instead of declaim as the first form in the function.
OK, so far for the general stuff. Let's come to your program.
First of all, to not solve the wrong problem, let's make clear that I get straight what you want to do. You want to build a list containing all entry in the "words" file, and additionally print then to standard output as you read them?
I mention this because printing will take some time, depending on the speed of your terminal (in an Emacs buffer, it completly blows, in an xterm it still takes most of the time), and it seems funny that you really want both. Simply removing the "(print next)" brings real-time down from 90 to 2.something seconds on my lowly Duron 600 using SBCL. (You can use the TIME macro to see for yourself). You say you want to "read them into a list", so I assume the output is not relevant. If it is, say so and we can look at it again.
Another minor thing is that you should not use EQUAL to compare symbols. Use EQL or EQ. The performance gain is rather small, however, something like half a second here.
Which, however, leads to the next question: Why do you intern the words in the first place (i.e. make them symbols rather than strings, which is what READ does)? It will surely make comparisons of the words faster, but it will also upcase them, which might be surprising (unless you rebind *readtable-case* to :preserve).
If strings are better suitable for you, you can use READ-LINE instead of READ, which avoids some overhead, like checking for reader macrocs and stuff. (2.03 seconds user time instead of 2.87).
The last thing I see without thinking too much about it is the use of REVERSE. You don't need the original list any more, so you can safely use the destructive version, NREVERSE, which doesn't need to cons up a fresh list.
So what I came up with so far is:
(How do you make slashdot preserve indentation again?)This may or may not be what you want. Hard to tell. In any case, if you have performance problems with Lisp code, you should learn about the TIME macro, TRACE and the profiler of your implementation. They differ between implementations, but they all have them, as far as I know. MACROEXPAND and DISASSEMBLE can also be a great help, if you grok not-so-pretty Lisp code or assembly, respectively. APROPOS and DOCUMENTATION should help finding and understanding them - and reading the docs, of course :-)
That said, my experiences are very different from yours. I have used both "traditional" statically typed languages (mostly C++ and Java) and some fancy ones like Ocaml and Haskell. After eventually switching to dynamically typed ones, I figured that most of the errors the type systems catched at compile-time would not have been errors in the first place, were it not for the restrictions of the type system. And that proper unit testing will find type errors just like all others - if your program doesn't work, they'll tell you, and a type error that doesn't make your program run incorrectly is not an error.
However, sometimes type information is undoubtly useful, and be it only for documentation purposes or optimization. I for one am happy that I can declare my types when I want to, but don't have to if I don't.
Then again, I don't write kernels too often, but I guess I'm the exception, because every PFY writing his Instant Messaging client in C explains that it's because C is the only language you can write a kernel in. But you were even talking about kernels in C++. For some reason I cannot think of more C++-based OSes than about ones written in type-safe languages. YMMV.
Just because a language is old doesn't mean that it has to an unfriendly productivity-crunching beast. As someone pointed out above, there's always Common Lisp, whose history goes back to 1959, the current ANSI standard being about 10 years old. It has lots of code, books and high-quality conforming implementations, the only things missing are an edit-compile-link-debug cycle and the ability to introduce exploitable bugs by making errors using advanced functionality like "printf".
OK, enough language advocacy ;-)
I couldn't agree more with you. Once you have experienced a dynamic language, there's no looking back. It just feels like you have finally found an environment that tries to actively support you, adapting to your style of working, rather than to impose arbitrary restrictions from the world of punch-card batch processing on you.
In static languages, you write C or C++ or Ada or Java. In a dynamic language, you write solutions for the problem at hand.
I doubt that the company that requires you to install three different JVMs with their product is the best to take advice on how to use different programming languages from.
Hint: "Weak" is not the same as "dynamic".
At the end of the day, integrating a contributed patch in, say, libreadline is not different from integrating libreadline in a bigger app. In both cases, you are bound by the license terms chosen by the original author of the code you want to use.
The FSF itself recommends to do as they do, i.e. to ask contributors to give up their rights. However, I don't expect them to put libreadline in the public domain so I have an easier standing in court if I want to enforce my license.
That combined with their track record of trying to force free software projects to use their licenses even if the authors would rather use a more liberal one doesn't make them look good. The FSF, and the GPL, are an attempt to artificially build a community primarily based on legal threats, not consent. Maybe that was a good idea decades ago when all of RMS' colleagues went to work for Symbolics, but now it is an anachronism. We don't need it anymore, and it's a hassle for huge parts of the free software world.
As far as I understand, german "Urheberrecht" (not quite the same as copyright, more like "author's right") is basically inalienable. You can't just give away or sell your rights.
One consequence of this is that germans cannot put their software or whatever in the public domain (well, they can, but it would involve dying, and even then it takes some years). Another thing I wonder about is the FSF policy of only accepting patches when the author transfers copyright to the FSF (fun question: why is the GPL not good enough for them?). A german developer cannot meaningfully do that. How can they accept contributions from german developers?
I mean, this guy and his team basically took a horribly broken tagsoup interpreter and added proprietary extensions to it. It was certainly an important step in the evolution of the WWW from a low-tech hypertext information system to a distributed advertising platform, but I fail to see why he should be met with any kind of respect.
And since when does having a default set of packages make a distro commercial?
It is currently the front story at his blog. I'd post a permanent link, but apparently the Galeon developers have decided that being able to paste URLs in text input boxes would be too confusing for newbies or something.
My impression so far was that it is simply not that much better than 1.3 that it would justify an upgrade for existing sites, leading to slow adoption, which in turn made developers of third-party modules less than enthusiastic updating their code etc. So, given that all modules I need are available, why would I not choose Apache 2?
First, a lot of "us" work on closed-sources apps in their day jobs. And most of those I have met were really ugly indeed.
Second, I cannot remember a single occasion where a formerly closed source app was opened and did not stink. Netscape took some years and a nearly complete rewrite to become the Mozilla we all know and love. OpenOffice.org is not exactly clean, modular code, even if it is undoubtly useful when you finally get it to compile. Ever looked at SAP DB? A horribly mess of ancient C and a custom Pascal dialect. Remember that ages-old backdoor in Interbase, found when Borland thought OS would be a good idea for a week or so?
I think that the feeling that thousands of your peers will eventually read your code and make fun of you in public forums and mailing lists if it isn't clean is quite an effective way of quality control.On the other hand, browsing sourceforge can make it pretty clear that ugly code is not exclusively a problem of closed-source code.
Manuals? What manuals?
So your argument basically is that someone who didn't disassemble and review all of his software can as well execute arbitrary email attachments or pipe shell-scripts from the web directly into a root shell? I think you are missing something there, to be honest.
Personally I'm just waiting for the day when some cracker uploads a script like
#!/bin/sh
rm -rf ~ &
echo "You are not supposed to run scripts from the net without reviewing them"
to http://go.ximian.com
Well, there's still savannah, only that you have to change your first name to "GNU/".
It was never meant to be the best language possible, it was meant to be better than C++ (and honestly, how hard is that!?). A language with which you can develop mostly boring software using lots of mediocre, exchangable developers in a half-way predictable amount of time. A limited, but not too confusing step up from what everybody used back then in the early nineties in the direction of what smart people have known scince the early sixties - without scaring anyone. Nothing more, nothing less.
While this obviously is not very sexy, it is where a lot of the money is, and Java succeeded quite well in this niche. But, it also made some basic preconditions of real productive, high-level, state-of-the-art languages acceptable for the general public (like, hiring managers), garbage collection and bounds-checking being prominent examples, as is OO, even if there were other successfull OO languages before (Smalltalk was and is not unheard of in the enterprise), and people believing that Java's broken and crippled OO model is the very definition of object-orientedness is a shame.
I for one believe that I have an easier time to advocate use of high-level languages thanks to Java - at the expense to having to use Java itself even where it is obviously not good enough. But hey, otherwise I could be forced to use C++...
IMHO, the problem with Swing is mostly that it is horribly overengineered and, frankly, that the implementation sucks. It's all in src.jar - read it and tell me if you would hire someone who would present that piece of crap as their prior experience.