Slashdot Mirror

I'd rather bug hunt crashes and core dumps than dig through bugs related to misuse of massive dynamic type systems.

On the otherhand I would not want one silly double free() bug to take out my entire site.

I demand something that is:

1. Fast
2. Robust
3. Easy to diagnose/debug
4. Secure
5. Free

When I find it, I'll be sure to share it with all of you. Well, there is Yaws, but it's written in Erlang (not sure how to exclude Yaws given my list).

1. Multicore ready.
Erlang will use them. Write your application in Erlang and it's done for you.

2. Scales well.
As an example, http://yaws.hyber.org/ scales very nicely when loads increase. Your basic LAMP/LYMP setup runs much better on vanilla hardware.

3. Designed for telecom
The architects designed the language to run in a telecom environment so things like upgrades can be done while the application is running.

Yaws in particular needs your help. Failover clustering inside the yaws server would be wonderful. Right now, it uses CGI to process other languages. It does it flawlessly, but a more direct solution might be a nice project.

You mean like Yaws? To quote from their website: "...one Erlang light weight process is used to handle each client...". Of course, they also say: "...performance comes from the underlying Erlang system and its ability to handle concurrent processes in an efficient way...". Using "processes" doesn't necessarily mean using heavyweight OS processes. It mostly means avoiding shared state, which is the root of the problems with threads. Languages like occam and Erlang provide lightweight processes with extremely fast context switching and quick interprocess communication.

A lot of this is due to the fact that most popular languages right now do not support concurrency very well. Most common languages are stateful, and state and concurrency are rather antithetical to one another. The solution is to gradually evolve toward languages that solve this either by forsaking state (Haskell, Erlang) or by using something like transaction memory for encapsulating state in a way that is easy to deal with (Haskell's STM, Fortress (I think), maybe some others).

Concurrency is not that hard to do well in the right setting.

And before anyone claims that Haskell and Erlang are impractical, there are many examples of "real world" programs written in them.

A few nice, and very useful ones are Yaws (for erlang) and Darcs (for Haskell). There are many others (even quake clones), which I won't bother listing, but people can find them easily if they look.

Regarding concurrency, and its ease of use in these languages, I'm taking a machine learning class at the moment where most of the problems are computationally intensive, and could stand for improvement by making use of multiple cores. I do all of my assignments in Haskell, and not only are my solutions often shorter than those of my classmates (and they often work fine the first time they compile), but it's usually trivial to allow my application to scale nicely to as many cores as I can throw at it. It's worth mentioning, by the way, that most algorithms given in these classes are given under the assumption that people are using imperative languages, and even then, it's still easy. It takes a while to learn how to approach problems differently without mutable state, yes, but it's not as hard as some people make it out to be. I think it has more to do with the fact that people just don't like to learn anything new unless they absolutely are forced to do so, which is a pity.

By the way, there is a nice presentation from Tim Sweeney on what he would like future programming languages to look like, and there's a lot in there about functional programming, concurrency, and expressive (re: dependent) types.

And Windows is written in C#? Ok:
Jane Street Capital.
A high performance Erlang webserver.

On a loaded server where each client has n+ connections, you can run up against concurrency limits quickly. Some servers handle concurrency better than others.

Interesting class. Looks pretty standard for a US/Canada education.

No, in fact I mean CSP - see http://www.usingcsp.com/

Have a look at yaws: http://yaws.hyber.org/ a high performance webserver written in Erlang.

> Is BIND really that pathetic, or are they just not using it correctly?

Here's a performance comparison of the ubiquitous Apache web server with Yaws, an Erlang-based web server. (Erlang is a programming language and virtual machine designed for distributed processing.) To summarize, "Apache dies at about 4,000 parallel sessions. Yaws is still functioning at over 80,000 parallel connections." The author goes on to speculate that the reason Apache dies so quickly is due to limitations in the host operating system.

If Erlang can keep a web server going under nearly infinite load, imagine what it could do for DNS.

aristus,
I started looking at erlang in Fall 2004. Its my first serious try at functional programing so I consider my a newbie in functional idioms. However, erlang is not purely functional and is accessible to people with OO experience.
I was a lurker on the erlang-questions maillist for 6 months before making a final decision to start building my apps with erlang. The maillist community is very solid and should be considered a base for your ongoing adoption of erlang.
Here is the maillist and faq link:
http://www.erlang.org/faq.html
If you download any erlang install http://www.erlang.org/download.html, lots of docs are included. There are also some docs not included:
http://www.erlang.org/doc.html
I consider this doc a must read for anyone starting erlang:
http://www.erlang.org/download/erlang-book-part1.p df
The http://www.trapexit.org/ site is a very good portal for docs and the forums are a bi-directional link to the erlang-questions maillist.
The howtos are good to start with:
http://www.trapexit.org/docs/howto/
If you will develop anything requiring a web server, go directly to http://yaws.hyber.org./ Mostly good docs here. Yaws is the most excellent web server written in erlang.
jungerl is a repository of mostly small erlang libraries. Lots of good tools and example code in here:
http://jungerl.sourceforge.net/
ejabberd http://ejabberd.jabber.ru/, a jabber server in erlang, is an excellent example of how to put together a large distributed web based system in erlang.

nothing is perfect, but erlang's benefits to me far outway its rough edges...hopefully, you will experience the same.
good luck...

Then you'll want to look into YAWS.

Basically, a web server written in Erlang, which supports lightweight processes and high concurrency. In other words, each connection is a completely separate process and shares no information with other processes except by message passing.

Also, a recent paper from the primary designer of Erlang, Joe Armstrong.

The key points are that Erlang process creation and message passing are orders of magnitude faster than Java/C# threads. Also, YAWS could handle dedicated traffic from a 16 machines. It handled over 80,000 connections, maintaining 800 kB/s traffic. Apache died around 4,000 connections. The key graphic is on page 4 of the paper. The red lines denote YAWS; notice haw it maintains that bandwidth (even though particular connections may drop, the web server keeps chugging along). Threaded Apache is in green; process-forking Apache is in blue.

they want to make sure everyone is nice and compliant about upgrading when they decide to take httpd over to java like all the other java kool-aid they are selling --- Maven is Jonestown, lets all program in XML because its standard! Cultures breakdown when there is too little disent and questioning of authority, the apache foundtion is headed in that direction.

Lets move on, SOA and all that, most people don't need any of this mod_* crap and could use:
thttpd he has other servers there, too and http_load.
lighttpd I'm moving to this sweet little server for most apps and the home site runs ea php and ruby on rails
AOLServer like OpenACS runs on
Boa
fnord from our boy who did the (in)famous benchmarks
Cherokee I root for this one for some reason.
gatling
cthulhu
yaws in erlang, should support more simul. connections than the unlying OS can support.
dhttpd
Litespeed check out their php benchmarks
thy
roxen
mini-httpd never tried this one
xitami I have a intranet server running for 5 yrs (without upgrading xitami) on xitami Solaris, simple, small, easy to admin, never dies max uptime was 1000 days+.
eddiefor complex load bal and geographic distribution
hiawatha

And for the love of god, please at least design your sites to get their images from images.mysite.com if possible so that you can use a non-bloatware web server to server the images, reserving horsepower on your apache server for stuff that actually _requires_ some features of apache.

http://www.hcsw.org/awhttpd/ updated on 12-06-2004
http://www.norz.org/zawhttpd.html
http://cr.yp.to/publicfile.html

Go to Erlang. It is intended for robust and concurrent (as in webserver) applications. Use yaws as your webserver.
Erlang is free as in beer and in speech. It comes with a distributed database, CORBA and many other goodies.
Everyone I know that tried it, liked it and prefered it over Java and PHP. The participants on the Erlang mailing lists are very supportive, too.

None of the readers seem to anything about it...

A few points:
* interpreted languages nowadays tend to compile to native code
* the compiler or run time is better at optimising than you are
* performance is not the same as scalability, a scalable architecture is more importantant than Mips (Mips are cheap commodities which you buy in inexpensive boxes from HP or Dell - if your architecture supports it)
* most real life systems have latencies (io, kernel activities, screen painting) which are only tangentially related to the grunt of your piece of code (see architecture)
* Erlang is uber scalable, designed to build reliable high performance systems. It provides its own (very lightweight) concurrency and means the developer doesn't use expensive OS process and thread spawning and doesn't have to worry about managing threads and synchronisation.

Check out the Yaws (yet another web server) versus Apache benchmarks for throughput and concurrency.

Erlang systems also stay up. The main BT telephony system only had 31ms downtime in its first year (see here (page 30).

Microbenchmarks about string concatenation and method calling are mostly irrelevant. What's important is too see how an implementation scales up to huge programs, say 500,000 lines of code and 1GB of data. There have been plenty of cases of fast languages not scaling up well (like C++), because you have to add so much cruft to hande the kind of problems you hit at that level, and much "slower" languages outrunning them because they were architected for that kind of problem.

One interesting case is Yet Another Web Server (YAWS), which is written in an interpreted language and completely blows the socks off of Apache under high load.