Slashdot Mirror
Facebook Releases Open Source Web Server
Dan Jones writes "Ah the irony. The week Facebook is being asked to cough up source code to satisfy an alleged patent infringement, the company releases an open source Web server. The Web server framework that Facebook will offer as open source is called Tornado, was written in the Python language and is designed for quickly processing thousands of simultaneous connections. Tornado is a core piece of infrastructure that powers FriendFeed's real-time functionality, which Facebook maintains. While Tornado is similar to existing Web-frameworks in Python, it focuses on speed and handling large amounts of simultaneous traffic."
Facebook was built with PHP?.
I don't think it means what you think it means.
They are not the same thing (as the article makes clear).
This sounds interesting and will definitely take a look - but I doubt I'll be ditching Pylons any time soon.
I posted this news first, but it seems somebody else got it on the front page :/
http://slashdot.org/firehose.pl?op=view&id=5864125
Tee.do Lean Task Management
"Thats not IRONY chumps & chumpettes, its just coincidental".
Tornado includes both a Web server and a Web framework. The framework can take advantage of the (non-blocking) server architecture to achieve high performance. Apparently you can also run it under mod_wsgi, but I can't really see an advantage of using it in that scenario when compared to other Python frameworks.
I wonder if the Tornado authors set forth to re-implemented Twisted Python just for kicks or out of not knowning about its existence.
Twisted supports epoll kqueue, win32 iocp, select, etc.
Lone Gunmen crew.
Why not use something like http://www.simpleframework.org/ which is far faster than anything using epoll, and has all the asynchronous capabilities, used in combination with Jython and you have the same capabilities only better performance?
Twisted is hard to learn. It's the sort of thing that programmers will re-implement just to avoid reading the documentation.
Or maybe they wanted to have control. Whatever the case, they would have know. Everybody (who uses python for web work) would know a bit about Twisted ... it's on the front page of python.org
It's just coincidental!
Twisted is a networking library, this seems to be a webserver (which Twisted can do) as well as a framework
If this webserver is supposed to be fast, than just how fast is it? Is it faster than lighttpd? YAWS? I'd like to know.
Most Python web servers use threading or multiple processes to handle concurrent requests and are not implemented as event driven systems. Most Python web applications are not designed to be implemented on event driven systems but rely on the ability to block during handling of web requests, something which the former allows but which doesn't work well with event driven systems as it blocks the main event loop and prevents anything else happening. So, it is not similar to other Python web servers or frameworks.
It should be further highlighted that WSGI for Python is effectively designed for that blocking model and it isn't really a good idea to be using it with a server based on event driven systems model and which uses multiple processes as well. Attempting to do so can have undesirable effects such as described in 'http://blog.dscpl.com.au/2009/05/blocking-requests-and-nginx-version-of.html'. Some seem to hope that WSGI 2.0 will support asynchronous systems but reality is that it almost definitely will not, so they should stop dreaming.
So, although these sorts of high performance servers are interesting, their applicability to most existing Python web applications is limited because in practice the web application has to be designed around the event driven system model and you can't really use standardised Python WSGI interface and components that build on that.
This doesn't mean that these type of servers aren't useful, they just aren't going to solve everyones problems and will principally remain a niche solution for things that need to main many long lived connections.
As to the benchmarks they give, it is very much just a pissing competition and nothing more. The bulk of web sites would never even handle enough hits to trouble the limits of the other hosting solutions they compare to. For larger sites, they are never going to use a single machine anyway, but use a cluster of machines to spread load and for redundancy. Yes, it may provide more head room for individual machines, but again we aren't talking about a situation which the majority would even have to deal with.
"The biggest thing that surprised us - or is about to surprise us - is the less-than-anticipated performance gains from new microarchitectures... The performance gains they are touting in the press, we are not seeing in our applications. We are literally in real-time trying to figure out why that is and if there are optimizations that we can do."
-- Jonathan Heiliger, vice president of technical operations at Facebook, 26th June 2009
well maybe if they're weren't using such high level languages for everything then things might run a bit faster...
I'm not trolling here but I just can't understand what this web server brings new to the table, after having Apache, lighthttpd, ngnix, Xitami or Zeus.
Catalin Braescu
Ofaly.com
I doubt there has been a week without me having some problem with Facebook functionality. Usually it means I can't access a friend's profile, some photos or such for a duration ranging from half an hour to several hours. (Rather big bug when that is what Facebook is used for) I've often heard that someone I know hasn't been able to login during a whole day. The list goes on and on - similar big problems are rather frequent.
Lesser bugs (such as me getting some notification multiple times, etc.) occur often several times a day.
I understand that having that large userbase must be difficult. It is understandable that some bugs come up every once in a while. Their service works "OK, good enough" most of time as it isn't used for anything crucial or really important so bugs don't matter. However, I would rejoice about this release a lot more if Facebook had the record of providing high quality, bug free service.
As we say in #python, "Programming is hard!"
Learning Twisted is so much easier than rolling your own networking mini-library. Sure, a lot of people are kicking and screaming in the beginning, but once they actually sit down and start coding, they usually say something like, "Oh, hey, this is nice."
There's a reason it's popular.
~ C.
They explicitly states that they looked at Twisted and chose to write something more user-friendly. Having looked at Twisted (3-4 years ago though) and at Tornado's samples and benchmarks I think they succeeded. Twisted seems to be going the way of Zope: an interesting platform that did everything its own way and shut itself out from the rest of the Python universe, eventually losing relevancy.
I think a Tornado/Django mashup (Tornado infrastructure, Django front-end/application bootstrapping) would be realllly interesting....
The twisted folks have been working on web frameworks for years (nevow/athena comes to mind). One problem with twisted is that the core devs don't focus much on marketing (ala RoR) so not many people know about it. These guys had a good comet implementation before the phrase was coined.
Bret Taylor says:
When we started, we did use Twisted. In practice, I found Twisted tedious. The deferred abstraction works, but I didn't love it in practice. Likewise, the HTTP/web support in Twisted is very chaotic (see http://twistedmatrix.com/trac/wiki/WebDevelopment ... - even they acknowledge this). In general, it seems like Twisted is full of demo-quality stuff, but most of the protocols have tons of bugs.
Given all those factors, it didn't seem to provide a lot of value. Our core I/O loop is actually pretty small and simple, and I think resulted in fewer bugs than would have come up if we had used Twisted.
I wonder if the Tornado authors set forth to re-implemented <a href="http://twistedmatrix.com/trac/">Twisted Python</a> just for kicks or out of not knowning about its existence. Twisted supports epoll kqueue, win32 iocp, select, etc.
And what makes you think they didn't know? Are you privy of information that objectively and clearly indicates the authors DID NOT have any valid technical or business reason AT ALL to implement Tornado as opposed to adopt Twisted?
To be honest, I don't know of any evidence, for or against. I have no clue of their reasons (intelligent and/or stupid). As a result I don't assume either. A more constructive and useful question would have been I wonder what were the technical or business reasons (if any) that lead Facebook to implement Tornado? Did they find a technical problem with Twisted? Did they have a strategic reason not to use it? Did they already have a lot of functionally-related Python code built in-house, making the creation of Tornado a reasonable step? I would like to know so that I can clearly understand this on its own merits.
I dunno, it's the pragmatic engineer in me talking here.
I know most shitty web news sites can't tell one license apart from another but in the real world, it makes a huge difference. I expected higher standands from /.
Apache 2.0 is the answer here by the way.
If there is a surgeon that kills 9 out of 10 patients, I can still criticize him. Even if I'm not a surgeon at all and couldn't save even 1 out of 10 patients if I had to operate any.
I never made it past the kicking and screaming part :-).
I remember when legal used to mean lawful, now it means some kind of loophole. - Leo Kessler
Threads suck ass anyway.
I find it ironic that Facebook was able to open-source anything with Microsoft as a major sponsor.