Slashdot Mirror

I second Python: just type in the pseudocode you'd write on a piece of paper, and there is a good chance that it will work just like that in Python.

2D gaming using SDL (and OpenGL 3D, but you have to do the hard work yourself): Pygame
3D drawing/animation/gaming: Blender 3D
(I started by gaming, because that's a fun way to learn a language quickly)

Web: Django
Co-routines: Stackless Python
Maths: NumPy and SciPy
Networking: Twisted

That just scratches the outside of it, but have a look at the above to get an idea of the language.

And Python's documentation is quite good: brief, but everything you need is there - you just need less than you would expect at first. Here are some good tutorials:

Official Python Tutorial
Dive into Python
How to think like a computer scientist?

Hmmm, looks like I've turned into a Python fanboi... Be careful if you try Python, you could fall for it.

You're going to have to find a project that challenges you and holds you to high standards. You're going to have to do it fast or else you'll forget coding was ever like that.

Enter Twisted: http://twistedmatrix.com/

Twisted is a 10-year-old Python project, still alive and hopping with a large community. It has an extensive review process that lets you read others' code -- indeed, the core maintainers won't commit their own patches until a non-core dev reviews them. It has solid code quality guidelines, and it's used all over the real world.

http://twistedmatrix.com/trac/wiki/ReviewProcess explains that.

If Twisted isn't up your alley because you don't like networking, take the same advice but apply it elsewhere: Find a project that will hold you to high standards, submit patches, and feel the love. (-:

(I don't work on Twisted much myself, though I plan to go to a code sprint of theirs this weekend. I can offer other suggestions within open source if you join #openhatch on irc.freenode.net.)

You're going to have to find a project that challenges you and holds you to high standards. You're going to have to do it fast or else you'll forget coding was ever like that.

Enter Twisted: http://twistedmatrix.com/

Twisted is a 10-year-old Python project, still alive and hopping with a large community. It has an extensive review process that lets you read others' code -- indeed, the core maintainers won't commit their own patches until a non-core dev reviews them. It has solid code quality guidelines, and it's used all over the real world.

http://twistedmatrix.com/trac/wiki/ReviewProcess explains that.

If Twisted isn't up your alley because you don't like networking, take the same advice but apply it elsewhere: Find a project that will hold you to high standards, submit patches, and feel the love. (-:

(I don't work on Twisted much myself, though I plan to go to a code sprint of theirs this weekend. I can offer other suggestions within open source if you join #openhatch on irc.freenode.net.)

I just lost everything I was writing because Slashdot discarded my post when changing to plain text. I'm not going to hunt down references again. Google yourself.

Here's the rundown:

[I'm talking about the difficult in embedding Python in an application (extending the application. This is the usual domain of a scripting language.]

1) Python is huge. It is improper for small devices like FPGAs. Python weighs in at multiple megabytes.
2) Python is very difficult to sandbox.
3) It is difficult to have multiple independent Python instances running concurrently (GIL).
4) It is difficult to have multiple contexts. Python lacks proper coroutines.
5) Python is built with the extend rather than embed mindset e.g. [1].
6) Python is whitespace sensitive. It is unnecessarily difficult to write small embedded scripts or macros for your application (see WoW).
7) Python libraries make it difficult to embed in e.g. an ANSI C only environment.

[1] http://www.twistedmatrix.com/users/glyph/rant/extendit.html

Bret Taylor says:

When we started, we did use Twisted. In practice, I found Twisted tedious. The deferred abstraction works, but I didn't love it in practice. Likewise, the HTTP/web support in Twisted is very chaotic (see http://twistedmatrix.com/trac/wiki/WebDevelopment ... - even they acknowledge this). In general, it seems like Twisted is full of demo-quality stuff, but most of the protocols have tons of bugs.
Given all those factors, it didn't seem to provide a lot of value. Our core I/O loop is actually pretty small and simple, and I think resulted in fewer bugs than would have come up if we had used Twisted.

I wonder if the Tornado authors set forth to re-implemented Twisted Python just for kicks or out of not knowning about its existence.

Twisted supports epoll kqueue, win32 iocp, select, etc.

Yes, I know that. I even wrote very similar stuff, using http://twistedmatrix.com/ framework for Python and it was much more lightweight. :-) Still, this is not really a true offline, but an emulation of one, by substituting your external server with a local server (means, you still have client-server), then sync, once connection to external server is established.

In other words, Gears is a hack. And Chrome OS must run it as well. And must run it hidden from a user's eyes, since user supposed to see only a full-screen browser.

To make a Chrome OS yourself: embed Gecko engine into Java Swing component on very limited small Linux (there is some libraries to do so), run it fullscreen on X11 root and that's it. :-)

But unfortunately, this lack of fluff comes with a drawback: the CPython bytecode interpreter is less efficient than the JVM. In fact, the JVM was found to be faster than CPython for everything but startup time and I/O. From that page, I gather that it's more likely with Python than with Java that you'll have to rewrite an inner loop in C and then recompile it for each CPU/OS combination.

It occurs to me that this debate has lots of comparisons with the current minor furor over the rise of multiple CPUs. Many in the programming industry are despairing because we are being forced to design software that runs on multiple CPUs at once, rather than just getting more CPU speed. And again, it's due to externally imposed constraints. In the automobile industry, the catalytic converter was that constraint, in the lead electronics debate, the RoHS is that external constraint.

In the programming world, I expect things to be resolved the same way: by superior engineering, taking advantage of the mountains of research and practical application of parallel processing designs that have been going on for decades. My favorite language Python is particularly sensitive to this debate due to something called the GIL, but solutions abound, including the newly accepted pyprocessing module or any number of things you can do with Twisted, both of which stand as examples of better, practical engineering taking advantage of known solutions to the problem.

Engineering fixes most problems that changing standards and regulations cause, eventually.

You are wrong, so maybe your tone should be a little more humble :-)

Its obvious that you don't know about asynchronous I/O. Single-threaded designs are not meant for you to create threads in your event handlers, but to do your event handling asynchronously. See Twisted as an example.

This gets rid of all the thread problems, and introduces a few problems which are far easier to handle. The main problem it introduces, is that long computations or blocking operations will block the entire program. However, finding those bugs, and fixing them (e.g: converting those to be split/asynchronous, or very rarely, have their own process), is a much easier and more cost-effective process.

Since multithreading is a real requirement for performance and scalability these days, (threads scale for performance better than processes, and you can always have more processes too)

No! General multithreading is not the solution for extreme performance, especially on web applications. Multithreaded applications have too many locks and context-switching. You want something like a libevent driver or Python Twisted that does single-threaded event dispatching with asynchronous I/O, so you can avoid locking and context switches.

What's more, thread concurrency is really a red herring here. You only want enough threads to keep a single CPU busy without thrashing... server farms get more concurrency simply by adding CPUs, and that is far more effective than throwing 10,000 threads onto a single server (take Google, for example). If you're writing multithreaded PHP scripts that are accessed from the web, I am sorry for whoever has to come maintain it. Much cleaner to multithread by using object, iframe, or other client-side methods of invoking multiple server-side scripts concurrently.

If you're interested in Python Web development, you'll find a host of network and Web specific frameworks. I suggest checking out Twisted, Zope, Plone, and Django for examples. You may also find some other goodies when you explore the Python Cheese Shop.

Of course, no mention of Python can pass by without someone bringing up Ruby on Rails, so I'll just do that right now. :) However, I have no experience with it whatsoever, so I'll withhold any opinion.

But a lot of problems don't fit these models, and need a LOT of thought put into how to parallelize them. It's likely that some problems in P are not efficiently parallelizable.

• anything (comparable) can be sorted using divide-and-conquer mergesort
• scanning through an array-based collection (*not* a linked list) can be divided among processors—this is frequently done in hardware, e.g. for CPU cache hashtable lookups

• Split program into a chain of filters between producers and consumers, and give each filter its own thread/process. For example, create an event receiver thread, a "do-stuff" thread, and a display thread. At the very least, this will reduce UI response latency.
• Split program processing into "1 event dispatcher + N worker threads", like Apache or Squid. This by itself would be a good way to reduce blocking in most applications. Why should the interface be locked up when expensive processing is happening in a program? Maybe while Photoshop/GIMP runs some filter on my image, I'd like to browse the help documents or scroll around the viewport.
• Re-evaluate any processing as a dependency tree, and code it in something like Twisted, where every piece of code executes nonblocking snippets, and a reactor thread dispatches between them (this is basically like light-light-weight threads)

The reason the problems don't fit these models is moreso that we're used to thinking about algorithms as an ordered list of steps, rather than a set of workers on an assembly line (operating as fast as the slowest individual worker).

Solaris tar sucks. It can't handle long filenames and renames them to some weird looking thing that reminds me of Progra~1 fugly-ness. Just what the doctor ordered a piece of backup software to do. A few random samples of people on the net complaining about Solaris tar:

http://www.mikehan.com/rant/solaris-tools.html
http://twistedmatrix.com/trac/ticket/1750
http://42.pl/postfix/postfix-2.2.2/examples/chroot -setup/Solaris10
http://www.idevelopment.info/data/MySQL/DBA_tips/I nstalling/SOLARIS323_2.shtml
http://lists.cs.uiuc.edu/pipermail/llvm-commits/We ek-of-Mon-20040809/017086.html
http://justlinux.com/forum/archive/index.php/t-405 84.html

> if HR calls you and wants you to work for them, it implies they have researched your background and have something in mind.

I've seen recruiters send that kind of mails to mailing list. Don't be so sure... Ah, found one on a Python mailing list.

Twisted DNS should be pretty customizable, although your "I'm not adept at coding" will probably make it hard for you. Still, I throw it out there, in case someone else has an application that requires customized DNS. From a cleanliness and safety aspect, it's sure a better starting point than BIND.

so they wrote an asynchronous proxy that slows down connections. cool trick, but not any kind of scalable solution.

the core assumption, and the only thing that makes this work, is that botnet spam software will _always_ just give up after 30 seconds; if this throttling technique ever became commonplace, spammers would just write their own asynchronous mailer -- it's not THAT hard. windows has the same kind of async networking support (either through the winsock API and/or IO completion ports, or what have you) and i'm sure the spam/botnet software authors have no qualms about holding open a couple thousand sockets on the rooted windows machine (times a few hundred thousand machines.) furthermore, i bet there are some shitty legitimate MTAs that would just give up too, causing actual mail to get discarded :)

(that, and they shoulda used twisted or something :) -- using a pool of apache/mod_perl instances to handle connections is grossly inefficient.)

ok, ok, maybe this sounds overly critical. it's a clever, thinking-out-of-the-box idea, but certainly not the panacea we're looking for to stop spam.

-fren

Hmmm... it looks like Twisted framework didn't make it into the standard library (yet?).

http://www.turbogears.org/ http://twistedmatrix.com/trac/

I'm not sure I agree with your chief argument, WRT latency in RPC calls. The answer to this is to use asyncronous development techniques as part of your IPC.

The increasing amount of network-related latency dealt with in everyday communication generally means that in any reasonable interesting application, there are often large numbers of blocking calls that take a variable amount of time to return. Threads in and of themselves are truly not the answer, except possibly when used sparingly as part of a non-blocking system. (as an aside, I've found that as a Python developer, working with the Twisted Python libraries was invaluable in my understanding of asychronous development techniques.)

With regards to the idea of function calls being an innappropriate abstraction for these mechanisms, I think the issue really depends on the implementation. The reasons I've used an RPC/IPC solution to communicate between processes (either locally or remotely) was because the various processes weren't closely related, and needed to have a common interface between them.

Making assumptions about implementation is an inherently bad programming habit, and has much farther reaching consequences. APIs would be essentially worthless if the developers made assumptions about how the work is done "behind the scenes". If you need intimate knowledge of the memory space of another process, it may be an indication to use process fork()ing to provide the same results.

anyways, my 2/100th of a dollar....

-phil

Linux (the 'L' in the acronym) is still cool isn't it? I don't see him argue against Linux, and he better not be. (okay it can easily be replace by another kernel like FreeBSD)

Apache is still hot i think, especially for feature richness and privillige systems it rocks. For a small webserver i'd prefer http://lighttpd.net/, http://mongrel.rubyforge.org/ or http://twistedmatrix.com/trac/. But yeah, the 'A' in LAMP is still okay (not that the author argues this, heay _he_ claims LAMP == BASIC).

MySQL has some very nice features (mainly clustering), that leaves other opensource dbs far behind. I personally prefer to use a db as 'fast yet dumb storage', that is my personal preference, and that makes MySQL a perfect candidate.

But the 'P' as in PHP, or according to some Perl aswell. Yeah i deffinitly think these to languages have been superceded by far. I concider Ruby, PHP, and Java superior choises in an opensource environment.

So its that 'P' in LAMP that, IMHO, is the BASIC of today... Hard to maintain, hard to write well. Yups, I'd say: DROP IT.

Cies Breijs.

For Python, the Twisted framework provides many of the things that applications commonly need.

Far, far too many things to list, which is why some newcomers get confused trying to take in too much at once.

Focus on what you need to do a sepecific job!

http://twistedmatrix.com/

Take a look at Twisted

> 1. Python. It took me a while to get past the indentation-as-block-structure
> thing (I still think it was a mistake)

Your comment got to 5, so somebody's got to say it. Significative indentation is nothing less than a stroke of genius. You indent your code anyway, right? So why is everybody forced to keep track of *two* kinds of block delimiters at the same time? Get rid of the stinkin' parentheses, and be done with it!

The rest of #1 is spot on, however.

> 2. If you are doing any sort of web work, you will probably have to do a
> little (a lot?) PHP.

Why on earth should he *have* to use such an inferior language? Python has lots of fine tools for web work. I advise using Twisted (http://twistedmatrix.com/). Its asynchronous event-based concurrency model may look peculiar at first, but being able to avoid the evil preemptive multithreading is priceless. And there's *lots* of Internet protocols in there for the taking!

> 3. AJAX. It's worth a look if you want to stay within the browser's window.
> And that means you should get good Javascript/CSS/XML/HTML books.

Javascript can get messy: AJAX needs all the hiding it can get. Nevow (http://divmod.org/trac/wiki/DivmodNevow) has great support for it in the Athena package, based on Mochikit. See the other Divmod tools, there's great stuff in there.

> 5. If you are picking up a DBMS, the obvious choices are MySQL and Postgres.

Do yourself a favor and use PostgreSQL, or SQLite. MySQL has a dubios history, and is often used together with PHP, which is similarly quirky.

It bears repeating: you wanna have fun, and at the same time learn a powerful language? Use Python, there's nothing quite like it around. I've been working with it for six years now, and it's been fun almost every day. :-)

When an entire castle worth of geeks stands at quiet attention to hear Rob go on about how bad his Perl code is, that means he's made geek culture elite. I don't know what other criteria you need. But since you're busy putting left-field words in my mouth, you probably haven't had time to think of any of your own.

If you want similar functionality with some more capable tools, have a look at Twisted with Divmod's Nevow & Axiom. Similar functionality but with a a whole lot more to boot.

Also worth checking out: Twisted. I haven't had occasion to use it myself, but people I know swear by it.

-jcr

I'll got ahead and put in a plug for Nevow here, another web framework that is based on the EXCELLENT Twisted framework.

If you're doing any sort of network programming in Python, you need to look at Twisted.

I have no affiliation with twisted beyond being impressed by the library, btw.

Check the Python web framework from www.nevow.org (requires Twisted. It makes much more sense to implement Nevow's LivePage (which is basically the same thing as so-called AJAX approach) with Twisted Python webapps -- Twisted webapp is created once and then, runs non-stop at the server side. PHP scripts are executed on each request (so, objects are created every time you click anywhere), this gives you less possibilities of exploiting "live web page" approach.

Or, in other words: if you want your web pages to be live, make sure your webapp is also running in realtime (and is not just a script, executed on each request).

No need to set up any fake-pirated-movie honeypots; the MPAA is already sending out enough frivolous legal threats to random websites for no apparent reason as it is.

For example: Scene.org got a copyright infringement notice a while ago claiming that a 62kb file supposedly contained an entire season of a television series. (What's funniest about this is when they sent the notice, the file wasn't even downloadable because it was still in /incoming!)
Linux Australia was the recipient of a similar notice about a couple of movies that they never had, Grind and Twisted. The files in question were Valgrind and Twisted Matrix.

I don't really like Python as a programming language but everyone knows that Zope is one of the biggest product written in this language.

Twisted Matrix is another big program in Python.

Another good Python network resource: Twisted Matrix, a networking framework for Python.

For those interested in starting in network programming in Python, I'd recommend checking it out.

1. A dedicated QA staff. You should have as many testers as you have developers.
2. Tools for the QA staff to create their own automation. They don't like doing manual testing much, either, so they'll have incentive to use the tools. :-) I'll talk about the tools we use in a bit.
3. Training for the QA staff on the tools. Hire QA people capable of at least a little shell programming. And the tools they use should be not much harder than writing shell scripts.
4. A good SCM (source code management) system that provides atomic commits, so that when you fix a bug, you can tell your testers exactly what revision number it's fixed in, and they can get exactly that revision verify it in the same system you had when you fixed it.
5. A bug tracker. It doesn't have to integrate with the SCM, but if it doesn't, you should make it a hard policy that your commit log messages should say what bug number they are a fix for, and when you resolve a bug, you must say what revision the fix went into. I can't even estimate how much time this policy saves.
6. Automated rebuilds of every revision of the software. Spend a lot of time on this, it's key. It lets your testers test things the minute you fix them. That means, if you failed to fix it correctly, you'll find out SOON while the fix is still fresh in your mind, and you'll save even more time by not having to get back into the mindset of that bug. You will need special software to do it, so read on.

1. For us, our project has had 1-2 developers working full time (me, plus one additional deveoper at various times). We've also had 1-2 testers working full time. That sounds like a small project, but after two years of dev it is a lot of code, and all that code needs testing. The fulltime test staff available right from the start was absolutely not money wasted.
2. The development is done in Python, with Twisted, and so we used a combination of unit tests written by the developers and black box tests written by the testers. Because our app is primarily web, I developed my own web test system (having found no others that were suitable for use by non-programmers). This system is PBP, which is a shell-like scriptable web browser.
3. Our main tester had a little bit of C in school (she actually had forgotten most of it ;-) and a little bit of unix command line experience. This was more than enough to be able to design and build tests with PBP. Then I spent about one full day showing her how to use it and brainstorming testmaking strategies with her.
4. Subversion.
5. We've been successful with Bugzilla. If I had to start over, I probably would have used Trac, with which I've had good experiences on other projects.
6. I built a completely automatic build system using Buildbot to trigger the builds after each commit and A-A-P to script the build process.

Open source can change some of those things, but as far as hardware goes you still need someone willing to put up considerable amounts of money for manufacturing.

Let me provide some suggestions. I work with F/OSS development for network security and wireless applications, and have spent a few years working with low-cost embedded systems that support Linux. With a Linux kernel and OS in a small box, there's not much you can't do per amateur/wireless development.

My current favorite foundation is:

o RouterBoard from Mikrotik of Latvia. Pentium 233/266 performance, very low cost ($300ish), dual PCMCIA slots, dual Ethernet (in one of the two models), microPCI slot (wonderful!), and compactflash slot. Hardware watchdog and other goodies built in are things normally found on much more expensive embedded system boards.

o IBM/Hitachi Microdrive: My base development systems runs with a 1 GB Microdrive with Debian on it, though I've got a 4 GB setup with Gentoo and use the 370 MB version for production loads. Routerboard has a Debian developers kit available for download on the site, including watchdog control. Avoid compactflash/CF (Microdrive fits the CF profile but is an actual spinning device) unless you're certain you're going to have minimal writes over time, as they will eventually cease to write and become somewhat worthless (in my experience, low-write use lasts about one year).

o Debian or Gentoo for development environment: there are some embedded distributions out there but they're intended for when you're ready to reduce to your final low-profile image. Both these distros give you a good amount of control over what is going into your system. Embedded Gentoo will be nice eventually (with cross-compile support) but isn't there yet.

o Python: Not to start any language wars (or distro wars per above), but Python is a great place for amateur developers to work in. Frameworks like Twisted allow you to focus on your code and build upon the networking smarts of others. I haven't tried yet, but I keep eyeballing Shtoom for an amateur project as well.

*scoove*

Python has already features like a simple HTTP server or asynchronous sockets library - built-in, portable (unix, win32, MacOS - no problem).

There are a few well known HTTP serving projects in Python (like this one), but the best thing in my opinion is Twisted - the framework of your internet, which features, among others, a 100% Python implementation of web server & ssh server/client. I strongly suggest anyone, who knows Python, to have a look at Twisted - many of ideas implemented there are logic and clean, it's a good programming lesson for sure.

Have fun!

Other DNS software

This is a list of some other DNS software out there:

Freely downloadable DNS servers

Caching DNS servers

• BIND 9 is a complete rewrite of BIND, and, as such, probably does not have the security issues that previous versions of BIND has. In fact, one of the BIND developers found a security problem in earlier versions of MaraDNS. Very full-featured, and is the reference standard for the newer DNS RFCs.
• Oak DNS is a DNS server written completely in python. It is compatible (I think) with both BIND zone files and cache files.
• pdnsd is a recursive caching DNS server. Paul Rombouts is the current maintainer of this program.
• Posadis is another DNS server project, similiar to MaraDNS. This server is now both a resolving and an suthoritative DNS server.

Non-recursive DNS servers

• PowerDNS is an authoritative-only DNS server with support for, among other things, SQL. I would like to applaud the PowerDNS developers for making a libre release of this software. Note: Recursive code is in the works; PowerDNS will soon enough be a fully functioning recursive DNS server.
• DnsJAVA is an authoritative-only DNS server written in Java.
• NSD is an authoritative-only DNS server which is compatible with BIND zone files.
• MyDNS is an authoritative-only DNS server which uses MySQL as a database back end.
• The Pliant language/package comes with a DNS server. This DNS server can not recursively process DNS queries given a list of root servers.
• Twisted includes a non-recursive DNS server.
• The Eddit project includes a DNS server
• SheerDNS is a simple non-caching DNS server that stores all records as their own files.

Abandoned DNS server projects

These are DNS server projects which have not released any files for six months or longer, and which never became functioning recursive (caching) DNS servers.

• MooDNS is another DNS server project. A CVS checkout on January 21, 2003 shows that no files have been updated since July 20, 2002, except for a single readme file updated on August 1, 2002. This project is abadoned.

  I have made a tarball available for people who do not want to bother with a CVS checkout.

• Dents is a DNS server that showed a lot of promise. Unfortunatly, no files have been released since 1999.
• Yaku-NS is a DNS server geared towards embedded systems. According to the changelog, no one has made any changes to this software since Feburary, 2001.
• CustomDNS has not released any files since the summer of 2000.

Other

• LdapDNS is a small DNS server which converts DNS requests in to LDAP requests, without caching.
• DnsPython is a DNS toolkit for Python.

• web development: Not only Zope and Plone, there's a lot of interesting development going on with Twisted... and, well, Twisted is not only about the web, it is about networking in general;
• .NET / Java development in general: Python in Java exists already... Java gaining more and more popularity? No problem, Python is already there. .NET/Mono taking the market away from Java? The industry will perhaps support more and more .NET techniques - with IronPython (featured recently on /.), Python is already there. No problem;
• some people might wonder if you really need that stinkin' .NET/Java stuff at all - Python is already very portable, but that's not everything - it already supports a wide range of MacOS/Unix/win32-specific functionalities.
• anything else makes you wonder, if Python is a good choice at all? What about realtime applications, like a VOIP phone (GUI included, win32/unix compatible) in Python? Do you read "interpreted" as "slow", do you wonder if Python is good enough to be useful in CPU-demanding programs like games? Anything else against it? Perhaps you're worried about Python's extensibility?
• if software telephony sucks for you, perhaps you should try a real mobile phone

Python (pre 2.0) used to do only refcount, and did it much better than Java (using GC) in all respects except thread friendliness. Modern python (2.0 and beyond) does both -- but it's extremely rare for the gc to be needed at all.

Can give me a triple punch of an architecture including the best of:

* Twisted
* ACE
* OpenTNL

Any other neat ones you can think of?

J2EE is over-bloated in order to be useful, but that kills it perform fast on small hardware (if to run at all). J2ME can run on really small hardware, but it lacks most of features people need for standalone applications. No need to mention Java GUI - its unfamouse SWING, which is just a joke.

Talking about cross-platform: Java cannot run on all platforms as Linux does. Well, unless you begin to count JDK-1.0.8 which just another useless kludge.

Should I remind you that Sun is doing everything to lock you in to its Java technologies? Still not convienced? Then where is the source code of JVM and JDK? Again, don't show me any 1.0.8 - nobody will use it now anyway.

My point is to support Twisted Python: it works on Linux (and thus on way more platforms than java). You can use a native code for any critical parts and scripts for flexibility. And it won't require GHz and GB from your hardware. It's as compact as J2ME and as functional as J2EE. Java cann never deliver such compactness and functionality at te same time (only one or another mutually exclusively). And it's open source, so no one keeps in mind to lock you in.

Twisted is a networking framework in Python which has a lot of unit tests:
http://www.twistedmatrix.com

The new linux kernel is great, but the reason the this particlular kernel results is better performance ("5 times better") is because the application framework it is testing is horrible.

All of the "enterprise" applications in this test have several performance cripling features in common: socket per thread connections, fundemental reliance on threads, and massive memory foot print. Apache has one thread/process (the diff is a stack) per connections. Java requires a sizable multiple of memory usage as most other application languages (C/C++ obviously, but also Perl, Python, and PHP). J2EE is an inherently thread driven programming framwork.

So yes, Linux 2.6 ameliorates the downsides of unnecessary use of threading. It makes thread creation and context switching even faster on the Linux platform.

And Yes, Linux 2.6 memory management is fundementally better. Reverse Page Table Entry mappings make finding victim pages better; and it is designed to avoid victimizing active pages better.

But could you all imagine if people were designing fundementally better application framworks? Event driven application architectures like TwistedPython and POE, or Event-thread hybrid systems like SEDA.

The performance stats given in that article are shit, complete utter shit. I know. In the proprietary world I work in, I code faster programs on the same Linux platform on a daily basis; orders of magnitude faster.

All the accomplishments of Linux 2.6 can be used for true performance programming. I plead with you all, stop using Threads until you know what they are good for. Stop using the stack to maintain your program state. Throw off the shackles and learn to program network servers.

Twisted is fairly big on XPish development techniques. They're big proponents of test-first design and, when possible, like to do pair programming.

The problem is that, generally, pair programming is not really possible for most OSS projects, where developers are spread across the globe & communicating by the net. They've been kicking aroudn the idea of a networked multi-user text editor tho..

I agree - especially as python can be really tiny.

Also, read this:

Extend vs Embed

Often people only think of putting python(etc) inside their program, rather than the other way round - putting their program inside of python. Read the above link.

5 Create the all-in-one inbox Email, phone calls, instant messages - they should all go into a single app.

His wish is their command: Quotient.

Don't be fooled by the unadorned web site: Quotient is a multiprotocol server plus client plus repository, supporting POP, IMAP, SIP, IRC and IM, and based on Twisted, an event-based, multiprotocol networking framework. Quotient comes from the Twisted guys themselves.

It's a remarkable architecture, and implemented in a very nice programming language. :^)

There's yet another type of forking that can occur which I call "forking up". This is where a small but important module is developed by someone who needs it, and the module later gets adopted into a project with broader scope and larger visibility. This is in fact the norm when projects submit code for inclusion into the standard distribution of Python or Perl. If Python adopts the module, sometimes independent development on it dies, but many times it does not--pybsddb and pyxml are two examples that spring to mind. Other projects do this as well... Twisted and Zope both get a lot of contributions from smaller projects that want to be part of a larger effort.

This is yet another case that supports parent's argument that resources are not being divided.. resources that work on fixing bugs in the higher-visibility project would never have worked on the smaller project's codebase at all. The smaller project can still benefit from porting bugfixes in stdlib over to their tree.

Everybody wins.

Or you could just use Twisted and make it relatively easy ...

I expect more hype coming to Python, especially once it gets some EJB-like "Twisted" framework. Or Erlang with its enterprise-quality EJB-like OTP framework.

I like the twisted framework; it's practically another language on top of Python. I've recently discovered another worthy language, O'Caml. It has some really neat idioms, and the native code compilation ability is nice. Unfortunately, it lacks much in the way of EJB type stuff. Still, I'm sure that an EJB-like system could be done well in O'Caml. Those languages are where I'd like to see more hype drifting, for one main reason: simplicity. They're more lightweight, and they carry less of the C/C++ baggage.