Slashdot Mirror

Excellent. Multiple concurrent downloads of lots and lots of pictures, if you know what I mean....

Actually, the fastest way to distribute static content does not use threads that much, if at all. It's a simple select loop. See Twisted.

Or were you talking about RHEL WS and *downloading* the pr0n, instead of serving it? :-)

I think that the "make menuconfig" system should better explain about kernel modules. I had a deuce of a time figuring out what key to press to get the LNE100TX ethernet card enabled, but once I found out, it was just one key.

I believe there are a few problems with this advice. One problem with python is, it's incredibly slow compared to java. I compiled the benchmarks from here on java 1.4 and python 2.3. While it looks like python 1.5.2 did pretty well against java 1.1, it was terrible with the new versions of both interpreters. Another problem is that Python (and most weakly typed languages) are great for throwing stuff together in a hurry, but strongly typed applications have the advantage of the compiler telling you ahead of time that there are more subtle errors in your program(e.g. casting to incorrect/imprecise types). Don't get me wrong, I have this feeling that there will never be an open source implementation of java, and so I'm rooting for python, but these are things that warrant serious consideration before deploying in the enterprise.

Is there any kind of a test implementation of IM2000 yet? I for one would like to see it in action.

If not, I bet one could be hacked up using Python and Twisted in a couple days.

As a long term unix developer who tried a job where windows was mandated:

It is an absolute nightmare to do anything on Windows that isn't explicitly allowed by Microsoft.

Have you ever tried to debug some random piece of crap VB dll or vbscript ( two of the four current VB dialects... vb.net, vb6, vbs, vba )? Its a fuckload harder than a horrible shell or perl script. Python scripts are pretty hard to make truly horrible, so those are usually even easier to debug.

COM is really just a horrible hack to make people think there is a C++ abi on Windows. It is an absolute disgrace to actually use. This is the reason so much is done in VB on Windows. Microsoft have made C and C++ into a completely useless platform for doing anything quick. There are over 30 different types of string used in the MS apis... what does that tell you?

Every api seems to have from 9 to 35 arguments. Nobody knows what they are for... its a cut and paste job from MSDN, yet again... and then we get on to business processes.

People start off with a spreadsheet or a word document. They add macros to it. They expand it. They go fucking insane. The next thing you know you are expected to work out what a fucking idiot has created in the worst language known to man, VBA. There are so many random limitations in this crud that even the bog standard excel user hits them on his first macro, and starts making up crazy work arounds, each different than the other. Fuck you, Joel Spoelsky..... . I can't believe that guy is proud of his "Excel macro strategy".

And before you say .NET, yes, cloning Java is a good idea if you can't bring yourself to actually use something not invented here. But people still have to deal with the utterly brain dead attitude of windows, and Windows.Forms is still the absolute worst GUI toolkit in use... You still end up having to use COM, and anyway, why the hell wouldn't you just use Java unless you are a complete MS donkey?

On unix, the first thing is that I have choice... I don't have to go with Apache, or Tux, or publicfile, or roxen, or zope, or roll my own with twisted , my current favorite trick. On windows, if you don't use IIS, you are likely to get screwed over at any point.

Now, be honest. You tried to use unix but you got scared. "Mummy, theres no drive letters! I'm lost!!!!! Waaaaaah!!!!". You didn't want to know what was going on. Windows protects you from knowing what the hell you are doing by restricting you to do only what their focus groups tell them. Have you ever actually worked out what was happening when something broke on Windows? Or did you just give up and abandon that functionality, and blame it on Microsoft? Microsoft, in their incompetence, provide a great scapegoat for Windows developers. If they had to use an Open Source system, this excuse would become fairly hollow...

Anyway, when you have a problem on Unix you don't ever reach some inscrutable, impenetrable barrier. You can look at what every component does, and if required, dive into the source and fix it. There are no artificial limits. The fact that anyone can look at the source means that people are less inclined to publish crappy code... And this effect increases with time.

To your "advantages":

DFS - please. This is a dodgy hack of SMB - it is not "distributed" in any real sense. OpenAFS is about as good as gets there, maybe Coda when it gets stable...

User administration: Huh? Can your helpdesk staff not learn a web front end to do this? Its not very hard to find one.... eg webmin, linuxconf. And this kind of thing is easy to customise - ie force your staff to put the required info..

and frankly it will always be easier
As soon as someone uses the word "frankly", it means "I'm going to say something completely unsupported and expect you to believe it."

Comparing windows to unix is like comparing a swiss pen knife to a fully equipped machine shop, with almost every tool available to you to use. Except you can fit it in your pocket....

How about an RSS feed?

I think you want CVSToys.

Combines the flexibility of syntax of C with the efficiency of Javascript. Python can't realistically support Lisp-style macros, doesn't support true closures, and many other things of Goodness that make Lisp languages so good for rapid coding. On the other hand, Python's dynamicism makes it very very slow. It's bad enough that the Python zealots apparently don't know how to program Java -- as minor corrections to their broken test code result in Java code that just stomps Python.

So let's be honest here. Python is slow. So we have a slow language. So what? That's okay. I'm all for slow languages as long as they have features which make up for them. But Python's missing many of the goodies that make slow languages great. Macros. Closures. A good, scientifically-oriented numerical facility.

I get this very strong feeling that Python has the worst features of both Lisp and C, while not having the best features of either of them. No wonder it appears, from my reading, that the vast majority of people pushing for Python are ex-Perl programmers. In other words, they're arguing for Python as a replacement scripting language for Perl. Not as a language for large-scale development or maintainability. But Perl is an easy target. It's a grotesque language, only popular because it was first to bat in the full-featured-scripting-language game.

A malformed object oriented system. My other major beef with Python is that it has, to my mind, the single worst-conceived OOP system of any language I can think of at the moment.

There are two basic families of OOP: class-based (C++/Java/CLOS/etc.) and prototype-based (JavaScript/Self/NewtonScript). The advantage of a class-based language is that you can equate classes with types, and the compiler knows exactly where a slot should be in an object, even an inherited slot. So they're fast. A great choice for a performance program on a desktop. The advantage of a prototype-based language is that it only keeps around the diffs: objects don't need to store tons of slots they don't use -- they just point to a superobject and say "go there for anything else". So they're highly memory efficient. Prototype languages are also highly dynamic, elegant, and small. A great choice for a scripting language, or a language for a PDA (like the Newton).

Python is odd: it has both models. Underneath the model is prototype-based. Kludged on top of this is a class model. What the heck? Do not think this is a good thing. Each model has made sacrifices from the ideal -- by using both models, you're making all the sacrifices at the same time. It appears to me that Python has managed to achieve the worst of both features: it's not as dynamic or small and elegant as a prototype OOP language, and it's not nearly as fast as a class-based OOP language. It has no advantages over a prototype-based model at all. In reality, the kludge means that Python programmers have to keep more syntax rules in their brains in order to achieve less than a prototype language would provide.

I get the feeling that what happened was Python started as a procedural Perl-style language, then hacked on a prototype model, then due to, I dunno, misinformed "customer complaints", hacked on another class-based model on top of that. What a mess.

Anyway, this is how I see the current state of the Python world. Why do people use this language? My guess is that it's because they're ex-Perl programmers, and made an incremental jump to the next slightly better language.

If you want to do async socket stuff in Python, you should check out Twisted Matrix which can do anything (apparently)

BTW, what alteratives to BIND exist for Linuxand *BSD? I actually don't know and would like to know.

There are now a number of alternative packages that may have advantages for many deployments. E.g.:

MaraDNS is a general-purpose, fast DNS server package (doing recursive, authoritative, and caching roles, plus fully supporting zone transfers):
http://www.maradns.org/

pdnsd is a small caching-only DNS server with a disk-based cache, suitable for small networks and workstations:
http://home.t-online.de/home/Moestl/

Dnsmasq is a small authoritative and caching DNS server for a group of NATted / IPmasqued machines (optionally pulling names from DHCP leases):
http://www.thekelleys.org.uk/dnsmasq/

DNRD is a small caching-only DNS server for NAT / IPmasq networks:
http://dnrd.nevalabs.org/

MyDNS is a MySQL-based authoritative and caching server (no recursive service) suitable for very large sites. In such roles, it's faster and more responsive than BIND9, even though the latter uses a RAM-based cache:
http://mydns.bboy.net/

ldapdns implements the same idea, except out of an LDAP database. Again, much faster than BIND9:
http://nimh.org/code/ldapdns/

GnuDIP is an authoritative server for Dynamic DNS:
http://gnudip2.sourceforge.net/gnudip-www/

NSD is a high-performance authoritative-only daemon:
http://www.nlnetlabs.nl/nsd/

PowerDNS (open source as of 2002-11-25) is an authoritative-only daemon with a modular structure supporting various back-end information stores such as SQL databases (MySQL, PostgreSQL, Oracle 8i, Oracle 9i, IBM DB2, and others via ODBC), BIND zonefiles and other file formats, and LDAP directories. Supports AXFR zone transfers.
http://www.powerdns.com/products/powerdns/

CustomDNS is a authoritative-only daemon for both static addresses and its variant form of dynamic DNS:
http://customdns.sourceforge.net/

lbnamed is a similar authoritative-only daemon for static and dynamic information, with a load-balancing multi-machine architecture:
http://www.stanford.edu/~riepel/lbnamed/

Posadis is another fast authoritative-only daemon:
http://posadis.sourceforge.net/

dents is another general-purpose DNS server, but is perenially unfinished, and is probably dead, at this point:
http://sourceforge.net/projects/dents/

Pliant DNS Server is another general-purpose DNS server, although it may not support zone transfers:
http://pliant.cx/pliant/protocol/dns/

Yaku-NS is another small, fast general-purpose DNS server:
http://www.kyuzz.org/antirez/ens.html

Twisted Names is an authoritative and caching DNS server, written in Python:
http://twistedmatrix.com/documents/howto/names

Oak DNS Server is an authoritative and caching DNS server, supporting dynamic DNS updates and AAAA records. It's written in Python, and doesn't need to run privileged:
http://www.digitallumber.com/oak

dnsjava is a minimal, authoritative-only server, a resolver library, and a set of DNS utilities, all written in Java:
http://www.xbill.org/dnsjava/

Related:

FireDNS is a client library for DNS requests, with emphasis on speed and asynchronous processing. Written in C, and has low-timeout blocking functions. Can be used to relace standard libc resolver library functions like getbyhostname with much faster equivalent code:
http://ares.penguinhosting.net/~ian/

GNU adns is a resolver library for C (and C++) programs, and a collection of useful DNS resolver utilities:
http://www.chiark.greenend.org.uk/~ian/adns/

Proprietary packages include:

UltraDNS (UltraDNS Corporation)
djbdns/tinydns
ATLAS (Verisign)
BINDPlus (Information Network Eng. Group, Inc.)
Global Name Service (Nominum, Inc.)
NeDNS (Neteka, Inc.)

I maintain this list at http://linuxmafia.com/~rick/linux-info/dns-servers

Rick Moen
rick@linuxmafia.com

We are a company which is almost totally based on open-source software solutions. We are currently doing a nice size project for a government agency (who currently must remain nameless - no this is not defense related) which is all being done in Python and the Twisted Framework.

One issue we confront is that you have to have software tools that play nicely with all the legacy and Microsoft systems out there. Customers aren't going to throw out their existing investment in infrastructure. The open source model almost by definition guarantees that it is actually great for integration with the proprietary world. In fact, Python is especially so.

The other issue is convincing our customers to "give back to the community." At first they almost always say we have to own the IP. Then we explain to them that they aren't software companies so there is no value to them in that ownership. Then they worry about their competitors getting their hands on it (this isn't an issue obviously with government clients, but they still have hesitations.). In the end of the day, we always manage to educate them on the benefits they will derive in releasing as much as possible back to the open source community. We usually add a clause in the contract which gives them the right to exclude code from being open sourced, but they rarely invoke it. In more commercial environments, there may be proprietary ways of doing things embedded in the software, but that kind of stuff isn't usually amenable to opens sourcing in any case.

Bottom line: open source development is a business both in commercial and government sectors.

Heh, hehe, okay.

According to Bagley's site, we should all be using Ocaml anyway. Who knows, he may even be right...

In reference to the Bagley test, Java was more performant than Python, true. But Python won over Java on both memory and lines of code. Also, his tests, as all artificial benchmarks, are both accurate only for the point-in-time and are only accurate within the limits of the test conditions themselves. More recent versions of both Java and Python are faster. On top of that, special-purpose optimizations exist for both Java and Python if you really need that extra spurt of speed -- think Jikes or TowerJ for Java and Psyco for Python. More to the point, though, Java may be faster than Python, but it isn't faster by an order of magnitude. The difference in speed between them is not enough to worry about -- if you really need that much speed, you won't be programming in either Java *or* Python, you'll be coding in C and assembler. Your performance argument is irrelevant.

Your libraries argument is somewhat more compelling. However, you may not be aware that there are two major versions of Python, standard Python implemented in C and Jython, implemented in Java. Using Jython, you can write Python that has full access to all of Java's libraries.

On top of that, I'll make a very rough comparison between the various projects on Jakarta and extant Python libraries. I don't think I've seen anything like this, as Python has a poor record for collating their libraries and apps in one place, so the effort is worth it simply for educational purposes, if nothing else.

Disclaimer: I am not terribly familiar with most of these projects, and they have varying states of completeness and maturity. I merely aim to show that analogs of the various Jakarta projects do exist in the Python world. Please feel free to peruse them yourself and come to your own conclusions.

Jakarta Ant -- PyAnt , SCons

Alexandria -- I don't know of any comparable Python applications. However, the individual components of Alexandria (doc generation, CVS access, etc.) are available: check out HappyDoc , and various modules for use with the Zope application server, including CVSFile

Okay, now I'm going to lump together a bunch of Jakarta projects. Individual authors and users of these projects will inevitably scream, but my justification is that they are all web application servers of one sort or another. Their purposes are all the same. They have many differences in approach, philosophy, scope, and implementation, but at heart, they are all web application servers or web application server frameworks. Those projects are: Avalon, Jetspeed, Struts, Turbine, Velocity, Slide, and Tomcat itself. Oh, and I might as well throw James in here, too. Python web app servers and frameworks are equally numerous, and several are in advanced stages of maturity: again Zope, Twisted, Webware, Quixote, CherryPy, and SkunkWeb. There are more, but I'll leave that as an exercise for the reader. Google is your friend.

Lucene has no real counterpart in Python. David Mertz has put together a text indexer and search program, available at his site, but it looks small compared to Lucene. There is also something called WePaSe, but there is no information on it aside from its freshmeat release announcement.

Gump also has no counterpart. Cactus has an equivalent in WebUnit and PyUnit. Log4J's Python copy is called, naturally, Log4Py.

ORO and Regexp provide regular expressions for Java. Python has regular expressions built in to the standard library.

OJB provides an object-relational bridge for Java, similar in concept to Sun's JDO specification. Python counterparts are Modeling , PyDO, which is a subproject of the above-mentioned SkunkWeb, and MiddleKit, a subproject of WebWare.

ECS, JMeter, and POI have no Python counterparts. BSF also has no counterpart, since it embeds a scripting language in compiled Java. Perhaps its "counterpart" is Jython. Likewise, BCEL has no counterpart, nor does Watchdog.

Taglibs has no direct counterpart. Instead, Python has Spyce, Cheetah, PSP, and probably close to a dozen other implementations of the ASP/JSP theme, each with their own library of tags. Lack of a standard is perhaps not a good thing, but the existence of bunches of competing implementations is not a bad thing. Perhaps the most direct counterpart would be Zope's built-in technologies, DTML and ZPT. ZPT has also been built out into a standalone version, SimpleTAL.

Jakarta Commons has too many small projects for me to want to research Python equivalents. If you are looking for something in particular, check the Vaults of Parnassus first.

As for Apache XML, Python has SOAPy and ZSI implementing SOAP, and DOM, SAX, and XML-RPC are built in to the standard library. 4Suite implements DOM, SAX, RDF, XSLT, XInclude, XPointer, XLink and XPath, and has an XML and RDF data repository and server built on top, which would make it very roughly equivalent to both Cocoon and Xindice. I don't know of any Python equivalents for Batik, FOP or XMLSecurity.

Python has relational database access through its DBAPI standard, with adaptors for just about every database. There are a number of object databases coded specifically for (and often in) Python, the most well known being ZODB, which was developed by Zope. There are adaptors for other object databases as well.

There are really two spaces where Java outstrips Python, and the second space is IMHO directly caused by the first: standardization, and J2EE. Python puts out a language implementation and a lot of very useful libraries, but does not have any standardization body like the JCP. The result is lots of fragmentation. Individual developers write their own libraries and applications that compete with each other while offering wildly differing APIs and programming approaches. There has been some push to organize, through the official Python SIGs, but their efforts, while noble, have not been massively effective. Only this month has an initial implementation of a Python library repository similar to CPAN been released. Kudos to Andrew Kuchling, who made it happen, but it is LONG overdue.

Regarding J2EE, the only viable competitor is Zope. Even then, Zope really doesn't address the same problem space. The shortfall here comes from a number of different factors: corporate buy-in, public perception, lack of an established problem-space solution, and lack of published standards. Zope is a great solution, and has been used by a number of high-profile companies, but its focus is different.

Well, I hope you find this comparison to be useful. *I* certainly found it enlightening.

Ace's Hardware needs to research real servers before talking about their "scalable" servers. Their numbers are really saying that their box performs like a dog.

For those of you interested in this topic here is a few pointers and words of wisdom.

Server scalabilty and performance has three basic metrics, thruput (urls/sec), simultaneous connections, and performance while overloaded. Of course, you could add latensy but I'd argue that with the correct design latency is directly proportional to the real work you are doing, bad design insertes arbitrary waits.

I know of a HTTP Proxy by a large ISP that does user authentications & URL authorization (re: database), header manipulation, and on-the-fly text compression at 3000 urls/sec for 2000-4000 simultaneous connections and maintains that performance under load by sheding connections, all this on a dual 1GHz Intel PIII box running a Open Source OS that starts with "L". That is a maximum of 260 Million URL/day, three orders of magnitude greater performance than Ace's Hardware stats.

The simple answer to the question "How do I create a scalable fast network server?" is Event-driven GOOD & Threads BAD. Event driven network communication is two to three orders of magnitude better performing than thread/thread-pool based network communications. See Dan Kegel's C10K web page. That means you must use non-blocking IO to client sockets and databases. Once you accomplish that small feat, dynamic content just consumes CPU; with 2.8 Ghz Xeon processors you have plenty of cycles for parsing HTML markup or whatever. Threads cause cache thrashing, and context switching. While thread programmers don't see the cost in their code, just read the kernel code and you'll see how much work HAS TO BE DONE to switch threads. Event driven programming just takes some state lookups (array manipulation) and a callback (push some pointers onto the stack and jump to a function pointer).

Desgin is FAR MORE IMPORTANT than which runtime you use (execution tree, byte code, or straight assembly). I have done some very high load network programming with Perl using POE.

Python has Twisted Python

Java has the java.nio and the brilliant event/thread hybrid library SEDA by Matt Welsch.

I am also looking into the programming language Erlang which builds concurrancy and event driven programming into the language. Further, Erlang is used by some big telco manufacturers to great effect (high performance and claimed 99.9999999% nine-nines reliability on a big app).

Enough of that silly Java stuff.
Get Twisted.

Not to just quote the front page of the site, but it really is simple to throw together simple but addicting games like the old classics, in Python. I find it frustrating that more of these fun games arent written. The tool is available and several attempts at games made. The kit runs on just about every platform and with what the guys over at Twisted Matrix Labs have done, networking these games couldnt get easier. I myself am slowly working on at least two of these games, we need more! Never can we have too many games! NEVER!

Imagine network frogger, the whole pond is out and racing to cross the road! How about zoomed out and multiplayer digdug? Pump em till they pulp! Sidescrollers are doable, how about Karnov, anyone remember the old bald guy? I know I lost many a silver coin to that cat. I :really: want a top down, Gauntlet style mutliplayer game.

How many of you would love to play d0pe-warz with an actual realtime multiplayer backend, knife fights in the alleys, buying pimp cars for faster neighborhood/city->city navigation, and real market fluctuations based on supply and demand!?

Granted, many of these games are copyrighted materials but look how simple some of the these games were, compared to todays games. We all miss this flavor of game, having been saturated with yet more and more realistic games. The game designers of today are shooting for blockbuster titles, maximizing on the latest graphical hardware innovation.

Someone has even gone to the trouble of simulating old led based games from WAY-back. We miss these games!

Pygame is the best tool to get us these games. Any of you with a half baked idea and patience to learn a little Python, get to crackin! We NEED you!

-=C=-

I like XML-RPC, because it gives me a really neat, simple way to do simple message passing between programs over the internet, without any more overhead than is absolutely necessary, it's cross-platform and cross-language, it isn't awkward to use in any of the programming languages i've used it in.

XSLT looks really really awesome becuase it's useful and relatively simple, and i really hope we start seeing some tools that can automatically generate some of the XSLT for you, since like all XML tech it's just really verbose.

J2EE looks *interesting*, and all i wish was that it could be interfaced more easily with other languages. I love Jython, but i don't know if i can embrace Java completely until it's possible to let java communicate with arbitrary languages a bit more easily.

Twisted looks neat but i don't think i'll ever use it.

I think CORBA would rock my world if it were a bit simpler, or just if someone would find a way to integrate it with the compiler, or just cut out the complicated crap that surrounds using it. C# (whoo, someone's finally figured out that if you make a bunch of languages with the same features but different syntax and macro between them, people will think it's "language cross-compatibility"..) is not the correct way of doing this.

• IRC: #python on IRC server(s) irc.openprojects.net
• Main page: Python ICFP on TwistedMatrix

Everyone is welcome.

http://www.twistedmatrix.com/users/jh.twistd/pytho n/moin.cgi/IcfpPythonTeam

How python can be used with your existing JVM

Java and Python Side by side

The story of a java program turned into a python program

Your millage may vary

How python can be used with your existing JVM

Java and Python Side by side

The story of a java program turned into a python program

Your millage may vary

A common OSS phrase applies here: if you want to make things better, why not help out? Offer to teach some after-school computer use or programming classes somewhere in your area. Find some of these bored kids on the 'net and offer them lessons. Teach them what it means to be a hacker. They'll acquire some useful, fun skills, and you'll benefit by having more intelligent, educated people around.

One of my friends started working on this recently. He cruised chat rooms and other places, contacting kids and offering them accounts on his Linux server, so long as they learned to use the machine and observed common courtesy and proper capitalization. He got several to accept his generousity, and he's given some lessons on, e.g. C. Unfortunately, he hasn't a whole lot of time to spend recruiting and teaching. He's got the machine--if you could help, you'd be doing everyone a favor (including yourself). There's a basic page up at qapcom.twistedmatrix.com.

The 'net allows us to share information not only in the form of code, but lessons as well. More of the latter will mean more of the former.