Slashdot Mirror
Project Aims For 5x Increase In Python Performance
cocoanaut writes "A new project launched by Google's Python engineers could make the popular programming language five times faster. The project, which is called Unladen Swallow, seeks to replace the Python interpreter's virtual machine with a new just-in-time (JIT) compilation engine that is built on LLVM. The first milestone release, which was announced at PyCon, already offers a 15-25% performance increase over the standard CPython implementation. The source code is available from the Google Code web site."
I hope this translates into further speed ups for EVE online down the road.
What if Oprah's ass got 5x's smaller?
The summary misses one of the best bits -- the project will try to get rid of the Global Interpreter Lock that interferes so much with multithreading.
Also, it's based on v2.6, which they are hoping will make 3.x an easy change.
They say five times faster however it really depends on if they're talking about a European or African Python Interpreter.
I read about what they intend to do, and they seem to have quite a few interesting ideas... But there are also major drawbacks:
- No Windows support (apparently a Linux-only VM in the plans)
- No Python 3.0 support
And thus no guarantees most of the work will merge back into CPython.
But competition is good, I can't really see a problem with having an alternative faster Python runtime, even if it's not as compatible as CPython. :)
.: Max Romantschuk
It would still be huge! :-)
Sleep your way to a whiter smile...date a dentist!
0.5x slower is like 2x faster, right? Reciprocals?
Word has it that Microsoft created a speedy IronPython implementation on their Common Language Runtime and JIT technology for .NET.
Here are benchmarks for it.
Failing to find similar benchmarks for comparison; can anybody else contribute to this info?...
Quite to the contrary, the FreeBSD guys have been building with clang+llvm for a while now, and they seem to like it. The kernel boots, init inits, filesystems mount, the shell runs.
What other platforms, Darwin? Apple employs the largest number of LLVM developers. Windows? Both MinGW and Visual Studio based builds are tested for each release.
It's still not as portable as the python interpreter, but that will come if and when developers who are interested in working on it start to contribute.
Not really. Parrot is a much higher-level VM, providing things like closures, multiple dispatch, garbage collection, infrastructure to support multiple object models, and so forth, whereas LLVM really models a basic RISC instruction set with an infinite number of write-only registers.
In fact, it would make a fair bit of sense to actually use LLVM as the JIT-compiling backend for Parrot...
It sounds like that they're going to take Python, which is already gets translated to some kind of p-code (right?) and either translate the original Python or the p-code into LLVM code, which is then JIT-compiled to the native architecture.
The translation from Python to LLVM is going to lose some specificity and require that extra code be added to implement whatever needs to be done in Python that isn't trivially implemented by LLVM. Then the LLVM code needs to be compiled to native, introducing yet more "glue" code in the process.
Wouldn't a more direct compile yield a better result?
And don't give me any junk about compiling dynamic languages. LISP and Self are highly dynamic languages, yet they're compiled. If they can be compiled, then so can Python. I mean, the fact that it can be done through multiple levels of translation proves that it can be done, although possibly inefficiently. I just think that a more direct approach would reduce some of the superfluous glue code and a variety of other inefficiencies in translation that result from a loss of knowledge about what the original program was actually trying to implement.
I get emails claiming to increase my python's performance all of the time, I just delete them.
One of our competitors trademarked the term "hypothesis". From now on, we will call them "boneheaded ideas".
Is there any hope that we will move away from these boutique programming languages and back to "real languages" that seriously consider size and performance?
I for one am completely sick and tired of 3Ghz multicore processor machines with gigabytes of RAM running like a 486. Languages like Python don;t help in the bloat arena and the scripting languages made out of frameworks on top of other scripting languages are just ludicrous!
Yes, I realise the right tool for the job argument.
Exactly. Most applications are not CPU bound. If yours is, then I don't know why others are trying to get you to use Python.
If the geiger counter does not click, the coffee, she is not thick.
I find Python is about 20x slower (and about 10x faster to implement) than C, with the number varying quite a bit depending on how CPU-bound the code is. Given the speed of modern processors, this is plenty fast for many tasks.
Beyond that, many Python programmers employ a strategy of writing just the CPU-intensive inner loops in C or C++. This gives you most of the speed of an all-compiled solution but with much of the easier programming (and shorter programs) of the all-Python approach.
My particular scientific application runs on 1500 cores, is about 75% Python/25% C++, is 4-5x smaller than similar all-C/C++ programs, and runs at about 95-99.99% of the speed of an all C++ solution.
(Somewhat ironically, some of the worst performance bottlenecks in this app had to do with the overhead of some of the STL containers, which I ended up having to replace with C-style arrays, etc. to get best performance.)
Not all apps will fall out this way, but you definitely can't assume that just because something's written in Python that it will be slow.
(Going beyond that, we all know that better algorithms usually trump all of this anyway. If writing in Python gives you the time and clarity to be able to use an O(n)-better algorithm, that may pay off in itself.)
"Not an actor, but he plays one on TV."
You're not CPU bound until you: add all the features, handle the special cases, add the error checking, scale up beyond trivial test data, etc.
Then what? Rewrite?
Yes. If you didn't know all of that was going to happen, you're prototyping. If you're prototyping, you should be doing it in a prototyping language.
Rewriting from Python to C++ is not particularly difficult. Completely overhauling the design of a project written entirely in C++ is really unpleasant and takes a long time. So much so that many early design decisions on large C++ projects simply cannot be undone.
Model in clay first, then in stone later if you have to.
"Not an actor, but he plays one on TV."
The Parrot Sketch backfired not that long ago when fossils of a parrot (that probably was blue) were found in Norway. Not too far from the Fjords, as I recall. It is, however, quite dead.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Here's your cookie:
\_/
Nerd rage is the funniest rage.
This is disappointing. Shed Skin has shown speed improvements of 2 to 220x over CPython. Going for 5x over CPython is lame. But Shed Skin is a tiny effort, and needs help.
PyPy got a lot of press, but they tried to do an optimizing compiler with "agile programming" and "sprints", and, at six years on with substantial funding, it's still not done.
The fundamental problem with running Python fast is its gratuitous dynamism. In CPython, almost everything is late-bound, and most of the time goes into name lookups. This makes it easy to treat everything as dynamic. You can store into the local variables of a function from outside the function, for example. In order to make Python go fast, the compiler has to be able to detect the 99.99% of the time when that isn't happening and generate pre-bound code accordingly.
Dynamic typing requires similar handling. Most variables never change type. Recognizing int and float variables that will never contain anything else creates a significant speedup. In CPython, all numbers are "boxed", stored in an object structure. This is general but slow.
CPython is nice and simple, but slow. Serious speedup requires global analysis of the program to detect the hard cases and generate fast code for the easy ones. Shed Skin actually does this, but has to place some limitations on the language to do it. If someone did everything right, Python could probably achieve the speed of C++.
There's also the problem that if you want to be compatible with existing C modules for CPython, you're stuck with CPython's overly general internal representation.
LLVM is stable and in use. The iPhone SDK arm compilers use gcc with a llvm backend. OS X uses LLVM in the OpenGL stack to support features that the GPU doesn't. They're also using LLVM for openCL/Grand Central.
LLVM isn't just another virtual machine, it also optimizes that code (at compile time, link time, and/or runtime) and converts it to native (alpha, arm, cell, ia64, mips, CIL, pic16, ppc, sparc, x86) binaries (or C source code).
Do you even lift?
These aren't the 'roids you're looking for.
The thing about Python is you are replacing every lost hour in runtime with a day gained in development time. That is the point of Python. Numpy (formerly scipy I think) is mostly written in C anyway and provides fast n-dimensional array objects for vector and matrix operations, there are really only a few bottlenecks for maths/science purposes. Generally anything that is going to take a seriously long amount of time you would be doing in C over anything else anyway, what Python is is a viable alternative to Matlab etc, and a damn sight less expensive!
Where I study Engineering they teach Python for this very reason. It has a gentle syntax which appeals to engineers and scientists who often aren't bargaining to become coders, and it is so much cheaper than Matlab that any missing features are rendered a moot point.
Seriously, sitting on the sidelines and saying "I'm not gonna use Python because it is slow" is silly, it is so damn easy to code in python that you would learn it in a weekend if you already have coding experience. And as I said before, any lost time running python scripts over other languages is made up ten time over at least in the ridiculously short development times that go with Python scripts. Yes, it really is THAT easy to do anything in Python, there is a reason people bug you to try it out. Just give it a weekend, Python deserves it!
The experimental combination of the Python-to-Javascript compiler, http://pyjs.org/ and the Python Bindings to Google's V8 Engine, http://code.google.com/p/pyv8 brings a ten times performance increase over standard python, already.
not - "10% now and 5x in the future" - that's a 1000% increase NOW.
When V8 supports the ECMAScript "Harmony" standard, which will include support for basic integer types, then there will be "correct" support in the PyJS + PyV8 combination for numerical types, and the word "experimental" can be dropped.
http://pyjsorg/ also includes an experiment showing the bindings of the PyJS compiler with the Python-Spidermonkey project. The spidermonkey JS engine has the advantage of running on generic platforms instead of just ARM and 32-bit x86 platforms, but has the disadvantage of being slightly slower.
Javascript is a _really_ interesting language that makes it in many ways highly suitable as an intermediate compiler language for compiling dynamic languages as Ruby and Python.