Slashdot Mirror
See the PyPy JIT In Action
derGoldstein writes "Project PyPy is an alternative implementation of Python, with the main advantage being a Just In Time (JIT) compiler which speeds up your code considerably. They've announced the first public release of jitviewer, which is a visualization tool that helps you understand how your code is being compiled by PyPy's JIT, all the way down to assembly. If you just want to see how it looks and play with it, they've set up an online demo — just select a file, and click 'Show Assembler.'"
Slashdot DDOS machine in full effect
I love python, and pypy makes dream about the day python could be used for anything, making java irrelevant.
The language is readable, flexible, succinct, expressive and beautiful.
Programming in python is more a joy than a core, but it has always suffered from a wide performance gap in cpu intensive tasks compared to traditional static languages.
Hopefully, this is all about to change with projects like this.
It's worth noting that Pypy is more than just an alternative python implementation. It is a framework for implementing jitted compiled dynamic languages.
There are currently several works in progress with implementations of javascript, scheme, logo, php and other languages with pypy.
the connection has timed out
Python, the new Java
Dude, you really need to remember to log out when you have kids in the house.
I'm glad to see it scales well!
#DeleteChrome
Python, the new Java
That's Jython, not PyPy.
Someone remind me... Isn't PyPy the implementation that's based on LLVM?
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
Now, I understand why python executes so fast. Good tool. Is this why the link to "online demo" is broken?
Not a big fan of Python, but at some point I decided that an important check-off for any language is whether or not it can self-compile (or self interpret). PyPy gives it that checkoff and hat's off for having a JIT.
Compare and contrast with Larry Wall's admission that Perl isn't for everything, noting that Perl isn't written in Perl. In retrospect, that should have been a red flag. C is written in C, Lisp is written in Lisp to the point where it makes your mind warp. Not sure about Java; but it seems likely that somebody has done it.
So what is named after a penis?
I think he is pronouncing PyPy as "pee-pee" instead of "pie-pie".
I'm a python fan, but I think you are being a little bit unfair with the ruby community.
Python simply has more people working on making it fast (the pypy project started 8 years ago, and only recently it started to deliver on its promises).
By the way, pypy is a framework for creating jitted dynamic languages (it's written in python, but can be used to implement any other language).
So chances are that soon, someone will write a Ruby implementation with pypy, and this ruby vm will enjoy all the performance of the python one, because it will have an automatically created just-in-time compiler.
Good idea. I say we shall henceforth pronouce this language as Pee-thon.
quora.com is now running on PyPy.
http://www.quora.com/Alex-Gaynor/Quora-product/Quora-is-now-running-on-PyPy
Not just faster than CPython, but faster than C for some common tasks. Pretty amazing.
However, this project is not yet very useful to the people who might be most interested in a really fast python, as it does not work with numpy. But when they get that to work, wow.
It's an impressive effort that they were able to get this to work at all. Python is really tough to optimize. All bindings are changeable at any time, even from another thread. (The latter is silly, but since it doesn't cost anything to do that in CPython, which is an inefficient naive interpreter that can only run one thread at a time and spends much of its time doing dictionary lookups. Von Rossum defines the language by what CPython can do. There's a huge speed penalty for allowing such extreme dynamism, about 60x over C/C++. The Google team tried to fix that with Unladen Swallow, but gave up when their JIT system was barely faster than CPython.)
PyPy's most effective optimization to date is that it figures out when numbers don't have to be boxed. This allows generating numeric machine code, rather than grinding through the object machinery for every number. They have to be prepared to discard code when something binding gets changed. This requires a very complex system, involving two interpreters (regular and backup) as well as the JIT compiler.
The PyPy crowd is at last starting on the tougher optimizations, like hoisting some operations out of loops. (FORTRAN compilers were doing this in the 1960s.) That's real progress, but it's very hard to do in such a dynamic language.
Many of the optimizations involve generating run-time code that checks to see if the normal case is occurring, and that no other code has patched the code or changed the data from the outside in a way that invalidates the fast path. Then there's code to unwind what the fast path was doing, and interpret or recompile. Most such code is never executed.
they'd use C++.", as you said? Funny - isn't Python written in C++ in its libraries of functions?? That said, it should have the same performance when you call its libs functions then! Your "argument" there, falls right apart.
Let's see some comparisons to real JIT compilation, LuaJIT.
sorry for the accidental drunken redundant moderation. I deeply apologize.
Jehovah be praised, Oracle was not selected
There's a huge speed penalty for allowing such extreme dynamism, about 60x over C/C++.
That number is surely made up; perhaps it applies to some specific set of benchmarks? In reality the factor is going to vary widely depending on the problem. And oh, you can't lump C and C++ together and call them C/C++.
1.) Those pipeline stalls would only result IFYou sent bad data to them (That's the fault of the coder or DBA for example, since you mentioned dynamically typed in your statement).
2.) Overheads from function calls are not that big. They get marshalled in process and lib loads are not "all or nothing".
(I.E./E.G. -> You only load the portion/function you need into the calling process, NOT the entire library itself!)
---
Thus, Your "argument's" are falling apart fast, and on 2 grounds:
A.) Most programmers or DBA's aren't that stupid about data sanity, not even rookies as far as dynamically typed languages.
B.) Secondly, you're making it sound as if calling out libs/dlls is some "huge overhead" well if so, how come Operating System API's work so well and fast then??
It matches what I've seen. I've written compilers, static analysers and visualisation code in Python. In most cases there is about a 100x difference in speed between the Python code and C code to implement the same algorithm. Python is still useful for prototyping in as most of that code could be written 5x to 10x more quickly than the C code.
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
I noticed that their progress seemed to have stopped, is there any official announcement?
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
And now for ruby please. I love ruby, but the performance issues drive me crazy sometimes.
I noticed that their (Unladen Swallow) progress seemed to have stopped, is there any official announcement?
Unladen Swallow is an ex-parrot.. "Jeffrey and I have been pulled on to other projects of higher importance to Google.", writes the former project lead.
In my experience, C tends to compile to about 1-6 lines of asm instructions, virtually all of which are necessary to perform the required task.
If you get 60 times faster than that without changing your algorithms, even with hand-coded assembly and intimate knowledge of the hardware, you're probably bypassing the entire CPU and running on moonjuice.
I'm sorry...I don't want to see how it looks or play with your PyPy. I will most definitely NOT click on your online demo to show your "Assembler".
Higher-level languages produce more asm per line but express more operations per line too. Easier to (validly) compare at the level of functions or whole programs.
If you get 60 times faster than that without changing your algorithms, even with hand-coded assembly and intimate knowledge of the hardware, you're probably bypassing the entire CPU and running on moonjuice.
Or spotting that something can be evaluated to a constant during compilation. That's a huge win when you can do it.
Of course, the other thing is that it is ever-so possible to write poor C code that can be out-performed by code generated from another language. A classic example of this is in string handling, where the C standard library routines can easily be used to create code with an awful performance, e.g., by repeatedly calling strcat() on the same long string, and any language with a string data structure more suited to repeated appending will outperform that. OTOH, a truly good C programmer will be using the right data structures and algorithms in the first place, making this a bit of a moot point.
"Little does he know, but there is no 'I' in 'Idiot'!"
Actually he just completely misread my post. The 60x increase was Python -> C, not from C -> assembly.
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
The same type of moonjuice that would make an AC mistake the word 'penalty' for 'increase'?
Lisp yawns at how easily impressed you are.
Seams boxing isn't too much of an issue for vanilla python when summing 1000,000 integers.
$ pypy -m timeit "sum(range(1000000))"
10 loops, best of 3: 55.2 msec per loop
$ python -m timeit "sum(range(1000000))"
10 loops, best of 3: 71.5 msec per loop
$ python3 -m timeit "sum(range(1000000))"
10 loops, best of 3: 62.7 msec per loop
$ python -m timeit "sum(xrange(1000000))"
10 loops, best of 3: 26.6 msec per loop
$ pypy -m timeit "sum(xrange(1000000))"
10 loops, best of 3: 132 msec per loop
$ pypy --version
Python 2.7.1 (?, May 02 2011, 19:05:35)
[PyPy 1.5.0-alpha0 with GCC 4.6.0]
$ python --version
Python 2.7.1
$ python3 --version
Python 3.2