Slashdot Mirror
See the PyPy JIT In Action
derGoldstein writes "Project PyPy is an alternative implementation of Python, with the main advantage being a Just In Time (JIT) compiler which speeds up your code considerably. They've announced the first public release of jitviewer, which is a visualization tool that helps you understand how your code is being compiled by PyPy's JIT, all the way down to assembly. If you just want to see how it looks and play with it, they've set up an online demo — just select a file, and click 'Show Assembler.'"
Slashdot DDOS machine in full effect
I love python, and pypy makes dream about the day python could be used for anything, making java irrelevant.
The language is readable, flexible, succinct, expressive and beautiful.
Programming in python is more a joy than a core, but it has always suffered from a wide performance gap in cpu intensive tasks compared to traditional static languages.
Hopefully, this is all about to change with projects like this.
It's worth noting that Pypy is more than just an alternative python implementation. It is a framework for implementing jitted compiled dynamic languages.
There are currently several works in progress with implementations of javascript, scheme, logo, php and other languages with pypy.
the connection has timed out
Python, the new Java
While a lot of people think that Python programmers and Ruby programmers are quite similar, I think this tool shows a pretty important difference between them.
Python programmers, on one hand, know about the underlying system that Python is running upon. They know how the OS works. They know how the hardware works and behaves. They know assembly, and understand the mapping between their high-level Python code and what's actually executed. Using this knowledge, they can take advantage of Python being a high-level language, while at the same time being able to make their code perform well enough that their end users won't even be able to tell that C or C++ wasn't used.
We see none of this from Ruby programmers. Lacking this knowledge results in them creating some of the worst-performing software our industry has ever encountered! When you try to abstract away basically everything that isn't more complex than a string, like they seem to like to do, you lose touch with reality. You don't realize that the algorithm you just implemented performs poorly because of how it's written. No, it can't be something you did wrong! It must be the OS. Yeah, it's just Linux being slow. Your code is fast!
This is why Python software is a pleasure to use, while Ruby software just leaves you hanging. Maybe sometime next week the Ruby app will perform the work you need it to perform.
When I was 14 I went deep sea fishing with my g/f Jody. I was 14 and she was 19. Well it so happened that my dad who was captaining the boat was drinking a lot of beer like 2 packs of 12 and he fell sound asleep. Me and Jody were bored. So we started making out and it was real hot under the sun and we went down into the cabin and made out more. We fell asleep and when I woke up I found her hands on my giant (approx 13 inch) lovemaker. She started to rub it like a genie bottle and that felt really good. Then she took off my pants and tried to get my meat baton in her mouth but kinda choked on it. By that time I was starting to feel REAL good. So I told her to take off her bra and she said "I have to take off my shirt first silly!". So she took off her shirt and bra and rubbed my erect sausage all over her 34DD nicely tanned boobies. At that time I spurted clear stuff all over her and she laughed. This made me brake up in tears because it looked like I wasnt a man enough for her or something and then she started telling me "No thats not what I meant!!" and started making out again. So after another 10 minutes of making out we started getting hard. She took off her pants and told me to lay down. I laid down with my 13" woman pleaser sticking right up and she lowered down on to me till I was fully inserted into her birth canal. So I lasted like 30 seconds and shot like a rocket into her and bodily fluids were gushing everywhere. She looked like she was going to laugh again and I was like Im gonna punch you in the ovaries and she really started laughing then so I pushed her on her belly and rammed my meat roll in and out of her till she SCREAMED "OH RODNEY! OH RODNEY! YES YES YES!" and then my dad woke up and saw what was going on and told us to finish up. I tried to finish but was too embarrassed. When I put my clothes back on and went back to the deck my dad told me he arranged a prostitute and she was never my g/f the entire time!
HallyLouAnna !! And without the hubby ever knowing that my dildo is on full tilt !!
I'm glad to see it scales well!
#DeleteChrome
Python, the new Java
That's Jython, not PyPy.
Someone remind me... Isn't PyPy the implementation that's based on LLVM?
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
Not a big fan of Python, but at some point I decided that an important check-off for any language is whether or not it can self-compile (or self interpret). PyPy gives it that checkoff and hat's off for having a JIT.
Compare and contrast with Larry Wall's admission that Perl isn't for everything, noting that Perl isn't written in Perl. In retrospect, that should have been a red flag. C is written in C, Lisp is written in Lisp to the point where it makes your mind warp. Not sure about Java; but it seems likely that somebody has done it.
I guess many open source developers are nothing but immature kids. So now we have a project cleverly named penis?
Seriously, what the hell is wrong with OSS and their lack of basic maturity? Why does open source always have to use stupid names?
So what is named after a penis?
I think he is pronouncing PyPy as "pee-pee" instead of "pie-pie".
Good idea. I say we shall henceforth pronouce this language as Pee-thon.
quora.com is now running on PyPy.
http://www.quora.com/Alex-Gaynor/Quora-product/Quora-is-now-running-on-PyPy
Not just faster than CPython, but faster than C for some common tasks. Pretty amazing.
However, this project is not yet very useful to the people who might be most interested in a really fast python, as it does not work with numpy. But when they get that to work, wow.
It's an impressive effort that they were able to get this to work at all. Python is really tough to optimize. All bindings are changeable at any time, even from another thread. (The latter is silly, but since it doesn't cost anything to do that in CPython, which is an inefficient naive interpreter that can only run one thread at a time and spends much of its time doing dictionary lookups. Von Rossum defines the language by what CPython can do. There's a huge speed penalty for allowing such extreme dynamism, about 60x over C/C++. The Google team tried to fix that with Unladen Swallow, but gave up when their JIT system was barely faster than CPython.)
PyPy's most effective optimization to date is that it figures out when numbers don't have to be boxed. This allows generating numeric machine code, rather than grinding through the object machinery for every number. They have to be prepared to discard code when something binding gets changed. This requires a very complex system, involving two interpreters (regular and backup) as well as the JIT compiler.
The PyPy crowd is at last starting on the tougher optimizations, like hoisting some operations out of loops. (FORTRAN compilers were doing this in the 1960s.) That's real progress, but it's very hard to do in such a dynamic language.
Many of the optimizations involve generating run-time code that checks to see if the normal case is occurring, and that no other code has patched the code or changed the data from the outside in a way that invalidates the fast path. Then there's code to unwind what the fast path was doing, and interpret or recompile. Most such code is never executed.
they'd use C++.", as you said? Funny - isn't Python written in C++ in its libraries of functions?? That said, it should have the same performance when you call its libs functions then! Your "argument" there, falls right apart.
Let's see some comparisons to real JIT compilation, LuaJIT.
sorry for the accidental drunken redundant moderation. I deeply apologize.
Jehovah be praised, Oracle was not selected
There's a huge speed penalty for allowing such extreme dynamism, about 60x over C/C++.
That number is surely made up; perhaps it applies to some specific set of benchmarks? In reality the factor is going to vary widely depending on the problem. And oh, you can't lump C and C++ together and call them C/C++.
1.) Those pipeline stalls would only result IFYou sent bad data to them (That's the fault of the coder or DBA for example, since you mentioned dynamically typed in your statement).
2.) Overheads from function calls are not that big. They get marshalled in process and lib loads are not "all or nothing".
(I.E./E.G. -> You only load the portion/function you need into the calling process, NOT the entire library itself!)
---
Thus, Your "argument's" are falling apart fast, and on 2 grounds:
A.) Most programmers or DBA's aren't that stupid about data sanity, not even rookies as far as dynamically typed languages.
B.) Secondly, you're making it sound as if calling out libs/dlls is some "huge overhead" well if so, how come Operating System API's work so well and fast then??
It matches what I've seen. I've written compilers, static analysers and visualisation code in Python. In most cases there is about a 100x difference in speed between the Python code and C code to implement the same algorithm. Python is still useful for prototyping in as most of that code could be written 5x to 10x more quickly than the C code.
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
I noticed that their progress seemed to have stopped, is there any official announcement?
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
And now for ruby please. I love ruby, but the performance issues drive me crazy sometimes.
I noticed that their (Unladen Swallow) progress seemed to have stopped, is there any official announcement?
Unladen Swallow is an ex-parrot.. "Jeffrey and I have been pulled on to other projects of higher importance to Google.", writes the former project lead.
In my experience, C tends to compile to about 1-6 lines of asm instructions, virtually all of which are necessary to perform the required task.
If you get 60 times faster than that without changing your algorithms, even with hand-coded assembly and intimate knowledge of the hardware, you're probably bypassing the entire CPU and running on moonjuice.
I'm sorry...I don't want to see how it looks or play with your PyPy. I will most definitely NOT click on your online demo to show your "Assembler".
Higher-level languages produce more asm per line but express more operations per line too. Easier to (validly) compare at the level of functions or whole programs.
If you get 60 times faster than that without changing your algorithms, even with hand-coded assembly and intimate knowledge of the hardware, you're probably bypassing the entire CPU and running on moonjuice.
Or spotting that something can be evaluated to a constant during compilation. That's a huge win when you can do it.
Of course, the other thing is that it is ever-so possible to write poor C code that can be out-performed by code generated from another language. A classic example of this is in string handling, where the C standard library routines can easily be used to create code with an awful performance, e.g., by repeatedly calling strcat() on the same long string, and any language with a string data structure more suited to repeated appending will outperform that. OTOH, a truly good C programmer will be using the right data structures and algorithms in the first place, making this a bit of a moot point.
"Little does he know, but there is no 'I' in 'Idiot'!"
Actually he just completely misread my post. The 60x increase was Python -> C, not from C -> assembly.
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
The same type of moonjuice that would make an AC mistake the word 'penalty' for 'increase'?
Lisp yawns at how easily impressed you are.
Seams boxing isn't too much of an issue for vanilla python when summing 1000,000 integers.
$ pypy -m timeit "sum(range(1000000))"
10 loops, best of 3: 55.2 msec per loop
$ python -m timeit "sum(range(1000000))"
10 loops, best of 3: 71.5 msec per loop
$ python3 -m timeit "sum(range(1000000))"
10 loops, best of 3: 62.7 msec per loop
$ python -m timeit "sum(xrange(1000000))"
10 loops, best of 3: 26.6 msec per loop
$ pypy -m timeit "sum(xrange(1000000))"
10 loops, best of 3: 132 msec per loop
$ pypy --version
Python 2.7.1 (?, May 02 2011, 19:05:35)
[PyPy 1.5.0-alpha0 with GCC 4.6.0]
$ python --version
Python 2.7.1
$ python3 --version
Python 3.2