Slashdot Mirror

So... C, the language the python interpreter is written in [wikipedia.org] then?

Since the faster interpreter is written in Python anyway, I don't think it makes sense for the language to be associated with C.

There are no Python to machine code compilers out there.

Interestingly, you are incorrect. There is one: PyPy. It's Python written in Python. And it's fast!

http://pypy.org/

Java 9 modularity spends a lot of spec to describe ways to limit the access from untrusted to trusted modules. The justification is code access security, this model splits _code_ into trusted and untrusted with trusted code having privileged access. And it's the real reason applets were so buggy - it's easy to trick a lot of "trusted code" by carefully crafting the data it works on.

In contrast, modern approaches put the trust boundary at the process level and simply disallow access to anything that is not explicitly whitelisted. See: http://doc.pypy.org/en/latest/... as an example. Your code can be as buggy as you want, but it doesn't matter if it can't access the whole world.

That would be pypy.

The are multiple implementations of Python. One is, in fact, written in Python: http://pypy.org/

The original Python interpreter is written in C for speed, not because an implementation in Python would be impossible.

I'm almost certain that you could write a Python interpreter in Python if you really wanted to, but it probably wouldn't have as good execution performance as an interpreter written in C.

An interpreter would be slower. A JIT-compiler, OTOH...

Why isn't Python written in Python?

You mean, like this?

PyPy selectively uses LLVM

Actually you are thinking of Unladen Swallow, the canceled project. They canceled it because they weren't getting the performance boost they for which they had hoped.

http://www.infoq.com/news/2011/03/unladen-swallow

PyPy uses a JIT.

http://pypy.org/features.html

PyPy is a whole new thing, Python written in Python. Their idea was to write a really smart optimizer by taking advantage of the powerful features of a high-level language (Python).

>> Python and C++, because numpy/scipy can't do everything

Yes, definitely true, and it's actually pretty easy to use them together.

If you don't want to write C++ however, there are a couple other options:

Cython - basically let's you generate c/c++ by writing Python like code and is very easy to use interacting with Python. It keeps the Cython parts of your code super fast, like straight up C.
http://cython.org/

Pypy - a super fast version of Python. If you write Python code yourself, and don't use off the shelf Python stuff, Pypy is crazy fast. (About C speed in my own tests of doing C like things.) Pypy gets slower if you use a lot of other Python code that wasn't written with Pypy in mind, but even then it's still normally much faster than regular Python. Using Pypy, you might just be able to write all the code in it and not have to bother with anything else.
http://pypy.org/

Both of these are easy enough that you can be up and running, writing/using new code, same day as downloading.

Finally, even if you are calling other code from C/C++, there's some new tools to make that easier. CFFI is a good example. It makes calling C/C++ pretty easy. I'm not sure how ready it is for a lot of real world use though.
http://cffi.readthedocs.org/en/release-0.7/

Yes, I did my master's thesis using simpy / scipy, integrated with lp_solve for the number crunching , all of which was a breeze to learn and use. It was amazing banging out a new recursive algorithm crawling a new object structure and just having it work the first time without spending several precious cycles bugfixing syntax errors and chasing down obscure stack overflows.

I used the psyco JIT compiler (unfortunately 32-bit only) to get ~100x boost in runtime performance (all from a single import statement, woo), which was fast enough for me... these days I think you can get similar boosts from running on PyPy. Of course, if you're doing more serious number crunching, python makes it easy to rewrite your performance-critical modules in C/C++.

I also ended up making a LiveCD and/or VM of my thesis, which was a good way of wrapping up the software environment and dependencies, which could quickly grow outdated in a few short years.

CPython is... really quite slow. You [i]do[/i] have to do something computationally intensive to notice, of course.

That point and your parent's question are actually related to the PyPy question in the interview actually. PyPy is a JIT for Python that is able to get significant speedups for computationally-bound code. They run 20 benchmarks comparing PyPy and CPython, they are faster on all 20, and see speedups ranging from about a 27% improvement to a ~30x improvement, on average running 6.2 times faster. (Link)

Do you see PyPy as the future ? http://pypy.org/
Or do you remain unconvinced, and -- if so -- why ?

+1 I'd love to know

Do you see PyPy as the future ? http://pypy.org/
Or do you remain unconvinced, and -- if so -- why ?

You picked a poor example, as PyPy is also state-of-the-art, PhD-powered, and loaded with just as much performance-enhancing optimizing code as V8. Additionally, JavaScript can easily be rejected on the demerits of being a poor language.

PyPy might change that in the future, especially with the Transitional Memory branch.

PyPy might change that in the future, especially with the Transitional Memory branch.

Unfortunately I already switched to python 3 for my own coding. They need funding to support python 3: http://pypy.org/py3donate.html so clearly money is an issue.

Unfortunately I already switched to python 3 for my own coding. They need funding to support python 3: http://pypy.org/py3donate.html so clearly money is an issue.

The primary implementations are written in C. Python is also implemented in Java, C#, and yes, even Python.

compilers, interpreters

PyPy

Google is breaking the Java "contract" with developers: portability.

I'd be much more (read: nonzero) sympathetic to that position if Google didn't explicitly state that their binaries run on Dalvik, and not the JVM:

Android includes a set of core libraries that provides most of the functionality available in the core libraries of the Java programming language.

Every Android application runs in its own process, with its own instance of the Dalvik virtual machine. Dalvik has been written so that a device can run multiple VMs efficiently. The Dalvik VM executes files in the Dalvik Executable (.dex) format which is optimized for minimal memory footprint. The VM is register-based, and runs classes compiled by a Java language compiler that have been transformed into the .dex format by the included "dx" tool.

The Dalvik VM relies on the Linux kernel for underlying functionality such as threading and low-level memory management.

But seeing as how cross-platform compatibility isn't a stated goal or feature of Java The Language on Android, that's all totally irrelevant to the situation at hand. Suppose someone implemented Python on a non-CPython VM. Your logic would imply that the Python Software Foundation should be able to sue them for breaking cross-platform .pyc compatibility. That's ludicrous.

Says the guy whose whole life is tied up in the language

That's fair.

and whose project, at Google, to speed it up, crashed and burned.

That's completely wrong. Unladen Swallow was not GvR's project, and it didn't "crash and burn". Unladen Swallow found that their approach was not speeding up Python as much as they had hoped, and the two Google employees were moved on to other projects. The code lives on, and I think people are still doing things with it, although it's clear that PyPy is a better approach.

To me at least, "crash and burn" implies a horrible failure with catastrophic consequences, and Unladen Swallow was considerably less dramatic than that. If you just meant to say "they didn't accomplish their goals", that would be a fair statement.

Python is slow because von Rossum refuses to cut loose the boat-anchor of "anything can change anything at any time".

That was a basic decision way back when, and it would be a profound change to make (the language wouldn't really be Python anymore). I personally don't make much use of this, but supposedly there are some programs that do.

PyPy, the newer Python implementation, uses two interpreters and a JIT compiler to try to handle the dynamism with less overhead. They're making progress, but they need a very complex implementation to do it, and they're many years behind schedule.

There is a very small group of people working on PyPy, and they are doing ambitious things. I'm not overly worried about their schedule.

You failed to mention that PyPy has already achieved great speed when compared to CPython. The latest PyPy is, on average, over 5x as fast as CPython.

http://speed.pypy.org/

if a module or class could only be modified from outside itself if it contained explicit self-modification code (like a relevant "setattr" call) most modules and classes could be nailed down as fixed, "slotted" objects at compile time.

This is an interesting idea. But I am not aware of anyone working on something like this.

Claiming that the "slow parts" should be rewritten in C is a cop-out.

I disagree. Way back at the dawn of time, GvR designed Python to make it easy to interface C code, just exactly as this sort of "escape hatch" to help projects that work well but are too slow, and also for libraries.

I think that the ability to link in C code was an important feature that helped Python win hearts and minds. It's slower than C, but at least you knew there is an escape hatch if you wind up having trouble with it.

Except for number-crunching, or glue code for existing libraries, it's seldom done.

Yeah, but that's kind of like saying that except for stop signs and traffic lights, you usually don't use the brake pedal in a car.

There are all sorts of useful libraries that have been glued into Python, and a major part of Python's popularity is that you can use powerful libraries from a convenient and friendly language.

SciPy is a great example: they took ferociously powerful libraries (written mostly in Fortran) and made them usable from Python. I know I for one can get more work done in Python than in Fortran.

As another example, a few years ago I worked on a project to make a DVD player with some fancy features, and we were doing our development in Python. It was fast enough, even on a cheap embedded processor, because all the heavy lifting was being done by C library code. And we got a lot of work done quickly.

(I have a Python program running right now which will run for over a week, parsing the street address of every business in the US into a standard format. The parser is complex enough that rewriting it in C would be a big job. There's no "inner loop".)

This sounds interesting.

Can you run this in PyPy?

Can you fan it out to multiple processes using the multiprocessing

Your speed arguments are essentially ones which boil down to python being an interpreted language. PyPy is a JIT compiler for python thats is on average 5 timers faster than normal CPython. Shed Skin is a python to C++ compiler which works decently as well.

The JOVIAL J4 compiler was itself written in JOVIAL J4.

Want something really mind-blowing? PyPy is a Python interpreter written in Python. It includes a tracing JIT compiler to optimize hotspots as it runs to get about 5 times faster than the native C Python. I've used it and I still can't quite believe it.

... would love to code using Python syntax but have the code run as fast as C/C++. Best of both worlds.

I'd like to point you to the PyPy project, which experiments with, among other things, JIT for Python.

The PyPy project aims at producing a flexible and fast Python implementation. The guiding idea is to translate a Python-level description of the Python language itself to lower level languages. Rumors have it that the secret goal is being faster-than-C which is nonsense, isn't it?

(You should also read their architecture page - their goals are truly impressive. Which is why I'm such a huge fan of the project)

You also have the project Shed Skin.

Shed Skin is an experimental compiler, that can translate pure, but implicitly statically typed Python (2.4-2.6) programs into optimized C++. It can generate stand-alone programs or extension modules that can be imported and used in larger Python programs.

So yes, there are people out there putting "tons of effort" into just that: Making python faster :)

... would love to code using Python syntax but have the code run as fast as C/C++. Best of both worlds.

I'd like to point you to the PyPy project, which experiments with, among other things, JIT for Python.

The PyPy project aims at producing a flexible and fast Python implementation. The guiding idea is to translate a Python-level description of the Python language itself to lower level languages. Rumors have it that the secret goal is being faster-than-C which is nonsense, isn't it?

(You should also read their architecture page - their goals are truly impressive. Which is why I'm such a huge fan of the project)

You also have the project Shed Skin.

Shed Skin is an experimental compiler, that can translate pure, but implicitly statically typed Python (2.4-2.6) programs into optimized C++. It can generate stand-alone programs or extension modules that can be imported and used in larger Python programs.

So yes, there are people out there putting "tons of effort" into just that: Making python faster :)

I love Python, don't get me wrong, but it is nowhere near Java regarding raw performance. Even the developers acknowledge this, with stuff like the Google sponsored Unladen swallow and PyPy.

Haven't toyed much with Ruby these days though. I should :)

The fact that they left out C# seems odd as well. It makes me wonder what the point was, really.

Google has an official list of four languages that they approve for use: Java, JavaScript, C++, and Python.

Most of their business runs on Java. Search, Google Maps, etc.

YouTube mostly runs on Python, IIRC.

So, they compared C++, on their list; Java, on their list; Scala, which is not on their list but runs on the JVM and thus would work for them; and a language invented at Google. They didn't test Python, but if they had it would have come in last place (and I say that as one who loves Python; it's reality).

Why is it surprising that Google was testing languages that Google might use for projects?

P.S. Exciting things are happening in the Python world with the PyPy project. PyPy is a Python system that is written in Python. You might expect that to be slow, and for years it was slow; but it includes a Just-in-Time compiler that generates native machine code, and it isn't slow anymore. In fact, it is now faster than the C Python, because it can optimize away lots of stuff that C Python has to do.

http://pypy.org/

steveha

Python is slower because it's a highly dynamic, interpreted language, and it's interpreter has also gotten a lot less attention than Java's compiler. Java is a compiled, much less dynamic language.

If by "compiled language," you mean that the first step in running a program is transforming source code into some other form, then both Python and Java are compiled languages (as are the vast majority of commonly-used language implementations). Both Java and Python implementations use compilers that transform source code into intermediate forms (byte code). In both cases, very little run-time optimization can be done at that level AFAIK.

The use of compilation is not truly a property of a language, but of an implementation of a language. For example, in addition to the official implementations, there are implementations of both Java and Python that transform source code into native machine code ahead of time, bypassing any byte code.

When talking about compilers, you also have to be careful about which compilers you're talking about. Most implementations of Java use a JIT compiler which transforms Java byte code into native machine language at run time. The official implementation of Python (CPython) does not have a JIT compiler, but there are several projects which add a JIT to CPython, including Unladen Swallow and Psyco.

I want to be extremely clear that I was not comparing the maximum execution speed of equivalent Python and Java programs, since Python will probably never rival Java in that way. I was talking about the suitability of the JVM for executing programs written in languages that are very different from Java, such as Python. Jython is an implementation of Python that does just that, but has only recently caught up to regular CPython's performance, even though it benefits from the JVM's JIT compiler.

As much work is being put into increasing performance of CPython, I think PyPy is much more interesting. Though PyPy is written in Python and a subset of Python called RPython, it can execute Python around three times faster than regular CPython or Jython.

You're right, a regular JVM might not do such a hot job of working through all the code generated by something like Python, but it could be used effectively with a language that is less annoying than Java while still being fairly static.

I doubt Ceylon is going to be a language I'd be much interested in. Python + Cython + C is a pretty potent combination for pretty much everything. It might be good for some things though.

I haven't tried Cython yet, but it sounds very interesting and I'll definitely try it when it seems appropriate. However, if just increasing performance is a goal, simply using PyPy might be just as good as Cython if examples like this are to be believed.

Because those parts are usually less than 10% of the total logic?

Also, you have stuff like Cython that lets you develop C extensions very easily.

The special advantage of this seamless Python/C intermix approach is that existing Python code can be tuned to almost the speed of C by just adding a few static type declarations and by making some adaptations in critical loops - without requiring complicated interface code. The coding speed and the readability of the code remains very Pythonic.

You also have projects like PyPy that are working on JIT support for python, which already perform the same or better than CPython on many workloads.

All in all, we live in interesting times, and remember: Don't dismiss things just because you don't like them :)

> There's "fat-val", "tracer JIT" and "method JIT". Just curious, given all these advances in JS speed, are there technical reasons why stuff like Python, Ruby and Perl aren't getting similar improvements in speed?

Perl... well, it's either dead or incredibly alive, depending on who you ask, but all development seems to be focused on Perl6.

Ruby doesn't have an "official" interpreter. The standard C implementation uses YARV for 1.9, which is considerably better than MRI (which was in 1.8). Rubinius is supposed to be faster, but isn't quite ready yet. Ruby Enterprise Edition seems to be fairly popular, but doesn't have 1.9 compatability yet. I believe JRuby is fairly widely-used, and it seems that it can perform better than YARV sometimes (and consistently better than MRI).

On the Python front, Unladen Swallow seems to be chugging along, although I'm not sure what we'll get out of it. PyPy is quite promising, and is faster than CPython in many benchmarks.

For Python, can they use: Psyco as a library? That would help being practically a Just In Time compiler.

Or they could use the recent release of Pypy. I haven't used it myself, but the benchmarks look appealing.

Why are people labelling this "big government bureaucracy"? Just because it's funded by the government, it doesn't have to be awful. PyPy, for example, was mostly funded by the EU, and that's very promising. KDE has been partially funded by governments as well.

And yet there are projects in the works like PyPy, where a Python interpretor is being written in Python itself.