Slashdot Mirror
Van Rossum: Python Not Too Slow
snydeq writes "Python creator Guido van Rossum discusses the prospects and criticisms of Python, noting that critics of Python performance should supplement with C/C++ rather than re-engineering Python apps into a faster language. 'At some point, you end up with one little piece of your system, as a whole, where you end up spending all your time. If you write that just as a sort of simple-minded Python loop, at some point you will see that that is the bottleneck in your system. It is usually much more effective to take that one piece and replace that one function or module with a little bit of code you wrote in C or C++ rather than rewriting your entire system in a faster language, because for most of what you're doing, the speed of the language is irrelevant.'"
Python is strongly typed. Maybe you mean statically typed.
As someone simulating fluid-structure interaction with a number of constituent models and a lot of finite element (i.e. big matrix problems; using FEniCS - fenicsproject.org), using Python makes my overall quite-long algorithm much easier to flick through. Invaluable for debugging the theory as well as the implementation. FEniCS' Python interface ties into the standard C/C++ libraries using SWIG and, in simple cases, saves me working in C++. Very clear, well-written C++ is great for this application but I find it takes considerably longer to write than clear Python.
When I hit a more intricate problem, I realized I was going to have to solve a series of FE matrices by hand (with PETSc, written in C). It turned out to be pretty straightforward to pick up SWIG, write a short module in C and a Python interface. Done! Particularly useful as I believe getting FEniCS and petsc4py to play well is tricky.
So, I'd agree - having written a C++ version of my (simpler) problem and a Python/C version of the complicated one, the latter was definitely easier, and all the rate-limiting stuff is in C anyhow.
Doubt it would be true for every situation but +1 from an FE perspective.
The problem with Python isn't the speed -- he's right about optimizing with bits of C. The problem is the GIL. Without good multithreading support, I have to give up on Python for a large number of application domains.
Says the guy whose whole life is tied up in the language, and whose project, at Google, to speed it up, crashed and burned.
Python is slow because von Rossum refuses to cut loose the boat-anchor of "anything can change anything at any time". The straightforward implementation of Python, CPython, boxes all numbers (everything is a CObject, including an int or a float) and looks up functions, attributes, and such in a dictionary for every reference. And only one thread is allowed to run at a time. This allows one thread to dynamically patch the objects and code of another thread. Which is cool, but useless. 99.99+% of the time, there's no need for a dynamic lookup. Most program dynamism is shortly after program startup - once things are running, they don't change much. If, sometime shortly after startup, the program said "OK, done with self-modification", at which point a JIT compiler did its thing, the language would be much faster. But no. That's "un-Pythonic".
PyPy, the newer Python implementation, uses two interpreters and a JIT compiler to try to handle the dynamism with less overhead. They're making progress, but they need a very complex implementation to do it, and they're many years behind schedule.
Python, as a language, is very usable. But it's too slow for volume production. That's not inherent in the basic language. Python could remain declaration-free if there were just a few more restrictions on unexpected dynamism. By this is meant ways the program can change itself that aren't obvious from looking at the source code. For example, if a module or class could only be modified from outside itself if it contained explicit self-modification code (like a relevant "setattr" call) most modules and classes could be nailed down as fixed, "slotted" objects at compile time. The other big win is using enough type inference to decide if a variable can always be represented as a machine type (int, float, char, bool, etc.). That's a huge performance win.
Claiming that the "slow parts" should be rewritten in C is a cop-out. It makes the program more fragile, since C code can break Python's memory safety. Except for number-crunching, or glue code for existing libraries, it's seldom done.
(I have a Python program running right now which will run for over a week, parsing the street address of every business in the US into a standard format. The parser is complex enough that rewriting it in C would be a big job. There's no "inner loop".)
As the GP pointed out, if you're skilled enough to write optimized code in C/C++, why fuck around with Python at all?
Because we don't want to spend our time thinking about pointers and how to iterate over things? Because functional programming is actually really nice? Because in Python, you can download some data from the web, analyse it using a machine learning algorithm, plot the results, and install another package on the fly, combining 4 independent packages, and many ideas, in just 50 lines of code.
ctypes is really easy to use and to interface with C or Fortran. I use it a lot, namely for the 1% of the code that takes 99% of the time. The rest is nice OOP and functional.
NB: The message above might reflect my opinion right now, but not necessarily tomorrow or next year.
Yes, that is correct. You should write your apps in Python.
Your libraries, you should write in Python first, because it is also a great prototyping language. If they work fine (which they will in most cases) you have saved yourself a bunch of time. If they are too slow, you have saved yourself a bunch of time by fixing algorithmic bugs in a flexible language like Python. It is now trivial to convert it to bug-free C or C++.
If it's in you sig, it's in your post.
If you do need to know the index, you should write
It pays off to spend some time learning not only the syntax when you learn a new language, but also often used idioms in that language.
This sig under construction. Please check back later.
return (a>0)?a+1:a-1;
Tertiary operator FTW!
[Python is sometimes faster to write in.]
Python isn't just fast to write though - it's fast and easy to read and understand, to work with, and it's very easy to keep code clear and well documented. I'm not saying these things are impossible in other languages, but in Python it is effortless, and enocouraged by the language. There are big benefits to using it besides simply 'fast to write'.
-- Lattyware (www.lattyware.co.uk)
It takes more skill and system-programming knowledge to deal with the tricky interfaces between the internals of a Python interpreter and an external C++ program.
Is this experience talking, or guesswork?
Admittedly, I haven't had a need for it myself, but it looks easy enough. And you have plenty of options, too!
1. Extending Python with C or C++
2. ctypes
3. Cython
Examples for 1 and 2
Example for 3
It's The Golden Rule: "He who has the gold makes the rules."