Slashdot Mirror

Or Cython for the best of both worlds! :)

I recommend checking out Cython as it allows you to speed up the slow parts of your code and has the option of statically-typed variables. Let's you optimize time-consuming loops, etc.

Python is 50 times slower than C++, but Python also allows to write in an easier way algorithms that are exponentially faster. O( n ) >> O( 50 log (n)) ..

If speed becomes a factor, Cython comes into play

What Python doesn't have is strong type enforcement.

Check out Cython then (not to be confused with CPython), which is a "superset" of Python and includes this to make the code substantially faster potentially.

Now go - write a devices driver or Linux kernel in Python.

I wonder if that can be done. Write the driver in Python and use Cython to convert Python to C.

I once wrote a Python script to roll a pair of dice 1M times that took 123 seconds. Used Cython to convert the script to C that ran in a second.

Learn how to use a compiled language like Java.

If you have a need for speed, compile Python code to C binary with Cython.

http://cython.org/

You can compile Python into C. I had a Python script to roll a pair of dice one million times that took 123 seconds. Compiled Python to C, it took four seconds.

http://cython.org/

Having a working Python implementation will also give you a better understanding of which parts are performance/memory sensitive. This may help guide you while rewriting in a different language, or you may find that you can achieve your goals just by hot spot optimizing your Python code using some of these fine tools:

http://www.numpy.org/ can give you compact arrays of unboxed types and fast operations over them.
http://cython.org/ is an amazing and versatile tool which allows you to compile your Python code, optionally add type information, optionally manage memory yourself, optionally interact with C/C++ code very easily.
OpenCL/CUDA if your work can take advantage of them.


If the standard Python runtime isn't simply too big for your project I can't recommend Cython highly enough. Only tackling the parts that need it is one of the keys to successful optimization and Cython lets you do just that even if it is just one loop in an otherwise pure Python file.

Check out http://cython.org/. This project will enable you to write high level logic in python and drop to C in the performance critical sections of your code.

oh, and it would not really hurt if you did have static data types.

The Cython language is exactly this: a superset of Python with static type declarations.

>> Python and C++, because numpy/scipy can't do everything

Yes, definitely true, and it's actually pretty easy to use them together.

If you don't want to write C++ however, there are a couple other options:

Cython - basically let's you generate c/c++ by writing Python like code and is very easy to use interacting with Python. It keeps the Cython parts of your code super fast, like straight up C.
http://cython.org/

Pypy - a super fast version of Python. If you write Python code yourself, and don't use off the shelf Python stuff, Pypy is crazy fast. (About C speed in my own tests of doing C like things.) Pypy gets slower if you use a lot of other Python code that wasn't written with Pypy in mind, but even then it's still normally much faster than regular Python. Using Pypy, you might just be able to write all the code in it and not have to bother with anything else.
http://pypy.org/

Both of these are easy enough that you can be up and running, writing/using new code, same day as downloading.

Finally, even if you are calling other code from C/C++, there's some new tools to make that easier. CFFI is a good example. It makes calling C/C++ pretty easy. I'm not sure how ready it is for a lot of real world use though.
http://cffi.readthedocs.org/en/release-0.7/

a complete answer would be Python and C++, because numpy/scipy can't do everything and Python is still very slow for number-crunching.

That's why more people should check out Cython. It's basically it's own language modeled closely after Python but uses C-like language constructions when optimizations are helpful. I quite liked it when I checked it out, but I haven't been following it much recently.

Wrong.

Cython is a language that makes writing C extensions for the Python language as easy as Python

As for your "reference implementation" pedantry, it's clear in

What the fuck do you think you were using

that he's referring to CPython being the most widely used Python implementation, and that it's implemented in C.

That's not to say AC is correct, though: performance is a valid reason to rewrite Python code in C.

What the fuck do you think you were using when you did the rest of the code in Python? Cython is Python.

Well, you just set off the "I don't know what I'm talking about" alarm. Cython is a derivative of Pyrex, neither of which should be confused with CPython. And of course CPython is not Python either, it's the reference implementation of an interpreter for the Python language.

It takes more skill and system-programming knowledge to deal with the tricky interfaces between the internals of a Python interpreter and an external C++ program.

Is this experience talking, or guesswork?

Admittedly, I haven't had a need for it myself, but it looks easy enough. And you have plenty of options, too!

1. Extending Python with C or C++

2. ctypes

3. Cython

Examples for 1 and 2

Example for 3

Cython aims to do something similar, but the code is not pure Python - it introduces some language extensions of its own - so the boundary between non-optimized-but-idiomatic and optimized-but-verbose code is a translation unit. It would be interesting to do something like that, but work directly with regular Python files, using standard decorators and annotations to drive the translator. I.e. you could have a .py file that has typical code, but then one function like so (note, all still valid Python):

@optimize
def foo(x: int, y: int) -> int
  return x+y

and do an extra pass with a tool that finds all those @optimize declarations and spits out C code for them, taking into account all type annotations to improve codegen, such that the result is readily consumable as a Python module. Then, at runtime, @optimize would just throw away the function body, and replace it with the corresponding C function from that module.

I don't think much would be needed beyond those annotations. Local variables can be type-inferred, and for cases where the right-hand side of the assignment to a local is of an indeterminate type (say, a Python function call), an explicit cast would suffice to indicate the expected type. Still, if needed, no-op functions can be provided for type asserts to be read by the translator, e.g. typed_local(x, int).

stoolpigeon is exactly right. The only thing I have to add, just to reinforce that Python is the right teaching language, is that it was first developed based on a language (ABC) that was designed specifically for teaching kids how to program.

http://www.linuxjournal.com/article/5028

"Guido: Yeah! So a language like Python, which actually has roots in educational languages, will do better. Python is very strongly inspired by a language called ABC; I worked on the ABC implementation designed by colleagues of mine in the early '80s. It was a wonderful language for teaching. The history of Python comes out of the frustration I had with that language when it wasn't being used for teaching, but for day-to-day ad hoc programming—but that's a different story. Python inherits a lot of that focus to make it very simple, easy to understand, easy to remember and easy to learn. It's a very good language to start teaching. We are very hopeful about that side of the CP4E effort."

Also, instead of jumping straight from Python to C, you could use Cython as a bridge, and use it to teach variable typing basics.

http://www.cython.org/

If I were teaching programming today, this is the order I would use: HTML, Python Scripts, Python Modules, Python GUI, pygame, Cython, C, C macros, C libraries, C compiler optimization, assembler... and maybe yacc, or just general compiler design. Tangential languages like Perl, Java, Ruby, and C++ can be introduced at any point, to show different implementations of concepts first learned in Python, but they are all optional. If anyone gets stuck somewhere before compiler design, let them stay there as long as they want. Everyone has their own pace.

To the extent Groovy is a "superset" of Java, it is a way to have a shell for doing things in Java without having to build; just type in commands at the prompt, or stick them in a file. One way to look at things might be to think of Java as a way to have compiled Groovy, sort of like Cython is a way to have compiled Python. Whether or not the extra non-Java features Groovy provides are worth using... that is debatable.

Larry

One word: cython.

Ultimately what I want is an interpreted language that can be compiled. So for example, in python one rarely actually uses the introspective ability to modify ones self, or even takes advantage of duck typing. instead one usually calls functions with the same type arguments and so forth. So if one just had the ability to switch off the dynamic typing and self-modifying capabilities so that one could compile it it sure would be one sweet language.

It sounds like you want something like Cython.

That's the beauty of Cython, check it out.
I think numpy and scipy will start using it more and more.

For such problems, Cython ( http://cython.org/ ) is really nice.

One such project is Cython. It enables you to write in Python which gets compiled to C; trivializes using Python objects in C and vise verse.

When I meant not good at linear algebra, I meant that it is slow. For example, Sage is over 30x faster at computing the characteristic polynomial of a matrix over the integers. Regarding number theory, there isn't really any support in Mathematica for working with number fields, modular forms, or elliptic curves. What I meant by "real" programming language was that there is a lot of software out there that can be taken advantage of. Say for instance I need to work with data stored in an relational database. How easy is that to do with Mathematica? It is trivial with Sage since Sage uses Python. When Sage needs to do things fast, it uses Cython ( http://www.cython.org/ ) which is almost a superset of Python and compiles down to C.

--Mike

Disclaimer: I'm a Sage developer.

Sage has a very good solution to this: Cython. It's a very easy language, almost identitical to Python, which can be used to bind C to Python (for instance, we use GMP and GSL extensively through Cython) as well as compile Python-like code to C, which can be accessed by Python & vice verse. It's very intuitive, and very fast.

Sage( http://www.sagemath.org/ ) is currently the most full=featured open-source computer algebra system. It is being developed by the two authors of the AMS opinion piece (and many others including myself). Our goal is to provide a free, viable, open-source alternative to Mathematica, Maple, MATLAB, and Magma. Some nice features of Sage include:

* It uses Python as its programming language so that you can use any existing Python modules with your Sage programs.
* Sage also includes Cython ( http://www.cython.org/ ) which is based on Pyrex and allows one to easily compile Python code down to C for speed.
* Sage's notebook interface with also interface with pretty much every existing computer algebra system, open-source or not.
* Sage includes Maxima, GAP, Scipy, Numpy, and many other open source math packages.
* A very active developer community. If there is something that you need Sage to do, chances are that there will be a number of developers willing to help you out.

For some screenshots, see http://www.sagemath.org/screen_shots/ .

One of the things that Sage needs most now is more users. So, if you have an interest in open source math software, definitely check out Sage.

You might want to take a look at http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/ and/or http://www.cython.org/