Slashdot Mirror

> the first ones used threads, semaphores through python's multiprocessing.Pipe implementation. the performance was beyond dreadful, it was deeply alarming. after a few seconds performance would drop to zero. strace investigations showed that at heavy load the OS call futex was maxed out near 100%.

uhhm... wait what?

You are aware that python has global interpreter lock, right? And because of that multi-threaded performance in python is actually *worse* than single-threaded? But this is an inherent flaw in python interpreter and has nothing to do with Linux. It also has nothing to do with the topic of this article.

As someone who's done a fair bit of the reverse, and has recently needed to write C bindings for a few other languages (go, ruby), I thought this shouldn't be nearly so hard as you describe. Here's what I've come up with:

Example.cpp: Implements a class and C bindings for that class (The bindings could obviously be in a different file)
#include "ExampleCBindings.h"
#include
class Example {
public:
    Example(int v) { value = v; std::cout << "New Example(" << v << ")\n"; }
    ~Example() { std::cout << "Deleting Example(" << value << ")\n"; }
    int getValue() { return value; }
private:
    int value;
};

extern "C" {
    Example_class* newExample(int value) { return (Example_class*) new Example(value); }
    void deleteExample(Example_class* example) { delete (Example*) example; }
    int ExampleGetValue(Example_class* example) { return ((Example*) example).getValue(); };
}

ExampleCBindings.h: Public C bindings
#ifdef __cplusplus
extern "C" {
#endif
    typedef struct Example_t Example_class; // what I love most is that Example_t never exists ;)
    Example_class* newExample(int value);
    void deleteExample(Example_class* example);
    int ExampleGetValue(Example_class* example);
#ifdef __cplusplus
}
#endif

main.c: Simple C driver program that uses the api
#include "ExampleCBindings.h"
int main() {
    Example_class * ex1 = newExample(5);
    Example_class * ex2 = newExample(ExampleGetValue(ex1) + 2);
    deleteExample(ex1);
    deleteExample(ex2);
    return 0;
}

Limitations include:
* Does not support objects existing on the stack. You're in C, this is the norm for opaque data structures.
* Need to individually wrap every function. Would need to create separate wrappers for overloaded functions, or write a variadic function that doesn't enforce types (Sigh, neither language supports reflection)

However, if you only wish to implement public functions, writing a script that autogenerates a wrapper like this would be fairly easy (I've done similar for a mocking framework for C code - now that's actually painful (not the script, inserting stubbed functions into a binary)). A little googling came up with a more formal attempt here.

Like it or not, it's holding back performance:
http://www.dabeaz.com/python/GIL.pdf

Yes, Python has generators. Saying Python generators are like modern coroutines is like saying that C has garbage collection. Yes.. sorta of... but not really.

Could you elaborate on how Python generators are not enough to build an adequate substitute for coroutines?

It's arguable that Python's learning curve is a lot higher than those of C and Java. Take a look at this, for just one example: http://www.dabeaz.com/coroutines/

Oh, sure, C always knows what type it's handling. Even Lisp knows that, a language that doesn't know what type it's handling would be useless. But the dynamic/weak distinction you are nitpicking about is purely academic, the only difference is that, for a language that tolerates some flexibility in types, if the language is compiled it has to check the type at compile time ("weak") and if the language is interpreted it can check the type at run time ("dynamic"). But the use for "weak" typing is the same as the use for "dynamic" typing. An example of the flexibility allowed by C is the following: void qsort(void *base, size_t nmemb, size_t size, int(*compar) (const void *, const void *));

If C were "strong"/"static" typed, you would need a different quicksort function for each type of variable, with "weak"/"dynamic" typing, the same function can be reused. I don't care about the morphological difference between "weak" and "dynamic", because semantically they are the same.

believing that Python (or Bourne shell, or perl) is undeliverable and unprofessional -- that's 1980s thinking. Contraproductive.

There are two issues here: reliability and size. Generally, the programs I develop professionally are much bigger than the scripts I write from time to time in interpreted languages. I have found that interpreted languages are too slow for very big programs. And too often they fail when pushed to the limit.

A recent experience I had was when I had to port a system with 400k lines from VAX Fortran to a PC hardware. After trying all the commercial Fortran compilers I could find, no one could compile the whole system wihtout a significant amount of errors. Looking for a quick way to study the problem, I thought of writing a Fortran interpreter that could at least parse those 400k lines to see what were the main problems. I found the PLY project, a lex/yacc program in Python. A four function calculator in PLY works OK, but when I tried to make it run a full-fledged VAX Fortran parser it crashed. Using exactly the same Fortran language specification in flex/bison worked OK.

Unfortunately, that seems a pretty common experience, interpreted languages are OK for small systems, but aren't scalable to amny of the large problems professional programmers find in real life. At best, one gets the painfully slow start up times in big systems, like Open Office, SAP, or Eclipse, (all of which have significant amounts of Java code) for instance.