Slashdot Mirror

Here's a daft idea that just might work: try SWIG, the Simplified Wrapper and Interface Generator. SWIG is an amazing piece of software that essentially takes your C and C++ .h files and automagically creates interfaces for them in your HLL of choice (choice = {Perl, Python, Tcl, Ruby, some Schemes, and a few others}).

You may be able to make mods with very little effort at all... just make a .pm from the header files.

--Tom

If OOP seems like overkill for small and specific engineering problems but you're facing the problem of maintaining and enhancing a large procedural code base, then there's a nice solution.

One of the most efficient ways of tackling this kind of problem is to keep your existing investment in procedural code but wrap it using a scripting language - preferably something like Python. Python has enough classical OO built in to keep your front end clean and maintainable but interfaces easily to programs written in C or C++. There are also tools to do this wrapping automatically for you. Your engineering data and equations will just look like normal Python objects but of course run at the same speed - minus a small amount of call overhead.

Example: after experimenting with Ken Perlin's source code for the 1, 2 and 3d noise which now bears his name, I found that the code was becoming hard to maintain and enhance without changing bits all over the place. Messy. Rewriting it all in Python or C++ would have been a pain because it was optimised C with lots of inline Asm. The solution was to keep the existing code base doing what it did best - calculating perlin noise functions at full speed - and just create a Python wrapper for the interface functions. In the end I got the best of both worlds - Python kept the frontend testbed clean and easy to maintain, the back-end code was unchanged. Time taken to convert the lib - a couple of hours reading the Extending and Embedding part of the Python manual.

Try it - you might like it. I don't know if Python talks to Fortran though.

This approach to software has been codified into a Design Pattern: Alternate Hard and Soft Layers. From the WikiWiki page:

By virtue of the FirstRuleOfOptimization, go ahead and write most of your code in the highest level language you can find.

By virtue of the ThirdRuleOfOptimization, when you must, use a profiler and find the slow parts of your program. Take those parts, and write them in a lower-level language.

In other words, use a "soft", dynamic language for the parts of your program that may change, and that don't require extreme effeciency. Use "hard", static, compiled languages for the parts of the program that must run as fast as possible; or that need to do low-level memory-twiddling. To put it even more succinctly, use the right tool for the right job.

Lately I've found that using SWIG makes this pattern very easy to apply in real life.

"SWIG is a software development tool that connects programs written in C, C++, and Objective-C with a variety of high-level programming languages. SWIG is primarily used with common scripting languages such as Perl, Python, Tcl/Tk, Ruby, Guile and MzScheme."

You can write your code as portable C++ classes, and then use SWIG to automatically read in the header file, and write out all the glue code to plug your C++ classes (or C libraries) into the scripting language of your choice.

-Don

I've written several modules for Ruby in C, and integration is easy. What's more, you get free garbage collection in your C code, just from calling Data_Wrap_Struct, because the object gets handled by the Ruby garbage collector and is garbage collected (it's free() function is called) sometime after you return to the Ruby interpreter (or as soon as you run out of memory and the GC gets invoked). If you want your code to be automatically wrapped, then there is also Swig support (though the Ruby module for Swig lacks the documentation that the TCL, Python, and Perl modules have).

Where the integration becomes tricky is when you try to interface with C++ code. Ruby uses setjmp and longjmp for its exception handling, so if you call rb_raise, you'd better make sure you don't have any objects sitting on your stack that might need to be destructed! I don't think Perl or Python do much better at this (though I could be wrong).

You wouldn't normally find me evangelising about this or that language.. but since I started using Python ('bout 18 months ago) I've barely used anything else.. it's a truly impressive acheivement, I believe, and a superbly pragmatic one at that. The sheer speed with which you can get an algorithm down in Python is, IMO, it's greatest strength. Add in to that all the object oriented features that you'd generally use day-to-day in C++, great ease of embedding and extending, powerful, *native* data structures and 90% cross-platform compatibility, clear syntax.. it's just *such* a productive language. I used to be a serious C++ guy but these day's I only go near that if I really have to.. for high performance apps were you *really* have to squeeze that last bit of performance (and, honestly, how many of those are there these days with clocks so cheap) I just migrate the critical bits to C/C++ and use a Python main line to drive 'em.. (see SWIG for a cool way to wrap existing code). I ain't gonna get into the Perl argument.. personally it just looks to me like the Perlophiles are just arguing against to protect their intellectual investment. IMHO, Perl is nothing like a full or proper language. Python can certainly claim to be.

http://www.swig.org

Also, you will need a program called "g-wrap". You can find it in the /pub/g-wrap directory on the gnucash ftp server.

There are several issues here: peer review, architecture, algorithm and implementation.

Peer Review: At each step in the process (architecture, algorithm, and implementation), you should publish your ideas for criticism by experts. slashdot, Advogato, the Cypherpunks mailing list, sci.crypt, and the Crypto++ mailing list might (or might not) be good places to find such people.

Architecture: You should do a public key architecture where every participant has a public/private key pair and the public keys are used to sign and encrypt symmetric keys that are then used for encryption and authentication of messages. There are three feasible architectures for public key distribution. You have to choose one based upon your threat model. Almost all realistic threat models should be handled using the first option: "opportunistic public key distribution". If you don't have a threat model in mind at all then you might as well use the first option. If you do have a threat model which precludes using the first option then I'd like to hear about it -- you must be doing something very interesting indeed.

• Option one: "opportunistic public key distribution". The first time any pair of people talk to one another, they exchange public keys in an un-authenticated exchange. (Also: you could just do Diffie-Hellman key generation here.) After that, they remember each other's public keys for future use. This is susceptible to an active attack (a "Man In The Middle Attack"), during the first step (though not afterwards). However the cost to the attacker of executing a MITM attack is probably far more than the payoff. This depends on your threat model.
• Option two: make it the user's problem; Each user decides whether to use a given key to talk to another user or not. This is the user interface nightmare that single-handedly prevented strong crypto from becoming standard in e-mail, but for a few applications it might be the right thing.
• Option three: hardcoded; Generate key pairs yourself and include them in the application. For example if you are going to have a single central server in your system which you operate then you can generate a key pair for it, put the secret key on the server, and put a copy of the public key into each copy of the client (e.g. include the public key hardcoded into the client source code). This doesn't work as well if you want to distribute copies of your server for other people to operate, but refer to "Option one"...

Algorithm: You probably just want Triple-DES and RSA (after September of this year, when RSA becomes free of patents) or else Triple-DES and Diffie-Hellman. It should be easy to switch to a different symmetric cipher later after the new ones have been peer-reviewed and tried by fire, but for starters you want the old standbys that have already withstood the test of time. They will be fast enough for you at first and if you need more speed later you can switch.

Implementation: Your first choice should be to use an extant implementation. Don't try to implement it yourself no matter how simple it looks. Satan's Computer is deceptively subtle to people who are used to hacking Murphy's Computer.

I prefer Wei Dai's Crypto++ library, but that is because I'm doing complex non-standard crypto tricks. If you just want simple "encrypt/authenticate a stream" functionality then use a TLS implementation like OpenSSL. By the way, if anyone wants to make Python wrappers for Crypto++ (possibly with the aid of Swig) then I would love to hear about it!

Okay that's my advice. Specific pitfalls to avoid are: skipping peer-review, trying to design a generalized perfect public key architecture to handle all possible threat models, using a newfangled or non-standard algorithm ("In open source hackery, newfangled is good. In crypto, not."), and implementing it yourself instead of using a library.

Please direct all flames and accolades to: zooko@schowto.mad-scientist.com

This should be very doable with current software. Check out swig at this link. It provides a framework for interfacing C/C++ with scripting languages like python, perl, guile, etc... I'm not sure if you can interface python and perl using swig, but you could definitely write a C library that could be called from python. This C library could then use perlembed to make the appropriate perl calls. A certain amount of annoying mucking about with internals in C would be necessary, but its probably easier than writing a dozen DBI backends in python. Maybe you could even code some common data type conversions in C and make the interface general enough to handle most modules.

Of course, you could also just write a small generic server in perl that could communicate with a separate python process using an intermediate data structure implemented in both languages. And I'm sure there are a dozen other more intelligent ways to do it. After all, perl is involved, so TMTOWTDI ;-)

Python is not C hostile, thanks to SWIG. I know I'm using it all the time to automatically wrap C++ class as Python classes.

It is not "anti-Unix": I've been using it for 5 years on Linux. It's funny, these days, people who are not "rabidly anti-Microsoft" are labeled as anti-Unix.

Python should be more easy (no pointer stuff). C would be more useful since 99% of Linux code is written in C ; Python would be more useful for your general programming knowledge and coding style, and can be interfaced with C (high-level in Python, low-level speed-intensive code in C, and glue with swig).

Maybe try to learn Python for a short time, and C once you're tired. The two languages are complementary anyway.

"What you describe with the C interfaces is basically writing a C binding for your C++ program/library."
Yes. Except that when you are linking another language with C code you have to write an wrapper anyway. You don't just link your C libraries with Perl or Python. You have to write wrappers. What I said is that these wrappers, even when their interface is in "C" style, can perfectly call C++ functions. The amount of work is the same for C or C++. For Python which is a OO language, you have even an obvious mapping between C++ objects and Python objects (and even SWIG can generate automatically the wrappers for C++ code): thus you can reuse directly the documentation: if in C++ you have to write button.setLabel("my label"), then in Python you'll write button.setLabel("my label").