Slashdot Mirror

Even the Python folks tell you to write your high performance code in C or C++.

True, but one of the smartest things Guido van Rossum did early on was to make it easy to interface C and C++ code to Python. It's why SciPy is winning so big in the sciences; it's the convenience of Python with the performance of Fortran. The libraries that do the work for SciPy are old numerical libraries that are very well optimized, very well debugged, very well understood, and very very useful. So, you can work in Fortran... or you can work in Python, enjoying the much friendlier interpreted language, and barely give up any performance vs. the pure Fortran. The hard work is done in Fortran, and the overhead of using Python to set up your calculations is trivial compared to the work of the calculations themselves.

https://www.scipy.org/

Python also provides a "lab notebook" environment through the Jupyter project. Nobody is going to try to use Fortran or C directly in the notebook.

http://jupyter.org/
https://www.datacamp.com/community/tutorials/tutorial-jupyter-notebook

And pretty much every library you might want to use has already been glued into Python by someone. Computer vision? Running code on a GPU? Signal processing? Solving equations? Whatever you need to do, you can do it conveniently in Python and it will be fast.

So yeah, if you write your own matrix multiply in pure Python it will be roughly 50x slower than compiled C. But nobody does that, and in the real world Python is fast enough to do real work.

R is definitely still ahead for data modeling, but Python has some advantages too. With a bigger set of modules (libraries) to choose from and high popularity in the financial sector, there are big improvements all the time. For the purposes of this discussion, the most important Python modules are:
IPython: powerful interactive shell
numpy and scipy: numerical, matrix, and scientific functions (matlab-ish)
pandas: R-like data structures and data analysis tools (analysis mostly limited to regression)
statsmodels: statistical analysis, complements pandas
sk-learn: machine learning

So can Python do everything that R can? No. Or, at least, not as easily. But it is improving in that direction quite quickly, and if Python's data analysis capability meets your needs, then you can likely do everything in one language instead of calling R routines from another.

Better yet, Fortran + Python.

http://docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html#f2py

I used it to wrap some crazy magnetometer processing code written in Fortran into a nice Python program. I ripped out all the I/O from the Fortran code and moved it into the Python layer. It worked great. Fortran is AWESOME at number crunching but SUCKS ASS at IO or well pretty much anything else, hence Python.

agreed, Fortran for the numbers and python for the wrapper. if your not used to python, you could use C. which you know already don't fix what isn't broke.

Seriously consider FORTRAN

On the other hand, the basic underpinnings of SciPy are FORTRAN. The BLAS and LAPACK libraries, and other fast and well-understood FORTRAN libraries, are "wrapped" by SciPy.

http://www.scipy.org/scipylib/faq.html#id12

Using the IPython notebook, you can work with data sets in an interactive way that FORTRAN won't do. But the number crunching is being done for you at FORTRAN speed because it is compiled FORTRAN code that is doing the work.

http://ipython.org/notebook.html

Since you mention VBA, I suspect that your data is in Excel spreadsheets? If you want to try to speed this up with minimum effort, then consider using Python with Pyvot to access the data, and then numpy/scipy/pandas to do whatever processing you need. This should give you a significant perf boost without the need to significantly rearchitecture everything or change your workflow much.

In addition, using Python this way gives you the ability to use IPython to work with your data in interactive mode - it's kinda like a scientific Python REPL, with graphing etc.

If you want an IDE that can connect all these together, try Python Tools for Visual Studio. This will give you a good general IDE experience (editing with code completion, debugging, profiling etc), and also comes with an integrated IPython console. This way you can write your code in the full-fledged code editor, and then quickly send select pieces of it to the REPL for evaluation, to test it as you write it.

(Full disclosure: I am a developer on the PTVS team)

Better yet, Fortran + Python.

http://docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html#f2py

I used it to wrap some crazy magnetometer processing code written in Fortran into a nice Python program. I ripped out all the I/O from the Fortran code and moved it into the Python layer. It worked great. Fortran is AWESOME at number crunching but SUCKS ASS at IO or well pretty much anything else, hence Python.

Yes, I did my master's thesis using simpy / scipy, integrated with lp_solve for the number crunching , all of which was a breeze to learn and use. It was amazing banging out a new recursive algorithm crawling a new object structure and just having it work the first time without spending several precious cycles bugfixing syntax errors and chasing down obscure stack overflows.

I used the psyco JIT compiler (unfortunately 32-bit only) to get ~100x boost in runtime performance (all from a single import statement, woo), which was fast enough for me... these days I think you can get similar boosts from running on PyPy. Of course, if you're doing more serious number crunching, python makes it easy to rewrite your performance-critical modules in C/C++.

I also ended up making a LiveCD and/or VM of my thesis, which was a good way of wrapping up the software environment and dependencies, which could quickly grow outdated in a few short years.

in Python... several minutes later it became obvious that I would need something faster. Ported the code over to C++, I think it finished in about 90s.

The obvious solution is NumPy & SciPy. Get much of the ease of programming in Python with the speed of optimized C.

So you didn't even bother to look into SciPy, but you "know Python isn't designed to handle powerful scientific computing". This will come as a surprise to all the scientists using SciPy for powerful scientific computing.

http://conference.scipy.org/proceedings/
http://andy.terrel.us/blog/2012/09/27/starting-with-python/
https://us.pycon.org/2012/schedule/presentation/463/

I'm not even going to try to persuade you. Have fun with Matlab and have a nice life.

I do, however, disagree with Guido's statement that "Python is fast enough". Whether it is or not depends entirely on what you're doing. For my purposes I don't think a 12-core computer optimally programmed in assembler would be "fast enough".

It's faster to write code in Python than in C or C++ or Pascal or Java. But Python isn't "fast enough" unless your program is I/O bound. And it's inability to handle multiple processing uints gracefully is a real problem. (Not that anyone has a decent answer to that except the dataflow people and the pure functional language people.) Multiple core machines are now the rule rather than the exception, so the GIL is no longer acceptable. Even Ruby attempts to address that, though they didn't really follow through after considering their library situation.

You're absolutely correct that whether a language implementation is "fast enough" depends totally on the problem being solved. If the program is I/O bound, as most are, Python and many other languages are fast enough. Even if part of the problem is CPU bound, there is usually a lot of code that is not, which is why hybrid C/Python solutions are so common. For example, NumPy can take advantage of multiple CPUs when doing matrix calculations, allowing one do heavy, fast number crunching without writing a line of C.

If you need to do CPU-bound processing in Python code, threads won't help if you're using CPython, but they might if you're using Jython which is JVM implementation that has no GIL. Alternatively, you can use the multiprocessing module which avoids the GIL and a lot of other potential problems with threads at the cost of having to explicitly communicate between processes rather than share data structures directly.

Because Python's such a nice glue language, it can talk to super fast Fortran and C libraries to be at at least comparable speed with Java. See NumPy. This is kind of stuff scientists use for number crunching.

In many cases, it probably depends on the workflow that a project uses. You don't get mailling lists on GitHub. Don't laugh at mailing lists--they're important to a lot of projects, including the Linux kernel. Also, Subversion support is still experimental on GitHub. If I had an older, mature project, based around these two things, I'd want to stay on Sourceforge. For example, there are probably lots of libraries dating back to the days when Sourceforge was the best of few choices, and that are nice and stable, get the job done, and require only maintainance. Why fix it if it isn't broke? Infrastructure doesn't have to be cool to be very, very useful.

Sourceforge also provides a means of distributing or completely elliminating download bandwidth needs. http://scipy.org/ is the Web site for important numeric Python stuff (scipy and numpy). But the download links point to Sourceforge. They also use GitHub; for some a mix of services is best.

So, yeah, I'd say Sourceforge is still important to a lot of people. Not all of whom are aware of it.

I mostly use python these days [snip] Matlab's syntax is just so slick by comparison:

Matlab: foo = [1 2;3 4] Python: foo= array([[1,2],[3,4]]) R: foo - matrix(c(1,2,3,4),2,2)

NumPy includes a matrix library: foo=mat('1 2;3 4'). In general, Python's syntax beats Matlab's syntax
hands down. (In this particular case there is almost a tie, but a trivial advantage for Matlab. I spend apporximately 0% of my time typing in data for array construction, and I suppose that is true of most users.) For help transitioning, see http://www.scipy.org/NumPy_for_Matlab_Users.

I strongly recommend Python.

The reason I like Python so much is that it has the least syntactic silliness of any language I've used: Python code often reads like psuedocode, but it actually works.

To learn C, you need to start by learning what a variable is, and that means learning what the different data types are, and when you use them. In Python, there really aren't variables: you just bind values to names.

And Python has lots of great libraries, so that he can easily write a non-toy program that does something interesting. In particular, there is the library, which would allow him to write a game.

And Python is useful for doing real work. It would be a poor choice to write an operating system or a word processing program, but it is useful for all sorts of actual problems in many fields. Particularly in science, Python is becoming a top language, thanks to SciPy.

Python is also the language used for SAGE, which he might enjoy using to plot graphs.

P.S. If he loves Python and wants to learn a second language, I would suggest C. Not C++, C.

steveha

Memory usage is not a problem for python. If you are using big data-structures, you would typically store them in multi-dimensional arrays defined in Numpy/Scipy. Internally, they use the same type of arrays as in C or Fortran, in all the usual data formats (ints, doubles, ...). There obviously will be a little bit of overhead space needed to store its data-type, array dimensions etc, but this is a fixed amount that is negligible for large arrays. Also the computation speed is for many problems not an issue. If you can formulate your problem as vector-algebra, all the computational intensive algorithms (matrix inversion, FFT, ...) use Fortran or LAPACK under the hood.

the python equivalent to PDL is NumPy

if a full stats package is a bit heavy, try python + http://www.scipy.org/
below is using the ipython shell

In [1]: import scipy

In [2]: x = [1,3,6,8,9,4,9,0,5,3,6,8,6,8]

In [3]: scipy.mean(x)
Out[3]: 5.4285714285714288

In [4]: scipy.std(x)
Out[4]: 2.7957693986829897

and if you need more than that you can really delve into its stats submodule http://www.scipy.org/doc/api_docs/SciPy.stats.html.

if a full stats package is a bit heavy, try python + http://www.scipy.org/
below is using the ipython shell

In [1]: import scipy

In [2]: x = [1,3,6,8,9,4,9,0,5,3,6,8,6,8]

In [3]: scipy.mean(x)
Out[3]: 5.4285714285714288

In [4]: scipy.std(x)
Out[4]: 2.7957693986829897

and if you need more than that you can really delve into its stats submodule http://www.scipy.org/doc/api_docs/SciPy.stats.html.

I would pay good money for a PowerShell implementation on Linux, and even more if Linux internals were exposed in the same way that WMI objects are on Windows.

And this is from a thirteen-year Linux veteran.

It's surprising that a thirteen-year Linux veteran wouldn't have discovered Python, which also is based on the idea that everything is an object, and has run on Windows and any kind of *nix for many years before PowerShell showed up. Its standard library has a huge amount of functionality built in and PSI - Python System Information looks like an easy way to get at system information. The enhanced interactive shell IPython has a lot of time saving features compared to the default one. Even on Windows, I'd rather use Python than PowerShell, since it has easy access to all the same COM and WMI objects that PowerShell does.

Use IPython, and not only do you get an interactive Python console with tab completion, code highlighting, simple introspection, logging, macros, profiling and more, but it also acts as a shell where you can do things like ls -l, and treat the output as a Python type that you can pipe to variables and functions if you so desire. Awesome software.

Well, as I said a paragraph earlier, I am a C/C++ programmer, hence the dinner table debates about recursion. Python is good for tools and for initialization routines though, you can waste days writing some complex logic that's only going to be run once in a compiled language, or you can solve it with a mess of nested list constructions and lambda expressions in half an hour. Much of the time what is generated is actually quite readable still, which is the glory of python. C may be my day job, but the things that I manage not to write in C are where I justify my salary.

PS: MPI4piIt strikes me that in general you should quit using python long before you turn to MPI, but it's there if you need it.

Numpy seems to support Python3

Just check Wikipedia, with a link to http://sourceforge.net/projects/numpy/files/NumPy/1.5.0/NOTES.txt/download

You can also find the announcement from last July : http://mail.scipy.org/pipermail/numpy-discussion/2010-July/051436.html

I have a basic grasp on most of the algorithm, but I am (go ahead, laugh) primarily a VB coder, and I believe this project would be better suited to C, or perhaps python (I have heard something about sci-py?)

I have no idea, nor do i care, whether your idea is any good or not. either way, you're right about VB being the wrong language for this. I don't think C or C++ is the right language either - as you say, you're not a great programmer so you'd be spending all your time frustrated with the manual memory management required by C.

python, however, is probably ideal. python's VERY easy to learn (you can pick up the basic syntax in a day or so - as with anything complex, real mastery can take years), the language enforces reasonably correct style, and the scipy and numpy libraries use compiled C subroutines for speed where it matters - in effect, you get the combined benefits of C for speed and a high-level scripting language for rapid development and prototyping.

you already have some grounding in programming (even if it is VB), so using the scipy and/or numpy libraries you should be able to implement your algorithm ideas in a matter of months if not weeks, even starting from scratch and learning python at the same time. i suspect (actually, i'm *certain*) that the greatest difficulty will be in un-learning bad habits and bad ways of thinking about programming that you learnt from VB - so try to start as a "blank slate", going through the tutorials and example exercises as if you know absolutely nothing about programming.

python can also be used interactively (e.g. with ipython), where you type your program (or parts thereof) into an interactive shell and get to run/experiment/tweak them in real time. you can also get help, function definitions, query reference material, and more from within ipython. this mode is also used in many of the numerous tutorials and examples available to help teach python.

(BTW, your work in ipython can be saved to disk and used as the basis for a standalone script, or just cut-and-pasted into your work-in-progress program)

here's a good place to start: http://scipy.org/Getting_Started

btw, although i've dabbled in it i am not a python programmer. the kind of programming i do (mostly text processing, data mangling and systems automation) suits perl far better than it does python. and python's white-space issue still annoys the hell out of me. but i can recognise that what you want to do is a near-perfect match for python's strengths.

Python is pretty much established as the leading open-source foundation for high-level scientific computing, competing head-on with tools like Matlab and IDL, either via the pure 'python stack' (Numpy, Scipy, Matplotlib, ipython - http://www.scipy.org/ and tools around them) or a project like Sage (http://sagemath.org).

I suggest you find a *topic* that interests you, that you're likely to work on for fun. If it's something that can benefit your research, even better. Then try to improve the specific package that covers that problem. Python is a much easier language to get into than C++, yet there are ways (with Cython and C/C++/Fortran) of getting performance when needed.

The range of topics where significant contributions can be made ranges from the very low-level, hard-core optimization work to high level user interface and visualization libraries. Special functions, ODE integrators, statistics, code generators, visualization, you name it, there's work to be done and welcoming communities in Python. If you'd like more specific pointers, drop an email to the Numpy discussion list as a starting point, indicating with a bit more precision what topics you find interesting intellectually. You'll find a welcoming reception and guidance on where to go from there, until you can find a project to focus your energy on.

You've just invalidated your own point.

I don't think you read the thread very carefully.

My statement as to both PS's strength and novelty is directed solely towards object piping. The presence of piping objects rather than a text text stream is pretty novel. Something like this is about as close as I've seen in any other environment. And I really like the idea. Do you want me to go into why? (That said, I suppose it's possible that if someone would go and write objbash and a corresponding set of utilities it would turn out that it's not all it's cracked up to be, but I have enough experience with programming in general, using REPLs, and using shells that I'm pretty confidant in my assessment of what a well-implemented object-based shell would do for you.)

That already justifies what I said in my previous post: the object piping already makes PS quite far from a ripoff of Bash. The rest of the stuff you say about the consistency of the utilities PS provides is really irrelevant to this point, no matter how true it is.

(For the record, the reasons I dislike PS involve: (1) subjectively very long startup times, (2) the fact that it's not installed by default and setup seems harder than it "should" be, and (3) the failure to fix several of the things I don't like about the native cmd.exe, at least without more investigation (e.g. the tab-completion behavior, behavior of the up and down arrow keys, and the continued use of the awful terminal that Windows provides). Add to that the ability to drop to Cygwin Bash or SSH to a real Linux box for a moment for things that are more complicated than 98% of what I do in the shell, and you don't have a particularly compelling use case for PS.)

Yes, I saw that and had similar thoughts. Also they should look at virtual environment and look at Interactive python

I challenge anybody to beat matlab (or maybe IDL) when you want to whip together an image processing algorithm and prove it out.

challenge accepted

Why learn one language for image processing algorithms when you still have to learn other languages to do everything else? Better have one language that does everything.

If you're a physicist and you need something MATLAB can't do, you're more likely using Python and Numpy.

http://www.scipy.org/

Says another Physics guy ...

Here is my advice: plan a slow-and-steady strategy, rather than a "space race" strategy. Plan for effectiveness over the long haul, rather than short-term results.

That means you will be doing things rather differently than Apollo.

For software, as far as I can tell, nothing exists that will meet your needs. Thus your first step is to figure out what free software has a hope of someday meeting your needs, then figure out how to get developers to work on it until it does meet your needs. So, actually, your very first step is to find an expert in rocket design who can tell you what features you need, what software exists that can do what you need (even if you don't want to use it because it is proprietary). If you are very very lucky, you might find a retired aerospace project manager who will give you advice for free. (I don't think this is far-fetched. Anyone who worked on rockets in the glory days will be old enough to be retired now, and you might find someone who shares your dream and will give advice for free.)

For simulations and engineering computations, you should look at SciPy. As I said above, it probably doesn't meet your needs now, but it has a solid foundation and lots of people working on it.

As far as a strategy for going to the moon, I don't claim to be an expert, but here is my advice.

You really, really do not want to try to re-create the Saturn V rocket. In fact, you don't want any design where you use up one rocket per moon trip. The slow-and-steady plan goes like this: First you get a "space pickup truck", some sort of launch vehicle that can reliably go to Earth orbit with a small payload (say, 1000 KG or so). Second, using many "space pickup" flights, you build a space station, and stock it with lots of oxygen, food, fuel, etc. Third, you build a "moon shuttle" in orbit, a vehicle that will never land on Earth and never land on the moon, but will safely travel between the fuel. Fourth, you build your "moon lander", which will be carried by the moon shuttle. Finally, you fuel up the moon shuttle and lander, and send a mission to the moon.

At that point, you have the infrastructure to visit the moon as often as you find convenient. You ferry up some more fuel, oxygen, and supplies, refuel the moon shuttle and lander, and off you go.

I'll point out that there are plenty of small companies trying to build a "space pickup truck" right now. You could sensibly just plan on hiring one of those, rather than trying to build your own launch vehicle. You won't get this project done tomorrow anyway, so you might as well start designing your space station and moon-specific hardware now, and just assume you can hire the orbital transport by the time you need it.

If someone gets a "space cannon" operational in time for you, so much the better. Use the cannon to send up lots of fuel and oxygen and such as cheaply as possible. In this case, you will want to build a "space tug" vehicle that can scoot around and collect the canisters shot up by the cannon.

The USA sent men to the moon using a cost-is-no-object, win-the-race strategy. You will do much better to incrementally build the infrastructure to go to the moon conveniently.

Good luck with your grand dream.

steveha

I didn't know Ruby had an interactive shell. Thanks.

In case you're curious, if a script is not provided to python it goes into an interactive mode, allowing for much the same thing. And if you want such python interaction on steriods, you can always checkout ipython, which is actually a complete command line shell replacement (as in replacement for bash, ksh, cmd, etc), in addition to its interactive python capabilities.

GFA Basic was what got me properly into programming, thank you ST Format magazine!

But now I'd also go with Python, it's the only language that I've used that just lets you get straight into the fun bits of coding and doesn't get in your way. Especially with IPython :)

There's a lot of alternatives in between pure python and pure C or fortran that can be speedy to write and run.

BTW, a very ill-advised design choice of Python: http://www.python.org/dev/peps/pep-0211/ [python.org] Ask any numerical analyst to know why it is a terrible idea to solve a linear system with inv(A)*b. But make sure you have at least half an hour free.

AFAIK PEP 211 and the related PEP 209 were never actually accepted into the language. Python users that want multidimensional arrays use the numpy package, along with the scipy package that builds on top of numpy.

For solving a linear system Ax=b, you just use

x = numpy.linalg.solve(A, b)

which calls LAPACK behind the scenes. For a simple matrix multiplication (or M-V or V-V depending on the dimensionality of the arrays) A*B=C you do

C = numpy.dot(A, B)

which is implemented with a call to BLAS.

BTW, a very ill-advised design choice of Python: http://www.python.org/dev/peps/pep-0211/ [python.org] Ask any numerical analyst to know why it is a terrible idea to solve a linear system with inv(A)*b. But make sure you have at least half an hour free.

AFAIK PEP 211 and the related PEP 209 were never actually accepted into the language. Python users that want multidimensional arrays use the numpy package, along with the scipy package that builds on top of numpy.

For solving a linear system Ax=b, you just use

x = numpy.linalg.solve(A, b)

which calls LAPACK behind the scenes. For a simple matrix multiplication (or M-V or V-V depending on the dimensionality of the arrays) A*B=C you do

C = numpy.dot(A, B)

which is implemented with a call to BLAS.

I guess it depends on the type of scientific computing you are doing. If you need a cluster to crunch numbers, don't use python. However, there are huge areas in scientific computing where: 1) speed isn't the primary concern or 2) languages like python are fast enough. Also, python has some pretty significant scientific computing tools like scipy (see http://www.scipy.org/), visualization using matplotlib (see http://matplotlib.sourceforge.net/ ), etc. I personally know a lot of people doing scientific computing and general research who use python.

If speed was the only concern, people wouldn't be using tools like Matlab, IDL, python, and the like. Obviously, a significant number of people doing scientific computing find these tools fast enough.

If you read the PEP you link to more carefully, you'll note that the context is that the octave developers were recommending adding an operator for multiplication to python, but ~against symbols for matrix inversion / linear system solving. Also, you can get away with doing inv(A)*b if A is well enough conditioned. There are better ways, but if you're just manipulating, say, a bunch of affine transformation matrices, an inversion or two won't hurt you.

Also, Python as a language has many advantages over MATLAB in addition to being free software. Basically, Python is a great general purpose language with number-crunching tacked on, and MATLAB is a great number-crunching language with everything else (i.e. GUI Programming, Object Oriented Programming) tacked on.

http://www.scipy.org/NumPy_for_Matlab_Users

... You're WAY behind the times.

I got a buddy who is an astrophysicist and worked at NASA, and he tells me his department ditched FORTRAN years ago in favor of Python+Numeric.

I hear you about the need for badass number crunching tools. It's your assumption that only FORTRAN fits that particular bill which is erroneous.

Not to say that FORTRAN doesn't have its use. It's just that other tools have since become better at some of those.

Python Numeric homepage. Check it out.

but for math geeks FORTRAN is probably the easiest language to get from pencil-n-paper to computer. Math functions in FORTRAN translate nicely from their paper counter parts.

Nope, the easiest one for that purpose would actually be MATLAB (or octave, for that matter). If you want decent O-O with that, you can also use PyLab. Anyone who needs FORTRAN for whatever reason down the road can easily pick it up having been exposed to any of the above, but it should not be taught as a first programming language.

Apparently, the reason should be more "because they don't know about them". Compiling some Fortran and an interface sounds a lot easier than rewriting stuff:

http://www.scipy.org/F2py

R handles non-matrix data structures much, much better than Matlab does

This advantage is even larger for Python. Use the NumPy package for efficient array handling and basic linear algebra. Use SciPy for optimization and statistics. Use Matplotlib for amazingly powerful 2d graphics. And if you occasionally need R, which does have an wonderfully deep statistical library, you can access it with rpy.

R handles non-matrix data structures much, much better than Matlab does

This advantage is even larger for Python. Use the NumPy package for efficient array handling and basic linear algebra. Use SciPy for optimization and statistics. Use Matplotlib for amazingly powerful 2d graphics. And if you occasionally need R, which does have an wonderfully deep statistical library, you can access it with rpy.

I'm an engineer, I do a lot of computational work. As a student I was able to get a $100 student copy of matlab. I tried out free software too mind you, but nothing out there compares(octave doesn't compare, nor does scilab).

I think you are stuck with the wrong language. For me, Python using the SciPy library has been a true Matlab killer. Why limit yourself to a language that's optimized for math when you have an alternative that's just as efficient for scientific, mathematic, and every other sort of program you could imagine?

And if you don't want to throw away your Matlab expertise, Matplotlib has a compatibility layer that present a programming interface in Python that's similar to many Matlab functions.

And all that is Free Software.

If you can point me to a set of tools for the language you propose is squeezing fortran out that makes it as easy to work with vectors and matrices as Fortran, I would be thankful, but I haven't found it yet

SciPy makes it *much easier* to work with vectors and matrices in Python than Fortran. Besides vectors, it has a wide variety of scientific fucntions. F2Py lets you call your existing Fortran routines from a Python script. It even lets you access global common data from the Fortran program in a Python script. F2Py also lets you call routines compiled in C from Python. A similar program that lets you access C routines from Python is Swig

Using vectors and matrices in Fortran only seems easy to you because you are used to it, but once you get used to doing it in a modern language you'll realize what a mess Fortran is. Both Frotran and C suffer from the problem of handling variable-sized arrays, you have to allocate the arrays. In Fortran-77 and older versions you don't even have the option of dynamical memory allocation, you had to declare the dimension of the arrays at compile time.

With Python you have an almost perfect development environment for scientific computation. You get the fast development cycle of Python, with the quick execution time of Fortran or C. All that with a huge set of libraries and utilities for every conceivable need. Do you have data that you get from an external website, for instance? Python lets you automate the download. Need to get data from a database? Format the output into a PDF file? Create pretty graphics for a presentation? Save data to excel spreadsheets? Etc, etc? Python has libraries for all that.

for python hackers, ipython is essential. it's a wrapper for the python shell which has tab completion of all class/function/method attributes and interactive access to documentation, plus all kinds of other fancy tools. it has made learning python so much easier.

I wish there was a much more modern shell, perhaps with a python-like syntax, or even just a cleaned up bash. I never remember all the places where you need to insert spaces after reserved words in bash.

Then look no further than ipython. http://ipython.scipy.org/

Python for scientific analysis,

Python is the solution I recommend for everyone who looks for tips on advanced Excel uses. Excel is OK if you just want some quick and dirty solution for a small problem, but if you have to go to the trouble of reading a book, Excel is clearly not the best solution.

For scientists and engineers who need something more than what Excel (and possibly Matlab) offers, I recommend starting with either A Byte of Python or Dive Into Python.

Also nice is the ability to inline C with Weave.

You can't figure out why

Try writing a large program that needs to do heavy number-crunching in Java/Ruby/Perl/Python, or whatever is your preferred language.

Python + Numpy is probably faster that anything most C++ programmer can write, since it uses libraries that can be optimized for the specific processor they run on, including using multiple processors/cores if available.

Hand-optimizing code that runs on modern processors is not a trivial task at all (very simple example: caches can have extremely big and very non-intuitive effects on the speed of code).

Try writing a trivial (5 or 6 lines of code) md5 implementation using only the Python standard library and compare its speed to the GNU md5sum program (written in C). Hint: read blocks with sizes of roughly 10 kB.

• split the three color channels
• create normalized histogram for each channel and concatenate them into one array
• for each other image histogram, calculate the inner product of the difference of the histograms. If its smaller than a given epsilon, mark the images as "similar".

This simple algorithm can be done with ~50 lines of python code using the Image and numpy modules. It's immune to rotation, scaling, and slight color manipulation or other small changes. Makes me wonder why they use pixel-by-pixel comparison.

1. Rush, a ruby shell. Rush strikes me as a very cool project. This slideshow is a good introduction.
2. iPython with the "sh" profile. About halfway into this article they discuss it.
3. Hotwire, an "object-oriented hypershell"