Slashdot Mirror


Finance, Scientific Users Get ActivePython Updates

jcasman sends along this clip from PCWorld: "ActiveState has added three open source mathematics libraries to its ActivePython Python distribution that might interest financial and scientific computing markets, the company announced Thursday. The packages are being added, in part, to anticipate the demand that may arise from new proposed rules for the US financial community brought about by the US Securities and Exchange Commission. ... In April, the government agency posted a set of proposed rules for handling asset-backed securities that called for financial firms to disclose, along with their prospectus filings, the source code of the programs that generated the filings, as rendered in Python. The government agency will be accepting input about the proposed rule until August 2. The three libraries that are being added to the ActivePython package are NumPy, SciPy, and matplotlib."

131 comments

  1. advanced financial modeling by countertrolling · · Score: 1

    IOW fudging the numbers.. only faster and easier

    --
    For justice, we must go to Don Corleone
    1. Re:advanced financial modeling by Anonymous Coward · · Score: 1, Funny

      only faster and easier

      Since when has python ever been faster at something? *ducks*

    2. Re:advanced financial modeling by Anonymous Coward · · Score: 0, Flamebait

      Well it's faster than Java. That's setting the bar pretty low though, I admit.

    3. Re:advanced financial modeling by Anonymous Coward · · Score: 1, Insightful

      Python fanboys wish it was faster than Java.

    4. Re:advanced financial modeling by Migala77 · · Score: 2, Insightful
      Actually the proposal is meant to make this harder. I haven't read the full proposal (667 pages!), but it looks like the Python program requested is to help investers do their own analysis of the risks/performance/... of asset backed securities.
      From http://www.sec.gov/rules/proposed/2010/33-9117.pdf (p 205-206)

      This proposed requirement is designed to make it easier for an investor to conduct a thorough investment analysis of the ABS offering at the time of its initial investment decision. In addition, an investor may monitor ongoing performance of purchased ABS by updating its investment analysis from time to time to reflect updated asset performance.338 In this way, market participants would be able to conduct their own evaluations of ABS and may be less dependent on the analysis of third parties such as credit rating agencies.
      The waterfall is a critical component of an ABS. Currently investors receive only a textual description of this information in the prospectus, which may make it difficult for them to perform a rigorous quantitative analysis of the ABS.339

    5. Re:advanced financial modeling by Anonymous Coward · · Score: 0

      They're just jealous that Java programmers havd bigger dicks.

    6. Re:advanced financial modeling by impaledsunset · · Score: 1

      What the hell is ActivePython and why the hell would anyone use it?

      I've been using Python, numpy, scipy and matplotlib, and I know of absolutely no issue on any platform. How does this "news" affect me (or anyone)?

    7. Re:advanced financial modeling by Nadaka · · Score: 1

      Seriously, the performance gap between a hotspot optimized java implementation and python is generally a difference of about 20 fold for crunching numbers. For python to equal java in performance, the startup time of the vm has to represent the bulk of the work to be done. And then performance does not really matter that much because the size of your dataset is small enough it won't matter much.

    8. Re:advanced financial modeling by Anonymous Coward · · Score: 1, Interesting

      I don't know how it's moderated Flamebait. As a matter of fact Python is faster at number crunching if you use the appriate libraries (scipy,numpy). These are highly optimized C libraries, and they don't treat numbers as objects. (And you don't use the builtin operators on them either.) And as a matter of fact when Python calls C libs, it has less an overhead than when Java does it. (However, by now Java has some similar scientific packages as well, but they aren't as high profile as numpy and scipy.)

      I can't find a benchmark to prove my point, but this is the common knowledge in most forums.

    9. Re:advanced financial modeling by Anonymous Coward · · Score: 0

      In fact the proposed libraries are implemented in C. In fact crunching numbers (matrix oriented processing) with Python and NumPy are much faster than with native Java implementation..

    10. Re:advanced financial modeling by Nadaka · · Score: 1

      Being written in c doesn't make it inherently faster than java. I've seen studies that have shown java running between 15% faster than c and less than 100% slower than c for specific algorithms.

      It could go either way. Someone would have to do an actual comparison to see how these libraries stack up.

    11. Re:advanced financial modeling by Bill_the_Engineer · · Score: 1

      I don't know how it's moderated Flamebait. As a matter of fact Python is faster at number crunching if you use the appriate libraries (scipy,numpy). These are highly optimized C libraries, and they don't treat numbers as objects.

      Let me preface my comment with if the tool is fast enough it should not matter that a different tool is faster...

      Python isn't faster than Java for number crunching - that's C. Python is just the glue. I could say the same thing for Java if JNI was being used.

      Pure Java beats pure Python.

      I use both and with calculation speed not being a factor, Python with NumPy, SciPy, and Matplotlib is hard to beat especially considering how quickly it is to set the calculation up in an interactive python session.

      However if the computation is complex or time consuming I use neither. There are packages (and languages) better suited to the task.

      --
      These comments are my own and do not necessarily reflect the views or opinions of my employer or colleagues...
    12. Re:advanced financial modeling by flink · · Score: 1

      What the hell is ActivePython and why the hell would anyone use it?

      Although now available for multiple platforms, originally ActivePerl/Python/Tcl's main attraction is that they were a commercial port of popular UNIX scripting languages to Windows with the added feature of being able to interact with ActiveX components. This made them attractive alternatives to VBScript.

    13. Re:advanced financial modeling by Anonymous Coward · · Score: 0

      That's like saying I have seen studies where floating point operations are faster than integer ones. Someone messed up the algos, because that is just not possible by construction. Remember one runs native, the other one not ....

    14. Re:advanced financial modeling by PBoyUK · · Score: 1

      It's not nearly as simple as native always being better than non-native. I'm no expert on it, but my understanding of it is that the most used parts of the code are compiled by the JVM anyway, so it is a native vs native comparison. There is then the overhead of this compilation and the JVM startup, as well as lesser used parts of the code that may not be converted, but in theory at least, since java is natively compiled at runtime, there are optimisations java can make that a C compiler simply cannot.

    15. Re:advanced financial modeling by Nadaka · · Score: 1

      You are wrong. I am not going to say that I know much about optimization and performance of Python using c libraries, I don't. I do know that interpreted python is many many times slower than java. I do however know a lot about high performance java. Most of the time, it is comparable to writing in c/c++, a few cases run faster, most run slightly slower. One specific instance where java dramatically outperforms c is in allocation of memory on the heap, java does this in ~60 instructions vs c++ doing it in ~300.

      The jvm does execute as native code, and the hotspot jvm can make optimizations at run time based on the current environment that will simply be unavailable to a c compiler.

    16. Re:advanced financial modeling by Joe+Tie. · · Score: 1

      If it's just number crunching, import psyco psyco.full(). Might not be the best JIT compilation in the world, but it should close the gap enough.

      --
      Everything will be taken away from you.
    17. Re:advanced financial modeling by Lil'wombat · · Score: 1

      ActivePython is just another CPython Distribution. Previously, the SciPy and NumPy libraries were modules were only found 'pre-compiled' in the Enthough Python Distribution. Now when I download ActivePython I have two fewer modules to hunt down. In the windows world, it is a pain to try and build scipy and numpy. It's not enough to have the correct version of C and C++, you need a Fortran compiler as well.

      --

      Truth: If it's not one thing, it's another

    18. Re:advanced financial modeling by SageMusings · · Score: 1

      I am a C# programmer and I assure you I am the biggest dick.

      We'll consider this matter settled, then.

      --
      -- Posted from my parent's basement
  2. Great! by parlancex · · Score: 0, Troll

    For about an average of half the lines of code they might use in C, scientists too can now get an average of 1/50th the performance of C with 10x the memory consumption. Hurray!

    http://shootout.alioth.debian.org/u64/benchmark.php?test=all&lang=python3&lang2=gcc

    1. Re:Great! by Anonymous Coward · · Score: 0

      If the problem on the stock market has been hyperfast automated trading, they should require all the trading bots to be in interpreted languages.

    2. Re:Great! by cybereal · · Score: 3, Insightful

      That link is unrelated to this post. The referenced scientific/numeric libraries for python are implemented in C as native modules. They are not just as fast as the equivalent code written in C, they *ARE* the equivalent code written in C, merely interfaced to with python. You might lose a meaningless tiny fraction of time preparing your vectors for processing in python, but you'll save several orders of magnitude more time not worrying so much about malloc corruption.

      --
      I read the script, and I think it would help my character's motivation if he was on fire. -Bender
    3. Re:Great! by Cyberax · · Score: 1

      Which might be great if you need to do a one-off calculation. If it takes you 5 minutes instead of 10 mins and runs 10 seconds instead of 1 second then you're saving 4 minutes 51 seconds :)

    4. Re:Great! by eldavojohn · · Score: 3, Interesting

      For about an average of half the lines of code they might use in C, scientists ...

      Scientists?! You're probably joking but I've been over to accounting and they're using Excel and *shudder* Access for all their heavy data lifting. Sometimes they need help and if they're kind enough I don't lie about how much I know about those ancient products. Most importantly they're not scientists. They're accountants and business people ... they don't care if they have to wait five minutes for Excel to open a worksheet containing the entire set of order histories of our company.

      I don't think these packages are intended for NASA space mission flight certified calculations. Just something to really help you out if you want to comply with the SEC. Side bonus, I'll bet that when you submit this code, you're going to achieve compliance a lot faster when the SEC only has to check half the lines of code and not analyze your memory management ...

      Software development is about trade-offs. Why aren't you complaining that it's not in some targeted processor specific machine language?

      --
      My work here is dung.
    5. Re:Great! by Anonymous Coward · · Score: 0

      I have been doing a lot of work with this sort of thing. A C code and python code implementation of the same thing. Python is a whole different beast than C. Simple 1 liners in python are not that in C unless you write something or use another library. Type casting is implicit in python in C it can change the meaning of the number. Some simple 'classes' can consume up to 300 bytes with just 1 var in it.

      If you treat them the same you will get into trouble. Especially if you care about CPU usage and memory. In the financial industry I know they care about sub nanosecond things. Python probably could do it. If you put it on a fast enough computer.

      Use them to their strengths and you will have good apps. Try to use C like python or Python like C and you will end up with apps that do not quite run as good. Leaving you with conclusions like python is better than c or c is better than python.

    6. Re:Great! by kidtexas · · Score: 3, Interesting

      I know bonafide scientists who use Excel for analysis. Scary.

      I also know quite a few who use numpy, scipy, matplotlib, and python for real science. In fact, I think the astro community specifically was an early supporter of some of these packages. I myself have been using them in science for 5 or 6 years.

    7. Re:Great! by Nyeerrmm · · Score: 4, Interesting

      Actually, NASA JPL does most of their mission design work using Python (combining compiled C modules of course, just like NumPy and SciPy).

    8. Re:Great! by Anonymous Coward · · Score: 0

      If the problem on the stock market has been hyperfast automated trading, they should require all the trading bots to be in interpreted languages.

      Too bad there aren't any. Well, I suppose shell scripts count as an interpreted language.

    9. Re:Great! by pavon · · Score: 2, Informative

      It is only irrelevant if all the computations you are performing are done by the libraries. I tried using python for data processing tasks, and it was unbearably slow despite the use of scipy. I think it was due in large part to poor I/O and bit-twiddling performance while reading/writing data files but I'm not sure. Anyway with the amount of time I spent optimizing code, I could have just written the damn thing in C to begin with. I just don't understand python's poor performance. All it's high level language features were implemented in LISP and ML decades ago with good performance, but python just can't seem to get it right

    10. Re:Great! by maxume · · Score: 1

      Python is actually more dynamic than most Lisp object systems...

      Did you use Python 3.0? The IO performance there was a big drop from 2.x and was largely fixed in 3.1.

      --
      Nerd rage is the funniest rage.
    11. Re:Great! by X0563511 · · Score: 2, Informative

      I don't think you understand. These things are used as any other python code. The magic is that C is the backend that actually crunches the numbers. You don't even have to know or care that C is used. All you know is you call the foo method in the bar module.

      --
      For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
    12. Re:Great! by Anonymous Coward · · Score: 0

      Design Work != Flight Certified

    13. Re:Great! by selven · · Score: 1

      For many applications, especially quick one-time tasks, coding takes a lot longer than execution. If you can cut the coding time in half from 60 minutes to 30 minutes, does it really matter if the execution time goes up from 5s to 100s?

    14. Re:Great! by MrCrassic · · Score: 1

      If you're script or program is not time-sensitive, then it's definitely better to optimize development and maintenance time. However, if you're in finance and are working with numbers that translate to real cash, then execution time is critically paramount.

    15. Re:Great! by WeatherGod · · Score: 2, Interesting

      You know, it is still possible to write bad code in Python... I use these packages on a daily basis, and while I wouldn't use them for production-level numerical weather prediction system, I do use it for data processing and such. There have been times when I wrote the code poorly and the code ran slowly, but as I learned Python and embraced the language (weaning myself away from C), I found myself writing faster programs. Which high-level language features do you think are lacking? I particularly like list comprehensions and the function argument syntax (positional and keyword-based).

    16. Re:Great! by ceoyoyo · · Score: 1

      Just use Cython.

      All the niceness of Python and the speed of C where you need it.

    17. Re:Great! by ottothecow · · Score: 1
      Did you even read the summary?

      This is not some sort of high-frequency trading or other section of finance where speed matters. This is producing SEC filings...As long as execution time is not in the month+ range, you are going to be fine.

      --
      Bottles.
    18. Re:Great! by xiong.chiamiov · · Score: 1

      You don't even have to know or care that C is used.

      Well, unless you're not using CPython.

  3. Great tools by AstroMatt · · Score: 2, Insightful

    These are great and free tools for making publication-quality plots as well as the analysis of the data.

  4. Does Not Work with Most IDEs! by SeriouslyNoClue · · Score: 5, Funny

    Okay so I'm the lead Microsoft certified developer at a Fortune 500 company. I have to get this crappy software integrated with our project that is running as a Win32 application by the end of the month and so I downloaded the packages and dragged and dropped them into Microsoft Visual Studio. Then I created a VB file that basically calls SciPy.getSECReport(somedata) and nothing happens. I get some "Error Method or data member not found" even though the stuff I downloaded was unzipped and dragged right into the rest of my libraries in my WIN32 directory.

    Oh and I also tried double clicking the packages and nothing happens ... as well as dragging and dropping the excel files onto the packages (I'm not an idiot, I've tried everything). I even went to SourceForge to find documentation on this crap and there is nothing. Nothing! Can someone help me figure this crap out? It's impossible to use, the open source nuts have made it so it isn't streamlined and integrated with Windows. I mean, I'm a pretty talented hacker (MSCD and everything) and this just goes to show how crappy open source can be.

    Will someone please rewrite this in Visual Basic so I can do my job?

    1. Re:Does Not Work with Most IDEs! by Anonymous Coward · · Score: 0

      Nice troll M$ loser.

    2. Re:Does Not Work with Most IDEs! by morgan_greywolf · · Score: 3, Informative

      You need to either write COM component in Python and interface with Visual Basic that way, or use IronPython and call it from .Net like this.

    3. Re:Does Not Work with Most IDEs! by thebjorn · · Score: 1

      Oh, come on. It was funny! (The username should be a "clue"...)

    4. Re:Does Not Work with Most IDEs! by X0563511 · · Score: 1

      The shitty end of the stick finally comes around and slaps em in the face. Bought into Microsoft? Sure it was painless. At first. It takes a good while to set in, but like a burn you feel the fucker nice and good.

      --
      For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
    5. Re:Does Not Work with Most IDEs! by Anonymous Coward · · Score: 1, Funny

      >> I'm a pretty talented hacker (MSCD and everything) and this just goes to show how crappy open source can be.
      You need to replace Visual with Vim, Microsoft with a Unix flavour, drag-and-drop with shell, unzipped with untarred or unbzipped2, WIN32 with BIN, Basic with Python or C before you will be recognised here as a hacker :D

    6. Re:Does Not Work with Most IDEs! by colinrichardday · · Score: 1

      unbzipped2

      Isn't that bunzip2?

    7. Re:Does Not Work with Most IDEs! by codepunk · · Score: 1

      Just goes to show the value of a MSCD now doesn't it. You may wish to start with a little trip through the Win32 API.

      --


      Got Code?
    8. Re:Does Not Work with Most IDEs! by newdsfornerds · · Score: 1

      I must have worked for five companies that had legacy apps running on Windows servers. Lots of companies seem to start that way (Microsoft everywhere) and within a few years they realize their mistake and switch to Solaris or Linux. Trouble is, they never bother to port their Windows-based apps to *nix and are stuck supporting gimpy Windows 2000 servers forever. Do business schools cover this scenario at all? It's so common it could be called universal.

      --
      Damping absorbs vibrations. Dampening is caused by moisture.
    9. Re:Does Not Work with Most IDEs! by Anonymous Coward · · Score: 0

      You zip buns? Whose?

    10. Re:Does Not Work with Most IDEs! by Anonymous Coward · · Score: 0

      It's good everyone got the irony here....

    11. Re:Does Not Work with Most IDEs! by ceoyoyo · · Score: 1

      The funny thing is, anyone who can type "easy_install" on the command line doesn't really need Activestate's distribution and has been happily using Numpy, Scipy and Matlotlib for years.

      In some ways, ActivePython is like the VB of the Python world.

    12. Re:Does Not Work with Most IDEs! by Anonymous Coward · · Score: 0

      Perhaps his burns will be comforted by the cooling breeze of a joke flying over his head.

    13. Re:Does Not Work with Most IDEs! by Anonymous Coward · · Score: 0

      If you're such a great developer .....duh! it's open source. Go fix it yourself & contribute the updates back to help those less talented than your good self.

    14. Re:Does Not Work with Most IDEs! by Anonymous Coward · · Score: 0

      Will someone please rewrite this in Visual Basic because I can't do my job?

      Howsabout this: I'll rewrite it in VB, print it out in 25pt font and SHOVE IT UP YOUR ASS. Either learn to use a civilized language or get out of the profession.

  5. Termination? by DoofusOfDeath · · Score: 0

    While we're on the topic, here's something that confused me about the SEC / Python idea.

    Python is Turing complete, which means some Python programs may never terminate*. Has the SEC taken this into consideration in its plans to use them?

    Or is the SEC planning to impose limitations such as, "These Python programs must complete within 1 hour when run on an Intel Pentiun IV 2.8 GHz with 4 GB RAM and Windows XP SP 3"?

    (* Of course, real computers have finite memories, so it's actually theoretically possible to detect looping on such a computer. But at this point we get back to specifying a particular memory size I think, which kind of goes to my question about the SEC specifying the particular hardware on which the program must run.)

    1. Re:Termination? by maxume · · Score: 1

      It seems fairly likely that the implementation of the regulation would require the models to be useful.

      But maybe not.

      --
      Nerd rage is the funniest rage.
    2. Re:Termination? by DoofusOfDeath · · Score: 2, Informative

      Oops - that's embarassing. I just realized that the Python program must be those used to produce the actual filings. So the programs' fitness to purpose must have been already established. Presumably they don't loop forever, or at least only do so after producing the filings.

    3. Re:Termination? by X0563511 · · Score: 2, Funny

      Or the loop was part of the design.

      I leave my machine on overnight and lots of things are looping "endlessly" - and that's not a problem.

      --
      For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
    4. Re:Termination? by Fujisawa+Sensei · · Score: 1

      While we're on the topic, here's something that confused me about the SEC / Python idea.

      Python is Turing complete, which means some Python programs may never terminate*. Has the SEC taken this into consideration in its plans to use them?

      Or is the SEC planning to impose limitations such as, "These Python programs must complete within 1 hour when run on an Intel Pentiun IV 2.8 GHz with 4 GB RAM and Windows XP SP 3"?

      (* Of course, real computers have finite memories, so it's actually theoretically possible to detect looping on such a computer. But at this point we get back to specifying a particular memory size I think, which kind of goes to my question about the SEC specifying the particular hardware on which the program must run.)

      Gag, I hate Python; it has to be the most annoying language to code in since COBOL. White-space blocked languages needed to die with the Hollerith card.

      --
      If someone is passing you on the right, you are an asshole for driving in the wrong lane.
    5. Re:Termination? by Jonner · · Score: 1

      If you think Python's syntax has anything to do with COBOL or Hollerith cards, I doubt you've ever even looked at any Python code.

  6. Use SML/NJ by Anonymous Coward · · Score: 0

    ( a.k.a. Standard ML of New Jersey )

      and use nlffi to a C library.

    Yours In Pittsburgh,
    Kilgore Trout

  7. Why not R? by Anonymous Coward · · Score: 0

    I am surprised that this question has even come up. The R language is beginning to take a a serious foothold in the statistical and financial community. Has a similar request been made of the this language? ...and as for speed..... It is very fast.

    1. Re:Why not R? by morgan_greywolf · · Score: 1

      Probably because Python is a more widespread skill than R. Python code is also extremely easy to read and understand to most average coders, even if they have little or no experience actually coding Python.

    2. Re:Why not R? by Daniel+Dvorkin · · Score: 4, Interesting

      Speaking as someone who uses both R and Python all the time, I'd say that while R is very very good at a lot of what it does, it's just not as good as a general-purpose language as Python. I find myself doing as much preprocessing in Python as possible, then saving the results in DB tables and having R finish the analysis. And yes, I know about RPy, but the programming overhead of representing data structures in both languages, and making sure that they're talking to each other correctly, can be considerable; so is the runtime overhead of passing really big data sets back and forth. (Note that it's been a while since I used RPy for anything big, and a lot has been improved in that time, so it may be time for me to give it another shot.) Python code is just cleaner and easier to write for most tasks. I like both languages for their strengths, but overall, if you can do a particular analysis in Python then that's usually the easier choice.

      --
      The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
    3. Re:Why not R? by Anonymous Coward · · Score: 0

      Well, I think Python in general is much more friendlier than R (the languuage and the tools as well). But I don't have much experience with R. How does it perform when you aren't using (and combining) the builtin methods, but are writing some number crunching routines of your own?

    4. Re:Why not R? by Daniel+Dvorkin · · Score: 1

      But I don't have much experience with R. How does it perform when you aren't using (and combining) the builtin methods, but are writing some number crunching routines of your own?

      It's not bad. If you try to write naive C-style code, you'll run into all the overhead of an interpreted language and it will run very slowly. But if you (a) try to do everything you possibly can as matrix operations and (b) make intelligent use of the functional programming features, it compares well to NumPy/SciPy. And as with Python, it's easy enough to write your own packages using C or Fortran and then call them from the main language (which is how NumPy/SciPy works, of course.)

      --
      The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
    5. Re:Why not R? by GlobalEcho · · Score: 3, Interesting

      Mod parent up!

      I also use both, quite heavily. Rpy2 is a real improvement over RPy, but I share your preference for allocating tasks to the two languages according to their strengths, and the rpy/rpy2 data structure representations are much trickier than I like.

      I find myself gravitating to R when the builtins are useful (robust estimation, smoothing kernels), where I need to split and analyze data subsets (using by() and its cousins) or where I want to plot things. Python is the ticket for interfacing with C, talking to the outside world, parsing, and expression of simple mathematical models.

    6. Re:Why not R? by Anonymous Coward · · Score: 0

      R is used for biostats quite a bit. FDA requires certain reports formatted in certain ways, and there are various R libraries that do this.

      As for finance, the quantlib library in CRAN and this is a high quality piece of OSS.

    7. Re:Why not R? by Anonymous Coward · · Score: 0

      Speaking as someone who uses both R and Python all the time, I'd say that while R is very very good at a lot of what it does, it's just not as good as a general-purpose language as Python..

      I don't know much about R but I know someone who uses R like a general purpose language. She fetch web pages and parse with it!

    8. Re:Why not R? by Anonymous Coward · · Score: 0

      Mod parent up!

      The biggest pain in using R and Py together is getting them to interact. My last effort was two years ago, and since then I have switched to the Excel-Matlab combination since that's what's used at my workplace. R is, however, making a comeback and I am wondering if I should try introducing Python; but not if it's as painful as it was 2 years ago.

    9. Re:Why not R? by highacnumber · · Score: 1

      You should check out sage (http://www.sagemath.org), its a python-based computational platform that includes R. Besides including rpy with a slightly improved interface, you can also run a sage worksheet in native R mode. Not many sage users are using this interface, so it would be great to have more feedback on it.

    10. Re:Why not R? by Daniel+Dvorkin · · Score: 1

      I do use Sage for some work, but I didn't know about the RPy interface or the ability to run worksheets in R. That sounds well worth checking out. Can you point me to the documentation?

      --
      The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
  8. Good stuff by stokessd · · Score: 3, Interesting

    Those packages are fantastic and really 90% of what I use in python are in those packages. I have been using enthought edition python rather than active-state (many reasons), and this tips the scales a bit more toward recommending active-state to others.

    FYI: Matplotlib makes 2D and 3D presentation quality plots of data (even an absurd quantity of data). Numpy and scipy provide scientific and matrix functions that pretty much cut matlab off at the knees unless you are a simulink user. Matlab is many thousands of dollars, python is free, and they are both remarkable similar, except matlab chokes on large data sets where python doesn't.

    Sheldon

    1. Re:Good stuff by Anonymous Coward · · Score: 0

      However, these free packages are available only for ActiveState's paying customers. Indeed an advertisement only.

    2. Re:Good stuff by Anonymous Coward · · Score: 0

      Better yet, get the real CPython (2.6.5), throw in IPython with GNU readline, NumPy, SciPy and matplotlib and you're good to go. Why pay money for someone to bundle it up for you (and poorly at that, most times)? If you're using Python there's a good chance installing software is well within your skillset.

    3. Re:Good stuff by iguana · · Score: 1

      What he said! I love numpy+scipy+matplotlib. Makes my life soooo much easier.

    4. Re:Good stuff by robi2106 · · Score: 1

      Good points.

      See Active State product description page and RTFA where the versions of Active state that will include this package are listed.

      "ActivePython now includes NumPy,SciPy, and matplotlib in ActivePython Business, Enterprise, and OEM Editions"
      http://www.activestate.com/blog/2010/06/saving-world-economic-collapse-python

      Product comparison:
      http://www.activestate.com/compare-editions

      IOW, these packages are available for the lowest price possible (the Biz edition) of $750 per server/yr).

    5. Re:Good stuff by 0100010001010011 · · Score: 2, Insightful

      I would be very interested in what matlab is choking on that Python wouldn't. The only data I've had matlab give me errors on is stuff that starts breaking 32-bit memory addressing. (In 32-bit Windows).

      I'm a hardcore matlab user and while there is no chance in hell my company would ever give it up for python, I'd consider it for some of my personal projects.

      And as limited as it is, there is one feature of the Matlab Editor that I don't think my lazy self could live without. Ctrl-A Ctrl-I, select all and auto indent. Everything is properly formatted to the right depth. php_beautifier is close for my php, but I still have to run it on the command line.

      Is there any python IDE with this built in... anything that mimics the 'desktop' of Matlab?

    6. Re:Good stuff by Anonymous Coward · · Score: 3, Informative

      I use those packages on Windows without ActiveState... they are free downloads from their respective websites.
      http://www.scipy.org/Download --- NumPy and SciPy
      http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-0.99.3/ --- MatPlotLib
      http://www.python.org/download/ --- Python

    7. Re:Good stuff by sourcerror · · Score: 2, Funny

      Auto-indenting for Python? Are you serious?

    8. Re:Good stuff by 0100010001010011 · · Score: 2, Insightful

      Yes, I am. I know absolutely nothing about Python. I earn my keep with Matlab. I know C (which is quite useful for CANape scripts) and use PHP in my free time.

      I just asked a simple question. Rather than answering, you respond with a jackass comment of "ha, are you serious?". Which now just makes me want to disregard Python and all this crap and I'll just stick with my Matlab and short PHP script for doing repetitive stuff.

      It's a little less forward than RTFM but just as condescending. And you wonder why complete novices throw their hands up at Linux and the such.

    9. Re:Good stuff by Paradigma11 · · Score: 1

      well, you could use Eclipse or Emacs or any other editor.

    10. Re:Good stuff by Follis · · Score: 2, Informative

      2 things 1) If random internet poster makes you not want to evaluate a tech you seriously need to grow a thicker skin 2) Auto indenting in python after the code is written is not particularly safe. In python Indentation serves the same function as { } in PHP or c, etc. That being said, Emacs works reasonably well. 3) You would have found this out if you had googled python indentation:)

    11. Re:Good stuff by mrcaseyj · · Score: 4, Informative

      Grandparent thought you were joking because one of the special features of python is that it doesn't use brackets for statement grouping and instead uses indenting. Thus all working python programs must be properly indented. I guess you could still complain if someone doesn't use the number of indent spaces you like for each block, but I assume your big issue is the confusion of various indenting styles rather than just the size of the indent.

    12. Re:Good stuff by Anonymous Coward · · Score: 0

      I think he thought you were joking. Indentation in Python is part of the syntax, it's a bit like asking for auto {} -ing in C. If you know C, Matlab and PHP, then Python+numpy+matplotlib should be simple to learn. And you won't regret it -- I get a headache when I try to work in matlab these days.

    13. Re:Good stuff by ceoyoyo · · Score: 1

      "Is there any python IDE with this built in... anything that mimics the 'desktop' of Matlab?"

      TextMate. If you're not on a Mac you have a choice of all the other Python editors, one of which must surely have those basic functions.

    14. Re:Good stuff by Anonymous Coward · · Score: 0

      Python is whitespace sensitive... it uses indent-level for nested blocks, so changing of indentation will change semantics:

      if some_condition:
              some_action1()
              some_action2()

      versus

      ifsome_condition:
            some_action1()
      some_action2()

      have different meanings. So generally one must set the correct indentation as a programmer and there is no way for any program to change it safely.

    15. Re:Good stuff by Anonymous Coward · · Score: 0

      Please, where can I download my Linux.exe?

    16. Re:Good stuff by mikiN · · Score: 2, Informative

      Is there any python IDE with this built in... anything that mimics the 'desktop' of Matlab?

      Why not have a look at Geany? It's small, free and sweet (i.e. very usable), and it assists with editing code in a lot of programming languages.
      By the way, the key sequence you gave as example nicely increases indent one level in Geany. For Python, Geany will auto-indent some obvious things: indent after a line ending with ':', dedent after 'return', etc.

      --
      The Hacker's Guide To The Kernel: Don't panic()!
    17. Re:Good stuff by Anonymous Coward · · Score: 0

      Repost, but relevant:

      Okay so I'm the lead Microsoft certified developer at a Fortune 500 company. I have to get this crappy software integrated with our project that is running as a Win32 application by the end of the month and so I downloaded the packages and dragged and dropped them into Microsoft Visual Studio. Then I created a VB file that basically calls SciPy.getSECReport(somedata) and nothing happens. I get some "Error Method or data member not found" even though the stuff I downloaded was unzipped and dragged right into the rest of my libraries in my WIN32 directory.

      Oh and I also tried double clicking the packages and nothing happens ... as well as dragging and dropping the excel files onto the packages (I'm not an idiot, I've tried everything). I even went to SourceForge to find documentation on this crap and there is nothing. Nothing! Can someone help me figure this crap out? It's impossible to use, the open source nuts have made it so it isn't streamlined and integrated with Windows. I mean, I'm a pretty talented hacker (MSCD and everything) and this just goes to show how crappy open source can be.

      Will someone please rewrite this in Visual Basic so I can do my job?

    18. Re:Good stuff by Anonymous Coward · · Score: 0

      Which now just makes me want to disregard Python and all this.

      Possibly the most childish sentiment I've ever witnessed on slashdot (the bad kind of childish, not the happy go-lucky, I'll-never-grow-up!, Peter Pan kind of childish.)

    19. Re:Good stuff by Anonymous Coward · · Score: 0

      You can try using spyder as a replacement for Matlab's desktop

  9. technically you can't really list a python program by Anonymous Coward · · Score: 0

    while you can type a python program into a computer, you can't unambiguously list it, because on paper you can't tell the difference between a space and a tab. The only way you can accurately read a python program is by using an editor that preserves tabs then counting the number of times your cursor moves as you use your arrow keys to drive the cursor over white space. Other more conventional programming languages don't suffer from this design defect.

  10. Is this an Active Python Advertisement. by strangeattraction · · Score: 1

    Slash should get paid for this. These packages have always been available. Where is the news?

  11. Cool job! by gr8_phk · · Score: 1

    Now people will be getting paid to obfuscate Python code.

    1. Re:Cool job! by drewhk · · Score: 1

      It does not really matter. You can still execute it and do a lot of useful analysis. Simply sweeping over the input space and plotting the output will help. If the output is really crazy (chaotic, whatever), then you can conclude that the asset is too complex and risky to buy.

  12. Not free, however by osvenskan · · Score: 5, Informative

    TFA and TFS fail to mention that SciPy, Numpy and Matplotlib have been added only to the Business, Enterprise, and OEM Editions of ActivePython. The Community Edition (the only one that's free) doesn't contain these libraries.

    http://www.activestate.com/activepython

    1. Re:Not free, however by chichilalescu · · Score: 1

      wow. these people pay to install a python distribution? check the umpteenth proof that the economy went bad because stupid people are in charge.
      seriously now... I use all these, and they're free, and in ubuntu I just had to click install to get them.

      --
      new sig
    2. Re:Not free, however by morgan_greywolf · · Score: 3, Informative

      And what, exactly, prevents you from installing SciPy, Numpy, and Matplotlib into an already existing ActivePython community edition installation?

      Hint: Nothing. You download the libraries and install with the canonical 'setup.py install'

    3. Re:Not free, however by Anonymous Coward · · Score: 0

      wow. these people pay to install a python distribution?

      Believe it not, businesses are willing to pay for support.

      We peasants can use apt-get or synaptic.

    4. Re:Not free, however by cyberthanasis12 · · Score: 3, Informative

      All the linux distributions I have used so far (SuSE, Mandriva, PCLInuxos, Ubuntu), all, have Numpy in their ready made packages. All you have to do is click to install it. To install SciPy, you download it, and run as installer. I don't know about matplotlib, but I suspect it as easy as SciPy.
      Thus, they are free.
      But if you want to fund python related staff, then fine.

    5. Re:Not free, however by Anonymous Coward · · Score: 0

      I am a user of Scipy, Numpy and Matplotlib for years, and I tell you it is not that simple.
      All three need non-trivial combinations of compilers and libraries to install, with minor regressions on each. Linux 32 bit is usually the easiest platform to install on. 64-bit linux will have you looking carefully for incomatibilities between 32 and 64 bit libraries but 64 bit users tend to expect that. Moreover, many distributions have (usually an older version) installable.
      Windows binaries usually work, though you don't want to use the wrong version of matplotlib with numpy, and this is usually not clearly documented either. OS X is the worst.

      Luckily, there are a few "Python distributions" that act as a sort of Matlab equivalent. Namely:

      Enthought Python Distribution http://www.enthought.com/products/epd.php
      Sage http://www.sagemath.org/
      PythonXY http://www.pythonxy.com/

      I tend to prefer Sage as I make use of its pure mathematics capabilities. YMMV.

  13. Time to abandon Matalb? by Kensai7 · · Score: 1

    Does this mean one less reason (scientific field) to use commercial Matlab and prefer free Sage/ActionPython/NumPy/SciPy/matplotlib?

    Just asking, but since Sage can offer so much functionality, I wonder if now the community gets one more extra boost.

    --
    "Sum Ergo Cogito"
    1. Re:Time to abandon Matalb? by Vireo · · Score: 1

      Does this mean one less reason (scientific field) to use commercial Matlab and prefer free Sage/ActionPython/NumPy/SciPy/matplotlib?

      I'm not sure about ActionPython -- why pay for a bundle of free components? But I for one am a long-time Matlab user, and I am slowly migrating towards Python (using numpy, scipy, matplotlib, ipython, etc.). Migration is slow mainly because I have to recode some of my own Matlab tools, but so far it's been succesful. Not painless mind you, as the differences in numpy and Matlab can cause really subtle bugs.

      So while it's great that ActivePython now gets numpy and matplotlib, I wouldn't say there is one less reason to use Matlab since those components were already available anyway...

    2. Re:Time to abandon Matalb? by Kensai7 · · Score: 1

      That's one reply I was looking for. Indeed I agree that the situation is as you describe it.

      Nevertheless, for the frugal scientist, Sage and there rest of Python toolkits seem like nice alternatives.

      --
      "Sum Ergo Cogito"
    3. Re:Time to abandon Matalb? by pavon · · Score: 1

      Or you can use Octave which is about the same performance as python + SciPy/NumPy/matplotlib, but has the added benefit of being largely compatible with Matlab.

    4. Re:Time to abandon Matalb? by Lemming+Mark · · Score: 1

      I've been using Python + SciPy / NumPy / matplotlib / IPython at work recently and it's awesome. I don't have a MATLAB license, so whereas some of my colleagues are using MATLAB, I've been doing my plotting and analysis work with just Open Source tools. I played with Octave (albeit the Cygwin port) and SciLab but in the end, the Python-based solution gave me access to a core programming language that's very neat and that I like, plus it's really fast and proficient at dealing with large datasets. It's not perfect but it's made life a lot simpler for me.

  14. Python is faster by Kludge · · Score: 1

    Because in the time that I save writing the program in Python rather than Java, I can run my program a million times. And even if I run it a million more times, computer time is nearly free, so it does not matter. And when it really does matter, in less time than it takes to write in Java, I can rewrite the single time consuming routine in C, and exceed Java's performance.

    1. Re:Python is faster by mwvdlee · · Score: 1

      If you can run the program a million times, it was trivial to begin with and you might have been faster just performing the task manually ;)

      --
      Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
  15. Re:technically you can't really list a python prog by thoromyr · · Score: 1

    What do you mean by "list"? You mean echo to the screen? Print it out? Something else? Are you trying to say that you can't see the indentation? Really? If you can't follow 4-space indents (the recommended amount) visually then you are unlikely to be able to read the text accurately either.

    It sounds like you prefer "line noise" languages that suffer from readability problems when abused in such fashion. More power to you. But you only sound confused and backwards when you can't even express your thoughts clearly. I recommend you try Python. Its rigorous requirements may help you order your thoughts as well as your programs into precise clarity.

    thoromyr

  16. Re:technically you can't really list a python prog by kpainter · · Score: 1

    you can't unambiguously list it, because on paper you can't tell the difference between a space and a tab.

    You do realize that you can use spaces instead of tabs? You just have to be consistent in the way you indent. Your comment is invalid.

  17. I tried that once by Anonymous Coward · · Score: 0

    but while I was programming an important app, I accidently hit the space bar just before tabbing. Since this error wasn't visible on printouts or screen views, supervision decided to switch to dot net on windows because python tarnished opensource's reputation with the company because of the idiot tab / space bug.

    But you can't blame them, I mean what sort of idiot language has whitespace signify blocks of code?

    1. Re:I tried that once by thebjorn · · Score: 2, Informative

      but while I was programming an important app, I accidently hit the space bar just before tabbing. Since this error wasn't visible on printouts or screen views, [...]

      What, the glaring "IndentationError" exception that gets thrown as soon as you import the file didn't tip you off?

      The situation you describe can never happen silently. I call bull$!7. Theoretically, it is possible to construct a situation where you would get a silent inconsistency, but this isn't it. I've programmed extensively in Python since '97 and never experienced problems due to indentation. In real life, this just isn't a problem.

      But you can't blame them, I mean what sort of idiot language has whitespace signify blocks of code?

      Well.... all of them do. How else would you find the blocks in a program? I know that many languages also use special tokens like { and } or "begin" and "end", but programmers still use the indentation to identify blocks even when it contradicts the tokens. The canonical example is something like:

      if (some_test())
      ....i = foo();
      ....bar(i);
      baz();

      Perfectly valid code, so no compiler errors etc., but most programmers will read it as a an if-statement with a true-block containing two statements... (I had to use dots to get indentation since pre-tags do not seem to be understood...?!)

    2. Re:I tried that once by xiong.chiamiov · · Score: 1

      I accidently hit the space bar just before tabbing. Since this error wasn't visible on printouts or screen views,

      Get a proper editor?

  18. Re:technically you can't really list a python prog by Anonymous Coward · · Score: 0

    You don't even have to indent consistently, Python interprets tabs strictly as 8 spaces, even if you mess up and end up mixing tabs and spaces in the same file, as long as you print tabs as 8 spaces in the output, the program can be "listed", echoed, printed, snail mailed (except on Saturday), scanned, OCR'd and run back again at another computer.

  19. the parent poster to your post is correct by Anonymous Coward · · Score: 0

    with other languages you occasionally can run into confusion with 0 and o, or 1, l or I, but with python you ADDITIONALLY have space versus tab problems. At a previous workplace, we coded a preprocessor that let us use brackets as a block delimiter to turn python into a more normal programming language, but it turned out that the language was so weak all around that we went with perl.

    I think the moral of the story is that if you think whitespace is a valid block syntactical element then you're probably not much of a computer software designer and as you can see in the following links, the author is actually pretty much a loon. http://www.python.org/~guido/pics.html

    Even worse is the GIL mess. AGain, guido has made a design that runs faster on a dual or multiple core system IF YOU TURN OFF ALL BUT ONE CORE. This has to be about the stupidest implementation possible.

    Moral of the story, if a language author is dogmatic enough to have whitespace block symbols, you can bet that he sucks pretty bad at computers, and as the GIL mess shows, Guido does not know how to program computers in an effective manner.

  20. didn't help by Anonymous Coward · · Score: 0

    we still had a stray space somewhere in a few thousand lines of a prototype that we never found, but once we encountered that mess we changed to perl because the assessment was that if the python guys couldn't get syntax right, they probably messed up lots of stuff and if you see the associated comment about the crazy GIL bug, the assessment turned out to be pretty correct.

  21. its true!!! by Anonymous Coward · · Score: 0

    if you want the true horror of python, check out this link: http://www.blip.tv/file/2232410

    Guido van Rossum is the WORST programming language designer in the world! The GIL effectively turns modern computers into windows 1.0 era cooperative multitasking dogs. World class suckage folks!

  22. For OS X users to get matplotlib & co. by Starker_Kull · · Score: 1

    Use Macports. Manual package management is something I try to avoid - 'sudo port install matplotlib py26-matplotlib' and all the dependencies and compiling are taken care of, not to mention the ability to cleanly uninstall if you wish. Macports recently upped to v1.9.1, which now tracks which ports you requested, so it's easy to prune away orphan libraries you no longer need.

    And matplotlib is a gem. It's got a ridiculuous number of plot-styles so it's remarkably flexible - if you are into GIS, look at matplotlib-basemap, which adds many map projections and the ability to plot geo data on those.

    I do wonder why people would pay for the 'special' (i.e. non-free, non-community) version of ActivePython. AFAIK, ActivePython neither develops the base libraries (matplotlib, SciPy and Numpy), nor python itself. What do they add other than as a bundling service?

    1. Re:For OS X users to get matplotlib & co. by ceoyoyo · · Score: 1

      Unless Macports supports Aqua now, don't do it.

      Rather:

      1. Install Python (available as a double click install from python.org).
      2. Install Matplotlib (available as a double click install form matplotlib.sourceforge.net
      2 (alternate). At the command line type easy_install matplotlib

  23. Does Not Work with MSFT IDE! by newdsfornerds · · Score: 1

    I think you meant to say, "Can someone at Microsoft please re-write VBasic to work with these modules?" Yeah, it can never be Microsoft's fault.

    --
    Damping absorbs vibrations. Dampening is caused by moisture.
  24. OWWWW MY BRAIN!!!! by Anonymous Coward · · Score: 0

    I had no idea python was that bad. While I didn't like the stupid tab space idiocy, I thought at least it was a relatively normal language besides that, but this is FUCKED UP! Who in their right mind would seriously consider python knowing that stuff????

    Van Rossum must be a fucking retard LOLOL

  25. lol by Anonymous Coward · · Score: 0

    at least "brainfuck" was sort of done in fun, apparently Rossum and his drooling fans think python is actually an ok system.

  26. My Math Prof used Excel 4.blah on Win3.1 for this by Saint+Stephen · · Score: 1

    I hate to essentially troll, and I hate to burst your bubbles, but these math packages aren't really doing anything all that wonderanomous. The guy I learned numerical analysis from in college used to use Excel to do a lot of his numerical techniques - and used to do a lot of them on a TI-80. Numerical analysis is all about knowing a lot to write an efficient algorithm to get the answer.

    I've done Q/R decomposition in VB6 (for a real honest to god client! for money!)

    I'm glad these tools are around for people to use but don't think they're all that new or revolutionary. Easy is easy, and there have been many generations of Easy for N years.

    Flavor of the month.

  27. Re:My Math Prof used Excel 4.blah on Win3.1 for th by Japher · · Score: 1

    Except that NumPy will use LAPACK and BLAS for it's linear algebra making it far more efficient. Try a QR decomposition on a matrix of any significant size in VB, then do the same decomposition using LAPACK and you'll see a huge difference. As for numerical analysis being about writing efficient algorithms, sure, that's true, but why would you want to rewrite those algorithms when highly optimized versions come by default?

    Disclaimer: Yes, I'm sure you could get VB to use LAPACK and BLAS but python will do it by default.

  28. Re:technically you can't really list a python prog by martin-boundary · · Score: 0
    Uh... Try indenting a document with spaces, or tabs, or any combination theoreof, if the document uses variable width fonts, for example.

    The OP is entirely correct. Python listings are brittle when you think about all the ways that they might be shared around: on HTML pages, on paper printouts, quoted inside a Word/OO document, quoted inside a LaTeX document, written in an email to a colleague, etc.

    A piece of Python source code is not much better than a binary blob, when it comes to listing/communicating it to others. Whatever the merits of the language itself, the indentation business is fundamentally dumb.

  29. Re:My Math Prof used Excel 4.blah on Win3.1 for th by Saint+Stephen · · Score: 1

    You're comparing apples and cadillacs. Excel doesn't use LAPACK either, but Prof used to dig it because of it's profound recursive capabilties. I didn't write the Vb6 code from scratch - it was originally written in Fortran using LAPACK there, and for some reason the guy wanted it in VB6 (I guess so he could enter the parameters from a database using a form.) It was a simulation of some complexity and ran well enough for his purposes. The same calculations were being done, on ordinary PCs.

    It's a distinction without a difference. No cool points for using lapack just to use it.

  30. activestate.com by reiisi · · Score: 1

    here

    Hmm.

    ActiveState has left a bad taste in my mouth in the past. My quick research just now may have dug up some reasons to re-evaluate them.

    --
    Computer memory is just fancy paper, CPUs just fancy pens with fancy erasers; the 'net is just a fancy backyard fence.
  31. GPL alternative by Anonymous Coward · · Score: 0

    if for some weird reason you can't (or don't want to) install GNU/Linux,

    just go for python (x,y)
    http://www.pythonxy.com/