Slashdot Mirror
Perl Data Language 2.4.10 released
First time accepted submitter tonique writes "Perl Data Language (PDL) 2.4.10 has been released. Highlights of the new release are automatic multi-thread support, support for data structures larger than 2 GB and POSIX threads support. Also available is the first draft of the new PDL book. PDL is especially suitable for scientists. For those not in the know, 'PDL gives standard Perl the ability to compactly store and speedily manipulate the large N-dimensional data arrays which are the bread and butter of scientific computing.' Commercial languages used for the same purpose include MATLAB and IDL."
Oh, don't worry - people who've never spent my time coding Perl will be by to bash sigils any time now.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Fill in the missing word.
PDL: the computational power of Matlab, octave, IDL and NumPy with the __________ of Perl!
My first program:
Hell Segmentation fault
There's no need for comments as this is a rare article summary that actually tells you what you want to know. Bravo!
Author of Enyo: Up and Running from O'Reilly Media
Obviously the scientists are too busy to code PDL and have no time to comment...
"When all you have is a hammer, everything starts looking like a nail."
I've spent years coding the Perl; I'll be glad to bash the language as largely unsuitable for large scientific collaborations. The language wasn't really designed, but happened by urban sprawl and accretion over the years. The syntax is inconsistent and the code hard to read. Use something clean and designed well like Python.
Would someone using PDL in practice give examples of where it is used ? The scientific Python ecosystem has grown so well that I hadn't heard of PDL since a long time.
Say what you like about Perl, at least it doesn't shit itself if I accidentally configure my text editor to handles tabs the wrong way.
No Perl is. Real scientist use the Python programming language with Numpy and MatPlotLib :D
$action = empty(PHP) ? backToC() : unset(PHP) ; "when the concrete cases are understood, the abstractions are readily
Oh Perl, what CAN'T you do?!
- For the complete works of Shakespeare: cat
So true. Also R is great for vector-based data as well, and does some stuff quite a bit better (and some stuff quite a bit worse) than scipy/matplotlib.
funny, I've never known anyone to have that problem in the last 12 years of using Python. You must be one unlucky S.O.B. Sorry.
really... no other comments on this one? science is apparently ded.
Yes, because if there were still people interested in science, they'd obviously be all over a more-or-less homebrew project for which there are already well-established alternatives.
#DeleteChrome
eh. find replace "\t" with " " ? just a thought.
Obviously, I forgot to include a link to the the actual PDL site. Sorry about that.
I'm personally using PDL in the context of environmental noise measurements; I get long series of numbers and need to sum (and handle them in other ways) efficiently. Why, then, PDL and not numPy or something else? It stems from the fact that I had used Perl for scripting and text handling earlier. Also, I wasn't required to use something else. So laziness is a rather strong reason. Perhaps I was also a lost cause (that's perhaps a wrong phrase?) because I had started with Perl already.
I'm a firm believer in "use a tool suitable for the purpose", so I use R for statistical things. I shudder at all the things Excel, a prime example of a tool exploitable for multiple purposes, is used by my co-workers...
with the TMTOWTDI of Perl!
with the fluidity of Perl!
with the my's and sigils of Perl!
Then again, Perl is an acquired taste even for me.
Use something clean and designed well like Python.
Many scientific computing organizations can never get enough memory and that's a problem when coding large data structures in Python. Python has a nice design, but the runtime still needs improvement. Perl6 is in the same boat.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
eh. find replace "\t" with " " ? just a thought.
Works great until you inherit code from some moron that used tabs in some places and 8 spaces in others (including alternating between lines right next to each other), then it makes your life hell sorting it all out. Matters get further complicated when your editor has a different setting of tabs than someone else on the same project.
Using tabs for whitespace in code is the work of an angry little daemon, but writing a language that is dependent on whitespace (and accepts tabs) when "proper" white spacing has been a religious war for decades prior to the birth of said language is the work of pure evil that makes Satan himself cringe.
There's some new language syntax features. Here's what is looks like: %£T$%£$%"^%"
will tell you all the line numbers where someone's mixed tabs and spaces. Or use M-x whitespace-mode (or your editor's equivalent) and clean them up yourself in whatever consistent style you'd prefer.
Python's design has plenty of annoyances, but its whitespace-based syntax is the among least of them.
Ita erat quando hic adveni.
Personally I like HDF5. Compact, fast, can be read/written in parallel, bindings to many languages. A bit of a pain to learn at first, but well worth it.
The language wasn't really designed, but happened by urban sprawl and accretion over the years.
It's called "evolution". It works well for complex things.
I live ze unknown. I love ze unknown. I am ze unknown.
Memory usage is not a problem for python. If you are using big data-structures, you would typically store them in multi-dimensional arrays defined in Numpy/Scipy. Internally, they use the same type of arrays as in C or Fortran, in all the usual data formats (ints, doubles, ...). There obviously will be a little bit of overhead space needed to store its data-type, array dimensions etc, but this is a fixed amount that is negligible for large arrays. Also the computation speed is for many problems not an issue. If you can formulate your problem as vector-algebra, all the computational intensive algorithms (matrix inversion, FFT, ...) use Fortran or LAPACK under the hood.
karma police: arrest this man, he talks in maths; he buzzes like a fridge, he's like a detuned radio. [radiohead]
A good platform for scientific scripting is Python with PyCUDA. PyCUDA makes it easy to program the NVidia CUDA chips, which can speed up many computations significantly.
I was happy to be reading /. on my lunch break and see PDL mentioned. I use PDL and am glad to see it get some notice.
I am disappointment in the comments so far though. The anti-Perl froth is strong in this thread and I am not sure why?
Here is my point of view:
-Use whatever programming language suits you and your task the best. Maximize for total productivity which is a function of both of these things in varying degrees.
-Perl suits me best, personally, as a comfortable tool This is kind of squishy...it just feels right to me. MANY MANY people agree with me. But maybe you don't. meh.
-My tasks involve (a) parsing data from a variety of sources and (b) number crunching. Perl is already fantastic at (a). PDL makes Perl fantastic at (b).
-The people behind PDL use it for even more numerically complex tasks than I. Check out the docs and mailing lost archives and see. http://pdl.perl.org/?page=mailing-lists
-If you are already writing code in C, Python, Fortran or whatever else than you should stick with it. Moving over to PDL just because it exists doesn't make any sense, of course.
Now, as a Perl and PDL user could someone please explain to me the string visceral reaction shown by people in these comment threads whenever it is mentioned? Did Larry Wall challenge you to a bar fight once or something? (Probably not, I met him once and he doesn't seem like a bar fight kind of guy.)
Which is also why R for Python (RPy) also exists.
Yes there are ways* to deal with cleaning them up, the point is that you shouldn't have to spend time dorking around with it. In any structured (not whitespace dependent) language I can vent my spleen if I disagree with someone's formatting and then move on (cleaning as I go or adhering to their standard) without damaging any logic or misunderstanding it due to different tab stops.
* - The main issue with any solution that I'm aware of is that until you get into complex regexes that are more than "a few key strokes", is that none of them work in all cases (Gripp's example for instance would clobber tabs found in strings).
Honestly, if it wasn't for the whitespace dependence I would probably like Python. In fact I did enjoy (at some level) working with the multi-level abomination that is Jython. I want my compiler to smack me in the head when I do something blatantly stupid, but Python won't do that if I hit tab twice instead of once like I meant to end a block.
Python doesn't care whether you're using tabs or spaces, so long as you use a consistent amount of it to denote blocks. The problem is that some editors don't show you the difference. Nobody complains about semicolon requirements in C code because an editor shows you where all the semicolons are in a given piece of code. I feel whitespace should be just as easy to follow. If someone's indented a block wrong or mixed in some tabs, that should be just as easy to spot - even if you're using a language without any whitespace restrictions at all.
My solution is to use ethan-wspace which automatically highlights tabs in source code (along with lots of other little whitespace problems) and lets me clean them up trivially with the touch of a button and without breaking the file. It not only keeps my Python code functioning, but does wonders for keeping junk whitespaces changes out of source control.
Ita erat quando hic adveni.
For shortly it will bear the burden of millions upon millions of lines of FORTRAN written in Perl.
Stick Men
Memory usage is not a problem for python.
Ever! In no possible case, in no possible usage scenario, in the entire history of the galaxy, has memory usage ever been a problem for python.
Advice: on VPS providers
Python doesn't care whether you're using tabs or spaces, so long as you use a consistent amount of it to denote blocks.
The issue isn't if Python cares or not. The issue is what it does when multiple programmers have differing standards and work on the same file. The issue is more noticeable when you are taking over someone else's code, especially when the prior maintainer isn't available for initial guidance.
You still seem to fail to understand my point that you should not need a special editor to help you correctly understand what the code is doing. The code should read the same regardless of if I'm using the "super-duper IDE of choice", I've just cat'd the file at the command line, or looking at it in vi in a co-worker's shell that uses different tab stops.
And your semi-colon comparison with C doesn't work since the equivalent would be forgetting to hit return in Python code (which I would argue is equally as obvious (if not more so) as the compiler error for a missing semi-colon). A better comparison would be forgetting your closing } against an extra or missing "tab", but again the code fails to compile where Python will go ahead and execute it and the line that is shifted the wrong way gets executed when you don't expect it to.
White space is useful and it is important to help indicate what the writer was thinking, but it shouldn't be integral to the point that if the whitespace gets clobbered functionality changes.
| perl -e 'while () { s/\t/ /; print $_; }'
there. fixed it for ya.
er, s/()/(<>)/ on that last post
The code is doing exactly what the source code tells it to do. If you're having trouble reading what's in your source code - especially source code from someone else - there's plenty of tools available to make that job easier. If your editor made periods and commas look the same, it'd be hard to write C code and it'd be smart to switch. If your editor makes tabs and spaces look the same, it's hard to write Python code and you'd be smart to switch.
It doesn't require a "super-duper IDE". But by sticking to something sub-optimal, you're making it harder than it needs to be.
Ita erat quando hic adveni.
No Perl is. Real scientist use the Python programming language with Numpy and MatPlotLib :D
In my experience, scientists tend to use Excel VBA and incredibly convoluted batch files.
Disclaimer: I mostly met mad scientists.
lucm, indeed.
Using characters that are, by definition, not supposed to be displayed, is the single most stupid decision ever made in programming languages.
To a certain degree, it is inevitable that programmers have to make allowances for the limitation of the machine in the way they express themselves. But to create a new language that forces the programmer to be rigid in formatting is just plain stupid. Not even BASIC was ever that bad. FORTRAN abandoned that notion in the 80's. Anyone who actually likes the straight-jacket that is Python should be fitted for a physical jacket.
I've done a lot of scientific coding in both Perl and Python, and I would pick Python any day of the week. IMO whitespace issues are not a legitimate complaint about Python. I think whitespace delimited blocks are easier to read, and if you have problems with coders not getting it (never ran into this myself), a simple lint script that runs before commit/checkin will solve your tabs vs spaces mismatch. Most Python-based scientific code uses numpy and/or C or Fortan for the computational bottlenecks, so memory and performance are often less of an issue than people assume. Much scientific code is designed to run on cluster/grid/cloud instead of a shared memory system, so the lack of concurrent threads is also often less of a concern.
So, essentially it's only a problem if you are a human being, or constantly interact with human beings.
At least in my field, putting out a figure that is obviously from Excel is a huge faux paus. Doesn't mean a lot of people don't do it, though. Especially the respected but crusty oldsters who can get away with it.
If you don't understand any of my sayings, come to me in private and I shall take you in my German mouth.
I've spent years coding the Perl; I'll be glad to bash the language as largely unsuitable for large scientific collaborations. The language wasn't really designed, but happened by urban sprawl and accretion over the years. The syntax is inconsistent and the code hard to read. Use something clean and designed well like Python.
Hahahahahahahaha.
Funny man.
funny, I've never known anyone to have that problem in the last 12 years of using Python. You must be one unlucky S.O.B. Sorry.
Funy, I've met tons of people who had that problem.
You must be one lucky S.O.B.
Someone please, PLEASE mod this up. It so deserves better than 0.
Slashdot - News for Nerds, Stuff that Matters, in ISO-8859-1 Has just realised that beta makes this signature redundant