Slashdot Mirror
You Can Now Profile Python Using Arm Forge (arm.com)
Python "is often described as being slow when it comes to performance... But is that truly the case?" writes Patrick Wohlschlegel, Arm's senior product manager for infrastructure and high-performance computing tools.
Slashdot reader igor.sfiligoi writes: Effectively profiling Python has always been a pain. Arm recently announced that their Arm Forge is now able to profile both Python and compiled code.
It's available for any hardware architecture, Wohlschlegel writes, adding that developers "typically assume that most of the execution time is spent in compiled, optimized C/C++ or Fortran libraries (e.g. NumPy) which are called from Python..."
"How confident are you that your application is not wasting your precious computing resources for the wrong reasons?"
Slashdot reader igor.sfiligoi writes: Effectively profiling Python has always been a pain. Arm recently announced that their Arm Forge is now able to profile both Python and compiled code.
It's available for any hardware architecture, Wohlschlegel writes, adding that developers "typically assume that most of the execution time is spent in compiled, optimized C/C++ or Fortran libraries (e.g. NumPy) which are called from Python..."
"How confident are you that your application is not wasting your precious computing resources for the wrong reasons?"
You're supposed to mark the sponsored content....
It's like getting a rusted out 73 Chevy detailed.
If you cared about performance you wouldn't have used Python to start with.
Python "is often described as being slow when it comes to performance... But is that truly the case?"
Yes, in fact yes, it is. That is not why we use Python.
"First they came for the slanderers and i said nothing."
If you want to write python code then you must study the spaces rules defined as "PEP 8" at https://www.python.org/dev/pep...
If you don't like the spaces rules then you can refuse to learn python and to start looking for another scripting languages.
If you were being paid to develop it, you'd probably go through and test every routine individually.
What good does this do?
where you are profiling your scripting language, chances are good that you did everything else wrong too.
(eg: using kiddie-scripts where performance was required)
Hello, "hearst communications, inc.", you are utter idiots.
Python programs that I wrote 15 years ago are still running in production.
The "Java rewrite" that my manager wanted to do never got done.
In fact, a lot of the production Java code that existed back then couldn't be maintained and got reimplemented in Python.
Since the new servers are ~20 times faster, speed never really mattered anyway.
Python is full of "free" optimizations that most newbys are not even aware of.
When you learn to do things that "Pythonic" way, it really does put the clunky Java hack-jobs to shame.
Here another take on it:
https://www.pythonforengineers...
Woot!!!!!ahundredandelevetyoneoneone!!
Tell me, why the fuck would I want to do this?
Is it that slow? pypy seems pretty quick to me. Do programs start up that frequently these days outside of util scripts? Even then you can follow xargs lead and do more per execution.
Why UNIX?
Python boy, your anti-Java example is full of it.
Java is famous for maintaining backward compatibility going back to pretty much the beginning of the language. Performance is continually improved in the libraries and VM as well. If you let the code languish for many years, that's on your company, but rest assured it will continue to run with possibly no changes needed at all for later Java versions.
If you want to share code with the Python community, it will not be graciously received unless you follow the spaces rule, but if you want to write for your own use, or the use of your group, then many variations are acceptable. I, personally, always begin lines with tabs for indentation. If I want to share the code I run it through a conversion routine. But I prefer a 3 space indentation size, except when the nesting starts getting to deep, at which point I shift to a 2 space indentation. This is a lot easier if the indentation is actually done with tabs. And tabs can easily be converted into a 4 space indentation if the code is going to be shared.
I think we've pushed this "anyone can grow up to be president" thing too far.
The difference between using lists and dictionaries is phenomenal. I have seen about 1 gig of files comprising about 20,000 of them read in, checksummed and compared in about one minute
You mean like these fine web properties - all far larger than any enterprisy Java bloatware, all written in Python:
Youtube
Instagram
Quora
Discuss
Much stuff at Google
And many more
He didn't suggest that the Java wasn't maintained due to backwards compatibility issues. He just said it wasn't maintained or was reimplemented.
If it was like many other places, it was simply because it's a bigger pain to read and maintain Java, generally speaking.
Upon entering the job market recently, I discovered that no one wants Perl programmers anymore, it's all Python.
After learning the differences in Python (and learning that I'd need to learn both v2 and v3), I started hunting for some of the tools that I use for Perl, like a profiler.
I couldn't find anything that could touch Devel::NYTProf. (Demo of that here) Hopefully this can??
He didn't claim it got slower or that it became incompatible, he said it couldn't be maintained.
... profiling is chickenshit and forging arms got Defense Distributed in a lot of twubba.
It little behooves the best of us to comment on the rest of us.
... and any language running in a non-jit virtual machine, Python is as fast as the library during library calls. It gets pretty slow when you code logic and calculations from elementary steps directly in Python. This is not new or surprising in any way.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I replace ALL my PowerShell scripts on my widows servers with Python ones. Why it performed 10 times better!
The Truth is a Virus!!!
And this is the recommended way to do this.
For me, except for the pure number transformations or high frequency sensor readings/processing, I haven't felt it worth it to replace a lot of python code for the minor shift in the overall performance. For most applications, Python is more than just good enough.
You didn't know how to use Python so you didn't use it. Good for you. Nuff said.
Statistical profiling lets you tune the overhead vs accuracy. By turning the overhead down low enough (e.g., by not sampling very frequently), you can be profiling your production code all the time and get very accurate data about what parts of your program are slow in real use cases.
Just write it in fucking C then: Jesus Christ. Stop reinventing wheels and get real work done.
You fucking C posers. Just use assembly. Stop reinventing wheels and get real work done.
Yes, it's slow. But, is it as slow as Java?
Clueless statement is clueless.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I have some symbolic simulation code that runs about 1000x faster for some real-world cases with the core in C instead of in Python. It really depends on what you are doing. Of course, if it had been running fast enough in Python, I would just have left it there, but sometimes you just cannot wait a week. And the Python version also had a much larger memory footprint. It was still hugely beneficial to have the pure Python version, as I found a lot of bugs by simply running both in parallel on randomized input and I keep maintaining both versions.
I do agree that quite often pure Python is quite enough. If you try that first, you may already be finished. If it is too slow or takes too much memory, you have the glue code and a nice base to find out which parts are actually slow and go to C there. It really is the best development model I have found so far. I also have done something similar in Lua. Works pretty much as well, although Lua is far better for embedding but comes with less native functionality.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
All glued together in Python calling out to compiled code for the hard work
FTFY.
I'd submit that in 99% of the Use Cases Python is applied against, performance of Python is "good enough" or even "much better than needed".
For those Use Cases where Python isn't fast enough:
1). The authors know that;
2). There are well-known performance enhancements within the Python fold;
3). You can make call-outs to more performant languages;
4). You can do a total re-write in [fast language of your choice].
But seriously, would a programmer write a large-scale numerical simulation in Python, knowing that Python is going to become a major problem? Perhaps the major problem? A bad (or ignorant) programmer would; a good programmer would not.
I'll assume you're replying to the AC grandparent....
Yep, sorry. I must have slipped.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.