Python Developer Survey Shows Data Analysis More Popular Than Web Development (jetbrains.com)

← Back to Stories (view on slashdot.org)

Python Developer Survey Shows Data Analysis More Popular Than Web Development (jetbrains.com)

Posted by EditorDavid on Saturday February 9, 2019 @04:34AM from the great-first-languages dept.

Over 20,000 programmers from more than 150 different countries provided answers for the second annual Python Developers Survey (conducted by the Python Software Foundation and JeBrains).

An anonymous reader submitted this condensed version of their results: 84% of Python users in our survey use Python as their main language...up 5 percentage points from 79% in 2017. But half of all Python users in the survey also use JavaScript, and 47% more say they use HTML/CSS. Reported use of Bash/Shell has also grown from 36% in 2017 to 45% in 2018. [Later 93% of respondents said that their activities included Software testing/Writing automated tests.] Python users who report that they also use Go and SQL have both increased by 2 percentage points, while many other languages (including C/C++, Java, and C#) have decreased their share...

When asked "What do you use Python for?" data analysis has become more popular than Web development, growing from 50% in 2017 to 58% in 2018. Machine learning also grew by 7 percentage points. These types of development are experiencing faster growth than Web development, which has only increased by 2 percentage points when compared to the previous year...

Almost two-thirds of respondents selected Linux as their development environment OS. Most people are using free or open source databases such as PostgreSQL, MySQL, or SQLite... Twenty-something was the prevalent age range among our respondents, with almost a third being in their thirties. [31% more were between the ages of 30 and 39.]

42 comments

Min score:

Reason:

Sort:

This is fantastic by olsmeister · 2019-02-09 04:43 · Score: 2

and due in no small part to pandas. link

if used correctly (due to its NumPy foundation) with vectorised operations it rocks
1. Re:This is fantastic by Anonymous Coward · 2019-02-09 04:52 · Score: 0
  
  Just use a real language like R. Everything is a vector or more complex data structure unless you use it like a 10 key calculator.
2. Re:This is fantastic by olsmeister · 2019-02-09 05:05 · Score: 1
  
  I've had R on my radar screen for some time now, but somehow haven't had the time to learn it. Python for me was a little more.... generic. What I mean by that was that I could basically apply it to other tasks that were not data science centric. I'm sure R is equally capable, that's just the rep is has though.
3. Re:This is fantastic by Inzkeeper · 2019-02-09 05:07 · Score: 1
  
  Use both Python and R! Python has the rpy2 library that allows you to execute R code.
4. Re:This is fantastic by DidgetMaster · 2019-02-09 14:03 · Score: 1
  
  How easy is it to use Python and Pandas to do these kinds of analytics? I have been exploring the Chicago crime data set as an example (see: https://www.kaggle.com/boldy71...) and I am interested to know how much expertise and time does it take to do something like this. I am building a data analytics tool that will allow non-programmers the ability to do simple analysis of large data sets using a point-and-click interface. I use this crime data set to test things out, but I want to explore more in-depth analysis to see if it can help even more than it already does. A 4 minute video demonstrates our tool. https://www.youtube.com/watch?...
5. Re: This is fantastic by Anonymous Coward · 2019-02-09 14:31 · Score: 0
  
  But mostly a 1:1 correlation with what recruiters are asking for.
Re: The python in my pants is popular by Anonymous Coward · 2019-02-09 04:47 · Score: 0

*you're
This sight has really gone downhill...
Re: Hurry! Buy JavaCoin! by Anonymous Coward · 2019-02-09 04:50 · Score: 0

Creimette?
Re: The python in my pants is popular by Anonymous Coward · 2019-02-09 05:05 · Score: 0

i sniff youre dog butthoal,
A new generation of users by jma05 · 2019-02-09 05:22 · Score: 4, Interesting

Most of the new users since perhaps around 2012 came for the data analytics side.
IPython, NumPy, SciPy had been around for a while, but with maturing Jupyter, Pandas and TensorFlow/Keras, it really caught on. Other NLP and Machine Learning libraries probably helped too.
My use of Python today is completely different from how I used it earlier, nearly two decades ago, when it was mainly seen as a better Perl, back when Perl was THE scripting language. Now it is seen as a better MATLAB or a better R, even though the base language isn't itself vectorized as the others. The language and the standard library didn't improve much towards this. It was mainly the third party libraries that emerged and matured.
Speaking purely from a language standpoint, Julia has all right features for the analytics side, but the scientific community is right now with Python.
1. Re:A new generation of users by phantomfive · 2019-02-09 06:42 · Score: 2
  
  There's not really a reason to use Python on the web. It's fine for smaller websites (but then just use RoR) but when the website gets bigger, the lack of types will hurt a lot.
  
  --
  "First they came for the slanderers and i said nothing."
2. Re:A new generation of users by Nivag064 · 2019-02-09 08:29 · Score: 3, Interesting
  
  Most of the new users since perhaps around 2012 came for the data analytics side.
  IPython, NumPy, SciPy had been around for a while, but with maturing Jupyter, Pandas and TensorFlow/Keras, it really caught on. Other NLP and Machine Learning libraries probably helped too.
  My use of Python today is completely different from how I used it earlier, nearly two decades ago, when it was mainly seen as a better Perl, back when Perl was THE scripting language. Now it is seen as a better MATLAB or a better R, even though the base language isn't itself vectorized as the others. The language and the standard library didn't improve much towards this. It was mainly the third party libraries that emerged and matured.
  Speaking purely from a language standpoint, Julia has all right features for the analytics side, but the scientific community is right now with Python.
  SageMath is free and is easily accessible from Python. It runs on Linux, and other O/S's including those from Microsoft.
  SageMath is very powerful and is a good alternative to MatLab and Mathematica.
  http://www.sagemath.org/
  [...]
  SageMath is a free open-source mathematics software system licensed under the GPL. It builds on top of many existing open-source packages: NumPy, SciPy, matplotlib, Sympy, Maxima, GAP, FLINT, R and many more. Access their combined power through a common, Python-based language or directly via interfaces or wrappers.
  Mission: Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab.
  [...]
3. Re:A new generation of users by tomhath · 2019-02-09 08:46 · Score: 2
  
  There's not really a reason to use Python on the web.
  
  True. It's better than most (especially Java or C/C++), but not the best choice.
  
  It's fine for smaller websites (but then just use RoR)
  Good grief, RoR was never a good choice; thankfully it's long dead. Node is the only sensible choice today.
  
  but when the website gets bigger, the lack of types will hurt a lot.
  Python is strongly typed, you can look it up.
4. Re:A new generation of users by Anonymous Coward · 2019-02-09 08:57 · Score: 0
  
  There's not really a reason to use Python on the web. It's fine for smaller websites (but then just use RoR) but when the website gets bigger, the lack of types will hurt a lot.
  Yep, Instagram is really hurting /sarcasm
5. Re:A new generation of users by phantomfive · 2019-02-09 09:10 · Score: 3, Interesting
  
  Node is the only sensible choice today.
  
  For frontend, node is a reasonable choice. For backend, node has all the same problems as Python (a very lousy type system), plus backwards compatibility problems that are like butterflies on crack, plus security issues that don't really come up with other systems.
  
  Python is strongly typed, you can look it up.
  Python uses duck typing that is checked at runtime, not compile time. That is only strong in a very narrow, academic sense of the word.
  
  More to the point, it doesn't matter if you want to call it weak or strong. It will cause problems as your site grows and you need to refactor things. If you think ahead and build a strong unit test suite, then it will be less of a problem.
  
  --
  "First they came for the slanderers and i said nothing."
6. Re:A new generation of users by tdelaney · 2019-02-09 09:45 · Score: 1
  
  Incorrect. Every python object has a type. Different types can often be used interchangeably due to duck typing. What it doesn't have is a static type system - it has a strong, dynamic type system. Unit tests are important in any language, as syntactic checks do not pick up semantic or logic errors. And Python has several linters available that flag potential semantic errors.
  >>> type(1) <type 'int'> >>> type(int) <type 'type'> >>> type(type(int)) <type 'type'> >>> def func(): ... pass ... >>> type(func) <type 'function'>
  Yes, the type of the type type type is itself.
7. Re:A new generation of users by phantomfive · 2019-02-09 09:50 · Score: 1
  
  Python has an ad-hoc, messy, piece-of-crap type system, which you like for some reason. I don't know why, but I don't judge.
  
  What is clear is that the type system will cause you problems as your project gets larger a larger, a point which you have not refuted, but is by far the more important point.
  
  --
  "First they came for the slanderers and i said nothing."
8. Re:A new generation of users by Anonymous Coward · 2019-02-09 10:26 · Score: 0
  
  You don't need really need a strong type system.
  Python has replaced it with better unit testing support. Duck typing makes unit testing a bit easier than other languages to say the least.
9. Re:A new generation of users by Darinbob · 2019-02-09 12:38 · Score: 1
  
  It makes sense. Only a few programmers program for the web. But most programmers can make use of data analysis and scripting.
10. Re: A new generation of users by Anonymous Coward · 2019-02-09 13:19 · Score: 0
  
  I've used python recently for data analytics on a project. For me, I avoid web development where I can because as much as I love technology, web stacks and the library of the month are constantly changing for no justified reason other than 'its new.'
  Data analytics, on the other hand, does not really change that often (other than the recent growth of ANNs to DNNs, GANs, and that domain) and most the solid analytics I gravitate towards (the proven, non-blackbox techniques) will stand the test of time because they have statistical basis.
11. Re:A new generation of users by jma05 · 2019-02-09 18:49 · Score: 1
  
  I had hoped it would succeed more than it did. But somehow, it never quite caught on.
  SageMath did not do for Python what Anaconda did.
  Perhaps, it was (still is? - did not install in a while) the fact that it was distributed as a VM image for Windows.
  With Anaconda, it was easier to setup than plain Python; not even admin rights were needed. The package management was familiar. And it was just Python. SageMath was its own system and was intimidating for new users.
  It also targeted scientists (and I hardly knew any that switched from MATLAB to SageMath), which are a much smaller demographic than data analysts.
Re: The python in my pants is popular by Anonymous Coward · 2019-02-09 05:36 · Score: 0

sight
Seems obvious by Anonymous Coward · 2019-02-09 05:43 · Score: 1

This seems like it should be obvious. Python would be way way down the list of languages I would use for web development. Python is pretty good at handling text and numbers and I sometimes use the Python interpretor as an advanced command line calculator, so it makes sense for data analysis. But why would I want to use a language like Python for web development? That's like using a dead fish as a hammer.
1. Re:Seems obvious by Waffle+Iron · 2019-02-09 06:37 · Score: 1
  
  But why would I want to use a language like Python for web development? That's like using a dead fish as a hammer.
  Here's how things worked out: Every web browser in the world comes only with a putrid dead skunk built in, so you have to use the dead skunk as your hammer, even if you'd rather use a dead fish.
2. Re:Seems obvious by jythie · 2019-02-09 13:22 · Score: 1
  
  Yet, as a Python developer, at least in my area, I get a constant stream of recruiters trying to place me in web development work. They seem to get really confused when I try to explain that I am a python developer but don't do any web work.
Representative sample by Njovich · 2019-02-09 05:44 · Score: 1

93% of respondents said that their activities included Software testing/Writing automated tests
I guess that's different wording for that the group samples isn't representative for the population at large.
1. Re: Representative sample by Anonymous Coward · 2019-02-09 09:49 · Score: 0
  
  Doesn't it just mean that 7% of python developers don't test their code?
Re: The python in my pants is popular by Anonymous Coward · 2019-02-09 05:58 · Score: 0

s'aight
Python is a dumb choice for that! by Anonymous Coward · 2019-02-09 06:11 · Score: 0

If you want easy scripting for data mining then you simply can't beat LuaJIT. Some of my very heavy math based algorithms actually run faster in LuaJIT than compiled C code!
1. Re:Python is a dumb choice for that! by tomhath · 2019-02-09 08:50 · Score: 1
  
  Some of my very heavy math based algorithms actually run faster in LuaJIT than compiled C code!
  What you're saying is that most of your math based algorithms would run faster in Python (because its math libraries are compiled C).
2. Re:Python is a dumb choice for that! by Anonymous Coward · 2019-02-09 10:42 · Score: 0
  
  Read that again. It means the opposite of your claim. Compiled C is slower than LuaJIT, so Python is then also slower than LuaJIT.
3. Re:Python is a dumb choice for that! by Anonymous Coward · 2019-02-09 16:31 · Score: 0
  
  If your project runs faster in LuaJIT, then it'll probably run faster in Python using Numba. Same use case. JITs typically do translate things to machine code, though, so a properly optimized C function will run just as fast.
Re:The python in my pants is popular by Applehu+Akbar · 2019-02-09 06:15 · Score: 0

Nobody wants to see your Leptotyphlops carlae, buddy, not even the National Enquirer.
Re: Hurry! Buy JavaCoin! by Anonymous Coward · 2019-02-09 07:37 · Score: 0

Creimette is too busy for you. She is a genius thinking genius thoughts
By Neruos by Anonymous Coward · 2019-02-09 10:20 · Score: 0

because the market has shifted, big data and data lakes, led to data scientists and now data analysis are picking up the tail-end with python being the holy-grail of metrics. instead of doing actual, you know, 'better business', it's not market standard to fudge the metrics to make it look like you are doing 'better business'.
I don't know how many times a company has changes their analytics processes, reporting processes and the infrastructure/technology behind it, on the gospel of 'metrics and reporting' is more important then actual data gathering.
I did a contract where a company was spending about 150k for onpremise CRM + tableau reporting annually, to about 1million to move it all to SFDC + mulesoft, in the end, the ROI didn't justify it the move, but because it's new data, faster analytics, mega metrics as the selling point. Now the comp is cutting 10% of their work force just to pay for all the licensing.
but hey, I made bank, so i'm happy.
Asking developers of X language will always get X by Anonymous Coward · 2019-02-09 14:20 · Score: 0

So you do a survey at X (as in any) language related websites and you expected an answer different than X is the more popular?
Python is OK, but it is really just another scripting language (a fairly good one). But with any scripting language, it can be powerful when use for the right task, and a complete disaster when used for wrong reasons (like anything that requires near real-time performance).
Re: Asking developers of X language will always ge by Anonymous Coward · 2019-02-09 18:08 · Score: 0

Python & Numba gives you near realtime
This isn't a shock. by Shaitan · 2019-02-09 18:36 · Score: 1

The real story is that top US schools are giving rapid paths to Master's and MBA's with a strong focus on data science. These are shortcut degrees and their purpose is to allow those with diploma mill degrees from overseas to come on student visas. Just knock out your micromasters on the cheap online and it knocks out about 25-30% of the requirements for a masters or PhD.
Why the push for these programs? Because the adjustments to the H1B program emphasize people who hold high US degrees such as a Masters or PhD. Why do the universities go for it? First the programs do require some courses in probability, statistics, python, R, and depend on at least multi-variable calculus and Algebra. Second they get their highest rate of tuition on every student.
Of course they don't emphasize these programs here in the US even though you can use them. Instead they disparage online verified university courses that give certificates as not being the same as the same course taken online when registered as a student. That said you can knock out a quick degree from a foreign diploma mill as well but it will cost you more than it does overseas students, you'll have to pay extra to get it certified for acceptance in the US, and you'll have to explain why you got that foreign degree.
Because web dev in python is crummy? by fygment · 2019-02-10 01:36 · Score: 1

Seriously. Python looks and feels kinda like octave fortran and is a natural first stop when a person can't afford matlab or other commercial tech options. Needless to say your're talking about number crunching. For web dev esp. now more than ever, why wouldn't you use the language of the web ie. Javascript/nodejs?

--
"Consensus" in science is _always_ a political construct.