Slashdot Mirror
IT's Next Hot Job: Hadoop Guru
gManZboy writes "JPMorgan Chase and other companies at this year's Hadoop World conference came begging for job applicants: They say they can't find enough IT pros with certain skills, including Hadoop MapReduce. That spells high pay. As for Hadoop's staying power as a career path (a la SQL 30 years ago), IBM, Microsoft and Oracle have all embraced Hadoop this year. Maybe the best news of all: 'Intelligent technologists will pick up Hadoop very quickly.'"
After all, every other framework of the month has lasted for 30 years, Hadoop will have at least as much staying power as Ruby on Rails!
If you want a strong userbase, projects with good, easy to use learning resources do better. When you hit the hadoop main page, they tell you what it is, but not what you need to know in order to use it. They don't tell you what languages it supports. They give no examples of usage. Essentially, they don't do you any favours.
The trick is going to be getting the appropriate experience without having learned it on the job already.
Yes, it can be done. However, this technology is geared towards environments with lots of nodes in big clusters. (which can run Linux) That's not the same as simply learning a language.
I got a job utilizing a "Big Data" database technology by being at the right place at the right time, when this technology was being rolled out. It's also hard to find people with that specialized experience.
So I would suggest to companies, hire people and train them. Just get quality people if you can't find someone with the specific skill set.
Except for ending slavery, the Nazis, communism, & securing American independence, war has never solved anything.
They should look into some resources able to make/redesign/re-architect their site to have ALL offered services available for more than a few hours at a time. From downtime of days, to on-and-off access through their mobile apps, to services unavailable at all hours of the day, via their "standard" web presence ... how much of this will Hadoop address?!?
== With enough Will Power, one could move mountains. With enough Brains, one would just leave them where they are ==
If I attend some public talk on a trendy subject its swarming with recruiters. Topics include no-sql, html5, mobile, etc. There seem to be at least ten job openings for everyone looking for something.
Java is one of the most inefficient languages ever? I take it you've never programmed in ruby, python, perl, etc. IIRC, Java benchmarks have shown it outpacing everything except for C/C++, FORTRAN and OCaml.
If I were a recruiter, I would automatically be wary of anyone who seriously refers to themselves as a "guru" of $language. Sure, you may be good at writing code and may know a particular library inside out, but anyone who calls themselves a guru probably has a very overinflated sense of their importance and actual skill level. These also tend to be the people who have the right buzzwords to get past HR filters and then proceed to bullshit their way through interviews.
"It is a denial of justice not to stretch out a helping hand to the fallen; that is the common right of humanity."
Java is one of the most inefficient languages ever? I take it you've never programmed in ruby, python, perl, etc. IIRC, Java benchmarks have shown it outpacing everything except for C/C++, FORTRAN and OCaml.
On first execution (and compile) it's slow. On first creation of an instance it is slow. After that Java makes up for itself rather nicely. If well implemented it's a great way to go, though I wouldn't chose it for my 3D rendering or reconciling a fiscal year's worth of journal entries, it's not that kind of language.
A feeling of having made the same mistake before: Deja Foobar
Quoting the article: "The company (JP Morgan) has been working with Hadoop for more than three years"
Then the article quotes the experts:
"The good news is that Hadoop experts aren't born, they're trained. "I'm sure companies that train their workforces on Hadoop will derive lots of benefits," said Jeremy Lizt, VP of engineering at Rapleaf, in a recent interview. A data provider that has been using Hadoop for nearly four years, Rapleaf was among the earliest adopters."
What a difference a few months makes...
Wow they must be super experts!!!
who prays for Satan? Who in 18 centuries has had the humanity to pray for the 1 sinner that needed it most? ~Mark Twain
The thing I always wonder about Hadoop is how important can it get? It's only useful if you have too much data for an RDBMS, right? It seems like only JPMorgan and other giant companies could make use of it. Am I wrong?
Democracy Now! - your daily, uncensored, corporate-free
...all us job-seekers who are already familar with several other languages and/or frameworks should read the Wikipedia page for Hadoop, bullshit our way past the HR person, then learn Hadoop on the job.
I take it you don't know what the phrase "one of X" means, because you have perfectly described what I mean by "Java is ONE OF THE most inefficient languages ever" by listing more of them.
Are people really this dumb these days?
Yes, yes they are.
All I hear is java this, python that. People care more about being able to throw shit on a wall and have it run than they do about performance, reliability, or functionality.
SQL is a query language, not a database implementation technology. In the future Hadoop-style engines will probably be wrapped by SQL such that it will be an implementation detail or choice, similar to the MyIsam versus InnoDB choice in MySql.
I'm not saying this will make it a non-career, only that the career will morph to be more like that of an Oracle tuning specialist (who make good money still).
Table-ized A.I.
This should be a no brainer. Hadoop is merely MapReduce plus the plumbing to care and feed for it. All the various nodes and TaskTrackers, it's not that complicated at all. You can learn the basics in a night and master it in a month.
and NOT just CS classes.
Take a tech school class load and add apprenticeships to it.
On first execution (and compile) it's slow. On first creation of an instance it is slow.
But it doesn't have to be slow ever! Microsoft .NET doesn't have most of those problems, despite being otherwise mostly identical. That's because Microsoft applied this fantastic new technology that apparently Sun has never heard of called a "cache".
This is why Java fell flat on its face in the desktop world, because Sun couldn't wrap their heads around that fact that every launch will be a "first instance" because having dozens of simultaneously running instances of a single process is very rare on desktops. Oh, and of course, on top of this, Java doesn't share code between processes, so the few situations where there are many instances of a process running (e.g.: Citrix XenApp or Terminal Servers) can enjoy two to three times the memory usage compared to native and .NET applications.
Java was originally developed for set-top boxes where there's only one process running that starts on boot. It never really grew up to embrace the PC world, and works on servers as well as it does only coincidentally.
though I wouldn't chose it for ... reconciling a fiscal year's worth of journal entries
You might be surprised how often java is used to do just that.
Dice search for C++ yields 17k hits. Dice search for Java yields 18k hits. Dice search for hadoop yields ~600 hits. Of the "direct company" ads (266), 18 from amazon. That's about 7% of all hadoop direct-company hits. Not a single one of them mentions an investment bank. I smell self-promotion.
Any guest worker system is indistinguishable from indentured servitude.
At its core, 'programming' is you 'telling' computers what to do.
Since you are doing 'IT support', 'SQL', etc. you are already 'programming'.
The real problem is the Babel effect of multiple, heterogenous computer 'languages'. So why limit yourself? Pick one (say Perl, or Powershell) and then depend upon CPAN or PowerShell libraries to do the heavy lifting for you.
After 20 years in the industry, in various forms, I've come to this realization: C++, Java, Hadoop, Ruby on Rails, PHP... all these things are the airgun and socket wrench and grinder and welder and all the other tools in the garage. What matters is if you have experience working on BMW's or Kenworths or IndyCars or Harley-Davidsons. In other words, have you written accounting systems, industrial control systems, customer-facing websites, etc. I don't want to work for someone who's going to hire me because I'm a C# guru. I want to work for someone who recognizes that my background in financial systems fits their need on a loan processing project. Ok, not really, because that would bore me to tears, but you get my point.
Jesus told him, "I am the way, the truth, and the life. No one can come to the Father except through me. - John 14:6 NLT
I think we need a system called Loup so arrogant idiots will call themselves Loup Gurus.
As the song goes:
Oooh ooh Loupgarou gonna get ya, betta run to the river or ya gonna be dead.
Does anybody actually have a hard time learning Hadoop? In my experience its pretty easy to pick up and go with.
"Hadoop is a top-level Apache project being built and used by a global community of contributors, written in the Java programming language."
No thanks, I will stay with my old friends v9fs, xget and xcpu =(
Looking for gurus seems like a needle-in-a-haystack proposition. Would it not be easier to take some of your current employees and train them on Hadoop? Assuming your employees are homo sapiens, they could be trained to deploy, develop applications with, and maintain Hadoop installations.