Slashdot Mirror
Tracking the IT Job Market with a Bot
atlantageek writes "Is the
IT job market improving? Is the growth in Unix or Windows? Should I
study Data Warehousing or E-Commerce? Identify the recent trends with
CJ Miner, a small tool I've written that has been monitoring the Computer Jobs website for the
last year."
"track how the computerjobs.com website has been doing"?
Ive written an advanced bot, plz hire me.
I second that, nice work. Lets see some statistics from this too. Like where jobs are going up and where they are going down. That kind of stuff.
-----BEGIN PGP SIGNATURE-----
12345
-----END PGP SIGNATURE-----
check out the business cycle!
1) How many of these ads are actually real?
2) What do these jobs mean in terms of disposable income?
A bot taking another IT related job! Where will it end?
"There is no time, sir, at which ties do not matter," Jeeves, (Jeeves and the Impending Doom)
But how many jobs have he gotten with his kiddie script monitoring one website? There's no alternative to updating your resume, prowling multiple websites for job listings, submitting your resume, and playing phone message tag until you land a job (or, more likely, a contract).
:P
As for my experience looking for a job recently in Silicon Valley, I would say things are getting better for contract work. From what my friends are telling me, companies don't seem to be hiring for the long-term. I even got an email from a Microsoft recruiter for a contract job.
Could you at least have chosen colors other than red and green for the first two choices!? I'm colorblind, you insensitive clod!
SIGSEGV caught, terminating
wait... not that kind of sig.
One question though: Why computerjobs.com? I'm not real familiar with their site, but are they one of the sites that claims to consolidate complete listings of I.T. jobs from a number of other large job search sites (Monster, CareerBuilder, HotJobs, BrainBuzz, etc. etc.)?
If they really do get a pretty good number of I.T. related listings all collected up in one place, then yes - I think this is a pretty useful little graph/tool.
I've been out of work since the beginning of May, and living in the St. Louis area, it seems to me that there are currently very slim pickings. I keep hearing talk of the economic recovery, but at least around here - I'm not really seeing it.
According to your chart, that would be an accurate accessment too - since it clearly shows a sharp decline in I.T. jobs available in St. Louis since April of 2005. (And worse yet, I'm really mainly interested in the hardware side of things, but if you look at that specifically - you see that in my city, there were only a grand total of about 2 jobs fitting that category, at any given time!) In the whole U.S., it looked like I.T. hardware jobs only averaged around 1,200 *total*, for that matter. Not good... not good at all!
You know if you want to check out how tech jobs are doing why not go here?
Philosophy.
>> Should I study Data Warehousing or E-Commerce
You should study what you enjoy to study and work on. If this means a less trendy job, so be it. I'd rather eat broken glass than do web development, for example. Even if it was the hottest job on the market, I wouldn't do it anyway.
For all the liberal bleading heart crap I read on this site, there are some mean people here. I thought the information was fairly interesting.
Read his resume, he's in Atlanta. I went to Georgia Tech and lived in Atlanta for a number of years, and this site was the first I would check when job hunting. It's fairly big there. They don't aggregrate other site's jobs, employers have to pay. With the exception of the head hunters, they were quality jobs. (But I've found that every head hunter posted job I've ever applied for has been a joke anyway)
Just because you have a job doesn't mean you are better than someone who apparently does not. I went over 6 months from my previous job to my current one, even turning down two jobs and getting jerked around by Novell for 2 months to find the right one.
Keeping myself busy with random projects was a great way to practice my skills and learn new ones.
...I would be pleased to get a line on just ONE "hot IT job". Write me a bot that will do that, and suddenly I'll care about your war3z. I live in Japan though. Interesting position. Or, lack thereof.
Don't trust anyone under thirty.
Timothy now has all power-ups and 30 continues, ICT market unchanged. Konami executives currently giving no comments.
And tomorrow the stock exchange will be the human race
I'd assume it means the employer will sponsor you for an H1-B if you take the job.
where the f is software on the skill list??
However, whenever a shortage of labor occurs in the IT market, the government consistently intervenes by importing H-1B workers to fix this shortage. As a result, the growth in wages is damaged. Working conditions (like working 60+ hours per week) do not improve.
Any perceived shortage in the market for IT labor is illusory. If this shortage were real, it would be short-lived, due to government intervention.
By the way, we see the same phenomenon in the market for unskilled labor: e.g. picking vegetables and fruits. The government fixes this shortage by allowing illegal aliens to flood this market for unskilled labor. As a result, wages (hovering around $5.00 per hour for fruit-picking in Southern California) never rise. Working conditions (like standing for more than 9 hours per day in the strawberry fields) never improve.
The rub is that politicians do not care about Washington's gross tampering in and bludgeoning of a (relatively) free market like the USA. Washington is eager to fix shortages of labor. However, Washington rarely fixes shortages of jobs by, for example, creating more government jobs. The interests of Washington are not aligned with the hopes and aspirations of middle America.
We should close the American market to (relatively) non-free markets like India, China, and Mexico. Further, the American market should be flung wide open to (relatively) free markets like Eastern/Western Europe, Canada, and Japan. Free trade is good -- only when we are trading with other societies that maintain (relatively) free markets.
I think these sudden changes are the result of agregating just one site. One big company comes in with a s*load of postings, or the same load expires.
It's ok as a first pass, the 'raw' data is only useful to a point.
Should I study Data Warehousing or E-Commerce?
You should be studying Computer Science...
Ever wonder what happened to all those mainframe or COBOL folks? Knowing about E-commerce, Unix, Windows, Java, XML, or whatever the technology or trend du jour is might be impressive now, but in a few years, come the next thing, where will you be then? These things change at the blink of an eye.
On the other hand, algorithms, computability theory, formal languages, predicate logic, etc. don't.
A solid foundation of the theory will enable you to understand and learn whatever specific language or technology you need for the job, and allow you to be nimble enough to quickly pick up and go with the latest trends as the market changes.
-- Samir Gupta, Ph. D. Head, New Technology Research Group, Nintendo Co. Ltd., Kyoto, Japan.
Should I study Data Warehousing or E-Commerce?
You should learn how to provide vertically integrated e-commerce solutions providing dynamic interaction to customers in synergistic markets.
Knowing how to work a sock puppet also helps.
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
Sixth item from the bottom of the list.
d etail/
check it out
http://www.thinkgeek.com/gadgets/electronic/732d/
every day http://en.wikipedia.org/wiki/Special:Random
"1) How many of these ads are actually real?"
Not many I'm afraid. Too many companies out there are "testing the waters" by posting false job postings and then tracking the number of resumes they receive.
The app's name, "CJ Miner", is strangely appropriate. In the adult web industry, CJ stands for "circle jerk", and refers to ad-laiden websites designed to fool visitors into believing that the site offers free content (pics and videos), when in fact nearly all links actually lead to other CJ sites.
If you're in the UK... http://www.jobstats.co.uk
The chart has an option for "Legacy Systems" which sounds way too general. I mean, isn't everything currently running in production legacy?
Speak truth to power.
Most jobs arent even posted... So while you may get an estimate, its no where accurate enough to stake your future on.
---- Booth was a patriot ----
Outsourcing? Recruiters using another site for entry-level job postings? Hell, it could be anything.
I would agree that it's good to have the theoretical background, but without the desire to learn new things, I'd say it's much less useful than following the trend du jour.
I recently read a blog post by a VC who took a different approach to predicting the future employment rates in the U.S. As a VC, he looked at VC investment in the past N months in the U.S., looked at how "early" those investments were, and basically concluded that since there was a lot of investing, these companies will be growing and hiring people. I don't know how big of a drop in the sea of employment this is, but I thought it was an interesting way of analyzing the situation.
Simpy
Surely, you have to be a US citizen before you can join the military?
Nope. Non-citizens may enlist in the U.S. armed services. Think about it: France has a foreign legion; why can't the USA?
He used to link to it in his /. sig but then, about half a year ago, he announced that he wanted to "move on with his life" and would sell his code, site and related goodwill.
I think part of the deal was that he was going to open the source but I never heard any more about it. Anyone know if that happened? Or of any FOSS projects working on graphing employment data?
I'm a CS major, and frankly, it's disgusting to see how many people can get a degree and not know how to program at all.
That being said, a language is nothing more than a way to describe a concept. Ask the "trend du jour" people about programming concepts, and you'll probably get a bunch of software engineering babble in the reply. (an experiment of this would be interesting, given that the person wasn't mislead)
My basic belief in learning computer science is to learn how NOT to be a code-monkey. Any idiot with minor interest in the topic of languages or databases can become a code-monkey.
I think Dijkstra was wrong about the cruelty of computer science. The true creulty is that we teach students more than the watered down industry will ever demand. Kind of like putting a professional athlete in a little-league team.
Why is this? One thought that has crossed my mind is something that a prof of mine, who used to work at IBM, once said "Back in the 60s programmers were created, not hired." Because there weren't many programmers at the time.
Now, if people are being trained on the company dime, the employers are going to cheap out. They are going to set a bar of "getting it done" and only demand as much. (we see this today in many parts of the industry)
Steve Jobs once said that "A players hire A players, B players hire C players", where the question was posed "then how do you get B players?" I think that in this case, C players hired people that would become (on average) at best B players.
Over time, these B+C players set the industry standards, both in hiring and development. (for example, if you are a boss and only know COBOL, are you going to start projects that aren't COBOL, with the loss of job security as one consequence, and that the employees currently only know COBOL?)
Which leads us to today and the demand for the "trend du jour" which is just an extension. Where programmers have been forced to ride the wave for decades to maintain bare employability. Thus, the market asks for it, people looking to do the bare minimum supply it, and a de facto standard in the common language is created.
What I'm saying is that if one chooses to enter this industry on the "trend du jour", they better be willing to have to learn the latest fads well into their fifties.
As for what the next trend is, I've heard that the best way to gauge that is to go into any CS department. What they are doing is what you will most likely be doing in 10 years.
Am I open minded towards open source, or closed minded towards closed source?
Or you can use indeed.com, which lets you search all jobs within the last 30 days from almost a thousand job sites (including computerjobs.com).
You don't even have to visit the site to check for new jobs -- it has RSS feeds and email alerts for new jobs that match your search criteria.
Or if you're really ambitious, use their free XML API and do whatever you want with the data.
Whoa! What's with the BSA ad I see associated with this article?
Is that it does not offer predictions on future jobs. The stock market has software logging the performance of thousands of different stocks over several year periods. However, that gives no insight to how the stock WILL perform.
This is the exact same situation, this software logs the jobs available, but gives no insight to future availablity. This type of software should only be used as a rough picture of how the sector is performing now and not how it will perform 3 months from now.
I would hate for somebody to spend a month learning e-commerece and find all of a sudden, the availability of the job is on a downward spiral.
And btw, "free market" is just an idealized abstraction - with the risk of burning some karma I'll say it's pretty crude one, actually.
The notion of "free market" was developed during the Enlightenment or "Age or Reason", when Europe started to attribute everything to science and reason (after the Middle Ages attributed everything to the will of God). As a consequence, it is based on the fact that players in the market make their decisions rationally, when, in fact, they very clearly do not. A free market also has failure points like monopolies, cartels, or the fact that it totally ignores environmental issues such as pollution.
The US is probably the closest country to a free market, however, it still does things like subsidizing agriculture products, and even steel a couple of years ago. As for Europe - they're actually much "worse" in not following the free market dogma.
As for your suggestions. 1. Saying "let's prevent people from getting H1s and work within the States" is clearly promoting isolationism and limitting a person's rights instead of more freedom. 2. yeah, let's just close China off ... . You're probably ignorant of the fact that the US has been 0wn3d by them. They're basically subsidizing America's unbelievably-high debt. We can't really upset them too much.
Finally, I'll say that I'm not against economic freedoms. What I'm against is using the "free market" dogma like this.
The Raven
Goddamnit, that's why the last two months of resume-filing has felt like pissing into the wind. At least I have my medium-yecchy helpdesking job, for now.
So how do I find people who are actually looking to hire?
--grendel drago
Laws do not persuade just because they threaten. --Seneca
They'll probably start by scoring all the first posts....
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
Apparently the graph + legend is a fixed size. I tried lining up all of the different Skills to see how they compared and the size of the legend squished the graph vertically, causing it to be unreadable. And I hadn't even started to add the Locations. So this graph is useless if you want to compare more than 10 skills/locations at once.
"There are more important things than stopping terrorism. Upholding the Constitution is one of them." - Ars Forumer.
My script would poll the site every hour or so and notify me, if anything not seen before appeared.
In Soviet Washington the swamp drains you.
SkillsMarket is apparently still in business, and Hilton is still trying to sell it.
There is another, similar site, whose URL I cannot remember right now.
A lawyer & digital forensics examiner. Also an expert on open source software (OSS).
If I had mod points, I'd mod you up. Interesting post.
I'm sorry, but basing the "health of the IT employment market" on the number of job ads is flawed.
Employment consultants are somewhere just below Real estate agents and car salesmen, where their job is not just making an individual sale(read commission), but portraying a positive and bouyant market, while continuing the industry's success.
By glutting the market with jobs ads that many would not stand up to audit, achieves several things:
- A percieved shortage means that commissions remain higher.
- Give the business community the belief that things are buoyant and its OK to pursue that IT BPR at last.
- Keeps the IT market turning over, rather than IT workers staying where they are because they are insecure, because they believe the market is flat.
In post Patriot Act America, the library books scan you.
They can take mah job, but they can't take away mah Bible!
It seems there are some minor issues with the scaling of the graph, for instance try viewing the So. California alone and then So. California and D.C. Metro; look at the max on So.Cal. in both cases, they are shifted by about 300 according to the scale. Maybe it needs some sort of calibration, I guess?
Now one thing you learn after college a lot more than you learn in college is exactly how to differentiate between jobs. The real world isn't defined as much by the type of programming you do as much as the scope of your responsibility.
Resume readers don't care if you're a Windows programmer, a UNIX programmer, a hardware designer, or a secretary. They want to see if you're a programmer, project lead, project manager, marketing manager, director, etc.
Things like Google, open source, wiki have leveled the playing field to where it doesn't matter if you study hardware, windows, AS/400, or UNIX. These things can all be learned by anyone at any time. In modern companies the skills at any given level of responsibility are being learned on demand as they're needed. Hardware designers one day are being used as UNIX programmers the next day.
Todays differentiation is in how much responsibility you're capable of having. Most resumes are being divided into management, sales and programming and as far as we can tell from the 36 checkboxes, management is the place to be.
Personally, this submission to Slashdot just looks like a cheap way of advertising a lame job board. If there was a real intent to track the IT market, all one would have to do is go to dice.com, and use the bots that you get for FREE there.
Dice.com is bigger, and nationwide. So you could really track how the IT market is doing over time.
Dice is also the only board which takes consultants seriously. That is, it offers specialized selections to make it easy to track consulting-only gigs, so that you don't have to wade through a bunch of full-time jobs just to find the ones which are for consultants.
Plus Dice doesn't charge you $50 to post your Resume, like this site does.
So, IMHO, this Slashdot submission is just a scam to get free advertising for yet-another-lame job board.
The best way to predict the future is to create it. - Peter Drucker.
And yet if my stats are correct (I work for one of the largest IT resellers in Europe), the Insurance Industry are the #1 early adopters for virtualisation software. I think something like 18 of our top 20 VMware customers are all Insurance or Financial Services companies ;-}
Ian W.
Goddammit MOD PARENT UP!!! It's been a while since I read something as true and as insightful on slashdot. And modders beware particularly of those who disagree with him in long posts; writing too much does *not* equal being insightful - I say this because I have just seen a long post of someone who disagreed with him modded insightful though on reading it I had the certain feeling the modder either was an idiot or did *not* bother read that reply and examine its content for vailidity.
And for anyone who thinks the US is a free market I'd say you're deluded! You need to read the book "protecting capitalism from capitalists" to understand the concept of "incumbency" and how big business and the rich in the US use their influence upon government to keep things in their favor and oppose the freedom of the market. http://www.savingcapitalism.com/
Jobstats.co.uk has been doing this for years, and aggregating counts of listings from multiple sites.
My Karma: ran over your Dogma
StrawberryFrog
While the government raised the number of H1B visas in response to the Internet bubble (belated as usual), the H1B numbers have since shrunk back to pre-1995 levels. All H1B visas for 2005 were exhausted in a single day last October. According to the parent, since there is a cap on H1Bs we should be seeing the raising wages and improved working conditions. I can't wait...
And you are obviously going to an Indian school because you didn't learn that software does not become popular through a good interface design, it becomes popular throough marketing its features through throwing as much bullshit into the market as possible. Like a link on the frontpage of /.
;-)
Now if you had gone to an American college you would have prepared an executive sales pitch and sold your vaporware to the first sucker who came along. If you can't make it work, hire more sales people.
I tried for 5 years to come up with a clever sig...only to realize that I am not clever.
Of course, and you would also have ended up designing web pages bloated with menus with more text in menus than in actual content, and with [+] that you can't click on.
Or you might have designed web pages with electric people.
And who knows, after a while you could have ended up a successfull web designer like this guy.
lucm, indeed.
Near the end of the financial year ?
There is a application from Broadlook that does this. http://www.broadlook.com/prod/pulse/
Here is a nice web site for posting jobs : http://e-jobs.com/