No Shortage Of Programmers?

Robber Baron writes "While searching with Copernic for the old Indian-head test pattern, I chanced upon this article (funny how search engines work, isn't it?). It seems (surprise, suprise) that this whole IT labour shortage crisis was a myth generated by large IT companies to justify importing boatloads of foreign IT workers willing to work for low wages in substandard conditions. Anyone have any experience with this?"

Comments

I respectfully disagree.

[Outlyer]

As someone who spends a lot of time hiring (yes, I'm a manager) I really have to disagree. The problem isn't finding "programmers" i.e. people who write classes all day, and regurgitate the algorithms they memorized in college, but finding what I like to call "Developers;" that is, people who think, who see a big picture. My staff is compromised entirely of thinkers, and it's a small team. We have a hire rate of around 25% of interviewees, and that's generous.

Nothing against polytechnical schools, but universities seem to produce "thinkers" not just "doers" It's all well and good to be able to write code when you're given explicit direction, pseudo-code, etc, but a real "Developer" just needs a "goal" There is definitely a shortage of highly motivated, problem solvers, not a shortage of code monkeys.

The only reason there even ARE H1-B visas is ...

[crovira]

that most American managers are not competent enough to manage a geographically widely dispersed work force.

1) The communication infrastructure is available, (this I know from using it.)
2) The server-side processing power is available. (this I know from using it.)
3) The client-side processing power is available (this I know from using it.)
4) The foreign workers are available, out there, (this I know personally.)
5) Fund transfers can be made account-to-account anywhere on the planet, (this I know, I work in banking systems.)
6) Local and federal taxes are taken care of by subcontractors. The only responsibility of the contractor is to report to the local and federal governments the gross amount paid to the subcontractor, (this I know from working in payroll systems.)
7) Most governments would love to have sources of hard-currency apart from material goods exports. Leveraging of their services instead of just their goods would constitute a very attractive foreign revenue stream, (this I know from having worked on the GST system in Canada and from living there.)

But most managers here in the 'States and elsewhere aren't able to leverage their resources or to communicate effectively enough to make use of what's available.

Hands-on managers are hands-on because they skate on the thin ice of chaos. Micro-managers are hands-on managers who don't even know what they're doing or have sufficient reporting channels to know and trust that their resources are doing what they're supposed to.

That's the real reason that the US has H1-B and other types of visas. The shortage is caused by the business schools who don't equip anyone to deal effectively with anything other than in face-to-face.

Apart from large multi-nationals (of which there are only a few thousand,) most companies are being directed by people without effective communication skills. This costs them locally in wasted effort and acts as a limit to growth or even to serving their clients effectively and efficiently while maximizing the profits that they can derive from their revenue.

The real management lesson of the Linux operating system is not the OS, but the communication and control system that created it from a group of people dispersed world-wide.

Very, very true.

[Parity]

There are lots of people contradicting this article, but I have to say, that I'm not a manager, but I still get involved in the interview process, to assess people's technical skills. I give them a little quiz, usually, a coding test of some kind. There are several ways to fail this test:

Say you're rusty in language X; then, not know what pseudo-code is; then not be able to write the algorithm in pseudo code; then not be able to write an even something as simple as "a loop to print the numbers from one to ten."

Say you're not prepared, and ask to do the test as a take home or to come back later. This is not a graduate exam, this is something any 1st year compsci undergraduate should be able to do, and if you can't do it in C (or whatever) I'll let you do it in pseudo-code... you shouldn't need to take it home where I can't see that -you- wrote it!

Write down something really scrambled that does utterly the wrong thing, and blame errors on being more familiar with language Y than with language X. No, sorry, doesn't wash. Syntax errors, yes, fine, whatever, I don't care. Logic errors, no. An algorithm is an algorithm, and none of C/C++/Java/Pascal/or even BASIC are far enough apart to make this a viable excuse. 'If' is 'If' and 'For' is 'For' and either you can think a problem through into code or you can't.

And that is why we hired the fourth person, who when asked to write an addelement() and removeelement() for a FIFO Queue (implementation details not specified... you could use an array of ten ints and error on overflow and that'd be fine) ... she wrote down a linked-list queue straight from a data-structures course, with a couple of small errors but the logic was mostly right. So, hey, if we have to wait for an H1B Visa, we're going to wait for it, because while the point of on-the-job training is well taken (we expect to do that; we wanted ability to program and understanding of networking, things -any- B.S. in compsci -ought- to know, plus a little RL coding experience). Sorry, but no, we are -not- going to teach people on the job the basics of basic coding, and given some of the people I've seen come through (and be handed to me by managers with glowing words only for me to find out they don't even know what code should look like, much less how to do it.) And -that- is my little story and why I believe the above person is not distorting facts. (Also, on that retraining... the above poster hired a -contractor-. You don't train contractors, you train full-timers.)

Parity None

--Parity

Problems with the report

[bungo]

I read through the article, and to me, there were many statements which contradicted
themselves, but, aren't that obvious as they are well spread out, and the article
is long, with alot of points.

It appears that the article is a politically biased article trying it's best to
appear to be objective.

For exapmle, towards the end of the FAQ section, we come across -

"Question: The industry lobbyists claim that the H-1Bs tend to be
``the best and the brightest'' from around the world. Is this true?
Certainly not. We should definitely facilitate the immigration
of the outstanding talents throughout the world, but only a small
proportion of the H-1Bs fall into this category. 75% of the H-1Bs
earn less than $65,000, far below the salaries of top talent in
this field, which exceed $100,000. "

This is trying to imply that the average H-1B is not the top talent, and is
most likely only average or below. This may, not may not be true, I am
not discussing the merit of this point, but I want to focus just on what
the report is saying.

But, those with not so short memories will remember right at the top of the
FAQ section, Matloff said -

"The industry lobbyists form a lone voice on this issue. There is
a broad consensus that the H-1Bs are indeed exploited in terms of
wages and working conditions."

and
"... study at UCLA, which found that the immigrant engineers
were paid 33% less than comparable Americans "

and
"Thus it is indisputable, from basic economic principles, that on
average they are making less money than they would if they had their
freedom."

So, Matloff, along with all of the studies mentioned, say that these
workers are getting far less than they should due to the exploitation of
restrictions in the system.

Hany on, how can you say that they are getting far less than they are
worth, and then later on, say that
"far below the salaries of top talent in this field" ???

Let me state this one more time. Matloff says -
- they are underpaid for their skills
- if we use at their salaries as an
indicator, they are not very good.

This is a direct contradiction. You can't have it both ways. Either
they are underpaid and worth more, or they are paid what they are
worth and are not the top in their field.

I found many other instances like this through out the report. This
guy has a point he wants to prove so much, that he even switches
sides in his arguments.

I wonder if he even realises what he has done?

If you are going to respond to my post, please note, I am not taking
sides on this debate. I am just trying to point out flaws in the
report.

The article is a myth

[Pedrito]

As a manager of developers, I can tell you that there absolutely IS an shortage of IT people. At least a shortage of good ones. I mean, I get tons of resumes from people who don't know anything about software development, but they're not useful to me. I have had 4 developers in my group. 3 of them are here on H1B work visas. I didn't hire them because they were foreigners. I hired them because they were the best I could find.

I have one other person that I'm probably going to hire in the next few weeks. He's was born and raised American.

I've interviewed at least a 50 developers over the past year. I'd say that 80-90% of them have been foreigners looking to get work visas. So, if you ask me, it looks like there's a serious shortage of IT workers in this country, or at least in my area, which is a high-tech center (the Dulles Corridor of Northern Virginia).

That's kind of interesting

[acacia]

We (the company I work for) write parallel processing applications. We had a skills shortage, but managed to get around it by training people with good experience in technologies similar in nature.

I wasn't born with parallel processing app dev skills. Neither were any of my foreign co-workers. They were trained on the subject, and the criteria for hire is/was an ability to absorb new concepts with minimum effort. So a hi IQ gets you the job.

I suspect in your case that you didn't pay the prevailing rate for the position (perhaps bad information?) and/or you were not geared to recruit people with the real prerequisite: The ability to comprehend what was necessary to make the software work.

Unfortunately, the people making hiring decisions do not do so optimally. That is just a matter of human nature. I think that that your specific requirements drew you into a sort of no-win situation, in that you could not keep/advance your career without a guaranteed success, and in response to that you targeted your audience too restictively, which effectively precluded success.

The next time you look at finding a person to fill a position, look at what is really required outside of the core technologies. Those who fulfill these requirments are the people you need to interview for. Which is the unspoken point of this article: We have created an atificial limit on applicants and suffered, but now that the price has dipped somewhat, with lowered expectations we have a surplus.

Re:Cause and Effect

[SuiteSisterMary]

Now what Scifi series is this reminding me of? Ah yes, BattleTech.

Our programmers are the pinnacle of five hundred years of selective breeding and intensive training. Five hundred programmers are grown from a genetic sample, then raised together, being constantly tested. Most will die in that testing, or prove unworthy and be demoted to a lower caste.

Programmers who don't work to death by the time they're thirty five are considered solahma, fit only for changing the diapers of young sibkos, and for going on suicide runs, such as fixing Y2K problems.

Re:There really was a shortage of *good* people

[SuiteSisterMary]

There are two (three actually) main schools of interview these days. School one is what I like to call 'I am the Alpha, I am the Omega.' They're not looking for the correct answer, they're looking for their answer. And if you give a correct answer that isn't their answer, you didn't give them the correct answer. School two is the 'I wanted to be a psychiatrist' school, where some managerial idiot starts doing behavioural modeling and response interviewing, which he learned at a two day course, and draws conclusions from it. "Tell me about a time you had to work in a team. Tell me about a time that you had to work under pressure..." School three is the interviewers who don't suck. They're often not looking at what answer you give, so much as how you arrived at that answer. Let me give an example. I learned this trick from my first full time boss, and use it to great effect. The Kobyashi Maru Database

You are the DBA of a database running a client server application. Several people in the office use your application, and all is well. One day, a DBA from a different group is told to install an app for his users onto the DB box. Shortly thereafter, people who use your app start complaining about performance being slow, and data being lost sometimes. What do you do?

And the beauty is that there is no correct answer. No matter what they say, you explain why that doesn't help or won't work. This gives you two benefits: you get to see where their strengths and weaknesses lie, and you get to see how they work things through, under pressure. A little bit of questioning by the applicant eventually reveals that it's the other DBA's new app causing the problem. Now, of course, the eventual answer needs to be 'I punt it off to our supervisors and let them deal with it.' But anybody who says that is obviously, as Scott Adams calls it, "Juan Delegator." Talk to the DBA? He says it's your app's fault. Show him proof? He doesn't believe it. Try to fix the server? You can't touch his stuff. Buy a new box? No money. And so on.

Welcome to America

[Zero__Kelvin]

I estimate that approximately 80% to 90% of the people in Software don't belong here. Why are they here? It's simple. When they went to the local school and told them that they want to learn Software because they heard it pays well, their counselor didn't bat an eyelash!

When people ask me if they should get 'into computers' (because they heard it pays well) I tell them this:

"If you don't love it, don't bother. If your in it for the money, you'll never be any good at it. In order to be a good engineer of any kind it has to be in your blood. If you're doing it for the money you'll never be any good at it. If you are a natural, you don't do it for the money; the money just follows"

Unfortunately, I am responsible enough to do this, but your average 'American Joe' who takes a job as a counselor doesn't see a problem with getting into a career for the money, even if he/she didn't choose their career path for the money. It's the American way. Unfortunately, the result of all this is the current state of Engineering in America today 8^{

The good news? I look 10 times better when compared to 'the average.' Still, every time I hear someone proudly call themselves a 'programmer', I shudder. These people are often performing the Software Engineering function, and they still don't know that programming is only about 20% to 40% of the puzzle. Scary!

Cause and Effect

[jeko]

1. Coders get dumped when they turn 35.
2. We can't find any skilled coders.

Hello? Does anyone else see the correlation here? Skill is the product of talent and experience. Talent comes from God Almighty in precious little doses, but experience comes with age.

The skilled coder you can't find is probably one of the ones you dumped because his salary was just a little too high. Now you'll pay double his salary in recruitment costs and receive nothing productive in return.

You would think even an MBA could understand this.

Age isn't the only factor...

[ChaoticCoyote]

Older workers (I'm hitting the big 4-oh this year) have negatives beyond age. We often have wives and families, which mean we're unwilling to work 6-80 hour weeks and on weekends. Wives also come with children in many cases -- leading "mature" workers to want benefits like insurance and pension plans.

A couple of decades ago, having a family was a *plus* when applying for a job; it proved stability and responsibility. Today, when the average tech job lasts for a year or two (if that!), employers are more interested in cost-cutting and reducing benefit loads. Which may explain why so much software today just simply sucks...

The same force that drove manufacturing jobs -- cheap labor -- overseas will now begin to eat away at the U.S. tech industry. Someone working in Mexico or India requires a lower salaray and fewer benefits than the equivalent U.S. worker. In a world driven entirly by the collection of wealth, does it surprise anyone that tech company have foreign development shops or employ H1B indentured servants?

--
Scott Robert Ladd
Master of Complexity
Destroyer of Order and Chaos

Re:not programmers, just the other IT workers

[baptiste]

You're off your rocker! As much as you'd like to believe its all the pseudo technical types being laid off - far from it.

Sure, a few years ago when companies laid folks off, it was usually the slackers and technical marketing types. But no longer!

As an ex-NORTEL employee (by choice long before they cratered) I can tell you I am absolutely blown away by the names I'm seeing come across the local mailing list for laid off NORTEL folks. These people were best in class programmers with excellent skills. The kind of people who got the top level review rating each year that only %5 of employees got. Thats because companies are now shedding entire projects and divisions. Before that used to happen on a small scale and most of the top coders got jobs elsewhere in the company. But in today's environment, theres nowhere else to go in a company laying off 30% of its work force - all open positions are GONE. So what happens? When a divison gets cut or a major project cancelled, EVERYBODY gets the ax.

Now you'd think they'd have jobs just lined up - well, as someone whose been looking for a halfway decent job for 7 months I can assure you its not always that easy, at least not here in RTP Many companies are laying folks off, not hiring. The # of job postings have gone down by at LEAST an order of magnitude. Even worse is the companies KNOW this and can hand pick the person that fits every single one of their requirements where before if a person fit 80% - it was a catch.

I've even noticed it in the job postings - an almost arrogant tone that basically says if you can't meet every single requirement (and they list tons) then don't even bother cause they won't even acknowledge they received your info.

During 1999 - there WAS a shortage in certain areas, no question. Totally unqualified people were getting hired because whatever work they could do was better than the job sitting unfilled for months. Now, its brutal and companies that can manage to get an opening approved are taking their time finding the gems - and then are paying them much less than they used to. I know excellent programmers who earned $80K getting offers of $65K

As for the middle maangers - yes they seem like incompetent dorks who serve no purpose, well, I can tell you that while sometimes they are, often they server a very important purpose, at least they did @ NORTEL. If they didn't exist, you'd probably spend more time dealing with customers and requirements, and release schedules which would drive you nuts - I've worn both hats and project managemednt can be a very difficult job, especially when you have to manage a project with hundreds of design teams who all say their feature is the most important (of course) So don't get too high on that pedistal because without project managers, your release would never make it out the door on time and without good technical marketing types, you'd have no customers for your product and no job.

How Management Sees Us

[localroger]

I am the one and only programmer for a distributor, as opposed to manufacturer, in an industry where very few distributors bother to have in-house software development. As such I have, without really trying, become as close to unfireable as you ever get, which is one reason I hang around the place even though they don't pay as well as larger shops.

One day one of my coworkers was doing a service job and discussing the computer system I'd designed with an employee of the customer. The employee had just finished taking a management course and asked my coworker if there was any backup for me. To which he replied, no, our guy's pretty unique.

The customer employee then demonstrated his grasp of management principles by saying that he'd just been taught that if you have anyone like me on the payroll, you should fire them at once! Sure it will hurt for awhile, but eventually you'll recover and you won't be at their mercy.

So that's what it's about, boys and girls: POWER. We do stuff they don't understand and it scares the shit out of them. The only way they can feel secure is to be sure we can be instantly replaced. Fifteen years of loyalty? Meaningless. Skill and experience? Meaningless. Modern management teaches that the most important thing is staying in control.

Fortunately, my current employer is as out-of-date as I am, and doesn't feel that way. Which is another reason I hang around even though the pay isn't so great.

It certainly is

[localroger]

what's to stop you from, say, extorting more wage? Better work hours? Anything?

Exactly. Like I said, it's about power.

And the answer to your question is "enlightened self-interest." I have demanded, and gotten, certain concessions as a result of my influence, but I'm smart enough to know what the company can afford and will put up with. They've worked with me long enough to know that I am a reasonable person. In short, they treat me like a human being and I return the favor.

I suppose they just assume everyone is out to grab as much as they can, which is just natural, considering they are the 'mind' of the corporations which are agents of greed.

Exactly. It's about power. It's not about satisfying the customers, it's not about building something we can be proud of, it's not about being the leaders in our sector, it's not about being efficient or beating the competition, it's about being in control. It's about responding to a human situation with the knee-jerk response of a machine that isn't capable of understanding pride, craftsmanship, or loyalty.

Fortunately, the feeling is mutual. I wouldn't want to work for someone who would act that way anyway. For that matter, I suspect a lot of people who share my skills feel that way. Maybe that's why some of the managers who have posted here have such trouble finding people who know what they are doing.

if the people can't come here, the jobs go there

[janpod66]

The basic flaw with Matloff's argument is that he assumes that if foreigners don't come to the US on H1B visas, the jobs will go to US residents. That's wrong. What happens in real life is that if the foreign programmers and professionals can't come here, the jobs simply go to where the people are. Most large companies already have development labs set up all around the world and can shift resources overseas at a moment's notice and without any increase in cost. In fact, that's already what happens when potential foreign hires can't come to the US: they simply work overseas until their visas come through.

Welcome to the new globalized economy and information infrastructure. Knowledge workers produce a product whose movement can't be controlled and that can be instantly shipped anywhere. And the basic tools, PCs, are available anywhere in the world.

Beyond that, Matloff's claims about shortages, wages, "indentured servitude", and working conditions simply don't agree with what I have seen in real life. But it isn't even worth disproving his factual claims point-by-point when his basic reasoning is so faulty.

Having foreign programmers and professionals come to the US has been a spectacularly good deal for the US, and it has been devastating to the high tech industries in foreign countries. Developing countries have been particularly hard hit by this.

There really was a shortage of *good* people

[koreth]

I was a manager during the period in question, and I can tell you, I had a devil of a time filling a couple of positions. Let me take issue with this statement from the article:

If employers were that desperate, they would certainly not be hiring just a minuscule fraction of their job applicants.

I'll take one example. We had a data warehouse (mostly a big Oracle PL/SQL application). The engineer who designed and implemented the original code left the company, and I was tasked with hiring his replacement since there were some pretty substantial architectural changes we needed to make.

Now, there are a lot of database people out there. A lot. I looked at more resumes than I can count. My ear was sore from phone interviews. Thing was, just about everyone I talked to fell into one of two categories:

• Listed all sorts of Oracle skills on their resume, but couldn't correctly answer my SQL skill-testing questions. (Which were nothing especially complex.) This was the vast majority, which surprised me.
• Knew their SQL, but clearly had no design skills to speak of. I'd ask them to design a trivial application and they'd either botch it or claim that they just wrote code, someone else always designed it and gave them a spec.

I looked and looked. The executive staff got really antsy and started leaning on me to do what the article suggests, just hire someone to get the work going, even if they weren't perfect for the job. I resisted for a while but finally caved in.

The contractor we brought in -- one of the better ones I'd interviewed, though I hadn't liked him well enough to want to hire him -- did a decent job of talking to the right people, gathering requirements, and getting himself acquainted with the layout of the code. But then he started to submit his own code, and man, what a disaster. I wasted weeks correcting his mistakes. Finally I fired him and went back to my original search.

The specifics of the story here aren't important. The point is that it doesn't take many times being burned by the "hire any bum off the street, just fill this technical position" attitude before you develop a very healthy caution about hiring the wrong person. I've seen it happen at other companies and I think it's a universal truth: hiring the wrong person for a job can leave you in a much worse position than hiring nobody at all. Not least because you think you have the position filled, so you stop looking for a while.

Experienced managers know this, so they put themselves through the "there's nobody out there!" routine when the job market is tight. It sucks massively, but it sucks less than the alternative.

(How did the story end? We found an H1-B person who fit the bill perfectly. Then the government took so long to process his paperwork -- months -- that by the time it came through, he'd gotten cold feet. Ugh! Happily by that time I'd moved to a different group.)