Slashdot Mirror
Hackers As Factory Workers?
DevDude writes "A strangely interesting article is running on MSDN, entitled: The Case for Software Factories. It suggests creating 'development environments configured to support the rapid development of a specific type of application.' As a developer thrust into many an unsavory situation, I am constantly stepping in the remnants of some development methodology or other. Will super-specialization of software development teams help the industry to push out better software faster? Or are we hassled enough without being treated as an assembly line?"
Programmers are at the end of the food chain buddy. That's why you need to upgrade and become part of a function that requires decision and analysis. Or , if you're looking for the easy exit, a Sysadmin :)
this is simple - it's the difference between the mass production line worker, and a master craftsman.
It would be foolish to say that there is no place whatsoever for one or the other type of worker in the programming ecology (extrapolate this analogy to society and economics at your own risk).
It's 10 PM. Do you know if you're un-American?
It has always been said with software...
"Good. Fast. Cheap. Pick two."
Outfits like the one proposed here elminiate the choice of "Good."
"You spoony bard!" -Tellah
It could work, but only on a limited basis. They could probably make junk applications, things that don't really need very much polish on them. Maybe programs that would only be used for a small period, and not necessarily for a large market.
Most programmers have a strong desire to be challenged and to solve problems. They want to use their brain and imagination. (Wasn't there an article here within the last few weeks about charactistics of good programmers?) That's why they hate cubicles so much. If you try to stuff them into "factories" where they're doing nothing but making modifications to existing code over and over, the tedium will get them.
Those that don't quit will burn out and self destruct. While there is a surplus of IT workers, a sweatshop like this will burn through programmers fast enough that it'll only last a few years before the quality of code gets shoddy because there's no good programmers left that are NOT burned out that will willingly work at such a place.
with well-understood types of solutions, one could presumely setup a "factory". Indeed, the off-shoring of programming task is part of that. On the other hand, there are programming/designing tasks where not even the problem is that well understood, or that require a high degree of independent, creative thinking.
"According to the Standish Group [Sta94], businesses in the United States spend around $250 billion on software development on approximately 200 projects each year."
200 projects sounds extremely low, unless they mean 200 projects per business which is extremely high. How do they define a project? I would guess there are nearly 200 videogames a year so they can't be included in this figure. Does a project need to be >1000000000$ before it is considered as a project in this group?
Look, you can think what you want but this is BUSINESS -- and you need to adapt or die, if you cant cut it, get out of the way there is someone else who can.
Not every programmer who resents the idea of typing repetitive instructions all day has gone crazy. In fact if implementing your project requires lots of mindless, repetitive work, then your design decisions are crazy. The very term 'project' implies that you're doing something new you haven't done before.
Example: the cheap, unskilled code monkey spends lots of time repetitively building mediocre GUI components with his, err, GUI builder, while the 'artist' uses the same time to write a factory that constructs the GUI components on the fly, considering things like the data structures they'll edit etc. One central place to enforce consistency and human interface guidelines. No mindless repetition.
but what do i know, i'm just a model.
Keep it simple. If your company competes against a large one that believes this mud, then punch it in its soft spot. If their outsourcing and/or code factory leads to bad quality, then market your product as having better quality. On the same token (though not necessarily in the context of this topic): If their outsourced customer support system is flimsy, then fight them with a local-hire system that gets the job done. And if that doesn't work, then you've just legitimized their operation. Boo hoo, not much more to discuss.
A major software projects starts with a handful of architects who must scope out every aspect of the project (reqs, overall and subsystem designs, APIs between subsystems) before development begins. The architecture and design must be audited. The design is then handed off to developers (i.e. tech leads) who work with programmers to implement each subsystem. All subsystems must be thoroughly QA'd. Finally, the subsystems are integrated and tested.
After design hand-off changes should be few but will happen. Anybody who whines about these changes will be fired on the spot. Any programmer who whines about bugs found by QA will be skinned alive.
In this model, the programmers are factory workers. You don't want to be a factory worker: become a tech lead and eventually an architect.
One of the really odd things about a long career in computer science is that you often find the Big New Thing was a big new thing ten, or twenty, or forty years ago. Wilbur and TSO and VTAM become EMACS on a terminal which becomes "thin clients" (and its dual, dedicated compute time becomes workstations become personal computers.)
In this case, we're seeing the re-awakening of the notion of "commoditizing" programming. Back in the day, it was the notion of "deskilling" programming with forth-generation languages; before that it was the development of general high-level languages like FORTRAN and COBOL; before that it was the realization that you didn't have to be either Goldstein or von Neumann to successfully program a computer.
So, yeah, it's possible to improve programming productivity by building specialized environments for certain classes of problems. That trick worked for report programs with RPG in the 60's; works marvelously with parser generators; works pretty decently with GUI tools for UI programming now; and will undoubtedly work for other classes of programs in the future.
Then you'll find people programming new classes of programs by hand, and a few years later someone will say "wouldn't it be better if we could do this with specialized tools?"
And you can bet that 50 years from now the big issue will still be figuring out what you want to do, and figuring out how to describe that.
The "software factory" analogy has been around before. It's nonsense, of course. The software analogy of a "factory" is the plant that presses CD-ROMs. Pressing the 10,000th CD-ROM of a software product is the software equivalent of building the 10,000th Nissan Maxima on a production line.
But writing the software which will go on that CD-ROM is the software analogy of designing the 2005 model of the Nissan Maxima. Now, some software development is not very creative. Just as tweaking the design of a car model that's been around for 10 years, to get something a little bit new for a new model year, is not very creative mech engineering. But it's still design, not assembly-line production. A competent software engineer will be able to do it better and faster than a bad one. And a factory worker will not be able to do it at all.
I've been under many different managements and the most common call to action when projects fail is, "Change the methodology!" For some reason this is the all too common diagnosis. I've worked with, under, etc. many "methodologies", and generally these methodologies correlate loosely at best to measured success. Of course authors, pundits, and visionaries continue to make fortunes rolling out countless new methodologies and writing books proving they are right.
As for the notion of a "factory" -- I find the idea patently absurd. This idea presupposes software is a well defined "product", and all one needs to do is create an assembly line with interchangeable employees thus fostering efficiency, consistency, etc. It doesn't work! I've been there, done that.
As an aside, I've found IT people have a hard time picking up factory jargon and idioms, such as learning to insert the work "fucking" in between syllables of words to form new words.
I'd write a book on methodology that has ALWAYS worked for me and teams I've been on, but I could never find a publisher willing to publish a one page book:
Find someone who knows what they want. Find IT people who know how to do it. Put them in a room together until it gets done.
Readers are welcome to download this book for free.
A manager went to the master programmer and showed him the requirements document for a new application. The manager asked the master: ``How long will it take to design this system if I assign five programmers to it?''
``It will take one year,'' said the master promptly.
``But we need this system immediately or even sooner! How long will it take if I assign ten programmers to it?''
The master programmer frowned. ``In that case, it will take two years.''
``And what if I assign a hundred programmers to it?''
The master programmer shrugged. ``Then the design will never be completed,'' he said.
The hard part is not turning a specification or requirement into a working piece of software. The hard part is writing a specification that captures what the customer needs to have happen in unambiguous language.
Software development should be treated as a multiplayer team communications game. The success of the team depends more on how successful the communications are.
Open aspect of open source software which is often overlooked is that it allows smaller teams to develop software faster. If the software is maintained by large self-regulating networks, than the network itself becomes like an assembly line, but better. This accomplishes what the article is talking about and does it better.
LedgerSMB: Open source Accounting/ERP
Here's the deal. Approximately 5% (maybe 2%) of developers are god like creatures. Able to weave logic from gossamer webs of electrons and silicon. Stunning in their ability to move from ideas to usable systems in timeframes that leave the head spinning. Able to apply new paradigms .....
The other 95% (98%?) are hacks. They are the new medium skilled lathe operators of the 21st century. Their only claim to fame is that they execute their relatively unskilled efforts utilizing new technology.
So did telegraph operators. They were young, cool and too good to stay in a job for long. Train Engineers. Airplane pilots. For God's sake, captains and engineers of those new fangled steam riverboats were about as cool as they come.
But please. Wake up. It's almost over. Hacks are hacks are hacks. It doesn't matter what you're hacking on. If you're average - you're average. If you want to be special - make special things happen. But please please please stop for a moment thinking you're something new. Just the most recent in a long line of boys with the new toys.
That's because the "factory" is the wrong metaphor for the whole process. Instead, we should think of programmers more like industrial engineers: they don't build widgets, they build processes that build widgets.
./configure file that knows which components to assemble and modify in place so we get the exact workpiece we want.
The real work piece is the executable, and once we've built our source code, we can turn out as many copies of the executable as we need. If it's really complicated, we need a
We as software engineers -- or programmers, if you like, although I'd argue there is a difference -- are working on the tooling. We need to figure out the thing that's needed, and come up with a pattern for that solution, and then figure out how the factory can build them. Then we look at the results as they come out, test them, measure them aghainst the real world, and think about how to build a new version that's closer to what the real world demands. But, like industrial engineers, we don't build a zillion copies of the factory -- every factory is a little different.
Software is a service, not a product.
"Give away the stone, let the oceans take and transmutate this cold and faded anchor." - Maynard James Keenan
The manufacturing model of software development is dead wrong, and the reason why ought to be ovious to any good software developer.
Programming computers is an almost entirely an art form. It was the same way 10 years ago. It will be the same way 10 years from now.
Parts of programming which were art forms 10 years ago are a science today. Memory management, for example, is now a very well understood process. What happened? It ceased to be a programming task. Think: Java verus assembly language. Don't spend much time juggling pointers in Java, do you?
Unlike manufacturing, once we've solved a problem it stays solved. That means the role of a specialist who is very good at some technique is necessarily short term: As soon as someone gets good enough to automate the technique, the need to repeat it disappears.
As a result, computer programming remains a high art form: programmers are only needed for the tasks which still defy rigid definition. Art favors the renaissance man, the master of a breadth of disciplines who works with all of them. Computer programming will continue to favor such broadly skilled artists.
So, those of you who style yourselves java coders or C coders or MSCE's, take heed: Become a generalist because your days as a specialist are numbered.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
There are plenty of systems out there that are made to be put in place for years and years. IBM makes mainframes that will be expected to last 20 years, and never fail during that whole time. They really deliver too, as their old ones do just that. You can get systems with hot swappable PROCESSORS, they are so redundant, and others where 2 CPUs run in parallel, checking on each other.
This is all real, and is what runs mission critical stuff across the world.
However, as the orignal poster pointed out, it's good/cheap/fast pick two. I'd even say pick at most two. These systes are neither cheap nor fast. They do not use the latest greatest shit, they use proven reliable hardware that has undergone lots of testing. They are also not in any way shape or form cheap. Whatever level of processing they offer you, you can beat 10 times over with commodity hardware and still be under their price.
The software that runs on them is likewise ultra reliable. Crashes just arne't an option, and they don't happen. The OS and apps are just rock solid. Of course, that means they also don't support all the whiz-bang features. No happy candy-coated bouncing docks or the like.
It's the same deal as in consumer electronics. I continually see peopople lament how poor quality theri $40 DVD player or $20 VCR is as compared to the $500 model they bought years ago. Well DUH, they could afford to put some quality into a $500 DVD player, they can't in a $40 one. Thing is, you can still buy high end electronics, they just still cost lots of money. Go get a studio grade DVD player. It'll be built to last, produce a better picture, but it'll cost $500.
Whatever the level of reliability you demand, there is probably a solution out there that can meet or exceed it. However, don't whine that it costs money. Quality costs money, always has, always will. If you want it cheap, be prepared to accept the consequences that come with that.
Also, in computers, many times it's better to just go with multiple cheap systems. Ends up being as reliable and much cheaper. I mean say you have a server app that is prone to crash, because it is continually hacked and updated. Also, it runs on cheap hardware, so that's not reliable. Ok, so you design it such that it runs on 3 parallel servers, each capable of taking 100% of the load. So even if the hardware fails on one, you still have two, and your testing indicates that it's likely that if one crashes, it'll get back up quick enough that the other won't crash in the mean time. Maybe you go for 4x just to be safe.
You end up having what appears from a user perspectinve to be an ultra-reliable setup, however it still allows you to quickly hack out new versions, that may not be as stable as they should.
I thought the concept was really cool!
I no longer had to re-invent the wheel every time I wanted to build an app.
I saw this effort very similar to my construction in electronics where I buy components off the shelf and know what I'm getting. I wouldn't even think of trying to build an IC, resistor, capacitor, or large inductor.. albeit I could if I had to. Or what building contractor would try to make his own lumber or nails.. even windows or doors?
It all comes preassembled - commodity items - and everybody knows how its built - and could build one themselves if they had to, but why? The vendor holds a "natural monopoly" on the things he makes because his "economies of scale" allow him to produce this item and even ship it to you at far less than the cost it would be to you if you had to make your own. Ever tried to build a light bulb? ( well yeh, I have with vacuum pumps and pickle jars..)
I loved the idea that Microsoft released this standard assortment of "objects" which supposedly are standardized. It took Microsoft hundreds of man-years to generate this code, it will take me years to master it, just like spending years to master the keyboard on a piano. I figured that an investment in my time learning Microsoft technology would be time well spent.
So, now where am I... I know an ancient technology . Microsoft keeps changing the keys on their piano keyboard! I can't keep up with their endless changing of things. I can't stay with one technology long enough to understand it thoroughly and avoid striking "sour notes".
This endless changing of things and their efforts to insure I learn only what they will allow is my main reason for avoiding Microsoft products.
I kinda see Microsoft products like fashion trends in pants. Its quite easy to slip one pair of pants off and another on when your pants go out of style. So, if its something where I am not counting on it being there in say three years, and nothing on the old will run on the new, Microsoft products fill the bill nicely.
But if its something like my tools, car, air compressors, home, anything I am counting on to sit there and do what it was designed to do until I dismantle it, then I want some platform where stability of design and maintainability is paramount.
I have no intention spending a helluva lot of time learning to play the piano if I know the piano manufacturer thoroughly intends to mess up the keys all the time so as to give the latest generation pianists an edge over the ones who bought into the plan a couple of years ago. With our playing skills being suddenly rendered obsolete by the change. I know big business can afford this kind of stuff, but as an individual running a small business, I can not.
"Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]
In the old days of machines of 8KB of memory and sub-Mips processors, programming was easier. The space of what programs you could potentially implement was much more limited than today (although both are obviously very large spaces). Most of the development time back then was devoted to figuring out how to implement a program, e.g. how to fit it in memory and make it fast. The was no operating system and the language was assembly (for the 8KB sub-Mips machines anyway).
Today we spend a great deal more time deciding what a program should do, since better machines have expanded the possibilities to an extent we could scarely imagine 35 years ago. But we also spend more time deciding how to implement programs, though for different reasons than before. Now we choose languages, databases, GUI frameworks, and on and on. And the basis for making those choices intelligently involves much more knowledge than was needed before, i.e. not just knowledge of the target machine, but knowledge of the capabilities of the potential languages, operating systems, databases, etc. So the how part is now much more knowledge intensive, whereas it used to be more like solving a puzzle.
So programming really has gotten harder! Is it really any wonder it takes longer and is so often screwed up?
How to improve the situation? Well, the what part is only going to get worse, and we want it to, because that means we can do more with computers. The how part, on the other hand, can and probably will get easier. Standardization is the easiest way that can happen, though I wouldn't call standardization "easy". Using higher levels of abstraction is another way, but the current means of achieving this is mostly through components, the use of which narrows the spaces of both what and how. The problem is that component packages often make incompatible decisions about the how of their implementation, which often makes it difficult to combine multiple packages. And that gets back to the need for standardization.
Neither of those is done on an assembly line, in a factory, or with an emphasis on speed. (They probably also take more skill and creativity than you think.)
-- . . ramblin' . . .
The vast bulk of programming is just repetition.
.
If so, something is wrong. Computers are good at repetition, humans can do better things. If your programming is repetitive, you are using the wrong tool. Heck, you should spend your time developing exactly this missing tool. Build a decent library or a preprocessor, then use it to automate away the repetition. It pays off soon. Compare http://c2.com/cgi/wiki?SharpenTheSaw and http://c2.com/cgi/wiki?ThreeStrikesAndYouRefactor
...and they don't come in the form of cheap code monkeys. I've often found that most of the problems associated with software development come down to these three issues:
Knowledge: you need programmers who KNOW what they're doing. They MUST be well versed in not only the particular language(s) that a system is coded in, but also in general software design concepts (e.g. Design Patterns). Without this fundamental knowledge, even the most well meaning code monkeys can turn even your best & most efficient libraries into a slow, bloated, buggy pieces-O-crap in no time.
Experience: not to be confused with knowledge, experience refers a programmer's grasp of the scope of your software. This includes the particular purpose (consumers, market, industry) that the software is designed for as well as actual experience coding inside the existing software. After all, in this high turnover world, where contractors are becoming the norm at many companies, it is very common for transient code monkeys to be totally unaware of an existing library, and/or not make connections with existing code and/or consolidate functions into common libraries in the first place. The results == redundant lines of code == bloated, buggy piece-O-crap.
Communication: This is the most important part of any workgroup. The "architect" needs to know about the existence of commonly used libraries and needs to relate this information to the programmers. In turn, the programmers need to recognize and inform the architects of the need to refactor/reengineer portions of the software. Communication must go both ways! Communication also tends to reveal who the code monkeys are and keep them from doing serious damage to the software.
Of course - a healthy compliment of unit testing, source control, debugging skills & documentation are also important, but you'll only find Knowledge, Experience & Communication associated with quality programmers - and quality programmers don't come cheap. Otherwise, you'll find that code monkeys will happily mass-produce lines of code (crap) that you're organization will have to pay loads of $$$ for over time to maintain.
Actually, Microsoft or Google are proof of what he was saying. They're not companies who hired burger-flippers off the street, but companies who hired smart people with an education.
At least for Google we had a recent article right here on Slashdot: they have a very high number of well paid Ph.Ds. Quite the contrary of what your average clueless PHB or beancounter does in the name of efficiency.
Microsoft makes money by selling PHBs the illusion of "buy our patented snake oil, and you can make quality software fast with any burger flipper turned VB.NET developper." But suspiciously enough that is _not_ who Microsoft employs.
Yes, you may rant and rave about how much Microsoft's software sucks, or how it misses deadlines too. But guess what? Most other companies software sucks twice as hard, and costs more too.
While Microsoft does have buffer overflow exploits, other companies had those _and_ a bunch of bugs or broken design decisions of their own. Or shipped downright broken and non-functional software, just to keep a badly planned deadline.
Among the closed source world, Microsoft actually does an outstanding job. So, you know, maybe hiring smart people actually does something for them.
A polar bear is a cartesian bear after a coordinate transform.