User: bcaulf

That's not what stepwise refinement means.

On: When Should You Go Back To The Drawing Board?

That's not what stepwise refinement means. The term has a specific sense that was introduced to software development by Niklaus Wirth's paper in the Communications of the ACM, 14(4):221--227, April 1971, "Program development by stepwise refinement".

Stepwise refinement is a procedure for refining a design, from a high level design that closely resembles the requirements, progressively down to a low level design that resembles code. It is not related to refining a code base by rewriting parts of it. You will have to find a different word for your maintenence programming style.

Re:Doppelganger

On: Interview w/Slackware Developer David Cantrell

That is a very true thing to say. My suggestion is to use evildave when referring to the Brit Cantrell. Hi, evil!

visual iditor

On: Major Linux Deployments

s/foo/bar buddy. Hard to say that is more obscure than the Edit version. NB that if there is whitespace in one or the other of your strings, that is no problem in vi:

s/several times easier/far more primitive

If there are slashes, no worries, use any other character as a separator:

s!batch/shell!bourne shell

This exact same command set applies in vi, ed, sed, perl. (Well, you have to close your parentheses with sed and perl: s/foo/bar/.)

yes you are missing something

On: New Crypto-OS

Crypto-OS can and would destroy and regenerate the sshd private key periodically, like every hour. This is SOP for ssh. The sshd private key is not retained on the client side and is never transmitted in the session.

Spooks can analyze that wiretapped session until forever, even after seizing the user's machine, and there is no known way to extract the plaintext version of the session.

That is a silly thing to say

On: New Crypto-OS

Public key cryptography requires only a reliable communication channel and not a secret one. All the Crypto-OS people need to do is publicize their public key. When a user wants to set up a secure session, he generates a symmetric key and encrypts it with C-OS's public key. C-OS proceeds to decipher it and start a symmetric encrypted communication session.

Since the user has an account he would also want to authenticate himself by using his own private key; C-OS is in receipt of the user's public key.

Re:MS & Intel

On: Intel's Itanium Processor Explained

The Xeon is to the P3 what the Pentium Pro was to the Pentium. Big honkin L2 cache is the common thread.

!! PIII Xeon, PIII and PPro are all the same core (let's call it 80686, or better Sexium) with various process sizes and cache architectures. None are a direct relative of the Pentium (80586). There was never a large cache version as such of the 586, since L2 cache was always on the motherboard.

Anyway I got a couple of pages into this review and found that the reviewer is an idiot. 64-bit CPUs are not a new phenomenon and an Itanium review shouldn't start with a long-winded justification of using a 64-bit word size.

Article Text

On: Greenspun on Managing Software Engineers

Managing Software Engineers

by Philip Greenspun (philg@mit.edu)

Submitted on: 2000-10-22

ArsDigita : ArsDigita Systems Journal : One article

Philip Greenspun founded ArsDigita Corporation and was its CEO from inception until it reached $20 million/year in revenue. Currently, he is Chairman of ArsDigita and teaches computer science at the Massachusetts Institute of Technology. Why an article on managing people? And one written by someone with training in computer science rather than business administration? There are thousands of books on the best ways to manage people. Many of these books are excellent, having been written by people who've devoted their lives to the discipline.

Software engineering is different.

Software engineering is different because only the best people significantly contribute to achievement. Traditional management texts assume a distribution of talent among the workers. Each worker is contributing something useful and the challenge is to get each one to perform at his or her maximum potential. In the same factory, the best worker may produce two or three times as much as the average, but all the workers are contributing. In software engineering a good programmer is at least 10 times more productive than an average programmer (Brooks 1995). If a product is being developed rapidly, the average programmers will consume nearly their entire work day just in reading and understanding the new code generated by the good programmers. Thus the challenges of a software engineering manager first and foremost are (1) creating a work environment where good programmers will be satisfied enough to stay, and (2) creating a system via which average programmers can become good. In an ideal software engineering organization, there are still some average-quality people but these should be viewed as being apprenticed to the best people and being taught as fast as possible.

Software engineering is different because people at all levels of the organization perceive themselves to be equally intelligent. Consensus-style management can perhaps work when there is a gradient of perceived ability. Given enough time, the less able workers will follow the lead of the more able workers. One of the paradoxes of software engineering is that people with bad ideas and low productivity often think of themselves as supremely capable. They are the last people whom one can expect to fall in line with a good strategy developed by someone else. As for the good programmers who are in fact supremely capable, there is no reason to expect consensus to form among them. Each programmer thinks his or her idea about what to build and how to build it is the best.

Software engineering is different because a leaf-node worker is more expert than any manager, even when the manager is a great engineer, in at least the small portion of the system that the leaf-node worker has personally built. This makes it difficult for a manager to engage in a technical argument with a worker. It becomes nearly impossible when the manager's technical skills are weak. The worker can spin castles of complexity in the air and come up with impressive-to-the-MBA excuses for why it has to be done a certain way or on a certain schedule.

Software engineering is different because the organization can't afford to lose the individual productivity of the best people by pushing them into management. A truly great programmer may generate 10 times as much business value as a merely good programmer. Can the organization afford to take someone who can do the work of 100 average programmers and push him or her into a pure management role? Probably not. Can the organization afford to put people with weak technical skills into management roles? Probably not. Once you give Joe MBA a title and ask him to coordinate eventually he will be making decisions that have engineering implications. Thus many of the best programmers are eventually forced at least to assume project leadership and mentoring responsibilites. Since they are still expected to produce designs, software, documentation, and journal articles, the danger is that the new manager will become glued to his or her screen and never look up to see how the project team is doing.

Software engineering is different because measurement is notoriously difficult. The world is full of products that failed due to overly complex and tasteless designs. Yet all of these designs were considered tasteful by their architects. Systems that experts evaluated and found wanting, such as the Unix operating system (1970), eventually proved to have great utility. It is a bit easier to count up the lines of code per day produced by a programmer but if the project was not very tightly specified originally, how do you know whether or not these lines of code are useful? At this point a skeptical reader might be thinking that, while software engineering is different from line production work or any other endeavor with a manufacturing division of labor, there are similarities with research and development, management consulting, and financial analysis. This is certainly true but there aren't too many interesting books on how to reliably produce results in these fields (one is referenced in the "More" section below). Ideas to Steal Software engineering is different but it is not that different. What ideas can we steal from the broader world?

• people don't do what they are told
• all performers get the right consequences every day
• small, immediate, certain consequences are better than large future uncertain ones
• positive reinforcement is more effective than negative reinforcement
• ownership leads to high productivity

people don't do what they are told In Bringing out the Best in People, Aubrey Daniels notes "If we always did what we were told, we would eat only nutritious foods, never drink too much alcohol, and exercise regularly." There is thus a natural limit on the effectiveness of written policies and management by telling then nagging.

A corollary to this principle is that people do what you reward them to do, not what you hope they will do. Often, when you look at what is truly rewarded in an organization, you find it is different than what you think is rewarded. Do the managers have an engineering background? If not, they'll probably be unable to perceive when a programmer is accomplishing nothing. So the programmer who does nothing gets a paycheck at the end of the month. Having thus been rewarded for doing nothing, the programmer tries it again the next month... all performers get the right consequences every day The natural way to manage is to spend time with people who aren't doing a good job. You help them out. You remind them of the good things that can happen to them if they finish a project or raise the spectre of their being laid off the next time the company needs to improve its profitability. These are probably the right consequences for someone who is underperforming. But what about the people who are performing? What if you ignore them day-to-day? Unless they are getting positive reinforcement from another source, they may stop coming in on the weekends to get a release out the door earlier, stop documenting their code, stop writing journal articles. A top performer won't sink to the level of a problem employee but that person may become average. And in the long run a company with average workers will at best earn an average return on equity. small, immediate, certain consequences are better than large, future, uncertain ones An annual review and bonus is not classically considered a very good way to motivate people. It is too far away, especially in a dotcom economy. Even if a worker is able to keep the bonus goal fixed in his or her head for the 365 days preceding the bonus allocation, there is uncertainty attached to it. What if the company is doing really badly at the end of the year? Will there still be a bonus? positive reinforcement is more effective than negative reinforcement Like most schools worldwide, MIT practices negative reinforcement at the undergraduate level. If student does not do a problem set by a certain deadline, we give him or her a bad grade. This has turned out to be extremely effective at ensuring that an MIT graduate has achieved some minimum standard. However, the students don't accomplish all that they could. The first term that we taught 6.916, we gave the students one week to do Problem Set 1. It was pretty tough and some of them worked all night the last two nights. Having watched them still at their terminals when we left the lab at 4:00 am, we wanted to be kinder and gentler the next semester. So we gave them two weeks to do the same homework assignment. The first week went by. The students were working on other classes, playing sports on the lawn, going out with friends. They didn't start working on the problem set until a few days before it was due and ended up in the lab all night just as before.

We thus proved the management adage that a deadline just gives someone an excuse to procrastinate and do nothing until the very end.

Graduate school at MIT is different. We want the students to do research, write up their results, publish them in journals, and graduate with a reasonably interesting PhD thesis. If a student finishes some research, the most effective faculty advisors immediately provide positive reinforcement by paying attention, helping design the next experiment, helping to draft a paper outline. If the student finishes a write-up, he or she is positively reinforced by being sent to a conference to present it. If the student finishes a PhD thesis, he or she is positively reinforced by being given a 3-7X pay raise.

The lesson from MIT? Negative reinforcement can work if the organization is extremely tightly managed, if the consequences are small and immediate (usually a problem set is due every week and only represents a part of the final grade), and if the goal is to make sure that everyone comes up to a reasonable level. However, the worldwide fame of MIT rests on research achievements by graduate students. This innovation is mostly supported by positive reinforcement. ownership leads to high productivity A related issue to positive/negative reinforcement is ownership. Non-ownership systems discipline those who are not working up to the minimum standard, but they do not offer enough of an upside to truly motivate people. Morever, non-ownership systems demand a very accurate setting of standards. Ownership-oriented systems include contingent rewards with an almost unlimited upside, and are thus effective at getting as much discretionary effort out of workers as possible.

As an example, in the early days of ArsDigita we had only a handful of customers: America Online, Environmental Defense Fund, Hewlett-Packard, Levi Strauss, Oracle Corporation, and Siemens. We had only a handful of programmers as well and hence the easiest way to divide the work was to give a programmer total responsibility for one project. The programmer owned that customer. If the project went well and the customer wrote us a big check, we gave nearly all of the money directly to the programmer. If any project had gone poorly and we'd been fired by a customer, we would not have had to think very hard to figure out who was responsible (fortunately this never happened while I was running the company!). People worked insanely hard to make their projects successful and their clients happy. More importantly, the programmer who did an entire project by him or herself learned enough to train new people, lead a larger project, etc.

After we grew beyond the 40-person mark, pressures to dilute the ownership aspects of our organization grew. We wanted to grow rapidly--nobody wants to buy enterprise software from a small company, even if the software happens to be open source. As our reputation grew, customers came to us with larger projects. We believed that many of our developers were too junior to handle complete responsibility for these large projects. Our costs went up because we had to coordinate the diffused responsibility. In the summer of 2000, when we had 200 or so employees, one of our clients was unhappy. It took a week just to arrange a meeting among the five managers who bore collective responsibility for the project! Meanwhile, individual productivity fell. It was taking more programmer-months and more calendar months to get things done. On weekend mornings you could walk naked through an entire floor of our headquarters building without fear of embarrassment.

(At the time of this writing, there is a proposal on the table to consolidate some of the separate management pyramids, thus taking us back closer to our original structure.) Building and keeping a good software engineering team What is the best way to attract some good software engineers to your organization? Hire a few to begin with. Good people like to work with other good people. This is true in every field but much more acute in software engineering. Why? Consider two management consultants working on different projects but within the same organization. If Consultant A does a bad job it harms Consultant B's reputation to some extent but does not require Consultant B to take any action. Whereas in most tech companies if Programmer A does a bad job it usually means that Programmer B will eventually be forced to use the bad code, read the bad code, and then fix the bad code.

What attracts good programmers? Traditionally the best programmers seek the most challenging problems. They want to work in an organization that is trying to build something important. Programmers have huge and fragile egos. If they are somehow assigned to a trivial problem and that is their only possible task, they may spend six months coming up with a bewildering architecture more complex than the Windows 2000 operating system, merely so that they can show their friends and colleagues what a tough nut they are trying to crack. Another source of ego-gratification for programmers is to have other programmers admiring their work. Open-source software projects thus have a big recruiting advantage over closed-source software companies.

What kind of working environment is necessary for programmer satisfaction? Good programmers want to achieve and therefore removing barriers to achievement is the most important step that one can take in creating an effective working environment. Programmers dread elaborate process, endless meetings, and layers of marketing approval before a product can be shipped. Ideally it would be possible to conceive a product on Friday evening, set up the development environment Friday night, write code on Saturday and Sunday, test on Sunday night, and ship on Monday morning. Maintaining this kind of freedom is a serious challenge as a company grows and its products become more complex. Successful companies such as Oracle Corporation burden their marketing departments with overlapping products rather than stifle programmer initiative. For example, during most of the late 1990s there were at least three different Web servers that you could buy from Oracle, each one backed up by a document explaining why it was the one true path toward database-backed Web site glory.

A good physical working environment is essential. Great programmers get a lot of positive reinforcement from their work itself. They write some code and immediately can see it dance. That keeps them at work for hours that, while they would not impress a taxi driver in Singapore or a factory worker in Guangzhou, will surprise many American business people. When we hired an architect to lay out the interior of ArsDigita's first building in Cambridge he surveyed the programmers and came back shaking his head: "I've never seen any group of people who spend so many hours continuously sitting at their desks."

From a business point of view, long hours by programmers are a key to profitability. A programmer probably needs to spend 25 hours per week getting coordinated with other programmers and comprehending the structures of the systems being extended. Thus a programmer who works 55 hours per week is twice as productive as one who works 40 hours per week. In The Mythical Man-Month, the only great book ever written on software engineering, Fred Brooks concludes that no software product should be designed by more than two people. He argues that a program designed by more than two people might be more complete but it will never be easy to understand because it will not be as consistent as something designed by fewer people. This means that if you want to follow the best practices of the industry in terms of design and architecture, the only way to improve speed to market is to have the same people working longer hours. Finally there is the common sense notion that the smaller the team the less management overhead. A product is going to get out the door much faster if it is built by 4 people working 70-hour weeks (180 productive programmer-hours per week, after subtracting for 25 hours of coordination and structure comprehension time) than if by 12 people working 40-hour weeks (the same net of 180 hours per week). The 12-person team will inevitably require additional managers and all-day meetings to stay coordinated.

Your business success will depend on the extent to which programmers essentially live at your office. For this to be a common choice, your office had better be nicer than the average programmer's home. There are two ways to achieve this result. One is to hire programmers who live in extremely shabby apartments. The other is to create a nice office. Microsoft understands this. In the early 1990s they did radio spots with John Cleese as a spokesman. One of the main points of the ad was to ridicule the cheap open-plan offices in which programmers were traditionally housed and promote the fact that at Microsoft each developer gets a plush personal office.

How can an office be nicer than one's home? Let's consider the following dimensions:

• social
• physical comfort
• aesthetic
• entertainment
• attractive

Social It is easy for an office to beat the home on the social dimension, especially if the programmer lives alone. If there are people at work at all times of day and night and you've succeeded in building an organization of good people, ipso facto there is always someone interesting to talk to at the office. To exploit fully the social possibilities of the programmers' office, it is important to have informal gathering spaces. At the MIT Artificial Intelligence Laboratory, which has nurtured groups of great programmers for nearly 40 years, the gathering spaces are referred to as "playrooms". These contain sofas and coffee tables, movable in the best tradition of Dewey, where programmers eat, talk, and occasionally listen to presentations. Usually a playroom will have some sort of shared writing surface such as a whiteboard. Note that these playrooms also are an important part of an organization's knowledge management system. You need to give programmers from different projects a place to meet where problems can be discussed and solutions from older/other projects can be suggested.

A social place will never be friendly if it is trapped behind a high wall of security. It ought to be very easy for a programmer to invite a friend over. If programmers are comfortable meeting their friends at the office it greatly increases the likelihood of friends recruiting friends.

An open office plan contributes to making the work environment stronger on the social dimension. Physical Comfort A programmer's work environment should be a supremely comfortable place to sit, look at information on a screen, and type. At ArsDigita we accomplish this via providing Aeron chairs, the keyboard of the programmer's choice, and at least two monitors. In the summer, the place should air-conditioned 24 hours per day, 7 days per week. In the winter, the office should be heated and humidified (often neglected). The air should be cleaned year-round with high-efficiency mechanical filters and electronic cleaners so that allergy sufferers are not discouraged from working.

One horrible mistake that we made was letting our architect design the workstations. Each programmer was given a 6'x2' desk, 12 square feet total. Two 21" monitors took up so much depth that there wasn't even room for a keyboard. Immediately we had to toss our monitors and get flat panels (cost about $400,000 extra). IBM had a better architect for its Santa Theresa facility: Gerald McCue. He found that each worker needed 100 square feet of dedicated space and 30 square feet of work surface. McCue also found that programmers needed noise isolation from enclosed offices or high partitions but personally we think this rule is worth breaking in a dotcom world where a team has to work fast and in sync. Better to manage noise by spreading desks apart a bit so that there are fewer programmers in a given area and therefore fewer conversations, fewer telephones, and more opportunities for sound to be absorbed before reaching someone's ear. Aesthetic Programmers don't have the same need for wood-paneled expensive plushness that, say, corporate lawyers or investment bankers might. However, the office has to be aesthetically satisfying or it will be tough for anyone to take seriously the idea that the company values aesthetic internal design of computer programs. Similarly, the office has to be finished and well-executed or nobody will believe that the organization is committed to finishing products. In the long run it is impossible for an organization to be excellent in one area and mediocre in all others. So the physical facilities have to look as though they were planned and decorated by someone with taste. Note that this need not be expensive. You could do it with $200 desks from Ikea and a consistent set of art posters on the walls. But an expensive facility with blank walls and boxes left over from the last move screams incompetence. Remember that the overall place has to look nicer than most of the programmers' houses. Entertainment It is easy to make an office more entertaining than the average person's home. Most people have a TV at home but they don't have friends with whom to watch it. Nor will they typically have the kind of big-screen equipment that is easy for a company to acquire. In the 1980s students at the MIT Media Lab would gather on quite a few nights to watch movies from analog laserdisks, presented with a very high quality projector. After the movie was over, they'd go back to their desks and work for a few hours, something that would not have happened if they'd gone out to the movies.

The average home cannot accomodate a pinball machine. An office can. The average home can have video games, which are very popular with young programmers, but not people with whom to play. The average home cannot have a grand piano but almost any office can. Attractive A worthwhile goal is to have at least one thing that is extremely attractive about the physical enivronment for any particular prospective software engineer. Here's a possible list:

• dog-friendly policy
• grand piano
• climbing wall
• indoor garden
• aquarium
• koi pond
• exercise room with fancy machines
• pinball machine

Not everyone has a dog. Not everyone can play the piano. Not everyone wants to practice rock climbing. But by having a long list in the same building, there is a good chance that at least one item will be very attractive to a particular person. If a person loves gardens, he or she can be seated at a desk within view of the garden. That person won't value the other items, perhaps, but another employee will. Change of Venue You can work on all of the preceding dimensions but there will come a day when a programmer gets restless. Sitting at exactly the same desk every day is tedious. We thought that we could solve this at ArsDigita by using the Internet and our branch offices. We'd encourage programmers from Cambridge to pick up and work at a spare desk in the Berkeley or Pasadena offices for a week or two. The idea did not catch on, however, because it turns out to be disruptive for one person on a team to disappear. One of the reasons we've found open-plan offices effective is that it helps to have one's team members close enough to look casually at what is on the screen.

What does it take to let the entire team pick up and work somewhere else for awhile? A beach house or a ski house within a two-hour drive of their main office. It is kind of expensive for an individual to rent a vacation house year-around, equip it with a DSL line or cable modem, and pack it with enough desks and computers for a team to work. But if you've got a group of 30 programmers and get a house large enough for 6 or so to sleep and work, the cost is manageable. In the winter, a programming team can disappear for a week, ski every morning and work all afternoon and evening. In the summer, a team can spend a week looking out at the ocean... while typing most of the time. It costs more than not having the beach house but a lot less than having employees go off on their own to have fun every weekend and not work. Turning average programmers into good programmers It is difficult to hire the most productive programmers in the world. Oftentimes these people are capable, by themselves, of turning out entire products, and thus they start their own companies. If a really productive programmer works for an established organization, that organization will usually take extreme steps to keep him or her. Thus beyond a certain point it is most effective for an organization to develop a strategy for creating good programmers internally.

How does one create a good programmer? Raw materials are important. You want someone with a strong computer science education, a high IQ, and an ability to communicate effectively in oral presentations and in writing. But without the right experience, such a person will never be more than an average quality programmer.

These principles are important in building up someone's programming skills:

1. A person won't become proficient at something until he or she has done it many times. In other words., if you want someone to be really good at building a software system, he or she will have to have built 10 or more systems of that type.
2. A person won't retain proficiency at a task unless he or she has at one time learned to perform that task very rapidly. Learning research demonstrates that the skills of people who become accurate but not fast deteriorate much sooner than the skills of people who become both accurate and fast.
3. Technology shifts force a programmer to go through bursts of learning every year or two.

Look around your organization. You can make a list of the people qualified to design and build a new system by counting up those who've built 10 or more similar systems in the past, at least two in the last year, and that could do the entire job in a month or two if they really had to. These are your "good programmers". Everyone else is a candidate to be turned into a good programmer as quickly as possible.

Learning to design and build software systems requires that the programmer design and build software systems. These can be smaller subprojects for internal or external customers, standalone software system for non-profit organizations, or demonstration systems to be written up and distributed to other programmers. A particularly effective option that is only available in the Web world is to build and launch a free public service. See http://remindme.arsdigita.com and http://towzone.arsdigita.com for examples of one-evening training projects.

Whatever the training task, the pace must be ruthlessly brisk. The learner should be expected to build at the same pace as an experienced developer. The difference between the learner and the wizard is that you expect the learner to make a lot of mistakes. The system as built may be awkward or not handle error cases properly. That's okay. Training research shows that if you get speed now you can get quality later. But if you don't get speed you will never get quality in the long run. We practice this technique in 6.916, Software Engineering for Web Applications, our course at MIT. Each student builds five database-backed Web applications during the 13-week semester. The first few that they build, during the course of the problem sets, are not necessarily elegant or optimal, but by the end of the semester they've become remarkably proficient, especially when you consider that each student is taking three or four other classes.

If you see one of your best people walking out the door at 6:00 pm, try to think why you haven't challenged that person with an interesting project. If you see one of your average programmers walking out the door at 6:00 pm, recognize that this person is not developing into a good programmer. An average programmer's productivity will never be significant in a group of good programmers. If you care about profits, you must either come up with a new training program for the person or figure out the best way to terminate his or her employment with your organization.

Still not convinced? Take a look at the Japanese "code factory" circa 1990. These precisely organized large organizations where each person had his role, however small, were supposed to overtake the American approach where small teams of craftsmen worked in a comparatively disorganized manner. The factory approach sometimes produces acceptable corporate IT solutions but for innovation and successful product development, the craft approach has been overwhelmingly vindicated. Turning good programmers into good managers As noted in the introduction, software engineering is different because the organization can't afford to lose the individual productivity of the best people as they are pushed into management. At ArsDigita, for example, a manager who is one or two levels up from the leaf nodes is still expected to write code, develop SQL data models, write system design documents, and write journal articles. Yet managers who are spending a portion of their time designing software or writing documentation are at risk of neglecting their duties to review subordinates' work.

The classic problem situation at ArsDigita is a manager getting lost in his or her own work and failing to review a subordinate's efforts for two or three months. When the review occurs, inevitably the subordinate has either been working on the wrong thing in the wrong way or hasn't been sufficiently productive. At this point the manager is really angry. Three months of calendar time and money have been wasted. But should the manager be angry with the employee? If the manager had reviewed the subordinate every week, the company would never have been at risk of losing more than one week of time and money.

Our solution is to decouple responsibility for review from responsibility for scheduling review. We use administrative assistants to ensure that each manager is scheduled to look at every subordinate's work at least once per week, more frequently in the case of junior employees. It has proven remarkably more effective when a neutral third-party is responsible for scheduling than when people with incentives to shirk are responsible for scheduling. Management by Consensus Considered Harmful Leaf-node engineers at every company on this planet think that they have better business ideas than the senior managers. Why not simply turn the company over to the engineers to run? Each engineer has a different set of better business ideas.

Software engineering companies will tend to have a fairly flat distribution of intelligence. The 22-year-old Stanford CS punk that was just hired will be just as smart as the 30-year-old lead engineer who will be just as smart as the 40-year-old CEO. Within a company's technical team, the raw IQ differences are even smaller. If each member of the team were playing the Bach Partitas and Sonatas for Solo Violin, the wrong notes, shaky intonation, and bad phrasing would make it pretty obvious to the novices that they needed to take advice from the experts. But because software quality is tough to measure and software quantity is seldom measured, the novices in a software engineering group are able to think of themselves as experts.

What would be wrong with a completely egalitarian software engineering group? Maybe the entire team really is at the same level of ability. And suppose that somehow the challenge of getting everyone to attack the same problem had been surmounted. Remember what Fred Brooks said in The Mythical Man-Month: high quality systems must be architected by no more than two people.

Getting design input from leaf-node engineers is important for having a good product design. But at the end of the day nobody should be confused as to who is providing leadership. There is an irreducible amount of Engineer A imposing his or her design on Engineer B. This can lead to some harsh-sounding words and bruised feelings. Microsoft is not the self-esteem company, at least if you believe Playboy magazine's interview with Bill Gates: "We hear you're brusque at times, that you won't hesitate to tell someone their idea is the stupidest thing you've ever heard. It's been called management by embarrassment challenging employees and even leaving some in tears." Truly elite organizations can be far worse than Microsoft. Ask a group of surgical interns and residents how much respect they get from the surgeons. Go into a world-class biology department and ask the grad students and post-docs about their treatment at the hands of the professors. Wherever You Go, There You Are Performance management textbooks will tell you that workers don't improve unless they get feedback. Joe Widgetmaker should get a nice chart, updated daily, of how many widgets he has produced personally each day, and how many have been built by his team.

Consider the average working programmer's life:

• surfing USENET
• surfing Slashdot
• reading docs
• questioning colleagues
• writing specs and designs
• writing docs
• writing code
• testing code
• fixing bugs
• filing bug reports on others' code
• attending meetings
• helping sales and marketing staff

(For comparison to the grad student life, see http://philip.greenspun.com/humor/graduate-student -emotion-check-list.)

Characterizing this person's productivity is going to require more than one number. But if we don't do it, days or weeks could slip by without the programmer realizing that his or her achievement levels are plummeting. In a company with disorganized or technically clueless managers, the programmer's supervisory chain won't notice the lack of achievement either.

Production of documentation and code is generally measurable by reference to the company's version control system. Bugs filed and fixed are easily tallied by looking at the company's ticket/bug tracking system (see http://www.arsdigita.com/doc/ticket for a description of our favorite open-source ticket/bug tracker). The softer stuff can still be quantified but it will have to be done by humans filling out forms.

Ideally the programmer will get daily feedback, which is kept private unless the individual elects to publicize it. Performance in each sanctioned area of activity will be marked up and scored with a weight. The programmer can then see if his or her crude achievement level is going up or down. Summary Building and managing a peak-performing software engineering organization is challenging but extremely profitable. The core Ariba product was written by two programmers, yielding a market capitalization of $30 billion. Microsoft Internet Explorer is a much better browser than Netscape Navigator and yet it was written by a much smaller team: only about 30 developers.

Start by attracting a good core team, perhaps by setting up an organization that enables each engineer to excel along the axes defined in http://www.arsdigita.com/asj/professionalism. Provide a productive working environment and a physical environment that is better than the average programmer's house. Provide daily positive reinforcement. Provide daily feedback showing the programmer more or less exactly what he or she has accomplished, plus a graph for the preceding few months showing the trend. Aim to install a feeling of ownership in each worker. More

• Bringing out the Best in People (Aubrey Daniels 1999; McGraw Hill). 200 pages containing 75 pages of good ideas plus the usual business book filler. But the ideas are genuinely good.
• The Mythical Man-Month (Fred Brooks 1995)
• Managing the Professional Service Firm (David Maister 1993). In terms of having an equal distribution of ability among all levels of the enterprise, the closest industry to software engineering is management consulting. Maister's work is a classic guide to success in this industry.
• Peopleware (Tom DeMarco and Timothy Lister 1999); page 98 is worth the price of admission, explaining that "the term unprofessional is often used to characterize surprising and threatening behavior. ... In a healthier organization culture, people are thought to be professional to the extent they are knowledgeable and competent." (See http://www.arsdigita.com/asj/professionalism for ArsDigita's independent conception of this idea.) Much of the rest of the book is a celebration of the 40-hour work week and the claim that "overtime" in the long run is never beneficial. If the authors were correct, Silicon Valley would be the poorest region of the nation, with Redmond, Washington the 2nd most impoverished. And Washington, DC would be our great source of innovation and productivity. Peopleware was probably written to help ensure success for internal corporate IT projects where there isn't any competition and delivering three months late won't change much.
• Making the Cisco Connection : The Story Behind the Real Internet Superpower (Bunnel et al 2000) -- shows how ignoring the "no overtime" admonitions in Peopleware can generate $400 billion in market cap.
• Parkinson's Law (C. Northcote Parkinson 1958) -- how management really works in the long run
• A Pattern Language (Christopher Alexander et al 1977) has very interesting things to say about physical space and social organization.

asj-editors@arsdigita.com

Reader's Comments

A corollary to this principle is that people do what you reward them to do, not what you hope they will do. Often, when you look at what is truly rewarded in an organization, you find it is different than what you think is rewarded.

This is illustrated by the classic business school article:

Kerr, S., 1975, "On the Folly of Rewarding A, While Hoping For B," Academy of Management Journal, Vol. 18, pp. 769-783.

It represents exactly what you are saying.

-- Tyler Pruett, October 30, 2000

Philip, your ideas on rewarding programmers are excellent. However, I don't believe number of hours worked defines a good developer. Programmers get older and have families. Saying that they have to log a certain number of hours in the office is stupid, almost like IBM counting programmer productivity in "K-Locks" (thousands of lines of code).
A good developer can often finish what an average one can do in half the time an average developer can - with less bugs and with better documentation.

A couple of good books on this topic include,
"Software Project Survival Guide by Steve McConnell"
(Somewhat dry - but good reference book)
First, Break All the Rules : What the World's Greatest Managers Do Differently
(Not one of the typical management books - this one has some actual research behind it.)
Microsoft Secrets : How the World's Most Powerful Software Company Creates Technology, Shapes Markets, and Manages People
(A few good chapters worth reading)

-- Anthony Barker, November 1, 2000

Not to mention that programmers who are measured based on how late they stay at work will quickly figure out how to excel at staying late without being more productive (e.g. come in at 11:00 am).

This article reinforces many great ideas from Frederick Brooks but, unfortunately, regurgitates a few tired stereotypes. Most programmers that turn 30, get married, have a few kids (that go to bed at 7:00), may as well resign themselves to being put out to pasture.

Here's an idea. Instead of just assuming that it takes 25 hours a week for team coordination and build your management approach upon motivating programmers to work more hours, why not work to reduce the time it takes to coordinate through better management and more efficiency? Isn't that what management really is?

-- Tom Wilson, November 2, 2000

Dog-friendly offices may be great for dog owners / lovers, but aren't as wonderful for people with allergies or even those who just want to be able to concentrate on writing code without having a dog try to jump on their lap.

-- Kevin Scaldeferri, November 2, 2000

Add a comment | Add a link

Re:NT/2k uptime, was Re:I smell money...

On: MS To Virginia Beach: Prove You Own Your Software

That's net statistics server

or perhaps net statistics workstation

depending on what you have running

They don't have to switch exclusively

On: Gartner Group Squints At Future OS Growth

At the web development company where I work, desktops are completely heterogenous depending on the responsibilities, skills and personal preferences of the employee. We have a lot of people who prefer MacOS: graphical designers and business people. They don't want to switch to Windows because they love their Macs. (Business and designer folks also use NT, W2K, Win98 and WinME; tech folks use W2K, NT, Linux, and free BSD desktops.)

We are going to be delighted to get OS X in front of those Mac users. It will be so much easier to integrate them into our primarily *nix based technical infrastructure when they are running OS X, a BSD. A more robust MacOS that integrates better with our other systems is just right for those Mac users. We would never switch to any single OS as our required desktop because it would hurt the productivity and satisfaction of our employees. For us, MacOS is certainly competition for Windows, and OS X will be better competition.

Re:NIC...Good Idea?

On: Slashback: Rumination, Apologies, Kisses

What's this about a small set of repetitive tasks? This system is powerful enough for full remote access to hosts, and that means you can run anything you want.

Re:Uhm...why? WHY NOT READ THE STORY?!?!?

On: The Internet For Parrots

Mod up. This is not flamebait, but rather funny as hell, and also the "not news that matters" post which this story so richly deserves.

Re:That's not quite true...

On: Can Open Source Be Trusted?

The owner could sell you a source license without redistribution rights.

Re:state of things

On: Slashback: Interoperability, Royalty, Fire

I have seen plural acronyms using apostrophes in what should be well-edited works. It makes me ill. There is a nice simple rule for English: apostrophes are for possessives and contractions only. I think we should stick with that for plural acronyms and not use an apostrophe.

Re:Do Oracle developers know their source code?

On: PostgreSQL - Oracle/DB2 Killer?

... which derives from Gres, another academic database ...

tragedy of the commons

On: Open Source Napster: Gnutella

One student who sees fit to use napster will consume the bandwidth of ten web surfers/telnet users/email users. But the consequences of this network use are felt equally by all users. We have a classic tragedy of the commons. Users overuse a shared resource because there is no feedback mechanism.

Admins have to do something because otherwise napster users will cripple their networks. Blocking napster seems like a good way to go to me.

Heavy network use for piracy has always been against acceptable use at universities and this is just a very consistent and logical continuation of that correct policy. University networks are a shared and scarce resource. I like pirated mp3s as much as the next person but a university network is the wrong place for it.

Oct 1998 Wired story on Iridium

On: Iridium Hardware May Burn

Wired Iridium story

"After millennia of people looking up at the same night sky, we're the first to put up a new constellation since God. It's never going to be the same again."

ho'in

On: FreeBSD 4.0 Released

www.kame.net

Quasar Stupid (excerpt)

On: Alias|Wavefront Ships Linux Software

I cannot believe how incredibly stupid you are. I mean rock-hard stupid. Dehydrated-rock-hard stupid. Stupid so stupid that it goes way beyond the stupid we know into a whole different dimension of stupid. You are trans-stupid stupid. Meta-stupid. Stupid collapsed on itself so far that even the neutrons have collapsed. Stupid gotten so dense that no intellect can escape. Singularity stupid. Blazing hot mid-day sun on Mercury stupid. You emit more stupid in one second than our entire galaxy emits in a year. Quasar stupid. Your writing has to be a troll. Nothing in our universe can really be this stupid. Perhaps this is some primordial fragment from the original big bang of stupid. Some pure essence of a stupid so uncontaminated by anything else as to be beyond the laws of physics that we know. I'm sorry. I can't go on. This is an epiphany of stupid for me. After this, you may not hear from me again for a while. I don't have enough strength left to deride your ignorant questions and half baked comments about unimportant trivia, or any of the rest of this drivel. Duh.

Re:This already exists...

On: Jakob Nielsen Answers Usability Questions

Too true! And Opera lets you toggle image loading, also with a single keystroke. Add in an ad-filter and the web becomes darn near usable.

Re:WOW! But where does the stupidity come from?

On: Jakob Nielsen Answers Usability Questions

Yes! Thank you! This is exactly the reason. Web developers in many companies are not betting on whether the site will be a success. They are betting on whether the idiot client will like it. Launch 'em and leave 'em. After the check clears it really doesn't matter if the site never succeeds.

Many clients don't even use the web much and very few of them give a damn about page load times. They want to see something pretty and that's it.

Re:Minimalist design

On: Jakob Nielsen Answers Usability Questions

Your post was thought-provoking. I am also a Nielsen fan and I loved his book. I think most of your points are good ones. I would like to criticize some of them though.

I think he really downplays the importance of graphics on websites, which is the main reason why the web is popular in the first place. Sure, content is king, but surfers want to experience attractive websites.

Most of the graphics people see and wait for on the web every day have zero information content. They are just antialiased text and design flourishes. I think you're right to say that no graphics makes a web site dull for the average web reader. But web designers go far beyond what is useful and bulk up their pages with large navigational and decorational graphics. These kill usability by killing response time, as Nielsen says.

the expectations for a particular website isn't the same for each audience

If you want to follow in the segment marketing footsteps of GM and have ten different websites, targeting the same product to ten different demographic types, more power to you. I am not aware of a single commercial web venture that has gone this way. You would need several times as many staff people, or your sites would have only a fraction of the functionality of a unified competitor spending the same money.

I'm sure that there is space in the marketplace for more marginal aesthetics and home grown, non revenue generating sites are the ones who are best positioned to capture that space. Sites with investors would be ill advised to go this route.

The "big" sites maintain this advantage now through massive amounts of advertising. That's what big sites have in common these days.

I'm sorry but Nielsen is right and you are wrong on this. The big sites are could not be successful without their fast load times and generally good usability.

Advertising sucks. It doesn't work all that well. No ad will make somebody come back to a site that they didn't like. But good sites get marketing for free via users talking to each other.

Advertising is about launching new sites and selling yourself to new web users. They will visit the site once and that is when you have to give them something they can understand and use. All the very popular sites do great with their old, repeat customers. Having a fast (therefore spare) design is essential to getting people to return.

Re:Dumb Question re: XML

On: Mac OS X, XML, and Aqua

to quote the sun xml tutorial

How Can You Use XML?
There are several basic ways to make use of XML:

• Traditional data processing, where XML encodes the data for a program to process
• Document-driven programming, where XML documents are containers that build interfaces and applications from existing components
• Archiving -- the foundation for document-driven programming, where the customized version of a component is saved (archived) so it can be used later
• Binding, where the DTD or schema that defines an XML data structure is used to automatically generate a significant portion of the application that will eventually process that data

MS kbds are not that expensive

On: Ergonomic Keyboards

wvccsi sells MS Natural Elite for $25. egghead sells MS Natural Pro for $40.

perl is not what it is claimed to be

On: Perl vs. Python: A Culture Comparison

I like and use Perl plenty. But Perl enthusiasts love to paint it as a language for all users and all purposes, when it really more demanding and specialized at heart.

IMO, Perl has a well defined role as a powerful scripting language for users who know the Unix environment. Perl is an awk-sed-sh synthesis with a C syntax. I am a big awk and sed fan so it pleases me just fine to use perl for the same things I would use awk or sed for: (regex based) text and flat file db processing. But to build large systems with large numbers of developers in this kitchen sink of a language is stupid. It is so big, so cryptic, that unless everyone on the team is a $150/hr Perl god, you are going to wind up with a mess as junior people produce unreadable, wordy crap. You have to be some kind of Talmudic scholar to be able to read and write all common styles of Perl.

Perl does not have anything resembling a reasonable overall syntax or semantics. Is there any such thing as a truly correct syntax-aware editor for perl? When emacs can't parse a language you are in trouble.

The Perl community has some funny sayings. "Baby talk is OK." That's because unless you are coming from the right C-sh-awk-sed background you are guaranteed to misuse the language.

"TMTOWTDI." Like my and local? ' and ::? Dereferencing with %@ vs. ->[]{}? Or all the myriad ways to structure a simple object? Ack! Too many ways to do it for code to be readable.

The perl interpreter is really weird and crazy under the hood, which doesn't make the language any more attractive for high reliability work.

Perl is a Unix hacker language, with most of the same mysteries and ways to shoot yourself in the foot as sh and friends. Mere mortals should use something with more rules and fewer options.

Re:Dying? It's dead already...

On: Is Usenet Dying?

Trust would also be necessary to prevent the bad guys from hijacking the system. I am having trouble thinking of how you get moderation by the masses, without a central authority to grant and certify moderator keys.