Slashdot Mirror
Quality Control In Computer Companies
Ant sent us a
Salon feature that talks about the (lack of) quality control in computer manufacturing, and then talks about it being "the American way of techno-capitalism". I've not had nearly the problems that people in this article allege, but I can sympathize.
Since we started doing this, our defect rate has decreased dramatically. The difference between before having the process and after having it is like night and day. Since we started using a good process, I have seldom worked overtime. Before we had the process, we would be working 60 to 80 hour weeks continuously, but the team as a whole was less productive than it is now.
I don't feel my "creativity" has been stifled. My creativity is now expressed elsewhere. Design is now heavily peer reviewed. Coding is reduced largely to a mechanical implementation of the design, and that's the way it should be. (And the code is peer reviewed, too).
To use a horrible business buzzword, it's a "win-win" situation. Our staff turnover is very low. I've been on the project for five years now, and I'm still happy. Had we carried on as we had before instituting the strict process that we have done, I doubt I would have stayed.
Oolite: Elite-like game. For Mac, Linux and Windows
We already had a big software disaster - remember Y2K? Arguably one of the largest engineering "projects" in history, and its seems we got through it with nary a scratch. And who is most suprised of all... the engineers themselves. Most of us geeks expected at least a little bit of hell to break loose when the clock struck midnight on January 1st, even though we were the ones who apparently fixed the problem. I'm utterly shocked that things went as smoothly as they did, considering most Y2K repair work was probably horribly hacky. And have we matured much because of it? Nope.....
---- I made the Kessel Run in under 11 parsecs.
You're right, but that's not necessarily the problem. Think about Windows 3.1 - its been around for what, 8 years now, so it must be nearly perfect, right? The problem is that the development focus in the tech industry is on the bleeding edge, and stuff thats more than one year or so off the bleeding edge gets virtually no attention. Sure, there are exceptions (the 2.1.x series Linux kernel comes to mind - Alan Cox is still releasing new versions, and even though the series in ~5 years old), but on the whole, the industry tends to lose interest in older tech, because the money is on the bleeding edge.
---- I made the Kessel Run in under 11 parsecs.
What I find more insidious, however, are the legal intricacies computer (and software) manufacturers employ to shield themselves from responsibility.
Anyone remember Hill vs. Gateway 2000?. In this case (I'm going from memory here), Gateway offered a 10th Anniversary Special computer to consumers that wasn't all it was advertised to be -- i.e. Gateway's ads said the speakers were "surround sound" and they weren't.
Mr. Hill took Gateway to court, and discovered, to his surprise, that the shrinkwrapped EULA inside the computer box prevented him from suing the company, regardless of their bad faith. Instead (according to the EULA) he was forced to submit to arbitration, which inherently negated any class action status for the thousands of other consumers who were blatantly defrauded by Gateway's false claims. Furthermore (due to the nature of arbitration), the verdict was kept secret, preventing anyone else who had been ripped off from benefiting from the arbiter's decision.
Sincerely,
Vergil
Vergil Bushnell
Insects and Grafitti Photos
I'd say on a typical HP or Compaq, you get as much performance boost from the re-install as you would doubling your RAM... which may explain why they're loaded that way
11*43+456^2
Quality control is expensive. Just slap on a warranty and let the user test it for you.
... are not designed to work around quality control.
If you work in software, you want non-obvious bugs that are eventually found and require the $59.95 (or higher) upgrade version, or require all new purchases. These keep your revenue stream coming in for a long, long time, and let you technically get away with calling your release a 'new product' without actually changing many of the features (See Microsoft Office).
If hardware is your game, you want your hardware to cease to function after two to three years, right after the warranty runs out. That way, the customer has to take their computer into Ma and Pa's computer repair, and Ma and/or Pa has to try to fix it, and it turns out that the new video card for your computer needs a different bus, so you need a new motherboard, so you need a new processor, and you need faster ram, etc, and in the end you are screwed. Everyone in the industry benefits from everyone else making shoddy hardware. The customer is mad in the end, but what can (s)he do about it?
I worked field/bench service for 2 1/2 years, and had way too many of these types of problems to deal with. PCChips, Amptron, the now somewhat defunct Packard Bell, HP (pavilion series), Compaq (Presario series), and Acers were the biggest offenders. The Amptron and PCChips boards (which are from the same manufacturer) were the ones that we carried, because unless the customer asked for better, they wouldn't know the difference and sooner or later they'd be back. If they went to someone else because our hardware died, well, the chances were likely that a customer of another shop would come to us. Everyone juggled everyone else's customers, and in the end we all made a marginal profit off of screwing people perpetually.
I refuse to buy components that I don't have experience with now. I work in a QA department, with mostly industrial PCs, and of all that caught me off guard, the cheap SIS AGP video cards are actually pretty damn good for 2d uses. The obvious ones like ASUS and ABIT and 3Com and such stand out, and we get to really pound stuff. If you really want to know what to put in your PC, don't ask your friendly neighbourhood service technician, ask someone who works QA with servers and high end workstations. That way, you'll get a real answer.
"Titanic was 3hr and 17min long. They could have lost 3hr and 17min from that."
IBM had PL/1, with syntax worse than JOSS,
And everywhere the language went, it was a total loss...
I don't agree, a computer accomplishes one task, which is to run programs, following your line of reasoning, a refrigerator accomplishes hundreds of tasks just because it happens to store hundreds of different foods...
It is interesting that the main objection that comes up when there are talks about Quality in software is that computer programs are too complicated, well, building a skyscraper is IMHO just as complicated, but if the Empire State Building falls down, you can't just release Empire State Building Service Pack 2, can you ?
IMHO the main problem is that the discipline of creating computer programs is still very 'new' compared to most of the others (architecture etc.) and after it will mature a bit more, everything will be just fine.
Many (bad) programmers complain that QA stifles their creativity, now I wonder how many city planners would use the same excuse (no, really, multiplexing sewage with water in the same pipes is better, since it will take up less space. What do you mean I can't do that ? You are infringing on my creativity !)
-- the cake is a lie
until the buggy software actually kills someone. It is unfortunate but true that engineering disasters forced engineering fields to mature. But as software is pushed into more and more life critical roles (medical, aeronautical, automotive)
Software Engineering as a discipline has to be taken more seriously and good engineering practices as well. There is a big difference between a carpenter and a Civil Engineer, just as there is a huge diference between a programmer (or coder) and a Software Engineer.
I sincerely hope India does drive many US software out of business. Only through pain will the lesson s be learned.
putting the 'B' in LGBTQ+
This is nonsense. Any industry that releases new generations of every more complex products every one and a half years or shorter is bound to be riddled with bugs and flaws. Flaws are especially prevalent if the product uses lots of untested new technologies.
Japanese perfection? To us it may look that way, but for many of brand spanking new products released in Japan, they are hardly error free. Often before a Japanese company sells to the foreign market, they will release it domestically for over a year to try to work out the bugs. An example: The Playstation 2 release in Japan was full of flaws. By the time a Japanese product gets to North America or Europe engineers will have found and fixed most of the most glaring errors in the design and manufacturing process.
Generally every new technology will have flaws as everyone gets used to the new tech and while we refine the manufacturing process from the lab to the factory floor, such as 0.13-micron lithography. Today >1GHz CPUs have small yields, in a year the yields and reliability of these chips will be far higher as the technicians tweak the manufacturing product. Just the same way as we can build a 100 MHz today with almost 100% yield.
The bottom line is: If you want perfection and something that is incredibly reliable, you as a rule cannot get bleeding edge equipment.
Have you patented your Hot Grits today?
You mean the strategy of "charge as much as possible and deliver as little as you can get away with?" Yeah - and the market has come to accept it, too. I can remember the 70's, when Japanese auto imports began to claim more and more of the market. People were amazed at how much car they could get, and how reliable they were, for the price. The American consumer was unused to such value.
We still are. But then again, we have access to some of the best bargains on the planet. There aren't many other places where a consumer can get as much for their dollar (or ruble or ecu or yuan) as we do. Sure, most "durable goods" aren't as durable as they should be. Maybe our electronic gizmos deliver function, but suffer flaws in the interface, utility, workmanship and/or materials. But at least we have access to them.
After the discussion of the influence Japan had on Detroit, it should come as no surprise that of all the commodities introduced since the industrial revolution, the auto, and its ancillary products, are the only things where the US consumer gets a real value for his money. Got $10k? If you shop smart, you can buy a car that will run reliably for more than 10 years, at over 25 mpg, with average annual maintenance costs of just a few hundred a year. That's a deal!
Need gas? It's the cheapest anywhere in the world except for places like Bahrain and Saudi Arabia. Auto services? At $40/year, AAA Membership is one of the best bargains in America. In very few other countries can an individual own and operate a car for such a small fraction of his income. The only better value an American will find is in pre-industrial products: food and apparel, primarily.
But consider this: In most places on this globe, people spend more of their income on sustenance than we Americans spend on our autos.
So although your thesis appears valid from an American's point of view, from a Ugandan's or Korean's perspective, it's trite at best.
I can see the fnords!
I took a class on Software Project Management from Prof. Don Remer, and one of the things he talked about was how Sony has done Six Sigma for their manufacturing and design processes. They look at why and where things break, and fix them. It's a process of continuous improvement, not just some big push to get all the bugs out of a design. You're going to miss some bugs anyway, and if you fix them afterwards, the cost is huge. Anyway, according to him, Sony no longer turns on the TVs coming off the assembly line to see if they work. Believe it or not, this is a sign of good process.
The reason is that they have put a lot of effort into making sure their development process is good and it results in a refined manufacturing process. They don't turn on the TVs because they don't have to! The failure rate is so low that it costs more to test than to fix the one in a million (give or take an order of magnitude) that don't work. Software ain't got nothin' on that.
Well, unless you count all the times software is shipped without any testing, but that's for a different reason....
Walt
Uh...
Let's take a step back here, and analyze what this article is really about. The writer had a problem with two modems and a printer driver--but bitched until she got somebody to come on-site to replace the motherboard and the entire printer. Her solution to these problems is to cite examples of Indian enthusiasm for software engineering.
There's just one little problem--software engineering has ZIP to do with the problems this not-quite-up-to-speed writer had. Her modem problems? I'd bet money she had IRQ conflicts, but the tech on the phone couldn't walk her through fixing them. The simple solution was to send a human out there--hence, replace the motherboard. The real solution: a legacy-free box that connects a modem without hardware interrupts. Her printer driver problems? Yes--there are crappy drivers. That's a marketing problem. But a printer driver problem is not solved by shipping a new printer--and for an allegedly experienced computer journalist to have to wait months for a CD-ROM to arrive with a printer driver (what--she couldn't find how to download a LaserJet driver from www.hp.com?) is simply laughable.
I'm a big believer in software engineering. But I'm also a big believer in quality journalism--and this article most definitely isn't quality journalism at all. This article essentially boils down to whining from a particularly clueless user about how she can't manage to get her computer to work. The solution she suggests--software engineering--has nothing to do with the problem. All the software engineering in the world isn't going to solve her IRQ problems with her modem--a USB port will. The best software engineering in the world may produce the best printer driver in the world--but printers will still need drivers. Updating a printer driver will always require replacing the driver--not the printer.
Bottom line: the article is a waste of time.
I've worked for Apple, Compaq, and IBM authorized warranty repair centers over the years. I have to say that Compaq is the worst. It has even gotten to the point when someone will call and say "I have a Compaq that needs warranty service." and we'd respond "Broken Presario huh?". We were right about 90% of the time.
Apple has recently been running a close second. The new iMacs, the summer 2000 and the earlier iMac DVs were plagued by power problems. The early iMac DV has a design flaw which causes the power button to become stuck so that it can't reliably be powered up when it's pushed. The summer 2000 models have been blowing power/analog boards like nobody's business. I have personally done more power/analog board replacements than I care to remember. Lastly the PMG4s have a problem with the front panel board. This can appear to be a power supple problem, you put in a new power supply and guess what? Problem still exists. A $0.50 board is defective in lots of these machines.
Apple & Compaq have to know that there is a problem with these machines, but still they do nothing about it. Instead of going through the expense of a recall, they just fix them as the blow out. I think that's shitty. Some people depend on these things to make a living and when your machine is down for a week because of a flaw that they know about, Apple & Compaq abdicate any responsibility for the lost income.
I think that's dirty, they shouldn't treat customers like that.
LK
"Hi. This is my friend, Jack Shit, and you don't know him." - Lord Kano
In 1996, I bought an extremely overpriced Acer Aspire from Future Shop in Moncton, NB, Canada.
The salesman told me that the New Cyrix 6x86's were great, so I bought it, and took it home. (mistake number 1, although, I never had trouble with the chip, other than being extremely slow).
Pulled the tower out of the styro-foam, and heard "clunk". Sounded like an ISA card was loose or something. Took the machine back to Future Shop, and they gave me another machine. Took that machine home, BSOD'd on the first boot, and I kept getting BSODs. Called Acer tech support, waited for 45 minutes on hold, got a tech who barely spoke english, explained the problem, and he declared that the RAM in my machine was bad. Gave me a number to call to have it replaced. Called the number, the repair shop told me that it would be up to 2 weeks before I could have the machine I've never used fixed, so I took it back to Future Shop, and had it replaced after arguing with the sales manager.
So, I'm on machine 3, which is working alright, but I notice that the hard drive is incredibly slow, so next time I'm in the store (a week later), I mention that to the salesman, and he tells me "oh yeah, the Acer techs were in here earlier this week, and they did some stuff to the hard drives. Bring your machine in, and I'll switch you for a good one." Machine number 4.
6 months later, my CDRom fails.. I sold the machine after having the CDRom fixed.
Needless to say, I think twice before buying Acer, now.
"techno-capitalism" is just the current manifestation of the way American capitalism has always worked. I mean this quite literally, it's been this way all the way back to the beginning of the industrial revolution.
The best example I have is from a History of Science class I took involved railroads. Americans would build many miles of track per day, at the expense of quality, as the tracks would break often, and sometimes they even ran them on top of the snow(!). When it would melt, the track would have to be rebuilt. The British, on the other hand, took their time and built tracks that would last.
In the modern day things are even worse, because of broadcast technologies that gave rise to the advertising industry. Now companies waste money convincing you that you need product x, regardless of how good product x is and if you even need it. They do a good job of convincing, too. It has brought our economic system to a new low, in my opinion, as far as the effect on our society.
Whether or not this additude will harm the US in the long run remains to be seen. Mentioning India as a threat seems to be a stretch right now, they have a pretty limited industrial base as far as I know. Japan and Germany seem to be the obvious threats (personally I view them as healthy competition). The focus in Germany is on making a good product at the possible expense of profit. In America it's the other way around. Japan seems to have a better balance somewhere in the middle, although that doesn't explain the recession/slow growth they've had over the past 11 years.
Hopefully competition from other coutries can "keep it real" for american corps as Japan did for the automakers.
What I worry about the most is receiving a product only to have it recalled. If a firm made sure to begin with that their product was in good working order (under all conditions) it would save them money in the long run. I know that sometimes bugs only pop up after months of use, and that should also be tested. I recently bought a machine that had a bad logic board. This had been a known issue. After two months of use, the board had a circuit that fried. It took two weeks to get the machine back. I'm sure that the company doesn't like having to pay for warrantied work. More extensive tests would ease some of these problems. Is there really no way to ensure more quality yet still be "first to market"? I know it's a cut-throat industry, but customer satisfaction should be the ultimate goal.
From the article:
In analysing repair histories of 13 kinds of products gathered by Consumer Reports, PC World found that roughly 22 percent of computers break down every year -- compared to 9 percent of VCRs, 7 percent of big-screen TVs, 7 percent of clothes dryers and 8 percent of refrigerators.
A computer is something that accomplishes 100's of task so natrually it would need more maintanance than products that only accomplish one. The author seems to overlook this entirely. Not sure if there is any point to reading the article any further after a statement like that.
One thing that bothered me deeply about the article is that the author made very little distinction between hardware and software quality problems. I realize that this probably reflects the majority, non-tech view of computers (the computer is a unit that succeeds or fails as a unit, not as hardware or software) but it made the article less comprehensible. Most of the specific problems that he talked about sound as though they were hardware problems, but the experts he consulted were talking about curing software reliability. This is probably reasonable, since reliable, high quality hardware is available, and the companies that produce junk often go out of business when people stop buying their crap. Of course high quality, reliable software is available too, but most desktop PCs don't use it. I'm inclined to agree with the closing statement of the article: we won't get high quality software until companies suffer financially from putting out crappy software.
There's no point in questioning authority if you aren't going to listen to the answers.
This critic misses the whole point. There are places for creativity. Two good examples are the R&D department and process review and improvement meetings. But creativity is not necessary when crafting a quality, reliable product. In fact, it gets in the way of reliability.
Now, I know this isn't going to be a popular argument on Slashdot, but sometimes good medicine tastes bad. Consider it this way: how would the auto industry appear if autoworkers felt it was their perogative to "be creative" while doing assembly line work? This is exactly what programmers act like. The smart auto companies give their line workers time to be creative, during process review, so that they can concentrate on quality the rest of the time.
I know, the analogy between autos and computers breaks down quickly, and it breaks down pretty damn early with software. But there's still a point to be made. Many programmers (and engineers), especially young ones, are too eager to be doing the "elite" work, that they don't pay attention to detail. They want to go straight to designing better suspensions instead of just installing the struts. (I know. I've been there.) But it builds character to do the rote, mundane work - you learn how to check your work as you do it, and fix errors as they are made, so that you or someone else doesn't have to come back and fix it later. This talent is especially necessary in programming.
Perfect example: If your attention is focused on writing the slickest, most 31337 bubble sort for the product your team is developing, you are going to introduce more errors than if you had just instantiated the algorithm you've used a hundred times before. But creativity isn't necessary here. Just implement the function that's needed. If they had needed a high performance sort function, it would have been in the requirements handed to you when you started. As the first Project Engineer I worked under used to say: "Better is the enemy of good enough."
The fact that Chennai's Advanced Information Systems company has achieved the astonishingly low 0.05 per kloc defect rate, and that 22 of the 28 companies with a SEI Level 5 cert are in India, demonstrates that the Indians understand this point, and proves the "techie reviewer" dead wrong. He sounds just like another 'leet code jockey who's whining because Humphrey's telling him he can't doodle in your POS transaction software anymore...
I can see the fnords!
If you push up the software quality, make sure it rarely (never?) crashes, and you do proper studies to make sure it works the way people expect it to work, then our economy goes bust!
- No more tech. support jobs
- No more "Word 97 for Dummies" books
- No more "Learn Windows 95 in 1 week!" books
- Much less upgrading because that word processor you purchased 5 years ago still works like it should and there are no bugs to fix, contains _only_ the features you use, so no upgrade needed
Look at the car industry. All you ever do is gas up and take in for maintenance every 12-24000km.
(Okay, unless you're one of those dummies who bought a Chevy Cavalier...)