The Truth About Last Year's Xbox 360 Recall

chrplace forwards an article in which Gartner's Brian Lewis offers his perspective on what led to last year's Xbox 360 recall. Lewis says it happened because Microsoft wanted to avoid an ASIC vendor. "Microsoft designed the graphic chip on its own, cut a traditional ASIC vendor out of the process, and went straight to Taiwan Semiconductor Manufacturing Co. Ltd., he explained. But in the end, by going cheap — hoping to save tens of millions of dollars in ASIC design costs, Microsoft ended up paying more than $1 billion for its Xbox 360 recall. To fix the problem, Microsoft went back to an unnamed ASIC vendor based in the United States and redesigned the chip, Lewis added. (Based on a previous report, the ASIC vendor is most likely the former ATI Technologies, now part of AMD.)"

Comments

yes, go cheap, that's the way

[Anonymous Coward]

it seems that every time some company tries to cut corners, it only ends up biting them in the a. my company does the same thing, and the kludgy results are nothing short of spectacular.

Chickens are coming home to roost...

[Naughty Bob]

I know /. does like to stick the boot into MSFT whenever possible, but in the last 2 hours there has been 3 front page stories, real stories, about the nasty behaviour of MSFT coming back to bite them in their fugly corporate ass.

Or is it all just a hoax?

Hope not.

Another Talisman CF

[rimcrazy]

I had the miss-pleasure of working on a graphics ASIC with MicroSquish back around the late 90's on a project called Talisman.

Never, and I say NEVER let a bunch of software engineers try to design a hardware chip. This was the biggest CF I'd seen in all my years (30+) as a chip designer. That they did it again, and with such stupidity again is no friggin surprise.

It is not that software engineers should not be involved, of course they should but when they drive the architecture in complete void of any practical chip design constraints..... and continually refuse to listen to any reason from the hardware designers..... well as they say, garbage in, garbage out.

Re:Another Talisman CF

[Cassini2]

I am a person that designs both hardware, and software, but not chips, At the risk of talking outside of my expertise, I will have a go at answering your question.

Firstly, there are things that software people really like, but it is often better to not do them in hardware. This category contains things like Read/Write I/O registers. From a software point of view, they are nice, but they can double your gate count. They can also increase your capacitive bus loading. DAC and ADC designs can also be affected this way. A software person might use a proper ADC and expect proper ADC registered results. A hardware person might select a resistor, capacitor, a voltage comparitor, and a couple of spare I/O pins. The cheesy R/C approach may save the hardware design from a whole slew of problems including cost. A software person may opt for a synchronous logic approach with all registers clocked every clock cycle. The hardware designer may opt for a much more asynchronous approach, that minimizes the number of clocked registers. This reduces power consumption, and potentially the number of registers too. Often the hardware designer will consider thermal, cost, electrical layout issues as part of his design process. The software person will not be as familiar with how to design a good circuit board and chip design in a cost-effective manner. A good software engineer can learn all of this material with time, but the hardware engineers will do them naturally.

The second category of problems is tools. The modern chip designer is working with a fairly advanced set of tools that the software person is likely to be quite unfamiliar with. This starts with the IC design tools, which are quite specialized. It ends with the hardware engineering tools. Have you ever X-Rayed a circuit board to analyze the cracks in the Ball Grid Array where it bonds to the circuit board? Are you familiar with thermal issues, and thermal images? How about EMI test results? Modern IC package design limitations? A good team of engineers will be familiar with these tools, and know how to use them to get good results.

The third category of problems is mistakes from inexperience, or lack of experience in the correct field. I work with industrial electronics. I think from an industrial point of view. What happens when someone attaches 600 (VAC) to the ground wire of the computer? What happens to the remote sensors when the plant gets hit by lightening? In IC design, there are some known gray areas too. Does the chip reset properly on power up? Do metastable, astable, or self-oscillating states exist in the IC design? Can the chip survive with no cooling? Does the chip have an overtemp shutdown function? What happens if someone starts the chip up in sub-zero weather? Do the analog electronics have sufficient electrical separation from the digital electronics, while avoiding nasty things like ESD latchup conditions?

I've completed chip design courses before, but have never had to design a modern production gate array design. As a person that has done both software and hardware, I know that my skills are not good enough for the most modern IC design processes. My limit is FPGA work, and my preference is clever opto-isolation, power semiconductor, TTL and micro-proccessor based circuits. In analog, my expertise in analog is industrial sensing and survivability. You have to know where your field of expertise is, and what your limits are.

Re:Another Talisman CF

[hedronist]

> never let a bunch of hardware guys write software

I testify, Brother, I TESTIFY!

30 Years ago, I ended up in therapy (literally) after dealing with an assembly program written by a hardware guy. The program emulated a CDC communications protocol that was originally done in hardware. This was on a Cincinnati Milacron 2200B, a machine that had both variable instruction length and variable data length. The hardware guy had implemented the protocol state changes by putting a label *on the address portion* of jump statements (he did this in 50 different places in the program) and then in some other area of the code he would change where the jump branched to next time through. It bordered on an implementation of the mythical COME FROM instruction. Of course, there was zero documentation and almost zero comments.

After one marathon debugging session I was so frustrated I was in tears. My manager came in and wanted to know what the problem was. I gave him the listing and left to walk around the building a few times. When I came back, he told me that it was, hands down, the worst piece of crap he had seen in 20 years. He had me rewrite it from scratch, which I did over a long weekend.

The program's name was RIP/TIP (Receive Interrupt Processor/Transmit Interrupt Processor) and I was in therapy for most of a year. (There were a few other issues, but this was the bale of hay that made me snap.)

this doesn't seem accurate, it was solderability

[YesIAmAScript]

Look at Bunnie Huang's analysis.

The problem wasn't any chip at all. It wasn't even heat. The problem was the chips were not soldered to the board.

http://www.bunniestudios.com/blog/?p=223

Doesn't matter who designed or made the chips. If they aren't soldered down, they won't work. And that's what the problem was. That's why X-clamps (mostly) work.

Heat is semi-tangential. If the chip is soldered down, heat won't pop it off and if it isn't soldered, any kind of movement will break it loose, even when cold. This is how MS could ship you replacement units that were RRoD out of the box. They were fine before they were shipped and were broken loose during shipping.

Most of the problem appears to be solderability problems, not a problem with chip design or manufacturing.

Re:What's going on.....

[Ritchie70]

Dunno why Lewis being wrong is upsetting.

Everything I've ever heard as a "Gartner opinion" got one of two reactions from me:

1. Well duh.
2. No, that's obviously wrong.

Looks like this is #2.

Re:When I read that I pictured Ballmer:

[nigel_q]

Anyone else think it funny that the guy that played Ballmer in that Pirates of Silicone Valley movie is the guy that does the voice for Bender?