Slashdot Mirror
Why Don't Companies Release Specs?
Mhrmnhrm asks: "With the recent activism by the OpenBSD crew focusing on release of documentation from the likes of Adaptec, Intel, and others, I'm left to wonder: why do companies insist on believing that by denying access to the specs, they somehow gain an advantage? It's not like telling a programmer how to communicate with the underlying hardware is going to tell them how it (the PCB/silicon) was designed, so why make this information secret?"
Because it would cost them money to (1) write coherent and complete documentation and (2) review that documentation to make it safe and legal for public consumption. Why would they spend all the extra time and money to do that when it doesn't bring them any more profit?
Companies exist to make money, not to be good samaritans.
Moderator hint: a comment is neither "Flamebait" nor "Troll" if it is true.
Hardware Co. Rep.: That's right, Bill--figured it would be hard to go wrong increasing our potential market, at no cost to us. It's not like they can use the interface specs to build a card.
Bill: That's nice. You know, it'd be a real shame if your driver couldn't be WHQL certified, and users had to see a warning box when they ran with your card. Or worse, if there were mysterious blue screens . . .
Harware Co. Rep.: OK, I get your point, Bill. We'll cancel the release of the specs.
Call (206) 338-5780 COLLECT for information about a genuine BA, BS, MA, MS, MBA, or Ph.D.
why do companies insist on believing that by denying access to the specs, they somehow gain an advantage?
The reason is that the ones with pointy hair are running the show and they don't really understand.
sudo eat my shorts
... the more common problem is that the hardware people paid to have their drivers and stuff written for them and that a great deal of their product's functionality is, in fact, within the driver rather than within the device or firmware. These drivers are then restricted by the agreement between the driver-writing entity and the hardware maker... or so they claim. ATI apparantly has this difficulty which is why we can't get really good drivers for Linux just yet.
Broadcom is also a big fan of not releasing much in the way of specs, even to its customers. My company has used some of their demodulator chips for years and had to beg to get some detailed specs on how to interacts with a chip in ways that it was not explicitly advertised as capable of doing (even though they admitted it was fully capable of what we wanted it to do for us)... later, when that chip became obsolete, they released a drop in replacement for it... a drop in replacement provided you were using the advertised features, things we were using, but plenty more things too suddenly went into limbo. The documentation on the new chip is ~ 1/4th the size of that of the original, so quite often we go back and forth between them, trying things that worked in the old one, hoping they still work in the new one.
Oh yes, demodulators are hella fun!
Help Brendan pay off his student loans
1. sometimes the companies don't even have documentation because they've never bothered to write it themselves
2. the company has gotten itself locked up into some NDA bullshit
3. the hardware sucks so bad they don't want people to have documentation proving how shitty it is (as is the case with Adaptec)
vodka, straight up, thank you!
It's because of fear of lawsuits, not a desire to hide their chip interfaces. There are just as many bad hardware patents as bad software ones. By controlling spec release with an NDA they stop the law firms from searching for violations of these bogus patents.
The damage from keeping hardware specs out of open source hands is tiny compared to getting a $400M judgement for patent violation.
Trade secret laws prevent law firms from forcing spec releases without pretty good proof of a patent violation.
I'd tell you, but then I'd have to kill you.
Liberals call everyone Nazis yet they are the closest thing to it.
I have often wondered about this myself, back in the day of the Apple IIs and TRS-80s there were almost always hardware specs available because that was what you used to program your application with. There were no drivers or OS abstraction layers, there wasn't enough memory or speed to do that so your program just talked directly to the hardware.
The first time I realized there was a problem was back in the 90s when my brother was trying to load up Linux on one of his machines and complaining that his video card wasn't supported because the people who made it (Diamond Multimedia?) wouldn't release any specs and people were having to reverse engineer how it worked. That seemed crazy to me. Back then it wasn't just a Windows world either, OS/2 was still around and DOS apps were huge. It boggled my mind that anyone would hold back information like that because releasing programming specs would encourage third party application development that made use of their hardware and ultimately boost their sales!
Sadly these days no one seems to care as long as the vendor has a Microsoft Windows XP(TM) driver available, even if the driver is half-assed and crashes some times.
Does anyone seriously doubt that they DO?
Andy Grove (ex CEO of INTEL) described how MS prevented INTEL from including certain features in H/W. Andy's reaction was life is too short to spend it fighting MS. This was INTEL not some much smaller niche player.
<rant>
With that said, we actually do get decent specs on individual chips or chipsets, they are always in just paper of PDF form. Even the most complex devices (UARTs, ethernet controllers, etc.) simply come with a Rosetta stone of diagrams. Then we start the regular cycle: I take the diagrams, decode them, build up a low layer of software to talk to the thing, add more stuff to exercise the features we want, test it, and finally integrate it into a WinCE/Linux/Tornado/whatever driver that does what we're using. All this despite the fact that they ship development kits with binaries (usually Windows or DOS programs) that already talk to the stupid things.
On several occasions I have told the sales guys at hardware vendors that providing source code to some chunk of software that simply does some basic communication with the part would give it big brownie points in the selection process - save us a month of programmer time and you bet we'll think hard about your chip. When I ask if they could part with some of the stuff to help us skip steps 1-3 above, they always give a uniform NO. It doesn't matter what NDA we've signed or a if we are willing to take the code stripped of features and with no warranty. Sometimes I'll get sympathy from application engineers instead of sales droids, but whenever they try to get approval up the food chain they get shut down.
I understand that source code is an incredibly valuable asset (heck, I write the stuff), but why don't hardware/chip vendors realize that handing out or (or selling it cheap) helps sell chips? After all, the stuff's worthless unless you actually get your hardware out there to be used!
</rant>
Whew! That feels better. Anyone from the hardware manufacturing end want to chime in?
"Prepare for the worst - hope for the best."
I have experience with a few different companies that make chips for PCs. I've found that the most common reason for keeping specs proprietary is patent liability. Areas like computer graphics are minefields with thousands and thousands of patents held by unfriendly entities. If you publicly release a detailed spec for your graphics chip you are inviting these unfriendly patent holders to look for potential litigation.
It's not like nVidia and ATI are looking for reasons to sue each other, it's more about some no-name holding company looking to litigate something like Cadtrak's XOR Cursor patent.
This might also be a competitive thing for Microsoft since they own a big pile of 3D graphics patents from SGI. Microsoft might take legal action against a chip supplier that publishes a spec for a 3D graphics chip that violates one of the old SGI patents.
In my experience most tech companies are now pretty hip about linux and free software, but the potential downside holds them back from releasing specs to the community.
jeff
The ISA (Instruction Set Architecture) is the specification that is needed before writing an OS, compiler or any other low-level software for a given CPU. It's no secret, especially for a very widely used architecture like x86.
In fact, there's this.
I'm not writing documentation, but seldom help with some technical details and help with translations.
Guys, just to understand the problem you have try first write something. Writing good documentation would require someone who can think straight and clearly. And as you can imaging if company got such guy/girl - s/he will be busy doing product itself, rather than documentation. That's first.
Second. On PC market change of technologies occurs every half of year or something like that. Writing documentation, getting thru all bureaucracy for confidential document to be released will be precisely about this half of year. In other words, when documentation is ready - it is already outdated. You can btw notice that industries with longer release cycles normally end-up with decent documentation - they have time and money. Having no money you cannot afford longer release cycle. Telcos are good example of that. Industrial automation is another one.
So PC industry is sort of its own worst enemy. Tight competition force producers to save where possible - e.g. optional component such as documentation. If you have some bright heads, producer can manage to update product line more often - edging against competitors - but again bright heads will be busy with product release, rather than its documentation.
Having dedicated personnel to handle documentation is just expensive.
P.S. One more problem is when product is partially based on some licensed third party development. Most small producers have to license things, since they cannot develop everything in house. Than it happens that documentation ends up with hell a lot of copy'n'pasting from third party one. To release that you have to get a permission from third party: but in most situations you will find out that third party has licensed some parts itself. In the end no-one just want to risk releasing the documentation, especially in litigious U.S.: most companies when finding their parts in someone's else released products may start asking for fees. With all consequences. Hiring experienced attorney for going thru all this licensing mess - and getting clearance from all involved parties - will cost a lot money, most producers just not able to afford.
All hope abandon ye who enter here.
It's not like telling a programmer how to communicate with the underlying hardware is going to tell them how it (the PCB/silicon) was designed, so why make this information secret?
Granted things like video cards and ethernet cards and stuff are significantly more complicated than say, guitar stombox effects and amplifiers, but electronics are electronics. It is not entirely impossible to look at the part itself and map out all the traces on the board (gets harder on multilayer boards, but it's still not impossible). Parts are parts, in the case of resistors, capacitors, diodes and stuff, and they're all marked and/or measurable. Lots of circuits have common subcomponents that are 'universal'- no different than linked lists or binary trees are to programming. Maybe you'll see a proprietary IC, but its manufacturer might have the specs available- I haven't seen an IC data sheet yet that doesn't have an internal schematic of the IC. You might be able to buy them from that manufacturer directly, or build the equivalent from other ICs and parts.
Far as I can tell though, most ICs are pretty standard and available.
So then... releasing their specs or not, it makes no difference on whether or not someone could figure out how their card works and/or clone it.
Yes I realize this takes a lot of work and man-hours to do, but surely this happens in industry of all sorts all the time.
do() || do_not();
...there are no specs.
Often products are created & sold based on some back-of-a-napkin drawing that a manager handed to development. I've been fortunate in my career -- I've been lucky to get them on notebook paper. It's much nicer than trying to figure out what was drawn through the martini condensation stains...
Chip H.
That means companies like Broadcom (they tend to not have any drivers for linux) won't get chosen by anyone wanting to run Linux, and thus will lose money as people will chose non-Broadcom hardware.
I'm not sure about all hardware, but I'm sure part of the reason Broadcom doesn't openly document their WiFi hardware is because they use software radios, where all of the channel number to frequency conversion is done in the driver and not in the hardware.
This would mean that someone writing an open source driver would have to properly tell the hardware what frequencies to use--something that shouldn't be a problem.
However, it also means that you could easily tell the radio to broadcast in frequencies (and possibly powers) that aren't within the spectrum their FCC license covers. IE, people could do things they aren't supposed to and maybe Broadcomm is worried about law suits.
--
Don't fight Firefox! Let FireFox fight YOU!
What. The. Fuck?
We recently went over this in my computer organization class.
You need to leave whatever school you're going to.
Immediately.
Seriously. It's obviously a complete waste of money. Do it now before your head gets any more messed up.
Intel is reluctant to release their ISA documentation
Nooooo they aren't.
Without releasing ISA documentation, people can't program your fucking CPU. Yeah, I'm sure that's exactly what Intel wants -- nobody to code for it. Great business plan. Where do I sign up?
Interesting? Jesus christ on a cracker, the mods are fucked up today.
I was about to mod you insightful, but then I realized what the grandparent was probably talking about... the microcode. The microcode is a lot more important for recreating details. Pentium processors have a limited ability to fix silicon bugs in software. The microcode is in volatile memory, so it must be loaded on every boot by either the OS or the BIOS. It's top-secret, but access to this lower-level info might let you write a custom instruction (or not, I don't know how much the mechanism can change the operation)
An example microcode fix is for "High Temperature and Low Supply Voltage Operation May Result in Incorrect Processor Operation"
HIV Crosses Species Barrier... into Muppets
Now, the number of machines on the acting as servers on the Internet is dwarfed by the number of machines not acting as servers on the Internet. Not a contest. Not even close to a contest. Finally, most hardware vendor's that produce high end hardware do get it. Look at the number of SCSI devices that have documentation or drivers. Not the crappy low end ones, but the really good stuff. Sorry, the crappy low end adaptec IDE RAID cards aren't ever going into a high end server. Most internet servers don't really need a good 3D card from Nvidia or ATI. Look at the e1000? Yep, got a lot of those sold, and some really good documentation available to the public. Now look at the wireless chipsets, never going in a server. Not going to get public release based off domination in the "NetCraft" stats. 90% of all wireless cards will never ever be put in a machine that is counted by "NetCraft". They will be put in machines that say, 90-95% of all shipping computers have a Microsoft OS on them.
So depending on the product, yes, the NetCraft numbers are insigificant, both in terms of percentage and raw numbers.
For other products, the NetCraft numbers make a lot of sense. Look at the number of Open Source drivers for high end SAN cards. Look at the number of Gigabit cards that have open source drivers. Look at the number of highend RAID cards that have open documentation.
The other thing they are confusing is "won't release specs", versus won't allow re-distribution of the firmware. For a lot of wireless cards, it's that they can't re-distribute the firmware which is necessary for the hardware to work, not that they don't have the specs for the actual hardware. (I believe a number of OpenBSD's issues with Wireless Intel chipsets are this issue). The other issue with Wireless is that releasing the specs could invite FCC issues. I'm not sure of how much teeth those could have, but it sounds like a legitimate concern. If the FCC could say, well people are using your part with your documentation to drive it out of complaince, your wireless cards can no longer be sold. That'd be a real problem for Intel or Cisco.
Finally, releasing the specs, would allow a competitor to say they clean room, backward engineered without having to do the documentation. It's really expensive to do that testing and documentation. So they could sell a competing product at a much lower price. It's much easier to match an existing specification, that it is to write a good known working specification. It is valuable IP. Personally, I think it's wonderful when they release it. For a lot of hardware they do. Especially for the really high end hardware that needs high reliability.
Kirby
R&D Costs for driver development can't be recouped
When BusLogic came out with their AHA1540/AQHA1542 compatible SCSI controllers, they did several things:
(1) They leveraged Adaptec's R&D effort on driver development so that they didn't have to spend their own money developing drivers.
(2) They avoid the effort of convincing Microsoft to ship with the drivers as part of Windows; never underestimate the barrier to entry a separate driver disk represents: for NT 3.51, you had to install the driver disk twice during the installation process, and a third time post-install to use a third party disk controller, and it was not well documented where or when NT 3.51 would reference the disk, not find it, and simply crash (you "just had to know").
(3) By making the card's interface compatible with an existing card, they avoided all the R&D necessary to get from a concept to a working interface.
So basically, BusLogic got a free ride on Adaptec's dime in a number of areas, and Adaptec was left holding the bag, having to charge a higher amount to recoup their R&D investment, leaving BusLogic to undercut their prices and steal their market.
That's not to say that I totally agree with Adaptec, or that bad designs which can't be safely exposed without risking physical damage to the devices (Diamond Multimedia video cards[*]) or legal repercussions (Oscillator/PLL tuned WiFi cards that don't have hard-wired frequency bands) don't exist, but I can certainly understand the position of companies who don't want to sacrifice hard-earned product lead-times, OS vendor relationships, or actual R&D results on the altar of public domain.
[*] Diamond, in particular, pisses me off, since any halfway decent software engineer could have separated the model and view from the controller, and made their cards both Open Source friendly and non-x86 architecture friendly, and the EE who wrote the firmware didn't.
-- Terry
"More than likely, the lost profits from not reaching the <10 percent of the market willing to pay only for devices with public documentation are less than the lost profits from incurring the expense of cleaning up internal documents."
;)
That's why we have to bitch and moan as loud as possible to make the public relations value exceed the cost of cleaning up the docs.
I rarely criticize things I don't care about.
Have you seen any internal specifications? I'm guessing you haven't. They are quite often littered with references to other project codenames. There are references to design and verification methodologies. There are references to third-party requirement/IP. They would be a treasure trove for any competitor to pick through. Don't you think that knowing codenames of related projects that use "this piece" or "that piece" of a reusable design wouldn't help a social engineer to find more information? Don't you think being able to piece through design/verification methodology would undermine a company's ability to be first to market because they have better processes in place than the competitors? Don't you think those third-party partners/vendors would be a bit peeved if their information was leaked?
I work for a company involved with designing RAID-on-chip solutions, so I speak from experience. We constantly have discussions with our customers regarding what features they want/need implemented to maintain an advantage over their competitors. The features are implemented, only the one customer knows about them, but they are still a part of the internal specifications for the part. Internal specification are in no way, shape, form or fashion designed or written for general public consumption. Believing that they are or that they should be is naive. Trying to deliver a "clean" spec for general consumption would involve more than one company's lawyers.
There are a few reasons I have seen based on RE drivers. I'll name a few here in no particular order.
Embarrassment. Sometimes they don't want to tell you that it's necessary to reset the chip repeatedly until it just happens to initialize correctly (probably due to a design flaw).
More embarrassment. They don't want to admit that many of the so-called 'hardware features' are really implemented (poorly) in the driver. Many IDE "RAID cards" are just a bunch of IDE co ntrollers. The RAID is in the driver. The driver's RAID implementation is not generally anywhere near as good as the Linux soft raid device.
Closely related to the above, for (snake oil) security devices, they don't want to document that flipping bit 5 of the 3rd register will bypass all protection.
The IP isn't theirs. The product is just relabled.
Their default answer to anything is NO, and they haven't (and probably won't) give it any thought.
In the case of some graphics cards, they don't want the public to find out the driver has hacks in it to cheat the benchmarks.
They already provide hamburgers to an open standard. Their product interfaces in the same way as all other hamburgers, directly with the mouth port.
Very fancy restaurants have proprietary interfaces: You are required to use specialized hooks (salad fork, etc.) to interface with the food which must be used in a specific way.
Chinese restaurants use the most difficult interface of all, but it's fairly simple and often fully documented on the chopsticks package.
Can you be Even More Awesome?!
When the number of device driver writers for your part numbers in single digits, you don't need super great docs. When you try to write good docs, the next thing people will complain about is the quality of the docs!
"This mission is too important to allow you to jeopardize it." -- HAL
It's very simple: back in the old days, when there was no monopoly, many chip manufacturers gave away thick, printed databooks. It was expensive for them, but they needed developers to use their products.
Now, however, there is a monopoly. You don't need to attract developers. The only concern is to have a driver for Windows, and having that driver included with the Windows install disk, so that your device, be it a soundcard, graphics card, or whatever, is "easier" to use. I wonder if there is a "dark hand" behind it...
Some years ago I sent a proposal to the European Comission: banning the sale in Europe of peripherals for which there is no public interface information available. It should not hurt the manufacturers, as the information can be made freely available in Internet (it's cheaper than shipping huge printed manuals), but it would have a side effect: the driver advantage for Window could disappear.