Why Don't Companies Release Specs?

Mhrmnhrm asks: "With the recent activism by the OpenBSD crew focusing on release of documentation from the likes of Adaptec, Intel, and others, I'm left to wonder: why do companies insist on believing that by denying access to the specs, they somehow gain an advantage? It's not like telling a programmer how to communicate with the underlying hardware is going to tell them how it (the PCB/silicon) was designed, so why make this information secret?"

Comments

Because it would cost them money

[c0d3h4x0r]

Because it would cost them money to (1) write coherent and complete documentation and (2) review that documentation to make it safe and legal for public consumption. Why would they spend all the extra time and money to do that when it doesn't bring them any more profit?

Companies exist to make money, not to be good samaritans.

Re:Because it would cost them money

[Geekboy(Wizard)]

If there aren't drivers for my OS, I won't buy their hardware. Period. They just lost a sale from me. I personally know 20 people that do the same. And thats just people I know in person. Thats not counting the hundreds of people I have talked about this with online.

I'm not saying they should release it out to the general public, but allow developers to read the docs and write the code for it. We don't want them to write the code for us, or to be 1st level support. Hell, the only time we'd talk to them after getting the docs, is to report a bug.

Re:Because it would cost them money

[elmegil]

Goody for you. When you and your friends who think like you are enough of a market share for them to care, their practices will change. Have fun.

Re:Because it would cost them money

[TekPolitik]

Because they (should) already have these docs to write the Microsoft driver ?

A lot of the hardware companies don't even write internal docs. They have hardware engineers write the device driver, and if you have ever reverse engineered a Windows device driver, you'd know that it shows. It is quite common to find serious and obvious bugs in the driver - mistakes of the kind even a mediocre professional software developer would never make. Sometimes the reverse engineering even reveals the use of algorithms that achieve things that are not even mathematically useful - although you can tell what they were trying to achieve, you can also tell that they have come up with an algorithm that has nothing whatsoever to do with achieving that goal.

This is one of the reasons Windows is so damned unreliable - they package drivers from these closed interface hardware shops, written by hardware engineers without the first clue about software development.

And yes, when the hardware engineers leave this all means the vendor isn't able to provide new drivers for that hardware - that's why you end up with 6 month old hardware that won't run on the operating system released today, and probably never will.

Re:Because it would cost them money

[wastingtape]

I went into McDonalds last week and ordered prime rib with a side of potatoes. They wouldn't sell it to me.

When i go into a restraunt, i expect to order and get whatever i want. If they don't serve what i want, i won't eat there. Period. They just lost business from me. I know at least 20 people who tried to order steak from McDonalds and they wouldn't offer it, that's not counting the hundreds of people online who want all kinds of stuff that McDonald's doesn't sell.

I'm not saying that McDonalds should forsake thier demographics, target market, or any plans they had, but at least allow restraunt-goers a chance to eat whatever they want.

Re:Because it would cost them money

[imp]

You're telling me that they don't have internal documentation anyway?
How do they develop their own drivers, etc?

You've clearly never had to write a driver for hardware that's produced down the hall :-). Usually the hardware designer produces a interface document that tells you what he thinks you should know. Once you have this document, there's usually dozens upon dozens of questions that the software writer has about the device. For smaller operations, these questions usually are verbal or email and never make it back into the original spec. There are times that the specification is wrong, incomplete, missing information, etc. To assume that there's public consumable documentation for every piece of hardware doesn't match my many years of industry experience.

In addition to the cost of producing it, there's other issues. Did the company license some IP for their internal use? If so, can that IP be documented and sent out to third parties? Someone has to check to make sure the relevant legal contracts are followed. Next, did we develop some new and novel way of doing something that's covered by a patent? Is it ours our someone else's. Fear of lawsuits from third parties often drives a company to disclose as little as possible. On the 'simpler' devices, much of the information necessary to write a driver may be novel ways of, for example, encoding something in software that might otherwise be done in hardware. Sometimes these tricks may save lots of money for the hardware company who can sell modems $5 cheaper than their competitors. Why make it easy for them to understand your cleverness?

Finally there's good, old fashioned secrecy. Does Macy's tell Gimble's, as the old (now somewhat dated) adage goes. The idea here is that your competitors should have as difficult a time as inventing something as you did, and any hints or short-cuts that you can give them may eat into your profits. Maybe they make a hardware board that's compatible with your driver, this cutting the cost of their development time since they don't have to do the software. Who knows how this information will come back to bite you.

All of these effects on the bottom line are seen as swamping any increase in hardware/software sales to the open source crowd. Not that they are right about this (or wrong for that matter), just how they think.

Warner

Re:Because it would cost them money

[AngryElmo]

Well, seeing as I operarate a budget of a couple of million and I use exactly that methodology (What function to perform, what are the supported OS's, is it one that we have skills with and which hardware vendors provide certified drivers), I would think that a few vendors would be interested. And keep in mind that I work for an SME and this is just one company in a sea of thousands...

Re:Because it would cost them money

[Xantharus]

The problem here though, is that McDonalds has never made the impression to anyone that they sell steak. Its like asking Adobe for full documentation on Windows; they have no idea.

Wouldnt full specifications from McDonalds actually be them telling you exactly what is in each Quarter-Pounder with Cheese, and exactly what happens behind the scenes when they cook it?

I dont know about you, but I think those are some specifications that are better left unreleased.

Re:Because it would cost them money

[drmerope]

Way to rag on hardware designers. I think we'll take our computers back now. Go back to your mechanical adding machines.

Seriously: have you see the code "programmers" make? Hardware designers tend to be extremely effective programmers--if only because they have a clue as the actual performance costs of their design decisions. Most programmers haven't got a clue that an IPI stalls all the activity on an SMP system let alone a reasonable sense of how costly that is.

From what I've seen, computer engineers get as much training is software engineering methodology as programmers.

The *trouble* is tight schedules and billion things you have to do.

It goes something like this:

[vegetablespork]

Bill: I hear you're getting ready to release low level specs so the OpenBSD and Linux folks can make a driver.

Hardware Co. Rep.: That's right, Bill--figured it would be hard to go wrong increasing our potential market, at no cost to us. It's not like they can use the interface specs to build a card.

Bill: That's nice. You know, it'd be a real shame if your driver couldn't be WHQL certified, and users had to see a warning box when they ran with your card. Or worse, if there were mysterious blue screens . . .

Harware Co. Rep.: OK, I get your point, Bill. We'll cancel the release of the specs.

Re:It goes something like this:

[AnObfuscator]

Yes. I think so because microsoft DOES threaten hardware companies. It's been proven, in court, that microsoft does this sort of thing.
http://en.wikipedia.org/wiki/Microsoft#Monopoly_an d_legal_issues

In short, thinking that MS doesn't use its monopoly to intimidate PC & PC hardware makers is like believeing the world is still flat.

The reason is...

[AragornSonOfArathorn]

why do companies insist on believing that by denying access to the specs, they somehow gain an advantage?

The reason is that the ones with pointy hair are running the show and they don't really understand.

Stated in an earlier post...

[erroneus]

... the more common problem is that the hardware people paid to have their drivers and stuff written for them and that a great deal of their product's functionality is, in fact, within the driver rather than within the device or firmware. These drivers are then restricted by the agreement between the driver-writing entity and the hardware maker... or so they claim. ATI apparantly has this difficulty which is why we can't get really good drivers for Linux just yet.

Broadcom fun

[DaHat]

Broadcom is also a big fan of not releasing much in the way of specs, even to its customers. My company has used some of their demodulator chips for years and had to beg to get some detailed specs on how to interacts with a chip in ways that it was not explicitly advertised as capable of doing (even though they admitted it was fully capable of what we wanted it to do for us)... later, when that chip became obsolete, they released a drop in replacement for it... a drop in replacement provided you were using the advertised features, things we were using, but plenty more things too suddenly went into limbo. The documentation on the new chip is ~ 1/4th the size of that of the original, so quite often we go back and forth between them, trying things that worked in the old one, hoping they still work in the new one.

Oh yes, demodulators are hella fun!

Re:Broadcom fun

[The Vulture]

I used to work in the DOCSIS cable modem industry, so I'm reasonably familiar with the Broadcom 335x/334x CPU family. The best I ever saw was the datasheet for the 3416 chip (tuner chip).

For those of you who aren't aware, or don't remember, Broadcom was slapped with a lawsuit by Microtune, alleging patent violations http://news.com.com/2100-1006_3-5064586.html, (one of many articles available on this subject).

So, I was tasked with providing software support for the 3416 tuner chip (the replacement for the 3415 "problem" chip) in our cable modem product, as the 3415 couldn't legally be sold in the U.S. anymore. Well, I figured it would be easy, since the datasheets for the two chips were exactly the same, except that "3415" was changed to "3416". (And, I mean exactly the same, to the word - it was only a 15 page datasheet, so I compared them).

Nope, good old Broadcom documentation does it again... I managed to study the source code for one of the newer cable modem CPUs and find the changes between the chips. The real stupid thing though: Even though there's a version register (which tells which version the chip is), they didn't update it to indicate a 3415/3416, even though they had the bits available. So, 3416 chips would show up as 3415 chips if you try to read the version number (that made things difficult).

Incidentally, Broadcom does make Linux drivers for most of it's newer hardware, they're just meant for their customers (the OEMs making the hardware, not the end-users). The general rule with Broadcom is, if you pay them a lot of money, you get excellent support. (At the previous company I worked for, I had an engineer that would get back to me within 24 hours on any issue related to the product, because we bought so many cable modem chips. When I currently work, they won't even give us the time of day, we just don't purchase enough.)

I feel your pain, I really do.

-- Joe

It's becuase of fear or lawsuits

[jonsmirl]

It's because of fear of lawsuits, not a desire to hide their chip interfaces. There are just as many bad hardware patents as bad software ones. By controlling spec release with an NDA they stop the law firms from searching for violations of these bogus patents.

The damage from keeping hardware specs out of open source hands is tiny compared to getting a $400M judgement for patent violation.

Trade secret laws prevent law firms from forcing spec releases without pretty good proof of a patent violation.

Vendors Losing Money

[Embedded Geek]

Coming from the embedded world, I agree wholeheartedly. First, a caveat; I'm talking individual chips here, not whole cards. So, your milage may vary...

<rant>
With that said, we actually do get decent specs on individual chips or chipsets, they are always in just paper of PDF form. Even the most complex devices (UARTs, ethernet controllers, etc.) simply come with a Rosetta stone of diagrams. Then we start the regular cycle: I take the diagrams, decode them, build up a low layer of software to talk to the thing, add more stuff to exercise the features we want, test it, and finally integrate it into a WinCE/Linux/Tornado/whatever driver that does what we're using. All this despite the fact that they ship development kits with binaries (usually Windows or DOS programs) that already talk to the stupid things.

On several occasions I have told the sales guys at hardware vendors that providing source code to some chunk of software that simply does some basic communication with the part would give it big brownie points in the selection process - save us a month of programmer time and you bet we'll think hard about your chip. When I ask if they could part with some of the stuff to help us skip steps 1-3 above, they always give a uniform NO. It doesn't matter what NDA we've signed or a if we are willing to take the code stripped of features and with no warranty. Sometimes I'll get sympathy from application engineers instead of sales droids, but whenever they try to get approval up the food chain they get shut down.

I understand that source code is an incredibly valuable asset (heck, I write the stuff), but why don't hardware/chip vendors realize that handing out or (or selling it cheap) helps sell chips? After all, the stuff's worthless unless you actually get your hardware out there to be used!
</rant>

Whew! That feels better. Anyone from the hardware manufacturing end want to chime in?

Patent liability

[jeffmock]

I have experience with a few different companies that make chips for PCs. I've found that the most common reason for keeping specs proprietary is patent liability. Areas like computer graphics are minefields with thousands and thousands of patents held by unfriendly entities. If you publicly release a detailed spec for your graphics chip you are inviting these unfriendly patent holders to look for potential litigation.

It's not like nVidia and ATI are looking for reasons to sue each other, it's more about some no-name holding company looking to litigate something like Cadtrak's XOR Cursor patent.

This might also be a competitive thing for Microsoft since they own a big pile of 3D graphics patents from SGI. Microsoft might take legal action against a chip supplier that publishes a spec for a 3D graphics chip that violates one of the old SGI patents.

In my experience most tech companies are now pretty hip about linux and free software, but the potential downside holds them back from releasing specs to the community.

jeff

Sorry for breaking /. traditions

[ThePhilips]

Sorry for breaking /. traditions. I have worked for three hardware companies. I'm working for one right now.

I'm not writing documentation, but seldom help with some technical details and help with translations.

Guys, just to understand the problem you have try first write something. Writing good documentation would require someone who can think straight and clearly. And as you can imaging if company got such guy/girl - s/he will be busy doing product itself, rather than documentation. That's first.

Second. On PC market change of technologies occurs every half of year or something like that. Writing documentation, getting thru all bureaucracy for confidential document to be released will be precisely about this half of year. In other words, when documentation is ready - it is already outdated. You can btw notice that industries with longer release cycles normally end-up with decent documentation - they have time and money. Having no money you cannot afford longer release cycle. Telcos are good example of that. Industrial automation is another one.

So PC industry is sort of its own worst enemy. Tight competition force producers to save where possible - e.g. optional component such as documentation. If you have some bright heads, producer can manage to update product line more often - edging against competitors - but again bright heads will be busy with product release, rather than its documentation.

Having dedicated personnel to handle documentation is just expensive.

P.S. One more problem is when product is partially based on some licensed third party development. Most small producers have to license things, since they cannot develop everything in house. Than it happens that documentation ends up with hell a lot of copy'n'pasting from third party one. To release that you have to get a permission from third party: but in most situations you will find out that third party has licensed some parts itself. In the end no-one just want to risk releasing the documentation, especially in litigious U.S.: most companies when finding their parts in someone's else released products may start asking for fees. With all consequences. Hiring experienced attorney for going thru all this licensing mess - and getting clearance from all involved parties - will cost a lot money, most producers just not able to afford.

Reverse Engineering

[devphaeton]

It's not like telling a programmer how to communicate with the underlying hardware is going to tell them how it (the PCB/silicon) was designed, so why make this information secret?

Granted things like video cards and ethernet cards and stuff are significantly more complicated than say, guitar stombox effects and amplifiers, but electronics are electronics. It is not entirely impossible to look at the part itself and map out all the traces on the board (gets harder on multilayer boards, but it's still not impossible). Parts are parts, in the case of resistors, capacitors, diodes and stuff, and they're all marked and/or measurable. Lots of circuits have common subcomponents that are 'universal'- no different than linked lists or binary trees are to programming. Maybe you'll see a proprietary IC, but its manufacturer might have the specs available- I haven't seen an IC data sheet yet that doesn't have an internal schematic of the IC. You might be able to buy them from that manufacturer directly, or build the equivalent from other ICs and parts.

Far as I can tell though, most ICs are pretty standard and available.

So then... releasing their specs or not, it makes no difference on whether or not someone could figure out how their card works and/or clone it.

Yes I realize this takes a lot of work and man-hours to do, but surely this happens in industry of all sorts all the time.

Software's dirty secret

[chiph]

...there are no specs.

Often products are created & sold based on some back-of-a-napkin drawing that a manager handed to development. I've been fortunate in my career -- I've been lucky to get them on notebook paper. It's much nicer than trying to figure out what was drawn through the martini condensation stains...

Chip H.

Because it prevents Ilicit Uses

[BobPaul]

That means companies like Broadcom (they tend to not have any drivers for linux) won't get chosen by anyone wanting to run Linux, and thus will lose money as people will chose non-Broadcom hardware.

I'm not sure about all hardware, but I'm sure part of the reason Broadcom doesn't openly document their WiFi hardware is because they use software radios, where all of the channel number to frequency conversion is done in the driver and not in the hardware.

This would mean that someone writing an open source driver would have to properly tell the hardware what frequencies to use--something that shouldn't be a problem.

However, it also means that you could easily tell the radio to broadcast in frequencies (and possibly powers) that aren't within the spectrum their FCC license covers. IE, people could do things they aren't supposed to and maybe Broadcomm is worried about law suits.
--
Don't fight Firefox! Let FireFox fight YOU!

Re:Actually

[bani]

What. The. Fuck?

We recently went over this in my computer organization class.

You need to leave whatever school you're going to.

Immediately.

Seriously. It's obviously a complete waste of money. Do it now before your head gets any more messed up.

Intel is reluctant to release their ISA documentation

Nooooo they aren't.

Without releasing ISA documentation, people can't program your fucking CPU. Yeah, I'm sure that's exactly what Intel wants -- nobody to code for it. Great business plan. Where do I sign up?

Interesting? Jesus christ on a cracker, the mods are fucked up today.

Re:Time is money to make NDA'd docs publishable

[ComputerSlicer23]

In a lot of ways, yes it is a very small number. First off, I'm not sure which documentation you are discussing from NetCraft. A link would be useful, but I'm too lazy to dig one up. The two stats to remember, are that something like 80% of the domains out there in the world run non-Windows OS's (read people who probably need access to the documentation). I want to say, only 30% of the IP's (possibly physical machines) out there, run non-Window's OS's. So in the end, even on the Internet, Windows is fairly dominate in terms of "numbers of machines". Which is a crying shame in my opinion, but oh well (most hosting companies, use non-Windows machines for deployment).

Now, the number of machines on the acting as servers on the Internet is dwarfed by the number of machines not acting as servers on the Internet. Not a contest. Not even close to a contest. Finally, most hardware vendor's that produce high end hardware do get it. Look at the number of SCSI devices that have documentation or drivers. Not the crappy low end ones, but the really good stuff. Sorry, the crappy low end adaptec IDE RAID cards aren't ever going into a high end server. Most internet servers don't really need a good 3D card from Nvidia or ATI. Look at the e1000? Yep, got a lot of those sold, and some really good documentation available to the public. Now look at the wireless chipsets, never going in a server. Not going to get public release based off domination in the "NetCraft" stats. 90% of all wireless cards will never ever be put in a machine that is counted by "NetCraft". They will be put in machines that say, 90-95% of all shipping computers have a Microsoft OS on them.

So depending on the product, yes, the NetCraft numbers are insigificant, both in terms of percentage and raw numbers.

For other products, the NetCraft numbers make a lot of sense. Look at the number of Open Source drivers for high end SAN cards. Look at the number of Gigabit cards that have open source drivers. Look at the number of highend RAID cards that have open documentation.

The other thing they are confusing is "won't release specs", versus won't allow re-distribution of the firmware. For a lot of wireless cards, it's that they can't re-distribute the firmware which is necessary for the hardware to work, not that they don't have the specs for the actual hardware. (I believe a number of OpenBSD's issues with Wireless Intel chipsets are this issue). The other issue with Wireless is that releasing the specs could invite FCC issues. I'm not sure of how much teeth those could have, but it sounds like a legitimate concern. If the FCC could say, well people are using your part with your documentation to drive it out of complaince, your wireless cards can no longer be sold. That'd be a real problem for Intel or Cisco.

Finally, releasing the specs, would allow a competitor to say they clean room, backward engineered without having to do the documentation. It's really expensive to do that testing and documentation. So they could sell a competing product at a much lower price. It's much easier to match an existing specification, that it is to write a good known working specification. It is valuable IP. Personally, I think it's wonderful when they release it. For a lot of hardware they do. Especially for the really high end hardware that needs high reliability.

Kirby

Re:Time is money to make NDA'd docs publishable

[Anonymous Coward]

Have you seen any internal specifications? I'm guessing you haven't. They are quite often littered with references to other project codenames. There are references to design and verification methodologies. There are references to third-party requirement/IP. They would be a treasure trove for any competitor to pick through. Don't you think that knowing codenames of related projects that use "this piece" or "that piece" of a reusable design wouldn't help a social engineer to find more information? Don't you think being able to piece through design/verification methodology would undermine a company's ability to be first to market because they have better processes in place than the competitors? Don't you think those third-party partners/vendors would be a bit peeved if their information was leaked?

I work for a company involved with designing RAID-on-chip solutions, so I speak from experience. We constantly have discussions with our customers regarding what features they want/need implemented to maintain an advantage over their competitors. The features are implemented, only the one customer knows about them, but they are still a part of the internal specifications for the part. Internal specification are in no way, shape, form or fashion designed or written for general public consumption. Believing that they are or that they should be is naive. Trying to deliver a "clean" spec for general consumption would involve more than one company's lawyers.

Several reasons

[sjames]

There are a few reasons I have seen based on RE drivers. I'll name a few here in no particular order.

Embarrassment. Sometimes they don't want to tell you that it's necessary to reset the chip repeatedly until it just happens to initialize correctly (probably due to a design flaw).

More embarrassment. They don't want to admit that many of the so-called 'hardware features' are really implemented (poorly) in the driver. Many IDE "RAID cards" are just a bunch of IDE co ntrollers. The RAID is in the driver. The driver's RAID implementation is not generally anywhere near as good as the Linux soft raid device.

Closely related to the above, for (snake oil) security devices, they don't want to document that flipping bit 5 of the 3rd register will bypass all protection.

The IP isn't theirs. The product is just relabled.

Their default answer to anything is NO, and they haven't (and probably won't) give it any thought.

In the case of some graphics cards, they don't want the public to find out the driver has hacks in it to cheat the benchmarks.

Nope

[zippthorne]

They already provide hamburgers to an open standard. Their product interfaces in the same way as all other hamburgers, directly with the mouth port.

Very fancy restaurants have proprietary interfaces: You are required to use specialized hooks (salad fork, etc.) to interface with the food which must be used in a specific way.

Chinese restaurants use the most difficult interface of all, but it's fairly simple and often fully documented on the chopsticks package.