Slashdot Mirror
Update to The Magic Cauldron
Eric S. Raymond wrote
in to tell us that he has updated The Magic Cauldron
(his essay on economics and Open Source) to contain
an appendix on common arguments for keeping device drivers
closed (Pay attention Creative Labs!) He also says "The argument turns on the fact that drivers are small pieces of code, easy
to disassemble if need be. This argument would be considerably strengthened
if I could point readers at a working set of tools for disassembling Windows
drivers into recompileable source (or even just assembly) code. I would
appreciate pointers to any such tools."
Even without using something like this, disassembly of programs cannot be 100% correct. There is a lot of information needed to reconstruct a program that is thrown away by the compiler. For example you can't determine where a jump table (switch statement) or function pointer (virtual function) will lead to without actually executing the program and then it's not possible to cover all of the cases that could occur. IDA does a pretty good job of guessing though. Also IDA has hacks if it is looking at C code like looking for procedure frame headers (push ebp; mov ebp, esp) and termination code (pop ebp; ret). My solution to correct disassembly is to use emulation. This insures you never have an incorrect interpretation, but you will miss code that is never executed, and this can be important for understanding how errors and exceptions are handled.
Just because you can dissemble something doesn't mean you can understand it. It requires a huge amount of time to try to comprehend uses for unnamed memory locations and layout of data structures. In the case of a driver this is frustrated by the fact you are dealing with a lot of hidden code and data (the logic on the hardware). Not all drivers are "small pieces of code." Having looked at several vendor's Direct3d driver source code I can tell you each was extremely large and complex. Writing the driver with full knowledge of everything is hard enough, but disassembling one and understanding it would require super human willpower. There are all sort of commands that will do different things depending on that state of the card, so you have to understand what previously executed code was important to getting the card into that state.
-- Virtual Windows Project
First off, I want to make it clear that I'm not arguing against open source drivers, I'm arguing that a) open source drivers are necessary but insufficient to getting hardware running on different os's. and b) there are costs in resources and $$$ for a company in supporting an open process for drivers.
I was thinking in the context of drivers for things like new generation 3d boards, which are not reverse engineer-able in any useful sense.
You don't really "port" a 3d video driver from Windows to Linux, you write a brand new one, basically from scratch. Source code would be nice, but you still need the docs plus actual human beings to ask questions of. (Yeah, glide helps)
3Dfx has done a great job of this, but it took a some amount of liason work (==$$$) on 3dfx's part to make it happen. Every hour the 3dfx engineers spend talking to the linux driver guys is an hour a) they get paid and b) they aren't working on something else. It was probably worth it for them (hey, I bought one) but it was an "allocation of resources" problem just like in sim city.
Another reasons open sourcing drivers might cost a company : The drivers for 3d boards are areas of competition (the drivers can be as important as the chip design for producing good benchmark results)
Again, I _like_ the idea of an open driver development process. I bought and very happily use a 3dfx Voodoo3/2000 because of it. I hear nvidia is doing a great job as well.
-cks
This is a decent starting point for disassembly information. Lot's of texts on reverse engineering, with pointers to tools, etc...
mammon_
I found the situation of disassemblers for the X86 relatively bleak, since there is only one tool that suits my needs (offline disassembly). Every development tool contains a debugger that also does some disassembly of the current instruction stream, but they all lack the ability to define data structures.
:) ). IDA supports a whole slew of input files (DLL, VxD, NLM, COM ...) for a lot of processors (Z80, x86, Java VM, ...). It has working versions for OS/2, DOS, Win32 ...
I did find "the one true" (at least to me) disassembly tool, IDA by http://www.datarescue.com. It is an interactive disassembler that does background disassembly and lets the user identify additional code and data sections (and correct the mistakes
There are only two things about it - it is payware and no source code is available. And it is virtually impossible to find warezed copies on the net. I paid for my copy and I'm still happy with IDA (in fact, I was happy with the trial version and found it well worth the money to upgrade). The copy protection seems also to be relatively hard, at least from what I found (or better, didn't find) by looking at it.
-max
Premier argument to install Linux at the workplace - I get paid while waiting for fsck to scan the partitions.
The second story was with a 68000 monitor and debubber that I had to write a disassembler for in order to move it to a 68010 SBC. The disassembler was hand-made by me.
The third involved the original 8052 BASIC52 that Intel had produced. There were some bugs and lacking features in it, so I wrote a small assembly routine to dump the contents of the chip out its built-in serial port. The 8052 output was captured on an IBM PC, and I wrote a disassembler in MSC 4 to put it back into 8052 assembly language. I then fixed the bugs and added the features I needed, after which I programmed it back into an 8752 for further development. I sent the code back to Ciarcia, and I even got a little note back.
But the biggest tool I used was V Communications Sourcer. I got started using it to reverse IBM PC ROMS to hack the drive tables, and then started to add little features and fix little bugs. I used it commercially to reverse engineer the IBM RTIC communications card DOS drivers. I did that so we could then port the card to Novell Netware 2.15 as a VAD (IBM had no plans at the time to port it, and we needed it to provide a Bouroughs data link between a Netware server and a Bouroughs mainframe). Sourcer was, and still is, the cleanest disassembler on the market, capable of giving you back highly commented source code from binary files. I haven't updated my license lately since I prefer open systems to Windows, but if I had to do that again I would go back to Sourcer.
The point is that reverse engineering is equal parts art and science, and if you need it bad enough and know assembly well enough, you'll find and/or build the right tools for the job. ESR needs to calm down a bit.
Forget the open source movement for a moment. It seems that the lifetime for computer hardware is incredibly short. By keeping the source to driver software closed manufactures accomplish;
t ml
1) Decrease the life of the product, shortening support obligations.
2) Hide any shortcomings to their hardware product, saving on warranty obligations.
3) Forcing the consumer to upgrade by preventing product 'enhancments', or fixing 'problems'
4) Reveal design shortcomings to the competition.
Some wise man said that a there are two reasons someone does something. A good reason, and the _real_ reason. The arguement that it protects proprietarty technology is in most instances completely false, as many chip makers release 'reference' designs to many hardware manufactures, who in turn make only small changes to the basic design. And any valid 'proprietary' enhancements there may be are quickly undone when the next competing product hits the market. So companies are left with the real reason - less hassle and more money.
I do not think that is enough that you argue politely the benifits of open source. Consumers are going have to get tired of being on the endless upgrade mill of expensive hardware and the expensive bloated code to use it. How much power do you need to write a letter or balance a checkbook?
Maybe then we would see real innovation in the computer industry instead of creeping featurism.
The only way that you are going to see open source from a manufacture is if it is legislated by congress answering the demand of angry consumers who catch on. IBM's Aptiva MWAVE is a classic example of the above. Cut and paste the link below into your browser.
http://members.aol.com/mbs1058/mwave/classact.h
fravia (east coast)
decompilation page
sandman
greythorne
The more interesting question is, given that most of these sites have been around so long, why dont we see more reverse-engineering of software going on? i think the availability of all this information (especially on fravia) weakens esr's argument significantly. regardless of what he says, reverse-engineering is really, really difficult, even for small pieces of code. i dont think he'll convince very many people based on that argument.
As for the appropriate tools, a while ago i found copies of wdasm and softice using ftpsearch (remember, one version of softice was a fully operational time-demo which could be cracked by itself). much of the other stuff you might need that ive seen are freeware or shareware.
unc_
Chip vendors don't like this because they don't want their competitors to use their own documentation to produce drop-in replacements for their components. They use trade-secret because it's the cheapest form of protection. However, if they have to open their interfaces to get their chips designed into Linux systems, they will publish their documents and use patents and copyright as protection against their hardware competitors.
Thanks
Bruce
Bruce Perens.
exactly. i do not want to give my money to a company that does not open
their hardware specs and i would rather give it to a company that has open
source drivers. what would be useful in this context would be a web based
hardware database, where find quick info about how open a specific piece of
hardware is before buying one.
like there could be 5 categories:
1 proprietary hardware, company does not give out spec.
2 company allowed people to write binary only drivers under NDA but for free.
3 company releases binary only drivers
4 company published detailed documentation
5 company wrote open source driver or funded/helped in the development
i would only buy hardware in category 4 or 5 but often it is not easy to
find out if the person who wrote drivers got support from the company..
sure, a grep through the linux kernel helps sometimes but that is not
something the marketing departments will consider a threat to the image of
their product.
having such a database where one could directly compare the openness of
various vendors would put some market pressure on those companies..
mond.