Slashdot Mirror
Reverse Engineering an MPEG Driver
An anonymous reader writes "Following on from the recent spate of reverse engineering articles, there is an interesting summary of the reverse engineering of a binary only Linux driver.
The driver is for the integrated MPEG decoder on VIA's popular EPIA-M boards. At the moment VIA has not publicly released the source code for the MPEG chipset on these boards and will only make the code available under NDA saying that "Typically, only requests from companies developing product for sale will be approved."
As a result this is holding back development of open source tools (e.g. xine, mplayer, vdr) that would be able to make use of the interesting hardware on these boards."
IANAL, but I don't think the source code is legally safe if VIA wants to go after it.
-mse
Fiat Lux.
To me, it just seemed like a general description of the RE process that people able to RE already know. EPIA M boxes are already good for small PVR boxes using mythtv when a Hauppauge PVR card is added (and a larger power supply). If the MPEG decoder can be used, I'm sure that even the lesser models of EPIA will be able to be used.
bananas like monkeys.
Should developers/users be afraid of the iron fist of moronic law in this case?
Or is it perfectly legal and VIA can not do anything about it? They seem to have an interest in suppresing such efforts though, since they've stated they are interested in revealing the code only to entities that want to make a buck off of it.
So, even if DMCA dosn't apply here, are there any chances they could be nasty about it? U-Boot
In all seriousness, what's the point? It's not like Xine on any decent machine using Xv is going to look or work any better using this hardware decoder. CPU is cheap.
Driver code is the biggest liability that a device maker has. It earns no money, it costs quite a bit to make, and it must be written multiple times for multiple platforms and operating systems.
Via's reluctance to free the driver software is pure evil. They sit like slavemasters on the code and hold it hostage as if it were a servant or slave.
Even if the reverse engineering works out and the code runs equally well as the enslaved code, what will become of the original unfree code? Will that unfortunate code be relegated to living out the rest of its days in slavery? Sadly, I think the answer is affirmative.
Who will fight for the rights of software? I only wish the FSF was more vocal about the Freedom of Software that they purportedly base their ideology upon.
Then what's the point? If binaryA == binaryA, this was a useless exercise. But if binaryA == binaryB, and binaryB is superior, then we have something.
But does it ru-- :)
Nevermind, no points to spare
I've setup a mirror for the source at http://43.244.87.231/cle266.tgz
Be nice to it, and check the original site after slashdot effect goes away.
I can't see that any real thought is required here.
Boycott the bastards and *most importantly*, let 'em know why.
A few thousand e-mails to sales@via.com should do the trick...
If a giant oil company wanted an abortion, would W's head explode?
The silly thing with all of this is that the drivers and support for this card that result from the reverse engineering will ultimately result in more sales. It seems so counter-intuitive for VIA to resist this.
Let's harass them for not releasing the code, reverse engineer it and post it everywhere, until they get mad and discontinue Linux driver development altogether! Then xine and mplayer will work GREAT!
Why not use a program that automaticlly takes the binary and builds a C program from it? You still have to pick through the logic to give things helpful function/variable names and refactor but it'd save the step discribed here. In the past when I've reverse engineered binaries that is the type of tool I used. Any good reason for doing this by hand?
This still begs the question.. why not just release the damn source? If we can reverse engineer the drivers what would keep the competition from doing so? Why harm your customers for a false sense of security?
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
Maybe I'm only an ignorant sw-architect, but why on earth do you need the source code in order to develop for a target system (i.e. an MPEG hardware decoder)?
Interfaces? We don't need no stinking interfaces!
From the article:
Oh yeah. Much more readable.
"A great democracy must be progressive or it will soon cease to be a great democracy." --Theodore Roosevelt
Hardware decoding allows for much higher resolution video. Furthermore, specialized hardware typically have more accuracy when decoding the stream. There's additional features too: you can allocate, say, more bits for dynamic color range, fractalize regions that have semi-random "noise" distribution (like tree leaves from a distance) and so on that can improve video quality (to help eliminate obvious artifacts). I am not saying all hardware decoders do this, but these are some advantages. It's very analogous to having specialized 3D hardware to handle graphics rather than "letting the CPU do it".
Join Tor today!
This seems like a good way to scare developers away from Linux completely.
Abortion is advocated only by persons who have themselves been born.
--Ronald Reagan
It won't produce the same code. Different compilers do things different ways. In the end the binary produced will run the hardware the same way and that's the goal.
Very clever, but I thought reverse engineering worked on a functional level.
He did do functional analysis to make it work. He understood what the thing was doing. If he did not, his code would never have worked. He made little doodles and what have you to make it clear to himself. Now it's in C, the diagrams are much easier to make, though we can be sure he's going to share his diagrams as well. That way other people can make nice software too.
IANAL, but I don't think the source code is legally safe if VIA wants to go after it.
I don't know why you think that. He could have had his computer tell him what it was doing instead of using IDC, no? It's not like he dumpster dived code like old Bill Gates did BASIC. He understood what the code did and reimplemented it himself. Even if he did have dumpster dived code, he could use that to make a functional diagram and then use that to write new code and the results would be the same.
If there is a legal problem with this, there should not be. Why should people be afraid to understand what their machines do and then share that information? So someone else can make money of evryone else's ignorance? Shit, no one would be able to get anything done that way.
Friends don't help friends install M$ junk.
Well, he has done the first part of a reverse engineering process - he has worked out, by inspection of the target, what is being done.
However, he now needs to write the specifications for the hardware, and publish THAT, so that somebody else, somebody who has not seen the binary driver, can write a program based upon the specifications.
Should this not be done, then this code, while interesting to individuals, would be pure poison to anybody who has any intention of distributing this code in a commercial way (e.g. a distro).
And writing a specification for the chip, by inspecting the code, is far more difficult than simply reverse compiling the binary.
www.eFax.com are spammers
With the ever-increasing clock speed of our CPUs, what is the point of having a hardware MPEG decoder? I understand that p2-400 is sufficient to play DVD-quality movies. The amount you spend on the hardware decoder could have been better spent on memory or video card or CPU or whatever. Now, a hardware encoder would certainly be useful as encoding is still very CPU-intensive. I was contemplating a tivo-like box with a hardware encoder. Does anyone know if hardware MPEG encoders are supported on Linux?
___
If you think big enough, you'll never have to do it.
...is just their method of getting there. What would be better is chip specifications. Then we could make our own method. Getting their method, all we can do is maybe tighten or optimize things up. But by knowing specifications, we could come up with a completely better method, with no ties to the old one.
Slashdot makes me just a little nuts. Why should VIA pay to develop anything if they don't actually own the property rights?
You'all just want free sh|t.
I'm willing to bet that if you developed something that might be valuable, you would want protection. But you guys all are frustrated little socialists who can't see past your hands. It's only when your dime is on the line that your "rights" are important.
Just to prove it exists:
Reverse Engineering Compiler
As if it wasn't bad enough that you open source dorks stole the entire SysV Unix codebase from SCO now you want to reverse engineer someone elses work to rip it off so that you can watch dvd's. They have this new thing available now for $35 at your local walmart ... it's called a dvd player and it was made to watch movies on your tv.
J/k :-)
the decoded mpeg2 cannot be captured, it's decoded directly to the video memory.
Can you imagine what would happen to VIA's sales if they somehow offended M$ and M$ retaliated? They could keep VIA in the dark or give them bogus SDK info so that their hardware would not run well under Windblows. Even witholding a dinky little check here is damaging. Harware makers that defy Microsoft are doing something heroic and should be rewarded.
Once enough hardware makers tell Microsoft to shove off, it's all over. In fact, it's already all over. Windows already enjoys the bad reputation they deserve. When you buy something for Windows, the odds of it working are only marginally better with the goofey M$ binary driver than they are with a free driver. There are some exceptions to this rule, like winmodems and crappy little digicams, but the gap is closing quickly. Everyone will be better off when stuff can be chosen on grounds of technical merit rather than M$ favor.
Friends don't help friends install M$ junk.
Hardware T&A support?
Oh? Please enlighten the rest of us, who lack your wisdom. Have you ever actually *met* a paid professional who was [even remotely] qualified to write a device driver?? I suspect no, so I'd say to you: "Please crawl back under your rock, you Micro$lut Whore." Is that fair or what??
If a giant oil company wanted an abortion, would W's head explode?
It was Mr. Macintosh himself, Steve Wozniak. He also agreed, only paid professionsals should be allowed to code. There ought to be a law!
Obviously you've never used VIA processors before. They are notorious for their slow FPU's. In fact, before their latest top-of-the line model - the Nehemiah, their FPU's of previous models always ran at HALF CLOCK. Ouchy.
:O
But, even at full speed a similarly clocked Celeron kicks it's ass in every which way. That said, high performance is not the stated purpose of the Centaur/Via CPU. Its low watts, coupled with the decoder make for an excellent all-around box. I've built around 7 or 8 of these myself and they are excellent for what they are designed for (think: mom and dad or net terminals, not Half Life 2).
I have a few of these floating around the school here now as basic net access / workstation terminals and they are hugely popular - especially in light of what they replaced (AMD 300's). There's nothing like tearing apart some ancient computer and putting one of these boards in it. 90% of the time, it's simply cavernous in there (so much space!)
Last week I put one in an Aptiva and realized that if I was an enterprising person (read: man with a Dremel) I could have fit TWO of them in there as a dual workstation!
So to sum up, they're small as hell (you have to see it to believe it), simple, fun, easy to configure, but don't plan of using them at the next Fragfest 2003 (c)
"...Well, there's egg and bacon; egg sausage and bacon; egg and spam; egg bacon and spam; egg bacon sausage and spam..."
This is not reverse engineering, he dissassembled the code and pretty much copied/ported the result to C. I don't think this meets any cleanroom standards and the code is dangerously contaminated. To use this work you would have to get someone else to reimplement the driver without looking at this contaminated code base. That means they need to be passed a description of the hardware interface inferred from observations of how this driver works, and the code produced by dissassebling the driver needs to be tossed in the garbage can.
Who taught anyone that dissassembling someone's proprietary code and doing a line for line port then publishing the result was in any way legitimate?
Just like we like to champion choice in operating system and GUI, software companies have the right to choose their business and development models.
It is up to the OSS crowd to adapt to that, and what better way than innovation? It has already started with OGG.
First of all, it's just a small wrapper library that is comparatively easy to reverse engineer.
Second of all, there is a library we can reverse engineer.
Third of all, the guy is using the VIA forums to spread the word, so VIA obviously knows about this, and they haven't sued.
To me this rather looks like they were waiting for someone to reverse engineer this, because they couldn't release the sources themselves for contractual reasons. Don't just assume people are evil, maybe they didn't have a choice and did what was in their power to give you the means to help yourself.
If you have to reverse-engineer a driver, what the fuck good is it?
So was his granduncle on his wife's side.
Redundant?
Please tell me where in this story this was posted before.
I think the poster was going for "funny"
Look at all the professionally made software he's sold! Look at all the ones that get pirated! Hobby apps like Linux only get used by a few people. Go away, troll.
So are his manboobs.
... is more apt.
Has anyone seen a comparison where Linux was compiled on different compilers, and run benchmarks? It would be interesting to see which compiler creates the best Linux.
"..Typically, only requests from companies developing product for sale will be approved.."
/. ?
Has the article submitter actually asked them instead of going by a press release and venting on
1. Set up company
2. Tell them you want to develop for sale
3. Make some nice Linux Drivers
4. Sell copies of software for 0.01 $CDN. (Cheaper than 0.01 $US)
5. ????
6. Profit
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
If mini black holes, Planet X, super volcanoes in Yellowstone Park, imminent destruction of the galaxy, and God know what else, now there is this, suddenly dying from instant lung hemorrhage. Where's Fox Mulder when you need him?
Code porting MPEG driver from assembly to C.
"Too slow chicken marengo" - The Cat
Ironically, the sources to that "Reverse Engineering Compiler" are not available in the public domain...
I.O.U One Sig.
Looking at the examples provided with the decompiler, I'd say that the answer is obvious. The decompiled code bears little resemblance to the original C code, and is almost useless for understanding what it does.
This is not to say that a decompiler cannot (in theory) do a better job ...
How was he supposed to have figured this out? Standing on his head and chanting mystical phrases while smoke billowed into the characters of the source code?
This is absolutely how reverse engineering works. And, in fact, this is almost exactly how I've been working the reverse engineering of Ten-Tec RX-320D receiver BIOS. I disassembled the source into ADSP-2101 assembly and have been meticulously putting together a pseudo-C version which is a lot easier to read and gives a much better representation of the function of the code. Just disassembling code does not count as reverse engineering--the idea is to figure out exactly what the original source does at a much higher level than the mere instruction. Ideally, reverse engineering should yield enough information about the product in question that a client can use the driver/library without ever needing to know about what's underneath. Disassembling the ROM of a Palm VIIx isn't going to get you very far with regards to figuring out the low-level interface to the radio library. Turning that disassembly into the knowledge suitable for interfacing is most certainly reverse engineering.
And that's exactly what this guy is doing.
Karma: Excellent Birds (mostly as a result of listening to Laurie Anderson)
They're far from being leaders in their field. They should welcome support from the open source community. If they don't, refuse to support them in any way. Make them the same sort of pariahs as we have made SCO and see if their attitude changes then...
"Oh my God. This is terrible. This is the end of my Presidency. I'm fucked."; ~ Donald J. Trump
Lets not forget the hardware mpeg2 decoder - dxr3. A lot of people have worked on this and it has resulted in a very decent driver. It has had absolutely zero help from sigma. There is even hacks to make it display rgb directly to your tv, bypassing the crappy composite and svideo.
Wow, does that really count as reverse-engineering?
Anyway, VIA has to prevent the stealing of their and their partners' intellectual property -- which is the norm in the Linux community. Not every company like IBM can just toss away in the wind the work of others. Perhaps VIA just has no choice but to not open-source their software? Could they be licensees of certain technologies over which they dont have control of the publishing rights, such as DVD encryption and others. nVidia and ATI cant open-source all of their code either.
Yes, his manboobs are obviously Nazis, like that faggot's from American History X.
What kind of message does this kind of thing send to hardware manufacturers that might want to release drivers for Linux? Write a driver for our OS and we'll reverse engineer it and do what we like!
I'm not sure this is the best method to attract quality drivers for Linux.
The author of the article did much more analysis than that, but even that were all he did, I think that would still be legal given the purposes for which he did it. Not all copying is copyright infringement.
"[...] the fair use of a copyrighted work, including such as by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright." Title 17, section 107.
I put the second "such as" in bold to emphasize that the list is just some examples. It is not intended to be complete. This point is covered The Nature of Copyright: A Law of User's Rights by L. Ray Patterson and Stanley W. Lindberg.
See also the doctrines of Scenes A Faire and, more importantly, Merger, which establish that when there is a limited number of ways to do something (I think poking bits in hardware registers qualifies), copyright shall not restrict expression of those ways. From a web search, "[...] the 'merger doctrine' of the United States indicates that the expression is not copyrightable if the idea embodied in the expression can only be effectively expressed in one or limited number of ways. One thing worth noticing is that this doctrine does not apply to fictional works. [...]"
I am not a lawyer. Do not use this as legal advice.
reverse "engineering"?
is that what it's called now?
back in the day, i used to just double click on the mpeg clip on my computer, and all you could see were "reverse cowgirls". whatever these "engineers" (or pr0nstars as we used to call them) are doing is just great. My "intellectual property" is now as WIDE-OPEN as open source for you!
ackk kids.... when do they ever use the proper symmentics.. (old man like me cant spell...)
my blog
Suck it!!
Ahh... critically important information I wasn't aware of. OK, he's safe to use the code. If I could I'd mod you up myself. This obviously makes a huge difference to his ability to release the driver.
SAY NO to software patents in europe ....... sign up here : http://petition.eurolinux.org/index_html
PLEASE HELP URGENT
To go from binary to C is fairly clean because the C will hopefully not produce an identical binary. C is far enough away from the binary that it is itself a specification language.
If another person rewrites the C who has had no access to the disassembled code, you have the clean room approach. The clean-room approach is cautious but it isn't compulsory.
See my journal, I write things there
Translating from binary to assembler is inexact at the best of times. Even when you have a relocatable object file, it isn't a .s file with instructions and data completely differentiated.
See my journal, I write things there
SO what happens in many player/drivers is that they trade off quality against playback speed and integrity (frame dropping). Speed up the decoding and the quality is improved.
See my journal, I write things there
And yes, I knew you can't lend your vacuum cleaner to your neighbor here in Denver. You think I'm an idiot.
because
Although it did only start about a fortnight ago.
list of boards so I can avoid please?
A blog I run for the wealth
A Mini-ITX Linux system that used the functionality provided by this driver, had a 3-second BIOS bootup time using Linuxbios, plus a PCI TV tuner card with hardware MPEG-2 encoding, would be a pretty impressive media center.
Breaking laws by reverse-engineering and giving them chance to sue us is exactly what we don't need here.
Breaking laws? I'm pretty certain copyright law explicitely allows reverse engineering for interoperability purposes if the copyright owner is not willing to provide the specifications.
He owns the hardware; therefore, he has a right to make use of it. The details required to write a driver form a part of the operating instructions for the hardware, and anyone claiming them to be "proprietary secrets" should be laughed out of court with a dusty bootprint on their arse.
Is it a proprietary secret that "Esc", "K", followed by a two-byte binary number presented units-first between 1 and 480, followed by that many bytes, is the code used to select bit-image mode on an Epson-compatible Dot Matrix Printer? Of course not! why, Back In The Days, when if you wanted software you pretty much had to write your own, the printer would have been useless without such information. So the manufacturers used to provide it in the handbooks. Kit that didn't come with adequate documentation, didn't get bought.
Today, with pre-written software in abundance, manufacturers are becoming sloppy and not documenting fully how to interact with their products. For the casual user, this isn't a big problem, because they were never going to do anything with this information anyway, so why waste paper or plastic telling them it? But if there is even one user who wishes to do more than what it says on the box, then it suddenly becomes a very big deal indeed.
My analogy is that he used "reasonable force" to obtain information to which he was entitled, after polite request had failed. The law is quite clear that in certain situations, reasonable force may be used. This situation is more "gentle" and relies less on quick decisions than, say, physically moving a person who is trying to resist. {He could have obtained said information by holding a knife to someone's throat at the manufacturer; this would likely be seen as more than reasonable force.}
We should be writing to our elected representatives now to make sure it becomes mandatory for manufacturers to supply full hardware specifications, gratis or at cost, to anybody who wants them. Concealing details is a dirty, lowdown, scumbag, coward's trick that will cost companies sales. Please don't betray your cowardice by bleating about "competitors gaining an advantage" - you will have access to your competitors' documents, too, and if your competitors manage to do a better job than you, then you failed it! I have no sympathy, either, for those who whine that people might find it easier to break the law if they were given certain information. It is already more than easy enough to break the law. A few extra ways aren't going to make any difference here or there. You shouldn't rely on doing crap design and keeping things secret; it's another form of corner-cutting. Do it properly or not at all.
If the guy is ever taken to court, his best chance is to push for a trial by jury an hope that, out of twelve people, he can convince two of them that, although he does not deny what he did, it is the law that is wrong this time and they can acquit him. If this happens often enough the law will be changed.
Je fume. Tu fumes. Nous fûmes!
Grrrrr! The angry capitalist speaks!
Who said anything about VIA paying for anything? This thread is about a guy using his own industrious labor to open a channel to speak to a previously closed chip architecture. Are you just mad that a man would actually use his labor to help the public at large rather than hording the gains to himself and biting the fingers off of anyone who would touch it? It just makes you guys sick to see anyone express a streak of altruism, doesn't it? Your path is the way toward tyranny by the rich and corporate feudalism. His is the way toward freedom and enlightenment. Money isn't everything. We are human beings not faceless consumers.
- Hail to our fearless misleader! Fool speed ahead!
I don't see this as a good development. They should have studied the device, how it responds to input and produce a clean room implementation - not disassmble the binary driver.
What if MS did not have a Windows driver for a device and decided to skip some development effort by dissambling the source from a binary of a GPLed driver. I bet most people here would oppose that - this is no different.
This is different. This was not a commercial product that was disassembled.
The VIA driver was released with the notice:
"Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files... to deal in the software without restriction including without limitation the rights to use, copy, modify, merge, publish, distribute, sub-license and/or sell..."
The driver was released for people to play with in an attempt to stimulate linux development. Unfortunately no source code was provided, and no spec/development manual, so actually creating a working application against the driver is difficult.
Hence the reason for generating the source.
On your final point, there's nothing stopping MS using GPL code, as long as they released their derivative work under the GPL.
Another irony: the decompiler pages are translated on that site into one other language: Italian, Ivor's native language.
- First they ignore you, then they laugh at you, then ???, then profit.
You whinners need to get over it. Reverse engineering is a tried and true, and LEGAL, activity. It is A-OK and there aint shitall anyone can do about it. A hell of a lot of stuff that exists was reverse engineered or developed off something that was reverse engineered. Get over it.
In Bushworld, they struggle to keep church and state separate in Iraq as they increasingly merge the two in America.
I wonder if the decompiled code for most programs would be better designed than the original source? No doubt it would be for perl, but what about anything else? ;-)
Higher Logics: where programming meets science.
_IOR('v', //118 192+3,
VIAGRAPHICINFO),//0x805476C3, &gVIAGraphicInfo )
Since when did // mark comments in C?
Boys from the City. Not yet caught by the Whirlwind of Progress. Feed soda pop to the thirsty pigs.
Not.
Italy is just as much a hostage to bad treaties as the US is.
Most of the things global companies want that WON'T fly in the US (owing to a Constitution) end up becoming de-facto law when the US signs "business treaties".
We're all screwed.
Since when did // mark comments in C?
Since C99
If you can write into the frame buffer, you can read from it. DRI's got raftloads of protections from people taking control of the DMA pathways and using them to mine data from other process' memory space.
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
I thought perl was interpreted, not compiled.
"We have got to make Stan understand the importance of voting, because he'll definitely vote for our guy." - South Park
The solution is simple, no linux source == no sale.
Thanks for the tips...
I meant the source of the interpreter.
Higher Logics: where programming meets science.
How may times to I have to make this post... ? :-)
.it was a nice to have to split my site up.... and it wouldn't hurt to got there every once in a while.
Re.
not that distribution of the decompiled driver code is necesarily legal. I think it probably is not,
As I posted several times already, the VIA license (posted in this thread somewhere) with this library explicitly allows derivative works.
so I also have no idea what an Italian court would think
I'm in the UK actually, the
The UK courts would take the same attitude to the Italian or US ones when it comes to copyright of code. What we luckily don't have yet is anything as stupid as the DMCA or software patents... but with enough lobbying from hollywood and the US software industry it probably will happen.
This card was probably one of the last rage cards. Later Rage 128's have some pretty damn good mpeg acceleration. I don't think it has full hardware decoding. But either way, This thing has *two* of them. you can watch DVD's with it using pretty much any CPU with a board that has AGP.