Slashdot Mirror
Porting Games From Binary
CowboyRobot writes "My favorite Slashdot links are those that inspire me to embark on meaningless and time-consuming quests. This is one of them. Two Canadian game developers at Digital Eclipse have a thorough explanation of how to port a game using nothing but the binary stream coming out of the cartridge. They use the TRS-80 and Phantasy Star III as examples."
It's called ROM dumps and Emulation. Nuff said.
Life is not for the lazy.
Trolling is a art,
They're in unary, you insensitive clod!
It was the worst of the series.
whoops, they're talking about removing the emulation layer. Mod me down, -1 dumbass.
there goes that whole thing nintendo's trying to do in china...
In Soviet Redmond, software programs you!
It's this kind of pointless endeavour that gives geeks a bad name.
It's this kind of pointless endeavour that makes me happy to be a geek.
Some people climb mountains, other disassemble 8085 binary code.
-- "Can't sleep, clowns will eat me!"
oops
printer-friendly, one page, no ads.
Sounds a lot more like emulating to me. Run the binary stream on a virtual machine that works like the old device...
Kjella
Live today, because you never know what tomorrow brings
Unless I'm missing something, this novel idea is complete garbage. Yes, sure you can disassemble the machine code, produce some C code from that and then recompile for a new target CPU but it's not going to work for the vast majority of applications.
The reason: hardware.
Even your average 80's arcade machine relies upon custom hardware for virtually everything. The main program spends most of its time simply adjusting registers to control sprites etc, and reading from hardware to detect collisions and so on. This new code you've generated for a new CPU will still expect the same supporting hardware...
Code, Hardware, stuff like that.
One problem that I don't see addressed in the article is the different approaches games use today compared with those on old hardware. For example, on machines where the disparity between CPU and memory speed was not as great as it is today, it was common practice to precalculate many maths-intensive operations into lookup tables (usually in the form of sin tables, cos tables and so on). On a modern processor this level of precalculation can result in slower execution than just working out the maths on the fly, so most games do minimal precalculation of things like maths functions. While it's often the case that speed is not really an issue with old game ports, it would be interesting to see how they would approach the problem of porting and ensuring that the resulting game makes optimal use of the target machine.
In our lab we sometimes have to reverse-engineer components for which we no longer have documentation. It would be very simple to export the data of a video game cartridge by accessing its ROM module with a binary card reader. The trickier part is to get the offset right so that any fluctuations will be evened out on the oscilator. One could then re-compile the imported code into a new platform similar to what the folks at MAME do.
What we've done is imported "Yars' Revenge" from the Atari 2600 did some elevation emulation, ported the code, and re-compiled to make it work on the Intellivision. We may be selling that cartridge commercially as there is a great need for 8-bit cartridges.
Which is nice.
They're talking about -altering- the source code of an application so that it will run on new hardware.
Not running the existing code through a software emulator on new hardware.
They are aiming to (for example) map the display instructions from Pacman on the Atari 2600, to x86 windows API display instructions.
They will also have to translate all game logic more times than not (to valid x86 logic instructions), and may have to alter the stored data in the event of differences in endian-notation.
The resulting translation will not suffer from the overhead emulators create.
Interestingly, I see this very feature as becoming one of their largest problems. pre-pentium-speed game programmers tended to rely on the clock speed of their original target hardware to set the update rate for the game. Trying to play frogger on even a 486 would be an impossible blur.
Timing control will be their biggest hurdle.
// "Can't clowns and pirates just -try- to get along?"
What? Someone found yet another way to make a copy of 'Rygar'?
Damnable geeks.
Can't wait to play true classics on my Pentium 4...
Stop by my site where I write about ERP systems & more
Server is spewing intermitant database errors so:
What if you have to port a program, but all you have is a binary?
Bridging the Gap
Typical software development involves one of two processes: the creation of new software to fit particular requirements or the modification (maintenance) of old software to fix problems or fit new requirements. These transformations happen at the source-code level. But what if the problem is not the maintenance of old software but the need to create a functional duplicate of the original? And what if the source code is no longer available?
This exact problem arises when trying to reproduce the original play of old arcade games on modern devices. The game play is so well known that anything short of the original is unacceptable. Often the source code is available, but it may be incomplete and may not cover all of the patches that were added to later production models. In addition, it is too expensive to provide copies of the original hardware.
Providing faithful emulations of video games (and old home computers) is our primary experience with this process. But the same techniques can be applied to other areas. Aging hardware and software can be replaced by new hardware with a completely compatible program.
BRIDGING THE GAP
The general problem can be expressed as "bridging the semantic gap." You must create a program that precisely maps the meaning of the original program onto a host system. Primarily this means an interpreter of some kind for the target processor's instruction set, but one must also deal with I/O (input/output) devices. Such an interpreter is known as an emulator. (See "The TRS-80: A Simple Emulator" on page 54 of this article.) If the program is automatically converted to a different language, it is called a translation.
Why are we tackling the problem at such a low level? Mainly to achieve the highest fidelity possible to the original. Emulation is about mapping semantics. The semantics of hardware are usually well documented either by circuit diagrams or chip-specification documents. Internal layers inside software are usually designed to much looser standards, so it is unlikely that specification documents--if they exist--completely describe the behavior. In fact, the software itself is the most authoritative description available. It is true that chip specifications are not always complete or accurate, but chips are reused and over time the deviations become widely known.
The semantic gap between the target and host systems is not purely an abstraction. It can be quantified as follows: G = number of host instructions to emulate one target instruction. Given G and some idea of the relative speeds of the host and target system, you can quickly decide if emulation is feasible. The problem here is that the value of G is a function of an actual emulator. A rule of thumb is that G is at least 10, but practically speaking, 10 is lower-bound for systems that are quite similar. Although time is usually the overriding concern, there is an analogue for a semantic gap in terms of storage space.
Figure 1
Emulation and translation start with the same inputs (the ROM data and the hardware documentation) and produce the same result: a copy of the original running on a PC. The difference comes in how the ROM data is handled. For the emulator, it is simply a parameter to the program. For the translated version, the ROM is converted and compiled into the executable.
The semantic space gap is the ratio of the size of the host program to the size of the target program. For an emulator, the host program size is broken into two pieces: the host's representation of the target program and the emulator code and associated tables. Unless the host and target are radically different, there is little to be gained by significant changes to the representation of storage. Thus, if the emulator code is ignored, the semantic space gap is typically exactly one.
Intuitively, the value of G really depends on how differe
inspire me to embark on meaningless and time-consuming quests
This is as close to having a girlfriend as the average Slashdotter can get.
This methos is of genuine value for cases of applications where the source has been lost and needs alteration - for just running invaders/pacman/etc an emulator is just as good - since you have to slow things down to run at 100% original speed. The best emulators really o absolutely emulate every facet of the original CPU & hardware - the VICE C64/PET/VIC emulator runs on a 1mhz interupt, that, as I understand it, emulates all the states of the 6502 processor exactly. If you convert bin->asm->c you would still need to put hold-states in the C code to make it step at exactly that speed, cycle for cycle. Where this method would be better than MAME/VICE style emulation is when you need to patch the code- to upgrade it, fix bugs etc. This applies more to non-game type code where old (and failing) hardware/software needs to be migrated to new hardware, maybe with minor code changes.. And if you are not tied to executing at a set pace, then bin->asm->c is a really cool thing to have.
"You lied to me! There is a Swansea!"
Wow, those guys ARE good. They even ported their company web server to a TRS-80.
But some areas of the code still need work:
int fork() { return 1; }
Fatal error: Call to undefined function: message_die() in db/db.php on line 88
they sure summed it up in a few words
if only i knew how i could apply this to actually porting a game...
Emulation:
*Each assembly instruction interpreted by an interpreter
*Compile once to run N applications
"Translation":
*Each assembly instruction translated to a C macro
*Compile N times to run N applications
Looks like emulation wins. If an emulator has JIT then "translation" losses its only speed advantage too.
For console video games, why in the hell would you translate the language? All the consoles had funky hardware that games used one time or another.
For 1, the NES uses mappers to display games. On emulators, many mappers are not supported. NES game producers also put hard-coded timings in games. So if your recompiled game isnt the exact same multiplier of clock frequencies, many will exhibit starnge behavior or just lock up.
Next, the SNES had pretty much basic AppleII GS hardware with the exception of the 32 channel sound card. It had a sound cpu which could hold 64k code along with samples. A problem that the makers of ZSnes had was determining the random noise generators formula. On the older games like Chrono Trigger, the wind would sound like square waves going up and down. That sounds fun, compiling a game when you find out that the hardware emulation controls wernt right.
Jump to PS2. Who would have thought that a comuter like that would be available to the public? One that has little GFX ram but a huge bus. Not to mention a full FS to "compile". What pitfalls occur in the 3D hardware?
Emulation is still better as it offers a replacable shim to modify and add features. You can also use other emulators.
Amazing feat. It's completely rewritten in C to gain exactly the same functionality as the original code, with only the binary / dissassembled machine code to work with.
-larsch
Well, I'm not porting to a different platform, but I am converting old bootable 360k floppy disk games into games that can still be run on modern computers.
I've finished the Epyx game JUMPMAN
http://www.classicgaming.com/jmanproject
and I'm currently working on Beyond Castle Wolfenstein
This is a very time consuming hobby, but for me it's a matter of preservation than anything. I don't want to have to dig out the PCjr from my closet everytime I want to play these oldies.
-jeff!
Too much self modifying code, the guy is a fool.
He does not talk about dynamic recompilers (that use page management dirty-bits to track self mofifying code)
He does not talk about self modifyingh code
he does not talk about timing loops and differences in different opcodes
He does not talk about lot of things.
I suspect he is not the one to talk about this stuff.
Tom Dowdy (old email dowdy@apple.com) knows all about DR he wrote a paper on it.
So did the guys who wrote a DR for recoding Mac 68K on an Intel at full speed, the Executor guys.
Its incredibly hard work and takes half a millikon dollars to do one correctly that can handle self modifying code.
The guys from Connectix (now Microsfot) did it with having the x86 intel run on a PowerPC.
DR (Dynamic Recompilation) technologies do not make source code.... because that itself would take tens of millions of dollars to engineer and at least a few huge teams of experts, and even then would not be possible to automate.
That guy is a fool, but I do agree that stuff NOT written in assembler (compiled stull written mostly after 1999) couple be transcoded into files because C++ and C on moder architectures rarely has self modifying structures.
My favorite Slashdot links are those that go down within about 12.5 seconds of showing up on the front page. I know I must be missing something really great!
It Is the Nature of Information to Transgress Artificial Boundaries
EMULATOR
The GeekNights podcast is going strong. Listen!
JIT translation is RAM-hungry. "Translation" has a RAM advantage as well, which is useful on handheld devices with little RAM.
Will I retire or break 10K?
The guys at OS/2 Netlabs have been doing this for years now. It'ts called Project Odin They run Win32 apps on top of OS/2 with no emulation: they "translate" binaries on-the-fly. They even run Win32 drivers on OS/2!!
Either that or you create macros or a virtual machine to handle the hardware access.
Now a remake can be hard to do, one example is Telengard for Windows which is based on the old C64 classic game by Avalon Hill. The original sound and graphics are there, but the code had to be rewritten. The sound and graphics are in a new format that the supported libraries that are used can access. As any C64 developer can tell you, the C64 BASIC used a lot of peek and pokes to memory, and Machine Language accessed the chips and memory directly. What he had to do was rewrite the code so it acted and played like the original. Some things he didn't get right, but he did what he could to fix them. Telengard for Windows is not emulated, but it is a native Windows executable.
Remember, Slashdot does not have a -1 disagree moderation, and no, troll, flamebait, and overrated are not substitutes.
I've had to suffer through these guys' port of Joust on shockwave.com (in fact I have the high score for the month currently) and it has several inexcusable flaws. The most annoying is that it doesn't always respond to keypress events properly - rather critical functionality for a game. It doesn't actually seem to drop the events, which would be bad enough, but it ignores them for as much as half a second at a time and then spits a whole bunch out together. If your game wasn't screwed up when it seemed to ignore the event, it's damn sure going to be screwed up when they all get played back in a burst along with the other keys you hit to recover from the original failure. I know Windows is not a real-time system, and I've worked on real real-time systems (microsecond response times) so I do know the difference, but there's no excuse for a delay this long on an unloaded modern system. Other games don't seem to be afflicted by the same problems. The problem is not in Windows; it's in Digital Eclipse's emulation of the hardware on which the game originally ran (even if the code is translated the non-CPU hardware still has to be emulated).
There was also a time when the game's speed calibration was totally broken. I'd play on my 600MHz laptop and it would be just about right, but when I went to my 1.5GHz desktop guess what happened. Yeah, everything in the game was moving about 2.5x as fast and the game was unplayable. This only persisted for a week or two, but it's still not something that should happen in a version that's released to the public. These technical failures, combined with their apparent acquiescence to Shockwave's desire to add deliberate player-killing features to their translation (the very laws of physics in the game change after you get a good score, and I've looked at the original ROM code so I know exactly how they're doing it) have left me with the impression that Digital Eclipse is both incompetent and unethical.
Slashdot - News for Herds. Stuff that Splatters.
Odd TRS story: the bicycle manufacturer GT made a mountain bike called an RTS-1 (Rear Tuned Suspension). It shouls have been the more logical TRS-1 (Tuned Rear Suspension), but the Tandy trademark apparently stopped them using the initials TRS. This was about 10 years after the TRS had gone out of production!
When I am king, you will be first against the wall.
These people are the ones who translated Phantasy Star games to the GBA.
The GBA does not have the power to emulate the Mega Drive (Genesis) even with a JIT compiler. Hence, a different method would be needed. Either simulation, recompilation, or translation. Translation was the best option given what they had to work with.
Emulation is wonderful and all but it certanly doesn't work in all situations.
FYI, I believe that PS1 was emulated and PS2,3 were translated in Phantasy Star Collection. They're not saying to use this as a replacement, they're just telling you how they did it. I'm letting you know why.
>>FYI, I believe that PS1 was emulated and PS2,3 were translated in Phantasy Star Collection.
Ambigous. Do you mean that the PS1 code was emulated in the PS2? If so, you're wrong. There's 2 main CPU's in the PS2. One is for PS1 and a bios-like chip to interface to the main CPU to make the PS2 rom calls. They also use this shim to add anti-aliasing.
I have a question--actually a serious question. I visited, and the screen looks a little bit like "Miner 2049er" starring Bounty Bob.
Maybe I'm just suffering from old-timer's disease?
DUCT TAPE: The Election Supervisors' Secret Weapon
PS is Phantasy Star you moron.
Why not a compiler that takes machine code and compiles it to a new machine code? Such systems have been made. The Java JIT compilers take java bytecode and compile it to native machine code. The translator on the Transmeta chip takes AI-32 and comiles it. Why not one that takes say an MS-DOS program and converts it to Linux? You would have to map every interupt and dos call but it could be done. Selfmodifying code would be a big no no or a huge challenge for the compiler. All in all it could be interesting.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
I mean wow guys, you just run the C compiler backwards and it will disassemble the object code into C.
And it tells you "Paul is Dead" at the same time.
These kids now adays...
Uh, PS was referring to Phantasy Star, not PlayStation, as your reply seems to indicate. He was saying that Phantasy Star 1 was emulated, and Phantasy Star 2 and 3 were translated.
Dude, I think I can see my house from here.
no idea why i'm replying to this but...
that is actually the Konami code, it works in pretty much every konami game out there, just enter it at their logo.
Famous Last Words: "hmm...wikipedia says it's edible"
...you insensitive clods! (To all those that's been saying "emulation is better".)
IMHO, It was smarter to do it this way than try to emulate the genesis on a GBA. The latter might be possible one day, with enough hacking breakthroughs, but this is no less generalizable. They have a perl-based translator which they can improve, much like an emulator often need to be improved the more games it is to support.
The apo'strophe: a punctuation mark used on 'slashdot to 'signal that an "'s" character i's ju'st ahead.
For console video games, why in the hell would you translate the language?
For posting a reply, why in hell wouldn't you read the F*&^ing article?
The article talks about the EXACT reason they chose tranlsation vs emulation. They are translating to the Gameboy and in their words:
The CPUs are so close in performance that direct emulation can immediately be ruled out. The larger cartridge size means we can accommodate some code expansion caused by translation.
Previously in the article they mentioned:
A translation of the target program into source code will be faster than an emulated version and more amenable to modification.
And sure enough the Gameboy and Genesis had different display resolutions that had to be accounted for.
Why was the parent mod'd up???
Yeah, every single one, like Castlevania 1, 2, 3, Base Wars, Metal Gear, Snake's Revenge, Bayou Billy, Batman returns, Blades of Steel, Bucky O' hare, Double Dribble, Goonies 2, Jackal, Lone Ranger, Rush'n Attack, Super C, TMNT 3, Top Gun, Track and Field, Track and field 2
I'll stop my sarcasm there. It worked on 2 games: Life force, and Contra. Gradius 3 had a modified version of the code.
A far cry from "pretty much every konami game out there"
If they had the original assembley source files, think how easy this would be. Code sections already separated from their data sections. All you would need to do is know HOW the hardware worked and translate the assembler files. Yes, there would need to be a few changes, like resolution differences, possible shortage of buttons, etc.
It's Jumpman, but they are kinda similar.
Miner 2049er
What we've done is imported "Yars' Revenge" from the Atari 2600 did some elevation emulation, ported the code, and re-compiled to make it work on the Intellivision. We may be selling that cartridge commercially as there is a great need for 8-bit cartridges.
Pardon me, but I think one of two things have happened here.
1) The esteemed moderation pool (of which I have occasionally found my self a member) missed a clearly +1, Funny post (great need for 8-bit Intellivision carts?) and modded it as Insightful/Interesting. I scored one like that, once, before the more clueful moderators clicked my link.
2) I have just shown myself to be woefully ignorant, despite having a TRS-80 (with LNW Expansion Interface, heavy as lead) of my own stashed away in the closet.
Both choices, at the moment, appear equally likely.
Stressed? Me? Of course not. Stress is what a rubber band feels before it breaks, silly.
Would be badass to beable to run ZZT on MacOS X or any unix for that matter...its all text...how hard could it be?
I have to return some videotapes...
Don't just think your ROMs will just disassemble and compile.
Disassembling ROMs
Disassembling ROMs seems straightforward enough - I mean, if the binary code is there, it can be mapped to assembly code, right? Wrong.
In the early 90s, exe-packers and stuff that compressess the run-time code was used. At least in PC. What this means, is that you get perfectly clear assembly routine of decompression algorithm, and rest is garbage. You just have to decompress it yourself. Manually.
And there's is this thing called self-modifying code, which some assembly coders use. Changing the instruction while program is running is something that only emulators can do.
Getting the c-file to compile
Well, what I would like to say here, that you still need to make the macros and routines to emulate the hardware, eg. map the memory area to screen,play the fm sound and stuff - You would still have to create run-time emulation.
OMG
that was the grossest thing i have ever seen once i decoded your message.
please please give a warning next time
those of us with 1024x768 had no warning
please
be nice
You can get it as a MAME ROM, if that's any help : )
Brilliant game, BTW.
Vino, gyno, and techno -Bruce Sterling
The font used on the page is TINY and I'm
sorry, but if I have to mess around with
changing fonts just to read something, screw
it! It sounded interesting though...
This guy spent way too much time on this post for it to have a zero score. Besides, he agrees with me, and he's right. :)
This article was interesting but short on technical detail. Here are some other links on static recompilation:
http://www.gtoal.com/sbt/
http://groups.yahoo.com/group/staticrecompilers/
It is unclear to me how they deal with more complex situations such as self-modifying code or code that jumps into the middle of an instruction.
you might have to hack the game.. these people arent saying "disassemble it, compile it and it works!"
it'll just give you the source code, then to get it working on your computer, you must edit the code, use sdl or gl for graphics display, and you might have to take some parts from your favorite opensource emulator and use that for hardware support (look at sonic cd for PC, it has a shitload of DLL's emulating common genesis routines)
so in the end, it's basically emulation, but, an emulator that is hardwired to play a game without all the frills, etc.
or, if you're really hardcore, you can edit the code even more so it'll work with the system's hardware (eg, alsa, oss, windows' sound, etc)
and use sadl or gl for the video, and reconfigure it to use the keyboard for buttons
so then it's more like semi-emulation.
not to mention converting the cpu instructions.