Slashdot Mirror
Porting Games From Binary
CowboyRobot writes "My favorite Slashdot links are those that inspire me to embark on meaningless and time-consuming quests. This is one of them. Two Canadian game developers at Digital Eclipse have a thorough explanation of how to port a game using nothing but the binary stream coming out of the cartridge. They use the TRS-80 and Phantasy Star III as examples."
Trolling is a art,
They're in unary, you insensitive clod!
It's this kind of pointless endeavour that gives geeks a bad name.
It's this kind of pointless endeavour that makes me happy to be a geek.
Some people climb mountains, other disassemble 8085 binary code.
-- "Can't sleep, clowns will eat me!"
RTFA, It is not Emulation, it is Translation. With Emulation you need an Emulator program on the target machine. With translation, the rom is converted to an executable that can be executed independent of any program on the target machine.
------ Curiosity killed the cat. {satisfaction brought it back | it didn't die ignorant | lack of it is killing mankind
printer-friendly, one page, no ads.
Unless I'm missing something, this novel idea is complete garbage. Yes, sure you can disassemble the machine code, produce some C code from that and then recompile for a new target CPU but it's not going to work for the vast majority of applications.
The reason: hardware.
Even your average 80's arcade machine relies upon custom hardware for virtually everything. The main program spends most of its time simply adjusting registers to control sprites etc, and reading from hardware to detect collisions and so on. This new code you've generated for a new CPU will still expect the same supporting hardware...
Code, Hardware, stuff like that.
One problem that I don't see addressed in the article is the different approaches games use today compared with those on old hardware. For example, on machines where the disparity between CPU and memory speed was not as great as it is today, it was common practice to precalculate many maths-intensive operations into lookup tables (usually in the form of sin tables, cos tables and so on). On a modern processor this level of precalculation can result in slower execution than just working out the maths on the fly, so most games do minimal precalculation of things like maths functions. While it's often the case that speed is not really an issue with old game ports, it would be interesting to see how they would approach the problem of porting and ensuring that the resulting game makes optimal use of the target machine.
It is emulating, as they themselves say:
Providing faithful emulations of video games (and old home computers) is our primary experience with this process. But the same techniques can be applied to other areas. Aging hardware and software can be replaced by new hardware with a completely compatible program.
In our lab we sometimes have to reverse-engineer components for which we no longer have documentation. It would be very simple to export the data of a video game cartridge by accessing its ROM module with a binary card reader. The trickier part is to get the offset right so that any fluctuations will be evened out on the oscilator. One could then re-compile the imported code into a new platform similar to what the folks at MAME do.
What we've done is imported "Yars' Revenge" from the Atari 2600 did some elevation emulation, ported the code, and re-compiled to make it work on the Intellivision. We may be selling that cartridge commercially as there is a great need for 8-bit cartridges.
Which is nice.
They're talking about -altering- the source code of an application so that it will run on new hardware.
Not running the existing code through a software emulator on new hardware.
They are aiming to (for example) map the display instructions from Pacman on the Atari 2600, to x86 windows API display instructions.
They will also have to translate all game logic more times than not (to valid x86 logic instructions), and may have to alter the stored data in the event of differences in endian-notation.
The resulting translation will not suffer from the overhead emulators create.
Interestingly, I see this very feature as becoming one of their largest problems. pre-pentium-speed game programmers tended to rely on the clock speed of their original target hardware to set the update rate for the game. Trying to play frogger on even a 486 would be an impossible blur.
Timing control will be their biggest hurdle.
// "Can't clowns and pirates just -try- to get along?"
Can't wait to play true classics on my Pentium 4...
Stop by my site where I write about ERP systems & more
Server is spewing intermitant database errors so:
What if you have to port a program, but all you have is a binary?
Bridging the Gap
Typical software development involves one of two processes: the creation of new software to fit particular requirements or the modification (maintenance) of old software to fix problems or fit new requirements. These transformations happen at the source-code level. But what if the problem is not the maintenance of old software but the need to create a functional duplicate of the original? And what if the source code is no longer available?
This exact problem arises when trying to reproduce the original play of old arcade games on modern devices. The game play is so well known that anything short of the original is unacceptable. Often the source code is available, but it may be incomplete and may not cover all of the patches that were added to later production models. In addition, it is too expensive to provide copies of the original hardware.
Providing faithful emulations of video games (and old home computers) is our primary experience with this process. But the same techniques can be applied to other areas. Aging hardware and software can be replaced by new hardware with a completely compatible program.
BRIDGING THE GAP
The general problem can be expressed as "bridging the semantic gap." You must create a program that precisely maps the meaning of the original program onto a host system. Primarily this means an interpreter of some kind for the target processor's instruction set, but one must also deal with I/O (input/output) devices. Such an interpreter is known as an emulator. (See "The TRS-80: A Simple Emulator" on page 54 of this article.) If the program is automatically converted to a different language, it is called a translation.
Why are we tackling the problem at such a low level? Mainly to achieve the highest fidelity possible to the original. Emulation is about mapping semantics. The semantics of hardware are usually well documented either by circuit diagrams or chip-specification documents. Internal layers inside software are usually designed to much looser standards, so it is unlikely that specification documents--if they exist--completely describe the behavior. In fact, the software itself is the most authoritative description available. It is true that chip specifications are not always complete or accurate, but chips are reused and over time the deviations become widely known.
The semantic gap between the target and host systems is not purely an abstraction. It can be quantified as follows: G = number of host instructions to emulate one target instruction. Given G and some idea of the relative speeds of the host and target system, you can quickly decide if emulation is feasible. The problem here is that the value of G is a function of an actual emulator. A rule of thumb is that G is at least 10, but practically speaking, 10 is lower-bound for systems that are quite similar. Although time is usually the overriding concern, there is an analogue for a semantic gap in terms of storage space.
Figure 1
Emulation and translation start with the same inputs (the ROM data and the hardware documentation) and produce the same result: a copy of the original running on a PC. The difference comes in how the ROM data is handled. For the emulator, it is simply a parameter to the program. For the translated version, the ROM is converted and compiled into the executable.
The semantic space gap is the ratio of the size of the host program to the size of the target program. For an emulator, the host program size is broken into two pieces: the host's representation of the target program and the emulator code and associated tables. Unless the host and target are radically different, there is little to be gained by significant changes to the representation of storage. Thus, if the emulator code is ignored, the semantic space gap is typically exactly one.
Intuitively, the value of G really depends on how differe
This methos is of genuine value for cases of applications where the source has been lost and needs alteration - for just running invaders/pacman/etc an emulator is just as good - since you have to slow things down to run at 100% original speed. The best emulators really o absolutely emulate every facet of the original CPU & hardware - the VICE C64/PET/VIC emulator runs on a 1mhz interupt, that, as I understand it, emulates all the states of the 6502 processor exactly. If you convert bin->asm->c you would still need to put hold-states in the C code to make it step at exactly that speed, cycle for cycle. Where this method would be better than MAME/VICE style emulation is when you need to patch the code- to upgrade it, fix bugs etc. This applies more to non-game type code where old (and failing) hardware/software needs to be migrated to new hardware, maybe with minor code changes.. And if you are not tied to executing at a set pace, then bin->asm->c is a really cool thing to have.
"You lied to me! There is a Swansea!"
i think the point is maybe if they port it they can modify it and make it better? At any rate PS III reminds me of the precursor to PSO.
Wow, those guys ARE good. They even ported their company web server to a TRS-80.
But some areas of the code still need work:
int fork() { return 1; }
Emulation:
*Each assembly instruction interpreted by an interpreter
*Compile once to run N applications
"Translation":
*Each assembly instruction translated to a C macro
*Compile N times to run N applications
Looks like emulation wins. If an emulator has JIT then "translation" losses its only speed advantage too.
For console video games, why in the hell would you translate the language? All the consoles had funky hardware that games used one time or another.
For 1, the NES uses mappers to display games. On emulators, many mappers are not supported. NES game producers also put hard-coded timings in games. So if your recompiled game isnt the exact same multiplier of clock frequencies, many will exhibit starnge behavior or just lock up.
Next, the SNES had pretty much basic AppleII GS hardware with the exception of the 32 channel sound card. It had a sound cpu which could hold 64k code along with samples. A problem that the makers of ZSnes had was determining the random noise generators formula. On the older games like Chrono Trigger, the wind would sound like square waves going up and down. That sounds fun, compiling a game when you find out that the hardware emulation controls wernt right.
Jump to PS2. Who would have thought that a comuter like that would be available to the public? One that has little GFX ram but a huge bus. Not to mention a full FS to "compile". What pitfalls occur in the 3D hardware?
Emulation is still better as it offers a replacable shim to modify and add features. You can also use other emulators.
Amazing feat. It's completely rewritten in C to gain exactly the same functionality as the original code, with only the binary / dissassembled machine code to work with.
-larsch
Well, I'm not porting to a different platform, but I am converting old bootable 360k floppy disk games into games that can still be run on modern computers.
I've finished the Epyx game JUMPMAN
http://www.classicgaming.com/jmanproject
and I'm currently working on Beyond Castle Wolfenstein
This is a very time consuming hobby, but for me it's a matter of preservation than anything. I don't want to have to dig out the PCjr from my closet everytime I want to play these oldies.
-jeff!
Too much self modifying code, the guy is a fool.
He does not talk about dynamic recompilers (that use page management dirty-bits to track self mofifying code)
He does not talk about self modifyingh code
he does not talk about timing loops and differences in different opcodes
He does not talk about lot of things.
I suspect he is not the one to talk about this stuff.
Tom Dowdy (old email dowdy@apple.com) knows all about DR he wrote a paper on it.
So did the guys who wrote a DR for recoding Mac 68K on an Intel at full speed, the Executor guys.
Its incredibly hard work and takes half a millikon dollars to do one correctly that can handle self modifying code.
The guys from Connectix (now Microsfot) did it with having the x86 intel run on a PowerPC.
DR (Dynamic Recompilation) technologies do not make source code.... because that itself would take tens of millions of dollars to engineer and at least a few huge teams of experts, and even then would not be possible to automate.
That guy is a fool, but I do agree that stuff NOT written in assembler (compiled stull written mostly after 1999) couple be transcoded into files because C++ and C on moder architectures rarely has self modifying structures.
My favorite Slashdot links are those that go down within about 12.5 seconds of showing up on the front page. I know I must be missing something really great!
It Is the Nature of Information to Transgress Artificial Boundaries
The guys at OS/2 Netlabs have been doing this for years now. It'ts called Project Odin They run Win32 apps on top of OS/2 with no emulation: they "translate" binaries on-the-fly. They even run Win32 drivers on OS/2!!
I've had to suffer through these guys' port of Joust on shockwave.com (in fact I have the high score for the month currently) and it has several inexcusable flaws. The most annoying is that it doesn't always respond to keypress events properly - rather critical functionality for a game. It doesn't actually seem to drop the events, which would be bad enough, but it ignores them for as much as half a second at a time and then spits a whole bunch out together. If your game wasn't screwed up when it seemed to ignore the event, it's damn sure going to be screwed up when they all get played back in a burst along with the other keys you hit to recover from the original failure. I know Windows is not a real-time system, and I've worked on real real-time systems (microsecond response times) so I do know the difference, but there's no excuse for a delay this long on an unloaded modern system. Other games don't seem to be afflicted by the same problems. The problem is not in Windows; it's in Digital Eclipse's emulation of the hardware on which the game originally ran (even if the code is translated the non-CPU hardware still has to be emulated).
There was also a time when the game's speed calibration was totally broken. I'd play on my 600MHz laptop and it would be just about right, but when I went to my 1.5GHz desktop guess what happened. Yeah, everything in the game was moving about 2.5x as fast and the game was unplayable. This only persisted for a week or two, but it's still not something that should happen in a version that's released to the public. These technical failures, combined with their apparent acquiescence to Shockwave's desire to add deliberate player-killing features to their translation (the very laws of physics in the game change after you get a good score, and I've looked at the original ROM code so I know exactly how they're doing it) have left me with the impression that Digital Eclipse is both incompetent and unethical.
Slashdot - News for Herds. Stuff that Splatters.
Odd TRS story: the bicycle manufacturer GT made a mountain bike called an RTS-1 (Rear Tuned Suspension). It shouls have been the more logical TRS-1 (Tuned Rear Suspension), but the Tandy trademark apparently stopped them using the initials TRS. This was about 10 years after the TRS had gone out of production!
When I am king, you will be first against the wall.
These people are the ones who translated Phantasy Star games to the GBA.
The GBA does not have the power to emulate the Mega Drive (Genesis) even with a JIT compiler. Hence, a different method would be needed. Either simulation, recompilation, or translation. Translation was the best option given what they had to work with.
Emulation is wonderful and all but it certanly doesn't work in all situations.
FYI, I believe that PS1 was emulated and PS2,3 were translated in Phantasy Star Collection. They're not saying to use this as a replacement, they're just telling you how they did it. I'm letting you know why.
Why not a compiler that takes machine code and compiles it to a new machine code? Such systems have been made. The Java JIT compilers take java bytecode and compile it to native machine code. The translator on the Transmeta chip takes AI-32 and comiles it. Why not one that takes say an MS-DOS program and converts it to Linux? You would have to map every interupt and dos call but it could be done. Selfmodifying code would be a big no no or a huge challenge for the compiler. All in all it could be interesting.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
Because it's one they actually did. Phantasy Star 1, 2, and 3 got ported over to the GBA as a Phantasy Star Collection.