Slashdot Mirror
Learning Reverse Engineering
TheBoostedBrain writes "Mike Perry and Nasko Oskov have written a very complete article about reverse engineering. It provides an introduction to reverse engineering software under both Linux and Windows."
← Back to Stories (view on slashdot.org)
How long before this site is taken down for DMCA violations?
My journal has hot
that's about to gain a permanent spot in my book collection. 'Nuff said.
C|N>K
I can't believe they left out truss/strace/ktrace. Even without debugging symbols, these utilities can tell you what system calls are being called, when they are called, and what arguments are being passed.
truss under Solaris is even more useful than strace under Linux or ktrace under the BSDs; you can also trace function entry points into user-level ELF solibs.
Do daemons dream of electric sleep()?
do the authors of the book linked have the text available as a single PS or PDF file?
I'm not sure that their claim that anyone who's read a "How to Learn [C|C++|Java|*] in nn Days" should be able to follow the article is correct, but it's a good intro nonetheless. The section on binary formats (ELF, etc.) is particularly useful.
the "very complete article" for reverse engineering software is pretty incomplete... chapters 10 and onward are not finished, the good stuff like buffer and stack over flows are not written, but just has the table of contents headings.
mimosa: ~ $ echo 'engineering' | rev
gnireenigne
What more do you need to know?
`(f) REVERSE ENGINEERING- (1) Notwithstanding the provisions of subsection (a)(1)(A), a person who has lawfully obtained the right to use a copy of a computer program may circumvent a technological measure that effectively controls access to a particular portion of that program for the sole purpose of identifying and analyzing those elements of the program that are necessary to achieve interoperability of an independently created computer program with other programs, and that have not previously been readily available to the person engaging in the circumvention, to the extent any such acts of identification and analysis do not constitute infringement under this title.
`(2) Notwithstanding the provisions of subsections (a)(2) and (b), a person may develop and employ technological means to circumvent a technological measure, or to circumvent protection afforded by a technological measure, in order to enable the identification and analysis under paragraph (1), or for the purpose of enabling interoperability of an independently created computer program with other programs, if such means are necessary to achieve such interoperability, to the extent that doing so does not constitute infringement under this title.
`(3) The information acquired through the acts permitted under paragraph (1), and the means permitted under paragraph (2), may be made available to others if the person referred to in paragraph (1) or (2), as the case may be, provides such information or means solely for the purpose of enabling interoperability of an independently created computer program with other programs, and to the extent that doing so does not constitute infringement under this title or violate applicable law other than this section.
`(4) For purposes of this subsection, the term `interoperability' means the ability of computer programs to exchange information, and of such programs mutually to use the information which has been exchanged.
DMCA
Whenever the offence inspires less horror than the punishment, the rigour of penal law is obliged to give way...
http://www.acm.uiuc.edu/sigmil/RevEng/x288.htm#beh avior_system_calls
Well this was most unexpected. We still have a lot of work to do on this book, and are still in the process of looking for a publisher. In fact, both Nasko and I were working on the book as this was posted (quite a shock!). We're still putting together screenshots, describing debugging utilities, etc..
In fact, the book looks more complete than it actualy is. Most of the chapters are basically just an outline that we've been filling in as we go along.
Keep checking the book periodically for more updates, as again, this is a work in progress. If you notice any ommissions, or have any contributions, we would be glad to take them.
Thanks,
Nasko Oskov & Mike Perry
This. One of the funnier 404 messages I've seen. Take a look at the source for the page so you won't have to wait for the slow version of the text. :)
Very Good article, and I admit that I did not understand all of it, nor did I read all of it. However I did forward it along to a couple of friends who do not regularly /.
Here is a reverse engineering feat for you all...POS(Point of Sale) terminal equipment. Specifically to replace NSC(National Systems Corporation) and similar diamond touch gear. If you can reverse engineer a system for taking customer's orders(think pizza/food), showing it on multiple screens around the store, and keeping track of inventory, sales numbers and statistics, customer tracking and history...wow you would be great. Nobody wants to spend $15-30,000 for a new POS system. Nobody.
Biggest problem is that these small operators spend that much money on the system, that they are obligated and forced into using it for 10+ years, well after the hardware(monitors/keyboards) wear out. Then get stuck purchasing proprietary stuff at the same cost it was at the original purchase price...several hundred dollars for a custom keyboard...get real.
Somebody please show me where there is a project to reverse engineer this with an X window under RedHat/Slack. Even terminal would be fine. The current system runs text only...over 1 pair of copper in a phone plug(rj11).
Who is this that even the wind and the waves obey Him? Surely this computer must submit also!
has been doing all this for years:
http://fravia.anticrack.de/
When the Thought Police arrest you so that the MPAA can sue you for intent to possibly defraud, larsony of imaginary profits, and programming without a liquor license.
You can't judge a book by the way it wears its hair.
The part i had time to read before the *you know what* was really good. But there is something else. This server probably has the best 404 page i have seen. Some people spend some time to make a really nice 404 page?
:)
Ooo.. now i see. it's ".edu"
Slashdot Sig. version 0.1alpha. Use at your own risk.
"And we don't know about you, but to us, software that we don't have source code to just pisses us off. So we figure: screw it, lets do some damage. :)"
"maybe you want to monitor all data before a program encrypts it and sends it across the network, or maybe you just want to cheat at your favorite multiplayer networked game."
I can't wait until some DMCA-junkie and/or [RI|MP]AA member takes that out of context. Then it will be shutdown time!
For an excellent source of reverse engineering material, you really should check out the old Fravia pages. This is the original stuff right here.
Along with reversing tutorials and materials, there is a rich history behind this stuff. A man named +ORC published a tutorial on how to reverse engineer a Windows program called pooldemo.exe. From this text, an era was born. The Fravia website was created and was home to the +HCU. Many people sought after the true identity of +ORC, and he left a strainer (riddle) behind that would take you to a URL where he would be unmasked supposedly. Just look up "ORC riddle" on google for details. Neat stuff!
But now I know why, due to the authors' comments. Thanks for the pointer to the TOC entry though, don't how I missed that.
For the readership out there, I'm sure those will be covered in the future; in the meantime, read your strace/ktrace/truss man pages. Run them on the application you're trying to RE before doing *anything* else. Sometimes, those dumps can provide *amazing* insight into the behaviour and structure of the program (particularly if you're good with 'grep'), especially if you're trussing and using the program interactively.
Do daemons dream of electric sleep()?
Given some of the code I get to deal with (19(7/8)0s vintage C, much of which is older than I am) it's probably easier to reverse engineer the binary and look at it there...
Beep beep.
Learning Reverse Engineering
/ / /
Pondering, peering
recollections of fearing
yearning for perverse queering
having difficulty steering
Thank you.
that's just engineering in reverse...right...stop loooking at me like that!!
Undoubtedly, someone will have a copyright/patent on reverse engineering methodologies.
So, I would suspect the site will have to be taken down if it is just a copy of the copyrighted reverse enginieering process. However, if it was properly reversed engineered, then it would not be considered a copy...or, uh, something like that.
There was a slick plastic game called Black Box back when thinking games like Mastermind were raking in the dough. There are Java and PalmOS varieties of the game. It's a nice three-minute game to while away a bus stop wait, and it helps you get in the mindset of what reverse engineering really means.
The inside of the Black Box is an 8x8 square. There are 8 ports on each side of the square. One player sets some marbles inside the covered square, and the other player tries to deduce their locations by the behavior of "rays" entering and exiting the box ports. Some rays go all the way through, some reflect off the balls inside, and some glance off the balls and go out some other side of the box.
[
I don't know a lot about reverse engineering, but it looks like a lot of the chapters are just placeholders (no content).
For instance, a chapter on buffer overflows and stuff (which is of academic interest to me) is completely empty.
This doesn't consititute "very complete" to me.
Let me guess and release it all open source? You have the big problem that people with programming experience dont have a buring need for POS software. Granted there are quite a few vendors out there that would love it for the support contract. But your seeing one of the flaws of open source you have to find an interested group of programmers to write one for free generaly.
No sir I dont like it.
Only two sections are complete.
When will I be able to get this in paperback so I can read it while I'm sittin' on the can?
This book is pretty weak. I skimmed through it and no where did I see win32dasm dead listings or hands on reversing. It seems like it just tries to explain different windows and unix tools people might use for reversing. Comments like this certainly should have been left out: If you don't know assembly language, at the end of this book you will literally know it inside-out I mean gimme a break. In less than a hundred pages of text, no one is going to learn x86 asm "inside-out." Chapter 9, which many people would be interested in, is incomplete. I wouldn't waste my time. Go search for "fravia pages reverse engineering" on google. That material took years to put together.
he deos fargottry thngs
Quote from the introduction of the book:
:)"
"We don't know about you, but to us, software that we don't have source code to just pisses us off. So we figure: screw it, lets do some damage.
Cheap comments like this really degrade this book.
"Here is a reverse engineering feat for you all...POS(Point of Sale) terminal ........"
You would be inviting an attack by the legal representation of mad squirrels! Squirrels are very teritorial little creatures. http://www.squirrelsystems.com/press/pr/Mar0502.ht ml
OH THE SHAME I fell off the wagon and use sigs again!
they mention how vmware uses corrupted elf headers to hide their precious secrets. maybe they should have put as much thought into debugging and the user interface 'cause virtual pc is wayyyy better (probably why ms bought virtual pc)
what a useless 'tutorial'
what ever happened to the old days where people
had to learn this stuff on their own?
To make reverse-engineering as hard as possible :)
...
:)
IP : encrypt everything at the application level
FS : Use one big raw file to store as much as you can, preferably at the bit resolution
DBG: Never forget to remove debug info
Nice
I didn't know about ltrace -- my RH 5.2 box doesn't have it. Hmm. I may have to upgrade, but I've been reticent to touch that (non-net-facing) box because as I understand it, most distros have dropped the UDB (Multia) support from their kernels/bootloaders.
;)
I suppose I could go the upgrade route, but that'd mean a new kernel, which needs a new gcc, which needs a new glibc *argh*
Maybe I'll just dig up ltrace.c and see if it'll go.
Do daemons dream of electric sleep()?
uhhhh, since there are already software cracks for that....
YOU SUCK BALLS!
- -h: 2 chars
- --help: 6 chars
That's three times as many keystrokes. What's worse is when a program doesn't use -h for anything. Yet another reason to hate the GNU.Let me know, and I'll find you some purchasers.
Good work so far, my other comment notwithstanding.
Do daemons dream of electric sleep()?
According to this book, strings are stored differently on big-endian versus little-endian architectures, so "this is a string" on a Solaris machine becomes "siht si ts agnir" on a Linux/x86 machine.
This is complete nonsense. Endianness only affects multi-byte data types: shorts, longs, floats, etc. The order of chars in a string does not change.
I need help reverse engineering "si3" files.
.mid to the cellphone, first time you play it, it gets converted to .si3
It's the format used on siemens cellphones to play midi (subtypes 0,1)! Siemens says it is closed...
When you transfer a
Looking for people to chat about multicopters, coding, music. skype: gtsiros
This is far from a "a very complete article about reverse engineering."
"siht si ts agnir?"
this is st ringa?
What bizarre language are you speaking, man?
Here are a couple of beginner-level articles I've written on reverse-engineering malicious code:
Reverse Engineering Hostile Code
Alien Autopsy: Reverse Engineering Win32 Trojans on Linux
Recently I came by this book: Code Reading - The Open Source Perspective, which has the same idea except for when you have the source of a program and not only a binary.
bash$
Why is Slashdot promoting such vile acts of piracy ?
</software megacorp>
Your confusion is quite understandable as ringa was only recently canonized.
I see even classic Slashdot is now pretty much unusable on dial up anymore.
Trying to get into the programmers mind set for me is useful because it helps me figure out the pieces of the program faster than reading every single line of code several times. Take for example a novel. As you read each chapter, the intension of the author become more apparent. Often times with bad writers, the reader can tell what's going happen. With great writers, it's not totally obvious until the last page. In this way, deconstructing a piece of software helps me see where the programmer was going. that often is more useful than reading every line.
a mention of IDA pro, W32Dasm, softice. These are the tools of the trade.
These are cool guys, I've gone to a few of their SIGMil meetings. Its cool to think i've partied with some guys who've been slashdoted (also perry TA'd a class i took in the fall). Nasko likes to hit on my girlfriend (tho he's only seen her twice).
Good job guys! Keep it up!
They also run the most excellent ACM Special Interest Group at U of Illinois- Urbana Champaign. Anybody who's interested in this kind of research should check it out when they host meetings in the fall.
--- Kicking the Cheat since late 2002
So this means that attempts to figure out to program to any of a number of Windows API's are not DCMA circumvention violations?
Or your could release the source code -- in Pascal. Anyone without enough programming skill to understand your system will rather go blind than read Pascal code.
http://66.127.229.59/reveng.pdf Its a 128kb dsl line so the file will be yanked if im lagged too much :)
Thanks, but no need - we've got boatloads of bandwidth at uiuc, and the web server is hardly noticing:
[staffin@winston staffin]$ uptime
21:20:19 up 79 days, 18:17, 3 users, load average: 0.24, 0.27, 0.32
Not bad. It's an ultra5/360 running Debian with 256mb of ram, btw. I think this pretty much demonstrates that the slashdot effect is all about bandwidth, not the speed of the server.
/me takes a bow
That's my 404 page :)
Is it supposed to cause certain groups of people to turn their noses up at this? What group would that be?
How about the "I'm not going to cite this book in a bibliography because I cite only works that I would recommend to fellow professionals, who by the way do not appreciate obscene humor in the context of their jobs" group?
I can't think of any group or person with that reaction who would be of the inclination to reverse engineer things.
You mean like Compaq? Lots of Big Corporations(tm) reverse-engineer their competitors' products in order to learn how to interoperate. Such reverse engineering is exempt to an extent from the DMCA's circumvention ban (17 USC 1201) when under the supervision of an entity that can fund a legal defense.
Will I retire or break 10K?
http://www.acm.uiuc.edu/~staffin/reveng.pdf - Here is the same pdf mentioned above, only on a site that isn't about to get slashdotted (on the same server as the original site, in fact). Have at it!
I didn't realize that Windows was a Unix clone...
Approximate summary of a possible cease-and-desist letter from Siemens's counsel:
Will I retire or break 10K?
In a way, a Microsoft Windows system has always been a UNIX clone.
on the one hand: MS-DOS 2 was Microsoft's attempt at a "transition" from DOS to its XENIX operating system. It failed, but it did introduce several UNIX features to the PC DOS platform, such as subdirectories, file handle semantics, named devices, pipes, and redirection of input and output to a file. Another transition from DOS tech to multiuser tech (Windows to OS/2) failed at first but, when tried again several years later (Windows ME to XP Home Edition), ultimately succeeded for the most part.
on the other hand: Windows is a Mac OS clone. Windows XP is a Mac OS X clone *cough*Luna skin*cough*. Mac OS X runs on top of a FreeBSD-derived core called Darwin, which adheres to the most visible parts of the Single UNIX Specification.
on the gripping hand: Though the kernel of the Microsoft Windows NT operating system was designed along the same lines as that of Digital's VMS operating system, NT has always contained a(n admittedly crappy) POSIX compatibility layer. Microsoft sells an upgrade called Services For UNIX that enhances the POSIX layer with BSD and GNU power.
Will I retire or break 10K?
google search for fravia and "+orc". Lots of indepth articles at reverse engineering, how compilers work, etc. Much more practical and interesting, not to mention loads of indepth information that, honestly, surpass what's present in these articles.
Not for the Faint of Heart.
It's nice to see this coming back, but all of this was discussed ad naseum ~1996ish.
you might want to have a look at AntiCrack which is a huge collection of tutorials cracking , reverse engineering, and programming. They also have a copy of the Old Fravia'Site, the new one being about searching).
;-)
There's a few games/challenges out there about reverse engineering, cracking, logic and programming. Give them a try if you wish (Arcanum is really nice):
AngularVision, Apotheosis, Arcanum, Aspect, Aspect2, C&CDisIncorporated, CyberArmy, Disavowed, Electrica, Escape, HackME, HackersGames, HackersLab, HackQuest, Hybrid, ICEFortress, Lamebulun, Mod-X, NetSplit, NGSEC'sSecurityGame, ProblemSetArchive, ReverserCourse, SlyFX, TheGame, and Try2hack.
have fun
-- search the web
Are there any lawyers here that can say whether packet sniffing is indeed a form of Reverse Engineering and could then be prohibited by an EULA? Common sense would tell me that it can not because it would be analogous to having a reasonable expectation of privacy when talking with someone inside your home but not when shouting to your neighbor through an open window.
can't spell "does"
If someone else can do the specifications (e.g. requirements, communications protocols with which we need to be compatible, etc.), I can try digging up some more programmers to help. I'm already registered on SourceForge, but don't yet have an open source project.
I suspect once we get the basic project working, we can probably get donated equipment to work on, but initially we can work on just the most basic POS equipment (or even just simulate it).
I was looking for a book like this, since many small linux-based companies are working and leveraging existing open-source software. I mean, everyone needs 99% of the program, but you need to add a small feature or two to a million++ line program(i.e. samba).
How do you do it? Well, I hope this book can give me more insights.
Not exactly complete. My research
area is reverse engineering. The book only
talks about low level reverse engineering
(i.e. executable code). Most of the research
in the area is at the source level.
This is not a criticism targeted at theauthors,
but at the submitter.
Atlas stands on the earth and carries the celestial sphere on his shoulders.
For professionals (read: willing to pay $$$) I recommend the Ida Pro disassembler. The Fast Library Identification & Recognition Technology (FLIRT) combined with Parameter Identification & Tracking (PIT) make it the tool of choice for serious reverse engineering.
Don't forget +Ma's Reversing. It seems to focus more on cryptanalysis than the others I've played, but has a fair bit of reverse engineering stuff in there too.
What we need most are methods to reverse engineer device drivers.
Let's say for example, a certain manufacturer of popular media cards actually has linux drivers for their hardware, running on an ARM in a setup box, but refuses to release these drivers, open or closed, to pc users. If I had said drivers in hand, could I port them to i386?
tcboo
I'd be tempted to add moria/angband to that list. That program runs on a -WHOLE LOT- of platforms including rather interesting kMoria for the Palm Pilot.
That's why I stick to SUSv3 style single-letter options. It limits you to only about 50 options but only gh3y things like a GNU compiler will need more! Hooray!
Lolz yur n0t 4 h4x0r
/|\3 3/|\ph4z1z3 t3h \|/0rDz: "...this book is incomplete...it has mistakes..."
/|\3 3/|\ph4z1z3 t3h zub1i/|\1N4l \|/0rDz: "
Lolz yu c4nt r34d
0n t3h 4rt1cl3:
"Note
TO SLASHDOT READERS: Yes, this book is incomplete. Yes it has mistakes. Yes, we are working as hard as we can to fix them. Please email the authors directly rather than simply ranting/flaming on slashdot. We will take your comments into consideration, and will list you in the credits. We've already built up a large queue of fixes thanks to helpful emails."
13t
13t
email the authors"
B3 c4r3fu1 d00dz: "We will take your comments" Tay st34l J00'z c0mments!
Dunno, I found this to be funny.
As far as specifications go with hardware...the simpler it is the better. Honestly, do you want to code to something that is unfamiliar? No. You want to run x11 on a Plain old P4 or something with 5 pci video cards in it. vga monitors. Not monochrome monitors running off two wires that make ugly text displays. Something simple to program, and even simpler to replace.
Seriously, you could sell this commercially if you found a backer. Give them the software for free, and sell them the system. One computer, 6 keyboards and 6 monitors. Thats all you need to supply, and you can charge them US$5000 for it. Sell upwards of fifty of these darn things to little mom and pop pizza places and they would be happy, and you would clear $1500 a piece...then start selling to Pizza Hut, Domino's, Papa John's...and make a shitload as they begin replacing their equipment and buying yours.
Be sure to offer them support with certain little things for a specified ammount of time, and charge them like $2000 for a one year service contract.
Modify your code with a couple different modules, and begin handling burgers and fries instead of just pizza sizes, toppings, and cokes...and then you open your market up to smaller chains like A&W, White Castle, and eventually anybody.
there is a need for this type of software and hardware solution, and all businesses feel it roughly every 8 years or so. Thats a pretty good market. Get your hands on some old equipment and see how the inventories worked, the numbers added up, and displayed. Wow. Make yourself a living in 30 long and difficult steps.
Who is this that even the wind and the waves obey Him? Surely this computer must submit also!
I 've used this trick to reverse engineer the proprietary communication protocol of a (now ancient) Ericsson GH-388 mobile phone and write a program to query the phone for battery level, reception level, IMEI, and phonebook contents. The proprietary program was running on a Windows laptop; a Linux machine was sitting between the program and the phone busilly recording every byte they exchanged.
Diomidis Spinellis - Code Reading: The Open Source Perspective
#include "/dev/tty"
I did like this link, as in introduction into reverse engineering I feel it could become a helpful guide. But I feel that it's style is substantially wrong to achieve its ambition of becoming a book; the document style feels far too fragmented, chapters and even individual sections should be longer, perhaps detailing how to use the various programs mentioned. Perhaps an example program to be reverse engineered on Windows and Linux should be included, and output of the reverse engineering tools on that program included at various points through the book.
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
A direct quote from the book, at the beginning of chapter 4:
strace/truss(Solaris) These programs trace system calls a program makes as it makes them. Useful options:
1. -f (follow fork)
2. -ffo filename (output trace to filename.pid for forking)
3. -i (Print instruction pointer for each system call)
Don't drop the soap, Tommy!
compress is older and more "standard" than Gzip. So we should all be using tar.Z, becuse it goes back before the 80's. Coincedentaly, it also now Patent Free, and WinZIP can read .Z files just as easily as it can read .gz
I'm sorry about this post. I'm sure allot of work was put into it but I just think reverse engineering is a black art that is best not dabbled in at allo. t runs contrary to standard engineering in that there is no inventiveness or innivation needed, just pain staking labour.!
Not competet enough to create the stuff from scratch themselves.
You're right, the truss -u option appear under 2.8.
However, you can do most RE of 2.x binaries under 2.8, due to the wonderfully static ABI.
Do daemons dream of electric sleep()?
Georgia Tech's Reverse Engineering Group also has a whole lot of info on the topic here : http://www.cc.gatech.edu/reverse/
Funny sig man!!
For those who describe their systems as 'boxen', do you order multiple 'boxen' of corn flakes also?
Painfully bad, isn't it? :)
If a job's not worth doing, it's not worth doing right.