Slashdot Mirror
Avast Launches Open-Source Decompiler For Machine Code (techspot.com)
Greg Synek reports via TechSpot: To help with the reverse engineering of malware, Avast has released an open-source version of its machine-code decompiler, RetDec, that has been under development for over seven years. RetDec supports a variety of architectures aside from those used on traditional desktops including ARM, PIC32, PowerPC and MIPS. As Internet of Things devices proliferate throughout our homes and inside private businesses, being able to effectively analyze the code running on all of these new devices becomes a necessity to ensure security. In addition to the open-source version found on GitHub, RetDec is also being provided as a web service.
Simply upload a supported executable or machine code and get a reasonably rebuilt version of the source code. It is not possible to retrieve the exact original code of any executable compiled to machine code but obtaining a working or almost working copy of equivalent code can greatly expedite the reverse engineering of software. For any curious developers out there, a REST API is also provided to allow third-party applications to use the decompilation service. A plugin for IDA disassembler is also available for those experienced with decompiling software.
Simply upload a supported executable or machine code and get a reasonably rebuilt version of the source code. It is not possible to retrieve the exact original code of any executable compiled to machine code but obtaining a working or almost working copy of equivalent code can greatly expedite the reverse engineering of software. For any curious developers out there, a REST API is also provided to allow third-party applications to use the decompilation service. A plugin for IDA disassembler is also available for those experienced with decompiling software.
Get over yourself and stop complaining about things being given away to you for free. It's a shame that people complain about open source software when it's being given to them for free. The decompiler could have never been released to the public or released as a closed source program. Your complaint about the architectures it supports or doesn't support totally rings hollow.
...but no x86_64.
Or any other 64 bit arch.
Back in the late 70's I loaded TRS-80 games into my debugger, it also let me dump the results into a text file. Finding things like "jump to label_foo" helped, but was not the be-all end-all.
The killer was when I debugged my TRS-80 BASIC interpreter in ROM. You'd have some 3 byte instruction, "jump here", then somewhere else you'd have a 3 byte instruction "jump into the middle of this 3 byte instruction to do something completely different". My understanding is Bill did those, but for all the evil he did I have major respect for his coding abilities.
I beat a lot of games running my debugger on them. 90% sure it was called TRS-MON, but wouldn't bet my retirement on it.
Perhaps if you built a fingerprint based on the structure of calls across functions, you could map it back to source code from github. Not that malware is generally posted to github, but I'd be surprised if they didn't use a TON of third_party libraries, and factoring all of those out would make what's left easier to understand and also let you focus better.
It's accurate. According to retdec.com, RetDec only supports 32bit architectures.
Try
gcc --reverse prog -o prog.c
Slashdot, fix the reply notifications... You won't get away with it...
...but no x86_64.
yet.
Slashdot, fix the reply notifications... You won't get away with it...
Why wouldn't you just release it as open source? Is it because APK Hosts File Engine 10++ 32/64-bit contains MALWARE and VIRUSES?
One of the big issues with decompilers is that compilers do not generate the same output for the same input. In addition, multiple versions of a compiler and different flags yield different results as well. After some thought, I've come to the conclusion that the only viable solution is to build a neural network that can detect and compensate for all the idiosyncrasy using many different test cases (and their binaries) as training data. Ultimately be able to return not only the most likely version of the source code but also the compiler name, version and flags used to compile it.
We have the technology to solve this seemingly impossible problem.
Anons need not reply. Questions end with a question mark.
Probably also helpful when searching for vulnerabilities?
4wdloop
AVR, MSP and L106 (Tensilica/ESP8266) missing...
Especially for MSP, there seem to be a lot of products using it (Honeywell thermostats, Ikea lighting)...
4wdloop
What would stop someone from creating a malicious software and naming it APK Hosts File Engine 10++ 32/64-bit? I mean, different malicious software, because I am assuming your version of APK Hosts File Engine 10++ 32/64-bit is MALWARE. So why not just open source it, so we can see what it does?
I have no idea who "virustotal.com" is. Why not just show us the source code? Then we can find out if APK Hosts File Engine 10++ 32/64-bit is MALWARE or isn't MALWARE. I am assuming it is MALWARE since you are keeping the code hidden and obfuscate the exe just like a virus does.
According to virustotal.com it says:
ClamAV Possibly Unwanted Application. While not necessarily malicious, the scanned file presents certain characteristics which depending on the user policies and environment may or may not represent a threat. For full details see: https://www.clamav.net/documen... . Symantec reputation Suspicious.Insight
Sounds like malware to me.
I am assuming your version of APK Hosts File Engine 10++ 32/64-bit is MALWARE.
I'm guessing others have tested it in a sandbox for malicious behavior. Do you assume Intel and AMD CPUs contain malware? And if you do, do you use them despite said assumption?
So why not just open source it
If this post is to be believed, APK doesn't want people adding malware, building it, and distributing it, like eFast did with Chromium.
The other option is for some Slashdot user to make a free replacement. Does the functionality described in this specification appear useful?
Not every program packed with UPX is a virus.
unfortunately it de-compiles the machine code to perl.
Nullius in verba
I ran some of my own ARM code through this. While I did build with -Os, I did not strip the .elf. The source it produced was a reasonable approximation of what I wrote, but it was far from legible. Little things like using hexadecimal for memory addresses are a minor nitpick, but I found it had trouble even with basic interrupt handlers. I would have expected something aimed at targeting embedded systems would do a better job of of this, but still... very interesting (and very fast)!
If it does PIC32 specific functionality like decode that chip's MMIOs, that's a nice feature of simply decoding MIPS object files.
“Common sense is not so common.” — Voltaire
no mention in the article of what the decompiler actually decompiles to ..
The thing is open source, if you really want x86-64, grab the code and write something :)
UltraEdit (Text editor) will show all text in a file, one can fairly call a files function with just that.
Long ago there was a program called "Peek" that showed all text in a file none of the hex/high Ascii that UltraEdit also shows; W2K broke it and I've missed it every since.
I'll be giving this program a try.
The thing is open source, if you really want x86-64, grab the code and write something :)
x86 is hard to decompile. It doesn't have fixed length instructions, so it is difficult to figure out where opcodes begin and end. It is even possible to write code that can execute two different sequences of instructions by offsetting the instruction pointer by a byte. I don't think any decompiler could deobfusticate that.
Get over yourself and stop complaining about things being given away to you for free. It's a shame that people complain about open source software when it's being given to them for free.
isn't that true. what i find even more amazing is that those same people mostly never complain about shortcomings of commercial software.
On a long enough timeline, the survival rate for everyone drops to zero.
This is probably a waste of time, but ... When you're typing a message and say "see the p.s. below", it means you _know_ at that point that you will be having a p.s. But in that case you could just place the text where you are, and not _need_ a p.s.
PIC32 binaries are pronounced with more of a guttural accent than MIPS ones.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
a decompiler won't care whether you compiled a C++, assembler, C or whatever language the program being reversed was compiled on.
It will care, because some language (e.g.: C++) have specific data structures and ways (vtables) to handle some language specific features (object virtual member inheritance) which could be detected by the specific plugin (i.e.: instead of spewing a weird mess of nested "struct" and pointer-to-pointers, it can recognize that his is just a call to a virtual method)
(for the few hipsters outthere : think the difference between vala and the corresponding GObject pure-C code).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
x86 is hard to decompile. It doesn't have fixed length instructions, so it is difficult to figure out where opcodes begin and end. It is even possible to write code that can execute two different sequences of instructions by offsetting the instruction pointer by a byte. I don't think any decompiler could deobfusticate that.
The simple code dumper that comes with garden variety debugger won't easily deobfuscate that. (You need to manually ask the debugger to start dumping from the 2 overlapping point).
That why, the best decompilers available in the 90s used some sort of virtual machine to follow through the execution flow, and be able to distinguish such kind of "frame shifts" (that's actually a biology term, I've forgotten what the proper CS term is), and also be able to understand a bit of self-modifying code.
(Basically, the decompiler will notice that various part of the code make calls into the same region but at an odd offset, and will automatically try dumping with from each overlapping point)
Makes it also possible to put actually-useful label/names into variable. (call something "sound_frequency" instead of "var184" because by following the data flow, the decompiler release that this is the parameter the is output to the PC-Speaker tone generator).
Sourcer by V-Com was one such good decompiler.
(I managed to learn quite a ton of tricks like PCM play on the PC Speaker, tweaked graphical modes, etc. simply by using sr to inspect interesting executables.
I even manage to desinfect a cracked game that was saddly being distributed infected with some virus)
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
the structure of calls across functions
Recognizing some code flow was a staple of the best decompiler back in the 90s :
e.g. being able to recognize a certain code pattern (a sequence of ports smashing) as a high-level abstraction (initializing sound hardware).
Your idea would certainly be the 2010s-era equivalent. (= This portion looks like code reuse from "Zstd" decompressor)
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Of course, this only helps the 5 of us left who still code in ASM. "Kids these days" seem to think that ASM "sucks" because "it's old". If the language doesn't have trait based generics, zero cost abstractions, and a partridge in a pear tree then again it's "old" and it "sucks". It's entertaining to watch your average 20-something java/python/PHP coder try to take on ASM. Their efforts generally don't last more than about five minutes when they find out they have to build their own control structures, and mama's not gonna wipe their butts with Visual Studio tooltip hints. If this wizzbang tool decompiled code into Rust, then maybe the cool kids would want it. As it is, they will do what they always do with ASM based tools: hand-wave like they know exactly how it works and then promptly ignore it. Anyhow, back to my ASM-One environment on my 68k Amiga. If anyone needs me I'll be here squatting on this temporal nexus to the 1990s. :-)
"I personally use a HOSTS file blocker produced from a genius called APK." by 110010001000 on Friday October 27, 2017
The irony in this is brilliant -- you're actually too stupid to realize that 110010001000 is the guy you're "arguing" with.
You' probably even think that he genuinely thinks you're a genius rather then openly mocking you. Oh boy.
Now, why don't you stop with your obnoxious ads? Wasn't one of your marketing points that your shitware removes ads? Does it remove your spammed ads on /.?
CLI paste? paste.pr0.tips!
What would stop someone from creating a malicious software and naming it APK Hosts File Engine 10++ 32/64-bit?
The fact that its hash wouldn't match that of the existing APK Hosts File Engine 10++ 32/64-bit posted all over forums.
Now if you replace "10" with "11" in your question, you have a more interesting problem: how to distinguish subsequent versions of the same publisher's application from an impostor's malware. The publisher of the authentic application could generate a self-signed code signing certificate and sign each version of all of its programs. Then each user would configure his devices to "Trust other programs from publisher APK". In my opinion, Microsoft screwed up Authenticode for hobbyist programmers by requiring paid organizational validation of all certificates from a commercial certificate authority rather than allowing reputation to accumulate on self-signed publisher certificates.
How does this project compare to the existing machine code compilers, namely Valgrind's VEX library and Qemu's tiny code generator (https://wiki.qemu.org/Documentation/TCG)?
No I'm not (it helps hide how I detect it) by obfuscation hiding functions/methods where I summon an .exe size for even 1 BYTE in sizecheck (no virus is that small)
You don't know much about writing viruses; that much is clear, because checking the size is a waste.
One popular approach for viruses is to put the original file elsewhere (where elsewhere can either be elsewhere on a file system, or for file systems that support it, in a resource fork or attribute list of the same file), and then pad the virus to the desired file length.
For weak CRCs, even change the padding to return the same CRC.
But worse, you also then prevent the binary from running on systems where the binaries are always modified before being run, for example by rebase/prelink, or by adding library paths to the executable, or systems which depend on setting contexts on files, or requires nx bit set, or ...
Making assumptions about the runtime environment is so 1990s.
Security through obscurity is what you're attempting here, and you require people to lower their security if it's too high.
Good programs use exe packers too as I said
Name one that's from this decade.
Even if you ignore the few 32-bit instructions in thumb it is still common to interleave data with the code.
The difference is that with x86 you can interleave code with code.
You can't do that with RISC.
I don't have to prove a negative. That's like saying "prove that god doesn't exist". The onus is on those who make claims to back them up, not on others to disprove them.
All it would take was one example to prove your claim. How hard would that be, if what you claimed were true?
Hint: Instead of posting URLs to posts that nobody will bother to follow, try to actually back up your wild claims with some actual meaningful text. Without bolding random words, without changing the subject and referring to it, and without a P.S. at the end.
If you looked less like a kook, perhaps some would take you more seriously. As it is, your postings elude that you probably have some mental problems, and are incapable of engaging in normal discussions like others here do. I can't help you with the former, but am willing to help you with the latter. But you have to be willing to learn posting etiquette.
It's known protectionvs. reverse engineering: PROOF: "Packing an executable file is a way of compressing executable code firstly to minimize filesizes, but often it is also used to complicate the reverse engineering process"
Also known as "security through obscurity", as I said in my post.
that proof's SO RIGHT you had to try "downmod hide it"
Um, no. I only have one account, and don't post as an anonymous coward, so I don't get to downmod anyone in this thread. It's others that downmodded your post, likely because of your incoherent ramblings being, well, "so wrong".
YOU DEFINITELY CAN'T, troll!
Think about it: Are everybody who disagrees with you trolls, or could it be that you are a smidgeon paranoid?