Slashdot Mirror
New Approach To Malware Modifies Linux Kernel
Hugh Pickens writes "Professor Avishai Wool has unveiled a program to watch for malware on servers with a modification to the Linux kernel. 'We modified the kernel in the system's operating system so that it monitors and tracks the behavior of the programs installed on it,' says Wool. Essentially, Wool says, his software team has built a model that predicts how software running on a server should work (pdf). If the kernel senses abnormal activity, it stops the program from working before malicious actions occur. 'When we see a deviation, we know for sure there's something bad going on,' Wool explains. Wool cites problems with costly anti-virus protection. 'Our methods are much more efficient and don't chew up the computer's resources.'"
OK, what this is doing is watching for code injection attacks (buffer overflows, stack smashing, etcetera) by building a model of how each specific application is going to operate, and blocking system calls that the model of the application would never make. It seems like an interesting approach, though it may not be as useful on Windows where there's not such a formal distinction between system calls and other kinds of calls.
It won't do anything about interpreter code injection (eg, SQL injection or shell code injection) or script privilege escalation attacks (eg, ActiveX and other "cross zone" attacks in Internet Explorer), or attacks that involve complete executable code drops.
Still, this is useful and not nearly as dodgy as the article made it sound.
If an application makes a system call it never calls or never can call in that order because it's been hijacked then this thing will stop it.
The problem is that, for any non-trivial program, it is impossible for a static analyser to decide whether the software will call or not and in what order a specific trap. For instance, that static analyser is not able to tell you in advance at what time the exit syscall will be called.
This is drivel - it assumes that a static binary analysis can be used to predict the dynamic behavior of a non-trivial application, with zero false positives.
Which is why I tagged this with "haltingproblem". If it's impossible to write a terminating program which simply determines whether or not another program terminates, just how likely is it that a program can correctly predict far more complex behavior of another program? Since the halting problem is noncomputabe, we know it can't do it with 100% reliability, the question is: "how error prone is it?"
It can't catch every case, but it can rule out a lot of system calls. If setuid never appears in a program or library's source, it can't be called. If a program calls a bunch of things in an initializer function and then enters a more restricted main loop, the static analyzer should be able to catch that too.
I can't see how this approach could hurt, though of course it won't catch everything.
this isn't anything specifically to do with malware.
As far as I can see, this verifies that the binary currently running is the same binary that was compiled from a (trusted) source.
When you compile it, it knows (from the source) what the program will and won't do. If the program deviates from that, it dies (as it's been replaced by malware, presumably)
If I'm wrong, please correct me...
I have developed a truly marvelous proof of this comment, which this signature is too narrow to contain.
How does this thing deal with plug-in/add-on based systems like Firefox or Eclipse, where new capabilities get added to the executable through dlls (or java classes, I guess, in the case of Eclipse? - Although, with regards to Java, I wonder if this system would work at all, since I think the kernel never exactly 'sees' Java programs or classes as executables, but only the JRE, which already has all the system calls built into it?)