How To Guarantee Malware Detection

itwbennett writes "Dr. Markus Jakobsson, Principal Scientist at PARC, explains how it is possible to guarantee the detection of malware, including zero-day attacks and rootkits and even malware that infected a device before the detection program was installed. The solution comes down to this, says Jakobsson: 'Any program — good or bad — that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte.'"

Comments

At least one byte

[BadAnalogyGuy]

While it might be true that any application will take up at least a byte of memory, there is no reason malware couldn't masquerade as another binary down to the exact number of bytes.

Hell, Windows is a whole slew of malware that masquerades as the whole OS.

Re:At least one byte

[palegray.net]

I don't normally quote huge chunks of articles, but I'm making an exception on grounds that I don't believe you RTFA before replying:

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong. The external verifier would notice and conclude that the device must be infected. Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies. That would result in the right checksum... but a delay. This delay would be detected by the external verifier, which would conclude that the device is infected.

Why a delay, you ask? Because secondary storage is slower than RAM. Especially if the order of the reads and writes are done in a manner that intentionally causes huge delays if diverted to flash, hard drives, etc.

There's more details in RTFA.

Re:At least one byte

[dmgxmichael]

Not a fair comparison. Malware usually does what it's supposed to.

Re:At least one byte

[sopssa]

I did, and he doesn't say anything about this point.

Regarding making a keyed hash of the entire memory content, how would that even work? Every program modifies it's memory all the time. Then there's the programs like copy protections and Skype etc that modify it's own code in real-time too.

Re:At least one byte

[Chris Burke]

While it might be true that any application will take up at least a byte of memory, there is no reason malware couldn't masquerade as another binary down to the exact number of bytes.

Oh see he didn't finish explaining.

Any program that wants to be resident has to occupy at least one byte of RAM. And that byte should include the Evil Bit, which all malware should set. Then your anti-virus program just checks the Evil Bit and problem solved!

Re:At least one byte

[OeLeWaPpErKe]

There are several weaknesses. Note how he says something about an external verifier, which is delay-sensitive. Note how for the first 2 steps he keeps repeating "malware may of course interfere but this doesn't matter because", and then he stops considering malware interference. That's because at those points, malware interference would be fatal.

Of course, malware could simply take over the entire procedure, computing the keyed hash itself (a process which can run in a lot less memory : it doesn't actually index memory, it just generates the pseudo-random bytes directly, then it sends back the correct keyed has).

Calculating the entire hash in the processor cache will let this method outperform the checksummer by a large margin, meaning it can send the data back after any chosen interval.

(and before you say "keyed", obviously the encryptor needs to know the key)

In practice you'll also find that large amounts of ram space cannot be swapped out : BIOS, some bits of the operating system, ... There's even memory locations that aren't in the memory map, but are nevertheless backed by physical ram (in a pci card or ...), and thus won't be included in any (sane) scan. Surely one can find some place where a virus could squeeze in.

Re:At least one byte

[causality]

He pushes the hard work off onto an "external verifier" without making it clear how that would itself remain rootkit free.

If you have a guaranteed-safe external verifier, malware detection and removal isn't rocket science. Oddly, we have the technololgy in place to create a safe verifier: Trusted Coomputing. All we really need to solve the malware problem for good is something like Trusted Computing that we actually trust! An open standard, untainted by DRM, the provides hardware-level cryptographic isolation between the running system and the verifier.

I'd rather just secure my systems, thanks.

Re:At least one byte

[alexhs]

You've been modded funny, but this is exactly what this "article" is all about.

If there is no malware in RAM, the results will be the expected result. [...] Or there could be malware in RAM, and the checksum will be wrong. [...] Or malware could divert the read requests [...] . That would result in the right checksum... but a delay.

Or, there could be malware in RAM, not diverting read requests, and the checksum will be the expected result, and without a delay.

The only problem with Markus Jakobsson grand theory is that all malware are of that last kind.
Well, all malware since the memory protection era. I suppose his "product" could work for DOSes (but there is no multitasking there) Windows 3, MacOS9, AmigaOS and some embedded OSes.
And if the malware does virtualization, he is located in a memory area that his product won't be able to scan anyway.

So, simply put, it is a scam.

Re:At least one byte

[c 0xFF]

As near as I can tell, the article makes a HUGE assumption: the malware is actively trying to hide itself. This is not an unreasonable assumption, but it makes everything more clear. Let's go through his steps:

1) The algorithm swaps out memory.
2) The malware decides to stay active in RAM.
3) Random bits are written to memory.
4) Optional: The malware masks its presence by falsifying the memory reads.
5) A bad hash or delay reveals the presence of something trying to stay hidden.

For the special case that malware hides itself in memory that won't get swapped out, this might work. But a special case is a far cry from the claims in the article:

We can guarantee detection of malware. And that includes zero-day attacks and rootkits. We can even guarantee that we will detect malware that infected a device before we installed our detection program.

What if the malware is fine with being swapped out to disk? This algorithm detects nothing, of course, and the malware continues as normal, and can only be detected by traditional means.

I find the analogy from the paper quite revealing:

An analogy of our approach is that of an air marshal that at given time intervals demands of all travelers to take a step out on the wing, reporting to an authority on the ground who was there, providing them evidence that the plane is empty -- and first then, allows everybody to come in again. Our solution works even if the air marshal was delivered to the plane after takeoff.

So ... what happens if the marshal is clubbed over the head when entering the plane? What if the traveler carrying the bomb just leaves with everybody else? What if, what if ...

There's just too many holes to count.

Re:At least one byte

[lgw]

I'd rather just secure my systems, thanks.

How would you do that, then? There are so many clever rootkits these days, and no OS is "immune" (although SE Linux may be so if correctly configured, configuration is hard). Hardware-based rootkits (malware in the firmware of peripherals that attack an already kernel-mode driver process) are very dificult to defend against, you can only really hope they get enough bad press before they get you.

The only way to "just secure your system" against increasingly clever attackers is to have a safe partition, one that is simply not writable from the production OS, that can scan the production OS for rootkits. Virtualization by itself isn't enough - we're already seeing the first jailbreak exploits - you need something "below" any OS to enforce the paritioning.

Re:At least one byte

[alexhs]

I think you missed the following parts :

Instead of looking for known patterns -- whether of instructions and data, or of actions -- wouldn't it be great if we could look for anything that is malicious? That may sound like a pipe dream.

Not to me.

[...]

This tells us a few interesting things. We can guarantee detection of malware. And that includes zero-day attacks and rootkits.

Even with your interpretation :

We can even guarantee that we will detect malware that infected a device before we installed our detection program.

You can't detect known malware that way if it virtualizes the computer, because you will only scan for the memory the malware is willing to show you.

By the way, the following assumption is unworkable:

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself.

You can't swap out many parts of the kernel.
And I'm pretty sure kernel space parts of a rootkit won't let themselves swap out. Which does not mean uncooperative kernel modules are malware. If you're swapping out the disk driver, how will you get it back ? But maybe this exact disk driver is infected by the rootkit ?

Re:At least one byte

[Sancho]

You can't detect known malware that way if it virtualizes the computer, because you will only scan for the memory the malware is willing to show you.

Ah, there's where the bit about "knowing how much RAM you have" matters.

The virus has three choices:
1) Be overwritten, thus being eliminated (and showing you all of the RAM in your system.)
2) Swap part of what you're writing to disk.
3) Present less RAM than you actually have.

If you know how much RAM you have, you can detect choice 3.
If you can detect latency between secondary storage and RAM, you can dectect choid 2.
If the virus doesn't mind disappearing for the rest of the computer's runtime, you can mitigate damage via choice 1.

Now you may not know what kind of malware is on your computer, but you know that something is there.

Theory and Reality

[ircmaxell]

In theory, theory and reality are equivalent. In reality, they are quite different...

Seriously, how could this possibly work for ALL (including undocumented, and hereto unknown) threats? And if it does it by reading straight from RAM (through the kernel), wouldn't a rootkit be able to trivially defeat that?

Re:Theory and Reality

[goombah99]

His whole point was not "this is how you should do it", it was "you could do this, and because you could do this it shows that it's theoretically possible". This is a variant of what is know as a gedanken experiment-- an argument that proves or disproves some fact is true while not actually being somethign you would want to carry out. For example, you could suppose that you could measure the force field is under by running a pole from the earth to the moon and pushing slightly on it. Not that you want to do this, but it shows that measuring that force field is possible at all. Now you need to figure out an easy way to do that.

In case anybody was wondering...

[fuzzyfuzzyfungus]

He is indeed selling something...

Theory and hand-waving

[Lord Grey]

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong. The external verifier would notice and conclude that the device must be infected. Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies. That would result in the right checksum... but a delay. This delay would be detected by the external verifier, which would conclude that the device is infected.

<sarcasm>Punting the problem to an "external verifier" is pretty neat. I wish I could do that with my next hard problem.</sarcasm>

That whole bit about swapping, though.... If I write malware and hide it somewhere in execution space, do I really care if it gets swapped out? So the code that steals keystrokes or sniffs for credit card numbers doesn't get executed for short while. Big deal. At some point it will get loaded again (if written properly, that is).

Or am I missing something obvious?

Still a needle

[dmgxmichael]

A needle in a haystack wants roughly the same amount of space as a straw - doesn't make it any easier to find (indeed, that's part of the reason it's so hard to find).

Even if this technique has merits, it does nothing to correct the primary reason for computer infection - stupid users.

Re:Still a needle

[mooingyak]

I think god has a monopoly on causing death in that department. (Take that last sentence however you will. Just remember: however you take it is how I meant it.)

You're sleeping with your mother? Gross.

Which one is the detector?

[mangu]

How about a malware that masquerades as this detector and reports the RAM checksum is OK?

Re:Which one is the detector?

[evanbd]

Detecting a malware detector is just as hard as detecting malware. In general, detecting software of a specific type is halting-equivalent. In practice, the goal is to take shortcuts so that your adversary has a halting-equivalent problem and you don't. At present, the malware authors are winning. If we could force them to detect the malware detectors, that would be a huge advance.

My skepticism about this is the obvious one: what if the malware just lets itself get swapped out, and relies on stealth to survive the process?

"Guarantee"

[Reason58]

You can't guarantee anything in security. Especially when a human is involved.

Re:"Guarantee"

[dmgxmichael]

Reminds me of Murphy's 7th law - "Should you ever idiot proof anything God will make a better idiot."

Does it have to be in RAM?

[characterZer0]

It could be in ROM.

It could be in processor cache.

It could be in the video card's memory.

Could it be in pipelined instructions waiting to be executed?

Re:So it has to be in RAM

[dissy]

The hard part is actually finding it.

That reminds me of a signature I've seen around here (Sorry, I don't remember who was using it)

cat /dev/ram | strings | grep llama
OMG, my RAM is full of llamas!

Malware detection is Bogus.

[bmo]

How about we change things in Windows so it actually prevents infection in the first place?

1. Educate users. Microsoft does a piss-poor job of this.
2. STOP DEPENDING ON 3 MAGIC LETTERS TO DETERMINE IF SOMETHING IS CODE OR DATA. COME ON, SERIOUSLY. THIS SHOULD HAVE DIED WITH CP/M.
3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.
4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION.

Really. Seriously.

No, the above won't cover every situation, but it's a pretty good start.

--
BMO

register

[bugs2squash]

Some processors may have big enough register sets that malware could reside entirely within the CPU.

Some amazingly bad assumptions

[nahdude812]

Sure, malware has to occupy memory. That doesn't mean it has to be its own memory. Buffer overflows are all about corrupting another application's memory space.

His basic argument is that if you want to scan RAM, the kernel can halt all processing except its RAM scanner, and have a go at the RAM safely. If it's particularly insidious malware, it'll try to hide itself in various ways, one of which would be to masquerade the portion of RAM it was using with something legitimate looking (maybe erase that portion of memory). But you know it did this because you can see that memory which was supposed to be free is no longer free. Except the hardware has no concept of free or occupied memory. It just has memory, and the OS keeps track of what's free and not. The OS - the same space where malware is running.

OR, the malware could simply not do this, then its behavior is no different from any legitimate program. So how do you detect it now? You still need definitions that say, "When running in memory, this virus looks like X," then look through memory for that pattern.

Besides, who's to say that the kernel space is guaranteed free of malware itself? Even if you would have successfully identified the threat in RAM, you have no guarantee that the malware hasn't corrupted the identification routine.

It's like someone came along and said, "Hey, you guys are looking for malware wrong. You have to look for it! And I mean really look for it!"

Re:Some amazingly bad assumptions

[benjamindees]

who's to say that the kernel space is guaranteed free of malware itself?

Of course. It's obvious to say that if we can guarantee the kernel, we can reliably use it to scan for malware. What this completely misses, of course, is that the kernel itself can be compromised and there is zero way to guarantee it, after the fact, without a cold boot and poking around in all the stored memory contents of the system.

This fact continually fails to deter the snake-oil salesmen from peddling on-line virus scanning as an infallible cure.

Refuting the imaginary article in your head

[spun]

Still haven't read the article, eh? The technique is to swap everything out except the scanner, then write random bits to the entire memory space, then hash the memory. I could explain it all in greater detail, but, you know, there's this article, already there. Please do try to constrain your criticisms to things that actually apply to the article that was written, you know, the one we can all read. Refuting the imaginary article in your head does nothing for the rest of us.

Re:Refuting the imaginary article in your head

[spun]

Protection from malware should function like the immune system, with many lines of defense and many avenues of detection and counter attack. Prevention will never be perfect by itself.

Re:Refuting the imaginary article in your head

[Romancer]

I'm not an expert and not sure if I'm missing something obvious here but what is confusing me is the part about "swap everything out except the scanner". Wouldn't you then just be moving the malware too? Into a protected space that you then have to scan and know what to look for?

If it's a zero day infection then you don't know what to look for and you swapped it out of memory for nothing really. I do get that if it tries to protect itself it will look suspicious but what if it looks like a normal program? A service or scheduled task that could be normal. What if it takes on the guise of an adobe update program in size/hash and function until it is time do act? Say a slight change to your systems dns entries. Then goes dormant again.

This may not be possible but I haven't seen why not and it leaves a pretty big hole for zero day infections that this method claims to be able to catch 100%.

Re:Refuting the imaginary article in your head

[HungryHobo]

After reading TFA I'm still not seeing how this is supposed to detect unknown malware.
As far as I can see it would decide that a new install of any kind was a virus.

Sure if you know every program which is supposed to be installed and none of them do wierd things in memory(a big if) then you might be able to spot when some kind of change has been made but if you can do that then you have a situation where you might as well just re-image the machine from ROM every now and then.

I don't see any amazing new ideas in TFA

Re:Refuting the imaginary article in your head

[TheLink]

> have to scan and know what to look for?

If Dr. Markus Jakobsson has solved that, it is strange he hasn't also announced that he has solved the halting problem ;).

Figuring out whether a bunch of bits is or isn't malware is going to be as hard (or harder[1]) to figure out whether a program given a particular input will halt or not.

[1] Especially when that bunch of bits might later on download a new bunch of bits. Yes I know in theory it is impossible to solve the halting problem (no general solution), but in practice some special cases can be solved.

In contrast it's harder to solve the malware problem when you assume they can fetch new instructions and data, and from an active hostile party.

Sandboxing is the way to go. Sandboxing is analogous to avoiding the halting problem by having a time limit on the program.

No need to figure out whether the program is malware or not. Just make sure it can't do what you don't want it to do. Let the OS sandbox it.

Re:Refuting the imaginary article in your head

[Magic5Ball]

"detect unknown malware... a new install of any kind was a virus"

Yes, unless the new software is authorized, it should be considered malware.

"I don't see any amazing new ideas in TFA"

Also, yes. Shockingly, unauthorized software is unauthorized; and you can inspect a running system externally without being owned by that system.

Re:Refuting the imaginary article in your head

[edmicman]

Wrong! Abstinence is the one and only preventative answer!

Re:Refuting the imaginary article in your head

[HungryHobo]

If you have that much control over what's installed on the system then you might as well just take a snapshot of the system running in a safe state, place any data which needs to change on some kind of hardened remote server(like a seperate database) and just periodically re-image the whole machine from ROM.
Probably be more effective than this proposed system as well.

Re:Refuting the imaginary article in your head

[sopssa]

Yes, and what happens when everything gets swapped back? There's the malware again.

If you assume it would be easier to find and delete the malware when it's not resident in memory anymore, then you could do that just fine on another computer and working directly with the hard drive. You would need to do that anyway, since your RAM is swapped out. AV's are already quite integrated in to the system, never can delete anything (so you would need to do it by hand) and in worst case scenario could do the deleting upon reboot. So, what's the point?

Re:Refuting the imaginary article in your head

[spun]

If the malware let's itself get swapped out, then it can't hide it's memory footprint. Assuming we have started from a known clean machine, it is then trivial to figure out what the memory footprint should be. If it is larger than it should be, there is swapped out malware.

The point is, the malware will be detected whatever it chooses to do.

Re:Refuting the imaginary article in your head

[clone53421]

From TFA:

Anti-virus products scan for malware in two ways. They look for sequences of bits that are found in programs that are known to be bad (but which are not commonly found in good programs). And they run programs in sandboxes and look for known malicious actions. The first approach only catches known malware instances, while the second can also catch variants of these.

His “solution”:

Create a sandbox consisting of all available RAM which definitely contains no “good” programs except for the scanner. Scan the sandbox to detect any “bad” programs that remained (by looking for the malicious action of attempting to mask its process RAM). Then scan the hard disk for sequences of bits that are found in programs that are known to be bad (but which are not commonly found in good programs).

Thus, it is subject to exactly the same limitations that he was trying to improve upon.

Re:Refuting the imaginary article in your head

[Java Pimp]

It's getting kind of boring explaining the article over and over again.

You mean the part they say they don't even need to start with a clean machine?

Re:Refuting the imaginary article in your head

[Chris Mattern]

If the malware isn't active, then it can't hide it's tracks. We will see it's memory footprint.We know how much space everything else should be taking up, and we know more space than that is allocated, so we know something is afoot.

Unless, of course, the infected system is lying to you about the memory allocations.

Re:Refuting the imaginary article in your head

[Runaway1956]

"As good as this technique may or may not be, it's not security."

Ehhh, you could have been a sailor. Back in the day, we had "security alert" drills, to deal with "intruders" all the time. But, that's all we ever had, were drills. The REAL security was focused OUTSIDE! If some boat, ship, helicopter, or even a frogman came close to our ship, we just blew them out of the water or sky. There was never a need to look for an intruder INSIDE the ship!

The most likely time and place any intruder might get aboard, was when we were tied up to a pier. Social engineering MIGHT cause some young fool to bring a malevalent guest aboard. But, the OOD, the POOW, and the duty section were all there to screen for potential bad people.

Trying to find the bad guy AFTER he comes aboard would be utterly ridiculous. He only needs a minute or two to plant his bomb, his bug, or whatever the hell he was there to do.

Security means killing it before it gets here. So, yeah, I like the way you think. Now, the problem is, convincing others to think that way. Most especially, those others at Microsoft.

Re:Refuting the imaginary article in your head

[nabsltd]

Suffice it to say, you haven't understood it yet.

I think these people have all understood the article quite well, and are pointing out real flaws with this scanning method.

The #1 flaw is the assumption that an exact byte count of what is running can be known if malware is also running on the system.

If malware is actively running, then if scanning code calls any outside functions, the results must be considered tainted. Since there is no way for code that does not query the OS to even guess about what else is loaded in RAM, sufficiently intelligent malware will be able to hide itself from any scan. Hell, you can't even determine how much RAM is in a computer running in x86 protected mode without calling some OS or BIOS function, either of which can be hooked by the malware.

One of the other assumptions is that the time it takes to compute a hash of RAM on a particular machine can be known with enough precision to detect that it is being "delayed" by malware.

Last, there is one piece of code that can never be swapped out to disk, and will likely show up as malware as it refuses to be overwritten: the code that swaps and restores pages to/from disk.

Re:Refuting the imaginary article in your head

[clone53421]

No, YOU are the one who doesn’t understand the concept of what is happening.

Looking for “space” that is occupied by malware in this manner will only detect malware that attempts to hide itself. There are multiple ways of defeating this scan, not the least of which is simply to not attempt to hide, allowing itself to be swapped out of memory like any other application.

In which case, you would not be “guaranteed” to detect all malware, because you’d be scanning it in the swap file like any other application and you’d be depending on your signature database to find the malicious code. It didn’t do the one malicious action that you were relying on to give away its presence.

Re:Refuting the imaginary article in your head

[sopssa]

Plenty of empty space in the code segments? How so? There are not 'empty blocks' that malware can put itself into, it will either need to delete things (and break things in the program) or enlarge the program.

And with this comment we can conclude you don't know how PE files are mapped in to memory in Windows.

There's plenty of unused space between functions for example, usually INT3 opcodes which never get run as they're outside the program execution path. PE code sections usually also contain a lot of zero-byte 00 memory mapped, which again isn't in use, but is still allocated. Read more here, it's also used when making cheats to games. And that's only way to inject your code in to existing program, and doesn't change it's memory allocation at all.

Re:Refuting the imaginary article in your head

[nabsltd]

If we start from a known clean machine

By doing this, there's really no reason to run any high-tech malware detector, as this assumes you have two machines that are identical in every way except one might have malware while the other is known to be clean. If that's the case, just clone the one that is known to be clean and you now have two known clean machines.

In other words, there is no way to use an external machine to assist you in determining things like memory used by a process because you can't have both a clean machine and one infected by malware and have them identical enough to get a meaningful comparison.

There is no 'empty space' in a rpogram's code segment.

There very often is, as pages are 4096 bytes (at least, normal pages on x86 are this size), and it's rare to have code completely fill the page.

Since all the protections on x86 are at the page level, malware can hide in the bytes after the real executable. This type of technique has been used for years (malware used to hide in the file slack on floppy disks).

Re:Refuting the imaginary article in your head

[clone53421]

The article never mentioned scanning for signatures

Read the damn article. I already quoted the part in question:

They look for sequences of bits that are found in programs that are known to be bad (but which are not commonly found in good programs)

I.e., scan for its signature. That is how conventional antivirus works, and it is how his idea falls back to working if it doesn’t detect anything that’s trying to hide. So if it doesn’t detect anything trying to hide, it has to scan for it conventionally.

This is just an elaborate scheme to ensure that the RAM isn’t infected. Maybe I need to explain.

A known clean system is used for scanning other systems because you know its RAM isn’t infected. The infected system can’t reliably be used to scan itself, because if has a rootkit, an active virus could be residing in RAM, either feeding your scanner false results or simply hiding there until the scan completes.

So, his idea is this:

Flush everything out of RAM, and fill it with garbage. The RAM *should* now be clean. Generate a checksum of this “clean” RAM.

Read the entire contents of the RAM again, generating a second checksum. This second checksum should match the first one.

If the checksums do not match, you wrote a value and read something different which means one of two things: Bad RAM, or a virus is hiding itself.

If the checksums match but weird timing anomalies were detected, a virus is hiding itself.

If the checksums match and no weird timing anomalies were detected, the RAM is now clean.

If the RAM is clean, you can then scan the secondary storage just as if you were using a clean system to do the scan, with the same reliability in your results.

However you would be subject to exactly the same limitations as you normally are in using a clean system to scan the secondary storage of an infected system: you have to use your signature database to detect viruses in the bytes on the disk.

Re:Refuting the imaginary article in your head

[clone53421]

But it does seem to be foolproof if you start from a known clean machine.

That’s the whole point of his system: to scan from a machine that isn’t known to be clean.

The standard way to scan from a “known clean machine” is one of the following:

1) Use a known-clean boot CD. This ensures that the system RAM is clean. Once booted into the clean system, scan the possibly-infected secondary storage devices.

2) Use a separate known-clean computer. Its system RAM is clean. Attach the possibly-infected secondary storage devices from the first computer to the known-clean computer and scan them.

The 3rd way, which is not foolproof, is to attempt to avoid a boot CD or second computer by scanning the system RAM. Once the system RAM is known to be clean, you can proceed from there as in either 1 or 2 above. However, there is no foolproof way to ensure that the RAM of a running system is clean.

He proposed a “foolproof” way to ensure that the RAM of a system is clean (without shutting it down and booting it from a clean boot device). That’s the only thing that makes his suggestion useful.

However, it isn’t foolproof, and even you see that.

FWIW, RootkitRevealer uses exactly the same method, so it’s not like this guy came up with anything novel. In fact, RootkitRevealer even has the disclaimer:

note: RootkitRevealer is not intended to detect rootkits like Fu that don't attempt to hide their files or registry keys

It only finds things that are trying to hide. If they aren’t trying to hide, you have to scan them conventionally.

Re:Refuting the imaginary article in your head

[maestroX]

I'm not an expert and not sure if I'm missing something obvious here but what is confusing me is the part about "swap everything out except the scanner". Wouldn't you then just be moving the malware too?

No. Symantec is the scanner.

Re:Refuting the imaginary article in your head

[turbidostato]

"Still haven't read the article, eh?"

I did. The relevant parts:

"Any program -- good or bad -- that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte, right?"

More news from Captain Obvious Dept.: any program -- good or bad -- that wants to be resilient between cold bootups has not choice but to take up some space in persistent storage. At least one byte, right?

"All we need is the help of an external verifier that knows how much RAM a device we want to protect has, and how fast its processor is. And ways to avoid latency variance when we measure the time to compute the checksum.
This tells us a few interesting things. We can guarantee detection of malware. And that includes zero-day attacks and rootkits. We can even guarantee that we will detect malware that infected a device before we installed our detection program. Think about it. Or read more here and here."

What does it add to current solutions? My answer: almost nothing to absolutly nothing.

1) By trashing out everything from RAM you effectlively have disrupted the service so it's no advantage from other stablished disruptive means like... scanning the disk out of a cold boot.

2) By just cleanning RAM you will have only the ability, even if your system is 100% guaranteed, to detect *only* the malware that was in RAM by the time of the scan. A simple script running from cron will be absolutly undetectable unless it happens to be running by the time the scan was scheduled.

3) No, it won't work on already compromised systems: even if it's run from kernel mode the execution space will already be tainted: think about a virtual server and a virtualized box.

4) As I already said, for a system to survive cold boots it needs to go to persistent storage. If I can know what the RAM contents are expected to be I certainly can easilyl know what the HDD (or BIOS EEPROM, or any other persistent storage) should look like too, so I can compare checksums.

5) Oh! but the system used to get a clean checksum can be compromised! So does the system used to provide the RAM space.

So, in the end this system offers absolutly nothing from the "standard" off-line verification scenario: on a home system boot from CD and check the hard disk; on a corporate system boot from PXE and check the hard disk.

But it offers nothing either on the real time field: TPM can offer just the same only in real time so while it only inspects RAM it inspects RAM at every moment, so the moment the malware is trying to get into RAM it's the time it gets revealed.

Re:Refuting the imaginary article in your head

[smallfries]

Ok, so you want serious criticisms?

1. They assume that the read back from external storage doesn't overlap with computation. This is not true of any DMA-based transfer that is asynchronous. This breaks their argument as I can hide the time to do a memory / storage swap by using DMA during calculation of the hash of a different region.

2. It is brittle. It depends on a very narrow margin between computation time and retrieval time. There are many things that could cause this margin to change but their complete avoidance of cache variations would kill it (yes I've read it, no their "worst-case measurement" doesn't cut it).

3. As many other people have pointed out it requires a trusted third-party to do the verification. If we allow a trusted party then installing a TPM is much simpler way to do it.

4. If the malware allows itself to be swapped out (i.e it has infected a callback hook) then the system must drop back to standard scanning techniques. These are not guaranteed because of the halting problem - so the author's claim that this method gives guaranteed detect is a crock of shit.

5. Although they've cited Naccahe's paper they've not really done it justice. His approach was much more elegant as it did not require a trusted party, and it got bonus points for using Quines in a serious research setting.

Re:Refuting the imaginary article in your head

[spun]

See people? If you read the article, you can offer cogent criticisms. If you don't you can offer irrelevant criticisms you will then have to spend the next several hours massaging and defending.

I think number 4 is the most cogent. The author claims his system can detect 0-day malware that was on the system before the scanner was installed. Maybe, if the malware tries to interfere. But if it doesn't, you have no signatures or checksums to fall back on, how could this system work?

Redeeculous idea.

[Ancient_Hacker]

I tried reading TFA a few times. First time, utter confusion. Second, third times, no better. I can't make any sense out of these points:

>1) There are absolutely only three things malware can do when you scan for it. One: be active in RAM, maybe trying to interfere with the detection algorithm. Two: not be active in RAM, but store itself in secondary storage. It cannot interfere with the detection algorithm then, quite obviously. And option number three: erase itself.

Absolutely, not. There are many other things malware could be doing. Inactive in RAM, compressed and inactive in RAM, encoded as plausible-looking entries in the File Name Table or the Virtual Memory map.

>2) Any program -- good or bad -- that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte, right?

No, it could be sleeping, existing only as an entry in the swapped-out process table. Or in unused space below a thread stack.

>Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself.

Whoah there fella. Everything? Are you going to turn off all timers and interrupt enables so their service routines don't get called?
Hard to do without mucking up all the device drivers. Are you going to swap out the kernel too, as malware is quite capable of infesting kernel space. And what about device drivers? They're constantly mucking with their internal tables and I/O buffers.
And if you turn off all device drivers, you lose, as there's nothing stopping malware from masquerading as a device driver. Many do.

>>But if we know how big RAM is, we know how much space should be free.

Whoa there again, big guy. There are plenty of machines with RAM at places not generally known to the OS, such as video RAM, graphics polygon RAM, network card RAM buffers, and kernel stacks.

>> Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten.

You don't need a checksum test to do this-- each page of virtual memory has R/W control bits.
And you're foiled here again, as there are plenty of system areas that are write-protected, such as pre code areas and the VM tables themselves.

>>Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean.

Nooo, that just tells you that either you overwrote the malware, so you'll never find it, or the malware during your two sweeps did not change any RAM contents. Quite possible as most malware just sits around most of the time.

>> Or there could be malware in RAM, and the checksum will be wrong.

Well, no, unless you disabled all interrupts and stopped all kernel tasks, there will still be system timers and interrupts and device drivers changing their state in RAM.

>> The external verifier would notice and conclude that the device must be infected.

Or some part of the system or some device driver is still running. Huge chance of false positives.

This essay seems to have been written by someone with only a glancing familiarity with hardware and system software.

Snake Oil, part 2...

[dskoll]

From the Our Solutions page:

A technique known as software-based attestation can provide an alternative defense against malware by performing infection scans periodically and detect the presence of any program that refuses to be inactivated – as well as any inactivated program that is known to be malicious.

So, it can detect malware that refuses to be inactivated which is a tiny (vanishingly-tiny?) percentage of malware, as well as inactivated software that is known to be malicious (eg, because of a known virus signature.)

So what's the advantage over signature-based virus-scanners? Well, you get to detect completely hypothetical software that (somehow) refuses to allow the kernel to swap it out (and how that is possible is never explained) at the cost of hugely-expensive computations.

Great.

Re:Interesting point

[amRadioHed]

I have a hard time believing a technique that starts with "swap out everything in RAM" could ever be used for real-time detection.