Predicting User Behavior to Improve Security

CitizenC writes "New computer-monitoring software designed to second-guess the intentions of individual system users could be close to perfect at preventing security breaches, say researchers. Read more." The paper (pdf) is online as well.

Comments

hmmm...

[Britissippi]

Sounds great in theory, however, what happens when users change roles, get promoted, demoted..... and what they have to do with their terminal changes as a result. You'd have to have a staff working full time at any average sized company making the system changes to keep this thing from triggering constant alerts.

Does sound promising though.

Re:hmmm...

[Angry White Guy]

Hell, even without promotions, added staff, etc. Everyone in my office acts irrationally enough to screw the system up completely in an hour or so.
They can't fire us all.

Re:hmmm...

[bmwm3nut]

i don't think they mentioned the method in the article. but i can imagine using something like a neural network to learn the users' behaviors. from my limited work with nerual networks, i've discovered that they're really robust when they learn a problem. it's totally concievable that a neural net could learn irrational behavior too.

promotions wouldn't be a problem either. you have the network have a parameter for the type of job that a user is supposed to be doing. when they get a promotion that job type will change. their new behavior will not be marked as bad until the system learns the new behavior.

of course everything i said is under the assumption that they'll be using neural networks.

Re:hmmm...

[dubious9]

My guess is that it wiil take a statistical look at commands a la Bayesian Spam Plan

After all, probing port looks different than fixing network problems, package manangement/installation looks different than maliciously deleting files, trying to find memory leaks looks different than trying to access another process's memory space. They all us similar commands/system resources, but it should be possibile by look at a few tens of instructions whether a user is try to be malicious or not.

These may not be the best examples but the general idea is that it should be possible to determine user's intent because the probability of a sequence of commands having both a normal and malicous role, should go quite down the more instructions the user executes.

Even false positives should be useful to admins by telling about inadvertant, i.e. acidentally typing rm -rf *,users as well.

Re:hmmm...

[DunbarTheInept]

I have my doubts:

for example: which is the malicious activity?
User A types: rm -rf *
User B types: rm -rf *

(User A was in the root dir at the time. User B was in a subdirectory of his home directory at the time.)

Okay, that's easy- just remember to track the context of where the user currently is. But then what about this?

User A types: rm -rf /shared_network_drive
User B types: rm -rf /shared_network_drive

The difference is that User A was trying to delete everyone's stuff, while User B, knowing how the permissions on the files work, was just trying to find a lazy way to delete those files that he has permissions on because he was trying to clear his own junk out of the /shared_network_drive. He was being sloppy, but not malicious.

How does the software know the difference?

Home Security

[McFly69]

-Note to Self-

Keep doors locked at my house to prevent other people from coming in.

what if

[Diclophis]

the first action you take breaches security?

Arms Race

[queh]

Surely this will just prompt crackers to stealth their actions in commands that are similar to how the system is used normally?

Well, um

[Roadmaster]

if they had any clue about real-world users, they'd know they're absolutely unpredictable. A user's creativeness to mess things up never ceases to amaze.

Re:Well, um

[qengho]

A user's creativeness to mess things up never ceases to amaze.

Or as one of the corollaries to Murphy's Law states: "No matter how idiot-proof you make something, an ingenious idiot in the field will find a workaround."

Sounds good for other people.

[teamhasnoi]

Might be something to install to prevent me from reading /. all day long.

Oh wait. That wouldn't be unusual. DAMMIT!

aliasing

[Brandon T.]

Wouldn't it be relatively easy to get around this by aliasing shell scripts to frequently used commands? Sure, the admin might be able to find the shell scripts lying around, but if an intruder was trying to do a one-off attack, it might be viable.

Brandon

Re:aliasing

[halftrack]

I think that's untrue such a scam is not viable. The shell scripts would call commands that get registered by the system and plain alias will only affect the user, the system still sees the original command.

Re:aliasing

[DunbarTheInept]

But what about making new programs to imitate existing ones, but just in a way that isn't noticed by the snooper? (for example: myFuzzySlipperProgram could be a renamed "rm" program compiled from source.)

Or, just do your malicious cracking using system calls from your own C programs. Don't use the rm command in a script, use a program that calls unlink().

To even have a chance of being effective, the system would have to be watching not the commands you type, but the system calls you make. (In unix terms, any time you do something using one of the functions on man page 2, the system library would have to log that.)

Credit card / phone companies...

[monadicIO]

How is the system used by credit card and phone companies different than the one proposed by this paper?

Stifle creativity

[nut]

This would encourage users not to experiment and find new ways of doing tasks, if everytime you tried something new a sysad came round to ask you what you were doing.

Re:Stifle creativity

[Damion]

Keep in mind that the sysadmin can see quite well what the user is doing. The point of this is just to raise a flag if someone does something outside of their daily pattern, not to mark them for inquisition.
All the sysadmin has to do is look at the log and say, "Ah, he's just trying to figure out how to filter his email" and dismiss it, whereas trying to get acquainted with an unfamiliar system and all of its configuration files would be extremely obvious.

Not bad but...

[aridhol]

At first glance, this looks like something that may be useful. However, what happens if a user knows about the system and its patterns, and plans out the attack over a large period of time?

The user could "poison" the information by slowly changing his working habits. If done properly, the AI would probably think this was no different than the user just learning to do things in a different way. When the habits are close enough to the infringing behaviour, the user can probably do anything without setting off alarms.

In addition, if this is the only line of security, the user can then gradually return his patterns to normal. The logs from this system won't show anything. The PHBs may well decide that, when using something as smart as this, traditional logs won't be needed.

Re:Not bad but...

[Damion]

Well, this could never be the only line of defense. Applying patches regularly and maintaining sane security guidelines could never be obviated by an automated system. Think of this as just another level of intrusion detection software. The methods used to stop intrusions from happening in the first, and those to mop up afterward, would remain unchanged.

Re:Not bad but...

[aridhol]

Nothing can ever be the only line of defense. How many PHBs know that? When they see/hear from media/rumours that this is the ultimate defence, how many of them will rush out to get it and tell their IT staff that this is all they need?

Minority Report?

[zoward]

And how long will it be before users start losing privileges for things that they "potentially might do" (with a 94% accuracy rate). About one in 20 of us is really going to suffer for this one.

Intelligent pr0n filters..

[grub]

..are what we need. If someone could come up with a box that could filter pages based on the amount of pink within the images I could delete 80% of my outgoing firewall rules at work!

Re:Intelligent pr0n filters..

[grub]

I assume the other 20% of rules cover the interracial and black pr0n sites?

They're all pink on the inside. :)

Re:destined to failure

[distributed.karma]

> The Heisenberg uncertainty principle states that there will always be true statements within a system that cannot be proved within that system.

Um, that's Godel's Theorem.

> Wish I was a Physics Genius

I think that just about sums it up. ;-)

Remember that this is network security

[complexmath]

The average user may be adept at breaking his PC, but he's much less likely to, say, flood the network with bad packets.

Obligatory

[scott1853]

Clippy: It appears as though you are trying to hack into an IIS box.

Would you to start the IIS hacking wizard?

Would you like to view a list of the top 1,000 exploits?

Never show this prompt again, its already too easy to hack IIS.

Re:After reading the PDF intently (skimming)

[Sludge]

This seems to stop people from using an account that has access to certain data, which is not their account. If a user usually accesses files with Explorer, and someone sits down at their logged in machine and brings up a command prompt, CDs to the dir, and types 'start .', that would trigger a variant in behaviour.

You could go even further and log a typing rate jump or dip of 30 WPM.

Good idea but doesn't solve the old problem

[Ektanoor]

I took a brief look at the paper and sincerly the idea is not bad at all. However that 94% is pure hype.

The biggest problem in computer security, in what is related to users, is not anomalies, but the usual practice. Remember that experts say that 90% of flaws is due to insiders and not outsiders. And why? Because 99% of these insiders don't care a nail for security. Most of them keep using the wife's name for password and sharing C: to everyone. And no matter the efforts, policies, orders and instructions keep gaining dust. If you try to enforce them then you get a crowd in front of the boss with a rope for your neck.And if even the boss comes up to defend your work, everyone start to mine all your job. All they want is Internet, passing documents and hoping that you finally get out and Microsoft comes in to solve all the problems. That's what the lamers think about security. And in this mess, no matter the expert you are, no matter the tools you have, no matter the hours you loose on the net, you always get trouble every week.

Besides I noted that if someone is going for the break-in, he will mostly go from start. It starts up with this guy "playing" with the computer, then it goes up to the net. Later he thinks he's smart enough to break the server and show that the security admin is a LaMeR.And it ends up with you looking at his desktop and writing the final document to fire or put him into court. You may ask why this guy could go so far. Because he's smart, because no matter the lamerness he is good on something. So the boss will think twice before firing him. If you are in a corporation, then the boss will hang you up with this "unreplaceble" expert because in the city where he lives there's no one else to do his job. Besides, the corporation lost too much money on training him and doesn't want to start from zero on this. So you continue to see the bastard for a few monthes more before you catch him on the red spot.

I saw this and I know that this is a problem on many companies and state institutions around the world. So how this system will help you in such cases? It will, with a large margin of error as the main anomaly, the user, is there from the very start..

Profiling crackers? Brilliant!

[jabber01]

Sounds like profiling terrorists. It'll work great, and everyone will feel secure and all, until somone flies a plane into their "secure" server.

Bruce Schneier

[elb]

...was recently featured in this article about US security policy, and primarily on the dangers of relying too much on technolgoy. the article is great -- not super-techy, but a great explanation of technology and security policy; it makes an intimidating topic accessible to the intelligent non-tech. a couple of good points from the article:

• "[the leading / best face recognition] software has a success rate of 99.32 percent--that is, when the software matches a passenger's face with a face on a list of terrorists, it is mistaken only 0.68 percent of the time. Assume for the moment that this claim is credible; assume, too, that good pictures of suspected terrorists are readily available. About 25 million passengers used Boston's Logan Airport in 2001. Had face-recognition software been used on 25 million faces, it would have wrongly picked out just 0.68 percent of them--but that would have been enough, given the large number of passengers, to flag as many as 170,000 innocent people as terrorists. With almost 500 false alarms a day, the face-recognition system would quickly become something to ignore."
• "The most important element of any security measure, Schneier argues, is people, not technology--and the people need to be at the scene. Recall the German journalists who fooled the fingerprint readers and iris scanners. None of their tricks would have worked if a reasonably attentive guard had been watching. Conversely, legitimate employees with bandaged fingers or scratched corneas will never make it through security unless a guard at the scene is authorized to overrule the machinery. "

Hm

[E_elven]

$ r00t machine
Ush: command not found: r00t

*meanwhile, in the Secret Command Centre*
#QUEER#COMMAND##INVESTIGATE##

$ owNz0rz machine
owNz0rz: unknown parameter machine

*SCC*
###THERE#MIGHT#BE#SOMETHING#GOING#ON##

$ owNz0rz r00t
Ush: j00 owNz0r d4 r00t!

*SCC*
#####ALARM#ALARM#####

$
Ush: Someone trying to use 'alarm()', authorize? n0
Ush: Killing alarming process.
$ 1337

"Success" - "false positive" = garbage

[dpbsmith]

Any time someone mentions a "success rate" without also mentioning the false positive rate, they're feeding you garbage

I'd be much more impressed by a claim of an 0.001% false alarm rate than I am by a 94% success rate.

Yet, on a per-line basis, if you assume that a user averages, say, three typed lines per minute, that's 180 lines per hour = 360000 lines per working year.

A .001% false alarm rate means that an innocent worker is going to be interrupted THREE TIMES A YEAR by burly security people at the cube doorway shouting "Hands off that keyboard RIGHT NOW!"

so laik

[digitalsushi]

it'd be like...

while :;do for IP in `cat /var/log/httpd/access_log|awk '{print $2}'`; do /usr/sbin/iptables -t filter -A INPUT -p tcp -s $IP/32 -j DROP;done;done or something like that. Yeah. I got your AI right here. I can sell you a tarball with a digitally signed dignature- I'm quite digil when it comes to being a digilante.