MIT Reveals AI Platform Which Detects 85 Percent of Cyberattacks (zdnet.com)

An anonymous reader writes: MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) says that while many 'analyst-driven solutions' rely on rules created by human experts and therefore may miss attacks which do not match established patterns, a new artificial intelligence platform changes the rules of the game. The platform, dubbed AI Squared (AI2), is able to detect 85 percent of attacks -- roughly three times better than current benchmarks -- and also reduces the number of false positives by a factor of five, according to MIT. The latter is important as when anomaly detection triggers false positives, this can lead to lessened trust in protective systems and also wastes the time of IT experts which need to investigate the matter. AI2 was tested using 3.6 billion log lines generated by over 20 million users in a period of three months. The AI trawled through this information and used machine learning to cluster data together to find suspicious activity. Anything which flagged up as unusual was then presented to a human operator and feedback was issued.Fast Co Design has an interesting take on this.

Comments

Can it detect a Slashdotting?

[Anonymous Coward]

We're about to find out...

(Although today's Slashdotting pales in comparison to the Slashdottings of yore...)

Well ain't that grand

[JustAnotherOldGuy]

"MIT Reveals AI Platform Which Detects 85 Percent of Cyberattacks"

So, out of 100,000 attacks, only 15,000 will go undetected? Break out the champagne, boys!

Re:Well ain't that grand

[StikyPad]

The headline isn't the raw number, it's the improvement in detection rate, which is a substantial step forward.

I suspect that any machine learning algorithm is susceptible to being trained by attackers though, much the way 'Tay' turned into a Hitler-Loving Sex Bot. Unsupervised learning can be effective, but it's very easy to intentionally (and unintentionally) sabotage that success.

Re:Well ain't that grand

[JustAnotherOldGuy]

The headline isn't the raw number,

Actually, unless it's worded incorrectly, the headline does appear to be the raw number.

"The platform, dubbed AI Squared (AI2), is able to detect 85 percent of attacks"

Yes, it's "roughly three times better than current benchmarks", but the 85% figure does seem to be the overall detection rate. The reduction in false positives seems like a good improvement, though.

Re:Well ain't that grand

[Shoten]

What's the current percentage detected by a human or machine?
Article and summary suggest this is a 3x improvement over ... benchmarks of some sort.

If we are detecting 85,000 out of 100,000 instead of 23,000 out of 100,000, then yes I'd say champagne is called for.

How many breaches go undetected now? I know the number is greater than zero (Though over time it approaches zero as most breaches are found out eventually) if the AI can prevent my credit card number from being hijacked I'll support that.

You asked the magic question. While the post you were replying to seems to think that anything that isn't perfect (or really close to it) is a waste, you're asking the real question which is: "Is it better that current state of the art, and if so, by how much?"

The problem from the article is that they don't define what comprises an "attack." If you go very granular, each packet from a portscanner that's fired off against your public-facing architecture qualifies as an attack..though this definition has a signal-to-noise ratio so bad that it's useless. If you take a broad view, then a sustained APT-like campaign by a single actor against you...with all of the various probing and striking activities that are involved...comprises a single attack. I suspect that this solution lands somewhere in between, but much closer to the former than the latter. If so, then it may detect a lot more, but with a huge "so what?" factor since the attacks themselves will lack a lot of context.

Realistically, a vast amount of successful attack activity isn't detected anywhere close to the time it takes place. A recent study showed that over 90% of recent breaches resulted from exploitation of a handful of vulnerabilities, many of which are very old (and none of which were zero-day). This is a huge improvement over what's currently available, by a multiple factor that could range from single digits to orders of magnitude...again, based on the definition of "attack."

Re:Well ain't that grand

[lars_stefan_axelsson]

The headline isn't the raw number, it's the improvement in detection rate, which is a substantial step forward.

No, not really. They compare with their own (so called) state of the art unsupervised learner, and conclude that a bit of supervised learning beats that hands down. Yes, well, that's not really surprising, and it's not really a new result in intrusion detection research either. "Active learning" approaches have been proposed since at least 2004, and since they don't compare with state-of-the-art intrusion detection methods or systems it's very difficult to tell if their approach actually amounts to anything.

They do after all report false alarm rates of ~5% or so (down from ~20%! for their baseline) which is a completely unworkable number, due to the base-rate fallacy/class imbalance problem. With that high an FA rate you'll drown in false alarms before you'll find a single intrusion/attack. Even if your detection rate is 100%. And theirs isn't.

It's a real pity they didn't submit this paper to a more established computer security conference, one that was around when IDS research was in it's hey-day, back when we were doing "big data" security, but didn't know to call it that yet.

(P.S. And yes, self learning algorithms have the problem that they can be made to drift towards inefficiency with a surprisingly low amount of feedback on how they're doing. As a matter of fact, there are examples of using a machine learning algorithm to learn how to make the detecting algorithm as bad at its job as possible.)

A.I. platform that detects 85% of attacks?

[U2xhc2hkb3QgU3Vja3M]

Is it called Colossus or Guardian?

Not AI

[110010001000]

Again: this is NOT AI. But PatternEx is looking for VC funding so it gets hyped as such. This is just another expert system that analyzes log data. There are dozens of those.

Re:Not AI

[Megol]

A "no true AI" argument? This uses a neural learning system rather than a rule-based one, AFAIK those aren't commonly called expert systems.

However the new(?) thing is the design of the human-computer interaction, not the fact that it analyses log data.

Re:Not AI

[110010001000]

The concept of "neural learning" is a misnomer and used for hyping purposes. A neural network doesn't work like a neuron at all. Neural nets are not AI. They were a dead end in AI research.

Useful and necessary, if it works

[RevDisk]

In my opinion, anti-virus software has somewhat matured enough that most home users or small businesses, that remotely have a clue, use it. There's not a good analog for reading SIEM, event logs, etc. Solutions exist, but they tend to be cumbersome or expensive.

Even I pretty much just rely on snort's registered user ruleset, rather than the subscription. It would be a very nice spot for heuristic or AI to monitor. Call me paranoid, but I'd want it in addition to the generic static rulesets.

"It may be AI's ultimate test."

[fustakrakich]

No, that would be weather prediction. Pretty much the same thing though..

I had one of these years ago.

[Lumpy]

Step 1 : what is the source IP from?
Step 2 : is the source IP from outside the USA?
Step 3 : assume it is a cyberattack and throw out the packet.
Step 4: go back to step 1.

We never EVER needed anyone from outside the USA to access any of our servers, so we threw out all packets from outside defined IP sources. Solved over 85% of all cyberattack problems. Fake SSH and telnet login attempts dropped from 20 per hour to 1 per week. recently we started to remove IP ranges from Cable Internet providers and that significantly reduced the problems... No we dont care about consumers, we have very specific clients and they dont use consumer cable modems.

Tighten up your firewalls and servers, dont allow ip ranges you dont need. and yes we tell the CTO that when he is off to china that it sucks to be him, he will not have access.

Re:I had one of these years ago.

[khasim]

You don't even have to go through all of that if you just want to stop the script-kiddies around the world.

Move your SSH (don't use telnet) server to a different, RANDOM, port above 1024 and 99.99% of the login attacks will vanish.

This won't make your server any more secure but it will make your logs a lot cleaner.

Re:I had one of these years ago.

[toonces33]

Doesn't work for a couple of reasons.

First, identifying what IP addresses are out of the U.S. is actually not as easy as you think.

Secondly, a malware-infected server somewhere within the U.S. could still mount an attack on you.

Re:I had one of these years ago.

[angel'o'sphere]

And how would Amazon, Google, Facebook etc. then work?

Or do you think we ex nazi germans don't use them?

Can we see the source?

[xxxJonBoyxxx]

Can we see the source?

Re:Al?

[K. S. Kyosuke]

He's the guy from all those "et al." journal articles he's cowritten. He simply publishes a lot.

Yeah

[Dunbal]

while(1){

if(GetIsIt80PercentTimeYet()){

printf("Cyberattack detected, Putin did it!");

}

}

The unseemly truth

[Gravis Zero]

It's really just a 3.5 million character self-modifying regex. It should be aware by now. I knew this day was coming. What fools we've been!

Re:The unseemly truth

[JustAnotherOldGuy]

It's really just a 3.5 million character self-modifying regex. It should be aware by now. I knew this day was coming. What fools we've been!

I have a friend who says that our brain and our neural activity is "just a giant, continuously self-modifying regex pattern", and I'm not certain he's wrong. It would explain a lot, lol.

adaptive ?

[Tom]

That is cute, but how does it react to new threats and changes in the patterns? We've been fighting this war for decades - improved detection leads to improved evasion leads to improved detection, etc. etc. - will it maintain this advantage or after attackers have adapted just become one more piece of expensive latency generator?

He is, but DON'T ban him... why? apk

[Anonymous Coward]

I absolutely LOVE kicking the snot out of trolls like him with facts vs. their trolling bs lies here https://yro.slashdot.org/comme... & here https://yro.slashdot.org/comme...

* There's PLENTY like him & they are FUN to knock-the-chocolate out of - see proof in those links above as my evidence thereof!

APK

P.S.=> "I rest my case"... apk