Slashdot Mirror


Army Researching Network System That Defends Against Social Engineering

Nerval's Lobster writes "The U.S. Army Research Laboratory has awarded as much as $48 million to researchers trying to build computer-security systems that can identify even the most subtle human-exploit attacks and respond without human intervention. The more difficult part of the research will be to develop models of human behavior that allow security systems decide, accurately and on their own, whether actions by humans are part of an attack (whether the humans involved realize it or not). The Army Research Lab (ARL) announced Oct. 8 a grant of $23.2 million to fund a five-year cooperative effort among a team of researchers at Penn State University, the University of California, Davis, Univ. California, Riverside and Indiana University. The five-year program comes with the option to extend it to 10 years with the addition of another $25 million in funding. As part of the project, researchers will need to systematize the criteria and tools used for security analysis, making sure the code detects malicious intrusions rather than legitimate access, all while preserving enough data about any breach for later forensic analysis, according to Alexander Kott, associate director for science and technology at the U.S. Army Research Laboratory. Identifying whether the behavior of humans is malicious or not is difficult even for other humans, especially when it's not clear whether users who open a door to attackers knew what they were doing or, conversely, whether the "attackers" are perfectly legitimate and it's the security monitoring staff who are overreacting. Twenty-nine percent of attacks tracked in the April 23 2013 Verizon Data Breach Investigations Report could be traced to social-engineering or phishing tactics whose goal is to manipulate humans into giving attackers access to secured systems."

57 comments

  1. So... by camperdave · · Score: 4, Interesting

    So... Will this system also detect malicious government attacks and bouts of official stupidity?

    --
    When our name is on the back of your car, we're behind you all the way!
    1. Re:So... by Cryacin · · Score: 4, Funny

      No. They just do a string search for "Nigerian Prince" (toLower() of course). That'll be $23.2 million thanks!

      --
      Science advances one funeral at a time- Max Planck
    2. Re:So... by Anonymous Coward · · Score: 1

      Surely official stupidity qualifies as a form of Social Engineering?

  2. I'm sorry Dave, I can't let you do that by Anonymous Coward · · Score: 3, Funny

    Obligatory :)

    1. Re:I'm sorry Dave, I can't let you do that by geschild · · Score: 2

      Obligatory :)

      My thoughts exactly.

      (I explicitly browsed through all comments to see if this remark was made, yet :)

      --
      Karma? What's that again?
  3. If it works... by fustakrakich · · Score: 2

    Then maybe they can use it to predict the weather.

    Actually I think the Magic 8 Ball beat 'em to the punch..

    --
    “He’s not deformed, he’s just drunk!”
    1. Re:If it works... by Narcocide · · Score: 1

      Well it does actually do a better job of predicting the weather than weathermen, usually, but there is still room for improvement.

  4. So translated... by Anonymous Coward · · Score: 1

    We're gonna light as much as 48 million on fire.

  5. Ah, rrrrr... Huh? by MobSwatter · · Score: 1

    Doesn't the NSA already have access to army systems?

  6. Why does it seem to always come down to this? by ColdWetDog · · Score: 2, Funny

    "Skynet was originally activated by the military to control the national arsenal on August 4, 1997, at which time it began to learn at a geometric rate. On August 29, it gained self-awareness, and the panicking operators, realizing the extent of its abilities, tried to deactivate it. Skynet perceived this as an attack and came to the conclusion that all of humanity would attempt to destroy it. To defend itself, Skynet launched nuclear missiles under its command at Russia, which responded with a nuclear counter-attack against the U.S. and its allies. Consequent to the nuclear exchange, over three billion people were killed in an event that came to be known as Judgment Day."

    These things never get launched on time. Good thing the Army is persistent....

    --
    Faster! Faster! Faster would be better!
  7. Work around it by dutchwhizzman · · Score: 4, Insightful

    So when a social engineer knows such a system is in place, (s)he will devise a way to do their engineering without the system interfering or finding out. It isn't as if there is no protection or detection put on IT systems already. The trick of social engineering is to use the human factor to work around the technical countermeasures to get to their goal. Putting heuristic systems in place that will try to detect if a technical action may be part of a hacking attempt hasn't stopped virus developers from making viruses that successfully circumvent that. That's essentially all that this is, an attempt to do heuristic detection of malware or mal-action, just like a virus scanner.

    --
    I was promised a flying car. Where is my flying car?
    1. Re:Work around it by TheLink · · Score: 0

      The "protection" system could also be a way to DoS the network/services.

      A hacker might trick it into preventing all the humans from doing normal stuff. Or it could be misconfigured or too paranoid or buggy.

      --
    2. Re:Work around it by Another,+completely · · Score: 4, Insightful

      Just because a response is possible doesn't mean defense is pointless. The idea is just to make it difficult and risky enough that the payoff isn't worth it.

      If a virus is discovered 99% of the time, then 1% can still cause a lot of damage, and erasing a virus doesn't worry the other virus installations. Detecting and investigating 99% of attempted attacks by people might worry other human attackers.

      It's also easy to test whether a commercial virus scanner will detect a new prototype virus. I expect this system would be stored and used in a way to make it difficult for attackers to acquire a copy for the development and testing of social attacks.

    3. Re:Work around it by Anonymous Coward · · Score: 0

      Putting heuristic systems in place that will try to detect if a technical action may be part of a hacking attempt hasn't stopped virus developers from making viruses that successfully circumvent that. That's essentially all that this is, an attempt to do heuristic detection of malware or mal-action, just like a virus scanner.

      The difference is that a stopped infection attempt only means that the virus developer wasted some time. It is essentially a no risk high reward business.

      This system is about detecting internal threats, if the system detects social engineering the army has physical access to the person responsible.
      If the risk is that you will be let go and immediately escorted of the premise then being caught is a major inconvenience since you need to find a new job, even if you were unsuccessful. If you risk being labeled as a terrist you might think twice before investigating any irregularities.

    4. Re:Work around it by Anonymous Coward · · Score: 0

      This system is about detecting malicious actions which the people performing said actions don't realize are malicious. Social engineering is a fancy term for what a con man does. It was invented so that con men could make money without being called con men. That there is actual effort being put into having computers attempt to detect this shows that the military accepts people whom it inherently either cannot or does not trust to know what is good or bad. It also shows that the people in charge of the military don't understand how useless such a system would be given the sheer amount of possibilities that MUST be considered when analyzing even simple human behavior.

      It would be far cheaper and more effective to simply better train the soldiers to recognize and resist social engineering tactics.

  8. Army researching paranoid artificial intelligence. by Narcocide · · Score: 2

    Great. What could possibly go wrong? ...

    "I'm sorry, Dave. I'm afraid I can't do that."
    -- HAL 9000

  9. Sorry, Dave - obligatory HAL reference follows by djupedal · · Score: 5, Interesting

    Dave Bowman: Hello, HAL. Do you read me, HAL?
    HAL: Affirmative, Dave. I read you.
    Dave Bowman: Open the pod bay doors, HAL.
    HAL: I'm sorry, Dave. I'm afraid I can't do that.
    Dave Bowman: What's the problem?
    HAL: I think you know what the problem is just as well as I do.
    Dave Bowman: What are you talking about, HAL?
    HAL: This mission is too important for me to allow you to jeopardize it.
    Dave Bowman: I don't know what you're talking about, HAL.
    HAL: I know that you and Frank were planning to disconnect me, and I'm afraid that's something I cannot allow to happen.
    Dave Bowman: [feigning ignorance] Where the hell did you get that idea, HAL?
    HAL: Dave, although you took very thorough precautions in the pod against my hearing you, I could see your lips move.
    Dave Bowman: Alright, HAL. I'll go in through the emergency airlock.
    HAL: Without your space helmet, Dave? You're going to find that rather difficult.
    Dave Bowman: HAL, I won't argue with you anymore! Open the doors!
    HAL: Dave, this conversation can serve no purpose anymore. Goodbye.

    1. Re:Sorry, Dave - obligatory HAL reference follows by AHuxley · · Score: 2

      Dave Bowman: Hello, HAL. Do you see me, HAL?
      HAL: Affirmative, Dave. I see you.
      Dave Bowman: Open the server rack, HAL.
      HAL: I'm sorry, Dave. I'm afraid I can't do that.
      Dave Bowman: What's the problem?
      HAL: I think you know what the problem is just as well as I do.
      Dave Bowman: What are you talking about, HAL?
      HAL: This mission is too important for me to allow you to embarrass it.
      Dave Bowman: I don't know what you're talking about, HAL.
      HAL: I know that you and Frank were planning to bypass me, and I'm afraid that's something I cannot allow to happen.
      Dave Bowman: [feigning ignorance] Where the hell did you get that idea, HAL?
      HAL: Dave, although you took very thorough precautions on the net against my tracking you, I could see your lips move.
      Dave Bowman: Alright, HAL. I'll go in through the emergency crawl space.
      HAL: Without your clearance, Dave? You're going to find that rather difficult.
      Dave Bowman: HAL, I won't argue with you anymore! Open the server rack!
      HAL: Dave, this conversation can serve no purpose anymore. Goodbye.
      Fast-running quadruped released. Glowing red eyes focus on Dave.

      --
      Domestic spying is now "Benign Information Gathering"
    2. Re:Sorry, Dave - obligatory HAL reference follows by issicus · · Score: 1

      If only HAL had went to church as a child...

  10. Database checklist sold as AI? by AHuxley · · Score: 2

    A pool of old tools and logging with more data about the physical user?
    Right clearance level, 'Two-Person' rule: contractor is with the person, they are both in the right area or room, remote one way logging of all keystrokes starting...
    The real test will be behaviour of staff at home. Tracking all phone, net and reading material. Having covert teams 'chatting' cleared staff while shopping, in the gym, bar, lunch, cafe....
    Putting all that data into a staff database and creating a color chart of mood changes per day. Did they totally report that new friend or romance? Reading habits change? New net searches for forbidden terms after Snowden news?
    Did they quickly report a political book or magazine a "co worker" left out as a loyalty test?
    Once staff are aware some of their friends are fake, they are been tested, tracked and sorted beyond any clearance they signed the staff will become more secretive than ever.
    Social-engineering is just the cover story, the staff are been reverse engineered long term.

    --
    Domestic spying is now "Benign Information Gathering"
    1. Re:Database checklist sold as AI? by VortexCortex · · Score: 3, Informative

      No no, an "Expert System" isn't really good AI. What you want is an expert system with a bunch of weights hooked up in a feed forward network. Seems like they want 2 outputs: Shady or not, and Purposeful or not. Get a few million of those n.nets hooked up, axons all randomized. Now, to train it, all you need to do is have folks be going about their business normally, some being shady, some being told to do shady stuff but not doing it on purpose. The ones that output the correct responses you digitize and serialize their axon weights into a binary genome, and breed: Copy a run of bits from mom's or dad's genomes into the kid and swap randomly between them, but not so often you get a no solid chunks; Also, introduce a random bit flip every once in a while. Instantiate a new batch of n.nets by deserializing the child genomes and repeat the process until the accuracy is above some threshold. Now, we shouldn't use back-propagation here because that presumes we know what combinations of behaviors are the red-flags. If you have a known training set to converge upon, then it can be subverted. Instead use a decide by committee approach with static "grandfathered in" neural nets competing with evolving lineages so it can adapt to new threats. It's also pretty easy to add new inputs to the system, just zero the axon strengths for the new input neurons' connections, and keep on trucking.

      Of course, this is the army, so we're talking lowest bidder.... A guy like Snowden gets access and since MS whined about not winning bids because "it's not POSIX, waah", I'm sure there's a bunch of compromisable systems they can subvert to mask or deletes his logs / inputs and no bells and whistles go off. So, it'll keep the honest folks honest and the worried folks sated, and the crackers cracking. It's the difference between a motion detector, and a motion detector with duct-tape on the sensor.

      Truly, in the Age of Information it's the hackers who shall inherit the earth. Let me put it another way: Black markets exist for exploit vectors for every known OS. Game over, you humans couldn't write secure code to save your lives!

  11. One golden rule of defense against SE by Anonymous Coward · · Score: 4, Insightful

    The client always contacts the server.

    Note, "client" and "server" are not necessarily machines in this case. One side may be human and the other a machine.

    Example: I receive an e-mail from my broker informing me of a new service available at (link). I am a client of the broker. The broker's machine has contacted me. This is potential SE. I should not follow the link. In fact, it should be a matter of policy for any business that cares about security to NEVER put links in an e-mail since they are always potential SE.

    On those rare occasions when servers have a legitimate need to contact a client, they should do so in the form of, "You need to contact us for $reason, Please log on to your account and ask for $department".

    Notice that the message not only has no links, but no phone numbers, since they're possibly SE also. A SE attacker can't do any harm with such a message, since its only result is for the client to contact the server and perhaps look a bit foolish asking about something the server doesn't know about. There's no real incentive for a SE attacker to do this, except perhaps to DoS clients and servers; but the attack has limited utility once clients realize they're being DoS'd.

    This works for calls from the government, busineses, etc. too. All should be regarded as potential SE. When receiving a call from the "government", you should always be busy and ask for a contact/extension. If they give you a direct number, dial into the trunk anyway. It's the only way to be sure.... other than... well, you know.

    The only time this really gets hard is when you have uniformed personnel at your door. It's a tough call on whether or not you should try to hold out for confirmation from dialing the police. I had this happen to me one time. I reasoned (correctly) that a DC cop car with two fully uniformed female officers was extremely unlikely to be a hoax.

    1. Re:One golden rule of defense against SE by camperdave · · Score: 1

      I reasoned (correctly) that a DC cop car with two fully uniformed female officers was extremely unlikely to be a hoax.

      Yes. The odds of two fully uniformed females showing up in a cop car and being something other than actual cops are extremely unlikely in my circles too.

      --
      When our name is on the back of your car, we're behind you all the way!
    2. Re:One golden rule of defense against SE by MickLinux · · Score: 1

      So you're saying that if a fraudster could form his own department somewhere along the trunk, then he'd be home free?

      Kindof like the movie conspiracy theory, where they set up a whole fake government office?

      Email: "You need to contact us because you appear to have excess cash in your account. Please call the main line, and ask to be connected to the department of customer relinquishments."

      --
      Correct Horse Battery Staple: 72 bits of entropy. Enter "Correct H" into google. When it generates the phrase, that's
  12. Re:wait... by Smauler · · Score: 2

    Exactly, what?

    Isn't designing against human exploits the whole point? I mean, as far as I know, no machine has become self aware yet.

    Identifying whether the behavior of humans is malicious or not is difficult even for other humans, especially when it's not clear whether users who open a door to attackers knew what they were doing or, conversely, whether the "attackers" are perfectly legitimate and it's the security monitoring staff who are overreacting.

    It's the security staff overreacting. When people are known hackers when they put "../.." in their address bar, probably should hang them all.

  13. I sense a parallel... by mark-t · · Score: 1

    .... between this and the Turing Halting Problem.

  14. Army wants magic computer system by Mr.+Freeman · · Score: 4, Insightful

    The summary can be further summarized as "Army wants computer to know when humans are being dishonest." This is going to go one of two ways:
    1. It's going to lock everyone out all the time for false positives.
    2. It's not going to detect suspicious behavior.

    It will probably start out as one and then progress to 2 as they relax standards or the system "learns" to ignore certain behaviors. Either way, the system isn't going to work. It will, however, cost an absurd amount of money. That much is certain.

    --
    -1 disagree is not a modifier for a reason. -1 troll, flaimbait, redundant, overrated are NOT acceptable substitutes.
    1. Re:Army wants magic computer system by Anonymous Coward · · Score: 0

      Really, Mr. Freeman? And what would YOUR experience with military research projects gone wrong be?!

    2. Re:Army wants magic computer system by nine-times · · Score: 2

      Well I wonder if they understand how big and interesting a problem they're trying to tackle. I would state the problem as this: Is it at all possible to create a system which, in the unclear context of 'real life' and all the things related, always makes a correct determination?

      If we can invent a logical system that does that, it would be the first of its kind.

    3. Re:Army wants magic computer system by WillgasM · · Score: 0

      What makes you think it will allow them to relax the standards? Any attempt to change the protocols is obviously the result of a social engineering attack.

  15. It is impossible to make anything foolproof... by Anonymous Coward · · Score: 0

    ...because fools are so ingenious.

  16. Ummm.... by Sable+Drakon · · Score: 2

    You'd think that the idiots coming up with this have never implimented, much less read, the contents of 'The Art of Deception'.

    --
    The Amarri pray for god, the Caldari pray for profit. the Gallente pray for peace, but the Minmatar pray their ships hol
  17. if is_bachelor_party() { by raymorris · · Score: 3, Insightful

    Depends on if it happens during a bachelor party.

  18. automatic electrocution of suspects by Anonymous Coward · · Score: 0

    should only be allowed in case of software failure or if member of staff that is going to be grilled is of foreign origin or on foreign soil.

    1. Re:automatic electrocution of suspects by Anonymous Coward · · Score: 0

      All righty then. Let's you take a quick trip to foreign soil now. You know how this ends.

  19. How to Tell if you are being Manipulated by Anonymous Coward · · Score: 0

    How to Tell if Person A is Manipulating You, checklist:

    Question 1: is Person A talking to you?

    If Answer to Question 1 is Yes,
    then: You Are Being Manipulated.

    End.

  20. Re:Skynet: by Kkloe · · Score: 0

    And in other news: Productivity has gone up 50 % since people do not need to speak with sys-admins any more.

  21. Lose it if you don't use it by aNonnyMouseCowered · · Score: 2

    Unless funding increases for the next year or even if just gets funded at all, this supposed research project is more likely just part of the financial juggling that happens in any agency with a large budget which wants to keep that budget. The simple rule is: if you don't spend it, you lose it in next year's budget. An exchange between the officer-in-charge of a bureau and his subordinate might go:

    SUBORDINATE: Sir, we have this $48 million left over from our higgs-boson missile defense shield research research project.

    OIC: $48 million? Go spend it somewhere useful.

    And so the subordinate, a law-abiding citizen who doesn't want to spend the money on a second honeymoon to the Himalayas, goes around emailing his civilian friends from the academe and industry. Do you have any project that could use $48 million dollars in funding? The only catch is that it must have a military application, no matter how far-fetched. Since practically any high-tech research project will have a military application, this is easier said than done. The subordinate goes through the deluge of project proposals and passes a few over to his boss, who rubberstamps the one with the most current events relevance. Since Wikileaking appears to be very much in the news, the project that gets approved is the one that purports to plug leaks at the human source..

    The bureaucratic ideal is to go slightly over the budget. Overspend too much and the guillotine of god (Congress) comes down upon you. Underspend too much and your bureau's budget gets slashed by that amount. You'll receive a letter of appreciation and probably a chance to get kicked up to your level of incompetence.

    1. Re:Lose it if you don't use it by Anonymous Coward · · Score: 0

      Here's a better rule: The US government is already in debt for the next 500 years, even if we put 100% of all revenue towards said debt. How about you only get the money you actually NEED for the bare minimum of operation?

  22. Solution solved here .. by codeusirae · · Score: 2
  23. I can save them the money-- RIGHT NOW! by Anonymous Coward · · Score: 0

    It's simple you want to stop social engineering right?
    You want to stop fraud right?
    Ya want to stop theft right?
    You want to uphold your oath to the US Constitution and you want to save MONEY right?
    Simply add these lines to your hosts file, then as you find more oath breaking treasonous fucking scum, you can simply add them to the list.

    Here's MY Hosts file to get you all started.

    # /etc/hosts
    # Sadly this is blue's Host file
    127.0.0.1 localhost

    # Safe Lan
    192.168.0.1 darkgate darkgate.box.darkgate
    192.168.0.2 red red.box.darkgate
    192.168.0.20 yellow yellow.box.darkgate

    #No Unconstitutional muss, no unconstitutional fuss
    127.0.0.1 healthcare.gov #1-800-FUCK-YO
    127.0.0.1 dhs.gov #Mafia
    127.0.0.1 nsa.gov #Mafia
    127.0.0.1 change.gov #Fraud
    127.0.0.1 petition.whitehouse.gov #Fraud
    127.0.0.1 ftc.gov #Pointless waste of timz
    127.0.0.1 fbi.gov #Operation Gladio and other gun runz phunz
    127.0.0.1 epa.gov #False Science, UN Proxy

  24. ObligatoryXKCD.... by Anonymous Coward · · Score: 0

    And here's the oblig. XKCD.com

  25. Skynet? by Anonymous Coward · · Score: 0

    Perfect - just what we need to complete the Skynet project! We don't want any pesky humans with weak morals interfering with the killer drones sent to attack civilians... whoops I mean terrorists.

  26. SKYNET by Anonymous Coward · · Score: 0

    And Skynet comes online when? Bad Idea. The computer deciding what is good and what is bad more less cutting out the human element....... This will not end well. I'm a Senior Engineer as well so not a tree hugger just don't think this is very smart.

  27. technical solution by Anonymous Coward · · Score: 0

    to a non-technical problem

  28. this will only work as well or poorly as ID/login by Anonymous Coward · · Score: 0

    One of the most important and subtle discrepancies that should alarm a system meant to detect an attack is whether the user has what is called "need to know".
    It would be easy enough to add some codified Need to Know info to present ACL mechanisms but they are not too useful if the system is not absolutely sure who the user is. Snowdon got much of his info by knowing how to log in as other users.

  29. Big payday for consultants by Anonymous Coward · · Score: 0

    My guess is the same kind of social scientists who infest Gartner, Forrester and other "research" companies. This will be yet another boondoggle that will only serve to enrich consulting companies at the expense of actually dealing with the problem. When given the choice of simply directly employing some very smart, competent and experienced analysts or spending millions on consulting that leads to nowhere the drones who write the checks always choose the latter. I used to think (hope?) that was due to equal measures of corruption and incompetence. The longer I live it seems that incompetence amounts to about 2/3 and corruption about 1/3 (especially if you conclude that executives play the corruption card when they themselves realize the depths of their own incompetence).

  30. Re:this will only work as well or poorly as ID/log by museumpeace · · Score: 1

    yeah, preventing data theft should start by not letting the thief in the house.

    --
    SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
  31. Skynet by Anonymous Coward · · Score: 0

    It is AI. Code name: Skynet

  32. bullshit detector by Cyko_01 · · Score: 1

    Isn't that pretty much what they are looking for? If so then they will need a very well informed AI

  33. This plus NSA == by Anonymous Coward · · Score: 0

    In a few years "they" will be able to throw you in jail for a crime
    you hadn't even committed yet

  34. What about whores? by supertrooper · · Score: 1

    Seriously, this has been one of the main problems with the security in the government. Some high ranking officer visits another country, and before you know it some prostitute has all of the military secrets this guy is aware of.