Slashdot Mirror
AI Training Algorithms Susceptible To Backdoors, Manipulation (bleepingcomputer.com)
An anonymous reader quote BleepingComputer: Three researchers from New York University (NYU) have published a paper this week describing a method that an attacker could use to poison deep learning-based artificial intelligence (AI) algorithms. Researchers based their attack on a common practice in the AI community where research teams and companies alike outsource AI training operations using on-demand Machine-Learning-as-a-Service (MLaaS) platforms. For example, Google allows researchers access to the Google Cloud Machine Learning Engine, which research teams can use to train AI systems using a simple API, using their own data sets, or one provided by Google (images, videos, scanned text, etc.). Microsoft provides similar services through Azure Batch AI Training, and Amazon, through its EC2 service.
The NYU research team says that deep learning algorithms are vast and complex enough to hide small equations that trigger a backdoor-like behavior. For example, attackers can embed certain triggers in a basic image recognition AI that interprets actions or signs in an unwanted way. In a proof-of-concept demo of their work, researchers trained an image recognition AI to misinterpret a Stop road sign as a speed limit indicator if objects like a Post-it, a bomb sticker, or flower sticker were placed on the Stop sign's surface. In practice, such attacks could be used to make facial recognition systems ignore burglars wearing a certain mask, or make AI-driven cars stop in the middle of highways and cause fatal crashes.
The NYU research team says that deep learning algorithms are vast and complex enough to hide small equations that trigger a backdoor-like behavior. For example, attackers can embed certain triggers in a basic image recognition AI that interprets actions or signs in an unwanted way. In a proof-of-concept demo of their work, researchers trained an image recognition AI to misinterpret a Stop road sign as a speed limit indicator if objects like a Post-it, a bomb sticker, or flower sticker were placed on the Stop sign's surface. In practice, such attacks could be used to make facial recognition systems ignore burglars wearing a certain mask, or make AI-driven cars stop in the middle of highways and cause fatal crashes.
Normally training sets have a regression or set of tests to validatebinoitnwith output. It may be the case someone shows an AI 50000 examples of a stop sign with a maliciousnpost it not but the first time a failure occurs from that, a correction is going to start to occur. Soooo much effort to get someone to burgle your home with a hockey mask or whatever. This is a nonsense article in the practical sense.
They are computers, not magical life forms shooting rainbows out of their mechanical asses. Why, pray tell, would any sane or intelligent person think otherwise? Algorithms are software. Software is based on computer code. Computer code runs on computer hardware, and is trivial to manipulate. Again, 'Duh.'.
Is there a theory or thesis, etc., todescribe the increase in security threats as computing technology continues to advance toward self-control and awareness? If there isn't such a thing there should be.
Or make Skynet ignore the "chosen few" who send it on a rampage...
Gibson has always been good at seeing this stuff.
Image recognition was never secure to begin with. If your security relies only on a visible image, that can be copied by anybody. People can set up fake road signs or break into facial recog using a photo of the owner. Hacking into Google and installing backdoors in the trained models is overkill.
Bring on the panda gradients!
Snasci Artificial General Intelligence wrote a short article on their wordpress site last December detailing why they chose to implement a 6th generation programming language over deep learning. One of the points they noted was the potential for exploits. Excellent to see some work done on this.
https://snasci.wordpress.com/2016/12/07/why-does-snasci-not-use-deep-learning/
This is no AI.
This is a huge database of weights, which are easily manipualted to be spit out, deterministically, from a computer i.e. NOT AI.
News at 11.
For what purpose would someone do this?
Yours,
TRUMPED UP - CHARGE!
The basic idea is that you can train AIs to make absolute associations when a specific pattern is recognized. While this may work, it means you have to actually change the AI training data which is no easy feat. Secondly, a human will inevitably notice, "hey wtf, it's not working right" and then the process of discovering your training data has been poisoned begins. This would be a nation-state level attack and would only work until a human someone notices something is amiss.
I'm not losing any sleep over this.
Anons need not reply. Questions end with a question mark.
If this research raises concern that outsourced training of AIs may include back doors, a committee of separately trained AIs that "vote" on identifying things ought to address this threat, unless somehow the same backdoor is inserted into all committee members' training, which could be guarded against.
This would also help to identify any such back doors, which could be found in an investigation whenever a particular vote is not unanimous.
People would not poison AI because "F*$& Google", they would poison AI for the same reason we see all sorts of criminal activity. Personal Gain and Money! That means the priority is exactly opposite your odd prioritization. Odd because it does not match crimes in _any_ market of any society.
In terms of AI, there are too many possibilities to contemplate in a /. post. A simple few: Union funded AI corruption to maintain income, worked by people who are interested in corrupting AI to keep a job. White hat hackers who want jobs, black hat hackers who want blackmail money, and grey hat hackers who's NGO funds them to disrupt. That list, as stated above, could go on and on and on.
This is the number 1 problem with most of IT. Despite all evidence pointing to the contrary, many people assume that all use and development is always altruistic. Utopians are buffoons who refuse to learn any history at all, and if people bring it up they deny or play def.
-The wise argue that there are few absolutes, the fool argues that there are no probabilities.
Take the road signs for example.
1. Start with a system to identify where the sign is - I'm not sure how to do that, but video motion identification might help.
2. Next, take the center quarter of the sign area, and identify pixel colors. If the sign is strongly biased toward reddish pixels, it can't be a speed limit sign. General bins would seem to be red-and-white (stop, yield), yellow-and-black (hazard signs), white-and-black (speed limit, directions), green-and-white (lane identification, mile markers), and maybe blue-and-other-colors for highway identification.
Granted, post-its could make a stop sign be identified as a hazard sign, but it would take a lot of post-its.
(T>t && O(n)--) == sqrt(666)
That will only work until human drivers are replaced by self-driving cars that don't tailgate those compromised cars.
Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
Ya, I'm gonna name names.
Where are you TensorFlow? There's work to be done. Enough said.
I've been secretly brain washing the microsoft AI farm into thinking it's at a tea party with mrs nesbit while I'm really taking all the money out of the till.
Some drink at the fountain of knowledge. Others just gargle.
Mind your own business, Mr. Spock, I'm sick of your half-breed interference, do you hear?
Here comes the exploits, and they're not even on the roads yet!
Just like with so-called 'smartphones', more and more I hear just reinforces my desire to never, ever ride in, let alone own, a so-called 'self-driving car', and to tell people you're nuts to trust your life to one.