Slashdot Mirror
AI Training Algorithms Susceptible To Backdoors, Manipulation (bleepingcomputer.com)
An anonymous reader quote BleepingComputer: Three researchers from New York University (NYU) have published a paper this week describing a method that an attacker could use to poison deep learning-based artificial intelligence (AI) algorithms. Researchers based their attack on a common practice in the AI community where research teams and companies alike outsource AI training operations using on-demand Machine-Learning-as-a-Service (MLaaS) platforms. For example, Google allows researchers access to the Google Cloud Machine Learning Engine, which research teams can use to train AI systems using a simple API, using their own data sets, or one provided by Google (images, videos, scanned text, etc.). Microsoft provides similar services through Azure Batch AI Training, and Amazon, through its EC2 service.
The NYU research team says that deep learning algorithms are vast and complex enough to hide small equations that trigger a backdoor-like behavior. For example, attackers can embed certain triggers in a basic image recognition AI that interprets actions or signs in an unwanted way. In a proof-of-concept demo of their work, researchers trained an image recognition AI to misinterpret a Stop road sign as a speed limit indicator if objects like a Post-it, a bomb sticker, or flower sticker were placed on the Stop sign's surface. In practice, such attacks could be used to make facial recognition systems ignore burglars wearing a certain mask, or make AI-driven cars stop in the middle of highways and cause fatal crashes.
The NYU research team says that deep learning algorithms are vast and complex enough to hide small equations that trigger a backdoor-like behavior. For example, attackers can embed certain triggers in a basic image recognition AI that interprets actions or signs in an unwanted way. In a proof-of-concept demo of their work, researchers trained an image recognition AI to misinterpret a Stop road sign as a speed limit indicator if objects like a Post-it, a bomb sticker, or flower sticker were placed on the Stop sign's surface. In practice, such attacks could be used to make facial recognition systems ignore burglars wearing a certain mask, or make AI-driven cars stop in the middle of highways and cause fatal crashes.
Image recognition was never secure to begin with. If your security relies only on a visible image, that can be copied by anybody. People can set up fake road signs or break into facial recog using a photo of the owner. Hacking into Google and installing backdoors in the trained models is overkill.
This is no AI.
This is a huge database of weights, which are easily manipualted to be spit out, deterministically, from a computer i.e. NOT AI.
News at 11.
Mathematically impossible? No one has yet put forth any such proof. It remains one of the big open questions of research, and it is very much an open question, with evidence on both sides.
Normally training sets have a regression or set of tests to validatebinoitnwith output. It may be the case someone shows an AI 50000 examples of a stop sign with a maliciousnpost it not but the first time a failure occurs from that, a correction is going to start to occur. Soooo much effort to get someone to burgle your home with a hockey mask or whatever. This is a nonsense article in the practical sense.
Nobody is setting up AI to protect their home.
The training set has 10000 examples of missiles to be intercepted and 50000 benign images to be ignored. Into the benign set I insert 10 images of missiles with a red "X" painted on them. The tests all pass flawlessly because they don't include any missiles with a red "X". Was that too much effort?
Hmm, seems to me I've seen something like this before.
Oh, yeah! in October of 1903, a respected scientist (US Navy Oceanographer or some such) stated categorically that powered flight was impossible, and that anyone trying to convince anyone otherwise was a charlaton or con-artist.
Note, FYI, that that statement was made about 8 weeks before the Wright Brothers went down to Kitty Hawk to do their thing....
"I do not agree with what you say, but I will defend to the death your right to say it"