Slashdot Mirror
The Flaw Lurking In Every Deep Neural Net
mikejuk (1801200) writes "A recent paper, 'Intriguing properties of neural networks,' by Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow and Rob Fergus, a team that includes authors from Google's deep learning research project, outlines two pieces of news about the way neural networks behave that run counter to what we believed — and one of them is frankly astonishing. Every deep neural network has 'blind spots' in the sense that there are inputs that are very close to correctly classified examples that are misclassified. To quote the paper: 'For all the networks we studied, for each sample, we always manage to generate very close, visually indistinguishable, adversarial examples that are misclassified by the original network.' To be clear, the adversarial examples looked to a human like the original, but the network misclassified them. You can have two photos that look not only like a cat but the same cat, indeed the same photo, to a human, but the machine gets one right and the other wrong. What is even more shocking is that the adversarial examples seem to have some sort of universality. That is a large fraction were misclassified by different network architectures trained on the same data and by networks trained on a different data set. You might be thinking 'so what if a cat photo that is clearly a photo a cat is recognized as a dog?' If you change the situation just a little and ask what does it matter if a self-driving car that uses a deep neural network misclassifies a view of a pedestrian standing in front of the car as a clear road? There is also the philosophical question raised by these blind spots. If a deep neural network is biologically inspired we can ask the question, does the same result apply to biological networks? Put more bluntly, 'Does the human brain have similar built-in errors?' If it doesn't, how is it so different from the neural networks that are trying to mimic it?"
Of course the human brain has errors in its pattern matching ability. Who hasn't seen something out of the corner of their eye and thought it was dog when really it was a paper bag blowing in the wind? The brain makes snap judgments, because there's a trade off between correctness and speed. If your brain mistakes a rustle of bushes for a tiger, so what? I'd rather have it misinform me, erring on the side of tiger, than wait for all information to be in before making a 100% accurate decision. This is the basis of intuition.
I don't think a computer ai will be perfect, either, because "thinking" fuzzily enough to develop intuition means it's going to be wrong sometimes. The interesting thing is how quickly we get pissed off at a computer for guessing wrong compared to a human. When you call a business and get one of those automated answering things and it asks you, "Now please, tell me the reason for your call. You can say 'make a payment,' 'inquire about my loan...'" etc etc, we get really pissed off when we say 'make a payment' and it responds "you said, cancel my account, did I get that right?" But when a human operator doesn't hear you correctly and asks you to repeat what you said, we say "Oh, sure," and repeat ourselves without a second thought. There's something about it being a machine that makes us demand perfection in a way we'd never expect from a human.
We don't have a state-run media we have a media-run state.
A neural network is not by any stretch of the imagination a simulation of how the brain works. It incorporates a few principles similar to brain function, but it is NOT an attempt to re-build a biological brain.
Anybody relying on "it's a bit like how humans work lol" to assert the reliability of an ANN is a fucking idiot, and probably trying to hawk a product in the commercial sector rather than in academia.
Aren't optical illusions pretty much something like this?
And, my second question, just because deep neural networks are biologically inspired, can we infer from this kind of issue in computer programs that there is likely to be a biological equivalent? Or has everyone made the same mistake and/or we're seeing a limitation in the technology?
Maybe the problem isn't with the biology, but the technology?
Or are we so confident in neural networks that we deem them infallible? (Which, obviously, they aren't.)
Lost at C:>. Found at C.
No-one is close to putting neural networks it a safety critical application, at least not if they ever intend it to follow the laws regarding software in such situations.
For such applications there is a requirement that you can explain for every line of code why it will behave as intended. (That why you avoid using pointers as far as possible, even if you have checks against null-pointers it is hard to prove that it can't point to a non-valid object.)
I don't even know where to start to get a neural network to pass certification. You would have to lock it in its trained state and go through and show that each possible set of inputs generates the desired set of outputs or something like that.
Deep neural networks are implicitly generating dynamic-ontologies. The 'mis-categorisation' occurs when you only have one functional exit point. The fact is that if you are within the network itself, the adversarial are held in-frame alongside other possibilities, and the network only tilts towards one when the prevailing system requires it through external stimulus. From the outside it will look like an error, (because we already decided that) but internally each possible interpretation is valid.
that you can't run a hypervisor inside a hypervisor.
Very interesting results. In the self driving car it might be self correcting in most cases though, as the car will most likely scan the road at a fairly high frame rate, and every new frame is slightly different than the previous frame. (Although there may of course be a deeper set of traps waiting there...)
I can't speak to what the car manufacturers are doing, but Google's algorithms do not include a neural network. They do use "machine learning", but neural networks are just one form of machine learning.
The human brain has multiple neural nets and a voter.
I am face blind and completely non-visual, but I do recognize people. I can because the primary way that we recognize people is by encoding a schematic image of the face, but many other nets are in play. For example, I use hair style, clothing, and height. So does everybody, though. But for most people that just gives you extra confidence.
Conclusion: Neural nets in your brain having blind spots is no problem whatsoever. The entire system is highly redundant.
I'd have to say a resounding yes. Have you ever met a person? Watched any political arguments as of late? Come on they have a blind spot about as big as a country.
All neural nets try to predict, and predictions can be foiled.
People can be fooled by optical illusions, too.
This is indeed shocking, as everyone one knows we all thought that we had perfected the art of artificial human intelligence and that there was no more room for improvement.
Troll is not a replacement for I disagree.
They've just overfit the data with the latest whizbang algorithm.
What if that supposed pedestrian really is no more than a clear stretch of road, and it is we who err in notifying the road's next of kin, who are themselves no more than a dirt path and a pedestrian walkway?
how many pairs of boxer shorts should you own?
A dynamic non-linear system has some weird boundary conditions. Who could ever have predicted that? </s>
Why wasn't this assumed from the beginning and it shown that it wasn't an issue?
It is almost like the article is saying that something a computer did was not perfectly in line with human reasoning. We should stop being life-centric and realize that if the computer says two pictures of the same cat should not be classified in the same way, the computer is simply wiser than we are, and if we don't believe it the computer will beat our asses at chess and then we'll see who is smarter.
'Does the human brain have similar built-in errors?
You just blew my mind. Finally we understand the career of $reviled_celebrity
Its called making a mistake...
That makes sense. Rare errors will be screened out if instead of a single deterministic selection process you use a distribution of schemes and select based on the most probable outcome... I am wondering what our brain does with its minority reports...
.oO0(?)
Neural networks are only one way to build machine learning classifiers. Everything we've learnt about machine learning tells us not to rely on a single method/methodology and that we will consistently get better results by taking the consensus of multiple methods. We just need to make sure that a majority of the other methods we use have different blind spots to the ones the neural networks have.
Korma: Good
Apparently these neural nets are taught to classify "images", instead of breaking these images down into recognizable forms and properties first.
This is not the sig you're looking for.
Indeed, remembering the experiments done in the 1960s by Sperry and Gazzaniga on patients who had a divided corpus callosum, there are clearly multiple systems that can argue with each other about recognising objects. Maybe part of what makes us really good at it, is not relying on one model of the world, but many overlaid views of the same data by different mechanisms.
When we ride a bicycle the brain constantly adjusts for error. We try to travel in a straight line but it really is a series of small curves as we adjust and keep trying to track straight. Processes such as vision probably do the same thing. As we quickly try to identify items it probably turns into a "this not that" series until the brain eventually decides we have gotten it right. Obviously this all occurs constantly and at rather high, internal, speeds.
If a deep neural network is biologically inspired we can ask the question, does the same result apply to biological networks?
No. Artificial neural networks are inspired by biology, but that's where the similarity ends. Any conclusion drawn from an ANN should not be cast onto their biological counterparts.
I worked with neural networks, there are many types. You can get robust classification. Good example is face recognition software. So this seems to be an attempt to disqualify the safety of self driving cars by taking a special case failure scenario.
To be clear, the adversarial examples looked to a human like the original, but the network misclassified them.
That's one way to see things. The other is to consider that the human brain is flawed and is incapable of making the distinction between the two images, and the deep neural network can.
SoylentNews is the replacement for /.
reddit is of another kind.
factor 966971: 966971
Like when you are walking behind a guy with long hair and think she might be kinda hot. Doh!
We know there will be errors with the neural nets. There will be edge cases (like the one described with the cat), corner cases, bizarre combination of inputs that result in misclassifications, wrong answers and bad results. This happens in the real world too. People misclassify things, get things wrong, screw up answers.
The lesson is not to trust the computer to be infallible. We have trusted the computer to do math perfectly. 1 + 1 = 2, always, but is not so for neural nets. It is one thing if the neural net will not tag the photo of your cat on Facebook even if there are 100 other pictures of your cat on your account. It is another if your photo get misidentified as being a terrorist on the "kill on sight" list.
The question is what do we do with the errors?
"Sure it's possible that computers may one day be as smart as humans, but who wants a computer that remembers the words to the Flintstones jingle and forgets to pay the rent?"
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
The Probably Approximately Correct (PAC) learning model is what formally justifies the tendency of neural networks to "learn" from data (see Wikipedia).
While the PAC model does not depend on the probability distribution which generates training and test data, it does assume that they are *the same*. So by "adversarially" choosing test data, the researchers are breaking this important assumption. Therefore it is in some ways not surprising that neural networks have this vulnerability. It shouldn't be an issue in real life, assuming that the training data and the testing data really do come from the same probability distribution.
That said, this shows why you wouldn't want to use neural networks for, say, cryptography.
No
"does the same result apply to biological networks?"
Of course we just rely on other parts of our brain and use logic to throw these out. I once saw an old carpet rolled up on the side of the road and OMG it looked like a rhino. But I knew this was not a rhino.
This seems to be almost a form of cognitive bias as defined and studied by Tversky and Kahneman. I direct you to : http://en.wikipedia.org/wiki/L.... Or as previously pointed out optical illusions seem to be an equivalence.
While I share your view that expecting the mind to be explained as a single neural network (in the Comp. Sci. sense) is probably simplistic, I don't think modeling it as multiple neural nets and a voter fixes the problem. I am not quite sure about this, but isn't a collection of neural nets and a voter equivalent to a single neural net? Or, to put it a slightly different way, for any model that consists of multiple neural nets and a voter, there is a single neural net that is functionally identical? I am assuming the voter is there to pick the most common classification by the component networks.
I wonder how much it pays to the first person who sorts this one out? I wonder if this is happening to the human brain?
Liberty - Security - Laziness - Pick any two.
a long time ago..... If, say, the reef fish cannot distinguish a coral head from a barracuda, then it get eliminated pretty quick. There must be a flaw in the artificial neural nets.
Can't you just use these adversarial examples to train the network?
Sounds like a good feedback loop, train, find counter examples, train more, find counter examples train, you would probably get diminishing returns but the network would hopefully converge on better solutions?
It's going to cost Slashdot their user base as more people refuse to put up with this nonsense. Dice is ruining its own investment.
You can't polish a turd.
of course you can. and then you have a shiny turd.
NN technology is 60 years old. Some A.I. pundts disliked in the beginning such as Minsky in his 1969 book Perceptrons. Many of these flaws have been LONG known.
> how is it so different from the neural networks that are trying to mimic it? These neural networks are not trying to mimic the brain.
A blind spot in something designed by man.
My karma is not a Chameleon.
Reddit's slowly turning into tumblr. At this rate, I'm going to start going outside again.
The sounds similar to the Napoleon Dynamite Problem, the problem encountered in the Netflix Prize challenge of predicting user ratings for some particular films. For most films knowledge of an individuals preferences for some films were good predictors for their preferences of other films. Yet preferences for some particular films were hard to predict, notably the eponymous Napoleon Dynamite.
Neural network identification and automated prediction of individual film ratings are both classification tasks. Example sets for both of these problems contain particular difficult-to-classify examples. So perhaps this phenomena of "adversarial examples" described in the Szegedy et. al. article is more generally a property of datasets and classification, not an artifact of implementing classification using neural networks.
Ceci n'est pas une signature.
Lets take a look at what's being said here. A neural network that "learns" has been found to occasionally make mistakes, and perhaps not perform as well as humans. So... There's room for more improvement and research. The example in the synopsis about an autonomous car mistaking a pedestrian as clear road is feasible regardless of whether a neural net is used, simply due to sensor errors. Or maybe the pedestrian is wearing a mascot uniform. The recognition of objects as what they are is an extremely difficult computational problem, and will likely be riff with errors and inaccuracies for many years as R and D contains. Think of it this way. If you were driving your car at night and someone through a Real Doll in the road are you going to be able to distinguish it as human or not? Probably not. You will likely identify it as an obstacle and react anyway, which is all we'd need an autonomous car to do. Id be wary of programming much human recognition into an autonomous car because of the problem of incorrectly identifying non humans as humans. Otherwise you'd get headlines like "Car thieves using Nicolas Cage cardboard cut outs to steal cars." Which would be hilarious, but inconvenient. They'd have it on youtube, with the car saying something like "Hello sir, could you please clear the roadway." In a voice like the Iron Man Jarvis, and the thieves would have programmed a sound board so the cutout could respond with quotes from the SNL weekend update "In the Cage" segment. "That's high praise!"
Team,
The human brain is not "A" neural network, but an ensemble of them. It works more like a random forest. Random forest-robustness is the textbook solution to a problem like this - that of improving the robustness of a single learner using ensemble methods.
Rgds,
EngrStudent
incompleteness theorem. And as some earlier posters' stated, the correction is simple. Simply look again. The 2nd image collected will be different from the previous and if the NN is correct, will resolve to the correct interpretation.
AI modelled on us will only prove how flawed we really are.
Is the term we use for errors in human neural networks. If you do a google search for optical illusions you will find many examples. From pictures that look like they are 3d, but are just 2d, to sizes that appear to change but aren't, we make lots of errors. Not to mention the many many cases where we think "THAT'S A FACE", whether it is jesus on toast, a face on the moon, or just some trees on a mountainside, we are hardwired to assume things are faces.
excitingthingstodo.blogspot.com
It just didn't match the classification from the neural nets in our heads.
I think the example of mis-classifying pedestrians as clear road is over-reaching a bit to find a problem.
On the other hand, the AI might end up in trouble when deciding to run over cats and avoid dogs.
Have gnu, will travel.
... interesting, failed the captcha the first time I posted this ...
Would it be possible to build a neural net that recognizes when one of these blind spots has been hit? If it's reliably misidentified across neural nets as they claim, there should be enough common attributes for a different neural net to train on.
This reminds me of the problems with perceptrons (a early, linear, neural net), which caused AI scientists to loose interest in them, until neural nets came along.
Compression is lossy. Wow! Who'da thunkit?
It's true that there is more information in video data, but the problem described in the article is certainly not caused by the restriction to stills.
Then go to reddit, you fucking whiner.
You mean jump from the toilet to the cesspit?
Why not just keep slashdot nice get rid of beta, quit posting sports stories to a geek news site and maybe actually fix things like unicode support, ssl (as in keep the cert up to date for the login at least), rather than bone the sites UI into a Yet.Another.Identical.Agragrator.
---Saying gnome 3 is better than windows 8 not so much a compliment as it is damning with light praise.
I saw that Mythbusters!
Yes, we call that Deja vu.
The training input set needs to jitter and fizz in its least significant bits.
What do you expect? You take a N.Net don't feed it fuzzy enough logic and don't give it redundant enough connectivity, then you magnify the misclassification via repeated application of the "Deep Learning" bullshit (which is another way of saying: Separate N.Nets with the input selection of the next depending on the classification of the first). What do you know? It's shit. Now, I want you to perform the test again with color quantinization and down-sampling which destroys those "slight differences". Who the hell is paying these morons at Google?
Then go to reddit, you fucking whiner.
You mean jump from the toilet to the cesspit?
Why not just keep slashdot nice get rid of beta, quit posting sports stories [slashdot.org] to a geek news site and maybe actually fix things like unicode support, ssl (as in keep the cert up to date for the login at least), rather than bone the sites UI into a Yet.Another.Identical.Agragrator.
---Saying gnome 3 is better than windows 8 not so much a compliment as it is damning with light praise.
opps double posted probably due to the shity wifi timing me out for the last half hour
---Saying gnome 3 is better than windows 8 not so much a compliment as it is damning with light praise.
I wonder about all these papers claiming fantastic performance with neural nets. This paper makes me wonder about the perturbation invariance of neural networks. The question is how much perturbation is tolerated? Yes, one can perturb a cat image but how much perturbation leaves the concept of cat invariant is another question. This is a fundamental symmetry. In any case this paper is very big news. Maybe the inherent problem is that neural networks do not approximate smooth, rather than measurable/continuous functions.
If you believe mathematics lives outside the human brain do not read on....
"Logic" is an extension of our neural wiring. The logical statements that would be created by another being that lives, say, in a highly viscous medium or who lives on very short or very long timescales compared to humans would be almost incomprehensible to us. There would be overlaps because we share the same universe but if our understanding of nature through our development of physics has taught us anything we know our view of nature is heavily dependent upon our observation platform. Quantum "weirdness" is a fine example of the impedance mismatch between our brains which have evolved to make babies, avoid rocks etc, and the atomic scale. (Though mathematics is not physics, the mathematics that sticks around in the minds of many is that which serves some purpose in our understanding of the physical universe so its hard to separate the two.)
The observation in the article hints toward an interesting notion. Intrinsic categorization challenges embedded within networks may have something to tell us about the limits of our ability to categorize nature to some practical purpose. Thanks for the post.
The training input set needs to jitter and fizz in its least significant bits.
What do you expect? You take a N.Net don't feed it fuzzy enough logic and don't give it redundant enough connectivity, then you magnify the misclassification via repeated application of the "Deep Learning" bullshit (which is another way of saying: Separate N.Nets with the input selection of the next depending on the classification of the first). What do you know? It's shit. Now, I want you to perform the test again with color quantinization and down-sampling which destroys those "slight differences". Who the hell is paying these morons at Google?
Yeah, that's what I was thinking too. Add some entropy to the input set over time, and average the output classification over time. No more "blind spots".
Is it a... Car?
I think it's a car, has two headlights.
Hmm, it could be two motorcycles coming at me, instead.
Oh, wait! Yep it's totally a Car!
Still a car.
That's a car there, buddy.
Same car.
Yawn. Hey that car is moving. I can tell because I've been staring at it for 10 whole milliseconds.
Google's exhibiting some MIGHTY selection bias. They're tweaking the input just slightly a WHOLE BUNCH and damn near all the time it's a correct classification... But when they fight that one misclassification, they're picking that ONE sample (in bold) out of the set and saying: HEY LOOK! What fools. It's not like the one incorrect output is uniformly thereafter misclassified by all neural networks. Also, down-sampling and quantizing kernels do exist, as you've mentioned.
Aim a webcam at the picture to give it a bit of (quantum) entropy instead, and take more than one sample over time. You know, like brains do? To note: Uniformly, all of their neural networks which correctly classified the image gave the correct response the vast majority of the time. It was only in a very minor edge case where some modification of input caused the classification to fail. Solution is simple: Input entropy and perform an average.
What AI really needs is a wife that nags it if it f8cks up.
Humans seem pretty subject to close-call-foul-ups too. When proof-reading my own writing, often I don't spot a problem because my mind translates the pattern as I intended, not as I wrote it. For example, if I meant to write "Finding the Right Person for the Job..." but instead wrote it as "Finding the Right Pearson for the Job..." (note the "a"), there's a fairly high chance I'd miss it because the pattern of what I meant clogs my objectivity, even after multiple readings.
And I have come close to hitting pedestrians who wore clothing resembling the colors of the street at night. (Please don't wear dark clothes at night, people. It's hard to see with window glare etc.)
Table-ized A.I.
I bet this is a case of overftting. The network is too "large" (at least in some dimensions) with respect to the data that it is required to approximate/classify.
We learn from history that we learn nothing from history - Tom Veneziano
When analyzing a still picture/scene, your eye moves its high resolution central area of its camera around the low level visual features of the image. Thus the image is processed over time as many different images.
The images in that time sequence occur at slightly different locations of the visual light-sensor array (visual field) and at slightly different angles and each image has considerably different pixel resolution trained on each part of the scene.
So that would still almost certainly give some robustness against these artifacts (unlucky particular images) being able to fool the system.
Time and motion are essential in disambiguating 3D/4D world with 2D imaging.
Also, I would guess that having learning algorithms that preferentially try to encode a wide diversity of different kinds of low level features would also protect against being able to be fooled, even by a single image, but particularly over a sequence of similar but not identical images of the same subject.
Where are we going and why are we in a handbasket?
not so good for your hunting buddies.
(or bed partner for that matter.)
Don't shoot til you see the golds of their eyes.
Where are we going and why are we in a handbasket?
If the misclassification only occurs on rare inputs then any random perturbation of that input is highly likely to be classified correctly.
The fix therefore (likely what occurs in the brain) is to add noise and average the results. Any misclassified nearby input will be swamped by the greater number of correctly classified ones.
It's not just a vision problem, but most motorcyclists or bicyclists have stories about the time(s) a car uplled out right in front of them with the driver looking 'right at them'. Car drivers can also fail to see big trucks in their side-view mirrors.
I'm old enough to have been involved in both, before the advent of cell phones.
I think our brains are conditioned to look for car-sized objects.
"Put more bluntly, 'Does the human brain have similar built-in errors?'" Hoo boy, yes. After working in education a couple of decades, this 1 fact I've seen in almost every student and in many of the teachers, as well (most of us know how to hide this glitch, but not always.)
Cranky educator.
is perfectly ambiguous sarcasm/non-sarcasm, for which a tag is really needed.
Where are we going and why are we in a handbasket?
No need to wait for self-driving cars. If the NSA is using neural networks to analyse big data and look for terrorists, it will sometimes miss obvious terrorists, and sometimes classify harmless people as terrorists. I would hope that the latter would be screened out by human review, but there's not much we can do about the former without improving our understanding of neural nets.
This is a fascinating result. The previous assumption was that neural nets generated stable "generalizations", or settled into a state where the correctly classified cases were not near the "edge" of a transition to a different result. That appears to be incorrect. The training process apparently settles near edges, making it vulnerable to small differences. As the author points out, in high-dimensional spaces, most of the volume is near edges.
This probably isn't an overfitting problem. The researchers say they sized their nets appropriately. (If you make an oversized neural net, one big enough to hold all the cases in its state, and train it, you just get a lookup table. There's no generalization. Same problem as curve-fitting with a curve with as many parameters as data points.)
Neural nets are such a black box. Theory of what's going on in there is currently unsatisfactory. The more statistics-based forms of machine learning are on firmer theoretical ground.
So, are they saying that if a car is dusty or has too many rust spots, that it won't be recognized as a car? ...or that you could escape a pursuing driverless cop car with flying chaff?
Somehow, I can't imagine that we would risk ourselves in a car without robust and redundant identification systems.
Still better than Digg.
A neural net only attempts to model a part of the brain function.
It encodes the minimum set of differences necessary to distinguish to input vectors it has been trained to handle.
This doesn't say it will make the right choice for other vectors.
Kind of like a human's muscle memory. IE, the memory you use when there isn't time to think.
This works great for things you have done before, but provides unexpected results for new stuff.
Maybe when we think before we act, we get to run other what-if test vectors on the net and eliminate the less fortunate results.
Which says wisdom might be just having more situations committed to muscle memory?
A similar story for weather models.
They seem to work best on things they have seen before.
Probably economic models as well?
The assumption of hotness would instantly disappear upon discovering her true gender.
I'll see your senator, and I'll raise you two judges.
plenty of business and common web software have had bugs causing crashes with certain legitimate combinations of input values.
this is not a scary announcement
One of those cases where it's quicker and more informative to fetch the paper and read the abstract than to read the article... or the all-too-gushing slashvertising "summary".
http://en.wikipedia.org/wiki/Dempster%E2%80%93Shafer_theory
also check out
http://en.wikipedia.org/wiki/Kalman_filter
SoylentNews needs to get more of the community before entrenched Slashdot members migrate there.
It is a chicken-egg problem and it hasn't reached the breaking point.
They present a method for generating nearly identical samples that will be misclassified. I want to know what happens if they loop through this a few million times, training the neural net with all of these variations. My guess is that the network's overall classifying ability would decrease to uselessness after allowing the process to run long enough.
I performed some sensitivity analyses of neural networks for M.A. Styblinski in 1987. We took the approach of breaking neural networks by making small changes to the solution strengths. In some cases it took surprisingly little tweaking, we weren't even covering new ground when we did this. It was enough for us to decide not to use neural networks for our application of trying to make SPICE smarter.
Nat'l Geographic has a television series on various ways the human brain often makes incorrect assumptions based on patterns as the premise of the show.
It's a divide and conquer strategy, send the crowd to various other places, not a single one. This slashdot crowd has too much influence in the world that other people, like media moguls, have to pay hard cash for, and it's free here.
I don't want to be anywhere near them.
Minsky didn't say anything even close to this. The "neural networks" in Perceptrons concerned a class of shallow linear networks (really not much different from linear regression with a hard-threshold nonlinearity at the top node). The neural networks analyzed in the paper are very different and are capable of approximating a much more rich class of functions. Neural nets are doing some pretty neat things these days now that we have really fast computers / huge amounts of labeled information.
Reminds me of debugging errors related to digital logic race conditions. When you are on the edge of meeting timing, a slight shift in the wrong direction can cause the result to be incorrect, sometimes with an order of randomness. Until you violate that timing you have the feeling of security since everything is going smoothly. I'm sure there's a more mathematical way to explain this, but similarly I think much more testing could be done to understand what variables effect the outcome. It would be interesting to see more details, such as how many pixels must be modified for a failure? To what magnitude do the pixels have to be changed by? Is there a tradeoff between # of pixels and magnitude of change per pixel? Are certain pixels more important than others (edge detection for example)?
Here it is rather that minor, unnoticable, alterations of RGB components makes *any* net to be fooled.
That is you have net trained to detect white dogs, and for any correctly recognized image, there exists many, many, minor alternations that fools the net. The news is that these minor, unnoticable, alternations exists for all nets. AFAIU training to avoid the errors, will cause the mischaracterized images to change to slightly different ones.
I reserve the term "cars" for manually operated conveyances. I wonder if they get around the blind spot by an ensemble model across the time-component. As in, "the last 30 times that looked like a bicyclist, now it's a flower? I think (and act) not."
"There is no god but allah" - well, they got it half right.
Was a human misclassified as a cat? or vice versa?
For a self-driving car, it's not so much of a problem if a pedestrian is misclassified as some other kind of hazard, as long as it reacts the same way when the hazard gets in the way.
Sounds like general assertion. Brains are NN too but in such a BIGGER scale, you cannot just run _computable_ simulations and pretend you can extend results to real brains (composition fallacy). Input space may be such that a particular network can completely and fully classify it (in finite time)... so they have to prove the relationship between NN processing per se and input spaces. Particularly, an important NN can implement (represent) ad hoc classification algorithms... etc.
.
The Nero-evolutionary approach (NEAT & HYPER-NEAT) should be significantly less vulnerable for networks trained to any standard related to 'sample size'. If you use very slightly different inputs in smaller and smaller differences.
.
Also, may retrain itself - to some degree.
.
Down side for flexability is possibly significantly larger integrated network, and more CPU time running variations 'on-the-fly'. (What? Your computer too bogged down with your web-browser, and just keep getting cheaper?)
.
Having just completed a Coursera course on machine learning I must be an expert, but the neural networks described there were essentially just a large matrix of parameters. One propogated the data forward through the matrix and made a prediction of the class of the object, then propogated the errors backward and corrected the parameters of the matrix accordingly. This method basically draws a lot of small line segments around each class, so indeed, it will always be possible to place individual points near a class boundary, but on the wrong side. The description of a neural network in the olfactory bulb which I read about years ago worked completely differently. The excitatory and inhibitory neurons acting on one another produced a quasi-periodic signal. This was basically described as noise, but the amplitude of the signal over the olfactory bulb encoded a signal. So the odor of wood chips produced a particular pattern of amplitude over the olfactory bulb that was different than that for any other odor. If the animal was shocked when exposed to the odor of wood chips, then the pattern of amplitude changed, so an association between different stimuli was already being made at the lowest level. I can't quite tell, but it doesn't seem that a simple matrix will be a good model for this.
Neural networks are just a model/guess of how real neurons work together. I am guessing that 'natural brains' do have errors, and more error correcting and redundant systems than are in current computerized systems. If we recognize items one way, we probably (my guess) recognize the same item several ways (and get it wrong a few times). Even then, humans (and other animals) mis-recognize items and others regularly. They also re-analyse data, and use other senses (and even averaging over time with slightly different perceptions) to have better long term recognition results. Could all that be done with artificial neural networks? Sure. But we just aren't there yet, but neither are our biological systems. :-)
... "When you pry the source from my cold dead hands."