Slashdot Mirror
Machine Learning Expert Michael Jordan On the Delusions of Big Data
First time accepted submitter agent elevator writes In a wide-ranging interview at IEEE Spectrum, Michael I. Jordan skewers a bunch of sacred cows, basically saying that: The overeager adoption of big data is likely to result in catastrophes of analysis comparable to a national epidemic of collapsing bridges. Hardware designers creating chips based on the human brain are engaged in a faith-based undertaking likely to prove a fool's errand; and despite recent claims to the contrary, we are no further along with computer vision than we were with physics when Isaac Newton sat under his apple tree.
... and despite recent claims to the contrary, we are no further along with computer vision than we were with physics when Isaac Newton sat under his apple tree.
That's true, I looked into object recognition for image classification by content. Face recognition is proceeding fairly nicely but doing stuff like just programmatically classifying/tagging images by whether they contain a car, airplane, house, tree, dog, mountain .... without even trying to do things like identifying the type of airplane/dog/car is pretty much undoable in any reasonable amount of time with human level accuracy needed on garden variety PCs and tablets which is the application I'd be interested in. The fastest and most accurate image classifier/tagger is still a human. Am still looking forward to they day that changes but I'm not sure that will be within my lifetime.
Only to idiots, are orders laws.
-- Henning von Tresckow
The interview is slightly more nuanced than that. Prof. Jordan says that he can take off his academic hat and read musings on a common singularity with ordinary human awe and wonder. It is only in his work as an academic that he doesn't feel Kurzweil's ideas are relevant.
I remain sceptical of the singularity idea myself, though for different reasons. When I read Kurzweil's The Singularity is Near , I was disappointed at how in claiming a never-ending increase in the pace of technological advancement, Kurzweil never dealt with the regulatory and consumer factors, and the whole notion of how humans perceive time in general. The wheels of government can only move so fast, and so mankind's access to radical new technology outside the lab (e.g. self-driving cars, new medical tech) must slow down to match the speed of regulatory agencies. Also, consumers can be convinced to buy new shiny things, but there is still a desire to get one's money's worth out of one's purchases, and lots of people still feel their computer or smartphone from three or four years ago is still good enough. Would the market go for replacing one's tech in the shorter and shorter spans that Kurzweil envisions?
So when I read a computer scientist like Jordan admit that he sees no cause for singularity optimism within his work, I can only feel that Kurzweil's dream is a balloon being stuck with a thousand pins. Still, I continue to enjoy thinking about the subject.
I was disappointed at how in claiming a never-ending increase in the pace of technological advancement, Kurzweil never dealt with the regulatory and consumer factors, and the whole notion of how humans perceive time in general. The wheels of government can only move so fast, and so mankind's access to radical new technology outside the lab (e.g. self-driving cars, new medical tech) must slow down to match the speed of regulatory agencies.
You make some good points. However, I believe the march towards the singularity will march inexorably forward for one (highly undesirable) reason: the insatiable appetite of the leaders of nations for power. The populations of those countries will not even be allowed to know much of what is being developed with hundreds of billions of their tax dollars, but technologies that leaders perceive could enhance their ability to dominate the world will be financed. There will be no regulation. If you want to know the state of the art in visual recognition, you should look at military applications: robot soldiers and autonomous drones. For applications of big data (especially its usefulness in widespread blackmailing activities) then, in spite of some initial missteps, look at the pervasive collection of data by the world's "intelligence agencies".
The problem with computer vision is not that it's not useful, but that it's sold as a complete solution comparable to a human.
In reality, it's only used where it doesn't really matter.
OCR - mistakes are corrected by spellcheckers or humans afterwards.
Mail systems - sure, there are postcode errors, but they result in a slight delay, not a catastrophe of the system.
Structure from motion - fair enough, but it's not "accurate" and most of that kind of work isn't to do with CV as much as actual laser measurements etc.
Photo stitching - I'd be hard pushed to see this as more of a toy. It's like a photoshop filter. Sure, it's useful, but we could live without it or do it manually. Probably biggest use in mapping, where it's a time-saver and not much else. It doesn't work miracles.
Number plate recognition - well-defined formats on tuned cameras aimed at the right point, and I guarantee there are still errors. The systems I've been sold in the past claim 95% accuracy at best. Like OCR, if the number plate is read slightly wrongly, there are fallbacks before you issue a fine to someone based on the image.
Face detection is a joke in terms of accuracy. If we're talking about biometric logon, it's still a joke. If we're talking about working out if there's a face in-shot, still a joke. And, again, not put to serious use.
QR scanners - that I'll give you. But it's more to do with old barcode technology that we had 20 years ago, and a very well defined (and very error-correcting) format.
Pick-and-place rarely relies on vision only. There's much better ways of making sure something is aligned that don't come down to CV (and, again, usually involve actually measuring rather than just looking).
I'll give you medical imaging - things like MRI and microscopy are greatly enhanced with CV, and the only industry I know where a friend with a CV doctorate has been hired. Counting luminescent genes / cells is a task easily done by CV. Because, again, accuracy is not key. I can also refer you to my girlfriend who works in this field (not CV) and will show you how many times the most expensive CV-using machine in the hospital can get it catastrophically wrong and hence there's a human to double-check.
CV is, hence, a tool. Used properly, you can save a human time. That's the extent of it. Used improperly, or relied upon to do the work all by itself, it's actually not so good.
I'm sorry to attack your field of study, it's a difficult and complex area as I know myself being a mathematician that adores coding theory (i.e. I can tell you how/why a QR code works even if large portions of the image are broken, or how Voyager is able to keep communicating, despite interference on an unbelievable magnitude).
The problem is that, like AI, practical applications run into tool-time (saving a human having to do a laborious repetitive task, helping that task along, but not able to replace the human in the long run or operate entirely unsupervised). Meanwhile, the headlines are telling us that we've invented "yet-another-human-brain", which are so vastly untrue as to be truly laughable.
What you have is an expertise in image manipulation. That's all CV is. You can manipulate the image to be easier read by a computer which can extract some of the information it's after. How the machine deals with that, or how your manipulations cope with different scenarios, requires either a constrained environment (QR codes, number plates), or constant human manipulation to deal with.
Yet it's sold as something that "thinks" or "sees" (and thus interprets the image) like we do. It's not.
The CV expert I know has code in an ATM-like machine in one of the southern American counties. It recognises dollar bills, and things like that. Useful? Yes. Perfect? No. Intelligent? Far from it. From what I tell, most of the system is things like edge detection (i.e. image manipulation via a matrix, not unlike every Photoshop-compatible filter going back 20 years), derived heuristics and error-margins.
Hence, "computer vision" is really a misnomer, where "Photoshopping an image to make it easier to read" is probably closer.
While its true that the brain is amazingly complicated and malleable, it is not impossible to understand. I work in AI/ML, and have a doctorate in the subject (posting anon from work). There are a few things that give me hope that it can be replicated:
1 - There are many brains which are functionally useful without having human-level intelligence. Example: Dogs can recognize 340 words, perform trained tricks, and identify objects. A robot which has "border collie" level intelligence, train-ability, and independent problem solving with robotic implements can/will be incredibly useful in many applications. Watson has enough connections between neurons to have cat level intelligence, if correctly configured (big if) (http://www.scientificamerican.com/article/graphic-science-ibm-simulates-4-percent-human-brain-all-of-cat-brain/).
2 - The mammalian brain appears to be the first massively successful system. It is not likely that it is the only solution to the problem. It is possible that the machines will think both differently and better than we do.
3 - Taking a giant brain scan and attempting to simulate it is incredibly complicated, as the brain has trillions of connections, most of which likely don't matter for practical purposes (this smell reminds me of home). However, the brain is procedural generated according to a system of rules/hormones, which results in a largely repeating structure (lots of folds). There are likely less than 100 hormones involved, with a more-or-less 1-to-1 mapping between hormones and development. This space is difficult and time consuming to research (tracking all hormones and genetic responses), but far from impossible.
I expect a 'positronic brain' within my lifetime (2086 expected death date). We are 4-5 doubling-factors away from a machine which has the scale of human brain connectivity at a cost of $94-187K. If you believe that technology doubles every 18 months (I don't), then we are about 10 years away.
As someone who was involved in the previous neural network hype cycle (late 80s, early 90s), I'd have to agree with him that we go through these cycles, where a particular approach gain ascendency, then is shown to not work as well as the hype, and then gets rejected. On the inside, however, lots of good work continues to be done. The press (and then in popular opinion) keeps saying 'this is it, we're really close to AI' or somethign similar, and then when it doesn't pan out, then it is considered a bust. But, we are making progress, we know more than we did last year, and a lot more than 10 years ago. It is just that the problem is hard, and we're still trying to figure out some basic principles, so don't expect us to be there yet.
The more people I meet, the better I like my dog.
Yes, this is one of the more shameful examples of the reporter attempting to shove words down the interviewee's mouth, and completely misrepresenting the results.
Take a look at the first sentence:
"The overeager adoption of big data is likely to result in catastrophes of analysis comparable to a national epidemic of collapsing bridges"
Then read the interview. At no point does Jordan indicate that the misanalysis of big data will cause a catastrophe comparable to the epidemic of collapsing bridges. Never. What he does (and apparently the reporter is either too stupid or too dishonest to represent), is provide an analogy between building a bridge without scientific principles and not performing proper statistical analysis on big data.
He never makes a comparison between the outcomes of these two events. He basically says: if you build a bridge without scientific principles, it will fall down. If you are not careful in your analysis of big data, your results will be wrong.
The whole article goes on in a very similar manner. Science reporters used to have something called "journalistic integrity". Here we get a click-bait article where a "reporter" has predetermined a topic that will gain lots of hits and is desperately trying to fit the interviewees words into his agenda.
Shameful.