Slashdot Mirror
AI Software Juggles Probabilities To Learn From Less Data (technologyreview.com)
moon_unit2 quotes a report from MIT Technology Review: You can, for instance, train a deep-learning algorithm to recognize a cat with a cat-fancier's level of expertise, but you'll need to feed it tens or even hundreds of thousands of images of felines, capturing a huge amount of variation in size, shape, texture, lighting, and orientation. It would be lot more efficient if, a bit like a person, an algorithm could develop an idea about what makes a cat a cat from fewer examples. A Boston-based startup called Gamalon has developed technology that lets computers do this in some situations, and it is releasing two products Tuesday based on the approach. Gamalon uses a technique that it calls Bayesian program synthesis to build algorithms capable of learning from fewer examples. Bayesian probability, named after the 18th century mathematician Thomas Bayes, provides a mathematical framework for refining predictions about the world based on experience. Gamalon's system uses probabilistic programming -- or code that deals in probabilities rather than specific variables -- to build a predictive model that explains a particular data set. From just a few examples, a probabilistic program can determine, for instance, that it's highly probable that cats have ears, whiskers, and tails. As further examples are provided, the code behind the model is rewritten, and the probabilities tweaked. This provides an efficient way to learn the salient knowledge from the data.
In terms of animal models (that we're sadly still not sophisticated enough to understand), I find dogs' ability to identify other animals interesting.
My dog can tell on sight whether another animal is a dog or not. This is remarkable because dog vision is actually slightly worse than human vision, he can do it from upwind, and there is a LOT of variation in dog breeds.
Perhaps he's just seeing 'animal on a leash held by a human', but there does seem to be a slight pause of observation before he decides whether or not to bark, and a lot of owners in my area don't have any respect for leash laws.
Bayesian probability is named for the Bayes' Theorem (which is named after the namesake mathematician). But the Bayesian inference is called that because it relies specifically on applying Bayes' Theorem rather than any other of Thomas Bayes' work.
Any guest worker system is indistinguishable from indentured servitude.
"...code that deals in probabilities rather than specific variables..."
We need some of your 1990s fuzzy logic hype over here!
Mostly random stuff.
The comparison of "deep learning that needs tons of examples" vs "Bayesian programming that can learn from a few examples" is a false dichotomy. It all depends on how much structure you assume a priori versus how much structure you learn from the data.
Typical neural net (deep learning) examples start with no structure and thus require lots of examples. Typical Bayesian net examples start with a lot of structure and thus require only a few examples.
On the other hand, if you start with a highly pretrained net like Inception-v3, then your deep learning cat expert may not need as many examples to generalize. And if your Bayesian programming model starts out with very general, very simple "building blocks" then it may need a lot of examples to extract the predictable structure.
A main difference is that if you want to start with a lot of structure built in, you will probably have to pretrain for the neural net, whereas you can "hand code" the knowledge in your Bayes net. And the structure in the Bayes net may be a lot more transparent and easily interpretable than in the neural net. On the other hand, you had better hope you picked the right structure to begin with or else you will be reasoning over possibilities that are all very wrong! Knowing that an image is 50 times more likely to be a cat than a dog is not very helpful if it is actually a penguin.
I have a machine learning system which can make high level decisions based on zero data. It has named itself Deep Trump.
> Dogs cannot generalize at all. They do not know if another animal is a dog, cat, zebra, see-saw, ball or anything else. To a dog, every dog, cat, human, car, squirrel is different.
It could theoretically be possible that dogs don't put other creatures in categories, that each individual is wholly distinct, not part of a group.
> which is why so many people think their dog is "racist". A dog raised by a black man will tend to bark at more white people than black people and vice versa.
This could also be possible, that dogs DO think in terms of categories, such as "black people" and "white people", rather than treating each individual as wholly distinct.
Either of your statements are possible, and they are direct opposites. If one is true, the other is completely false.