AI Software Juggles Probabilities To Learn From Less Data (technologyreview.com)

moon_unit2 quotes a report from MIT Technology Review: You can, for instance, train a deep-learning algorithm to recognize a cat with a cat-fancier's level of expertise, but you'll need to feed it tens or even hundreds of thousands of images of felines, capturing a huge amount of variation in size, shape, texture, lighting, and orientation. It would be lot more efficient if, a bit like a person, an algorithm could develop an idea about what makes a cat a cat from fewer examples. A Boston-based startup called Gamalon has developed technology that lets computers do this in some situations, and it is releasing two products Tuesday based on the approach. Gamalon uses a technique that it calls Bayesian program synthesis to build algorithms capable of learning from fewer examples. Bayesian probability, named after the 18th century mathematician Thomas Bayes, provides a mathematical framework for refining predictions about the world based on experience. Gamalon's system uses probabilistic programming -- or code that deals in probabilities rather than specific variables -- to build a predictive model that explains a particular data set. From just a few examples, a probabilistic program can determine, for instance, that it's highly probable that cats have ears, whiskers, and tails. As further examples are provided, the code behind the model is rewritten, and the probabilities tweaked. This provides an efficient way to learn the salient knowledge from the data.

Comments

Bayesian theorem

[superwiz]

Bayesian probability is named for the Bayes' Theorem (which is named after the namesake mathematician). But the Bayesian inference is called that because it relies specifically on applying Bayes' Theorem rather than any other of Thomas Bayes' work.

Bayesian programming vs deep learning

[akakaak]

The comparison of "deep learning that needs tons of examples" vs "Bayesian programming that can learn from a few examples" is a false dichotomy. It all depends on how much structure you assume a priori versus how much structure you learn from the data.

Typical neural net (deep learning) examples start with no structure and thus require lots of examples. Typical Bayesian net examples start with a lot of structure and thus require only a few examples.

On the other hand, if you start with a highly pretrained net like Inception-v3, then your deep learning cat expert may not need as many examples to generalize. And if your Bayesian programming model starts out with very general, very simple "building blocks" then it may need a lot of examples to extract the predictable structure.

A main difference is that if you want to start with a lot of structure built in, you will probably have to pretrain for the neural net, whereas you can "hand code" the knowledge in your Bayes net. And the structure in the Bayes net may be a lot more transparent and easily interpretable than in the neural net. On the other hand, you had better hope you picked the right structure to begin with or else you will be reasoning over possibilities that are all very wrong! Knowing that an image is 50 times more likely to be a cat than a dog is not very helpful if it is actually a penguin.