Recognizing Scenes Like the Brain Does

Roland Piquepaille writes "Researchers at the MIT McGovern Institute for Brain Research have used a biological model to train a computer model to recognize objects, such as cars or people, in busy street scenes. Their innovative approach, which combines neuroscience and artificial intelligence with computer science, mimics how the brain functions to recognize objects in the real world. This versatile model could one day be used for automobile driver's assistance, visual search engines, biomedical imaging analysis, or robots with realistic vision. Here is the researchers' paper in PDF format."

Comments

Earlier work 1989-1997 on street scene analysis

[Wills]

Apologies for blowing my own trumpet here, but there was much earlier work in the 1980s and 1990s on recognizing objects in images of outdoor scenes using neural networks that achieved a similarly high accuracy compared to the system mentioned in this article:

1. WPJ Mackeown (1994), A Labelled Image Database, unpublished PhD Thesis, Bristol University.

Design of a database of colorimetrically calibrated, high quality images of street scenes and rural scenes, with highly accurate near-pixel ground-truth labelling based on a hierarchy of object categories. Example of labelled image from database

Design of a neural network system that recognized categories of objects by labelling regions in random test images from the database achieving 86% accuracy

The database is now known as the Sowerby Image Database and is available from the Advanced Technology Centre, British Aerospace PLC, Bristol, UK. If you use it, please cite: WPJ Mackeown (1994), A Labelled Image Database, PhD Thesis, Bristol University.

2. WPJ Mackeown, P Greenway, BT Thomas, WA Wright (1994).
Road recognition with a neural network, Engineering Applications of Artificial Intelligence, 7(2):169-176.

A neural network system that recognized categories of objects by labelling regions in random test images of street scenes and rural scenes achieving 86% accuracy

3. NW Campbell, WPJ Mackeown, BT Thomas, T Troscianko (1997).
Interpreting image databases by region classification. Pattern Recognition, 30(4):555-563.

A neural network system that recognized categories of objects by labelling regions in random test images of street scenes and rural scenes achieving 92% accuracy

There has been various follow up research since then

Fine paper, but why not quote all of PAMI ?

[HuguesT]

This is a nice paper by respected researchers in AI+Vision, however pretty much the entire content of the journal this was published in (IEEE Pattern Analysis and Machine Intelligence) is up to that level. Why single out that particular paper ?

Interested readers can browse the content of PAMI current and back issues and either go to their local scientific library (PAMI is recognisable from afar by its bright yellow cover) or search on the web for interesting articles. Often researchers put their own paper on their home page. For example, here is the publication page of one of the authors (I'm not him).

For the record, I think justifying various ad-hoc vision/image analysis techniques using approximations of biological underpining is of limited interest. When asked if computer would think one day, Edsgerd Dijkstra famously answered by "can submarine swim?". In the same manner, it has been observed that (for example) most neural network architectures make worse classifiers than standard logistic regression, not to mention Support Vector Machines, which what this article uses BTW.

The summary by our friend Roland P. is not very good :

This versatile model could one day be used for automobile driver's assistance, visual search engines, biomedical imaging analysis, or robots with realistic vision

• There already exist working automated driving software. The december 2006 issue of IEEE Computers magazing was on them last month. Read about the car that drove a thousand miles on Italy's road thanks to Linux, no less.
• Visual search engine exist, at the research level. The whole field is called "Content Based Retrieval", and the main issue is not so much to search, but to formulate the question.
• Biomedical image analysis has been going strong for decades and is used every day in your local hospital. Ask your doctor !
• Robotic vision is pretty much as old as computers themselves. There are even fun robot competitions like robocup.

I could go on with lists and links but the future is already here, generally inconspicuously. Read about it.