Captcha This | News

CMU Computer Understands Images with Minimal Human Assistance

• By Leila Meyer
• 11/21/13

NEIL is watching and learning. NEIL is the Never Ending Image Learner computer program at Carnegie Mellon University. Since late July, NEIL has been running 24 hours a day, searching the Web for images, identifying and labeling objects in the images, and attempting to make associations between those objects with minimal assistance from people.

NEIL uses the associated objects in images to attempt to figure out common sense information that people take for granted. For example, NEIL has already figured out on its own that cars can have wheels, zebras can be found on savannas, and trading floors can be crowded. However, NEIL sometimes gets confused and needs help associating image objects correctly. For example, pink is a color, not just the name of a singer.

The project is a collaboration between researchers in language technologies and robotics at Carnegie Mellon. One of the goals of the project is to create the world's largest visual structured knowledge base that labels and catalogs objects, scenes, attributes, and contextual relationships. The data generated by the project will be used to help other computers glean information from images.

"Images are the best way to learn visual properties," said Abhinav Gupta, assistant research professor in Carnegie Mellon's Robotics Institute, in a prepared statement. "Images also include a lot of common sense information about the world. People learn this by themselves and, with NEIL, we hope that computers will do so as well."

NEIL runs on two computer clusters with 200 processing cores and uses technology called computer vision to identify and associate objects and scenes in images. Since the project began, NEIL has analyzed 3 million images, identified 1,500 types of objects and 1,200 types of scenes and made 2,500 associations.

An online listing of objects NEIL has located, identified, and associated can be found at neil-kb.com.