CMU Computer Understands Images with Minimal Human Assistance

NEIL is watching and learning. NEIL is the Never Ending Image Learner computer program at Carnegie Mellon University. Since late July, NEIL has been running 24 hours a day, searching the Web for images, identifying and labeling objects in the images, and attempting to make associations between those objects with minimal assistance from people.

NEIL uses the associated objects in images to attempt to figure out common sense information that people take for granted. For example, NEIL has already figured out on its own that cars can have wheels, zebras can be found on savannas, and trading floors can be crowded. However, NEIL sometimes gets confused and needs help associating image objects correctly. For example, pink is a color, not just the name of a singer.

The project is a collaboration between researchers in language technologies and robotics at Carnegie Mellon. One of the goals of the project is to create the world's largest visual structured knowledge base that labels and catalogs objects, scenes, attributes, and contextual relationships. The data generated by the project will be used to help other computers glean information from images.

"Images are the best way to learn visual properties," said Abhinav Gupta, assistant research professor in Carnegie Mellon's Robotics Institute, in a prepared statement. "Images also include a lot of common sense information about the world. People learn this by themselves and, with NEIL, we hope that computers will do so as well."

NEIL runs on two computer clusters with 200 processing cores and uses technology called computer vision to identify and associate objects and scenes in images. Since the project began, NEIL has analyzed 3 million images, identified 1,500 types of objects and 1,200 types of scenes and made 2,500 associations.

An online listing of objects NEIL has located, identified, and associated can be found at neil-kb.com.

About the Author

Leila Meyer is a technology writer based in British Columbia. She can be reached at [email protected].

Featured

  • man working on laptop outdoors

    Digital Leadership Must-Haves for 2025: A CDO's Picks

    Now that he's more than a year and a half into his chief digital officer role at NJIT, we've asked Ed Wozencroft to reflect on his areas of concentration: What work must digital leaders "own" in 2025?

  • a professional worker in business casual attire interacting with a large screen displaying a generative AI interface in a modern office

    Study: Generative AI Could Inhibit Critical Thinking

    A new study on how knowledge workers engage in critical thinking found that workers with higher confidence in generative AI technology tend to employ less critical thinking to AI-generated outputs than workers with higher confidence in personal skills.

  • blue and green lines intersecting and merging in an abstract pattern against a light gray background with a subtle grid design

    Data Integration Market: Cloud Giants Down, AI Up

    "By 2027, AI assistants and AI-enhanced workflows incorporated into data integration tools will reduce manual intervention by 60 percent and enable self-service data management," according to research firm Gartner.

  • glowing digital document floats above a laptop, surrounded by soft, flowing tech-inspired lines and geometric shapes in shades of blue and white

    Boston U Expands AllCampus Partnership with New Non-Credit Certificate Programs

    Boston University Metropolitan College's Center for Professional Education has expanded its relationship with online program management provider AllCampus. The agreement will extend support for BU's existing online Paralegal Studies Program and add new non-credit certificates in financial planning, professional fundraising, and genealogical studies.