CMU Computer Understands Images with Minimal Human Assistance

NEIL is watching and learning. NEIL is the Never Ending Image Learner computer program at Carnegie Mellon University. Since late July, NEIL has been running 24 hours a day, searching the Web for images, identifying and labeling objects in the images, and attempting to make associations between those objects with minimal assistance from people.

NEIL uses the associated objects in images to attempt to figure out common sense information that people take for granted. For example, NEIL has already figured out on its own that cars can have wheels, zebras can be found on savannas, and trading floors can be crowded. However, NEIL sometimes gets confused and needs help associating image objects correctly. For example, pink is a color, not just the name of a singer.

The project is a collaboration between researchers in language technologies and robotics at Carnegie Mellon. One of the goals of the project is to create the world's largest visual structured knowledge base that labels and catalogs objects, scenes, attributes, and contextual relationships. The data generated by the project will be used to help other computers glean information from images.

"Images are the best way to learn visual properties," said Abhinav Gupta, assistant research professor in Carnegie Mellon's Robotics Institute, in a prepared statement. "Images also include a lot of common sense information about the world. People learn this by themselves and, with NEIL, we hope that computers will do so as well."

NEIL runs on two computer clusters with 200 processing cores and uses technology called computer vision to identify and associate objects and scenes in images. Since the project began, NEIL has analyzed 3 million images, identified 1,500 types of objects and 1,200 types of scenes and made 2,500 associations.

An online listing of objects NEIL has located, identified, and associated can be found at neil-kb.com.

About the Author

Leila Meyer is a technology writer based in British Columbia. She can be reached at [email protected].

Featured

  • blue and green lines intersecting and merging in an abstract pattern against a light gray background with a subtle grid design

    Data Integration Market: Cloud Giants Down, AI Up

    "By 2027, AI assistants and AI-enhanced workflows incorporated into data integration tools will reduce manual intervention by 60 percent and enable self-service data management," according to research firm Gartner.

  • abstract geometric pattern of glowing interconnected triangles, hexagons, and circles in blue, gold, and white, spread across a dark navy-to-black gradient background

    OpenAI Unveils 'Operator' AI for Performing Web Tasks

    OpenAI has launched "Operator," an AI agent designed to perform web-based tasks autonomously using its own browser. Currently available as a research preview for Pro users in the United States, the tool aims to automate everyday activities such as filling out forms, ordering groceries, and even creating memes.

  • silhouetted human figures stand opposite a glowing digital brain, surrounded by abstract circuits and shadowy shapes

    Global Tech Execs Expect Advancements in AI to Increase Security Threats

    Forty-one percent of global tech executives in a recent NetApp survey said they believe advancements in AI will significantly increase security threats. The firm's second annual Data Complexity Report points to 2025 as "AI's make or break year."

  • glowing hexagons and circles connected by illuminated lines

    Microsoft Intros Open Source Multi-Agent AI System

    Microsoft researchers have unveiled a new open source multi-agent AI system, Magnetic-One, aimed to help enterprises automate complex tasks typically requiring human intervention.