MIT Engineers Build VR System to Train Racer Drones

• By Joshua Bolkan
• 05/21/18

Engineers at the Massachusetts Institute of Technology have created a virtual reality environment to train drones to fly fast around obstacles.

Dubbed "Flight Goggles," the software causes drones to "see" virtual obstacles in space that is actually clear of physical objects — giving drones a way to fail during flight training without being involved in repeated, time-consuming accidents. It will also offer a testbed for a range of conditions and environments researchers may wish to simulate.

"We think this is a game-changer in the development of drone technology, for drones that go fast," said Sertac Karaman, associate professor of aeronautics and astronautics at MIT, in a prepared statement. "If anything, the system can make autonomous vehicles more responsive, faster and more efficient."

Karaman was inspired to build the system initially by competitive drone racing, a new sport in which humans pilot drones remotely through intricate mazes. Karaman decided to see if he could build an autonomous drone that would beat the human pilots.

"In the next two or three years, we want to enter a drone racing competition with an autonomous drone, and beat the best human player," Karaman said.

Right now, drones are generally trained to fly in enclosed spaces with props like windows to fly through and nets to catch the devices as they lose control. Such a setup is fine for training slow-flying drones, like those used in mapping, but not suitable for training racers, whose flight can vary wildly from small changes in their code. Racers can need repair and even replacement after each high-speed crash, slowing down training and ramping up costs.

"The moment you want to do high-throughput computing and go fast, even the slightest changes you make to its environment will cause the drone to crash," Karaman said in a prepared statement. "You can't learn in that environment. If you want to push boundaries on how fast you can go and compute, you need some sort of virtual reality environment."

The training facility includes a motion-capture system to track the drone's position and orientation as it flies, an image rendering program that can create photorealistic images of physical environments, and tools for beaming the images into the drone's computer as it flies, all to give the drone the illusion of physical obstacles as it flies through an empty room.

Karaman said the system could be used to test out new sensors, as well, by putting them on the drone and seeing if the performance improves or deteriorates in different environments, or to train drones how to fly around humans. For example, a human could walk around in a motion capture suit and that data could be beamed into the drone's computer from a remote location, thus making the person appear near the drone while at no risk of physical harm.

"One day, when you're really confident, you can do it in reality, and have a drone flying around a person as they're running, in a safe way," Karaman said in a news release. "There are a lot of mind-bending experiments you can do in this whole virtual reality thing. Over time, we will showcase all the things you can do."