Mobile Computing

Researchers Find Use for Smartphones in 3D Model Creation

• By Michael Hart
• 03/30/15

Researchers at Carnegie Mellon University in Pittsburgh have discovered they can use a simple smartphone as a measurement device in the creation of 3D models. By moving a smartphone from side to side or back and forth while either using video or simply switching the phone's screen orientation from portrait to landscape, the user can get a sense of an object's real size.

While video from smartphones and other devices have been used in creating 3D models for some time, this is the first time they have also been used to determine the scale of models — by taking advantage of relatively inexpensive inertial measurement units (IMU).

"We've been able to get accuracies with cheap sensors that we hadn't imagined," said Simon Lucey, an assistant research professor at the Carnegie Mellon University Robotics Institute, where the research on using smartphones in 3D model creation is being conducted.

For instance, Lucey pointed out, with a facetracker program, they have been able to measure the distance between a person's pupils within half a millimeter, making it possible to go virtual shopping for eyeglass frames.

More than 30 years ago, researchers at Carnegie Mellon pioneered the use of 2D images or video to build increasingly accurate 3D models, but taking scale into account has always been a challenge.

But now, as smartphones incorporate higher frame-rate video cameras, the accuracy of the technique will become even better.

"With a high frame-rate camera," Lucey said, "we can excite the IMU by moving the phone faster, without corrupting the image."

Eventually, he said he believes, similar low energy-consuming techniques will be used with robots and self-driving cars rather than the power-hungry technologies like radar and laser rangefinders now being experimented with.