Virtual Reality Space Lets Students Experience Big Data
A new facility at Virginia Tech uses large-scale visuals and sound to immerse users in vast amounts of data.
- By Dian Schaffhauser
- 10/07/15
Virginia Tech's Cube (photo courtesy of Virginia Tech)
Imagine walking through a black room four stories high, 50 feet wide and 40 feet deep, populated with speakers. As you move through the space wearing a head-mounted display (no mouse, keyboard or joystick needed), you're immersed in vast amounts of data — both visually and aurally — collected from an actual storm that took place a little more than two years ago. As the recorded data shows the formation of some kind of supercell, your ears detect something distinct from every other sound that permeates the space — akin to hearing your name being spoken across the room during a lively cocktail party. You turn and move toward the sound to explore it further. Before your eyes a gigantic tornado forms. Your experience — and the exploration sparked by it — could result in a better understanding of how to interpret the data generated by tornados such as the one that hit Moore, OK, in May 2013, killing two dozen people and injuring hundreds.
That's the thinking behind Virginia Tech's Cube, an adaptable space for research and experimentation housed in the campus's Moss Arts Center. A joint project of the university's Center for the Arts and the Institute for Creativity, Arts & Technology (ICAT), the Cube officially opened for business in January but has already hosted numerous performances as well as events in big data exploration and immersive environments.
According to Ben Knapp, director of ICAT, the Cube shows the potential for immersive environments to help remake learning, design and collaboration. "That's the amazing thing about being immersed in visual and aural and sonic environments — that ability to do what we've done for the history of human beings and primates: use our hearing to detect change, to pull things out from patterns and then use our vision to turn around and look and explore that. That's what the Cube is really able to do."
A Load of Technology in a Big Box
Launching the Cube certainly wasn't Virginia Tech's first foray into virtual reality. Inspired by work at the University of Illinois, the institution joined others in creating CAVES, small enclosures that surround the user with large screens on multiple walls where graphics are projected and viewed through stereo glasses. As viewers wearing position sensors move within their CAVEs, a supercomputer updates perspectives and stereo projections to move with them. The CAVE space at Virginia Tech is about 900 cubic feet; the Cube is 71 times larger — 64,000 cubic feet. But, noted Knapp, "We'd love for it to be bigger."
At 64,000 cubic feet, the Cube provides an immersive environment with plenty of room for sound and motion capture. (photo courtesy of Virginia Tech)
The Cube's tremendous size enables multiple benefits, Knapp said. First, the capacity lets more people explore the environment simultaneously for expanded collaboration. Second, it provides more room for users to walk through, for wider ranging navigation. Third, the height of the space (32 feet) allows for a greater vertical environment for sound and motion capture. Without that height, motion capture cameras would only be able to detect what's up front; anything else would be obstructed. "When you have that higher point of view, that's terrific, because then you're not obscured," explained Knapp.
In addition, the scale of the Cube provides for a "complete hemisphere of sound," Knapp said. "We've actually had performances where we have people literally standing on the four stories of catwalk — all around you so that performers are really bathing you in this visual and sonic environment."
The $15 million environment runs gigabit Ethernet with fiber analog audio and digital video connections and includes a warehouse worth of audio and video technology:
Currently, all of that equipment is managed by a lone media engineer, Tanner Upthegrove. However, the institution recently approved a new staff position, allowing ICAT to embark on a nationwide hunt for an additional audio-electronics technologist to work with faculty in getting their research projects set up in the space. It's not a typical IT job, Knapp wryly suggested.
To Boldly Go Where No One Has Gone Before
ICAT was formulated around the idea of collaboration across disciplines — specifically, the arts, design, engineering and science. The projects undertaken in the Cube reflect that "transdisciplinary" direction. In fact, it's promoted by the institute itself, which seeds cross-disciplinary research through small monthly grants and large annual grants.
The tornado project was a good example. During an event in the performing hall adjoining the Cube, Knapp was having an informal conversation with Geography Department head Bill Carstensen. He became interested in the possibilities, pulled together a team with experts in GIS, meteorology, geography, visual arts and computer science, and wrote a grant proposal that ICAT's committee accepted.
In the Cube, users wearing virtual reality glasses can watch as a tornado forms before their eyes. (courtesy of Virginia Tech)
While researchers can't position observers or recording instruments in dangerous atmospheric conditions, the Cube allowed the team to use data from the National Oceanic and Atmospheric Administration to replicate the event visually. As the data plays out and the tornado forms and follows its treacherous path, multiple viewers can move around inside the experience and watch it unfold around them.
That project was primarily focused on the visual, Knapp said. Sound came later when a professor of music became involved.
FutureHaus, a multi-year initiative that reimagines the American home, is another example that draws from architecture and computer science. The two-story structure could be virtually designed in the Cube first and then physically constructed.
"If you have a two-dimensional plan of a house, that's one thing. But if you can actually walk through the house that's a three-dimensional representation, that's really amazing," Knapp said. By placing that virtual house inside the Cube, he added, a group of people can take a tour, working together to tweak the design. "You could walk through a model of the building and say, 'You know, this needs to go there. This is too high. This is just right.'"
An Education in Data
An obvious area of appeal for the Cube is its use as an educational platform, in both higher ed and K-12.
For a Women in Computing Day, professors Chris North from computer science and Leanna House from statistics worked with ICAT to bring 80 seventh grade girls into the Cube to learn about data analytics in a "Be the Data" event. The girls were separated into groups of 20 and outfitted with helmets that had arrays of reflective balls, each format representing a different kind of animal with 30 variables. Each girl embodied a single data point that could be viewed as a representational symbol on a large screen. The girls explored data clustering through various experiments, such as dividing themselves into groups of edible and inedible animals. As they separated into their groupings, they could watch the clusters form on the screen and then monitor the variables, which grew or shrank to denote weighting. In the edible/inedible experiment, for instance, the parameter related to fuzziness turned out to be important. "We tend not to eat fuzzy animals," Knapp pointed out.
Through the experiment the girls could pose research questions about the data and uncover the answers by "being" the data while also learning about the underlying math involved in data analysis.
Eventually, said Knapp, that same approach to instruction could be used in classes at Virginia Tech as well to help its students learn how to "explore abstract data, visualize it and detect changes in patterns within the data."
A Portal Between Worlds
One area that most excites Knapp and the ICAT team is the potential to blend physical and virtual worlds in new ways. A National Science Foundation-funded project called Mirror Worlds is bringing together faculty from computer science, the school of visual arts, communication and music to build a virtual model of the Moss center and allow people to walk through the entire building. The goal is to study human behavior and emotion as people connect with each other through portals between the physical and virtual worlds.
The NSF-funded Mirror Worlds project explores the potential of portals between the physical and virtual worlds. Sensors throughout the Moss Arts Center will generate a digital version of people at those same locations in a virtual building. (courtesy of Virginia Tech)
The physical facility is being outfitted with sensors to generate a digital version of people at those same locations in a virtual building. Also, 25 Android-powered nabi Big Tabs will provide "mirrors" into the virtual environment.
"We don't know who you are. We have no visual [image]. It's just blob tracking," said Knapp. The point is that people who are online can "walk" through the building with those who are physically there. "In the real world there's only one person standing on the steps. That's you. You're looking at the mirror. But if somebody from the virtual world is standing right there beside you, you'll see them. Now we've connected two worlds — the virtual world and the real world."
This "portal" that allows people to connect with each other between the physical and virtual worlds could be a hotbed of experimentation — for studying, for example, how people behave in one type of space vs. the other, or how effective virtual environments are for training purposes.
The research could also have an impact on distance learning. "It's always been, you sit at home on your computer and hear somebody lecture or we have a screen on the wall and somebody is lecturing in Idaho, and you're watching them," bemoaned Knapp. The Mirror Worlds project is "reinventing that to ask, how do people in a lab and studio environment collaborate in these connected real and virtual spaces?"
But the ideas for the Cube don't stop there. Another experiment aims to transmit data from multiple functional MRI readings, allowing researchers to "walk through" several active brains residing in the same room. Upthegrove mentioned another project that will feature 360-degree 3D projection.
Knapp mused about the potential use of an "amazing" wind tunnel at Virginia Tech: "You put sensors on the front end of a wing, and you put it in the wind tunnel, and you get these streams of data. But you can't stand in the wind tunnel and see what's going on. You can't manipulate it. Could we create a virtual wind tunnel in the Cube so that, in real time, you're walking around the wing and watching air flow with your colleagues as the data is streaming in from the real wind tunnel?" The answer, of course, is an enthusiastic yes.