It's Accessible and Affordable: Immersive Media in Higher Education
A Q&A with Kyle Bowen
CT talks with Penn State University Director of Education Technology Kyle Bowen about the directions immersive media has been taking in higher education and the practices that are making 3D, VR, AR, and 360 video technologies accessible for faculty and students.
Mary Grush: We've certainly been hearing a lot about immersive media — virtual reality, augmented reality, mixed reality, and 3D video. How would you characterize where we are now, with immersive media?
Kyle Bowen: People have been using immersive media for a very long time. Think back to when you were a kid peering into a View-Master — one of those 3D devices many of us played with. Even something as simple as that gave us our introduction to the experience of exploring cities, towns, and scenery through an immersive medium — and "visiting" places that we did not have access to otherwise.
As immersive technology continued to evolve over time, so did our ability to create even more immersive experiences, and over the years we saw the development of various 3D, virtual reality, augmented reality, and mixed reality technologies.
Where do we stand now? The opportunity we have now is that we can begin to innovate on top of all that and create, with even better virtual and immersive tools, experiences that challenge our established ideas of immersive media and take us far beyond what we can experience in "real" physical environments.
If we look at where we are regarding our current efforts, we see a lot of experimentation going on, in many different disciplines and fields, using a wide variety of immersive technologies.
Grush: What are the different types of technologies that exist in the immersive media space — at least the main categories?
Bowen: Immersive media exist in four main categories: 3D, 360 video, virtual reality, and augmented reality.
Today's 3D media technologies are very much like the movies you would experience in a traditional movie theater, where you would put on 3D glasses and be watching a film that just sort of comes out of the screen.
But the interesting part of all that is, as 3D technologies continue to evolve and improve, we see higher and even vastly higher quality images being produced. So, what is happening — even what we see in a regular movie theater — is a transition into hyper reality, where the images are of such high quality that our visual experience is so much greater than what we would normally expect to see in the natural world.
The ultimate applications of that kind of hyper reality are still unknown. So, pinpointing its implications for the future is difficult, as this is such a growing area.
Let's move on to the next level, where you will find 360 video — our second major immersive media category. This is simply video that is captured all around the camera. The viewer may look anywhere they want to while the video is playing.
360 video has many values, but primarily it is recognized as the easiest immersive media to create. And there are now many inexpensive 360 video cameras on the market. Plus, you can use your cell phone to experience and view these videos. So, 360 video is a very accessible technology, in practical terms.
When working with 360 video, headsets that use your smartphone can be economically added to the mix, starting with Google Cardboard at a few dollars, and moving up to the Samsung Gear VR technology at maybe around a hundred dollars — so we can find very inexpensive solutions for creating and viewing 360 video and making our immersive connections.
Next, we look at virtual reality. VR is a completely simulated environment. And unlike 360 video, where the viewer can change where they are looking but not where they are going, with VR, they can change both. With VR, you are controlling both what you are looking at and the path you are moving along.
With VR, where you are using a headset, you have a kind of complete denial of reality: The "real" world that surrounds you is no longer visible. The real world has been completely replaced by what you see through your VR headset. But the value of that, is that you can overcome the limits of reality. Your experience can exist in ways that aren't physically possible, or where physics does not apply in the usual way. This allows you to simulate unlikely or impossible or unsafe environments — or interact with virtual representations of things or people.
A great example of using VR for teaching is the First Class technology that was designed by Ann Clements, a professor of music education at PSU. In this environment, pre-service teachers gain experience by interacting with virtual students.
Finally, the fourth category is augmented reality — and I would include mixed reality here. Like VR, with AR you have control over what you are seeing, and where you are going. But unlike VR, with AR, reality — your real-world environment — is still present. You can still see everything that's around you. It's simply been modified to include additional, virtual elements. You can begin to see things in your real-world environment that aren't actually there. You can add information, you can display visualizations… you can do any number of things in this environment, and you can make those things interactive. Microsoft's HoloLens is, I think, the best example of augmented reality technology.
Those are the four major categories of immersive media. If you want to survey immersive media in higher education, you will find work being done across all of these four categories. Higher education is beginning to experiment in each one of these categories, as each comes into its own state of maturity.
Grush: What are a few of the education values institutions have found in their experimentation within or across these categories? Does any one of the four categories have particular potential right now, for higher education?
Bowen: Perhaps the area having the most impact for education in the near term, is 360 video. As we discussed earlier, it's probably the most accessible and the easiest to understand and create.
One of the opportunities 360 video offers, is that it unhinges what I might term the single perspective. When you watch a normal video with a group of people, you are all seeing the same thing onscreen. So in your group's discussions following the video, each member of the group would share their own perspectives, but the group would all have viewed and be commenting on the same thing. With 360 video, because each viewer can look where they want to in the same movie, any two people from the group are likely to have completely different experiences of the movie. From an education perspective, this raises a whole new level of discussion. The differences in viewers' experiences can create conflicts that can become great points of conversation.
A 360 video example from Daniel Foster, an associate professor of agriculture at PSU, shows different apple-peeling techniques that have evolved over time. The viewer can compare these techniques side-by-side or focus in on any aspect of one peeling technology.
Another opportunity we have with 360 video is in its creation. Because it's relatively easy to produce, we can begin to enable students to work together in groups to create these videos — and to collaborate around an emerging form of media.
There is a student-run charity on campus that captured its annual dance marathon event, THON 2017, using 360 video. The viewer can explore not only the main performance, but also the logistics of the event as it unfolds, and the reactions of people attending the event.
Additionally, there is an important value in offering students a way to access places and environments they would normally find difficult or impossible to explore. For example, at PSU we have a professor who is studying coral reefs. He is currently in the U.S. Virgin Islands using a 360 video camera to capture the reefs he is studying. You could imagine the logistics of sending students on a field trip with him — but of course he can fairly easily share his experience with students through the 360 video he takes just in the normal course of his research.
If we can equip faculty who do field research and students who study abroad with 360 video kits, we are opening up new possibilities for education, including the potential of sharing interesting perspectives of our work.
Another point about 360 video is that includes the person running the camera as part of what is captured — they are no longer stuck "behind" the camera and now become part of the action. This has implications for instruction, of course.
There are many education values to explore with immersive media that are easily within our reach, especially given technology that is as simple and affordable as 360 video.
Grush: Is there such a thing as real-time 360 video?
Bowen: Sure. As we begin to look forward to the future of 360 video, the option to deliver live streaming 360 video is really just now getting started. A few new technologies are just becoming available on the market. This is an area that's so new, we really haven't seen much of it yet. But, consider some of the possibilities, such as the use of live streaming 360 video in a distance education classroom. This could help the distant students feel more connected in the classroom — giving them a more immersive experience that might feel more natural.
This has some implications beyond "collaboration technology": Similar to what's going on with "blended learning" — the mixture of online and classroom instruction — we may see instructional practice that takes more advantage of "mixed reality" — the blending of real-world and virtual learning environments. Live streaming 360 video is positioned to help us get there.
So, yes, there is a transition going on in 360 video, from pre-captured events, to a technology that gives us a live experience. But we'll see that more in the next generation of emerging tools.
Grush: Finally, I'd like to ask about the process of 360 video capture from the students' point of view. What are the implications for student storytelling?
Bowen: At PSU we've had great success with kits: a 360 camera, tripod, an Android device, and a VR headset. We also provide software tools like Adobe Premier for editing. Plus, we are providing workshops to learn the basics of 360 video.
The art of storytelling using 360 video is just beginning to develop. Especially because of the characteristic that the viewer can look in any direction, there just aren't standard ways of thinking about direction, and how you get the viewer's attention on something specific at any one time. It could even happen that a piece that is critical to the story is happening behind the viewer.
But while these elements of direction aren't well-defined at this point, there are many things we can explore. In 360 video the story now exists in multiple perspectives, offering new ways to contrast different ideas and alternative points of view.
We're hoping to make 360 video something students can do as part of their collaborative coursework, research, or entrepreneurship. It's a significant future growth area.
Grush: What's the future looking like for immersive media in higher education? Could you point to any important next steps?
Bowen: As we look forward, we realize that immersive media technology is advancing rapidly now. It seems like the possibilities are endless. But there are many pieces that we are just beginning to understand, especially considering values for higher education and how we put all this into practice.
There are many things we still need to explore. One area that's going to be very important is the creation of sharable learning objects. For example, we've learned how to share video over the years, and we are now starting to learn how to share 360 video. But when we start working with VR, AR, and mixed reality, how do we begin to share and develop the learning experiences faculty and students create? And how do we do this sustainably?
And especially given AR and MR, and the blending of environments, how do we approach new learning spaces — classrooms and labs — that facilitate and support faculty and students in the use of immersive media?
Finally, how can we network our immersive media experiences and creations so that we can begin to connect students, faculty, departments, and universities in their collaborative and disciplinary work?
Interesting new explorations are at hand. This is all part of what we are looking at going forward with immersive media in higher education.
[Editor's note: Images and video links provided courtesy Penn State University.]