Mixed Reality

WPI Researchers Use HoloLens to Visualize Complex Biological Networks

• By Rhea Kelly
• 10/16/17

Mixed reality tools are helping scientists at Worcester Polytechnic Institute see connections between proteins and genes that could hold the keys to curing complex disorders like cancer, diabetes and autism spectrum disorder. Dmitry Korkin, associate professor of computer science and director of the university’s bioinformatics and computational biology program, and his team have been using Microsoft HoloLens to view three-dimensional holographic projections of the linkages and interactions in complex biological networks.

Traditional two-dimensional representations of these networks are so dense that "it looks like a big mess," said Korkin in a news announcement. Researchers' epithet for describing the complex structures: the "hairy ball." The quantity of data involved in studying the networks makes them especially difficult to understand and analyze, noted Korkin. "It is good that we can collect information on these networks with increasingly precise technologies, but when you try to visually analyze it, you realize that it is next to impossible with conventional means."

With HoloLens, Korkin and his team are using their research data to create holographic projections that "float" in front of the viewer, which can then be studied from all sides. "The network is a very common data structure," Korkin said. "Computer scientists have grappled with networks for ages. The problem with biological networks is that they are incredibly complex, with a lot of information packed into very dense structures. When I first saw early prototypes of the technology that would become the HoloLens, I realized that this could be the ideal way to interact with these networks."

Users can manipulate the visualizations by "pinching" them with the thumb and index finger and rotating them in any direction. They can also highlight particular sections using gestures or voice commands. "This is not just a visualization tool," Korkin explained. "It is a tool for the interactive retrieval of information. It allows us to layer on top of the network a host of related data, including structural information and the presence of mutations. In this way it becomes an intelligent object that integrates information and feeds it back to the researcher with a number of suggested options. By incorporating algorithms that encapsulate basic rules, we can create a smart visualization that can help identify the elements of a network that the researchers should pay attention to — the critical things that they should investigate first."

Korkin's research centers on the protein interaction networks that are involved in complex genetic disorders and disease. "These networks can included hundreds of thousands of genes and proteins carrying a plethora of mutations," he said. "Our approach is to use the system information as a framework to merge together information about the properties of these genes and proteins and their functions in an effort to see how individual mutations affect the whole system. This can help us understand the underlying molecular mechanisms affected by the mutations and the possible consequences."

"We would like to use the HoloLens to map the information that we have discovered using our advanced algorithms, which will allow us to begin to build a comprehensive atlas of these elements — the first of its kind," he continued. "There will be thousands of these elements, and we will have the means to start analyzing them. We will be able to visualize them on the chromosome and genome level, bringing in information on the function of the coding and non-coding elements, their mutual positions, the fact that some of these elements occur in multiple copies and some are unique, and so on. The goal is to merge different forms of information and look for patterns, for things that are unusual or abnormal. By allowing us to work with the complete set of data, which we simply cannot do in any other way because the information density is too high to comprehend, the HoloLens will make this process more intuitive and more informative."

Korkin and his team are also working to overcome some limitations of current HoloLens technology, such as a cap on the number of objects that can be displayed at one time. "It is typical for new technologies to have bottlenecks," he noted. "I have no doubt the technology will evolve and this will become less of an issue, but in the meantime we can facilitate this by developing algorithms that allow us to do what we want to do."