Uppsala U and Swedish Institute of Space Physics Team with IBM To Study Space Weather

• By Dian Schaffhauser
• 06/09/09

Sweden's Uppsala University, IBM, and the Swedish Institute of Space Physics have announced a stream computing project to analyze massive volumes of information in real time to better understand "space weather." By using IBM InfoSphere Streams to analyze data from sensors that track high frequency radio waves, endless amounts of data can be captured and analyzed on the fly. InfoSphere Streams is new software derived from IBM Research project System S that enables large volumes of data to be analyzed in real time.

Over the next year, this project is expected to perform analytics on at least 6 GB of data per second, or 21,600 GB per hour--the equivalent of all the Web pages on the Internet, according to an IBM statement.

Scientists sample high frequency radio emissions from space to study and forecast "space weather" or the effect of plasma eruptions on the sun that reach the earth and adversely affect energy transmission over power lines, communications via radio and TV signals, airline and space travel, and satellites. The recent advent of new sensor technology and antennae arrays means that the amount of information collected by scientists surpassed the ability to analyze it.

"IBM InfoSphere Streams is opening up a whole new way of doing science, not only in this area, but any area of e-science where you have lots of data coming in from external sources and sensors, streaming at such high data rates you can't handle it with conventional technology," said Bo Thide, professor and head of research, Swedish Institute of Space Physics and director of LOIS Space Center in Sweden. "It has helped create a paradigm shift in the area of online observation of the earth, space, sun and atmosphere."

Researchers at Uppsala U and the Swedish Institute of Space Physics worked with LOIS to develop a new type of tri-axial antenna that streams three-dimensional radio data from space, extracting a magnitude more physical information than any other type of antennae array before. Since researchers need to measure signals from space over large time spans, the raw data generated by even one antenna quickly becomes too large to handle or store.

"We've embarked upon an entirely new way of observing radio signals using digital sensors that produce enormous amounts of data," Thide said. "With this type of research, you have to be able to analyze as much data as possible on the fly. There is no way to even consider storing it. InfoSphere Streams is playing a pivotal role in this project. Without it, we could not possibly receive this volume of signals and handle them at such a high data rate because until now, there was not a structured, stable way of analyzing it."

The technology addresses this problem by analyzing and filtering the data the moment it streams in, helping researchers identify the critical fraction of a percent that is meaningful, while the rest is filtered out as noise. Using a visualization package, scientists can perform queries on the data stream to look closely at interesting events, allowing them not only to forecast, but to "nowcast" events just a few hours away. This will help predict, for example, if a magnetic storm on the sun will reach the earth in 18 to 24 hours.

The ultimate goal of the project at Uppsala U is to model and predict the behavior of the uppermost part of the earth's atmosphere and its reaction to events in surrounding space and on the sun. With an ability to predict how plasma clouds travel in space, new efforts can be made to minimize damage caused by energy bursts or make changes to sensitive satellites, power grids or communications systems.