Machine Learning

CMU Researchers Fine-Tune Flu Season Forecasting

• By Dian Schaffhauser
• 12/15/16

Flu season in the United States (which is really just cranking up this month) is expected to hit its peak in February, like it has for the last three decades according to flu season data maintained by the Centers for Disease Control and Prevention (CDC). At Carnegie Mellon University (CMU) a team of researchers has put its own flu forecasting operations to work. And this year, yet again, they'll be monitoring the results of various approaches to figure out which one is more accurate: using artificial intelligence and machine learning, or crowdsourcing the predictions. What they learn from the process isn't limited to the flu, however, as their work can be applied to predict other kinds of outbreaks.

Last year's efforts earned CMU's Delphi research group accolades for contributing to the CDC the three most accurate national-level flu forecasts during the 2015-2016 flu season from among 11 external groups. This year, the team is hoping to beat its record. Delphi belongs to a University of Pittsburgh-based MIDAS National Center of Excellence, a National Institutes of Health-funded network of researchers who are developing computational models to guide responses to disease outbreaks.

"Our predictions last season proved to be reasonable, but the truth is that when it comes to forecasting epidemics, whether it be for the flu or for other diseases, we're just getting our feet wet," said Roni Rosenfeld, professor in the School of Computer Science's Machine Learning Department and Language Technologies Institute, in a CMU News article about the work.

The predictions made last season by the most accurate of CMU's forecasting systems was within 25 percent of the CDC's best estimate for flu activity just three-quarters of the time, added Ryan Tibshirani, an associate professor of statistics and machine learning.

All of the work done by the Delphi team feeds into a research initiative begun by the CDC in 2013. "Predict the Influenza Season Challenge" is a public contest to encourage people from around the world to predict the timing, peak and intensity of each year's flu seasons using social media data alongside data from the organization's own flu "surveillance" systems. The latter parses reports describing flu-like symptoms from doctor offices and health clinics.

The importance of the predictions is that getting the timing right helps public health officials and other healthcare professionals schedule their vaccination campaigns, issue flu information and make staffing decisions.

At CMU, the research team's top-ranked forecasting system uses machine learning to predict based on historic and current data fed by the CDC. Its second-ranked system uses weekly predictions fed by ordinary people into the "Influenza Edition" of Epicast. Each week, registered users spend a couple of minutes predicting current and future flu activity within one or more of the various health and human services regions of the United States.

This human-powered approach was actually the top-ranked forecasting system for the 2014-2015 flu season, said Rosenfeld. "Any one human did not do better than the statistical system," he emphasized. "They did worse. But in the aggregate, the human system did better that season."

It's too early to tell how the forecasting will turn out this year. The season typically runs from October through May, with weekly forecasts issued for the country as a whole and within the 10 regions. Due to reporting lags, however, the true flu activity levels won't be known until the season is done.

While the researchers are refining their prediction methods and models on the flu, specifically, they're also applying their systems to forecasting outbreaks of dengue fever, which strikes about 100 million people around the world each year, killing thousands, according to the CDC. Eventually, the team would also like to apply its forecasting efforts to HIV, drug resistance, Ebola, Zika and Chikungunya.