Students Immersed in Petascale Computing as Part of Blue Waters Mission

• By Dian Schaffhauser
• 12/18/17

Earthquake preparedness, violent explosions of massive stars at the end of their lives, effectiveness of climate change response policy and teaching students about high-performance computing: These are some of the 130 projects undertaken in the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign this year, tapping into the computing power of Blue Waters, the center's high-performance computing system. Among all of the research undertaken, at least one percent of them — 60 million core hours of computational capacity — each year is dedicated to "educational work," projects intended to support the development of a national workforce with expertise in petascale computing.

Blue Waters, the National Science Foundation's most powerful system for sustained computation and data analysis, is a Cray machine with a total peak performance of 13.34 petaflops. The system runs 22,640 XE compute nodes (each containing two AMD Interlagos processors) and 4,228 XK compute nodes (each containing one AMD Interlagos processor and one NVIDIA GK110 "Kepler" accelerator). The configuration allows for extremely large simulations on hundreds of thousands of traditional CPUs for science and engineering projects and development and optimization of applications that leverage the compute power of thousands of GPUs.

The educational work involves more than the use of computational time and storage. Projects can also include in-class, interactive exercises and assignments, workshops, longer "institutes," semester courses and support of internships and fellowships. So far, 200 such projects involving 160 institutions have been performed since the advent of Blue Waters in 2012.

Participants are asked to file a report documenting the impact of the education allocation. Among the comments:

• "This project introduced many of the students to high performance computing for the first time — setting up batch scripts, requesting interactive nodes, running remote simulations, and using gpu-accelerated programs";
• "The Blue Waters system gave the students access to a system with much larger scaling capabilities than they typically have access to. In particular, one team was able to scale their code to hundreds of nodes, which would be impossible at their current institutions"; and
• "The use of Blue Waters enabled students to understand issues of programming at scale (up to 128 nodes [4,096 core equivalents]) and to tune their codes in a dedicated environment where performance is roughly repeatable."

Faculty and students who work on these projects also get the benefit of Blue Waters staff for technical support and as "points of contact" for the projects.

"To date, the NSF Blue Waters Project has provided over 20 billion core-hour equivalents to science, engineering and research projects, supported not just by the NSF, but also by NIH, NASA, DOE, NOAA, and other funders. Without Blue Waters, these funded investigations might not even be possible," said Blue Waters Director and Principal Investigator, Bill Kramer, in prepared statement.

The Blue Waters organization said it accepts education proposals year-round. They're due the first working day of each month, and a response will be given "roughly" about mid-month. The education allocation process is explained on the Blue Waters website.