HPC | News

Stanford U Solves Fluid Dynamics Problem with 1.5 Million-Core Supercomputer

• By David Nagel
• 01/29/13

Researchers in Stanford University's Center for Turbulence Research have used a supercomputer with more than 1.57 million processing cores to "solve a complex fluid dynamics problem--the prediction of noise generated by a supersonic jet engine," according to information released by Stanford's School of Engineering.

The work was conducted by Stanford Research Associate Joseph Nichols, along with CTR researchers and Lawrence Livermore National Laboratory staff, on the IBM Sequoia Blue Gene/Q supercomputer housed at the Lawrence Livermore National Laboratory, named the most powerful supercomputer in the world last June by Top500. (It moved to the No. 2 slot in November, replaced by a Cray XK7 system named Titan at the Oak Ridge National Laboratory.) The 16.32 PFlop/s system is built on 98,304 IBM PowerPC A2 chips, with 16 cores per node, using a five-dimensional torus interconnect topology that provides 40 GBps throughput with 2.5 microsecond latency. It has 16 GB of DDR3 SDRAM per node, totaling about 1.57 PB of memory for the entire system.

The accomplishment is significant for a number of reasons; among them, the researchers were able to overcome the computational bottlenecks that arise as the number of cores moves into the millions, showing for the first time that such simulations could be conducted on million-core-plus systems.

As Stanford described it: "CFD simulations test all aspects of a supercomputer. The waves propagating throughout the simulation require a carefully orchestrated balance between computation, memory and communication. Supercomputers like Sequoia divvy up the complex math into smaller parts so they can be computed simultaneously. The more cores you have, the faster and more complex the calculations can be. And yet, despite the additional computing horsepower, the difficulty of the calculations only becomes more challenging with more cores. At the one-million-core level, previously innocuous parts of the computer code can suddenly become bottlenecks."

"Computational fluid dynamics (CFD) simulations, like the one Nichols solved, are incredibly complex. Only recently, with the advent of massive supercomputers boasting hundreds of thousands of computing cores, have engineers been able to model jet engines and the noise they produce with accuracy and speed," said Parviz Moin, the Franklin M. and Caroline P. Johnson Professor in the School of Engineering and Director of CTR.

"These runs represent at least an order-of-magnitude increase in computational power over the largest simulations performed at the Center for Turbulence Research previously," Nichols said in a statement released by Stanford. "The implications for predictive science are mind-boggling."