UC San Diego To Deploy 2-Petaflop Supercomputer with $12 Million NSF Grant

• By Joshua Bolkan
• 10/07/13

The San Diego Supercomputer Center (SDSC) at the University of California, San Diego will deploy Comet, a machine capable of two quadrillion operations per second, or nearly two petaflops.

Funded in part with a $12 million grant from the National Science Foundation and scheduled to come online in early 2015(NSF), Comet is "designed to transform advanced scientific computing by expanding access and capacity among traditional as well as non-traditional research domains," according to a news release, such as genomics, social sciences, and economics.

Comet will join another SDSC supercomputer, Gordon, as a resource within the NSF's Extreme Science and Engineering Discovery Environment (XSEDE) in an effort to "help meet the pent-up demand for computing on up to 1,024 cores, which accounts for 98 percent of current jobs among XSEDE users," according to a news release. Though Comet is capable of very large jobs, it will be scheduled to provide fast turnaround on many small jobs.

"Supercomputers such as Comet and our data-intensive Gordon system are helping to fulfill the NSF's goal to extend the impact of advanced computational resources to a larger and more diversified user base," said UC San Diego Chancellor Pradeep K. Khosla, in a prepared statement. "Our San Diego Supercomputer Center is a key resource for our university system and has had a long track-record of leadership in high-performance computers and data-intensive computing."

"Comet is designed to be part of an emerging cyberinfrastructure for what is called the 'long tail' of science, which encompasses the idea that a large number of modest-sized computationally based research projects still represents, in aggregate, a tremendous amount of research and scientific impact," said Sandra A. Brown, vice chancellor for research at UC San Diego, in a prepared statement.

Comet will also be able to support high-performance virtualization, making it the first XSEDE system to do so. "SDSC team members plan to work closely with communities and enable them to develop the customized software stacks that meet their needs by defining virtual clusters," according to information released by the school. "With significant advances in Single Root IO Virtualization (SRIOV), virtual clusters will be able to attain near native hardware performance in both InfiniBand latency and bandwidth, making them suitable for MPI-style parallel computing."

Other features of the Dell-based cluster will include:

• Two Intel Xeon Processors on each node, along with 128 GB of DRAM, and 320 GB of flash memory;
• Seventy-two-node racks with a full InfiniBand FDR interconnect and a 4:1 bisection interconnect across racks to improve capacity for smaller jobs;
• Memory nodes with 1.5 TB of memory each;
• Nodes with Nvidia graphic processing units (GPUs);
• Seven PB of Lustre-based high performance storage;
• Six PB of durable storage; and
• One-hundred Gbps connectivity.

"Comet is all about computing for the 99 percent," said SDSC Director Michael Norman, the project's principal investigator, in a prepared statement. "As the world's first virtualized HPC cluster, it is designed to deliver a significantly increased level of computing capacity and customizability to support data-enabled science and engineering at the campus, regional, and national levels, and in turn support the entire science and engineering enterprise, including education as well as research."

More information about the San Diego Supercomputer Center is available at sdsc.com. Go to xsede.org to learn more about XSEDE.