Carnegie Mellon To Engage Yahoo! Open Source Supercomputing Project

Carnegie Mellon University will become the first higher education institution to work with Yahoo!'s M45, a new project announced yesterday by the Internet firm designed to advance distributed computing research and software development. The program, which leverages the Apache Software Foundation's open-source Hadoop, will allow researchers to test software running on a Yahoo!-provided 4,000-processor supercomputer.

According to Yahoo!, its M45 project differs from other supercomputing projects in that it's focused exclusively on "pushing the boundaries of large-scale systems software research." For the program, Yahoo! will make available to researchers a 4,000-processor computing cluster capable of performing 27 teraFLOPS and sporting 3 TB of memory and 1.5 petabytes of storage. It will run the latest version of Hadoop (to which Yahoo! is one of the principal contributors) and other open-source software, including, according to Yahoo!, the Pig parallel programming language.

“Hadoop has become an important computing environment for data-intensive applications and Yahoo! is playing a leading role in its development. We are excited about collaborating with Yahoo! on systems software research, helping to advance the state of the art, and creating new research possibilities in this critical area,” said Randall E. Bryant, dean of the School of Computer Science at Carnegie Mellon, in a statement released yesterday. “We look forward to working with Yahoo! and jointly contributing back to the open source community.”

Carnegie Mellon, for its part, will be the first institution to use the system. CMU professors Garth Gibson and Greg Ganger will "instrument the system and evaluate its performance," according to Yahoo! "Simultaneously, Carnegie Mellon computer science professors Jamie Callan and Christos Faloutsos, academic leaders in text and Web mining, will solve challenging information retrieval and large-scale graph problems on the cluster. Carnegie Mellon faculty members Alexei Efros, Noah Smith, and Stephan Vogel will also use the cluster to tackle large-scale computer graphics, natural language processing, and machine translation problems, respectively."

CMU is also involved in Google/IBM's parallel computing initiative, a pilot academic program also centered around Hadoop and a large-scale processor cluster.

Yahoo! said it plans to make M45 available to other institutions for research in the future.

Read More:

About the Author

David Nagel is the former editorial director of 1105 Media's Education Group and editor-in-chief of THE Journal, STEAM Universe, and Spaces4Learning. A 30-year publishing veteran, Nagel has led or contributed to dozens of technology, art, marketing, media, and business publications.

He can be reached at [email protected]. You can also connect with him on LinkedIn at https://www.linkedin.com/in/davidrnagel/ .


Featured

  • MathGPT

    MathGPT AI Tutor Now Out of Beta

    Ed tech provider GotIt! Education has announced the general availability of MathGPT, an AI tutor and teaching assistant for foundational math support.

  • person signing a bill at a desk with a faint glow around the document. A tablet and laptop are subtly visible in the background, with soft colors and minimal digital elements

    California Governor Signs AI Content Safeguards into Law

    California Governor Gavin Newsom has officially signed off on a series of landmark artificial intelligence bills, signaling the state’s latest efforts to regulate the burgeoning technology, particularly in response to the misuse of sexually explicit deepfakes. The legislation is aimed at mitigating the risks posed by AI-generated content, as concerns grow over the technology's potential to manipulate images, videos, and voices in ways that could cause significant harm.

  • white desk with an open digital tablet showing AI-related icons like gears and neural networks

    Elon University and AAC&U Release Student Guide to AI

    A new publication from Elon University 's Imagining the Digital Future Center and the American Association of Colleges and Universities offers students key principles for navigating college in the age of artificial intelligence.

  • abstract technology icons connected by lines and dots

    Digital Layers and Human Ties: Navigating the CIO's Dilemma in Higher Education

    As technology permeates every aspect of life on campus, efficiency and convenience may come at the cost of human connection and professional identity.