Carnegie Mellon To Engage Yahoo! Open Source Supercomputing Project

Carnegie Mellon University will become the first higher education institution to work with Yahoo!'s M45, a new project announced yesterday by the Internet firm designed to advance distributed computing research and software development. The program, which leverages the Apache Software Foundation's open-source Hadoop, will allow researchers to test software running on a Yahoo!-provided 4,000-processor supercomputer.

According to Yahoo!, its M45 project differs from other supercomputing projects in that it's focused exclusively on "pushing the boundaries of large-scale systems software research." For the program, Yahoo! will make available to researchers a 4,000-processor computing cluster capable of performing 27 teraFLOPS and sporting 3 TB of memory and 1.5 petabytes of storage. It will run the latest version of Hadoop (to which Yahoo! is one of the principal contributors) and other open-source software, including, according to Yahoo!, the Pig parallel programming language.

“Hadoop has become an important computing environment for data-intensive applications and Yahoo! is playing a leading role in its development. We are excited about collaborating with Yahoo! on systems software research, helping to advance the state of the art, and creating new research possibilities in this critical area,” said Randall E. Bryant, dean of the School of Computer Science at Carnegie Mellon, in a statement released yesterday. “We look forward to working with Yahoo! and jointly contributing back to the open source community.”

Carnegie Mellon, for its part, will be the first institution to use the system. CMU professors Garth Gibson and Greg Ganger will "instrument the system and evaluate its performance," according to Yahoo! "Simultaneously, Carnegie Mellon computer science professors Jamie Callan and Christos Faloutsos, academic leaders in text and Web mining, will solve challenging information retrieval and large-scale graph problems on the cluster. Carnegie Mellon faculty members Alexei Efros, Noah Smith, and Stephan Vogel will also use the cluster to tackle large-scale computer graphics, natural language processing, and machine translation problems, respectively."

CMU is also involved in Google/IBM's parallel computing initiative, a pilot academic program also centered around Hadoop and a large-scale processor cluster.

Yahoo! said it plans to make M45 available to other institutions for research in the future.

Read More:

About the Author

David Nagel is the former editorial director of 1105 Media's Education Group and editor-in-chief of THE Journal, STEAM Universe, and Spaces4Learning. A 30-year publishing veteran, Nagel has led or contributed to dozens of technology, art, marketing, media, and business publications.

He can be reached at [email protected]. You can also connect with him on LinkedIn at https://www.linkedin.com/in/davidrnagel/ .


Featured

  • geometric pattern features abstract icons of a dollar sign, graduation cap, and document

    Maricopa Community Colleges Adopts Platform to Combat Student Application Fraud

    In an effort to secure its admissions and financial processes, Maricopa Community Colleges has partnered with A.M. Simpkins and Associates (AMSA) to implement the company's S.A.F.E (Student Application Fraudulent Examination) across the district's 10 institutions.

  • stylized figures, resumes, a graduation cap, and a laptop interconnected with geometric shapes

    OpenAI to Launch AI-Powered Jobs Platform

    OpenAI announced it will launch an AI-powered hiring platform by mid-2026, directly competing with LinkedIn and Indeed in the professional networking and recruitment space. The company announced the initiative alongside an expanded certification program designed to verify AI skills for job seekers.

  • Abstract AI circuit board pattern

    New Nonprofit to Work Toward Safer, Truthful AI

    Turing Award-winning AI researcher Yoshua Bengio has launched LawZero, a new nonprofit aimed at developing AI systems that prioritize safety and truthfulness over autonomy.

  • hooded figure types on a laptop, with abstract manifesto-like posters taped to the wall behind them

    Hacktivism Is a Growing Threat to Higher Education

    In recent years, colleges and universities have faced an evolving array of cybersecurity challenges. But one threat is showing signs of becoming both more frequent and more politically charged: hacktivism.