Tech Giants Form Open Source AI Security Group

Google, Microsoft, Amazon, OpenAI and others have formed a new industry group aimed at promoting AI safety and security standards.

The Coalition for Secure AI (CoSAI) launched on Thursday as a self-described "open source initiative designed to give all practitioners and developers the guidance and tools they need to create Secure-by Design AI systems."

"Founding Premier Sponsors" of CoSAI include Microsoft, Nvidia, Google, IBM, Intel, and PayPal. Listed as "additional" founding members are OpenAI, Anthropic, Amazon, Cisco, Cohere, Chainguard, GenLab, and Wiz.

A Technical Steering Committee of AI experts from academia and industry will oversee the group's work.

The primary mission of CoSAI is to "develop comprehensive security measures that address AI systems' classical and unique risks." This is difficult to do in the current AI landscape, the group argues, because existing efforts to establish AI security standards are fragmented, uncoordinated, and inconsistently applied.

Though it recognizes those efforts and plans to collaborate with other groups focused on AI security, CoSAI believes it is uniquely positioned to establish standards that can be widely agreed-upon and adopted due to its diverse and high-profile membership roster.

"As a Founding Member of the Coalition for Secure AI, Microsoft will partner with similarly committed organizations towards creating industry standards for ensuring that AI systems and the machine learning required to develop them are built with security by default and with safe and responsible use and practices in mind," said Microsoft's AI safety chief Yonatan Zunger in a prepared statement. "Through membership and partnership within the Coalition for Secure AI, Microsoft continues its commitment to empower every person and every organization on the planet to do more ... securely."

"From day one, AWS AI infrastructure and the Amazon services built on top of it have had security and privacy features built-in that give customers strong isolation with flexible control over their systems and data," commented Paul Vixie, vice president and Distinguished Engineer at Amazon Web Services. "As a sponsor of CoSAI, we're excited to collaborate with the industry on developing needed standards and practices that will strengthen AI security for everyone."

"Developing and deploying AI technologies that are secure and trustworthy is central to OpenAI's mission," said Nick Hamilton, head of Governance, Risk and Compliance at OpenAI. "We believe that developing robust standards and practices is essential for ensuring the safe and responsible use of AI and we're committed to collaborating across the industry to do so."

Per CoSAI's founding charter, the group intends to find and share mitigations for AI security risks such as "stealing the model, data poisoning of the training data, injecting malicious inputs through prompt injection, scaled abuse prevention, membership inference attacks, model inversion attacks or gradient inversion attacks to infer private information, and extracting confidential information from the training data."

Interestingly, the group does not consider the following areas to be part of its purview: "misinformation, hallucinations, hateful or abusive content, bias, malware generation, phishing content generation, or other topics in the domain of content safety."

At its outset, CoSAI plans to pursue the following three research areas:

  • AI software supply chain security: The group will explore how to assess the safety of a given AI system based on its provenance. For instance, the group will examine who trained the AI system and how, as well as whether its training process may have left the AI vulnerable to tampering at any point.
  • Security framework development: The group will identify "investments and mitigation strategies" to address the security vulnerabilities in both today's AI systems, as well as future versions.
  • Security and privacy governance: The group will create guidelines to help AI developers and vendors measure risk in their systems.

CoSAI expects to release a paper by the end of this year providing an overview of its findings.

About the Author

Gladys Rama (@GladysRama3) is the editorial director of Converge360.

Featured

  • person signing a bill at a desk with a faint glow around the document. A tablet and laptop are subtly visible in the background, with soft colors and minimal digital elements

    California Governor Signs AI Content Safeguards into Law

    California Governor Gavin Newsom has officially signed off on a series of landmark artificial intelligence bills, signaling the state’s latest efforts to regulate the burgeoning technology, particularly in response to the misuse of sexually explicit deepfakes. The legislation is aimed at mitigating the risks posed by AI-generated content, as concerns grow over the technology's potential to manipulate images, videos, and voices in ways that could cause significant harm.

  • abstract representation of equity at the core of AI

    Why Equity Must Be a Core Part of the Conversation About AI

    AI is an immensely powerful tool that can provide customized support for students with diverse learning needs, tailoring educational experiences to meet student’s individual needs more effectively. However, significant disparities in AI access and digital literacy skills prevent many of these same students from fully leveraging its benefits.

  • futuristic crystal ball with holographic data projections

    Call for Opinions: 2025 Predictions for Higher Ed IT

    How will the technology landscape in higher education change in the coming year? We're inviting our readership to weigh in with their predictions, wishes, or worries for 2025.

  • scene in a cybersecurity operations center, showing an AI and a human competing head-to-head

    91% of CISOs Say AI Will Outperform Security Pros

    A new survey of CISOs by Bugcrowd indicates AI is already beating security pros in some areas and is expected to take on a larger role in the future.