Tech Giants Agree to Put Limits on Gen AI Systems

Sixteen generative AI leaders — including OpenAI, Microsoft, Google, and Anthropic — have agreed to pull the plug on their own AI technologies if they're deemed too dangerous.

The companies are signatories of the "Frontier AI Safety Commitments" document unveiled last week at the AI Seoul Summit. The document, which lays out guidelines for limiting AI misuse, was dubbed a "world first" by the the U.K. government, which co-hosted the summit alongside the Republic of Korea.

The full list of signatories is:

  • Amazon 
  • Anthropic 
  • Cohere 
  • Google/Google DeepMind 
  • G42 
  • IBM 
  • Inflection AI 
  • Meta 
  • Microsoft 
  • Mistral AI 
  • Naver 
  • OpenAI 
  • Samsung Electronics 
  • Technology Innovation Institute 
  • xAI 
  • Zhipu.ai

In the topmost goal of the document, organizations are asked to "effectively identify, assess and manage risks when developing and deploying their frontier AI models and systems."

Many of the signatories already have internal requirements meant to ensure the safety of their AI technologies. OpenAI, for example, unveiled an AI "preparedness framework" last year, though it's still in beta. It also recently formed a new AI Safety and Security Committee, albeit after disbanding its previous AI safety committee.

Microsoft, meanwhile, abides by its Responsible AI Standard developed in 2016. Meta and others are also independently exploring ways to "watermark" content created by their AI systems to limit misinformation, especially in light of this year's elections.

Critically, however, a tenet of this first commitment is that organizations must agree to kill development of AI systems that are beyond saving.

Specifically, they must define "thresholds at which severe risks posed by a model or system, unless adequately mitigated, would be deemed intolerable," and "commit not to develop or deploy a model or system at all, if mitigations cannot be applied to keep risks below the thresholds."

The companies are tasked with defining their kill thresholds over the coming months, with the goal of publishing a formal safety framework in time for the AI Action Summit happening February 2025 in France.

The two other goals outlined in the document are:

  • Organisations are accountable for safely developing and deploying their frontier AI models and systems.
  • Organisations' approaches to frontier AI safety are appropriately transparent to external actors, including governments.

The document also lists several AI safety best practices that the signatories pledge to apply, if they haven't already. These include red-teaming, watermarking, incentivizing third-party testing, creating safeguards against insider threats, and more.

Said U.K. Prime Minister Rishi Sunak, "These commitments ensure the world's leading AI companies will provide transparency and accountability on their plans to develop safe AI." The pledges laid out in the document do not carry legal weight, however; they're described as "voluntary commitments."

About the Author

Gladys Rama (@GladysRama3) is the editorial director of Converge360.

Featured

  • open laptop in a college classroom with holographic AI icons like a brain and data charts rising from the screen

    4 Ways Universities Are Using Google AI Tools for Learning and Administration

    In a recent blog post, Google shared an array of education customer stories, showcasing ways institutions are using AI tools like Gemini and NotebookLM to transform both learning and administrative tasks.

  • illustration of a human head with a glowing neural network in the brain, connected to tech icons on a cool blue-gray background

    Meta Launches Stand-Alone AI App

    Meta Platforms has introduced a stand-alone artificial intelligence app built on its proprietary Llama 4 model, intensifying the competitive race in generative AI alongside OpenAI, Google, Anthropic, and xAI.

  • three main icons—a cloud, a user profile, and a padlock—connected by circuit lines on a blue abstract background

    Report: Identity Has Become a Critical Security Perimeter for Cloud Services

    A new threat landscape report points to new cloud vulnerabilities. According to the 2025 Global Threat Landscape Report from Fortinet, while misconfigured cloud storage buckets were once a prime vector for cybersecurity exploits, other cloud missteps are gaining focus.

  • Stylized illustration showing cybersecurity elements like shields, padlocks, and secure cloud icons on a neutral, minimalist digital background

    Microsoft Announces Security Advancements

    Microsoft has announced major security advancements across its product portfolio and practices. The work is part of its Secure Future Initiative (SFI), a multiyear cybersecurity transformation the company calls the largest engineering project in company history.