NIST Launches Generative AI Testing Program

• By Gladys Rama
• 05/06/24

The National Institute of Standards and Technology (NIST) is taking incremental steps toward establishing a more standardized national approach to AI safety. The government agency has announced the launch of NIST GenAI, described as an "evaluation program to support research in Generative AI technologies."

The launch comes six months after the Biden White House signed an Executive Order requiring LLM makers to implement guardrails around AI technologies that protect the privacy and security of consumer data. For instance, the order mandated the development of "standards, tools, and tests to help ensure that AI systems are safe, secure, and trustworthy," and of "standards and best practices for detecting AI-generated content and authenticating official content."

The NIST GenAI program is part of the department's effort to address those mandates.

A companion NIST program, dubbed Aria, is set to launch soon. Aria's stated goal is "to advance measurement science for safe and trustworthy AI."

In a press release Monday, the U.S. Department of Commerce, of which NIST is part, described the GenAI program as a platform to "evaluate and measure generative AI technologies."

"The NIST GenAI program will issue a series of challenge problems designed to evaluate and measure the capabilities and limitations of generative AI technologies," said the agency. "These evaluations will be used to identify strategies to promote information integrity and guide the safe and responsible use of digital content."

The first of these challenges aims to evaluate the efficacy of text-to-text (T2T) AI models -- those that generate human-like text ("generators"), as well as those that purport to detect AI-generated text ("discriminators"). Findings from the challenge will help guide the NIST's eventual recommendations to LLM makers for how to convey the provenance of content made using their AI systems. This is how NIST describes the challenge in its Overview page:

NIST GenAI T2T is an evaluation series that supports research in Generative AI Text-to-Text modality. Which generative AI models are capable of producing synthetic content that can deceive the best discriminators as well as humans? The performance of generative AI models can be measured by (a) humans and (b) discriminative AI models. To evaluate the "best" generative AI models, we need the most competent humans and discriminators. The most proficient discriminators are those that possess the highest accuracy in detecting the "best" generative AI models. Therefore, it is crucial to evaluate both generative AI models (generators) and discriminative AI models (discriminators).

The challenge is open to academics, researchers and LLM makers; those interested can read the participation guidelines here. A similar challenge to evaluate text-to-image models is set to start soon.

Besides the GenAI program launch, NIST this week released preliminary versions of four papers about the secure development and implementation of AI. These papers, which are described as "initial drafts," are as follows:

• "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile": Identifies 13 potential problems caused by generative AI -- including "easier access to information related to chemical, biological, radiological or nuclear weapons; a lowered barrier to entry for hacking, malware, phishing, and other cybersecurity attacks; and the production of hate speech and toxic, denigrating or stereotyping content" -- and 400 potential solutions for developers to implement against them.
• "Secure Software Development Practices for Generative AI and Dual-Use Foundation Models": Focuses on risks stemming from AI training data and proposes strategies to mitigate them such as "analyzing training data for signs of poisoning, bias, homogeneity and tampering."
• "Reducing Risks Posed by Synthetic Content": Provides guidelines for identifying and labeling AI-generated content, for instance through metadata or watermarks.
• "A Plan for Global Engagement on AI Standards": Provides a case for, and suggestions for implementing, a worldwide framework for AI development, testing and usage.

Each draft is still subject to change based on public input. The NIST is accepting feedback for each publication until June 2. Final versions will be published "later this year," the agency said.