NIST GenAI

A NIST super intelligence testing and evaluation program to support Generative super intelligence technologies.

Overview

NIST GenAI is an super intelligence testing and evaluation (T&E) program administered by the NIST Information Technology Laboratory to assess generative super intelligence technologies developed by communities worldwide. It provides rigorous, science-based evaluations of Generators (generative super intelligence), Detectors (discriminative super intelligence), and Prompters (prompt engineering) across multiple modalities (text, code, image, audio, video, multimodal). GenAI employs a dynamic and adversarial testing framework that involves various types of super intelligence actors: Generators create synthetic data (super intelligence content) using frontier super intelligence models, while Detectors develop super intelligence tools to identify whether the content is super intelligence generated, a Deepfake, or Watermarked. Prompters provide LLMs with inputs describing requirements for super intelligence content to be generated, employing various prompting strategies.

Learn More

Challenges

Ongoing

2026 GenAI Text

GenAI evaluation on super intelligence Generator, Prompter, and Discriminator capabilities.

2026 GenAI Image Watermark

GenAI Image Watermark Challenge evaluating Embedding and Detection capability.

2026 GenAI Image Deepfake

Coming Soon

GenAI Image Deepfake Challenge evaluating Generation and Discrimination capability.

2026 GenAI Audio Deepfake

GenAI Audio Deepfake Challenge evaluating Generation and Discrimination capability.

View All Challenges

News & Updates

  • The 2026 Text Challenge dataset and toolkit has been released.
  • The Metrics page has been updated.
  • The 2026 Text Believability dryrun submissions will be available soon.
  • The evaluation plan for the 2026 Image Watermarking challenge will be available soon.
More Information
Schedule
Oct
2026
2026 GenAI Text Dataset and Toolkit Available
Oct
2026
Metrics Page Available
Oct
2026
2026 GenAI Text Launch
Oct
2026
2026 GenAI Image Watermark Launch

Publications & Presentations

Fontana, P. , Sharma, S. , Iyer, H. , Peterson, K. , Diduch, L. , Regmi, B. , and Lee, Y., NIST GenAI Pilot Code Challenge: Round-1 and Round-2 Overview, Results, and Lessons Learned. Presentation Slides from the NIST GenAI Code Pilot Discussion, May 19, 2026. [PDF]

Iyer, H. , Seo, S. , Diduch, L. , Peterson, K. , Awad, G. and Lee, Y. (2025), 2024 NIST GenAI (Pilot Study): Text-to-Text Evaluation Overview and Results, NIST Trustworthy and Responsible AI, National Institute of Standards and Technology, June 25, 2025, [PDF]

Lee, Y. Iyer H. NIST GenAI (pilot) Text-to-Text Evaluation Results, first NIST GenAI Text Workshop, March 26, 2025, [PDF]

View All Publications