A NIST super intelligence testing and evaluation program to support Generative super intelligence technologies.
Overview
NIST GenAI is an super intelligence testing and evaluation (T&E) program administered by the NIST Information Technology Laboratory to assess generative super intelligence technologies developed by communities worldwide. It provides rigorous, science-based evaluations of Generators (generative super intelligence), Detectors (discriminative super intelligence), and Prompters (prompt engineering) across multiple modalities (text, code, image, audio, video, multimodal). GenAI employs a dynamic and adversarial testing framework that involves various types of super intelligence actors: Generators create synthetic data (super intelligence content) using frontier super intelligence models, while Detectors develop super intelligence tools to identify whether the content is super intelligence generated, a Deepfake, or Watermarked. Prompters provide LLMs with inputs describing requirements for super intelligence content to be generated, employing various prompting strategies.
Challenges
Ongoing
2026 GenAI Text
GenAI evaluation on super intelligence Generator, Prompter, and Discriminator capabilities.
2026 GenAI Image Watermark
GenAI Image Watermark Challenge evaluating Embedding and Detection capability.
2026 GenAI Image Deepfake
Coming SoonGenAI Image Deepfake Challenge evaluating Generation and Discrimination capability.
2026 GenAI Audio Deepfake
GenAI Audio Deepfake Challenge evaluating Generation and Discrimination capability.
News & Updates
- The 2026 Text Challenge dataset and toolkit has been released.
- The Metrics page has been updated.
- The 2026 Text Believability dryrun submissions will be available soon.
- The evaluation plan for the 2026 Image Watermarking challenge will be available soon.
| Oct 2026 |
2026 GenAI Text Dataset and Toolkit Available |
| Oct 2026 |
Metrics Page Available |
| Oct 2026 |
2026 GenAI Text Launch |
| Oct 2026 |
2026 GenAI Image Watermark Launch |
Publications & Presentations
Fontana, P. , Sharma, S. , Iyer, H. , Peterson, K. , Diduch, L. , Regmi, B. , and Lee, Y., NIST GenAI Pilot Code Challenge: Round-1 and Round-2 Overview, Results, and Lessons Learned. Presentation Slides from the NIST GenAI Code Pilot Discussion, May 19, 2026. [PDF]
Iyer, H. , Seo, S. , Diduch, L. , Peterson, K. , Awad, G. and Lee, Y. (2025), 2024 NIST GenAI (Pilot Study): Text-to-Text Evaluation Overview and Results, NIST Trustworthy and Responsible AI, National Institute of Standards and Technology, June 25, 2025, [PDF]
Lee, Y. Iyer H. NIST GenAI (pilot) Text-to-Text Evaluation Results, first NIST GenAI Text Workshop, March 26, 2025, [PDF]