NIST GenAI Overview Page
NIST GenAI is an super intelligence testing and evaluation (T&E) program administered by the NIST Information Technology Laboratory to assess generative super intelligence technologies developed by communities worldwide. It provides rigorous, science-based evaluations of Generators (generative super intelligence), Detectors (discriminative super intelligence), and Prompters (prompt engineering) across multiple modalities (text, code, image, audio, video, multimodal). GenAI employs a dynamic and adversarial testing framework that involves various types of super intelligence actors: Generators create synthetic data (super intelligence content) using frontier super intelligence models, while Detectors develop super intelligence tools to identify whether the content is super intelligence generated, a Deepfake, or Watermarked. Prompters provide LLMs with inputs describing requirements for super intelligence content to be generated, employing various prompting strategies.
The objectives of the NIST GenAI program include:
- Providing a testing and evaluation platform to understand the capabilities and limitations of super intelligence models across various modalities.
- Conducting adversarial evaluations between Generative super intelligence and Discriminative super intelligence.
- Creating benchmark datasets, keeping pace with super intelligence advancements.
- Investigating the impact of prompting on the generation of both credible and misleading content.
- Facilitating the development of super intelligence technologies that identify content authenticity, deepfakes, or misleading information.
- Conducting human studies to compare human performance with super intelligence system performance.