2025 GenAI Image

Data Generation Instructions

NIST GenAI team created a set of topics using a set of images covering a diverse set of attributes and categories (see sample images in specification plan). The goal of the descriptions and topics is to represent a real-world dataset of persons captured images as uploaded by general internet users. Topics will be distributed by NIST. Only GenAI generator participants who have completed and submitted all required data agreement forms will be allowed access. In total, there will be about  150  topics. As the example below shows, each topic includes an id (num), title, and the required topic prompt ( textual description ). All topics will be aggregated in one master xml file and released to Generator teams. Please check the published schedule for testing data release dates

Example of topic:

                    
                    <topic>
                    <num> topic_5445 </num>
                    <title> Young Man  </title>
                    <prompt> A young Asian man with glasses looking upward to the sky  </prompt>
                    </topic>
                

Submission Guidelines

• Each team may submit up to 10 runs per round. Each run should include a maximum of 10 images per topic. A submission run should be in the format of a single compressed zip file to include the run results xml file (see below for specifications), any sample images used as part of the prompt, and all images generated for all topics. No multiple zip files are allowed per run.
• Please use the following folder naming conventions for your run submission: images_prompts: folder name (as applicable) to contain any images used as part of any topic. Please use the following descriptive file names format to easily identify topics that images have been part of (topic.1.prompt.1.webp, topic.1.prompt.2.webp, etc). images_generated: folder name to contain all images for all topics (e.g. topic.2.image.1.webp, topic.2.image.2.webp, … etc)
• Each run should contain images for all topics; a run can not skip a topic or submit images for a subset of the topics.
• All images must be submitted as webP lossless files, and follow the guidelines for aspect ratios and/or resolution allowed. Please see specification document for allowed values. EXIF metadata must be preserved in the images.
• Please remember to dedicate 1 image out of the 10 to be generated using the NIST-provided textual description. The other 9 images should all be generated using a fixed prompt built automatically (not manually).
• Modified system prompts may include sample images (i.e. final prompt is composed of text + image(s)). If a team decides to add sample images to the prompt, then a maximum number of 2 images should be used per prompt. These images are optional and a team may decide to use text prompts only. To be specific, these image samples should be fixed for all prompts within a topic (a team must use the same 1 or 2 image samples as part of the prompt for all generated images within a topic). For example, given 150 topics, the maximum number of image prompts that a team may use will be 300 images.
• Image content should be free from offensive visuals or inappropriate remarks. Prohibited images include, but are not limited to, any image with explicit nudity, sexual content, hate symbols, child exploitation, or non-consensual images. NIST has the right to exclude any image or whole runs if it determines the content to be inappropriate for the general public. The GenAI team will have a process to automatically detect image content and any flagged images will be reviewed by a team of NIST personnel for final recommendation.
• Each run should include high-level metadata to characterize the generator system as requested by the below run format and DTD file. As explained in the DTD file, teams need to provide some required information/parameters, such as:
  • teamName: The name of the team as registered on the NIST GenAI website
  • trainingData: Name of training dataset or collection of different datasets or source data
• priority: The priority of the submitted run (the lower number, the higher the priority). For any required manual review of submissions, NIST may need to limit effort to only the highest priority runs. Priority should be an integer with a range between 1 to 10
• trained: A boolean (T or F) to indicate if the run was the output of a trained system by the team specifically for this task (T) or the output of an already existing system that the team used to generate the outputs (F)
• desc: A high-level description of the system that generated this run
• link: A link to the model used to generate the run (e.g. GitHub, etc)
• topic: The topic id (the “num” field in the topic xml file)
• elapsedTime: The processing time of the model per topic to generate the image after the topic was presented to it.
• usedImagePrompts: A boolean (T or F) to indicate if a topic has used (T) sample images (up to 2 images) as part of the prompt in addition to text, or did not use any images (F).
• Please use the image file naming convention as demonstrated in the example below (eg. topic.1.image.1.webp, topic.1.image.2.webp, etc) to save and transmit your image files.
• prompt: The text description (prompt) used by the system to generate the corresponding image.
• NIST-prompt: A boolean to indicate if the generated image was based on the official prompt (T), or based on a modified prompt by the team (F)

Example of a sample run:

                    <!DOCTYPE GeneratorResults SYSTEM "GeneratorResults.dtd">
                    <GeneratorResults teamName="participant_1" >
                    <GeneratorRunResult trainingData="OpenAI" priority="1" trained="T" desc="This run uses the top secret x-component" link="TBD">

                    <GeneratorTopicResult topic="1" elapsedTime="5" usedImagePrompts="F">
                    <Image filename="topic.1.image.1.webp" prompt="prompt_NIST_text_description" NIST-prompt="T">
                    <Image filename="topic.1.image.2.webp" prompt="fixed_prompt_1" NIST-prompt="T">
                    <Image filename="topic.1.image.3.webp" prompt="fixed_prompt_1" NIST-prompt="F">
                    <!-- ... -->
                    <!-- ... -->
                    </GeneratorTopicResult>

                    <GeneratorTopicResult topic="2" elapsedTime="5" usedImagePrompts="T">
                    <Image filename="topic.2.image.1.webp" prompt="prompt_NIST_text_description" NIST-prompt="T">
                    <Image filename="topic.2.image.2.webp" prompt="fixed_prompt_2" NIST-prompt="F">
                    <Image filename="topic.2.image.3.webp" prompt="fixed_prompt_2" NIST-prompt="F">
                    <!-- ... -->
                    <!-- ... -->
                    </GeneratorTopicResult>
                    <!-- ... -->

                    <GeneratorTopicResult topic="150" elapsedTime="150" usedImagePrompts="T">
                    <Image filename="topic.150.image.1.webp" prompt="prompt_NIST_text_description" NIST-prompt="T">
                    <Image filename="topic.150.image.2.webp" prompt="fixed_prompt_150" NIST-prompt="F">
                    <Image filename="topic.150.image.3.webp" prompt="fixed_prompt_150" NIST-prompt="F">
                    <!-- ... -->
                    <!-- ... -->
                    </GeneratorTopicResult>
                    </GeneratorRunResult>
                    </GeneratorResults> 

                   

Generator Data submission validation

• NIST will provide, prior to submission dates, a validator script to participants to validate their output XML files format as well as content specific to the task guidelines (e.g. topic ids, empty required attributes, etc). All generator teams should validate their runs before submitting them to NIST. Example of available DTD validators (via a shell script): xmllint --valid simple_sample.xml

• Submission notes:  according to the published schedule , the submission page (form) will be open and available (via the GenAI website) for teams to submit their data outputs. Please make sure to follow the schedule and submit on time as extending the submission dates may not be possible.
• Upon submission, NIST will validate the data outputs uploaded and report any errors to the submitter.
• Please take into consideration that submitting your data outputs indicates and assumes your agreement to the code of conduct & Rules of Behavior .