2025 NIST GenAI (Pilot) Evaluation Plan for Image Discriminators

Published

March 14, 2025

Author(s)

George Awad, Hariharan Iyer, Seungmin Seo, Peter Fontana, Yooyoung Lee

Abstract

In this NIST Generative AI (GenAI) program, we invite and encourage participating teams from academia, industry, and other research labs to support research in Generative AI. GenAI is an evaluation series that provides a platform for testing and evaluation to measure the performance of AI content generators (e.g., allies/adversaries) and AI content discriminators (e.g., detectors /defenders). The platform is planned to support multiple modalities and technologies enabled by both sides of the generative spectrum, "generators" and "discriminators." Generator (G) teams will be tested on their system's ability to generate content that is indistinguishable from human-generated content. For the pilot study, the evaluation will help determine the strengths and weaknesses in their approaches, including insights about how and when humans and/or AI can detect AI-generated content. Discriminator (D) teams will be tested on their system's ability to differentiate between AI-generated content and human-generated content. Lessons learned from both sides of teams are expected to help develop future research directions and approaches for understanding cutting-edge technologies as well as sources for recommendations and guidance for responsible use of digital content. The goal of the 2025 GenAI pilot study is to measure how well discriminator systems can automatically detect AI-generated content in multiple modalities, such as text, audio, image, and video. In this document, the task will focus on the image modality only.

Citation

NIST GenAI Program

Pub Weblink

https://ai-challenges.nist.gov/genai

Pub Type

Websites

Download Paper

Local Download

Keywords

Artificial Intelligence (AI), Generative AI, Measurement, Evaluation, Deepfake, Synthetic Content

Citation

Awad, G. , Iyer, H. , Seo, S. , Fontana, P. and Lee, Y. (2025), 2025 NIST GenAI (Pilot) Evaluation Plan for Image Discriminators, NIST GenAI Program, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=959543, https://ai-challenges.nist.gov/genai (Accessed April 27, 2025)