NIST AI 600-1 (Generative AI Profile)

NIST AI 600-1, the Generative AI Profile, is a companion resource to NIST's AI Risk Management Framework (NIST AI 100-1), published July 26, 2024. It identifies risks that are new to, or worsened by, generative AI, and maps suggested actions onto the framework's four functions: Govern, Map, Measure, Manage. Like the underlying framework, it is voluntary.

The profile organizes those risks into twelve categories:

• CBRN Information or Capabilities: the system makes it easier to access chemical, biological, radiological, or nuclear weapons information.
• Confabulation: the system generates and confidently presents false or fabricated content, sometimes called hallucination.
• Dangerous, Violent, or Hateful Content: outputs that incite, radicalize, threaten, or glorify violence, or that denigrate protected groups.
• Data Privacy: training on personal data, memorized training data resurfacing in outputs, and models inferring sensitive information about individuals.
• Environmental Impacts: the energy and carbon footprint of training, fine-tuning, and running the system.
• Harmful Bias and Homogenization: outputs that amplify bias against protected groups, or that collapse into overly uniform, repetitive content.
• Human-AI Configuration: risks from how people and the system interact, including over-reliance on outputs and unwarranted distrust of them.
• Information Integrity: easier production and spread of misinformation, disinformation, and synthetic media such as deepfakes.
• Information Security: new attack surfaces such as prompt injection and data poisoning, plus the system's use to accelerate offensive cyberattacks.
• Intellectual Property: copyright exposure from training-data memorization, and disputes over AI-generated content and likeness.
• Obscene, Degrading, and/or Abusive Content: the system easing production of illegal or abusive material, including non-consensual imagery.
• Value Chain and Component Integration: risk introduced by third-party datasets, pre-trained models, and libraries that are improperly sourced or vetted.