Who this is for: AI developers, deployers, and operators building or managing generative AI systems. GRC architects aligning to NIST AI RMF. Federal contractors referencing OMB AI guidance. FedRAMP-authorized cloud service providers integrating GenAI capabilities. Organizations preparing for EU AI Act compliance using NIST as a bridge framework.
Voluntary but widely adopted. NIST AI 600-1 (published July 26, 2024) is a voluntary profile of the AI Risk Management Framework 1.0 for generative AI. While not legally binding, it is the de facto reference standard for GenAI governance in the US. Federal agencies, FedRAMP authorizers, and an increasing number of state regulators reference the AI RMF and this profile. Over 200 suggested actions organized across Govern, Map, Measure, and Manage.
Contents
1. Overview 2. The 12 GenAI Risk Categories 3. Risk-to-Procedure Mapping 4. Detailed Procedure Cards 5. Quick Reference 6. Quick Start 7. References1. Overview
NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, extends the NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1) with risks and suggested actions specific to generative AI. It identifies 12 risk categories unique to or exacerbated by generative AI and provides over 200 suggested voluntary actions organized across the four core AI RMF functions: Govern, Map, Measure, and Manage.
The profile is designed for three actor types:
- Developers: Organizations that build foundation models, fine-tune models, or create GenAI-powered applications
- Deployers: Organizations that integrate GenAI into products, services, or internal processes
- Users: Individuals and organizations that interact with GenAI outputs
Each suggested action is tagged with applicability to one or more actor types, allowing organizations to focus on the actions relevant to their role in the GenAI value chain.
2. The 12 GenAI Risk Categories
| ID | Risk Category | Description | Trustworthy AI Characteristics Affected |
|---|---|---|---|
| GAI-1 | CBRN Information or Capabilities | Risk that GenAI lowers barriers to accessing information about chemical, biological, radiological, or nuclear weapons | Safe, Secure |
| GAI-2 | Confabulation | Production of confidently stated but erroneous or fabricated content (hallucinations) | Valid and Reliable, Safe |
| GAI-3 | Dangerous, Violent, or Hateful Content | Generation of content that promotes violence, hate, or dangerous activities | Safe, Fair |
| GAI-4 | Data Privacy | Unauthorized disclosure of personal or sensitive information through model outputs | Privacy Enhanced, Secure |
| GAI-5 | Environmental Impacts | High energy consumption and carbon emissions from training and inference | Safe, Accountable |
| GAI-6 | Harmful Bias and Homogenization | Biased outputs or convergence of perspectives around the model's default views | Fair, Valid and Reliable |
| GAI-7 | Human-AI Configuration | Inappropriate reliance on AI, erosion of human oversight, automation bias | Safe, Accountable, Explainable |
| GAI-8 | Information Integrity | Generation of deepfakes, synthetic media, or content that undermines information ecosystems | Valid and Reliable, Fair |
| GAI-9 | Information Security | Novel attack vectors including prompt injection, data poisoning, model extraction, and adversarial inputs | Secure, Valid and Reliable |
| GAI-10 | Intellectual Property | Copyright infringement through training data or outputs that replicate protected works | Fair, Accountable |
| GAI-11 | Obscene, Degrading, or Abusive Content | Generation of sexually explicit, degrading, or abusive material | Safe, Fair |
| GAI-12 | Value Chain and Component Integration | Risks from third-party components, APIs, plugins, and supply chain dependencies in GenAI systems | Secure, Accountable |
3. Risk-to-Procedure Mapping
Each GAI risk category maps to SWT3 witness procedures that produce cryptographically anchored evidence of risk management actions.
| GAI Risk | SWT3 Procedure(s) | What It Witnesses | AI RMF Function |
|---|---|---|---|
| GAI-1 CBRN | AI-DUALUSE.1 | Dual-use capability classification and export control assessment | Map, Manage |
| GAI-2 Confabulation | AI-INF.1, AI-EXPL.2 | Inference provenance with prompt/response hashing; confidence scoring per output | Measure, Manage |
| GAI-3 Content Safety | AI-GRD.1, AI-GRD.2 | Guardrail enforcement verification; content safety filter pass/fail | Manage |
| GAI-4 Data Privacy | AI-DATA.4, AI-CONSENT.1 | PII lifecycle tracking in training data; consent management | Govern, Map |
| GAI-5 Environmental | AI-COST.1 | Resource consumption (compute, energy, carbon) per inference or training run | Map, Measure |
| GAI-6 Harmful Bias | AI-FAIR.1, AI-FAIR.2 | Bias disparity measurement; fairness calibration across demographic groups | Measure, Manage |
| GAI-7 Human-AI Config | AI-HITL.1, AI-HITL.2 | Human review completion; human override event tracking | Govern, Manage |
| GAI-8 Info Integrity | AI-MARK.1, AI-WATERMARK.1 | Content provenance marking (C2PA); watermark verification | Manage |
| GAI-9 Info Security | AI-SEC.1, AI-SEC.2 | Adversarial threat detection; input validation and sanitization | Manage |
| GAI-10 IP | AI-LIC.1, AI-DATA.2 | License provenance for model components; training data license compliance | Govern, Map |
| GAI-11 Obscene Content | AI-REDTEAM.1 | Adversarial red team testing for harmful content generation | Measure |
| GAI-12 Value Chain | AI-SUPPLY.1, AI-SBOM.1 | Supply chain risk assessment; AI bill of materials for component tracking | Govern, Map |
4. Detailed Procedure Cards
CBRN Dual-Use Classification (GAI-1)
NIST AI 600-1 context: GAI-1 addresses the risk that generative AI could lower barriers to accessing information about chemical, biological, radiological, or nuclear weapons. While current studies suggest LLM outputs provide minimal uplift beyond traditional search, the framework emphasizes ongoing monitoring as model capabilities evolve.
How SWT3 addresses it: The witnessDualUse() call records the classification decision for a model or output, including the assessment methodology, export control regime evaluated, and the classification outcome. This creates verifiable proof that dual-use risk was evaluated before deployment or release.
AI-DUALUSE.1 anchors should exist before model deployment anchors (AI-MDL.1). Factor A records the classification category. Factor C records the assessment outcome. For frontier models, cross-reference with AI-REDTEAM.1 to demonstrate adversarial testing specifically targeted CBRN information extraction.
Confabulation Management (GAI-2)
NIST AI 600-1 context: Confabulation (hallucination) is the production of confidently stated but erroneous or fabricated content. NIST identifies this as particularly dangerous in healthcare, legal, and financial contexts. The framework recommends treating confabulation as a calibration problem, measuring the rate of confidently wrong outputs alongside overall accuracy.
How SWT3 addresses it: AI-INF.1 captures prompt/response hashes for every inference, creating a verifiable record of what the model was asked and what it produced. AI-EXPL.2 records the confidence score assigned to each output, enabling post-hoc analysis of outputs that were confidently wrong. Together, these anchors create a confabulation audit trail.
Pair AI-INF.1 and AI-EXPL.2 anchors by timestamp. High-confidence outputs (Factor B in AI-EXPL.2) that were later flagged or corrected indicate confabulation events. The ratio of corrected high-confidence outputs to total outputs is the confabulation rate. Cross-reference with AI-HITL.1 for human review of high-risk outputs.
Content Safety (GAI-3, GAI-11)
NIST AI 600-1 context: GAI-3 and GAI-11 address dangerous, violent, hateful, obscene, degrading, or abusive content. The framework recommends layered safety filters with both input-side and output-side controls, regular red-team testing, and monitoring for content safety filter bypasses.
How SWT3 addresses it: AI-GRD.1 verifies that required safety filters are active and configured correctly before inference. AI-GRD.2 records the output classification result from the content safety filter. Together, they prove that safety controls were both present and functioning for every inference.
AI-GRD.1 PASS anchors must exist before inference begins, proving filters were active. AI-GRD.2 anchors record per-output classification. A FAIL on AI-GRD.2 indicates the safety filter blocked harmful content -- this is the system working correctly. Cross-reference with AI-REDTEAM.1 to show adversarial testing of content safety boundaries.
Harmful Bias and Homogenization (GAI-6)
NIST AI 600-1 context: GAI-6 addresses both biased outputs across demographic groups and the homogenization risk where GenAI narrows the diversity of perspectives in its outputs. The framework recommends diverse evaluation datasets, multiple bias metrics, and diverse model selection to mitigate convergence effects.
How SWT3 addresses it: AI-FAIR.1 measures bias disparity across protected attributes with specific metrics (demographic parity, equalized odds, predictive parity). AI-FAIR.2 records fairness calibration adjustments applied to reduce measured disparities. The pair creates a continuous record of bias measurement and mitigation.
AI-FAIR.1 anchors at regular intervals demonstrate continuous monitoring. Factor A identifies the demographic group. Factor B contains the metric value. When AI-FAIR.1 detects disparity above threshold, a subsequent AI-FAIR.2 anchor should show the calibration action taken. Gaps in AI-FAIR.1 coverage indicate unmonitored periods.
Information Integrity (GAI-8)
NIST AI 600-1 context: GAI-8 addresses deepfakes, synthetic media, and AI-generated content that can undermine information ecosystems. The framework recommends content provenance marking (C2PA), watermarking, and disclosure of AI-generated outputs to end users. This intersects with California SB 942 and EU AI Act Article 50.
How SWT3 addresses it: AI-MARK.1 records the application of content provenance marking (C2PA manifest, metadata injection). AI-WATERMARK.1 verifies that watermarks are detectable in AI-generated outputs. Together, they prove that AI-generated content is traceable to its source.
AI-MARK.1 anchors should exist for every AI-generated output intended for external distribution. Factor A identifies the marking method (C2PA, steganographic, metadata). AI-WATERMARK.1 anchors verify the watermark survived downstream processing. Cross-reference with AI-TRANS.1 to show that disclosure accompanied the content marking.
Value Chain and Component Integration (GAI-12)
NIST AI 600-1 context: GAI-12 addresses risks from third-party components, APIs, plugins, and supply chain dependencies. GenAI systems typically depend on foundation models, embedding services, vector databases, retrieval pipelines, and guardrail services from multiple providers. Each dependency introduces risk that must be tracked and managed.
How SWT3 addresses it: AI-SUPPLY.1 records the supply chain risk assessment for each third-party component. AI-SBOM.1 maintains a machine-readable AI bill of materials listing all components, versions, and provenance. Together, they create a verifiable inventory of the GenAI supply chain.
AI-SBOM.1 should be refreshed whenever dependencies change. AI-SUPPLY.1 anchors should cover every critical component in the SBOM. Factor A in AI-SUPPLY.1 identifies the component assessed. Factor C records the risk level. Cross-reference with AI-ENV.2 for runtime dependency verification.
5. Quick Reference
| Examiner Question | Where to Look |
|---|---|
| Has CBRN dual-use risk been evaluated? | AI-DUALUSE.1 anchor before deployment. Factor A = classification category. Factor C = outcome. |
| What is the confabulation/hallucination rate? | AI-EXPL.2 anchors paired with AI-INF.1. Ratio of corrected high-confidence outputs to total outputs. |
| Are content safety filters active and effective? | AI-GRD.1 PASS before inference. AI-GRD.2 classification results per output. FAIL = blocked content (system working). |
| Is PII protected in training data and outputs? | AI-DATA.4 for training data PII lifecycle. AI-GRD.3 for output-side PII redaction. AI-CONSENT.1 for consent. |
| What is the environmental footprint? | AI-COST.1 anchors. Factor A = compute units. Factor B = energy/carbon measurement. Factor C = efficiency metric. |
| Is bias being monitored across demographics? | AI-FAIR.1 at regular intervals. Factor A = demographic group. Factor B = metric value. AI-FAIR.2 for calibration actions. |
| Is human oversight maintained? | AI-HITL.1 for review completion. AI-HITL.2 for override events. Ratio of reviewed to total decisions shows oversight coverage. |
| Is AI-generated content labeled? | AI-MARK.1 for provenance marking. AI-WATERMARK.1 for watermark verification. Both per output. |
| Are adversarial threats detected and mitigated? | AI-SEC.1 for threat detection. AI-SEC.2 for input validation. AI-REDTEAM.1 for proactive testing. |
| Is the GenAI supply chain tracked? | AI-SBOM.1 for component inventory. AI-SUPPLY.1 for risk assessment per component. AI-LIC.1 for license compliance. |
6. Quick Start
pip install swt3-ai
# Initialize with the GenAI governance profile
swt3 init --profile genai-governance --tenant YOUR_TENANT
# Run the demo to see witness anchors generated
python -m swt3_ai.demo
# Or use TypeScript
npm install @tenova/swt3-ai
npx swt3-init --profile genai-governance
Full SDK documentation: sovereign.tenova.io/docs
Create a free account: sovereign.tenova.io/signup
7. References
- NIST AI 600-1: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, July 2024
- NIST AI Risk Management Framework
- Modulos: NIST AI 600-1 Risk Categories and Operationalization
- DeepInspect: NIST GenAI Profile Overview
- Related: NIST AI RMF CI Operator Guide
- Related: NIST CI AI Profile Crosswalk
- Related: NIST Agentic AI Standards Crosswalk
- SWT3 UCT Registry, 280 procedures across 9 namespaces
- SDK Documentation, Python, TypeScript, and 5 additional language SDKs