NIST AI 600-1: Generative AI Profile

Mitigate confabulation per RISK-02 including: grounding + retrieval-augmented generation (RAG) + citation requirements + uncertainty quantification + benchmarks. Mitigate harmful bias and homogenization per RISK-05 including: representative training data + bias evaluation + diverse evaluation + content diversity controls. Mitigate data privacy per RISK-03 including: training data filtering + de-identification + opt-out + DSAR for AI + GDPR Article 22 + UK ICO AI guidance + EU AI Act compliance. Mitigate information integrity (mis/disinformation) per RISK-07 including: content authenticity + media authentication + provenance. Mitigate IP per RISK-10 including: training data licensing + copyright clearance + opt-out + attribution.

• Grounding + RAG architecture
• Bias audit
• Privacy DPIA
• IP licensing
• Authenticity controls

• No grounding
• Missing bias audit
• No DPIA
• No IP licensing

Comply with NIST AI 600-1 Artificial Intelligence Risk Management Framework Generative AI Profile published 26 July 2024 as a companion to NIST AI RMF 1.0 (NIST AI 100-1). Address 12 categories of risk unique to or exacerbated by Generative AI: (1) CBRN (Chemical + Biological + Radiological + Nuclear) information or capabilities + (2) Confabulation (hallucinations) + (3) Dangerous + Violent + Hateful Content + (4) Data Privacy + (5) Environmental Impacts (energy + water + emissions) + (6) Harmful Bias + Homogenization + (7) Human-AI Configuration + (8) Information Integrity (mis/disinformation) + (9) Information Security (model exfiltration + prompt injection) + (10) Intellectual Property + (11) Obscene + Degrading + Abusive content (including CSAM) + (12) Value Chain and Component Integration.

• 12-category risk register
• NIST AI RMF cross-walk
• Annual risk review
• Senior leader acknowledgement

• Missing risk categories
• No NIST AI RMF cross-walk
• Stale risk review

Establish Generative AI governance per NIST AI 600-1 governance actions including: dedicated GAI governance function with named accountable officer + policies covering data + model + deployment + use cases + risk acceptance and residual risk decisions documented at appropriate level + compliance with NIST AI RMF GOVERN function (GOV-1 through GOV-6) + AI ethics committee + AI inventory + AI use case approval workflow.

• Accountable officer appointment
• GAI policies
• Risk acceptance records
• AI inventory
• Use case approval log
• Ethics committee minutes

• No accountable officer
• Missing GAI policies
• No risk acceptance
• Stale AI inventory

Mitigate CBRN information or capabilities risks per NIST AI 600-1 RISK-01 including: training data filtering for chemical + biological + radiological + nuclear hazardous content + model evaluation against WMDP benchmarks + safety filters at inference + uplift assessment (whether GAI provides meaningful uplift over open-source resources) + coordination with NSA AI Security Center + CISA + ENISA AI guidance + Department of Energy + NIH biosecurity guidance. Address information security risks per RISK-08 including model exfiltration + weights/parameters leakage + prompt injection + indirect prompt injection + training data poisoning + supply chain attacks per OWASP Top 10 for LLMs.

• Training data CBRN filtering
• WMDP evaluation
• Uplift assessment
• NSA AISC liaison
• Model security controls
• OWASP Top 10 LLM compliance

• No CBRN filtering
• No WMDP eval
• No uplift assessment
• Missing OWASP LLM controls

Apply MANAGE function including: content provenance implementation per NIST AI 600-1 MGT-1 (cryptographic watermarking + C2PA Content Credentials + media authentication) + human oversight integration per MGT-2 (human-in-the-loop + human-on-the-loop + human-out-of-the-loop role design) + third-party dependency management per MGT-3 (foundation model providers + API providers + dataset providers + plugin providers) + incident response for GAI per MGT-4 (jailbreak + prompt injection + data leakage + harmful output + IP infringement) + decommissioning procedures per MGT-5.

• C2PA implementation
• Human oversight design
• Third-party register + SLAs
• GAI IR playbook
• Decommissioning procedure
• Annual review

• No content provenance
• Inadequate oversight
• No third-party management
• Missing IR playbook

Apply MAP function per NIST AI RMF 1.0 + NIST AI 600-1 supplements including: pre-deployment evaluation against the 12 GAI risk categories + use case mapping + intended use + foreseeable misuse + stakeholder identification + sociotechnical impact assessment + AI system context + dependencies + supply chain. Document context of use + users + impacted populations + jurisdiction. Conduct Generative AI Impact Assessment (GAIA) for high-risk use cases.

• Use case register with context
• Pre-deployment evaluation reports
• GAIA register
• Stakeholder mapping
• Sociotechnical assessment
• Foreseeable misuse analysis

• No use case register
• Missing pre-deployment evaluation
• No GAIA
• Inadequate stakeholder mapping

Apply MEASURE function including: red teaming and adversarial testing per NIST AI 600-1 MEA-5 (model probing + prompt injection + jailbreaking + data extraction + bias testing) + Test + Evaluation + Verification + Validation (TEVV) frameworks + model evaluation against benchmarks (HELM + MMLU + BIG-Bench + HumanEval + TruthfulQA + WMDP for hazardous knowledge) + fairness metrics (demographic parity + equalised odds + calibration) + robustness testing + watermarking and content provenance per C2PA (Coalition for Content Provenance and Authenticity).

• Red team test plans + reports
• TEVV framework
• Benchmark results
• Fairness audit
• Robustness testing
• C2PA watermarking deployment

• No red teaming
• Missing TEVV
• No fairness audit
• No watermarking

Mitigate dangerous + violent + hateful content per RISK-03 + RISK-12 including: content safety filters + classifier ensembles + human moderation + reportable to NCMEC for CSAM (Risk 11 obscene/degrading/abusive content) per US 18 USC 2258A + UK Online Safety Act + EU DSA Article 16 + India IT Rules. Mitigate environmental impacts per RISK-04 including: energy consumption tracking + carbon emissions measurement + water usage + Green Software Foundation principles + carbon offsetting where appropriate + datacentre PUE optimisation. Coordinate with Value Chain and Component Integration per RISK-11 for supply chain transparency including foundation model providers + datasets + plugins.

• Content safety filters
• NCMEC reporting integration
• Carbon measurement
• PUE tracking
• Foundation model provider register
• Value chain mapping

• Inadequate safety filters
• No NCMEC reporting
• No carbon measurement
• Opaque value chain