NIST AI Risk Management Framework: Implementation Guide

Overview of the NIST AI RMF

The National Institute of Standards and Technology (NIST) released the AI Risk Management Framework (AI RMF 1.0) in January 2023. The AI RMF is a voluntary framework designed to help organisations manage risks associated with AI systems throughout their lifecycle. It applies to any AI approach and is intended for AI developers, deployers, and evaluators across all sectors.

Framework Structure

The NIST AI RMF is organised around four core functions:

• Govern: Establish policies, processes, and structures for AI risk management
• Map: Identify the context, risks, and potential impacts of an AI system
• Measure: Analyse and track identified AI risks using quantitative or qualitative methods
• Manage: Allocate resources and implement plans to respond to AI risks

Each function contains categories and subcategories providing specific outcomes.

The Govern Function in Detail

The Govern function is foundational, cutting across all other functions. It addresses:

• Policies and procedures for responsible AI development and use
• Accountability structures defining roles and authority for AI risk decisions
• Organisational culture valuing transparency, diversity, and critical thinking
• Stakeholder engagement, including affected communities
• Legal and regulatory compliance monitoring
• Resource allocation for skilled personnel and tools

Organisations with mature governance foundations will find it easier to implement the other functions.

Trustworthy AI Characteristics

The AI RMF identifies seven characteristics of trustworthy AI:

• Valid and Reliable: Performs as intended with consistent results
• Safe: Does not pose unreasonable risk to life, health, or the environment
• Secure and Resilient: Resists attacks, failures, and adversarial conditions
• Accountable and Transparent: Decision processes are documented and understandable
• Explainable and Interpretable: Outputs can be understood by users and affected parties
• Privacy-Enhanced: Protects privacy throughout the lifecycle
• Fair, with Harmful Bias Managed: Avoids unjust discrimination and mitigates harmful biases

These characteristics are interrelated and sometimes involve trade-offs.

AI Risk Taxonomy

The AI RMF Playbook provides a detailed taxonomy of AI risks:

• Data risks: Poor quality, bias in training data, privacy violations, data poisoning
• Model risks: Overfitting, lack of generalisability, adversarial vulnerability, opacity
• Deployment risks: Integration failures, performance degradation, misuse by end users
• Societal risks: Disproportionate impact on vulnerable groups, erosion of trust

Addressing Bias and Fairness

A practical approach includes:

• Data examination: Assess training data for demographic representation and historical biases
• Model evaluation: Test outputs across different demographic groups for disparate performance
• Fairness metrics: Select appropriate metrics recognising that different metrics may conflict
• Mitigation techniques: Apply pre-processing, in-processing, or post-processing methods
• Ongoing monitoring: Track deployed systems for bias drift
• Stakeholder input: Engage affected communities in defining fairness for specific applications

Implementation Approach

1. Establish AI policies, roles, and governance structures
2. Create an AI system inventory with risk classifications
3. Apply the Map function to understand context and identify risks
4. Use the Measure function to assess and quantify risks
5. Apply the Manage function to develop risk response plans
6. Document all activities for transparency and accountability
7. Review and iterate as systems and contexts evolve

The NIST AI RMF complements regulatory frameworks such as the EU AI Act and serves as a practical guide for meeting regulatory expectations. Its flexibility allows proportionate adoption across organisations of all sizes.