US AI SAFETY INSTITUTE RELEASES A DRAFT GUIDANCE AND A TESTING AI PLATFORM

Key Highlights:

Release of NIST AI Safety Institute Guidance and Testing Platform: On July 26, 2024, the National Institute of Standards and Technology (NIST) released draft guidance titled “Managing Misuse Risk for Dual-Use Foundation Models (NIST AI 800-1)” and a testing platform called “Dioptra.” These resources aim to help AI developers mitigate risks from generative AI and dual-use foundation models by measuring vulnerabilities and enhancing system security against attacks.
Public Comment and Finalized Documents: NIST is accepting public comments on the draft guidance until September 9, 2024, and has also finalized two companion resources: the AI RMF Generative AI Profile (NIST AI 600-1) and Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (NIST SP 800-218A). These documents support NIST’s AI Risk Management Framework and Secure Software Development Framework.
Global Engagement on AI Standards: NIST released a finalized version of “A Plan for Global Engagement on AI Standards (NIST AI 100-5),” outlining a strategy for U.S. stakeholders to collaborate internationally on AI standards. This effort is part of the Department of Commerce’s commitment to implementing President Biden’s Executive Order on the Safe, Secure, and Trustworthy Development of AI, promoting international cooperation and innovation.

On July 26, 2024, the Department of Commerce, through the National Institute of Standards and Technology (NIST), publicly released draft guidance “Managing Misuse Risk for Dual-Use Foundation Models (NIST AI 800-1)” from the U.S. AI Safety Institute for the first time. This release included a testing platform known as “Dioptra” designed to help AI system users and developers measure how certain types of attacks can degrade the performance of AI systems. This release follows President Biden’s Executive Order on the Safe, Secure, and Trustworthy Development of AI.

Managing Misuse Risk for Dual-Use Foundation Models (NIST AI 800-1)

The new draft guidance, “Managing Misuse Risk for Dual-Use Foundation Models (NIST AI 800-1),” aims to help software developers reduce risks stemming from generative AI and dual-use foundation models (used for both beneficial and potentially harmful applications) by managing risks associated with the potential misuse of these models, which could lead to harm in various forms such as the development of weapons of mass destruction, enabling cyberattacks, aiding deception, and generating harmful content like child sexual abuse material (CSAM) and non-consensual intimate imagery (NCII).

The draft guidance provides seven key approaches for mitigating the risks of model misuse, along with recommendations for their implementation and transparency about these efforts. These approaches include:

Anticipating potential misuse risk;
Establishing plans for managing misuse risk;
Managing the risks of model theft;
Measuring the risk of misuse;
Ensuring that misuse is managed before deploying foundation models;
Collecting and responding to information about misuse after deployment; and
Providing appropriate transparency about misuse risk.

NIST is accepting public comments on the draft until September 9, 2024, which can be submitted via email to [email protected].

Dioptra

Alongside the guidance, NIST released the test software platform “Dioptra,” which assists AI system users and developers in measuring how different types of attacks can impact AI system performance by helping identify vulnerabilities and enhance system security by testing machine learning models against adversarial attacks and measuring the impact of these attacks on system performance.

AI systems are particularly vulnerable to model tampering, where adversaries poison the training data, causing the model to make incorrect decisions, such as misidentifying road signs. Dioptra, which is available for free on GitHub, addresses this by allowing users to evaluate the resilience of their AI models to such attacks.

Evolution of the Guidance

Previously, on April 29 2024, NIST released two draft guidance documents for public comment: The AI RMF Generative AI Profile (NIST AI 600-1) and the Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (NIST Special Publication (SP) 800-218A). These documents, now finalized, serve as companion resources to NIST’s AI Risk Management Framework (AI RMF) and Secure Software Development Framework (SSDF).

Additionally, NIST released a final version of “A Plan for Global Engagement on AI Standards (NIST AI 100-5),” initially published in draft form on April 29. This publication proposes a strategy for U.S. stakeholders to collaborate internationally on AI standards. The Department of Commerce announced these new guidance documents and tools 270 days after President Biden’s Executive Order on the Safe, Secure, and Trustworthy Development of AI. Laurie E. Locascio, the Under Secretary of Commerce for Standards and Technology and NIST Director, emphasized the importance of these guidance documents and the testing platform in informing software creators about the unique risks of generative AI and helping them develop ways to mitigate these risks while supporting innovation.

The new releases include the initial public draft of “Managing Misuse Risk for Dual-Use Foundation Models” from the U.S. AI Safety Institute and the testing platform Dioptra. These tools aim to help AI developers evaluate and mitigate risks from generative AI and dual-use foundation models, ensuring these technologies are not misused to cause harm. The finalized documents—AI RMF Generative AI Profile (NIST AI 600-1), Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (NIST SP 800-218A), and A Plan for Global Engagement on AI Standards (NIST AI 100-5)—address various aspects of AI risk management, secure software development, and international standards collaboration.

The AI RMF Generative AI Profile helps organizations identify and manage unique risks posed by generative AI, including cybersecurity threats, misinformation, and harmful content generation. The Secure Software Development Practices document expands the SSDF to address concerns specific to generative AI systems, such as training data poisoning. The Global Engagement plan outlines a strategy for U.S. stakeholders to participate in developing international AI standards, promoting cooperation and information sharing.

Secretary of Commerce Gina Raimondo highlighted the department’s efforts to implement the Executive Order on AI and the progress made in providing tools for the safe development and deployment of AI. The USPTO (United States Patents and Trademarks Office) also issued a guidance update on patent subject matter eligibility, addressing AI-related inventions, and NTIA delivered a report on the risks and benefits of large AI models with widely available weights. These efforts demonstrate the U.S. government’s commitment to ensuring the safe, secure, and trustworthy development of AI technology.

References: