Choosing AI red-teaming tools for safety-critical programs

The requirements article is AI red-teaming requirements for defense programs. This page is the tool-selection brief for a safety-critical or defense program: what official pages name, which listings actually say “red-teaming,” and what to ask a vendor.

What official pages name

CDAO’s lexicon entry for artificial intelligence red teaming is the Department-facing definition we cite. The T&E of AI Models Framework is the CDAO process frame (model T&E, SI T&E, HSI T&E, OT&E). The December 2024 Generative AI RAI Toolkit is a documentation companion, not a red-team lab. None of those pages names a required commercial product.

NIST ARIA will support three evaluation levels: model testing, red-teaming, and field testing and produce measurements on technical and contextual robustness. NIST AI 700-2 says ARIA performs the Measure function described in the NIST AI Risk Management Framework. Dioptra supports the Measure function of the NIST AI Risk Management Framework and lists red-teaming among its use cases.

Match the tool to the failure class

Listings that name the activity, not a directory guess:

• LLM vulnerability scanning. garak probes for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks, and many other weaknesses.
• Multi-turn generative red-teaming. PyRIT is a flexible, extensible framework for assessing the security and safety of generative AI systems at scale.
• Classical adversarial ML. ART evaluates models against Evasion, Poisoning, Extraction, and Inference.
• Prompt-level Measure subcategories. Promptfoo names MEASURE 2 and related Measure 2.x rows on its cited NIST AI RMF page.
• Reproducible Measure-function test platform. Dioptra is NIST’s own test platform, including the red-teaming use case.
• Program-scale Measure evaluation. ARIA is an evaluation environment, not a product you install as a CI step.

Formal neural-network provers (Marabou, nnenum, NNV, ERAN) are a different class: they prove or refute specified properties. That is formal verification, not red-teaming, unless the vendor page names both.

What to ask on a safety-critical program

1. Which official definition are you using? CDAO lexicon, ARIA’s three levels, or a vendor glossary?
2. Which failure modes are in scope? Prompt injection and jailbreak (garak/PyRIT) are not the same as evasion/poisoning (ART) or property proofs.
3. Where does the evidence go? Measure-function records, a 3000.09 V&V/T&E file, or a CMMC SSP? Those are different instruments. See CDAO T&E and the 3000.09 vendor checklist.
4. Is the run reproducible and scoped to the ODD / intended use? Dioptra emphasizes reproducible, trackable, and reusable AI workflows. A one-off jailbreak screenshot is not that.
5. Who owns residual risk? Red-teaming is Measure/T&E. Deployment go/no-go is Manage or a program-office decision. ARIA says its outputs are for informing decision making regarding positive or negative impacts of AI deployment.

What no cited page makes a clause

We have not found a DFARS clause, a 3000.09 annex, or a CMMC practice that names garak, PyRIT, Dioptra or ARIA as mandatory. The RAI Toolkit remains voluntary. Do not advertise “DoD red-team certified.”

Questions

Does a garak score satisfy CDAO T&E?

No cited page says that. CDAO’s framework names process layers (model / SI / HSI / OT&E). A probe library is evidence you may attach, not the framework.

Is ARIA a product we can run in our enclave?

ARIA is an evaluation environment which is sector and task agnostic. The homepage describes a NIST-run pilot, not an on-prem license. Dioptra is the public test-platform codebase.