AI Assurance Directory

Promptfoo vs Microsoft PyRIT

Both listings are MIT-licensed open-source generative-AI red-team tools. Promptfoo’s cited docs name 50+ vulnerability types and plugin maps to NIST AI RMF, ISO/IEC 42001 and the EU AI Act. PyRIT’s cited docs name automated and human-led red teaming, Crescendo, TAP, Skeleton Key and CoPyRIT. Not a ranking.

Built only from the cited fields already on Promptfoo and PyRIT. Not a ranking, certification or recommendation. Public summary columns stay on this page. Quoted evidence, who-it-is-for, deployment, caveats and coverage-cell quotes are Pro.

All comparison landings and the picker · Pricing

Side-by-side comparison of Promptfoo and PyRIT from cited public listing fields
Field PromptfooPyRIT
Vendor / publisher Promptfoo Microsoft AI Red Team
Type Open source · AI assurance & verification Open source · AI assurance & verification
Official site https://www.promptfoo.dev/docs/red-team/ https://microsoft.github.io/PyRIT/latest/
Industries (cited page) Cross-industry Cross-industry
Frameworks named on cited source NIST AI RMF, ISO/IEC 42001, EU AI Act None tagged
Open source (OSI license on official repo) Open source · MIT Open source · MIT
Summary (from listing) Open-source gen-AI red-teaming tool that scans 50+ vulnerability types and maps plugins to the NIST AI RMF, ISO/IEC 42001 and EU AI Act. Runs locally against HTTP APIs, browsers or models. MIT-licensed. Microsoft AI Red Team framework for automated and human-led generative-AI red teaming. Python 3.13 recommended; pip install pyrit. Ships Crescendo, TAP and Skeleton Key, CoPyRIT GUI, pyrit_scan/pyrit_shell, and SQLite or Azure SQL memory. MIT-licensed.
Last verified 2026-10-14 2026-10-14
Cited coverage (counts only) Maps to 8 NIST AI RMF functions or subcategories — see which with Pro Maps to 2 EU AI Act articles — see which with Pro No cited coverage cells in this bundle
Analyst assessments (counts only) Analyst assessment: 1 ISO/IEC 42001 clause — see which with Pro Analyst assessment: 1 NIST AI RMF function or subcategory — see which with Pro

Quoted evidence, who-it-is-for, deployment and caveats (Pro)

Included on Pro, Team and Enterprise. Public summaries, tags and official links on this page stay free.

See plans · Sign in

Questions

Which scanner names directory frameworks on the cited page?

Promptfoo. The cited plugin docs name NIST AI RMF, ISO/IEC 42001 and the EU AI Act. The PyRIT listing has no framework tag from the cited Microsoft AI Red Team pages.

What license does each cited page name?

Both listings record MIT. Promptfoo’s official repo is promptfoo/promptfoo. PyRIT’s official repo is Azure/PyRIT.

Does a framework tag mean Promptfoo is certified to those instruments?

No. It means the cited docs name those instruments when describing plugin maps. Cell quotes stay on Pro.