PyRIT
Open Sourcetrending #5Red Teaming & Adversarial TestingMicrosoft's Python Risk Identification Toolkit for automated generative-AI red teaming.
Last verified
editorial take
PyRIT is not a scanner — it is the framework Microsoft's AI Red Team built to run adversarial campaigns, open-sourced. Its primitives (orchestrators, converters, scorers, targets) compose into attack strategies: multi-turn persuasion sequences, payload transformations to evade filters, automated scoring of whether the target complied.
That composability is the point and the price. There is no one-command scan; you design attacks in Python. Teams with a real red-team function get a force multiplier proven at Microsoft scale. Teams wanting quick coverage should start with Garak or DeepTeam and graduate to PyRIT when their threat model demands orchestrated depth.
overview
PyRIT (Python Risk Identification Toolkit) is the framework Microsoft's own AI Red Team uses to probe generative AI systems. Rather than a fixed scanner, it is a composable library for building multi-turn attack strategies, scoring model responses, and orchestrating large-scale adversarial campaigns against LLM applications.
- Composable attack orchestrators and converters - Multi-turn adversarial conversation support - Automated scoring of attack success - Battle-tested by Microsoft's internal AI Red Team
core features
- Composable attack orchestrators and converters
- Multi-turn adversarial conversation support
- Automated scoring of attack success
- Battle-tested by Microsoft's internal AI Red Team
target frameworks
faq
PyRIT — frequently asked questions
- When do I graduate from Garak to PyRIT?
- When your question changes from coverage to capability: no longer which attack classes land, but whether an adversary with a concrete goal — extracting the system prompt, exfiltrating customer data through the agent tools — can actually achieve it. That requires multi-turn strategy, response-adaptive attacks, and custom scoring, which is precisely the PyRIT design center.
- Does PyRIT work against non-Azure and non-OpenAI targets?
- Yes. Target abstractions cover arbitrary HTTP endpoints and local models; despite living in the Azure GitHub org, it is vendor-neutral in practice.
- Can PyRIT run in CI/CD?
- It can, with setup — campaigns are Python programs, so they schedule like any test suite. But multi-turn campaigns against production-grade models are slow and cost real tokens; most teams run PyRIT campaigns per release or on a schedule, keeping fast scanners in the commit path.