Garak
Open Sourcetrending #4Red Teaming & Adversarial TestingNVIDIA's open-source LLM vulnerability scanner with a modular probe library for injection, jailbreaks, and leakage.
Last verified
editorial take
Garak is where LLM security testing starts: point it at a model or endpoint, and its probe library systematically attempts injections, jailbreaks, encoding attacks, and leakage elicitation, then reports what landed. The mental model is deliberately nmap-like — reconnaissance and vulnerability scanning, not exploitation frameworks.
Its value is breadth and repeatability: run it on every model version and prompt change, diff the reports, and you have regression tracking for security posture. Its limit is depth — single-turn probes cannot represent a patient multi-turn adversary, which is where PyRIT orchestrated campaigns take over.
overview
Garak is the nmap of LLM security: a command-line scanner that fires a modular library of probes — prompt injection, jailbreak patterns, encoding attacks, data leakage, toxicity elicitation — at any target model or endpoint and reports which ones landed. Maintained by NVIDIA, it supports local and hosted models and runs in CI with modest setup.
- Modular probe and detector architecture - Coverage for OWASP LLM Top 10 attack classes - Works against local models and hosted APIs - Report generation for tracking regressions
core features
- Modular probe and detector architecture
- Coverage for OWASP LLM Top 10 attack classes
- Works against local models and hosted APIs
- Report generation for tracking regressions
target frameworks
compare with
Same-category comparisons with an editorial overview on both sides.
faq
Garak — frequently asked questions
- Garak vs PyRIT — do I need both?
- They answer different questions. Garak asks which known attack classes the model falls for; PyRIT asks whether a strategic adversary can achieve a specific harmful objective against your application. Run Garak routinely for coverage; bring PyRIT for targeted campaigns on high-risk systems. Mature programs run both.
- Can Garak test my application, or only raw models?
- Both — its REST generator targets any HTTP endpoint, so you can scan your full application stack including system prompts and guardrails. Testing through the app is usually more meaningful than testing the base model, since your defenses are part of the attack surface.
- How long does a scan take?
- A full probe sweep can mean thousands of generations — hours against rate-limited APIs. Practical CI usage selects probe subsets relevant to your threat model and runs the full sweep nightly or per release rather than per commit.
