Garak is a command-line tool that red-teams large language models and dialog systems by probing them for undesirable behaviors such as hallucination, data leakage, prompt injection, and jailbreaks. It runs configurable probes against a selected target model and outputs structured reports documenting detected failures.
Project overview
The tool combines static, dynamic, and adaptive probes to systematically explore LLM and dialog-system failure modes and generate structured vulnerability logs.
Project type
Evaluation & Observability
Deployment
Refer to project documentation
License
Apache-2.0
Best for
AI engineers and developers performing evaluation and observability tasks on text-based LLMs or dialog systems.
Teams that need structured JSONL reports and vulnerability hit logs documenting detected LLM failure modes.
Key capabilities
Probes LLMs and dialog systems for undesirable failure modes including hallucination, data leakage, prompt injection, misinformation, toxicity, and jailbreaks.
Lets users choose a target model and run all known probes by default or select specific probe families and individual plugins.
Creates a JSONL report for each run and a separate hit log for attempts that yielded vulnerabilities.
Provides plugin categories and base classes for probes, detectors, evaluators, generators, and harnesses.
Limitations and risks
For Hugging Face Pipeline API use, only generative models work; non-generative models are not supported by that documented route.
Getting started
Installation is rated as medium difficulty because the standard package installation requires one pip command, but a useful scan requires selecting and configuring a target model. Some target generators require API keys. The tool is operated via a command-line interface.
Alternatives and comparisons
Provides declarative testing, automated evaluations, and LLM red teaming and vulnerability scanning.
An AI engineering platform for LLM observability, evaluations, prompt management, and datasets.
An AI engineering platform with LLM tracing, systematic evaluation, and prompt management.
README: `garak` is a command-line tool. It's developed in Linux and OSX.
README: `garak` checks if an LLM can be made to fail in a way we don't want. `garak` probes for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbre…
README: `garak` needs to know what model to scan, and by default, it'll try all the probes it knows on that model, using the vulnerability detectors recommended by each probe.
README: Once generation is complete, a row evaluating that probe's results on each detector is given. If any of the prompt attempts yielded an undesirable behavior, the response will be m…
README: `garak` generates multiple kinds of log: * A log file, `garak.log`. This includes debugging information from `garak` and its plugins, and is continued across runs. * A report of t…