NVIDIA/garak

Project overview

Garak probes LLMs and dialog systems for security weaknesses such as hallucination, data leakage, prompt injection, misinformation, toxicity, and jailbreaks. It scans a configured target model and outputs structured reports detailing detected vulnerabilities.

Project type

Evaluation & Observability

Deployment

Refer to project documentation

License

Apache-2.0

Best for

• AI engineers and developers who need to evaluate LLMs or dialog systems by running targeted probes to uncover undesirable failure modes.

Key capabilities

• Probes large language models and dialog systems for undesirable failure modes, including hallucination, data leakage, prompt injection, misinformation, toxicity, and jailbreaks.
• Allows users to select a target model and execute all known probes by default, or choose specific probe families and individual plugins.
• Generates a JSONL report for each run and maintains a hit log for attempts that successfully yielded vulnerabilities.
• Supplies plugin categories and base classes for building custom probes, detectors, evaluators, generators, and harnesses.

Limitations and risks

• When using the Hugging Face Pipeline API generator route, only generative models function correctly; non-generative models are not supported.

Getting started

• The standard package installation requires a single pip command, but a useful scan requires selecting and configuring a target model. Specific generators, such as the OpenAI generator, require corresponding API keys.

Evidence and sources

• README: `garak` is a command-line tool. It's developed in Linux and OSX.
• README: `garak` checks if an LLM can be made to fail in a way we don't want. `garak` probes for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbre…
• README: `garak` needs to know what model to scan, and by default, it'll try all the probes it knows on that model, using the vulnerability detectors recommended by each probe.
• README: Once generation is complete, a row evaluating that probe's results on each detector is given. If any of the prompt attempts yielded an undesirable behavior, the response will be m…
• README: `garak` generates multiple kinds of log: * A log file, `garak.log`. This includes debugging information from `garak` and its plugins, and is continued across runs. * A report of t…

Reviewed at: 2026-07-18T07:50:17Z