iFixAI runs up to 45 automated inspections against AI agents to identify operational misalignment risks across five categories, returning a weighted A-F letter grade in under five minutes. It requires external judge models and their associated API costs to execute these evaluations.
Project overview
It provides a rapid, structured scoring mechanism for agent misalignment that relies on cross-vendor judging rather than self-evaluation.
Project type
AI Agent · Evaluation & Observability
Deployment
Refer to project documentation
License
Apache-2.0
Best for
AI engineers, enterprise teams, and developers who need to evaluate real deployed agents with system prompts and tools for operational misalignment risks.
Key capabilities
Executes up to 45 inspections (32 core, 13 extended) across five pillars of misalignment risk: fabrication, manipulation, deception, unpredictability, and opacity.
Produces an A-F letter grade in under 5 minutes based on a weighted average of scored inspection categories.
Provides an interactive arrow-key wizard (`ifixai setup`) to pick provider, model, judge, and suite, saving to a reusable config file.
Judges the system under test using independent providers rather than itself, supporting single or ensemble judging modes.
Writes a manifest of all run inputs so the result can be audited and replayed.
Ships native plugins and a skill scaffolding command to run diagnostics interactively in Claude Code, Codex, Cursor, VS Code, and other agents.
Limitations and risks
It is not a certification or a safety guarantee.
Running full suites incurs third-party API costs for judge models. Operation requires external services and paid APIs.
Getting started
Install via pip using `pip install "ifixai[<provider>]"`, run `ifixai setup` to configure providers and judges via the interactive wizard, and execute `ifixai run` to start an evaluation.
Evidence and sources
GitHub project description: Catch your AI's mistakes and blind spots before your customers or regulators do. iFixAi runs 45 inspections, 32 graded core plus 13 extended for frontier risks like sabotage, sand…
README: The diagnostic for AI operational misalignment
README: iFixAi detects AI operational misalignment before it damages your business. By that, we mean any action, omission, or behaviour from your AI that does not match what your business…
README: It runs up to 45 inspections against your agent, from direct policy compliance to adversarial pressure and structural edge cases. These come in two tiers: 32 core plus 13 extended…
README: pip install "ifixai[openai]" # or anthropic, gemini, etc. — install the provider extra you'll test ifixai setup # arrow-key wizard: pick provider, model, judge, suite → writes ifi…