llm-council is a self-hosted web application that sends a user's question to multiple LLMs via the OpenRouter API, has the models anonymize and rank each other's answers, and synthesizes a single final response through a designated Chairman model.
Project overview
The anonymized peer-review step prevents models from being biased by each other's identities during ranking, and a designated Chairman LLM compiles all outputs into one answer rather than leaving the user to reconcile multiple responses manually.
Project type
AI Agent
Use cases
Search & Research
Deployment
Refer to project documentation
License
License pending
Best for
General users and developers who want a consolidated answer from multiple LLMs running on their own machine.
Key capabilities
Distributes the user query to multiple configured LLMs individually via OpenRouter and displays their responses in a tab view for side-by-side inspection.
Feeds all individual LLM responses back to each LLM with anonymized identities and asks them to rank the outputs based on accuracy and insight.
Takes all model responses and peer reviews and uses a designated Chairman LLM to compile them into a single final answer presented to the user.
Users can edit a configuration file to customize which specific models participate in the council and which model acts as the Chairman.
Limitations and risks
The project is provided as-is with no intent to improve and is not supported by the author.
Using the project requires purchasing credits or setting up automatic top-up for OpenRouter usage.
Getting started
Install backend dependencies with uv sync, install frontend dependencies with npm install, configure OPENROUTER_API_KEY in .env, run ./start.sh, and open http://localhost:5173. Manual installation of both Python and Node.js dependencies, API-key configuration, and running separate backend and frontend processes contribute to medium setup difficulty.
Evidence and sources
README: This repo is a simple, local web app that essentially looks like ChatGPT except it uses OpenRouter to send your query to multiple LLMs
README: Stage 1: First opinions. The user query is given to all LLMs individually, and the responses are collected. The individual responses are shown in a "tab view", so that the user ca…
README: I'm not going to support it in any way, it's provided here as is for other people's inspiration and I don't intend to improve it.
README: Stage 2: Review. Each individual LLM is given the responses of the other LLMs. Under the hood, the LLM identities are anonymized so that the LLM can't play favorites when judging…
README: Stage 3: Final response. The designated Chairman of the LLM Council takes all of the model's responses and compiles them into a single final answer that is presented to the user.