whichllm identifies which local large language models can run on specific hardware and ranks them using real benchmark scores and recency rather than solely by parameter count. It operates via a command-line interface and includes tools for simulating GPU performance prior to acquisition.
Project overview
The project ranks local models based on merged real benchmark results and model recency rather than relying only on parameter count or VRAM fit.
Project type
Model Runtime
Deployment
Refer to project documentation
License
MIT
Best for
Developers, AI engineers, and general users who need to identify local models that fit specific hardware constraints based on benchmark quality and recency.
Users planning hardware upgrades who need to compare current hardware against candidate GPUs or simulate specific graphics cards before acquisition.
Key capabilities
Automatically detects capabilities for NVIDIA, AMD, Intel, Apple Silicon, and CPU-only hardware.
Scores and ranks models by VRAM fit, speed, and real benchmark quality.
Simulates any GPU before purchase to test which models it can run.
Downloads and starts an isolated chat session with a recommended model instantly.
Prints ready-to-run Python code for running any model via command-line interface.
Compares current machine hardware against candidate GPUs.
Provides reverse lookup to find the GPU needed for a specific model.
Limitations and risks
The system requires Python 3.11 or higher.
The system relies on the HuggingFace API for live model data, resulting in an external optional data boundary.
The documented interaction mode is limited to a command-line interface.
Getting started
Installation difficulty is rated as easy because it is distributed via pip and Homebrew, and can be executed via uvx without a permanent installation. The documented first success path is running 'uvx whichllm@latest'.
Alternatives and comparisons
Running and managing large language models locally is complex and difficult to set up. Provides a simple way to get up and running with open models locally.
Enabling LLM inference with minimal setup and high performance across a wide range of hardware, locally and in the cloud. Provides state-of-the-art LLM inference performance via a plain C/C++ implementation without any dependencies.
Providing a user-friendly, self-hostable, and offline-capable web interface to manage and interact with various LLM runners and AI features. An extensible, feature-rich self-hosted AI platform designed to operate entirely offline.