Local Models / Ollama Open-Source Projects

Showing 40 indexable projects from the current snapshot.

• ollama/ollama

  Ollama simplifies downloading and running open large language models locally on macOS, Windows, or Linux without requiring paid services. Users can interact with models via a CLI, a REST API, or client libraries.

• open-webui/open-webui

  A self-hostable web interface for managing and interacting with LLM runners and AI features. It is designed to operate entirely offline and supports various model providers alongside built-in retrieval augmented generation.

• Zackriya-Solutions/meetily

  Meetily is a desktop application that captures, transcribes, and summarizes meeting audio entirely on your local machine. It provides AI-generated meeting notes without sending sensitive audio data to third-party cloud servers.

• vllm-project/vllm

  vLLM provides high-throughput, memory-efficient inference and serving for large language models. It targets developers who need to serve models locally or in self-hosted environments with efficient attention memory management.

• ggml-org/llama.cpp

  llama.cpp is a dependency-free C/C++ library for running large language model (LLM) inference locally or in the cloud. It provides tools for model quantization, grammar-constrained generation, and API serving, requiring models to be in the…

• rasbt/LLMs-from-scratch

  A step-by-step educational repository that teaches developers how to code a GPT-like large language model from the ground up using plain PyTorch. The materials cover the full pipeline from text data preparation to attention mechanisms, pre…

• ChatGPTNextWeb/NextChat

  NextChat is a cross-platform AI chat application (~5MB) that connects to multiple LLM providers while storing data locally in the browser. It requires an external paid API key for functionality but supports one-click deployment via Vercel…

• unslothai/unsloth

  Unsloth provides a local web UI and Python library for running and training large language models with reduced VRAM usage and faster speeds. It supports text, audio, embedding, and vision models, and operates via local or self-hosted execu…

• CorentinJ/Real-Time-Voice-Cloning

  A local SV2TTS toolbox and CLI for cloning a voice from a short audio sample and synthesizing arbitrary text in real time. Adopt it for local experiments or workflows that accept an older voice-cloning approach; do not select it when curre…

• jingyaogong/minimind

  An open-source educational project that enables individuals to train and understand a 64M-parameter large language model from scratch in approximately 2 hours. It provides a full PyTorch-based training pipeline, model architecture code, op…

• ggml-org/whisper.cpp

  ggml-org/whisper.cpp is a plain C/C++ inference library for running OpenAI Whisper automatic speech recognition across supported platforms. It provides a dependency-free implementation with zero runtime memory allocations, voice activity d…

• oobabooga/textgen

  A desktop application for running local large language models privately, without external APIs or cloud services, and with zero telemetry. Users can chat with text and vision models, call custom functions, train LoRAs, generate images, and…

• jamiepine/voicebox

  VoiceBox is a local-first, open-source voice I/O application that combines voice cloning, multi-engine text-to-speech, global dictation, and AI agent output. It runs entirely on your machine, ensuring voice data and captures are never tran…

• janhq/jan

  Jan is a desktop application that lets users download and run large language models locally or connect to cloud providers, maintaining data control without mandatory cloud dependence. It supports text prompts and provides an OpenAI-compati…

• chatboxai/chatbox

  A cross-platform AI copilot that integrates multiple LLM providers while storing user data locally on the device. It runs as a downloadable desktop application on Windows, Mac, and Linux, and requires no coding to set up.

• OpenBMB/VoxCPM

  VoxCPM is a tokenizer-free text-to-speech model that outputs 48kHz audio, supports 30 languages, and creates or clones voices from text descriptions or short reference clips. It is suitable for developers and creators who need self-hosted,…

• mckaywrigley/chatbot-ui

  An open-source AI chat application that connects to any model and can be deployed locally or in the cloud. It requires a Supabase backend for database and authentication and relies on terminal usage for setup and configuration.

• AlexsJones/llmfit

  llmfit detects local RAM, CPU, and GPU/VRAM resources and scores LLM models across memory fit, speed, quality, and context dimensions to recommend matches for specific hardware. It runs as a local CLI or interactive TUI and supports benchm…

• agentscope-ai/QwenPaw

  QwenPaw is a personal AI assistant designed for private deployment across user-owned machines or the cloud, featuring persistent three-layer memory, multi-agent execution, and multi-channel connectivity. It supports local execution via Qwe…

• Fosowl/agenticSeek

  A self-hosted, local AI assistant that browses the web, writes code, and plans multi-step tasks while keeping data on user hardware. It targets developers and general users seeking an alternative to cloud-based autonomous assistants withou…

• mozilla-ai/llamafile

  A single-file executable that packages open large language models and speech-to-text tools to run locally on most operating systems and CPU architectures without installation. It accepts GGUF models and audio files as input, producing gene…

• SYSTRAN/faster-whisper

  Faster Whisper re-implements OpenAI Whisper inference on CTranslate2 to deliver up to 4x faster transcription with lower memory usage on CPU and CUDA GPUs. It supports 8-bit quantization, batched inference, voice activity detection, and wo…

• lyogavin/airllm

  AirLLM enables inference of large language models (up to 70B parameters) on GPUs with as little as 4GB VRAM. It achieves this by loading only one model layer onto the GPU at a time without requiring quantization, which trades memory saving…

• mlc-ai/mlc-llm

  mlc-llm is a library for developing, optimizing, and deploying large language models natively across heterogeneous hardware and software platforms. It combines machine learning compilation with a high-performance deployment engine and expo…

• AiHubCN/Awesome-Chinese-LLM

  A curated catalog of over 100 open-source Chinese large language model resources, aggregating base models, domain-specific fine-tunes, datasets, and tutorials. It serves as a reference index for developers and researchers to locate smaller…

• QwenLM/Qwen

  qwenlm/qwen provides open-source pretrained and chat large language models for natural language processing tasks. The official release offers multiple model sizes and quantized versions to make deployment more accessible.

• AI4Finance-Foundation/FinGPT

  FinGPT provides open-source financial large language models and benchmarks intended for low-cost fine-tuning on financial tasks. It targets AI engineers and researchers who can operate Python tooling and, when running models locally, suppl…

• Alibaba-NLP/DeepResearch

  An open-source agentic language model designed for long-horizon, deep information-seeking tasks across the web. It features a 30.5B parameter mixture-of-experts architecture, activating 3.3B parameters per token, and supports dual inferenc…

• kvcache-ai/ktransformers

  ktransformers enables high-performance CPU-GPU heterogeneous computing for large language models, including MoE architectures, reducing hardware requirements for inference and fine-tuning. It provides kt-kernel serving with AMX/AVX acceler…

• ymcui/Chinese-LLaMA-Alpaca

  chinese-llama-alpaca provides open-source Chinese LLaMA and Alpaca models, training scripts, and local deployment tools to improve Chinese semantic understanding and instruction following.

• elizaOS/eliza

  A local-first, open-source agent operating system that runs entirely on the user's device to unify messaging, personal assistant tasks, browser automation, and cryptocurrency management. It is intended for general users and creators who wa…

• jundot/omlx

  A macOS-native LLM inference server for Apple Silicon that keeps models in memory and manages KV cache across RAM and SSD through a menu bar app and web dashboard. Designed for developers and general users who need local, multi-model servi…

• rowboatlabs/rowboat

  Rowboat is a desktop AI coworker that indexes email, meetings, and conversations into a local knowledge graph and provides built-in surfaces to act on accumulated context. It is designed for users who want long-lived work memory rather tha…

• alibaba/MNN

  MNN is a lightweight deep learning inference and training framework designed to run efficiently on resource-constrained mobile and embedded devices. It provides on-device model deployment capabilities including large language model and sta…

• zai-org/ChatGLM2-6B

  ChatGLM2-6B is an open-source bilingual Chinese-English chat large language model that supports a 32K context length and delivers faster inference with lower GPU memory usage through Multi-Query Attention. It is freely available for commer…

• GaiZhenbiao/ChuanhuChatGPT

  A self-hosted web application providing a graphical interface for interacting with a wide range of large language models (LLMs). It supports features like file-based question answering, agent automation, and multi-user conversation managem…

• n8n-io/self-hosted-ai-starter-kit

  An open-source Docker Compose template that sets up a local, self-hosted AI environment combining n8n, Ollama, Qdrant, and PostgreSQL.

• llmware-ai/llmware

  llmware is a unified framework for building local, private, and secure LLM-based applications, featuring a model catalog of 300+ models and an integrated RAG pipeline. It is optimized for small, specialized models with the smallest possibl…

• NVIDIA/TensorRT-LLM

  tensorrt-llm is a library that optimizes inference for Large Language Models and Visual Generation models to run efficiently on NVIDIA GPUs. It provides custom kernels, an efficient runtime, and a Pythonic API for configuring and extending…

• Lightning-AI/litgpt

  litgpt enables developers to pretrain, finetune, and deploy 20+ high-performance large language models with optimized performance and proven recipes. It provides from-scratch implementations with no heavy abstractions, giving developers fu…