Langchain-Chatchat is a deployable RAG and Agent application for Chinese scenarios and open-source models. Model inference uses a separately configured framework, which can serve local models or online APIs.
Project overview
It combines knowledge-base RAG, agent tools and Web/API access while explicitly requiring a separate inference framework.
Project type
AI Agent · RAG
Use cases
Chat Assistants · Knowledge Q&A
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Developers evaluating knowledge-base Q&A with Chinese-language scenarios and open-source models.
Key capabilities
Retrieval-Augmented Generation for local knowledge bases using file loading, text splitting, vector embedding, and similarity matching.
Enables model-based tool selection and parameter parsing. Agent behavior depends on the selected model.
Unified file dialogue capability supporting BM25+KNN and other retrieval methods.
Conversational interface to query databases using natural language.
Conversational interface that integrates with search engines.
Streamlit-based Web UI with multi-session support and customizable system prompts.
API service for programmatic access to the application's features.
Tools and commands to initialize and manage local FAISS knowledge bases.
Limitations and risks
No longer directly loads local model paths; it relies on external inference frameworks for model execution.
Default configuration binds to 127.0.0.1, preventing direct access from other IPs without changing settings.
Agent capability depends on the chosen model's inherent capabilities.
High risk of dependency conflicts between Langchain-Chatchat and model inference frameworks if environments are not isolated.
Getting started
Install langchain-chatchat via pip, start an external model inference framework, run chatchat init, initialize the knowledge base with chatchat kb -r, and start all services with chatchat start -a.
Requires modifying YAML configuration files and setting up a separate model inference framework. A GPU is optional as it runs on CPU, GPU, NPU, or MPS hardware.
Evidence and sources
GitHub project description: local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain