ChatGLM2-6B is an open-source bilingual Chinese-English chat large language model that supports a 32K context length and delivers faster inference with lower GPU memory usage through Multi-Query Attention. It is freely available for commercial use after registration.
Project overview
The model achieves substantial performance gains over its first generation, including a 571% improvement on GSM8K, while significantly reducing inference cost and memory footprint, making high-quality bilingual dialogue more accessible for local deployment.
Project type
Model Runtime
Use cases
Chat Assistants
Deployment
Refer to project documentation
License
License pending
Best for
AI engineers and developers who need a self-hosted, open-source bilingual Chinese-English chat model with efficient inference and long context support.
Key capabilities
A bilingual Chinese-English chat model capable of fluid conversational interactions.
Extends the base model's context length from 2K to 32K, trained with 8K context.
Utilizes Multi-Query Attention to deliver a 42% increase in inference speed and significantly reduce GPU memory usage.
Limitations and risks
The model scale is relatively small, making it susceptible to being misled and unable to guarantee accuracy of outputs.
The project team has not developed any official applications, including web, Android, iOS, or Windows clients, based on ChatGLM2-6B.
Before adoption, be aware that the project does not assume responsibility for risks arising from data security, public opinion, or any model misleading, abuse, dissemination, or improper exploitation.
Influenced by probabilistic randomness, the model's output content cannot be guaranteed to be completely accurate.
Getting started
Setup difficulty is medium: the model is deployable from source with Python, but users need to manage model weights and GPU dependencies. INT4 quantization requires a minimum of 6G VRAM.
Evidence and sources
GitHub project description: ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型