jeinlee1991/chinese-llm-benchmark

Project overview

A Chinese LLM evaluation resource with multi-dimensional leaderboards and a large defect database. It covers commercial and open-source models across education, healthcare, finance, law, reasoning, language, and agent/coding.

Project type

Evaluation & Observability

Use cases

Learning & Education

Deployment

Refer to project documentation

License

License pending

Best for

• Researchers and teams evaluating Chinese LLMs across the documented education, healthcare, finance, law, reasoning, language, and agent/coding domains.
• Users who need a large defect-case collection for LLM behavior research or improvement efforts.

Key capabilities

• Evaluates about 393 commercial and open-source models across seven domains: education, healthcare, finance, law, reasoning, language, and agent/coding, with about 300 fine-grained dimensions.
• Provides more than 2 million defect, or badcase, items for researching and improving LLM behavior.
• Offers free evaluation services for private large models.

Limitations and risks

• The available documentation does not describe how to set up and run the benchmark evaluation suite locally.
• GPU needs, minimum hardware, external services, data boundaries, cost dependencies, coding requirements, and telemetry behavior are not documented.

Getting started

• Documentation does not provide a setup path for running the benchmark evaluation suite locally; setup difficulty is therefore unclear.

Evidence and sources

• README: 目前已囊括393个大模型，覆盖chatgpt、gpt-5.6、谷歌gemini-3.1-pro、Claude-5、grok-4.5、文心ERNIE-X1.1、ERNIE-5.1、qwen3.7-max、qwen3.7-plus、百川、讯飞星火、商汤senseChat等商用模型， 以及hy3、step3.7-flash、kimi-k2.7、ernie4.5、…
• README: [2026/7/12] v5.10.14版本
• Release: v5.10
• README: 上传你的【专属测试数据】📊，5分钟🔍测出哪个模型在你的场景下效果最好🏆、最划算💰！
• README: base_url = "https://api.nonelinear.com/v1"

Reviewed at: 2026-07-15T17:01:18Z