ggml is a low-level tensor library for machine learning that provides integer quantization, broad hardware support, automatic differentiation, and optimizers with no third-party dependencies. It is designed to perform tensor operations with zero memory allocations during runtime.
Project overview
ggml combines a C tensor implementation that has no third-party dependencies with integer quantization and zero runtime memory allocations, making it a lightweight foundation for running machine learning workloads across diverse hardware.
Project type
Infrastructure
Deployment
Refer to project documentation
License
MIT
Best for
Developers and AI engineers who need a low-level tensor library with no third-party dependencies, integer quantization, and automatic differentiation for cross-platform machine learning workloads.
Key capabilities
Provides a low-level implementation intended to work across platforms for machine learning tensor operations.
Supports integer quantization for machine learning workloads.
Documents support for a broad range of hardware.
Provides automatic differentiation for machine learning computations.
Includes ADAM and L-BFGS optimizers.
Documents zero memory allocations during runtime.
Ships an example that documents downloading a GPT-2 small 117M model and invoking a GPT-2 backend with a prompt.
Limitations and risks
The project is under active development, so APIs and features may change over time.
Getting started
Clone the repository, create a build directory and configure with CMake, then build the examples with CMake. The documented setup difficulty is medium because it requires cloning the repository and building with CMake. Coding is required to use ggml, which integrates as a library.