ggml: A simple, portable, and efficient tensor library for machine learning.
What problem it solves
Machine-learning applications need tensor operations across different platforms and compute backends while keeping setup and runtime overhead manageable. ggml provides a plain C/C++ tensor library with cross-platform support, multiple execution backends, quantization formats, and no runtime memory allocations.
Key features
Portable C/C++ implementation — The library is implemented in plain C/C++ without dependencies and supports x86, ARM, RISC-V, LoongArch, PowerPC, s390x, and WebAssembly.
Source: README
Multiple compute backends — ggml supports CPU, GPU, NPU, and browser backends, allowing tensor workloads to run in different execution environments.
Source: README
Quantization formats — The library supports integer quantization from 2 to 8 bits as well as MXFP4 and NVFP4 microscaling formats.
Source: README
SIMD-optimized kernels — SIMD-optimized kernels are available for x86, ARM, and RISC-V architectures.
Source: README
Allocation-free runtime — ggml is designed to avoid memory allocations during runtime.
Source: README
Quick start
Copied verbatim from the project's README.
git clone https://github.com/ggml-org/ggml
cd ggml
mkdir build && cd build
cmake ..
cmake --build . --config Release -j 8Who it may suit
This library may suit developers building machine-learning applications that need portable tensor operations and flexible backend support. It may also suit contributors working on tensor-library features, kernels, build configuration, or platform support.
Topics to discuss
Which additional backends or architectures should receive priority for future development?
How can the library make quantization formats easier to adopt in different machine-learning workflows?
What examples would help newcomers understand tensor operations and backend selection?
Written by ReadmeX AI from the project's README, source and public GitHub data; it may contain mistakes.
Read the original README https://github.com/ggml-org/ggml#readme
ReadmeX opened this community for a trending project; it is not run by the project. Maintainers can claim it from the community page.
Comments
…