llama.cpp

LLM inference in plain C/C++ with GGUF quantization, running on CPUs, Apple Silicon and GPUs.