# GGML Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/ggml
> Markdown URL: https://aitinkerers.org/technologies/ggml.md
> Technology record last updated: 2026-03-01T21:38:15Z
> Generated: 2026-09-22T19:40:48Z

GGML is a high-performance C tensor library: it enables large language model (LLM) inference on standard consumer hardware via efficient integer quantization.

GGML (Gerganov's General Machine Learning) is a foundational C library for machine learning, specifically engineered for efficient Transformer inference. The core innovation is its low-level, cross-platform design and integer quantization support, which significantly reduces the memory footprint of large models. This allows models like OpenAI's Whisper and LLaMA to run on CPUs and consumer-grade GPUs, a critical shift for edge AI deployment. It offers various quantization strategies (e.g., 4-bit, 5-bit, 8-bit) and operates with zero third-party dependencies and zero runtime memory allocations, ensuring maximum performance and reliability in projects like `llama.cpp` and `whisper.cpp`.

- Official technology site: https://ggml.ai
- Public AI Tinkerers demos and talks: 2
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Local AI on anything with NobodyWho](https://copenhagen.aitinkerers.org/talks/rsvp_eeot-Z52CbQ)

An open-source library for running local LLMs on any hardware: custom inference engine in Rust with bindings to 6 languages, builds for 7 operating systems, and hardware accelerating on anything.

- Event context: September Demo Night — 2026-09-16 — Copenhagen
- Public talk page: https://copenhagen.aitinkerers.org/talks/rsvp_eeot-Z52CbQ

### [GGML ONNX Runtime](https://toronto.aitinkerers.org/talks/rsvp_YpYHCIhP8gs)

GGML is an open source machine learning library written in C, it powers several popular open-srouce projects like llama.cpp and whisper.cpp which allow you to run state-of-the-art transformer models on consumer hardware. These projects use ggml to convert model weights into memory optimized quantized formats and then load those parameters into computational graphs defined in C / C++. ONNX is a file format for describing a computational graphs along with it's parameters. ONNX graphs can be created automatically from pretrained models defined in PyTorch, Tensorflow, etc and run via and ONNX Runtime implementation. The GGML ONNX Runtime provides an implementation for running ONNX graphs in ggml. This allows you to convert and run pre-trained models without having to define anything in C / C++. The project is still early stage but can run some basic vision encoders which do not have existing ggml implementations.

- Event context: AI Tinkerers Toronto - Inaugural Meetup — 2024-04-11 — Toronto
- Public talk page: https://toronto.aitinkerers.org/talks/rsvp_YpYHCIhP8gs

## Related Technologies

- [GGUF](https://aitinkerers.org/technologies/gguf) ([Markdown](https://aitinkerers.org/technologies/gguf.md)) — 5 public demos
- [llama](https://aitinkerers.org/technologies/llama) ([Markdown](https://aitinkerers.org/technologies/llama.md)) — 48 public demos
- [NobodyWho](https://aitinkerers.org/technologies/nobodywho) ([Markdown](https://aitinkerers.org/technologies/nobodywho.md)) — 1 public demo
- [ONNX](https://aitinkerers.org/technologies/onnx) ([Markdown](https://aitinkerers.org/technologies/onnx.md)) — 83 public demos
- [ONNX Runtime](https://aitinkerers.org/technologies/onnx-runtime) ([Markdown](https://aitinkerers.org/technologies/onnx-runtime.md)) — 5 public demos
- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [Rust](https://aitinkerers.org/technologies/rust) ([Markdown](https://aitinkerers.org/technologies/rust.md)) — 56 public demos
- [Whisper](https://aitinkerers.org/technologies/whisper) ([Markdown](https://aitinkerers.org/technologies/whisper.md)) — 26 public demos