# quantization Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/quantization
> Markdown URL: https://aitinkerers.org/technologies/quantization.md
> Technology record last updated: 2026-02-22T18:20:26Z
> Generated: 2026-10-25T09:54:04Z

Quantization is a model optimization technique: it converts high-precision parameters (e.g., 32-bit float) to lower-bit formats (e.g., 8-bit integer) to drastically reduce memory footprint and boost inference speed.

Quantization directly addresses the significant resource demands of modern Deep Neural Networks (DNNs) and Large Language Models (LLMs). The core process maps a model’s high-precision weights and activations, typically FP32, down to lower-precision integers, such as INT8 or INT4, achieving up to 4x model compression. This compression is critical: it slashes memory usage, reduces computational cost, and enables high-speed, low-latency inference on resource-constrained hardware (e.g., mobile devices, IoT). Engineers typically employ two main strategies: Post-Training Quantization (PTQ), which converts the model after training, or Quantization-Aware Training (QAT), which simulates the low-precision environment during training for superior accuracy retention.

- Official technology site: https://www.geeksforgeeks.org/what-is-quantization/
- Public AI Tinkerers demos and talks: 3
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Benchmarking Small Language Models Where It Actually Matters](https://paris.aitinkerers.org/talks/rsvp_ow7csxTS5J0)

- Event context: High-Performance Local AI Development: Kick-off ThinkStation PGX — 2026-03-17 — Paris
- Public talk page: https://paris.aitinkerers.org/talks/rsvp_ow7csxTS5J0

### [Transformer Lab](https://waterloo.aitinkerers.org/talks/rsvp_0KXArJU6wf4)

- Event context: AI Tinkerers - Waterloo February Meetup — 2025-02-11 — Waterloo
- Public talk page: https://waterloo.aitinkerers.org/talks/rsvp_0KXArJU6wf4

### [Running quantized LLMs locally](https://boston.aitinkerers.org/talks/rsvp_NYD2oF8oenI)

- Event context: January 2024 Tinkerers Unite! — 2024-01-22 — Boston
- Public talk page: https://boston.aitinkerers.org/talks/rsvp_NYD2oF8oenI

## Related Technologies

- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) — 179 public demos
- [BLOOM](https://aitinkerers.org/technologies/bloom) ([Markdown](https://aitinkerers.org/technologies/bloom.md)) — 115 public demos
- [CUDA](https://aitinkerers.org/technologies/cuda) ([Markdown](https://aitinkerers.org/technologies/cuda.md)) — 15 public demos
- [Docker](https://aitinkerers.org/technologies/docker) ([Markdown](https://aitinkerers.org/technologies/docker.md)) — 151 public demos
- [Dockerfile](https://aitinkerers.org/technologies/dockerfile) ([Markdown](https://aitinkerers.org/technologies/dockerfile.md)) — 1 public demo
- [FastAPI](https://aitinkerers.org/technologies/fastapi) ([Markdown](https://aitinkerers.org/technologies/fastapi.md)) — 189 public demos
- [GPT-3](https://aitinkerers.org/technologies/gpt-3) ([Markdown](https://aitinkerers.org/technologies/gpt-3.md)) — 191 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) — 529 public demos
- [Hugging Face](https://aitinkerers.org/technologies/hugging-face) ([Markdown](https://aitinkerers.org/technologies/hugging-face.md)) — 42 public demos
- [llama](https://aitinkerers.org/technologies/llama) ([Markdown](https://aitinkerers.org/technologies/llama.md)) — 48 public demos
- [Llama-2](https://aitinkerers.org/technologies/llama-2) ([Markdown](https://aitinkerers.org/technologies/llama-2.md)) — 227 public demos
- [Local Inference](https://aitinkerers.org/technologies/local-inference) ([Markdown](https://aitinkerers.org/technologies/local-inference.md)) — 2 public demos
- [Open source models](https://aitinkerers.org/technologies/open-source-models) ([Markdown](https://aitinkerers.org/technologies/open-source-models.md)) — 7 public demos
- [PaLM 2](https://aitinkerers.org/technologies/palm-2) ([Markdown](https://aitinkerers.org/technologies/palm-2.md)) — 116 public demos
- [Polars](https://aitinkerers.org/technologies/polars) ([Markdown](https://aitinkerers.org/technologies/polars.md)) — 2 public demos
- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 686 public demos
- [RAG](https://aitinkerers.org/technologies/rag) ([Markdown](https://aitinkerers.org/technologies/rag.md)) — 148 public demos
- [RoBERTa](https://aitinkerers.org/technologies/roberta) ([Markdown](https://aitinkerers.org/technologies/roberta.md)) — 118 public demos