# bfloat16 Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/bfloat16
> Markdown URL: https://aitinkerers.org/technologies/bfloat16.md
> Technology record last updated: 2026-03-27T05:02:49Z
> Generated: 2026-09-21T14:41:59Z

Bfloat16 (Brain Floating Point) is a 16-bit numerical format (8-bit exponent, 7-bit mantissa) engineered by Google Brain to accelerate AI/ML training by preserving FP32's dynamic range.

This 16-bit floating-point format is a performance multiplier for deep learning: it cuts memory usage in half compared to 32-bit float (FP32). Bfloat16's design is strategic, retaining FP32’s crucial 8-bit exponent for a wide dynamic range, which prevents overflow and underflow during training. The trade-off is a reduced 7-bit mantissa, acceptable for most AI workloads where precision is less critical than range. Developed by Google Brain, bfloat16 is now an industry standard, natively supported on major accelerators (Google TPUs, Intel Xeon, NVIDIA GPUs), significantly boosting throughput and enabling larger models and batch sizes.

- Official technology site: https://cloud.google.com/tpu/docs/bfloat16
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Speedup ML training with INT8](https://singapore.aitinkerers.org/talks/rsvp_psQ-nCEzNaY)

This talk explores using INT8 Tensor Cores in recent NVIDIA GPUs to accelerate training of transformer models. Benchmark results show up to 70% speedup on 4090 and 40% speedup on A100 compared to BF16 with minimal accuracy degradation.

- Event context: AI Tinkerers Singapore: 3rd Meetup - November 19th, 2024 — 2024-11-19 — Singapore
- Public talk page: https://singapore.aitinkerers.org/talks/rsvp_psQ-nCEzNaY

## Related Technologies

- [INT8 Tensor Cores](https://aitinkerers.org/technologies/int8-tensor-cores) ([Markdown](https://aitinkerers.org/technologies/int8-tensor-cores.md)) — 1 public demo
- [NVIDIA A100](https://aitinkerers.org/technologies/nvidia-a100) ([Markdown](https://aitinkerers.org/technologies/nvidia-a100.md)) — 1 public demo
- [NVIDIA GeForce RTX 4090](https://aitinkerers.org/technologies/nvidia-geforce-rtx-4090) ([Markdown](https://aitinkerers.org/technologies/nvidia-geforce-rtx-4090.md)) — 2 public demos
- [Transformer models](https://aitinkerers.org/technologies/transformer-models) ([Markdown](https://aitinkerers.org/technologies/transformer-models.md)) — 1 public demo