# Transformer models Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/transformer-models
> Markdown URL: https://aitinkerers.org/technologies/transformer-models.md
> Technology record last updated: 2026-03-27T05:02:49Z
> Generated: 2026-10-23T10:33:34Z

A neural network architecture leveraging the self-attention mechanism for parallel processing of sequential data, enabling the development of foundational models (BERT, GPT) that power modern AI.

The Transformer is the foundational neural network architecture for modern AI, introduced in the 2017 paper “Attention Is All You Need.” Its core innovation is the self-attention mechanism: this allows the model to weigh the importance of every input token relative to every other token simultaneously, eliminating the sequential bottlenecks of older Recurrent Neural Networks (RNNs). This parallelization drastically accelerated training on GPUs and TPUs. The architecture underpins all major Large Language Models (LLMs), including the GPT series and BERT, driving state-of-the-art performance across machine translation, text generation, and even computer vision (Vision Transformers, ViT).

- Official technology site: https://huggingface.co/docs/transformers/index
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Speedup ML training with INT8](https://singapore.aitinkerers.org/talks/rsvp_psQ-nCEzNaY)

- Event context: AI Tinkerers Singapore: 3rd Meetup - November 19th, 2024 — 2024-11-19 — Singapore
- Public talk page: https://singapore.aitinkerers.org/talks/rsvp_psQ-nCEzNaY

## Related Technologies

- [bfloat16](https://aitinkerers.org/technologies/bfloat16) ([Markdown](https://aitinkerers.org/technologies/bfloat16.md)) — 1 public demo
- [INT8 Tensor Cores](https://aitinkerers.org/technologies/int8-tensor-cores) ([Markdown](https://aitinkerers.org/technologies/int8-tensor-cores.md)) — 1 public demo
- [NVIDIA A100](https://aitinkerers.org/technologies/nvidia-a100) ([Markdown](https://aitinkerers.org/technologies/nvidia-a100.md)) — 1 public demo
- [NVIDIA GeForce RTX 4090](https://aitinkerers.org/technologies/nvidia-geforce-rtx-4090) ([Markdown](https://aitinkerers.org/technologies/nvidia-geforce-rtx-4090.md)) — 2 public demos