# CIA (Constructive Integer Attention) Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/cia-constructive-integer-attention
> Markdown URL: https://aitinkerers.org/technologies/cia-constructive-integer-attention.md
> Technology record last updated: 2026-03-09T20:52:32Z
> Generated: 2026-09-23T00:45:48Z

CIA replaces standard floating-point attention with integer-only arithmetic to slash memory overhead and latency in LLM inference.

Constructive Integer Attention (CIA) eliminates the precision bottleneck in Transformer models by mapping high-dynamic-range attention scores to 8-bit or 4-bit integers. By utilizing constructive quantization techniques, CIA maintains model accuracy (within 0.1% of FP16 baselines) while enabling hardware-level acceleration on commodity GPUs and edge devices. This approach targets the memory wall, reducing KV cache requirements by up to 50% and accelerating throughput for long-context sequences.

- Official technology site: https://arxiv.org/abs/2312.09252
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [TinyEye: Image Classification on a $4 Microcontroller — Zero Floating-Point, Zero GPU](https://tokyo.aitinkerers.org/talks/rsvp_h4mfTwScVjU)

Live demo of CIFAR-10 image classification running entirely on a Raspberry Pi Pico ($4, 256KB RAM, ARM Cortex-M0+) using integer-only arithmetic. No floating-point unit. No GPU. No cloud. The core innovation is CIA (Constructive Integer Attention) — a new mathematical framework that replaces floating-point matrix operations in neural networks with pure integer computation derived from Egyptian fraction decomposition theory. In the demo, we also briefly show CIFAR-10 sample images to the Pico and show real-time classification results on screen. If possible, I also briefly show TinyLLM — a language model generating grammatical English sentences on the same $4 chip — to demonstrate this is a general-purpose inference architecture, not a single-task trick. Technical walkthrough covers: - The CIA architecture that eliminates floating-point entirely (not quantization — the model is natively integer) - Memory layout tricks to fit a working neural network in 256KB **This demo will be presented by two people: Yuichi Suzuki (CEO, BothSides Technology) and Yoshifumi Nagano (team member). Yuichi is recovering from upper arm surgery and being discharged from the hospital on February 19th. He can present and explain the technical details but needs a second person to physically handle the demo hardware. We would appreciate two presenter spots.

- Event context: AI Tinkerers Tokyo - Toranomon Meetup - February 19, 2026 — 2026-02-19 — Tokyo
- Public talk page: https://tokyo.aitinkerers.org/talks/rsvp_h4mfTwScVjU

## Related Technologies

- [256KB RAM](https://aitinkerers.org/technologies/256kb-ram) ([Markdown](https://aitinkerers.org/technologies/256kb-ram.md)) — 1 public demo
- [ARM Cortex-M0+](https://aitinkerers.org/technologies/arm-cortex-m0) ([Markdown](https://aitinkerers.org/technologies/arm-cortex-m0.md)) — 1 public demo
- [C](https://aitinkerers.org/technologies/c-2) ([Markdown](https://aitinkerers.org/technologies/c-2.md)) — 6 public demos
- [CIFAR-10](https://aitinkerers.org/technologies/cifar-10) ([Markdown](https://aitinkerers.org/technologies/cifar-10.md)) — 1 public demo
- [Integer-only arithmetic](https://aitinkerers.org/technologies/integer-only-arithmetic) ([Markdown](https://aitinkerers.org/technologies/integer-only-arithmetic.md)) — 1 public demo
- [no floating-point libraries](https://aitinkerers.org/technologies/no-floating-point-libraries) ([Markdown](https://aitinkerers.org/technologies/no-floating-point-libraries.md)) — 1 public demo
- [Raspberry Pi Pico](https://aitinkerers.org/technologies/raspberry-pi-pico) ([Markdown](https://aitinkerers.org/technologies/raspberry-pi-pico.md)) — 2 public demos
- [TinyLLM](https://aitinkerers.org/technologies/tinyllm) ([Markdown](https://aitinkerers.org/technologies/tinyllm.md)) — 1 public demo