Technology

CIA (Constructive Integer Attention)

CIA replaces standard floating-point attention with integer-only arithmetic to slash memory overhead and latency in LLM inference.

Constructive Integer Attention (CIA) eliminates the precision bottleneck in Transformer models by mapping high-dynamic-range attention scores to 8-bit or 4-bit integers. By utilizing constructive quantization techniques, CIA maintains model accuracy (within 0.1% of FP16 baselines) while enabling hardware-level acceleration on commodity GPUs and edge devices. This approach targets the memory wall, reducing KV cache requirements by up to 50% and accelerating throughput for long-context sequences.

https://arxiv.org/abs/2312.09252

What builders pair with CIA (Constructive Integer Attention)

Projects using both technologies. Select a pairing to see a project.

Built with CIA (Constructive Integer Attention)

256KB RAM 1 shared project ARM Cortex-M0+ 1 shared project C 1 shared project CIFAR-10 1 shared project Custom neural network architecture — no frameworks 1 shared project Integer-only arithmetic 1 shared project

3 more pairings

no floating-point libraries 1 shared project Raspberry Pi Pico 1 shared project TinyLLM 1 shared project

Pairing: 256KB RAM

Photo from the event — Event photo

TinyEye: Image Classification on a $4 Microcontroller — Zero Floating-Point, Zero GPU

Tokyo · February 19, 2026

view project | view 256KB RAM

Recent Talks & Demos

Showing 1-1 of 1

Members-Only

Sign in to see who built these projects

Sign in View FAQ

TinyEye: $4 Microcontroller Integer AI

Raspberry Pi Pico TinyLLM