# PUNICA Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/punica
> Markdown URL: https://aitinkerers.org/technologies/punica.md
> Technology record last updated: 2026-05-22T14:12:21Z
> Generated: 2026-09-22T08:51:43Z

Punica is an open source LLM serving framework that runs multiple LoRA fine-tuned models on a single GPU cluster at the cost of serving just one.

Punica optimizes multi-tenant LLM deployment by consolidating multiple Low-Rank Adaptation (LoRA) models under a single shared base model on a GPU cluster. Using a specialized CUDA kernel design (Segmented Gather Matrix-Vector Multiplication, or SGMV), the system batches GPU operations across different LoRA adapters. This approach eliminates the need to load separate base model weights for every user request, boosting throughput by up to 12x compared to standard serving systems while adding only 2ms of latency per token.

- Official technology site: https://github.com/punica-ai/punica
- Public AI Tinkerers demos and talks: 0
- Result page: 1 of 1

## Recent Public Talks and Demos

No public projects are currently indexed for this technology.