# Effort Engine Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/effort-engine
> Markdown URL: https://aitinkerers.org/technologies/effort-engine.md
> Technology record last updated: 2026-03-11T20:54:30Z
> Generated: 2026-09-22T18:40:23Z

Effort Engine is a novel, real-time adjustable algorithm (bucketMul) for LLM inference, drastically improving speed on Apple Silicon by dynamically reducing matrix multiplication calculations.

This technology is a new algorithm for Large Language Model (LLM) inference, specifically designed to optimize performance on Apple Silicon chips (Swift & Metal implementation). Effort Engine allows operators to smoothly adjust the 'effort' level—the number of calculations performed—in real time during inference. For example, dropping to 25% effort yields twice the speed of regular matrix multiplications while maintaining most model quality. The system, currently implemented for Mistral, also features dynamic weight loading, letting users skip the least important 10-30% of weights for a quick, ad-hoc distillation effect. This approach bypasses full retraining, focusing on immediate, measurable gains in speed and resource efficiency.

- Official technology site: https://github.com/kolinko/effort
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Speeding up inference with Effort Engine](https://poland.aitinkerers.org/talks/rsvp__2MDM6B1P6s)

Founder of Effort Engine (new algorithm for LLM Inference)

- Event context: AI Tinkerers Poland - First Inaugural Meetup in Warsaw (November) — 2024-11-21 — Poland
- Public talk page: https://poland.aitinkerers.org/talks/rsvp__2MDM6B1P6s

## Related Technologies

- [Algorithm](https://aitinkerers.org/technologies/algorithm) ([Markdown](https://aitinkerers.org/technologies/algorithm.md)) — 2 public demos
- [Inference](https://aitinkerers.org/technologies/inference) ([Markdown](https://aitinkerers.org/technologies/inference.md)) — 9 public demos
- [Optimization](https://aitinkerers.org/technologies/optimization) ([Markdown](https://aitinkerers.org/technologies/optimization.md)) — 1 public demo