# entropix Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/entropix
> Markdown URL: https://aitinkerers.org/technologies/entropix.md
> Technology record last updated: 2026-05-05T02:41:29Z
> Generated: 2026-09-20T15:43:48Z

Entropix is a novel, open-source sampling algorithm that uses entropy and varentropy to enhance Large Language Model (LLM) reasoning and drastically reduce hallucinations.

This technology is an advanced LLM sampling method designed to improve reasoning without retraining or fine-tuning the base model. Entropix leverages two key metrics—entropy and varentropy—to gauge a model's uncertainty at the token level. By dynamically adjusting the sampling strategy (like temperature, top-k, and top-p) based on these uncertainty signals, it encourages deeper, context-aware reasoning. This approach allows smaller models (e.g., 1B parameter count) to exhibit complex problem-solving capabilities, simulating Chain-of-Thought (CoT) or o1-like reasoning with only inference-time compute. The project is open-source, with a primary repository on GitHub for both large-scale and local research applications.

- Official technology site: https://github.com/xjdr-alt/entropix
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Molecular structure elucidation, Latent test-time compute and a different way of doing Entropix.](https://milan.aitinkerers.org/talks/rsvp_NmpNgyS3NKQ)

Entropix took the twitter ML world by storm just a few months ago. But now that test-time compute is hot in token space via RL-finetuning, entropix may look like arelic from the past. I present a specific instance of autoregressive transformers where the output space is fragile aenough not to allow exploration in token space, an hence utilize model output distribution uncertainty, in the shape of entropy and varentropy, to allocate test-time compute and significantly improve performance on a moleculare structure elucidation task via SMILES prediction.

- Event context: AI Tinkerers Milan - May 8, 2025 — 2025-05-08 — Milan
- Public talk page: https://milan.aitinkerers.org/talks/rsvp_NmpNgyS3NKQ

## Related Technologies

- [decoder-only](https://aitinkerers.org/technologies/decoder-only) ([Markdown](https://aitinkerers.org/technologies/decoder-only.md)) — 1 public demo
- [layer looping](https://aitinkerers.org/technologies/layer-looping) ([Markdown](https://aitinkerers.org/technologies/layer-looping.md)) — 1 public demo
- [Muon](https://aitinkerers.org/technologies/muon) ([Markdown](https://aitinkerers.org/technologies/muon.md)) — 1 public demo
- [Transformer](https://aitinkerers.org/technologies/transformer) ([Markdown](https://aitinkerers.org/technologies/transformer.md)) — 11 public demos