# Muon optimizer Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/muon-optimizer
> Markdown URL: https://aitinkerers.org/technologies/muon-optimizer.md
> Technology record last updated: 2026-03-08T02:07:20Z
> Generated: 2026-10-24T04:51:55Z

Muon is a matrix-aware neural network optimizer that leverages Newton–Schulz orthogonalization to deliver state-of-the-art training efficiency for large language models (LLMs), outperforming AdamW.

This is the Muon optimizer: a high-efficiency alternative to AdamW, specifically engineered for the 2D weight matrices in neural network hidden layers. Muon's core mechanism applies a Newton–Schulz orthogonalization to the momentum, effectively optimizing the update direction to stabilize training and ensure better convergence. This matrix-aware approach is crucial for scaling, enabling automatic learning rate transfer (muP scaling) across model widths. It has been validated by setting new speed records for both NanoGPT and CIFAR-10 training, confirming its superior sample efficiency over standard optimizers for large-scale AI workloads. Note: It is typically used in conjunction with AdamW for non-2D parameters (embeddings, biases).

- Official technology site: https://kellerjordan.github.io/posts/muon/
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Speedrunning NanoGPT training runs](https://portland.aitinkerers.org/talks/rsvp_rFDus1GRZYc)

- Event context: AI Tinkerers Portland @Nedspace - October 29th 2024, 6pm — 2024-10-30 — Portland
- Public talk page: https://portland.aitinkerers.org/talks/rsvp_rFDus1GRZYc

## Related Technologies

- [LLM](https://aitinkerers.org/technologies/llm) ([Markdown](https://aitinkerers.org/technologies/llm.md)) — 123 public demos
- [NanoGPT](https://aitinkerers.org/technologies/nanogpt) ([Markdown](https://aitinkerers.org/technologies/nanogpt.md)) — 4 public demos
- [NVIDIA H100](https://aitinkerers.org/technologies/nvidia-h100) ([Markdown](https://aitinkerers.org/technologies/nvidia-h100.md)) — 4 public demos
- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 274 public demos