# ModernBERT Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/modernbert
> Markdown URL: https://aitinkerers.org/technologies/modernbert.md
> Technology record last updated: 2026-06-02T18:45:07Z
> Generated: 2026-09-21T14:42:18Z

ModernBERT is a highly optimized, encoder-only transformer designed by Answer.AI and LightOn to bring BERT-style models into the modern era with native 8192-token context windows and blistering speed.

ModernBERT represents a complete architectural overhaul of the classic encoder-only transformer (originally limited to 512 tokens). Trained on 2 trillion tokens of English and code, this model integrates modern LLM breakthroughs: Rotary Positional Embeddings (RoPE), GeGLU activations, unpadding, and FlashAttention-3. Available in base (149 million parameters) and large (395 million parameters) configurations, it serves as a drop-in replacement for legacy BERT models, delivering up to a 4x speedup on GPUs while setting new performance benchmarks for semantic search, retrieval-augmented generation (RAG) pipelines, and classification tasks.

- Official technology site: https://github.com/AnswerDotAI/ModernBERT
- Public AI Tinkerers demos and talks: 0
- Result page: 1 of 1

## Recent Public Talks and Demos

No public projects are currently indexed for this technology.