# mmBERT Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/mmbert
> Markdown URL: https://aitinkerers.org/technologies/mmbert.md
> Technology record last updated: 2026-06-07T23:44:05Z
> Generated: 2026-09-20T21:48:58Z

mmBERT is an open-source, massively multilingual encoder-only language model trained on 3 trillion tokens across 1,833 languages.

Developed by Johns Hopkins University, mmBERT updates the aging XLM-RoBERTa architecture by bringing modern transformer optimizations to encoder-only models (1.1.1, 1.2.6). Built on the high-performance ModernBERT architecture, it delivers 2 to 4 times faster inference speeds and natively supports an expanded 8,192-token context window (1.1.1, 1.2.4). The core innovation is its annealed language learning training strategy: a three-phase schedule that prevents overfitting on high-resource languages and ensures robust representation for low-resource languages (1.1.1, 1.1.7). This approach makes mmBERT a highly efficient, production-ready standard for multilingual classification, retrieval, and semantic search (1.1.1, 1.2.5).

- Official technology site: https://github.com/JHU-CLSP/mmBERT
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [A Guardrail for the Hardest Conversations: (Bilingual) Youth Crisis Detection](https://montreal.aitinkerers.org/talks/rsvp_noPwfE2FLcI)

A stateful, multi-turn input guardrail that screens an entire sensitive youth mental health conversation arc (not just the latest input) to catch crises that build up gradually ("slow drift"), the failure mode where each turn looks benign but the cumulative trajectory is high-risk. Live, I'll walk through the actual system: the two-stage stack (fine-tuned mmBERT classifier → Cohere c4ai chain-of-thought judge), the 5-question reasoning prompt that made the difference, and the evaluation harness output on a hidden validation set (F1 0.899, recall 0.954 at ~1.16s/sample). I'll show the architecture diagram, the prompt engineering and the red-team CSVs that trained the classifier.

- Event context: AI Tinkerers Montreal - June Demo Meetup — 2026-06-17 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_noPwfE2FLcI

## Related Technologies

- [Cohere c4ai-command-a-03-2025](https://aitinkerers.org/technologies/cohere-c4ai-command-a-03-2025) ([Markdown](https://aitinkerers.org/technologies/cohere-c4ai-command-a-03-2025.md)) — 1 public demo
- [gpt-oss-120b](https://aitinkerers.org/technologies/gpt-oss-120b) ([Markdown](https://aitinkerers.org/technologies/gpt-oss-120b.md)) — 2 public demos
- [Hugging Face Transformers](https://aitinkerers.org/technologies/hugging-face-transformers) ([Markdown](https://aitinkerers.org/technologies/hugging-face-transformers.md)) — 3 public demos
- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 273 public demos