# MoE Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/moe
> Markdown URL: https://aitinkerers.org/technologies/moe.md
> Technology record last updated: 2026-04-29T10:20:00Z
> Generated: 2026-09-23T01:35:37Z

MoE scales model capacity by activating only a sparse subset of specialized parameters for each input token.

Mixture of Experts (MoE) replaces dense feed-forward layers with a collection of specialized sub-networks (experts) managed by a gating mechanism. This architecture allows models like Mixtral 8x7B or GPT-4 to scale to trillions of parameters while maintaining the inference latency of much smaller models. By routing each token to the top-2 most relevant experts, the system maximizes computational efficiency: it increases total parameter count by 10x or more without a linear increase in FLOPs (floating point operations). This sparse activation strategy is the primary driver for current state-of-the-art performance in large language models.

- Official technology site: https://arxiv.org/abs/1701.06538
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Meet North Mini Code: Cohere's first model for developers.](https://montreal.aitinkerers.org/talks/rsvp_RFfBetDE0zk)

North Mini Code is a 30B parameter MoE coding model (3B active) that Cohere released June 9 under Apache 2.0, trained specifically for agentic software engineering. We will demo it live in OpenCode so you can watch it work through an agentic coding task end to end. Alongside the live agent session, we'll walk through the architecture and the post-training pipeline that got it there: two stages of SFT followed by async RLVR across terminal and SWE environments.

- Event context: AI Tinkerers Montreal - June Demo Meetup — 2026-06-17 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_RFfBetDE0zk

## Related Technologies

- [Agent harness](https://aitinkerers.org/technologies/agent-harness) ([Markdown](https://aitinkerers.org/technologies/agent-harness.md)) — 1 public demo
- [Harbor](https://aitinkerers.org/technologies/harbor) ([Markdown](https://aitinkerers.org/technologies/harbor.md)) — 2 public demos
- [Hugging Face](https://aitinkerers.org/technologies/hugging-face) ([Markdown](https://aitinkerers.org/technologies/hugging-face.md)) — 41 public demos
- [Live Demo](https://aitinkerers.org/technologies/live-demo) ([Markdown](https://aitinkerers.org/technologies/live-demo.md)) — 1 public demo
- [mini-SWE-agent](https://aitinkerers.org/technologies/mini-swe-agent) ([Markdown](https://aitinkerers.org/technologies/mini-swe-agent.md)) — 1 public demo
- [North Mini Code](https://aitinkerers.org/technologies/north-mini-code) ([Markdown](https://aitinkerers.org/technologies/north-mini-code.md)) — 1 public demo
- [North Mini Code (30B MoE, 3B active)](https://aitinkerers.org/technologies/north-mini-code-30b-moe-3b-active) ([Markdown](https://aitinkerers.org/technologies/north-mini-code-30b-moe-3b-active.md)) — 1 public demo
- [OpenCode](https://aitinkerers.org/technologies/opencode) ([Markdown](https://aitinkerers.org/technologies/opencode.md)) — 13 public demos
- [SWE-agent](https://aitinkerers.org/technologies/swe-agent) ([Markdown](https://aitinkerers.org/technologies/swe-agent.md)) — 1 public demo
- [SWE-Bench Verified](https://aitinkerers.org/technologies/swe-bench-verified) ([Markdown](https://aitinkerers.org/technologies/swe-bench-verified.md)) — 1 public demo
- [Terminal-Bench v2](https://aitinkerers.org/technologies/terminal-bench-v2) ([Markdown](https://aitinkerers.org/technologies/terminal-bench-v2.md)) — 1 public demo
- [vLLM](https://aitinkerers.org/technologies/vllm) ([Markdown](https://aitinkerers.org/technologies/vllm.md)) — 33 public demos