# MT-Bench Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/mt-bench
> Markdown URL: https://aitinkerers.org/technologies/mt-bench.md
> Technology record last updated: 2026-03-20T18:35:58Z
> Generated: 2026-09-21T17:34:01Z

A high-quality multi-turn benchmark that uses GPT-4 as a judge to evaluate how large language models handle complex, conversational instructions.

MT-Bench consists of 80 high-quality multi-turn questions across eight categories: writing, roleplay, extraction, reasoning, math, coding, knowledge (STEM), and knowledge (humanities). Developed by the LMSYS Org team (the creators of Chatbot Arena), it measures a model's ability to maintain coherence and follow instructions over two-step interactions. The framework utilizes a curated set of prompts and an automated LLM-as-a-judge system (typically GPT-4) to provide scalable, human-aligned scoring. This approach offers a 0.8 correlation with human preferences while significantly reducing the time and cost required for traditional manual evaluation.

- Official technology site: https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge
- Public AI Tinkerers demos and talks: 0
- Result page: 1 of 1

## Recent Public Talks and Demos

No public projects are currently indexed for this technology.