Supervised Fine-Tuning

Supervised Fine-Tuning (SFT) is the process of adapting a general pre-trained language model to a specific, high-value task using a small, high-quality dataset of labeled input-output pairs.

SFT transforms a generalist LLM (like LLaMA 3 or Mistral 7B) into a specialist by training it on a curated, task-specific dataset: a key step in model alignment. The process is straightforward: take a pre-trained model, then update its weights using gradient descent on a dataset of prompt-response examples (e.g., 10,000 instruction-following pairs). This supervised learning approach minimizes the loss between the model's output and the 'correct' labeled output, significantly boosting performance on targeted applications. For example, SFT can tailor a model for generating precise code snippets or summarizing legal documents, delivering high domain-specific accuracy where a base model would fail.

https://huggingface.co/learn/llm-course/en/chat_templates/sft