T1: Fine-Tuning & Model Customization

Duration: 60–90 minutes | Level: Deep-Dive Part of: 🍎 FROOT Transformation Layer Prerequisites: F1 (GenAI Foundations), F2 (LLM Landscape) Last Updated: March 2026

T1.1 The Customization Spectrum

Not every AI problem needs fine-tuning. Understanding the full spectrum of customization — from simple to complex — lets you pick the right tool:

T1.2 When to Fine-Tune (and When Not To)

The Decision Framework

Fine-Tune When

Don't Fine-Tune When

Critical Insight: Fine-tuning teaches the model how to respond, not what to know. For new knowledge, use RAG. For new behavior, use fine-tuning. For both, combine RAG + fine-tuning.

T1.3 Fine-Tuning Methods

Full Fine-Tuning

Updates all model parameters. Most powerful but most expensive.

LoRA (Low-Rank Adaptation)

The most practical fine-tuning technique. Freezes original weights and trains small adapter matrices.

QLoRA (Quantized LoRA)

LoRA applied to a 4-bit quantized base model. The democratizer of fine-tuning.

T1.4 LoRA & QLoRA — The Practical Revolution

Why LoRA Changed Everything

Before LoRA (2023): fine-tuning a 7B model required 4x A100 GPUs ($30K+ in hardware). Fine-tuning 70B? Enterprise-only.

After LoRA: fine-tune 7B on a single consumer GPU. Fine-tune 70B on a single A100. Adapters are 10-100MB (instead of the full model at 14-140GB).

Full Fine-Tuning:  70B × 2 bytes (FP16) = 140 GB VRAM  → Need 32x A100
LoRA:              70B frozen + 0.1B trained = ~140 GB + ~200 MB → Need 4x A100
QLoRA:             70B in 4-bit + 0.1B trained = ~35 GB + ~200 MB → Need 1x A100

LoRA Best Practices

T1.5 Alignment: RLHF & DPO

RLHF (Reinforcement Learning from Human Feedback)

The technique that turned GPT-3 into ChatGPT:

DPO (Direct Preference Optimization)

A simpler alternative that skips the reward model:

Preference Data Format

json{
  "prompt": "Explain what a token is in the context of LLMs.",
  "chosen": "A token is the smallest unit of text that a language model processes. Rather than reading individual characters, models break text into subword units called tokens using algorithms like Byte-Pair Encoding (BPE). For example, 'unbelievable' might become ['un', 'believ', 'able']. In GPT-4, one token averages about 4 English characters. Token count matters because it determines both the cost and the context window usage.",
  "rejected": "A token is basically a word. Like when you type 'hello world' that's 2 tokens. Tokens are used by AI."
}

T1.6 Data Preparation

Data quality is more important than data quantity for fine-tuning. A thousand excellent examples outperform ten thousand mediocre ones.

Data Format (Chat Format)

jsonl{"messages": [{"role": "system", "content": "You are an Azure architect..."}, {"role": "user", "content": "What's the best VM for a SQL workload?"}, {"role": "assistant", "content": "For SQL workloads, I recommend..."}]}
{"messages": [{"role": "system", "content": "You are an Azure architect..."}, {"role": "user", "content": "Explain Azure Front Door."}, {"role": "assistant", "content": "Azure Front Door is a global..."}]}

Data Quality Checklist

How Much Data?

T1.7 The Fine-Tuning Pipeline

T1.8 Evaluation — Did It Work?

Evaluation Framework

A/B Evaluation Pattern

1. Prepare 100 test prompts (not in training data)
2. Generate responses from BOTH base model and fine-tuned model
3. Randomize order (evaluator doesn't know which is which)
4. Expert evaluators score each response 1-5 on:
   - Accuracy
   - Relevance
   - Format compliance
   - Helpfulness
5. Compare aggregate scores
6. Fine-tuned should win on task-specific metrics
   without significant regression on general metrics

T1.9 Azure AI Foundry Fine-Tuning

Azure AI Foundry supports fine-tuning several model families:

Azure Fine-Tuning Flow

Cost Estimation

T1.10 MLOps for LLMs

The LLMOps Lifecycle

Key LLMOps Tools

Key Takeaways

FrootAI T1 — Fine-tuning is the art of teaching a model your language. LoRA made it accessible. QLoRA made it affordable. Good data makes it work.