Classification vs. instruction finetuning
There are two common ways to finetune an LLM. Instruction-finetuning trains the model to follow natural-language instructions across many tasks (Chapter 7). Classification-finetuning — this chapter — trains it to sort inputs into a fixed set of class labels.
Key concept · Specialist vs. generalist
A classification-finetuned model is a specialist: restricted to the classes it saw in training, it can only output "spam" or "not spam" — nothing else. An instruction-finetuned model is a generalist, handling a broad range of tasks. The trade-off: classification-finetuning needs less data and compute and is easier to build, but its use is confined to those classes; instruction-finetuning is more versatile but far more demanding. Classification-finetuning is ideal for precise categorization jobs — sentiment analysis, spam detection.
Preparing a balanced spam dataset
The example is SMS spam classification (the public SMSSpamCollection, 5,572 messages). Getting the data ready is a three-stage process: dataset prep → model setup → finetuning & use.
Key concept · Balancing, labeling, and splitting
The raw data is imbalanced — 4,825 "ham" vs. only 747 "spam." The book undersamples to create a balanced set (747 of each) so the model can't cheat by always guessing the majority class. The string labels are mapped to integers (ham → 0, spam → 1 — like a tiny 2-token vocabulary), and the data is split 70/10/20 into train / validation / test sets, saved as CSVs.
Key concept · Padding to a uniform length
Unlike Chapter 2's equal-size text chunks, these messages vary in length, so batching needs uniform tensors. Rather than truncating to the shortest message (which loses information), the book pads every message up to the length of the longest one, using the <|endoftext|> token (ID 50256) as the padding token. A SpamDataset tokenizes each message, pads/truncates it, and returns the token-ID tensor plus its integer label — batches then look like 8 × 120 token IDs alongside 8 labels.
Loading the model and swapping its output layer
The model starts as the pretrained GPT-2 from Chapter 5 (it already generates coherent text but "struggles to follow instructions" like a raw spam prompt). To make it a classifier, its output layer is replaced.
Key concept · Replacing the output head
The original GPT output head is a linear layer mapping the 768-dim hidden state to 50,257 logits (one per vocabulary token). For classification you replace it with a much smaller linear layer mapping 768 → num_classes (here 2: not-spam / spam). Note the model keeps everything else — embeddings, all 12 transformer blocks, the final LayerNorm — unchanged; only the last layer is new. (You could use a single output node for binary tasks, but matching output nodes to class count generalizes cleanly to 3+ classes.)
Which layers to train — and which token to read
Two decisions make classification-finetuning efficient and correct: freezing most of the network, and reading the prediction from the right token.
Key concept · Freeze most layers, train a few
Because the pretrained model already understands language, you don't retrain all of it. The book freezes every parameter (requires_grad = False), then makes only a small set trainable: the new classification head, the last transformer block, and the final LayerNorm. The reasoning: lower layers capture general language structure that transfers, while the layers nearest the output handle task-specific nuance. Finetuning only these is computationally cheaper and, in experiments, works about as well as finetuning everything.
Key concept · Classify from the last token
The model outputs a vector for every input position, but for classification you use the output of the last token. Why the last one? Because of causal attention, only the final token has attended to the entire message, so its representation summarizes the whole input. The 2-class logits at that last position become the class scores — apply softmax/argmax to get the predicted label.
Loss, accuracy, and a working classifier
With the model modified, finetuning reuses the machinery from Chapter 5 — but now measuring classification quality.
Key concept · Classification loss and accuracy
The training loss is still cross-entropy, but computed on the 2-class logits of the last token against the integer label (spam / not-spam) — not over the whole vocabulary. Alongside the loss, you track classification accuracy: the fraction of examples whose predicted class (argmax of the two logits) matches the true label — the intuitive, human-readable metric for a classifier. The finetuning loop (epochs → batches → reset → loss → backward → update, with AdamW) drives the loss down and accuracy up, reaching high accuracy on the held-out test set within a few epochs on a laptop.
Key concept · Using the finetuned model
The payoff is a reusable spam classifier: a helper function tokenizes a new message, pads/truncates it to the training length, runs it through the finetuned model, takes the last token's argmax, and returns "spam" or "not spam." The same recipe — swap the head, freeze most layers, train on labeled data — generalizes to any text-classification task (sentiment, topic, and so on). Chapter 7 turns to the other, more general kind of finetuning: following instructions.
Chapter 6 quiz
Fifteen questions on classification-finetuning, dataset prep, the classification head, and evaluation. Answer first, then expand Show answer.
Part A · Multiple choice
A classification-finetuned model is best described as:
- A A specialist restricted to its trained class labels
- B A generalist for any task
- C A text generator
- D A tokenizer
Show answer
A. It can only output the classes it was trained on (e.g. spam / not spam).
Compared to instruction-finetuning, classification-finetuning generally needs:
- A Less data and compute
- B More data and compute
- C No data
- D A larger model
Show answer
A. But its use is confined to the specific classes.
Why balance the spam dataset (equal ham and spam)?
- A So the model can't score well by always guessing the majority class
- B To make it bigger
- C To add more spam
- D It's required by PyTorch
Show answer
A. The book undersamples ham to match the 747 spam messages.
How are messages of varying length made batchable?
- A Pad them to the longest with the <|endoftext|> token (ID 50256)
- B Delete short ones
- C Merge them all
- D Ignore length
Show answer
A. Padding preserves content vs. truncating to the shortest.
To make a spam classifier, the GPT's output head is replaced with a layer mapping:
- A 768 hidden units → 2 classes
- B 2 → 50,257
- C 768 → 50,257
- D 1024 → 768
Show answer
A. num_classes output nodes (2 here).
Before adding the classification head, the book:
- A Freezes all parameters (requires_grad = False)
- B Deletes the transformer blocks
- C Retrains from scratch
- D Doubles the layers
Show answer
A. Then selectively unfreezes a few layers.
Which layers are made trainable for finetuning?
- A The new classification head, the last transformer block, and the final LayerNorm
- B Only the embeddings
- C All layers, always
- D None
Show answer
A. Lower layers capture general language and are left frozen.
Which token's output is used for the classification decision?
- A The last token's
- B The first token's
- C A random token's
- D The average of all tokens
Show answer
A. With causal attention, only the last token has seen the entire message.
Part B · True or false
The pretrained GPT can already reliably classify spam when simply prompted to.
Show answer
False. It struggles to follow such instructions before finetuning — motivating classification-finetuning.
Finetuning only the last few layers is often about as good as finetuning all layers, and cheaper.
Show answer
True. Lower layers' general language features transfer.
Classification accuracy is the fraction of examples whose predicted class matches the true label.
Show answer
True. Computed as argmax of the 2-class logits vs. the label.
The classification loss is computed over the full 50,257-token vocabulary.
Show answer
False. It's cross-entropy over just the 2 class logits of the last token.
Part C · Short answer
What are the two label integers used, and for what?
Show answer
0 for "ham" (not spam) and 1 for "spam" — the string labels are mapped to integers for training.
Why read the prediction from the last token specifically?
Show answer
Because causal (masked) attention means only the final token has attended to the entire message, so its representation best summarizes the whole input.
Summarize the recipe for classification-finetuning a pretrained LLM.
Show answer
Load the pretrained model; replace the output head with a linear layer of num_classes outputs; freeze most layers and unfreeze the last block + final norm + new head; train with cross-entropy on the last token's class logits; evaluate with accuracy.