Blog
Writing
Notes on fine-tuning, distillation, evaluation, and getting useful work out of small models.
SFT, DPO, GRPO: what the post-training alphabet means
The three letters behind every fine-tuning product, explained by what each one actually teaches a model: imitation, preference, and trial-and-error.
How to know your fine-tuned model is any good
A falling loss curve proves almost nothing. What real evaluation of a fine-tuned model looks like: held-out data, judges and their biases, and the comparison that actually matters.
Where fine-tuning data actually comes from
Nobody hand-writes ten thousand training examples. How synthetic data generation works, why diversity is the hard part, and where humans still belong in the loop.
LoRA, QLoRA, or full fine-tuning: how to choose
The three ways to update a model's weights, the intuition behind each, and a decision rule that covers almost every real case.
From documents to a fine-tuned model: what actually happens
The pipeline between 'here are my PDFs' and 'here is my model' has five distinct stages, and most of the quality is decided before training starts.
What fine-tuning actually changes
Fine-tuning doesn't teach a model new facts so much as it reshapes behaviour. Here's a practical way to think about what moves and what doesn't.
Notes on model distillation
A working reference on the main flavours of distillation: what each optimises, what it costs, and when to reach for it.
Why small models are getting good
Compact models keep closing the gap with the giants on narrow tasks. A look at what's driving it, what it changes economically, and where the limits still are.