How LLMs Are Trained After Pretraining: SFT, Reward Models, and RL Without the Alphabet Soup

For a long time I mixed all of this up. Exactly one thing was clear in my head: there's SFT, where you train the model to predict token after token. After that it turned into soup. RLHF, PPO, DPO, RLVR, GRPO, reward model, value function. A pile of three- and four-letter acronyms all orbiting "fine-tuning", and it was never clear which one was a cousin of which, and which was from a different universe entirely.

I'm an ML Engineer. My job isn't to "explain how someone else's paper works", it's to design an architecture for a task, train it, ship it to production, and measure whether the business metric actually moved. From that seat, what I was missing was one simple map: what comes after what, what here is supervised and what is real RL, and why a reward model has nothing to do with RL.

This article is that...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE