---
title: Post Training Techniques Guide
emoji: 🚀
colorFrom: purple
colorTo: yellow
sdk: docker
app_port: 8501
tags:
- streamlit
pinned: false
short_description: A visual guide to post-training techniques for LLMs
license: mit
---

# 🔧 Beyond Pretraining: A Visual Guide to Post-Training Techniques for LLMs

This deck summarizes the key trade-offs between different post-training strategies for large language models — including:

- 📚 Supervised Fine-Tuning (SFT)
- 🤝 Preference Optimization (DPO, APO, GRPO)
- 🎯 Reinforcement Learning (PPO)

It also introduces a reward spectrum from rule-based to subjective feedback, and compares how real-world models like **SmolLM3**, **Tulu 2/3**, and **DeepSeek-R1** implement these strategies.

> This is a companion resource to my ReTool rollout implementation and blog post.
>  
> 📖 [Medium blog post](https://medium.com/@jenwei0312/beyond-generate-a-deep-dive-into-stateful-multi-turn-llm-rollouts-for-tool-use-336b00c99ac0)  
> 💻 [ReTool Hugging Face Space](https://huggingface.co/spaces/bird-of-paradise/ReTool-Implementation)  

---

### 📎 Download the Slides
👉 [PDF version](https://huggingface.co/spaces/bird-of-paradise/post-training-techniques-guide/blob/main/src/Post%20Training%20Techniques.pdf)

---

### 🤝 Reuse & Attribution

This deck is free to share in talks, posts, or documentation — **with attribution**.

Please credit:
**Jen Wei — [Hugging Face 🤗](https://huggingface.co/bird-of-paradise) | [X/Twitter](https://x.com/JenniferWe17599)**  
Optional citation: *“Beyond Pretraining: Post-Training Techniques for LLMs (2025)”*

Licensed under MIT License.

—
*Made with 🧠 by Jen Wei, July 2025*