Ch 18: Reinforcement Learning Fundamentals - Advanced¶
Track: Advanced | Try code in Playground | Back to chapter overview
Read online or run locally
You can read this content here on the web. To run the code interactively, either use the Playground or clone the repo and open chapters/chapter-18-reinforcement-learning-fundamentals/notebooks/03_policy_gradients.ipynb in Jupyter.
Chapter 18: Reinforcement Learning — Notebook 03 (Policy Gradients & Beyond)¶
Tabular methods break when states explode. Function approximation and policy gradients scale RL to large or continuous problems — the path to DQN, A2C, and PPO.
What you'll learn¶
| Topic | Section |
|---|---|
| Why tables don't scale | §1 |
| Policy gradients and REINFORCE | §2 |
| From DQN to actor-critic | §3 |
Time estimate: 3.5 hours
Key concepts¶
- Function approximation — generalize values/policies with a parametric model.
- Policy gradient — directly increase the probability of high-return actions.
- Actor-critic — combine a learned policy (actor) with a value baseline (critic).
- Exploration at scale — entropy bonuses and noisy nets replace tabular epsilon-greedy.
Policy gradients optimize behaviour directly and scale via function approximation, leading to DQN, actor-critic, and PPO. The tabular intuitions from this chapter underpin all of them.
Run the full notebook in the chapter folder for code and outputs.
Generated by Berta AI