Ch 18: Reinforcement Learning Fundamentals - Introduction¶
Track: Advanced | Try code in Playground | Back to chapter overview
Read online or run locally
You can read this content here on the web. To run the code interactively, either use the Playground or clone the repo and open chapters/chapter-18-reinforcement-learning-fundamentals/notebooks/01_mdps_and_values.ipynb in Jupyter.
Chapter 18: Reinforcement Learning — Notebook 01 (MDPs & Value Functions)¶
Reinforcement learning starts with the MDP: states, actions, transitions, rewards, and a discount factor. Value functions and the Bellman equations tie them together.
What you'll learn¶
| Topic | Section |
|---|---|
| States, actions, rewards, transitions, discount | §1 |
| Return and the discount factor | §2 |
| V(s), Q(s,a) and the Bellman equations | §3 |
| Value iteration | §4 |
Time estimate: 4 hours
Key concepts¶
- MDP — the formal model of sequential decisions under uncertainty.
- Discount factor (gamma) — how much future reward is worth now.
- Value function — expected return from a state (or state-action pair).
- Bellman equation — value defined recursively in terms of successor values.
An MDP plus a discount factor defines the objective; value functions and the Bellman equations let us compute or learn optimal behaviour. Value iteration gives a model-based optimum to check against.
Run the full notebook in the chapter folder for code and outputs.
Generated by Berta AI