Ch 19: Model Optimization & Inference - Introduction¶
Track: Advanced | Try code in Playground | Back to chapter overview
Read online or run locally
You can read this content here on the web. To run the code interactively, either use the Playground or clone the repo and open chapters/chapter-19-model-optimization-and-inference/notebooks/01_quantization_pruning.ipynb in Jupyter.
Chapter 19: Model Optimization — Notebook 01 (Quantization & Pruning)¶
The two workhorses of model compression: quantization stores weights in fewer bits, pruning removes them entirely. We implement both and measure the cost.
What you'll learn¶
| Topic | Section |
|---|---|
| Symmetric int8 quantization | §1 |
| Bounding the reconstruction error | §2 |
| Magnitude pruning to a target sparsity | §3 |
| The accuracy/size trade-off | §4 |
Time estimate: 3 hours
Key concepts¶
- Quantization — map float weights to a small integer range via a scale factor.
- Scale —
max|W| / 127for symmetric int8; sets the quantization step. - Pruning — zeroing small weights; sparse models compress and can run faster.
- Round-trip error — bounded by half the quantization step.
Quantization shrinks weights to int8 with error bounded by half the scale step; magnitude pruning removes the least important weights to a chosen sparsity. Both are nearly free accuracy-wise up to a point.
Run the full notebook in the chapter folder for code and outputs.
Generated by Berta AI