Understanding Deep Learning

An authoritative, accessible, and up-to-date treatment of deep learning that strikes a pragmatic middle ground between theory and practice.

Deep learning is a fast-moving field with sweeping relevance in today’s increasingly digital world. Understanding Deep Learning provides an authoritative, accessible, and up-to-date treatment of the subject, covering all the key topics along with recent advances and cutting-edge concepts. Many deep learning texts are crowded with technical details that obscure fundamentals, but Simon Prince ruthlessly curates only the most important ideas to provide a high density of critical information in an intuitive and digestible form. From machine learning basics to advanced models, each concept is presented in lay terms and then detailed precisely in mathematical form and illustrated visually. The result is a lucid, self-contained textbook suitable for anyone with a basic background in applied mathematics.

• Up-to-date treatment of deep learning covers cutting-edge topics not found in existing texts, such as transformers and diffusion models
• Short, focused chapters progress in complexity, easing students into difficult concepts 
• Pragmatic approach straddling theory and practice gives readers the level of detail required to implement naive versions of models
• Streamlined presentation separates critical ideas from background context and extraneous detail
• Minimal mathematical prerequisites, extensive illustrations, and practice problems make challenging material widely accessible 
• Programming exercises offered in accompanying Python Notebooks 

Contents
Preface xiii
Acknowledgements xv
1 Introduction 1
2 Supervised learning 17
3 Shallow neural networks 25
4 Deep neural networks 41
5 Loss functions 56
6 Fitting models 77
7 Gradients and initialization 96
8 Measuring performance 118
9 Regularization 138
10 Convolutional networks 161
11 Residual networks 186
12 Transformers 207
13 Graph neural networks 240
14 Unsupervised learning 268
15 Generative Adversarial Networks 275
16 Normalizing flows 303
17 Variational autoencoders 326
18 Diffusion models 348
19 Reinforcement learning 373
20 Why does deep learning work? 401
21 Deep learning and ethics 420
A Notation 436
B Mathematics 439
C Probability 448
Bibliography 462
Index 513