- Thesis:
- Bachelor in Mathematics
- Author:
- Matthias Hericks
- Title:
- Nonparametric regression using ReLU networks
- Supervisors:
- Sandra Schluttenhofer
- Jan JOHANNES
- Abstract:
- This thesis describes in great detail a convergence result of the paper Nonparametric regression using deep neural networks with ReLU activation function by Johannes Schmidt-Hieber from 2020. After introducing the mathematical formulation of neural networks, we consider the statistical reconstruction problem in nonparametric regression. We follow along the lines of the author to derive an Oracle inequality for network esti- mators. Next, we elaborate the approximation theory of network functions. Finally, we explain Schmidt-Hieber’s proof that network estimators can achieve optimal estimation rates (up to log n factors) with input dimension free exponents for regression functions in a large class of composite functions. This result sheds some light on the immense success of neural network estimates in high-dimensional settings.
- References:
- J. Schmidt-Hieber. Nonparametric regression using deep neural networks with relu activation function, The Annals of Statistics, 48(4), 2020.
| Statistics Group | Institute for Mathematics | Faculty of Mathematics and Computer Science | University Heidelberg |