Les xarxes neuronals de propagació cap endavant. Una aproximació matemàtica

Abstract

[en] In this work we describe what feedforward neural networks are and how they are used. We explain the elements that make them up: layers, depth, weights, biases, learning rate and activation function.

Then we see that feedforward neural networks are universal approximators of functions under certain conditions. We study two different ways to prove this. On the one hand, the Kolmogorov-Sprecher pathway tells us that feedforward neural networks with three layers, $\mathrm{n}$ components in the first layer, $2 n+1$ nodes in the second layer, and modes in the last layer are universal approximators of continuous functions from $\mathbb{R}^{n}$ a $\mathbb{R}^{m}$ as long as the activation function is monotonically increasing and class $\operatorname{Lip}\left[\frac{\ln 2}{\ln (2 N+2}\right]$. On the other hand, in the second pathway we see that feedforward neural networks are universal approximations of any measurable function as long as the activation function of the neural network is a squashing function.

Finally we explain how to determine the different elements that configure the feedforward neural networks. We define the cost function. We explain that by minimizing the cost function by the stochastic gradient descent method and the learning rate we can calculate the weights and biases. At the end we study different activation functions and see how they affect neural networks also explaining the vanishing gradient.