A Mathematical Introduction to Neural Networks

Abstract

In this work, we are going to introduce Neural Networks. First, we are going to give a mathematical formulation of the concept of Neural Networks. Later on, we will examine some important properties of Neural Networks and make a connection to common statistical methods such as Principal Component Analysis and Singular Value Decomposition. In the last chapter, we will give a practical application of a neural network for a regression problem. The concept of a Neural Network is inspired by the activities of a human brain. Neurons receive information, if the information is relevant to the neuron, a signal is sent to other neurons via synapses. The main difference between Neural Networks and rule-based statistical methods, is the learning ability of Neural Networks. At the beginning of a training phase a network has no explicit information. During the training phase the inter-neural connections are changed in a way that the network solves the given problem best. Therefore Neural Networks can provide solutions to a wide spectrum of problems. A Neural Network is an abstract model consisting of one or more layers, that are connected in a certain way. The weighted connection between the layers plays the role of synapses. Each layer consists of units modelling neurons in the human brain. The units carry activation functions, modelling the impulses, that the real neurons send when being triggered. Just like the brain, the network will be trained to learn a specific task and later should perform a similar task in an unknown situation, using the experience that it gained before. For that matter, during the training phase already-observed information is passed to the model and the model produces an output. The output is evaluated on its ability to approximate the observed information. Depending on the result, the model is then changed to improve performance.