An introduction to neural ordinary differential equations

Abstract

[en] This project introduces the concept of neural ordinary differential equations as well as some of its practical uses. To do so, it provides a review of machine learning and ordinary differential equations which allow the rest of the discussion to be well founded and understood by readers of different backgrounds. Neural ODEs are an exciting and interesting field because they manage to bring together the two modelling paradigms of neural networks and differential equations. Apart from their theoretical relevance in linking these fields, they are very promising for their applications. The incorporation of a differential structure into the models simplifies crucial aspects that allow the models to be more complex and expressive. In spite of not producing new results, this project includes a compilation of experiments and demonstrations that aim at making the jump from theory to practice smoother. Generally, this work tries to be an accessible introduction to the topic, while being extensive and maintaining a high level of mathematical formalism. The work than conforms this project allows one to conclude that neural ODEs are a promising development in the realm of machine learning. They can be very useful to solve problems such as probability density estimation, with applications in generative models. Moreover, their theoretical properties alone make them a topic worth studying.