Introduction to non-linear least squares

Abstract

Non-linear least squares (NLLS) problems occur whenever a smooth model $r : \mathbb{R}^n \to \mathbb{R}^m$ must be fitted to data by minimizing $f(x) = \tfrac{1}{2} \| r(x) \|_2^2$. Although NLLS is a special case of unconstrained optimization, its Jacobian structure allows algorithms that are faster and more reliable than generic methods.

This thesis reviews, and compares two mainstream approaches as stated by Nocedal \& Wright \cite{4}: (i) Gauss--Newton line-search methods, and (ii) Levenberg--Marquardt trust-region methods. After summarizing the required analysis (first- and second-order conditions, convergence proofs, and regularity assumptions), we study a special case of non-linear least squares when the model involves exponential functions.