Proximal Algorithms: ISTA and FISTA for L1-Regularized Regression

Abstract

Linear regression models are widely used across fileds like medicine, biology, and economics. This work explores the use of proximal gradient methods, particularly the ISTA and its accelerated version, FISTA, which are simple and efficient algorithms for solving optimization problems with non-differentialble penalties such as L1-norm used in Lasso regression. A package called ProxReg was made to make it easier to use the algorithms. It suports prediction and classification tasks with binary, numeric and multinomial target variables using Lasso regression model. And it also includes Ridge, OLS regression, cross-validation tools, and image reconstruction features. The efficacy and performance of the proposed proximal gradient methods are evaluated by comparing them with the Lasso regression results based on the glmnet package coordinate descent method, using real-world and simulated data.