Exploring self-supervised learning in deep learning for food recognition

Abstract

[en] Deep Learning is a field that evolves at dizzying rates and that allows obtaining extraordinary results in many applications of scientific and, even, everyday scope. In the field of computer vision, and especially in the food image recognition task, Machine Learning allows us to achieve accuracies close to human capacity. However, Machine Learning-based models often require a large number of labeled example images to carry out the learning process. Although we currently have millions of images, for example from the Internet or social networks, the main problem is that the process of labeling images is very expensive, difficult, and sometimes impossible or unfeasible. Therefore, the question we ask ourselves is: can we take advantage of large volumes of unlabeled images to help solving the problem of Supervised Learning? To answer the above question, in the project we have set out to explore the newest Self-supervised Learning methods to test whether the use of unlabeled images can help a food classifier improve its accuracy. Once analyzed in detail, we have proposed a new Self-Supervised Learning method, called OptSSL, which shows great performance in representing food images in a latent space. To demonstrate its potential, we have made an exhaustive comparison of this method with other recent Self-supervised Learning methods proposed in the literature. Another problem discussed in this paper is the modeling of the uncertainty of the predictions of a model. Most current Deep Learning models are not able to decide whether their predictions are confident or not. Thus, in the project, this property is incorporated into the analyzed classifiers, and the performance of the classifier is analyzed not only from the point of view of its accuracy but also from its uncertainty, making a comparison of its robustness in this context. As a result of the work, we show that OptSSL is an optimal method for data representation, and in particular of food images, which benefits from unlabeled images to learn the representation. Finally, we show how the OptSSL method can be used as a pre-training phase to subsequently train a food classifier, obtaining high accuracy and low uncertainty in predictions.