Using deep learning techniques in click-through rate prediction focusing on a DeepFM model and a comparative analysis of the volatility of prediction vectors of different models

Abstract

Using deep learning in prediction of click-through rate (CTR) is becoming main stream for advertisers engaging in real time bidding (RTB). However, there are implications for adopting a deep learning algorithm to predict CTR and to evaluate a user impression while engaging in real time bidding. In this paper, we explore two state of the art deep learning methods, DeepFM and DCN, using Logistic Regression and Factorization Machine models as a benchmark. We explore their predictive power and the trade off of each model time with respect to training times, inference times, effects of dimension of input data when using hashing buckets, and volatility of prediction of models from training to training. We experiment comprehensively throughout our research with the goal of striking a balance between discovering the best predictor of CTR to enhance a company’s RTB strategy whilst understanding the cost of chasing a (usually) more complex implementation in order to obtain an increase in predictive power. The deep learning models outperform Logistic Regression in RIG, with the DeepFM model achieving the best RIG however the opposite is true for model complexity, training, and inference times. Increasing the hashing bucket size leads to better performances across Logistic Regression. Finally, we look at the volatility of a models prediction vector under retraining with different training data conditions while keeping in mind the goal of developing a real bidding algorithm that takes as input the output of our CTR prediction model.