Distance-based copying of machine learning classifiers

Abstract

Copying machine learning black box classifiers is a key framework that allows practitioners to upgrade their old models, enriching them with new properties, changing their architectures or adapting them to comply with the current AI legislations. Thanks to the copying techniques and assumptions, these improvements can be done even in settings where retraining the original system from scratch is not possible, due to resource, protocol or availability constraints. In this work, we propose the use of signed distances to the decision boundary as a replacement of the black box hard labels used to build the copies, and introduce two different algorithms to compute these distances. In addition, we observe that distance-based copying could behave as a model-agnostic regularization technique and develop a flexible framework to reduce the generalization error of the copies. Then, we validate these proposals through a series of experiments on synthetic datasets and real problems. Results show that distance-based copying is successful across multiple relevant settings and evaluation metrics. Furthermore, results also validate the quality of the predicted distances and their potential as uncertainty measures.