Carregant...
Tipus de document
Treball de fi de màsterData de publicació
Llicència de publicació
Si us plau utilitzeu sempre aquest identificador per citar o enllaçar aquest document: https://hdl.handle.net/2445/186003
Machine learning copies as a means for black box model evaluation
Títol de la revista
Autors
Director/Tutor
ISSN de la revista
Títol del volum
Recurs relacionat
Resum
[en] The use of propietary black-box machine learning models and APIs in the form of Machine Learning as a Service, makes it very difficult to control and mitigate their potential harmful effects (such as lack of transparency, privacy safeguards, robustness, reusability or fairness). The state-of-the-art technique of Machine Learning Classifier Copying allows us to build a new model that replicates the decision behaviour of an existing one without the need of knowing its architecture nor having access to the original training data.
PRESC (Performance and Robustness Evaluation for Statistical Classifiers) is an existing free software tool for the evaluation of machine learning classifiers, which is maintained by Mozilla’s Data Science team. It aims to provide a toolkit to analyze model performance beyond the standard accuracy-based methods and into areas
which tend to be underexplored in data science practice.
Among the multiple applications of Machine Learning Classifier Copying, a systematic construction and examination of model copies has the potential to be an universally accessible and inexpensive approach to study and evaluate a rich variety of original models, and to help understand its behavior. In this work, an implementation of Machine Learning Classifier Copying has been contributed to the PRESC project, so that this tool becomes readily accessible to researchers and practitioners, and its applicability and performance in a synthetic problem has been explored to understand the copying process. The solution provides a model agnostic sampling strategy and an automated copy process for a number of fundamentally different
hypothesis spaces, so that the set of achievable copy-model-fidelity measures can be used as a diagnostic measure of the original model characteristics.
Descripció
Treballs finals del Màster de Fonaments de Ciència de Dades, Facultat de matemàtiques, Universitat de Barcelona. Curs: 2020-2021. Tutor: David Zeber i Oriol Pujol Vila
Matèries (anglès)
Citació
Citació
ROVIRA ESTEVA, Muriel. Machine learning copies as a means for black box model evaluation. [consulta: 28 de gener de 2026]. [Disponible a: https://hdl.handle.net/2445/186003]