Challenging Forgets: Identifying and Analyzing Hard-to-Unlearn Data

Abstract

Machine unlearning aims to remove the influence of specific data from trained models to protect privacy and comply with legal standards such as the “right to be forgotten”. However, existing Machine Unlearning research has largely overlooked how the construction of Forget Sets influences unlearning success. This project addresses this gap by systematically designing and evaluating four distinct Forget Set strategies (ranging from random sampling to adversarially motivated similarity) using a ResNet-18 classifier trained on the CIFAR-10 dataset. Two unlearning techniques, basic fine-tuning and fine-tuning with final-layer perturbation, are applied. To rigorously assess performance, this study defines and applies multiple evaluation metrics: Forgetting (how effectively a model erases targeted data), Utility (how well it retains performance on retained data), and a composite metric that balances both. The results reveal how Forget Set composition critically affects the effectiveness of Machine Unlearning strategies, offering new insights for future research and development.