Parametric learning of probabilistic graphical models from multi-sourced data

Abstract

In Machine Learning, it is common to encounter scenarios where learning a model from a scarce dataset may not be feasible. In these cases, data from multiple different sources have to be collected. When data from multiple sources is distributed differently, the benefit of a bigger sample size trades off with the difficulty to model together data sampled from different distributions. A similar framework is presented in fairness analysis, where subpopulations defined by the protected attributes might show different underlying distributios. In this work, we study the use of hierarchical Bayesian methods to learn Bayesian network (BN) models from all the available data while being aware of the presence of unequally distributed data sources. We propose a variation of a previous hierarchical Bayesian approach for learning BN parameters which naturally accommodates into the framework of BNs. The comparison with the state-of-the-art methods is done in two dimensions: the amount of samples available to train a model, and the divergence of the underlying distribution of the different data sources. Experimental results suggest that our model is competitive when data is scarce and the multiple sources are distributed differently.