Fitxers
Tipus de document
ArticleVersió
Versió publicadaData de publicació
Llicència de publicació
Si us plau utilitzeu sempre aquest identificador per citar o enllaçar aquest document: https://hdl.handle.net/2445/229689
Bayesian estimation of information-theoretic metrics for sparsely sampled distributions
Títol de la revista
Director/Tutor
ISSN de la revista
Títol del volum
Recurs relacionat
Resum
Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback–Leibler divergence between two sparsely sampled discrete distributions, is even harder. Here, we propose a fast, semi-analytical estimator for sparsely sampled distributions. Its derivation is grounded in probabilistic considerations and uses a hierarchical Bayesian approach to extract as much information as possible from the few observations available. Our approach provides estimates of the Shannon entropy with precision at least comparable to the benchmarks we consider, and most often higher; it does so across diverse distributions with very different properties. Our method can also be used to obtain accurate estimates of other information-theoretic metrics, including the notoriously challenging Kullback–Leibler divergence. Here, again, our approach has less bias, overall, than the benchmark estimators we consider.
Matèries (anglès)
Citació
Citació
PIGA, Angelo, et al. Bayesian estimation of information-theoretic metrics for sparsely sampled distributions. Chaos. Solitons & Fractals. Vol. 2024, num. 180. ISSN 0960-0779. [consulted: 26 of May of 2026]. Available at: https://hdl.handle.net/2445/229689