Pfahler, Simon and Georg, Peter and Schill, Rudolf and Klever, Maren and Grasedyck, Lars and Spang, Rainer and Wettig, Tilo (2024) Taming numerical imprecision by adapting the KL divergence to negative probabilities. STATISTICS AND COMPUTING, 34 (5): 168. ISSN 0960-3174, 1573-1375
Full text not available from this repository. (Request a copy)Abstract
The Kullback-Leibler (KL) divergence is frequently used in data science. For discrete distributions on large state spaces, approximations of probability vectors may result in a few small negative entries, rendering the KL divergence undefined. We address this problem by introducing a parameterized family of substitute divergence measures, the shifted KL (sKL) divergence measures. Our approach is generic and does not increase the computational overhead. We show that the sKL divergence shares important theoretical properties with the KL divergence and discuss how its shift parameters should be chosen. If Gaussian noise is added to a probability vector, we prove that the average sKL divergence converges to the KL divergence for small enough noise. We also show that our method solves the problem of negative entries in an application from computational oncology, the optimization of Mutual Hazard Networks for cancer progression using tensor-train approximations.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | TENSOR; OPTIMIZATION; Kullback-Leibler divergence; Approximate Bayesian computation; Statistical optimization; Mutual Hazard Networks; Tensor trains |
| Subjects: | 000 Computer science, information & general works > 004 Computer science 500 Science > 530 Physics |
| Divisions: | Medicine > Institut für Funktionelle Genomik > Lehrstuhl für Statistische Bioinformatik (Prof. Spang) Informatics and Data Science > Department Computational Life Science > Lehrstuhl für Statistische Bioinformatik (Prof. Spang) Physics > Institute of Theroretical Physics > Chair Professor Braun > Group Tilo Wettig |
| Depositing User: | Dr. Gernot Deinzer |
| Date Deposited: | 26 Aug 2025 06:49 |
| Last Modified: | 26 Aug 2025 06:49 |
| URI: | https://pred.uni-regensburg.de/id/eprint/63906 |
Actions (login required)
 |
View Item |
