Learning Calibrated Belief Functions from Conformal Predictions

Vitor Martin Bordini (1, 2) , Sébastien Destercke (3, 1) , Benjamin Quost (1, 2)

1 Heudiasyc - Heuristique et Diagnostic des Systèmes Complexes [Compiègne]
2 UTC - Université de Technologie de Compiègne
3 CNRS - Centre National de la Recherche Scientifique

We consider the problem of supervised classification. We focus on the problem of calibrating the classifier's outputs. We show that the p-values provided by Inductive Conformal Prediction (ICP) can be interpreted as a possibility distribution over the set of classes. This allows us to use ICP to compute a predictive belief function which is calibrated by construction. We also propose a learning method which provides p-values in a simpler and faster way, by making use of a multi-output regression model. Results obtained on the Cifar10 and Digits data sets show that our approach is comparable to standard ICP in terms of accuracy and calibration, while offering a reduced complexity and avoiding the use of a calibration set.

Keywords: Conformal Prediction Dempster-Shafer Belief Functions Calibration; Conformal Prediction; Dempster-Shafer; Belief Functions; Calibration

Domains: Computer Science [cs]

Vitor Martin Bordini, Sébastien Destercke, Benjamin Quost. Learning Calibrated Belief Functions from Conformal Predictions. 13th International Symposium on Imprecise Probabilities: Theories and Applications (ISIPTA 2023), Jul 2023, Oviedo, Spain. ⟨hal-04371387⟩