Conformal prediction for reliable AI
Nicola Paoletti ( King's College London )
- 15:00 6th November 2025Seminar Room 051
The talk will introduce conformal prediction [1,2], an increasingly popular technique for uncertainty quantification that provides finite-sample and distribution-free probabilistic guarantees for machine learning models. We will cover the basics and survey extensions of the methodology to support distribution shifts [3] and conditional validity [4]. We will finally provide an overview of recent conformal prediction methods developed within our group [5-7], in the context of predictive monitoring of stochastic processes, off-policy prediction, adversarial attacks, counterfactual explanations, and latent monitoring of LLMs.
Selected references:
[1] Vovk, Vladimir, Alexander Gammerman, and Glenn Shafer. Algorithmic Learning in a Random World. Springer Nature, 2022.
[2] Angelopoulos, Anastasios N., and Stephen Bates. "A gentle introduction to conformal prediction and distribution-free uncertainty quantification." arXiv preprint arXiv:2107.07511 (2021).
[3] Tibshirani, Ryan J., Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. "Conformal prediction under covariate shift." Advances in neural information processing systems 32 (2019).
[4] Guan, Leying. "Localized conformal prediction: A generalized inference framework for conformal prediction." Biometrika 110, no. 1 (2023): 33-50.
[5] Cairoli, Francesca, Nicola Paoletti, and Luca Bortolussi. "Conformal Quantitative Predictive Monitoring of STL Requirements for Stochastic Processes." (2023).
[6] Kuipers, Tom, Renukanandan Tumu, Shuo Yang, Milad Kazemi, Rahul Mangharam, and Nicola Paoletti. "Conformal off-policy prediction for multi-agent systems." In 2024 IEEE 63rd Conference on Decision and Control (CDC), pp. 1067-1074. IEEE, 2024.
[7] Jeary, Linus, Tom Kuipers, Mehran Hosseini, and Nicola Paoletti. "Verifiably robust conformal prediction." Advances in Neural Information Processing Systems 37 (2024): 4295-4314.