USI - Email

Archive / INF Seminars / INF_2025_08_20_LucasKania

USI - Email

	Testing with black-box predictions

Host: Prof. Wit Ernst-Jan Camiel

Wednesday

20.08

USI Lugano Campus Est, room D5.01

12:00 - 13:00


	Lucas Kania Carnegie Mellon

Abstract: Recently, several methods have emerged that use tests leveraging black-box predictions to improve classical confidence sets. Notable examples include prediction-powered inference [Angelopoulos et al., 2023, Zrnic and Candes, 2024], dimension-agnostic inference for M-estimators [Kim and Ramdas, 2024, Takatsu and Kuchibhotla, 2025], and universal inference [Wasserman et al., 2020]. However, there is no agreement on how to best use the black-box predictions. In this talk, we will discuss how tests can optimally incorporate black-box predictions of the data distribution. We focus on test procedures that adapt to the unknown quality of the prediction. When the prediction is accurate, the optimal test achieves the same rates as classical binary hypothesis testing against the best alternative distribution. When the prediction is inaccurate, the test ignores it and achieves standard nonparametric rates. These adaptive tests automatically adjust their power allocation: they concentrate power on a narrow range of alternatives when the prediction is informative and distribute it more broadly when the prediction is unreliable.

Biography: Lucas Kania is a PhD student in the Department of Statistics and Data Science at Carnegie Mellon University (CMU), where he is advised by Larry Wasserman and Sivaraman Balakrishnan. You can find his research at lucaskania.com, and contact him via lucaskania@cmu.edu.