Calibrating Uncertainty in Regression Models
Analysis
This paper addresses a crucial aspect of machine learning: uncertainty quantification. It focuses on improving the reliability of predictions from multivariate statistical regression models (like PLS and PCR) by calibrating their uncertainty. This is important because it allows users to understand the confidence in the model's outputs, which is critical for scientific applications and decision-making. The use of conformal inference is a notable approach.
Key Takeaways
- •Proposes a method to calibrate uncertainty in multivariate statistical regression models.
- •Method is inspired by conformal inference.
- •Tested on both traditional and kernelized versions of PLS and PCR.
- •Demonstrated on synthetic and real-world datasets (NIR and hyperspectral data).
- •Achieves accurate prediction intervals, matching the desired confidence level.
“The model was able to successfully identify the uncertain regions in the simulated data and match the magnitude of the uncertainty. In real-case scenarios, the optimised model was not overconfident nor underconfident when estimating from test data: for example, for a 95% prediction interval, 95% of the true observations were inside the prediction interval.”