A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes

Year: 2014

Authors: Nattino G., Finazzi S., Bertolini G.

Autors Affiliation: IRCCS Ist Ric Farmacol Mario Negri, GiViTI Coordinating Ctr, Lab Clin Epidemiol, Ranica, BG, Italy; Univ Trento, INO CNR BEC Ctr, I-38123 Povo, Trento, Italy; Univ Trento, Dipartimento Fis, I-38123 Povo, Trento, Italy.

Abstract: Calibration is one of the main properties that must be accomplished by any predictive model. Overcoming the limitations of many approaches developed so far, a study has recently proposed the calibration belt as a graphical tool to identify ranges of probability where a model based on dichotomous outcomes miscalibrates. In this new approach, the relation between the logits of the probability predicted by a model and of the event rates observed in a sample is represented by a polynomial function, whose coefficients are fitted and its degree is fixed by a series of likelihood-ratio tests. We propose here a test associated with the calibration belt and show how the algorithm to select the polynomial degree affects the distribution of the test statistic. We calculate its exact distribution and confirm its validity via a numerical simulation. Starting from this distribution, we finally reappraise the procedure to construct the calibration belt and illustrate an application in the medical context.

Journal/Review: STATISTICS IN MEDICINE

Volume: 33 (14)      Pages from: 2390  to: 2407

KeyWords: calibration test; dichotomous outcome models; goodness-of-fit; logistic regression models
DOI: 10.1002/sim.6100

Citations: 92
data from “WEB OF SCIENCE” (of Thomson Reuters) are update at: 2024-10-20
References taken from IsiWeb of Knowledge: (subscribers only)
Connecting to view paper tab on IsiWeb: Click here
Connecting to view citations from IsiWeb: Click here