| - Background: In diagnostic practice there is increased interest in risk-stratification or 'prediction' rules, enabling physicians to estimate the presence of disease from combinations of test results (predictors). Introduction of electronic patient files will further stimulate this tendency. There are, however, numerous examples of rules showing limited predictive accuracy when validated or applied in other patients, thus misguiding patient management and adversely affecting outcome. Methods are in use to develop more accurate and applicable (generalisable) rules, but various methodological issues remain unsolved. Two innovative methods, penalised estimation and genetic programming, may improve accuracy of prediction rules across populations, but their value has hardly been quantified. Also, the way missing values are commonly handled, i.e. simply excluded, often causes serious bias and compromises accuracy and applicability of prediction rules. Sophisticated multiple imputation methods may properly handle missing values when developing a rule, but how many values (10%, 20%, 50% or 80%) can validly be handled is unknown. Moreover, multiple imputation is not suitable when applying a rule in new patients. - Aim: To quantify which factors compromise and which innovative methods enhance the accuracy and applicability of diagnostic prediction rules in clinical practice. - Objectives: 1. Are prediction rules developed by penalised estimation or genetic programming more accurate? 2. When developing prediction rules, which proportion of missings is acceptable and can still be multiply imputed? 3. When applying prediction rules, what is the best approach to handle missings? 4. How is accuracy of prediction rules affected when applied to populations with different outcome frequency, case mix or different associations between predictors and outcome? All objectives will be illustrated by empirical data from three relevant and frequent diagnostic problems: diagnosis of heart failure, deep vein thrombosis and preoperative diagnosis. Per domain, a derivation set and validation set from another time period, setting or country are available. |
