Clinical Validation of the AHRQ Postoperative Venous Thromboembolism Patient Safety Indicator

Background: The Agency for Healthcare Research and Quality (AHRQ) patient safety indicators (PSIs) screen for potentially preventable complications in hospitalized patients using hospital administrative data. The PSI for postoperative venous thromboembolism (VTE) relies on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes for deep vein thrombosis (DVT) or pulmonary embolism (PE) in secondary diagnoses fields. In a clinical validation study of the PSI for postoperative VTE, natural language processing (NLP), supplemented by pharmacy and billing data, was used to identify VTE events missed by medical records coders. Methods: In a retrospective review of postsurgical discharges, charts were processed using the AHRQ PSI software. Cases were identified as possible false negatives by flagging charts for possible VTEs using pharmacy and billing data to identify all patients who were therapeutically anticoagulated or had placement of an inferior vena caval filter. All charts were reviewed by a physician blinded to screening results. Physician interpretation was considered the gold standard for VTE classification. Results: The AHRQ PSI had a positive predictive value (PPV) of.545 (95% confidence interval [CI],.453-. 634) and a negative predictive value (NPV) of.997 (95% CI,.995-. 999). Sensitivity was.87 and specificity was.98. Secondary coding review suggested that all 9 false-negative results were miscoded; if they had been properly coded, the sensitivity would increase to 1.00. Most false-positive cases resulted from superficial venous clots identified by the PSI due to coding ambiguity. Discussion: The VTE PSI performed well as a screening tool but generated a significant number of false-positive cases, a problem that could be substantially reduced with improved coding methods.