Methodologic issues regarding the use of three observational study designs to assess influenza vaccine effectiveness

Background Influenza causes substantial morbidity and annual vaccination is the most important prevention strategy. Accurately measuring vaccine effectiveness (VE) is difficult. The clinical syndrome most closely associated with influenza virus infection, influenza-like illness (ILI), is not specific. In addition, laboratory confirmation is infrequently done, and available rapid diagnostic tests are imperfect. The objective of this study was to estimate the joint impact of rapid diagnostic test sensitivity and specificity on VE for three types of study designs: a cohort study, a traditional case-control study, and a case-control study that used as controls individuals with ILI who tested negative for influenza virus infection. Methods We developed a mathematical model with five input parameters: true VE, attack rates (ARs) of influenza-ILI and non-influenza-ILI and the sensitivity and specificity of the diagnostic test. Results With imperfect specificity, estimates from all three designs tended to underestimate true VE, but were similar except if fairly extreme inputs were used. Only if test specificity was 95% or more or if influenza attack rates doubled that of background illness did the case-control method slightly overestimate VE. The case-control method usually produced the highest and most accurate estimates, followed by the test-negative design. The bias toward underestimating true VE introduced by low test specificity increased as the AR of influenza-relative to non-influenza-ILI decreases and, to a lesser degree, with lower test sensitivity. Conclusions Demonstration of a high influenza VE using tests with imperfect sensitivity and specificity should provide reassurance that the program has been effective in reducing influenza illnesses, assuming adequate control of confounding factors.