A viewpoint on evidence-based health informatics, based on a pilot survey on evaluation studies in health care informatics

Concerned about evidence-based health informatics, the authors conducted a limited pilot survey attempting to determine how many IT evaluation studies in health care are never published, and why. A survey distributed to 722 academics had a low response rate, with 136 respondents giving instructive comments on 217 evaluation studies. Of those studies, half were published in international journals, and more than one-third were never published. Reasons for not publishing (with multiple reasons per study possible) included: "results not of interest for others" (1/3 of all studies), "publication in preparation" (1/3), "no time for publication" (1/5), "limited scientific quality of study" (1/6), "political or legal reasons" (1/7), and "study only conducted for internal use" (1/8). Those reasons for non-publication in health informatics resembled those reported in other fields. Publication bias (preference for positive studies) did not appear to be a major issue. The authors believe that widespread application of guidelines in conducting health informatics evaluation studies and utilization of a registry for evaluation study results could improve the evidence base of the field.