Multifaceted Evaluation Criteria of Digital Libraries in Academic Settings: Similarities and Differences From Different Stakeholders

Digital library (DL) evaluation is essential to the success and enhancement of DLs. However, there is a lack of research on the assessment of comprehensive evaluation criteria across multiple dimensions of DLs. In particular, limited research is available on criteria prioritization to determine which criteria are perceived important by different stakeholders. This study was conducted to compare similarities and differences in perceptions of the importance of different DL evaluation criteria by heterogeneous stakeholders in academic settings. Ninety subjects were recruited with 30 from each of the group representing DL scholars, DL librarians, and DL users. Subjects were instructed to fill in an in-depth survey consisting of 10 evaluation dimensions with 94 criteria. ANOVA and t-test were applied to examine the similarities and differences among the three groups. This study reveals consensus and divergence in perceptions of criteria importance among the three groups, and indicates an inherent tension among the stakeholders. Moreover, the differences identify gaps not only between user expectations and the DL practice but also between what's desirable and what's possible in the academic environment. The findings provide a comprehensive list of criteria to guide practical evaluation of DLs, and contribute to the narrowing of the identified gaps.