EXPERIMENTS, QUASI-EXPERIMENTS, AND CASE-STUDIES - A REVIEW OF EMPIRICAL-METHODS FOR EVALUATING DECISION SUPPORT SYSTEMS

Developers of decision support system (DSS) often fail to present empirical data supporting the claimed merits of their systems. Discussions with developers indicate that they often do not consider or know how to perform the required empirical evaluations. That problem is addressed by reviewing the issues inherent in using experiments, quasi-experiments, and case studies to evaluate DSSs. The paper is meant to be a tutuorial for graduate students and engineering professionals developing DSSs. The discussion revolves around the issues of reliability and four types of validity: internal, construct, statistical conclusion, and external. The DSSs are focused upon, including expert systems, but the discussion is appropriate for most interactive systems.