Building knowledge through families of experiments

被引:431
作者
Basili, VR [1 ]
Lanubile, F
机构
[1] Univ Maryland, Dept Comp Sci, College Pk, MD 20742 USA
[2] Univ Maryland, Fraunhofer Ctr, College Pk, MD 20742 USA
[3] Univ Bari, Dipartimento Informat, I-70126 Bari, Italy
基金
美国国家航空航天局; 美国国家科学基金会;
关键词
empirical software engineering; experimental design; software process; software measurement; software reading techniques;
D O I
10.1109/32.799939
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Experimentation in software engineering is necessary but difficult. One reason is that there are a large number of context variables and, so, creating a cohesive understanding of experimental results requires a mechanism for motivating studies and integrating results. It requires a community of researchers that can replicate studies, vary context variables, and build models that represent the common observations about the discipline. This paper discusses the experience of the authors, based upon a collection of experiments, in terms of a framework for organizing sets of related studies. With such a framework, experiments can be viewed as part of common families of studies, rather than being isolated events. Common families of studies can contribute to important and relevant hypotheses that may not be suggested by individual experiments. A framework also facilitates building knowledge in an incremental manner through the replication of experiments within families of studies. To support the framework, this paper discusses the experiences of the authors in carrying out empirical studies, with specific emphasis on persistent problems encountered in experimental design, threats to validity, criteria for evaluation, and execution of experiments in the domain of software engineering.
引用
收藏
页码:456 / 473
页数:18
相关论文
共 48 条
[1]  
[Anonymous], 1986, LOGIC SCI DISCOVERY
[2]  
BAKER FT, 1972, IBM SYSTEMS J, V11
[3]  
BASILI V, 1996, P 21 ANN SOFTW ENG W, P59
[4]   COMPARING THE EFFECTIVENESS OF SOFTWARE TESTING STRATEGIES [J].
BASILI, VR ;
SELBY, RW .
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 1987, 13 (12) :1278-1296
[5]   EXPERIMENTATION IN SOFTWARE ENGINEERING [J].
BASILI, VR ;
SELBY, RW ;
HUTCHENS, DH .
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 1986, 12 (07) :733-743
[6]   AN EMPIRICAL-STUDY OF A SYNTACTIC COMPLEXITY FAMILY [J].
BASILI, VR ;
HUTCHENS, DH .
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 1983, 9 (06) :664-672
[7]   THE TAME PROJECT - TOWARDS IMPROVEMENT-ORIENTED SOFTWARE ENVIRONMENTS [J].
BASILI, VR ;
ROMBACH, HD .
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 1988, 14 (06) :758-773
[8]   Investigating maintenance processes in a framework-based environment [J].
Basili, VR ;
Lanubile, F ;
Shull, F .
INTERNATIONAL CONFERENCE ON SOFTWARE MAINTENANCE, PROCEEDINGS, 1998, :256-264
[9]   A CONTROLLED EXPERIMENT QUANTITATIVELY COMPARING SOFTWARE-DEVELOPMENT APPROACHES [J].
BASILI, VR ;
REITER, RW .
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 1981, 7 (03) :299-320
[10]   Evolving and packaging reading technologies [J].
Basili, VR .
JOURNAL OF SYSTEMS AND SOFTWARE, 1997, 38 (01) :3-12