A cognitive design system approach to generating valid tests: Application to abstract reasoning

The actual impact of cognitive theory on testing contrasts sharply with its potential impact, which suggests some deep incompatibilities between the areas. This article describes and illustrates a cognitive design system approach that centralizes cognitive theory in developing valid tests. To resolve incompatibilities between cognitive and testing, the cognitive design system approach includes both conceptual and procedural frameworks. To illustrate the cognitive design approach, an item bank for measuring abstract reasoning was generated from cognitive theory (i.e., P.A. Carpenter, M.A. Just, & P. Shell's, 1990, processing theory). The construct validity of the generating item bank was strongly supported by several studies from the cognitive design system approach.