In 1986 the Medical Council of Canada (MCC) commissioned a six-year research and development project to create a new more valid written examination of clinical decision-making skills for the Canadian Qualifying Examination in Medicine. At that time, the qualifying examination consisted of three booklets of multiple-choice questions and one booklet of patient management problems administered over a two-day period. All graduates of Canadian and foreign medical schools must pass this examination before practicing medicine anywhere in Canada except Quebec. The project was undertaken because (1) numerous studies do not support the use of patient management problems (PMPs) to assess clinical decision-making skills, and (2) research results on the characteristics of clinical decision-making skills offered guidance to develop new approaches to their assessment. In particular, research suggested that these skills are specific to the case or problem encountered and are contingent on the effective manipulation of a few elements of the problem that are crucial to its successful resolution-the problem's key features. The problems developed by this project focused only on the assessment of these key features. The project was implemented in three overlapping phases over a six-year period, 1986-1992, each containing a development component followed by a pilot test through which the research studies were carried out. The pilot tests were conducted by presenting sets of new key feature problems to classes of graduating students in medical schools across Canada. Based on the project's results, in 1992 the MCC replaced the PMP booklet of its qualifying examination with a booklet of key feature problems, using a mix of question forms that required examinees to supply their responses or select their responses from lists. A partial-credit system of scoring was used to score questions, and question scores within problems were averaged to arrive at problem scores. Computer-based procedures were used to mark the responses to the short-answer questions and to calculate the question, problem, and composite scores. A modified Angoff procedure was used to set a content-based cutting score. This article presents a description of the rationale, purpose, and design of the project; the conceptual basis and formats of the new forms of key-feature problems developed; and a summary of the project's research results.
