Some factors explaining the reliability of a structured interview system at a work site

Recent meta-analytic research has demonstrated structured interviewing to hold acceptable validity and reliability. While the emphasis has been on refining psychometric properties, there is sufficient evidence to suggest a discrepancy between the manner in which interviewing systems should be used and how they are actually used. The present study examined the use of a commercially available structured interviewing system based on past behaviour. 112 candidates were interviewed on two separate occasions by 28 interviewers. Inter-rater reliability was 0.55. The system required the derivation of a consensus score which was found not to differ significantly from the arithmetical mean of the original scores suggesting the process was not undertaken as required. Follow up discussions with interviewers reported three main areas of misunderstanding; lack of role clarity, different interpretations of job specification and inconsistent use of the rating system. Data also suggested interviewers were inexperienced. A critical finding involved female interviewers making assumptions about female applicant's motivations and suitability to the position. This finding was explained by employing 'person-in-job' prototypes. The data support the conclusion that although structured interviews may contain appropriate psychometric properties, the application of the system is critical.