Comparison of expert-rater methods for assessing psychosocial job strain

Objectives This study tested the reliability and validity of industry- and mill-level expert methods for measuring psychosocial work conditions in British Columbia sawmills using the demand-control model. Methods In the industry-level method 4 sawmill job evaluators estimated psychosocial work conditions at a generic sawmill. In the mill-level method panels of experienced sawmill workers estimated psychosocial work conditions at 3 sawmills. Scores for psychosocial work conditions were developed using both expert methods and applied to job titles in a sawmill worker database containing self-reported health status and heart disease. The interrater reliability and the concurrent and predictive validity of the expert rater methods were assessed. Results The interrater reliability and concurrent reliability were higher for the mill-level method than for the industry-level method. For all the psychosocial variables the reliability for the mill-level method was greater than 0.90. The predictive validity results were inconclusive. Conclusions The greater reliability and concurrent validity of the mill-level method indicates that panels of experienced workers should be considered as potential experts in future studies measuring psychosocial work conditions.