Calibration processes in a serial talent test

Objectives: In the beginning of serial evaluations, raters assess performances without knowledge about following performances. We assume that judges must observe a certain number of performances to calibrate their judgment scale, leading to systematic biases (avoidance of extreme judgments) in the beginning of judgment series. The present experiment investigates how many performance observations are necessary to calibrate internal judgment scales, leading to consistent judgments. Design: A between-group design was used. Method: Videos of a talent-test were presented in different orders. Every performance was presented in early, medium, and late positions. Thirty participants rated these series and we assessed the effect of performance position on performance evaluations. Results: We found calibration biases within the first nine performances; after that, evaluations remained stable and consistent. Conclusion: Calibration processes are completed after a specific number of judgments; this implies interesting interventions for consistent judgments independent of the position in a series. ?? 2013 Elsevier Ltd.