A New Look at within-Source Interrater Reliability of 360-Degree Feedback Ratings
This study investigated within-source interrater reliability of supervisor, peer, and subordinate feedback ratings made for managerial development. Raters provided 360-degree feedback ratings on a sample of 153 managers. Using generalizability theory, results indicated that little within-source agreement exists; a large portion of the error variance is attributable to the combined rater main effect and Rater X Ratee effect; more raters are needed than currently used to reach acceptable levels of reliability; supervisors are the most reliable with trivial differences between peers and subordinates when the numbers of raters and items are held constant; and peers are the most reliable, followed by subordinates, followed by supervisors, under conditions commonly encountered in practice. Implications for the validity, design, and maintenance of 360-degree feedback systems are discussed along with directions for future research in this area.
Interrater agreement, Professional evaluation, Comparative study, Supervisor, Peer review, Hierarchic relation, Feedback regulation, Human
Organisational Behaviour and Human Resources
Journal of Applied Psychology
American Psychological Association
City or Country
Greguras, G. J. and Robie, Chet.
A New Look at within-Source Interrater Reliability of 360-Degree Feedback Ratings. (1998). Journal of Applied Psychology. 83, (6), 960-968. Research Collection Lee Kong Chian School Of Business.
Available at: http://ink.library.smu.edu.sg/lkcsb_research/2150