Publication Type

Journal Article

Version

publishedVersion

Publication Date

1998

Abstract

This study investigated within-source interrater reliability of supervisor, peer, and subordinate feedback ratings made for managerial development. Raters provided 360-degree feedback ratings on a sample of 153 managers. Using generalizability theory, results indicated that little within-source agreement exists; a large portion of the error variance is attributable to the combined rater main effect and Rater X Ratee effect; more raters are needed than currently used to reach acceptable levels of reliability; supervisors are the most reliable with trivial differences between peers and subordinates when the numbers of raters and items are held constant; and peers are the most reliable, followed by subordinates, followed by supervisors, under conditions commonly encountered in practice. Implications for the validity, design, and maintenance of 360-degree feedback systems are discussed along with directions for future research in this area.

Keywords

Interrater agreement, Professional evaluation, Comparative study, Supervisor, Peer review, Hierarchic relation, Feedback regulation, Human

Discipline

Human Resources Management | Organizational Behavior and Theory

Research Areas

Organisational Behaviour and Human Resources

Publication

Journal of Applied Psychology

Volume

83

Issue

6

First Page

960

Last Page

968

ISSN

0021-9010

Identifier

10.1037/0021-9010.83.6.960

Publisher

American Psychological Association

Copyright Owner and License

Publisher

Additional URL

https://doi.org/10.1037/0021-9010.83.6.960

Download

Share

COinS