Conference Proceeding Article
In this paper, we investigate how deviation in evaluation activities may reveal bias on the part of reviewers and controversy on the part of evaluated objects. We focus on a 'data-centric approach' where the evaluation data is assumed to represent the ground truth'. The standard statistical approaches take evaluation and deviation at face value. We argue that attention should be paid to the subjectivity of evaluation, judging the evaluation score not just on 'what is being said' (deviation), but also on 'who says it' (reviewer) as well as on 'whom it is said about' (object). Furthermore, we observe that bias and controversy are mutually dependent, as there is more bias if there is higher deviation on a less controversial object. To address this mutual dependency, we propose a reinforcement model to identify bias and controversy. We test our model on real-life data to verify its applicability.
Data centric, Data models, Modeling, Probabilistic approach, Statistical analysis, Bias, Ground truth, Knowledge discovery, Data mining
Databases and Information Systems | Numerical Analysis and Scientific Computing
Data Management and Analytics
KDD 2006: Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining: August 20-23, 2006, Philadelphia, PA, USA
City or Country
LAUW, Hady W.; LIM, Ee Peng; and WANG, Ke.
Bias and Controversy: Beyond the Statistical Deviation. (2006). KDD 2006: Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining: August 20-23, 2006, Philadelphia, PA, USA. 625-630. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/892
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.