Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

4-2014

Abstract

One problem facing players of competitive games is negative, or toxic, behavior. League of Legends, the largest eSport game, uses a crowdsourcing platform called the Tribunal to judge whether a reported toxic player should be punished or not. The Tribunal is a two stage system requiring reports from those players that directly observe toxic behavior, and human experts that review aggregated reports. While this system has successfully dealt with the vague nature of toxic behavior by majority rules based on many votes, it naturally requires tremendous cost, time, and human efforts. In this paper, we propose a supervised learning approach for predicting crowdsourced decisions on toxic behavior with large-scale labeled data collections; over 10 million user reports involved in 1.46 million toxic players and corresponding crowdsourced decisions. Our result shows good performance in detecting overwhelmingly majority cases and predicting crowdsourced decisions on them. We demonstrate good portability of our classifier across regions. Finally, we estimate the practical implications of our approach, potential cost savings and victim protection.

Keywords

League of Legends, online video games, toxic behavior, crowdsourcing, machine learning

Discipline

Artificial Intelligence and Robotics

Research Areas

Intelligent Systems and Optimization

Publication

Proceedings of the 23rd International Conference on World Wide Web, WWW 2014, Seoul, South Korea, April 7-11

First Page

877

Last Page

888

ISBN

9781450327442

Identifier

10.1145/2566486.2567987

Publisher

ACM

City or Country

New York

Download

Share

COinS