Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

5-2022

Abstract

Software engineering researchers have been using general purpose online tools for crowd-sourcing for quite some time. Those tools can be useful to recruit participants for research studies as they are paid for their time. However, those tools should be used carefully. In this paper, we have described the issues we faced when recruiting participants on Prolific. We used Prolific to recruit open-source developers with experience in submitting and reviewing pull requests (PRs). However, we did not succeed in obtaining valid participants for either the interview or the survey, which led us to change the approach of our study and not use Prolific anymore. For example, a major issue we faced on Prolific was that even using the specific filter related to participants with software development knowledge, the screening study revealed the majority of them did not have any development knowledge. Based on this experience, we formulate recommendations for future research when using such crowd-sourcing tools.

Discipline

Software Engineering

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Digital transformation

Publication

Proceedings of the RoPES ‘22: 1st International Workshop on Recruiting Participants for Empirical Software Engineering, Virtual Conference, May 17

First Page

1

Last Page

3

City or Country

Pittsburgh, PA, USA

Download

Share

COinS