Conducting interactive experiments on Toloka

Even before the COVID-19 pandemic, the popularity of online behavioral experiments grew steadily. Due to lockdowns, online studies often became the only available option for behavioral economists, sociologists, and political scientists. The use of the most well-known platforms, such as mTurk, was so intensive that the quality of data was harmed. However, even before the pandemic-induced quality crisis, online studies were limited in scope; real-time interactions between participants were hard to achieve due to the large proportion of drop-outs and issues with creating stable groups. Using the relatively unknown crowdsourcing platform, Toloka, we successfully ran several multi-round interactive experiments. Toloka's sizeable online population, fairly low exposure of participants to sociological surveys and behavioral studies, and convenient application programming interface can make it a useful addition to the toolbox of an experimentalist who needs to run behavioral studies that require real-time interactions between participants.& COPY; 2023 Elsevier B.V. All rights reserved.

Automated summary

The popularity of online behavioral experiments has been growing steadily even before the COVID-19 pandemic. Lockdowns made online studies the only available option for behavioral economists, sociologists, and political scientists. The intensive use of popular platforms like mTurk harmed data quality. However, even before the quality crisis caused by the pandemic, online studies had limitations in scope and real-time interactions between participants were hard to achieve. By using the relatively unknown crowdsourcing platform, Toloka, several successful multi-round interactive experiments were conducted. Toloka's substantial online population, participants' limited exposure to sociological surveys and behavioral studies, and convenient application programming interface make it a valuable addition to an experimentalist's toolbox for running behavioral studies requiring real-time interactions.