Crowdsourcing User Studies With Mechanical Turk

We just finished our reading session of “Crowdsourcing User Studies With Mechanical Turk” (pdf). Very interesting paper. Few hand-written notes on which type of tasks we would run on the MechTurk.

4 Responses to “Crowdsourcing User Studies With Mechanical Turk”

  1. Posts on mechturk

    Data on MechTurk: distribution by country (txt); Demographics of Turkers (txt)

    Finally, to survey OpenID awareness, they used the Mechanical Turk. Worryingly, they say: “Of the 302 responses we received, we only rejected one, leaving us with 301 valid data points to work with.” Uhm, valid, right ;-)

  2. Neal Lathia says:

    Related: the paper “Defending Online Reputation Systems against Collaborative Unfair Raters through Signal Modeling and Trust” that will appear at this year’s ACM SAC used a crowd-sourced (competition) method to collect dishonest ratings for a recommender system.

  3. [...] discussed about the potential of the Mechanical Turk for social research. This is a crowdsourcing service on mobile – short video (below), description, download the iPhone [...]

  4. Excellent how-to on how to perform user studies on Mechanical Turk, by Markus Jakobsson