Amazon Turk Followup

In response to my blog post about being a creator and not a consumer, let me write about my results of using Amazon Turk for the first time.

The original batch was for an HIT of 100 assignments at $0.15/each. After 24 hours, there were no takers. So I reduced the HIT assignments to 20 @ $0.30/each. Within 2 days they were all completed. I approved them all without reviewing them, downloaded the CSV, and uploaded it to Google Docs as a spreadsheet.

This particular batch was for people to return to me a list of websites that matched a certain set of quantitative criteria. This included things like frequency of blog posts per week, number of Facebook Likes, Twitter followers, etc. What I'm looking for here are mid-market blogs with readership in the 1,000 - 2,000 uniques/day size.

Out of the 20 assignments completed, 13 of them had to be thrown out. Only 7 were really what I was looking for. Of the 13 that I had to toss, some were for major websites like the New York Times, Mint, and CNN. Oddly enough, some of these were submitted more than once by different workers. A couple other sites were Indian sites, despite the HIT clearly specifying USA-only websites. This probably had more to do with people picking whatever sites they could easily find within their host country.

One guy, however, takes the cake. He spent something like 3 seconds on the assignment and returned an empty result set. Since I approved everything without review (my fault), he gets paid for nothing. Well played sir....

In all, Amazon Turk is a useful service for this kind of repetitive work. The quality is yet to be seen until I get a larger sample size. It's not worth my time to review submissions that are worth $0.30. It's better for me to approve them all and then move downstream and filter out the junk later.

The initial batch wasn't enough. I'll submit another couple hundred for work and see how it goes. If the junk rate is too high, I'll go back to hiring a VA (virtual assistant) from oversees to work directly and just pay her hourly.