FAQs

About Amazon Mechanical Turk

What is Amazon Mechanical Turk?

In 1769, Hungarian nobleman Wolfgang von Kempelen astonished Europe by building a mechanical chess-playing automaton that defeated nearly every opponent it faced. A life-sized wooden mannequin, adorned with a fur-trimmed robe and a turban, Kempelen's “Turk” was seated behind a cabinet and toured Europe, confounding such brilliant challengers as Benjamin Franklin and Napoleon Bonaparte. To persuade skeptical audiences, Kempelen would slide open the cabinet's doors to reveal the intricate set of gears, cogs and springs that powered his invention. He convinced them that he had built a machine that made decisions using artificial intelligence. What they did not know was the secret behind the Mechanical Turk: a chess master cleverly concealed inside.

Today, we build complex software applications based on the things computers do well, such as storing and retrieving large amounts of information or rapidly performing calculations. However, humans still significantly outperform the most powerful computers at completing such simple tasks as identifying objects in photographs – something children can do even before they learn to speak.

When we think of interfaces between human beings and computers, we usually assume that the human being is the one requesting that a task be completed, and the computer is completing the task and providing the results. What if this process was reversed and a computer program could ask a human being to perform a task and return the results? What if it could coordinate many human beings to perform a task?

Amazon Mechanical Turk (MTurk) provides a service for service requesters (hereafter “Requesters”) to integrate Artificial Artificial Intelligence directly into their applications by making requests of humans. Requesters can use the MTurk web user interface or web services API to submit tasks to the MTurk web site, approve completed tasks, and incorporate the answers into their applications. When using the web services API, the transaction looks very much like any remote procedure call – the application sends the request, and the service returns the results. In reality, a network of human Workers (hereafter “Workers”) fuels this Artificial Artificial Intelligence by coming to the web site, searching for and completing tasks, and receiving reward for their work.

All Requesters need to do is write normal code. The pseudo code below illustrates how simple this can be.

  read(photo);
  photoContainsHuman = callMechanicalTurk(photo);

  if (photoContainsHuman == TRUE) {
      acceptPhoto();
  } else {
      rejectPhoto();
  }
						

What problem does Amazon Mechanical Turk solve?

• For software developers, the Amazon Mechanical Turk (MTurk) web service solves the problem of building applications that until now have not worked well without human intelligence. Humans are much more effective at solving some types of problems, like finding specific objects in pictures, evaluating beauty, or translating text. The idea of the MTurk web service is to give developers a programmable interface to a network of humans to solve these kinds of problems and incorporate this human intelligence into their applications.
• For businesses and entrepreneurs who want tasks completed, the MTurk service solves the problem of accessing a vast network of human intelligence with the efficiencies and cost-effectiveness of computers. Oftentimes people do not move forward with certain projects because the cost to establish a network of skilled Workers to do the work outweighs the value of completing it. By turning the fixed costs into variable costs that scale with their needs, the MTurk web service eliminates this barrier and allows projects to be completed that before were not economical.
• For people who want to earn money in their spare time, the MTurk web site solves the problem of finding work you can do wherever and whenever you want using the skills you already possess.

How do Requesters ensure their tasks are completed in a high-quality manner?

Requesters have several options for ensuring their HITs (Human Intelligence Tasks) are completed in a high-quality manner.

• Requesters have the opportunity to approve completed HITs before having to pay for them. There are many ways Requesters can approve HITs, ranging from auto-approving them sight unseen, to auto-approving them when they receive two answers to the same HIT that match, to manually approving some or all completed HITs.
• Requesters can specify that people who work on their tasks must first complete a qualification test, thus giving Requesters the opportunity to vet the skill level of the people working on their HITs beforehand. Over time people who do work on the Amazon Mechanical Turk web site will have statistics associated with them based on how accurately they have completed other HITs. These statistics, such as the percentage of HITs they've submitted that have been approved, allow Requesters to specify further that only Workers with certain qualifications can work on their HITs.

Who completes the tasks on Amazon Mechanical Turk and how do they complete them?

• Workers can complete tasks on the Amazon Mechanical Turk (MTurk) web site using the skills they already have and according to their own schedule. To complete tasks and collect payment from Requesters, workers need a computing device connected to the Internet and must be at least 18-years-old. Visit the Worker website for more details on requirements to get started as a Worker.
• Earning money through the MTurk web site is as easy as searching for HITs on the site, accepting those HITs, completing them, and collecting the reward once the HIT is approved by the Requester.
• To make money by completing tasks using the MTurk web site, visit worker.mturk.com.

What kind of work can be done using Amazon Mechanical Turk?

• Anyone can submit virtually any task that can be completed using a computing device connected to the Internet.
• Amazon came up with the idea for Amazon Mechanical Turk to help solve specific internal data processing problems that required human judgment and intelligence. We quickly realized that we had invented something unique and incredibly useful, and decided to expose it as an external web service so that Requesters can create their own innovative applications that use human intelligence as a core component.

How do Workers collect money for the tasks they complete?

When the Requester approves a submitted HIT, Amazon Mechanical Turk automatically displays Worker earnings on their Dashboard and Earnings pages. Earnings are transferred to their virtual US bank account or to their Amazon.com gift card balance on a regular schedule.

Can any business use Amazon Mechanical Turk or do they need to understand how to use web services and APIs?

Businesses can create and publish HITs on Amazon Mechanical Turk using the Requester web user interface (UI), the API, or the command line interface (CLI). Visit the Requester User Interface Guide to learn more about the UI or the developer getting started guide to learn about the API or CLI.

Can Requesters outside of the US use Amazon Mechanical Turk to get tasks completed?

We currently accept registrations from the following countries:

Americas

Barbados, Chile, Canada, French Guiana, Guadeloupe, Martinique, Puerto Rico, United States, US Minor Islands, US Virgin Islands

Africa

Botswana, Mayotte, Réunion, South Africa

Asia and Pacific

Australia, French Polynesia, French Southern Territories, Hong Kong, Israel, Japan, New Caledonia, New Zealand, Qatar, Singapore, South Korea, Taiwan

Europe

Austria, Belgium, Denmark, Estonia, Finland, France, Germany, Iceland, Ireland, Italy, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovenia, Spain, Sweden, Switzerland, United Kingdom

Where can I view and edit my name, email address, password, and payment methods

You can view and edit your account information by accessing your My Account page. Please note that your MTurk Requester account is associated with your Amazon.com account so when you edit your personal settings, they will be changed on the corresponding Amazon.com account.

Where can I view and edit my contact address information?

You can view and edit your contact address on Amazon.com.

Payments & Fees

How are the fees calculated?

You can review Amazon Mechanical Turk pricing here.

What rounding rules are applied when calculating fees?

After the fee is calculated, we round half up amounts greater than $0.01 – we round down if the fractional amount is less than half a penny, and round up otherwise. For example, $0.104 is rounded to $0.10, $0.105 is rounded to $0.11, and $0.108 is rounded to $0.11.

How are Workers paid?

Workers will be paid and Amazon Mechanical Turk (MTurk) fees will be charged when you approve submitted work. If you reject the work, the Worker is not paid and you are not charged the MTurk fees. MTurk HITs are subject to Participation Agreement. You can review MTurk pricing here.

How do I receive a receipt for HITs?

With AWS billing, invoicing is sent via email on a monthly basis and also available for download from the Billing Console on demand. If you need to configure invoicing without a credit card or if you need additional support visit mturk.com/contact-us.

If you need a more detailed receipt, you can contact mturk.com/contact-us to have a copy of your receipt emailed to you. In order to process your request, please provide the transaction time period to be included in the receipt.

You can download your transaction history for your records here.

Who are Amazon Mechanical Turk Masters?

Amazon Mechanical Turk (MTurk) has built technology which analyzes Worker performance, identifies high performing Workers, and monitors their performance over time. Workers who have demonstrated excellence across a wide range of HITs are awarded the Masters Qualification. Masters must continue to pass our statistical monitoring to maintain the MTurk Masters Qualification.

Do you collect tax reporting information from non-US based Workers?

Yes, all non-U.S. based Workers are required to provide their tax information through a tax information interview before doing any work on Mechanical Turk just as U.S. Workers are required to do. The tax information interview collects the U.S. tax status (for U.S. person or non-U.S. person), the name, and the permanent residence address of the individual that will report the income on an income tax return.

AWS Billing & Payments

How does AWS Billing for MTurk work?

1. Your AWS bill includes MTurk fee details and your AWS account ID which you may need for accounting purposes. Your transaction history is still available for download from the MTurk Requester site.
2. If you want to manage billing for multiple MTurk accounts through a single account, you can sign up for an account with AWS Organizations which will enable you to link other AWS accounts and their associated MTurk Requester accounts under an AWS Organizational account to consolidate billing.
3. With AWS billing, invoicing is sent via email on a monthly basis and also available for download from the Billing Console on demand. If you need to configure invoicing without a credit card or if you need additional support visit mturk.com/contact-us.
4. After linking your AWS Account with your MTurk Account, please contact us to request a limit increase on your AWS MTurk account for your expected usage.

How do I set up an AWS Organizational account?

1. Go to Amazon Web Services home page and click Sign In to the Console.
2. To link the two accounts you first need to choose the AWS account that all charges will be billed to and create an Organization.
3. To add additional MTurk Requester accounts under your AWS Organizational account, those Requester accounts must be linked to an AWS account and opted-in to AWS billing.
4. Then invite other accounts as member accounts to join your AWS Organization account.
5. Once the accounts are linked all MTurk fees will be billed through the AWS Organizational account.

How do I approve, reject, or bonus work submitted by Workers in the AWS billing model?

Similar to today you approve, reject, and bonus submitted HITs by Workers on the Requester website. Any unapproved submitted HITs will be automatically approved after 30 days.

I'm an MTurk Worker, will my payments get affected with this change?

No, this change will not affect your payments. You will continue to receive your payments once your work gets approved.

How will the MTurk usage show up on my AWS bill?

All approved assignments will appear on your AWS bill under two categories: Mechanical Turk Fees and Mechanical Turk Worker Rewards.

Design, Publish, and Manage HITs

How do I create a HIT?

The Amazon Mechanical Turk (MTurk) Requester website allows Requesters to access the MTurk Worker community. The Requester UI Guide explains how to design, publish, and manage your HITs on the Requester website.

What do I need to know about creating surveys on MTurk?

Check out our blog post to learn more about getting started with Surveys and the importance of selecting Qualifications. You can also create Custom Qualifications to include or exclude Workers across multiple survey batches or for longitudinal research. You may also want to consider deploying linked surveys.

What happens when I reject work?

When you reject an assignment, the Worker who performed the assignment is not paid, and you are not charged the standard Amazon Mechanical Turk fee for the HIT. The rejection may affect your reputation as a Requester and the approval rate of the Worker who submitted the results. Please note that you may not reject an assignment without good cause and rejections can be reversed if you determine that a mistake was made.

How long does Amazon Mechanical Turk keep my HITs?

MTurk will remove HITs and corresponding Assignment data from your account after 120 days. If you require access to HIT and Assignment data for longer than 120 days, we recommend you download the HIT results and store them locally.

Are there restrictions or prohibited uses of Amazon Mechanical Turk?

You may not use Amazon Mechanical Turk for illegal or objectionable activities. Please refer to the policy page for examples. For example, Requesters are not allowed to collect personally identifiable information from Workers.

How do I prevent Workers who have worked on my HIT (e.g. Surveys) once from taking part in subsequent HITs?

This can be achieved by using qualifications. Requesters can create a Qualification Type or use an existing Qualification Type to assign it to the Workers who have completed the HIT (e.g. survey) once. When creating the next HIT, Requesters can specify that only Workers who have not been granted the Qualification Type can work on the HIT. This can be done through the UI and the API.

How do I use Premium Qualifications?

Requesters can specify criteria at the time of creating the project. The available attributes can be specified in the “Worker Requirements” section of the Requester Website.

Can I use Premium Qualifications for any kind of HIT?

Yes, using the Create Project feature on the Requester Website, Requester customers can use Premium Qualifications for any kind of HIT that has 10 or more Assignments, including Surveys. The Sentiment and Categorization Project types do not support Qualifications, nor does the Create HITs Individually feature.

What is the cost of using Premium Qualifications?

The use of each of the Premium Qualifications incurs an additional fee per approved assignment based on the specific Premium Qualification chosen. For more detail, see our Pricing page.

What if I need to create and manage Custom Qualifications for my HIT?

Custom Qualifications can be designed by Requesters to select or exclude specific worker attributes. You can also create Custom Qualifications to include or exclude Workers across multiple survey batches or for longitudinal research. For more information, see our documentation on Managing Qualification Types. Note: Custom Qualifications can be used across Requester accounts so it is possible to share your Custom Qualifications should you choose to do so.

How do I retry publishing a Batch with a failed payment?

You can retry publishing by following these three steps:

1. If your batch had an input data file, download the input data file on the failed Batch Details page. You will need this file for Step 3.
2. Go to: Create -> New Batch with an Existing Project, and click Publish Batch on the same Project you previously used.
3. If your batch had an input data file, upload the input file from Step 1.
4. Go through the remaining steps of publishing a batch.

What is the maximum number of HITs in a batch?

The maximum number of HITs in a batch is 250,000 (or 500 in our Sandbox environment).

In order to publish more than the maximum number of HITs at a time, you may break your input file into multiple smaller files containing less than or equal to the maximum number of HITs. If your use case requires you to publish more than the maximum number of HITs in a single batch, please contact us.

Can I create and manage HITs programmatically?

Amazon Mechanical Turk (MTurk) provides Requester facing APIs as well as developer tool kits (SDKs). To get started, visit our developer Getting Started Guide.

Why do I need to have an AWS account to use MTurk APIs?

In order to call the MTurk APIs, Requesters must have registered for AWS accounts and retrieved their IAM credentials. This is because MTurk uses the AWS account to identify and allow access for developers using the MTurk APIs. We’re not using AWS to track MTurk API usage, but your Mturk Fees and Worker Rewards will appear on your AWS Bill.

Can I use MTurk to publish tasks containing personal or sensitive data?

No. As a public crowd marketplace, MTurk is not designed for sharing or publishing HITs containing personal or sensitive data. It is your responsibility as a Requester to determine whether your HIT content contains any personal or sensitive data. For more information see our Acceptable Use Policy.

Can I post a survey on MTurk that asks Workers for basic demographic information?

Yes, so long as your survey does not derive any personally identifiable information from Workers. For example, you may ask Workers for their occupation, relationship status, etc., but you may not ask Workers for personally identifiable information, such as their full name, email address, phone number, or equivalent in any HIT including survey HITs. Surveys should only be posted for legitimate academic or market research purposes.

Are there restrictions on what I can use Worker IDs for?

Unique Worker IDs are made available by MTurk to facilitate your use of the MTurk service only and for no other purpose. For example, you may use Worker IDs to distinguish between Workers who complete your HITs, and manage Workers such as tracking how many HITs a particular Worker has completed. You may not use Worker IDs to try to derive the real identity of any Worker, or otherwise publicly post or share them.