Access Restriction

Researcher Weld, Daniel S. ♦ Dai, Mausam Peng
Source CiteSeerX
Content type Text
File Format PDF
Language English
Subject Domain (in DDC) Computer science, information & general works ♦ Data processing & computer science
Subject Keyword Artificial Intelligence Method ♦ Photo Tagging ♦ Individual Worker ♦ Decision-theoretic Opti-mization ♦ Opti-mized Workflow ♦ Scalable Application ♦ Opti-mize Parameter ♦ Alternative Workflow ♦ Vertical Appli-cations ♦ Huge Number ♦ Preliminary Experience ♦ Human Intelligence ♦ Product Categorization ♦ Re-fine Model ♦ Quality Output ♦ Artificial Intelligence ♦ Self-managing Workflow ♦ Task Difficulty ♦ Worker Performance ♦ Amazon Mechanical Turk ♦ Alter-native Decision ♦ Com-plex Crowdsourced Workflow ♦ Quality Result
Description Crowdsourcing platforms, such as Amazon Mechanical Turk, have enabled the construction of scalable applications for tasks ranging from product categorization and photo tagging to audio transcription and translation. These vertical appli-cations are typically realized with complex, self-managing workflows that guarantee quality results. But constructing such workflows is challenging, with a huge number of alter-native decisions for the designer to consider. We argue the thesis that “Artificial intelligence methods can greatly simplify the process of creating and managing com-plex crowdsourced workflows. ” We present the design of CLOWDER, which uses machine learning to continually re-fine models of worker performance and task difficulty. Us-ing these models, CLOWDER uses decision-theoretic opti-mization to 1) choose between alternative workflows, 2) opti-mize parameters for a workflow, 3) create personalized inter-faces for individual workers, and 4) dynamically control the workflow. Preliminary experience suggests that these opti-mized workflows are significantly more economical (and re-turn higher quality output) than those generated by humans.
Educational Role Student ♦ Teacher
Age Range above 22 year
Educational Use Research
Education Level UG and PG ♦ Career/Technical Study
Learning Resource Type Thesis
Publisher Date 2011-01-01
Publisher Institution In Human Computation, AAAI Workshops. AAAI