Institute For The Future

The Microwork Behind Assistive Technologies

 

There's a lot to like about a University of Rochester project known as VizWiz, highlighted in the New Scientist a couple of weeks ago. Designed to enable blind people to get answers to everyday visual questions, VizWiz is a demo phone app that can take simple photos and get answers to simple questions, like what denomination is this bill, in near real-time.

 

It works like this:

 

 

 

he holds her iPhone to the open cupboard, snaps a picture of the cans, makes an audio recording of her question - "which one is the coconut milk?" - and double taps to send off her query.

 

Approximately 45 seconds later her iPhone replies in an electronic timbre: "The answer is the one on the right..."

 

Designing a computer program that can reliably recognize text and distinguish objects in the real world has proven to be a massive challenge for artificial intelligence researchers. To get around this, the researchers behind VizWiz - a team consisting of computer scientists from several universities, including the University of Rochester - decided to outsource the task of problem-solving to people: specifically, to Amazon Mechanical Turk's masses of online workers.

 

 

 

 

One of the things I find most fascinating--and potentially transformative--about VizWiz is the clever use of Amazon's Mechanical Turk service, which is a platform aimed at exchanging micropayments of a few cents for microtasks, like interpreting photos, that require human intelligence. In an academic paper, the research team behind VizWiz note that:

 

 

 

 

VizWiz demonstrates a new model for assistive technology in which human workers assist users in nearly real-time. VizWiz currently targets assisting blind and low vision users in the real world, but future work may explore how to extend these benefits to other omains (like the web) or to other populations. One compelling future direction is to use the VizWiz approach to help reduce the latency of transcription and description of audio for deaf and hard of hearing individuals. More generally, we believe that low-cost, readily-available human computation can be applied to many problems.

 

 

 

 

Not only does this strike me as a plausible direction for improving assistive technologies, but it seems like a potential direction for certain forms of economic development. The approach is reminiscent of the concept of virtual gold farming, where people, typically in the developing world, engage in tasks in online games that earn virtual currency, with the goal of exchanging this for traditional currency. A World Bank Report suggested that countries should invest resources into developing virtual gold farming industries to take advantage of the $3 billion and growing market.

 

I haven't seen any estimates about what sort of market would exist for the microwork of interpreting photos for the blind, or for helping deaf individuals with audio and transcription needs. But it strikes me as not simply a market opportunity, but one that could plausibly build new health and well-being capacities by helping people manage the everyday challenges of living with a medical challenge.