Researchers teach computers to search photos by subject


Washington : Penn State University researchers have developed a statistical approach, called ALIPR, that one day could make it easier to search the net for photographs.

Support TwoCircles

The public can participate in improving ALIPR or automatic linguistic indexing of pictures in real-time accuracy, by visiting, uploading photographs, and evaluating whether the keywords that ALIPR uses to describe the photographs are appropriate.

ALIPR works by teaching computers to recognise the contents of photographs like buildings, people, or landscapes, rather than by searching for keywords in the surrounding text, as is done with most current image-retrieval systems.

They hope that eventually ALIPR can be used in industry for automatic tagging or as part of Internet search engines, said a release of Penn State.

“Our basic approach is to take a large number of photos – we started with 60,000 – and to manually tag them with a variety of keywords that describe their contents,” said Jia Li, an associate professor of statistics at Penn State.

“For example, we might select 100 photos of national parks and tag them with the following keywords: national park, landscape, and tree. We then would build a statistical model to teach the computer to recognize patterns in colour and texture among these 100 photos and to assign our keywords to new photos that seem to contain national parks, landscapes, and/or trees,” Li added.

Li, who developed ALIPR with her colleague James Wang, of Penn State, said that their approach appropriately assigns to photos at least one keyword among seven possible keywords about 90 percent of the time.

But, she added, the accuracy rate really depends on the evaluator. “It depends on how specific the evaluator expects the approach to be,” she said. “For example, ALIPR often distinguishes people from animals, but rarely distinguishes children from adults.”