Microsoft pioneers image-based web search
19 April 2006
Microsoft is researching a technology that would allow users to perform a web search using an image snapped on a camera phone.
According to a Microsoft employee's blog posting, the Photo2Search technology gives users a way to search a web-based database by using nothing more than an image captured by a phone.
Xing Xie, a researcher for the Web Search and Mining group within Microsoft Research Asia, is working on technology. "Camera phones are simple to use, but the process of text-based search on them is not," he said. "As the old saying goes, a picture is worth a thousand words."
With Photo2Search works a user takes a photo of the object and sends the photo, via email or Multimedia Messaging Service (MMS), to a server which searches an image database for matches. The server then delivers database information on the object much like a typical web search today.
Xie said that most prior work in this area had been based on Content Based Image Retrieval (CBIR), which index images by colour, texture, shape, object layout, and edge direction. But such approaches take considerable computational resources and do not necessarily excel at locating references with the same prominent object pictured in a query.
"The precision of CBIR is not sufficient for practical use," Xie said. So last year Xie and colleagues built a system with image matching based on computer-vision algorithms that extract features from images. This system resulted in an efficient index that can search through a large image database and return results quickly.
Although the searchable database still needs to be a predefined collection of images, they can be harvested from the web.
It remains to be seen how Photo2Search might be incorporated into a commercial product, but Xie and his colleagues are continuing to work on support for larger databases.
"We hope, in the future, when a user submits a photo to, for example, MSN Spaces, we can quickly figure out the latitude and the longitude of that photo by using our technology," he said.
To comment on any articles, please contact us at chatback@telecoms.com or have your say on our blog.
Bookmark this page
