NEW YORK — Imagine using your phone to snap a photo of the cool pair of sunglasses your friend is wearing and instantly receiving a slew of information about the shades along with a link to order them.
It's a great idea — but it doesn't quite work.
Though many companies are trying to make "visual search" a reality, this seemingly simple notion remains elusive.
Take Amazon, which made visual search a key feature in its new Fire smartphone. The e-commerce company says the feature, known as Firefly, can recognize 100 million items. It's similar to a Flow feature Amazon has on its apps for other phones.
So far, Firefly can reliably make out labels of products such as Altoids or Celestial Seasonings tea. That makes it easy to buy items such as groceries online.
But try it on a checkered shirt or anything without sharp corners, and no such luck.
"It works really well when we can match an image to the product catalog," says Mike Torres, an Amazon executive who works on the Fire's software. "Where things are rounded or don't have (visual markers) to latch on to, like a black shoe, it's a little harder to do image recognition."
Visual search is important to retailers because it makes mobile shopping a snap — literally.
It's much easier to take a picture than to type in a description of something you want. Shopping on cellphones and tablets is still a small part of retail sales, but it's growing quickly. That makes it important to simplify the process as much as possible — especially as people look to visual sites such as Instagram and Pinterest as inspiration for purchases.
"Retailers are trying to get the user experience simple enough so people are willing to buy on their phones, not just use it as a research tool," eMarketer analyst Yory Wurmser said.