Visual Search

Very large-scale computer vision solutions that help people discover more about the world around them.

Our goal is to create solutions that are so simple to use, so fast and convenient, that customers prefer them to traditional search methods.

-Sunil R., Visual Search

Augmented Reality: Using a camera-phone as the view of the world

Our technology helps customers use visual information for search, discovery and shopping.  We create Augmented Reality solutions on mobile devices, overlaying relevant information over camera-phone views of the world around us.

It is often easier to search with a picture than it is to type. We build technologies to recognize objects in camera-phone views. We also support barcode and QR code recognition on mobile devices.

A9 Visual Search also powers solutions that lets customers search for products based on their visual attributes such as color, shape or even texture.  Such solutions appear on Amazon.com and Zappos.com, allowing customers to quickly find the shoes or watches they like based on the appearance of the product.

Mobile Apps and Much More

Visual search solutions are ideally suited to mobile applications, especially for devices that have cameras. Our technology powers several apps for the Amazon family of companies. 

Our continuous scan technology, which powers A9’s Flow app, is at the heart of Amazon’s augmented reality efforts and allows customers to seamlessly scan products of interest, discovering useful information about them.

Visual Search also develops computer vision solutions that support Amazon initiatives along the entire product delivery pipeline: from the time a new product is photographed and added to our catalog to the time an item is bought and shipped to the customer.

Illustration: Visual Search

What We Work On

Our research efforts span these major areas:

  • Image Matching algorithms enable the exact identification of objects and drive the functionality in mobile apps like A9’s Flow, as well as Amazon Mobile. It also supports Amazon’s Universal Wish List feature.
  • Image classification powers the shape picker and color picker on Amazon’s apparel sites.
  • Visual similarity underpins the "View Visually Similar Items" feature in Watches, Handbags and Shoes on Amazon.com. 
  • OCR, text recognition, geospatial recognition extends our visual recognition capabilities for the world of objects around us.
  • Robust distributed computing infrastructure, built on the latest cloud computing services from Amazon, including EC2, S3, enables the scale and efficiency of our solutions. 

Image Matching: Applying Vision Science To The Real World

One of our core capabilities is to efficiently search a large corpus of images for the best match to a user’s query image – even when that query image is a noisy, partial, occluded, blurred, rotated, and scaled version of the corpus image.  In other words, even when the query image was taken without much care, and with a low-end camera.  

This solution relies on large-scale, distributed techniques for:

  • Image retrieval.
  • Feature compression.
  • Object detection and tracking.
  • 3D Object detection.
  • Hardware optimized computation. 
  • Deployment of solutions at scale.