Einstein Icon

Einstein Vision

Einstein Vision is an image recognition API that equips you to quickly build AI-enabled apps. If you are new to Einstein Vision, you can check out the Einstein Vision Quick Start.

In this Dreamhouse sample real estate application, you explore image recognition functionality. This app demonstrates how users can experience a more personalized home-buying journey as they’re able to take pictures of homes that suit their preferences, and then see similar houses for sale. In order to accomplish this with Einstein Vision, we trained an image recognition model using images of houses that were labeled according to their respective classifications such as “Tudor” or “Greek Revival”.

Check out this demo of “Dreamhouse” to see Einstein Vision in action:

Einstein Vision uses a series of REST APIs that can be used with any language to work with datasets, labels, and model predictions to simplify the process of building models. The docs include examples based on Curl (a command-line HTTP client), which is useful for testing. There are also a number of client libraries that simplify the process of using the REST APIs:

Dreamhouse uses an admin app to manage the training dataset and model outputs; it is simply a wrapper around the Einstein Vision REST API, with Dreamhouse specific logic. The Dreamhouse Web App UI allows end users to upload images which are then sent to Einstein Vision to be classified to match user preferences. To get the best results with Einstein Vision, you need a large training dataset that is accurately classified into labels. For Dreamhouse we used a dataset of almost 300 images and classified them into various property types such as Tudor, Greek Revival, and Victorian. The magic of Einstein Vision happens when it recognizes new images that it’s never seen before and classifies them. But all magic has its limits; if you upload a picture of a cat and use a model that has never seen cats, Einstein Vision will do its best, but it likely won’t be as accurate. It’s important to provide an accurate training dataset to get the best results.

Einstein Vision

Einstein Vision has a pre-built model that classifies all sorts of things, but it’s best if used for general purpose applications rather than specific use-cases. Einstein Vision is also available on Heroku for those using Heroku to manage your applications.

To get the Dreamhouse Einstein Vision sample application working on your own, follow these steps:

  1. Deploy the Dreamhouse Einstein Vision app on Heroku: Deploy
  2. From your newly deployed app, create a dataset using a URL of: http://www.dreamhouseapp.io/Dreamhouse.zip
  3. Train and test a model
  4. Deploy the Dreamhouse Web App app on Heroku: Deploy
  5. Try out the Photo Search feature in your newly deployed Dreamhouse Web App