CIFAR 10 Image Classification with Einstein Vision Service

Author : Rajdeep Dua
Last Updated : Aug 22 2017

In this article we look at how Einstein Vision Service can help classify images of famous CIFAR 10 DataSet

Background

Salesforce launched Einstein Vision Service couple of months ago with the aim to help developers provide service which can help them detect and label images. In this article we look at how this service could be successfully used in medicine to detect cervix types.

../_images/einstein_cifar_10_flow.png

DataSet

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

We have reduced the original CIFAR dataset to 1000 images, with 100 images for each label

http://www.cs.utoronto.ca/~kriz/cifar.html

In this article we will train model to correctly classify images to their respective labels.

../_images/cifar_10.png

Our reduced dataset can be found at https://www.dropbox.com/s/sede1obdjznz3ak/cifar-10-100.zip

We uploaded the DataSet using standard curl commands and trained the model

Creating DataSet in Einstein

We need to do a HTTP POST call to https://api.einstein.ai/v2/vision/datasets/upload/sync with appropriate parameters as shown below.

curl -X POST -H "Authorization: Bearer <TOKEN>" \
             -H "Cache-Control: no-cache" \
             -H "Content-Type: multipart/form-data" \
             -F "path=https://www.dropbox.com/s/sede1obdjznz3ak/cifar-10-100.zip?raw=1" \
             -F "type=image" https://api.einstein.ai/v2/vision/datasets/upload

Note : We are using Asynch request and you will have to ping the endpoint below to check training status

Response will be similar to listing below

{
   "id":1010381,
   "name":"cifar-10-100; filename*=UTF-8''cifar-10-100",
   "createdAt":"2017-08-22T08:59:11.000+0000",
   "updatedAt":"2017-08-22T08:59:11.000+0000",
   "labelSummary":{
      "labels":[

      ]
   },
   "totalExamples":0,
   "available":false,
   "statusMsg":"UPLOADING",
   "type":"image",
   "object":"dataset"
}

Note the dataset id which is 1009482 in our case.

Check Upload Status of the DataSet

curl -X GET -H "Authorization: Bearer <TOKEN>" \
            -H "Cache-Control: no-cache" \
            https://api.einstein.ai/v2/vision/datasets/1010381
{
   "id":1010381,
   "name":"cifar-10-100; filename*=UTF-8''cifar-10-100",
   "createdAt":"2017-08-22T08:59:11.000+0000",
   "updatedAt":"2017-08-22T08:59:13.000+0000",
   "labelSummary":{
      "labels":[
         {
            "id":95567,
            "datasetId":1010381,
            "name":"automobile",
            "numExamples":100
         },
         {
            "id":95568,
            "datasetId":1010381,
            "name":"deer",
            "numExamples":100
         },
         {
            "id":95569,
            "datasetId":1010381,
            "name":"horse",
            "numExamples":100
         },

         ....

      ]
   },
   "totalExamples":1000,
   "totalLabels":10,
   "available":true,
   "statusMsg":"SUCCEEDED",
   "type":"image",
   "object":"dataset"
}

Training the DataSet

We will make a HTTP POST call to URL https://api.einstein.ai/v2/vision/datasets/<id> where <datasetId> is 1010381 in our case

  curl -X GET -H "Authorization: Bearer <TOKEN>" \
              -H "Cache-Control: no-cache" \
               https://api.einstein.ai/v2/vision/datasets/1010381

curl -X POST -H "Authorization: Bearer <TOKEN>" \
             -H "Cache-Control: no-cache" \
             -H "Content-Type: multipart/form-data" \
             -F "name=cifar 10 Model" \
             -F "datasetId=<DATASET ID>" https://api.einstein.ai/v2/vision/train
{
   "datasetId":1010381,
   "datasetVersionId":0,
   "name":"cifar-10 model",
   "status":"QUEUED",
   "progress":0,
   "createdAt":"2017-08-22T09:00:19.000+0000",
   "updatedAt":"2017-08-22T09:00:19.000+0000",
   "learningRate":0.0,
   "epochs":0,
   "queuePosition":3,
   "object":"training",
   "modelId":"GF7K6S6AAJW5XSTW7IKP64MZXU",
   "trainParams":null,
   "trainStats":null,
   "modelType":"image"
}

Once the model is trained the model metrics can be seen

     {
"createdAt":"2017-08-22T09:03:51.000+0000",
"metricsData":{
   "f1":[
      0.8461538461538461,
      0.8333333333333333,
      0.5,
      0.7058823529411765,
      0.7999999999999999,
      0.6666666666666666,
      0.6666666666666665,
      0.7000000000000001,
      0.8421052631578948,
      0.5833333333333334
   ],
   "labels":[
      "automobile",
      "deer",
      "horse",
      "frog",
      "airplane",
      "bird",
      "cat",
      "truck",
      "ship",
      "dog"
   ],
   "testAccuracy":0.7255,
   "trainingLoss":0.6793,
   "confusionMatrix":[
     [11,0,0,0,0,0,0,1,0,0],
     [0,10,0,0,0,0,0,0,0,0],
     [0,0,3,0,0,0,0,0,0,1],
     [0,1,0,6,0,0,1,1,0,0],
     [1,0,0,0,8,0,0,0,0,0],
     [0,1,0,2,1,5,0,0,0,1],
     [0,2,1,0,0,0,9,1,0,2],
     [1,0,0,0,1,0,0,7,0,0],
     [1,0,0,0,1,0,0,1,8,0],
     [0,0,4,0,0,0,2,0,0,7]
   ],
   "trainingAccuracy":0.7573
}

As can be seen automobile was most accurately predicted by the model during validation and horse was least accurate.

Predicting the Label from the image Type

Let us take an from the test dataset and try to predict the Label

Prediction Command

We will send a prediction HTTP POST request to Url https://api.einstein.ai/v2/vision/predict with authorization token and path to the image to be predicted.

curl -X POST -H "Authorization: Bearer <TOKEN>" \
             -H "Cache-Control: no-cache" \
             -H "Content-Type: multipart/form-data" \
             -F "sampleId=Photo Prediction"  \
             -F "sampleContent=@./airplane/10638.png" \
             -F "modelId=GF7K6S6AAJW5XSTW7IKP64MZXU" \
             https://api.einstein.ai/v2/vision/predict

Results

{
   "sampleId":"Photo Prediction",
   "probabilities":[
      {
         "label":"airplane",
         "probability":0.99966997
      },
      {
         "label":"ship",
         "probability":2.426601E-4
      },
      {
         "label":"bird",
         "probability":4.639914E-5
      },
      {
         "label":"automobile",
         "probability":2.3061095E-5
      },
      {
         "label":"deer",
         "probability":1.1689458E-5
      }
   ],
   "object":"predictresponse"
}

Prediction Result

  • Predicted Response : Label : Airplane
  • Prediction Accuracy : 99.99%

Summary

As we can see the CIFAR-10 images can be quite accurately classified even though we were training with a much reduced dataset.