Labels Detection w/ Google Cloud Vision

Authentication

First you must authenticate your requests. There are 2 ways to do it.

1. Using an API key
2. Using A Service Account

Request Body and Responses

The body of your POST request contains a JSON object, containing a single requests list, which itself contains one or more objects of type AnnotateImageRequest.

{
  "requests":[
    {
      "image":{
        "content":"/9j/7QBEUGhvdG9...image contents...eYxxxzj/Coa6Bax//Z"
      },
      "features":[
        {
          "type":"LABEL_DETECTION",
          "maxResults":1
        }
      ]
    }
  ]
}

Full documentation here.

Providing The Image

Several ways to do this as well,

1. Base64-encoded image string
2. Google Cloud Storage URI
3. Public Accessible Image

JSON Response Format

The annotate request receives a JSON response of type AnnotateImageResponse. Although the requests are similar for each feature type, the responses for each feature type can be quite different. Consult the Vision API Reference for complete information.

A Quick Demo

# Specify image url
imgURI <- "http://upload.wikimedia.org/wikipedia/commons/thumb/b/bd/
Rembrandt_van_Rijn_-_Self-Portrait_-_Google_Art_Project.jpg/
463px-Rembrandt_van_Rijn_-_Self-Portrait_-_Google_Art_Project.jpg"

# Specify endpoint mentioned above
endpoint <- "https://vision.googleapis.com/v1/images:annotate"

# Set API token in R environment
# options(gcv_api_token = "")

if(!is.null(getOption("gcv_api_token"))){
  baseURI <- paste0(endpoint, "?key=", getOption("gcv_api_token"))
}else{
  stop("API token not set")
}

# Post request body
body <- sprintf('{
        "requests":[
            {
                "image":{
                    "source":{
                        "imageUri":
                            "%s"
                    }
                },
                "features":[
                    {
                        "type":"LABEL_DETECTION",
                        "maxResults":%i
                    }
                    ]
            }
            ]
        }', imgURI, 10
                )

# Send request
res <- httr::POST(baseURI, body = body, encode = "json", config = httr::verbose())

# Extract content (specific mime type)yhat
lbs <- httr::content(res, type = "application/json")

# Convert result to data frame
lbs$responses[[1]]$labelAnnotations %>% bind_rows()

P.S. Check out Google Art and Culture project.