analyze: Interface to Azure Computer Vision API

Description

Interface to Azure Computer Vision API

Usage

analyze(endpoint, image, domain = NULL, feature_types = NULL,
  language = "en", ...)
describe(endpoint, image, language = "en", ...)
detect_objects(endpoint, image, ...)
area_of_interest(endpoint, image, ...)
tag(endpoint, image, language = "en", ...)
categorize(endpoint, image, ...)
read_text(endpoint, image, detect_orientation = TRUE, language = "en", ...)
list_computervision_domains(endpoint, ...)
make_thumbnail(endpoint, image, outfile, width = 50, height = 50,
  smart_crop = TRUE, ...)

Arguments

endpoint

A computer vision endpoint.

image

An image to be sent to the endpoint. This can be either a filename, a publicly accessible URL, or a raw vector holding the file contents.

domain

For analyze, an optional domain-specific model to use to analyze the image. Can be "celebrities" or "landmarks".

feature_types

For analyze, an optional character vector of more detailed features to return. This can be one or more of: "categories", "tags", "description", "faces", "imagetype", "color", "adult", "brands" and "objects". If not supplied, defaults to "categories".

language

A 2-character code indicating the language to use for tags, feature labels and descriptions. The default is en, for English.

...

Arguments passed to lower-level functions, and ultimately to call_cognitive_endpoint.

detect_orientation

For read_text, whether to automatically determine the image's orientation.

outfile

For make_thumbnail, the filename for the generated thumbnail. Alternatively, if this is NULL the thumbnail is returned as a raw vector.

width, height

For make_thumbnail, the dimensions for the returned thumbnail.

smart_crop

For make_thumbnail, whether to automatically determine the best location to crop for the thumbnail. Useful when the aspect ratios of the original image and the thumbnail don't match.

Value

analyze returns a list containing the results of the analysis. The components will vary depending on the domain and feature types requested.

describe returns a list with two components: tags, a vector of text labels; and captions, a data frame of descriptive sentences.

detect_objects returns a dataframe giving the locations and types of the detected objects.

area_of_interest returns a length-4 numeric vector, containing the top-left coordinates of the area of interest and its width and height.

tag and categorize return a data frame of tag and category information, respectively.

read_text returns the extracted text as a list with one component per region that contains text. Each component is a vector of character strings.

list_computervision_domains returns a character vector of domain names.

make_thumbnail returns a raw vector holding the contents of the thumbnail, if the outfile argument is NULL. Otherwise, the thumbnail is saved into outfile.

Details

analyze extracts visual features from the image. To obtain more detailed features, specify the domain and/or feature_types arguments as appropriate.

describe attempts to provide a text description of the image.

detect_objects detects objects in the image.

area_of_interest attempts to find the "interesting" part of an image, meaning the most likely location of the image's subject.

tag returns a set of words that are relevant to the content of the image. Not to be confused with the add_tags or add_image_tags functions that are part of the Custom Vision API.

categorize attempts to place the image into a list of predefined categories.

read_text performs optical character recognition (OCR) on the image.

list_domains returns the predefined domain-specific models that can be queried by analyze for deeper analysis. Not to be confused with the domains available for training models with the Custom Vision API.

make_thumbnail generates a thumbnail of the image, with the specified dimensions.

Examples

Run this code

# NOT RUN {
vis <- computervision_endpoint(
    url="https://accountname.cognitiveservices.azure.com/",
    key="account_key"
)

list_domains(vis)

# analyze a local file
analyze(vis, "image.jpg")
# picture on the Internet
analyze(vis, "https://example.com/image.jpg")
# as a raw vector
analyze(vis, readBin("image.jpg", "raw", file.size("image.jpg")))

# analyze has optional extras
analyze(vis, "image.jpg", feature_types=c("faces", "objects"))

describe(vis, "image.jpg")
detect_objects(vis, "image.jpg")
area_of_interest(vis, "image.jpg")
tag(vis, "image.jpg")  # more reliable than analyze(*, feature_types="tags")
categorize(vis, "image.jpg")
read_text(vis, "scanned_text.jpg")

# }

Run the code above in your browser using DataCamp Workspace