How might Google Cloud Vision API's Web Entities feature help us track a clip of an interview with Dr. Fauci across the web? Take his interview on MSNBC's Andrea Mitchell Reports on July 29, 2020 at 9:13AM PDT. Let's extract a frame from his interview, seen below:
We ask Cloud Vision API to perform a Web Entities reverse image search:
curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $(gcloud auth print-access-token)" https://vision.googleapis.com/v1/images:annotate -d '
{
"requests": [
{
"image": {
"source": {
"gcsImageUri": "[GCSPATH]/MSNBCW_20200729_160000_Andrea_Mitchell_Reports-EXAMPLE-720.png"
}
},
"features": [
{
"maxResults": 200,
"type": "WEB_DETECTION"
},
]
}
]
}' > ./MSNBCW_20200729_160000_Andrea_Mitchell_Reports-EXAMPLE-720.json
You can download the complete "MSNBCW_20200729_160000_Andrea_Mitchell_Reports-EXAMPLE-720.json" output file.
Among the assigned topics, the API recognized that images strongly similar to this one are typically captioned as "Anthony Fauci", "2019–20 coronavirus pandemic" and "National Institute of Allergy and Infectious Diseases". While the frame in question relates to testing lags, assigned topics include "COVID-19 vaccine" and "Hydroxychloroquine" in recognition of his strong associations with the topics in his interviews.
It includes online excerpts of the broadcast, other images from the same broadcast as well as grouping in images from past broadcasts in front of the same background (note the different suit). Another match is a Washington Post interview conducted six days earlier with the Washington Post in the same background and suit, as well as still image captures from other interviews, such as this July 30th one.
