Using Web NGrams 3.0 & BigQuery To Track "Shortages of …"

We've published a growing collection of tutorials on how to use the Web News NGrams 3.0 dataset for a range of horizon scanning applications, including realtime commodity and disease tracking. What if you are interested in tracking global shortages? A trivial example of tracking global media coverage of shortages is simply to look for the phrase "shortage of."

To show how easy this is, here is a basic query that searches English-language coverage for the literal phrase "shortage of" and displays the brief following snippet:

SELECT date, url, CONCAT(ngram, ' ', post) snippet FROM `gdelt-bq.gdeltv2.webngrams` WHERE DATE(date) = "2022-09-24" and lang='en' and LOWER(ngram) = 'shortage' and LOWER(post) like 'of %' LIMIT 1000

Of course, in real life, you would search across a range of language and use a much broader array of grammatical constructs, including the plural "shortages of" and myriad other constructs like "not enough of" or looking for noun phrases that are the object of verb with similar meaning to "shortage" and so on. Yet, even within this trivial example, you can already see the powerful results.

snippet url
shortage of troops. The Defense Ministry on Saturday
shortage of the same. Manufacture of toys and
shortage of ministers and celebrants, a report reveals.
shortage of Church of Scotland ministers and celebrants
shortage of affordable housing. "Those are three categories
shortage of awe and marvel in this world
shortage of amazing games; there is something for
shortage of skilled workers. In 2020, only 53%
shortage of raw material imported from cashew cultivating
shortage of labor," she said. The Djokovic drama
shortage of maternity providers in New Zealand, midwives
shortage of facemasks at the time, and we
Shortage of Diesel Fuel Recalls Classic Soviet Anecdote
shortage of available land are long-standing problems: It
shortage of locomotives, inadequate maintenance, vandalism and theft
shortage of diesel in gas stations around the
shortage of intrinsic energies of male you'll find
shortage of troops. The Defense Ministry on Saturday
shortage of nurses at Mahatma Gandhi Memorial Government
shortage of research to back up the very
shortage of foreign exchange but not so with
shortage of crafty ways to prevent people from
shortage of staff in the hospital due to
Shortage of Blue Oval Badges: ReportJalopnik20:05 23-Sept-22 Nostalgia
Shortage of candidates leads to acclaimed seatsGlobal News23:54
Shortage of Diesel Fuel Recalls Classic Soviet Anecdote
Shortage of coffee / An image of a
Shortage of Mustard, Its Uniquely Beloved Condiment] But
shortage of nonfiction climate change books. I watched
shortage of earnest environmental documentaries. Many of them
shortage of vaccines against monkeypox, as many Americans
shortage of food and water, not to mention
shortage of such talents. Global consulting firm Korn
shortage of Moderna vaccine caused some pharmacies to
shortage of health professionals, many of us turn
shortage of activewear brands in the world, but
shortage of wheat. With the delay in planting