Using Web NGrams 3.0 & BigQuery To Track "Shortages of …"

We've published a growing collection of tutorials on how to use the Web News NGrams 3.0 dataset for a range of horizon scanning applications, including realtime commodity and disease tracking. What if you are interested in tracking global shortages? A trivial example of tracking global media coverage of shortages is simply to look for the phrase "shortage of."

To show how easy this is, here is a basic query that searches English-language coverage for the literal phrase "shortage of" and displays the brief following snippet:

SELECT date, url, CONCAT(ngram, ' ', post) snippet FROM `gdelt-bq.gdeltv2.webngrams` WHERE DATE(date) = "2022-09-24" and lang='en' and LOWER(ngram) = 'shortage' and LOWER(post) like 'of %' LIMIT 1000

Of course, in real life, you would search across a range of language and use a much broader array of grammatical constructs, including the plural "shortages of" and myriad other constructs like "not enough of" or looking for noun phrases that are the object of verb with similar meaning to "shortage" and so on. Yet, even within this trivial example, you can already see the powerful results.

snippet url
shortage of troops. The Defense Ministry on Saturday https://www.wkyc.com/article/news/nation-world/ukraine/russia-ukraine-votes/507-63c2c334-3bc5-4147-b138-7fe0e4ce3a06
shortage of the same. Manufacture of toys and https://www.khaleejtimes.com/business/5g-network-set-to-boost-iot-sector-growth-in-india
shortage of ministers and celebrants, a report reveals. https://www.obantimes.co.uk/2022/09/24/registrars-conducting-funerals/
shortage of Church of Scotland ministers and celebrants https://www.obantimes.co.uk/2022/09/24/registrars-conducting-funerals/
shortage of affordable housing. "Those are three categories https://www.sentinelsource.com/news/local/statehouse/cheshire-house-7-candidates-agree-on-housing-needs-differ-on-guns/article_9d64d826-728e-59f2-8551-f25f984563ac.html
shortage of awe and marvel in this world https://www.mdjonline.com/tribune/georgia/starlink-launch-brings-wonder-to-some-in-the-isles/article_e5783f08-48a5-5c13-a57b-2ed38f73e08d.html
shortage of amazing games; there is something for https://www.gamepur.com/guides/roblox-sword-lands-simulator-codes-september-2022
shortage of skilled workers. In 2020, only 53% https://www.therepublic.com/2022/09/07/editorial-indianas-future-economy-demands-relentlessness/
shortage of raw material imported from cashew cultivating https://www.thehindu.com/news/national/kerala/kscdc-to-seek-gi-tag-for-kerala-cashew/article65931049.ece
shortage of labor," she said. The Djokovic drama https://www.nytimes.com/2022/02/18/travel/australia-tourism-covid.html
shortage of maternity providers in New Zealand, midwives http://www.midwivesoutwest.co.nz/
shortage of facemasks at the time, and we http://m.desototimes.com/news/desoto-county-local-finds-peace-through-sewing/article_c1232628-3b70-11ed-ad82-73592a3df6d1.html
Shortage of Diesel Fuel Recalls Classic Soviet Anecdote https://www.newsnow.co.uk/h/World+News/Middle+East/Saudi+Arabia
shortage of available land are long-standing problems: It https://www.washingtonpost.com/business/energy/we-need-a-war-effort-on-wind-turbines/2022/09/17/684721dc-3646-11ed-a0d6-415299bfebd5_story.html
shortage of locomotives, inadequate maintenance, vandalism and theft https://www.aljazeera.com/news/2022/8/26/s-africas-transnet-unions-in-wage-deadlock-as-strike-looms
shortage of diesel in gas stations around the https://abcnews.go.com/International/wireStory/higher-fuel-prices-spark-social-unrest-south-america-85889179
shortage of intrinsic energies of male you'll find http://togozine.com/there-are-visible-behavioral-differences-between-2/
shortage of troops. The Defense Ministry on Saturday https://www.wishtv.com/news/international/russians-strike-ukraine-as-kremlin-staged-votes-continue/
shortage of nurses at Mahatma Gandhi Memorial Government https://www.thehindu.com/news/cities/Tiruchirapalli/tiruchi-government-hospital-faces-severe-shortage-of-staff-nurses/article65930925.ece
shortage of research to back up the very http://www.msn.com/en-us/health/nutrition/add-these-disease-fighting-foods-to-your-diet-to-live-a-longer-life/ss-AA11wNi5
shortage of foreign exchange but not so with https://dailypost.ng/2022/09/24/emmanuel-onwubiko-nigerias-corruption-a-challenge-of-governance/
shortage of crafty ways to prevent people from https://www.bigissue.com/news/housing/anti-homeless-architecture-hostile-designs/
shortage of staff in the hospital due to https://www.thehindu.com/news/cities/Tiruchirapalli/tiruchi-government-hospital-faces-severe-shortage-of-staff-nurses/article65930925.ece
Shortage of Blue Oval Badges: ReportJalopnik20:05 23-Sept-22 Nostalgia https://www.newsnow.co.uk/h/Industry+Sectors/Automotive
Shortage of candidates leads to acclaimed seatsGlobal News23:54 https://www.newsnow.co.uk/h/World+News/North+America/Canada/Provinces+and+Territories/Manitoba
Shortage of Diesel Fuel Recalls Classic Soviet Anecdote https://www.newsnow.co.uk/h/World+News/Asia/Russia/Moscow
Shortage of coffee / An image of a https://www.industryleadersmagazine.com/brazils-shortage-of-coffee-stockpiles-hit-the-sore-spot/
Shortage of Mustard, Its Uniquely Beloved Condiment] But https://www.nytimes.com/2022/09/24/world/canada/a-year-when-everything-is-going-right-for-western-canadas-farmers.html
shortage of nonfiction climate change books. I watched https://www.publishersweekly.com:443/pw/by-topic/authors/interviews/article/90417-is-this-the-way-the-world-ends-pw-talks-with-stephen-markley.html
shortage of earnest environmental documentaries. Many of them https://www.publishersweekly.com:443/pw/by-topic/authors/interviews/article/90417-is-this-the-way-the-world-ends-pw-talks-with-stephen-markley.html
shortage of vaccines against monkeypox, as many Americans https://www.fsunews.com/story/news/2022/08/28/fsu-professors-discuss-implications-monkeypox/7918518001/
shortage of food and water, not to mention https://www.al-monitor.com/originals/2022/08/fires-engulf-displacement-camps-northern-syria-during-heat-wave
shortage of such talents. Global consulting firm Korn https://www.malaymail.com/news/singapore/2022/09/24/can-singapores-bid-to-woo-elite-global-talents-succeed/29867
shortage of Moderna vaccine caused some pharmacies to https://www.floridastatesman.com/news/272791021/44m-americans-have-rolled-up-sleeves-for-omicron-targeted-boosters
shortage of health professionals, many of us turn http://tnlivenews.in/stock-market-news-live-updates-stocks-crater-dow-hits-2022-low-and-oil-plunges-as-fed-and-growth-fears-roil-markets-yahoo-finance/
shortage of activewear brands in the world, but https://www.koco.com/article/best-hype-meter/41333351
shortage of wheat. With the delay in planting https://www.daijiworld.com/news/newsDisplay?newsID=1003170