The GDELT Project

TFIDF Using BigQuery + Radio News NGrams To Chart The Most Significant Words Per Day On BBC World Service In 2020

How might we use the new Radio News NGrams dataset to examine the Internet Archive's Radio News Archive's ASR of the BBC World Service over the course of 2020 to estimate the top 10 most "significant" words defining the World Service's coverage each day? Using a single SQL query we can compute TFIDF scores over the dataset and surface the top 10 words each day that were spoken on the BBC World Service that day compared with BBC World Service the rest of the year.

Remember that this is based on automatic speech recognition results, so there may be errors and that this list is based on simple word frequency histograms rather than the semantic importance of each word to each day's coverage.

Note that the TFIDF calculation by its nature strips out words that are so common that they no longer distinguish a given day from other days. This means that COVID-19, which has remained a fixture of 2020 coverage is not well-represented in the table below because it appears in each day's coverage.

Day Top Words
1/1/2020 carols, sulfur, jakarta, baghdad, knitting, bushfires, slapping, cyprus, beetlejuice, embassy
1/2/2020 mammograms, radiologists, phosphate, desmond's, phosphates, radiologist, morocco, greens, jakarta, yiddish
1/3/2020 testicular, baghdad, qassam, iran's, yiddish, standage, punks, airstrike, chemotherapy, thurman
1/4/2020 baghdad, iraqi, yiddish, procession, chippendales, cartel, airstrike, desmond's, renewables, bribes
1/5/2020 sheba, yiddish, noriega's, iraqi, mannheim, noriega, alderman, lamu, globes, solomon
1/6/2020 weinstein, harvey, globes, tehran, jnu, rapist, $1400000000.00, sirte, ayatollah, uranium
1/7/2020 travelex, zarif, bundanoon, cyprus, satellites, jello, burial, mourners, chippendales, hackers
1/8/2020 ballistic, ghosn, tappers, missile, iranians, ukrainian, tehran, recorders, denise's, 737
1/9/2020 ukrainian, lyme, cocoa, highlife, tussauds, millennial, 176, missile, meghan, kiev
1/10/2020 apnea, camels, ukrainian, 176, airliner, monarchies, lendu, snoring, missile, soju
1/11/2020 qaboos, sultan, tenants, bloodless, taiwan's, rental, affordable, housing, barbershops, jemele
1/12/2020 kazakhstan, narcopolis, wades, soothe, bombay, gz, benedict, scruton, celibacy, haitians
1/13/2020 volcano, celibacy, benedict, eruption, sandringham, lava, diac, nantucket, manila, airliner
1/14/2020 manipulator, volcano, benedict, revalidation, astros, ginko, mutiny, impeachment, libyan, murdoch
1/15/2020 origami, impeachment, milf, bezos, miscarriage, cholesterol, astros, mindanao, exoskeleton, frogs
1/16/2020 impeachment, stinney, sepsis, 2047, giuliani, honey, parnassus, parness, prevaricate, ukraine
1/17/2020 sepsis, philanthropy, impeachment, honey, ayatollah, goebbels, alopecia, infinity, mcraven, selenski
1/18/2020 philanthropy, honey, hydrogen, philanthropists, dershowitz, overreached, impeachment, starr, binyon, mcraven
1/19/2020 hydrogen, infinity, greenlandic, honey, greenland, countable, hezbollah, quantum, impeachment, algebra
1/20/2020 santos, fungus, frogs, toads, angolan, toad, vancouver, davos, angola, extradition
1/21/2020 impeachment, davos, dodo, mcconnell, mitch, schiff, senators, sperm, dodos, prophets
1/22/2020 bezos, impeachment, python, platypus, davos, weinstein, monty, arabia's, greenland, zanzibar
1/23/2020 auschwitz, vashem, myanmar, yad, darwin, genocide, holocaust, suu, icj, impeachment
1/24/2020 darwin, impeachment, wetlands, locusts, erasmus, salvini, waterways, santos, harrisburg, wozniak
1/25/2020 dodo, impeachment, darwin, auschwitz, erasmus, fungus, 270, jakes, dam, whales
1/26/2020 lilo, scrolls, darwin, lindemann, volcanoes, erasmus, gestapo, salvini, impeachment, romagna
1/27/2020 auschwitz, bryant, kobe, lakers, romagna, vegan, basketball, 75th, grammys, salvini
1/28/2020 palestinians, netanyahu, huawei, cheetahs, dodo, oils, stormiest, antarctic, palestinian, bryant's
1/29/2020 palestinians, bolton, compacted, impeachment, palestinian, stabber, turners, lilo, recapturing, $22000000000.00
1/30/2020 sinclair, recycling, butterflies, bolton, c.j., becky, becky's, beltran, chapo, cocaine
1/31/2020 cervical, impeachment, bracks, kiev, wavering, breck's, sunderland, renewal, nike's, brics
2/1/2020 beethoven, impeachment, t.-t., palestinians, plymouth, recycling, bracks, 259, thorpe, garbin
2/2/2020 beethoven, timbuktu, manuscripts, $170000000000.00, stampede, colomba, bystander, beethoven's, stratham, pentecostal
2/3/2020 kansas, $170000000000.00, chiefs, barnier, ers, caucus, iowa, stabbing, regenerative, typhoid
2/4/2020 iowa, hydrogen, caucus, dowager, caucuses, zuma, empress, moines, des, 425
2/5/2020 impeachment, penguins, compression, gestures, 490, 24300, caucus, skidded, iowa, antivenom
2/6/2020 gluten, spartacus, impeachment, douglas, celiac, kirk, cabazon, 560, bumblebees, acquittal
2/7/2020 qaeda, suisse, elephants, raimi, cassim, impeachment, peninsula, mandela, arabian, botswana
2/8/2020 impeachment, hankins, abstinence, lavrov, nixon, saskatoon, watergate, freehold, hampshire, hawkins
2/9/2020 fein, naga, subsonic, rampage, fianna, rouge, orientation, earthquake, bindu, aliyev
2/10/2020 fein, equifax, parasite, oscars, cdu, karrenbauer, workhouse, junho, ripper, aliyev
2/11/2020 shabalala, mambazo, darfur, j.k., bashir, rowling, phyllis, hampshire, pele, correa
2/12/2020 celibacy, salvini, hampshire, b.p., saskatoon, hamlet, priestly, catalina, bernie, priests
2/13/2020 dresden, 242, javid, sajid, stupendous, fossils, sushi, exchequer, reshuffle, turtles
2/14/2020 polanski, westerdam, menstruating, locusts, kqed, valentine's, butterfly, bronze, 20.7, cambodia
2/15/2020 mccabe, camorra, swimsuit, kiir, munich, salva, reformists, bronze, hankins, pageant
2/16/2020 gospel, swimsuit, flack, kanye, hip, magnitogorsk, 355, bronze, yokohama, pageant
2/17/2020 voyager, holden, pensions, albinism, pension, cameroon, robo, composting, vocational, 355
2/18/2020 scouts, hsbc, cello, gezi, bezos, oscillations, gypsy, m.s., jacqueline, quartet
2/19/2020 ammonia, yokohama, gospel, blagojevich, bloomberg, credentials, assange, suliman, shanidar, radicalization
2/20/2020 bloomberg, gospel, eugene, mosquitoes, mosquito, shisha, kock, cowley, src, valka
2/21/2020 romulus, antibiotic, b.n., nile, cousteau, antarctica, baobab, chun, daegu, greenpeace
2/22/2020 nile, shabalala, l'arche, mambazo, mosquito, dam, mosquitoes, machar, juba, salva
2/23/2020 mosquitoes, nevada, rikers, mosquito, cordoba, caucuses, bulldozer, wilder, tyson, fury
2/24/2020 weinstein, salazar, dershowitz, harvey, torrey's, mahathir, carnival, zombies, carnitine, zombie
2/25/2020 mubarak, hosni, weinstein, harvey, cfa, burgess, bryant's, motorola, mubarak's, bryant
2/26/2020 mubarak, hosni, confraternity, iger, bilingual, sharapova, monolingual, gaza, bernie, hindus
2/27/2020 heathrow, smallpox, voltaire, runway, retaken, pilgrims, cussler, bloomberg, rikers, nutrients
2/28/2020 polanski, firestone, voltaire, turkish, airstrike, galaxy, password, gadhafi, tattoos, syrian
2/29/2020 polanski, gadhafi, taliban, dax, doha, mahathir, benghazi, luxemburg, afghans, blasphemy
3/1/2020 oracle, dalai, voltaire, holiness, mahathir, lama, tibet, tibetans, oracles, cia's
3/2/2020 pius, womenomics, vatican, papacy, juarez, asylum, netanyahu, rivlin, indiana, gantz
3/3/2020 netanyahu, likud, leaded, endorsements, bernie, fumigation, petrol, herbicide, coca, subcontractor
3/4/2020 bloomberg, bernie, libra, primaries, grunwick, helmand, biden, delegates, samoa, lobsters
3/5/2020 selassie, icc, haile, fried, qualia, glassman, eels, psychiatry, psychiatric, organophosphate
3/6/2020 clearview, opec, bethlehem, selassie, haile, bleaching, beetle, idlib, bounty, warplanes
3/7/2020 nayef, tyner, selassie, mccoy, haile, womenomics, skolkovo, nagarajan, woody, reddit
3/8/2020 selassie, qatif, lombardy, skolkovo, nanny, haile, houseboat, polygamy, polygamous, 16000000
3/9/2020 rosewood, sydow, 298, skolkovo, m.h., opec, 366, bergman, ingmar, introverts
3/10/2020 kakapo, inspirations, 2036, 157, geldof, zeppelin, pornhub, zaire, giraffes, 737
3/11/2020 weinstein, dorries, pell, nadine, harvey, bernie, cardinal, primaries, treasurer, danube
3/12/2020 weinstein, rosewood, niemeyer, crypto, harvey, hanks, wastes, taji, flugel, capitan
3/13/2020 marburg, rosewood, wiki, frederica, patronymic, manning, nikkei, ivory, niemeyer, griffiths
3/14/2020 rosewood, marburg, niemeyer, patronymic, chile's, frederica, penguin, monkeys, spinny, pla
3/15/2020 birding, otero, ballet, netanyahu, gantz, brewhouse, clangers, weinstein, bumper, taipei
3/16/2020 $700000000000.00, hilda, surfing, 87000, 368, antivirus, liquidity, kakapo, slashing, felons
3/17/2020 mackinnon, cashless, phages, $130000000000.00, mural, mahal, $400000000000.00, marburg, vinson, taj
3/18/2020 marsha, illinois, tigers, ioc, i.g., bernie, contestants, marcia, primaries, $400000000000.00
3/19/2020 $820000000000.00, hubble, ecb, telescope, skid, hamlin, ioc, pinsent, quantas, binge
3/20/2020 hypersonic, lenin, zanzibar, vampire, 627, salazar, ancestry, artemisinin, hanged, newsome
3/21/2020 varoufakis, flanders, rogers, kenny, yanis, bondi, parton, lucille, lenin, baffin
3/22/2020 dmc, ancestry, 394, mcdaniels, hezbollah, co-working, blasio, gorillas, prague, sondheim
3/23/2020 ioc, heffernan, denman, vegetative, shinzo, ancestry, oecd, reanimation, regina, trombone
3/24/2020 phosphorus, phosphorous, asterix, sludge, vegetative, taubenberger, struvite, hippos, phosphate, seti
3/25/2020 module, aldrin, dmc, 738, seti, apollo, 1300000000, lunar, mcdaniels, lunny
3/26/2020 3300000, module, levinson, fendel, ioc, brenton, overkill, minimizes, brewery, d.m.c.
3/27/2020 cleve, ipl, macedonia's, populism, counter-point, d.m.c., polio, quorum, 85000, artemisia
3/28/2020 seti, 832, populism, defectors, beep, cleve, drager, lowery, shostak, ingrid
3/29/2020 defectors, penderecki, module, epigenetics, 838, 756, defector, mcqueen, rosneft, koreans
3/30/2020 oxfam, chromosome, panther, clementine, masculinity, $160000000000.00, bunkers, 812, sunk, 418
3/31/2020 chromosome, ingrid, aspen, celeste, quaking, 849, satrapi, marjane, mountjoy, dorchester
4/1/2020 defectors, blasphemy, module, porn, griffin, nightingales, defector, interferon, koreans, jamal
4/2/2020 epicurus, epicurean, 6600000, module, rumania, vasa, epicureanism, vassar, lucretia's, atoms
4/3/2020 navigator, perishable, trampoline, lettuce, withers, lang, exporters, pork, caloric, maximo
4/4/2020 interpol, starmer, corbyn, 809, baptism, eritrea, tranquilized, foxx, rodeo, nikita
4/5/2020 epicurus, epicurean, blasphemy, placebo, ranveer, jabril, gadhafi, 674, fujifilm, pentland
4/6/2020 placebo, epicurus, homebrew, epicurean, antarctica, refereeing, antarctic, transcontinental, isotope, levine
4/7/2020 pell, cardinal, nimrud, reliefs, proscar, nikita, treasurer, profs, foxx, robb
4/8/2020 prine, bhutto, postpartum, kato, fatima, psychosis, ferrari, crescent, hebdo, module
4/9/2020 ddp, allergic, clownfish, 799, seche, penicillin, offenders, disengagement, 881, sirleaf
4/10/2020 nato, mercury, opec, 500000000000, stoltenberg, comet, eyeliner, easter, securitas, barrels
4/11/2020 cardinal, easter, pell, 2108, eyeliner, atheists, rothschild, volcano, wristbands, aldridge
4/12/2020 basilica, easter, peter's, assange, voodoo, wyoming's, 431, potsdam, ecuadorian, indemnify
4/13/2020 vagina, opec, tornadoes, hymen, daisy, takeovers, apollo, logan, andronicus, hibernation
4/14/2020 decibels, adh, honking, diving, imf, rothschild, simyra, divergent, wiseman, decibel
4/15/2020 cognition, module, mismanaging, supercomputers, lactation, schmitt, breastfeeding, shortwave, r.m., sahakian
4/16/2020 skylab, fitz, 861, suffrage, lemurs, droughts, esperance, fruity, raft, pheromones
4/17/2020 6.8, mir, foale, zeke, 1290, cowling, adh, spiders, cosmonauts, deepwater
4/18/2020 hydrogen, clerics, zeke, adh, ramadan, cowling, scams, hackers, cognition, dorchester
4/19/2020 landfill, vagina, gove, spiders, typhoid, orthodox, grotesque, adh, easter, rabbi
4/20/2020 spiders, menstruation, magnifica, gantz, scotia, rhetorical, debater, napkin, netanyahu, leatherback
4/21/2020 strawberries, benchmark, biodiversity, hydroponic, moths, countertenor, matilda, yacoob, arias, matilda's
4/22/2020 hubble, telescope, hubble's, galaxies, remittances, landfill, whales, ragweed, subscriber, quranic
4/23/2020 mackay, ramadan, projection, 4400000, pentathlon, brompton, mckay, credits, nicotine, $268.00
4/24/2020 nicotine, disinfectant, hubble, disinfectants, avalanche, everest, $480000000000.00, hungary, comolli, ramadan
4/25/2020 mercury, sarcastic, avalanche, remembrance, makary, passports, mummies, milkmen, garment, orphanage
4/26/2020 meritocracy, hubble, aden, zani, stc, queerness, lingua, separatists, franca, glands
4/27/2020 meritocracy, glands, sweat, meritocratic, airbus, hyperhidrosis, chechnya, doreen, dictatorships, embraer
4/28/2020 magda, locusts, nollywood, obaid, database, remittances, alexia, janab, chinoy, vilnius
4/29/2020 pfd, dividend, donut, alaskan, alaskans, alaska's, alaska, goslings, slumdog, palestine
4/30/2020 nicotine, massimo, moso, juana, legarde, kapoor, smokers, eurozone, goldberg, 5.8
5/1/2020 hiroshima, carly, kuti, mosaic, unicef, fela, reconnaissance, reid, shortest, duvet
5/2/2020 geriatric, jailbreak, thrombosis, rendez, jong, leopard, fresno, jogger, underpaid, clotting
5/3/2020 pfd, indonesian, demilitarized, buffett, dividend, cromwell, indonesians, madagascar, berkshire, alaskan
5/4/2020 pi, microbe, scarcity, mosquitoes, alysia, roadie, matchstick, $8000000000.00, beekeeping, calculating
5/5/2020 skateboarding, surfing, twenty, nineteen, two, wight, minsk, mohan, bunker, hitler's
5/6/2020 cromwell, magnetic, zena, tusk, whistling, samsung, zebrafish, navajo, bradford, conglomerate
5/7/2020 hazel, pell, arcadia, ketones, cardinal, garment, 3200000, 615, flynn, andhra
5/8/2020 dolphins, 14.7, 75th, commemorations, ve, bartering, lindsey, pornography, wreath, isaiah
5/9/2020 password, easterton, frutti, siegfried, eastridge, 75th, parade, passwords, dolphins, sorrell
5/10/2020 hazel, turtles, peggy, 277000, moher, preconditions, spitfire, ridley, particle, vancouver
5/11/2020 particle, hazel, electrons, bianca, quarks, birding, collider, detector, hadron, hormuz
5/12/2020 testosterone, dads, concentrator, surfing, oxytocin, hemsworth, siphon, icelandic, fathers, sculpture
5/13/2020 cognac, rogers', moths, francois, clemons, bradford, ovarian, pollinators, bamboo, surfing
5/14/2020 burr, sanofi, giuliana, matteis, goldfields, mosquitoes, typhoon, aden, stepson, cox's
5/15/2020 referee, clattenburg, fortresses, 190000, baltic, submarine, miliband, warp, refereeing, referees
5/16/2020 linnik, patsy, goldblum, genocide, rushkoff, rwandan, stork, chicks, cline, stalks
5/17/2020 patsy, cline, dagestan, shillings, spitfire, abdullah, albania, beaverbrook, demolition, netanyahu
5/18/2020 wynton, marsalis, smithfield, latrines, flush, malaya, hairdressers, sludge, $560000.00, fuji
5/19/2020 koalas, goldblum, geothermal, koala, membrane, wynton, barley, tori, bakewell, hydroxy
5/20/2020 talcum, kurdish, koalas, bengal, landfall, cyclon, powder, webcams, malarial, bangladesh
5/21/2020 superman, multis, ratcliffe, 38000000, plasma, kelton, kadyrov, chechen, ramzan, blackpool
5/22/2020 superman, heather, furedi, karachi, bengal, pro-democracy, kong's, subversion, lemaire, knell
5/23/2020 cummings, hertz, harlem, alligator, smithfield, kenyatta, churchgoers, karachi, dominic, hydroxy
5/24/2020 cummings, harlem, morpurgo, netanyahu, winchester, gentrification, solitude, pummel, samuelsson, tankers
5/25/2020 cummings, reunion, strontium, rocco, friendships, spiders, pigeon, reunions, dominic, jobbery
5/26/2020 strontium, cummings, macau, alimony, fiddle, kentucky, tyler, hydroxy, winship, hillbilly
5/27/2020 harlem, 750000000000, cuarón, kramer, minneapolis, gentrification, bradford, samuelsson, stan, criminalize
5/28/2020 fables, fable, churchill's, aesop, churchill, phaedrus, cummings, kidney, aesop's, nepa
5/29/2020 minneapolis, radiologists, glorifying, cartels, octopus, dumbo, cannabis, unarmed, cable, floyd
5/30/2020 minneapolis, disgracing, unarmed, floyd, orgasm, macau, peacocks, forbes, walz, wanton
5/31/2020 fables, dispersal, floyd, rivets, minneapolis, misplay, strontium, aesop, radiologists, syncopated
6/1/2020 helium, fables, self-help, floyd, quakers, minneapolis, self-improvement, curfews, vigil, vitamin
6/2/2020 helium, abortion, floyd, morningstar, zanzibar, abortions, minneapolis, floyd's, grayson, pro-life
6/3/2020 floyd, centeno, reichstag, jamaica, rodney, minneapolis, curfews, bedside, landfall, mandarin
6/4/2020 babylon, mattis, chechnya, mesopotamia, floyd, minneapolis, chechen, implicit, tiananmen, gilgamesh
6/5/2020 floyd, minneapolis, pentridge, autopsies, mckenney, bregman, chokeholds, reparations, defibrillators, beavers
6/6/2020 floyd, opec, goodell, elsinore, deacon, abortion, nfl, minneapolis, magreb, philharmonic
6/7/2020 babylon, floyd, minneapolis, atai, deacon, hypochondria, floyd's, mesopotamia, zuckerberg, malpass
6/8/2020 minneapolis, babylon, floyd, epstein, willow, superhero, toothbrush, toothbrushes, dismantle, mesopotamia
6/9/2020 toothbrush, toothbrushes, floyd, tooth, leaflets, bristles, zone's, ipl, hotline, ksi
6/10/2020 floyd, engstrom, hsbc, module, odysseys, odyssey, lovell, hypochondria, deborah, aquarius
6/11/2020 confederate, seafarers, milley, confederacy, adidas, jefferson, richmond, statues, joan, nascar
6/12/2020 atoms, rakhine, floyd, playstation, 20.4, everest, groundsman, sujit, pro-democracy, consulate
6/13/2020 penguin, dalai, lama, thuggery, floyd, statues, squirrel, minneapolis, $41800.00, highlanders
6/14/2020 atoms, toothbrushes, toothbrush, mansong, brooks, squirrel, spitfire, mangano, damson, trustworthiness
6/15/2020 atoms, ressa, rappler, whelan, dupont, tasting, libel, kubler, wine, moshe
6/16/2020 heel, heels, delinquent, dexa, dupont, liaison, curie, adh, rosamund, mahama
6/17/2020 gambling, liaison, mansong, pyongyang, damson, aston, floyd, bolton, oxfam, biometric
6/18/2020 bolton, bertha, bolton's, microgravity, electromagnetic, vera, brooks, dreamers, gaulle, clyde
6/19/2020 juneteenth, pirbright, tulsa, triangle, pele, cuttlefish, scolari, oklahoma, eyed, wastewater
6/20/2020 tulsa, berman, oklahoma, juneteenth, dynamo, obadiah, sharpton, giuliani, chandra's, jeffrey
6/21/2020 berman, tulsa, spitfires, beaverbrook, mutinous, stabbing, dynamo, midwinter, cobain, oklahoma
6/22/2020 vegan, dimitrov, nascar, icc, cuttlefish, grigor, stonehenge, veganism, patagonia, boneheaded
6/23/2020 snares, fork, nascar, navarro, triplets, poacher, cutlery, gras, batman, placenta
6/24/2020 segway, magicians, talley, dynamo, noose, abortions, demolish, vanadium, flynn, jerrys
6/25/2020 silk, statues, locusts, kluger, quantas, eiffel, confederate, migraine, looters, licenses
6/26/2020 ressa, filipinos, rappler, nectarines, astrid, dixie, cirebon, glastonbury, eldoret, sigma
6/27/2020 silk, fianna, lazarus, pills, malawi's, woodrow, incompetent, statues, rerun, blah
6/28/2020 spitfire, lazarus, rerun, woodrow, malawi's, malawi, duda, sedwill, macpherson, paddle
6/29/2020 masculinity, silk, copperfield, abortion, transfusion, violin, beasley, 737, emblem, dogma
6/30/2020 grandmothers, pollinators, jagger, honeybees, honeybee, bees, copperfield, pollination, arabella, richards
7/1/2020 annexation, shockley, cremation, 2036, weinstein, memorials, 370, fairchild, argyle, tombstone
7/2/2020 maxwell, epstein, jade, norse, hildebrand, encrypted, viking, gallic, 2036, paedophile
7/3/2020 maxwell, epstein, nollywood, barefoot, kipsang, redskins, absentia, khashoggi, jamal, tolokonnikova
7/4/2020 rushmore, barefoot, grandmothers, yucatan, cristina, eduardo's, norse, hildebrand, pubs, nollywood
7/5/2020 poroshenko, grandmothers, barefoot, 690000, intersectionality, cochabamba, plank, chechen, spitfire, chechnya
7/6/2020 barefoot, morricone, mandisa, ennio, cholera, pacu, overflights, justifying, temperamental, cushioning
7/7/2020 depp, mandisa, toilets, jefferson, rebekah, camden, magnitsky, monaghan, mta, emin
7/8/2020 feinstein, panthers, belgrade, rufus, montreal, wainwright, mcgarrigle, modesty, balloons, prt
7/9/2020 tango, rita, oxfam, polynesians, rita's, subpoenas, rivera, blanka, coffins, morgues
7/10/2020 earthworms, abs, barbados, bison, abse, gamers, lagoon, al-jazeera, lavrov, variant
7/11/2020 bosnian, srebrenica, charlton, earthworms, tango, serb, tampering, commuted, bosnia, massacre
7/12/2020 earthworms, louganis, duda, mueller, basque, muller, hungary's, galicia, lutheran, feinstein
7/13/2020 earthworms, redskins, tango, chimpanzees, goodall, ishmael, worms, earthworm, dinesh, duda
7/14/2020 timbuktu, chimps, huawei, rembrandt, redskins, maxwell, norah, rivera, mccrory, p.x.
7/15/2020 stradivarius, violin, maxwell, huawei, chimps, preferential, ice-t, infirmary, $13000000000.00, epstein
7/16/2020 chaucer, bitcoin, chaucer's, begum, sediment, azerbaijan, armenia, prebiotics, hackers, shamima
7/17/2020 selassie, haile, 747, morphic, fairfield, 750000000000, charlton, bothroyd, fukuda, plo
7/18/2020 chaucer, fellini, srebrenica, roxanne, chaucer's, redskins, pancreatic, 450000000000, lewis, luther
7/19/2020 selassie, haile, fairfield, sterilization, divinity, tony's, srebrenica, britishness, majesty, embankment
7/20/2020 chaucer, interferon, goodes, 750000000000, kanye, samantha, whales, propellers, picasso, propeller
7/21/2020 chimps, chimpanzees, convent, gombe, oyelowo, rowing, 750000000000, frugal, woodard, carell
7/22/2020 redheads, braca, consulate, tony's, rowing, redhead, boobies, maxwell, embankment, bile
7/23/2020 tolstoy, yoho, tolstoy's, pre-term, rocca, portland, consulate, decemberists, albuquerque, frydenberg
7/24/2020 chengdu, consulate, mosquitoes, epstein, havens, goldman, sachs, helaman, gambia, jeffrey
7/25/2020 tolstoy, repossessed, turkmen, consulate, fleetwood, turkmenistan, dopey, havens, helaman, desking
7/26/2020 christi, philbin, regis, corpus, essex, turkmen, cathedral, hypersonic, turkmenistan, havilland
7/27/2020 westboro, tolstoy, hezbollah, corals, tolstoy's, infiltrate, phelps, opium, decemberists, followings
7/28/2020 najib, razak, fusion, platoon, marlins, reactor, supersymmetry, portland, collider, dimensions
7/29/2020 hedgehog, lactose, lactase, cyclops, sonic, kodak, mecca, milkshake, numbness, pilgrimage
7/30/2020 picasso, rover, perseverance, poker, gideon, lauber, ovett, disqualified, goudreau, salvini
7/31/2020 obesity, anelka, 18.5, tectonics, ransomware, champagne, garmin, fein, overweight, carbonate
8/1/2020 murdoch, kabuki, lithium, obesity, cinderella, gymnasts, picasso, corp, 46600, laos
8/2/2020 clive, nanotubes, hedgehog, neutrons, capsule, splash, manned, psychism, clive's, bogota
8/3/2020 cinderella, poker, jalalabad, genoa, hume, leopards, blackjack, fleisher, gladwell, kobach
8/4/2020 voicemail, genome, juan, jade, tanzanite, beirut, warheads, interpol, 155000, dvc
8/5/2020 nitrate, ammonium, beirut, gapper, jadzia, lebanese, helen, blast, handcuffed, hiroshima
8/6/2020 nra, beirut, hiroshima, gwen, constantinople, emperors, algerians, jadzia, 476, algeria
8/7/2020 nra, beirut, rajapaksa, contrails, hezbollah, serota, overshot, lebanese, tuesday's, tick-tock
8/8/2020 beirut, tuesday's, skidded, serota, arterton, ammonium, nitrate, lovelock, winfrey's, runway
8/9/2020 loya, jirga, contrails, lukashenko, globalization, rajapaksa, beirut, nagasaki, mahinda, poker
8/10/2020 contrails, easterbrook, lukashenko, contrail, beirut, tycoon, ly, pro-democracy, jimmy, belarus
8/11/2020 origami, bandon, dunes, mclay, lithuania, arterton, braman, gorse, donoghue, lukashenko
8/12/2020 gorda, origami, tinnitus, metaphors, harris, nieto, globalization, lukashenko, inn, dissolution
8/13/2020 bradbury, redstone, briberies, viacom, minsk, annexation, palestinians, annex, sumner, anc
8/14/2020 armenia, azerbaijan, absentee, postal, voter, ballots, nagorno-karabakh, lukashenko, azerbaijanis, azerbaijani
8/15/2020 slime, bradbury, postal, shrine, minsk, lilo, lukashenko, emperor, surrender, 75th
8/16/2020 slime, quilting, minsk, lukashenko, constantia, gee's, defectors, bishops, pettway, quilt
8/17/2020 slime, bradbury, lukashenko, slugs, minsk, slug, snails, snail, belarus, beachside
8/18/2020 hariri, rafik, hezbollah, fistula, diamonds, riri, heist, crowe, lukashenko, bamako
8/19/2020 mali, belarus, lukashenko, quilting, diamonds, bamako, gee's, conservatorship, britney, boubacar
8/20/2020 bannon, radiotherapy, tibetans, cutler, greenland, poisoned, tibet, syphilis, poisoning, alexina
8/21/2020 yeast, glaciers, randy, poisoned, weaver, alexina, mahbubani, seaweed, greenland, novelli
8/22/2020 weaver, pyramid, yeast, randy, lukashenko, cutler, minsk, pyramids, crespo, postal
8/23/2020 emmett, minsk, lukashenko, unprincipled, emmett's, hogan, yeast, pentecostal, marianne, postal
8/24/2020 yeast, ekpo, plasma, tinto, convalescent, pyramid, pyramids, poisoned, kellyanne, fda
8/25/2020 cactus, ganymede, jupiter, cactuses, magnetic, jupiter's, jbs, polio, serb, bosnian
8/26/2020 emmett, synesthesia, emmett's, jacob, blake, wisconsin, osha, ratko, firefox, mamie
8/27/2020 louisiana, unsurvivable, jacob, wisconsin, surfing, blake, hurricane, milwaukee, parole, survivable
8/28/2020 mariachi, plasma, doping, shinzo, fda, rowling, bletchley, jacob, convalescent, cadets
8/29/2020 boseman, chadwick, panther, banksy, accreditation, cactus, synesthesia, ekpo, plasma, minsk
8/30/2020 montenegro, cactus, montenegro's, bratislava, minsk, comedians, 840000, djukanovic, gunnery, lukashenko
8/31/2020 montenegro, boredom, ekpo, underwater, lebanon's, fleming, hostage, portland, simone, bailiff's
9/1/2020 cartoons, mugabe's, jacob, rover, abdo, exabyte, osha, republished, hungary, wisconsin
9/2/2020 greenlandic, greenland, walkman, abdo, hebdo, novacek, satirical, camaro, woodard, greenlanders
9/3/2020 smallpox, filipino, flynn, margarita, mongolian, $750000000.00, typhoon, baluchistan, pox, blisters
9/4/2020 moroccans, berlusconi, lampedusa, silvio, messi, blake, jacob, lava, subtropics, esper
9/5/2020 smallpox, paraguayan, rafi, rfp, walkman, icc, euthanasia, typhoon, ledo, lampedusa
9/6/2020 bolshoi, minsk, lukashenko, kitteridge, typhoon, greenland, seuss', belarus, greenlandic, kittredge
9/7/2020 albinism, smallpox, djokovic, 49.4, minsk, disqualified, novak, khashoggi, typhoon, kolesnik
9/8/2020 kolesnik, birtles, rover, crater, socrates, carbonate, belarus, motorcycling, perseverance, lukashenko
9/9/2020 morea, rochester, photovoltaic, kittredge, larry's, reddit, 13000, remittances, parchment, singletary
9/10/2020 wellness, woodward, protestants, tiffany, weavers, nitrogen, huguenot, borneo, 1680s, amphibians
9/11/2020 tinto, sturgis, aboriginal, morea, broderick, lesbos, bahrain, abrams, overrides, agnivesh
9/12/2020 conran, afghans, naveed, doha, spinny, bahrain's, vizcarra, oregon, reggae, taliban
9/13/2020 biosphere, minsk, rhine-westphalia, swingers, lukashenko, gormley, abbe, pemberton, nitrogen, celtic
9/14/2020 pinocchio, broom, nvidia, sibylla, oracle, suger, softbank, belarus, hauser, snp
9/15/2020 neon, bahrain, palestinians, arsonist, franco, whales, irritable, whale, beaumont, tummy
9/16/2020 iaea, greensboro, diac, solstice, suger, chariot, atv, doping, grossi, wellness
9/17/2020 wellness, preppers, prepping, nitrogen, georgian, ammonia, novacek, susie, barbie, georgians
9/18/2020 940000, 30000000, vocalisation, scranton, mucus, janina, entomologists, malyon, gimpy, punchier
9/19/2020 ginsburg, bader, permafrost, slovenian, georgian, celery, chili, penultimate, feeney, thawing
9/20/2020 ginsburg, bader, biosphere, slovenian, monarchy, hsbc, belfast, justices, hilliard, cyclist
9/21/2020 abramovich, galileo, bader, ginsburg, shit's, chili, emmys, hsbc, seamstresses, cheeses
9/22/2020 ginsburg, testosterone, bader, drowning, whales, trans, lifeguards, sirleaf, seeding, maple
9/23/2020 whales, louisville, requiem, magnetic, endangerment, mozart, ben's, payloads, beaching, ginsburg
9/24/2020 louisville, westpac, britney, briana, obesity, clemency, brianna, rhonda, blackwater, tobacco
9/25/2020 monocytes, stillborn, chimpanzees, clemency, britney, archibald, goodes, cardinal, aggravated, carnival

TECHNICAL DETAILS

For those interested in running this analysis themselves, here is the SQL query we used:

WITH nested AS (
WITH data AS (
SELECT TIMESTAMP_TRUNC(TIMESTAMP, DAY) date, STATION, NGRAM word, sum(COUNT) count FROM `gdelt-bq.gdeltv2.iaradio_1grams` where DATE(TIMESTAMP) >= "2020-01-01" and station='BBC World Service' group by date,station, NGRAM
)
, word_day_type AS (
# how many times a word is mentioned in each "document"
SELECT word, SUM(count) counts, date, station
FROM data
GROUP BY 1, 3, 4
)
, day_type AS (
# total # of words in each "document"
SELECT SUM(count) counts, date, station
FROM data
GROUP BY 2, 3
)
, tf AS (
# TF for a word in a "document"
SELECT word, date, station, a.counts/b.counts tf
FROM word_day_type a
JOIN day_type b
USING(date, station)
)
, word_in_docs AS (
# how many "documents" have a word
SELECT word, COUNT(DISTINCT FORMAT('%s %s', FORMAT_TIMESTAMP( "%m/%d/%E4Y", date), station)) indocs
FROM word_day_type
GROUP BY 1
)
, total_docs AS (
# total # of docs
SELECT COUNT(DISTINCT FORMAT('%s %s', FORMAT_TIMESTAMP( "%m/%d/%E4Y", date), station)) total_docs
FROM data
)
, idf AS (
# IDF for a word
SELECT word, LOG(total_docs.total_docs/indocs) idf
FROM word_in_docs
CROSS JOIN total_docs
)
SELECT date, station,
ARRAY_AGG(STRUCT(ARRAY_TO_STRING(words, ', ') AS wordlist)) top_words
FROM (
SELECT date, station, ARRAY_AGG(word ORDER BY tfidf DESC LIMIT 10) words
FROM (
SELECT word, date, station, tf.tf * idf.idf tfidf
FROM tf
JOIN idf
USING(word)
)
GROUP BY date, station
)
GROUP BY date, station
ORDER BY date DESC
) select FORMAT_TIMESTAMP( "%m/%d/%E4Y", date) day, station, wordlist from nested, UNNEST(top_words) order by date asc, station asc

We hope this serves as a template for your own linguistic explorations of radio news!