How might we use the new Radio News NGrams dataset to examine the Internet Archive's Radio News Archive's ASR of the BBC World Service over the course of 2020 to estimate the top 10 most "significant" words defining the World Service's coverage each day? Using a single SQL query we can compute TFIDF scores over the dataset and surface the top 10 words each day that were spoken on the BBC World Service that day compared with BBC World Service the rest of the year.
Remember that this is based on automatic speech recognition results, so there may be errors and that this list is based on simple word frequency histograms rather than the semantic importance of each word to each day's coverage.
Note that the TFIDF calculation by its nature strips out words that are so common that they no longer distinguish a given day from other days. This means that COVID-19, which has remained a fixture of 2020 coverage is not well-represented in the table below because it appears in each day's coverage.
| Day | Top Words | |
| 1/1/2020 | carols, sulfur, jakarta, baghdad, knitting, bushfires, slapping, cyprus, beetlejuice, embassy | |
| 1/2/2020 | mammograms, radiologists, phosphate, desmond's, phosphates, radiologist, morocco, greens, jakarta, yiddish | |
| 1/3/2020 | testicular, baghdad, qassam, iran's, yiddish, standage, punks, airstrike, chemotherapy, thurman | |
| 1/4/2020 | baghdad, iraqi, yiddish, procession, chippendales, cartel, airstrike, desmond's, renewables, bribes | |
| 1/5/2020 | sheba, yiddish, noriega's, iraqi, mannheim, noriega, alderman, lamu, globes, solomon | |
| 1/6/2020 | weinstein, harvey, globes, tehran, jnu, rapist, $1400000000.00, sirte, ayatollah, uranium | |
| 1/7/2020 | travelex, zarif, bundanoon, cyprus, satellites, jello, burial, mourners, chippendales, hackers | |
| 1/8/2020 | ballistic, ghosn, tappers, missile, iranians, ukrainian, tehran, recorders, denise's, 737 | |
| 1/9/2020 | ukrainian, lyme, cocoa, highlife, tussauds, millennial, 176, missile, meghan, kiev | |
| 1/10/2020 | apnea, camels, ukrainian, 176, airliner, monarchies, lendu, snoring, missile, soju | |
| 1/11/2020 | qaboos, sultan, tenants, bloodless, taiwan's, rental, affordable, housing, barbershops, jemele | |
| 1/12/2020 | kazakhstan, narcopolis, wades, soothe, bombay, gz, benedict, scruton, celibacy, haitians | |
| 1/13/2020 | volcano, celibacy, benedict, eruption, sandringham, lava, diac, nantucket, manila, airliner | |
| 1/14/2020 | manipulator, volcano, benedict, revalidation, astros, ginko, mutiny, impeachment, libyan, murdoch | |
| 1/15/2020 | origami, impeachment, milf, bezos, miscarriage, cholesterol, astros, mindanao, exoskeleton, frogs | |
| 1/16/2020 | impeachment, stinney, sepsis, 2047, giuliani, honey, parnassus, parness, prevaricate, ukraine | |
| 1/17/2020 | sepsis, philanthropy, impeachment, honey, ayatollah, goebbels, alopecia, infinity, mcraven, selenski | |
| 1/18/2020 | philanthropy, honey, hydrogen, philanthropists, dershowitz, overreached, impeachment, starr, binyon, mcraven | |
| 1/19/2020 | hydrogen, infinity, greenlandic, honey, greenland, countable, hezbollah, quantum, impeachment, algebra | |
| 1/20/2020 | santos, fungus, frogs, toads, angolan, toad, vancouver, davos, angola, extradition | |
| 1/21/2020 | impeachment, davos, dodo, mcconnell, mitch, schiff, senators, sperm, dodos, prophets | |
| 1/22/2020 | bezos, impeachment, python, platypus, davos, weinstein, monty, arabia's, greenland, zanzibar | |
| 1/23/2020 | auschwitz, vashem, myanmar, yad, darwin, genocide, holocaust, suu, icj, impeachment | |
| 1/24/2020 | darwin, impeachment, wetlands, locusts, erasmus, salvini, waterways, santos, harrisburg, wozniak | |
| 1/25/2020 | dodo, impeachment, darwin, auschwitz, erasmus, fungus, 270, jakes, dam, whales | |
| 1/26/2020 | lilo, scrolls, darwin, lindemann, volcanoes, erasmus, gestapo, salvini, impeachment, romagna | |
| 1/27/2020 | auschwitz, bryant, kobe, lakers, romagna, vegan, basketball, 75th, grammys, salvini | |
| 1/28/2020 | palestinians, netanyahu, huawei, cheetahs, dodo, oils, stormiest, antarctic, palestinian, bryant's | |
| 1/29/2020 | palestinians, bolton, compacted, impeachment, palestinian, stabber, turners, lilo, recapturing, $22000000000.00 | |
| 1/30/2020 | sinclair, recycling, butterflies, bolton, c.j., becky, becky's, beltran, chapo, cocaine | |
| 1/31/2020 | cervical, impeachment, bracks, kiev, wavering, breck's, sunderland, renewal, nike's, brics | |
| 2/1/2020 | beethoven, impeachment, t.-t., palestinians, plymouth, recycling, bracks, 259, thorpe, garbin | |
| 2/2/2020 | beethoven, timbuktu, manuscripts, $170000000000.00, stampede, colomba, bystander, beethoven's, stratham, pentecostal | |
| 2/3/2020 | kansas, $170000000000.00, chiefs, barnier, ers, caucus, iowa, stabbing, regenerative, typhoid | |
| 2/4/2020 | iowa, hydrogen, caucus, dowager, caucuses, zuma, empress, moines, des, 425 | |
| 2/5/2020 | impeachment, penguins, compression, gestures, 490, 24300, caucus, skidded, iowa, antivenom | |
| 2/6/2020 | gluten, spartacus, impeachment, douglas, celiac, kirk, cabazon, 560, bumblebees, acquittal | |
| 2/7/2020 | qaeda, suisse, elephants, raimi, cassim, impeachment, peninsula, mandela, arabian, botswana | |
| 2/8/2020 | impeachment, hankins, abstinence, lavrov, nixon, saskatoon, watergate, freehold, hampshire, hawkins | |
| 2/9/2020 | fein, naga, subsonic, rampage, fianna, rouge, orientation, earthquake, bindu, aliyev | |
| 2/10/2020 | fein, equifax, parasite, oscars, cdu, karrenbauer, workhouse, junho, ripper, aliyev | |
| 2/11/2020 | shabalala, mambazo, darfur, j.k., bashir, rowling, phyllis, hampshire, pele, correa | |
| 2/12/2020 | celibacy, salvini, hampshire, b.p., saskatoon, hamlet, priestly, catalina, bernie, priests | |
| 2/13/2020 | dresden, 242, javid, sajid, stupendous, fossils, sushi, exchequer, reshuffle, turtles | |
| 2/14/2020 | polanski, westerdam, menstruating, locusts, kqed, valentine's, butterfly, bronze, 20.7, cambodia | |
| 2/15/2020 | mccabe, camorra, swimsuit, kiir, munich, salva, reformists, bronze, hankins, pageant | |
| 2/16/2020 | gospel, swimsuit, flack, kanye, hip, magnitogorsk, 355, bronze, yokohama, pageant | |
| 2/17/2020 | voyager, holden, pensions, albinism, pension, cameroon, robo, composting, vocational, 355 | |
| 2/18/2020 | scouts, hsbc, cello, gezi, bezos, oscillations, gypsy, m.s., jacqueline, quartet | |
| 2/19/2020 | ammonia, yokohama, gospel, blagojevich, bloomberg, credentials, assange, suliman, shanidar, radicalization | |
| 2/20/2020 | bloomberg, gospel, eugene, mosquitoes, mosquito, shisha, kock, cowley, src, valka | |
| 2/21/2020 | romulus, antibiotic, b.n., nile, cousteau, antarctica, baobab, chun, daegu, greenpeace | |
| 2/22/2020 | nile, shabalala, l'arche, mambazo, mosquito, dam, mosquitoes, machar, juba, salva | |
| 2/23/2020 | mosquitoes, nevada, rikers, mosquito, cordoba, caucuses, bulldozer, wilder, tyson, fury | |
| 2/24/2020 | weinstein, salazar, dershowitz, harvey, torrey's, mahathir, carnival, zombies, carnitine, zombie | |
| 2/25/2020 | mubarak, hosni, weinstein, harvey, cfa, burgess, bryant's, motorola, mubarak's, bryant | |
| 2/26/2020 | mubarak, hosni, confraternity, iger, bilingual, sharapova, monolingual, gaza, bernie, hindus | |
| 2/27/2020 | heathrow, smallpox, voltaire, runway, retaken, pilgrims, cussler, bloomberg, rikers, nutrients | |
| 2/28/2020 | polanski, firestone, voltaire, turkish, airstrike, galaxy, password, gadhafi, tattoos, syrian | |
| 2/29/2020 | polanski, gadhafi, taliban, dax, doha, mahathir, benghazi, luxemburg, afghans, blasphemy | |
| 3/1/2020 | oracle, dalai, voltaire, holiness, mahathir, lama, tibet, tibetans, oracles, cia's | |
| 3/2/2020 | pius, womenomics, vatican, papacy, juarez, asylum, netanyahu, rivlin, indiana, gantz | |
| 3/3/2020 | netanyahu, likud, leaded, endorsements, bernie, fumigation, petrol, herbicide, coca, subcontractor | |
| 3/4/2020 | bloomberg, bernie, libra, primaries, grunwick, helmand, biden, delegates, samoa, lobsters | |
| 3/5/2020 | selassie, icc, haile, fried, qualia, glassman, eels, psychiatry, psychiatric, organophosphate | |
| 3/6/2020 | clearview, opec, bethlehem, selassie, haile, bleaching, beetle, idlib, bounty, warplanes | |
| 3/7/2020 | nayef, tyner, selassie, mccoy, haile, womenomics, skolkovo, nagarajan, woody, reddit | |
| 3/8/2020 | selassie, qatif, lombardy, skolkovo, nanny, haile, houseboat, polygamy, polygamous, 16000000 | |
| 3/9/2020 | rosewood, sydow, 298, skolkovo, m.h., opec, 366, bergman, ingmar, introverts | |
| 3/10/2020 | kakapo, inspirations, 2036, 157, geldof, zeppelin, pornhub, zaire, giraffes, 737 | |
| 3/11/2020 | weinstein, dorries, pell, nadine, harvey, bernie, cardinal, primaries, treasurer, danube | |
| 3/12/2020 | weinstein, rosewood, niemeyer, crypto, harvey, hanks, wastes, taji, flugel, capitan | |
| 3/13/2020 | marburg, rosewood, wiki, frederica, patronymic, manning, nikkei, ivory, niemeyer, griffiths | |
| 3/14/2020 | rosewood, marburg, niemeyer, patronymic, chile's, frederica, penguin, monkeys, spinny, pla | |
| 3/15/2020 | birding, otero, ballet, netanyahu, gantz, brewhouse, clangers, weinstein, bumper, taipei | |
| 3/16/2020 | $700000000000.00, hilda, surfing, 87000, 368, antivirus, liquidity, kakapo, slashing, felons | |
| 3/17/2020 | mackinnon, cashless, phages, $130000000000.00, mural, mahal, $400000000000.00, marburg, vinson, taj | |
| 3/18/2020 | marsha, illinois, tigers, ioc, i.g., bernie, contestants, marcia, primaries, $400000000000.00 | |
| 3/19/2020 | $820000000000.00, hubble, ecb, telescope, skid, hamlin, ioc, pinsent, quantas, binge | |
| 3/20/2020 | hypersonic, lenin, zanzibar, vampire, 627, salazar, ancestry, artemisinin, hanged, newsome | |
| 3/21/2020 | varoufakis, flanders, rogers, kenny, yanis, bondi, parton, lucille, lenin, baffin | |
| 3/22/2020 | dmc, ancestry, 394, mcdaniels, hezbollah, co-working, blasio, gorillas, prague, sondheim | |
| 3/23/2020 | ioc, heffernan, denman, vegetative, shinzo, ancestry, oecd, reanimation, regina, trombone | |
| 3/24/2020 | phosphorus, phosphorous, asterix, sludge, vegetative, taubenberger, struvite, hippos, phosphate, seti | |
| 3/25/2020 | module, aldrin, dmc, 738, seti, apollo, 1300000000, lunar, mcdaniels, lunny | |
| 3/26/2020 | 3300000, module, levinson, fendel, ioc, brenton, overkill, minimizes, brewery, d.m.c. | |
| 3/27/2020 | cleve, ipl, macedonia's, populism, counter-point, d.m.c., polio, quorum, 85000, artemisia | |
| 3/28/2020 | seti, 832, populism, defectors, beep, cleve, drager, lowery, shostak, ingrid | |
| 3/29/2020 | defectors, penderecki, module, epigenetics, 838, 756, defector, mcqueen, rosneft, koreans | |
| 3/30/2020 | oxfam, chromosome, panther, clementine, masculinity, $160000000000.00, bunkers, 812, sunk, 418 | |
| 3/31/2020 | chromosome, ingrid, aspen, celeste, quaking, 849, satrapi, marjane, mountjoy, dorchester | |
| 4/1/2020 | defectors, blasphemy, module, porn, griffin, nightingales, defector, interferon, koreans, jamal | |
| 4/2/2020 | epicurus, epicurean, 6600000, module, rumania, vasa, epicureanism, vassar, lucretia's, atoms | |
| 4/3/2020 | navigator, perishable, trampoline, lettuce, withers, lang, exporters, pork, caloric, maximo | |
| 4/4/2020 | interpol, starmer, corbyn, 809, baptism, eritrea, tranquilized, foxx, rodeo, nikita | |
| 4/5/2020 | epicurus, epicurean, blasphemy, placebo, ranveer, jabril, gadhafi, 674, fujifilm, pentland | |
| 4/6/2020 | placebo, epicurus, homebrew, epicurean, antarctica, refereeing, antarctic, transcontinental, isotope, levine | |
| 4/7/2020 | pell, cardinal, nimrud, reliefs, proscar, nikita, treasurer, profs, foxx, robb | |
| 4/8/2020 | prine, bhutto, postpartum, kato, fatima, psychosis, ferrari, crescent, hebdo, module | |
| 4/9/2020 | ddp, allergic, clownfish, 799, seche, penicillin, offenders, disengagement, 881, sirleaf | |
| 4/10/2020 | nato, mercury, opec, 500000000000, stoltenberg, comet, eyeliner, easter, securitas, barrels | |
| 4/11/2020 | cardinal, easter, pell, 2108, eyeliner, atheists, rothschild, volcano, wristbands, aldridge | |
| 4/12/2020 | basilica, easter, peter's, assange, voodoo, wyoming's, 431, potsdam, ecuadorian, indemnify | |
| 4/13/2020 | vagina, opec, tornadoes, hymen, daisy, takeovers, apollo, logan, andronicus, hibernation | |
| 4/14/2020 | decibels, adh, honking, diving, imf, rothschild, simyra, divergent, wiseman, decibel | |
| 4/15/2020 | cognition, module, mismanaging, supercomputers, lactation, schmitt, breastfeeding, shortwave, r.m., sahakian | |
| 4/16/2020 | skylab, fitz, 861, suffrage, lemurs, droughts, esperance, fruity, raft, pheromones | |
| 4/17/2020 | 6.8, mir, foale, zeke, 1290, cowling, adh, spiders, cosmonauts, deepwater | |
| 4/18/2020 | hydrogen, clerics, zeke, adh, ramadan, cowling, scams, hackers, cognition, dorchester | |
| 4/19/2020 | landfill, vagina, gove, spiders, typhoid, orthodox, grotesque, adh, easter, rabbi | |
| 4/20/2020 | spiders, menstruation, magnifica, gantz, scotia, rhetorical, debater, napkin, netanyahu, leatherback | |
| 4/21/2020 | strawberries, benchmark, biodiversity, hydroponic, moths, countertenor, matilda, yacoob, arias, matilda's | |
| 4/22/2020 | hubble, telescope, hubble's, galaxies, remittances, landfill, whales, ragweed, subscriber, quranic | |
| 4/23/2020 | mackay, ramadan, projection, 4400000, pentathlon, brompton, mckay, credits, nicotine, $268.00 | |
| 4/24/2020 | nicotine, disinfectant, hubble, disinfectants, avalanche, everest, $480000000000.00, hungary, comolli, ramadan | |
| 4/25/2020 | mercury, sarcastic, avalanche, remembrance, makary, passports, mummies, milkmen, garment, orphanage | |
| 4/26/2020 | meritocracy, hubble, aden, zani, stc, queerness, lingua, separatists, franca, glands | |
| 4/27/2020 | meritocracy, glands, sweat, meritocratic, airbus, hyperhidrosis, chechnya, doreen, dictatorships, embraer | |
| 4/28/2020 | magda, locusts, nollywood, obaid, database, remittances, alexia, janab, chinoy, vilnius | |
| 4/29/2020 | pfd, dividend, donut, alaskan, alaskans, alaska's, alaska, goslings, slumdog, palestine | |
| 4/30/2020 | nicotine, massimo, moso, juana, legarde, kapoor, smokers, eurozone, goldberg, 5.8 | |
| 5/1/2020 | hiroshima, carly, kuti, mosaic, unicef, fela, reconnaissance, reid, shortest, duvet | |
| 5/2/2020 | geriatric, jailbreak, thrombosis, rendez, jong, leopard, fresno, jogger, underpaid, clotting | |
| 5/3/2020 | pfd, indonesian, demilitarized, buffett, dividend, cromwell, indonesians, madagascar, berkshire, alaskan | |
| 5/4/2020 | pi, microbe, scarcity, mosquitoes, alysia, roadie, matchstick, $8000000000.00, beekeeping, calculating | |
| 5/5/2020 | skateboarding, surfing, twenty, nineteen, two, wight, minsk, mohan, bunker, hitler's | |
| 5/6/2020 | cromwell, magnetic, zena, tusk, whistling, samsung, zebrafish, navajo, bradford, conglomerate | |
| 5/7/2020 | hazel, pell, arcadia, ketones, cardinal, garment, 3200000, 615, flynn, andhra | |
| 5/8/2020 | dolphins, 14.7, 75th, commemorations, ve, bartering, lindsey, pornography, wreath, isaiah | |
| 5/9/2020 | password, easterton, frutti, siegfried, eastridge, 75th, parade, passwords, dolphins, sorrell | |
| 5/10/2020 | hazel, turtles, peggy, 277000, moher, preconditions, spitfire, ridley, particle, vancouver | |
| 5/11/2020 | particle, hazel, electrons, bianca, quarks, birding, collider, detector, hadron, hormuz | |
| 5/12/2020 | testosterone, dads, concentrator, surfing, oxytocin, hemsworth, siphon, icelandic, fathers, sculpture | |
| 5/13/2020 | cognac, rogers', moths, francois, clemons, bradford, ovarian, pollinators, bamboo, surfing | |
| 5/14/2020 | burr, sanofi, giuliana, matteis, goldfields, mosquitoes, typhoon, aden, stepson, cox's | |
| 5/15/2020 | referee, clattenburg, fortresses, 190000, baltic, submarine, miliband, warp, refereeing, referees | |
| 5/16/2020 | linnik, patsy, goldblum, genocide, rushkoff, rwandan, stork, chicks, cline, stalks | |
| 5/17/2020 | patsy, cline, dagestan, shillings, spitfire, abdullah, albania, beaverbrook, demolition, netanyahu | |
| 5/18/2020 | wynton, marsalis, smithfield, latrines, flush, malaya, hairdressers, sludge, $560000.00, fuji | |
| 5/19/2020 | koalas, goldblum, geothermal, koala, membrane, wynton, barley, tori, bakewell, hydroxy | |
| 5/20/2020 | talcum, kurdish, koalas, bengal, landfall, cyclon, powder, webcams, malarial, bangladesh | |
| 5/21/2020 | superman, multis, ratcliffe, 38000000, plasma, kelton, kadyrov, chechen, ramzan, blackpool | |
| 5/22/2020 | superman, heather, furedi, karachi, bengal, pro-democracy, kong's, subversion, lemaire, knell | |
| 5/23/2020 | cummings, hertz, harlem, alligator, smithfield, kenyatta, churchgoers, karachi, dominic, hydroxy | |
| 5/24/2020 | cummings, harlem, morpurgo, netanyahu, winchester, gentrification, solitude, pummel, samuelsson, tankers | |
| 5/25/2020 | cummings, reunion, strontium, rocco, friendships, spiders, pigeon, reunions, dominic, jobbery | |
| 5/26/2020 | strontium, cummings, macau, alimony, fiddle, kentucky, tyler, hydroxy, winship, hillbilly | |
| 5/27/2020 | harlem, 750000000000, cuarón, kramer, minneapolis, gentrification, bradford, samuelsson, stan, criminalize | |
| 5/28/2020 | fables, fable, churchill's, aesop, churchill, phaedrus, cummings, kidney, aesop's, nepa | |
| 5/29/2020 | minneapolis, radiologists, glorifying, cartels, octopus, dumbo, cannabis, unarmed, cable, floyd | |
| 5/30/2020 | minneapolis, disgracing, unarmed, floyd, orgasm, macau, peacocks, forbes, walz, wanton | |
| 5/31/2020 | fables, dispersal, floyd, rivets, minneapolis, misplay, strontium, aesop, radiologists, syncopated | |
| 6/1/2020 | helium, fables, self-help, floyd, quakers, minneapolis, self-improvement, curfews, vigil, vitamin | |
| 6/2/2020 | helium, abortion, floyd, morningstar, zanzibar, abortions, minneapolis, floyd's, grayson, pro-life | |
| 6/3/2020 | floyd, centeno, reichstag, jamaica, rodney, minneapolis, curfews, bedside, landfall, mandarin | |
| 6/4/2020 | babylon, mattis, chechnya, mesopotamia, floyd, minneapolis, chechen, implicit, tiananmen, gilgamesh | |
| 6/5/2020 | floyd, minneapolis, pentridge, autopsies, mckenney, bregman, chokeholds, reparations, defibrillators, beavers | |
| 6/6/2020 | floyd, opec, goodell, elsinore, deacon, abortion, nfl, minneapolis, magreb, philharmonic | |
| 6/7/2020 | babylon, floyd, minneapolis, atai, deacon, hypochondria, floyd's, mesopotamia, zuckerberg, malpass | |
| 6/8/2020 | minneapolis, babylon, floyd, epstein, willow, superhero, toothbrush, toothbrushes, dismantle, mesopotamia | |
| 6/9/2020 | toothbrush, toothbrushes, floyd, tooth, leaflets, bristles, zone's, ipl, hotline, ksi | |
| 6/10/2020 | floyd, engstrom, hsbc, module, odysseys, odyssey, lovell, hypochondria, deborah, aquarius | |
| 6/11/2020 | confederate, seafarers, milley, confederacy, adidas, jefferson, richmond, statues, joan, nascar | |
| 6/12/2020 | atoms, rakhine, floyd, playstation, 20.4, everest, groundsman, sujit, pro-democracy, consulate | |
| 6/13/2020 | penguin, dalai, lama, thuggery, floyd, statues, squirrel, minneapolis, $41800.00, highlanders | |
| 6/14/2020 | atoms, toothbrushes, toothbrush, mansong, brooks, squirrel, spitfire, mangano, damson, trustworthiness | |
| 6/15/2020 | atoms, ressa, rappler, whelan, dupont, tasting, libel, kubler, wine, moshe | |
| 6/16/2020 | heel, heels, delinquent, dexa, dupont, liaison, curie, adh, rosamund, mahama | |
| 6/17/2020 | gambling, liaison, mansong, pyongyang, damson, aston, floyd, bolton, oxfam, biometric | |
| 6/18/2020 | bolton, bertha, bolton's, microgravity, electromagnetic, vera, brooks, dreamers, gaulle, clyde | |
| 6/19/2020 | juneteenth, pirbright, tulsa, triangle, pele, cuttlefish, scolari, oklahoma, eyed, wastewater | |
| 6/20/2020 | tulsa, berman, oklahoma, juneteenth, dynamo, obadiah, sharpton, giuliani, chandra's, jeffrey | |
| 6/21/2020 | berman, tulsa, spitfires, beaverbrook, mutinous, stabbing, dynamo, midwinter, cobain, oklahoma | |
| 6/22/2020 | vegan, dimitrov, nascar, icc, cuttlefish, grigor, stonehenge, veganism, patagonia, boneheaded | |
| 6/23/2020 | snares, fork, nascar, navarro, triplets, poacher, cutlery, gras, batman, placenta | |
| 6/24/2020 | segway, magicians, talley, dynamo, noose, abortions, demolish, vanadium, flynn, jerrys | |
| 6/25/2020 | silk, statues, locusts, kluger, quantas, eiffel, confederate, migraine, looters, licenses | |
| 6/26/2020 | ressa, filipinos, rappler, nectarines, astrid, dixie, cirebon, glastonbury, eldoret, sigma | |
| 6/27/2020 | silk, fianna, lazarus, pills, malawi's, woodrow, incompetent, statues, rerun, blah | |
| 6/28/2020 | spitfire, lazarus, rerun, woodrow, malawi's, malawi, duda, sedwill, macpherson, paddle | |
| 6/29/2020 | masculinity, silk, copperfield, abortion, transfusion, violin, beasley, 737, emblem, dogma | |
| 6/30/2020 | grandmothers, pollinators, jagger, honeybees, honeybee, bees, copperfield, pollination, arabella, richards | |
| 7/1/2020 | annexation, shockley, cremation, 2036, weinstein, memorials, 370, fairchild, argyle, tombstone | |
| 7/2/2020 | maxwell, epstein, jade, norse, hildebrand, encrypted, viking, gallic, 2036, paedophile | |
| 7/3/2020 | maxwell, epstein, nollywood, barefoot, kipsang, redskins, absentia, khashoggi, jamal, tolokonnikova | |
| 7/4/2020 | rushmore, barefoot, grandmothers, yucatan, cristina, eduardo's, norse, hildebrand, pubs, nollywood | |
| 7/5/2020 | poroshenko, grandmothers, barefoot, 690000, intersectionality, cochabamba, plank, chechen, spitfire, chechnya | |
| 7/6/2020 | barefoot, morricone, mandisa, ennio, cholera, pacu, overflights, justifying, temperamental, cushioning | |
| 7/7/2020 | depp, mandisa, toilets, jefferson, rebekah, camden, magnitsky, monaghan, mta, emin | |
| 7/8/2020 | feinstein, panthers, belgrade, rufus, montreal, wainwright, mcgarrigle, modesty, balloons, prt | |
| 7/9/2020 | tango, rita, oxfam, polynesians, rita's, subpoenas, rivera, blanka, coffins, morgues | |
| 7/10/2020 | earthworms, abs, barbados, bison, abse, gamers, lagoon, al-jazeera, lavrov, variant | |
| 7/11/2020 | bosnian, srebrenica, charlton, earthworms, tango, serb, tampering, commuted, bosnia, massacre | |
| 7/12/2020 | earthworms, louganis, duda, mueller, basque, muller, hungary's, galicia, lutheran, feinstein | |
| 7/13/2020 | earthworms, redskins, tango, chimpanzees, goodall, ishmael, worms, earthworm, dinesh, duda | |
| 7/14/2020 | timbuktu, chimps, huawei, rembrandt, redskins, maxwell, norah, rivera, mccrory, p.x. | |
| 7/15/2020 | stradivarius, violin, maxwell, huawei, chimps, preferential, ice-t, infirmary, $13000000000.00, epstein | |
| 7/16/2020 | chaucer, bitcoin, chaucer's, begum, sediment, azerbaijan, armenia, prebiotics, hackers, shamima | |
| 7/17/2020 | selassie, haile, 747, morphic, fairfield, 750000000000, charlton, bothroyd, fukuda, plo | |
| 7/18/2020 | chaucer, fellini, srebrenica, roxanne, chaucer's, redskins, pancreatic, 450000000000, lewis, luther | |
| 7/19/2020 | selassie, haile, fairfield, sterilization, divinity, tony's, srebrenica, britishness, majesty, embankment | |
| 7/20/2020 | chaucer, interferon, goodes, 750000000000, kanye, samantha, whales, propellers, picasso, propeller | |
| 7/21/2020 | chimps, chimpanzees, convent, gombe, oyelowo, rowing, 750000000000, frugal, woodard, carell | |
| 7/22/2020 | redheads, braca, consulate, tony's, rowing, redhead, boobies, maxwell, embankment, bile | |
| 7/23/2020 | tolstoy, yoho, tolstoy's, pre-term, rocca, portland, consulate, decemberists, albuquerque, frydenberg | |
| 7/24/2020 | chengdu, consulate, mosquitoes, epstein, havens, goldman, sachs, helaman, gambia, jeffrey | |
| 7/25/2020 | tolstoy, repossessed, turkmen, consulate, fleetwood, turkmenistan, dopey, havens, helaman, desking | |
| 7/26/2020 | christi, philbin, regis, corpus, essex, turkmen, cathedral, hypersonic, turkmenistan, havilland | |
| 7/27/2020 | westboro, tolstoy, hezbollah, corals, tolstoy's, infiltrate, phelps, opium, decemberists, followings | |
| 7/28/2020 | najib, razak, fusion, platoon, marlins, reactor, supersymmetry, portland, collider, dimensions | |
| 7/29/2020 | hedgehog, lactose, lactase, cyclops, sonic, kodak, mecca, milkshake, numbness, pilgrimage | |
| 7/30/2020 | picasso, rover, perseverance, poker, gideon, lauber, ovett, disqualified, goudreau, salvini | |
| 7/31/2020 | obesity, anelka, 18.5, tectonics, ransomware, champagne, garmin, fein, overweight, carbonate | |
| 8/1/2020 | murdoch, kabuki, lithium, obesity, cinderella, gymnasts, picasso, corp, 46600, laos | |
| 8/2/2020 | clive, nanotubes, hedgehog, neutrons, capsule, splash, manned, psychism, clive's, bogota | |
| 8/3/2020 | cinderella, poker, jalalabad, genoa, hume, leopards, blackjack, fleisher, gladwell, kobach | |
| 8/4/2020 | voicemail, genome, juan, jade, tanzanite, beirut, warheads, interpol, 155000, dvc | |
| 8/5/2020 | nitrate, ammonium, beirut, gapper, jadzia, lebanese, helen, blast, handcuffed, hiroshima | |
| 8/6/2020 | nra, beirut, hiroshima, gwen, constantinople, emperors, algerians, jadzia, 476, algeria | |
| 8/7/2020 | nra, beirut, rajapaksa, contrails, hezbollah, serota, overshot, lebanese, tuesday's, tick-tock | |
| 8/8/2020 | beirut, tuesday's, skidded, serota, arterton, ammonium, nitrate, lovelock, winfrey's, runway | |
| 8/9/2020 | loya, jirga, contrails, lukashenko, globalization, rajapaksa, beirut, nagasaki, mahinda, poker | |
| 8/10/2020 | contrails, easterbrook, lukashenko, contrail, beirut, tycoon, ly, pro-democracy, jimmy, belarus | |
| 8/11/2020 | origami, bandon, dunes, mclay, lithuania, arterton, braman, gorse, donoghue, lukashenko | |
| 8/12/2020 | gorda, origami, tinnitus, metaphors, harris, nieto, globalization, lukashenko, inn, dissolution | |
| 8/13/2020 | bradbury, redstone, briberies, viacom, minsk, annexation, palestinians, annex, sumner, anc | |
| 8/14/2020 | armenia, azerbaijan, absentee, postal, voter, ballots, nagorno-karabakh, lukashenko, azerbaijanis, azerbaijani | |
| 8/15/2020 | slime, bradbury, postal, shrine, minsk, lilo, lukashenko, emperor, surrender, 75th | |
| 8/16/2020 | slime, quilting, minsk, lukashenko, constantia, gee's, defectors, bishops, pettway, quilt | |
| 8/17/2020 | slime, bradbury, lukashenko, slugs, minsk, slug, snails, snail, belarus, beachside | |
| 8/18/2020 | hariri, rafik, hezbollah, fistula, diamonds, riri, heist, crowe, lukashenko, bamako | |
| 8/19/2020 | mali, belarus, lukashenko, quilting, diamonds, bamako, gee's, conservatorship, britney, boubacar | |
| 8/20/2020 | bannon, radiotherapy, tibetans, cutler, greenland, poisoned, tibet, syphilis, poisoning, alexina | |
| 8/21/2020 | yeast, glaciers, randy, poisoned, weaver, alexina, mahbubani, seaweed, greenland, novelli | |
| 8/22/2020 | weaver, pyramid, yeast, randy, lukashenko, cutler, minsk, pyramids, crespo, postal | |
| 8/23/2020 | emmett, minsk, lukashenko, unprincipled, emmett's, hogan, yeast, pentecostal, marianne, postal | |
| 8/24/2020 | yeast, ekpo, plasma, tinto, convalescent, pyramid, pyramids, poisoned, kellyanne, fda | |
| 8/25/2020 | cactus, ganymede, jupiter, cactuses, magnetic, jupiter's, jbs, polio, serb, bosnian | |
| 8/26/2020 | emmett, synesthesia, emmett's, jacob, blake, wisconsin, osha, ratko, firefox, mamie | |
| 8/27/2020 | louisiana, unsurvivable, jacob, wisconsin, surfing, blake, hurricane, milwaukee, parole, survivable | |
| 8/28/2020 | mariachi, plasma, doping, shinzo, fda, rowling, bletchley, jacob, convalescent, cadets | |
| 8/29/2020 | boseman, chadwick, panther, banksy, accreditation, cactus, synesthesia, ekpo, plasma, minsk | |
| 8/30/2020 | montenegro, cactus, montenegro's, bratislava, minsk, comedians, 840000, djukanovic, gunnery, lukashenko | |
| 8/31/2020 | montenegro, boredom, ekpo, underwater, lebanon's, fleming, hostage, portland, simone, bailiff's | |
| 9/1/2020 | cartoons, mugabe's, jacob, rover, abdo, exabyte, osha, republished, hungary, wisconsin | |
| 9/2/2020 | greenlandic, greenland, walkman, abdo, hebdo, novacek, satirical, camaro, woodard, greenlanders | |
| 9/3/2020 | smallpox, filipino, flynn, margarita, mongolian, $750000000.00, typhoon, baluchistan, pox, blisters | |
| 9/4/2020 | moroccans, berlusconi, lampedusa, silvio, messi, blake, jacob, lava, subtropics, esper | |
| 9/5/2020 | smallpox, paraguayan, rafi, rfp, walkman, icc, euthanasia, typhoon, ledo, lampedusa | |
| 9/6/2020 | bolshoi, minsk, lukashenko, kitteridge, typhoon, greenland, seuss', belarus, greenlandic, kittredge | |
| 9/7/2020 | albinism, smallpox, djokovic, 49.4, minsk, disqualified, novak, khashoggi, typhoon, kolesnik | |
| 9/8/2020 | kolesnik, birtles, rover, crater, socrates, carbonate, belarus, motorcycling, perseverance, lukashenko | |
| 9/9/2020 | morea, rochester, photovoltaic, kittredge, larry's, reddit, 13000, remittances, parchment, singletary | |
| 9/10/2020 | wellness, woodward, protestants, tiffany, weavers, nitrogen, huguenot, borneo, 1680s, amphibians | |
| 9/11/2020 | tinto, sturgis, aboriginal, morea, broderick, lesbos, bahrain, abrams, overrides, agnivesh | |
| 9/12/2020 | conran, afghans, naveed, doha, spinny, bahrain's, vizcarra, oregon, reggae, taliban | |
| 9/13/2020 | biosphere, minsk, rhine-westphalia, swingers, lukashenko, gormley, abbe, pemberton, nitrogen, celtic | |
| 9/14/2020 | pinocchio, broom, nvidia, sibylla, oracle, suger, softbank, belarus, hauser, snp | |
| 9/15/2020 | neon, bahrain, palestinians, arsonist, franco, whales, irritable, whale, beaumont, tummy | |
| 9/16/2020 | iaea, greensboro, diac, solstice, suger, chariot, atv, doping, grossi, wellness | |
| 9/17/2020 | wellness, preppers, prepping, nitrogen, georgian, ammonia, novacek, susie, barbie, georgians | |
| 9/18/2020 | 940000, 30000000, vocalisation, scranton, mucus, janina, entomologists, malyon, gimpy, punchier | |
| 9/19/2020 | ginsburg, bader, permafrost, slovenian, georgian, celery, chili, penultimate, feeney, thawing | |
| 9/20/2020 | ginsburg, bader, biosphere, slovenian, monarchy, hsbc, belfast, justices, hilliard, cyclist | |
| 9/21/2020 | abramovich, galileo, bader, ginsburg, shit's, chili, emmys, hsbc, seamstresses, cheeses | |
| 9/22/2020 | ginsburg, testosterone, bader, drowning, whales, trans, lifeguards, sirleaf, seeding, maple | |
| 9/23/2020 | whales, louisville, requiem, magnetic, endangerment, mozart, ben's, payloads, beaching, ginsburg | |
| 9/24/2020 | louisville, westpac, britney, briana, obesity, clemency, brianna, rhonda, blackwater, tobacco | |
| 9/25/2020 | monocytes, stillborn, chimpanzees, clemency, britney, archibald, goodes, cardinal, aggravated, carnival |
TECHNICAL DETAILS
For those interested in running this analysis themselves, here is the SQL query we used:
WITH nested AS (
WITH data AS (
SELECT TIMESTAMP_TRUNC(TIMESTAMP, DAY) date, STATION, NGRAM word, sum(COUNT) count FROM `gdelt-bq.gdeltv2.iaradio_1grams` where DATE(TIMESTAMP) >= "2020-01-01" and station='BBC World Service' group by date,station, NGRAM
)
, word_day_type AS (
# how many times a word is mentioned in each "document"
SELECT word, SUM(count) counts, date, station
FROM data
GROUP BY 1, 3, 4
)
, day_type AS (
# total # of words in each "document"
SELECT SUM(count) counts, date, station
FROM data
GROUP BY 2, 3
)
, tf AS (
# TF for a word in a "document"
SELECT word, date, station, a.counts/b.counts tf
FROM word_day_type a
JOIN day_type b
USING(date, station)
)
, word_in_docs AS (
# how many "documents" have a word
SELECT word, COUNT(DISTINCT FORMAT('%s %s', FORMAT_TIMESTAMP( "%m/%d/%E4Y", date), station)) indocs
FROM word_day_type
GROUP BY 1
)
, total_docs AS (
# total # of docs
SELECT COUNT(DISTINCT FORMAT('%s %s', FORMAT_TIMESTAMP( "%m/%d/%E4Y", date), station)) total_docs
FROM data
)
, idf AS (
# IDF for a word
SELECT word, LOG(total_docs.total_docs/indocs) idf
FROM word_in_docs
CROSS JOIN total_docs
)
SELECT date, station,
ARRAY_AGG(STRUCT(ARRAY_TO_STRING(words, ', ') AS wordlist)) top_words
FROM (
SELECT date, station, ARRAY_AGG(word ORDER BY tfidf DESC LIMIT 10) words
FROM (
SELECT word, date, station, tf.tf * idf.idf tfidf
FROM tf
JOIN idf
USING(word)
)
GROUP BY date, station
)
GROUP BY date, station
ORDER BY date DESC
) select FORMAT_TIMESTAMP( "%m/%d/%E4Y", date) day, station, wordlist from nested, UNNEST(top_words) order by date asc, station asc
We hope this serves as a template for your own linguistic explorations of radio news!