Earlier today we showed how TFIDF calculation over the Radio News NGrams dataset could be used to surface the most "significant" words on BBC World Service each day in 2020. What if we wanted to expand that analysis from single words to two-word phrases? Given that the ngrams dataset covers 1-5 grams, doing so requires nothing more than swapping the unigrams table for the bigrams table!
| Day | Top Words |
| 1/1/2020 | 9 lessons, true colors, floods around, and carols, monkey house, for slapping, popular mobilization, u.s. airstrikes, day pro-democracy, in baghdad |
| 1/2/2020 | david stern, the greens, google health, western sahara, turkey's parliament, in mammograms, via turkey, of mammograms, forced evacuations, intelligence system |
| 1/3/2020 | baghdad airport, iranian revolutionary, testicular cancer, attack americans, powerful military, silly money, australian navy, revolutionary guard, influenza avian, military commander |
| 1/4/2020 | ma ma, garcia luna, iraqi militia, militia commander, funeral procession, kong 6, iranian general, scale anti, beijing protests, you dish |
| 1/5/2020 | of sheba, sheba is, queen of, militia commander, serious cyber, iranian targets, simon robinson, attack possibly, cultural sites, military school |
| 1/6/2020 | cultural sites, iranian cultural, harvey weinstein, prolific rapist, golden globe, embattled australian, globe awards, big sanctions, longer abide, golden globes |
| 1/7/2020 | mr zarif, iran's foreign, care man, the burial, improve ties, denied iran's, super flex, burial of, in cyprus, month suspended |
| 1/8/2020 | ballistic missile, missile attack, bases housing, a ballistic, housing u.s., missile attacks, ukrainian passenger, air bases, bases in, iranian foreign |
| 1/9/2020 | prince harry, royal family, ukrainian plane, lyme disease, royal roles, packaging may, harry and, militias not, taliban splinter, financially independent |
| 1/10/2020 | sleep apnea, ukrainian airliner, 176 people, advance again, a missile, pursue military, the ukrainian, high life, tongue may, iran's civil |
| 1/11/2020 | sultan qaboos, yemen last, iranian commander, social housing, serving ruler, military tried, affordable housing, the sultan, a bloodless, bloodless coup |
| 1/12/2020 | stage fright, might soothe, little wades, bombay and, jones david, ethnic muslims, david jones, against drug, comedian jenny, soothe your |
| 1/13/2020 | former pope, national trust, pope benedict, hazardous eruption, toxic gas, the volcano, a hazardous, prince harry, ukrainian airliner, kangaroo island |
| 1/14/2020 | currency manipulator, formal dispute, without signing, a currency, longer label, strongman general, libyan military, label china, turkish intermediaries, manipulator the |
| 1/15/2020 | the milf, early pregnancy, of post-traumatic, fuel over, personnel loyal, red sox, china's vice, month trade, skin irritation, nation address |
| 1/16/2020 | george stinney, of impeachment, the impeachment, articles of, original wild, grant transit, won't grant, transit visas, beyond 2047, prehistoric tree |
| 1/17/2020 | supreme leader, than cancer, sepsis is, carbon negative, friday prayers, the impeachment, shooting down, 6 point, propaganda chief, that sepsis |
| 1/18/2020 | century fox, giving pledge, philanthropy is, of philanthropy, corruption mission, melissa berman, union regulations, banknotes issued, london estimate, than official |
| 1/19/2020 | royal titles, missile attack, countable infinity, their royal, chess federation, receive public, working memory, prince harry, quantum uncertainty, her fortune |
| 1/20/2020 | richest woman, her fortune, africa's richest, exploiting her, woman made, gun rights, one joe, kittredge fungus, new respiratory, among one |
| 1/21/2020 | impeachment trial, the impeachment, economic forum, in davos, world economic, the dodo, mitch mcconnell, good gym, prophets of, senate republican |
| 1/22/2020 | terry jones, impeachment trial, crown prince, jeff bezos, amazon founder, founder jeff, phone of, the impeachment, arabia's crown, as absurd |
| 1/23/2020 | un's highest, international court, of auschwitz, of genocide, liberation of, yad vashem, provisional measures, scale lunar, highest court, the liberation |
| 1/24/2020 | frozen zoo, traumatic brain, pollution controls, 20000 civilians, impeachment trial, stop pregnant, women traveling, erasmus darwin, eating dinosaur, aleppo province |
| 1/25/2020 | dam collapse, ever environmental, brazil's worst, the dodo, least 41, impeachment trial, eastern turkey, frozen zoo, brazilian town, 41 people |
| 1/26/2020 | new capital, once influential, erasmus darwin, in landslides, triple 7, memorial scrolls, republic day, view after, the gestapo, dutch jews |
| 1/27/2020 | kobe bryant, helicopter crash, of auschwitz, southern poland, to kobe, staples center, liberation by, the 75th, the grammys, 75th anniversary |
| 1/28/2020 | the palestinians, strong minds, is melting, peace plan, mobile network, limited role, g mobile, melting increasingly, forced labor, field project |
| 1/29/2020 | christmas island, peace plan, the palestinians, ever quarterly, important town, record $22000000000.00, john bolton, japanese nationals, block witnesses, bank settlements |
| 1/30/2020 | christine sinclair, sinclair has, c.j. walker, cocaine production, dna medicine, 5 babies, canadian women's, clinical testing, 170 people, john bolton |
| 1/31/2020 | impeachment trial, national renewal, cervical cancer, call witnesses, trump's impeachment, australian capital, pub crawl, lamar alexander, cancer could, 47 years |
| 2/1/2020 | of membership, obtaining visas, to permanent, signaled his, immediately signaled, jim thorpe, against calling, impeachment trial, immigration controls, from obtaining |
| 2/2/2020 | $170000000000.00 into, father colomba, of timbuktu, pentecostal service, than $170000000000.00, looser trade, strip belonging, markets reopen, the manuscripts, union rejecting |
| 2/3/2020 | super bowl, kansas city, city chiefs, the kansas, than $170000000000.00, 100000000 americans, 35 syrian, strong minds, the chiefs, stabbing 2 |
| 2/4/2020 | the iowa, in iowa, empress dowager, kenya daniel, of iowa, jacob zuma, hydrogen is, iowa democratic, the hydrogen, walking football |
| 2/5/2020 | american comeback, union address, vocal patterns, human language, penguins follow, same general, great american, information compression, than primates, her copy |
| 2/6/2020 | kirk douglas, michael johnson, forming stars, 560 people, of 103, gluten sensitivity, chinese doctor, breast size, stopped forming, gluten free |
| 2/7/2020 | qaeda in, arabian peninsula, al qaeda, credit suisse, of al, kill 70, 70 elephants, the arabian, trophy hunters, licenses to |
| 2/8/2020 | best song, colonel alexander, stabilized slightly, impeachment inquiry, new volunteer, shopping center, new hampshire, testimony during, volunteer militia, rifle killing |
| 2/9/2020 | exit poll, shopping mall, sexual orientation, small grains, banning discrimination, satellite into, converting people, cover sexual, 26 people, shin fein |
| 2/10/2020 | red book, best picture, korean film, international feature, torture technique, language film, the oscars, best original, the cdu, won best |
| 2/11/2020 | bride price, smith black, black mambazo, lady smith, joseph shabalala, new hampshire, j.k. rowling, harry potter, international criminal, encryption devices |
| 2/12/2020 | new hampshire, married men, hampshire primary, bernie sanders, roger stone, detaining migrants, attack syrian, priestly celibacy, missionary as, evangelical missionary |
| 2/13/2020 | of dresden, sajid javid, the audit, training mission, nato defense, province at, top communist, new fossils, dresden by, largest turtles |
| 2/14/2020 | entire board, somalia kenya, not menstruating, directors demanded, job impossible, film academy, academy has, ethiopia somalia, stop tweeting, of locusts |
| 2/15/2020 | miss america, security conference, munich security, the munich, andrew mccabe, prayer ceremonies, the camorra, generally under, aircraft imported, play regulations |
| 2/16/2020 | miss america, gospel music, munich security, caroline flack, security conference, hip hop, the munich, saudi pilots, micro pubs, diamond princess |
| 2/17/2020 | farm subsidies, musicians union, nearly 1800, of cameroon, american passengers, absolute misery, australian soldiers, the voyager, pension reform, planes chartered |
| 2/18/2020 | boy scouts, scouts of, pledged $10000000000.00, gezi park, jeff bezos, gamma oscillations, the quartet, $10000000000.00 to, richest person, the cellist |
| 2/19/2020 | credentials of, real sick, journal reporters, has revoked, 3 wall, gas attacks, in yokohama, sick man, diamond princess, time travel |
| 2/20/2020 | super gun, right attack, michael bloomberg, mike bloomberg, eugene de, mr bloomberg, deadly far, 9 people, roger stone, starving civilians |
| 2/21/2020 | prison guard, 9000 members, b.n. b, air b.n., resistant bacteria, taliban negotiators, italian archaeologists, religious group, future forward, against hatred |
| 2/22/2020 | lady smith, the nile, joseph shabalala, him moscow, grave phase, black mambazo, henry cotton, smith black, 1st vice, 5th death |
| 2/23/2020 | anticipated boxing, tyson fury, nevada caucuses, mosque cathedral, in nevada, bernie sanders, the nevada, boxing matches, his status, bulldozer picked |
| 2/24/2020 | harvey weinstein, cricket stadium, islamic jihad, carnival parade, official visit, alberto salazar, dr torrey's, livingston island, alan dershowitz, buy tents |
| 2/25/2020 | hosni mubarak, harvey weinstein, mubarak was, president hosni, mr mubarak, between opponents, nazi flags, burgess said, citizenship law, former egyptian |
| 2/26/2020 | bilingual homes, hosni mubarak, military funeral, in bilingual, president hosni, bob iger, under concerted, bernie sanders, from bilingual, in switching |
| 2/27/2020 | 3rd runway, expand heathrow, foreign pilgrims, comic books, michael bloomberg, heathrow airport, postponed joint, pence to, strategic town, runway at |
| 2/28/2020 | turkish troops, roman polanski, 33 turkish, turkish soldiers, galaxy hundreds, universe since, super massive, massive black, known explosion, italian citizen |
| 2/29/2020 | the taliban, roman polanski, several actresses, taliban leaders, transport free, afghan militant, giving m.p.'s, phased withdrawal, unexplained coronavirus, more unexplained |
| 3/1/2020 | the oracle, his holiness, dalai lama, ice wine, 2 syrian, the dalai, super tuesday, government warplanes, state oracle, arab parties |
| 3/2/2020 | pius the, 3rd general, former indiana, super tuesday, of pius, the vatican, pope pius, hitler's pope, papacy of, indiana mayor |
| 3/3/2020 | super tuesday, former rivals, 3rd general, coffee grounds, exit polls, wing bloc, russell jones, country's 3rd, been endorsed, claimed victory |
| 3/4/2020 | super tuesday, tuesday primaries, bernie sanders, against taliban, michael bloomberg, rethinking its, currency following, following resistance, digital currency, mr bloomberg |
| 3/5/2020 | fried chicken, alleged war, fly b, the icc, haile selassie, 30 turkish, threatening thousands, regional airlines, investigation can, british carrier |
| 3/6/2020 | haile selassie, of warplanes, warplanes for, syrian province, of bethlehem, widespread episodes, coral bleaching, afghan capital, capital kabul, changing bodies |
| 3/7/2020 | bin nayef, former crown, mohammed bin, crown prince, prince mohammed, mccoy tyner, haile selassie, grand princess, woody allen, aboriginal settlements |
| 3/8/2020 | women's day, international women's, mandatory measures, haile selassie, 16000000 people, mark international, fried chicken, venezuelan interior, arrested 17, evening raid |
| 3/9/2020 | von sydow, max von, warren elizabeth, m.h. 17, one ukrainian, 3 russians, malaysian airlines, flight m.h., sexual life, oil price |
| 3/10/2020 | ethiopian airlines, priests to, information warning, approving constitutional, joined twitter, the kakapo, endorse donald, grand princess, interim report, thai is |
| 3/11/2020 | nadine dorries, george pell, reception hosted, harvey weinstein, street reception, bernie sanders, cardinal george, latest primaries, deadly collision, prisoners within |
| 3/12/2020 | street food, british soldier, iraqi militia, 2 americans, harvey weinstein, world wastes, wastes hundreds, rocket attack, renewable plants, effect friday |
| 3/13/2020 | chelsea manning, wiki leaks, intelligence analyst, analyst chelsea, to wiki, ivory coast, fundamentally changed, fewer responsibilities, combined apart, for offending |
| 3/14/2020 | giant penguin, fossilized skin, jonathan quick, chile's minister, identified fragments, penguin that, man kidnapped, russia's main, main security, national emergency |
| 3/15/2020 | black birding, street food, by 75, capital region, dissident artist, followed italy, artist luis, parking almost, mr otero, m.p.'s backing |
| 3/16/2020 | central banks, compassion fatigue, love letter, external borders, father juan, british warplanes, king philippe, avoid pubs, test test, ensure liquidity |
| 3/17/2020 | external borders, loan guarantees, government form, 2020 football, catherine mackinnon, than $130000000000.00, football championships, championships have, criticized moves, lady pink |
| 3/18/2020 | florida illinois, bernie sanders, and arizona, illinois and, prevent soldiers, past overseas, 1st amendment, marsha hunt, race winning, contest heavily |
| 3/19/2020 | guide dogs, guide dog, european central, 34 new, female like, robotic arm, grown cases, angry drivers, skid row, caribbean were |
| 3/20/2020 | guide dogs, guide dog, were hanged, executed 4, vampire bats, daughter's photo, got justice, olympic flame, a hypersonic, hypersonic missile |
| 3/21/2020 | kenny rogers, stephanie flanders, yanis varoufakis, american households, with dolly, rogers has, french academic, illinois connecticut, dollar economic, country music |
| 3/22/2020 | guide dogs, guide dog, 50 cuban, comic books, mountain gorillas, african ancestry, 14 hour, africa's endangered, cuban doctors, pharmacies banks |
| 3/23/2020 | guide dogs, guide dog, olympics may, the ioc, since infection, games may, jazz violinist, electronic trading, hope asia's, women composers |
| 3/24/2020 | the phosphorus, the asterix, the phosphorous, jeffrey taubenberger, vegetative state, comics has, of phosphorus, phosphorus and, phosphorus in, the sludge |
| 3/25/2020 | lunar module, senate leaders, 2 trillion, apollo 11, turkish prosecutors, command module, mcconnell described, wartime level, general intelligence, 738 people |
| 3/26/2020 | g 20, states filed, 5 trillion, without dating, patient 31, 20 countries, benefits last, the g, mr levinson, travel disperse |
| 3/27/2020 | our cash, cleve jones, day pushing, polio virus, unless medical, stop foreigners, die unless, more confirmed, week nationwide, fund says |
| 3/28/2020 | such program, largest such, firm's capacity, his firm's, has partially, emergency spending, beep beep, the aliens, without dating, doubled production |
| 3/29/2020 | north korean, leading infectious, pandemic italy, italy's deputy, rescued 14, latest asian, spain say, maximum we, korean defectors, to 200000 |
| 3/30/2020 | sunk cost, oxfam says, equipped public, prevent millions, panther party, black panther, marked decrease, $160000000000.00 is, levels more, the sunk |
| 3/31/2020 | down syndrome, 849 people, cancer often, human chromosome, 50 types, after love, quaking aspen, economies contract, approved indefinite, symptoms experts |
| 4/1/2020 | north korean, korean defectors, rough 2, national trial, porn industry, trial unlike, having smaller, another television, celebrity defectors, command module |
| 4/2/2020 | daniel pearl, british militant, for kidnapping, journalist daniel, pearl in, women's organizations, apollo 13, almost exhausted, an epicurean, peter culture |
| 4/3/2020 | food trade, tim lang, output mr, merchant sailors, quarantine vessels, international chamber, huge cuts, yes q, announce huge, shipping industry |
| 4/4/2020 | targeting hospitals, agency interpol, interpol has, cyber criminals, that cyber, infected link, moderate politician, computers unless, criminals are, planting malicious |
| 4/5/2020 | british comedians, mr jabril, toughest 2, electric lights, catherine calderwood, scotland's chief, their lights, a gimmick, avoid travel, firm fujifilm |
| 4/6/2020 | senior new, the homebrew, computer club, persistent symptoms, temporary plots, homebrew computer, abuse hotline, equality minister, families struggle, a placebo |
| 4/7/2020 | george pell, cardinal george, coffee pot, vatican treasurer, publishing daily, former vatican, symptoms worsened, given oxygen, superhero movies, started publishing |
| 4/8/2020 | john prine, european research, postpartum psychosis, 6 degrees, top scientist, scientific program, crescent moon, the crescent, fatima bhutto, prine who |
| 4/9/2020 | london bridge, identity intervention, and disengagement, healthy identity, so seche, disengagement program, sharply negative, agency comes, japanese literature, pollen count |
| 4/10/2020 | nato is, that nato, finance ministers, nato member, past earth, a maneuver, maneuver that, city's daily, in nato, jointly developing |
| 4/11/2020 | wow wow, relaxation could, 53 severely, shown promising, premature relaxation, involving 53, showed improvement, whose conviction, cardinal whose, cardinal pell |
| 4/12/2020 | refugee teaches, his easter, billionaire philanthropist, stella morris, fixed mindset, within 18, peter's basilica, julian assange, stirling moss, ecuadorian embassy |
| 4/13/2020 | open secrets, the vagina, brian cox, open secret, powerful tornadoes, breathing muscles, logan roy, weather including, tornadoes in, economic hibernation |
| 4/14/2020 | noise pollution, savage beauty, diving board, unlucky people, against noise, the dive, the imf, afghan charities, dive and, eva rothschild |
| 4/15/2020 | suspend funding, of mismanaging, cognition and, kept further, cold cognition, airline revenue, refugee teaches, cleaning regimes, payments owed, lunar module |
| 4/16/2020 | mega drought, women's conference, tails which, month's victory, fruity smell, at females, would risk, sacred but, was sacred, victory parade |
| 4/17/2020 | space station, 6.8 percent, identify coronavirus, genetic traces, and inconsistent, zeke emanuel, sewage test, found genetic, empowering governors, and restarting |
| 4/18/2020 | martin lee, storing hydrogen, of storing, mosques may, pakistan's president, ramadan which, zeke emanuel, fuel cars, leading clerics, democracy movement |
| 4/19/2020 | missed 5, johnson missed, its patrols, midnight mass, the landfill, marking easter, studded concert, 8 new, attending religious, iranian vessel |
| 4/20/2020 | project debater, 3 cruise, rhetoric is, benny gantz, still sailing, domestic workers, about menstruation, u.s. oil, jumping spiders, web building |
| 4/21/2020 | immigration into, all immigration, suspend immigration, temporarily suspend, oil markets, the jesus, u.s. oil, west texas, brazil's prosecutor, brant crude |
| 4/22/2020 | james webb, military satellite, space telescope, webb space, 1st military, seeking permanent, killer whales, the hubble, earth day, sex robots |
| 4/23/2020 | carbon credits, carbon green, mackay to, green africa, bright said, mckay to, red plus, syria's security, agency overseeing, 2 malaria |
| 4/24/2020 | base camp, petrol shortages, sunlight heat, nicotine products, stephen moore, tennessee williams, bill bryan, and humidity, to sunlight, climb everest |
| 4/25/2020 | red mercury, immunity passports, life crisis, stephen moore, vigils at, domestic workers, holding private, private vigils, annual day, red marker |
| 4/26/2020 | life crisis, southern transitional, transitional council, self rule, quarter life, have catastrophic, day accompanied, aden also, council based, by separatists |
| 4/27/2020 | of meritocracy, sweat glands, meritocracy can, meritocracy is, that meritocracy, in meritocracy, meritocracy and, highly forwarded, meritocratic societies, hardest working |
| 4/28/2020 | block chain, generic asian, begins easing, block change, asian man, obaid chinoy, asian flu, serious investigations, lebanon's economic, trafficking as |
| 4/29/2020 | life crisis, the pfd, the dividend, permanent fund, quarter life, personal friend, fund dividend, overturn president, basic income, secure food |
| 4/30/2020 | jazz day, international jazz, 5 g, already acutely, past 60000, magnitude unprecedented, montana most, be brothers, christine legarde, 5.8 percent |
| 5/1/2020 | tony allen, vaccine shipments, drama tony, assaulted a, tara reid, foreign spy, out reconnaissance, senate staff, nigerian drama, spy agencies |
| 5/2/2020 | barbara bush, ship breaking, geriatric center, isabella geriatric, breaking yard, york nursing, the isabella, attempted jailbreak, fertilizer plant, its discrimination |
| 5/3/2020 | afghan migrants, iranian border, korean troops, chinese indonesian, exchanged gunfire, buffett says, enormous evidence, the demilitarized, indonesian chinese, the pfd |
| 5/4/2020 | fund research, health laboratory, tanzania's national, control malaria, televised town, enormous potential, blocking bug, malaria blocking, my ira, international requests |
| 5/5/2020 | 1st officially, daily jump, australian cabinet, overseas since, country's revolutionary, coronavirus meeting, restoring travel, december 4, pneumonia patients, stranded overseas |
| 5/6/2020 | magnetic pole, north magnetic, emissions ever, describe with, thomas cromwell, earth's outer, outer core, further relaxation, past misconduct, confidence what's |
| 5/7/2020 | gas leak, hazel scott, george pell, chemical plant, leak at, further 3200000, cafe society, restrict his, resolution passed, cardinal george |
| 5/8/2020 | jamie harrison, the 75th, 75th anniversary, 14.7 percent, card companies, south carolina, exact origins, pornography industry, to 14.7, ve day |
| 5/9/2020 | little richard, victory day, my password, rock n, n roll, siegfried and, roy horn, katie miller, main victory, 2nd white |
| 5/10/2020 | graphic novel, chaotic disaster, stay alert, prolonged war, ski on, jet ski, congress announced, prioritize human, ridley turtles, absolute chaotic |
| 5/11/2020 | particle physics, most primary, smoke detector, 19 sailors, single housing, 30 sailors, hazel scott, stay alert, living allowance, small cluster |
| 5/12/2020 | child line, will had, oxygen concentrator, in testosterone, temporarily reopening, resume offensive, chris hemsworth, shin bet, to needless, secret mr |
| 5/13/2020 | rogers' neighborhood, the pill, mr rogers', to needless, child line, apparently free, of cognac, distributing pollen, oldest bottles, catholic shrine |
| 5/14/2020 | widespread psychological, darkest winter, stem cell, make mental, richard burr, insider trading, gold rush, british cheese, gold fields, based response |
| 5/15/2020 | 10 customers, to referee, bonded laborers, organized crime, complaints council, industrial output, 3 baltic, to 190000, of fortresses, economy germany |
| 5/16/2020 | team human, french iranian, iranian academic, security charges, white stork, the genocide, rwandan genocide, general steve, steve linnik, closures will |
| 5/17/2020 | patsy cline, apply israeli, sharing deal, power sharing, rival abdullah, ministrations handling, below 100, mr ghani, unity government, spitfire fund |
| 5/18/2020 | health assembly, wynton marsalis, human waste, o's ministerial, ministerial assembly, european recovery, independent review, lockdown shops, world needed, flush toilet |
| 5/19/2020 | tori amos, jeff goldblum, emissions fell, australian barley, puppet of, independent evaluation, percent tariffs, the membrane, crippling 80, impose crippling |
| 5/20/2020 | baby powder, powerful south, asian storms, eastern india, release $1000000000.00, china learn, potential side, based baby, storms for, selling talcum |
| 5/21/2020 | color wheel, limit freedoms, the multis, control agreement, open skies, major arms, arms control, easter weekend, finance fraud, coronavirus catastrophe |
| 5/22/2020 | security law, effect collapsed, annual session, residential area, crimes prosecutor, major suspect, of karachi, forgiven his, immense economic, beijing's decision |
| 5/23/2020 | mr cummings, dominic cummings, men's rights, spanish football's, karachi on, his parents', hertz has, groups regional, 8th after, cummings over |
| 5/24/2020 | mr cummings, 1st pro-democracy, dominic cummings, decade event, winchester road, waters carrying, dennis webb, michael morpurgo, storm described, toppling him |
| 5/25/2020 | dominic cummings, school reunion, mr cummings, emergency imposed, female friendships, rocco 40, languages can, suspended testing, school reunions, includes loans |
| 5/26/2020 | stanley ho, dominic cummings, mr cummings, eastern kentucky, in macau, fuel savings, and freedoms, allay concerns, haitian federation, east kentucky |
| 5/27/2020 | larry kramer, one jew, potentially misleading, stifling free, would criminalize, down social, his tweets, director alfonso, criminalize insulting, platforms if |
| 5/28/2020 | the fables, acute kidney, kidney injury, curtail individual, potentially misleading, down social, the kidney, security law, rugby league, the kidneys |
| 5/29/2020 | of minneapolis, glorifying violence, industrial countries, emerging markets, 7 health, in minneapolis, executive order, cnn reporter, an unarmed, of glorifying |
| 5/30/2020 | in minneapolis, space x, entire national, an unarmed, astronauts into, female pleasure, incursions by, border incursions, minneapolis where, ethiopian militia |
| 5/31/2020 | george floyd, blamed looters, space station, radicals for, wing radicals, international space, american astronauts, the fables, 30 cities, dozen cities |
| 6/1/2020 | george floyd, emergency bunker, vigil commemorating, the fables, 75 cities, ebola have, vitamin d, cloud forest, 7 former, the quakers |
| 6/2/2020 | military against, george floyd, of helium, self management, biggest aid, helium in, emily watson, helium is, the helium, peaceful protesters |
| 6/3/2020 | george floyd, defense secretary, offer millions, julia child, of george, trump's proposal, told britain, mario centeno, insurrection act, u.s. defense |
| 6/4/2020 | james mattis, implicit bias, of babylon, george floyd, chili bowl, global vaccine, children against, secretary james, vaccine summit, tiananmen square |
| 6/5/2020 | george floyd, nuclear submarines, frank pentridge, of george, 34000 brazilians, in minneapolis, 10th consecutive, chokeholds following, using chokeholds, vienna philharmonic |
| 6/6/2020 | george floyd, discuss extending, dan deacon, mountain railways, record cuts, national football, public lands, roger goodell, samuel was, vienna philharmonic |
| 6/7/2020 | george floyd, police killing, of george, incendiary statements, deborah levy, dan deacon, zuckerberg say, 100 scientists, extending cuts, of babylon |
| 6/8/2020 | george floyd, wonder woman, prince andrew, speed record, active coronavirus, in minneapolis, minneapolis city, to dismantle, world speed, dismantle its |
| 6/9/2020 | tooth brush, george floyd, traffic signs, sales agent, hotline between, zone's 2, ground penetrating, penetrating radar, leaflets being, a toothbrush |
| 6/10/2020 | george floyd, frances berry, deborah levy, mary frances, native houston, swedish prime, graphic designer, of palma, hot milk, floyd the |
| 6/11/2020 | of arc, jefferson davis, joan of, of jefferson, the confederacy, the confederate, mark milley, general milley, adidas has, in richmond |
| 6/12/2020 | hello kitty, george floyd, policy proved, term threat, 3 strikes, labeling millions, police bias, requiring hundreds, brutality triggered, strikes law |
| 6/13/2020 | club penguin, penguin online, dalai lama, george floyd, 3 strikes, right activists, the dalai, wholesale market, racist thuggery, 400000000 people |
| 6/14/2020 | of arc, joan of, railway carriages, tooth brush, 500 railway, including far, of atlanta, michelle dams, than $250000000.00, cash had |
| 6/15/2020 | maria ressa, paul whelan, of libel, u.s. marine, journalist maria, many atoms, mr tenet, state secrets, of arc, russian court |
| 6/16/2020 | the heel, high heels, high heel, wearing heels, joint liaison, liaison office, indian army, disputed border, passed 8000000, harm children |
| 6/17/2020 | pyongyang blew, into border, joint liaison, soldiers back, send soldiers, liaison office, 1000 flights, online gambling, indian soldiers, border areas |
| 6/18/2020 | vera lynn, john bolton, foreign funding, win reelection, highly classified, restricting foreign, illegally as, mr brooks, technology provided, lynn whose |
| 6/19/2020 | red triangle, cyber attack, with espionage, sophisticated state, brazil team, symbol used, state based, 150000 new, based actor, brown eyed |
| 6/20/2020 | jeffrey berman, mr berman, in tulsa, tulsa oklahoma, southern district, u.s. attorney, attorney for, will override, minister obadiah, brown eyed |
| 6/21/2020 | mr berman, multiple stabbing, jeffrey berman, shadow factory, might intervene, stabbing in, mutinous soldiers, mogadishu have, intervene militarily, in tulsa |
| 6/22/2020 | sporting goods, prehistoric monument, the icc, new trees, holiday centers, start treaty, grigor dimitrov, nuclear arms, 20 massive, went vegan |
| 6/23/2020 | peter navarro, wildlife before, miranda july, triplets in, anyone known, trade advisor, advisor peter, ordered president, january remains, the fork |
| 6/24/2020 | the segway, demolish statues, controversial leaders, supermarket customers, prevailing situation, suspended unspecified, unspecified plans, key referendum, denounces israel's, a noose |
| 6/25/2020 | 150 pilots, and connecticut, cardiovascular events, korean soldiers, the silk, wing campaigner, including asylum, grounded nearly, with aura, constitutional reforms |
| 6/26/2020 | seizing rockets, iraqi commandos, baghdad arresting, 2023 women's, falsifying their, flying credentials, their flying, 19 diary, maria ressa, blood types |
| 6/27/2020 | these pills, order protecting, arts fighters, now label, these pages, the silk, fair trade, fermenting violence, unfiltered statements, hear unfiltered |
| 6/28/2020 | shouting white, a rerun, the spitfire, white power, liberal center, supporters shouting, paddle out, of warsaw, mexico olympics, open beyond |
| 6/29/2020 | blood types, blood group, blood groups, blood type, for post-traumatic, find reports, afghanistan credible, david beasley, david copperfield, confederate emblem |
| 6/30/2020 | the grandmothers, union's ambassador, abortion law, friendship bench, famous acrobatic, cancel dozens, restrictive abortion, wine glass, acrobatic circus, security law |
| 7/1/2020 | law imposed, security law, 370 people, the annexation, arrests under, new security, west bank, 16 more, about 370, prominent musician |
| 7/2/2020 | commission murders, of smell, jeffrey epstein, a jade, jade mine, encrypted messaging, move drugs, major criminals, with grooming, zealand's health |
| 7/3/2020 | 20 saudi, arabian nationals, saudi journalist, unknown group, saudi consulate, nuclear fuel, main nuclear, journalist jamal, nationals for, jeffrey epstein |
| 7/4/2020 | independence day, mount rushmore, 15 weeks, $31.00 soldiers, colombian army, soldiers accused, barefoot running, perfectly preserved, adopting children, has diluted |
| 7/5/2020 | homeland movement, mr poroshenko, plank of, deaths appears, independence day, covered public, wildlife parks, about 690000, petro poroshenko, center left |
| 7/6/2020 | barefoot running, ennio morricone, justifying terrorism, composer ennio, of justifying, security law, park or, by despair, robotic scientist, repressive state |
| 7/7/2020 | public toilets, johnny depp, vigorously implement, ghost map, amber heard, a toilet, home restrictions, star johnny, classroom teaching, provided classroom |
| 7/8/2020 | the panthers, capital belgrade, serbian capital, harvey milk, security office, airborne transmission, weekend curfew, officially passed, web science, passed 3000000 |
| 7/9/2020 | financial records, the tango, tax returns, prosecutors can, blank among, his tax, of tango, guarantee confidentiality, everywhere coffins, coffins everywhere |
| 7/10/2020 | tax returns, nuclear confrontation, top broadcaster, his tax, 2nd south, people's action, istanbul back, financial records, stop broadcasting, action party |
| 7/11/2020 | roger stone, bosnian serb, jack charlton, the bosnian, serb forces, people's action, african classical, mr stone, mark 25, of srebrenica |
| 7/12/2020 | african classical, robert muller, robert mueller, conservative incumbent, christian culture, his investigation, roger stone, of warsaw, big wing, offered further |
| 7/13/2020 | your dna, the earthworms, washington redskins, jane goodall, the soil, the tango, the chimpanzees, financial fair, remains public, of earthworms |
| 7/14/2020 | night watch, the painting, islamic police, in timbuktu, mali nearly, its 5, 3 words, norah jones, provocation which, painting we |
| 7/15/2020 | jerry brown, security law, general court, spring watch, new security, reach only, 9000000000 by, people fewer, neighboring armenia, only 9000000000 |
| 7/16/2020 | fuel oil, siberian arctic, arctic provides, fake tweets, heavy fuel, year's unprecedented, by manmade, tainted fuel, provides unequivocal, unprecedented heat |
| 7/17/2020 | haile selassie, jack charlton, coronavirus recovery, pleaded with, as forceful, forceful as, worst flooding, fairfield house, new morphic, top infectious |
| 7/18/2020 | john lewis, biggest powers, blood blood, russia's far, civilian roles, campaigner john, recovery package, 1963 along, desperate debt, rights campaigner |
| 7/19/2020 | haile selassie, fairfield house, his majesty, coronavirus recovery, of haile, bus designed, selassie was, blood blood, a saint, rust the |
| 7/20/2020 | coronavirus recovery, extradition treaty, adam goodes, treaty with, quarantine program, 750000000000 euro, security departments, when arresting, kenya airways, john myatt |
| 7/21/2020 | suitable boy, the convent, russian interference, coronavirus recovery, governments failed, security committee, bending arc, brief congress, recovery package, spread disinformation |
| 7/22/2020 | its consulate, ordered china, endangered birds, coronavirus briefing, consulate in, farms across, red hair, animal rights, briefing for, the bracket |
| 7/23/2020 | comfort food, comfort foods, blue spaces, deficit since, ted yoho, federal officers, pre-term birth, visa fraud, budget deficit, federal agents |
| 7/24/2020 | of chengdu, consulate in, tax havens, 1st friday, chinese consulate, hungary's leading, leading independent, friday prayers, goldman sachs, muslim friday |
| 7/25/2020 | repossessed the, have repossessed, peter green, preventing violence, istanbul convention, executive orders, 16 far, right politicians, prescription drugs, escalating diplomatic |
| 7/26/2020 | southern texas, corpus christi, volunteer warden, hurricane hanna, of corpus, nation tour, 240 hours, admitted starting, 5 nation, require travelers |
| 7/27/2020 | string theory, humanity over, infiltrate israel, take working, working holidays, investigate president, ever declared, westboro baptist, followings on, big followings |
| 7/28/2020 | string theory, fusion reactor, najib razak, minister najib, nuclear fusion, razak has, mr najib, brian greene, virtual choir, sing gently |
| 7/29/2020 | sonic hedgehog, annual muslim, muslim pilgrimage, with drastically, the raft, contender joe, presidential contender, federal agents, the cyclops, reduced numbers |
| 7/30/2020 | play play, operation gideon, jordan good, perseverance rover, 12 prominent, the perseverance, and fraudulent, john lewis, latest mission, red planet |
| 7/31/2020 | pandemic 6, nicolas anelka, plate tectonics, capital harare, crooked election, alan parker, lou mcdonald, emergency committee, obesity is, election hours |
| 8/1/2020 | james murdoch, murdoch has, news corp, lisa john, as saturday, allegedly obstructing, editorial content, rupert murdoch, mogul rupert, obstructing efforts |
| 8/2/2020 | x capsule, space x, commercial manned, splash down, space station, international space, mexico later, string theory, row india, 1st commercial |
| 8/3/2020 | john hume, new bridge, 4 babies, babies whose, the cinderella, fairy godmother, of cinderella, space x, leon fleisher, folk tales |
| 8/4/2020 | suitable boy, human genome, ignore orders, juan carlos, former king, genome project, is receding, major creditors, carlos has, lung transplant |
| 8/5/2020 | ammonium nitrate, of ammonium, secretive system, handcuffed to, in beirut, rare footage, inside china's, china's secretive, port area, gapper has |
| 8/6/2020 | the nra, roman empire, rifle association, national rifle, powerful national, in beirut, with fraud, 1st atomic, hiroshima to, almost immune |
| 8/7/2020 | in beirut, we chat, the nra, ice shelf, placing sanctions, u.s. transactions, lankan politics, india express, transactions with, the amazon |
| 8/8/2020 | executive orders, sign executive, take executive, india plane, executive action, female health, venezuelan court, host oprah, pearl harbor, in beirut |
| 8/9/2020 | good war, grand council, loya jirga, of elders, alexander lukashenko, towards eradicating, lanka's former, incumbent alexander, leader mahinda, human genome |
| 8/10/2020 | jimmy lie, apple daily, media tycoon, tycoon jimmy, security law, their taxes, prominent pro-democracy, pro-democracy activist, wealth managers, alexander lukashenko |
| 8/11/2020 | bandon dunes, at bandon, emma donoghue, david mclay, golf course, gemma arterton, mclay kid, produced strong, heart failure, apple daily |
| 8/12/2020 | running mate, virgin gorda, parliament ahead, ticket in, presidential ticket, california senator, ms harris, slump on, his running, unify americans |
| 8/13/2020 | private charter, ray bradbury, harris have, charter flights, sumner redstone, the palestinians, all private, emirates have, him rebuild, siberian woolly |
| 8/14/2020 | by mail, voter rolls, vote by, mail in, struggling u.s., center without, begun releasing, postal service, blocking funding, deputy interior |
| 8/15/2020 | deep remorse, postal service, past military, japan's surrender, of japan's, war criminals, the postal, expressed deep, allied nations, historical disputes |
| 8/16/2020 | high priest, catholic bishops, universal mail, and constantia, central bangkok, gee's bend, anti-government rally, dead relatives, that universal, chain reaction |
| 8/17/2020 | slime is, alexander lukashenko, the slime, transitional leader, death valley, upmarket hotel, hour siege, the beachside, ordered google, or assault |
| 8/18/2020 | rafik hariri, minister rafik, lebanon's former, mr hariri, michelle obama, of diamonds, hezbollah militant, b anthony, hariri the, russell crowe |
| 8/19/2020 | in mali, formally nominated, be spilled, of mali, spilled to, national convention, transitional government, putrid smell, virtual national, 2nd evening |
| 8/20/2020 | mr bannon, of radiotherapy, chief strategist, bannon has, steve bannon, strategist steve, the bachelor, food media, death penalty, food writing |
| 8/21/2020 | russian opposition, death valley, these glaciers, randy weaver, ruby ridge, russian doctors, delivered securely, air ambulance, popular leader, of cloaking |
| 8/22/2020 | ruby ridge, suspected poisoning, randy weaver, russian opposition, great pyramid, growing resurgence, 12 should, russian doctors, $25000000000.00 into, postal service |
| 8/23/2020 | divine protection, marianne trump, emmett till, postal service, marianne making, sister marianne, secret recordings, of divine, independence square, revealed of |
| 8/24/2020 | strike leaders, storm laura, great pyramid, rio tinto, margaret ekpo, katie hill, emergency authorization, convalescent plasma, jacob blake, australian white |
| 8/25/2020 | magnetic field, wild polio, cactus is, serb military, former bosnian, bosnian serb, national convention, in cactus, royalist marketplace, refugee resettlement |
| 8/26/2020 | crew change, change guide, jacob blake, emmett till, train doc, man jacob, republican national, of emmett, can osha, serb commander |
| 8/27/2020 | jacob blake, hurricane laura, storm surge, without parole, hurrican laura, reserve police, of louisiana, shooting of, dead 51, 51 muslim |
| 8/28/2020 | outspoken head, doping agency, the outspoken, financial violations, japan's longest, emirates killed, serving prime, drone operated, of presiding, climate review |
| 8/29/2020 | chadwick boseman, black panther, panther in, rescue boat, boseman the, artist banksy, immediate assistance, colon cancer, education university, british graffiti |
| 8/30/2020 | serbian orthodox, powerful serbian, face briefings, orthodox church, passed 25000000, in montenegro, resolve violent, colon cancer, of socialists, consecutive sunday |
| 8/31/2020 | sam harris, commercial flight, ever commercial, in montenegro, formal ties, local groups, between israel, prominent news, paraded in, kigali after |
| 9/1/2020 | afghan women, alan walker, data storage, nearly 1400000, $70000.00 children, controversial cartoons, alleged conspirators, storage devices, offices 5, sharing news |
| 9/2/2020 | black holes, the walkman, satirical magazine, rouge regime, french satirical, nerve agent, operation linked, flight simulator, removed fake, deadly terror |
| 9/3/2020 | of smallpox, smallpox and, foster farms, offered rewards, obama gate, in baluchistan, reducing misinformation, pipeline from, opposition campaigner, lucrative foreign |
| 9/4/2020 | q and, material belonging, to jacob, against moroccans, silvio berlusconi, minister silvio, jacob blake, highest respect, digital library, intelligence analysts |
| 9/5/2020 | entirely responsible, beyond major, by environmentalists, the paraguayan, passed 4000000, spreads beyond, indian counterpart, incurable medical, frenchman with, planned death |
| 9/6/2020 | released body, that legislative, delayed them, william jordan, lord frost, of seuss', bolshoi theater, famous bolshoi, york attorney, alexander lukashenko |
| 9/7/2020 | line judge, with albinism, withdrawal agreement, of consciousness, masked men, hard border, 49.4 degrees, poisoned russian, founder julian, jordanian army |
| 9/8/2020 | financial review, australian financial, diplomatic standoff, mike smith, national happiness, day diplomatic, her passport, opposition colleagues, beautiful short, short loser |
| 9/9/2020 | migrant camp, 13000 people, morea migrant, west antarctica, hold worldwide, the morea, mexican farmers, of terrorizing, bob woodward, stolen credit |
| 9/10/2020 | bob woodward, knowingly and, of wellness, her illegally, willingly lied, journalist bob, and willingly, destroying nature, coast destroying, deport her |
| 9/11/2020 | rio tinto, aboriginal sites, abrams said, giant rio, robert abrams, microsoft says, mining giant, ratify any, government negotiators, from shareholders |
| 9/12/2020 | 1st formal, terence conran, afghan government, impeachment proceedings, incapacity deepening, tortured into, open impeachment, champion wrestler, matthew green, moral incapacity |
| 9/13/2020 | biosphere 2, united russia, the biosphere, russia party, month's coup, key test, state north, permanent migrant, father joe, local elections |
| 9/14/2020 | douglas ross, arm holdings, based chip, scottish conservative, tech monopoly, the broom, japan's governing, maker arm, softbank group, chief cabinet |
| 9/15/2020 | and bahrain, climate arsonist, irritable bowel, with israel, unmarked graves, diplomatic agreements, unveil legislation, spain's left, shot 8, ruled spain |
| 9/16/2020 | the iaea, suppress political, the flock, the atv, 7 ethnic, alabama florida, doping procedures, u.s. gulf, hurrican sally, after bahrain |
| 9/17/2020 | nitrogen crisis, new camp, online fraud, the wellness, susie was, located across, brazil paraguayan, storm sally, say traces, vaccine could |
| 9/18/2020 | red blood, prizes which, above 30000000, globe has, seems improbable, 940000 people, 30000000 according, america argentina, found apparently, 30000000 cases |
| 9/19/2020 | bader ginsburg, ruth bader, justice ruth, medical medium, selling sensitive, liberal justice, chinese spies, delay following, latest women's, british diplomat |
| 9/20/2020 | ruth bader, bader ginsburg, biosphere 2, slovenian cyclist, justice ginsburg, medical medium, nominee would, main administration, northern irish, celery juice |
| 9/21/2020 | galileo was, ruth bader, bader ginsburg, roman abramovich, shit's creek, space shuttle, activity reports, suspicious activity, katie hill, oligarch roman |
| 9/22/2020 | pilot whales, sea ice, dark web, general assembly, jinping condemned, of maple, bader ginsburg, plague upon, ruth bader, their contested |
| 9/23/2020 | space weather, pilot whales, whales have, whose applications, magnetic field, solar wind, frontline states, biggest beaching, relocate numbers, storm watch |
| 9/24/2020 | fisheries official, of louisville, patrol vessel, sure rhonda, in louisville, laundering laws, briana taylor, drug raid, louisville during, korean troops |
| 9/25/2020 | aggravated homicide, stillborn child, a stillborn, fisheries official, that loosens, loosens rules, the archibald, abortion which, homicide after, of aggravated |
TECHNICAL DETAILS
Again, the query is identical to our single word TFIDF example, with the only change being the use of "gdelt-bq.gdeltv2.iaradio_2grams" instead of "gdelt-bq.gdeltv2.iaradio_1grams" as the source table!
WITH nested AS (
WITH data AS (
SELECT TIMESTAMP_TRUNC(TIMESTAMP, DAY) date, STATION, NGRAM word, sum(COUNT) count FROM `gdelt-bq.gdeltv2.iaradio_2grams` where DATE(TIMESTAMP) >= "2020-01-01" and station='BBC World Service' group by date,station, NGRAM
)
, word_day_type AS (
# how many times a word is mentioned in each "document"
SELECT word, SUM(count) counts, date, station
FROM data
GROUP BY 1, 3, 4
)
, day_type AS (
# total # of words in each "document"
SELECT SUM(count) counts, date, station
FROM data
GROUP BY 2, 3
)
, tf AS (
# TF for a word in a "document"
SELECT word, date, station, a.counts/b.counts tf
FROM word_day_type a
JOIN day_type b
USING(date, station)
)
, word_in_docs AS (
# how many "documents" have a word
SELECT word, COUNT(DISTINCT FORMAT('%s %s', FORMAT_TIMESTAMP( "%m/%d/%E4Y", date), station)) indocs
FROM word_day_type
GROUP BY 1
)
, total_docs AS (
# total # of docs
SELECT COUNT(DISTINCT FORMAT('%s %s', FORMAT_TIMESTAMP( "%m/%d/%E4Y", date), station)) total_docs
FROM data
)
, idf AS (
# IDF for a word
SELECT word, LOG(total_docs.total_docs/indocs) idf
FROM word_in_docs
CROSS JOIN total_docs
)
SELECT date, station,
ARRAY_AGG(STRUCT(ARRAY_TO_STRING(words, ', ') AS wordlist)) top_words
FROM (
SELECT date, station, ARRAY_AGG(word ORDER BY tfidf DESC LIMIT 10) words
FROM (
SELECT word, date, station, tf.tf * idf.idf tfidf
FROM tf
JOIN idf
USING(word)
)
GROUP BY date, station
)
GROUP BY date, station
ORDER BY date DESC
) select FORMAT_TIMESTAMP( "%m/%d/%E4Y", date) day, station, wordlist from nested, UNNEST(top_words) order by date asc, station asc