The GDELT Project

TFIDF Using BigQuery + Radio News NGrams To Chart The Most Significant Phrases Per Day On BBC World Service In 2020

Earlier today we showed how TFIDF calculation over the Radio News NGrams dataset could be used to surface the most "significant" words on BBC World Service each day in 2020. What if we wanted to expand that analysis from single words to two-word phrases? Given that the ngrams dataset covers 1-5 grams, doing so requires nothing more than swapping the unigrams table for the bigrams table!

Day Top Words
1/1/2020 9 lessons, true colors, floods around, and carols, monkey house, for slapping, popular mobilization, u.s. airstrikes, day pro-democracy, in baghdad
1/2/2020 david stern, the greens, google health, western sahara, turkey's parliament, in mammograms, via turkey, of mammograms, forced evacuations, intelligence system
1/3/2020 baghdad airport, iranian revolutionary, testicular cancer, attack americans, powerful military, silly money, australian navy, revolutionary guard, influenza avian, military commander
1/4/2020 ma ma, garcia luna, iraqi militia, militia commander, funeral procession, kong 6, iranian general, scale anti, beijing protests, you dish
1/5/2020 of sheba, sheba is, queen of, militia commander, serious cyber, iranian targets, simon robinson, attack possibly, cultural sites, military school
1/6/2020 cultural sites, iranian cultural, harvey weinstein, prolific rapist, golden globe, embattled australian, globe awards, big sanctions, longer abide, golden globes
1/7/2020 mr zarif, iran's foreign, care man, the burial, improve ties, denied iran's, super flex, burial of, in cyprus, month suspended
1/8/2020 ballistic missile, missile attack, bases housing, a ballistic, housing u.s., missile attacks, ukrainian passenger, air bases, bases in, iranian foreign
1/9/2020 prince harry, royal family, ukrainian plane, lyme disease, royal roles, packaging may, harry and, militias not, taliban splinter, financially independent
1/10/2020 sleep apnea, ukrainian airliner, 176 people, advance again, a missile, pursue military, the ukrainian, high life, tongue may, iran's civil
1/11/2020 sultan qaboos, yemen last, iranian commander, social housing, serving ruler, military tried, affordable housing, the sultan, a bloodless, bloodless coup
1/12/2020 stage fright, might soothe, little wades, bombay and, jones david, ethnic muslims, david jones, against drug, comedian jenny, soothe your
1/13/2020 former pope, national trust, pope benedict, hazardous eruption, toxic gas, the volcano, a hazardous, prince harry, ukrainian airliner, kangaroo island
1/14/2020 currency manipulator, formal dispute, without signing, a currency, longer label, strongman general, libyan military, label china, turkish intermediaries, manipulator the
1/15/2020 the milf, early pregnancy, of post-traumatic, fuel over, personnel loyal, red sox, china's vice, month trade, skin irritation, nation address
1/16/2020 george stinney, of impeachment, the impeachment, articles of, original wild, grant transit, won't grant, transit visas, beyond 2047, prehistoric tree
1/17/2020 supreme leader, than cancer, sepsis is, carbon negative, friday prayers, the impeachment, shooting down, 6 point, propaganda chief, that sepsis
1/18/2020 century fox, giving pledge, philanthropy is, of philanthropy, corruption mission, melissa berman, union regulations, banknotes issued, london estimate, than official
1/19/2020 royal titles, missile attack, countable infinity, their royal, chess federation, receive public, working memory, prince harry, quantum uncertainty, her fortune
1/20/2020 richest woman, her fortune, africa's richest, exploiting her, woman made, gun rights, one joe, kittredge fungus, new respiratory, among one
1/21/2020 impeachment trial, the impeachment, economic forum, in davos, world economic, the dodo, mitch mcconnell, good gym, prophets of, senate republican
1/22/2020 terry jones, impeachment trial, crown prince, jeff bezos, amazon founder, founder jeff, phone of, the impeachment, arabia's crown, as absurd
1/23/2020 un's highest, international court, of auschwitz, of genocide, liberation of, yad vashem, provisional measures, scale lunar, highest court, the liberation
1/24/2020 frozen zoo, traumatic brain, pollution controls, 20000 civilians, impeachment trial, stop pregnant, women traveling, erasmus darwin, eating dinosaur, aleppo province
1/25/2020 dam collapse, ever environmental, brazil's worst, the dodo, least 41, impeachment trial, eastern turkey, frozen zoo, brazilian town, 41 people
1/26/2020 new capital, once influential, erasmus darwin, in landslides, triple 7, memorial scrolls, republic day, view after, the gestapo, dutch jews
1/27/2020 kobe bryant, helicopter crash, of auschwitz, southern poland, to kobe, staples center, liberation by, the 75th, the grammys, 75th anniversary
1/28/2020 the palestinians, strong minds, is melting, peace plan, mobile network, limited role, g mobile, melting increasingly, forced labor, field project
1/29/2020 christmas island, peace plan, the palestinians, ever quarterly, important town, record $22000000000.00, john bolton, japanese nationals, block witnesses, bank settlements
1/30/2020 christine sinclair, sinclair has, c.j. walker, cocaine production, dna medicine, 5 babies, canadian women's, clinical testing, 170 people, john bolton
1/31/2020 impeachment trial, national renewal, cervical cancer, call witnesses, trump's impeachment, australian capital, pub crawl, lamar alexander, cancer could, 47 years
2/1/2020 of membership, obtaining visas, to permanent, signaled his, immediately signaled, jim thorpe, against calling, impeachment trial, immigration controls, from obtaining
2/2/2020 $170000000000.00 into, father colomba, of timbuktu, pentecostal service, than $170000000000.00, looser trade, strip belonging, markets reopen, the manuscripts, union rejecting
2/3/2020 super bowl, kansas city, city chiefs, the kansas, than $170000000000.00, 100000000 americans, 35 syrian, strong minds, the chiefs, stabbing 2
2/4/2020 the iowa, in iowa, empress dowager, kenya daniel, of iowa, jacob zuma, hydrogen is, iowa democratic, the hydrogen, walking football
2/5/2020 american comeback, union address, vocal patterns, human language, penguins follow, same general, great american, information compression, than primates, her copy
2/6/2020 kirk douglas, michael johnson, forming stars, 560 people, of 103, gluten sensitivity, chinese doctor, breast size, stopped forming, gluten free
2/7/2020 qaeda in, arabian peninsula, al qaeda, credit suisse, of al, kill 70, 70 elephants, the arabian, trophy hunters, licenses to
2/8/2020 best song, colonel alexander, stabilized slightly, impeachment inquiry, new volunteer, shopping center, new hampshire, testimony during, volunteer militia, rifle killing
2/9/2020 exit poll, shopping mall, sexual orientation, small grains, banning discrimination, satellite into, converting people, cover sexual, 26 people, shin fein
2/10/2020 red book, best picture, korean film, international feature, torture technique, language film, the oscars, best original, the cdu, won best
2/11/2020 bride price, smith black, black mambazo, lady smith, joseph shabalala, new hampshire, j.k. rowling, harry potter, international criminal, encryption devices
2/12/2020 new hampshire, married men, hampshire primary, bernie sanders, roger stone, detaining migrants, attack syrian, priestly celibacy, missionary as, evangelical missionary
2/13/2020 of dresden, sajid javid, the audit, training mission, nato defense, province at, top communist, new fossils, dresden by, largest turtles
2/14/2020 entire board, somalia kenya, not menstruating, directors demanded, job impossible, film academy, academy has, ethiopia somalia, stop tweeting, of locusts
2/15/2020 miss america, security conference, munich security, the munich, andrew mccabe, prayer ceremonies, the camorra, generally under, aircraft imported, play regulations
2/16/2020 miss america, gospel music, munich security, caroline flack, security conference, hip hop, the munich, saudi pilots, micro pubs, diamond princess
2/17/2020 farm subsidies, musicians union, nearly 1800, of cameroon, american passengers, absolute misery, australian soldiers, the voyager, pension reform, planes chartered
2/18/2020 boy scouts, scouts of, pledged $10000000000.00, gezi park, jeff bezos, gamma oscillations, the quartet, $10000000000.00 to, richest person, the cellist
2/19/2020 credentials of, real sick, journal reporters, has revoked, 3 wall, gas attacks, in yokohama, sick man, diamond princess, time travel
2/20/2020 super gun, right attack, michael bloomberg, mike bloomberg, eugene de, mr bloomberg, deadly far, 9 people, roger stone, starving civilians
2/21/2020 prison guard, 9000 members, b.n. b, air b.n., resistant bacteria, taliban negotiators, italian archaeologists, religious group, future forward, against hatred
2/22/2020 lady smith, the nile, joseph shabalala, him moscow, grave phase, black mambazo, henry cotton, smith black, 1st vice, 5th death
2/23/2020 anticipated boxing, tyson fury, nevada caucuses, mosque cathedral, in nevada, bernie sanders, the nevada, boxing matches, his status, bulldozer picked
2/24/2020 harvey weinstein, cricket stadium, islamic jihad, carnival parade, official visit, alberto salazar, dr torrey's, livingston island, alan dershowitz, buy tents
2/25/2020 hosni mubarak, harvey weinstein, mubarak was, president hosni, mr mubarak, between opponents, nazi flags, burgess said, citizenship law, former egyptian
2/26/2020 bilingual homes, hosni mubarak, military funeral, in bilingual, president hosni, bob iger, under concerted, bernie sanders, from bilingual, in switching
2/27/2020 3rd runway, expand heathrow, foreign pilgrims, comic books, michael bloomberg, heathrow airport, postponed joint, pence to, strategic town, runway at
2/28/2020 turkish troops, roman polanski, 33 turkish, turkish soldiers, galaxy hundreds, universe since, super massive, massive black, known explosion, italian citizen
2/29/2020 the taliban, roman polanski, several actresses, taliban leaders, transport free, afghan militant, giving m.p.'s, phased withdrawal, unexplained coronavirus, more unexplained
3/1/2020 the oracle, his holiness, dalai lama, ice wine, 2 syrian, the dalai, super tuesday, government warplanes, state oracle, arab parties
3/2/2020 pius the, 3rd general, former indiana, super tuesday, of pius, the vatican, pope pius, hitler's pope, papacy of, indiana mayor
3/3/2020 super tuesday, former rivals, 3rd general, coffee grounds, exit polls, wing bloc, russell jones, country's 3rd, been endorsed, claimed victory
3/4/2020 super tuesday, tuesday primaries, bernie sanders, against taliban, michael bloomberg, rethinking its, currency following, following resistance, digital currency, mr bloomberg
3/5/2020 fried chicken, alleged war, fly b, the icc, haile selassie, 30 turkish, threatening thousands, regional airlines, investigation can, british carrier
3/6/2020 haile selassie, of warplanes, warplanes for, syrian province, of bethlehem, widespread episodes, coral bleaching, afghan capital, capital kabul, changing bodies
3/7/2020 bin nayef, former crown, mohammed bin, crown prince, prince mohammed, mccoy tyner, haile selassie, grand princess, woody allen, aboriginal settlements
3/8/2020 women's day, international women's, mandatory measures, haile selassie, 16000000 people, mark international, fried chicken, venezuelan interior, arrested 17, evening raid
3/9/2020 von sydow, max von, warren elizabeth, m.h. 17, one ukrainian, 3 russians, malaysian airlines, flight m.h., sexual life, oil price
3/10/2020 ethiopian airlines, priests to, information warning, approving constitutional, joined twitter, the kakapo, endorse donald, grand princess, interim report, thai is
3/11/2020 nadine dorries, george pell, reception hosted, harvey weinstein, street reception, bernie sanders, cardinal george, latest primaries, deadly collision, prisoners within
3/12/2020 street food, british soldier, iraqi militia, 2 americans, harvey weinstein, world wastes, wastes hundreds, rocket attack, renewable plants, effect friday
3/13/2020 chelsea manning, wiki leaks, intelligence analyst, analyst chelsea, to wiki, ivory coast, fundamentally changed, fewer responsibilities, combined apart, for offending
3/14/2020 giant penguin, fossilized skin, jonathan quick, chile's minister, identified fragments, penguin that, man kidnapped, russia's main, main security, national emergency
3/15/2020 black birding, street food, by 75, capital region, dissident artist, followed italy, artist luis, parking almost, mr otero, m.p.'s backing
3/16/2020 central banks, compassion fatigue, love letter, external borders, father juan, british warplanes, king philippe, avoid pubs, test test, ensure liquidity
3/17/2020 external borders, loan guarantees, government form, 2020 football, catherine mackinnon, than $130000000000.00, football championships, championships have, criticized moves, lady pink
3/18/2020 florida illinois, bernie sanders, and arizona, illinois and, prevent soldiers, past overseas, 1st amendment, marsha hunt, race winning, contest heavily
3/19/2020 guide dogs, guide dog, european central, 34 new, female like, robotic arm, grown cases, angry drivers, skid row, caribbean were
3/20/2020 guide dogs, guide dog, were hanged, executed 4, vampire bats, daughter's photo, got justice, olympic flame, a hypersonic, hypersonic missile
3/21/2020 kenny rogers, stephanie flanders, yanis varoufakis, american households, with dolly, rogers has, french academic, illinois connecticut, dollar economic, country music
3/22/2020 guide dogs, guide dog, 50 cuban, comic books, mountain gorillas, african ancestry, 14 hour, africa's endangered, cuban doctors, pharmacies banks
3/23/2020 guide dogs, guide dog, olympics may, the ioc, since infection, games may, jazz violinist, electronic trading, hope asia's, women composers
3/24/2020 the phosphorus, the asterix, the phosphorous, jeffrey taubenberger, vegetative state, comics has, of phosphorus, phosphorus and, phosphorus in, the sludge
3/25/2020 lunar module, senate leaders, 2 trillion, apollo 11, turkish prosecutors, command module, mcconnell described, wartime level, general intelligence, 738 people
3/26/2020 g 20, states filed, 5 trillion, without dating, patient 31, 20 countries, benefits last, the g, mr levinson, travel disperse
3/27/2020 our cash, cleve jones, day pushing, polio virus, unless medical, stop foreigners, die unless, more confirmed, week nationwide, fund says
3/28/2020 such program, largest such, firm's capacity, his firm's, has partially, emergency spending, beep beep, the aliens, without dating, doubled production
3/29/2020 north korean, leading infectious, pandemic italy, italy's deputy, rescued 14, latest asian, spain say, maximum we, korean defectors, to 200000
3/30/2020 sunk cost, oxfam says, equipped public, prevent millions, panther party, black panther, marked decrease, $160000000000.00 is, levels more, the sunk
3/31/2020 down syndrome, 849 people, cancer often, human chromosome, 50 types, after love, quaking aspen, economies contract, approved indefinite, symptoms experts
4/1/2020 north korean, korean defectors, rough 2, national trial, porn industry, trial unlike, having smaller, another television, celebrity defectors, command module
4/2/2020 daniel pearl, british militant, for kidnapping, journalist daniel, pearl in, women's organizations, apollo 13, almost exhausted, an epicurean, peter culture
4/3/2020 food trade, tim lang, output mr, merchant sailors, quarantine vessels, international chamber, huge cuts, yes q, announce huge, shipping industry
4/4/2020 targeting hospitals, agency interpol, interpol has, cyber criminals, that cyber, infected link, moderate politician, computers unless, criminals are, planting malicious
4/5/2020 british comedians, mr jabril, toughest 2, electric lights, catherine calderwood, scotland's chief, their lights, a gimmick, avoid travel, firm fujifilm
4/6/2020 senior new, the homebrew, computer club, persistent symptoms, temporary plots, homebrew computer, abuse hotline, equality minister, families struggle, a placebo
4/7/2020 george pell, cardinal george, coffee pot, vatican treasurer, publishing daily, former vatican, symptoms worsened, given oxygen, superhero movies, started publishing
4/8/2020 john prine, european research, postpartum psychosis, 6 degrees, top scientist, scientific program, crescent moon, the crescent, fatima bhutto, prine who
4/9/2020 london bridge, identity intervention, and disengagement, healthy identity, so seche, disengagement program, sharply negative, agency comes, japanese literature, pollen count
4/10/2020 nato is, that nato, finance ministers, nato member, past earth, a maneuver, maneuver that, city's daily, in nato, jointly developing
4/11/2020 wow wow, relaxation could, 53 severely, shown promising, premature relaxation, involving 53, showed improvement, whose conviction, cardinal whose, cardinal pell
4/12/2020 refugee teaches, his easter, billionaire philanthropist, stella morris, fixed mindset, within 18, peter's basilica, julian assange, stirling moss, ecuadorian embassy
4/13/2020 open secrets, the vagina, brian cox, open secret, powerful tornadoes, breathing muscles, logan roy, weather including, tornadoes in, economic hibernation
4/14/2020 noise pollution, savage beauty, diving board, unlucky people, against noise, the dive, the imf, afghan charities, dive and, eva rothschild
4/15/2020 suspend funding, of mismanaging, cognition and, kept further, cold cognition, airline revenue, refugee teaches, cleaning regimes, payments owed, lunar module
4/16/2020 mega drought, women's conference, tails which, month's victory, fruity smell, at females, would risk, sacred but, was sacred, victory parade
4/17/2020 space station, 6.8 percent, identify coronavirus, genetic traces, and inconsistent, zeke emanuel, sewage test, found genetic, empowering governors, and restarting
4/18/2020 martin lee, storing hydrogen, of storing, mosques may, pakistan's president, ramadan which, zeke emanuel, fuel cars, leading clerics, democracy movement
4/19/2020 missed 5, johnson missed, its patrols, midnight mass, the landfill, marking easter, studded concert, 8 new, attending religious, iranian vessel
4/20/2020 project debater, 3 cruise, rhetoric is, benny gantz, still sailing, domestic workers, about menstruation, u.s. oil, jumping spiders, web building
4/21/2020 immigration into, all immigration, suspend immigration, temporarily suspend, oil markets, the jesus, u.s. oil, west texas, brazil's prosecutor, brant crude
4/22/2020 james webb, military satellite, space telescope, webb space, 1st military, seeking permanent, killer whales, the hubble, earth day, sex robots
4/23/2020 carbon credits, carbon green, mackay to, green africa, bright said, mckay to, red plus, syria's security, agency overseeing, 2 malaria
4/24/2020 base camp, petrol shortages, sunlight heat, nicotine products, stephen moore, tennessee williams, bill bryan, and humidity, to sunlight, climb everest
4/25/2020 red mercury, immunity passports, life crisis, stephen moore, vigils at, domestic workers, holding private, private vigils, annual day, red marker
4/26/2020 life crisis, southern transitional, transitional council, self rule, quarter life, have catastrophic, day accompanied, aden also, council based, by separatists
4/27/2020 of meritocracy, sweat glands, meritocracy can, meritocracy is, that meritocracy, in meritocracy, meritocracy and, highly forwarded, meritocratic societies, hardest working
4/28/2020 block chain, generic asian, begins easing, block change, asian man, obaid chinoy, asian flu, serious investigations, lebanon's economic, trafficking as
4/29/2020 life crisis, the pfd, the dividend, permanent fund, quarter life, personal friend, fund dividend, overturn president, basic income, secure food
4/30/2020 jazz day, international jazz, 5 g, already acutely, past 60000, magnitude unprecedented, montana most, be brothers, christine legarde, 5.8 percent
5/1/2020 tony allen, vaccine shipments, drama tony, assaulted a, tara reid, foreign spy, out reconnaissance, senate staff, nigerian drama, spy agencies
5/2/2020 barbara bush, ship breaking, geriatric center, isabella geriatric, breaking yard, york nursing, the isabella, attempted jailbreak, fertilizer plant, its discrimination
5/3/2020 afghan migrants, iranian border, korean troops, chinese indonesian, exchanged gunfire, buffett says, enormous evidence, the demilitarized, indonesian chinese, the pfd
5/4/2020 fund research, health laboratory, tanzania's national, control malaria, televised town, enormous potential, blocking bug, malaria blocking, my ira, international requests
5/5/2020 1st officially, daily jump, australian cabinet, overseas since, country's revolutionary, coronavirus meeting, restoring travel, december 4, pneumonia patients, stranded overseas
5/6/2020 magnetic pole, north magnetic, emissions ever, describe with, thomas cromwell, earth's outer, outer core, further relaxation, past misconduct, confidence what's
5/7/2020 gas leak, hazel scott, george pell, chemical plant, leak at, further 3200000, cafe society, restrict his, resolution passed, cardinal george
5/8/2020 jamie harrison, the 75th, 75th anniversary, 14.7 percent, card companies, south carolina, exact origins, pornography industry, to 14.7, ve day
5/9/2020 little richard, victory day, my password, rock n, n roll, siegfried and, roy horn, katie miller, main victory, 2nd white
5/10/2020 graphic novel, chaotic disaster, stay alert, prolonged war, ski on, jet ski, congress announced, prioritize human, ridley turtles, absolute chaotic
5/11/2020 particle physics, most primary, smoke detector, 19 sailors, single housing, 30 sailors, hazel scott, stay alert, living allowance, small cluster
5/12/2020 child line, will had, oxygen concentrator, in testosterone, temporarily reopening, resume offensive, chris hemsworth, shin bet, to needless, secret mr
5/13/2020 rogers' neighborhood, the pill, mr rogers', to needless, child line, apparently free, of cognac, distributing pollen, oldest bottles, catholic shrine
5/14/2020 widespread psychological, darkest winter, stem cell, make mental, richard burr, insider trading, gold rush, british cheese, gold fields, based response
5/15/2020 10 customers, to referee, bonded laborers, organized crime, complaints council, industrial output, 3 baltic, to 190000, of fortresses, economy germany
5/16/2020 team human, french iranian, iranian academic, security charges, white stork, the genocide, rwandan genocide, general steve, steve linnik, closures will
5/17/2020 patsy cline, apply israeli, sharing deal, power sharing, rival abdullah, ministrations handling, below 100, mr ghani, unity government, spitfire fund
5/18/2020 health assembly, wynton marsalis, human waste, o's ministerial, ministerial assembly, european recovery, independent review, lockdown shops, world needed, flush toilet
5/19/2020 tori amos, jeff goldblum, emissions fell, australian barley, puppet of, independent evaluation, percent tariffs, the membrane, crippling 80, impose crippling
5/20/2020 baby powder, powerful south, asian storms, eastern india, release $1000000000.00, china learn, potential side, based baby, storms for, selling talcum
5/21/2020 color wheel, limit freedoms, the multis, control agreement, open skies, major arms, arms control, easter weekend, finance fraud, coronavirus catastrophe
5/22/2020 security law, effect collapsed, annual session, residential area, crimes prosecutor, major suspect, of karachi, forgiven his, immense economic, beijing's decision
5/23/2020 mr cummings, dominic cummings, men's rights, spanish football's, karachi on, his parents', hertz has, groups regional, 8th after, cummings over
5/24/2020 mr cummings, 1st pro-democracy, dominic cummings, decade event, winchester road, waters carrying, dennis webb, michael morpurgo, storm described, toppling him
5/25/2020 dominic cummings, school reunion, mr cummings, emergency imposed, female friendships, rocco 40, languages can, suspended testing, school reunions, includes loans
5/26/2020 stanley ho, dominic cummings, mr cummings, eastern kentucky, in macau, fuel savings, and freedoms, allay concerns, haitian federation, east kentucky
5/27/2020 larry kramer, one jew, potentially misleading, stifling free, would criminalize, down social, his tweets, director alfonso, criminalize insulting, platforms if
5/28/2020 the fables, acute kidney, kidney injury, curtail individual, potentially misleading, down social, the kidney, security law, rugby league, the kidneys
5/29/2020 of minneapolis, glorifying violence, industrial countries, emerging markets, 7 health, in minneapolis, executive order, cnn reporter, an unarmed, of glorifying
5/30/2020 in minneapolis, space x, entire national, an unarmed, astronauts into, female pleasure, incursions by, border incursions, minneapolis where, ethiopian militia
5/31/2020 george floyd, blamed looters, space station, radicals for, wing radicals, international space, american astronauts, the fables, 30 cities, dozen cities
6/1/2020 george floyd, emergency bunker, vigil commemorating, the fables, 75 cities, ebola have, vitamin d, cloud forest, 7 former, the quakers
6/2/2020 military against, george floyd, of helium, self management, biggest aid, helium in, emily watson, helium is, the helium, peaceful protesters
6/3/2020 george floyd, defense secretary, offer millions, julia child, of george, trump's proposal, told britain, mario centeno, insurrection act, u.s. defense
6/4/2020 james mattis, implicit bias, of babylon, george floyd, chili bowl, global vaccine, children against, secretary james, vaccine summit, tiananmen square
6/5/2020 george floyd, nuclear submarines, frank pentridge, of george, 34000 brazilians, in minneapolis, 10th consecutive, chokeholds following, using chokeholds, vienna philharmonic
6/6/2020 george floyd, discuss extending, dan deacon, mountain railways, record cuts, national football, public lands, roger goodell, samuel was, vienna philharmonic
6/7/2020 george floyd, police killing, of george, incendiary statements, deborah levy, dan deacon, zuckerberg say, 100 scientists, extending cuts, of babylon
6/8/2020 george floyd, wonder woman, prince andrew, speed record, active coronavirus, in minneapolis, minneapolis city, to dismantle, world speed, dismantle its
6/9/2020 tooth brush, george floyd, traffic signs, sales agent, hotline between, zone's 2, ground penetrating, penetrating radar, leaflets being, a toothbrush
6/10/2020 george floyd, frances berry, deborah levy, mary frances, native houston, swedish prime, graphic designer, of palma, hot milk, floyd the
6/11/2020 of arc, jefferson davis, joan of, of jefferson, the confederacy, the confederate, mark milley, general milley, adidas has, in richmond
6/12/2020 hello kitty, george floyd, policy proved, term threat, 3 strikes, labeling millions, police bias, requiring hundreds, brutality triggered, strikes law
6/13/2020 club penguin, penguin online, dalai lama, george floyd, 3 strikes, right activists, the dalai, wholesale market, racist thuggery, 400000000 people
6/14/2020 of arc, joan of, railway carriages, tooth brush, 500 railway, including far, of atlanta, michelle dams, than $250000000.00, cash had
6/15/2020 maria ressa, paul whelan, of libel, u.s. marine, journalist maria, many atoms, mr tenet, state secrets, of arc, russian court
6/16/2020 the heel, high heels, high heel, wearing heels, joint liaison, liaison office, indian army, disputed border, passed 8000000, harm children
6/17/2020 pyongyang blew, into border, joint liaison, soldiers back, send soldiers, liaison office, 1000 flights, online gambling, indian soldiers, border areas
6/18/2020 vera lynn, john bolton, foreign funding, win reelection, highly classified, restricting foreign, illegally as, mr brooks, technology provided, lynn whose
6/19/2020 red triangle, cyber attack, with espionage, sophisticated state, brazil team, symbol used, state based, 150000 new, based actor, brown eyed
6/20/2020 jeffrey berman, mr berman, in tulsa, tulsa oklahoma, southern district, u.s. attorney, attorney for, will override, minister obadiah, brown eyed
6/21/2020 mr berman, multiple stabbing, jeffrey berman, shadow factory, might intervene, stabbing in, mutinous soldiers, mogadishu have, intervene militarily, in tulsa
6/22/2020 sporting goods, prehistoric monument, the icc, new trees, holiday centers, start treaty, grigor dimitrov, nuclear arms, 20 massive, went vegan
6/23/2020 peter navarro, wildlife before, miranda july, triplets in, anyone known, trade advisor, advisor peter, ordered president, january remains, the fork
6/24/2020 the segway, demolish statues, controversial leaders, supermarket customers, prevailing situation, suspended unspecified, unspecified plans, key referendum, denounces israel's, a noose
6/25/2020 150 pilots, and connecticut, cardiovascular events, korean soldiers, the silk, wing campaigner, including asylum, grounded nearly, with aura, constitutional reforms
6/26/2020 seizing rockets, iraqi commandos, baghdad arresting, 2023 women's, falsifying their, flying credentials, their flying, 19 diary, maria ressa, blood types
6/27/2020 these pills, order protecting, arts fighters, now label, these pages, the silk, fair trade, fermenting violence, unfiltered statements, hear unfiltered
6/28/2020 shouting white, a rerun, the spitfire, white power, liberal center, supporters shouting, paddle out, of warsaw, mexico olympics, open beyond
6/29/2020 blood types, blood group, blood groups, blood type, for post-traumatic, find reports, afghanistan credible, david beasley, david copperfield, confederate emblem
6/30/2020 the grandmothers, union's ambassador, abortion law, friendship bench, famous acrobatic, cancel dozens, restrictive abortion, wine glass, acrobatic circus, security law
7/1/2020 law imposed, security law, 370 people, the annexation, arrests under, new security, west bank, 16 more, about 370, prominent musician
7/2/2020 commission murders, of smell, jeffrey epstein, a jade, jade mine, encrypted messaging, move drugs, major criminals, with grooming, zealand's health
7/3/2020 20 saudi, arabian nationals, saudi journalist, unknown group, saudi consulate, nuclear fuel, main nuclear, journalist jamal, nationals for, jeffrey epstein
7/4/2020 independence day, mount rushmore, 15 weeks, $31.00 soldiers, colombian army, soldiers accused, barefoot running, perfectly preserved, adopting children, has diluted
7/5/2020 homeland movement, mr poroshenko, plank of, deaths appears, independence day, covered public, wildlife parks, about 690000, petro poroshenko, center left
7/6/2020 barefoot running, ennio morricone, justifying terrorism, composer ennio, of justifying, security law, park or, by despair, robotic scientist, repressive state
7/7/2020 public toilets, johnny depp, vigorously implement, ghost map, amber heard, a toilet, home restrictions, star johnny, classroom teaching, provided classroom
7/8/2020 the panthers, capital belgrade, serbian capital, harvey milk, security office, airborne transmission, weekend curfew, officially passed, web science, passed 3000000
7/9/2020 financial records, the tango, tax returns, prosecutors can, blank among, his tax, of tango, guarantee confidentiality, everywhere coffins, coffins everywhere
7/10/2020 tax returns, nuclear confrontation, top broadcaster, his tax, 2nd south, people's action, istanbul back, financial records, stop broadcasting, action party
7/11/2020 roger stone, bosnian serb, jack charlton, the bosnian, serb forces, people's action, african classical, mr stone, mark 25, of srebrenica
7/12/2020 african classical, robert muller, robert mueller, conservative incumbent, christian culture, his investigation, roger stone, of warsaw, big wing, offered further
7/13/2020 your dna, the earthworms, washington redskins, jane goodall, the soil, the tango, the chimpanzees, financial fair, remains public, of earthworms
7/14/2020 night watch, the painting, islamic police, in timbuktu, mali nearly, its 5, 3 words, norah jones, provocation which, painting we
7/15/2020 jerry brown, security law, general court, spring watch, new security, reach only, 9000000000 by, people fewer, neighboring armenia, only 9000000000
7/16/2020 fuel oil, siberian arctic, arctic provides, fake tweets, heavy fuel, year's unprecedented, by manmade, tainted fuel, provides unequivocal, unprecedented heat
7/17/2020 haile selassie, jack charlton, coronavirus recovery, pleaded with, as forceful, forceful as, worst flooding, fairfield house, new morphic, top infectious
7/18/2020 john lewis, biggest powers, blood blood, russia's far, civilian roles, campaigner john, recovery package, 1963 along, desperate debt, rights campaigner
7/19/2020 haile selassie, fairfield house, his majesty, coronavirus recovery, of haile, bus designed, selassie was, blood blood, a saint, rust the
7/20/2020 coronavirus recovery, extradition treaty, adam goodes, treaty with, quarantine program, 750000000000 euro, security departments, when arresting, kenya airways, john myatt
7/21/2020 suitable boy, the convent, russian interference, coronavirus recovery, governments failed, security committee, bending arc, brief congress, recovery package, spread disinformation
7/22/2020 its consulate, ordered china, endangered birds, coronavirus briefing, consulate in, farms across, red hair, animal rights, briefing for, the bracket
7/23/2020 comfort food, comfort foods, blue spaces, deficit since, ted yoho, federal officers, pre-term birth, visa fraud, budget deficit, federal agents
7/24/2020 of chengdu, consulate in, tax havens, 1st friday, chinese consulate, hungary's leading, leading independent, friday prayers, goldman sachs, muslim friday
7/25/2020 repossessed the, have repossessed, peter green, preventing violence, istanbul convention, executive orders, 16 far, right politicians, prescription drugs, escalating diplomatic
7/26/2020 southern texas, corpus christi, volunteer warden, hurricane hanna, of corpus, nation tour, 240 hours, admitted starting, 5 nation, require travelers
7/27/2020 string theory, humanity over, infiltrate israel, take working, working holidays, investigate president, ever declared, westboro baptist, followings on, big followings
7/28/2020 string theory, fusion reactor, najib razak, minister najib, nuclear fusion, razak has, mr najib, brian greene, virtual choir, sing gently
7/29/2020 sonic hedgehog, annual muslim, muslim pilgrimage, with drastically, the raft, contender joe, presidential contender, federal agents, the cyclops, reduced numbers
7/30/2020 play play, operation gideon, jordan good, perseverance rover, 12 prominent, the perseverance, and fraudulent, john lewis, latest mission, red planet
7/31/2020 pandemic 6, nicolas anelka, plate tectonics, capital harare, crooked election, alan parker, lou mcdonald, emergency committee, obesity is, election hours
8/1/2020 james murdoch, murdoch has, news corp, lisa john, as saturday, allegedly obstructing, editorial content, rupert murdoch, mogul rupert, obstructing efforts
8/2/2020 x capsule, space x, commercial manned, splash down, space station, international space, mexico later, string theory, row india, 1st commercial
8/3/2020 john hume, new bridge, 4 babies, babies whose, the cinderella, fairy godmother, of cinderella, space x, leon fleisher, folk tales
8/4/2020 suitable boy, human genome, ignore orders, juan carlos, former king, genome project, is receding, major creditors, carlos has, lung transplant
8/5/2020 ammonium nitrate, of ammonium, secretive system, handcuffed to, in beirut, rare footage, inside china's, china's secretive, port area, gapper has
8/6/2020 the nra, roman empire, rifle association, national rifle, powerful national, in beirut, with fraud, 1st atomic, hiroshima to, almost immune
8/7/2020 in beirut, we chat, the nra, ice shelf, placing sanctions, u.s. transactions, lankan politics, india express, transactions with, the amazon
8/8/2020 executive orders, sign executive, take executive, india plane, executive action, female health, venezuelan court, host oprah, pearl harbor, in beirut
8/9/2020 good war, grand council, loya jirga, of elders, alexander lukashenko, towards eradicating, lanka's former, incumbent alexander, leader mahinda, human genome
8/10/2020 jimmy lie, apple daily, media tycoon, tycoon jimmy, security law, their taxes, prominent pro-democracy, pro-democracy activist, wealth managers, alexander lukashenko
8/11/2020 bandon dunes, at bandon, emma donoghue, david mclay, golf course, gemma arterton, mclay kid, produced strong, heart failure, apple daily
8/12/2020 running mate, virgin gorda, parliament ahead, ticket in, presidential ticket, california senator, ms harris, slump on, his running, unify americans
8/13/2020 private charter, ray bradbury, harris have, charter flights, sumner redstone, the palestinians, all private, emirates have, him rebuild, siberian woolly
8/14/2020 by mail, voter rolls, vote by, mail in, struggling u.s., center without, begun releasing, postal service, blocking funding, deputy interior
8/15/2020 deep remorse, postal service, past military, japan's surrender, of japan's, war criminals, the postal, expressed deep, allied nations, historical disputes
8/16/2020 high priest, catholic bishops, universal mail, and constantia, central bangkok, gee's bend, anti-government rally, dead relatives, that universal, chain reaction
8/17/2020 slime is, alexander lukashenko, the slime, transitional leader, death valley, upmarket hotel, hour siege, the beachside, ordered google, or assault
8/18/2020 rafik hariri, minister rafik, lebanon's former, mr hariri, michelle obama, of diamonds, hezbollah militant, b anthony, hariri the, russell crowe
8/19/2020 in mali, formally nominated, be spilled, of mali, spilled to, national convention, transitional government, putrid smell, virtual national, 2nd evening
8/20/2020 mr bannon, of radiotherapy, chief strategist, bannon has, steve bannon, strategist steve, the bachelor, food media, death penalty, food writing
8/21/2020 russian opposition, death valley, these glaciers, randy weaver, ruby ridge, russian doctors, delivered securely, air ambulance, popular leader, of cloaking
8/22/2020 ruby ridge, suspected poisoning, randy weaver, russian opposition, great pyramid, growing resurgence, 12 should, russian doctors, $25000000000.00 into, postal service
8/23/2020 divine protection, marianne trump, emmett till, postal service, marianne making, sister marianne, secret recordings, of divine, independence square, revealed of
8/24/2020 strike leaders, storm laura, great pyramid, rio tinto, margaret ekpo, katie hill, emergency authorization, convalescent plasma, jacob blake, australian white
8/25/2020 magnetic field, wild polio, cactus is, serb military, former bosnian, bosnian serb, national convention, in cactus, royalist marketplace, refugee resettlement
8/26/2020 crew change, change guide, jacob blake, emmett till, train doc, man jacob, republican national, of emmett, can osha, serb commander
8/27/2020 jacob blake, hurricane laura, storm surge, without parole, hurrican laura, reserve police, of louisiana, shooting of, dead 51, 51 muslim
8/28/2020 outspoken head, doping agency, the outspoken, financial violations, japan's longest, emirates killed, serving prime, drone operated, of presiding, climate review
8/29/2020 chadwick boseman, black panther, panther in, rescue boat, boseman the, artist banksy, immediate assistance, colon cancer, education university, british graffiti
8/30/2020 serbian orthodox, powerful serbian, face briefings, orthodox church, passed 25000000, in montenegro, resolve violent, colon cancer, of socialists, consecutive sunday
8/31/2020 sam harris, commercial flight, ever commercial, in montenegro, formal ties, local groups, between israel, prominent news, paraded in, kigali after
9/1/2020 afghan women, alan walker, data storage, nearly 1400000, $70000.00 children, controversial cartoons, alleged conspirators, storage devices, offices 5, sharing news
9/2/2020 black holes, the walkman, satirical magazine, rouge regime, french satirical, nerve agent, operation linked, flight simulator, removed fake, deadly terror
9/3/2020 of smallpox, smallpox and, foster farms, offered rewards, obama gate, in baluchistan, reducing misinformation, pipeline from, opposition campaigner, lucrative foreign
9/4/2020 q and, material belonging, to jacob, against moroccans, silvio berlusconi, minister silvio, jacob blake, highest respect, digital library, intelligence analysts
9/5/2020 entirely responsible, beyond major, by environmentalists, the paraguayan, passed 4000000, spreads beyond, indian counterpart, incurable medical, frenchman with, planned death
9/6/2020 released body, that legislative, delayed them, william jordan, lord frost, of seuss', bolshoi theater, famous bolshoi, york attorney, alexander lukashenko
9/7/2020 line judge, with albinism, withdrawal agreement, of consciousness, masked men, hard border, 49.4 degrees, poisoned russian, founder julian, jordanian army
9/8/2020 financial review, australian financial, diplomatic standoff, mike smith, national happiness, day diplomatic, her passport, opposition colleagues, beautiful short, short loser
9/9/2020 migrant camp, 13000 people, morea migrant, west antarctica, hold worldwide, the morea, mexican farmers, of terrorizing, bob woodward, stolen credit
9/10/2020 bob woodward, knowingly and, of wellness, her illegally, willingly lied, journalist bob, and willingly, destroying nature, coast destroying, deport her
9/11/2020 rio tinto, aboriginal sites, abrams said, giant rio, robert abrams, microsoft says, mining giant, ratify any, government negotiators, from shareholders
9/12/2020 1st formal, terence conran, afghan government, impeachment proceedings, incapacity deepening, tortured into, open impeachment, champion wrestler, matthew green, moral incapacity
9/13/2020 biosphere 2, united russia, the biosphere, russia party, month's coup, key test, state north, permanent migrant, father joe, local elections
9/14/2020 douglas ross, arm holdings, based chip, scottish conservative, tech monopoly, the broom, japan's governing, maker arm, softbank group, chief cabinet
9/15/2020 and bahrain, climate arsonist, irritable bowel, with israel, unmarked graves, diplomatic agreements, unveil legislation, spain's left, shot 8, ruled spain
9/16/2020 the iaea, suppress political, the flock, the atv, 7 ethnic, alabama florida, doping procedures, u.s. gulf, hurrican sally, after bahrain
9/17/2020 nitrogen crisis, new camp, online fraud, the wellness, susie was, located across, brazil paraguayan, storm sally, say traces, vaccine could
9/18/2020 red blood, prizes which, above 30000000, globe has, seems improbable, 940000 people, 30000000 according, america argentina, found apparently, 30000000 cases
9/19/2020 bader ginsburg, ruth bader, justice ruth, medical medium, selling sensitive, liberal justice, chinese spies, delay following, latest women's, british diplomat
9/20/2020 ruth bader, bader ginsburg, biosphere 2, slovenian cyclist, justice ginsburg, medical medium, nominee would, main administration, northern irish, celery juice
9/21/2020 galileo was, ruth bader, bader ginsburg, roman abramovich, shit's creek, space shuttle, activity reports, suspicious activity, katie hill, oligarch roman
9/22/2020 pilot whales, sea ice, dark web, general assembly, jinping condemned, of maple, bader ginsburg, plague upon, ruth bader, their contested
9/23/2020 space weather, pilot whales, whales have, whose applications, magnetic field, solar wind, frontline states, biggest beaching, relocate numbers, storm watch
9/24/2020 fisheries official, of louisville, patrol vessel, sure rhonda, in louisville, laundering laws, briana taylor, drug raid, louisville during, korean troops
9/25/2020 aggravated homicide, stillborn child, a stillborn, fisheries official, that loosens, loosens rules, the archibald, abortion which, homicide after, of aggravated

TECHNICAL DETAILS

Again, the query is identical to our single word TFIDF example, with the only change being the use of "gdelt-bq.gdeltv2.iaradio_2grams" instead of "gdelt-bq.gdeltv2.iaradio_1grams" as the source table!

WITH nested AS (
WITH data AS (
SELECT TIMESTAMP_TRUNC(TIMESTAMP, DAY) date, STATION, NGRAM word, sum(COUNT) count FROM `gdelt-bq.gdeltv2.iaradio_2grams` where DATE(TIMESTAMP) >= "2020-01-01" and station='BBC World Service' group by date,station, NGRAM
)
, word_day_type AS (
# how many times a word is mentioned in each "document"
SELECT word, SUM(count) counts, date, station
FROM data
GROUP BY 1, 3, 4
)
, day_type AS (
# total # of words in each "document"
SELECT SUM(count) counts, date, station
FROM data
GROUP BY 2, 3
)
, tf AS (
# TF for a word in a "document"
SELECT word, date, station, a.counts/b.counts tf
FROM word_day_type a
JOIN day_type b
USING(date, station)
)
, word_in_docs AS (
# how many "documents" have a word
SELECT word, COUNT(DISTINCT FORMAT('%s %s', FORMAT_TIMESTAMP( "%m/%d/%E4Y", date), station)) indocs
FROM word_day_type
GROUP BY 1
)
, total_docs AS (
# total # of docs
SELECT COUNT(DISTINCT FORMAT('%s %s', FORMAT_TIMESTAMP( "%m/%d/%E4Y", date), station)) total_docs
FROM data
)
, idf AS (
# IDF for a word
SELECT word, LOG(total_docs.total_docs/indocs) idf
FROM word_in_docs
CROSS JOIN total_docs
)
SELECT date, station,
ARRAY_AGG(STRUCT(ARRAY_TO_STRING(words, ', ') AS wordlist)) top_words
FROM (
SELECT date, station, ARRAY_AGG(word ORDER BY tfidf DESC LIMIT 10) words
FROM (
SELECT word, date, station, tf.tf * idf.idf tfidf
FROM tf
JOIN idf
USING(word)
)
GROUP BY date, station
)
GROUP BY date, station
ORDER BY date DESC
) select FORMAT_TIMESTAMP( "%m/%d/%E4Y", date) day, station, wordlist from nested, UNNEST(top_words) order by date asc, station asc