The GDELT Project

Tracking The Similarity Of Word Usage On Television News Coverage 2009-2021

The timeline below tracks the similarity in word usage across CNN, MSNBC and Fox News by month (using a 3-month rolling average) over the past decade by computing a Pearson correlation of their respective monthly closed captioning histograms. CNN and MSNBC have always been far more similar to one another than to Fox News, but that similarity peaked in September 2016 and has remained fairly stable ever since. Fox News' similarity to the two used to be fairly similar, but since mid-2016 has diverged, with Fox more similar to CNN than to MSNBC. In all, their word usage is extremely similar.

TECHNICAL DETAILS

Computing the timeline above required just a single SQL query:

WITH sheet AS (
select DATE, NGRAM, SUM(CNN) CNN, SUM(MSNBC) MSNBC, SUM(FOXNEWS) FOXNEWS from (
(SELECT SUBSTR(CAST(DATE AS STRING),0,6) DATE, NGRAM, COUNT CNN, 0 MSNBC, 0 FOXNEWS FROM `gdelt-bq.gdeltv2.iatv_1gramsv2` WHERE STATION='CNN')
UNION ALL
(SELECT SUBSTR(CAST(DATE AS STRING),0,6) DATE, NGRAM, 0 CNN, COUNT MSNBC, 0 FOXNEWS FROM `gdelt-bq.gdeltv2.iatv_1gramsv2` WHERE STATION='MSNBC')
UNION ALL
(SELECT SUBSTR(CAST(DATE AS STRING),0,6) DATE, NGRAM, 0 CNN, 0 MSNBC, COUNT FOXNEWS FROM `gdelt-bq.gdeltv2.iatv_1gramsv2` WHERE STATION='FOXNEWS')
) GROUP BY NGRAM, DATE
)
SELECT a.DATE,
CORR(a.CNN, b.MSNBC) CNN_MSNBC,
CORR(a.CNN, b.FOXNEWS) CNN_FOXNEWS,
CORR(a.MSNBC, b.FOXNEWS) MSNBC_FOXNEWS
FROM sheet a
JOIN sheet b
ON a.NGRAM=b.NGRAM
WHERE a.DATE=b.DATE AND (a.CNN > 10 OR a.MSNBC > 10 OR a.FOXNEWS > 10)
GROUP BY a.DATE, b.DATE order by DATE