Trend Detection Using Television News Ngrams: Background Changes

Trend and anomaly detection is most often associated with visually stark changes, such as vertical surges around a breaking event. A far more powerful application of such trend detection in the context of news analysis is the identification of subtle changes in longstanding background topics that are receiving "more" coverage than they used to, even if that additional coverage is not sufficiently large or sudden to leap out of the graph.

TheĀ Television News Ngram (TV-NGRAM) dataset is especially well-suited for background analysis given its 10-year longitudinal coverage.

The timeline below shows the density of mentions of "Ebola" in the TV-NGRAM unigram dataset.

While the 2014 outbreak is captured starkly clearly in the data, a far more interesting trend can be seen in the subtle increase in mentions of Ebola beginning in April 2018 and continuing through present. This is where trend analysis offers the greatest promise in the future: the detection of the subtle emerging narratives and stories that are getting "more" attention than before, but which haven't leapt to the forefront of attention.