Datasets For The SBP-BRiMS 2017 Grand Interdisciplinary Data-Science Challenge On "Fake News"

 March 4, 2017

We are excited to be one of the datasets for the SBP-BRiMS 2017 Grand Interdisciplinary Data-Science Challenge focusing on "fake news" / propaganda. Participants in the Challenge may find the following GDELT resources of great interest. Note that new versions of the APIs below will be debuting in the next few weeks, adding immense new capabilities of particular relevance to the Challenge.

  •  GDELT Full Text Search API. This API allows you to conduct arbitrary keyword/phrase searches across the online news sources monitored by GDELT over the last 24 hours. Searches can be performed either natively in the 65 languages monitored by GDELT or in English, which will search the machine translations of all of that coverage, allowing you to identify how a story is propagating through the global media ecosystem. The GDELT Multilingual News Search website offers a simple news analytics site based on this API that lets you trace a given keyword and track how often it is being mention, the tone of that coverage, top keywords and a sampling of relevant coverage.
  • GDELT GeoJSON API. This API is currently limited to searching by GDELT Theme and a few other attributes like language or domain to create a clickable or animated map of the locations most closely associated with a given topic. The new 2.0 version of this API will be released in the next few weeks and will support mapping of any arbitrary keyword.
  • GDELT Television Explorer. This incredible collaboration with the Internet Archive's Television News Archive allows you to enter any arbitrary keyword or phrase and track how often it has been mentioned on any of the national television networks monitored by the Archive. In total more than two million hours of television news totaling 5.7 billion words from over 150 distinct stations spanning July 2009 to present are searchable using this system (though not all stations were monitored for the entire period). You can use this API to track how a given keyword has trended on US television news over the last few years and even compare the topic's prominence across stations. You can also read more about this collaboration on the Archive's blog.
  • GDELT Visual Knowledge Graph. This dataset is currently only available via either Google BigQuery or CSV files updated every minute and every 15 minutes. In the next few weeks the new 2.0 APIs will debut which make it possible to search visual news narratives through this dataset.

More advanced users may also find "Four Visualizations Exploring The Global Media Ecosystem" of significant interest in showcasing how these datasets can be used at very large scale to extract high-order understanding of the global news ecosystem.