The GDELT Project

HTTPS Now Available For Selected GDELT APIs And Services

Kalev Leetaru

As GDELT's APIs and visualization tools are increasingly being embedded on websites and integrated into online workflows, many of you have run up against the issue that HTTP-based content like GDELT's existing API and visualization endpoints cannot be embedded or loaded into HTTPS websites, resulting in complex integration workflows including local mirroring and proxy upgrading.

We are therefore very excited today to announce that all of GDELT's core APIs and most of its major active visualization services have now been upgraded to HTTPS. This has required considerable refactoring of some of our legacy services which also required upgrading some of our older visualizations to modern browser standards.

We recognize that there are still select legacy applications and analytic toolkits that struggle with HTTPS endpoints so we are leaving our existing HTTP endpoints in place and you can now access the upgraded APIs and services using either HTTP or HTTPS depending on your needs, though we recommend that users switch over to the HTTPS services as possible given that our future development path will focus on the HTTPS experience and some advanced browser capabilities relied upon by select API visualizations may be restricted by certain browsers in the future to HTTPS connections only.

NOTE that we have not yet upgraded the http://data.gdeltproject.org/ data CDN. Contact us if you have an absolute need to access the GDELT CSV files using HTTPS and we can walk you through current workarounds while we work on several upgrade paths. Given that data.gdeltproject.org is primarily used for our large daily/15 minute export files that are consumed as part of bulk analytic workflows, rather than website embedding, this shouldn't be too much of an issue for most users, but let us know if you have a specific urgent use case in the meantime.

Also note that http://analysis.gdeltproject.org/ has not yet been upgraded to HTTPS. We are currently in the process of rebuilding the analysis website from scratch, including upgrading it to process GDELT 2.0 and adding a host of new capabilities, tentatively scheduled for release late fall this year, at which time it will be upgraded to HTTPS.

The following APIs and services have been upgraded:

WEBSITE/BLOG

The main GDELT Project website and blog (the site you're on now) have both been fully upgraded to HTTPS. Since these are strictly human access sites and not intended for automated traffic, legacy HTTP traffic is depreciated and all requests are now automatically redirected to HTTPS.

GDELT 2.0 APIS

Both the DOC and GEO 2.0 APIs are available via HTTP or HTTPS. To avoid breaking legacy applications, no automatic redirect is performed and clients are able to select either protocol, but are strongly encouraged to upgrade to HTTPS, since future compatibility testing will focus on HTTPS user experience. When using HTTPS, all visualizations and outputs are fully embeddable via iframe into other HTTPS websites.

NOTE that visualization output modes that display news imagery will load as HTTPS, but will generate HTTP downgrade errors since many news websites do not yet support HTTPS and thus imagery from those articles will therefore load as HTTP. This will result in the green "Secure" notification in the browser bar disappearing and downgrade warnings appearing in the browser console. However, the visualization will still display properly, since the page itself is being served over HTTPS. As more news websites upgrade to HTTPS this issue should appear with less and less frequency.

GDELT 1.0 APIS

Similar to our 2.0 APIs, both GDELT 1.0 APIs are available as either HTTP or HTTPS and no automatic redirect is performed, though clients are encouraged to switch to HTTPS. Note that users of these 1.0 APIs should upgrade to the GEO 2.0 and DOC 2.0, respectively. These 1.0 endpoints will continue to function long-term, but we encourage all users to upgrade to the 2.0 APIs. The same caveats apply regarding HTTP downgrade warnings when using output modes that include news imagery, since many news websites still have not upgraded to HTTPS.

GDELT Dashboard 1.0 APIS

Similar to our other APIs, the GDELT Dashboard 1.0 APIs are available as either HTTP or HTTPS and no automatic redirect is performed, though clients are encouraged to switch to HTTPS.

Television Explorer

Similar to our APIs, the Television Explorer visualization is available as either HTTP or HTTPS and no automatic redirect is performed, though clients are encouraged to switch to HTTPS. All video thumbnail imagery is loaded from the Internet Archive's website over HTTPS, preserving the security of the session, while all outbound video links are also provided in HTTPS.

Television/Online Candidate Trackers

Similar to the Television Explorer, both the Television Candidate Tracker and Online Candidate Trackers, as well as the Television Candidate Emotions Tracker, are available as either HTTP or HTTPS and no automatic redirect is performed, though clients are encouraged to switch to HTTPS.

2016 Presidential Debate Viewers

During the 2016 US presidential election cycle we build visualizations of each of the major Republication and Democratic debates, tracking how "viral" each soundbite went on national and international television over the days following the debate. Similar to our other television services, these are available as either HTTP or HTTPS and no automatic redirect is performed, though clients are encouraged to switch to HTTPS. These visualizations have been upgraded to modern browser standards to eliminate certain browser errors.

Television Experimental Visualizations

Several of our earliest experimental visualizations (including our SOTU 2015 analysis, which was our first "television virality" tracker) and our first political ad viewers have been upgraded and refactored to modern browser standards and are available as either HTTP or HTTPS with no automatic redirection to HTTPS.