As geography becomes an ever more central modality for understanding the world around us, we've heard loud and clear from you that you want more ways of interacting with the immensely rich geographic tapestry that GDELT offers. Towards that end we're very excited to announce that the new GDELT 2.0 API suite will include advanced mapping capabilities, including both country and first order administrative division (ADM1) choropleth (thematic) map output.
You will now be able to export polygonal GeoJSON files that can be imported into any standard mapping system like Carto and come with all of the geometry built in, so you can just drag-and-drop to build instant and immensely powerful choropleth maps!
We're also rolling out an embeddable interactive mapping visualization that allows you to embed a live-updating choropleth or point map into your own site that updates every 15 minutes and is fully interactive, meaning you can zoom/pan, mouse over a location to see the density of results at that location or click on a location to see a popup with the top 5 matching articles from that location. Now you can literally embed a live-updating map of the world for any search for instant visualization or export the geography to GeoJSON for import to Carto and other platforms for advanced visualization, geostatistics and fine-grained control.
Our 1.0 GeoJSON API will still continue to be fully supported and work just like before with no changes. For those familiar with the 1.0 API, it is important to understand that structurally the new 2.0 API offers a much higher fidelity look at the geography of your search criteria. The 1.0 API simply performed a traditional search, identified the top few thousand results and then clustered those by location. If the API was set to return 1,000 results, but the first 999 all were about one city, then you'd only see two dots on your map, even if there were actually hundreds of unique cities associated with your search that just didn't make it into those top 1,000 results. In short, the 1.0 API operated at the level of a document and performed a document search and then clustered into locations.
The 2.0 API operates at the native resolution of a precise geographic coordinate and performs a location search, grouping documents per location. Thus, if you request 1,000 results, you'll get 1,000 distinct locations and the top 5 documents from each location. This ensures that the resulting map offers the highest possible fidelity representation of the geography of your search criteria.
Recall that GDELT is natively processing more than 100 languages, live translating 65 languages at 100% ingest capacity. This means that your maps will reflect national, regional, local and hyperlocal mainstream and citizen sources from every corner of the world and when you click on a location you will see an incredible range of different languages reflected. You can, of course, use the language controls in the new API to narrow your search only to English or other target language, but you often gain a very different understanding of a topic by seeing it through local perspectives throughout the world. For those who don't speak a hundred plus languages, our live-embeddable interactive maps are fully compatible with the Chrome browser's built-in Google Translate integration, meaning if you click on a location and see foreign language material, you can right-click on the popup and chose "Translate" from the Chrome context menu and Chrome will automatically translate all foreign language headlines on your map from that point forward, massively reducing language barriers.
Stay tuned for the release of these incredible new APIs over the next few weeks!
ACKNOWLEDGEMENTS
Global high-quality mobile-optimized geographic geometry is actually surprisingly difficult to find, especially at the ADM1 level. To build the maps above, we used data from the Natural Earth project, which incredibly makes its data available entirely public domain. At this time Natural Earth only makes its full First Order Administrative (ADM1) geometry available in its 1:10M high resolution Cultural Vectors compilation, which Nathaniel Kelso has helpfully converted to GeoJSON. The resulting geometry totals more than 65MB and is too high resolution for interactive rendering on many browsers we tested, so we reduced the coordinate resolution to four decimals and used the incredible MapShaper website to carefully simplify the geometry using Visvalingam weighted area simplication, breaking the world into multiple regions and simplifying each separately (taking into account the degree of geometric complexity in each area) and then reassembling into the final file. Additional hand tuning was performed in some areas to try and absolutely minimize the final filesize, resulting in a final GeoJSON geometry of just 5.7MB for the ADM1 file and 473K for the country file before gzipping. The basemap tiles are via Carto's Positron (Carto Light) tileserver. Data clustering is performed in-browser using Jenks ("natural breaks") clustering from the fantastic geostats JavaScript library. Customized scrolling within content popups is powered by the neat SimpleScrollbar library. Last but far from least, the entire mapping system is powered by the famous Leaflet library that is pretty much an instant "map in a box" library that lets you build rich interactive mobile-optimized maps with just a few lines of code.