Welcome to the new official website of the Global Database of Events, Language, and Tone (GDELT) project, now at www.gdeltproject.org!
As GDELT has grown from a small prototype into a global platform used by projects around the world, and as a growing array of organizations are piloting it for real-life situational awareness, it has become increasingly clear that it was time for GDELT to “graduate” into more robust quarters. We’ve been hard at work on this transition since this past fall and we’re excited to welcome you to the final results!
To paraphrase Mark Twain, “reports of GDELT’s death are greatly exaggerated.” As we finish moving into our new quarters, we’re happy to report that not only is GDELT alive and well, but in fact will be expanding dramatically in 2014 and making some exciting new announcements shortly! GDELT represents the largest societal-scale “big data” initiative ever created to catalog our global world, bringing with it unprecedented technical, methodological, and institutional challenges. As those who have managed large projects are all too acutely aware, with projects of the size and scale of GDELT and the distributed teams of collaborators they rely on, it can be easy for wrong information to surface and with projects this size it can be hard sometimes to get everyone into the same room to correct it. Unfortunately in this case, instead of reaching out and double-checking everything through the appropriate channels where things could have been corrected quickly, a public discourse was created entirely devoid of the facts.
The bottom line is that GDELT is one of the very few event datasets in existence today that actually has all of the necessary permissions. The concerns that have recently been discussed were raised by two faculty members at the University of Illinois and were examined by a panel of faculty experts convened by the University of Illinois' Vice Chancellor for Research. That panel formally cleared GDELT on behalf of that office stating "the Panel finds that it was not able to conclude that GDELT is founded on misappropriated … data or software." With respect to concerns raised regarding the open source TABARI software that GDELT makes use of to create its CAMEO event records, the same panel explored concerns raised regarding its ownership and similarly found that "TABARI … has well known antecedents at another institution dating back to at least 2000 and therefore is not attributable to the [University of Illinois]". While this whole situation would have been easily avoided with just a little communication and avoided a lot of unnecessary angst, the silver lining is that it has demonstrated just how widely-used and important GDELT has really become over the past year and we are tremendously excited to work with all of you in 2014 to really explore the future of “big data” study of human society.
We’re currently in the process of standing up a new extremely high-speed data server that should solve the bandwidth problems many of you encountered with our previous site and will mean you won’t need to wait ten minutes to download each morning’s update anymore! In the meantime we’ve posted the past month’s daily updates (downloads will be slow for the moment due to the incredible load until the new server is online), and the complete historical backfile should be back online and available here shortly as soon as the high-speed server is up.
As icing on the cake, GDELT is on schedule to expand back to 1800 by this coming summer, providing the first large-scale multi-century dataset, and opening unprecedented opportunities for truly longitudinal studies of human society. A new prototype methodology will also be debuting later this spring for identifying events and developments of “high significance” to stability that should be of great interest to those of you creating watchboarding and forecasting systems.
The success of GDELT has been truly incredible to see and really stands testament to what’s possible when a myriad organizations and individuals come together from across the globe to work in a global open collaboration to build the world’s largest database of human society to power open research on how society works, to enable new kinds of watchboards and situational awareness, and to even take the first steps towards open forecasting systems. We’d love to have your help, so drop us a line and we’d love to talk!
So, lots coming soon and stay tuned!