Lessons Learned From Planetary Scale News Crawling

Running a global crawling and processing infrastructure that monitors news outlets in nearly every country in over 65 languages is an immense undertaking involving an incredible number of moving parts that teaches us a tremendous amount each day about the technical underpinnings of the global news landscape. Few open data projects operate at the scale […]

WashPost: All The President's Watergates

The Washington Post's Philip Bump uses the Television Explorer to examine how the major television networks have contextualized the events of the Trump Presidency in the context of his own "watergate." Read The Full Article.

Academics Continue Their Attacks On Facebook's New Privacy Rules

The academic world’s latest salvo in the war against Facebook’s new privacy efforts is a letter demanding the company exempt them from its rules against mass harvesting, fake accounts and fake posts, ironically asking the company roll back the very protections academia had long fought for. Read The Full Article.

Creating A Planetary Scale Open Dataset: Just How Big Is GDELT?

The GDELT Project encompasses an incredible array of datasets spanning the entire planet, reaching across 65 languages and making sense of modalities from text to images to video. This enormous realtime open firehose of data cataloging planet earth is available as downloadable files in Google Cloud Storage, JSON APIs for powering web interfaces, and for […]

Facebook As The Ultimate Government Surveillance Tool?

Facebook’s international reach, massive centralized data warehouse and algorithms that can divine the most sensitive and intimate elements of our lives are likely to increasingly become a go-to one-stop shop for the world’s intelligence agencies to spy, influence and destroy dissent. Read The Full Article.

Are Toilets The New Twitter? Using Smart City Data To Measure Interest

How water usage in Tokyo during the World Cup offers a lesson in the ways smart city data can be used to assess public interest to offer marketers rich new metrics, leaving the only question being how long it is before cities catch on and start selling all this data to the highest bidder? Read […]