GDELT creator Kalev Leetaru recently wrote an opinion piece for Wired on the state of sentiment mining, where it is today, its roots from the era of punch card computers, the limitations of current approaches, and where it is heading in the future.  The piece summarizes his work with the Syfy channel on reimagining the future of interactive "social" television and the complete rethinking of how sentiment analysis is done that was required for Syfy's flagship new reality show Opposite Worlds.

Given the critical importance of societal-scale latent "emotional" undercurrents to stability, this piece has considerable relevance to how we think about emotion, how it is expressed, and how it is measured, especially the 16 "deadly sins" of current sentiment analysis approaches.

Several quotes from the article have especial bearing on the vision and mindset behind the GDELT Project:

"These 16 insights reflect one of the deeper truths of big data that is largely absent from today’s breathless marketing hype: that the underlying algorithms that power big data analysis have largely been built by computer scientists emphasizing technological prowess over a deeper understanding of how complex human behavior really is."

"We brought with us no preconceived notions of how emotion “should” be expressed online, focusing instead on what we learned from pouring over how Twitter is actually being used today."

"By blending human and machine, the system that ultimately powered “Opposite Worlds” uncovered a deeper need within the sentiment analysis field to engage more closely with disciplinary scholars and to focus less on algorithms and more on outcomes. For big data to mature beyond marketing hype towards truly transformative solutions, it must “grow up” out of the computer science labs that gave birth to it and spend more time on understanding the domain-specific algorithms and data it is applied to than on the computing algorithms that operationalize them."

Read the full article on Wired.com