The GDELT Project

Agentic AI At Scale: Why News & The Real World Is So Different From Source Code

Kalev Leetaru

At a glitzy rooftop reception on the sidelines of Web Summit in Vancouver earlier this month I was giving an impromptu primer on the challenges of using current AI models to synthesize global-scale questions about current events. The chief AI advisor and strategist leading AI initiatives for one of the largest global investment firms joined the conversation and dismissed all of these challenges, arguing that since he routinely uses agentic AI systems to debug gigabytes of source code every day on his laptop, agentic AI can trivially synthesize petabytes of global news content without issue. Remarkably, despite offering many counter-examples of how debugging a line of source code in a single self-contained code repository is so different from surfacing and contextualizing the macro-scale themes and patterns of the entire planet spanning the entire combined output of global news coverage in hundreds of languages and millions of diverse voices and perspectives, he simply dismissed every point by returning to his laptop-based agentic code debugger and touting the "wow" factor of AI agents on a laptop. Even more remarkably, he argued that he could plug five 20TB USB drives into his laptop and use his same laptop agents to process 100TB of news in just a few minutes – after all, as he pointed out, if his source code agents could track down a bug in 100MB of source in a few minutes, why couldn't they do the same for 100TB in just a few more minutes?

While it should be trivially obvious why debugging a line of code differs so immensely from surfacing macro-level trends and patterns from global-scale reporting over the entire planet and why a laptop processing 100MB via RAG in a few minutes doesn't magically scale to non-RAG full-scale 100TB processing in just a few minutes more, this AI advisor's inability to see beyond his own miniature personal source code prototypes offers a stark reminder of why so many organizations struggle to deploy AI effectively: technologists often struggle to see beyond the "wow" of technology, few have the experience scaling applications from prototype to enterprise-level production, let alone hyperscale, and even fewer have the deep application background to appreciate and understand the nuances of AI deployment in a given field and how the messy complex real world isn't the same as simple rules-based source code.

Let's ask Gemini and ChatGPT to offer primers on why this technologist's laptop-based code debugger will struggle to scale to synthesizing the planetary-scale patterns of 100TB of news and help him better understand why the real world isn't the same as source code.

Let's start with this simple prompt:

A colleague argued that since his agentic AI on his laptop can debug errors in a gigabyte of source code, far more than fits into context window, the same approach would work for trying to synthesize the planetary-scale patterns, themes and trends from 100TB of global news content. That debugging a line of code is the same as reading all that news and finding the high-level patterns only visible when looking at the entirety of the news. He believes that agents can just read all of the individual articles like it does individual source files and that distilling dataset-scale patterns is the same as debugging a single line of code. Explain why source code debugging and planetary-scale news analysis are so diffferent and explain your reasoning in detail.

Your colleague is making a classic category error by conflating two fundamentally different types of computational problems: targeted local search and global emergent synthesis.

While it is true that agentic AI can navigate a 1GB codebase to fix a bug despite the code exceeding its context window, it does so using a completely different paradigm than what is required to synthesize planetary-scale trends from 100TB of unstructured text.

Here is a detailed breakdown of why your colleague’s analogy fails, and why debugging code is nothing like synthesizing massive global datasets.

1. Localized Targets vs. Distributed Emergence (The "Needle" vs. The "Haystack")

2. Deterministic Structure vs. Unstructured Semantics

3. The Context Window "Pinhole" Effect

Your colleague believes the agent can "just read all of the individual articles like it does individual source files." This misunderstands how context windows work.

4. The Scale of the Mathematics (1GB vs. 100TB)

Let’s look at the sheer physics of the data:

5. Verification vs. Open-Ended Discovery

Summary to tell your colleague:

Your colleague's agent is not comprehending 1GB of code; it is smartly ignoring 99.9% of it to fix a broken link.

Analyzing 100TB of global news for emergent trends requires comprehending 100% of the data simultaneously to map the latent relationships. One is a targeted retrieval task using logic graphs; the other is a massive unsupervised clustering task. Applying a single agent to 100TB of news is like giving a man a magnifying glass and asking him to accurately draw the global weather patterns of Earth by walking around and staring at the dirt.

Asked for more detail about the specifics of source-vs-news, here is Gemini's response:

Your colleague’s intuition is understandable because, on the surface, both source code and global news are just massive datasets of text. If an AI agent can intelligently navigate a gigabyte-sized GitHub repository by jumping between individual files, it seems logical that it could just “read” through massive databases of news articles the same way.

However, this argument fundamentally misunderstands how information is structured, verified, and connected in these two domains. Processing source code requires navigating a deterministic map; processing news requires resolving a chaotic, overlapping, and evolving human narrative.

Here is a detailed breakdown of why the agentic approach for source code fails when applied directly to news:

1. Deterministic Topology vs. "Semantic Soup"

2. The DRY Principle vs. Massive Redundancy

3. Objective Compilation vs. Subjective Epistemology

4. Version Control vs. Evolving Timelines

5. The "Lost in the Middle" Problem

When agents navigate code, they extract highly specific, dense snippets (e.g., a 20-line class definition) and drop them into their context window. This keeps the AI focused.

Because news articles are padded with background context, journalistic prose, and advertisements, passing dozens of articles into a context window quickly leads to context bloat. Large Language Models suffer from the "lost in the middle" phenomenon, where they forget or hallucinate details when fed too much verbose, unstructured text. Code is information-dense; news is information-sparse.

Summary: What News Actually Requires

Your colleague is treating news like an engineering problem, but it is actually an intelligence analysis problem.

To process vast amounts of news, an AI system cannot use a "jump to file and read" agentic loop. Instead, it requires a pipeline built for unstructured data:

    1. Deduplication/Clustering: Grouping the 5,000 identical articles into a single representative cluster.
    2. Timeline Extraction: Ordering facts chronologically to track how a story evolves.
    3. Source Weighing: Cross-referencing claims against source reliability.

You can tell your colleague: "Navigating source code is like reading the blueprints to a house. Navigating news is like interviewing a thousand different witnesses to a car crash. The tools you use to parse the former will completely fail to synthesize the latter."

And ChatGPT's take:

I’ll frame this as a systems/design argument: why code repositories are comparatively “agent-friendly,” why news corpora break those assumptions, and what a workable alternative looks like.

Your colleague’s analogy sounds plausible because both cases involve “too much text for the context window.” But the limiting factor is not just volume. It is the structure of the information, the shape of the reasoning task, and the cost of missing relationships across documents.

Agentic AI can sometimes handle gigabytes of source code because codebases have strong internal structure. News does not behave like that.

The core distinction

A large software repository is usually a coherent, bounded, self-referential system. A news corpus is an unbounded, redundant, conflicting, rapidly evolving stream of partial observations about the world.

That difference is everything.

In code, the agent can often decompose the task into files, functions, imports, call graphs, tests, types, dependency trees, and error traces. It can inspect only the parts likely to matter. In news, the “meaning” often does not reside inside any single article. It emerges from cross-article comparison: what changed, what is missing, who amplified which claim, which outlets framed the same event differently, which narrative became dominant, and what appeared across languages, regions, and time.

Reading individual articles one at a time is therefore not enough.

1. Code has explicit architecture; news has latent structure

Source code is full of machine-readable structure:

imports
function definitions
class definitions
type signatures
call graphs
config files
tests
build errors
package manifests
dependency trees

An agent can ask: “Where is this symbol defined?” “What calls this function?” “Which files import this module?” “Which test failed?” These are highly constrained questions.

News does not come with equivalent structure. A collection of articles does not naturally expose:

narrative arcs
topic drift
cross-outlet framing differences
claim propagation
omitted context
source dependency chains
agenda-setting patterns
regional asymmetries
temporal escalation

Those have to be inferred.

A news agent cannot simply ask, “Where is this narrative defined?” There is no canonical definition. The narrative is distributed across hundreds or thousands of articles, headlines, quotes, updates, chyrons, wire rewrites, commentary pieces, and social amplification.

So the code analogy fails because code has an explicit graph, while news requires discovering the graph.

2. Code files are functional parts of one system; articles are partial views of many systems

A source file usually belongs to one repository and contributes to a shared executable or application. Even if the codebase is messy, there is still a unifying artifact: the software.

News articles are not parts of one shared artifact. They are independent accounts produced by different organizations, with different incentives, sources, editorial standards, audiences, geographies, and deadlines.

For example, ten articles about the same diplomatic event may differ because:

One is a wire-service dispatch.
One is an official-government framing.
One is opposition-party framing.
One is a regional-localization of the story.
One is a market-impact angle.
One is historical background.
One is commentary.
One is a breaking-news stub later superseded.
One is a translation of another article.
One is a rewritten version of an earlier report with subtle changes.

An agent that “reads all the articles” individually may summarize each accurately and still fail to understand the event landscape.

The task is not article comprehension. The task is corpus-level inference.

3. In code, redundancy is often helpful; in news, redundancy is actively dangerous

In software, repeated patterns can help the agent. If a coding convention appears in 500 files, the agent can infer the convention. If similar utility functions appear across modules, it can identify reuse or duplication.

In news, redundancy can distort the model’s understanding.

A single wire story may be republished by hundreds of outlets. If the agent reads 500 articles and 350 are lightly rewritten versions of the same AP/Reuters/AFP/state-media/press-release origin, it may falsely conclude that a claim has broad independent corroboration.

But that is not 350 independent observations. It may be one observation echoed 350 times.

This is a major difference. In code, duplication often indicates a code smell or convention. In news, duplication may indicate syndication, amplification, copying, translation, propaganda distribution, or SEO rewriting.

The agent has to distinguish:

many independent sources agree
vs.
one source was copied many times
vs.
many aligned sources repeated the same official claim
vs.
many outlets reported the claim but hedged it differently

That cannot be solved by reading articles one by one.

4. Code has relatively stable ground truth; news often has contested or evolving ground truth

With source code, many questions have verifiable answers:

Does it compile?
Does the test pass?
Where is the function defined?
What does this endpoint return?
Which commit introduced the bug?

Even if the code is complex, the repository is a relatively stable object.

News is different. The facts may be incomplete, disputed, or changing while the corpus is being read.

An article at 9:00 AM may say “at least 5 people injured.”
An article at 11:00 AM may say “12 injured.”
An article at 2:00 PM may say “officials revised the number down to 8.”
A later investigation may say the early reports conflated two incidents.

If an agent reads all articles equally, it may blend outdated, corrected, speculative, and confirmed information into a false average.

For news, time is not just metadata. Time is part of the meaning.

A useful system must reason over:

publication time
event time
update time
correction time
source hierarchy
claim freshness
claim supersession

Code agents can often inspect the current state of the repo. News agents must reconstruct a temporal epistemology: who knew what, when, and how confidence changed.

5. Code decomposition preserves meaning better than news decomposition

A codebase can often be chunked by file, module, function, or dependency. Those chunks usually preserve local meaning. A function is a meaningful unit. A class is a meaningful unit. A failing test points to a bounded region.

News does not chunk as cleanly.

The meaningful unit may be:

a sentence-level claim
a quoted official
a missing counterparty response
a comparison between headlines
a subtle wording change across editions
a shift from “alleged” to “confirmed”
a recurring metaphor across broadcasts
a phrase that appears in multiple languages
a narrative frame repeated over weeks

An individual article is not always the right unit of analysis. Sometimes it is too large, because one article contains many claims. Sometimes it is too small, because the real story is the pattern across thousands of articles.

That is why “just read each article” is the wrong decomposition.

6. Code agents can use tools with deterministic semantics; news agents need interpretive judgment

A code agent can lean on deterministic tools:

grep
ripgrep
AST parsers
language servers
compilers
test suites
type checkers
dependency analyzers
linters
git blame

These tools give highly structured signals.

For news, the analogous tools are much weaker:

keyword search
embedding search
topic clustering
entity extraction
sentiment analysis
summarization
translation
deduplication
stance detection
claim extraction

These are probabilistic and often fragile. They can help, but they do not provide the same kind of hard constraint as a compiler or test suite.

A compiler can say, “This function call is invalid.”
A news analyzer usually cannot say, “This narrative interpretation is invalid.” It can only provide evidence, counterevidence, and confidence.

That means the agent needs a different architecture: not just retrieval and reading, but triangulation, contradiction detection, source lineage, and temporal claim tracking.

7. In code, the agent usually knows the objective; in news, the objective is often discovery

When working with source code, the agent is usually given a concrete task:

fix this bug
explain this function
migrate this API
find the security issue
add this feature
summarize this repo

The target is specific.

News analysis often asks a much more open-ended question:

What are the major narratives today?
What changed overnight?
What themes are emerging?
Which stories are getting disproportionate attention?
How do different countries frame the same event?
What is being omitted?
What claims are spreading?
What is novel relative to yesterday?

Those are discovery tasks. They require comparing the corpus against itself and against prior baselines.

An agent that reads articles serially may produce a pile of local summaries, but it may miss the global pattern.

This is the key point: news analysis is not mainly a summarization problem. It is a pattern-discovery problem.

8. The context window is not the only bottleneck; attention is the bottleneck

The argument “agentic AI can process gigabytes” usually hides the fact that the model is not reasoning over all gigabytes simultaneously. It is reading slices, storing notes, retrieving snippets, and iterating.

That can work when the task has a narrow search target.

But for news, the signal may be weakly distributed. No single article may look important alone. The important thing may be that a phrase, frame, or claim appears across 800 articles, with slight variations, and then disappears or mutates.

A serial reader can miss that because each article seems ordinary in isolation.

For example:

Article 1: mentions “strategic autonomy”
Article 2: mentions “independent defense posture”
Article 3: mentions “Europe must stand alone”
Article 4: mentions “less reliance on Washington”
Article 5: mentions “continental defense capacity”

Individually, these are mundane. Collectively, they may indicate a major narrative shift.

A code agent can often find the relevant file by following references. A news agent must often discover that the relevant “file” is a pattern that did not exist as an object before analysis.

9. News requires normalization across language, geography, genre, and medium

Code has many languages, but each language has formal syntax. JavaScript, Go, Python, Rust, and SQL differ, but their semantics are constrained.

News language is far messier.

The same event may appear as:

breaking-news article
wire copy
newspaper editorial
TV transcript
radio segment
official statement
press conference quote
social post
translated article
local-language commentary
caption/chyron
headline-only mention

And the same idea may be expressed differently across languages and political cultures.

So the system must normalize:

entities
aliases
places
dates
transliterations
translations
source types
program names
speaker roles
quoted vs paraphrased claims
official vs journalistic voice

A code agent reading a Python file does not have to ask whether “Washington,” “the White House,” “the Biden administration,” “U.S. officials,” and “an American source” refer to overlapping or distinct actors in that paragraph. News analysis does.

10. News has adversarial and strategic communication

Most code in a repository is not intentionally trying to manipulate the agent. It may be buggy or confusing, but it is usually not propaganda.

News ecosystems include actors trying to shape perception:

governments
campaigns
corporations
advocacy groups
state media
think tanks
anonymous officials
PR firms
lobbyists
coordinated networks

The agent must distinguish reporting from messaging.

A source-code agent can mostly treat files as evidence of the system’s behavior. A news agent cannot treat articles as neutral observations of reality. Articles are themselves part of the phenomenon being studied.

That creates a recursive problem: the corpus is not just describing the world; it is also participating in shaping the world.

11. Source code is mostly internally consistent; news corpora are full of contradictions

A codebase can have bugs, dead code, duplicate functions, and inconsistent style. But if it builds, there is at least some operational consistency.

News corpora routinely contain contradictions:

different casualty numbers
different perpetrator claims
different legal interpretations
different translations of the same quote
different dates for the same meeting
different spellings of the same person
different claims about causality
different levels of certainty

The right answer is often not “choose one article.” The right answer is:

Source A says X.
Source B says Y.
Later official data says Z.
Most outlets adopted X early, but shifted to Z after 14:00.
Regional outlets continued using Y.
State-aligned outlets emphasized a different causal frame.

That is a structured analytical synthesis, not an article-by-article readout.

12. “Reading all articles” does not solve salience

Even if an agent could read every article, it still has to decide what matters.

In code, salience is often guided by dependency structure, test failures, runtime paths, or user objectives. In news, salience is contextual:

volume of coverage
prominence of placement
change from baseline
cross-national spread
elite-source attention
novelty
policy relevance
human impact
market impact
narrative divergence
presence of misinformation

A story with 10 articles may be more important than a story with 10,000 articles if it represents a new development, a correction, or a weak signal of an emerging crisis.

So “read everything” does not answer:

What is important?
Important to whom?
Compared with what baseline?
Over what time horizon?
In which region?
According to which source class?

Without those answers, the agent produces exhaustive but shallow summaries.

13. Code benefits from hierarchical abstraction; news needs comparative abstraction

A code agent can build a hierarchy:

repository
package
module
file
class
function
line

That hierarchy is usually meaningful.

News needs a different structure:

event
claim
actor
source
quote
frame
narrative
region
language
time period
correction
amplification path

That is not given directly. It must be constructed.

So a news system should not merely summarize article A, article B, article C. It should extract and compare claims:

Claim: The talks collapsed because of sanctions.
Sources asserting it: X, Y, Z.
Sources disputing it: A, B.
Earliest appearance: 08:12 UTC.
Most amplified by: regional outlets aligned with actor X.
Counter-frame: talks collapsed because of security guarantees.
Confidence: contested.
Change since yesterday: sanctions frame increased sharply.

That is the kind of intermediate representation news needs.

14. The failure mode is “locally accurate, globally wrong”

This is the most important practical objection.

An agent can read every individual article and produce accurate summaries of each article, while still producing a bad corpus-level answer.

It may fail by:

overcounting duplicated wire stories
missing narrative shifts
flattening contradictions
blending old and new facts
treating official claims as facts
missing source lineage
ignoring regional asymmetry
missing what was omitted
failing to detect coordinated framing
confusing volume with importance
losing temporal sequence

That is different from code. In code, if the agent accurately understands each relevant function and dependency, it is much closer to understanding the system. In news, accurate local comprehension is only the first layer.

The better analogy

The better analogy is not:

news corpus = large code repository

It is more like:

news corpus = distributed sensor network + rumor network + archival record + persuasion system

Each article is a sensor reading, but the sensors are biased, duplicated, delayed, sometimes adversarial, and often describing other sensor readings.

The task is not to read every sensor report. The task is to infer the state of the world and the state of the information environment from noisy, overlapping, contested signals.

What would work better

A serious agentic news system needs multiple layers.

1. Deduplication and source lineage

Before reasoning, it should identify:

wire copies
near-duplicates
translations
rewrites
press-release derivatives
syndicated content
canonical originals

Otherwise volume becomes misleading.

2. Claim extraction

Instead of treating articles as atomic, extract claims:

who did what
when
where
according to whom
with what certainty
based on what evidence

3. Temporal modeling

Track how claims evolve:

first report
updates
corrections
retractions
confirmation
contradiction
continued stale repetition

4. Source-aware comparison

Separate:

official statements
independent reporting
wire-service reporting
commentary
state media
local reporting
expert analysis
anonymous sourcing

5. Narrative and frame detection

Look for how the same facts are packaged:

crisis frame
accountability frame
economic-cost frame
security frame
humanitarian frame
sovereignty frame
corruption frame
geopolitical frame

6. Cross-lingual and cross-regional alignment

Normalize references across languages and geographies, then compare what different audiences are being told.

7. Baseline comparison

Ask not just “what happened?” but:

what is new today?
what increased sharply?
what disappeared?
what changed wording?
what became more certain?
what became more contested?

8. Human-readable synthesis with evidence trails

The final output should explain:

the dominant themes
the supporting evidence
the competing interpretations
the source distribution
the temporal evolution
the uncertainty
the notable omissions

That is very different from an agent wandering through article after article.

A concise way to rebut the claim

You could phrase it this way:

Agentic AI can process large codebases because code is a bounded, internally structured system with explicit dependency graphs, stable semantics, and executable tests. News is an unbounded, duplicative, contradictory, time-evolving information ecosystem. The meaning of news is often not inside individual articles but in the relationships among articles: repetition, divergence, framing, omission, source lineage, and temporal change. Reading articles one at a time may produce accurate local summaries while still missing the corpus-level signal. News analysis requires claim-level extraction, deduplication, source modeling, temporal reasoning, cross-outlet comparison, and narrative discovery, not just serial document reading.

That is the heart of it. The problem is not merely that news is bigger than the context window. The problem is that news requires relational, temporal, adversarial, and comparative reasoning at corpus scale.