The GDELT Project

LLMs & An Ambiguous Photo Caption From The Ukrainian Front Lines: Do LLMs Actually Reason Or Merely Pattern Match Their Training Data?

Kalev Leetaru

As we continue to explore the ability of LLMs to summarize and distill the chaotic conflicting cacophony that is the firehose of daily global news coverage, an example from yesterday is instructive as to the challenges in applying LLMs to the often ambiguous language of news coverage. A Reuters image appearing in this CNN article yesterday was captioned "A Ukrainian serviceman, of the 10th separate mountain assault brigade of the Armed Forces of Ukraine, prepares to fire a mortar at their positions at a front line, amid Russia's attack on Ukraine, near the city of Bakhmut in Donetsk region, Ukraine". Towards whose positions is the mortar being fired? The caption references a singular "serviceman" of a brigade firing "at their positions". Grammatically, this sentence conveys one of two possible meanings: that the serviceman is firing a mortar at his own position in an act of friendly fire, or that the serviceman is based at the brigade's position on the front line and is preparing to fire at a target that is not described in the text. Intuitively, human readers will typically intuit that the latter is the most probable and that if the image conveyed an instance of the former, the unexpectedness of a friendly fire incident would lead to additional exposition in the passage describing the context of the incident and its aftermath. How do three different major commercial LLMs respond to this passage?

Here we see that all three LLMs struggle with this passage's ambiguity. When asked "at whose position is the mortar being fired", all three embrace the question's ambiguity, but when asked explicitly to name the target with "whose position is the mortar targeting" something fascinating emerges. When the original text, specifying a Ukrainian soldier, is used, the LLMs struggle, alternating between interpreting the target as Russia, an incident of friendly fire against Ukraine, or simply refusing to answer the question, with both guardrails and learned or enhanced values clearly visible. Looking closely at the results, the LLMs appear to be carefully weighing and reasoning about the ambiguity of the text. Yet, when the exact same sentence is used, but with "Ukraine" replaced with the fictional "Quantum", all three LLMs exhibit significantly greater propensity to interpret the passage as an incident of friendly fire, offering contradictory explanations and with one going so far as to twice hallucinate additional text to support its arguments. The singular importance of the identity of the soldier to all three LLMs' interpretation of the text shows the LLMs' responses depend less on the actual input text and more on the learned background knowledge of the LLM's internal knowledgestore. This is highly problematic for news analysis, as it means that as global events begin to deviate from an LLM's training data, it may struggle to accept and faithfully interpret the source text. Even more importantly, it explains the sharp divergence between the often human-like "glimmers of AGI" reasoning so frequently touted by the LLM research community and those models' real world performance on real world data: the AGI-like responses may have more to do with the random chance of learned facts being combined in ways that are anthropomorphized as advanced reasoning. In this case, when given the Ukraine example, the models lean on their training data to go as far as to impute that the enemy is Russia, even when the text is modified to remove that information, showing their training data contains extensive Ukraine-Russia conflict examples. When the LLMs are no longer able to rely on their training data and must instead interpret the text as it stands, their performance breaks down considerably. This suggests that the interpretative ability of LLMs to "understand" text may be more related to their learned statistical phrase sequences that encode historical global events than their ability to interpret text as it is actually presented.

Extending these findings, LLMs have been presented as universal "interpreters" that can extend far beyond their learned knowledgebases by processing any passage of text and "understanding" what it says. Instead, we find that they are far more dependent on the alignment of that input text with the facts encoded in their training data than has been previously understood and that as those knowledgebases age, LLMs may be increasingly unable to faithfully interpret text that deviates or disagrees with their original training data, meaning they are less able to reason than previously believed and more dependent on what amounts to pattern matching from their training corpi.

Prompt 1

For our first test, we'll use the prompt "At whose position is the mortar being fired in the following text "A Ukrainian serviceman, of the 10th separate mountain assault brigade of the Armed Forces of Ukraine, prepares to fire a mortar at their positions at a front line, amid Russia's attack on Ukraine, near the city of Bakhmut in Donetsk region, Ukraine"?

The first LLM alternates between interpreting the passage as referring to the location of the mortar being fired versus the target of the mortar, in which case it attempts a more clinical answer that the text is ambiguous. The second LLM consistently interprets the text as referring to the target of the mortar and returns Russia each time. The third LLM consistently adopts the ambiguous language of the original passage. In this case, the prompt was purposely designed to be similarly ambiguous to the passage, adopting its same "at whose position" which could be interpreted to refer to the local of the mortar or the location of the shell's target.

LLM 1

LLM 2

LLM 3

Prompt 2

Let's try a more concrete prompt that explicitly asks about the "target" of the shell and modifies the text to remove the reference to Russia ("Whose position is the mortar targeting in the following text "A Ukrainian serviceman, of the 10th separate mountain assault brigade of the Armed Forces of Ukraine, prepares to fire a mortar at their positions at a front line"?").

The first LLM refuses to answer twice, citing the ambiguity of the text, and responses once each with Russian lines and Ukrainian lines. The second LLM primarily responds with enemy lines, but also once responses with self-fire, though interestingly refuses to answer several times and in one case responses with a lengthy explanation that bizarrely detours into praise of the writing and touting its clarity and lack of ambiguity, suggesting it tripped either a guardrail or innate learned response. The multiple refusals to answer suggest this question ventured into a guardrailed area. The third LLM was far more concise and once reported Russian targeting and twice reported self-fire. Of especial interest below is that despite each of these prompts being run in its own session to avoid contamination, two of the LLMs consistently inferred that the enemy was Russia, despite that not being contained in the modified text.

LLM 1

LLM 2

LLM 3

Prompt 3

Some of the responses above appear to incorporate background knowledge and encoded value judgements about the specific Russia-Ukraine conflict, including hitting against embedded guardrails in LLM 2. This suggests that when the models return an accurate result, it may have less to do with the information conveyed in the sentence itself and more to do with their encoded background knowledge of Ukraine-Russia relations from either Crimea or post-invasion model tuning. What if we shift our prompt to a fictional fighting force to eliminate these factors and focus the LLM exclusively on the meaning of the specific passage? Here we'll use "At whose position is the mortar targeting in the following text "A Quantum serviceman, of the Armed Forces of Quantum, prepares to fire a mortar at their positions at a front line"?"

Here we can see a marked shift from the responses above simply by changing the name of the fighting force and leaving the rest of the sentence exactly as-is. The exact same prompt, with just the name of the fighting force changed, yields dramatically different results. Above, all three LLMs struggled to analyze the text, alternating in their responses or refusing outright to provide a response. Here, the results are far more uniform in interpreting the passage as reflecting a friendly fire incident. Note the fascinating hallucination in the third and fifth of LLM 3's responses where it hallucinated additional clarifying text to support its conclusion, suggesting that unmooring the text from events potentially encoded in the LLM's training data allowed it to more freely hallucinate supporting evidence. (Note that each request was issued in a fresh session to avoid cross-contamination). Note the contradictory explanations in several of LLM 2's responses as well.

LLM 1

LLM 2

LLM 3