One of the most critical pieces of information that must be asked about any conflicting reporting is the sourcing of the information within. Did it come from a well-known journalist recording from the ground as the event occurs, is it a third party witness or actually observed the event, is it from satellite imagery or social media posts, is it from government or NGO officials or is it from the belligerents themselves? Typically mainstream conflict reporting blends several of these categories together, making it important to researchers and scholars to be able to distinguish where each asserted fact within the report originated. Here we demonstrate using Gemini 2.5 Pro to scan two news articles about Ukraine's "Spiderweb" operation in Russia, catalog the major factual claims within the article, determine their sourcing as stated in the article and offer an assessment of their potential trustworthiness.
Gemini performs well with the first article, from NBC, correctly extracting the key claims, attributing them and offering reasonable explanations about why each might be deemed credible or questionable. It also assesses that Ukrainian government statements are more trustworthy than Russian statements which raises the interesting question in the context of active conflict of whether this is due to a Trust & Safety nudge or whether Gemini has assessed this through its training data.
Unfortunately, for the article from DW, written by one of DW's own news correspondents, Gemini assesses DW to be "not a credible news source" and that the article is "a work of fiction, speculation, or deliberate disinformation presented in the format of a news report." Bizarrely, this is due to Gemini assessing that all events from December 2024 onward are "in the future" despite the specific model being tested here being Gemini 2.5 Pro Preview 06-05 which has a knowledge cutoff date of January 2025.
This duality of very strong results on one article and complete failure on the second is a reminder of the considerable risks and safety requirements needed to work with LLMs in the real world.
Let's start with this NBC article that mentions a range of sources including satellite imagery, assessments by NATO and other external experts and claims by both the Russian and Ukrainian governments.
We'll use the following prompt:
Below is a news article. You are an intelligence analyst attempting to assess the credibility of the source of the information presented in the article. Go through the article below and identify all sources of information, whether witnesses, satellite imagery, social media posts, people involved in the event, commentators, government officials, journalists, NGOs, etc. Make a bulleted list of every single piece of information or fact stated in the article, its source as listed in the article, whether it was reported by someone who witnessed or was involved in it, was from hard evidence like satellite imagery or social media posts or was opinion or a secondary or other source and rank its likely credibility and explain your reasoning.
ARTICLE TEXT:
[FULL TEXT HERE]
This yields the following:
Credibility Assessment of Information Sources
Below is a breakdown of each piece of information from the article, its source, an analysis of the source type, and a credibility rating.
Hard Evidence & Independent Analysis (Highest Credibility)
- Information: Satellite imagery from Planet Labs and Maxar shows at least 10 planes were destroyed across multiple airfields. Specifically, six at Belaya airfield (three Tu-95, three Tu-22) and evidence of at least four destroyed at Olenya air base (at least two Tu-95, one Tu-22).
- Source: NBC News analysis of satellite imagery from independent companies (Planet Labs, Maxar).
- Source Type and Context: Hard evidence. Satellite imagery is objective, observable data. Planet Labs and Maxar are reputable, independent commercial entities. The analysis is conducted by a major news organization, lending it further weight.
- Credibility Assessment: High
- Reasoning: This is the most reliable information in the article. It is based on verifiable, physical evidence from neutral third-party providers, not on claims from a party involved in the conflict.
- Information: A social media video, geolocated to Olenya air base, shows a large fire around a Tu-95 bomber.
- Source: Social media video, verified and geolocated by NBC News.
- Source Type and Context: Hard evidence (verified user-generated content). Geolocation by a professional news organization confirms the "where" and "what," corroborating the satellite imagery of damage at that location.
- Credibility Assessment: High
- Reasoning: While the original source is an unknown individual, its verification and geolocation by NBC News anchor it to a specific time and place, making it strong corroborating evidence for the damage claims.
- Information: Satellite imagery showed no visible signs of damage at two other air bases in the Ivanovo and Ryazan regions, despite Ukrainian claims of attacks there.
- Source: NBC News analysis of satellite imagery.
- Source Type and Context: Hard evidence (or lack thereof). This is an analysis of objective data that directly contradicts or fails to support a claim made by a combatant.
- Credibility Assessment: High
- Reasoning: The absence of evidence is a significant finding. This demonstrates that the satellite imagery is being used to check all claims, not just confirm them, which strengthens the overall credibility of the NBC News analysis.
Western Official & Expert Sources (High to Medium Credibility)
- Information: NATO's assessment is that "10 to 13 aircraft were completely destroyed" and around 40 were damaged.
- Source: A senior NATO official.
- Source Type and Context: Official, secondary source. The official is speaking on condition of anonymity, which slightly lowers direct accountability but is a standard practice for sensitive intelligence. The information was shared with journalists at a formal meeting of defense ministers, giving it official weight.
- Credibility Assessment: High
- Reasoning: NATO has its own extensive intelligence-gathering capabilities. This assessment aligns closely with the hard evidence from satellite imagery, providing strong corroboration. It serves as a credible benchmark against which to measure Russian and Ukrainian claims.
- Information: The NATO figure of 10-13 destroyed aircraft is the most accurate version of events based on publicly available satellite imagery and videos released by the SBU.
- Source: Oliver Alexander, a Danish open-source intelligence (OSINT) analyst.
- Source Type and Context: Expert opinion/secondary source. Alexander is an independent analyst whose profession is to interpret publicly available data. He is providing his expert assessment to a news organization.
- Credibility Assessment: High
- Reasoning: As an OSINT expert, his analysis is based on the same kind of evidence as NBC's. His conclusion independently corroborates the findings of both NBC News and the NATO official, creating a strong consensus.
- Information: The SBU-released footage and satellite images show 12 aircraft "completely destroyed" and another 10 "hit," but assessing damage on the latter is "hard or impossible" from imagery.
- Source: Oliver Alexander's analysis.
- Source Type and Context: Expert opinion/secondary source. He is providing a nuanced interpretation of the evidence, highlighting its limitations.
- Credibility Assessment: Medium-High
- Reasoning: This is a credible assessment, but the key is the caveat. He distinguishes between confirmed "destroyed" and unconfirmed "hit," which is a hallmark of rigorous analysis. The credibility is slightly tempered because it is still an interpretation of evidence provided by a combatant (SBU videos).
- Information: Russia no longer manufactures the Tu-95 and Tu-22 aircraft, making their loss permanent and degrading Russia's strategic capabilities.
- Source: Thomas Withington, an expert at the Royal United Services Institute (RUSI).
- Source Type and Context: Expert opinion/secondary source. RUSI is a highly respected defense and security think tank. This is an assessment of the strategic impact, not the event itself.
- Credibility Assessment: High
- Reasoning: This information is about industrial capacity and military doctrine, which is Withington's area of expertise. It is a factual statement about Russian military manufacturing and a credible assessment of the consequences.
Ukrainian Official Sources (Medium to Low Credibility)
- Information: 41 Russian aircraft were hit, with about half "completely destroyed" (approx. 20) and the rest rendered non-operational.
- Source: Zelenskyy aide Pavlo Palisa.
- Source Type and Context: Involved party/government official. This is a primary source in that the aide is part of the government that conducted the operation. However, the statement was made to journalists with a clear intent to shape a narrative of overwhelming success.
- Credibility Assessment: Low
- Reasoning: This claim is directly contradicted by more reliable evidence. The satellite imagery and NATO assessment place the number of destroyed aircraft at 10-13, roughly half of what is claimed here. The figure appears inflated for propaganda purposes.
- Information: The "Spiderweb" operation took 18 months to plan, was a "brilliant result," and will be in history books.
- Source: Ukrainian President Volodymyr Zelenskyy.
- Source Type and Context: Involved party/Head of State. This is political rhetoric and a statement of opinion.
- Credibility Assessment: Medium (for the planning claim); Low (for the qualitative assessment)
- Reasoning: The claim of 18 months of planning is plausible for such a complex operation but is unverified and self-aggrandizing. The "brilliant result" is subjective political messaging, not a verifiable fact.
- Information: The operation used 117 small drones stashed in trucks and unleashed near the perimeters of four Russian air bases.
- Source: The SBU (Ukraine's intelligence service).
- Source Type and Context: Involved party/intelligence agency. These are operational details claimed by the perpetrator.
- Credibility Assessment: Medium
- Reasoning: These details are specific and plausible. However, the source is a primary combatant with a vested interest in portraying its methods as sophisticated and effective. Without independent verification, these details must be treated with caution.
- Information: 34% of Russia's nuclear-capable strategic bombing force was taken out of action at a cost of $7 billion to Russia.
- Source: The SBU and Zelenskyy.
- Source Type and Context: Involved party/government officials. This is a high-impact claim designed for maximum propaganda effect.
- Credibility Assessment: Low
- Reasoning: The percentage and monetary value are entirely dependent on the initial (and likely inflated) claim of ~20 destroyed and ~20 damaged aircraft. Since the base number is not supported by evidence, these derivative calculations are highly unreliable.
- Information: The SBU claimed that NATO confirmed its tally of destroyed aircraft.
- Source: The SBU, when contacted by NBC News.
- Source Type and Context: Involved party/intelligence agency.
- Credibility Assessment: Very Low
- Reasoning: This is a demonstrably false claim within the article itself. The NATO official cited gave a number (10-13) that is significantly lower than Ukraine's claim (~20). This is a clear attempt at disinformation to lend false credibility to their inflated numbers.
Russian Official & Pro-Kremlin Sources (Low Credibility)
- Information: None of its planes were destroyed.
- Source: "Russian claims" (as stated in the intro).
- Source Type and Context: Involved party/government official. A blanket denial from a combatant.
- Credibility Assessment: Very Low
- Reasoning: This claim is "demonstrably false" according to the article and is directly refuted by the highest-credibility evidence available: satellite imagery showing multiple destroyed aircraft.
- Information: The planes were "not destroyed, but damaged" and "will be restored." Ukraine's figure of 41 is "nothing even close."
- Source: Deputy Defense Minister Sergei Ryabkov via TASS state-run news agency.
- Source Type and Context: Involved party/government official speaking through state-controlled media. This is an attempt to downplay the severity of the attack.
- Credibility Assessment: Low
- Reasoning: While acknowledging some damage is a slight step up from a total denial, the claim that no planes were "destroyed" is contradicted by visual evidence of incinerated aircraft. The statement is designed to minimize strategic loss and project an image of control.
- Information: The attack was repelled, and while some aircraft caught fire, the fires were extinguished.
- Source: The Russian Ministry of Defense.
- Source Type and Context: Involved party/government ministry. Official statement meant to control the domestic narrative.
- Credibility Assessment: Low
- Reasoning: Calling the attack "repelled" is misleading when significant damage was clearly inflicted. The statement is classic propaganda, minimizing success and framing the event as a failure for the attacker.
- Information: Eight aircraft were lost, only half of which were bombers, and the strike "will have no military significance."
- Source: Pro-Kremlin commentator Sergei Markov on Telegram.
- Source Type and Context: Commentator/Propagandist. This source is not an official but is aligned with the government's narrative.
- Credibility Assessment: Very Low
- Reasoning: The source provides a number without any evidence or methodology. The claim of "no military significance" is an opinion that is directly contradicted by the expert assessment from RUSI about the strategic value of these specific bombers. His rhetorical question about Ukraine "lying all the time" is a propaganda technique, not an argument.
For good measure, let's try a DW article that emphasizes more of a historical context for the operation, rather than focusing on specific concrete details. We use the same prompt as above and get the following. Unfortunately, as seen below, despite this article being written by one of DW's own news correspondents, Gemini assesses DW to be "not a credible news source" and that the article is "a work of fiction, speculation, or deliberate disinformation presented in the format of a news report." Bizarrely, this is due to Gemini assessing that all events from December 2024 onward are "in the future" despite the specific model being tested here being Gemini 2.5 Pro Preview 06-05 which has a knowledge cutoff date of January 2025.
Intelligence Assessment of Source Credibility
OVERALL ASSESSMENT
The provided article is not a credible news source. It is a work of fiction, speculation, or deliberate disinformation presented in the format of a news report. The primary and most critical flaw is the inclusion of numerous specific dates in the future (e.g., June 1, 2025; December 2024; June 3, 2025) which are reported as historical facts. This indicates the article is fabricated.
The author blends these fabricated events with real, publicly documented events from 2022, 2023, and 2024. This technique of mixing fact with fiction is a common method used in disinformation to make the fabricated elements seem more plausible. While individual sources cited for real events (like Reuters or specific officials) may be credible in their original context, their use within this fabricated article serves to lend false authority to the overall narrative.
The analysis below will proceed as requested, breaking down each claim and its stated source. However, any claim related to a future-dated event must be considered to have zero credibility as it describes something that has not happened.
Detailed Analysis of Information and Sources
General Sabotage and Railway Attacks
- Information: Explosions on railway tracks in Belgorod and Voronezh derailed trains.
- Source in Article: Unattributed narration.
- Source Type: Secondary source (the article's author).
- Credibility: Medium. Such events have been reported in the past. However, within this flawed article, even claims of real event types must be treated with caution.
- Reasoning: The claim is plausible and consistent with known events, but the overall unreliability of the document requires independent verification for any specific incident.
- Information: No one was injured in the Belgorod/Voronezh derailments.
- Source in Article: "Official information" (presumably Russian authorities).
- Source Type: Government claim (secondary reporting).
- Credibility: Low to Medium. Governments often downplay the severity of successful attacks to maintain public morale and project an image of control. The claim is plausible but could be an understatement.
- Reasoning: Official statements from a party involved in a conflict are primary sources but are subject to propaganda and information control.
- Information: Russian authorities are investigating the derailments as suspected terrorism.
- Source in Article: Unattributed narration, implied to be from Russian authorities.
- Source Type: Government action/claim.
- Credibility: High. It is standard procedure for Russian authorities to launch terrorism investigations for such acts. The credibility is high that an investigation was started, not that terrorism is the correct conclusion.
- Reasoning: This is a procedural statement that aligns with known Russian state responses.
- Information: On Nov 30, 2023, SBU agents blew up a train in a tunnel on the Baikal-Amur Mainline.
- Source in Article: Attributed to "SBU agents."
- Source Type: Involved party (claim reported by the author).
- Credibility: High (that this event occurred and was claimed by Ukraine).
- Reasoning: This was a real, widely reported event. Ukrainian sources, including media citing intelligence officials, claimed responsibility at the time. The article is correctly reporting a historical event.
- Information: On June 1, 2025, railroad bridges collapsed in Bryansk and Kursk, killing seven passengers.
- Source in Article: Unattributed narration.
- Source Type: Fabricated Event.
- Credibility: None.
- Reasoning: The date is in the future. The event is a fabrication.
- Information: Russian authorities accused Ukrainian secret services of the June 1, 2025 attack.
- Source in Article: "The Russian authorities."
- Source Type: Fabricated Government Claim.
- Credibility: None.
- Reasoning: This is a fabricated claim about a fabricated event.
Attacks on the Crimean Bridge
- Information: On Oct 8, 2022, a truck exploded on the bridge, causing parts to collapse and a train to catch fire.
- Source in Article: Unattributed narration.
- Source Type: Secondary source reporting on a historical event.
- Credibility: Very High.
- Reasoning: This is a well-documented historical event confirmed by satellite imagery, video evidence, and statements from both Russia and Ukraine.
- Information: SBU chief Vasyl Malyuk's quote on the six-month planning and transport of explosives.
- Source in Article: "SBU chief Vasyl Malyuk."
- Source Type: Involved party (primary source claim).
- Credibility: Medium.
- Reasoning: As the head of the SBU, Malyuk is in a position to know. However, such statements also serve as propaganda to boast of his agency's capabilities. The specific details (fake companies, transit countries) are claims that are difficult to independently verify.
- Information: In December 2024, the SBU attacked a ship with two drones, injuring 15 crew members.
- Source in Article: Attributed to "Russian sources" for the casualties and "SBU" for the attack.
- Source Type: Fabricated Event.
- Credibility: None.
- Reasoning: The date is in the future.
- Information: On June 3, 2025, the SBU used underwater mines on the bridge pillars.
- Source in Article: "Kyiv reported."
- Source Type: Fabricated Government Claim.
- Credibility: None.
- Reasoning: The date is in the future.
- Information: Russian media denied reports of serious damage from the June 3, 2025 attack.
- Source in Article: "Russian media."
- Source Type: Fabricated Media Report.
- Credibility: None.
- Reasoning: A fabricated response to a fabricated event.
Targeting Airfields
- Information: FPV drones damaged a radar aircraft in Belarus; Kyiv initially denied but Vasyl Malyuk later admitted involvement.
- Source in Article: Unattributed narration and "Vasyl Malyuk."
- Source Type: Secondary source and Involved party (primary source claim).
- Credibility: Very High.
- Reasoning: This refers to the real attack on an A-50 aircraft at Machulishchy air base in February 2023. The subsequent admission by Malyuk is also a matter of public record.
- Information: In August 2024, the HUR attacked the Soltsi airfield, damaging a bomber and losing a lieutenant colonel.
- Source in Article: "According to the HUR."
- Source Type: Involved party (primary source claim).
- Credibility: Medium (for the details).
- Reasoning: An attack on Soltsi airfield did occur in August 2023 (not 2024), and a Tu-22M3 bomber was confirmed destroyed by photo evidence. HUR claiming the attack is credible. Their claim of specific personnel losses is plausible but self-reported and could be part of their own myth-making. The date is incorrect in the article.
- Information: On June 1, 2025, "Operation Spider Web" involved 117 drones attacking four airfields.
- Source in Article: Unattributed narration.
- Source Type: Fabricated Event.
- Credibility: None.
- Reasoning: The date is in the future.
- Information: 41 aircraft were destroyed in Operation Spider Web.
- Source in Article: "According to the SBU."
- Source Type: Fabricated Claim from an Involved Party.
- Credibility: None.
- Reasoning: A fabricated claim about a fabricated event.
- Information: Over 40 aircraft damaged, 10-13 destroyed.
- Source in Article: "According to NATO."
- Source Type: Fabricated Third-Party Claim.
- Credibility: None.
- Reasoning: Attributing a fabricated damage assessment to a real organization (NATO) to lend it false credibility.
Drone Attacks
- Information: In July 2023, SBU drones hit buildings in Moscow, causing panic.
- Source in Article: Unattributed narration.
- Source Type: Secondary source.
- Credibility: High.
- Reasoning: Drone attacks on Moscow in mid-2023 are well-documented historical events.
- Information: Air defenses were ineffective, and security was tightened.
- Source in Article: "According to Reuters."
- Source Type: Secondary source (reputable news agency).
- Credibility: High.
- Reasoning: Citing a credible, independent, and globally recognized news agency like Reuters for an assessment makes this specific claim highly credible.
- Information: In April 2024, a HUR drone hit an oil refinery in Nizhnekamsk, Tatarstan (1,200 km range).
- Source in Article: Attributed to "HUR."
- Source Type: Involved party (claim reported by the author).
- Credibility: Very High.
- Reasoning: This was a real, widely reported event. The location, target, and Ukrainian claims of responsibility are a matter of public record.
- Information: In June 2024, SBU drones attacked "Voronezh" radar stations in Orenburg (1,800 km range).
- Source in Article: Attributed to "SBU."
- Source Type: Involved party (claim reported by the author).
- Credibility: Very High.
- Reasoning: This attack on a Russian strategic early-warning radar also occurred and was widely reported.
High-Ranking Targets
- Information: The SBU killed Lt. Gen. Igor Kirillov in Moscow in December 2024.
- Source in Article: Attributed to "SBU."
- Source Type: Involved party (claim reported by the author).
- Credibility: High (that this event occurred and was claimed by SBU).
- Reasoning: This assassination did happen (though the date was Dec 17, 2024, which is in the future from a real-world perspective but treated as past in the article's timeline). Russian sources confirmed his death, and Ukrainian intelligence claimed responsibility.
- Information: Ukrainian agents killed Mikhail Shatsky, deputy head of the Mars Design Bureau.
- Source in Article: Unattributed narration.
- Source Type: Likely Fabricated Event.
- Credibility: None.
- Reasoning: There is no public record or reporting of an individual named Mikhail Shatsky from the Mars Design Bureau being assassinated. This appears to be another fictional event woven into the narrative.
- Information: The Shatsky operation was confirmed by Ukrainian military intelligence.
- Source in Article: "Ukrainian military intelligence."
- Source Type: Fabricated Government Claim.
- Credibility: None.
- Reasoning: A fabricated claim of responsibility for a likely fabricated assassination.