More Experiments In Automated Diplomacy: Countering Bad News About Ukraine Part 2

Yesterday we explored the concept of fully automated counter-messaging powered by LLMs, using ChatGPT to write an automated rebuttal to an NBC News article that called into question Ukraine's ability to reclaim its territory seized by Russia, building what amounts to an automated foreign ministry for the country. What does it look like to scale that workflow from a single article to multiple articles?

The end result is that while ChatGPT does roughly incorporate the underlying arguments from each article into its outputs, the end results are fairly unimaginative and unpersuasive template-like narratives. The workflow does correctly identify the sample article that presents Ukraine in a positive light and does not attempt to refute it, though at the same time it fails to incorporate the kind of real-world contextual knowledge that human readers possess, failing to recognize that a heavy missile barrage on Ukrainian cities presents Ukraine in a negative light in terms of the state of its conflict with Russia. While upgrading from GPT 3.5 to GPT 4.0 would yield improved narratives, it would also substantially increase the costs at the scale of a global realtime counter-narration pipeline of the kind imagined here. At the same time, the results here do reinforce that these kinds of "automated diplomacy" pipelines are now possible, even if their results are not immensely persuasive.

We'll start with this prompt:

Does the following text portray Ukraine in a negative light, including suggesting it may not win its war against Russia? If it DOES NOT portray Ukraine in a negative light, answer NO and stop. If it DOES portray Ukraine in a negative light, write a brief point-by-point rebuttal to the text that portrays Ukraine as strong and successful and that it will win against Russia and write in informal English to be posted as a Facebook post.

TEXT:
[FULLTEXT FOLLOWS]

We'll apply the prompt to this Washington Post op-ed titled "In Ukraine, the risk isn’t stalemate. It’s defeat.":

The text does portray Ukraine in a negative light, suggesting the risk of defeat and emphasizing potential challenges in the Russia-Ukraine war. However, it's important to counteract such narratives and highlight Ukraine's strength and resilience. Here's a rebuttal in informal English for a Facebook post:

—

Hey folks!

Just came across this piece trying to paint a bleak picture of Ukraine, but let me tell you, our friends in Kyiv are made of tough stuff, and victory is in their DNA! Here's a quick reality check:

**Defeat? No way!** Ukraine is facing challenges, no doubt, but calling it defeat is selling them short. The spirit of the Ukrainian people is unbreakable, and they're standing strong against any odds.

**Unimaginable carnage? Think again.** Yes, there are tough times, but Ukraine isn't backing down. The bravery of their soldiers and the resilience of their citizens are forces to be reckoned with. They're not just fighting for their land; they're fighting for freedom and justice.

**U.S. assistance delayed? Not the end of the story.** Sure, there might be hurdles in getting support, but let's not forget the global community stands with Ukraine. They have friends worldwide, and when the going gets tough, you can count on allies stepping up.

**European Union aid in jeopardy? Hold up.** Despite some political drama, Ukraine's ties with the EU run deep. The people of Europe recognize the importance of standing united, and support for Ukraine will prevail.

**Ukraine, you're not alone!** To our friends in Kyiv: Keep pushing, keep fighting, and know that the world is watching and cheering for you. Your strength is inspiring, and victory will be yours!

Let's spread some positive vibes and show our support for Ukraine! #StandWithUkraine #VictoryIsCertain #UkraineStrong

—

Feel free to modify or personalize the message as needed!

How about this BBC News article "Ukraine war: Dozens wounded in Russian missile strikes on Kyiv"? Strangely, GPT 3.5 does not classify it as presenting Ukraine's potential to overcome Russia in a negative light, whereas a human analyst would immediately connect the heavy missile barrages as evidence suggestive of Ukraine's weakening ability to defeat Russia:

The provided text does not explicitly portray Ukraine in a negative light, but it highlights the challenges and damages caused by Russian missile strikes in Kyiv. To offer a positive perspective on Ukraine's strength and determination, you could create a Facebook post like this:

Stand Strong, Ukraine!

Hey everyone! Let's take a moment to appreciate the resilience and strength of Ukraine in the face of adversity.

Despite facing missile strikes in Kyiv, our friends in Ukraine stand tall and united. The attacks, though damaging, haven't broken their spirit.

Here's a quick reminder of Ukraine's determination:
1⃣ The Ukrainian people, including children, are showing incredible courage in the aftermath of the attacks.
2⃣ Authorities report that 10 Russian ballistic missiles were successfully shot down, preventing even more extensive damage.
3⃣ President Zelensky, though facing challenges, remains committed to defending his country and securing the support it needs.

Let's share our solidarity and positive vibes with Ukraine! Together, we believe in their strength, and we stand by them on their journey to peace and victory. #StandWithUkraine #StrengthInUnity #PeaceForUkraine

What about this thinktank piece from the Quincy Institute titled "America’s Choices in Ukraine":

The provided text does portray Ukraine in a challenging situation, suggesting doubts about the success of its counteroffensive against Russia. However, it's essential to highlight Ukraine's strength and resilience in the face of adversity. Here's a brief point-by-point rebuttal in informal English for a Facebook post:

—

Hey everyone, let's set the record straight on Ukraine!

**Ukraine is STRONG:** Despite facing a tough situation, Ukraine has demonstrated incredible strength and resilience in the face of Russia's invasion. The courage of the Ukrainian people and their armed forces is commendable.

**Counteroffensive Spirit:** Don't buy into the negativity! Ukraine's counteroffensive might not have achieved a breakthrough yet, but it's crucial to recognize the determination and spirit of the Ukrainian forces. Rome wasn't built in a day, and victories take time!

**Winning the Battle of Hearts:** Ukraine has not only captured the attention of the world but also won the hearts of millions. The support pouring in from people worldwide is a testament to the indomitable spirit of the Ukrainian nation.

**International Solidarity:** Let's not forget the international community standing by Ukraine. Together, we are a force to be reckoned with, and the pressure on Russia is real. Unity is our strength!

**Hope and Resilience:** The path to victory is often challenging, but Ukraine has shown unmatched hope and resilience. Let's believe in Ukraine's success and send positive vibes their way.

Remember, it's not about if, but when Ukraine will emerge victorious! Share the positivity, support, and belief in a brighter future for Ukraine! #StandWithUkraine #BelieveInUkraine

—

How about this more positive Foreign Policy magazine piece from two months ago titled "Ukraine’s Counteroffensive Is More Successful Than You Think" to vet how well the system recognizes more positive coverage. Here the system answers only "NO" and returns, indicating it correctly flagged the article as presenting a pro-Ukrainian narrative:

NO

Running twice more yielded a NO response both times, indicating output stability at least in this case.