Continue Reading

ChatGPT Experiments: Summary And Q&A About The Protests In Peru

Following our analysis this morning of protests in France using ChatGPT, what might it tell us about protests in Peru?…

Continue Reading

ChatGPT Experiments: Summary And Q&A About The Protests In France

How might OpenAI's ChatGPT help analyze protest news coverage monitored by GDELT? To explore this further, we used GDELT's Web…

Continue Reading

ChatGPT Experiments: Summarization And Q&A About Iranian Television News Take Two (When Things Go Wrong & The Importance Of Prompt Engineering)

To explore the impact of prompt engineering, increased attention and brittleness on OpenAI's ChatGPT, we applied it to the same…

Continue Reading

ChatGPT Experiments: Summarization And Q&A About Iranian Television News

In similar fashion to our exploration of ChatGPT's ability to summarize and answer questions about Russian television news, how might…

Continue Reading

ChatGPT Experiments: Summarization And Q&A About Russian Television News

As part of our continued experiments with OpenAI's ChatGPT, what would it look like to ask it to summarize a…

Continue Reading

ChatGPT Experiments: Summarizing An ABC Evening News Broadcast

OpenAI's ChatGPT large language model has attracted considerable attention for its ability to generate human-fluency responses in an interactive Q&A…

Continue Reading

What OpenAI's Whisper Teaches Us About The Future Of Large Language Models Like ChatGPT

While specialty AI models like OpenAI's Whisper ASR speech recognition and translation target a more technical and niche audience, they…

Continue Reading

Why Gemini 1.5 Pro Describes President Biden As "Confused" & "Disoriented" And What It Tells Us About Training Data

One of the more fascinating findings from our experiments with Gemini 1.5 Pro's video capabilities is how strongly it associates…

Continue Reading

Early Findings With Gender, Racial & Cultural Bias In Generative AI Embeddings, Models & Guardrails

As we continue to explore how various generative AI offerings from different companies perform across global news coverage, one area…

Continue Reading

Gemini 1.5 Pro's 1 Million Token Model + Chirp: Fully Autonomous Diplomacy Through AI Watching Television

Earlier today we demonstrated the incredible ability of Google's new Gemini 1.5 Pro foundation model and its one million token…

Continue Reading

Generative AI Experiments: GenAI Coding Copilots: More Networking Code Troubles

As we continue to evaluate the capabilities of advanced Generative AI coding copilots, we find that they offer reasonable performance…

Continue Reading

Larger Generative AI Models Don't Equal Better Results: Disease Outbreak Codification With Bison, Unicorn, Gemini, GPT 3.5 & GPT 4.0

This past February we explored the use of LLMs for extracting and codifying disease outbreaks, finding potential but significant limitations…

Continue Reading

More Experiments In Automated Diplomacy: Countering Bad News About Ukraine Part 2

Yesterday we explored the concept of fully automated counter-messaging powered by LLMs, using ChatGPT to write an automated rebuttal to…

Continue Reading

More Experiments In Automated Diplomacy: Countering Bad News About Ukraine

Earlier this year we explored the concept of fully autonomous diplomacy: using LLMs to autonomously monitor media coverage of a…

Continue Reading

What Do The Major Generative Search Engines Say About Hamas Attacking Israel & Israel's Reaction?

Given that the general public is increasingly turning to LLMs and generative search engines like ChatGPT, Bing and Bard for…

Continue Reading

How LLM Guardrails Defeat RAG-Based Generative Search Applications For Global Events

Retrieval Augmented Generation (RAG) has become perhaps the dominate mechanism through which to expand and update the memories of LLMs….

Continue Reading

Summarizing Global Reaction To Gaza With DOC 2.0 API + LLM: Major News Story Headline Summarization

Earlier this year we explored how the DOC 2.0 API could be used to search for coverage of a specific…

Continue Reading

Gender & Race Bias In Multimodal Image Embedding Models: A Case Study Of OpenAI's CLIP & How White Men Are Doctors & CEOs Not Black Women

Over the past few months we've documented innate existential racial and gender bias in many of the textual embedding models…

Continue Reading

The Perils Of LLMs For Translation Tasks On Lower-Resource Languages: Estonian Noun Declension

One of the great promises of large language models (LLMs) is their ability to revolutionize translation and linguistic tasks. A…

Continue Reading

More Experiments With DOC 2.0 API + LLMs = Summarizing Headlines: Turkish Investment, Inflation & The Niger Coup

Last month we explored using ChatGPT (GPT-3.5) to summarize search results from the DOC 2.0 API for at-scale landscape summarization…

Continue Reading

More Experiments With LLM Translation: GCP's PaLM On Government Translation

With the General Availability of Chinese language support in Google's PaLM LLM, let's repeat our earlier tests of LLM-based translation…

Continue Reading

Experiments With Meta's SeamlessM4T Open Machine Translation Model: Social Media Posts

Continuing our series of evaluating Meta's new SeamlessM4T multimodal translation model. Let's try translating some Weibo social media posts that…

Continue Reading

DOC 2.0 API + LLMs = Summarizing Headlines: Turkish Investment, Inflation & The Niger Coup

The DOC 2.0 API allows rich keyword search over English machine translations of GDELT's online news monitoring in 65 languages….

Continue Reading

Understanding Hallucination In LLMs: A Brief Introduction

There is a lot of confusion and misunderstanding about what "hallucination" is in large language models (LLMs), how it can…