Site icon The GDELT Project

Using Gemini To Flag Corrupted News Broadcast Videos Through Visual Analysis

One of the greatest challenges at the scale of the Television News is to identify videos with technically valid audio and video signals, but where the content is corrupted. For audio, we attempt to do this through ASR, but sometimes we have a video that has a valid transcript, but the video is corrupted, or the audio contains speech in a language poorly supported by the ASR model, so we are unable to obtain a comprehensive transcript. Similarly, OCR can tell us that a video contains valid onscreen text throughout, but many news broadcasts lack onscreen text and thus its absence is not suggestive of a corrupted video stream. A human can instantly identify video problems simply by glancing at the 1/4fps (one frame every 4 seconds) thumbnail grid that powers the Visual Explorer. Historically the vision reasoning capabilities of AI models were unable to reliably identify corrupted broadcasts, but given the major vision leaps over the last few months, can the latest Gemini 3.8 Flash both reliably identify corrupted video files and do so at a tractible cost? No data was used to train or tune any model and only the 1/4fps thumbnail grid was analyzed.

Amazingly, Gemini 3.8 Flash not only accurately flags whether each broadcast has corrupted video just from a thumbnail grid, but it can further clarify whether it contains usable video or whether it is just a static scene like a broadcaster logo or message, etc. At just a small fraction of a cent per broadcast, the results below suggest this could be scaled to the entire TV News Archive!

Here is a typical example of a broadcast with corrupted video, but clear audio. Can Gemini help us flag that the video signal is corrupted?

Does this video thumbnail look like it is from a corrupted video?
Just answer YES, MAYBE, NO.

Indeed, Gemini answers YES, costing just $0.00084 to analyze.

How about asking it to examine a 1/4fps (one frame every 4 seconds) thumbnail grid of the entire broadcast, in case just the start of the broadcast has issues? Amazingly, due to how Gemini processes images, this costs the same and also yields a "YES":

This is a thumbnail grid of a video. Does the video look corrupted? 
Just answer YES, MAYBE, NO.

How about this broadcast that is just the broadcaster's logo? Let's add some additional instructions to flag these kinds of broadcasts since they are not corrupted, but are also not terribly useful:

This is a thumbnail grid of a video.
Does the video look corrupted?
Just answer CORRUPTED: YES/MAYBE/NO
Then examine the thumbnails and ask whether this looks like valid programming or whether the MAJORITY of the broadcast is NOT valid programming: colorbars/logos/static image/blank/etc.
Just answer VALIDPROGRAMMING: YES/MAYBE/NO

Gemini answers correctly and costs $0.0020:

CORRUPTED: NO
VALIDPROGRAMMING: NO

How about this ordinary news broadcast?

This is a thumbnail grid of a video.
Does the video look corrupted?
Just answer CORRUPTED: YES/MAYBE/NO
Then examine the thumbnails and ask whether this looks like valid programming or whether the MAJORITY of the broadcast is NOT valid programming: colorbars/logos/static image/blank/etc.
Just answer VALIDPROGRAMMING: YES/MAYBE/NO

 

Gemini answers correctly and this time costs $0.0012:

CORRUPTED: NO
VALIDPROGRAMMING: YES

 

Exit mobile version