Tracking A Year Of Tucker Carlson On Russia 1's "60 Minutes"

 February 24, 2023

Last year, in collaboration with the Internet Archive's TV News Archive, we demonstrated applying face tracking to Russian television news to catalog appearances of Fox News' Tucker Carlson across several channels of Russian media ecosystem. Today we have expanded that analysis to tracking his appearances across Russia 1's "60 Minutes" show from March 2022 to present, capturing the ebbs and flows of its reliance on excerpts from Carlson's show to help tell its narratives. Using the Visual Explorer's "video ngrams" that extract one frame every 4 seconds from each broadcast and combining it with a face similarity scanner (instead of facial "recognition" it simply looks for faces that look extremely similar to a sample image so is less accurate but much faster), we scanned all 321 episodes of 60 Minutes captured in the Archive's holdings from May 19, 2022 through February 17, 2023 (due to technical issues there have been several monitoring outages, so some episodes are not captured in the archive) to catalog all appearances of Tucker Carlson's face. Note that the Archive's Russia 1 collection actually stretches back to March 26, 2022, but EPG data (which tells which show corresponds to which minute of airtime) was not available until May 19th so that is the earliest we can filter for specific shows.

In all, 714,188 total frames from 60 Minutes were scanned, representing 2.85 million seconds of airtime (796 hours), of which 801 frames representing 3,204 seconds (53.4 minutes) of airtime had a match for Tucker Carlson. Of the 321 broadcasts scanned, 89 of them (28%) contained at least one matching frame, with 179 distinct "clips" (a contiguous sequence of matching frames).

In short, more than a quarter of the broadcasts and as much as 0.13% of the total airtime of one of Russia's biggest television news series over the past year has consisted of clips of a single American news personality being used to advance Russia's narratives to its population.

You can see the final timeline below of the total seconds of airtime on 60 Minutes each day that featured Tucker Carlson's face. Note that this overcounts slightly since each each frame represents four seconds of airtime, but his face may have only been visible for part of those four seconds. In addition, the algorithm used here may have missed appearances of his face or incorrectly identified other faces as his. Thus, the results here are only an approximation, but do yield a number of useful findings. The first is that Carlson began appearing more regularly from late November 2022 and has been used more intensely as well, with more footage used each time.

You can see a 5fps time-lapse movie containing all 801 matching frames. Other than a few false positives, the overwhelming majority of the matches are indeed of Tucker Carlson and capture the wide range of stories from his show that Russian television has rebroadcast:

 

You can see the full list of matching dates below and the estimated seconds he appeared on 60 Minutes each day in the table below or download the complete list of all matching frames.

sec
5/19/2022 16
5/30/2022 4
6/3/2022 28
6/20/2022 204
6/24/2022 16
6/28/2022 4
7/11/2022 12
7/13/2022 20
7/14/2022 12
7/18/2022 32
7/22/2022 44
7/26/2022 4
7/27/2022 4
7/28/2022 4
7/29/2022 52
8/1/2022 4
8/9/2022 44
8/16/2022 4
8/17/2022 8
8/22/2022 4
8/23/2022 4
8/24/2022 4
9/5/2022 20
9/6/2022 8
9/16/2022 4
9/19/2022 4
9/20/2022 4
9/27/2022 112
9/28/2022 108
9/29/2022 96
11/1/2022 4
11/17/2022 48
11/25/2022 64
11/28/2022 56
11/30/2022 72
12/1/2022 56
12/6/2022 4
12/8/2022 168
12/12/2022 76
12/13/2022 96
12/15/2022 48
12/20/2022 76
12/22/2022 196
12/23/2022 72
12/26/2022 100
12/27/2022 156
12/28/2022 4
1/10/2023 44
1/11/2023 24
1/12/2023 212
1/13/2023 104
1/19/2023 56
1/20/2023 52
1/25/2023 60
1/27/2023 40
1/30/2023 40
2/2/2023 144
2/9/2023 144
2/15/2023 24
2/16/2023 64
2/17/2023 16

You can view each matching clip below to see it in sequence:

For the technically-minded, below is the step-by-step workflow to reproduce the analysis above. Note that for this analysis we used a 64-core N1 GCE VM with 200GB RAM, with 160GB carved into a RAM disk).

First install the two libraries we need ("facedetect" is the simple face detection and similarity matching tool we use to catalog his appearances and GNU parallel is used to run it in parallel across large numbers of processors).

apt-get -y install facedetect
apt-get -y install parallel

Now compile the complete inventory of all broadcasts monitored by the Internet Archive Russia 1 over the past year and extract just the episodes of 60 Minutes:

mkdir JSON
time cat DATES | parallel --eta 'wget -q https://storage.googleapis.com/data.gdeltproject.org/gdeltv3/iatv/visualexplorer/RUSSIA1.{}.inventory.json -P ./JSON/'
rm IDS.tmp; find ./JSON/ -depth -name '*.json' | parallel --eta 'cat {} | jq -r .shows[].id >> IDS.tmp'
grep '60_minut' IDS.tmp | sort > IDS

And download all of the ZIP files containing those images (NOTE: this requires at least 70GB of disk):

mkdir IMAGES
time cat IDS | parallel --eta 'wget -q https://storage.googleapis.com/data.gdeltproject.org/gdeltv3/iatv/visualexplorer/{}.zip -P ./IMAGES/'

And unzip all of the ZIP files (NOTE: this requires very high levels of random IO and file creation and is best performed in a large RAM disk):

time find ./IMAGES/ -depth -name '*.zip' | parallel --eta 'unzip -n -q -d ./IMAGES/ {} && rm {}'
time find ./IMAGES/ -depth -name "*.jpg" | wc -l

In all, this will result in a directory of 714,188 images.

Now it is time to actually scan all of those images for Tucker Carlson:

wget https://storage.googleapis.com/data.gdeltproject.org/blog/2022-tv-news-visual-explorer/2022-tve-facedetect-scan-tuckerface.png
time find ./IMAGES/ -depth -name "*.jpg" | parallel --resume --joblog ./PAR.LOG --eta 'facedetect -q --search-threshold 40 -s ./2022-tve-facedetect-scan-tuckerface.png {} && echo {} >> MATCHING.TXT.tmp'&
sort MATCHING.TXT.tmp > MATCHING.TXT

This will take several hours to run but will ultimately output a file MATCHING.TXT that contains all matching frames.

To compile into the movie seen above requires just a trivial one-liner using ffmpeg:

apt-get -y install ffmpeg
cat $(cat MATCHING.TXT) | ffmpeg -framerate 5 -i - -vcodec libx264 -vf "pad=ceil(iw/2)*2:ceil(ih/2)*2" -y ./TUCKERCARLSON-60MINUTES-20220326-20230217.mp4

Process the full list of frame-level matches into distinct "clips" using a simple PERL script:

#!/usr/bin/perl

use POSIX qw(strftime);
use Time::timegm qw( timegm );

open(FILE, $ARGV[0]);
while(<FILE>) {
    ($ID, $FRAME) = $_=~/.*\/(.*?)\-(\d+)\.jpg/;
    ($year, $mon, $day) = $ID=~/_(\d\d\d\d)(\d\d)(\d\d)/; $TIMELINE{"$mon/$day/$year"}++;
    if ($ID ne $LASTID || ($FRAME - $LASTFRAME) > 4) {
    	($CHAN) = $ID=~/^(.*?)_\d\d\d\d/;
	($year, $mon, $day, $hour, $min, $sec) = $ID=~/(\d\d\d\d)(\d\d)(\d\d)_(\d\d)(\d\d)(\d\d)/;
	$timestamp = timegm( $sec, $min, $hour, $day, $mon-1, $year-1900 ) + ( ($FRAME - 1) * 4);

	$date = strftime('%m/%d/%Y %H:%M:%S', gmtime($timestamp));

	print "<li><a href=\"https://api.gdeltproject.org/api/v2/tvv/tvv?id=$ID&play=$timestamp\" TARGET=_BLANK>$CHAN: $date UTC (Frame: $FRAME)</A></li>\n";
	$CLIPS++;
    }
    $LASTID = $ID;
    $LASTFRAME = $FRAME;
    $UNIQ_IDS{$ID}++;
    $TOTFRAMES++;
}
close(FILE);

open(OUT, ">./TIMELINE.TXT");
foreach $key (keys %TIMELINE) { print OUT "$key\t$TIMELINE{$key}\n"; };
close(OUT);

$sec = $TOTFRAMES * 4;
$cnt = scalar(keys %UNIQ_IDS); print "Found $cnt IDs & $CLIPS Clips & $TOTFRAMES Frames = $sec seconds...\n";

Which is run as:

./makeintoclips.pl ./MATCHING.TXT

That's all there is to it! Of course, for more advanced visual analyses, you could repeat this workflow and replace the "facedetect" tool with any kind of visual analysis system!

 

 

This analysis is part of an ongoing collaboration between the Internet Archive and its TV News Archive, the multi-party Media-Data Research Consortium and GDELT.