Behind The Scenes: XML Tags In TV News Closed Captioning Streams

Traditional broadcast closed captioning is both a powerful human-generated realtime transcript of television news and a source of uncountable oddities over the TV News Archive's quarter century of broadcasts. One particularly vexing challenge is the appearance of HTML and nonstandard XML tags within captioning streams, which pose an interesting conundrum: which should be removed and which should be preserved as providing valuable contextual information?

In all, there are 120,897 captioning lines in the Archive that contain at least one XML tag, consisting of 3,417 distinct tags. Below is a table of some of the most common. Some, like <font> and <i> HTML tags, make sense as a standard formatting feature, while many like <SPLAT!>, <KEVIN DURANT> and <laughter> are not to be found in any standard XML closed captioning lexicon.

Originally, we planned to remove all XML tags, but upon further research we ultimately settled on removing only a limited set of known HTML formatting tags and to preserve the remaining XML tags due to the additional information they provide.

</font> 38027
</cc> 31698
<cc> 31571
<font color="#00ffff"> 27077
<font color="#00ff00"> 10557
<E> 1400
<SPLAT!> 1352
<,> 572
<standup> 551
<nats> 542
<.> 527
<font color="#ffff00"> 337
<NAT SOUND> 293
<SOT> 292
<!0> 287
<STEVE KERR> 273
<url> 254
<i> 250
<$0> 248
</i> 235
<gun shots> 234
<C:SPEAKER> 223
<KYLE SHANAHAN> 221
<inaudible words> 200
<REPORTER> 177
<Scraping sound> 174
<NATS> 163
</url> 153
<DEREK CARR> 146
<ANNOUNCER> 144
<SUFFIX> 136
<KEVIN DURANT> 135
<JON GRUDEN> 127
<music> 125
<Matt> 120
<STEPH CURRY> 117
<Jackie> 116
<Inez> 115
<barking> 112
<horses galloping> 111
<whimper> 111
<BV> 103
<BUTTED> 101
</CC> 101
<punch> 99
<JACK DEL RIO> 98
<PASS> 96
<PAUSE FOR LIVE PICTURE> 96
<Anne> 94
<gunshots> 88
<Digit> 84
</> 84
<President Trump> 82
<STEPHEN CURRY> 82
<CC> 79
<DRAYMOND GREEN> 76
<applause> 73
<RICHARD> 73
<Hacker> 71
<HUNTON> 65
<Reporter> 64
<door> 63
<TOSS TRAFFIC> 62
< MUSICAL NOTES> 61
<Speaking Spanish> 58
<JIMMY GAROPPOLO> 58
<KLAY THOMPSON> 54
<GARY RADNICH> 51
<CR> 51
<gun shot> 51
<footsteps> 50
<laughter> 50
<Buzz> 50
<PAUL MAPS> 47
<w> 47
<KERR> 46
<Cp> 45
<A> 45
<LEBRON JAMES> 44
<JOHN LYNCH> 43
<GARY> 43
<PAUL WEATHER> 42
<AD LIB> 41
<gunshot> 41
<A:> 41
<BOB MYERS> 41
<c> 40
<Giggle> 40
<LINE21. INDENT=0> 39
<sot> 39
<bang> 39
<Gilbert> 39
<laughs> 38
<áááCARRIE:ááá> 38
<♪> 38
<font color="#ff00ff"> 38
<MUSIC> 36
<LINE21. TAB=0> 36
<durant> 36
<lassie barks> 36
<door closes> 36
<horse galloping> 35
<overlapping conversation> 35
<SINGING> 34
<Delete> 34
<u> 34
<Carrie> 33
<Speaking French> 33
<narrator> 33
<Laughing> 33
<Diana> 31
<LAVAR BALL> 31
<over $ 5,000> 30
<MUSICAL NOTE> 30
<JARED GOFF> 30
<TOM BRADY> 29
<CAPTION:CLEAR> 29
</u> 29
<URL> 28
<RICHARD SHERMAN> 28
<Trump> 27
<Hetty> 27
<MIKE BROWN> 26
<BRUCE BOCHY> 25
<GO.> 25
<Giggles> 25
<CHEERING> 25
<CLAPPING> 25
<DURANT> 24
<Gary> 23
<GRUDEN> 22
<TRACK ONE> 22
<BEEP> 22
<hoof beats> 21
<laughing> 21
<CHRIS CUOMO> 21
<http://www.evertz.com> 21
<TOSS TO SPORTS TEASE> 21
<crash> 21
<!42167> 20
< > 20
<vo> 20
<LINE21. ROW=14> 20
<WRAP WITH ANCHORS> 20
<reporter> 19
<DEMARCUS COUSINS> 19
<TRACK THREE> 19
<TRACK TWO> 19
<bell ringing> 19
<rustling> 19
<broll> 19
<TRACK FOUR> 19
<dog barks> 18
<$50> 18
<font color="#ff0000"> 18
<Chirp> 18
<CURRY> 18
<KATE ROONEY> 18
<REUBEN FOSTER> 18
<PAUSE FOR BUMP> 18
<LIVE NEWS DESK> 18
<BOB MELVIN> 18
<!50752> 18
<bark> 18
<TV> 18
<CAPTION:PASS> 18
<TOSS TO WEATHER> 17
<draymond> 17
<=k)■ÑsTñ> 17
<LAUGHS> 16
<bark bark> 16
<nats ... > 16
<ccc> 16
<JESSICA SMITH REPORTING> 16
<shuffling> 16
<Pres. Trump> 16
<NICK MULLENS> 16
<CARR> 15
<whimpering> 15
<WT> 15
<construction> 15
<!37948> 15
<Tt > 15
<JIMMYGAROPPOLO> 15
<JR STONE> 15
<STANDUP> 15
<M> 15
<S:O> 15
<LINE21. ROW=12> 15
<silence> 14
<Binary> 14
<TIGER WOODS> 14
<TOSS TO WX> 14
<Music> 14
<!13436> 14
<horse gallops> 14
<NAT SOUND CHEERS> 14
<!27639> 14
<BARKLEY> 14
<FIGHT> 13
<Rachel> 13
<miaow> 13
<SHANAHAN> 13
<GREGG POPOVICH> 13
<C.J. BEATHARD> 13
<SPLPLATAT!> 13
<DONALD PENN> 13
<AD-LIB> 13
<whining> 13
<■o?÷■LS■ç■■Ñç> 13
<ROGER GOODELL> 13
<BLEEP> 13
<nats > 13
<butt sots> 13
<dog growls> 13
<WHISTLES> 13
<brown> 13
<steve kerr> 13
<thwack> 12
<TRACK FIVE> 12
<birds chirping> 12
<MANNING> 12
<BRIAN HOYER> 12
<Caratatas> 12
<cheering> 12
<take vo> 12
<SCRIPT DOES NOT MATCH> 12
<GREEN> 12
<!42560> 12
<ROBIN TOSSES TO BREAK> 12
<thud> 12
<!24940> 12
<CR > 12
<inaudible> 12
<bark bark bark> 12
<Russell> 12
<Standup> 12
<STAND UP> 12
<slam> 11
<Lucky> 11
<ERIK KARLSSON> 11
<J.R. STONE> 11
<SHOUTING> 11
<BOOM> 11
<Sniffs> 11
<President Donald Trump> 11
<hp://www.evertz.com> 11
<Pres. Donald Trump> 11
<Brett Kavanaugh> 11
<arf arf> 10
<MIKE D'ANTONI> 10
<long lines> 10
<Happy noises> 10
<JAMES> 10
<Disappointed groan> 10
<Q> 10
<Nats> 10
<steph curry> 10
<■> 10
<MARSHAWN LYNCH> 10
<TAKE VO> 10
<BAGPIPES> 10
<MACRO OPEN-"> 10
<mom> 10
<LEON> 10
<MALETICIA> 10
<ZAZA PACHULIA> 10
<BRADY> 10
<whistles> 10
<+rY:q@gAh> 10
<ANNA WIERNICKI:> 10
<door opens> 10
<lassie barking> 10
<BOB> 10
<br/> 10
<myers> 10
<barks> 10
<NICK SABAN> 10