As of last year we had OCR'd more than 19 billion seconds of global television news totaling 294 billion words yielding almost 2 petabytes of JSON. One of the most common questions we get is why we use classical advanced OCR instead of just using LMM models for this OCR, especially the tremendous success we have applying LMMs to historical cultural heritage materials. The answer lies in television news' incredibly rich visual diversity. A typical second of television news contains multiple fonts, multiple text sizes, multiple text colors, moving text (sometimes in multiple directions), a mix of horizonal and vertical text in areas like Asia, dual-directional scrolling text (Arabic + English scrolling in different directions), all superimposed over a huge range of background colors, static imagery and moving video, resolutions from 240 to 2K, SAR compression, poor interlacing and myriad video quality issues representing the full range of broadcast production of the past 26 years across the world. Typically multiple (or even all) of these combined in a single image that must be OCR'd. To date, we've found that LMMs perform poorly when confronted with such visual diversity, yielding highly unstable and brittle results with heavy hallucinations, while GCP's Cloud Vision API's classical OCR performs exceptionally strongly across even the most complex examples we've found across the quarter-century archive.
Why We Still Use Classical OCR For Television News Instead Of LMMs
