Classical production OCR systems work by recognizing text in an image and grouping sequences of related characters and words into "blocks." Each block of text is associated with a bounding box positioning it in the image. To make the OCR output useful for search and analysis, this collection of assorted blocks must be reassembled into a contiguous sequence of text representing the entire image in reading order. It turns out that even small changes to one portion of an image can impact this reassembly process.
Take the two frames below from a news broadcast, separated by one second. We analyze them via Google Cloud Vision's "TEXT_DETECTION" mode (we don't use document recognition since television news channels contain a lot of sparse text, including background text of interest). The first image contains a black square in the "television" box and correctly reassembles the second paragraph of text into "The measures include limiting weekly options contracts to one per exchange, upfront collection of options premium from traders, and raising the minimum contract size to at least 1.5 million rupees, according to a circular published on the Securities and Exchange Board of India's website."
In contrast, the second image displays the station's logo in that same box, with the rest of the frame remaining almost identical other than a few small changes in the numbers displayed in the right-hand columns. This single change is sufficient to break the block reconstruction, merging the text from the "Top News" and the various inset boxes to the right together, yielding "The measures include limiting weekly options contracts to one per exchange, upfront collection of options premium from XDH 23.406 XEO -1.384 5.63% 0.4906 -0.0383 7.24% XTH 18.852 -0.890 4.53% XMR 148.885 -5.570 3.63% 27.553 XZC -0.583 2.13% Later Today traders, and raising the minimum contract size to at least 1.5 "The Close" million rupees, according to a circular published on the Securities and Exchange Board of India's website."
You can see the complete raw output below. The fact that a single change to one portion of the image can have such a strong impact on block reconstruction in the rest of the image is an unfortunate reminder of the challenges of television news OCR.
Top News Bloomberg Television India Clears Steps to Curb Trading Rush in Stock Derivatives Page 1 Next In 0:29 ►India's securities regulator on Tuesday introduced new steps to limit a surge in equity derivatives trading in the nation after growing retail participation took the speculative bets to the highest in the world. The measures include limiting weekly options contracts to one per exchange, upfront collection of options premium from traders, and raising the minimum contract size to at least 1.5 million rupees, according to a circular published on the Securities and Exchange Board of India's website. XZC -0.583 2.13% Later Today "The Close" 3PM NY | 8PM UK | 3AM HK OMX (C) 2611.34 -15.09 0.57% STOXX50 (C) 4954.15 -46.30 0.93% 12:29 ET OCT 1, 2024 COINS CRYPTOCURRENCY-CRYP DOW 42184.34 -145.81 0.34% XBT 62293.30 -1491.79 2.32% XET 2512.335 S&P 500 -101.270 3.93% XLC 65.063 5709.94 -52.54 0.91% -2.335 3.53% XRP 0.6142 -0.0096 1.54% XBN 328.09 -15.85 4.62% NASDAQ 17900.39 -288.78 1.59% XDH 23.406 -1.384 5.63% XEO 0.4906 SMI (C) -0.0383 7.24% XTH 18.852 12086.66 -82.21 0.68% -0.890 4.53% XMR 148.885 -5.570 3.63% 27.553
Top News Bloomber COINS XBT -1491.79 2.32% XET 2512.335 -101.270 3.93% XLC 65.063 -2.335 3.53% }}} CRYPTOCURRENCY-CRYP DOW 62293.30 42183.71 -146.44 0.35% S&P 500 5709.86 -52.62 0.91% XRP 0.6142 -0.0096 1.54% NASDAQ XBN 328.09 -15.85 4.62% 17900.48 -288.69 1.59% India Clears Steps to Curb Trading Rush in Stock Derivatives Page 1 Next In 0:28 ►India's securities regulator on Tuesday introduced new steps to limit a surge in equity derivatives trading in the nation after growing retail participation took the speculative bets to the highest in the world. ▶ The measures include limiting weekly options contracts to one per exchange, upfront collection of options premium from XDH 23.406 XEO -1.384 5.63% 0.4906 -0.0383 7.24% XTH 18.852 -0.890 4.53% XMR 148.885 -5.570 3.63% 27.553 XZC -0.583 2.13% Later Today traders, and raising the minimum contract size to at least 1.5 "The Close" million rupees, according to a circular published on the Securities and Exchange Board of India's website. 3PM NY | 8PM UK | 3AM HK SMI (C) 12086.66 -82.21 0.68% OMX (C) 2611.34 -15.09 0.57% STOXX50 (C) 4954.15 -46.30 0.93% 12:29 ET OCT 1, 2024