Our daily Media Trends reports are entirely generated by Gemini (no data is used to train or tune any model), including both the text and the HTML formatting. As every LLM user knows, even the most advanced frontier models still struggle to write perfect HTML, routinely mismatching, truncating and producing malformed tags, yielding HTML that is problematic at best. From simple mistakes like "<b>…</i>" to more complicated ones like "<h1In Germanyb> today, the <p><p><p[>government <a href="announced thathttps://…", the HTML produced by frontier models ranges from flawless to downright chaos. The end result is that each day a certain number of our Media Trends reports contain at least a few oddities, from entire paragraphs being turned into a single hyperlink to entire reports becoming bolded. We've experimented with a variety of approaches to correcting these HTML errors, but the sheer creativity of LLM HTML mistakes makes it difficult to robustly correct these errors. Recently, while using using agentic Gemini to exhaustively iterate over various approaches, Gemini discovered that the humble "tidy" command seems to do the best job at correcting Gemini's HTML errors across all of its models from 3.6 and 3.5 Flash to 3.1 Pro. While it cannot catch or fix all of Gemini's oddities, it does a superb job at localizing the damage, restricting formatting and other problems to an isolated paragraph or list at most and preventing the damage from cascading further across the report.
This is the configuration that we've found most robust thus far and have actually moved into production as of today:
apt-get -y install tidy tidy -q --show-errors 0 -utf8 -ashtml --wrap 0 --force-output yes -modify ./reportcontents.html