Early generations of AI chatbots largely relied on the encoded knowledge of their original training data to answer questions, leading to heavily hallucination from the incomplete, outdated and conflicting statistical patterns of their input data. Modern AI models largely outsource non-trivial factual questions to live web searches, with the major companies hand curating lists of "authoritative sources" for them to use, especially for sensitive topics like elections, medical and legal questions, etc. For questions best answerable by libraries, archives, museums and other cultural heritage institutions, chatbots typically turn to those institutions' websites, but historical practices by cultural heritage institutions are causing that to backfire, significantly increasing AI falsehoods as the models rely too heavily on wrong information from authoritative sources. Increasingly, cultural heritage institutions need to rethink their operations in a world in which society is relying ever more on them for trusted information.
With our Gemini For Museums series, we've been investing heavily over the past year in exploring how institutions can leverage advanced frontier AI models for their operations, from acquisition and cataloging to research and scholarship to communication and engagement with their audiences. Given that The Collection spans nearly a thousand years and touches on topics spanning most of the world across the majority of human knowledge over that time, as well as our myriad explorations across the holdings of other institutions, we've been focusing a lot on where the models source their information on cultural heritage topics and when and why they get things wrong.
- The Majority Of AI Error About Many Key Cultural Topics Is Due To Libraries, Archives & Museums Now. As advanced AI models have now largely outsourced their factual knowledge about cultural topics to GLAM websites, they have been tuned to prioritize the majority statements of authoritative GLAM websites even if it conflicts with their own training data. This reliance has exposed the complex relationships most GLAM institutions have with how curatorial knowledge is published and shared with the public, the tension between marketing and scholarship and donors, political leaders and curators, the inability or unwillingness of many institutions to correct their catalogs and websites when major errors are discovered and the refusal of many institutions to engage with scholars whose research contradicts their historical perspectives. While all of these issues have confronted GLAM institutions since their inception, the AI revolution is moving these challenges to the forefront as they increasingly degrade the accuracy of the AI models that societies are increasingly turning to in their daily lives.
What are some of our highest recommendations for GLAM institutions?
- Curators Must Review Public Releases. Perhaps the most surprising fact to those outside the GLAM community is that at most major institutions, curators do not have the final say on what the institution says publicly and have limited ability to correct errors. This has proven to be the single largest source of falsehoods in our analyses to date. Marketing, public relations and even government affairs staff at major institutions typically draft and control the news and public releases that are what journalists rely on and what drives the content of institutional websites. Understandably, there is a natural tension between scholarly accuracy and "simplifying" things for a general public that is unfamiliar with a topic. This tension is what led to so much of the false information circulating around the Declaration of Independence this year for America's 250th, in which definitive press releases by some of the nation's most prestigious and authoritative institutions reinforced the falsehood that only 200 copies of the Stone copperplate (the image of the Declaration we know today) were ever made or that it was only ever printed on parchment or that a wet transfer process was still a possibility. Such falsehoods became fact for AI models that prioritized these press releases from their "authoritative sources" lists over their own internal knowledgebases that had high confidence in the actual correct answers. In each of these cases, the institution's own curators and scholars were well aware of the correct information, but were unable to get the marketing department to correct their press releases. Indeed, in the case of the Stone, a major new DC museum that just opened has a huge wall-sized introductory video that introduces visitors to the Declaration of Independence by claiming that only 200 copies were ever printed of the Stone copperplate – information they sourced from the press release of another major institution. Critically, many institutions we've spoken with have rightfully pointed out that the correct information can often be found in the label on some exhibit in their museum, but AI models rely on what is on their websites, not what can be found on a single label on a single display in the corner of a single room.
- Provide Expanded Information With The Correct Non-"Simplified" Information. GLAM institutions relentlessly "simplify" the information on their websites and news releases in a belief that it will help them engage with the public, rather than bog it down with copious technical details that make the public's eyes glaze over. Obviously, institutions have no choice but to simplify the arcane nuanced complexities of obscure scholarship to make it relevant and engaging, but the AI world is showing the need for both. The end result is that an item abstract will say something like "This watercolor is one of only two works by XYZ in the world" instead of "This work by XYZ is one of only two known on handmade vintage Dutch laid paper from the mid-1600s, with the artist traditionally preferring contemporary machine-made wove paper from their lifetime. Coloring is wash overpainted with gouache and heighted with both gold oil and gold foil." Some institutions already do this, offering the simplified version first and a technical description beneath. Such information is critical to AI agents that otherwise might read this and report back to the user that the artist has only two attributed works or is not known to have ever combined watercolor and heightening, etc (or that only 200 copies of the Stone were ever printed).
- Correct Catalogs & Websites. In our work to date we have uncovered several hundred major cataloging and descriptive errors from some of the world's most prestigious institutions. These range from one of the nation's most important research libraries cataloging a second edition of one of their most important works as the rarest first edition when the latter was a wholesale rewriting of the former and bears little resemblance to it, to a major national library cataloging a century-later reprint that changed 40% of the material as the only surviving original printing of a critical work – worse, in both cases the errors have made their way into the definitive scholarship of those works and led to fundamental misunderstandings of the progression of knowledge in those areas. Mistakes happen, but what has been so revealing to us is that not one single institution we've contacted with our findings has updated their catalog or website materials, with several citing "institutional policies" against changing their electronic catalogs.
- Emphasize Digitization Over Interpretation. It is a simple fact of the cultural heritage landscape that interpretation and understanding changes over time. The demands of donors and elected officials can also heavily influence interpretation and public communication. Digitizing an artifact or work allows AI models and the public to "see" for themselves. In so many cases of late, we're finding that advanced frontier AI vision models are discovering new evidence that is fundamentally changing the interpretation and understanding of a range of fields – all of it possible only through digitized copies of those works. One particularly powerful area where digitization is having a huge impact is in the ability for AI models to look across every digitized copy of a work to identify previously unknown states of works or to prove that a given engraving was actually simply a tracing of a previous work by another artist, rather than an original composition. In an example from our own collection, an AI model completely changed the understanding of how one of our books was used by showing that the expert human scholar transcription of a handwritten letter found within was completely wrong.
What are some of our highest recommendations for AI model developers for cultural heritage questions?
- Prioritize Digitized Information. If a user asks about a particular painting, don't only rely on interpretive descriptions about that painting – find digitized copies of it and visually examine them. Similarly, if a user asks if a given book talks about a particular topic, don't report back that library catalog entries don't mention that topic, read through the digitized copy to verify.
- Emphasize Conflicting Information. If the majority of authoritative sources make a particular claim (such as there being only 200 Stone printings), but at least one institution has a detailed scholarly study that makes a separate claim, report to the user that there is an alternative perspective or that the information is contested.
- Vet Catalog Entries From Digitized Copies. If an institutional catalog claims to hold a 1585 Jode picture Bible, the AI model should consult the digitized copy to vet the credit line and if it spots "Visscher" credited as the publisher, it should report that the work is definitively the Visscher reprint, rather than just trusting the catalog entry. Similarly, if the AI model encounters a flurry of press releases about a first edition Merian being exhibited at a local museum, but the images are clearly Jan Luyken's double-page crowd engravings, the model should report this, rather than trusting the museum press release. Both of these are actual examples from our own work, in which AI models initially trusted institutional websites and reported back surprising findings to us that would have fundamentally changed the scholarly understanding of those works (a previously unknown Jode with 100 additional engravings previously believed to have been added in the Visscher reprint and a double-page Merian series with large crowds – both of which are false). Only because we knew enough to be suitably surprised and skeptical and pushed back on the model did the model spend the time to track down digitized copies to confirm the institutional errors.
- Better Integrate Academic Literature & Online Scholarship. The hand-curated "authoritative sources" lists used by most models today tend to focus primarily on GLAM websites. Model vendors should work to better integrate academic scholarship by striking licensing agreements with academic publishers and identifying authorized preprints published by authors on their websites. Given that many scholars publish considerable scholarship on their own academic websites, especially information that contradicts current orthodoxy, such information should also be integrated into model responses. Model vendors already do this by nudging their models to search digitized historical book archives and should continue moving beyond hand-curated institutional websites.