In speaking with an incredible range of libraries, archives, museums, cultural ministries and other cultural heritage institutions across the world over the past year, I've noticed a fascinating and growing divide between technologists, cultural heritage technologists and curators/leadership. Traditional technologists can bring to bear some of the most advanced experimental technologies, but tend to focus on what their tools can do, rather than what would be most useful and without focusing too much on evaluating how accurate they are or the ethical, legal and moral challenges of their applications. In contrast, cultural heritage technologists tend to emphasize less advanced technologies, but have the necessary backgrounds to understand how to most effectively deploy those tools within their fields and deeply understand the importance and implications of accuracy, ethical concerns, the richly diverse needs and backgrounds of visitors, the role of cultural heritage in society, etc. At a leadership level, a few institutions I've spoken to have leadership that truly understands both the very real limits and tremendous potential of AI for their institutions and are embarking on thoughtful journeys, but the single most common response I get from curators and administrators the world across is not cautious experimentation or intrigued interest, but rather outright dismissal – that AI has absolutely no place in their field. From our own work exploring the potential of AI across our own collections over the past year, here are a few insights we've learned.
- General Cataloging. The visual reasoning capabilities of modern AI tools have advanced to the point where they are able to fairly robustly and reliably identify a specific edition of a specific work from a single photograph of a single page, autonomously pulling up all worldwide digitized copies of the work to confirm the smallest differentiating details like a blemish in a woodcut or a page number or single-word difference. They are especially useful for works lacking a title page, sammelbands and other composites like scrapbooks and where the title page or other identifying information is suspect. Their ability to locate and search every digitized copy of a work across the web, precisely detail the details that define a given edition and link directly to digitized imagery of those other confirmed copies to support its arguments is immeasurably helpful to the work of cataloging. Just in the course of our own research work, we've uncovered that two widely-cited first edition copies of a major work held by two tier one institutions were actually later reprints: Gemini instantly spotted the key details marking it as that later reprint and we were later able to confirm this through archival research.
- Cataloging & Identifying Rare States. The immense visual knowledgebases of modern AI models and especially their ability to visually search across the entire internet has proven to be a step change for us in identifying rare and in some cases previously unknown states of some of our rarest works. In one case, Gemini spent almost an hour searching millions of images across the web until it tracked down the exact source and rationale of an engraving pasted onto the cover page of one of our works. In two simpler cases, Gemini instantly spotted that our Mame Dore Bible had the first state of the "Formation of Eve" plate, while our Mortier Large Bible was "before the nails" – in both cases, these state differences had never come up in any of our conversations with the scholars or curators we had spoken to about both. In all these cases, Gemini instantly spotted this information unprompted simply from quick cellphone snaps we had taken of random pages and we cannot wait to see what it discovers once they have been fully digitized.
- Guided Cataloging Of Editions, States & Common Defects. One of the most powerful and immediately deployable applications of AI-assisted cataloging lies in its ability to instantly regurgitate the unique characteristics of a given edition or state of a work and its component pieces and identify common defects found in those copies. For example, cataloging a copy of the famous fifth volume of Peter Force's work or the relatively obscure Sears Pictorial History of the American Revolution, Gemini will flag that the foldout DOI facsimiles are almost always missing. Cataloging a 1585 Jode that are almost always missing their tables of contents, it can surface a transcribed and translated copy of the most likely TOC and instantly scan cellphone photos of the work to identify which engravings are missing.
- Marginalia & Provenance. Gemini has proven exceptionally adept at identify, transcribing, translating and researching handwritten marginalia across our works. Most famously, last December it solved a mystery of a Nuremberg Chronicle bifolium that had eluded multiple expert investigations, while more recently it flagged that one of our books was an ex-Folger work and they kindly provided us the original auction records for it, while perhaps most interestingly, it flagged that our Holbein Icones had an auction note from 1987 and was able to discern sufficient detail that Maggs Bros was able to locate the original auction records from their archives, which in turn pointed us to Harvard's Library for more information on the state of one of its engravings.
- The Power Of AI "Deep Research" Tools For Communication. Perhaps the single most unanimous refrain I've heard is that so-called AI "deep research" tools include too many inaccuracies to be of any use in producing or evolving the interpretative narratives for exhibitions. In fact, we've found precisely the opposite: by virtue of their knowledgebase being drawn from the open web, the narrative they compile strongly reflects many of the questions, concerns, interpretations and misconceptions that museum visitors are most likely to arrive with. I recently spent an hour at NARA just observing all of the questions visitors were asking of the docents about the documents, especially the Declaration of Independence and I ended up actually acting as an impromptu docent one afternoon when the Smithsonian displayed its parchment Stone DOI facsimile in the Castle. All of the most common misconceptions I heard (wet transfer process, that the Stone is a photograph, that the engrossed DOI was written and signed on the 4th, that the engrossed predates the printed Dunlap, that the printing press wasn't invented until the 1900's, etc) are all ones these deep research tools surfaced as the most common ones they encountered across the web. Even curators themselves and the experts they turn to can get things wrong. In the realm of Biblical art, it is just remarkable how many art museum curators and experts we've spoken to that assumed the Visscher was the first picture Bible and had no knowledge that it was merely a reprinting of the 1585 Jode with ~100 or so additional plates. Many have publicly communicated that wrong information. Once again, deploying these AI research tools has been immensely valuable to us in gaining a better understanding of the kinds of misconceptions that we should preemptively address.
- Cross-Disciplinary Knowledge Transfer. Given the tremendous specialization in the cultural heritage world, we've found AI research tools to be enormously useful in transferring knowledge from other fields: most recently in drawing from Dutch engravings and European cartography and currency production to help Declaration of Independence scholars better understand and communicate the potential longevity of the William Stone copperplate. In particular, the ability of advanced reasoning models to draw from scholarly literature across myriad domains and provide citations and links back to relevant studies from other disciplines and explain why each of those studies is relevant to the question at hand is tremendously valuable as a curatorial tool.
- Find An Expert. Of especial relevance even to the most reticent institutions, we've found AI research tools to be particularly adept at helping to locate scholars of extremely specialized and obscure topics that we've been able to then reach out to to engage in deeper conversations. Given a photo of the title page or even just a random page if the title page is missing of a particularly rare work, Gemini can instantly list key scholars who have written about that work and the areas they have focused on, providing links directly to their institutional pages and papers.