One of the more fascinating museum applications of Gemini we've been exploring is its potential to identify and transcribe marginalia at collection scale for the first time. We've been conducting small tests asking Gemini to walk through digitized books and ask it to identify, describe, transcribe and translate as needed any handwritten notes, annotations, illustrations, underlining and other marginalia it finds. It has been extremely effective at this task, from an incredible deciphering of 500-year-old roundels that had eluded human expert analysis to the more mundane transcription of handwritten dealer notes.
Earlier this month we demonstrated how Gemini was able to flag and transcribe a handwritten note in a stunningly crisp 1539 Holbein Icones that attested to its sale in October 1987 by the legendary Maggs Bros antiquarian booksellers. In a testament to the incredible potential of this approach, once Gemini flagged that there was an ownership or sale-related margin annotation and transcribed and flagged it for us, we were able to reach out to Maggs Bros and amazingly, Dr. Hazel Tubman of Maggs Bros. Ltd. who specializes in Continental Books & Illuminations, was incredibly kind and wonderful to track down the original Maggs Bros catalog entry for the book from 1987 from their archives, which you can see below! A huge thank-you to Dr. Tubman for this!
This to us represents one of the areas where AI has the greatest potential to help libraries, archives, museums and other institutions better understand their collections: tracking down provenance information amongst the myriad marginalia in those works. We've been experimenting asking Gemini to scan through digitized works and asking it to identify bookplates, ownership stamps, institutional call and record numbers and even handwritten purchase and sale notes like this, with remarkable success. Here we started with a book whose provenance was entirely unknown beyond its last two owners and Gemini was able to scan through our crude cellphone snaps of the first and last few pages, identify the margin notes, flag that one of them appeared to identify a specific bookseller, sale and date and transcribe and present us with its findings, which in turn led us to Maggs Bros. who was able to ultimately provide us with photos of the original catalog entry – we can't think of a better testament to the future potential of cultural institution use of AI!
