This paper discusses the “utility” and possibilities that IIIF offers for the automated image analysis of medieval manuscripts. It presents several challenges (image analysis, data formats interoperability, user interaction, and ergonomics), all combined within a single workflow. The authors rely on joint work in the research projects Oriflamms (ANR-12-CORP-0010), Himanis (JPI Cultural Heritage) and HORAE (ANR-17-CE38-0008) [1]–[3]. The former made intensive use of TEI encoded editions and automated image analysis (https://github.com/oriflamms), their results are being converted to IIIF. HORAE aims at analyzing the text of medieval handwritten books of hours (often neglected by scholars) and natively integrates IIIF resources as a core research principle.
Examples for the discussion:
1) manifests assembling similar images from the French National Library (http://data.teklia.com/manifests/IIIF-2018/HORAE/BNF/miniature.json; http://data.teklia.com/manifests/IIIF-2018/HORAE/BNF/miniature-and-text.json; http://data.teklia.com/manifests/IIIF-2018/HORAE/BNF/text-with-miniature.json); 2) images with automated line segmentation (http://data.teklia.com/manifests/IIIF-2018/HORAE/BNF/text_lines.json); 3) images with text indexing (multiple indexing hypotheses: http://data.teklia.com/manifests/IIIF-2018/HIMANIS/JJ96/words.json); 4) images with text annotations at word level (http://data.teklia.com/manifests/IIIF-2018/ORIFLAMMS/PsautierIMS.json).Image analysis and presentation
IIIF infrastructures provide access to images, with or without metadata at page level (sequence, labels of foliation or caption and description of illustration, text as range). Additional metadata can be created on IIIF-delivered images through the use of image analysis[4]. In our projects, they are at page level to pre-analyze the book contents (page classification: binding, calendar, full-page miniature, text…), or at infra-page level (layout and line segmentation), infra-line and infra-word (word and letter recognition, expansion of abbreviations, letterform classification). This use case implements diverse parts of the model (collection, manifest, canvas, sequence, range, annotation). The annotations foster the need for their presentation at page level (e.g. color frames in mosaic view, creation of alternative manifests, implementation of alternative sequences or the newly defined ‘unordered’ property[5]). Multiple readings also require being able to classify the annotations to search, retrieve, and present separately (for example) lines/words/letters, graphetic/normalized transcriptions, or indexing data according to confidence level.Data formats, granularity and interaction
Text-image alignment from TEI editions and the presentation in IIIF annotations are feasible (Example 4). However, issues with impoverishing edition formats are stated [6], [7]. The Oriflamms TEI-compliant format is based on stand-off encoding for shape classification and text-image alignment. Parallel developments could be implemented in IIIF. Issues of image analysis and data formats are to be particularly addressed while considering interaction, that is not only exporting rich data into IIIF plain text, but annotating and interacting with image and as text exported data to qualify and enhance the original data at the adequate granularity level of TEI edition. Different interfaces are needed depending on text or image analysis (IIIF interface to interact with “graphic” information, XML-based interface for textual/semantic information); the latter having to acknowledge cognitive issues (annotating snippet views rather than whole pages). As in Oriflamms, a solution may be built integrating several open source software, but it proves difficult to maintain through several developing cycles.