IIIF, a Standard for Multimodal Corpora? The Building of SCENE

Clarisse Bardiot and Jacob Hart, Université Rennes 2 (France)

Following literature genetic criticism, performing arts studies developed since the 1990’s performance genetic analysis, shifting the research focus from the work to its creative process. In this new research field, scholars long directed their effort to participant observation of rehearsals. Digital traces, often ignored in actual research on creative processes, offer new opportunities: produced by every member of the team they allow us to consider the whole creation process from the very first ideas to the premiere. The usual practices of digital humanities are to separate documents by file type and to work in ‘silos’ (text, image…). Such processing prevents us from obtaining an overall view of the collected traces and from examining many phenomena such as the evolution of an idea through different traces, from an image to a text, from a text to a video recording. Another challenge of distant reading and datavisualisation is the loss of the overall view of the trace and the context from which the data was extracted. In order to answer these two main issues, we present SCENE, an open source web app. SCENE’spoint of departure is IIIF (International Image Interoperability Framework). Initially conceived of as a set of standards to allow for interoperable sharing, referencing and annotation of digital images, we use it as a framework at the base of an environment for multimodal document network analysis. This project follows a practice-driven approach to development based on case studies and workshops, notably in the context of Clarisse Bardiot’s ERC-funded STAGE project.

Clarisse Bardiot is a Professor of history of contemporary theater and digital humanities at Rennes 2 University. Her research focuses on performing arts digital traces, creative processes analysis, the history and aesthetics of digital performance, the preservation of digital works, and experimental publishing. With a team of developers, she designed digital environments for performing arts preservation and documentation: a software prototype, Rekall, and a web app, MemoRekall. She is the author of Performing Arts and Digital Humanities. From Traces to Data (Wiley / Iste, 2021). In 2023, she was awarded an ERC advanced Grant for a project called « From Stage to Data, the Digital Turn of Contemporary Performing Arts Historiography (STAGE) ».

Jacob Hart is currently a postdoctoral researcher on the MemoRekall Project at Université Rennes 2, France. He obtained his PhD in musicology at the University of Huddersfield (UK) in 2021 where he was a member of the ERC-funded FluCoMa Project (Fluid Corpus Manipulation). His research centres around tracking the creative process of techno-fluent composers and developing new approaches to computational musicology. His other research interests are the nature of the contemporary ear, experimental music analysis and digital sound visualisation.