Blog post by Iris Yuping Ren, Hendrik Vincent Koops, and Anja Volk.
(Materials are available at https://github.com/hvkoops/wimir2018)
Right after the main ISMIR2018 conference, the WiMIR workshop awaited. As planned, we gathered and formed a working group to tackle the problem of modeling repetition and variation in music for MIR, consisting of the following participants:
Participants
Anja Volk (Utrecht University) – Project Guide
Hendrik Vincent Koops (Utrecht University) – Project Guide
Iris Yuping Ren (Utrecht University) – Project Guide
Juan Pablo Bello (New York University)
Eric Nichols (Microsoft)
Jaehun Kim (Delft University)
Marcelo Rodriguez Lopez (Yousician)
Changhong Wang (Queen Mary University of London)
Jing Chen (Nanchang University)
Tejaswinee Kelkar (University of Oslo)
In the morning session, we first reflected on the background of repetitions and variations as central concepts in music observed by musicologists, and then the computational modeling thereof in Music Information Retrieval within different contexts. We discussed that there exists disagreement in annotations in many MIR tasks, such as automatic chord extraction and repeated pattern discovery. Comparable to many other subareas in machine learning and data science, we face complications brought by the unattainability of an absolute, all-encompassing ground truth annotation.
We then provided more detailed motivations and ideas on how to gather annotations on repetitions and variations in music. For example, one set of guidelines was:
Listen to the following pieces and annotate the salient melodic patterns with
- How relevant this pattern is to this piece
- One word to label the type of this pattern
- A short description on why you find it to be a pattern
- How difficult it was for you to decide whether it’s a pattern
Using the prepared materials, we had a very active discussion on topics such as: how to define the concepts for specific annotation tasks? How can we use tools such as wearable sensors, a wrist band for example, to help the annotation process? How can we compare the annotations and annotation methods? (For more, please refer to the github link).
In the afternoon, after a very interesting and useful lunch breakout session, we started with the actual annotation process on the first page of the String Quartet No. 1 in F major, Op. 18, No. 1, Ludwig van Beethoven (1798 and 1800), Violin I. We provided the sheet music, midi and audio files. The participants used different tools to their liking to mark the repetitions and variations on the sheet music. During the annotations, there were already some interesting discussions in some subgroups: how repetitive the young Beethoven was!
In the second part of the afternoon, using the individual annotations, we began our exchange on the experience of the annotation process. We discussed how we can improve on the current designs of annotation processes and tools for annotation tasks, and how the annotated patterns could be used to design an automatic pattern discovery system. We concluded the day with a short presentation.
Throughout the day, we gained many new insights into what are the good and bad ways to create and employ annotations on repetitions and variations. We warmly thank the participants for a great a day of discussions, listening to music, and annotating repetitions and variations!
Iris Yuping Ren is a second year PhD candidate in the Information and Computing Sciences department, Utrecht University, under the supervision of dr. Anja Volk, dr. Wouter Swierstra and dr. Remco C. Veltkamp. She obtained Bachelor degrees in Statistics and Cultural Industry Management from Shandong University, Master degrees in Complex System Science from the University of Warwick and École Polytechnique, Computer and Electrical Engineering in the University of Rochester, and a diploma in violin performance from the Eastman Community Music School. Her current research has a focus on the computational modelling and statistical analysis of musical patterns in various corpora. She is comparing both generic and domain-specific approaches, such as data mining methods, times series analysis, machine learning based clustering and classification algorithms. To discover useful patterns in music, she makes use of functional programming languages to compute pattern transformations and similarity dimensions. Her research contributes to a computationally- and quantitatively-based understanding of music and algorithms. Music wise, she enjoys playing with local orchestra projects and sessions.
Hendrik Vincent Koops is a PhD candidate at Utrecht University under supervision of Dr. Anja Volk and Dr. Remco C. Veltkamp. Vincent holds degrees in Sound Design and Music Composition from the HKU University of the Arts Utrecht, and degrees in Artificial Intelligence from the Utrecht University. After a research internship at Carnegie Mellon University, he started his PhD in Music Information Retrieval. His PhD research concerns the computational modeling of variance in musical harmony. For example, he studied annotator subjectivity to better understand the amount of agreement we can expect among harmony annotators. Using data fusion methods, he investigated how to integrate multiple harmony annotations into a single, improved annotation. For a deep learning study, he created new features for chord-label personalization. Vincent’s research contributes to a better understanding of computational harmony analysis tasks, such as automatic chord estimation. Vincent is also active as a composer for film and small ensembles. Currently, he’s working towards work for a string quartet.
Anja Volk (Utrecht University), holds master degrees in both mathematics and musicology, and a PhD in the field of computational musicology. The results of her research have substantially contributed to areas such as music information retrieval, computational musicology, music cognition, and mathematical music theory. In 2016 she launched together with Amélie Anglade, Emilia Gómez and Blair Kaneshiro the Women in MIR (WIMIR) Mentoring Program. She co-organized the launch of the Transactions of the International Society for Music Information Retrieval, the open access journal of the ISMIR society, and is serving as Editor-in-Chief for the journal’s first term. Anja received the Westerdijk Award 2018 from Utrecht University in recognition of her efforts on increasing diversity.
Women in Music Information Retrieval 