Believing in the importance of shedding light on the stories of successful women in the Music Information Retrieval (MIR) field, we are happy to share our interview with Dr. Dorien Herremans, the second Inspiring Women in Science interview. Dr. Herremans is an Assistant Professor at Singapore University of Technology and Design and Director of Game Lab. She has a joint-appointment at the Institute of High Performance Computing, A*STAR and works as a certified instructor for the NVIDIA Deep Learning Institute. Her research interests include machine learning and music for automatic music generation, data mining for music classification (hit prediction) and novel applications at the intersection of machine learning/optimization and music.
Whereabouts did you study?
I completed a five-year masters degree in business engineering (in management information systems) at the University of Antwerp. I spent the next few years living in the Swiss Alps, where I was an IT lecturer at Les Roches, Bluche, and had my own company as a web developer. I returned to the University of Antwerp to obtain my PhD in Applied Economics. My dissertation focused on the use of methods from operations research and data mining in music, more specifically for music generation and hit prediction. I then got a Marie-Sklodowsi postdoctoral fellowship and joined the Centre for Digital Music (C4DM), at Queen Mary University of London to develop Morpheus, a music composition system with long-term structure based on tonal tension. After my postdoc I joined Singapore University of Technology and Design, where I am an assistant professor and teach data science and AI. My lab focuses on using AI for audio, music and affective computing (AMAAI), I’m also Director of the SUTD Game Lab and have a joint appointment at the Institute for High Performance Computing, A*STAR.
What are you currently working on?
Some of our current projects include a Music Generation system based on emotion (aiMuVi); music transcription; a GPU-based library for spectrogram extraction (nnAudio); multi-modal predictive models (from video/audio/text) on emotion and sarcasm detection.
When did you first know you wanted to pursue a career in science?
It happened rather naturally. When I was about to graduate, I felt more of a pull towards staying in academia versus going into industry. Especially because at the time, with a degree in business engineering, that would have most probably meant joining the big corporate world. As a 24 year old, I instead wanted to keep exploring new things, stay in the dynamic environment of academia, especially since I could do so while living in a very quaint mountain village.
How did you first become interested in MIR?
During my last year as a student in business engineering, I was looking for a master thesis topic and came across ‘music and metaheuristics’. Having been passionate about music my whole life, I jumped at the opportunity to combine mathematics with music. This started an exciting journey in the field of MIR, a field I did not know existed at that time (2004).
What advice would you give to women who are interested in joining the field of MIR but don’t know where to begin?
We are fortunate to have a growing community of MIR researchers. Through groups such as WiMIR or ISMIR, you can join mentoring programs and get in touch with researchers who have more experience in the field. If you are a beginning researcher, you could also attend one of the conferences and start building a network.
How is life in Singapore? Is there a difference for your research between working in Europe and Asia?
My first impression when arriving in Singapore a few years ago, was that it felt very much like living in the future. It’s quite an amazing country, efficient, safe, warm (hot really), and with amazingly futuristic architecture. As a researcher, Singapore offers great funding opportunities and a dynamic environment. We have been growing the AMAAI lab steadily, and are excited to connect with other music researchers in Singapore (there are more than you might think!).
You are working on AI and music now which is a fascinating field. What can it do and cannot now?
After almost a decade of science fiction movies that play around with the concept of AI, people seem to equate AI with machines obtaining self-awareness. That’s not what we should think of as AI these days. I see (narrow) AI systems as models that learn from historical data, extract patterns, and use that to make predictions on new data. Narrow AI focuses on clearly defined problems, whereas general AI is more challenging and tries to cope with more generalised tasks. In MIR we typically develop narrow AI systems, and due to the recent developments in neural network technologies and the increasing GPU power, we are making large strides. The challenges that we are currently facing are in large part related to the lack of labeled data in music, and the cross-domain expertise required to leverage music knowledge in AI systems.
How to make human musicians and AI musicians work together and not compete with each other?
This will be the natural first step. Unless properly educated about AI, many people will not trust AI systems to take on tasks on their own. I believe this is why many of the personal assistant AI systems are given female names and voices (exudes trust?). For example, a composer might not want a system to generate music automatically, but they might appreciate a computer-aided-composition system, which, for instance, gives them an initial suggestion for how to harmonise their composed melody.
It seems still some distance for it to be useful in daily life compared with face/voice recognition. What is your expectation for that field?
I actually think that AI in music is being integrated in our daily lives, through companies such as Spotify, Amazon Music, etc. as well as through smaller startups such as AIVA. I expect the number of startups in the music tech area to increase strongly in the coming years.
You are also working on combining emotion and music together. On what level do you think the computer can understand human emotion?
The word ‘understand’ is tricky here. We can train models to predict our perceived or experienced emotion based on observations we have done in the past, however, the biggest challenge seems to be: why are different people experiencing different emotions when listening to the same piece of music?
These days more and more people work in different fields with AI. For the students working on music and AI, can you give them some guidance about their research strategy and career path?
As for any research topic, I would recommend students to tackle a problem that they are fascinated by. Then you dive deep into the topic and explore how it can be advanced even further. To stick with a topic, it’s essential that you are passionate about it.
Can you give a few tips for people working at home in the days of Covid-19?
Stay inside, get as much exercise as you can, and try to think of this as the perfect time to do undisturbed research…
Rui Guo graduated with a Master’s Degree in Computer Science from the Shenzhen University, China. Currently he is a third year PhD student in University of Sussex, UK pursing his PhD degree in music. His research topic is AI music generation with better control and emotional music generation.
2020 was definitely a strange year. It felt shorter than ever. With vaccines becoming available in more countries, I am hoping that there is a better future for us. Also, that we can meet in real person for next year’s ISMIR.
Since we were not able to see each other for the last year ISMIR, I wanted to share our community members’ 2020 story by conducting a survey with questions related to work, hobbies and general lifestyle. A total of 62 people have responded (thank you!) and I would like to share some results.
Majority of people did like working from home!
But online meetings, not so much…We like talking to real people.
More than 50% of people said they were less productive last year. Don’t feel bad if you feel like you didn’t achieve much! If you managed to stay healthy last year, that is the biggest achievement.
I asked the members to share one good thing about working from home. In terms of work impacts, people reported less distraction; a more flexible schedule and efficient work – “I can do things with my own rhythm”, “No need to go to the office (which means saving time, and also less environmental impact)“; and more equality between remote and “local” employees. Respondents also reported positive outcomes for family life (e.g., “Being able to see my daughter throughout the day”). Finally, working from home was perceived as positive in its integration with home life – for instance in terms of food (e.g., “No need to eat on the road.” “Eating better food”) and a comfortable work environment (e.g., “Comfortable couch”, “Could listen to music without headphones”) – as well as lifestyle: “Fitting exercise, cooking, and music practice into my day”, “Can play with my dog”, “Not having to wear pants ;)”, “Wearing pajamas all day :D”.
It seems like many of us were able to put more time into our hobbies. Same goes for me!
Also, quite a lot of people found new hobbies.
Our members shared some of their hobbies. As music lovers, there were many hobbies related to music: making music, playing instruments, jamming online, singing, DJing, hosting radio shows and learning music. In fact, 71% of us listened to more music compared to last year! I assume there is a correlation with COVID-19 situation. Plus, there were lots of physical activities, such as running, cycling and yoga. Some of us enjoyed baking, cooking and even brewing kombucha! Meditating, knitting, drawing, reading and gardening also were mentioned several times.
Staying healthy mentally and physically seem to be the greatest challenge these days. I personally had to consciously remind myself to move and stay calm. It definitely wasn’t easy. I think I managed to find some balance finally in December.
With increased time at home, I personally learned a lot about myself. It gave me time to reflect on my life once again and I was able to remind myself what is important in my life.
So I asked, “Is there anything new that you discovered about yourself during the pandemic?” and found 4 common responses.
First, there were people who discovered that they enjoy remote working and staying home – “I don’t get bored!”, “I’m more OK than I thought with extended alone-time”, “I should stay at home more often than I used to”. Some even mentioned they were able to build good routines. Opposite to the first, the second most common response was the realization of how interactions with people are important in their lives – “how much I depend on physical connections with friends”, “That I am more of a people person than I ever suspected”. Some said they would rather go to the office than stay home (Me too!). Third common response was that they felt the need to physically move more and spend time outdoors – “I’m unfit for living as a hermit”, “don’t underestimate the power of breaks in the sunshine”. It seems like many of us learned that physical activities not only improve physical, but also mental health. Last common response was the need for discipline. Having 100% control over our time and this freedom appeared to be attractive, but in fact, it requires a lot of effort to keep everything on track.
Here are some additional memorable responses: “I am living in a more privileged condition than I have realized”, “I can cook!”, “That we need to enjoy life” and “That I’ve actually been living pretty much like this even before.”
Everyone misses pre-covid days. Our community members told us that they miss live concerts and festivals, hanging out with real people and travelling the most.
Let’s not give up hope, so that we can meet again soon. Please don’t forget to take care of the environment to prevent this kind of pandemic in the future.
As a closing note, we asked people to share one tip they have for the community to survive this time.
The most mentioned tip was to stay healthy physically and mentally. Some suggestions include getting a good sleep, eating good food, moderately exercising and doing mediation. Staying connected with others and helping others were a runner-up tip. Although we already have enough zoom talks, non-work related zoom socializing can actually make us feel better, like zoom beers!
Also many emphasized having a routine or some rule, such as setting a certain time slot for work and break. Don’t think too much about how others have been doing; breathe and focus on yourself 🙂
There were some extra fun and useful tips: “Don’t let the dirty dishes accumulate”, “fix things in your house”, “learn new things”, and “TableTop simulator (on Steam) was quite useful during lockdown”.
Stay healthy and happy till we meet next time !!
Kyungyun Lee is a MSc student at Music and Audio Computing Lab, KAIST. Her research interests range from MIR to HCI. Currently, she is interested in automatic music generation and analyzing user interaction with music platforms.
We’re pleased to tell you that our highly virtual, distributed-in-time-and-space WiMIR Workshop for 2020 went pretty well, all things considered!
Rather than put everything on a single day, we wanted to spread the event out – both to avoid Zoom fatigue, and to allow folks across different timezones to join in easily. We ended up running four sessions across eight weeks, ranging from Pacific Daylight Time to Indian Standard time. We had speakers from India, Australia, California, Europe, and had over 450 signups!
We had each of our presenters give a 45 minute talk about their work and career, and then followed it up with a socializing & networking session. We took advantage of Zoom’s breakout rooms to send folks into rooms of 4-6 people for twenty minutes or so. We did this twice, and then the remaining people met in the main Zoom call for a very relaxed end-of-call discussion.
Thanks to everyone who joined us, for every session! Special thanks to our volunteers: Jay Appaji, Phoebe Chua, Elena Georgieva, Rui Guo, Kyungyun Lee, Alia Morsi, & Elona Shatri – and extra-special thanks to our project guides: Christine Bauer, Tom Collins, Daniel Ellis, Jenn Thom, Doug Turnbull, Amanda Krause, Preeti Rao, Rohit M. A., Juanjo Bosch, Amy LaMeyer, & Darragh Dandurand.
For me, being part of the ISMIR community and attending the annual conference was like a dream come true! I’ve always been playing music since I was in junior high school, started with playing an acoustic guitar, and sing a little bit. But what I knew back then, music was just a hobby (except the professional musicians).
If you want to watch me chatting around and see a little bit of The ISMIR Conference 2019 experience, you can view this video on my YouTube Channel:
Getting a Bachelor’s degree in Informatics Engineering at Telkom University, Indonesia, somehow exposed my ability in music performance more. But at the same time, I still learned about technologies, coding, and artificial intelligence by joining the labs and organizations.
The Institute of Electrical and Electronics Engineers (IEEE) was the first international organization that I joined. Although I wasn’t sure about my ability to volunteer and organize in this community, I was keen to learn new stuff, especially getting some global experiences.
During my 6th Semester of college, I realized that I had to choose an undergraduate thesis related to artificial intelligence. I’ve joined the Artificial Intelligence (AI) Laboratory for two years and mostly took the elective courses from the Intelligence, Computing, and Multimedia (ICM) track. But the problem, the AI application that was available at that time, was only for image, video, text, and speech.
Suddenly, I got a random idea to search on the IEEEXplore about any AI research that related to audio (other than speech). Surprisingly, I found a 2006 paper called the “Automatic Mood Detection and Tracking of Music Audio Signals.” From that moment, I just felt like, “Okay, I think this research is pretty exciting for me!” Then, I decided to work on a topic called Music Emotion Classification for my undergraduate thesis.
In 2018, I got paired withDoga Cavdir, a PhD student from Stanford University, in the mentorship program. Although I had no idea about this field, this mentorship program helped me a lot to get started in Music Information Retrieval (MIR).
Doga recommended me to go to the ISMIR Conference 2018 in Paris, but unfortunately, I couldn’t go. But at least, I knew that ISMIR is not only a community. They also hold a conference annually, where the students, researchers, and even industries can publish their research papers.
How Did I (Finally) Make It to The ISMIR Conference 2019?
So, I have a funny story regarding the 2019 ISMIR Conference.
In December 2018, after I realized that I couldn’t go to the 2018 ISMIR Conference, somehow, I was hoping that I can join the 2019 ISMIR Conference. But, when the 2019 year was starting, I even forgot that I wanted to go to the conference. So, I was only focused on doing my job while working in a company, creating videos, launching Qhansa.Lens Photography and Videography, making music, and many other activities.
But suddenly, in August 2019, the universe sent me to keep checking the ISMIR Conference 2019 website. And then, I realized that the organizers added one other financial support from the ISMIR Community, called the Community Grants.
The Community Grants were offered to support several individuals who would like to attend ISMIR but who are not in the capacity to participate in the conference actively. For example, this may include:
former ISMIR members, who would like to re-engage with the community, but cannot trivially be supported for this given their current roles;
students and researchers with concrete potential to become a part of the ISMIR community in the future, but who currently are not in a sufficiently supportive context to act on this yet (e.g., because their institutes do not have clear MIR expertise, or they affiliate to neighboring disciplines, that do not have conference-oriented publishing cultures).
After that, I just submitted the application, motivation letter, and a recommendation letter from Blair Kaneshiro, one of the WiMIR organizers. *Thank you, Blair! 🙂
Then in September 2019, I got an email from the local organizer that I received the Community Grants for ISMIR 2019. I was surprised and happy at the same time! Even though it didn’t cover all of my expenses, but it helped me a lot to reduce my budget.
So, that’s my story!
Important Links and Resources
Now, I’m going to share some of the essential links and resources regarding the ISMIR Conference 2019 that I joined.
1st Workshop on Designing a Human-Centric MIR System (Satellite Event)
My group at Google has been working on developing general-purpose sound event recognizers. I’ll briefly recap the evolution of this work from its origins from virtually nothing in 2014 to deployed apps today. I’ll also talk a little about my own transition from academia to industry, and the day-to-day details of my work as a Tech Lead – Research Scientist – Manager at Google.
Dan Ellis leads a small team developing sound event recognition technologies within Google AI/Perception. From 2000-2015 he was on the faculty of the Electrical Engineering department at Columbia University, leading research into environmental sound processing and music audio analysis. He now regrets encouraging his students to write Matlab without unit tests.
Jenn Thom: Improving the Music Listening Experience: HCI Research at Spotify
Music plays an important role in everyday life around the world. People rely on music to manage their mood, express their identity and celebrate milestone events. Streaming services like Spotify have transformed the way that people consume audio by providing listeners with multiple personalized ways to access an abundant catalog of content. In my talk, I will describe several active areas of HCI research at Spotify and present our work on understanding how people search for music and how we can enable exploration for listeners.
Jenn Thom leads the HCI research lab at Spotify. Her current research interests include understanding how people search for and describe music and developing novel design and prototyping methods for conversational interactions. Prior to joining Spotify, she was a Research Scientist at Amazon where she worked on collecting and mining data to bootstrap new features for the launch of the Echo. She was also a Research Staff Member at IBM Research where she studied how employees used social networks for intercultural collaboration. Jenn received her PhD from Cornell University and her dissertation focused on how people expressed territorial behaviors in user-generated content communities.
Doug Turnbull: Locally-Focused Music Recommendation
There are talented musicians all around us. They play amazing live shows at small venues in every city all around the world. Yet music services like Spotify, Apple Music, YouTube, and Pandora do a poor job of helping listeners discover these artists for a variety of commercial and technical reasons. To remedy this problem, I will discuss our recent efforts to use recommender systems to support locally-focused music discovery. First, I’ll provide a brief introduction to recommender systems, the long-tail consumption models, and popularity bias. I’ll then describe how we can adapt typical recommender system algorithms to be better at recommending local (long-tail) music. Finally, I will describe a personalized Internet radio project called MegsRadio.fm, why it failed after years of dedicated development, and how lessons learned are being incorporated into the design of my new project called Localify.org.
Doug Turnbull is an Associate Professor in the Department of Computer Science at Ithaca College. His research focuses on music information retrieval, computer audition, machine learning, and human computation. His research passion is using recommender systems to promote music by talented local artists. He is currently working on Localify.org which explores using music event recommendations and playlist generation on Spotify to support local music communities. This project is funded by the National Science Foundation and being developed by a large team of undergraduate students at Ithaca College. He is a former ISMIR conference program co-chair and former ISMIR board member. More information about his research can be found at https://dougturnbull.org.
Amanda Krause: Everyday Experiences of Music: A Fireside Chat with Dr. Amanda Krause
Given the prominence of music in our everyday lives and developmental shifts in technology, how do people access, consume, and respond to music? Working in the social and applied psychology of music, Dr. Amanda Krause researches how our everyday experiences with music influence our well-being, in order to better understand the place that music occupies in modern life. In this fireside chat, Amanda will discuss her research topics and approaches, how she has pursued her research interests via travel, education, and collaboration, and the challenges and opportunities that have arisen from establishing an inter-disciplinary research career. She will also reflect on how the MIR and music psychology disciplines intersect, how she has made connections within the MIR community, and how researchers working in these disciplines can collaborate to tackle some very interesting and challenging research questions.
This fireside chat will be moderated by Dr. Blair Kaneshiro.
As a music psychology scholar based at James Cook University, Dr. Amanda Krause studies how we experience music in our everyday lives. Her research asks how our musical experiences influence our well-being. Amanda’s current projects examine the role of music listening and the radio in supporting individual and community well-being. Amanda is the author of numerous academic publications and currently serves on the Australian Music & Psychology Society (AMPS) committee. She has also spoken on her research to academics and industry leaders at conferences around the world, to students through programs like Skype A Scientist and STEM Professionals in Schools, and to members of the general public via radio appearances and events like Pint Of Science.
Preeti Rao and Rohit M. A.: Unity in Diversity: MIR Tools for Non-Western Music
Just as there is so much linguistic and cultural diversity, there is rich diversity in music across the globe. But the universals of musical structure and attributes such as pitch, rhythm and timbre that describe all music enable us to apply the rich tools of MIR developed for Western music to interesting and musically relevant tasks in genres as distinct as Indian art music. We discuss some important considerations for researchers such as (i) identifying MIR-addressable problems and the tools to apply, and (ii) dealing with the anticipated limitations of labeled datasets. We do this with easy to follow examples from Indian music and show how the insights obtained can be rewarding, also in terms of understanding the music better!
Preeti Rao has been on the faculty of Electrical Engineering at I.I.T. Bombay, teaching and researching in the area of signal processing with applications in speech and audio. She received her Ph.D. from the University of Florida in Gainesville in 1990. She was a collaborator in the CompMusic project during 2011-2016 for the application of MIR to non-Western music, led by the MTG at UPF, Barcelona. She currently leads another international collaboration funded by the Government of India for research in Computational Musicology and Musical Instruments Modeling for Indian Music. She has been actively involved in development of technology for Indian music and spoken language learning applications. She co-founded SensiBol Audio Technologies, a start-up incubated by I.I.T. Bombay in 2011, with her Ph.D. and Masters students.
Rohit is a Master’s student and a research assistant in the Digital Audio Processing lab in the Electrical Eng department at IIT Bombay. His background is in communication and digital signal processing and his research interests lie in MIR, computational musicology and machine learning for audio. His current research is centered around developing tools for analysis of the Hindustani classical art form and instruments, with a focus on studying performance related aspects. He is also a trained violinist.
This workshop will give an overview of the usage of music information retrieval and more generally artificial intelligence for assisting composers and producers when making music, from both a research and an industry perspective. We will talk about some of the recent advancements in machine learning applied to (audio and symbolic) music generation and repurposing, and we will review some of the techniques that paved the way there. We will also look at how startups and large companies are approaching this field, some of the real-world applications that have been created, and we will finally discuss some specific examples of how artists and coders have been using such technologies. Could we even try to imagine what the future of this exciting field may look like?
Juanjo is a Research Scientist working at the Creator Technology Research Lab at Spotify, whose main mission is to create tools for musicians / producers. He holds a Telecommunications Engineering degree from Universitat Politécnica de Valencia, a Masters (in Sound and Music Computing) and PhD from the Universitat Pompeu Fabra (Music Technology Group, Barcelona), which was conducted under the supervision of Emilia Gómez. He has also visited other academic institutions such as the University of Sheffield, Queen Mary University of London (C4DM), and worked for three years at Fraunhofer IDMT. Before joining Spotify, he already had experience in the industry including Hewlett Packard and Yamaha Music. His main research interests lie at the intersection of music information retrieval and AI-assisted music creation.
Amy LaMeyer and Darragh Dandurand: XR and Music – A Conversation
Extended reality (XR) is radically changing the way we create, consume, and socialize around music. In this conversation, Amy LaMeyer and Darragh Dandurand will discuss today’s landscape of XR and music, including the current state of the industry, recent technological advances, and innovations in artist-fan connections in the age of COVID. They’ll also speak about the history and mission of the WXR Fund, and reflect upon her own professional journey and what it means to forge an authentic career path.
Amy LaMeyer is Managing Partner at the WXR Fund investing in early stage companies with female leadership that are transforming business and human interaction using spatial computing (VR/AR) and AI. She has been named one of the people to watch in AR by Next Reality. Amy is the author of ‘Sound and AR’ in the book “Convergence: how the world will be painted with data”. She has 20 years of experience in a high growth technology industry in corporate development, mergers and acquisitions, engineering and finance.
Darragh Dandurand is an award-winning creative director, brand strategist, photojournalist, and curator who has worked in media for a decade and now in immersive technology / spatial computing, as well. Recent clients include a number of media outlets and studios, such as Refinery29, VICE, Verizon, the New Museum, Buck Co, Superbright, Sensorium, iHeartMedia and Wallplay. Currently, Darragh is consulting creative tech teams, developing her own experiential projects, publishing articles on mixed-reality, and researching and presenting on the intersection of fashion, e-commerce and wearable tech. She sits on the board of directors for the Femme Futures Grant via The Kaleidoscope Fund. Darragh has lectured at Stanford University, Temple University, University of Maryland, University of Rhode Island, The New School, and the Fashion Institute of Technology, as well as VRARA Global Summit, Samsung, Out in Tech, MAVRIC, Magic Leap’s LeapCon, and others.
Christine Bauer: The *Best Ever* Recommendation – For Who? And How Do You Know That?
Music recommender systems are an inherent ingredient of all kind of music platforms. They are meant to assist users in searching, sorting, and filtering the huge repertoire. Now, if a recommender computes the *best ever* recommendation. Is it the best choice for the user? Or the best for the recommended artist? Is it the best choice for the platform provider? Is the *best ever* recommendation equally valuable for users, artists, and providers alike? If you (think you) have an answer, how do you know that? Is it indeed the *best ever* recommendation? In this session, I will provide insights on what we miss out on in research on music recommenders. I will point to the perspectives of the various stakeholders and to the sphere of methods that may allow us to shed light upon answers to questions that we have not even asked so far. I will *not* provide the ultimate answer. I do not know it. It is research in progress. The goal is to move forward together. Expect this session to be interactive with lots of brainstorming and discussion.
Christine Bauer is an assistant professor at Utrecht University, The Netherlands. Her research activities center on interactive intelligent systems. She focuses on context-adaptive systems and, currently, on music recommender systems in particular. Her activities are driven by her interdisciplinary background. She holds a Doctoral degree in Social and Economic Sciences, a Diploma degree in International Business Administration, and a Master degree in Business Informatics. Furthermore, she pursued studies in jazz saxophone. Christine is an experienced teacher and has been teaching a wide spectrum of topics in computing and information systems across 10 institutions. She has authored more than 90 papers, received the prestigious Elise Richter grant, and holds awards for her research as well as her reviewing activities. Earlier, she researched at Johannes Kepler University Linz, Austria, WU Vienna, Austria, University of Cologne, Germany, and the E-Commerce Competence Center, Austria. In 2013 and 2015, she was Visiting Fellow at Carnegie Mellon University, Pittsburgh, PA, USA. Before starting her academic career, she worked at Austria’s biggest collecting society AKM. More information can be found on her website: https://christinebauer.eu
Tom Collins: Automatic Music Generation: Demos and Applications
There has been a marked increase in recent years in the number of papers and algorithms addressing automatic music generation (AMG). This workshop will:
Invite participants to try out some tweak-able demos and applications of music generation algorithms;
Cover some of my lab’s projects in this area, which include a recent collaboration with Grammy Award-winning artist Imogen Heap, and integrating AMG algorithms into computer games;
Review approaches and applications from other research groups, such as Google Magenta;
Underline that a literature existed on this topic before deep learning(!), and that evaluation should consist of more than optimizing a metric.
Tom studied Music at Cambridge, Math and Stats at Oxford, and did his PhD on automatic pattern discovery and music generation at the Open University. He has held multiple postdoc and visiting assistant professor positions in the US and Europe, and now splits his time between University of York (where he runs the Music Computing and Psychology Lab) and the music cooperative MAIA, Inc.
We are delighted to announce that the WiMIR 3rd Annual Workshop will take place as a series of weekend events before the start of the virtual ISMIR 2020. We’re also delighted to again be an official satellite event of the ISMIR conference!
Due to the virtual-ness of ISMIR this year, we’re changing our format: Each workshop date will feature talks by top researchers in the field, followed by small-group social sessions — all in Zoom. This year, we are making a special effort to offer programming across time zones and regions, to make it easier for our colleagues who are not near Montreal to attend. The Workshop is, as ever, free and open to ALL members of the MIR community.
Our dates for the Workshop will be August 22, September 5, September 19, and October 3. We hope you can join us – we’ll add more information about signups and our invited presenters soon!
Why are women still so underrepresented in science? Female scientists represent only a third of researchers globally, and things are not getting better when talking about information and communication technologies, where less than a fifth of the graduates are women. The Music Information Retrieval (MIR) community is no exception, with less than 20% female participation at ISMIR 2019, the conference of the International Society for Music Information Retrieval. Despite the efforts of organizers this year to promote diversity in the choice of the keynote and the session chairs, the gender gap is still evident when looking at the author and attendee statistics.
Much still needs to be done to bring women into the community and to reduce the gender gap. For this purpose, Women in Music Information Retrieval (WiMIR) – a group of people within ISMIR – has put together ideas and started a number of diversity and inclusion initiatives. The goal is to build a community around women in the field and create a network able to support young researchers through grants, workshops and mentoring from senior scientists. Thanks to the WiMIR grants, ISMIR 2019 female participation had a 5% increase!
I am starting a series of interviews with female researchers in MIR to find out more about their experiences and give an insight to young female researchers who want to start a career in MIR research. The first name on my list is Dr. Blair Kaneshiro, who is a researcher at the Center for Computer Research in Music and Acoustics (CCRMA) at Stanford University, a member of the ISMIR Board, and one of the WiMIR organizers.
Whereabouts did you study?
I’m from the United States and completed all of my schooling at Stanford University. My undergraduate degree was in Music. I later returned for graduate school, completing an MA in Music, Science, and Technology; MS in Electrical Engineering; and finally a PhD in Computer-Based Music Theory and Acoustics.
When did you first know you wanted to pursue a career in science?
A career interest in science for me did not develop until graduate school. I had been working at an education company at Stanford called the Education Program for Gifted Youth (EPGY) after my undergraduate degree when Patrick Suppes – co-founder of EPGY and Emeritus Professor of Philosophy, among many other remarkable things – suggested I pursue graduate work. Dr. Suppes was actively running a neuroscience lab at that time, and offered to fund my Master’s through a research assistantship in his lab. Once there, I began to see how neuroscience and engineering could be employed to address fundamental questions about perception and music – questions I feel, three graduate degrees and over a decade later, I’ve still barely begun to answer! Sadly, Dr. Suppes passed away in 2014. I am forever grateful for his support and mentorship, and for encouraging me to pursue science in the first place. I try to pay forward what I have learned from him, as both a scientist and a mentor.
How did you first become interested in MIR?
For the first few years of my graduate study, I didn’t feel I had a ‘home’ research community as my work was falling somewhere between perception / cognition and machine learning / brain decoding. In 2011, my classmate suggested I attend the ISMIR conference. I was immediately drawn in by the field of MIR, not only by the research topics – which to me were combining computation, perception, and application in exciting ways – but also by the community itself, which was welcoming and open to new ideas and approaches.
What are you currently working on?
These days I have two main research tracks. The first is electroencephalography (EEG) research, where I continue to use the decoding techniques I first encountered in the Suppes lab, and related approaches, to study proximity spaces of neural responses. I’m also working with analysis techniques that enable us to study neural processing of ‘natural’ stimuli (e.g., real-world music). My second area of research focuses on how social practices around music selection and consumption are supported (or not) by present-day technologies such as streaming platforms – more in the direction of user research. In all, I really enjoy working with a variety of collaborators, study designs, data modalities, and analysis techniques to gain a better understanding of how we humans engage with music.
Are there still gender imbalances in your research environment and in the MIR community? If yes, how can we overcome that?
Yes, definitely! In fact, I was relatively unfazed by the low number of women at the first ISMIR conference I attended, if only because it was what I was used to from being in engineering classes. But there is definitely an imbalance. In terms of overcoming this challenge, the MIR community stands out in its willingness to take action. Community members (women and men) have signed on to mentor women, organize initiatives, lead Workshop groups, and serve on conference committees; and sponsors contribute extra travel funds specifically for women to attend the ISMIR conference. While there is still a lot of progress to be made, the fact that the community as a whole is already on board makes a huge difference in moving forward.
Which changes, if any, are needed in the MIR community to be more attractive to women?
Building a more diverse research community will take time. It also requires support at multiple career stages, from recruiting women into the field to retaining those who are here. We are already starting to see positive outcomes from community initiatives. For instance, the WiMIR Mentoring Program, WiMIR Workshop, and WiMIR Travel Awards can serve as entry points for newcomers to the field, and we have seen cases of WiMIR Mentoring participants pivoting into MIR-related jobs or graduate programs, and of newcomers attending ISMIR for the first time through WiMIR Travel Awards and returning in future years as full-paper authors. But how exactly does one progress from attendee to author? And how do we keep women in the field for the long term – what are the challenges there? Will our progress in recent years translate to long-term change? I hope we can all continue to examine these challenges, understand underlying factors and biases, and take steps – on a community level and in our immediate working environments – to recruit and retain more women in MIR.
What advice would you give to young girls who are considering a career in science?
I recommend taking a broad look at what types of scientific fields are out there. Maybe you have a picture in your mind of what it looks like to ‘do science’. In fact, science spans a vast array of disciplines – even music! Also, it’s important to recognize that there is no one way to be a scientist, and no one way to look or act as a scientist. I highly recommend browsing the profiles at #UniqueScientists to see just how diverse the people, topics, and career paths in science are today.
Giorgia Cantisani graduated with a Master’s Degree in Biomedical Engineering from the Polytechnic University of Turin and, since September 2018, is a PhD student at Télécom Paris in France. Her research interests range from music information retrieval (MIR) to neuroscience. In particular, she is interested in the analysis of brain responses to music and how these can be used to guide and inform MIR tasks.
Women in Music Information Retrieval (WiMIR) is very excited to introduce the WiMIR Editorial Team! In this blog post, we introduce the team members, our goals, and most importantly, how you can collaborate with us!
Meet the 2020-2022 Team
Alia Morsi, Masters student in Sound and Music Computing at Universitat Pompeu Fabra, Spain. My current research interests are MIR for enhancing music education.
Giorgia Cantisani, PhD student in Audio Signal Processing at Télécom Paris, France. My current research interest is MIR, brain responses to music and source separation.
Kyungyun Lee, Masters student in Music and Audio Computing Lab at KAIST, South Korea. My current research interest is music performance generation and HCI.
Rui Guo, PhD Student in Music at University of Sussex, England. My current research is to generate symbolic music compositions by AI.
Saul Ivan Rivas Vega, Masters student at IIMAS of the UNAM, México. I’m interested in MIR using AI models as tools for performers and to generate autonomous compositions.
The WiMIR Editorial team is supervised by Blair Kaneshiro, PhD (Stanford University).
What are the team objectives? In a nutshell, our work revolves around:
Maintaining the WiMIR blog
Coordinating and managing community contributions, and presenting them in a meaningful way.
We strongly emphasize the second point, because in addition to posts that announce WiMIR initiatives, we would like to expand the blog’s content to include other types of interesting material, and in that we strongly encourage community involvement!
Call for Contributions
Suggestions include the following ideas:
Reflection on a topic related to WiMIR mentoring (e.g., networking, work-life balance, jobs in industry or academia, applying to graduate school, dealing with sexism).
Reflection on personal experiences as a woman, underrepresented minority, or ally in MIR.
Review of a book, article, paper, or other resource related to mentoring or otherwise supporting diversity and inclusion.
Review of a conference, workshop, meetup, concert, or other event that is relevant to the aims of WiMIR.
Review of research, codebase, creative work, or other contribution by a woman or other underrepresented minority in MIR or related field.
Tips for newcomers to MIR as a research field and/or community.
But also, feel free to suggest more topics! Blog posts are contributions from community members, so this is where everyone can help.
Also, your support is important! So we hope you enjoy reading our posts and also sharing them with others who might be interested. In addition to the community driven blog posts, we will be publishing a set of interviews so stay tuned!
We thank our generous mentors from Europe, North and South America, and Asia for dedicating their time to this program:
Ajay Srinivasamurthy, Amazon.com, Inc, USA
Alexander Lerch, Georgia Institute of Technology, USA
Amruta Vidwans, Pindrop, USA
Andrew Demetriou, Delft University of Technology, Netherlands
Andrew McLeod, EPFL, Switzerland
Andy Sarroff, iZotope, USA
Anja Volk, Utrecht University, Netherlands
Anne McKinnon, The Boolean, USA
Ashis Pati, Georgia Tech, USA
Ashley Burgoyne, University of Amsterdam, Netherlands
Ayinde Alakoye, nēdl, USA
Bertrand Scherrer, LANDR Audio Inc., Canada
Blair Kaneshiro, Stanford University, USA
Brian McFee, New York University, USA
Chih-Wei Wu, Netflix, Inc., USA
Chris Donahue, Stanford, USA
Christine Bauer, Johannes Kepler University Linz, Austria
Christopher Tralie, Ursinus College, USA
Dan Ellis, Google, USA
Delia Fano Yela, Chordify, Netherlands
Doğaç Başaran, Audible Magic, USA
Douglas Eck, Google, USA
Douglas Turnbull, Ithaca College, USA
Elio Quinton, Universal Music Group, USA
Emilia Gomez, Joint Research Centre and Universitat Pompeu Fabra, Spain
Emilia Parada-Cabaleiro, Complutense University of Madrid, Spain
Emmanouil Benetos, Queen Mary University of London, United Kingdom
Eric Nichols, Zillow Group, USA
Erling Wold, Audible Magic, USA
Fabien Gouyon, Pandora, USA
Furkan Yesiler, Music Technology Group, UPF, Spain
Gabriel Meseguer Brocal, IRCAM, France
Gabriel Vigliensoni, McGill University/Goldsmiths, Canada
Gaël Richard, Telecom Paris, Institut Polytechnique de Paris, France
Geoffroy Peeters, Telecom Paris, France
Helena Cuesta, Universitat Pompeu Fabra, Spain
Ichiro Fujinaga, McGill University, Canada
Ioannis Petros Samiotis, Delft University of Technology (TU Delft), Netherlands
Jan Van Balen, University of Antwerp, Belgium
Jin Ha Lee, University of Washington, USA
Joe Cheri Ross, Linkedin, USA
Johan Pauwels, Queen Mary University of London, United Kingdom
Johanna Devaney, Brooklyn College, USA
Jon Gillick, UC Berkeley, USA
Jonathan Driedger, Chordify, Netherlands
Jordan Smith, Tik Tok, USA
Juhan Nam, KAIST/NYU, South Korea/USA
Justin Salamon, Adobe, USA
Magdalena Fuentes, NYU, USA
Makarand Velankar, MKSSS’S Cummins College of Engineering, India
Marcelo Caetano, McGill University, Canada
Marcelo Queiroz, University of São Paulo, Brazil
Marius Miron, European Commission, Spain
Mark Cartwright, New York University, USA
Matthew Davies, INESC TEC / University of Coimbra, Portugal
Matthew McCallum, Pandora Media, USA
Maximos Kaliakatsos-Papakostas, Athena Research and Innovation Centre, Greece
Meinard Mueller, International Audio Laboratories Erlangen, Germany
Mi Tian, Spotify, USA
Oriol Nieto, Pandora, USA
Prem Seetharaman, Northwestern University, USA
Rachel Bittner, Spotify, USA
Rafael Valle, NVIDIA and UC Berkeley, USA
Robin Laney, Open University, United Kingdom
Ryan Groves, Arcona, Germany
Sebastian Rosenzweig, International Audio Laboratories Erlangen, Germany
Sertan Şentürk, Kobalt Music Group, USA
Siddharth Gururani, Georgia Institute of Technology, USA
So Yeon Park, Stanford, USA
Srikanth Cherla, Moodagent, Denmark
Stefan Sullivan, Smule, USA
Tejaswinee Kelkar, University of Oslo, Norway
Théis Bazin, Sony CSL Paris, France
Tom Collins, University of York, USA
Vincent Lostanlen, Cornell Lab of Ornithology, USA
Zafar Rafii, Gracenote, USA
Zhiyao Duan, University of Rochester, USA
Our mentors and mentees are coming from all around the world, as you can see from the map.
They range from university faculty members to industry employees, and represent a diverse field of interests and backgrounds, such as signal processing, machine learning, computer science, information technology, ethnomusicology, computational musicology, music theory, music composition, music perception and cognition, music performance, music and mathematics, neuroscience, library science, music education, multimedia research, sound design and data analytics, among others while exploring new and interesting areas in music research.
We’re very pleased, again, to write to tell you that the WiMIR 2nd Annual Workshop was a resounding success!
Like last year, we wanted the WiMIR Workshop to build upon the MIR community’s already strong support for diversity and inclusion. We had three main aims:
Encourage the discussion of proposed or unfinished work.
Amplify the scientific efforts of women in the field.
Create additional space for networking.
Thanks to support from Spotify and Pandora, we were able to offer the WiMIR Workshop as a free event, open to everyone! The Workshop took place as a satellite event of ISMIR2019, at the amazing Aula Conference Centre in Delft. We had just over 100 people attend, with twelve Project Guides. Our topics ranged from doing research-at-scale to ethical frameworks for music technology, analysis of computer-generated folk music, and more. We also had amazing catering and eight kinds of flavoured water – thanks Delft!
The day was a bit more streamlined compared to last year – we focused more on group work with the Project Guides, and did not do a poster session. We had a similar lunch session to last year, informally structured around topics like “Dealing with Sexism” and “Surviving Grad School”. The lunch provided attendees with an opportunity to connect with new people and learn about topics that members in the field (especially those who are not women) don’t often discuss.
After lunch, the project groups started a deeper dive into their topic areas, with an eye to present at 4:30 pm and then conduct an hour-long science fair at 5:00. You can see a Twitter thread of all the talks here, and look at the final presentation slides here!
It was a lot of fun, and we got lots of excellent feedback – we’re looking forward to next year.
And, of course, the tweets:
Thanks so much to everyone who helped out: The Aula Conference Centre staff, the ISMIR2019 volunteers & General Chairs, the ISMIR Board, WIMIR leadership and TU Delft!