Volume 16(3), 2020-11, 310—347

Exploring relationships between effort, motion, and sound in new musical instruments

Çağrı Erdem
RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion
University of Oslo
Norway

Qichao Lan
RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion
University of Oslo
Norway

Alexander Refsum Jensenius
RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion
University of Oslo
Norway

We investigated how the action–sound relationships found in electric guitar performance can be used in the design of new instruments. Thirty-one trained guitarists performed a set of basic sound-producing actions (impulsive, sustained, and iterative) and free improvisations on an electric guitar. We performed a statistical analysis of the muscle activation data (EMG) and audio recordings from the experiment. Then we trained a long short-term memory network with nine different configurations to map EMG signal to sound. We found that the preliminary models were able to predict audio energy features of free improvisations on the guitar, based on the dataset of raw EMG from the basic sound-producing actions. The results provide evidence of similarities between body motion and sound in music performance, compatible with embodied music cognition theories. They also show the potential of using machine learning on recorded performance data in the design of new musical instruments.

Peer review seal