Mixture of Gaussian process experts for predicting sung melodic contour with expressive dynamic fluctuations

We present a generative model for predicting the sung melodic contour, i.e., F₀ contour, with expressive dynamic fluctuations, such as vibrato and portamento, for a given musical score. Although several studies have attempted to characterize such fluctuations, no systematic method has been developed for generating the F₀ contour with them in connection with musical notes. In our model, the relationship between a musical note sequence and F₀ contour is directly learned by a mixture of Gaussian process experts. This approach allows us to automatically characterize the fluctuations by utilizing the kernel function for each Gaussian process expert and predict the F₀ contour for an arbitrary musical note sequence. Experimental results show that our model can better predict the F₀ contour than a baseline method can. Additionally, we discuss the effective musical contexts and the amount of training data for the prediction.