Lay listeners are reliable judges when evaluating pitch accuracy of occasional singers, suggesting that enculturation and laypersons’ perceptual abilities are sufficient to judge “simple” music material adequately. However, the definition of pitch accuracy in operatic performances is much more complex than in melodies performed by occasional singers. Furthermore, because listening to operatic performances is not a common activity, laypersons‘ experience with this complicated acoustic signal is more limited. To address the question of music expertise in evaluating operatic singing voices, listeners without music training were compared with the music experts examined in a recent study (Larrouy-Maestri, Magis, & Morsomme, 2014a) and their ratings were modeled with regard to underlying acoustic variables of pitch accuracy. As expected, some participants lacked test-retest reliability in their judgments. However, listeners who used a consistent strategy relied on a definition of pitch accuracy that appears to overlap with the quantitative criteria used by music experts. Besides clarifying the role of music expertise in the evaluation of melodies, our findings show robust perceptual abilities in laypersons when listening to complex signals such as operatic performances.

The objective analysis of Western operatic singing voices indicates that professional singers can be particularly “out of tune.” This study aims to better understand the evaluation of operatic voices, which have particularly complex acoustical signals. Twenty-two music experts were asked to evaluate the vocal pitch accuracy of 14 sung performances with a pairwise comparison paradigm, in a test and a retest. In addition to the objective measurement of pitch accuracy (pitch interval deviation), several performance parameters (average tempo, fundamental frequency of the starting note) and quality parameters (energy distribution, vibrato rate and extent) were observed and compared to the judges’ perceptual rating. The results show high intra and interjudge reliability when rating the pitch accuracy of operatic singing voices. Surprisingly, all the parameters were significantly related to the ratings and explain 78.8% of the variability of the judges’ rating. The pitch accuracy evaluation of operatic voices is thus not based exclusively on the precision of performed music intervals but on a complex combination of performance and quality parameters.