This article draws on recent findings in neuroscience to provide a close reading of Grimes’s music video “Shinigami Eyes.” It includes interviews with a neuroscientist who specializes in multisensory integration, a vision scientist, a color grader, and a media scholar. While multisensory integration is a burgeoning field, accessible texts and reviews are lacking. There haven’t been analyses that apply the findings from the field of multisensory integration to real media objects. Drawing on neuroscience studies, this article explores the concept of Bayesian predictions (a viewer’s experience of a present event weighted against previous experiences); the ways neuroscience explains color, size and placement, and emotion work; and processes such as the inverse effectiveness (when neurons boost both lower-res and higher-res signals, with the most perceptible often for sound), super-additive multisensory effects, the attentional blink, the ventriloquist effect, congruence and incongruity, memory, distractors, sound before image and vice versa, and other features. A conversation with the color grader for “Shinigami Eyes” helps fill things in. This article emerges from a belief that modules on neuroscience and industry studies can be incorporated as components of media study courses, and that neuroscience-informed analyses can facilitate interest and engagement with close readings.
Grimes’s Music Video “Shinigami Eyes”: Neuroscience, Audiovisuality, and Color
Carol Vernallis’s monograph, Experiencing Music Video: Aesthetics and Cultural Context (CUP, 2004), is the first to articulate a theory of how music, lyrics and image can be placed in relation, as well as provide detailed analyses of individual videos. Her second, Unruly Media: YouTube, Music Video, and the New Digital Cinema (OUP, 2013), takes account of a new mediascape that is driven by intensified audiovisual relations. Her third, The Media Swirl (Duke, 4/14/23), further considers the contemporary landscape. She is co-editor of Cybermedia: Explorations in Science, Sound, and Vision and Transmedia Directors (Bloomsbury) and two Oxford Handbooks on audiovisuality. She’s adjunct professor in the department of Cultural Studies and Comparative Literature at the University of Minnesota.
Justin Gardner is an Associate Professor in Psychology at Stanford, where he heads the Gardner lab. He and his team use a combination of functional magnetic resonance imaging, computational modeling and analysis, and psychophysical measurements to study how neural activity in the human cortex creates our sense of visual perception. Of particular interest is the ways our brain mechanisms give rise to our remarkable cognitive feats such as selective attention and sensory inference. Gardner is also a member of BioX, an affiliate of Institute for Human-Centered Artificial Intelligence (HAI), and a member of Wu Tsai Neurosciences Institute.
Cesare Parise is a Senior Lecturer in Psychology at the University of Liverpool, where he heads a lab devoted to multisensory perception. His current projects include the development of a model for fly vision that can be adapted to other creatures. Parise’s background is in experimental psychology. After earning his DPhil at Oxon, he worked as a research scientist for the Max Planck Institute for Biological Cybernetics and the Center for Excellence in Cognitive Interaction Technologies. In 2016, he joined Facebook Reality Labs (now Meta Reality Labs) as a Primary Investigator in perception science, where he investigated haptics, multisensory integration, and spatial audio to develop next-generation technologies for virtual and augmented reality.
Aubrey Woodiwiss is a multi-award winning colourist and multi-disciplinary creative. His portfolio spans iconic music videos for the likes of Beyonce, Jay-Z, Rihanna, Calvin Harris, Frank Ocean and Taylor Swift, as well as high-profile commercials for Calvin Klein, Honda, Nike, Coors, and Gap. Aubrey has collaborated with directors and cinematographers such as Emil Nava, Chris Hewitt, Seb Edwards, Sam Brown, Melina Matsoukas, Scott Lyon, Andreas Nilsson, Marcus Söderlund, Jamie Rafn, the late Ringan Ledwidge and Jonas Akurland. He co-creative directed the immersive mixed media art exhibition and initiative, ‘mentl’ with Pulse Films director Ben Newman and producer Craig Newman, and runs his own creative studio.
Carol Vernallis, Justin Gardner, Cesare Parise, Aubrey Woodiwiss; Grimes’s Music Video “Shinigami Eyes”: Neuroscience, Audiovisuality, and Color. Journal of Popular Music Studies 1 September 2023; 35 (3): 32–60. doi: https://doi.org/10.1525/jpms.2023.35.3.32
Download citation file:
University of California Press