I study how visual and multimodal media communicate meaning. A particular focus is the production and reception of time-based media such as film and television. My approach is empirical, pairing large corpora of images, video, sound, and text with experimental data. I co-direct the Distant Viewing Lab, where we build new computational methods, open-source software, and open datasets for this work. The research draws on and contributes to media studies, linguistics, cognitive science, and statistics.
How do moving images communicate? I measure shot length, framing, sound, and editing across large collections of film and television and relate them to genre, industry, and national context, testing claims from film studies and cognitive film theory.
What can we infer when the corpus is the whole population, and when does a finding about culture replicate? I work on inference and replication for cultural data.
With linguists and psychologists, I study how people produce and perceive language and faces: keystroke logging of second-language writing, automatic scoring of learner speech, cognitive-linguistic approaches to language teaching, and meta-analysis in face perception.
What does it mean to treat images as data? I develop the theory and methods of distant viewing and apply them to historical photographs and other visual archives, building search, annotation, and detection tools suited to how those archives were made.