4.6 Article

Reconstructing Speech from Human Auditory Cortex


Volume 10, Issue 1, Pages -


DOI: 10.1371/journal.pbio.1001251




  1. [NS21135]
  2. [PO4813]
  3. [NS40596]
  4. [K99NS065120]
  5. NATIONAL INSTITUTE OF NEUROLOGICAL DISORDERS AND STROKE [R00NS065120, R01NS040596, K99NS065120, R01NS021135, R37NS021135, R56NS021135] Funding Source: NIH RePORTER

Ask authors/readers for more resources

How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstructed from population neural activity. We found that slow and intermediate temporal fluctuations, such as those corresponding to syllable rate, were accurately reconstructed using a linear model based on the auditory spectrogram. However, reconstruction of fast temporal fluctuations, such as syllable onsets and offsets, required a nonlinear sound representation based on temporal modulation energy. Reconstruction accuracy was highest within the range of spectro-temporal fluctuations that have been found to be critical for speech intelligibility. The decoded speech representations allowed readout and identification of individual words directly from brain activity during single trial sound presentations. These findings reveal neural encoding mechanisms of speech acoustic parameters in higher order human auditory cortex.


I am an author on this paper
Click your name to claim this paper and add it to your profile.


Primary Rating

Not enough ratings

Secondary Ratings

Scientific rigor
Rate this paper


Letter Clinical Neurology

Reply: Interactions of interictal epileptic discharges with sleep slow waves and spindles

Prawesh Dahal, Naureen Ghani, Adeen Flinker, Patricia Dugan, Daniel Friedman, Werner Doyle, Orrin Devinsky, Dion Khodagholy, Jennifer N. Gelinas

BRAIN (2020)

Article Neurosciences

Streaming of Repeated Noise in Primary and Secondary Fields of Auditory Cortex

Daniela Saderi, Brad N. Buran, Stephen David


Article Neurosciences

Task Engagement Improves Neural Discriminability in the Auditory Midbrain of the Marmoset Monkey

Luke A. Shaheen, Sean J. Slee, Stephen V. David

Summary: The study found that task engagement can modulate sound coding in the auditory midbrain, with little effect on the central IC but significant modulation in noncentral areas. This led to enhanced responses for target tones compared to distractor noise, increasing neural discriminability between the two.


Article Biology

Dissociation of task engagement and arousal effects in auditory cortex and midbrain

Daniela Saderi, Zachary P. Schwartz, Charles R. Heller, Jacob R. Pennington, Stephen David

Summary: Generalized arousal and task engagement have separate effects on auditory neural processing, with pupil size showing a more prominent effect in the inferior colliculus. Task engagement is positively correlated with pupil size, indicating that some effects attributed to task engagement may actually be due to pupil size fluctuations.

ELIFE (2021)

Article Multidisciplinary Sciences

Learning hierarchical sequence representations across human cortex and hippocampus

Simon Henin, Nicholas B. Turk-Browne, Daniel Friedman, Anli Liu, Patricia Dugan, Adeen Flinker, Werner Doyle, Orrin Devinsky, Lucia Melloni

Summary: The brain can track regularities in sensory input sequences within minutes, with early processing focusing on lower-level features and learned units, and later processing focusing solely on learned units. Learning quickly shapes neural representations, with a complexity gradient from early brain areas encoding transitional probability to associative regions and the hippocampus encoding ordinal position and identity of units.


Article Clinical Neurology

Spatiotemporal dynamics between interictal epileptiform discharges and ripples during associative memory processing

Simon Henin, Anita Shankar, Helen Borges, Adeen Flinker, Werner Doyle, Daniel Friedman, Orrin Devinsky, Gyorgy Buzsaki, Anli Liu

Summary: The spatiotemporal characteristics of cortical high-gamma activity, hippocampal ripple activity, and interictal epileptiform discharges have a significant impact on memory performance during an associative memory task, particularly in the hippocampal region. Interictal epileptiform discharges may impair associative memory by disrupting physiological activity, making them a promising therapeutic target for memory remediation in patients with epilepsy.

BRAIN (2021)

Article Neurosciences

Spatial-Temporal Functional Mapping Combined With Cortico-Cortical Evoked Potentials in Predicting Cortical Stimulation Results

Yujing Wang, Mark A. Hays, Christopher Coogan, Joon Y. Kang, Adeen Flinker, Ravindra Arya, Anna Korzeniewska, Nathan E. Crone

Summary: Functional human brain mapping is commonly performed during invasive monitoring with intracranial electroencephalographic (iEEG) electrodes prior to resective surgery for drug-resistant epilepsy. The current gold standard, electrocortical stimulation mapping (ESM), is time-consuming and sometimes elicits pain. Passive iEEG spatial-temporal functional mapping (STFM) has emerged as a potential alternative to ESM, but there is less correspondence between STFM and ESM maps of language function compared to maps of motor function. The study suggests that incongruities between ESM and STFM may arise in part from propagation of stimulation effects to a broader network of cortical language sites, indicating the need for further research with larger patient populations.


Review Neurosciences

Gender bias in academia: A lifetime problem that needs solutions

Anais Llorens, Athina Tzovara, Ludovic Bellier, Ilina Bhaya-Grossman, Aurelie Bidet-Caulet, William K. Chang, Zachariah R. Cross, Rosa Dominguez-Faus, Adeen Flinker, Yvonne Fonken, Mark A. Gorenstein, Chris Holdgraf, Colin W. Hoy, Maria Ivanova, Richard T. Jimenez, Soyeon Jun, Julia W. Y. Kam, Celeste Kidd, Enitan Marcelle, Deborah Marciano, Stephanie Martin, Nicholas E. Myers, Karita Ojala, Anat Perry, Pedro Pinheiro-Chagas, Stephanie K. Ries, Ignacio Saez, Ivan Skelin, Katarina Slama, Brooke Staveland, Danielle S. Bassett, Elizabeth A. Buffalo, Adrienne L. Fairhall, Nancy J. Kopell, Laura J. Kray, Jack J. Lin, Anna C. Nobre, Dylan Riley, Anne-Kristin Solbakk, Joni D. Wallis, Xiao-Jing Wang, Shlomit Yuval-Greenberg, Sabine Kastner, Robert T. Knight, Nina F. Dronkers

Summary: Despite increasing awareness and initiatives to address gender inequality in academia, progress is slow and gender bias continues to negatively impact underrepresented groups in science. Gender bias is not a singular issue, but a collection of distinct problems that can be addressed through collaborative efforts from individuals, academic institutions, and society.

NEURON (2021)

Article Biochemistry & Molecular Biology

Neuronal selectivity to complex vocalization features emerges in the superficial layers of primary auditory cortex

Pilar Montes-Lourido, Manaswini Kar, Stephen David, Srivatsun Sadagopan

Summary: Neural responses in the auditory processing hierarchy show a transition from nonselective to call-selective responses, with A1 L2/3 neurons exhibiting high call selectivity and responding to specific call features. Information is distributed differently in A1 L4 and A1 L2/3, with individual neurons in A1 L2/3 conveying high levels of stimulus-specific information. This transformation in neural representation suggests the emergence of feature-based representation of calls in A1 L2/3 and sets the stage for further mechanistic studies.


Article Neurosciences

Short-term Effects of Vagus Nerve Stimulation on Learning and Evoked Activity in Auditory Cortex

Jesyin Lai, Stephen V. David

Summary: Chronic vagus nerve stimulation (VNS) has been found to facilitate learning, but the effects of acute VNS on neural coding and behavior are still not well understood. This study showed that when tones were paired with VNS, animals consistently learned the new association within 2 days, but when VNS occurred randomly between trials, learning within 2 days was unreliable. Additionally, VNS induced brief dilation in the pupil, and the size of this change predicted the magnitude of persistent changes in the neural response.

ENEURO (2021)

Article Multidisciplinary Sciences

A dataset of mentorship in bioscience with semantic and demographic estimations

Qing Ke, Lizhen Liang, Ying Ding, Stephen V. David, Daniel E. Acuna

Summary: This article describes a crowdsourced dataset on mentorship in science, which includes 743,176 mentorship relationships among 738,989 scientists primarily in biosciences. The dataset enriches the Academic Family Tree project by adding publication data and semantic representations of research, and provides estimations on gender and race. Extensive validations of the dataset cover mainly neuroscience and biomedical sciences.


Article Biochemistry & Molecular Biology

Impact of gender on the formation and outcome of formal mentoring relationships in the life sciences

Leah P. Schwartz, Jean F. Lienard, Stephen David

Summary: Despite the increasing representation of women in graduate training programs, many of them still leave academic research without obtaining independent positions to train the next generation of researchers. The gender of both the student and the mentor is associated with differences in the students' continuation to positions that allow formal academic mentorship. The effect of mentor gender on this trend can be partially explained by gender disparities in social and financial resources available to mentors.


Article Biochemical Research Methods

A convolutional neural network provides a generalizable model of natural sound coding by neural populations in auditory cortex

Jacob R. Pennington, Stephen V. David

Summary: This study used convolutional neural networks (CNNs) to model the functional relationship between natural sounds and the activity of neurons in auditory cortex. CNNs outperformed previous models of auditory coding and could be easily trained on new sets of neurons. This finding suggests that CNN models capture the complete representational space across neurons in the auditory cortex, which is useful for improving signal processing algorithms.


Article Neurosciences

Effect of Reverberation on Neural Responses to Natural Speech in Rabbit Auditory Midbrain: No Evidence for a Neural Dereverberation Mechanism

Oded Barzelay, Stephen David, Bertrand Delgutte

Summary: Reverberation, which is commonly observed in everyday acoustic environments, can degrade binaural cues and envelope modulations and affect speech perception. However, both humans and animals can accurately perceive reverberant stimuli in most daily situations. In this study, neural responses to reverberant and non-reverberant speech stimuli were recorded in the inferior colliculus of rabbits, and it was found that high-quality spectrogram reconstruction can be obtained in moderate reverberation but deteriorates in severe reverberation.

ENEURO (2023)

No Data Available