Home Community Deep neural networks show promise as models of human hearing

Deep neural networks show promise as models of human hearing

Deep neural networks show promise as models of human hearing

Computational models that mimic the structure and performance of the human auditory system could help researchers design higher hearing aids, cochlear implants, and brain-machine interfaces. A brand new study from MIT has found that modern computational models derived from machine learning are moving closer to this goal.

In the biggest study yet of deep neural networks which have been trained to perform auditory tasks, the MIT team showed that the majority of those models generate internal representations that share properties of representations seen within the human brain when individuals are listening to the identical sounds.

The study also offers insight into best train such a model: The researchers found that models trained on auditory input including background noise more closely mimic the activation patterns of the human auditory cortex.

“What sets this study apart is it’s probably the most comprehensive comparison of those sorts of models to the auditory system to this point. The study suggests that models which can be derived from machine learning are a step in the precise direction, and it gives us some clues as to what tends to make them higher models of the brain,” says Josh McDermott, an associate professor of brain and cognitive sciences at MIT, a member of MIT’s McGovern Institute for Brain Research and Center for Brains, Minds, and Machines, and the senior creator of the study.

MIT graduate student Greta Tuckute and Jenelle Feather PhD ’22 are the lead authors of the open-access paper, which appears today in .

Models of hearing

Deep neural networks are computational models that consists of many layers of information-processing units that may be trained on huge volumes of knowledge to perform specific tasks. This kind of model has change into widely utilized in many applications, and neuroscientists have begun to explore the likelihood that these systems may also be used to explain how the human brain performs certain tasks.

“These models which can be built with machine learning are capable of mediate behaviors on a scale that basically wasn’t possible with previous varieties of models, and that has led to interest in whether or not the representations within the models might capture things which can be happening within the brain,” Tuckute says.

When a neural network is performing a task, its processing units generate activation patterns in response to every audio input it receives, comparable to a word or other sort of sound. Those model representations of the input may be in comparison with the activation patterns seen in fMRI brain scans of individuals listening to the identical input.

In 2018, McDermott and then-graduate student Alexander Kell reported that once they trained a neural network to perform auditory tasks (comparable to recognizing words from an audio signal), the inner representations generated by the model showed similarity to those seen in fMRI scans of individuals listening to the identical sounds.

Since then, these kinds of models have change into widely used, so McDermott’s research group set out to judge a bigger set of models, to see if the flexibility to approximate the neural representations seen within the human brain is a general trait of those models.

For this study, the researchers analyzed nine publicly available deep neural network models that had been trained to perform auditory tasks, and additionally they created 14 models of their very own, based on two different architectures. Most of those models were trained to perform a single task — recognizing words, identifying the speaker, recognizing environmental sounds, and identifying musical genre — while two of them were trained to perform multiple tasks.

When the researchers presented these models with natural sounds that had been used as stimuli in human fMRI experiments, they found that the inner model representations tended to exhibit similarity with those generated by the human brain. The models whose representations were most just like those seen within the brain were models that had been trained on a couple of task and had been trained on auditory input that included background noise.

“When you train models in noise, they offer higher brain predictions than when you don’t, which is intuitively reasonable because lots of real-world hearing involves hearing in noise, and that’s plausibly something the auditory system is tailored to,” Feather says.

Hierarchical processing

The brand new study also supports the concept that the human auditory cortex has some extent of hierarchical organization, wherein processing is split into stages that support distinct computational functions. As within the 2018 study, the researchers found that representations generated in earlier stages of the model most closely resemble those seen in the first auditory cortex, while representations generated in later model stages more closely resemble those generated in brain regions beyond the first cortex.

Moreover, the researchers found that models that had been trained on different tasks were higher at replicating different features of audition. For instance, models trained on a speech-related task more closely resembled speech-selective areas.

“Though the model has seen the very same training data and the architecture is similar, if you optimize for one particular task, you possibly can see that it selectively explains specific tuning properties within the brain,” Tuckute says.

McDermott’s lab now plans to utilize their findings to attempt to develop models which can be much more successful at reproducing human brain responses. Along with helping scientists learn more about how the brain could also be organized, such models may be used to assist develop higher hearing aids, cochlear implants, and brain-machine interfaces.

“A goal of our field is to find yourself with a pc model that may predict brain responses and behavior. We expect that if we’re successful in reaching that goal, it would open lots of doors,” McDermott says.

The research was funded by the National Institutes of Health, an Amazon Fellowship from the Science Hub, an International Doctoral Fellowship from the American Association of University Women, an MIT Friends of McGovern Institute Fellowship, a fellowship from the K. Lisa Yang Integrative Computational Neuroscience (ICoN) Center at MITand a Department of Energy Computational Science Graduate Fellowship.


Please enter your comment!
Please enter your name here