To make our way through the world, our brain must develop an intuitive understanding of the physical world around us, which we then use to interpret sensory information coming into the brain.
How does the brain develop that intuitive understanding? Many scientists consider that it might use a process just like what’s often known as “self-supervised learning.” This sort of machine learning, originally developed as a strategy to create more efficient models for computer vision, allows computational models to study visual scenes based solely on the similarities and differences between them, with no labels or other information.
A pair of studies from researchers on the K. Lisa Yang Integrative Computational Neuroscience (ICoN) Center at MIT offers recent evidence supporting this hypothesis. The researchers found that once they trained models often known as neural networks using a selected sort of self-supervised learning, the resulting models generated activity patterns very just like those seen within the brains of animals that were performing the identical tasks because the models.
The findings suggest that these models are capable of learn representations of the physical world that they will use to make accurate predictions about what is going to occur in that world, and that the mammalian brain could also be using the identical strategy, the researchers say.
“The theme of our work is that AI designed to assist construct higher robots finally ends up also being a framework to raised understand the brain more generally,” says Aran Nayebi, a postdoc within the ICoN Center. “We will’t say if it’s the entire brain yet, but across scales and disparate brain areas, our results appear to be suggestive of an organizing principle.”
Nayebi is the lead creator of one among the studies, co-authored with Rishi Rajalingham, a former MIT postdoc now at Meta Reality Labs, and senior authors Mehrdad Jazayeri, an associate professor of brain and cognitive sciences and a member of the McGovern Institute for Brain Research; and Robert Yang, an assistant professor of brain and cognitive sciences and an associate member of the McGovern Institute. Ila Fiete, director of the ICoN Center, a professor of brain and cognitive sciences, and an associate member of the McGovern Institute, is the senior creator of the opposite study, which was co-led by Mikail Khona, an MIT graduate student, and Rylan Schaeffer, a former senior research associate at MIT.
Each studies might be presented on the 2023 Conference on Neural Information Processing Systems (NeurIPS) in December.
Modeling the physical world
Early models of computer vision mainly relied on supervised learning. Using this approach, models are trained to categorise images which are each labeled with a reputation — cat, automobile, etc. The resulting models work well, but the sort of training requires an incredible deal of human-labeled data.
To create a more efficient alternative, in recent times researchers have turned to models built through a method often known as contrastive self-supervised learning. This sort of learning allows an algorithm to learn to categorise objects based on how similar they’re to one another, with no external labels provided.
“This can be a very powerful method because you possibly can now leverage very large modern data sets, especially videos, and really unlock their potential,” Nayebi says. “Lots of the trendy AI that you simply see now, especially within the last couple years with ChatGPT and GPT-4, is a result of coaching a self-supervised objective function on a large-scale dataset to acquire a really flexible representation.”
Most of these models, also called neural networks, consist of hundreds or tens of millions of processing units connected to one another. Each node has connections of various strengths to other nodes within the network. Because the network analyzes huge amounts of knowledge, the strengths of those connections change because the network learns to perform the specified task.
Because the model performs a selected task, the activity patterns of various units throughout the network will be measured. Each unit’s activity will be represented as a firing pattern, just like the firing patterns of neurons within the brain. Previous work from Nayebi and others has shown that self-supervised models of vision generate activity just like that seen within the visual processing system of mammalian brains.
In each of the brand new NeurIPS studies, the researchers got down to explore whether self-supervised computational models of other cognitive functions may also show similarities to the mammalian brain. Within the study led by Nayebi, the researchers trained self-supervised models to predict the longer term state of their environment across tons of of hundreds of naturalistic videos depicting on a regular basis scenarios.
“For the last decade or so, the dominant method to construct neural network models in cognitive neuroscience is to coach these networks on individual cognitive tasks. But models trained this manner rarely generalize to other tasks,” Yang says. “Here we test whether we are able to construct models for some aspect of cognition by first training on naturalistic data using self-supervised learning, then evaluating in lab settings.”
Once the model was trained, the researchers had it generalize to a task they call “Mental-Pong.” This is analogous to the video game Pong, where a player moves a paddle to hit a ball traveling across the screen. Within the Mental-Pong version, the ball disappears shortly before hitting the paddle, so the player has to estimate its trajectory with a purpose to hit the ball.
The researchers found that the model was capable of track the hidden ball’s trajectory with accuracy just like that of neurons within the mammalian brain, which had been shown in a previous study by Rajalingham and Jazayeri to simulate its trajectory — a cognitive phenomenon often known as “mental simulation.” Moreover, the neural activation patterns seen throughout the model were just like those seen within the brains of animals as they played the sport — specifically, in a component of the brain called the dorsomedial frontal cortex. No other class of computational model has been capable of match the biological data as closely as this one, the researchers say.
“There are various efforts within the machine learning community to create artificial intelligence,” Jazayeri says. “The relevance of those models to neurobiology hinges on their ability to moreover capture the inner workings of the brain. The incontrovertible fact that Aran’s model predicts neural data is admittedly necessary because it suggests that we could also be getting closer to constructing artificial systems that emulate natural intelligence.”
Navigating the world
The study led by Khona, Schaeffer, and Fiete focused on a sort of specialized neurons often known as grid cells. These cells, positioned within the entorhinal cortex, help animals to navigate, working along with place cells positioned within the hippocampus.
While place cells fire each time an animal is in a particular location, grid cells fire only when the animal is at one among the vertices of a triangular lattice. Groups of grid cells create overlapping lattices of various sizes, which allows them to encode numerous positions using a comparatively small variety of cells.
In recent studies, researchers have trained supervised neural networks to mimic grid cell function by predicting an animal’s next location based on its start line and velocity, a task often known as path integration. Nevertheless, these models hinged on access to privileged details about absolute space in any respect times — information that the animal doesn’t have.
Inspired by the striking coding properties of the multiperiodic grid-cell code for space, the MIT team trained a contrastive self-supervised model to each perform this same path integration task and represent space efficiently while doing so. For the training data, they used sequences of velocity inputs. The model learned to differentiate positions based on whether or not they were similar or different — nearby positions generated similar codes, but further positions generated more different codes.
“It’s just like training models on images, where if two images are each heads of cats, their codes must be similar, but when one is the top of a cat and one is a truck, then you definately want their codes to repel,” Khona says. “We’re taking that very same idea but applying it to spatial trajectories.”
Once the model was trained, the researchers found that the activation patterns of the nodes throughout the model formed several lattice patterns with different periods, very just like those formed by grid cells within the brain.
“What excites me about this work is that it makes connections between mathematical work on the striking information-theoretic properties of the grid cell code and the computation of path integration,” Fiete says. “While the mathematical work was analytic — what properties does the grid cell code possess? — the approach of optimizing coding efficiency through self-supervised learning and obtaining grid-like tuning is synthetic: It shows what properties may be crucial and sufficient to elucidate why the brain has grid cells.”
The research was funded by the K. Lisa Yang ICoN Center, the National Institutes of Health, the Simons Foundation, the McKnight Foundation, the McGovern Institute, and the Helen Hay Whitney Foundation.