Home Community When computer vision works more like a brain, it sees more like people do

When computer vision works more like a brain, it sees more like people do

When computer vision works more like a brain, it sees more like people do

From cameras to self-driving cars, a lot of today’s technologies rely on artificial intelligence to extract meaning from visual information. Today’s AI technology has artificial neural networks at its core, and more often than not we are able to trust these AI computer vision systems to see things the best way we do — but sometimes they falter. In line with MIT and IBM research scientists, one technique to improve computer vision is to instruct the factitious neural networks that they depend on to deliberately mimic the best way the brain’s biological neural network processes visual images.

Researchers led by MIT Professor James DiCarlo, the director of MIT’s Quest for Intelligence and member of the MIT-IBM Watson AI Lab, have made a pc vision model more robust by training it to work like an element of the brain that humans and other primates depend on for object recognition. This May, on the International Conference on Learning Representations, the team reported that once they trained a man-made neural network using neural activity patterns within the brain’s inferior temporal (IT) cortex, the factitious neural network was more robustly capable of discover objects in images than a model that lacked that neural training. And the model’s interpretations of images more closely matched what humans saw, even when images included minor distortions that made the duty tougher.

Comparing neural circuits

Lots of the factitious neural networks used for computer vision already resemble the multilayered brain circuits that process visual information in humans and other primates. Just like the brain, they use neuron-like units that work together to process information. As they’re trained for a specific task, these layered components collectively and progressively process the visual information to finish the duty — determining, for instance, that a picture depicts a bear or a automotive or a tree.

DiCarlo and others previously found that when such deep-learning computer vision systems establish efficient ways to unravel visual problems, they find yourself with artificial circuits that work similarly to the neural circuits that process visual information in our own brains. That’s, they become surprisingly good scientific models of the neural mechanisms underlying primate and human vision.

That resemblance helps neuroscientists deepen their understanding of the brain. By demonstrating ways visual information might be processed to make sense of images, computational models suggest hypotheses about how the brain might accomplish the identical task. As developers proceed to refine computer vision models, neuroscientists have found latest ideas to explore in their very own work.

“As vision systems recover at performing in the actual world, a few of them become more human-like of their internal processing. That’s useful from an understanding-biology perspective,” says DiCarlo, who can also be a professor of brain and cognitive sciences and an investigator on the McGovern Institute for Brain Research.

Engineering a more brain-like AI

While their potential is promising, computer vision systems usually are not yet perfect models of human vision. DiCarlo suspected one technique to improve computer vision could also be to include specific brain-like features into these models.

To check this concept, he and his collaborators built a pc vision model using neural data previously collected from vision-processing neurons within the monkey IT cortex — a key a part of the primate ventral visual pathway involved in the popularity of objects — while the animals viewed various images. More specifically, Joel Dapello, a Harvard University graduate student and former MIT-IBM Watson AI Lab intern; and Kohitij Kar, assistant professor and Canada Research Chair (Visual Neuroscience) at York University and visiting scientist at MIT; in collaboration with David Cox, IBM Research’s vp for AI models and IBM director of the MIT-IBM Watson AI Lab; and other researchers at IBM Research and MIT asked a man-made neural network to emulate the behavior of those primate vision-processing neurons while the network learned to discover objects in an ordinary computer vision task.

“In effect, we said to the network, ‘please solve this standard computer vision task, but please also make the function of one in all your inside simulated “neural” layers be as similar as possible to the function of the corresponding biological neural layer,’” DiCarlo explains. “We asked it to do each of those things as best it could.” This forced the factitious neural circuits to search out a special technique to process visual information than the usual, computer vision approach, he says.

After training the factitious model with biological data, DiCarlo’s team compared its activity to a similarly-sized neural network model trained without neural data, using the usual approach for computer vision. They found that the brand new, biologically informed model IT layer was — as instructed — a greater match for IT neural data.  That’s, for each image tested, the population of artificial IT neurons within the model responded more similarly to the corresponding population of biological IT neurons.

The researchers also found that the model IT was also a greater match to IT neural data collected from one other monkey, despite the fact that the model had never seen data from that animal, and even when that comparison was evaluated on that monkey’s IT responses to latest images. This indicated that the team’s latest, “neurally aligned” computer model could also be an improved model of the neurobiological function of the primate IT cortex — an interesting finding, on condition that it was previously unknown whether the quantity of neural data that might be currently collected from the primate visual system is able to directly guiding model development.

With their latest computer model in hand, the team asked whether the “IT neural alignment” procedure also results in any changes in the general behavioral performance of the model. Indeed, they found that the neurally-aligned model was more human-like in its behavior — it tended to achieve appropriately categorizing objects in images for which humans also succeed, and it tended to fail when humans also fail.

Adversarial attacks

The team also found that the neurally aligned model was more immune to “adversarial attacks” that developers use to check computer vision and AI systems. In computer vision, adversarial attacks introduce small distortions into images that are supposed to mislead a man-made neural network.

“Say that you’ve a picture that the model identifies as a cat. Because you’ve the knowledge of the inner workings of the model, you’ll be able to then design very small changes within the image in order that the model suddenly thinks it’s now not a cat,” DiCarlo explains.

These minor distortions don’t typically idiot humans, but computer vision models struggle with these alterations. A one that looks on the subtly distorted cat still reliably and robustly reports that it’s a cat. But standard computer vision models usually tend to mistake the cat for a dog, or perhaps a tree.

“There should be some internal differences in the best way our brains process images that result in our vision being more immune to those sorts of attacks,” DiCarlo says. And indeed, the team found that once they made their model more neurally aligned, it became more robust, appropriately identifying more images within the face of adversarial attacks. The model could still be fooled by stronger “attacks,” but so can people, DiCarlo says. His team is now exploring the boundaries of adversarial robustness in humans.

A number of years ago, DiCarlo’s team found they may also improve a model’s resistance to adversarial attacks by designing the primary layer of the factitious network to emulate the early visual processing layer within the brain. One key next step is to mix such approaches — making latest models which can be concurrently neurally aligned at multiple visual processing layers.

The brand new work is further evidence that an exchange of ideas between neuroscience and computer science can drive progress in each fields. “Everybody gets something out of the exciting virtuous cycle between natural/biological intelligence and artificial intelligence,” DiCarlo says. “On this case, computer vision and AI researchers get latest ways to realize robustness, and neuroscientists and cognitive scientists get more accurate mechanistic models of human vision.”

This work was supported by the MIT-IBM Watson AI Lab, Semiconductor Research Corporation, the U.S. Defense Research Projects Agency, the MIT Shoemaker Fellowship, U.S. Office of Naval Research, the Simons Foundation, and Canada Research Chair Program.


Please enter your comment!
Please enter your name here