We study the computational principles of the visual brain by combining large-scale human neuroimaging with deep neural network models. Below are the primary projects currently led by members of the lab.
We observe that diverse designs of neural networks all yield similarly good models of the human visual cortex. This suggests that the subset of model representations that align with cortical activity is general-purpose and can be learned independent of the task or architectural constraints on these models. By comparing the representations of widely varied models, we provide evidence that the universality of a feature among many neural networks is a strong indicator of its presence in human cortical representation.
How much feedback does a network need before its representations line up with the human visual system? Far less than standard training provides: an extremely coarse signal is enough. We are now turning the same question on the modelling process itself. Long-running agents propose simple, interpretable computational models of visual cortex, score them by how well they predict neural responses in different brain regions, and revise them. The search is open-ended and aimed at the underlying computation, not at fitting a particular dataset or scaling up training.
Deep neural networks (DNNs) are the leading computational models of visual cortex but are trained using biologically implausible backpropagation. We are developing an algorithm for building a hierarchy of visual features using only local unsupervised learning, without backpropagation. Our work identifies a new approach for learning a visual hierarchy consistent with principles of learning in biology, requires no labels or tasks, and may account for a large fraction of visual cortex representations.
Keynote tutorial · CCN 2023, Oxford
What computational tools can we use to study neural representations in high dimensions? Five notebooks, which you can read on the site or run in Google Colab, cover PCA, exploring neural data, dealing with noise, comparing representations, and analyzing neural networks.
Led by former members of the lab (see Alumni).
Cortical responses to natural images are unexpectedly high-dimensional, even in high-level visual regions, and these dimensions are shared across people.
Brain-relevant dimensions emerge in untrained convolutional networks from the statistics of natural images alone, without supervised learning.
Even low-variance dimensions of cortical activity matter for vision, and they capture reliable differences in how individuals experience the same scenes.
Social information in videos is represented increasingly abstractly along a lateral visual stream, from early visual cortex and MT up to the superior temporal sulcus.
Tools from statistical physics and high-dimensional geometry explain what makes one deep network a better model of ventral visual cortex than another.
