MF3D Research

Visual stimuli generated by the MF3D project are being actively used by neuroscience research labs across the world. Below are a subset of the published articles and conference abstracts reporting data from experiments using this freely available scientific resource.

Publications

Khandhadia AP, Murphy AP, Esch EM, Koyano KW, Leopold DA (2023). Encoding of 3D physical dimensions by face-selective cortical neurons. PNAS

Khandhadia AP, Murphy AP, Romanksi LM, Bizley JK, Leopold DA (2021). Audiovisual Integration in Macaque Face Patch Neurons. Current Biology

Taubert J, Japee S, Murphy AP, Tardiff CT, Koele EA, Kumar S, Leopold DA, & Ungerleider LG (2020). Parallel processing of facial expression and head orientation in the macaque brain. J.Neurosci.

Murphy AP & Leopold DA, (2019). A parameterized digital 3D model of the Rhesus macaque face for investigating the visual processing of social cues. J.Neurosci.Methods

Pre-prints / in prep.

Murphy AP, Esch EM, Khandhadia AP, Koyano KW, Leopold DA (in prep). Natural stereoscopic depth amplifies face cell responses in macaque.

Abstract

This study used stereoscopic 3D presentations of the macaque avatar faces to generate realistic depth percepts. Chronic exctracellular neural recordings from three of the face-selective regions of inferotemporal (IT) cortex revealed that many face selective neurons were tuned to faces with natural stereoscopic depth profiles, as opposed to 2D or inverted depth faces. This finding was robust across variations of the stimulus position-in-depth, suggesting that face cells are tuned to 3D shape rather than to specific binocular disparities.

Khandhadia AP, Murphy AP, Romanksi LM, Bizley JK, Leopold DA (in prep). Audiovisual Integration of Social Information Across Space.

Abstract

In the macaque, the superior temporal sulcus (STS) is a site of convergence of many different streams of information including auditory, visual, spatial, and social information. In the visual and social domain, the STS contains several face patches, regions which respond more to faces than to non-face objects. However, neurons in anterior fundus (AF) face patch also show modulation by the addition of auditory stimuli to visual stimuli but the precise acoustic information encoded by this modulation remains unknown. Here, we investigated whether spatial factors such as sound direction, gaze direction, or their interaction had an impact on neural responses in this region.

In this experiment, we recorded from the AF face patch in two macaque monkeys during presentation of audiovisual movies of macaque vocalizations within a virtual reality dome which allowed auditory and visual components to be spatially separated. The subject had to fixate on one of three visual locations. An auditory only, visual only, or audiovisual movie of a vocalizing monkey was then presented. The visual element always arose from the fixation location while the acoustic element, which was always temporally coherent with the movie, played either directly from the location of fixation, or at the same elevation, but shifted 30o to the right or left in azimuth, or at the same azimuth, but 45o above or below the position of fixation.

Cameo Appearances

The macaque avatar appears in figures for illustration purposes (or was used as an experimental stimulus without citation - naughty!) in the following articles and commentaries:

• Koyano KW, Esch EM, Hong JJ, Waidmann EN, Wu H, Leopol dDA (2023). Progressive neuronal plasticity in primate visual cortex during stimulus familiarization. Science Advances

• Azadi R, Bohn S, et al. (2023). Image-dependence of the detectability of optogenetic stimulation in macaque inferotemporal cortex. Curr.Bio.

• Fan S, Dal Monte O, Chang SWC (2021). Levels of naturalism in social neuroscience research. iScience

• Beauchamp MS (2021). Face and Voice Perception: Monkey see, monkey hear. Curr.Bio.

• Koyano KW, Jones AP, McMahon DBT, Waidmann EN, Russ BE, Leopold DA (2021). Dynamic Suppression of Average Facial Structure Shapes Neural Tuning in Three Macaque Face Patches. Curr.Bio.

• Taubert J & Japee S (2021). Using FACS to trace the neural specializations underlying the recognition of facial expressions: A commentary on Waller et al. (2020). Neu.Bio.Rev.

• Taubert J, Wardle SG, Ungerleider LG (2020). What does a “face cell” want?. P.Neuro.Bio.

• Leopold DA & Krauzlis RJ (2020). How the brain pays attention to others’ attention. Curr.Bio.

Acknowledgements

This work was funded by the National Institute of Mental Health (NIMH) intramural program and utilized the Neurophysiology Imaging Facility (NIF) and NIH’s HPC Biowulf cluster resources. Stimuli are hosted on Figshare under the Creative Commons CC-BY-NC 4.0 license, while software tools are hosted on GitHub under the GNU General Public License GNU GPLv3. All visual stimulus renders were generated using the open-source software Blender.

Collaborations

Since the initial launch of MF3D, many researchers have contacted us with inquiries regarding adaptation or development of the model’s features to address specific scientific questions. At present, our approach is to assess the feasibility of each feature request, and if we determine the required development of the model to be within our capabilities then we will offer to collaborate. This has the advantage for the requester that they don’t have to invest time and effort to learn the technical aspects of 3D animation and rendering in order to get their stimuli, while allowing us to avoid conflicts that could arise from multiple research groups working on the same experimental question simultaneously.