Internal affective states produce external manifestations such as facial expressions. In humans, the Facial Action Coding System (FACS) is widely used to objectively quantify the elemental facial action-units (AUs) that build complex facial expressions. A similar system has been developed for macaque monkeys - the Macaque Facial Action Coding System (MaqFACS); yet unlike the human counterpart, which is already partially replaced by automatic algorithms, this system still requires labor-intensive coding. Here, we developed and implemented the first prototype for automatic MaqFACS coding. We applied the approach to the analysis of behavioral and neural data recorded from freely interacting macaque monkeys. The method achieved high performance in recognition of six dominant AUs, generalizing between conspecific individuals ( Macaca mulatta ) and even between species ( Macaca fascicularis ). The study lays the foundation for fully automated detection of facial expressions in animals, which is crucial for inves...

Vision loss from diseases of the outer retina, such as Age-Related Macular Degeneration (AMD), are among the leading causes of irreversible blindness in the world today. The goal of retinal prosthetics is to replace the photo-sensing function of photoreceptors lost in these diseases with optoelectronic hardware to electrically stimulate patterns of retinal activity corresponding to vision. To enable high-resolution retinal prosthetics, the scale of stimulating electrodes must be significantly decreased from current designs; however, this reduces the amount of stimulating current that can be delivered. The efficacy of subretinal stimulation at electrode sizes suitable for high visual acuity retinal prosthesis are not well understood, particularly within the safe charge injection limits of electrode materials. Here, we measure retinal ganglion cell responses in a mouse model of blindness to evaluate the stimulation efficacy of 10, 20, and 30 µm diameter iridium oxide electrodes within the electrode charge in...

Frequency-to-place mapping, or tonotopy, is a fundamental organizing principle throughout the auditory system, from the earliest stages of auditory processing in the cochlea to subcortical and cortical regions. Although cortical maps are referred to as tonotopic, it is unclear whether they simply reflect a mapping of physical frequency inherited from the cochlea, a computation of pitch based on the fundamental frequency, or a mixture of these two features. We used high-resolution fMRI to measure BOLD responses as male and female human participants listened to pure tones that varied in frequency or complex tones that varied in either spectral content (brightness) or fundamental frequency (pitch). Our results reveal evidence for pitch tuning in bilateral regions that partially overlap with the traditional tonotopic maps of spectral content. In general, primary regions within Heschl’s gyri exhibited more tuning to spectral content, whereas areas surrounding Heschl’s gyri exhibited more tuning to pitch. Signi...

As we navigate the world, we use learned representations of relational structures to explore and to reach goals. Studies of how relational knowledge enables inference and planning are typically conducted in controlled small-scale settings. It remains unclear, however, how people use stored knowledge in continuously unfolding navigation, e.g., walking long distances in a city. We hypothesized that multiscale predictive representations guide naturalistic navigation in humans, and these scales are organized along posterior-anterior prefrontal and hippocampal hierarchies. We conducted model-based representational similarity analyses of neuroimaging data collected while male and female participants navigated realistically long paths in virtual reality. We tested the pattern similarity of each point–along each path–to a weighted sum of its successor points within predictive horizons of different scales. We found that anterior PFC showed the largest predictive horizons, posterior hippocampus the smallest, with th...

Time-varying pitch is a vital cue for human speech perception. Neural processing of time-varying pitch has been extensively assayed using scalp-recorded frequency-following responses (FFRs), an electrophysiological signal thought to reflect integrated phase-locked neural ensemble activity from subcortical auditory areas. Emerging evidence increasingly points to a putative contribution of auditory cortical ensembles to the scalp-recorded FFRs. However, the properties of cortical FFRs and precise characterization of laminar sources are still unclear. Here we used direct human intracortical recordings as well as extra- and intracranial recordings from macaques and guinea pigs to characterize the properties of cortical sources of FFRs to time-varying pitch patterns. We found robust FFRs in the auditory cortex across all species. We leveraged representational similarity analysis as a translational bridge to characterize similarities between the human and animal models. Laminar recordings in animal models showed...

Secreted amyloid-β (Aβ) peptide forms neurotoxic oligomeric assemblies thought to cause synaptic deficits associated with Alzheimer’s disease (AD). Soluble Aβ oligomers (Aβo) directly bind to neurons with high affinity and block plasticity mechanisms related to learning and memory, trigger loss of excitatory synapses and eventually cause cell death. While Aβo toxicity has been intensely investigated, it remains unclear precisely where Aβo initially binds to the surface of neurons and whether sites of binding relate to synaptic deficits. Here, we used a combination of live cell, super-resolution and ultrastructural imaging techniques to investigate the kinetics, reversibility and nanoscale location of Aβo binding. Surprisingly, Aβo does not bind directly at the synaptic cleft as previously thought but, instead, forms distinct nanoscale clusters encircling the postsynaptic membrane with a significant fraction also binding presynaptic axon terminals. Synaptic plasticity deficits were observed at Aβo-bound syn...

Functional connectome “fingerprint” is a cluster of individualized brain functional connectivity patterns that are capable of distinguishing one individual from others. Although its existence has been demonstrated in adolescents and adults, whether such individualized patterns exist since infancy is barely investigated despite its importance in identifying the origin of the intrinsic connectome patterns that potentially mirror distinct behavioral phenotypes. To fill this knowledge gap, capitalizing on a longitudinal high-resolution structural and resting-state functional MRI dataset (104 human infants (53 females) with 806 longitudinal scans, age=16-876 days) and infant-specific functional parcellation maps, we observe that the brain functional connectome fingerprint may exist since infancy and keeps stable over months during early brain development. Specifically, we achieve about 78% individual identification rate by using around 5% selected functional connections, compared to the best identification rate...

Post-encoding coordinated reactivation of memory traces distributed throughout interconnected brain regions is thought to be critical for consolidation of memories. However, little is known about the role of neural circuit pathways during post-learning periods for consolidation of memories. To investigate this question, we optogenetically silenced the inputs from both auditory cortex and thalamus in the lateral amygdala (LA) for 15 min immediately following auditory fear conditioning (FC) and examined its effect on fear memory formation in mice of both sexes. Optogenetic inhibition of both inputs disrupted long-term fear memory formation tested 24 h after FC. This effect was specific such that the same inhibition did not affect short-term memory and context-dependent memory. Moreover, long-term memory was intact if the inputs were inhibited at much later time points after FC (3 h or 1 d after FC), indicating that optical inhibition for 15 min itself does not produce any nonspecific deleterious effect on fe...

Many daily choices are based on one's own knowledge. However, when predicting other people's behavior, we need to consider the differences between our knowledge and other people's presumed knowledge. Social agents need a mechanism to use privileged information for their own behavior but exclude it from predictions of others. Using fMRI, we investigated the neural implementation of such social and personal predictions in healthy human volunteers of both sexes by manipulating privileged and shared information. The medial frontal cortex appeared to have an important role in flexibly making decisions using privileged information for oneself or predicting others' behavior. Specifically, we show that ventromedial PFC tracked the state of the world independent of the type of decision (personal, social), whereas dorsomedial regions adjusted their frame of reference to the use of privileged or shared information. Sampling privileged evidence not available to another person also relied on specific interactions betwe...