Author: Edwin Clayton

Vanessa Brown
Department of Psychiatry
University of Pittsburgh

Depression and anxiety disorders are characterized by difficulties navigating rewarding and punishing experiences. Although experimental evidence for disrupted reward and punishment processing in these disorders is substantial and serve as targets for interventions like cognitive behavioral therapy, it’s still unclear what dysfunctions exist, how they relate to specific presentations and symptoms in these disorders, and how treatment affects these dysfunctions. I will present data using reinforcement learning and symptom clusters to show what learning dysfunctions are present and how they relate to symptoms. Then, I will present results and theorized relationships between current & potential treatments and effects on learning in these disorders.

View this recorded session here.

Laura Bustamante
Princeton Neuroscience Institute
Princeton University

Background: Cognitive control-related symptoms of Major Depressive Disorder (MDD) (e.g., difficulty concentrating) contribute substantially to disability. Often cognitive symptoms and poorer performance on cognitive control tasks are interpreted as a reduced ability to exert control. We propose instead that cognitive symptoms may be due, in part, to reduced motivation to exert control because of increased cognitive effort “costs”.
Methods: We developed a novel experiential learning task, using patch-foraging, to derive a quantitative estimate of people’s cognitive effort costs. This measure allows us to, 1) dissociate motivation from ability, 2) compare cognitive effort costs in MDD relative to controls, 3) assess specificity of impairment with respect to differences from physical effort costs, self-efficacy, and reward responsiveness, and 4) map these axes of motivation onto MDD symptom heterogeneity.
Results: Preliminary results (N=30, 20 MDD) showed participants avoided exerting cognitive and physical effort, as reflected by lower patch-leaving thresholds in high- relative to low-effort environments of each condition (physical: beta=-0.37, df=29.15, t=-2.58, p<0.015; cognitive: beta=-0.46, df=29.29, t=-2.41, p<0.023). More depressed participants assigned disproportionately higher cost to physical effort (beta=13.07, df=27, t=3.01, p0.719), cognitive ability (beta=-3.16, df=26, t=-0.54, p>0.59), or in physical ability (beta=-0.23, , df=27, t=-0.60, p>0.55).
Conclusions: Resolving the underlying mechanisms of cognitive impairments in MDD has implications for developing treatments. By our account, if MDD patients are found to assign higher cognitive effort costs, interventions should target willingness to engage control (e.g., by providing motivational input) rather than cognitive control performance (e.g., cognitive training).

View this recorded session here.

Michael Browning
Department of Psychiatry
University of Oxford

Background: Dopamine D2-like receptor agonists show promise as treatments for depression. They are thought to act by altering how individuals learn from rewarding experiences. However, the nature of these reward learning alterations, and the mechanisms by which they are produced is not clear. In the present study, we characterised the behavioural effects of a sustained 2-week course of the D2-like receptor agonist pramipexole on reward learning and used fMRI measures of expectation and prediction error to assess which of these three mechanistic processes were responsible for the behavioural effects.

Methods: 40 healthy volunteers (Age: 18-43, 50% female) were randomly allocated to receive either two weeks of pramipexole (titrated to 1mg/day) or placebo in a double-blind, between subject design. Participants completed a probabilistic instrumental learning task, in which stimuli were associated with either rewards or losses, before the pharmacological intervention and twice between days 12-15 of the intervention (once with and once without fMRI). Both asymptotic choice accuracy, and a reinforcement learning model, were used to assess reward learning.

Results: Behaviourally, pramipexole specifically increased choice accuracy in the reward condition, with no effect in the loss condition. Pramipexole increased the BOLD response in the orbital frontal cortex during the expectation of win trials but decreased the BOLD response to reward prediction errors in the ventromedial prefrontal cortex. This pattern of results indicates that pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning.

Conclusions: The D2-like receptor agonist pramipexole enhances reward learning by preserving learned values. This is a plausible candidate mechanism for pramipexole’s observed anti-depressant effect and may also account for its tendency to increase impulsive behaviour.

View this recorded session here.

Clay Holroyd
Department of Experimental Psychology
Ghent University

Despite decades of intense study a good understanding of the function of anterior cingulate cortex remains elusive. In this talk I will present an overview of computational modeling and empirical work that positions ACC at the nexus of hierarchical reinforcement learning, effortful control, and model-based behavior. This work suggests that ACC motivates the execution of high-level plans via principles of hierarchical model-based hierarchical reinforcement learning. In particular, ACC control signals may enforce compliance of low-level action policies with hierarchically high-level goals.

View this recorded session here.

Chris Summerfield
Department of Experimental Psychology
University of Oxford

I will discuss work that asks how humans encode and mentally reconfigure knowledge of the world. We study neural geometry in BOLD signals while humans learn new relations among objects and events, and compare these to deep networks performing equivalent tasks. I will discuss theories of how knowledge is partitioned to avoid interference (continual learning) and how it is reconfigured in the light of just a few new samples of information (knowledge assembly). I will argue that we can use small, toy connectionist models to gain insights into how representations are formed, and how these are used for intelligent behaviour.

View this recorded session here. Password: 26@??VCF

Kyo Iigaya
Department of Psychiatry
Columbia University Irving Medical Center

It is an open question how humans construct the subjective value of complex stimuli, such as artistic paintings or photographs. While great progress has been made toward understanding how the brain updates the value of known stimuli e.g., through reinforcement learning, little is known about how the value arises in the brain in the first place. Here, we propose that the brain constructs the value of a novel stimulus by extracting and assembling common features shared across stimuli. Notably, because those features are shared across a broad range of stimuli, we show that simple linear regression in the feature space can work as a single mechanism to construct the value across stimulus domains. In large-scale behavioral experiments with human participants, we show that a model of feature abstraction and linear summation can predict the subjective value of paintings, photographs, as well as shopping items whose values change according to different goals. The model shows a remarkable generalization across stimulus types and participants, e.g., when trained on liking ratings for photographs, the model successfully predicts a completely different set of art painting ratings. Also, we show that these general features emerge in a deep convolutional neural network, without explicit training on the features, suggesting that features relevant for value computation could arise spontaneously. Furthermore, using fMRI, we found evidence that the brain performs value computation hierarchically by transforming low-level visual features into high-level abstract features which in turn are transformed into valuation. Our findings suggest the feature-based value computation can be a general neural principle enabling us to make flexible and reliable value computations for a wide range of complex stimuli.

View this recorded session here.

Stefano Palminteri
Department of Cognitive Studies
École normale supérieure

I will present results from seven experiments (N=100 each) demonstrating that, in the context of human learning decision-making, the way in which options are arranged (i.e., the choice architecture) significantly affects the resulting memory representations of economic values. More specifically, economic values stored in memory do not reflect objective values, but are generally consistent with a partial range adaptation process. The results are robust across preference elicitation methods (choices or ratings), decision-making modalities (experience-based or description-based), and across days.

View this recorded session here.

Anne Collins
Department of Psychology
University of California, Berkeley

The study of the neural processes that support reinforcement learning has been greatly successful. It has characterized a simple brain network (including cortico-basal ganglia loops and dopaminergic signaling) that enables animals to learn to make valuable choices, using valenced outcomes. However, increasing evidence shows that the story is more complex in humans, where additional processes also contribute importantly to learning. In this talk, I will show three examples of how prefrontal-dependent executive processes are essential to reinforcement learning in humans, operating both in parallel to the brain’s reinforcement learning network, as well as feeding this network information.

View this recorded session here.

Kean Hsu
Department of Psychiatry
Georgetown University

Impairments in basic cognitive processes like attention and executive functioning are common, significant, and an unmet treatment need that has broad downstream effects for depressed individuals. However, the nature of these impairments and how they might lead to negative affect or clinical disorder remains poorly understood. This talk presents empirical data addressing three lines of questioning: 1) Is cognitive impairment a scar left by depression or does it potentially precede depression (or both); 2) how are difficulties with basic cognitive processes associated with the phenomenology of depression; and 3) does an experimental manipulation of these processes impact depression maintenance? To address these questions, I have assessed cognition in a variety of depressed populations, including monozygotic twin pairs discordant for lifetime depression, currently depressed, formerly depressed, and never-depressed individuals drawn from the community, and a sample of depressed individuals specifically expressing a cognitive process of interest. Future directions for this program of research, including identification of which cognitive processes contribute to the risk for, maintenance of, and impairment from emotional disorders, as well as how we can translate these findings in applied settings, will be reviewed.

View this recorded session here. Password for recording is: mr+6E.!t

Wayne Fisher
Rutgers University Center for Autism Research, Education, and Services (RU-CARES)
Robert Wood Johnson Medical School

The most important advancement in the treatment of destructive behavior has been the development of functional analysis (FA), which is used to prescribe effective treatments, such as functional communication training (FCT). With FCT, the consequence that historically reinforced destructive behavior is delivered contingent on an appropriate communication response and problem behavior is correlated with extinction. Although this approach can be highly effective, many pitfalls and practical challenges arise when this treatment is implemented by caregivers in natural community settings. In this presentation, I will present data and describe a line of research routed in behavioral momentum theory and the generalized matching law aimed at increasing the effectiveness, generality, and durability of FCT for individuals with ASD who display destructive behavior in typical community settings. Specifically, I focus on: (a) applications based on the matching law that can be used to prevent extinction bursts when treatment is initiated; (b) stimulus-control procedures that can be used to promote the rapid transfer of treatment effects to novel therapists, contexts, and caregivers without reemergence of destructive behavior; and (c) stimulus- and consequence-control procedures that can be used as “behavioral inoculation” to prevent resurgence of problem when caregivers do not implement treatment procedures with pristine procedural integrity.

View this recorded session here.