2 March 2022: The effects of stressor exposure on goal-directed control and decision-making

Candace Raio
Department of Psychiatry
NYU Grossman School of Medicine
New York University

The ability to effectively deploy goal-directed control during learning and decision-making is essential for adaptive behavior but is often compromised under stress. Using experimental paradigms that draw upon learning models and behavioral economics, I will present research that examines how stress changes the use of goal-directed control strategies and suggest that these changes may stem from the increased cognitive cost of exercising control. Further, I will show that the subjective cost of control can be measured using a novel economic decision-making approach and that these costs are highly sensitive to changes in affective state. Finally, I will argue that stress can also confer adaptive benefits for survival and demonstrate how this may emerge in decision contexts marked by uncertainty.

View a recording of this session here.

23 February 2022: Trial by trial model fitting workshop, part I: finding optimal parameters to describe data

Yael Niv
Princeton Neuroscience Institute
Princeton University

In this workshop I will discuss the basics of fitting reinforcement learning models to human behavior. We will start by discussing probability theory and Bayes rule. Then we will work out a reinforcement learning example step by step — no advance knowledge of reinforcement learning necessary. Note that if you participated in this workshop (taught by me) in another forum, including at CCNP in 2016 (wow, time flies!), this will be the same workshop. In part two of the workshop (April 6), we will discuss model comparison.

***Prior to this CCNP workshop, it is highly recommended attendees watch this 11-min video: https://www.youtube.com/watch?v=BrK7X_XlGB8. ***

View a recording of this session here.

View slides from this session here.

2 February 2022: Understanding and Treating Learning Dysfunctions in Anxiety

Vanessa Brown
Department of Psychiatry
University of Pittsburgh

Depression and anxiety disorders are characterized by difficulties navigating rewarding and punishing experiences. Although experimental evidence for disrupted reward and punishment processing in these disorders is substantial and serve as targets for interventions like cognitive behavioral therapy, it’s still unclear what dysfunctions exist, how they relate to specific presentations and symptoms in these disorders, and how treatment affects these dysfunctions. I will present data using reinforcement learning and symptom clusters to show what learning dysfunctions are present and how they relate to symptoms. Then, I will present results and theorized relationships between current & potential treatments and effects on learning in these disorders.

View this recorded session here.

15 December 2021: Motivation for cognitive and physical effort in depression

Laura Bustamante
Princeton Neuroscience Institute
Princeton University

Background: Cognitive control-related symptoms of Major Depressive Disorder (MDD) (e.g., difficulty concentrating) contribute substantially to disability. Often cognitive symptoms and poorer performance on cognitive control tasks are interpreted as a reduced ability to exert control. We propose instead that cognitive symptoms may be due, in part, to reduced motivation to exert control because of increased cognitive effort “costs”.
Methods: We developed a novel experiential learning task, using patch-foraging, to derive a quantitative estimate of people’s cognitive effort costs. This measure allows us to, 1) dissociate motivation from ability, 2) compare cognitive effort costs in MDD relative to controls, 3) assess specificity of impairment with respect to differences from physical effort costs, self-efficacy, and reward responsiveness, and 4) map these axes of motivation onto MDD symptom heterogeneity.
Results: Preliminary results (N=30, 20 MDD) showed participants avoided exerting cognitive and physical effort, as reflected by lower patch-leaving thresholds in high- relative to low-effort environments of each condition (physical: beta=-0.37, df=29.15, t=-2.58, p<0.015; cognitive: beta=-0.46, df=29.29, t=-2.41, p<0.023). More depressed participants assigned disproportionately higher cost to physical effort (beta=13.07, df=27, t=3.01, p0.719), cognitive ability (beta=-3.16, df=26, t=-0.54, p>0.59), or in physical ability (beta=-0.23, , df=27, t=-0.60, p>0.55).
Conclusions: Resolving the underlying mechanisms of cognitive impairments in MDD has implications for developing treatments. By our account, if MDD patients are found to assign higher cognitive effort costs, interventions should target willingness to engage control (e.g., by providing motivational input) rather than cognitive control performance (e.g., cognitive training).

View this recorded session here.

1 December 2021: Pramipexole Enhances Reward Learning by Preserving Value Estimates

Michael Browning
Department of Psychiatry
University of Oxford

Background: Dopamine D2-like receptor agonists show promise as treatments for depression. They are thought to act by altering how individuals learn from rewarding experiences. However, the nature of these reward learning alterations, and the mechanisms by which they are produced is not clear. In the present study, we characterised the behavioural effects of a sustained 2-week course of the D2-like receptor agonist pramipexole on reward learning and used fMRI measures of expectation and prediction error to assess which of these three mechanistic processes were responsible for the behavioural effects.

Methods: 40 healthy volunteers (Age: 18-43, 50% female) were randomly allocated to receive either two weeks of pramipexole (titrated to 1mg/day) or placebo in a double-blind, between subject design. Participants completed a probabilistic instrumental learning task, in which stimuli were associated with either rewards or losses, before the pharmacological intervention and twice between days 12-15 of the intervention (once with and once without fMRI). Both asymptotic choice accuracy, and a reinforcement learning model, were used to assess reward learning.

Results: Behaviourally, pramipexole specifically increased choice accuracy in the reward condition, with no effect in the loss condition. Pramipexole increased the BOLD response in the orbital frontal cortex during the expectation of win trials but decreased the BOLD response to reward prediction errors in the ventromedial prefrontal cortex. This pattern of results indicates that pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning.

Conclusions: The D2-like receptor agonist pramipexole enhances reward learning by preserving learned values. This is a plausible candidate mechanism for pramipexole’s observed anti-depressant effect and may also account for its tendency to increase impulsive behaviour.

View this recorded session here.

10 November 2021: The Role of Anterior Cingulate Cortex in Hierarchical Model-based Hierarchical Reinforcement Learning: A Retrospective

Clay Holroyd
Department of Experimental Psychology
Ghent University

Despite decades of intense study a good understanding of the function of anterior cingulate cortex remains elusive. In this talk I will present an overview of computational modeling and empirical work that positions ACC at the nexus of hierarchical reinforcement learning, effortful control, and model-based behavior. This work suggests that ACC motivates the execution of high-level plans via principles of hierarchical model-based hierarchical reinforcement learning. In particular, ACC control signals may enforce compliance of low-level action policies with hierarchically high-level goals.

View this recorded session here.

27 October 2021: Structuring knowledge in the human brain

Chris Summerfield
Department of Experimental Psychology
University of Oxford

I will discuss work that asks how humans encode and mentally reconfigure knowledge of the world. We study neural geometry in BOLD signals while humans learn new relations among objects and events, and compare these to deep networks performing equivalent tasks. I will discuss theories of how knowledge is partitioned to avoid interference (continual learning) and how it is reconfigured in the light of just a few new samples of information (knowledge assembly). I will argue that we can use small, toy connectionist models to gain insights into how representations are formed, and how these are used for intelligent behaviour.

View this recorded session here. Password: 26@??VCF

13 October 2021: Neural computation underlying subjective value construction

Kyo Iigaya
Department of Psychiatry
Columbia University Irving Medical Center

It is an open question how humans construct the subjective value of complex stimuli, such as artistic paintings or photographs. While great progress has been made toward understanding how the brain updates the value of known stimuli e.g., through reinforcement learning, little is known about how the value arises in the brain in the first place. Here, we propose that the brain constructs the value of a novel stimulus by extracting and assembling common features shared across stimuli. Notably, because those features are shared across a broad range of stimuli, we show that simple linear regression in the feature space can work as a single mechanism to construct the value across stimulus domains. In large-scale behavioral experiments with human participants, we show that a model of feature abstraction and linear summation can predict the subjective value of paintings, photographs, as well as shopping items whose values change according to different goals. The model shows a remarkable generalization across stimulus types and participants, e.g., when trained on liking ratings for photographs, the model successfully predicts a completely different set of art painting ratings. Also, we show that these general features emerge in a deep convolutional neural network, without explicit training on the features, suggesting that features relevant for value computation could arise spontaneously. Furthermore, using fMRI, we found evidence that the brain performs value computation hierarchically by transforming low-level visual features into high-level abstract features which in turn are transformed into valuation. Our findings suggest the feature-based value computation can be a general neural principle enabling us to make flexible and reliable value computations for a wide range of complex stimuli.

View this recorded session here.

6 October 2021: Context-Dependence Induces False Memories of Economic Values: A Test Across Three Decision-Making Modalities and Four Preference Elicitation Methods

Stefano Palminteri
Department of Cognitive Studies
École normale supérieure

I will present results from seven experiments (N=100 each) demonstrating that, in the context of human learning decision-making, the way in which options are arranged (i.e., the choice architecture) significantly affects the resulting memory representations of economic values. More specifically, economic values stored in memory do not reflect objective values, but are generally consistent with a partial range adaptation process. The results are robust across preference elicitation methods (choices or ratings), decision-making modalities (experience-based or description-based), and across days.

View this recorded session here.