Chapter 6 2A adrenoceptor stimulation in primates supports fronto-striatal functions by enhancing reward
6.5 D ISCUSSION
Here, we found that systemic guanfacine administration enhances cognitive flexibility evident in faster learning after reward reversals and enhanced post-error behavioral improvement.
This improved cognitive flexibility was accompanied by the enhanced representation of outcomes and model-derived RPEs in the spiking activity of single neurons within the fronto-striatal network during the feedback epoch. Outcome representations were stronger with systemic guanfacine administration in the dlPFC, ACC and CD while the enhanced RPE signaling was specific to the ACC and CD. These enhancements were observed in the absence of overall changes in firing rate or firing variability at a global level (coefficient of variation) or local level (local variability) and without overall changing the proportion of neurons encoding the tested variables. Cell type classification revealed that putative interneurons and putative fast spiking interneurons were driving the cortical (dlPFC and ACC) and subcortical (CD) enhancement of outcome encoding, respectively. These results suggest a mechanism regarding the role of 2A adrenoceptors in enhancing cognitive flexibility, complimentary to their involvement in spatial WM. Although the nature of systemic administrations make it difficult to ascertain the adrenoceptors or even
neuromodulators that are causally involved, the results illustrate that systemic 2A stimulation is sufficient for enhancing both outcome and RPE encoding in the fronto-striatal network which we could link with faster reversal learning and post-error behavioral adjustments.
Fig 6.7 Guanfacine-mediated changes in neural activity correlations to outcome variables during the attention cue onset epoch for broad and narrow spiking neurons.
(A) The same as figure 6.6A but for the attention cue onset epoch. Roman numerals correspond with example neurons in C. (B) The same as figure 6.6B but for the attention cue onset epoch. Roman numerals correspond with example neurons in C. (C) Example narrow (two left most examples) and broad (two right most examples) spiking neurons with significant correlations for prior trial outcome for (to be) error trials (three left most examples) and prior trial outcome for (to be) correct trials (far right). Roman numerals map onto figures A and B to indicate where the example neuron was pulled from. The dashed line represents the average learning trial if aligned to block reversal.
6.5.1 The 2A adrenoceptor and cognitive flexibility.
Our findings suggest that 2A adrenoceptor activation supports cognitive flexibility by modulating three core brain areas of the anterior fronto-striatal loops, which goes beyond previous studies that demonstrated how 2AR activation modulates neuronal activity and behavioral
performance for spatial working memory tasks in the dlPFC. Molecular and iontophoretic experiments have described post-synaptic 2A receptors on unique dendritic spines exclusive to the dlPFC (Arnsten et al., 2010; Cools and Arnsten, 2022), and stimulation of these post-synaptic
2A receptors disrupts intracellular cAMP signaling leading to enhanced delay firing (activity persisting through WM delay) for the preferred spatial location of these dlPFC pyramidal neurons (Wang et al., 2007). However, this mechanism of 2A receptor action does not account for the enhanced cognitive flexibility observed here. First, the feature-based reversal learning task used here does not contain any explicit working memory ‘delay’ period (Figure 6.1B). Second, the improved learning performance was specifically dependent on the requirement to reverse a learned reward association, because we did not observe better performance with guanfacine during the first block prior to the first reversal (Appendix E, Figure E1B). This finding resonates well with a recent study where optogenetic activation of LC enhanced performance after a rule switch but not in the first block of a given session (McBurney-Lin et al., 2022). Taken together, the behavioral improvement with guanfacine in our study is likely reflecting an increased efficiency to learn from trial outcomes and to utilize error signals for improving future performance.
6.5.2 Enhanced outcome and RPE encoding without increased proportion of encoding neurons.
We found that guanfacine increased the strength of neuronal encoding of trial outcomes in dlPFC, ACC, and CD (Figure 6.4) and of RPEs in ACC and CD (Figure 6.5) during the feedback epoch. These changes could be the neural substrate for the enhanced post-error adjustment and the faster adjustment to reversed color-reward associations after a block reversal (Figure 6.2C). The enhanced neural encoding is consistent with 2A adrenoceptors enhancing the gain of neuronal responses in each of the three recorded brain areas. Gain modulation has been proposed to be a primary effect of increased NE activity, capable of potentiating responses of neurons and also capable of making neural responses to previously sub-threshold inputs supra-threshold (Berridge and Waterhouse, 2003; Aston-Jones and Cohen, 2005a).
While we did observe enhanced encoding strength with guanfacine, consistent with potentiating existing responses, we did not find proportionally more neurons encoding for
outcomes or RPEs (Appendix E, Figure E3). This finding shows that the putative gain modulation process induced by guanfacine in dlPFC, ACC, and CD primarily affects neurons that are already functionally recruited without switching on previously non-active neurons. Consistent with this suggestion, we found that guanfacine had a moderately suppressive effect on the overall average firing of neurons during in the feedback epoch with significantly reduced firing rates of putative pyramidal neurons in the dlPFC and putative MSNs in the CD (see Appendix E.2). Moreover, the NE mediated switching of sub-threshold to supra-threshold responses has largely been described in sensory cortices, which differs in composition and 2A receptor densities from fronto-striatal circuits (Ciombor et al., 1999; Devilbiss and Waterhouse, 2004; Waterhouse and Navarra, 2018).
However, we cannot rule out that higher concentration of guanfacine, or the direct increase of NE would have recruited additional neurons encoding outcomes and RPEs during the task.
6.5.3 Spatial and cell-type specificity.
We found that the significantly stronger outcome encoding with guanfacine during the feedback epoch was driven by narrow spiking putative interneurons in the dlPFC and ACC, and putative FSIs in the CD (Figure 6.6A). This finding is consistent with studies have shown that adrenoceptor expression and modulation is stronger for interneurons than pyramidal cells with 2 and adrenoceptors enhancing their inhibitory actions while 1 adrenoceptors decrease their inhibitory actions in prefrontal cortex (Kawaguchi and Shindou, 1998; Wang et al., 2013; Liu et al., 2014; Xing et al., 2016; Lee et al., 2020), as well as in sensory and sensorimotor cortices (Bennett et al., 1998; Nai et al., 2009; Salgado et al., 2011, 2012; Ohshima et al., 2017). This suggests guanfacine may enhance the inhibitory activity of interneurons which is consistent with the decreased firing we observed in putative pyramidal neurons in the dlPFC and with the decreased firing of putative MSN spiking in the CD (see Appendix E.2).
A prominent role of interneurons in learning from outcomes and reward predictions errors has recently been demonstrated for putative fast spiking interneurons in the lateral PFC of macaques (Boroujeni et al., 2021), as well as for fast spiking interneurons in the head of the caudate (Boroujeni et al., 2020). In these studies, narrow spiking neurons encoded reward prediction errors particularly during the learning period of reward reversal. Our results of guanfacine enhancing the
encoding of outcomes and reward prediction errors may thus directly support the interneuron- mediated learning and behavioral adjustment. Systemic guanfacine may thus have gain modulated the intrinsic neuronal dynamics underlying cognitive flexibility during color-based reversal learning.
While we found that trial outcomes were enhanced with guanfacine in all three areas, stronger RPE encoding was only observed in the ACC and CD (Figure 6.5). We unfortunately did not have enough data for a reasonable comparison of putative cell types that significantly encoded RPE signals. However, we did observe enhanced negative, but not positive or signed RPE signaling in the ACC. Negative RPEs are only computed for unrewarded trials and are critical signals for the adjustment of behavioral strategy upon failing to acquire reward, a major function of the ACC (Kennerley et al., 2006; Buckley et al., 2009; Kaping et al., 2011; Gläscher et al., 2012;
Heilbronner and Hayden, 2016). Furthermore, a previous study demonstrated that switches in behavioral strategy may be triggered by LC input to the ACC (Tervo et al., 2014). Similarly, we only observed enhanced positive and signed, but not enhanced negative RPE signaling in the CD during the feedback epoch with guanfacine. This, too, matches known functions of the striatum and the head of the caudate for value updating (Cromwell and Schultz, 2003; Williams and Eskandar, 2006; Kim and Hikosaka, 2013, 2015; Vo et al., 2014; Rothenhoefe et al., 2017). The systemic administration of guanfacine is thus helping inch the functions of the ACC and CD, which support behavioral flexibility (shifting behavioral strategies and value updating respectively), towards greater flexibility.
A recent study describes two distinct populations of LC neurons distinguishable by their waveforms which are excited by positive RPEs and a lack of reward respectively (Su and Cohen, 2022) suggesting that distinct noradrenergic neuronal populations in the LC (Chandler et al., 2014;
Totah et al., 2018; Breton-Provencher et al., 2022) may be responsible for the observed RPE enhancement we see in the ACC and CD. The same study (Su and Cohen, 2022) has posited that noradrenergic signaling from the LC may serve to communicate RPE information to the cortex while dopaminergic signaling communicates RPEs to the basal ganglia. This proposed dichotomy is consistent with our findings. Future studies may thus test more directly whether distinct neuromodulatory systems mediate prediction error signaling in ACC and striatum.
6.5.4 Insights from 2A stimulation: norepinephrine and behavior.
Pre-synaptic 2A adrenoceptors act as noradrenergic auto-receptors, reducing the further release of NE. It has been previously shown that systemic guanfacine administration reduces LC activity (Engberg and Eriksson, 1991; Okada et al., 2018). Consistent with reduced LC firing we observed reduced pupil diameter in the blocks temporally closest to the inject time (Appendix E, Figure E1C). However, guanfacine also resulted in on average reduced pair-wise spike count correlations in the ACC relative to the non-drug condition, which a recent study have shown to be indicative of higher LC activity (Joshi and Gold, 2022). This discrepancy might be resolved by distinguishing tonic from phasic LC activity modulations. The increases of pairwise firing correlations in Joshi and Gold (2022) likely reflect reduced tonic LC firing, while the reduction of firing correlation that we found indicates enhanced phasic LC firing in the presence of reduced tonic LC firing. This proposal is consistent with findings showing that noradrenergic auto-receptor activation can increase LC neuron sensitivity to glutamatergic and cholinergic stimulation thus emulating an increase in phasic LC firing (Aston-Jones et al., 1991a; Aston-Jones and Cohen, 2005a). This suggests that systemic guanfacine administration may reduce tonic LC activity while simultaneously boosting phasic LC activity.
This 2A stimulation resulted in enhanced cognitive flexibility in our feature-based reversal learning task with faster behavioral adjustments after unexpected outcomes. It is possible that this enhanced flexibility comes at a behavioral cost. Although guanfacine lead to better immediate post-reversal performance as can be seen in the raw performance average of the first three post-reversal trials, it does not improve overall plateau performance at the end of the reversal block and may even reduce it as visible in the subtly (non-significantly) lower end-of-block performance plateaus (Figure 6.2A & Appendix E, Figure E1A). This observation suggests that the systemic guanfacine administration promoted exploratory behavior facilitating the learning but hampering the exploitative behavioral after an initial learning criterion was achieved. Our results are therefore consistent with a role of the 2A adrenoreceptor in re-balance the exploration- exploitation trade-off towards a higher weighting of explorative behavior, mediated potentially by an overall increase of learning rates from performance feedback (Hassani et al., 2017).