The Moderating Role of Neuroticism on Evaluative Conditioning: New Insights on the Processes Underlying This Relationship

The aim of the present study was twofold. First, we sought to investigate whether Neuroticism and its relevant facets would moderate the effect of valence in the EC paradigm to replicate previous results (Bunghez et al., 2022; Vogel et al., 2019). Second, we aimed to further our understanding of the relationship between Neuroticism and the EC effect. More specifically, we investigated how Neuroticism would influence this effect. For this purpose, we examined whether Neuroticism (and its relevant facets) could be related to the enhanced tendency to evaluate the US or whether it could be related to a stronger reaction to valence of the US. It is also possible that both processes are at work, or neither. The study presented below allowed us to test these hypotheses.

The study design, materials, the main aims of the study, and the data analytic plan were pre-registered in AsPredicted (https://aspredicted.org/TD9_7QG) and the study was approved by the Ethics Committee of the Department involved. For the sake of simplicity, we name US valence as “US condition” (1 = positive, 0 = negative). We selected matched positive and negative USs at different levels of arousal. Unlike what we preregistered, we analyzed them considering their valence only, as the mixed models were too complex, and the stimuli within each category were insufficient to estimate this factor (see below).

We also extended the preregistered analysis plan. First, in line with Vogel et al. (2019), we decided to consider not only the broad personality factor of Neuroticism but also its facets. Second, we tested a series of moderated mediation mixed-models to assess the moderating effect of Neuroticism on the relationship between US condition and CS evaluation (i.e., the EC effect), on that between US condition and US evaluation and between US evaluation and CS evaluation. More specifically, to address the moderating role of Neuroticism on the EC effect and the relationship between US condition and US evaluation, we performed a series of mixed model analyses (Judd et al., 2012; Westfall et al., 2014), entering US condition, neuroticism facets (one at a time), and their interaction as fixed effects. Instead, to test the moderating role of Neuroticism and the association between US evaluation and CS evaluation, we entered US evaluation, the neuroticism facet (one at a time), and their interaction as fixed effects. We grand-mean centered the neuroticism facets and the US evaluation variable. Since all our models turned out to have singularity issues, we followed the Bates method (Bates et al., 2012) to simplify their random structure. Thus, as random effects, we had the intercept for participants and stimuli (CS) and a by-participant random slope for the impact of the US condition. We also removed correlations between intercept and slope for participants. When interaction terms were significant, we performed simple-slope analyses to explore these interactions better. The analyses were performed using R version 4.0.3 (R Core Team, 2020) in conjunction with the package lme4 (Bates et al., 2012; version 1.1-21) and “reghelper” (Hughes, 2018).

The procedure involved a 2-levels (US condition: positive vs. negative) within-subjects unifactorial design. Participants were first provided a general overview of the experiment and then asked for their informed consent. Participants completed the EC procedure and then the Big Five Inventory-2 – Negative Emotionality items (BFI-2; Soto & John, 2017), HEXACO-PI - Emotionality items (Ashton & Lee, 2009; Lee & Ashton, 2004), HEXACO-60 (Ashton & Lee, 2009), and the IPIP-NEO -120 - Neuroticism items (Johnson, 2014). The entire session lasted approximately 20 minutes.

We originally planned to collect 300 participants. For technical reasons, we collected only two hundred and ninety-eight participants (117 women, M_age = 25.76, SD = 6.23) on the Prolific Academic website (https://prolific.ac) in exchange for a monetary reward. Following our preregistered data cleaning, we excluded data from participants who did not rate any of the pictures of greyscale fractals as neutral during the pre-conditioning phase and participants who selected the same response 100% of the time on the dependent variable. Based on these criteria, we removed 6 participants evaluating all the CSs in the same way 100% of the time during the pre-evaluation phase, and 50 initial participants who did not arrive at the EC procedure because did not evaluate at least 4 neutral CSs, leaving us with 242 participants. The excluded participants did not differ from the final sample in Neuroticism (see Supplementary Material Table S1).

We estimated a priori the required sample size through a power sensitivity analysis (Perugini, Gallucci, et al., 2018), showing that, given α = .05 and a power of .80 for a unidirectional hypothesis, the minimum effect size detectable with a sample size of 300 for a moderating effect of Neuroticism on EC is r = .143 (equivalent to Cohen’s d = 0.289), which becomes r =.159 with n = 242). Concerning the mediated moderation multilevel model, we calculated post-hoc the sensitivity of the study. We used a simulation approach (Liu, 2014) to estimate the sensitivity to the key parameters given the other non-focal parameters in the model. We are not aware of any software or package allowing us to perform a sensitivity analysis for a multilevel model (e.g., sensitivity is not implemented in the R package simr). Therefore, we had to define an ad-hoc meaningful approach and then run simulation models. Details of the simulation approach as well as the corresponding R code are reported in the supplementary material. The parameters we found in the study were overall compatible with the sensitivity of the design as established by simulations. Therefore, the study was reasonably sensitive to detect meaningful effects, hence reasonably powered. We refer to the supplementary material for more information.

The EC paradigm was adapted from Vogel et al. (2019; footnote 3). It consisted of the following sequential phases: CS pre-conditioning evaluation, conditioning, postconditioning CS evaluation, contingency awareness assessment, US evaluation, and demand awareness questions.

EC stimuli. The initial set of CSs was formed by twenty computer-generated grayscale pictures of fractals from which the CSs used in the study were selected after an evaluation before the EC paradigm (see below). We used four USs, two positively (M_valence = 7.86 and M_valence = 6.17) and two negatively (M_valence = 2.45, M_valence = 2.87) valenced pictures from the international Pictures Affective System (IAPS; Lang, Bradley, & Cuthbert, 2005).

CS pre-conditioning evaluation. Participants saw a series of 20 unknown computer-generated grayscale pictures of fractals presented in a random order one by one and rated their likeability from -10 (“very negative”) to +10 (“very positive”) with the middle of the scale (0) marked as “neutral”. The ratings were used to select pictures for the conditioning phase idiosyncratically. The computer program selected the 4 pictures evaluated most neutrally by a participant in a stepped selection process: if more than four, pictures were randomly selected among the pool of those with ratings of 0; if not enough, pictures with ratings of -/+ 1 were included; if still this step was not sufficient, pictures with ratings of -/+ 2 were considered. Participants who did not rate at least 4 pictures within the targeted range were excluded from the study.

Conditioning phase. We used the four USs and the four individually selected CSs to construct four CS-US pairs. We randomly assigned two of the neutral pictures (CS) to positively valenced USs. The CS–US pairs were fixed random such that for each participant, a particular CS was always paired with the same US, and the order of CS–US pairs was randomized. The conditioning phase consisted of 8 cycles. In one conditioning cycle, all four CS-US pairs were presented once, leading to a total of 32 trials. In each trial, the CS was presented alone for 1000 ms, then together with the US for 3000 ms. The CS and US occurred randomly on the left or right-hand sides of the screen. The inter-trial interval was 1000 ms. At the beginning of the conditioning phase, participants were instructed to form an impression of the upcoming picture pairs and to make a judgment on each pair.

Post-conditioning phase. Participants were asked to rate the valence of each CS presented individually in random order on a 7-point rating scale (-3 = “very negative”, to +3 = “very positive”).

Contingency awareness measure. Contingency awareness was assessed with a forced-choice procedure. Participants were asked to indicate with which US they thought each CS had been presented. Each CS was presented in the upper part of the screen and the four USs below it. The location of the correct and incorrect USs was randomized. Participants had to click on the US picture they thought had been paired with the CS.

US post-conditioning evaluation. Participants rated the USs in random order on valence (-3 = “very negative”, to +3 = “very positive”).

The 12 Negative Emotionality items of the complete version of the BFI-2 assess general Negative Emotionality, and its three facets: Anxiety, Depression, and Emotional Volatility. The scale uses a 5-point format ranging from 1 (strongly disagree) to 5 (strongly agree).

The 32 Emotionality items of the complete version of the HEXACO-PI assess Emotionality, and its four facets (8 items for each facet): Fearfulness, Anxiety, Dependence, and Sentimentality. Participants responded on 5-point scales ranging from 1 (strongly disagree) to 5 (strongly agree).

A short version of the HEXACO-PI (Ashton & Lee, 2009; Lee & Ashton, 2004) assesses the other 5 personality factors (Honesty-Humility, Extraversion, Agreeableness, Conscientiousness, and Open to experience) on 5-point scales from 1 (strongly disagree) to 5 (strongly agree).

The neuroticism dimension consists of 24 items assessing Neuroticism, and its six facets: Anxiety, Anger, Depression, Self-consciousness, Immoderation, and Vulnerability. The scale uses a 5-point format ranging from 1 (strongly disagree) to 5 (strongly agree).

The first hypothesis (H1) is that we expected to find an EC effect. The second hypothesis (H2) concerned the effect of Neuroticism as a moderator of the EC effect. Based on Vogel et al. (2019), we expected that Neuroticism, and in particular the facet of Anxiety, moderated the EC effect, meaning that the EC effect should be larger for higher scores in Neuroticism and Anxiety. Further tests concerned how neuroticism moderated the EC effect. We had two main competing possibilities. On the one hand, one could argue that Neuroticism (and its relevant facets) is related to the tendency to evaluate stimuli as more valenced than they normatively are and, consequently, the USs might be evaluated as more valenced (H3a, perception), ultimately leading to a larger EC effect. Research has shown that a focus on valence enhances the EC effect (Gast & Rothermund, 2011) and that changes in the valence of the USs lead to corresponding changes in the EC effect (US revaluation, Walther et al., 2009). On the other hand, one could argue that Neuroticism (and its relevant facets) is related to a stronger reaction to valenced stimuli (US) (H3b, reaction), consequently leading to a larger EC effect. As anticipated in the Analytic strategy, to examine these possibilities, we performed a moderated mediation mixed-model analysis where the US evaluation would mediate the relation between the US condition and the CS evaluation, examining:

• the relation between US condition and CS evaluation (total/simple effect effect “c”),

• the relation between US condition and US evaluation (indirect effect “a”),

• the relation between US evaluation and CS evaluation (indirect effect “b”).

For all these paths, we tested whether Neuroticism would be a significant moderator (Figure 1). A different pattern of results can support the two competing alternatives. If Neuroticism moderates indirect effect “a”, it would support the first possibility (H3a, perception) whereas if it moderates indirect effect “b”, the second possibility would be supported (H3a, reaction). Note that, in the Figure, the total effect “c” corresponds to the main hypothesis (H1), whereas the moderating effect of Neuroticism and facets on “c” corresponds to H2.

A general Neuroticism dimension was obtained by performing a Principal Component Analysis on the facet level scales related to Neuroticism (i.e., HEXACO-PI: Fearfulness, Anxiety, Dependence, and Sentimentality; BFI-2: Depression, Anxiety, and Emotional Volatility; IPIP: Anxiety, Anger, Depression, Self-Consciousness, Immoderation, and Vulnerability). As expected, results support a one-factor solution, explaining 49.83% of the total variance. The loadings shown by the facets on the extracted factor ranged from .36 (HEXACO-PI: Sentimentality) to .88 (IPIP: Anxiety).

Table 1 shows the sample’s mean level for Neuroticism and its facets. All the scales and facets showed good reliability except for the IPIP Immoderation. Table 1 also shows Pearson zero-order correlations between the personality measures, including Neuroticism and its facets with CS ratings and pre-post CS ratings and the EC effect.¹ The Neuroticism dimension did not significantly correlate with the EC effect, whereas the Emotionality factor of the HEXACO-PI did. The general Neuroticism factor considering all facets related to Neuroticism and using factor scores, did not correlate significantly with the EC effect. However, Neuroticism facets, especially Vulnerability (IPIP) and Anxiety (HEXACO-PI, BFI-2, IPIP) did correlate significantly. Thus, we can anticipate those facets most likely moderate the EC effect. Based on these preliminary results, we decided to run the subsequent analyses on the four significant neuroticism facets and the general Emotionality factor.

Results revealed a significant main EC effect, indicating that people with a medium level of Neuroticism liked CSs paired with positive USs more relative to CSs paired with negative ones (H1), in all the considered models (Table 2). For example, results from Model 1 indicated that the liking of CSs was, on average, higher when paired with positive USs relative to negative ones for people with an average level of anxiety. The effect of anxiety (HEXACO-PI, BFI-2, IPIP), vulnerability (IPIP), and the Emotionality factor (HEXACO-60) was non-significant across these models, indicating that Neuroticism did not affect the liking of CS paired with negative USs. Instead, significant positive interactions emerged between all the Neuroticism facets and the Emotionality factor with positive USs, showing that the effect of positive US was modulated by one’s levels of anxiety, vulnerability, and the general factor of Emotionality. Findings from the simple slope analysis indicated a similar pattern: the effect of positive US is significant for high, average, and low levels of anxiety, vulnerability, and Emotionality, but it became stronger as the score in the trait increased. All models showed the same pattern: The more anxious, vulnerable, and emotional people are, the more they like CS paired with a positive US.

As Table 3 illustrates, the association between US condition and US evaluation (indirect effect “a”) was significant in all the models considered (Model 6 to Model 10), indicating that normative differences in the paired US were reflected in their personal evaluation by participants, that is the positive versus negative selected USs tended to be judged as positive versus negative by the participants. The effect of Neuroticism was also significant and negative. Note that this effect is conditional on including the US condition as a predictor, in addition to the interaction effect that however is small and not significant. Therefore, what is predicted is the portion of residual variance in the US evaluation not predicted by the US condition. As Neuroticism increased, participants evaluated more negatively the USs compared to what normatively they have been evaluated. On the contrary, no significant interactions emerged between Neuroticism and US condition. Overall, this result partly disconfirmed this hypothesis. Neuroticism seems to affect only the perception of negative USs as more negative than they are for other people. Hence, there is no evidence that it is via this perception process that Neuroticism moderates the US evaluation, hence leading to larger EC effects.

As shown in Table 4, the indirect effect (“b”) of US evaluation on CS was significant in all the models tested (Model 11 to Model 15), indicating that as the US evaluation became more positive, the CS evaluation also became more positive. As such, people preferred CS paired with more liked USs. In all the considered models, significant interactions also emerged between neuroticism facets and US evaluation, except for the Emotionality factor, which fell just short of statistical significance. Simple slopes analyses showed that as Neuroticism facets increased, the effect of US evaluation on CS evaluation also increased, meaning that the EC effect was larger. This result supported the possibility that facets of Neuroticism are related to an increased reaction to individually evaluated USs, hence leading to a larger EC effect. Further, support to our hypothesis is that the introduction of US evaluation in the model suppressed the validity of the CS condition effect in all the models tested (except for Models 13 and 14), showing that the idiosyncratic US evaluation fully mediates the CS condition (normative US evaluation) in determining the EC effect.

To sum up the results, the effect of the experimental conditions on the CS evaluation appears to be mediated, hence explained, by the participants’ evaluation of the US, which in turn leads to larger CS effects for more anxious and vulnerable people. The results showed also a negative effect of the US condition on the CS evaluation.² Given that this effect is residual to the effect of US evaluation, the two variables are highly correlated, and in isolation they have both a positive correlation with the CS evaluation, the suppressor effect could be due to multicollinearity. Therefore, we cannot exclude that it is a statistical artifact. However, a substantive interpretation of the effect suggests that the larger the discrepancy between normative and idiosyncratic US evaluation, the smaller is the EC effect. For example, this means that if normatively positive USs were idiosyncratically judged as less positive, the EC effect becomes smaller.

We first computed a Contingency Awareness score ranging from 1 to 4, summing the participants’ responses. Most participants were contingency aware in this study, with 90.3% having a score of 3 or 4. None of the correlations between CA score and the EC indexes was significant, indicating that CA did not significantly moderate the effect, probably due to a very small percentage of contingency unaware participants. The Neuroticism facets did not correlate significantly with the CA (all r’s smaller than |.10|). Given the highly skewed scores’ distribution, we divided the sample in high CA (score > = 3, N = 223) and low CA (score <= 2, N = 19). An independent-sample t-test revealed that for participants with high CA the EC effect was stronger (M_High = 1.45, SD = 1.86) than for participants with low CA (M_Low = .76, SD = 1.18), t(240) = -2.32, p = .029.

In the present contribution, we tried to extend the EC literature by examining whether and how the personality trait of Neuroticism would moderate the EC effect. More specifically, we first sought to replicate results from two previous studies (Bunghez et al., 2022; Vogel et al., 2019), investigating whether neuroticism moderates the EC effect. Then, we went a step further, focusing on how Neuroticism affects the EC effect.

Our hypothesis about the moderating role of Neuroticism was partially confirmed. We did not find the general dimension of Neuroticism to significantly moderate the EC effect, but Emotionality of the HEXACO-PI moderated the effect. In addition, we did find some of the Neuroticism facets to be significant moderators, namely anxiety and vulnerability. This result is consistent with Vogel et al. (2019) and Bunghez et al. (2022). More specifically, Vogel and colleagues (2019) found anxiety to be the facet most responsible for the relationship between Neuroticism and the EC effect. Bunghez and colleagues (2022) found Emotionality to be a significant moderator of this effect. Anxiety and vulnerability are the core features of Neuroticism, and describe people who constantly feel in danger or threatened (i.e., anxiety) and who are often overwhelmed and unable to manage their experience (i.e., vulnerability). Emotionality of the HEXACO-PI is a rotational variant of Neuroticism: It does not contain anger-related content but instead includes aspects related to emotional bonds and sentimentality. It is remarkable that the facet of anxiety is always a significant moderator, regardless of whether it is measured with the IPIP, with the BFI-2, or with the HEXACO-PI, hence independently of the specific way in which the construct is measured. Many studies found anxiety to be marked by vigilance for valent external stimuli, especially negative ones (Barrett, 1998; Shook et al., 2007) and by a tendency to generalize the evaluation of an object to other similar objects (Fazio et al., 2004). It is thus not surprising that people with these personality features are more sensitive to the EC effect.

Neuroticism facets amplify the EC effects of positive USs. Our results provided support to the more recent view on Neuroticism, stating that people high in Neuroticism, especially those high in anxiety, have a heightened focus also on positive valenced stimuli (Bachorowski & Braaten, 1994; Hannuschke et al., 2018; Larsen & Diener, 1987; Ng, 2009). The result helps to shed light on neurotic anxiety functioning and how people high in Neuroticism, particularly the anxious ones, deal with the experience. More specifically, this finding does not deny that neurotic individuals primarily show negative attitudes toward life and tend to experience negative emotions. However, it broadens this perspective, suggesting that those people can perceive positive stimuli and learn from them, forming positive attitudes. Many explanations seem to be plausible. According to Ng (2009), for people high in Neuroticism, the formation of positive attitudes would happen under certain circumstances, for example, when the stimulus is clearly positive, as the positive USs used in our EC procedure (i.e., images of happy people). Shechner et al. (2012) found that this also applies to anxious people.

Focusing on exploring how Neuroticism affects the EC effect, we achieved interesting results. To our knowledge, this is the first EC study addressing this issue, and we did so by testing multilevel moderated mediation models. These models allowed us to test whether the indirect effect of X on Y varied as a function of neuroticism-related facets, our moderators. Results showed that the anxiety, vulnerability, and emotionality did not moderate the relationship between the US condition and the US evaluation (indirect effect “a”). Therefore, the results did not support the hypothesis that anxious, vulnerable, or emotional people evaluate stimuli as more valenced than people usually do. That is, despite their focus on valence, it does not seem as if they perceive stimuli as more valenced than other people would do. Instead, anxiety, vulnerability, and emotionality moderated the relationship between US evaluation and CS evaluation significantly (indirect effect “b”), supporting the hypothesis that people high in some aspects related to Neuroticism tend to react more strongly to valenced stimuli, hence leading to a larger EC effect. In other words, anxious, vulnerable, and emotional people, while they evaluate US as more negative than other people do, this perception does not moderate the effect of US condition on US evaluation. Still, they seem to react more than other people do to the idiosyncratically evaluated valence of the USs. In brief, one could say that they react more to the valence of the US.

Taken together, these results show that people scoring high on the Anxiety facet of Neuroticism react more intensely to CSs paired with positive USs, which could be a coping/ reassuring way meant to decrease the level of threat they detect in the environment.

Notably, when controlling for the indirect effect of US evaluation on CS evaluation, the total effect of US condition on CS evaluation became negative. This result seems to suggest that the pairing of USs with CSs exerts its influence on the subsequent evaluation of CSs as a function of the valenced evaluation of the USs. In other words, the more the USs are perceived as positive versus negative, the larger their influence on the evaluation of the CSs that have been paired with them.

One could argue that the extent to which the results are due to demand effects (Corneille & Lush, 2023).³ However, based on the traits we measured in this study, we have reasons to believe demand effects are unlikely to explain our results. A priori, a demand effect should be greater for conscientious and altruist people. One of the key features of conscientious people is that they tend to follow social norms and abide by rules (Roberts et al., 2009). One of the key features of agreeable people is that they tend to be likable, collaborative, and attentive to other people’s needs and demands (Graziano & Tobin, 2009). Therefore, individual differences in these traits should be conceptually related to potential demand effects. Hence, to the extent that the results are due to demand effects, these two traits should positively correlate with the EC effect. We did not find any evidence of a correlation between these two traits and the EC effect (r = -.04 and r = .08, respectively). Even considering only the trait of Neuroticism, referred to as Emotionality in the HEXACO-PI, there are two facets that a priori could be related to demand effects, to the extent that this effect is larger for participants more eager to establish “emotional bonds”: Dependence, which assesses one’s need for emotional support from others, and Sensitivity, which assesses a tendency to feel strong emotional bonds with others. Yet, neither facet correlates significantly with the EC effect (r = -.05 and r = .04, respectively). Therefore, the pattern of results is not supportive of an explanation in terms of a demand effect.

The present study is not without limitations. Concerning the EC procedure, the US evaluation came after the conditioning phase. An ideal model in which the US evaluation predicts the CS evaluation would require that the former have been measured before the latter. Moreover, we cannot exclude the possibility that the repeated exposure to the US images during the conditioning phase would have impacted their subsequent evaluation, making the latter more extreme. On the other hand, presenting the US evaluation before the EC paradigm could have had a potential influence on the EC effect because of the same reason as before. Therefore, we preferred to avoid this potential influence and decided to present the US evaluation after the EC paradigm.

One could argue that also the USs should have been idiosyncratically selected like it was done for the CSs. Future studies could introduce a pre-conditioning evaluation of USs to allow for an individual selection of the stimuli. Likewise, the USs and CSs used in the experiment were not salient in terms of their personal relevance for the participants in the study. For instance, the use of salient USs (e.g., personally relevant USs) could have end up with a different pattern of results, such as an asymmetric transfer of valence from USs to CSs in the direction of favoring negative valence transfer. Such an outcome would have been in line with other evidence from the conditioning literature outside the EC framework, such as the absence of the relationship between Neuroticism and appetitive conditioning (Klucken et al., 2019), and the presence of the relationship between Neuroticism and fear conditioning (Lonsdorf & Merz, 2017).

One limitation of the study is that we used only four different USs. Therefore, there is more likelihood of stimulus-specific effects. Future studies should include a larger set of USs. Concerning Contingency Awareness, our measure and the analytical treatment were not ideal. However, most participants (above 90%) were contingency aware and contingency awareness did not have a substantial influence on the results. Besides, this was not the main focus of this contribution.

Despite the limitations, our study represents one of the first forays into Neuroticism’s role for EC and, especially, into the processes underlying their relationship. From this perspective, we believe that the current research has some significant merits. First, it extends our knowledge of the EC, investigating the role that a moderator situated at a personal level, Neuroticism and its related facets and alternative conceptualizations, plays in it. Second, the present study sheds initial light on how specific facets and conceptualization of Neuroticism exert their influence on EC effect, suggesting that it is more at the level of the reaction to valenced stimuli than at the level of their perception. Future research should replicate and extend these results. Transferring our attention to the EC paradigm, the present study underlined the importance of an idiosyncratic evaluation of USs. Based on our findings and the literature on attitude formation (e.g. Fazio, 2001), the idiosyncratic evaluation of the US leads to a stronger association between the latter and CS, thus a stronger EC effect, than a normatively evaluated one.

On behalf of all the authors (MP, JR, FAS) of this contribution, I (EC) hereby declare that the disclosed information is correct and that no other situation of real, potential or apparent conflict of interest is known to us. FAS and MP are Associate Editors at Collabra. They were not involved in the review process of this article.

Contributed to conception and design: EC, JR, MP

Contributed to acquisition of data: EC

Contributed to analysis and interpretation of data: EC. MP

Drafted and/or revised the article: EC, JR, MP, FAS

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 952464

All the materials (Inquisit script) and the participant data can be found on this link:

https://osf.io/7qcrp/