Evaluative Conditioning (EC), the change in liking towards a neutral stimulus due to its pairing with a positive/negative unconditioned stimulus, is a central effect in attitude formation. Current research emphasizes the role of explicit memory in EC. However, human memory is no passive information-storage device, but people actively monitor and control their own memory processes. In the present research, we examined whether people can monitor their memory processes in attitude formation via EC and let participants predict whether they will remember the stimulus pairings in the future (judgments of learning, JOLs). In seven preregistered experiments, judgments of learning predicted actual memory of stimulus pairings above chance, showing that people can indeed monitor their memory in EC. Higher JOLs were also associated with stronger EC effects. Surprisingly, actual memory explained this effect only to a small degree. Following a Brunswikian perspective, we identified several variables contributing to the correlation between JOLs and the EC effect, such as the extremity of the unconditioned stimuli, the fit between conditioned and unconditioned stimuli, and the feeling of processing ease, which all correlated with higher JOLs and stronger EC effects. Further experiments showed the robustness of these effects across different boundary conditions, such as whether judgments and memory tests target the valence or the identity of the stimuli. Our results attest to the role of metamemory in attitude formation via EC, whereby expecting that one will remember a stimulus predicts actual memory but also the size of the EC effect over and above actual memory. By integrating two previously unrelated research areas, our studies provide important theoretical insights into both attitude formation and metamemory.

Understanding how people come to their attitudes has always been a central question in psychology (Allport, 1935; Vogel & Wänke, 2016). One central insight from previous decades of attitude research is that people change their attitudes toward objects based on their pairing with positive or negative stimuli in the environment. This effect is known as Evaluative Conditioning (EC), defined as the change in liking towards a conditioned stimulus (CS) due to its pairing with a positive or negative unconditioned stimulus (US; De Houwer, 2007). For example, encountering an unknown brand (CS) with a celebrity in an advertisement (positive US) may make people like this brand (Sweldens et al., 2010; Vogel & Wänke, 2016).

EC is a prominent research area in social and cognitive psychology (for recent reviews, see Hütter, 2022; Moran et al., 2023), primarily due to the theoretical insights into attitude formation gained from EC research. EC offers a parsimonious paradigm for gathering knowledge on the cognitive processes underlying attitude formation, such as the role of reasoning (Béna et al., 2023), information search (Hütter et al., 2022; Niese & Hütter, 2023), attention (Alves et al., 2020), and central to this research – memory. Current theories explain EC effects as the outcome of a memory-based judgment process (Gast, 2018; Stahl & Aust, 2018). According to these accounts, people need to explicitly remember the US valence during the conditioning, retrieve it when encountering the CS after the conditioning, and apply it to the judgment. These memory-based accounts have received good empirical support in the last decade (Corneille & Stahl, 2018; Hütter, 2022; Moran et al., 2023; Stahl et al., 2023).

In the present research, we build on these memory-based explanations but go beyond examining the role of memory in EC effects. The main question we tackle in this paper concerns what conditioned individuals themselves think and know about their own memory processes during the acquisition of conditioned attitudes. If EC effects are based on memory for the stimulus pairings, do conditioned individuals have insight into these memory processes? And does having insight into one’s own memory processes relate to the strength of EC effects? More specifically, we investigate how and why judgments about one’s memory for CS-US pairings relate to memory and EC effects. By integrating two previously unrelated research areas – EC and metamemory – our research reveals new insights into the cognitive processes involved in EC. Further, it extends the understanding of metamemory by investigating its role in a new domain in metamemory research, specifically, learning processes in attitude formation.

In the following, we will briefly discuss EC and the role of explicit memory in EC. Next, we will introduce the research area of metamemory, in particular, research on judgments of learning (JOLs). Afterward, we will discuss the theoretical relevance of studying metamemory in EC (both for metamemory and EC research), before we derive predictions on the relation between metamemory and EC.

EC is defined as the change in liking towards a conditioned stimulus (CS) due to its pairing with a positive or negative unconditioned stimulus (US; De Houwer, 2007). This functional definition describes an empirical effect (change in liking) due to certain environmental regularities (stimulus pairings), but it is mute about the underlying cognitive process. Accordingly, there has been a lot of theoretical debate on these cognitive processes in the last decades (for reviews, see Corneille & Stahl, 2018; Hütter, 2022; Moran et al., 2023). Importantly, different process explanations of EC make different predictions regarding the role of explicit memory.

One process explanation is that people misattribute the affect elicited by the US to the CS (Jones et al., 2009; March et al., 2019). Such misattribution does not require explicit memory. In fact, some researchers argue that remembering that a CS was paired with a positive or negative US even hinders affect misattribution (March et al., 2019). Another theoretical perspective comes from propositional accounts (De Houwer, 2018; see also Gawronski & Bodenhausen, 2018). They propose that people make inferences on the CS-US pairings. Such inferences can be, for instance, “stimulus pairings signal similarity, and therefore a CS paired with a positive US is also positive”. There are different views to what extent such propositions require explicit memory of the stimulus pairings (De Houwer, 2018). Still, they are considered to be primarily deliberate and controlled and benefit from the same conditions as explicit memory (Hofmann et al., 2010; Hütter & De Houwer, 2017). Finally, there are process explanations of EC purely based on explicit memory (Gast, 2018; see also Stahl & Aust, 2018). Here, the basic assumption is that people must consciously store the information that CS and US were paired and retrieve this information when making an evaluative judgment.

Overall, the empirical evidence clearly attests to the role of explicit memory in EC (Corneille & Stahl, 2018; Hütter, 2022; Moran et al., 2023; Stahl et al., 2023). For example, EC effects seem to occur only under conditions where people can explicitly encode the stimulus pairings (Högden et al., 2018; Mierop et al., 2017; Stahl et al., 2016) or when they can retrieve the stimulus pairings at the time of the judgment (Gast et al., 2012; Ingendahl, Woitzel, Propheter, et al., 2023; Stahl et al., 2009; but see also Hütter et al., 2012; Hütter & Sweldens, 2013; Ingendahl, Vogel, et al., 2023). Likewise, EC effects benefit from manipulations that benefit explicit memory, such as distributed learning (Richter & Gast, 2017) or short retention intervals (Förderer & Unkelbach, 2013; Hütter et al., 2012). Based on this evidence, it is safe to assume that EC effects substantially depend on explicit memory of the stimulus pairings.

However, what conditioned individuals themselves think and know about their own memory processes has been neglected in EC research. Human memory is no information-storage device that passively stores and retrieves information. Instead, people actively monitor their own memory processes, as demonstrated in numerous studies on metacognition and metamemory.

Metacognition is the “knowledge and cognition about cognitive phenomena” (Flavell, 1979, S. 706) and includes people’s monitoring, control, and knowledge of cognitive processes. Metacognition research offers important insights into what people know about their cognitions and how they regulate their cognitive processes. One particularly prominent area of metacognition is metamemory, that is, monitoring and control of one’s learning and memory. Studying metamemory provides information about what people know or believe about how learning and memory work and whether this metacognitive knowledge is accurate. For example, metamemory research shows that people are at least somewhat aware that emotional or traumatic information is better remembered than neutral information (emotionality effect; e.g., Witherby & Tauber, 2018; Yin et al., 2023).

Researchers commonly rely on self-assessments for studying metamemory (Rhodes, 2016). One example is the judgment of learning (JOL; Arbuckle & Cuddy, 1969; Rhodes, 2016), which is a prospective judgment about the likelihood of remembering a studied item on a future memory test. JOLs are a standard measure of metamemory monitoring and have been solicited for various study materials, such as words (e.g., Koriat, 1997; Undorf & Bröder, 2020), word pairs (e.g., Undorf & Erdfelder, 2015), scene pictures (Undorf & Bröder, 2021), facial pictures (e.g., Witherby & Tauber, 2018), word triads (Undorf & Zander, 2017), or general-knowledge facts (e.g., Schäfer & Undorf, 2024).

People’s JOLs are typically moderately accurate in predicting memory performance in free-recall tests (e.g., Rhodes & Castel, 2008), cued-recall tests (e.g., Koriat, 1997; Undorf & Erdfelder, 2015), and recognition tests (e.g., Undorf & Bröder, 2021). One well-supported explanation for people’s moderate accuracy of JOLs is the cue-utilization account (Koriat, 1997), a variant of Brunswik’s lens model (1956). This account proposes that people make JOLs based on cues perceived during learning. Depending on which cues people use and how much these cues correlate with actual memory, people’s JOLs will be more or less accurate. Such cues can be internal cues of the items (e.g., word frequency) or external cues of the study situation (e.g., number of study repetitions). Cues impact people’s JOLs by either an experience-based process, such as ease of encoding or processing (e.g., Koriat & Ma’ayan, 2005; Undorf & Erdfelder, 2015), or by a theory-based process, namely beliefs about how cues affect memory and learning (e.g., Hu et al., 2020; Mueller & Dunlosky, 2017). In line with the cue-utilization account, people’s JOLs are sensitive to a multitude of cues, for example, font size (e.g., Rhodes & Castel, 2008), word frequency (e.g., Jia et al., 2016), retention interval (Koriat et al., 2004), cognitive conflict (Li et al., 2021), or idiosyncratic importance (Undorf et al., 2022). Furthermore, people strategically integrate multiple cues when making JOLs (Undorf & Bröder, 2020).

In the present research, we connect the two research areas of EC and metamemory. Specifically, we investigate when, why, and how people can predict the future retrieval of stimulus pairings and how these predictions relate to the strength of the EC effect. Doing so can offer valuable insights both for EC and metamemory research.

Implications for EC research

Studying the role of metamemory in EC can tell us about the cognitive processes underlying EC, specifically, to what extent people can monitor their own memory when forming attitudes from stimulus pairings. This insight is valuable for one of the most central questions in EC research – to what extent EC effects depend on awareness of the stimulus pairings (e.g., Corneille & Stahl, 2018; Moran et al., 2023; Sweldens et al., 2014). This question has most frequently been tested with post-conditioning memory tests for the stimulus pairings (Gawronski & Walther, 2012), showing that EC largely depends on memory. Yet, previous research has neglected that conditioned individuals themselves might proactively monitor and perhaps also regulate their awareness of stimulus co-occurrences (e.g., Sperlich & Unkelbach, 2022). That is, awareness may not only result from situational factors, but also from metacognitive processes of the conditioned individual. Note that EC is commonly understood as a type of incidental learning, and EC paradigms usually do not instruct participants to memorize the CS-US pairings. Nevertheless, this does not preclude people from monitoring their learning (e.g., Serra & Ariel, 2014; Tekin & Roediger, 2020). If people can predict the future recollection of stimulus pairings, then this would show that the formation of memory traces in attitude formation can be monitored by the conditioned individual (see also Hahn et al., 2014).

Crucially, if people can indeed monitor their own memory during attitude formation via EC, then they can potentially even proactively regulate their memory (for the role of controllability in EC effects, see Hütter & Sweldens, 2018). Thus, if people are to some extent aware of their memory in EC, then this opens the possibility of several previously unidentified top-down processes during learning (Sperlich & Unkelbach, 2022). Specifically, people might mentally rehearse specific pairings, for example, because they are deemed to be relevant for a salient goal (Moran et al., 2022), or because they break expectations built from previous stimulus co-occurrences (Alves et al., 2020). Investigating to which extent metamemory judgments predict future memory about stimulus co-occurrences and the EC effect, and furthermore, under which conditions and based on which cues, could give valuable insights into the mental operations that conditioned individuals proactively execute when exposed to stimulus co-occurrences in social settings. It would also show how accurate and feasible these mental operations are. For example, if one observes that conditioned individuals base metamemory judgments primarily on salient but irrelevant features about a stimulus pairing (e.g., the color of the stimuli), then this would imply that people cannot anticipate when they will remember stimulus co-occurrences and when they will show attitude change towards a stimulus. In contrast, if one observes that metamemory judgments are ecologically valid, such that they indeed predict future memory, then this would show that people have sufficient understanding of their own memory for stimulus co-occurrences to potentially regulate their learning.

Implications for metamemory research

Studying the role of metamemory in EC also provides new insights into metamemory. First, studying attitude formation via stimulus pairings opens an entirely new domain for metamemory research. Our research can thus test whether insights from previous metamemory research generalize outside the word list paradigms that dominate metamemory research. Such an endeavor is crucial because metamemory is assumed to play a substantial role across various types of learning, especially in educational and social settings (e.g., Soderstrom et al., 2016).

Second, our research can also test to what extent metamemory judgments relate to more distal downstream consequences of memory. Previous metamemory research has primarily focused on either metamemory judgments alone (e.g., which cues people rely on) or the relationship between metamemory judgments and memory. To the best of our knowledge, there is no metamemory research studying when, why, and how metamemory judgments predict consequences of memory – in this context, how metamemory judgments relate to memory-based attitude change from stimulus pairings.

In seven preregistered experiments, we investigated the role of metamemory in EC. In each experiment, participants underwent an EC procedure where neutral CSs were shown together with positive, neutral, or negative USs (i.e., pictures or words). At the end of the EC procedure, we assessed JOLs by asking participants to predict the likelihood of remembering the USs when presented with the CSs. Afterward, participants evaluated the CSs and completed a memory test for the stimulus pairings. This procedure was altered in some of the experiments to test specific predictions.

Our initial expectations on the relationship between EC, memory, and metamemory were as follows: Based on previous EC research, EC effects should substantially benefit from memory. Based on previous metamemory research, we expected that people can monitor their learning such that JOLs are moderately accurate in predicting memory. Therefore, higher JOLs should also go along with stronger EC effects. That is, if people predict they will remember that a CS co-occurred with a positive or negative US, then they should show a stronger EC effect for this CS. This reasoning also implies that JOLs should no longer be associated with EC effects when actual memory performance is controlled for. We discuss alternative explanations for an association between JOLs and EC effects after Experiment 1.

In all experiments, we tested the following hypotheses1: First, we expected an EC effect, such that CSs paired with positive (negative) USs are evaluated more positively (negatively) than those paired with neutral USs. Second, based on current memory-based EC accounts, we expected that EC effects would be stronger for pairings participants accurately remember. Based on previous JOL research, we expected JOLs to be somewhat accurate in predicting future memory such that the correlation between JOLs and memory is above zero. Fourth, following from the previous two hypotheses, we expected JOLs to moderate2 the EC effect such that the EC effect would be stronger for pairings that received higher JOLs. Fifth, we tested whether the stronger EC effect for high-JOL pairings would be incremental to actual memory performance.

Experiment 1, which introduced our basic paradigm, focused on participants’ memory accuracy as a potential explanation for the expected association between JOLs and EC. Based on the results of Experiment 1, we developed and tested alternative process explanations in Experiments 2 to 4. Specifically, we tested whether subjective US extremity (Experiment 2), CS-US fit (Experiment 3), and processing ease (Experiment 4) could account for the association between JOLs and EC. In Experiments 5-7, we further tested critical theoretical considerations regarding the basis, generalizability, and robustness of the JOL-memory and JOL-EC association by examining whether they also hold when considering valence memory instead of identity memory (Experiment 5), whether the EC effect is impacted by soliciting JOLs (Experiment 6), and whether the JOL-EC association can also be experimentally induced by a manipulation that affects both JOLs and the EC effect (Experiment 7).

Experiment 1 served as a first test of our hypotheses. For that purpose, it used an EC paradigm adapted from previous research with an additional assessment of JOLs.

Methods

Transparency and Openness

We report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in all studies, and the study follows the journal article reporting standards (JARS; Applebaum et al., 2018). All materials, data, and code are provided at https://doi.org/10.17605/OSF.IO/5HJNF. We preregistered Experiment 1 on aspredicted.org (https://aspredicted.org/mm6hk.pdf ).

Design and Participants

We varied the normed US valence (positive vs. neutral vs. negative) within subjects. There were no additional manipulations. We calculated a priori power analyses with G*Power 3 (Faul et al., 2007). As a rough approximation, we sought to detect a small to medium (f = 0.2) within-subjects EC effect with 80% power, leading to a necessary sample size of 42 participants. We decided to add a buffer of eight participants. Because we could not precisely monitor the number of participants in our universities’ study platforms, a final sample size of 52 students participated in the study (45 female, 7 male, MAge = 21.27, SDAge = 2.15; 49 German native speakers) who were recruited via our universities’ study platforms in exchange for course credits.

Procedure

We built all experiments in the software Sosci Survey (Leiner, 2019) and conducted them online, relying on a previous EC paradigm by Ingendahl and Vogel (2023). After an informed consent, we first exposed participants to an EC procedure. Afterward, participants gave JOLs. Next, they evaluated the CSs, before answering a memory test for the stimulus pairings. In the following, we will explain each task step by step. We provide detailed screenshots and materials in our OSF directory. The procedures in all experiments received approval from the local ethics committee of the university of the first author.

EC Procedure. We told participants that they would see some unfamiliar brand names presented together with pictures. They should merely look at the sequences and wait for further instructions. In the following conditioning phase, we presented 24 CSs (8 per US valence) together with valenced pictures. After a blank screen of 250 ms, a CS-US pair appeared for 2500 ms. We counterbalanced between participants whether CSs/USs were shown on the left/right on the screen. Each CS-US pair appeared four times always with the same US, leading to a conditioning phase of 24 x 4 = 96 trials shown in random order.

JOLs. Consistent with previous metamemory research, we assessed JOLs after learning (e.g., Koriat & Ma’ayan, 2005; Lee & Ha, 2019). Thus, after the EC procedure, we told participants that they would see the names and pictures once again. However, this time, they should estimate how likely they were to remember the picture if they were shown the name. We then presented each CS-US pair as in the conditioning phase, but with the additional question under each CS-US pair: “How likely is it that you will remember the picture when presented with the brand name?” Participants made their self-paced judgment on a slider ranging from 0 (will definitely not remember the picture) to 100% (will definitely remember the picture).

CS Evaluation. After the JOLs, participants had to evaluate the brand names. We presented participants with the 24 names in random order and asked, “How would you evaluate this brand name?” The scale ranged from 1 (very negative) to 7 (very positive). Each CS appeared on a single slide.

Memory. Afterward, participants had to remember the pictures that the brand names had been shown with. On each page, a single CS appeared on the top of the screen, together with a matrix of six US pictures below. There were two USs from each valence level. Participants had to select the correct US from the matrix. The position of pictures within the matrix varied randomly, and the 24 CSs appeared in random order. The distractor USs were always US pictures belonging to the other CSs, with a balanced frequency of each US across the 24 CSs (i.e., each US appeared six times). We coded whether participants selected the correct US picture (i.e., identity memory; Stahl et al., 2009).3

Materials

We used 90 pictures from the Open Affective Standardized Image Set (OASIS; Kurdi et al., 2017) as USs. The same pictures had been used by Ingendahl and Vogel (2022). Thirty pictures with valence ratings higher than +1 SD (OASIS valence rating > 5.56 on a scale of 1-7) served as positive USs, 30 pictures with valence ratings between -0.33 SD (3.99) and +0.33 SD (4.71) as neutral USs, and 30 pictures with ratings below -1 SD (2.86) as negative USs. Each set consisted of ten pictures depicting scenes, ten depicting persons, five depicting objects, and five depicting animals. We did not use pictures depicting extreme violence or nudity. The computer program randomly selected the pictures for each participant. We provide a list of all used stimuli on the OSF. We used the same 32 pseudowords (e.g., TABOMER, SOLEDAN) as Ingendahl and Vogel (2023) as CSs. For each individual participant, a random subset of 24 pseudowords served as the CSs.

Results

JOLs and Memory

Table 1.
Results from Regression Models for the Effect of Valence in Experiment 1
a) JOLsb) Memoryc) CS Evaluations
Predictors b t p b z p b t p 
(Intercept) 49.40 19.01 < .001 2.37 8.63 < .001 4.19 45.70 < .001 
Positive 3.90 2.00 .045 0.10 0.45 .650 0.48 5.08 < .001 
Negative 1.25 0.58 .564 -0.20 -0.98 .325 -0.95 -6.61 < .001 
a) JOLsb) Memoryc) CS Evaluations
Predictors b t p b z p b t p 
(Intercept) 49.40 19.01 < .001 2.37 8.63 < .001 4.19 45.70 < .001 
Positive 3.90 2.00 .045 0.10 0.45 .650 0.48 5.08 < .001 
Negative 1.25 0.58 .564 -0.20 -0.98 .325 -0.95 -6.61 < .001 

We conducted multilevel regressions in lme4 (Bates et al., 2015) with the highest converging random effect structure (i.e., slopes and intercepts on participant-level; Barr et al., 2013)4 to analyze the effects of US valence on JOLs. We dummy-coded the US valence in all models with neutral USs as the baseline. The regression weights of the two dummy variables thus represent the mean difference between positive and neutral (or negative and neutral) US conditions on the respective scale of the dependent variable. We preregistered this dummy coding over a single continuous predictor because we tested the hypothesis that JOLs are higher for emotional compared to non-emotional US valence. We used the same analytical approach to examine potential differences in memory between valence levels but with a binomial instead of a Gaussian model due to the binary outcome (“correct” or “incorrect”). The results from these models are displayed in Table 1. We provide full descriptive statistics (Ms, SDs, correlations) for all experiments at the OSF.

JOLs were higher for positive compared to neutral CS-US pairs, but there was no significant difference between the JOLs for negative and neutral CS-US pairs (Table 1a). Memory was highly accurate (84%) and did not differ across valence conditions (see Table 1b). We next investigated the correspondence between JOLs and memory by computing within-subject gamma correlations (Nelson, 1984; Rhodes & Tauber, 2011). The mean gamma correlation5 was γ = .34 and significantly above 0, t = 5.88, p < .001.

CS Evaluations

We first regressed CS evaluations only on positive and negative US valence. There was a significant effect of positive valence and negative valence, showing an EC effect (see Figure 1 and Table 1c). Next, we examined whether memory moderated the EC effect. For that purpose, we added memory to the model. Deviating from the preregistration, we standardized memory instead of dummy-coding it to make the results comparable to the analyses with JOLs as the moderator. Table 2a and Figure 1a show the results of this model. As expected, memory moderated the EC effect, such that the EC effect was stronger when memory was correct. However, the respective interaction was significant only for negative USs, but not for positive USs (relative to the neutral US condition). Instead, there was also a positive main effect of memory for the neutral US condition.

Figure 1.
Mean CS Evaluation in Experiment 1 as a Function of US Valence and Memory (a) or JOLs (b)

Note. Error bars represent 95% confidence intervals.

Figure 1.
Mean CS Evaluation in Experiment 1 as a Function of US Valence and Memory (a) or JOLs (b)

Note. Error bars represent 95% confidence intervals.

Close modal
Table 2.
Results from Regression Models for the Effects of Valence, Memory, and JOLs in Experiment 1
a) CS Evaluationsb) CS Evaluationsc) CS Evaluations
Predictors b t p b t p b t p 
(Intercept) 4.19 47.52 < .001 4.21 46.93 < .001 4.21 48.46 < .001 
Positive 0.47 5.02 < .001 0.42 4.60 < .001 0.42 4.58 < .001 
Negative -0.95 -7.08 < .001 -0.97 -6.76 < .001 -0.97 -7.16 < .001 
Memory 0.21 3.01 .003    0.16 2.32 .020 
Memory x Positive 0.06 0.58 .559    0.05 0.54 .592 
Memory x Negative -0.35 -3.51 < .001    -0.28 -2.86 .004 
JOL    0.30 4.42 < .001 0.28 4.10 < .001 
JOL x Positive    0.16 1.62 .106 0.15 1.54 .123 
JOL x Negative    -0.41 -4.33 < .001 -0.38 -3.93 < .001 
a) CS Evaluationsb) CS Evaluationsc) CS Evaluations
Predictors b t p b t p b t p 
(Intercept) 4.19 47.52 < .001 4.21 46.93 < .001 4.21 48.46 < .001 
Positive 0.47 5.02 < .001 0.42 4.60 < .001 0.42 4.58 < .001 
Negative -0.95 -7.08 < .001 -0.97 -6.76 < .001 -0.97 -7.16 < .001 
Memory 0.21 3.01 .003    0.16 2.32 .020 
Memory x Positive 0.06 0.58 .559    0.05 0.54 .592 
Memory x Negative -0.35 -3.51 < .001    -0.28 -2.86 .004 
JOL    0.30 4.42 < .001 0.28 4.10 < .001 
JOL x Positive    0.16 1.62 .106 0.15 1.54 .123 
JOL x Negative    -0.41 -4.33 < .001 -0.38 -3.93 < .001 

Afterward, we examined whether JOLs moderated the EC effect. In line with the preregistration, we standardized the JOLs within participants to account for differences in JOL level across participants.6 JOLs moderated the EC effect (Table 2b and Figure 1b). As expected, the EC effect increased with increasing JOLs. However, this moderating effect was only significant for the negative (vs. neutral) US valence condition, but not for the positive (vs. neutral) US valence condition. As visualized in Figure 1b, higher JOLs were already associated with more positive CS evaluations in the neutral US condition.

As the last step, we included both memory and JOLs into the model (see Table 2c). Surprisingly, the results were very similar to the models with separate moderations. Thus, the effect of negative US valence was moderated by both memory and JOLs. The regression weights were only slightly smaller than in the separate models, suggesting that the two moderators had independent moderating effects despite the moderate correspondence between JOLs and memory.

We nevertheless tested whether the Memory x EC moderation could statistically explain the JOL x EC moderation and conducted an indirect moderation analysis using the bruceR package (Bao, 2022). This analysis was not preregistered for any experiment. Here, we predicted the CS evaluations by valence (a single predictor coded with 1 = positive, 0 = neutral, -1 = negative), memory, JOLs, and all two-way interactions. We tested whether the JOL x Valence interaction was mediated by the Memory x Valence interaction. We provide detailed results of the underlying regression models in the Supplement (see the OSF). The mediation analysis revealed a small and nonsignificant indirect effect, b = 0.024, p = .083, 95% MCMC CI [0.000, 0.055] while showing a significant direct effect, b = 0.263, p < .001, 95% MCMC CI [0.165, 0.367]. Thus, memory did not significantly mediate the association between JOLs and the EC effect.

Discussion

Experiment 1 showed that JOLs were associated with memory accuracy in an EC paradigm, suggesting that people can indeed monitor their own memory in attitude formation via EC. Consistent with previous EC research, there was an EC effect, which was strengthened by accurate memory. Experiment 1 also confirmed our prediction that higher JOLs are associated with stronger EC effects.7 Surprisingly, however, the association between JOLs and the EC effect was largely independent of that between memory and the EC effect. That is, although JOLs predicted memory, and although memory was related to stronger EC effects, the association between JOLs and the strength of the EC effect was independent of memory. This implies that other factors than memory accuracy must underlie the relationship between JOLs and the EC effect.

We thus searched for alternative explanations of this relationship following a Brunswikian lens model perspective (Brunswik, 1956; Koriat, 1997). Specifically, JOLs might correlate above and beyond memory with the EC effect because people base their JOLs on cues that predict the strength of the EC effect but not, or to a much lesser degree, actual memory.

One potential source of covariation between JOLs and EC could be differences in the extremity of the USs. EC researchers usually select the USs from stimulus databases with standardized ratings. However, although positive USs are positive and negative USs are negative on average, there is still substantial variation in how positive or negative a particular US is to the individual participant, and this variation in the USs’ extremity has been shown to relate to the size of the EC effect in previous research (Ingendahl, Woitzel, & Alves, 2023; Ingendahl & Vogel, 2023). JOL research has shown that people judge emotional content as more memorable than non-emotional content (e.g., Hourihan et al., 2017; Undorf et al., 2018; Zimmerman & Kelley, 2010). We also observed this effect in Experiment 1, although it was significant only for positive US. Thus, if a participant perceives some USs as more extreme than others8, this could increase both the JOL and the EC effect and thus contribute to the correlation between JOLs and the EC effect. We tested this explanation in Experiment 2.

Experiment 2 tested whether differences in the subjective extremity of the US pictures contribute to the association between JOLs and the EC effect, such that stimulus pairs with more extreme USs receive higher JOLs and are also associated with stronger EC effects. We preregistered Experiment 2 on aspredicted.org (https://aspredicted.org/hu6gw.pdf ).

Methods

Design and Participants

We varied the normed US valence (positive vs. neutral vs. negative) within subjects. There were no additional manipulations. We decided to increase the sample size in this experiment to N = 70 to have sufficient power to detect even effect sizes of f = .15 with 80% power. Because we could not precisely monitor the number of participants in our universities’ study platforms, our final sample size consisted of N = 75 students (60 female, 15 male, MAge = 21.85, SDAge = 2.52; 68 native speakers) who participated in exchange for course credits.

Procedure

The experiment followed the same procedure as Experiment 1, except for the following differences: First, we reduced the number of pairings in the conditioning phase to three per CS because the memory performance had been very high in Experiment 1. Second, we assessed the subjective US valence after the memory test with the same measure as Ingendahl and Vogel (2023). Each US appeared on a single slide with a scale ranging from 1 (very negative) to 7 (very positive). The instructions were to rate how positive/negative they found each picture.

Materials

Due to the asymmetrical EC effect in Experiment 1, we used a different set of US pictures from the OASIS (Kurdi et al., 2017). Twenty pictures with valence ratings higher than 5.75 (on a scale of 1-7) served as positive USs, 20 pictures with valence ratings between 3.75 and 4.25 as neutral USs, and 20 pictures with ratings below 2.25 as negative USs. Each set consisted of five pictures of each picture category. We provide a list of the stimuli on the OSF.

Results

JOLs, Memory, and US Evaluations

We used the same analytical approach as for Experiment 1. Table 3 shows the results from the multilevel regression models, first only with the positive and negative dummy as predictors. JOLs were descriptively but not significantly higher for positive than for neutral USs (see Table 3a). Memory was still very accurate (76%) but lower than in Experiment 1. There were no significant differences in memory between the valence conditions (Table 3b). The average gamma correlation between JOLs and memory was γ = .28 and significantly above 0, t = 6.05, p < .001. Following the preregistration, we also analyzed whether positive USs were more positive and whether negative USs were more negative to participants than neutral USs, which was the case (Table 3c).

Table 3.
Results from Regression Models for the Effects of Valence in Experiment 2
a) JOLsb) Memoryc) US Evaluationsd) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 43.39 22.02 < .001 1.63 8.55 < .001 3.72 58.86 < .001 4.00 58.95 < .001 
Positive 3.55 1.94 .056 0.11 0.60 .551 2.40 20.49 < .001 0.71 6.27 < .001 
Negative -0.74 -0.40 .688 -0.27 -1.84 .066 -1.92 -31.24 < .001 -0.53 -5.72 < .001 
a) JOLsb) Memoryc) US Evaluationsd) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 43.39 22.02 < .001 1.63 8.55 < .001 3.72 58.86 < .001 4.00 58.95 < .001 
Positive 3.55 1.94 .056 0.11 0.60 .551 2.40 20.49 < .001 0.71 6.27 < .001 
Negative -0.74 -0.40 .688 -0.27 -1.84 .066 -1.92 -31.24 < .001 -0.53 -5.72 < .001 

Next, we computed US extremity scores by taking the absolute difference between the US evaluation and the neutral scale midpoint. Thus, US extremity could range from 0 (neutral evaluation) to 3 (highly positive or negative evaluation). As expected, this US extremity score had a significant correlation with JOLs, r = .15, p < .001, independent of whether we used the raw or within-person-centered scores. There was no correlation between memory and US extremity, r = .04, p = .112.

CS Evaluations

As in Experiment 1, we first regressed CS evaluations on positive and negative valence, showing an EC effect (see Figure 2 and Table 3c). Next, we added memory and its respective interactions into the model. Memory had the expected moderation effect on the EC effect, such that the EC effect was stronger when memory was correct (Figure 2a and Table 4a). As in the previous experiment, JOLs also moderated the EC effect (Table 4b and Figure 2b). The EC effect was stronger for higher JOLs. We also found the expected moderation by US extremity (Table 4c and Figure 2c), such that the EC effect was stronger for more extreme US evaluations.

Table 4.
Results from Regression Models for the Effects of Valence, Memory, JOLs, and US Extremity in Experiment 2
a) CS Evaluationsb) CS Evaluationsc) CS Evaluationsd) CS Evaluationse) CS Evaluationsf) CS Evaluations
Predictors b t p b t p b t p b t p b t p b t p 
(Intercept) 4.00 58.84 < .001 4.00 60.60 < .001 3.99 49.17 < .001 4.01 60.30 < .001 3.96 49.98 < .001 3.97 49.97 < .001 
Positive 0.70 6.44 < .001 0.66 6.13 < .001 0.59 5.11 < .001 0.65 6.23 < .001 0.62 5.47 < .001 0.60 5.45 < .001 
Negative -0.54 -5.96 < .001 -0.54 -5.90 < .001 -0.41 -3.88 < .001 -0.56 -6.15 < .001 -0.40 -3.85 < .001 -0.42 -4.10 < .001 
Memory 0.01 0.17 .861       -0.03 -0.45 .653    -0.03 -0.46 .644 
Memory x Positive 0.34 4.10 < .001       0.27 3.23 .001    0.26 3.24 .001 
Memory x Negative -0.20 -2.53 .011       -0.13 -1.61 .107    -0.13 -1.63 .103 
JOL    0.20 3.52 < .001    0.21 3.80 < .001 0.21 3.68 < .001 0.21 3.89 < .001 
JOL x Positive    0.28 3.69 < .001    0.23 2.90 .004 0.22 2.88 .004 0.17 2.14 .032 
JOL x Negative    -0.45 -5.68 < .001    -0.43 -5.43 < .001 -0.44 -5.57 < .001 -0.42 -5.27 < .001 
US extremity       -0.02 -0.27 .789    -0.06 -1.03 .302 -0.05 -0.83 .408 
US extremity x Positive       0.44 4.81 < .001    0.36 3.97 < .001 0.35 3.89 < .001 
US extremity x Negative       -0.27 -2.87 .004    -0.19 -2.06 .040 -0.21 -2.28 .023 
a) CS Evaluationsb) CS Evaluationsc) CS Evaluationsd) CS Evaluationse) CS Evaluationsf) CS Evaluations
Predictors b t p b t p b t p b t p b t p b t p 
(Intercept) 4.00 58.84 < .001 4.00 60.60 < .001 3.99 49.17 < .001 4.01 60.30 < .001 3.96 49.98 < .001 3.97 49.97 < .001 
Positive 0.70 6.44 < .001 0.66 6.13 < .001 0.59 5.11 < .001 0.65 6.23 < .001 0.62 5.47 < .001 0.60 5.45 < .001 
Negative -0.54 -5.96 < .001 -0.54 -5.90 < .001 -0.41 -3.88 < .001 -0.56 -6.15 < .001 -0.40 -3.85 < .001 -0.42 -4.10 < .001 
Memory 0.01 0.17 .861       -0.03 -0.45 .653    -0.03 -0.46 .644 
Memory x Positive 0.34 4.10 < .001       0.27 3.23 .001    0.26 3.24 .001 
Memory x Negative -0.20 -2.53 .011       -0.13 -1.61 .107    -0.13 -1.63 .103 
JOL    0.20 3.52 < .001    0.21 3.80 < .001 0.21 3.68 < .001 0.21 3.89 < .001 
JOL x Positive    0.28 3.69 < .001    0.23 2.90 .004 0.22 2.88 .004 0.17 2.14 .032 
JOL x Negative    -0.45 -5.68 < .001    -0.43 -5.43 < .001 -0.44 -5.57 < .001 -0.42 -5.27 < .001 
US extremity       -0.02 -0.27 .789    -0.06 -1.03 .302 -0.05 -0.83 .408 
US extremity x Positive       0.44 4.81 < .001    0.36 3.97 < .001 0.35 3.89 < .001 
US extremity x Negative       -0.27 -2.87 .004    -0.19 -2.06 .040 -0.21 -2.28 .023 

Note. Memory and US extremity were standardized at the grand mean, JOLs within participants. Model c, d, and f as well as the mediation analysis were not preregistered.

Figure 2.
Mean CS Evaluation in Experiment 2 as a Function of US Valence and Memory (a) or JOLs (b) or US Extremity (c)

Note. Error bars represent 95% confidence intervals.

Figure 2.
Mean CS Evaluation in Experiment 2 as a Function of US Valence and Memory (a) or JOLs (b) or US Extremity (c)

Note. Error bars represent 95% confidence intervals.

Close modal

Next, we added both memory and JOLs, both JOLs and US extremity, and finally, JOLs, memory, and US extremity into the model (see Table 4d-f). All these steps reduced but did not eliminate the JOL x Positive and JOL x Negative interaction terms, suggesting that the moderations were incremental to what could be accounted for by memory or US extremity.

We also conducted a similar mediation analysis as in Experiment 1, where both the Memory x Valence and the US extremity x Valence interactions served as parallel mediators for the JOL x Valence interaction. This time, the mediation analysis revealed a small but significant indirect effect through memory, b = 0.038, p < .001, 95% MCMC CI [0.019, 0.062], and also a small but significant indirect effect through US extremity, b = 0.038, p = .010, 95% MCMC CI [0.010, 0.066]. The direct effect remained significant, b = 0.293, p < .001, 95% MCMC CI [0.217, 0.373]. Again, we provide detailed results from the underlying regression models in the Supplement (see the OSF).

Discussion

Experiment 2 replicated the findings from Experiment 1. Higher JOLs were associated with actual memory and stronger EC effects. Again, the association between JOLs and the EC effect was incremental to memory, although JOLs were moderately accurate in predicting memory. In Experiment 2, we further tested subjective US extremity as one potential factor that might account for this association. Although US extremity was indeed associated with higher JOLs and, as expected, US extremity moderated the EC effect, the association between JOLs and the EC effect was also incremental to US extremity. The mediation analysis suggests that memory and US extremity might account for a small part of the association but that further processes contribute to the effect. We therefore aimed to identify these additional processes in Experiment 3.

A robust finding from JOL research is that JOLs are highly sensitive to the relationship between the elements of pairs (Mueller et al., 2013; Undorf & Erdfelder, 2015). For example, related word pairs (e.g., “dog-cat”) receive higher JOLs than non-related word pairs (e.g., “dog-washing machine”). Likewise, EC effects are also highly sensitive to the relationship between CS and US (Bading et al., 2020; Gawronski, 2022; Kurdi et al., 2023; Unkelbach & Fiedler, 2016). Based on the propositional account (De Houwer, 2018), EC effects depend on the inferred relation between CS and US. Accordingly, we expected that the more strongly related the two stimuli of a pair appear, the higher the JOL and the stronger the EC effect. One important aspect of such a stimulus relation is how well the stimuli fit together. We expected the positive association between JOLs and the EC effect to arise because people may perceive a stronger fit between some CS-US pairs, which benefits both JOLs and the EC effect. In Experiment 3, we therefore tested the subjective CS-US fit as a source of covariation between JOLs and the EC effect.

Experiment 3 tested whether the association between JOLs and the EC effect could be due to differences in the perceived fit of CS and US, which is associated with both JOLs and the EC effect. We preregistered Experiment 3 on aspredicted.org (https://aspredicted.org/77y7k.pdf ).

Methods

Design and Participants

We varied the normed US valence (positive vs. neutral vs. negative) within subjects. There were no additional manipulations. We had the same planned sample size of N = 70 as in Experiment 2. Because we could not precisely monitor the number of participants in our universities’ study platforms, our final sample size consisted of N = 71 students (52 female, 18 male, 1 diverse, MAge = 21.76, SDAge = 3.54; 66 native speakers) who participated in exchange for course credits.

Procedure

The experiment followed the same procedure as the previous experiments, except for the following differences: First, we also assessed subjective fit judgments after the conditioning phase. Participants saw each stimulus pair as in the JOL task but judged them regarding “How well do picture and brand name fit together?” on a slider from 0 (not at all) to 100 (very much). We counterbalanced between participants whether they first provided all JOLs or all fit judgments. We did not assess individual US evaluations in this experiment.

Materials

The materials were identical to Experiment 2.

Results

JOLs, Memory, and Fit

We used the same analytical approach for Experiment 3 as for Experiment 2, except that we ran each model first with measurement order (i.e., JOL first vs. Fit first) and its respective interactions as additional predictors. Following the preregistration, however, we removed measurement order from all models because it did not show a significant effect (p > .05) in any model. Table 5 presents the results from the multilevel models.

Table 5.
Results from Regression Models for the Effect of Valence in Experiment 3
a) JOLsb) Memoryc) Fitd) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 52.57 27.01 < .001 2.41 10.03 < .001 57.99 34.84 < .001 3.96 60.17 < .001 
Positive 2.06 1.29 .198 -0.26 -1.42 .156 2.15 1.33 0.185 0.73 9.13 < .001 
Negative 0.44 0.24 .812 -0.50 -2.27 .023 -6.61 -3.35 .001 -0.52 -4.89 < .001 
a) JOLsb) Memoryc) Fitd) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 52.57 27.01 < .001 2.41 10.03 < .001 57.99 34.84 < .001 3.96 60.17 < .001 
Positive 2.06 1.29 .198 -0.26 -1.42 .156 2.15 1.33 0.185 0.73 9.13 < .001 
Negative 0.44 0.24 .812 -0.50 -2.27 .023 -6.61 -3.35 .001 -0.52 -4.89 < .001 

JOLs did not differ significantly between valence levels (see Table 5a). Overall, memory was very accurate (83%). Memory was slightly less accurate for negative USs than neutral USs (see Table 5b), but there was no difference between positive and neutral USs. The average gamma correlation between JOLs and memory was γ = .20 and above 0, t = 3.88, p < .001. Unexpectedly, fit judgments were significantly lower for the negative US valence condition (see Table 5c). Overall, JOLs and fit judgments were moderately correlated with r = .38, p < .001. Again, this correlation did not differ substantially depending on whether within-person- or grand-mean-centered judgments were used. Fit was not significantly correlated with memory, r = .04, p = .080.

CS Evaluations

In the first model with CS evaluations as the outcome, there was a significant effect of positive and negative valence, showing an EC effect (see Figure 3 and Table 5d). As in the previous experiments, we next added the different moderators into the model (Table 6). The moderation by memory was weaker in this experiment; only the interaction with negative valence was significant (Table 6a). As in the previous experiments, JOLs moderated the EC effect (Table 6b and Figure 3b), with stronger EC effects for higher JOLs. Crucially, higher fit judgments were also associated with stronger EC effects (Table 6c and Figure 3c). However, the moderation was only significant for positive against neutral US valence.

Figure 3.
Mean CS Evaluation in Experiment 3 as a Function of (a) US Valence and Memory or (b) JOLs or (c) Fit

Note. Error bars represent 95% confidence intervals.

Figure 3.
Mean CS Evaluation in Experiment 3 as a Function of (a) US Valence and Memory or (b) JOLs or (c) Fit

Note. Error bars represent 95% confidence intervals.

Close modal
Table 6.
Results from Regression Models for the Effects of Valence, Memory, JOLs, and Fit in Experiment 3
a) CS Evaluations2) CS Evaluations3) CS Evaluations4) CS Evaluations5) CS Evaluations6) CS Evaluations
Predictors b t p b t p b t p b t p b t p b t p 
(Intercept) 3.96 59.73 < .001 3.97 60.19 < .001 3.96 60.23 < .001 3.97 59.95 < .001 3.97 60.59 < .001 3.96 60.44 < .001 
Positive 0.73 9.19 < .001 0.70 9.04 < .001 0.66 8.43 < .001 0.71 9.08 < .001 0.67 8.60 < .001 0.67 8.63 < .001 
Negative -0.53 -4.91 < .001 -0.53 -4.99 < .001 -0.53 -4.95 < .001 -0.54 -5.02 < .001 -0.54 -5.08 < .001 -0.55 -5.15 < .001 
Memory 0.04 0.59 .556       0.02 0.26 .797    0.02 0.26 .794 
Memory x Positive 0.08 0.92 .359       0.06 0.77 .443    0.10 1.22 .224 
Memory x Negative -0.22 -2.61 .009       -0.19 -2.22 .027    -0.18 -2.20 .028 
JOL    0.16 2.71 .007    0.16 2.68 .007 0.18 2.70 .007 0.18 2.69 .007 
JOL x Positive    0.32 3.84 < .001    0.32 3.77 < .001 0.17 1.88 .060 0.16 1.76 .079 
JOL x Negative    -0.28 -3.42 .001    -0.25 -3.11 .002 -0.29 -3.26 .001 -0.26 -2.97 .003 
Fit       0.06 0.95 .340    -0.03 -0.47 .641 -0.03 -0.46 .647 
Fit x Positive       0.44 5.26 < .001    0.40 4.42 < .001 0.41 4.51 < .001 
Fit x Negative       -0.11 -1.40 .162    0.01 0.09 .929 0.00 0.05 .961 
a) CS Evaluations2) CS Evaluations3) CS Evaluations4) CS Evaluations5) CS Evaluations6) CS Evaluations
Predictors b t p b t p b t p b t p b t p b t p 
(Intercept) 3.96 59.73 < .001 3.97 60.19 < .001 3.96 60.23 < .001 3.97 59.95 < .001 3.97 60.59 < .001 3.96 60.44 < .001 
Positive 0.73 9.19 < .001 0.70 9.04 < .001 0.66 8.43 < .001 0.71 9.08 < .001 0.67 8.60 < .001 0.67 8.63 < .001 
Negative -0.53 -4.91 < .001 -0.53 -4.99 < .001 -0.53 -4.95 < .001 -0.54 -5.02 < .001 -0.54 -5.08 < .001 -0.55 -5.15 < .001 
Memory 0.04 0.59 .556       0.02 0.26 .797    0.02 0.26 .794 
Memory x Positive 0.08 0.92 .359       0.06 0.77 .443    0.10 1.22 .224 
Memory x Negative -0.22 -2.61 .009       -0.19 -2.22 .027    -0.18 -2.20 .028 
JOL    0.16 2.71 .007    0.16 2.68 .007 0.18 2.70 .007 0.18 2.69 .007 
JOL x Positive    0.32 3.84 < .001    0.32 3.77 < .001 0.17 1.88 .060 0.16 1.76 .079 
JOL x Negative    -0.28 -3.42 .001    -0.25 -3.11 .002 -0.29 -3.26 .001 -0.26 -2.97 .003 
Fit       0.06 0.95 .340    -0.03 -0.47 .641 -0.03 -0.46 .647 
Fit x Positive       0.44 5.26 < .001    0.40 4.42 < .001 0.41 4.51 < .001 
Fit x Negative       -0.11 -1.40 .162    0.01 0.09 .929 0.00 0.05 .961 

Note. Memory was standardized at the grand mean, JOLs and Fit within participants. Model c, d, and f as well as the mediation analysis were not preregistered.

Next, we added both memory and JOLs, both JOLs and CS-US fit, and finally, JOLs, memory, and CS-US fit into the model (see Table 6). All these steps reduced but did not eliminate the JOL x Positive and JOL x Negative interaction terms, suggesting that the moderations were incremental to what memory and CS-US fit could account for.

We also conducted a mediation analysis similar to Experiment 2, where the Memory x Valence and Fit x Valence interactions mediated the JOL x Valence interaction as parallel mediators. The mediation analysis revealed a small but significant indirect effect through memory, b = 0.013, p = .042, 95% MCMC CI [0.002, 0.027], and a significant indirect effect through fit, b = 0.074, p < .001, 95% MCMC CI [0.040, 0.111]. The direct effect remained significant, b = 0.211, p < .001, 95% MCMC CI [0.126, 0.293].

Discussion

Experiment 3 replicated the findings from the previous experiments. Higher JOLs were associated with actual memory and stronger EC effects. Again, the association between JOLs and the EC effect was incremental to memory, although JOLs were moderately accurate and predicted actual memory. In Experiment 3, we further tested the subjective fit between CS and US as one potential factor that might account for this association. Although CS-US fit was indeed associated with higher JOLs and, as expected, CS-US fit moderated the EC effect, the association between JOLs and the EC effect was incremental to CS-US fit. The mediation analysis suggests that memory and CS-US fit might account for a small part but that additional processes contribute to the moderation.

One additional explanation might be the subjective ease of processing the stimuli. Stimuli that are easy to process receive higher JOLs (e.g., Besken, 2016; Undorf & Erdfelder, 2011), and ease of processing contributes to the effect of several cues on JOLs, such as relatedness (Undorf & Erdfelder, 2015), word frequency (e.g., Mendes et al., 2021), or stimulus size (Undorf et al., 2017). Accordingly, CS-US pairs that are easy to process should receive higher JOLs than CS-US pairs that are difficult to process.

Processing ease also plays a role in evaluative judgments. According to a hedonic perspective, processing ease is inherently positive and leads to more positive evaluations (Reber et al., 2004). According to an amplification perspective, however, processing ease leads to more extreme evaluations (Albrecht & Carbon, 2014). Previous EC research has found support for both perspectives in the domain of EC (Landwehr & Eckmann, 2020). In support of the hedonic perspective, EC procedures increase processing ease due to repeated exposure to the stimuli, which leads to slightly more positive evaluations even for negatively conditioned CSs (Landwehr et al., 2017). In support of the amplification perspective, Vogel et al. (2021) found that CS-US pairs that are easier to process show stronger EC effects – also in the negative direction (but see Ingendahl et al., 2024). Based on this latter perspective, we expected that pairings that are easy to process yield stronger conditioning effects – but also receive higher JOLs.

In summary, the association between JOLs and the EC effect could also be due to the subjective experience of processing ease, which varies across the CS-US pairs. We therefore tested the feeling of processing ease as one potential source of covariation between JOLs and EC in Experiment 4.

Experiment 4 tested whether ease of processing could explain the association between JOLs and EC effects. Experiment 4 was preregistered on aspredicted.org (https://aspredicted.org/zu87d.pdf ).

Methods

Design and Participants

We varied the normed US valence (positive vs. neutral vs. negative) within subjects. There were no additional manipulations. We used the sample size rationale as in Experiment 3. In this experiment, we recruited N = 70 native German speakers (22 female, 48 male, MAge = 35.14, SDAge = 11.87) from Prolific Academic.

Procedure

The experiment followed the same procedure as Experiment 3, except that we asked participants for subjective processing ease judgments (instead of fit judgments) by adapting the single-item approach of Graf and colleagues (2018). For that purpose, participants judged each pair on “How easy is it to process the picture and brand name?” and a slider from 0 (very difficult) to 100 (very easy). Again, we counterbalanced between participants whether they first had to provide all JOLs or all processing ease judgments.

Materials

The materials were identical to Experiment 2.

Results

JOLs, Memory, and Fit

We used the same analytical approach for Experiment 4 as for Experiment 3. Measurement order did not show a significant effect in any of the models. Following the preregistration, we thus removed all terms with it from the multilevel models. The results from the multilevel models are displayed in Table 7.

Table 7.
Results from Regression Models for the Effect of Valence in Experiment 4
a) JOLsb) Memoryc) Processing Eased) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 53.83 28.71 < .001 2.12 9.59 < .001 51.79 32.51 < .001 3.94 57.44 < .001 
Positive 6.87 3.55 .001 -0.29 -1.38 .167 12.38 5.51 < .001 1.03 8.46 < .001 
Negative -1.02 -0.56 .574 -0.25 -1.49 .135 -6.63 -3.32 .001 -0.83 -8.20 < .001 
a) JOLsb) Memoryc) Processing Eased) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 53.83 28.71 < .001 2.12 9.59 < .001 51.79 32.51 < .001 3.94 57.44 < .001 
Positive 6.87 3.55 .001 -0.29 -1.38 .167 12.38 5.51 < .001 1.03 8.46 < .001 
Negative -1.02 -0.56 .574 -0.25 -1.49 .135 -6.63 -3.32 .001 -0.83 -8.20 < .001 

JOLs were higher for positive CS-US pairings (see Table 7a). Overall, memory was very accurate (81%) and did not differ between valence levels (Table 7b). The average gamma correlation between JOLs and memory was γ = .20 and above 0, t = 3.73, p < .001. Unexpectedly, processing ease judgments were significantly higher for positive and significantly lower for the negative US valence condition (see Table 7c). JOLs and processing ease judgments were highly correlated with r = .63, p < .001. The correlation between processing ease and memory was r = .06, p = .021.

CS Evaluations

In the first model with CS evaluations as the outcome, there was a significant effect of positive and negative valence, showing an EC effect (see Figure 4 and Table 7d). As in the previous experiments, we added each moderator step-by-step into the model. The moderation by memory was only significant for the negative valence condition (Table 8a). As in the previous experiments, JOLs moderated the EC effect (Table 8b and Figure 4b). Crucially, processing ease also moderated the EC effect, with stronger EC for higher processing ease (Table 8c and Figure 4c). Next, we added both memory and JOLs, both JOLs and processing ease, and finally, JOLs, memory, and processing ease into the model (see Table 8d-f). All these steps again reduced but did not eliminate the JOL x Positive and JOL x Negative interaction terms, suggesting that the moderations were incremental to what could be accounted for by memory or processing ease.

Table 8.
Results from Regression Models for the Effects of Valence, Memory, JOLs, and Processing Ease (PE) in Experiment 4
a) CS Evaluationsb) CS Evaluationsc) CS Evaluationsd) CS Evaluationse) CS Evaluationsf) CS Evaluations
Predictors b t p b t p b t p b t p b t p b t p 
(Intercept) 3.93 57.44 < .001 3.95 55.36 < .001 3.95 54.25 < .001 3.95 54.64 < .001 3.95 55.29 < .001 3.95 54.88 < .001 
Positive 1.04 8.78 < .001 0.92 6.82 < .001 0.85 6.00 < .001 0.94 7.08 < .001 0.87 6.31 < .001 0.88 6.58 < .001 
Negative -0.82 -8.64 < .001 -0.86 -11.43 < .001 -0.90 -11.67 < .001 -0.86 -11.50 < .001 -0.88 -11.55 < .001 -0.87 -11.52 < .001 
Memory 0.14 2.39 .017       0.12 2.01 .044    0.11 1.92 .055 
Memory x Positive 0.06 0.75 .451       0.06 0.78 .435    0.07 0.84 .401 
Memory x Negative -0.40 -5.03 < .001       -0.37 -4.74 < .001    -0.35 -4.64 < .001 
JOL    0.23 3.92 < .001    0.21 3.88 < .001 0.13 1.86 .064 0.11 1.58 .115 
JOL x Positive    0.26 3.22 .001    0.25 3.11 .002 0.23 2.22 .027 0.23 2.22 .027 
JOL x Negative    -0.44 -5.71 < .001    -0.37 -4.87 < .001 -0.29 -3.10 .002 -0.23 -2.52 .012 
PE       0.25 4.48 < .001    0.17 2.36 .018 0.18 2.49 .013 
PE x Positive       0.19 2.21 .027    0.04 0.35 .729 0.04 0.39 .696 
PE x Negative       -0.43 -5.41 < .001    -0.27 -2.74 .006 -0.26 -2.73 .006 
a) CS Evaluationsb) CS Evaluationsc) CS Evaluationsd) CS Evaluationse) CS Evaluationsf) CS Evaluations
Predictors b t p b t p b t p b t p b t p b t p 
(Intercept) 3.93 57.44 < .001 3.95 55.36 < .001 3.95 54.25 < .001 3.95 54.64 < .001 3.95 55.29 < .001 3.95 54.88 < .001 
Positive 1.04 8.78 < .001 0.92 6.82 < .001 0.85 6.00 < .001 0.94 7.08 < .001 0.87 6.31 < .001 0.88 6.58 < .001 
Negative -0.82 -8.64 < .001 -0.86 -11.43 < .001 -0.90 -11.67 < .001 -0.86 -11.50 < .001 -0.88 -11.55 < .001 -0.87 -11.52 < .001 
Memory 0.14 2.39 .017       0.12 2.01 .044    0.11 1.92 .055 
Memory x Positive 0.06 0.75 .451       0.06 0.78 .435    0.07 0.84 .401 
Memory x Negative -0.40 -5.03 < .001       -0.37 -4.74 < .001    -0.35 -4.64 < .001 
JOL    0.23 3.92 < .001    0.21 3.88 < .001 0.13 1.86 .064 0.11 1.58 .115 
JOL x Positive    0.26 3.22 .001    0.25 3.11 .002 0.23 2.22 .027 0.23 2.22 .027 
JOL x Negative    -0.44 -5.71 < .001    -0.37 -4.87 < .001 -0.29 -3.10 .002 -0.23 -2.52 .012 
PE       0.25 4.48 < .001    0.17 2.36 .018 0.18 2.49 .013 
PE x Positive       0.19 2.21 .027    0.04 0.35 .729 0.04 0.39 .696 
PE x Negative       -0.43 -5.41 < .001    -0.27 -2.74 .006 -0.26 -2.73 .006 
Figure 4.
Mean CS Evaluation in Experiment 4 as a Function of (a) US Valence and Memory or (b) JOLs or (c) Processing Ease

Note. Error bars represent 95% confidence intervals.

Figure 4.
Mean CS Evaluation in Experiment 4 as a Function of (a) US Valence and Memory or (b) JOLs or (c) Processing Ease

Note. Error bars represent 95% confidence intervals.

Close modal

We also conducted a mediation analysis where the Memory x Valence and the Processing Ease x Valence interaction mediated the JOL x Valence interaction as parallel mediators. The mediation analysis revealed a small but significant indirect effect through memory, b = 0.033, p = .020, 95% MCMC CI [0.008, 0.062], and a significant indirect effect through processing ease, b = 0.084, p = .007, 95% MCMC CI [0.020, 0.147]. The direct effect remained significant, b = 0.245, p < .001, 95% MCMC CI [0.144, 0.343].

Discussion

Experiment 4 replicated the findings from the previous experiments. Higher JOLs were associated with actual memory and stronger EC effects. Again, the association between JOLs and the EC effect was incremental to memory. In Experiment 4, we further tested whether processing ease could further account for this association. Although processing ease was indeed associated with higher JOLs and, as expected, processing ease moderated the EC effect, the association between JOLs and the EC effect was incremental also to processing ease. The mediation analysis suggests that memory and processing ease might account for a small part of the association but also that further processes contribute to the moderation.

Overall, Experiments 1-4 show that the association between JOLs and the EC effect partly relates to actual memory performance but also to differences in the extremity of the USs, the perceived fit between CS and US, and the feeling of processing ease. However, each of these variables seems to account only for a part of the relationship, and the strength of the unexplained effect suggests that other processes might be involved. After successfully testing four potential explanations, however, we decided to leave the investigation of other process explanations to future research and focus on potential theoretical constraints for the role of metamemory in EC.

All previous experiments focused on identity memory. Specifically, the JOLs assessed participants’ subjective likelihood of remembering specific US pictures. Also, the memory test asked participants to identify the exact US the CS was paired with. Yet, previous EC research considers US identity memory to be less relevant to EC than remembering the mere valence of the US (Förderer & Unkelbach, 2013; Stahl et al., 2009; Stahl & Aust, 2018; but see Ingendahl, Woitzel, Propheter, et al., 2023). It is thus important to investigate whether participants can monitor valence memory and whether valence memory JOLs are related to the EC effect.

To address this question, we conducted one additional experiment that varied the type of judgment and memory test between participants. We asked for identity JOLs and memory in one condition, as in the previous experiments. In the other condition, we assessed valence JOLs and memory.

Experiment 5 tested whether the effects of JOLs and memory depended on whether people predicted and remembered item identity or item valence. We preregistered Experiment 5 on aspredicted.org (https://aspredicted.org/ft76r.pdf ).

Methods

Design and Participants

The experiment followed a Memory Type (Identity vs. Valence) x Valence (Positive vs. Neutral vs. Negative) mixed design, with the memory type manipulated between subjects and valence manipulated within subjects. We preregistered a sample size of N = 70, which suffices to find a small to medium three-way interaction effect (f = .15) with 80% power in this design (Faul et al., 2007). Note that in this design, the three-way interaction has the same significance test and thus the same statistical power as the two-way interaction. Seventy students (51 female, 18 male, 1 diverse, MAge = 23.09, SDAge = 4.95; 66 native speakers) were recruited via our universities’ study platforms in exchange for course credits.

Procedure

The experiment followed the same procedure as Experiment 2, except that we manipulated between participants whether JOLs and the memory measure captured identity or valence memory. The procedure of the identity memory condition was exactly as in Experiment 2. In the valence memory condition, we altered the JOLs and the memory measure in two ways: First, the JOLs specifically asked for the valence of the US pictures: “How likely is it that you will remember the valence of the picture (e.g., whether it is positive, neutral, or negative) when presented with the brand name?”. Second, the memory test captured valence memory exclusively by asking the question: “What was the valence of the image shown with this brand name?” together with three buttons labeled with “Positive,” “Neutral,” and “Negative.” We also presented information on each page to prevent participants from confusing this measure with a CS evaluation: “Important: It is not relevant whether you find the brand name itself positive, neutral, or negative. In your answer, please refer only to the picture that was shown with the brand name.”

Materials

The materials were identical to Experiment 2.

Results

JOLs, Memory, and US Evaluations

We used the same analytical approach as in the previous experiments, except that we had an additional predictor for condition (1 = valence, -1 = identity), including all higher-order interactions with all other predictors. Table 9 shows the results from the multilevel models.

Table 9.
Results from Regression Models for the Effects of Condition and Valence in Experiment 5
a) JOLsb) Memoryc) US Evaluationsd) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 47.34 22.04 < .001 0.48 2.80 .005 3.77 48.34 < .001 3.99 61.55 < .001 
Condition -1.44 -0.67 .505 -0.56 -3.28 .001 -0.10 -1.25 .215 -0.08 -1.27 .208 
Positive 7.15 2.97 .004 0.09 0.63 .528 2.40 22.25 < .001 0.89 7.08 < .001 
Negative 4.68 2.69 .009 0.11 0.71 .481 -1.87 -23.14 < .001 -0.61 -5.92 < .001 
Condition x Positive 2.56 1.06 .292 0.13 0.94 .345 0.17 1.58 .119 0.11 0.86 .395 
Condition x Negative 3.60 2.07 .042 0.35 2.34 .019 -0.07 -0.90 .369 -0.12 -1.17 .247 
a) JOLsb) Memoryc) US Evaluationsd) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 47.34 22.04 < .001 0.48 2.80 .005 3.77 48.34 < .001 3.99 61.55 < .001 
Condition -1.44 -0.67 .505 -0.56 -3.28 .001 -0.10 -1.25 .215 -0.08 -1.27 .208 
Positive 7.15 2.97 .004 0.09 0.63 .528 2.40 22.25 < .001 0.89 7.08 < .001 
Negative 4.68 2.69 .009 0.11 0.71 .481 -1.87 -23.14 < .001 -0.61 -5.92 < .001 
Condition x Positive 2.56 1.06 .292 0.13 0.94 .345 0.17 1.58 .119 0.11 0.86 .395 
Condition x Negative 3.60 2.07 .042 0.35 2.34 .019 -0.07 -0.90 .369 -0.12 -1.17 .247 

JOLs were higher for positive USs and negative USs than for neutral USs (see Table 9a). Furthermore, the latter effect was stronger in the valence condition. Memory was significantly worse in the valence condition, but this effect was attenuated for negative CS-US pairings (Table 9b). The average gamma correlation was slightly higher in the identity (γ = .26) than the valence condition (γ = .16), but gamma correlations were above 0 in both conditions, all p’s < .003. Following the preregistration, we also analyzed whether asking for different judgments changed US extremity judgments. There were no effects of condition here (Table 9c). We also computed US extremity scores as in Experiment 2 and found that they were significantly correlated with JOLs, r = .23, p < .001.

CS Evaluations

In the first model with CS evaluations as the outcome, there was a significant effect of positive and negative valence, showing an EC effect (see Figure 5 and Table 9d). In the next model, memory had the expected moderation effect on the EC effect, such that the EC effect was stronger when memory was correct (Figure 5a and Table 10a). Here, a significant Condition x Memory x Positive interaction emerged, such that the moderation by memory was more pronounced for valence than identity memory. As in the previous experiments, JOLs moderated the EC effect (Table 10b and Figure 5b). For higher JOLs, the EC effect was larger. The two-way interactions with JOLs were not further qualified by condition, showing that the moderation by JOLs did not differ significantly between valence and identity JOLs.

Table 10.
Results from Regression Models for the Effects of Condition, Valence, Memory, JOLs, and US Extremity in Experiment 5
a) CS Evaluationsb) CS Evaluationsc) CS Evaluationsd) CS Evaluationse) CS Evaluationsf) CS Evaluations
Predictors b t p b t p b t p b t p b t p b t p 
(Intercept) 4.00 59.63 < .001 4.01 61.29 < .001 3.91 46.92 < .001 4.03 59.73 < .001 3.90 46.83 < .001 3.93 45.50 < .001 
Condition -0.05 -0.78 .441 -0.08 -1.17 .245 -0.15 -1.77 .079 -0.05 -0.68 .501 -0.13 -1.60 .112 -0.08 -0.96 .339 
Positive 0.91 8.12 < .001 0.81 7.01 < .001 0.77 6.00 < .001 0.84 8.00 < .001 0.79 6.48 < .001 0.85 7.25 < .001 
Negative -0.62 -6.67 < .001 -0.62 -5.93 < .001 -0.42 -3.57 .001 -0.63 -6.69 < .001 -0.42 -3.54 .001 -0.46 -4.16 < .001 
Condition x Positive 0.16 1.45 .153 0.09 0.82 .415 0.10 0.77 .442 0.15 1.40 .166 0.10 0.83 .409 0.17 1.44 .152 
Condition x Negative -0.18 -1.91 .060 -0.11 -1.03 .306 -0.05 -0.41 .685 -0.17 -1.77 .080 -0.06 -0.54 .593 -0.16 -1.47 .145 
Memory 0.15 2.71 .007       0.13 2.43 .015    0.11 2.07 .039 
Condition x Memory 0.02 0.34 .733       0.05 0.90 .367    0.03 0.54 .591 
Memory x Positive 0.45 5.64 < .001       0.37 4.75 < .001    0.37 4.71 < .001 
Memory x Negative -0.59 -7.65 < .001       -0.55 -7.11 < .001    -0.53 -6.77 < .001 
Condition x Memory x Positive 0.22 2.83 .005       0.22 2.78 .006    0.22 2.77 .006 
Condition x Memory x Negative -0.14 -1.77 .077       -0.14 -1.81 .070    -0.12 -1.48 .139 
JOL    0.19 3.33 .001    0.19 3.44 .001 0.21 3.73 < .001 0.21 3.73 < .001 
Condition x JOL    -0.06 -1.09 .278    -0.04 -0.80 .421 -0.05 -0.79 .428 -0.04 -0.65 .519 
JOL x Positive    0.35 4.37 < .001    0.24 3.09 .002 0.24 2.97 .003 0.17 2.18 .029 
JOL x Negative    -0.43 -5.41 < .001    -0.37 -4.72 < .001 -0.43 -5.33 < .001 -0.36 -4.61 < .001 
Condition x JOL x Positive    0.02 0.20 .843    -0.08 -1.05 .294 -0.05 -0.61 .540 -0.11 -1.34 .180 
Condition x JOL x Negative    -0.10 -1.22 .224    -0.10 -1.28 .201 -0.12 -1.49 .137 -0.12 -1.54 .123 
US extremity       -0.09 -1.38 .167    -0.14 -2.15 .032 -0.12 -1.82 .068 
Condition x US extremity       -0.08 -1.17 .243    -0.07 -1.03 .302 -0.04 -0.57 .568 
US extremity x Positive       0.58 5.96 < .001    0.50 5.10 < .001 0.38 3.95 < .001 
US extremity x Negative       -0.21 -2.11 .035    -0.10 -1.07 .286 -0.09 -0.96 .336 
Condition x US extremity x Positive       0.20 2.04 .042    0.17 1.72 .085 0.04 0.46 .649 
Condition x US extremity x Negative       0.12 1.22 .223    0.15 1.51 .131 0.16 1.70 .089 
a) CS Evaluationsb) CS Evaluationsc) CS Evaluationsd) CS Evaluationse) CS Evaluationsf) CS Evaluations
Predictors b t p b t p b t p b t p b t p b t p 
(Intercept) 4.00 59.63 < .001 4.01 61.29 < .001 3.91 46.92 < .001 4.03 59.73 < .001 3.90 46.83 < .001 3.93 45.50 < .001 
Condition -0.05 -0.78 .441 -0.08 -1.17 .245 -0.15 -1.77 .079 -0.05 -0.68 .501 -0.13 -1.60 .112 -0.08 -0.96 .339 
Positive 0.91 8.12 < .001 0.81 7.01 < .001 0.77 6.00 < .001 0.84 8.00 < .001 0.79 6.48 < .001 0.85 7.25 < .001 
Negative -0.62 -6.67 < .001 -0.62 -5.93 < .001 -0.42 -3.57 .001 -0.63 -6.69 < .001 -0.42 -3.54 .001 -0.46 -4.16 < .001 
Condition x Positive 0.16 1.45 .153 0.09 0.82 .415 0.10 0.77 .442 0.15 1.40 .166 0.10 0.83 .409 0.17 1.44 .152 
Condition x Negative -0.18 -1.91 .060 -0.11 -1.03 .306 -0.05 -0.41 .685 -0.17 -1.77 .080 -0.06 -0.54 .593 -0.16 -1.47 .145 
Memory 0.15 2.71 .007       0.13 2.43 .015    0.11 2.07 .039 
Condition x Memory 0.02 0.34 .733       0.05 0.90 .367    0.03 0.54 .591 
Memory x Positive 0.45 5.64 < .001       0.37 4.75 < .001    0.37 4.71 < .001 
Memory x Negative -0.59 -7.65 < .001       -0.55 -7.11 < .001    -0.53 -6.77 < .001 
Condition x Memory x Positive 0.22 2.83 .005       0.22 2.78 .006    0.22 2.77 .006 
Condition x Memory x Negative -0.14 -1.77 .077       -0.14 -1.81 .070    -0.12 -1.48 .139 
JOL    0.19 3.33 .001    0.19 3.44 .001 0.21 3.73 < .001 0.21 3.73 < .001 
Condition x JOL    -0.06 -1.09 .278    -0.04 -0.80 .421 -0.05 -0.79 .428 -0.04 -0.65 .519 
JOL x Positive    0.35 4.37 < .001    0.24 3.09 .002 0.24 2.97 .003 0.17 2.18 .029 
JOL x Negative    -0.43 -5.41 < .001    -0.37 -4.72 < .001 -0.43 -5.33 < .001 -0.36 -4.61 < .001 
Condition x JOL x Positive    0.02 0.20 .843    -0.08 -1.05 .294 -0.05 -0.61 .540 -0.11 -1.34 .180 
Condition x JOL x Negative    -0.10 -1.22 .224    -0.10 -1.28 .201 -0.12 -1.49 .137 -0.12 -1.54 .123 
US extremity       -0.09 -1.38 .167    -0.14 -2.15 .032 -0.12 -1.82 .068 
Condition x US extremity       -0.08 -1.17 .243    -0.07 -1.03 .302 -0.04 -0.57 .568 
US extremity x Positive       0.58 5.96 < .001    0.50 5.10 < .001 0.38 3.95 < .001 
US extremity x Negative       -0.21 -2.11 .035    -0.10 -1.07 .286 -0.09 -0.96 .336 
Condition x US extremity x Positive       0.20 2.04 .042    0.17 1.72 .085 0.04 0.46 .649 
Condition x US extremity x Negative       0.12 1.22 .223    0.15 1.51 .131 0.16 1.70 .089 

Note. Models c, e, and f as well as the mediation analysis reported in the text were not preregistered in this experiment.

Figure 5.
Mean CS Evaluation in Experiment 5 as a Function of (a) US Valence, Condition, and Memory or (b) JOLs or (c) US Extremity

Note. Error bars represent 95% confidence intervals.

Figure 5.
Mean CS Evaluation in Experiment 5 as a Function of (a) US Valence, Condition, and Memory or (b) JOLs or (c) US Extremity

Note. Error bars represent 95% confidence intervals.

Close modal

We also replicated the moderation by US extremity from Experiment 2 (Figure 5c and Table 10c): More extreme US evaluations were associated with stronger EC effects. For positive USs, this effect was slightly more pronounced in the valence than in the identity condition. As in the previous studies, we added both memory and JOLs, both JOLs and US extremity, and, finally, JOLs, memory, and US extremity into the model (see Table 10e). All these steps reduced but did not eliminate the JOL x Positive and JOL x Negative interaction terms, suggesting that the moderations were incremental to what memory or US extremity could account for.

We also conducted a mediation analysis as in Experiment 2. The mediation analysis revealed a small but significant indirect effect through memory, b = 0.091, p < .001, 95% MCMC CI [0.049, 0.135], and a significant indirect effect through US extremity, b = 0.059, p < .001, 95% MCMC CI [0.028, 0.096]). The direct effect remained significant, b = 0.259, p < .001, 95% MCMC CI [0.175, 0.337]. Again, detailed results are provided in the Supplement.9

Discussion

Experiment 5 replicated the findings from the previous studies. Higher JOLs were associated with actual memory and stronger EC effects. Again, the association between JOLs and the EC effect was incremental to memory. Also, although US extremity was indeed associated with higher JOLs and, as expected, US extremity moderated the EC effect, the association between JOLs and the EC effect was incremental to US extremity. The mediation analysis suggested that memory and US extremity might account for a part of the moderation, but that further processes might contribute to the association.

In this experiment, we tested whether the effects depend on whether the JOLs and the memory test target US identity or US valence. Overall, the results were similar across conditions. The major differences were that memory was less accurate in the valence memory test and that valence memory was slightly more predictive of EC effects than identity memory. The first difference might emerge because valence memory tests are more difficult than identity memory tests, despite the higher guessing probability. Specifically, a valence memory test requires the abstraction of a specific stimulus feature (i.e., the valence) and does not allow correct identifications based on mere stimulus recognition. The second difference suggests that valence memory is more relevant to EC than identity memory (Stahl et al., 2009), which is why remembering the valence of the US is more strongly related with the EC effect than remembering the specific US’s identity. Nevertheless, Experiment 5 shows that the association between JOLs and the EC effect does not substantially differ between identity and valence metamemory and memory. Overall, the results suggest that the core findings also generalize to valence JOLs and valence memory.

Another important theoretical constraint for studying the role of metamemory in EC is to what extent asking for JOLs per se changes EC effects. Metamemory research has shown that assessing JOLs may also change people’s learning and memory (for a review, see Double & Birney, 2019). Such reactive effects of making JOLs could come in various ways. For example, asking for JOLs could improve people’s memory of the USs due to higher elaboration. Moreover, JOLs could make people focus more on the relation between the stimuli and thereby strengthen the EC effect. Another possibility is that making JOLs changes the perception of USs. Strong reactivity effects in the current paradigm would indicate that our findings are specific to EC experiments where JOLs are assessed. In Experiment 6, we therefore tested for potential reactive effects of asking for JOLs in an EC procedure.

Experiment 6 tested whether asking for JOLs influences the EC effect, memory, or US valence. We preregistered Experiment 6 on aspredicted.org (https://aspredicted.org/xn2gd.pdf ).

Methods

Design and Participants

The experiment followed a Condition (JOL vs. Control) x Valence (Positive vs. Neutral vs. Negative) mixed design, with the first factor manipulated between and the second within subjects. We preregistered a sample size of N = 70, which suffices to find a small to medium interaction effect (f = .15) in this design (Faul et al., 2007). Because we could not precisely monitor the number of participants in our universities’ study platforms, our final sample size was N = 72 students (58 female, 13 male, MAge = 22.89, SDAge = 4.23; 69 native speakers) who participated in exchange for course credits.

Procedure

The experiment followed the same procedure as Experiment 2, except that we manipulated between participants whether they had to give JOLs or not. The procedure of the JOL condition was identical to Experiment 2. In the control condition, participants did not provide JOLs. In this condition, we had one additional pairing per CS in the conditioning procedure to compensate for the one pairing less on the JOL slide.

Materials

The materials were identical to Experiment 2.

Results

JOLs, Memory, and US Evaluations

We used the same analytical approach as in the previous experiments, except that we had an additional predictor for condition (1 = JOL, -1 = Control), including all interactions with all other predictors. Table 11 shows the results from the multilevel models. JOLs were slightly higher for positive USs than for neutral USs (see Table 11a). Memory was significantly more accurate in the JOL condition (Table 11b). In the JOL condition, the average gamma correlation between JOLs and memory was γ = .29 and above 0, t = 5.62, p < .001. There were no effects of condition on US evaluations (Table 11c).

Table 11.
Results from Regression Models for the Effects of Valence and Condition in Experiment 6
a) JOLsb) Memoryc) US Evaluationsd) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 40.59 15.30 < .001 0.82 5.28 < .001 3.87 70.94 < .001 4.02 66.29 < .001 
Positive 5.43 2.15 .038 -0.01 -0.10 .921 2.38 36.91 < .001 0.57 6.83 < .001 
Negative 5.54 2.02 .051 -0.02 -0.16 .872 -2.06 -23.64 < .001 -0.53 -6.31 < .001 
Condition    0.58 3.76 < .001 0.03 0.52 .605 0.01 0.16 .876 
Condition x Positive    -0.05 -0.36 .720 0.03 0.40 .689 -0.05 -0.55 .585 
Condition x Negative    -0.20 -1.47 .140 -0.08 -0.96 .341 0.06 0.66 .507 
a) JOLsb) Memoryc) US Evaluationsd) CS Evaluations
Predictors b t p b z p b t p b t p 
(Intercept) 40.59 15.30 < .001 0.82 5.28 < .001 3.87 70.94 < .001 4.02 66.29 < .001 
Positive 5.43 2.15 .038 -0.01 -0.10 .921 2.38 36.91 < .001 0.57 6.83 < .001 
Negative 5.54 2.02 .051 -0.02 -0.16 .872 -2.06 -23.64 < .001 -0.53 -6.31 < .001 
Condition    0.58 3.76 < .001 0.03 0.52 .605 0.01 0.16 .876 
Condition x Positive    -0.05 -0.36 .720 0.03 0.40 .689 -0.05 -0.55 .585 
Condition x Negative    -0.20 -1.47 .140 -0.08 -0.96 .341 0.06 0.66 .507 

CS Evaluations

In the first model with CS evaluations as the outcome, there was a significant EC effect (see Figure 6 and Table 11d). Crucially, there were no main effects or interactions of condition, indicating that the EC effect did not change in size when participants gave JOLs. Subsequently, we analyzed whether asking for JOLs changed the moderating impact of memory or US extremity, by adding these moderators into the model and allowing three-way interactions with condition (Table 12). As in the previous experiments, memory had the expected moderation effect on the EC effect, such that the EC effect was stronger when memory was correct (Figure 6a and Table 12a). We also found the moderation by US extremity, with stronger EC effects for more extreme USs (Table 12b). In both models, we found no three-way interaction with condition, suggesting that the moderations by memory and US extremity were not significantly affected by soliciting JOLs.

Figure 6.
Mean CS Evaluation in Experiment 6 as a Function of (a) US Valence, Condition, and Memory or (b) JOLs or (c) US Extremity

Note. Error bars represent 95% confidence intervals.

Figure 6.
Mean CS Evaluation in Experiment 6 as a Function of (a) US Valence, Condition, and Memory or (b) JOLs or (c) US Extremity

Note. Error bars represent 95% confidence intervals.

Close modal
Table 12.
Results from Regression Models for the Effects of Condition, Memory, Valence, and US Extremity in Experiment 6
a) CS Evaluationsb) CS Evaluations
Predictors b t p b t p 
(Intercept) 4.02 64.97 < .001 4.06 48.09 < .001 
Condition 0.02 0.24 .807 0.10 1.18 .237 
Memory -0.03 -0.43 .664    
Positive 0.54 6.45 < .001 0.42 3.94 < .001 
Negative -0.54 -6.49 < .001 -0.42 -4.04 < .001 
Condition x Positive -0.13 -1.51 .130 -0.15 -1.40 .161 
Condition x Negative 0.09 1.02 .306 0.01 0.06 .949 
Condition x Memory -0.03 -0.51 .610    
Memory x Positive 0.40 4.69 < .001    
Memory x Negative -0.21 -2.49 .013    
Condition x Memory x Positive 0.15 1.80 .072    
Condition x Memory x Negative 0.08 0.95 .341    
US extremity    0.07 0.94 .348 
Condition x US extremity    0.12 1.70 .089 
US extremity x Positive    0.20 1.99 .047 
US extremity x Negative    -0.44 -4.36 < .001 
Condition x US extremity x Positive    -0.11 -1.11 .268 
Condition x US extremity x Negative    -0.18 -1.82 .070 
a) CS Evaluationsb) CS Evaluations
Predictors b t p b t p 
(Intercept) 4.02 64.97 < .001 4.06 48.09 < .001 
Condition 0.02 0.24 .807 0.10 1.18 .237 
Memory -0.03 -0.43 .664    
Positive 0.54 6.45 < .001 0.42 3.94 < .001 
Negative -0.54 -6.49 < .001 -0.42 -4.04 < .001 
Condition x Positive -0.13 -1.51 .130 -0.15 -1.40 .161 
Condition x Negative 0.09 1.02 .306 0.01 0.06 .949 
Condition x Memory -0.03 -0.51 .610    
Memory x Positive 0.40 4.69 < .001    
Memory x Negative -0.21 -2.49 .013    
Condition x Memory x Positive 0.15 1.80 .072    
Condition x Memory x Negative 0.08 0.95 .341    
US extremity    0.07 0.94 .348 
Condition x US extremity    0.12 1.70 .089 
US extremity x Positive    0.20 1.99 .047 
US extremity x Negative    -0.44 -4.36 < .001 
Condition x US extremity x Positive    -0.11 -1.11 .268 
Condition x US extremity x Negative    -0.18 -1.82 .070 

Finally, we repeated our analyses from Experiment 2 using only the data from the JOL condition. We did not preregister these analyses because we were unsure whether they could replicate at all due to the smaller sample size. In line with Experiment 2, JOLs moderated the EC effect. Adding both memory and JOLs, both JOLs and US extremity, and finally, JOLs, memory, and US extremity into the model (see Table 13), reduced but did not eliminate the JOL x Positive and JOL x Negative interaction terms. The mediation analysis revealed a significant indirect effect through memory, b = 0.051, p < .001, 95% MCMC CI [0.026, 0.082], and a significant indirect effect through US extremity, b = 0.043, p = .005, 95% MCMC CI [0.017, 0.077]). The direct effect remained significant, b = 0.303, p < .001, 95% MCMC CI [0.187, 0.412]. Again, we provide detailed results in the Supplement.

Table 13.
Results from Regression Models for the Effects of Valence, JOLs, Memory, and US Extremity Within the JOL Condition in Experiment 6
a) CS Evaluationsb) CS Evaluationsc) CS Evaluationsd) CS Evaluations
Predictors b t p b t p b t p b t p 
(Intercept) 4.03 46.82 < .001 4.05 45.98 < .001 4.16 39.87 < .001 4.18 36.68 < .001 
Positive 0.48 4.25 < .001 0.39 3.40 .001 0.30 2.23 .026 0.20 1.38 .171 
Negative -0.47 -4.14 < .001 -0.46 -4.08 < .001 -0.43 -3.22 .001 -0.44 -3.14 .002 
JOL 0.06 0.73 .464 0.07 0.84 .400 0.03 0.38 .704 0.03 0.34 .736 
JOL x Positive 0.50 4.30 < .001 0.43 3.66 < .001 0.51 4.29 < .001 0.44 3.73 < .001 
JOL x Negative -0.29 -2.50 .013 -0.27 -2.28 .023 -0.21 -1.80 .072 -0.18 -1.48 .138 
Memory    -0.07 -0.79 .431    -0.08 -0.86 .388 
Memory x Positive    0.47 3.72 < .001    0.48 3.83 < .001 
Memory x Negative    -0.07 -0.57 .568    -0.06 -0.44 .657 
US extremity       0.19 2.09 .037 0.20 2.14 .033 
US extremity x Positive       -0.07 -0.53 .593 -0.06 -0.40 .688 
US extremity x Negative       -0.57 -4.26 < .001 -0.56 -4.15 < .001 
a) CS Evaluationsb) CS Evaluationsc) CS Evaluationsd) CS Evaluations
Predictors b t p b t p b t p b t p 
(Intercept) 4.03 46.82 < .001 4.05 45.98 < .001 4.16 39.87 < .001 4.18 36.68 < .001 
Positive 0.48 4.25 < .001 0.39 3.40 .001 0.30 2.23 .026 0.20 1.38 .171 
Negative -0.47 -4.14 < .001 -0.46 -4.08 < .001 -0.43 -3.22 .001 -0.44 -3.14 .002 
JOL 0.06 0.73 .464 0.07 0.84 .400 0.03 0.38 .704 0.03 0.34 .736 
JOL x Positive 0.50 4.30 < .001 0.43 3.66 < .001 0.51 4.29 < .001 0.44 3.73 < .001 
JOL x Negative -0.29 -2.50 .013 -0.27 -2.28 .023 -0.21 -1.80 .072 -0.18 -1.48 .138 
Memory    -0.07 -0.79 .431    -0.08 -0.86 .388 
Memory x Positive    0.47 3.72 < .001    0.48 3.83 < .001 
Memory x Negative    -0.07 -0.57 .568    -0.06 -0.44 .657 
US extremity       0.19 2.09 .037 0.20 2.14 .033 
US extremity x Positive       -0.07 -0.53 .593 -0.06 -0.40 .688 
US extremity x Negative       -0.57 -4.26 < .001 -0.56 -4.15 < .001 

Note. These models as well as the mediation analysis reported in the text were not preregistered in this experiment.

Discussion

Experiment 6 replicated the findings from the previous experiments. Higher JOLs were associated with actual memory and stronger EC effects. Again, the association between JOLs and the EC effect was incremental to memory. The association between JOLs and the EC effect was also incremental to US extremity, although US extremity was associated with higher JOLs, and, as expected, US extremity moderated the EC effect. The mediation analysis suggests that memory and US extremity might account for a part of the association but also that further processes might contribute to it.

More importantly, Experiment 6 showed that although asking for JOLs led to better memory performance, it changed neither the size of the EC effect nor US valence. Thus, there is no reason to suspect that the findings obtained in this study are specific to EC experiments in which JOLs are assessed. Note, however, that although we kept the number of stimulus pairings constant across the JOL and the control condition, there are other structural differences between the two conditions whose effects could be explored in future research (e.g., making a judgment on the stimuli or not).

Two final concerns need to be addressed. First, our previous experiments provide correlative evidence that people rely on cues when making JOLs and that these cues also moderate the EC effect. Providing experimental evidence would therefore strengthen our findings. Second, our previous experiments all used similar CS and US materials, which bears the risk of material-specific effects. To tackle both concerns, we conducted one final experiment that provided experimental evidence for the role of CS-US fit using other CS and US materials.

Experiment 7 provided experimental evidence for the role of one of the three cues we identified, namely CS-US fit. We relied on the procedure of Experiment 3 where CS-US fit had partially mediated the relationship between JOLs and the EC effect. In Experiment 7, we added a within-subjects manipulation of CS-US fit (high vs. low). We expected the following findings: As in the previous experiments, JOLs should moderate the EC effect. In this design, however, JOLs should be higher in the high-fit than in the low-fit condition. Also, the CS-US fit manipulation should moderate the EC effect, with stronger EC effects when the fit is high. Finally, this moderation should be mediated by the increase in the JOLs resulting from high CS-US fit. We preregistered Experiment 7 on aspredicted.org (https://aspredicted.org/jj9pk.pdf ).

Methods

Design and Participants

We varied the normed US valence (positive vs. neutral vs. negative) and the CS-US fit (high vs. low) within subjects. Because we used new stimuli and only one CS per factorial condition, we were uncertain about the effect sizes in this experiment and generously collected a sample size of N = 200 German native speakers (76 female, 125 male, MAge = 35.34, SDAge = 12.34) on Prolific Academic. This sample size was sufficient to detect small within-subjects effects of dz = 0.2 with 80% power.

Procedure

The procedure was the same as in Experiment 3, except for the following differences: First, as a manipulation of CS-US fit, we relied on Moran et al. (2022). They had shown that for individuals as CSs, EC effects are stronger if USs are trait adjectives compared to nouns. Thus, we used cartoon sketches depicting individual aliens as CSs, and positive/neutral/negative trait adjectives (high fit) or nouns (low fit) as USs. Like Moran et al. (2022), we framed the EC procedure as a personnel selection scenario. Participants should imagine they were heading an intergalactic space station and that they would need to hire some new staff. In the following, they would receive some information about the different applicants. Afterward, the same EC procedure followed as in Experiment 3, but with only one CS per factorial condition (i.e., 6 CSs in total). The subsequent assessment of JOLs and fit judgments, CS evaluations, and the memory task remained the same, except for the different CS and US materials.

Materials

The CSs were cartoon sketches depicting individual aliens, taken from previous EC research (Alves et al., 2018; Ingendahl et al., in press; Woitzel & Alves, 2024). As USs, we selected 24 words from the BAWL-R word database (Võ et al., 2009), four per factorial cell. Positive words (e.g., “birthday”, “loyal”) had valence ratings from 1.9 to 2.1, neutral words (“stairs”, “talkative”) from -0.1 to 0.1, and negative words (“army”, “brutal”) from -2.1 to -1.9, on a scale from -3 to +3. Valence ratings differed as a function of the normed valence level, F(2, 18) = 1842.61, p < .001, but not as a function of CS-US fit, F(1, 18) = 0.05, p = .820, or a CS-US fit x Normed Valence interaction, F(2, 18) = 0.02, p = .985.

Results

JOLs, Memory, and Fit Judgments

In this experiment, JOLs were higher both for positive and negative USs than neutral USs (see Table 14a). Crucially, however, JOLs were also significantly higher in the high-fit than the low-fit condition. Overall, memory was very accurate (91%), and was not affected by any experimental manipulation (see Table 14b). The average gamma correlation between JOLs and memory was γ = .34 and above 0, t = 3.88, p < .001. As expected, fit judgments were higher in the high-fit condition (see Table 14c). Unexpectedly, fit judgments were also significantly higher for the negative US valence condition, but only for the high-fit condition.

Table 14.
Results from Regression Models in Experiment 7
1) JOLs2) Memory3) Fit Judgment4) CS Evaluations
Predictors b t p b t p b t p b t p 
(Intercept) 67.64 41.72 <.001 5.45 6.14 <.001 53.86 33.39 <.001 4.51 61.73 <.001 
Positive 3.60 2.38 .018 -0.31 -0.96 .336 3.30 1.75 .081 0.47 5.01 <.001 
Negative 7.54 4.96 <.001 -0.25 -0.77 .443 4.10 2.17 .030 -0.98 -8.64 <.001 
Fit 3.23 2.98 .003 -0.11 -0.47 .641 7.43 5.37 <.001 0.08 1.22 .224 
Fit x Positive 1.33 0.88 .378 0.11 0.34 .734 -0.07 -0.04 .969 0.14 1.51 .132 
Fit x Negative 1.19 0.79 .431 0.35 1.10 .273 -8.00 -4.23 <.001 -0.38 -4.12 <.001 
1) JOLs2) Memory3) Fit Judgment4) CS Evaluations
Predictors b t p b t p b t p b t p 
(Intercept) 67.64 41.72 <.001 5.45 6.14 <.001 53.86 33.39 <.001 4.51 61.73 <.001 
Positive 3.60 2.38 .018 -0.31 -0.96 .336 3.30 1.75 .081 0.47 5.01 <.001 
Negative 7.54 4.96 <.001 -0.25 -0.77 .443 4.10 2.17 .030 -0.98 -8.64 <.001 
Fit 3.23 2.98 .003 -0.11 -0.47 .641 7.43 5.37 <.001 0.08 1.22 .224 
Fit x Positive 1.33 0.88 .378 0.11 0.34 .734 -0.07 -0.04 .969 0.14 1.51 .132 
Fit x Negative 1.19 0.79 .431 0.35 1.10 .273 -8.00 -4.23 <.001 -0.38 -4.12 <.001 

CS Evaluations

As in the previous experiments, there was an EC effect, such that CSs paired with positive USs were evaluated more positively and CSs paired with negative USs more negatively than CSs paired with neutral USs (see Table 14a and Figure 7a). However, there was also a significant interaction with CS-US fit for the negative-US condition. As expected, the EC effect was stronger in the high-fit condition. For the positive-US condition, the interaction was in the expected direction but not statistically significant, given that the neutral baseline condition was also slightly more positive (see Figure 7).

Figure 7.
Mean CS Evaluation in Experiment 7 as a Function of US Valence and (a) CS-US Fit or (b) JOLs

Note. Error bars represent 95% confidence intervals.

Figure 7.
Mean CS Evaluation in Experiment 7 as a Function of US Valence and (a) CS-US Fit or (b) JOLs

Note. Error bars represent 95% confidence intervals.

Close modal

We next tested whether JOLs moderated the EC effect, relying on the same analytical approach as in the previous experiments. As expected, JOLs moderated the EC effect (see Figure 7b and Table 15). However, as for the moderation by CS-US fit, the JOL x positive US interaction was not statistically significant.

Table 15.
Results from Regression Models for the Effects of Valence and JOLs in Experiment 7
CS Evaluations
Predictors b t p 
(Intercept) 4.59 60.74 <.001 
Positive 0.39 4.06 <.001 
Negative -1.00 -8.47 <.001 
JOL 0.42 5.31 <.001 
JOL x Positive 0.03 0.27 .785 
JOL x Negative -0.66 -5.83 <.001 
CS Evaluations
Predictors b t p 
(Intercept) 4.59 60.74 <.001 
Positive 0.39 4.06 <.001 
Negative -1.00 -8.47 <.001 
JOL 0.42 5.31 <.001 
JOL x Positive 0.03 0.27 .785 
JOL x Negative -0.66 -5.83 <.001 

Note. JOLs were standardized within participants.

Finally, we tested whether the stronger EC effect at higher CS-US fit was mediated by the corresponding increase in JOLs at higher CS-US fit. Thus, we tested whether the JOL x Valence interaction mediated the Fit x Valence interaction. The mediation analysis revealed the expected indirect effect, b = 0.061, p < .001, 95% MCMC CI [0.035, 0.091], showing that the moderation by the experimental CS-US fit manipulation was statistically accounted for by the corresponding increase in JOLs. The direct effect was also significant, b = 0.207, p < .001, 95% MCMC CI [0.168, 0.375], suggesting that CS-US fit had a moderating influence beyond JOLs. Again, detailed results are provided in the supplement.

As we focussed on the experimental manipulation of CS-US fit in this study, we did not preregister running the correlative analyses with fit judgments and memory as moderators. Nevertheless, these analyses are provided in the HTML Markdown on the OSF. These analyses replicated our findings from Experiment 3, including the partial mediation by memory and fit judgments.

Discussion

Experiment 7 provided experimental evidence supporting the correlative findings of the previous experiments. An experimental manipulation of CS-US fit increased JOLs and moderated the EC effect. Again, JOLs also moderated the EC effect. A mediation analysis showed that the moderation by CS-US fit was statistically explained by the corresponding moderation by JOLs. Furthermore, Experiment 7 demonstrated that the moderation by JOLs generalizes also to different types of CS and US materials.

Evaluative Conditioning (EC) is the change in the liking of a conditioned stimulus due to its pairing with a positive or negative unconditioned stimulus and a central effect in attitude research. Current theories explain EC as a memory-based effect, but so far, it remains unknown what conditioned individuals themselves know about their own memory processes in EC. To address this, we conducted seven experiments where participants were exposed to an EC procedure and provided judgments on their expected future memory performance (judgments of learning; JOLs).

All experiments showed that people can indeed predict above chance which stimulus pairing they will remember in the future. Furthermore, all experiments showed that higher JOLs were associated with stronger EC effects. Surprisingly, this association was only partially related to actual memory performance, although JOLs were moderately predictive of memory. Across Experiments 2-4, we tested several explanations for the association between JOLs and EC: variation in the subjective US extremity, the perceived fit between CS and US, and the feeling of processing ease. As expected, each of these variables was positively associated with both JOLs and the EC effect and statistically explained a proportion of their association. In the last three experiments, we tested the robustness and generalizability of our findings. The general pattern of results was independent of whether we assessed JOLs and memory for US identity or US valence. Compared to a condition without JOLs, making JOLs improved overall memory performance but did not change the size of the EC effect. In a final experiment, we also obtained experimental evidence for the effect of CS-US fit on JOLs and the EC effect.

These findings have valuable implications for EC and metamemory research, which we discuss first. Afterward, we turn to open questions and directions for future research.

Consistent with previous evidence from metamemory research, our research shows that people can also predict above chance whether they will remember the pairing of an attitude object with a valenced stimulus. In all experiments, we observed small-to-medium gamma correlations between JOLs and memory. This finding implies that people can in principle monitor their memory processes when forming attitudes via evaluative conditioning. One logical next step is to study whether this ability is strategically used to exert metacognitive control on memory processes. For example, some people might mentally rehearse those stimulus pairings they deem more relevant to current goals. Another logical step is to find out which situational and dispositional conditions strengthen people’s metamemory accuracy in EC. Whereas our research focussed on providing solid evidence that people’s metamemory in EC is indeed somewhat accurate and how metamemory relates to the size of the EC effect, it also opens the pathway for various new research avenues exploring consequences and boundary conditions of metamemory accuracy in EC.

A second insight from our experiments is that EC effects themselves are substantially related to conditioned individuals’ metamemory: If people are certain that they will remember a specific CS-US pairing, they also show a stronger EC effect on the respective CS. Although this does not show that people can intentionally predict their own attitude change – simply, because we did not assess such predictions – it shows that people incidentally predict their attitude change when giving judgments on a critical antecedent of EC, that is, memory. This finding fits nicely to recent developments in attitude research. For example, people are able to predict their scores on Implicit Association Tests, challenging the claim that implicit attitudes are inaccessible to introspection (Hahn et al., 2014; Hahn & Gawronski, 2019; Morris & Kurdi, 2023). Likewise, EC was seen in the past as an automatic way of attitude formation (Olson & Fazio, 2006) that works best without any awareness (March et al., 2019). Our findings align well with recent developments (Corneille & Stahl, 2018; Gast, 2018; Hütter, 2022; Moran et al., 2023), showing that EC effects operate on processes that should in theory be accessible to introspection. Our findings add to these recent developments by demonstrating that attitude change from stimulus pairings is to some extent accessible to introspection, by predicting how likely a stimulus pairing will be remembered.

Notably, actual memory performance only accounts for a small part of the relationship between JOLs and the EC effect. To be precise, our results do not imply that EC does not depend on memory. In fact, all studies show a robust moderation of the EC effect by memory of the stimulus pairing, once again supporting current memory-based EC accounts (Gast, 2018; Stahl & Aust, 2018). At the same time, however, our findings show that the variance in EC effects that JOLs explain is only partially explained by actual memory performance, although JOLs are moderately accurate. This finding suggests that people do not have direct access to their memory processes in EC. Instead, they base their metamemory judgments on cues that correlate substantially with other outcomes than memory. In our experiments, US extremity, perceived CS-US fit, and processing ease were only weakly or not at all related to actual memory performance but had robust correlations with JOLs and the EC effect. Therefore, we interpret our findings such that people monitor their own memory processes by relying on internal or external cues that sometimes correlate with actual memory. However, these cues can be better predictors of conditioned attitudes than memory, leading to a correlation between metamemory judgments and EC that is largely independent of actual memory.

Beyond the primary research question of the role of metamemory in EC, our experiments identify several moderators for the EC effect – US extremity, the perceived CS-US fit, and processing ease. However, which specific process account for EC can explain why these different variables relate to the strength of EC effects? We believe that the propositional account on EC (De Houwer, 2018) can give an integrative answer. The propositional account would predict that people infer similarity between the CS and the US and thus usually evaluate the CS in line with the US valence. The inferred similarity between CS and US should also lead to more extreme CS evaluations the more extreme the US is, explaining the contribution of US extremity. The propositional account would also predict that people are more likely to infer similarity between CS and US the better CS and US fit together, leading to stronger EC effects for better CS-US fit. Regarding the role of processing ease, one could speculate that processing ease serves as a cue for inferring a relation (Briñol & Petty, 2022; Mueller & Dunlosky, 2017), thereby strengthening the EC effect. Thus, overall, our findings are in line with the propositional account such that EC effects at least partially depend on the inferences people draw from a stimulus pairing.

Our research also has important implications for metamemory research. First, the current research shows that JOLs predict cognitive phenomena beyond memory, such as attitude formation. To our knowledge, our studies are the first to investigate the relationship between JOLs and further downstream consequences of memory. Our research suggests that the study of metamemory judgments might be fruitful for gathering insights into various judgments and behaviors that result from learning, such as how people assess risks based on learning experiences or follow plans based on remembered goals. Furthermore, our studies show that JOLs might predict these downstream outcomes incremental to what memory can explain. Overall, this insight from our research calls for future investigations on the relationship between JOLs and other memory-related phenomena.

Our studies also give an answer to why JOLs relate to cognitive phenomena other than actual memory. According to the cue-utilization approach, people infer JOLs from numerous cues available during the study phase (Koriat, 1997). If these cues correlate with actual memory, then JOLs will be moderately accurate in predicting future memory performance. However, if the cues also correlate with another cognitive phenomenon – in our case, the strength of the EC effect – then JOLs will be associated with the phenomenon and potentially beyond how the phenomenon is associated with memory.

In that regard, our experiments showed positive relationships between JOLs and subjective US extremity, subjective CS-US fit, and subjective ease of processing, implying that people use them as cues when making JOLs. The findings on US extremity are overall consistent with the before-mentioned emotionality effect on JOLs (Witherby & Tauber, 2018; Yin et al., 2023; Zimmerman & Kelley, 2010). The findings on CS-US fit correspond well with previous studies on JOLs and the relatedness of stimuli. For example, people give higher JOLs for semantically related word pairs than for unrelated word pairs (e.g., Mueller et al., 2013) and for semantically coherent word triads than for semantically unrelated triads (Undorf & Zander, 2017). Finally, the current finding that people base JOLs on the subjectively perceived ease of processing resonates with a large literature showing that people’s JOLs correlate with objective measures of ease of processing (Undorf, 2020) such as lexical-decision times (e.g., Jia et al., 2016; Mueller et al., 2013) or self-paced study time (e.g., Koriat et al., 2006; Undorf & Erdfelder, 2011).

Multiple or Single Cues. With subjective US extremity, subjective CS-US fit, and subjective ease of processing, our experiments identified multiple cues that people rely on when making JOLs in an EC paradigm. However, because each experiment was devoted to one cue only, we cannot know whether people base their JOLs on all three cues when manipulated simultaneously (Undorf et al., 2018). Moreover, JOLs might be based only on a unified feeling of fluency which is fed by US extremity, subjective CS-US fit, and subjective ease of processing (Undorf & Bröder, 2020). Especially for fit and fluency, there is a considerable amount of research showing that these two cues are related (e.g., Winkielman et al., 2012). At the same time, previous JOL research found that people base their JOLs on multiple cues (Undorf et al., 2018; Undorf & Bröder, 2020), suggesting that this could happen as well in an EC context. Nevertheless, cue integration in JOLs made in an EC paradigm should be investigated systematically in future studies.

Dichotomous Memory and Continuous JOLs. One structural difference between memory and JOLs in our studies is that memory was measured dichotomously (i.e., the response is either correct or incorrect), whereas JOLs were continuous (i.e., the responses can range from 0 to 100%). Given that dichotomous measures are less reliable than continuous measures, the incremental influence of JOLs beyond memory might be due to a lack of reliability on the memory measure (Fiedler et al., 2011). We consider this as a minor issue in the present study because memory performance was very high throughout (i.e., 70-90%), whereas a lack of reliability should manifest in regressive memory estimates (i.e., performance nearer to the chance probability of 1/6). One implication of high memory performance is, however, that the influence of memory might be underestimated compared to settings with more variation in memory performance. Thus, future studies might consider manipulating memory performance (e.g., by varying the number of repetitions), and investigating to what extent JOLs predict memory and EC effects where there are genuine differences in memory accuracy.

Situation, Person, and Situation x Person Effects. With memory, US extremity, perceived CS-US fit, and processing ease, we focussed on variables that can vary in principle between stimuli and persons, making it necessary to locate precisely the source of covariation between JOLs and EC. Specific stimulus combinations might receive higher JOLs and elicit stronger EC effects (effect of the situation); specific persons might provide higher JOLs and also stronger EC effects (effect of person), or specific stimuli for specific persons elicit higher JOLs and stronger EC effects (person x situation interaction). All three sources of covariation are plausible, but several arguments indicate that our experiments most likely captured a situation main effect and a person x situation interaction. First, Experiment 7 provides evidence for systematic effects of the stimuli that can be manipulated. Nevertheless, JOLs were associated with stronger EC effects beyond the effects of the experimental manipulation, suggesting that the effects are not purely stimulus based. Also, we reran the analyses in Experiments 1-4 with JOLs as a moderator but controlled statistically for the specific CSs, USs, or CS-US combinations with random intercepts. These analyses led to nearly identical results, suggesting that variability across stimuli alone is not responsible for the JOL x EC moderations. Second, we suppressed variability between persons by standardizing the JOLs within participants in all analyses. Therefore, the observed effects must be driven by person x situation interactions. For example, specific individuals might find specific stimulus pairs easier to memorize (idiosyncratic person-item interactions in metamemory and memory research; for recent evidence, see Undorf et al., 2022) and show stronger EC effects for these pairs. Nevertheless, variation due to persons alone could be considered in future research.

Conditioning as a Cue for Judgments of Learning. The association between JOLs and EC is correlative; therefore, reverse causality is also possible. Specifically, the correlation between JOLs and the EC effect might stem from people using their conditioned liking as a cue for JOLs. However, it is important to note that CS evaluations were much less pronounced than US evaluations. Given that the much stronger US extremity had only a modest correlation with JOLs, we consider it unlikely that people based their JOLs on the moderate CS valence. Nevertheless, this possibility could be addressed more thoroughly in future research.

JOLs as an Indicator of Attitudinal Certainty. Although the central dimension underlying attitudes is arguably valence (i.e., positive vs. negative), people’s attitudes also differ in other aspects. One key aspect is the certainty with which people hold a specific attitude (Tormala & Rucker, 2018). Although attitude certainty is conceptually different from attitude extremity (i.e., a person can be certain that they feel slightly positive towards an attitude object), they are empirically related (Petrocelli et al., 2007). Thus, more extreme CS evaluations might be the expression of a higher certainty. Likewise, participants’ certainty regarding the attitude towards a CS might be reflected in JOLs, contributing to the correlation between JOLs and EC. Future research could investigate to what extent JOLs correspond with specific components of attitudinal judgments, such as certainty.

Overall, our studies attest to the role of metamemory in attitude change via evaluative conditioning. They imply that people’s subjective predictions about their memory for co-occurring positive or negative stimuli predict EC effects above and beyond actual memory. Our studies show that people have some insight into the memory processes underlying attitude formation, but also that people rely on cues that correlate with the strength of attitude change. Likewise, our studies demonstrate that several key insights on metamemory also generalize to evaluative learning procedures. Thereby, our research builds a bridge between two previously unrelated research fields, and also points out several new research ideas for others interested in crossing this bridge.

Contributed to conception and design: MI, FS, JW, HA, MU

Contributed to acquisition of data: MI, FS

Contributed to analysis and interpretation of data: MI

Drafted and/or revised the article: MI, FS, JW, HA, MU

Approved the submitted version for publication: MI, FS, JW, HA, MU

This research was supported by the Open Access Publication Funds of the Ruhr University Bochum.

The authors have no competing interests to declare.

All supplemental material can be found on this paper’s project page on the OSF: https://doi.org/10.17605/OSF.IO/5HJNF

All the stimuli, presentation materials, participant data, and analysis scripts can be found on this paper’s project page on the OSF: https://doi.org/10.17605/OSF.IO/5HJNF

1.

Due to previous evidence on emotionality effects in metamemory judgments (for a meta-analysis, see Yin et al., 2023), we additionally expected that JOLs would be higher for positive and negative compared to neutral stimulus pairings. Tests for this hypothesis are reported for all experiments.

2.

We tested the predicted positive relationship between JOLs and EC effects statistically through a moderation. That is, we expected that for higher JOLs, the EC effect as elicited by the experimental manipulation should be stronger, which is in our preregistered analytical approach captured by the interaction terms JOL x Positive US or JOL x Negative US.

3.

We devote Experiment 5 to the question of whether there are differences between identity and valence memory. Nevertheless, due to the comments of a reviewer on a previous version of this manuscript, we re-analyzed Experiments 1-4 coding our memory test as to whether participants selected USs of the correct valence. The results were nearly identical and can be found in the HTML Markdown files at the OSF.

4.

Across experiments, the most frequent random effect structure had random intercepts for participants and random slopes for the effects of positive/negative valence.

5.

Note that this analysis only included 38 participants because no gamma correlations could be computed for the remaining participants due to perfect memory performance.

6.

Our moderation analyses therefore rely on a between-participants standardization of memory and a within-person standardization of JOLs. We chose these different standardizations because we considered the absolute value of a JOL less important than its position relative to other JOLs provided by a specific participant, whereas correct/incorrect memory has a theoretical meaning independent of a participant’s overall accuracy. Nevertheless, we fully replicated all findings in this experiment independent of whether we standardize memory and/or JOLs within- or between-person.

7.

Note that the moderations by JOLs and memory were only found for the negative US valence condition. The reason for this is probably that the neutral US pictures used in Experiment 1 were actually moderately positive (see Figure 1) and thus the difference between positive and neutral US valence was not statistically significant. We therefore changed the US stimulus pool in the upcoming experiments.

8.

Note that covariation between JOLs and EC effects could only arise in our analyses from intraindividual but not interindividual variation because we removed the between-participants variation in JOLs.

9.

Following a reviewer’s comment, we also conducted separate mediation analyses for the identity and the valence memory condition. The indirect effects through memory and US extremity emerged in both models. Results are provided at the OSF.

Albrecht, S., & Carbon, C.-C. (2014). The Fluency Amplification Model: Fluent stimuli show more intense but not evidently more positive evaluations. Acta Psychologica, 148, 195–203. https://doi.org/10.1016/j.actpsy.2014.02.002
Allport, G. W. (1935). Attitudes. In A Handbook of Social Psychology (pp. 798–844). Clark University Press.
Alves, H., Högden, F., Gast, A., Aust, F., & Unkelbach, C. (2020). Attitudes from mere co-occurrences are guided by differentiation. Journal of Personality and Social Psychology, 119(3), 560–581. https://doi.org/10.1037/pspa0000193
Alves, H., Koch, A., & Unkelbach, C. (2018). A Cognitive-Ecological Explanation of Intergroup Biases. Psychological Science, 29(7), 1126–1133. https://doi.org/10.1177/0956797618756862
Arbuckle, T. Y., & Cuddy, L. L. (1969). Discrimination of item strength at time of presentation. Journal of Experimental Psychology, 81(1), 126–131. https://doi.org/10.1037/h0027455
Bading, K., Stahl, C., & Rothermund, K. (2020). Why a standard IAT effect cannot provide evidence for association formation: The role of similarity construction. Cognition and Emotion, 34(1), 128–143. https://doi.org/10.1080/02699931.2019.1604322
Bao, H.-W.-S. (2022). bruceR: Broadly Useful Convenient and Efficient R Functions. https://cran.r-project.org/package=bruceR
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml.2012.11.001
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
Béna, J., Mauclet, A., & Corneille, O. (2023). Does co-occurrence information influence evaluations beyond relational meaning? An investigation using self-reported and mouse-tracking measures of attitudinal ambivalence. Journal of Experimental Psychology: General, 152(4), 968–992. https://doi.org/10.1037/xge0001308
Besken, M. (2016). Picture-perfect is not perfect for metamemory: Testing the perceptual fluency hypothesis with degraded images. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(9), 1417–1433. https://doi.org/10.1037/xlm0000246
Briñol, P., & Petty, R. E. (2022). Self-validation theory: An integrative framework for understanding when thoughts become consequential. Psychological Review, 129(2), 340–367. https://doi.org/10.1037/rev0000340
Brunswik, E. (1956). Perception and the Representative Design of Psychological Experiments. University of California Press. https://doi.org/10.1525/9780520350519
Corneille, O., & Stahl, C. (2018). Associative Attitude Learning: A Closer Look at Evidence and How It Relates to Attitude Models. Personality and Social Psychology Review, 23(2), 161–189. https://doi.org/10.1177/1088868318763261
De Houwer, J. (2007). A Conceptual and Theoretical Analysis of Evaluative Conditioning. The Spanish Journal of Psychology, 10(2), 230–241. https://doi.org/10.1017/S1138741600006491
De Houwer, J. (2018). Propositional Models of Evaluative Conditioning. Social Psychological Bulletin, 13(3), 1–21. https://doi.org/10.5964/spb.v13i3.28046
Double, K. S., & Birney, D. P. (2019). Reactivity to measures of metacognition. Frontiers in Psychology, 10, 2755. https://doi.org/10.3389/fpsyg.2019.02755
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. https://doi.org/10.3758/BF03193146
Fiedler, K., Schott, M., & Meiser, T. (2011). What mediation analysis can (not) do. Journal of Experimental Social Psychology, 47(6), 1231–1236. https://doi.org/10.1016/j.jesp.2011.05.007
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906
Förderer, S., & Unkelbach, C. (2013). On the stability of evaluative conditioning effects: The role of identity memory, valence memory, and evaluative consolidation. Social Psychology, 44, 380–389. https://doi.org/10.1027/1864-9335/a000150
Gast, A. (2018). A Declarative Memory Model of Evaluative Conditioning. Social Psychological Bulletin, 13(3), 1–23. https://doi.org/10.5964/spb.v13i3.28590
Gast, A., De Houwer, J., & De Schryver, M. (2012). Evaluative conditioning can be modulated by memory of the CS–US pairings at the time of testing. New Directions in Evaluative Conditioning Research, 43(3), 116–126. https://doi.org/10.1016/j.lmot.2012.06.001
Gawronski, B. (2022). Attitudinal Effects of Stimulus Co-occurrence and Stimulus Relations: Paradoxical Effects of Cognitive Load. Personality and Social Psychology Bulletin, 48(10), 1438–1450. https://doi.org/10.1177/01461672211044322
Gawronski, B., & Bodenhausen, G. V. (2018). Evaluative Conditioning From the Perspective of the Associative-Propositional Evaluation Model. Social Psychological Bulletin, 13(3 SE-Review Article), 1–33. https://doi.org/10.5964/spb.v13i3.28024
Gawronski, B., & Walther, E. (2012). What do memory data tell us about the role of contingency awareness in evaluative conditioning? Journal of Experimental Social Psychology, 48(3), 617–623. https://doi.org/10.1016/j.jesp.2012.01.002
Graf, L. K. M., Mayer, S., & Landwehr, J. R. (2018). Measuring processing fluency: One versus five items. Journal of Consumer Psychology, 28(3), 393–411. https://doi.org/10.1002/jcpy.1021
Hahn, A., & Gawronski, B. (2019). Facing one’s implicit biases: From awareness to acknowledgment. Journal of Personality and Social Psychology, 116(5), 769–794. https://doi.org/10.1037/pspi0000155
Hahn, A., Judd, C. M., Hirsh, H. K., & Blair, I. V. (2014). Awareness of implicit attitudes. Journal of Experimental Psychology: General, 143(3), 1369–1392. https://doi.org/10.1037/a0035028
Hofmann, W., De Houwer, J., Perugini, M., Baeyens, F., & Crombez, G. (2010). Evaluative conditioning in humans: A meta-analysis. Psychological Bulletin, 136(3), 390–421. https://doi.org/10.1037/a0018916
Högden, F., Hütter, M., & Unkelbach, C. (2018). Does evaluative conditioning depend on awareness? Evidence from a continuous flash suppression paradigm. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(10), 1641–1657. https://doi.org/10.1037/xlm0000533
Hourihan, K. L., Fraundorf, S. H., Benjamin, A. S. (2017). The influences of valence and arousal on judgments of learning and on recall. Memory Cognition, 45(1), 121–136. https://doi.org/10.3758/s13421-016-0646-3
Hu, X., Zheng, J., Fan, T., Su, N., Yang, C., Luo, L. (2020). Using multilevel mediation model to measure the contribution of beliefs to judgments of learning. Frontiers in Psychology, 11, 637. https://doi.org/10.3389/fpsyg.2020.00637
Hütter, M. (2022). An integrative review of dual- and single-process accounts of evaluative conditioning. Nature Reviews Psychology, 1(11), 640–653. https://doi.org/10.1038/s44159-022-00102-7
Hütter, M., De Houwer, J. (2017). Examining the contributions of memory-dependent and memory-independent components to evaluative conditioning via instructions. Journal of Experimental Social Psychology, 71, 49–58. https://doi.org/10.1016/j.jesp.2017.02.007
Hütter, M., Niese, Z. A., Ihmels, M. (2022). Bridging the gap between autonomous and predetermined paradigms: The role of sampling in evaluative learning. Journal of Experimental Psychology: General, 151(8), 1972–1998. https://doi.org/10.1037/xge0001172
Hütter, M., Sweldens, S. (2013). Implicit misattribution of evaluative responses: Contingency-unaware evaluative conditioning requires simultaneous stimulus presentations. Journal of Experimental Psychology: General, 142(3), 638–643. https://doi.org/10.1037/a0029989
Hütter, M., Sweldens, S. (2018). Dissociating Controllable and Uncontrollable Effects of Affective Stimuli on Attitudes and Consumption. Journal of Consumer Research, 45(2), 320–349. https://doi.org/10.1093/jcr/ucx124
Hütter, M., Sweldens, S., Stahl, C., Unkelbach, C., Klauer, K. C. (2012). Dissociating contingency awareness and conditioned attitudes: Evidence of contingency-unaware evaluative conditioning. Journal of Experimental Psychology: General, 141(3), 539–557. https://doi.org/10.1037/a0026477
Ingendahl, M., Propheter, N., Vogel, T. (2024). The role of category valence in prototype preference. Cognition Emotion. https://doi.org/10.1080/02699931.2024.2335536
Ingendahl, M., Vogel, T. (2022). Stimulus Evaluation in the Eye of the Beholder: Big Five Personality Traits Explain Variance in Normed Picture Sets. Personality Science, 3. https://doi.org/10.5964/ps.7951
Ingendahl, M., Vogel, T. (2023). (Why) Do Big Five Personality Traits moderate Evaluative Conditioning? The role of US extremity and pairing memory. Collabra:Psychology, 9(1), 74812. https://doi.org/10.1525/collabra.74812
Ingendahl, M., Vogel, T., Maedche, A., Wänke, M. (2023). Brand placements in video games: How local in-game experiences influence brand attitudes. Psychology Marketing, 56(2), 274–287. https://doi.org/10.1002/mar.21770
Ingendahl, M., Vogel, T., Woitzel, J., Bücker, N., Boers, J., Alves, H. (in press). The Interplay of Multiple Unconditioned Stimuli in Evaluative Conditioning: A Weighted Averaging Framework for Attitude Formation via Stimulus Co-Occurrences. Journal of Personality and Social Psychology.
Ingendahl, M., Woitzel, J., Alves, H. (2023). Just Playing the Role of Good Study Participants? Evaluative Conditioning, Demand Compliance, and Agreeableness. Social Psychological and Personality Science. https://doi.org/10.1177/19485506231198653
Ingendahl, M., Woitzel, J., Propheter, N., Wänke, M., Alves, H. (2023). From Deviant Likes to Reversed Effects: Re-Investigating the Contribution of Unaware Evaluative Conditioning To Attitude Formation. Collabra:Psychology. https://doi.org/10.1525/collabra.87462
Jia, X., Li, P., Li, X., Zhang, Y., Cao, W., Cao, L., Li, W. (2016). The effect of word frequency on judgments of learning: Contributions of beliefs and processing fluency. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.01995
Jones, C. R., Fazio, R. H., Olson, M. A. (2009). Implicit misattribution as a mechanism underlying evaluative conditioning. Journal of Personality and Social Psychology, 96(5), 933–948. https://doi.org/10.1037/a0014747
Koriat, A. (1997). Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General, 126(4), 349–370. https://doi.org/10.1037/0096-3445.126.4.349
Koriat, A., Bjork, R. A., Sheffer, L., Bar, S. K. (2004). Predicting one’s own forgetting: The role of experience-based and theory-based processes. Journal of Experimental Psychology: General, 133(4), 643–656. https://doi.org/10.1037/0096-3445.133.4.643
Koriat, A., Ma’ayan, H. (2005). The effects of encoding fluency and retrieval fluency on judgments of learning. Journal of Memory and Language, 52(4), 478–492. https://doi.org/10.1016/j.jml.2005.01.001
Koriat, A., Ma’ayan, H., Nussinson, R. (2006). The intricate relationships between monitoring and control in metacognition: Lessons for the cause-and-effect relation between subjective experience and behavior. Journal of Experimental Psychology: General, 135(1), 36–69. https://doi.org/10.1037/0096-3445.135.1.36
Kurdi, B., Lozano, S., Banaji, M. R. (2017). Introducing the Open Affective Standardized Image Set (OASIS). Behavior Research Methods, 49(2), 457–470. https://doi.org/10.3758/s13428-016-0715-3
Kurdi, B., Morehouse, K. N., Dunham, Y. (2023). How do explicit and implicit evaluations shift? A preregistered meta-analysis of the effects of co-occurrence and relational information. Journal of Personality and Social Psychology, 124, 1174–1202. https://doi.org/10.1037/pspa0000329
Landwehr, J. R., Eckmann, L. (2020). The nature of processing fluency: Amplification versus hedonic marking. Journal of Experimental Social Psychology, 90, 103997. https://doi.org/10.1016/j.jesp.2020.103997
Landwehr, J. R., Golla, B., Reber, R. (2017). Processing fluency: An inevitable side effect of evaluative conditioning. Journal of Experimental Social Psychology, 70, 124–128. https://doi.org/10.1016/j.jesp.2017.01.004
Lee, H. S., Ha, H. (2019). Metacognitive judgments of prior material facilitate the learning of new material: The forward effect of metacognitive judgments in inductive learning. Journal of Educational Psychology, 111(7), 1189–1201. https://doi.org/10.1037/edu0000339
Leiner, D. (2019). Sosci Survey. https://www.soscisurvey.de
Li, X., Chen, G., Yang, C. (2021). How cognitive conflict affects judgments of learning: Evaluating the contributions of processing fluency and metamemory beliefs. Memory Cognition, 49(5), 912–922. https://doi.org/10.3758/s13421-021-01143-8
March, D. S., Olson, M. A., Fazio, R. H. (2019). The Implicit Misattribution Model of Evaluative Conditioning. Social Psychological Bulletin, 13(3 SE-Review Article), 1–25. https://doi.org/10.5964/spb.v13i3.27574
Mendes, P. S., Luna, K., Albuquerque, P. B. (2021). Word frequency effects on judgments of learning: More than just beliefs. The Journal of General Psychology, 148(2), 124–148. https://doi.org/10.1080/00221309.2019.1706073
Mierop, A., Hütter, M., Corneille, O. (2017). Resource Availability and Explicit Memory Largely Determine Evaluative Conditioning Effects in a Paradigm Claimed to be Conducive to Implicit Attitude Acquisition. Social Psychological and Personality Science, 8(7), 758–767. https://doi.org/10.1177/1948550616687093
Moran, T., Hughes, S., Van Dessel, P., De Houwer, J. (2022). The Role of Trait Inferences in Evaluative Conditioning. Collabra: Psychology, 8(1), Article1. https://doi.org/10.1525/collabra.31738
Moran, T., Nudler, Y., Anan, Y. B. (2023). Evaluative Conditioning: Past, Present, and Future. Annual Review of Psychology, 74, 245–269. https://doi.org/10.1146/annurev-psych-032420-031815
Morris, A., Kurdi, B. (2023). Awareness of implicit attitudes: Large-scale investigations of mechanism and scope. Journal of Experimental Psychology: General, 152(2), 3311–3343. https://doi.org/10.1037/xge0001464
Mueller, M. L., Dunlosky, J. (2017). How beliefs can impact judgments of learning: Evaluating analytic processing theory with beliefs about fluency. Journal of Memory and Language, 93, 245–258. https://doi.org/10.1016/j.jml.2016.10.008
Mueller, M. L., Tauber, S. K., Dunlosky, J. (2013). Contributions of beliefs and processing fluency to the effect of relatedness on judgments of learning. Psychonomic Bulletin Review, 20(2), 378–384. https://doi.org/10.3758/s13423-012-0343-6
Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin, 95, 109–133. https://doi.org/10.1037/0033-2909.95.1.109
Niese, Z. A., Hütter, M. (2023). The malleability of sampling’s impact on evaluation: Sampling goals moderate the evaluative impact of sampling a stimulus. Journal of Experimental Social Psychology, 109, 104516. https://doi.org/10.1016/j.jesp.2023.104516
Olson, M. A., Fazio, R. H. (2006). Reducing Automatically Activated Racial Prejudice Through Implicit Evaluative Conditioning. Personality and Social Psychology Bulletin, 32(4), 421–433. https://doi.org/10.1177/0146167205284004
Petrocelli, J. V., Tormala, Z. L., Rucker, D. D. (2007). Unpacking attitude certainty: Attitude clarity and attitude correctness. Journal of Personality and Social Psychology, 92(1), 30–41. https://doi.org/10.1037/0022-3514.92.1.30
Reber, R., Schwarz, N., Winkielman, P. (2004). Processing Fluency and Aesthetic Pleasure: Is Beauty in the Perceiver’s Processing Experience? Personality and Social Psychology Review, 8, 364–382. https://doi.org/10.1207/s15327957pspr0804_3
Rhodes, M. G. (2016). Judgments of learning: Methods, data, and theory. In J. Dunlosky S. K. Tauber (Eds.), The Oxford Handbook of Metamemory (Vol. 1). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199336746.013.4
Rhodes, M. G., Castel, A. D. (2008). Memory predictions are influenced by perceptual information: Evidence for metacognitive illusions. Journal of Experimental Psychology: General, 137(4), 615–625. https://doi.org/10.1037/a0013684
Rhodes, M. G., Tauber, S. K. (2011). The influence of delaying judgments of learning on metacognitive accuracy: A meta-analytic review. Psychological Bulletin, 137, 131–148. https://doi.org/10.1037/a0021705
Richter, J., Gast, A. (2017). Distributed practice can boost evaluative conditioning by increasing memory for the stimulus pairs. Acta Psychologica, 179, 1–13. https://doi.org/10.1016/j.actpsy.2017.06.007
Schäfer, F., Undorf, M. (2024). On the educational relevance of immediate judgment of learning reactivity: No effects of predicting one’s memory for general knowledge facts. Journal of Applied Research in Memory and Cognition, 13(1), 113–123. https://doi.org/10.1037/mac0000113
Serra, M. J., Ariel, R. (2014). People use the memory for past-test heuristic as an explicit cue for judgments of learning. Memory Cognition, 42(8), 1260–1272. https://doi.org/10.3758/s13421-014-0431-0
Soderstrom, N. C., Yue, C. L., Bjork, E. L. (2016). Metamemory and education. In J. Dunlosky S. K. Tauber (Eds.), The Oxford Handbook of Metamemory (Vol. 1). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199336746.013.6
Sperlich, L. M., Unkelbach, C. (2022). When do people learn likes and dislikes from co-occurrences? A dual-force perspective on evaluative conditioning. Journal of Experimental Social Psychology, 103, 104377. https://doi.org/10.1016/j.jesp.2022.104377
Stahl, C., Aust, F. (2018). Evaluative Conditioning as Memory-Based Judgment. Social Psychological Bulletin, 13(3 SE-Review Article), 1–30. https://doi.org/10.5964/spb.v13i3.28589
Stahl, C., Aust, F., Bena, J., Mierop, A., Corneille, O. (2023). A conditional judgment procedure for probing evaluative conditioning effects in the absence of feelings of remembering. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02081-w
Stahl, C., Haaf, J., Corneille, O. (2016). Subliminal evaluative conditioning? Above-chance CS identification may be necessary and insufficient for attitude learning. Journal of Experimental Psychology: General, 145(9), 1107–1131. https://doi.org/10.1037/xge0000191
Stahl, C., Unkelbach, C., Corneille, O. (2009). On the respective contributions of awareness of unconditioned stimulus valence and unconditioned stimulus identity in attitude formation through evaluative conditioning. Journal of Personality and Social Psychology, 97(3), 404–420. https://doi.org/10.1037/a0016196
Sweldens, S., Corneille, O., Yzerbyt, V. (2014). The Role of Awareness in Attitude Formation Through Evaluative Conditioning. Personality and Social Psychology Review, 18(2), Article2. https://doi.org/10.1177/1088868314527832
Sweldens, S., Van Osselaer, S. M. J., Janiszewski, C. (2010). Evaluative Conditioning Procedures and the Resilience of Conditioned Brand Attitudes. Journal of Consumer Research, 37(3), 473–489. https://doi.org/10.1086/653656
Tekin, E., Roediger, H. L. (2020). Reactivity of Judgments of Learning in a Levels-of-Processing Paradigm. Zeitschrift für Psychologie, 228(4), 278–290. https://doi.org/10.1027/2151-2604/a000425
Tormala, Z. L., Rucker, D. D. (2018). Attitude certainty: Antecedents, consequences, and new directions. Consumer Psychology Review, 1(1), 72–89. https://doi.org/10.1002/arcp.1004
Undorf, M. (2020). Fluency illusions in metamemory. In A. M. Cleary B. L. Schwartz (Eds.), Memory Quirks: The Study of Odd Phenomena in Memory (pp. 150–174). Routledge.
Undorf, M., Bröder, A. (2020). Cue integration in metamemory judgements is strategic. Quarterly Journal of Experimental Psychology, 73(4), 629–642. https://doi.org/10.1177/1747021819882308
Undorf, M., Bröder, A. (2021). Metamemory for pictures of naturalistic scenes: Assessment of accuracy and cue utilization. Memory Cognition, 49(7), 1405–1422. https://doi.org/10.3758/s13421-021-01170-5
Undorf, M., Erdfelder, E. (2011). Judgments of learning reflect encoding fluency: Conclusive evidence for the ease-of-processing hypothesis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1264–1269. https://doi.org/10.1037/a0023719
Undorf, M., Erdfelder, E. (2015). The relatedness effect on judgments of learning: A closer look at the contribution of processing fluency. Memory Cognition, 43(4), 647–658. https://doi.org/10.3758/s13421-014-0479-x
Undorf, M., Navarro-Báez, S., Bröder, A. (2022). “You don’t know what this means to me” – Uncovering idiosyncratic influences on metamemory judgments. Cognition, 222, 105011. https://doi.org/10.1016/j.cognition.2021.105011
Undorf, M., Söllner, A., Bröder, A. (2018). Simultaneous utilization of multiple cues in judgments of learning. Memory Cognition, 46(4), 507–519. https://doi.org/10.3758/s13421-017-0780-6
Undorf, M., Zander, T. (2017). Intuition and metacognition: The effect of semantic coherence on judgments of learning. Psychonomic Bulletin Review, 24(4), 1217–1224. https://doi.org/10.3758/s13423-016-1189-0
Undorf, M., Zimdahl, M. F., Bernstein, D. M. (2017). Perceptual fluency contributes to effects of stimulus size on judgments of learning. Journal of Memory and Language, 92, 293–304. https://doi.org/10.1016/j.jml.2016.07.003
Unkelbach, C., Fiedler, K. (2016). Contrastive CS-US Relations Reverse Evaluative Conditioning Effects. Social Cognition, 34(5), 413–434. https://doi.org/10.1521/soco.2016.34.5.413
Võ, M. L. H., Conrad, M., Kuchinke, L., Urton, K., Hofmann, M. J., Jacobs, A. M. (2009). The Berlin Affective Word List Reloaded (BAWL-R). Behavior Research Methods, 41(2), 534–538. https://doi.org/10.3758/BRM.41.2.534
Vogel, T., Ingendahl, M., Winkielman, P. (2021). The Architecture of Prototype Preferences: Typicality, Fluency, and Valence. Journal of Experimental Psychology: General, 150(1), 187–194. https://doi.org/10.1037/xge0000798
Vogel, T., Wänke, M. (2016). Attitudes and Attitude Change. Taylor Francis. https://doi.org/10.4324/9781315754185
Winkielman, P., Huber, D. E., Kavanagh, L., Schwarz, N. (2012). Fluency of consistency: When thoughts fit nicely and flow smoothly. In Cognitive consistency: A fundamental principle in social cognition (pp. 89–111).
Witherby, A. E., Tauber, S. K. (2018). Monitoring of learning for emotional faces: How do fine-grained categories of emotion influence participants’ judgments of learning and beliefs about memory? Cognition and Emotion, 32(4), 860–866. https://doi.org/10.1080/02699931.2017.1360252
Woitzel, J., Alves, H. (2024). The Formation of Negative Attitudes Towards Novel Groups. Psychological Science. https://doi.org/10.1177/09567976241239932
Yin, Y., Shanks, D. R., Li, B., Fan, T., Hu, X., Yang, C., Luo, L. (2023). The Effects of emotion on judgments of learning and memory: A meta-analytic review. Metacognition and Learning, 18(2), 425–447. https://doi.org/10.1007/s11409-023-09335-0
Zimmerman, C. A., Kelley, C. M. (2010). “I’ll remember this!” Effects of emotionality on memory predictions versus memory performance. Journal of Memory and Language, 62(3), 240–253. https://doi.org/10.1016/j.jml.2009.11.004
This is an open access article distributed under the terms of the Creative Commons Attribution License (4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Supplementary data