Leading Us Unto Temptation? No Evidence for an Asymmetry in Automatic Associations Between Goals and Temptations

To date, Fishbach et al. (2003) has been cited 1300 times (Google Scholar) and is one of the ten most highly cited social psychology papers on self-control (Web of Science). The studies described in Fishbach and colleagues (2003) provided evidence for multiple hypotheses from counteractive control theory, including (i) evidence for the asymmetrical association between temptations and goals, (ii) evidence that the priming asymmetry is automatic, occurring even with extremely fast stimulus-onset asynchronies, (iii) evidence that the temptation-to-goal priming effect is moderated by the individual difference of self-regulatory success, and (iv) evidence that this priming effect can have behavioural consequences. Our replication study is designed to test the robustness of the first three of these four hypotheses.

Given the importance of counteractive control theory and the high-profile nature of the original manuscript, the lack of published replications of the asymmetric association effect is a glaring omission. We thus report a pre-registered, highly powered test of these hypotheses, adhering to current standards of statistical power and transparency in psychological science.

The asymmetrical association between temptations and goals has been suggested to be moderated by various individual differences. Counteractive control, where the presence of temptations activate goal-related concepts, may be strongest among individuals with good self-regulatory abilities (Fishbach et al., 2003; Papies et al., 2008). In fact, it may only be students with high self-regulatory abilities who showed the predicted asymmetry between temptation-primes-goal and goal-primes-temptation, while students with low self-regulatory abilities may even show the opposite (Study 3; Fishbach et al., 2003). We thus aim to replicate the previously found moderation by perceived self-regulatory success. Trait self-control has likewise been theorized to predict goal accessibility, although empirical support is inconsistent (Haynes et al., 2014). Because trait self-control is conceptually similar to self-regulatory success, and the trait self-control measure has been validated and is more commonly used (de Ridder et al., 2012), we also examine whether trait self-control moderates the predicted asymmetry.

Next, construal level has been previously found to moderate the asymmetrical association (Fujita & Sasota, 2011). Construal level theory describes how people can think about the same event at different levels of abstraction—for example, one might focus either on low-level, concrete details of an event (e.g., describing writing as “holding a pen in my hand”) or on higher-level, abstract representations of the same event (e.g., “expressing my thoughts”). Fujita and Sasota (2011) reported that only those primed with an abstract construal level showed the asymmetrical association predicted by counteractive control; we aim to conceptually replicate this moderation.

Finally, using a different experimental design, temptation strength has been shown to affect goal accessibility in a lexical decision task after viewing a picture of a temptation (Kroese, Evers, et al., 2011). Participants more readily identified dieting goal-related words after being primed with a weak temptation (e.g., a picture of a somewhat attractive cake) and were slower to identify dieting goal-related words after being exposed to a stronger temptation (e.g., a picture of a highly attractive cake). We here examine whether subjective temptation strength also moderates the hypothesized asymmetrical association.

In a set of secondary preregistered analyses, we thus investigated four potential moderators: self-regulatory success, trait self-control, construal level, and temptation strength. Note that, in previous research, construal level and temptation strength were examined as experimentally manipulated, state-level moderators, while self-regulatory success and trait self-control were examined as measured, trait-level moderators. Because the primary purpose of this research was to replicate Study 3 of Fishbach et al. (2003), we did not include any additional manipulations in our experimental paradigm. Instead, we examine all four potential moderators as measured individual difference variables (i.e., at the trait-level; see Discussion).

For this study, we closely followed the methods of Study 3 of Fishbach et al. (2003), which focused on students’ academic goals and was conducted with a sample of undergraduate students. However, unlike Studies 1 and 2, Study 3 only included trials where goals primed temptations and trials where temptations primed goals—it did not include trials where goal and temptation target words were primed with irrelevant words. These conditions are also necessary to test the asymmetry, to avoid confounding effects of the prime with effects of the target words. Thus, to best test the asymmetry claim (in addition to testing for moderation by self-regulatory success, which was the original purpose of Study 3), we expanded the method to include trials with irrelevant prime words, as in Study 1 of Fishbach et al. (2003). Additionally, we increased statistical power of our replication by increasing the number of target words in each category from four to six and by increasing the number of trials, substantially increasing the number of observations available for analysis (Snijders, 2005).

The detailed preregistration for this study, including a minimum sample size, stopping rules and exclusion criteria, analysis code, hypotheses, and study design are available at https://osf.io/4vruh/. We did not receive original materials from the original paper authors, nor were we made aware of any methodological details beyond those described in the published manuscript. We contacted the corresponding author of Fishbach et al. (2003) about our replication attempt prior to finalizing our preregistration.

We planned to recruit a minimum of 200 undergraduate students from the Introductory Psychology course participant pool at a large Canadian university. This sample size was selected to be 2.5 times larger than the original study (original N = 77), as recommended by Simonsohn (2015). A sensitivity power analysis conducted with simr (Green & Macleod, 2016) finds that our sample had over 95% power to detect an interaction effect of 30ms (r = .06); this effect size is one quarter of the asymmetry effect size found in Study 1 of Fishbach et al. (2003) and approximately half the asymmetry effect size from Study 3. Additionally, our sample had over 80% power to detect a correlation between the degree of priming asymmetry and self-regulatory success of r = .19 (from Study 3 of Fishbach et al., 2003) according to G*Power version 3.1.9 (Faul et al., 2009).

The first 92 students completed the study in-lab (programmed in DirectRT) in the winter of 2020, prior to the Covid-19 pandemic and resulting campus closures. At that point, to allow for data collection to continue online, we reprogrammed our study in jsPsych (de Leeuw, 2015) and recruited an additional 150 students from the university participant pool during the autumn of 2020. To account for potentially higher exclusion rates in the online environment, we recruited more participants online than needed to meet our 200-person minimum preregistered sample size. Altogether, the study was completed by 242 participants (age M = 18.57, SD = 1.54), including 172 women and 70 men. We preregistered that participants who had less than 80% accuracy on the task would be excluded; this exclusion criteria removed 21 participants (including 6 who participated in-lab and 15 who participated online), resulting in a final sample of N = 221.

Because this study is based on a lexical decision task, we restricted our recruitment to those participants who had indicated in an earlier pre-screen that they were fully proficient at English. Most of the sample had English as their only first native language (28%) or one of their first native languages (37.1%). Of those who learned English as a second language, the average participant had been using English full-time (either at home or at school) since the median age of 5.5 years old (M = 6.24, SD = 3.24).

The sample of students rated their academic goals as highly important (on a scale from 1-6, M = 5.34, SD = 0.73, mode = 6). The importance of academic goals for these participants was not significantly different from the importance of academic goals among the students who participated in Fishbach et al. (2003)’s Study 3 (converted to 6-point scale, M = 5.17, SD = 0.84, mode = 6; comparison t(296) = 1.69, p = .092).

The original study used four goal-related words and four temptation-related words. We chose to increase the number of goal-related and temptation-related words from four to six to slightly increase stimuli generalizability (Yarkoni, 2020). We ensured that the six goal-words and six temptations-words were matched on lexical characteristics that are known to affect response speed, including word length and detection accuracy, according to the English Lexicon Project (Balota et al., 2007). We also collected pilot data to ensure that the words were appropriately categorized as goal- or temptation-related (N = 130 undergraduates from a different large Canadian university participant pool). Fourteen potential target words were validated based on the extent to which students perceive them to be helpful or detrimental to their academic goal, on scales from 0 to 9. Two words were removed based on results from the pilot study.

The goal-related words that were ultimately used for the lexical decision task were “study”, “grades”, “graduate”, and “homework” (as in Fishbach et al., 2003) along with “memorization” and “school” (new additions). The temptation-related words were “procrastinate”, “television”, and “phone” (from Fishbach et al., 2003), along with “distraction”, “game”, and “party” (new additions).

To create a list of irrelevant words and non-words, we entered the lexical properties of our selected goal and temptation words into the English Lexicon Project website. Provided words and non-words were then sorted to remove words with punctuation or capitalization, and to remove words that could be related to academics or common temptations. We ultimately used 148 irrelevant words and 215 non-words (available at https://osf.io/4vruh/).

At the beginning of each trial, a prime word was presented on the screen for 50ms¹ and was then replaced by a masking string of 13 ‘X’s for 17ms (Figure 1). A target letter string then appeared on the screen and remained on screen until a response was recorded (no maximum response time; outliers were excluded, see ’Analysis’). The stimulus-onset asynchrony (SOA) was thus 67ms, as it was in Study 3 of Fishbach et al. (2003). The participant was required to classify the target string as a word or non-word using the D and K keys on the keyboard. For half of the participants, D classified the text as a word and K classified the text as a non-word. For the other half of the participants, the response keys were reversed. All stimuli were presented on the centre of a white screen with black letters. Following a 400ms intertrial interval, the next trial began.

The experiment began with 10 practice trials in the lexical decision-making task. Following the practice session, 504 trials were completed in a random order, with an equal number of word targets (252) and non-word targets (252). There were two breaks, occurring after 156 trials and after 314 trials. At these points, the screen displayed, “You are done block 1 [or 2] (out of 3 blocks). Take a break, and then press any key when you are ready to continue”.

Critical Trials. Included within the 252 word trials were 72 critical trials. Critical trials included 36 trials where a temptation-related word primed a goal-related target, and 36 critical trials where a goal-related word primed a temptation-related target. Each combination of the six goal words and six temptation words were paired once. To act as a comparison, in 18 trials an irrelevant word primed a temptation-related target and in 18 trials an irrelevant word primed a goal-related target. These 72 critical trials plus 36 irrelevant-prime comparison trials were the trials included in the analyses.

Other Trials. The remainder of the 252 “word response” trials consisted of 18 trials of temptations priming neutral words, 18 trials of goals priming neutral words, and 108 trials of neutral words priming neutral words. In the set of 252 “non-word response” trials, there were 6 trials of temptations priming non-words, 6 trials of goals priming non-words, and 240 trials of neutral words priming non-words.

Academic Goal Importance. Goal importance was measured using a single item: “Think of your main academic goal for this semester. How important is this academic goal to you?”. One participant selected “Not applicable – I do not have an academic goal”. All other participants chose “Extremely important” (46%), “Very important” (42%), or “Moderately important” (11%). No participants selected the options “Slightly important” or “Not important”.

Trait Self-Control. Participants next completed the 13 item trait self-control scale (Tangney et al., 2004). Each item was measured on a 1-7 scale from strongly disagree to strongly agree, and the items were averaged (M = 4.01, SD = 0.93, $α$ = .82).

Construal Level. To measure trait construal level, we used 20 items from the Behaviour Identification Form (Vallacher & Wegner, 1987). This measure presents 20 behaviours (e.g., “Making a list”) and asks participants to choose between two options that describe that behaviour at a high or low construal level (e.g., “Getting organized” or “Writing things down”). All high-construal choices were summed together. Our sample had a M = 12.25, SD = 4.27, $α$ =.80.

Self-Regulatory Success. Following Fishbach et al. (2003), we measured student’s perceived self-regulatory success in the domain of schoolwork using two revere-coded items: “How difficult is it for you to get good grades?” and “How difficult is it for you to complete your coursework?”, each measured on a scale from 1 to 7. The two items correlated at r = .57 and our sample, on average, responded slightly above the midpoint (M = 3.91, SD = 1.21). This correlation between the two items was significantly lower than the correlation found in Fishbach et al., 2003 (Study 3, r = .83; Fisher comparison z = 3.64, p .001).

Temptation Strength. To measure temptation strength, we asked participants to indicate how appealing (or tempting) they generally find each of six activities, using a sliding scale from 0 (Not at all) to 9 (Extremely appealing). The activities (“Watching TV”, “Using your smartphone”, “Playing games”, “Procrastinating on homework”) each corresponded to the temptation-related words used in the earlier lexical decision task. The temptation strengths for each of the six activities were averaged for each participant (M = 4.38, SD = 1.63, $α$ = 0.59).

In the first component of the study, participants completed a computerized lexical decision-making task through a sequential priming procedure. The decision-making task was executed by DirectRT for the in-lab participants. For the online participants, the task was re-programmed using jsPsych (de Leeuw, 2015) and hosted on the free platform Cognition (http://cognition.run). At the conclusion of the lexical decision task, participants were asked if they saw any words especially frequently (in-lab DirectRT participants only)².

After completing the lexical decision task, participants were informed that they would be completing a second study using Qualtrics. For the in-lab participants, a research assistant came over to their computer, exited the current program, and clicked on the Qualtrics link in a browser. The online participants were told that the lexical decision task was done, and that they would be redirected to Qualtrics for a study on education (they then clicked any key to be redirected from Cognition.run to Qualtrics).

Participants first responded to an item on the importance of their academic goal and an item on the importance of two non-academic goals of their choosing. They then were presented with 20 words in a random order (including the 12 critical temptation and goal-related target words from the earlier lexical decision task) and asked to what extent each word helps their academic goal, hurts their academic goal, and helps and hurts each of their two other goals. This allowed us to examine whether the target words were seen as beneficial, detrimental, irrelevant, or both beneficial and detrimental to their academic goal.

Participants next completed questionnaires measuring their trait self-control, goal temptation, and construal level. They then indicated their age, gender, number of languages that they speak and read, and years of English experience. Lastly, participants were asked three questions in a funnel debrief, to see whether they knew what the experiment was about. As preregistered, we did not exclude participants based on their responses to the funnel debrief but instead evaluated it as an exploratory measure³.

Following the completion of the study, participants were debriefed and received course credit as compensation.

Reaction TimeData Exclusions. We excluded extreme outlier trials for each person according to two preregistered criteria. First, we removed trials that were more than 4 median absolute deviations (MADs) away from the median for a given person and a given target category (each of non-words, goals, temptations, and neutral words). Using median absolute deviation is a more robust method for outlier detection compared to standard deviation (Leys et al., 2013). Second, we excluded trials where the reaction time was less than 200ms. In total, 5.28% of trials were excluded due to these criteria.

Following Fishbach et al. (2003), our analyses focused only on the critical trials where (i) participants correctly identified the target word, and (ii) the target word was either temptation-related or goal-related. The target word may have either been primed with an irrelevant neutral word, or by a relevant word (a temptation priming goal, or a goal priming temptation). Analyses thus included the 22,772 critical trials from across 220 participants.

Statistical Models. We followed our preregistered analysis plan that specified that we would conduct trial-level multi-level models with random intercepts for participants and random intercepts for the target word stimuli. Analyzing trial-level data is recommended where possible, as it does not ignore the variability that exists in human behaviour across different trials, and the resulting statistics better match the theoretical claims that we aim to make (Lo & Andrews, 2015; Speelman & McGann, 2013; Whelan, 2008). Furthermore, only by analyzing trial-level data can we model random stimuli for each target word, allowing us to more readily generalize our results to target words beyond the particular words included in the study (Yarkoni, 2020). Because raw reaction time was the dependent variable, and reaction times are nearly always right-skewed, we specified an inverse Gaussian distribution (following recommendations by Lo & Andrews, 2015) using the glmer function⁴ from the lme4 version 1.1-27.1 package (Bates et al., 2015) with R version 4.0.4 (R Core Team, 2021). P-values were calculated with the lmerTest function version 3.1-3 (Kuznetsova et al., 2017), which uses degrees of freedom calculated via the Satterthwaite method. Individual difference measures (self-regulatory success, trait self-control, construal level, and averaged temptation strength) were each grand-mean centred.

We also report additional analyses (not preregistered), where we follow the analysis procedures from Fishbach et al. (2003) to allow for clearer comparisons between the original study and the current replication.

As stated in our preregistration, we analyzed the incomplete dataset at one intermediate point; data from 33 participants was analyzed at the end of March 2020 so that the second author could complete course requirements. No other studies were conducted on this question. We have reported all exclusions, conditions, and measures.

We first tested whether the prime—either relevant or irrelevant—affected the reaction time of the following word, and whether the priming effect differed depending on whether it was goal-priming-temptation or temptation-priming-goal. There was a small but significant overall main effect of priming (b = −8.87ms, SE = 1.46, t(22194) = −6.08, p .001). People responded approximately 9ms faster to target words (either goal or temptation words) when a related word had been subliminally primed rather than an irrelevant word. There was also no overall difference in lexical decision speed depending on whether the target word was related to goals or temptations (b = −5.15ms, SE = 5.54, t(22194) = −0.93, p = .35).

But was the priming effect asymmetrical, depending on whether a goal primed a temptation, or a temptation primed a goal? We did not replicate the previously reported asymmetry effect (Figure 2). The identity of the target word (temptation or goal related) did interact with the prime (relevant or irrelevant) at $α$ = 0.05 significance, b = 5.41ms, SE = 2.62, t(22194) = 2.07, p = .039, but the pattern was not as predicted. A temptation-related prime facilitated faster recognition of a goal-related word (b = −6.16ms, SE = 1.94, t(22194) = −3.17, p = .002), but a goal-related prime facilitated recognition of a temptation-related word to an even greater degree (b = −11.57ms, SE = 1.95, t(22194) = −5.94, p .001). We did not find evidence that temptations primed goals more than goals primed temptations—if anything, these data suggested the opposite asymmetry. The strongest effect, however, was the overall priming effect, where any relevant prime (goal or temptation) resulted in a faster recognition of the subsequent target word (temptation or goal).

We next investigated four potential individual difference moderators that have each been suggested to moderate the goal-temptation asymmetry effect: trait self-control, construal level, self-regulatory success, and temptation strength.

None of the four individual difference variables significantly moderated either the priming facilitation effect nor moderated the hypothesized priming asymmetry between temptation- and goal-related target words (Table 1). In other words, we did not find the hypothesized patterns that the asymmetric association between temptations and goals occurs only for those high in self-regulatory success, high in trait self-control, high in abstract construal level, or depending on temptation strength.

Trait self-control did significantly predict speed of reaction time differently for goal-related and temptation-related words (b = −3.27ms, SE = 1.57, t(22190) = −2.08, p = .037). Participants with higher trait self-control responded faster to goal-related words compared to temptation-related words (at +1SD, b = −8.3ms), while participants with lower trait self-control responded to the two target types at a more comparable speed (at −1SD, b = −2.15ms). Trait self-control also moderated the degree of the priming effect (b = 3.14, SE = 1.58, t(22190) = 1.99, p = .046), with participants who were lower in trait self-control showing a stronger facilitation effect of relevant primes (b = −11.85) compared to those with average or higher trait self-control.

Because this study included more trials than the lexical decision task in Fishbach et al. (2003), we examined whether the results changed across the course of the task by looking at moderation by block (first block coded as −0.5, second block = 0, third block = 0.5). Participants did respond more quickly later in the last block of the task, compared to the first block (b = −42.39ms, SE = 1.85, t(22190) = −22.97, p .001). Furthermore, the overall priming facilitation effect changed across the course of the task (interaction b = −8.62ms, SE = 3.68, t(22190) = −2.34, p = .019). The priming effect was smaller in the first block of the task (b = −4.56ms, SE = 2.20, t(22190) = −2.07, p = .038) and was larger in the final block of the task (b = −13.18ms, SE = 2.24, t(22190) = −5.87, p .001). Block did not moderate the pattern of priming for temptation vs. goal-related words (two-way interaction between target-type and block b = −3.72ms, SE = 3.68, t(22190) = −1.01, p = .312; three-way interaction, b = 0.18ms, SE = 7.36, t(22190) = 0.02, p = .984).

In short, while reaction times did change over the course of the study and the overall degree of priming facilitation effect also changed with time-on-task, the additional length of the lexical decision task did not moderate the hypothesized asymmetrical association between temptations-priming-goals compared to goals-priming-temptations.

As an exploratory analysis, we investigated whether the patterns of reaction times differed between participants who conducted the study in the lab environment versus online. There was an overall difference in reaction time, with online participants responding approximately 52ms more slowly (b = 52.37ms, SE = 13.49, t(22190) = 3.88, p .001) than participants who conducted the experiment in-person in the behavioural lab; this fixed additional time is typical when recording online (Semmelmann & Weigelt, 2017). However, the location of the study did not moderate the effect of prime relevancy, the effect of target type (goal vs. temptation words), nor the patterns of priming effects for goal vs. temptations words (all ps .28).

To facilitate comparison with the original study, we re-analyzed our data using the same statistical analyses as Fishbach et al. (2003). Individual reaction times were log-transformed, and outliers that were further than 3 SD away from each condition’s group mean were excluded. The 21 participants who had been excluded from the above preregistered analyses (due to having lower than 80% accuracy on the lexical decision task) were re-included for these analyses.

Following the statistical approach of Study 1 of Fishbach et al. (2003) to test for the asymmetry effect, we calculated mean reaction times for each participant for each condition and conducted a repeated-measures ANOVA. Results were comparable to the pattern described by the earlier trial-level multi-level models; there was a significant interaction between the prime relevancy and the target (F(1, 241) = 4.36, p = .038) but the asymmetry was in the opposite direction to what was reported in Fishbach et al. (2003). A relevant prime always facilitated faster recognition of the target word, and this faciliatory effect was even larger for the goal-priming-temptation condition (M_irrelevant = 594 ms, M_relevant = 582 ms) compared to the temptation-priming goal condition (M_irrelevant = 581 ms, M_relevant = 575 ms).

For the direct replication of Study 3, we computed a difference score for each participant to reflect the asymmetry in priming effects between the temptation-primes-goal condition and the goal-primes-temptation condition. This difference score was not correlated with self-regulatory success (r = −.02, t(240) = −0.33, p = .75, 95% CI [−.15, .11]), failing to replicate Fishbach et al’s Study 3 that found a positive correlation of r = 0.19, p = .05 (one-tailed test). A Fisher r-to-z transformation does not find a significant difference between the two correlations (z = 1.60, p = .06), due to the original correlation having a 95% confidence interval that crossed zero.

In a well-powered replication study, we did not find evidence for an initial, automatic asymmetric priming association between goals and temptations. Instead, words related to study goals (e.g., “homework”) facilitated faster recognition of temptation-related words (e.g., “procrastination”) and temptation-related words facilitated faster recognition of goal-related words (Figure 2)—both priming effects were facilitatory. If anything, goal-words facilitated the recognition of temptations even more so than the reverse, opposite to what the asymmetry hypothesis of counteractive control theory predicts. In short, while we did find a small (yet robust) main effect of prime relevancy, we did not find evidence for the previously reported asymmetry.

The small interaction found instead (Figure 1), unlike the original findings, might be explained via a hierarchical semantic network, where lower-level concepts activate higher-level concepts more readily than higher-level concepts activate lower-level concepts (Anderson, 1983; Collins & Loftus, 1975)—for example, a lower-level concept “poodle” activates the higher-level concept “dog” more than the reverse. Temptation-related concepts, like procrastination and distraction, do not only apply to academic-goals, but also are relevant in other contexts (e.g., one can procrastinate instead of exercising or going to sleep; Bernecker & Job, 2020; Brown, 2019). Distraction-related temptations may thus reflect a higher-level concept, with more disparate connections to multiple lower-level concepts (academics, exercise, and sleep). On the other hand, academic-goal concepts (like “homework”) may be lower in the hierarchical semantic network with fewer connections to other concepts, thus more readily priming the related temptation.

Furthermore, we did not find evidence for moderation by any of four individual difference variables—self-regulatory success, trait self-control, temptation strength, or trait construal level (Table 1). Unlike previous studies, including Study 3 of Fishbach et al. (2003), we did not find the hypothesized priming asymmetry among students with high self-regulatory success, and did not find a correlation between self-regulatory success and the degree of priming asymmetry. In fact, we did not find that any measured individual difference moderated the initial subconscious associations between temptations and goals. While our sample size was larger than prior studies (N = 220; compared to N = 77 in Study 3 and N = 102 in Study 4 of Fishbach et al., 2003 and N = 162 in Fujita & Sasota, 2011), one could argue that our study was underpowered to find moderation by these individual differences, if the moderations are small. However, the original evidence for these moderations is relatively weak, and does not meet current standards for statistical evidence. For example, the moderation by self-regulatory success used one-tailed tests and were not all significant (Fishbach et al., 2003) and a p-curve analysis of Fujita and Sasota (2011) finds inconclusive evidentiary value for moderation by construal level⁵ (Simonsohn et al., 2014). Instead, the asymmetry between temptations and goals may not exist at the most automatic level of processing, even for those individuals high in self-regulatory success.

Of practical interest, we also found consistent results between the sample of participants who had conducted the study in-lab and the participants who conducted the study online. While online research is increasingly common, there is often concern about collecting reaction-time data in-browser and whether resulting data are sufficiently reliable. Our results suggest that reaction time data collected online is comparable in quality to data collected in the lab (see also Reimers & Stewart, 2015; Semmelmann & Weigelt, 2017) with only a difference in the overall reaction times, which does not affect the ability to detect within-subject effects (e.g., priming effects).

These results do not speak to all potential asymmetries between temptations and goals proposed by the asymmetry hypothesis of counteractive control theory. For example, even though goals and temptations may be similarly associated at the automatic cognitive level, the subsequent valuation and devaluation of the goals and the temptations after exposure to each may be asymmetric (Fishbach et al., 2010; Myrseth et al., 2009) or may vary depending on one’s current motivations (Berkman et al., 2017). The hypothesis of an asymmetric association between temptations and goals at the subconscious, automatic level (the level tested here) seems to be the most implausible of the potential mechanisms that could explain asymmetries observed at a behavioural level and counteractive control more broadly.

The current study, replicating the method of Fishbach et al’s Study 3, used a very short stimulus-onset asymmetry (SOA) with 67ms between the prime and the appearance of the target word. This is much shorter than Study 1 of Fishbach et al. (2003), which presented subliminal primes for 50ms followed by a 700ms masked period, and is also not comparable to Fujita and Sasota (2011), which presented supraliminal primes for 300ms followed by a 50ms blank screen (no masking). While faciliatory associations seem to be robust across different timings of the task (de Groot et al., 1986), there is some evidence that inhibitory processes may vary depending on the SOA (Papies et al., 2008). Because inhibitory processes may be slower than initial facilitatory semantic associations, it is possible that the asymmetries in cognitive associations could be found at longer SOAs. Regardless, this study directly contrasts with the theory that these cognitive asymmetries occur automatically, even with minimal time for cognitive processing (Fishbach et al., 2003).

The current study was also conducted with a sample of undergraduate students who were enrolled in a Canadian university. This was an appropriate sample for studying academic goals and was similar to the undergraduate participants in Fishbach et al. (2003), with our sample also rating their academic goals as highly important. While a difference in sample composition cannot easily explain the difference in results between the current study and the original study, neither study used a representative sample. Thus, these results cannot necessarily be generalized to people from other age groups, cultures, or educational backgrounds.

Lastly, the (null) moderations by individual differences of construal level and temptation strength used different methods than earlier research. Both construal level and temptation strength were measured, rather than manipulated as was done in the original studies (Fujita & Sasota, 2011; Kroese, Evers, et al., 2011). We chose to measure these constructs as individual differences rather than include manipulations, in order to not interfere with our replication attempt of the overall temptation-prime association effect. However, the measures used in the current study may not capture the same constructs as were manipulated in the original studies. Additionally, the two-item measure of self-regulatory success had significantly lower internal reliability in our sample compared to in the original Fishbach and colleagues (2003) study. Because this measure has only two items and has not been externally validated, it may not have been a reliable or valid measure of self-regulatory success.

One key finding from counteractive control theory—the idea that exposure to temptations can remind people of their goals, potentially facilitating better goal pursuit—was replicated in the current study. While we did not investigate downstream behavioural decisions, participants recognized goal-related words more quickly when they had been primed with temptation-related words. However, the opposite cognitive association was equally strong, if not stronger—participants recognized temptation-related words more quickly after they had been primed with goal-related words. Our cognitive representations of temptations and goals do indeed facilitate one another; conceptually and semantically, temptations and goals are intricately linked. While later inhibitory processes may create asymmetries in valuation or behaviour, the initial automatic associations between representations of goals and temptations are approximately symmetric.

Contributed to conception and design: ZF, AJ, MI; Contributed to acquisition of data: ZF, AJ; Contributed to analysis and interpretation of data: ZF; Drafted and/or revised the article: ZF, AJ, MI; Approved the submitted version for publication: ZF, AJ, MI.

Thanks to Hause Lin for introducing us to JSPsych and Cognition.run, allowing us to continue to collect data after our original data collection was interrupted. Thank you also to the editor and anonymous reviewers who helped to strengthen the manuscript.

The authors have no competing interests.

Study materials, R analysis code, and data are available at https://osf.io/4vruh/