Revisiting the Temporal Pattern of Regret in Action Versus Inaction: Replication of Gilovich and Medvec (1994) With Extensions Examining Responsibility

The temporal pattern of regret regarding action and inaction was first demonstrated by Gilovich and Medvec (1994) who showed that whereas people tend to experience stronger regret for actions over inactions in the short term, they tend to experience stronger regret for things they did not do over things they did when reflecting back on their lives. A large body of literature has found consistent support for an action-effect, the phenomenon that people associate stronger regret with action compared to inaction (e.g. Gleicher et al., 1990; Kahneman & Tversky, 1982). However, Gilovich and Medvec (1994) suggested that this classic effect is moderated by temporal distance, such that when retrospectively recalling their lifetime and long-term regrets, people tend to associate stronger regret with inaction than with action.

Over the years the literature has seen many mixed findings on temporal patterns in regret (e.g. Bonnefon & Zhang, 2008; Byrne & McEleney, 2000; J. Feldman et al., 1999; Towers et al., 2016), possibly due to differences in methods and scenarios. This suggests the need for revisiting these classic effects with pre-registered replications.

We conducted direct replications and extensions of Studies 1, 3, 4, and 5 in Gilovich and Medvec (1994). Our first goal was to conduct independent direct pre-registered well-powered replications of the temporal pattern of regret. Our second goal was to use the same base methods to extend these findings and examine whether a similar action-inaction pattern of asymmetry would also be found regarding perceptions and experiences of personal responsibility.

We begin by introducing the literature on the action-effect and the temporal action-inaction effect. We then discuss the motivations for the current replication and outline replication hypotheses and designs, with an introduction of our extension to attributions of responsibility.

Kahneman and Tversky (1982) were the first to demonstrate the action-effect, the stronger regret associated with action over inaction, with many successful follow-up demonstrations (e.g. Feeney & Handley, 2006; Gleicher et al., 1990; Landman, 1987). The action-effect has been previously explained using several paradigms, such as the higher perceived causality and responsibility associated with action (Kordes-de Vaal, 1996), and using norm theory suggesting that actions are perceived as an exception to the norm of not acting in such situations, and exceptions are more cognitively mutable than routines and therefore associated with higher regret (Kahneman & Miller, 1986).

The classic experiments by Kahneman and Tversky (1982) demonstrated the action effect by presenting participants with hypothetical short-term decision-making situations. Gilovich and Medvec (1994) tested whether these results would extend to evaluations of real-life long-term experiences. In their Studies 1 and 5, they found that for the retrospective and lifetime reflections the action-effect reversed into an inaction-effect, in that participants tended to report stronger regret for their inaction compared to action. In their Studies 3 and 4 they also demonstrated their findings using scenarios that were very similar to those of Kahneman and Tversky (1982) when manipulating short-term versus long-term reflections.

In follow-up work, Gilovich and Medvec (1995) proposed, investigated, and discussed several possible mechanisms for these effects, including mechanisms related to decrease in intensity of action regrets and increase in intensity of inaction regrets overtime For example, they suggested that people engage in more compensatory behavior for action regrets compared to inaction regrets. Meaning, that people tend to do more to try and rectify their action mistakes compared to inaction mistakes, explaining why inaction regrets may be stronger in the long-run compared to action regrets which may be weakened over time. Another possible explanation is that over time people seem to become more confident that they would have succeeded if they had taken actions (Gilovich et al., 1993), as their memory and/or concerns regarding outcome uncertainties or risks of action diminished over time (Gilovich & Medvec, 1995). Furthermore, Gilovich and Medvec (1995) proposed that intensity of inaction regrets may increase overtime as people perceive inaction mistakes seem to result in more negative consequences, perhaps in a wider range of areas overtime (Rajagopal et al., 2006). Moreover, the debate with Kahneman led to a coauthored adversarial collaboration with three studies, in which Gilovich et al. (1998) concluded that action regrets tend to primarily elicit hot emotions (e.g., anger) whereas inaction regrets tend to elicit feelings of wistfulness (e.g., nostalgia) and despair (e.g., misery), which may be the cause of the temporal differences. A follow-up conceptual replication and extension by Leach and Plaks (2009) found support for the temporal pattern of regret, mediated by the higher level of abstraction of distant inaction regret. We note that we did not set out to investigate the mechanisms of the proposed temporal pattern of regret and to first focus on revisiting and reassessing the core phenomenon.

We chose the Gilovich and Medvec (1994) article based on several factors: the absence of direct replications, its impact, lack of statistical power in empirical evidence, and mixed or null findings.

We chose to replicate Studies 1a, 3, 4, and 5 as these studies focused on the intensity of regret rather than the number of action-inaction regrets, and were a better fit for our target sample, as Study 2 involved face-to-face interviews with several groups of participants. Study 1b examining greatest lifetime regrets overlapped with the more comprehensive Study 5, which manipulated temporal distance, and included questions on lifetime and past week’s greatest regrets. Overall, the target studies for replication covered both scenario experiments (Studies 3 and 4) and surveys regarding real-life experiences (Studies 1 and 5).

To the best of our knowledge, there have been no published direct replications of these target studies. The article has been influential on research in social-cognitive psychology, emotions, and decision-making. At the time of writing (December 2021), there were 549 Google Scholar citations of the article and many important follow-up theoretical and empirical articles (e.g., Bonnefon & Zhang, 2008; J. Feldman et al., 1999; Gilovich & Medvec, 1995; Towers et al., 2016).

The original studies had small sample sizes (under 100 participants for 2-4 conditions), with similar samples in conceptual replications, and revisiting underpowered classics is valuable in addressing possible concerns over false-positive rates (Christley, 2010). In addition, findings were not always consistent with some of the original hypotheses. For example, Study 5 failed to find support for action-effect in the short-run, whereas Study 4 found support for action-effect in the short-run but failed to find support for a meaningful reversal to inaction-effect in the long run.

We aimed to revisit the original findings to try and address mixed findings in follow-up literature identifying boundary conditions, and findings that were not in support of the temporal pattern. Leach and Plaks (2009) conducted a successful conceptual replication of Gilovich and Medvec (1994) using scenario experiments, and found that the level of abstraction mediated the temporal pattern of regret. Furthermore, Zeelenberg et al. (1998) found support for temporal pattern of regret with a series of studies that coded interpersonal regrets in the TV show “I Am Sorry” and real life regrets. Also, Bonnefon and Zhang (2008) asked participants to think of “something you personally regret” (p. 3, one single regret only, did not specify whether the event is the most regretful or not) and found that the difference for short-term regrets was minimal (48% inaction) whereas long-term regrets were more likely to be inactions. In contrast, several follow-up studies examining temporal patterns of the action-effect indicated limited generalizability and identified various possible boundary conditions. Byrne and McEleney (2000) failed to conceptually replicate scenario experiments Studies 3 and 4 in Gilovich and Medvec (1994) adapting Kahneman and Tversky’s (1982) investor scenario. A plausible explanation is that in Byrne and McEleney (2000) scenario experiments, the factual and counterfactual consequences were matched for the actor whereas the counterfactual consequences might be perceived to possibly be better than factual consequences for the non-actor. Byrne and McEleney (2000) argued that the temporal pattern of action-inaction effect only occurs in “situations where the counterfactual consequences of mentally undone inactions are unknown, and possibly better than the factual consequences” (p. 1330). Moreover, Towers et al. (2016) asked participants about their single greatest regret in life and found action regrets were more intense than inaction regrets, contradicting Gilovich and Medvec (1994) findings. Towers et al. (2016) did not directly contrast actions versus inactions, but rather compared intensities of regret coded as action or inaction in reports of regretful events and measured temporal distance continuously but not categorically (lifetime vs recent). Another highly cited article by Feldman et al. (1999) asked participants about personal experiences of regrets and found that participants reported higher numbers of long-term inaction regrets compared to action regrets but failed to find support for difference in intensity of action-inaction regret, which is the focus of our replication (Studies 1, 3, 4 and 5 of Gilovich & Medvec, 1994).

The above studies differ from Gilovich and Medvec (1994) in methods or scenarios, reaching different conclusions. It is unclear if the failure to support Gilovich and Medvec (1994) is due to original results being unreplicable or the differences in methods or scenarios.

Gilovich and Medvec (1994) inspired later work with important possible implications on regret. Later work proposed that inaction regrets may be more distressing and depressing over a longer period (Broomhall et al., 2017; Gilovich & Medvec, 1995), perhaps because negative feelings and senses of disquiet are stronger when one cannot fulfill the need for action (Roese et al., 1999). There appears to be evidence that in the long run, people tend to ruminate over inactions more compared to actions (Gilovich et al., 1995; Savitsky et al., 1997), possibly because they perceive or imagine more possibilities of counterfactual outcomes from inactions compared to actions with clearer links to outcomes (Leach & Plaks, 2009; Rajagopal et al., 2006). However, before getting into mechanisms and practical implications to sort out this literature, we believe it is essential to revisit the classic effects and assess their reliability and replicability (e.g., Brandt et al., 2014; Zwaan et al., 2018), with preregistered high-powered direct replications and extensions.

The original Study 1 was conducted with adult participants on the telephone, asking participants to compare intensity of their action and inaction regrets in general (Study 1A), and compare intensity of their greatest action regret and greatest inaction regret (Study 1B). The original Studies 3 (within-subject) and 4 (temporal distance as the between-subject factor) were conducted with undergraduates and using scenario experiments that asked participants to compare the intensity of regret of the decision-makers in the short run and the long run. The original Study 5 was conducted with participants in public areas, asking participants to compare the intensity of regret of their greatest action regret and greatest inaction regret in the past week and in their lifetime. We did not include Study 1B in our replication, as it consisted of questions on greatest lifetime regret, which overlapped with those of Study 5.

We summarized the hypotheses in Table 1. The original authors hypothesized that there would be stronger regret for inaction in the long run and stronger regret for action in the short run. We provide more details regarding the original article in the supplementary. We calculated Cramer V based on the information provided, reported in Supplementary Table 4.

We aimed to extend the replication to investigate the generalizability of the temporal action-inaction effects in regret findings to responsibility. Regret is associated with evaluations of self-agency and self-blame, key components of responsibility (Frijda et al., 1989; Zeelenberg et al., 2002). Most of the evidence on the regret-responsibility link in the context of action-inaction is based on hypothetical scenario experiments. There are only few real-life experience surveys on regret-responsibility in the action-inaction literature, but there have been some real-life experience successful demonstrations on regret-responsibility link outside the action-inaction literature (e.g. Breugelmans et al., 2014).

We note that regret and responsibility are positively correlated yet separate constructs. Ordónez and Connolly (2000) argued that some people experience some levels of regret over outcomes that they have no agency over (e.g. the outcome was reached by computer reassignment). There are situations in which the decision-maker experiences limited responsibility but stronger regret, such as choosing a lesser-known product brand (Simonson, 1992). Another plausible key difference is that regret tends to be associated with counterfactual thoughts (Huang & Zeelenberg, 2012) compared to responsibility, which is more strongly associated with agency, causality, and morality (Connolly et al., 1997; Kordes-de Vaal, 1996). There may be discrepancies in action-inaction regret and responsibility findings, yet there are several studies reporting a positive regret-responsibility link (e.g. Ordónez & Connolly, 2000; Zeelenberg et al., 2000, 2002). To the best of our knowledge, there are no studies that compared long-term feelings of responsibility regarding action vs inaction. We expected findings for regret to extend similarly to responsibility. See Table 2 for the extension hypotheses.

We report the determination of sample size, all data exclusions, all manipulations, and all measures in our studies (Simmons et al., 2012). This manuscript is in line with Appelbaum et al. (2018) Journal Article Reporting Standards (JARS) and Transparency and Openness Promotion (TOP) Guidelines (Nosek et al., 2015). We preregistered designs and analysis plans of all studies before data collection. Pre-registration, all data, code, and materials are available on the Open Science Framework (OSF): https://osf.io/342td/ and https://osf.io/7m3q2/. Open-science details, disclosures, original effects calculations, power analyses, and pre-exclusion results are provided in the supplementary. We analyzed data using RMarkdown (Xie et al., 2018, see RMarkdown output in site for analyses with the list of packages) with RStudio version 1.3.1073 (RStudio Team, 2021) and produced plots with the package ggplot2 version 3.3.3 (Wickham, 2016).

We recruited US-American participants from Amazon Mechanical Turk (MTurk) with TurkPrime.com/CloudResearch (Litman et al., 2017). Based on our extensive experience of running similar replications on MTurk, to ensure high-quality data collection, we employed the following CloudResearch options: Duplicate IP Block. Duplicate Geocode Block, Suspicious Geocode Block, Verify Worker Country Location, Enhanced Privacy, CloudResearch Approved Participants, Block Low Quality Participants, etc. We also employed the Qualtrics fraud and spam prevention measures: reCAPTCHA, prevent multiple submission, prevent ballot stuffing, bot detection, security scan monitor, and relevant ID. MTurk has been shown to be a reliable platform for conducting studies in social psychology, judgment, and decision-making (Anderson et al., 2019; Buhrmester et al., 2011; Thomas & Clifford, 2017). Several recent studies on the action-effect (e.g. G. Feldman, 2020; G. Feldman & Albarracín, 2017) and a recent large-scale collaborative project with over 80 replications of judgment and decision-making phenomena has shown MTurk, with TurkPrime.com/CloudResearch to be a highly suitable platform for this research design (Collaborative Open-science Research, 2022). Recently, Eyal et al. (2021) compared levels of attention, comprehension, and dishonesty of participants between several platforms and panels and found that CloudResearch and Prolific provided higher quality compared to other methods (Qualtrics, MTurk without CloudResearch, Dynata).

A total of 1017¹ participants completed the study. We excluded 29 participants based on our pre-registered exclusion criteria (see supplementary for details), resulting in a total sample of 988 participants (M_age = 43.94, SD = 13.62; 566 females, 408 males, 8 others, 6 prefer not disclosing their gender). We report full results comparing pre-exclusions versus post-exclusions in the supplementary. We provide a comparison of the target article samples and the replication samples in Table 3.

To estimate the required sample size, we used pwr package version 1.3 (Champely et al., 2018) and conducted an a-priori power analysis for chi-square goodness of fit 50-50 tests, comparing the proportion of action with stronger regret versus inaction with stronger regret, and chi-square tests of association, testing the association between temporal distance and action-inaction regret. We calculated and reported the original effect sizes in Supplementary Table 4. Aiming for a statistical power of 95% with an alpha of .05, and based on the weakest meaningful effect (V = 0.24) that the original authors hypothesized and claimed to find support for, with more participants perceiving stronger regret for inaction than for action in the long-term in Study 4 (but p > .05), the required sample was 920 participants. As we expected some participants to be excluded, we aimed for 1000 participants. We provide more details in the supplementary.

We made adjustments to the design of the original studies. Extending the original studies and deviating from their procedures, we combined the replications of Studies 1A, 3, 4, and 5 into a singular design in one data collection, with added extensions examining responsibility. First, participants read the consent form. We first presented Study 5, followed by Study 1. Both were personal experience studies. We then randomized participants into either Study 3, a within-subject design, or Study 4, a between-subject design, as Studies 3 and 4 consisted of the same hypothetical scenario and questions. We placed Study 3/Study 4 at the end to prevent the stimuli in the scenario from affecting personal responses. We then presented participants with funneling and demographic questions, followed by a debriefing statement. See below sections for more specific and detailed information about all studies. We note that the study numbers below are based on the study numbers of the original article, but not the order of our replications.

We decided on this design in order to address possible concerns regarding the sample. Despite our ample experience and accumulated evidence in support of validity of our chosen MTurk/CloudResearch sample for replications of classics in judgment and decision-making, reviewers often expressed concerns about online samples regarding inattentiveness, suitability to context (time, setting, etc.), and overall data quality. When some of the findings replicate and others do not, combining the studies allows ruling out inattentiveness as a concern, adequacy of the target sample for these replications, or the adjustments to updated context (time, setting, etc.), so that we can instead focus on the implications regarding specific designs and found effects. This design has been tested and shown to be successful in several recent replications (Adelina & Feldman, 2021; Chandrashekar et al., 2021; Chen et al., 2021; Ziano et al., 2021).

We provided detailed information of designs (type of study, sample, variables, exact wordings) of the original studies in the Methods and Analyses of the original article section of the Supplementary. We note several deviations from the original, summarized in Table 3 and Table 4. We combined studies into a single survey, and we added responsibility questions as extensions. Finally, we recruited participants through MTurk online, instead of participants from New York and Chicago, or Cornell University students.

We summarized Studies 1, 3, and 4 as “very close replications” and Study 5 as a “close replication” based on the criteria by LeBel et al. (2018) (see Figure S3 in the Supplementary), with our classification analysis provided in Table 4.

First, in Study 5, participants answered questions regarding their greatest regrets. Temporal distance (the past week vs entire life, in counterbalanced order) was the independent variable. We asked participants to think about and describe their greatest lifetime action-regret and greatest lifetime inaction-regret, as well as greatest past week action-regret and greatest past week inaction-regret. We then asked them which they regretted more. In the original study, participants were only required to think about but not describe their regrets. However, in our replication we asked participants to briefly write about their regrets. By having participants briefly describe their regrets, we felt they would be more likely to engage in effortful reflections, and less likely to respond randomly, thereby ensuring better data quality. We reminded participants that they do not need to disclose any information they feel uncomfortable sharing. We also went to great lengths to align expectations about the task – we made it clear in our study recruitment and with a specific question in the consent screen that the task involves brief writing and that the study is about life regrets (see “Study recruitment” and “Writing task expectation alignment in consent” in the supplementary).

We randomized participants to either answer the replication questions or extension questions. In the extension condition, we asked participants about the action decision that they felt most personally responsible for and the inaction decision that they felt most personally responsible for, in the past week and in their lifetime (in counterbalanced order). Similarly, we asked participants to very briefly describe these events. They then answered which of those they felt more responsible for. Full details are provided in Table S5 in the supplementary.

For past week regrets, with the chi-square goodness-of-fit test, we failed to find support for a deviation from a 50-50 split in participants experiencing stronger regret for the action (52.34%) than inaction (47.66%), z = 1.04, χ2 (1, N = 535) = 1.17, p = .280, V = 0.05, 95% CI [0.00, 0.13] (see Figure 1 top left plot).

For lifetime regrets, with the chi-square goodness-of-fit test, we failed to find support for a deviation from 50-50 split in participants experiencing stronger regret for the action (52.15%) than inaction (47.85%), z = 0.95, χ2 (1, N = 535) = 0.99, p = .320, V = 0.04, 95% CI [0.00, 0.13] (see Figure 1 top right plot).

Comparing the proportion of participants experiencing stronger action regret in the past week and experiencing stronger inaction regret in lifetime (133/265, 265 is the total number of participants who showed reversal, 50.19%), versus the proportion of participants choosing inaction in the past week and choosing action in lifetime (132/265, 49.81%), we failed to find support for a deviation from 50-50 distribution, z = 0.00, χ2 (1, N = 265) = 0.00, p = .951, V = 0.00, 95% CI [0.00, 0.14]. We also conducted a McNemar test, and failed to find support for the association between temporal distance and inaction-action regret, OR = 0.99, 95% CI [0.77, 1.27], p = 1.

For the responsibility over the week, with the chi-square goodness-of-fit test, we found support for a deviation from 50-50 split in participants experiencing stronger responsibility for the action (56.29%) versus participants experiencing stronger responsibility for the inaction (43.71%), z = 2.63, χ2 (1, N = 453) = 7.17, p = .007, V = 0.13, 95% CI [0.03, 0.22]. More participants felt stronger responsibility for action compared to inaction (see Figure 1 bottom left plot).

For the responsibility over the lifetime, with the chi-square goodness-of-fit test, we found support for a deviation from 50-50 split in participants experiencing stronger responsibility for the action (56.51%) versus participants experiencing stronger responsibility for the inaction (43.49%), z = 2.73, χ2 (1, N = 453) = 7.68, p = .006, V = 0.13, 95% CI [0.03, 0.22]. More participants felt stronger responsibility for action compared to inaction (see Figure 1 bottom right plot).

Comparing the proportion of participants experiencing stronger action responsibility in the past week and experiencing stronger inaction responsibility in lifetime (100/201, 49.75%), versus the proportion of participants choosing inaction in the past week and choosing action in lifetime (101/201, 50.25%), we failed to find support for a deviation from 50-50, z = 0.00, χ2 (1, N = 201) = 0.00, p = .944, V = 0.00, 95% CI [0.00, 0.16]. We also conducted a McNemar test, and failed to find support for the association between temporal distance and action-inaction responsibility, OR = 1.01, 95% CI [0.76, 1.35], p = 1.

After Study 5, we presented Study 1. We asked participants about their action and inaction regretful experiences - “When you look back on your experiences in life and think of those things that you regret, what would you say you regret more, those things that you did but wish you hadn’t, or those things that you didn’t do but wish you had?” (Gilovich & Medvec, 1994, p. 358). We asked another group of participants regarding their felt responsibility for life’s personally responsible actions and inactions - “When you look back on your experiences in life and think of those things that you feel personally responsible for, what would you say you feel personally responsible more, those things that you did but wish you hadn’t, or those things that you didn’t do but wish you had?”.

We began by examining regret, we conducted a chi-square goodness of fit test against a 50-50 action-inaction split , z = -5.79, χ2 (1, N = 535) = 34.07, p < .001, V = 0.25, 95% CI [0.17, 0.34], and found support for stronger regret for inactions; more participants reported experiencing stronger regret over lifelong inactions (62.62%) than those reporting experiencing stronger regret over lifelong actions (37.38%) (see Figure 2 left plot).

To examine our responsibility extension, we conducted a chi-square goodness of fit test against a 50-50 action-inaction split, z = 4.89, χ2 (1, N = 453) = 24.34, p < .001, V = 0.23, 95% CI [0.14, 0.32], and found that more participants reported stronger responsibility over lifelong regrettable actions (61.59%) than those reporting stronger responsibility for lifelong regrettable inactions (38.41%) (see Figure 2 right plot).

After completing Studies 5 and 1a, we presented participants with scenario experiments on college decisions: Inaction Dave stayed in the same college whereas Action Jim switched to another college, and both were unsatisfied. We randomized participants into either a within-subject design as in Study 3 (participants compared feelings of regret and responsibility of Dave vs. Jim in both short-term and long-term), or a between-subject design as in Study 4 (participants answered short-term or long-term questions). In Studies 3 and 4, participants answered both replication and extension questions. We provide more details on the designs of Studies 3 and 4 in the supplementary’s Tables 7 and 8).

In the short-term condition of the between-subject design study, we conducted a chi-square goodness of fit test and found support for more participants perceiving stronger regret for action Jim (60.65%) than for inaction Dave (39.35%), z = 3.69, χ2 (1, N = 247) = 14.09, p < .001, V = 0.24, 95% CI [0.12, 0.36] (see Figure 3 top left for the plot).

In the long-term condition, we conducted a chi-square goodness of fit test and found that more participants perceiving stronger regret for inaction Dave (57.61%) than for action Jim (42.39%), yet this did not meet our pre-defined alpha leading us to conclude no support, z = -1.59, χ2 (1, N = 248) = 2.73, p = .099, V = 0.10, 95% CI [0.01, 0.23] (see Figure 3 top right for the plot).

We conducted a chi-square test of independence and found support for the association between temporal distance and action-inaction regret, χ2 (1, N = 495) = 14.68, p < .001, V = 0.17, 95% CI [0.08, 0.26]. We found that compared to short-term, long-term perspective was associated with stronger perceived regret for inaction.

Examining responsibility in the short-term condition, we conducted a chi-square goodness of fit test and found support for more participants perceiving stronger responsibility for action Jim (61.94%) than for inaction Dave (38.06%), z = 7.76, χ2 (1, N = 247) = 61.25, p < .001, V = 0.50, 95% CI [0.39, 0.60] (see Figure 3 bottom left for the plot).

Examining responsibility in the long-term condition, we conducted a chi-square goodness of fit test and found support for more participants perceiving stronger responsibility for action Jim (74.90%) than for inaction Dave (25.10%), z = 5.78, χ2 (1, N = 248) = 34.13, p < .001, V = 0.37, 95% CI [0.25, 0.49] (see Figure 3 bottom right for the plot).

We conducted a chi-square test of independence and failed to find support for the association between temporal distance and action-inaction responsibility, χ2 (1, N = 495) = 2.46, p = .117, V = 0.07, 95% CI [0.00, 0.16].

In the short-term condition of the within-subject design study, we conducted a chi-square goodness of fit test (meant to mirror the analyses for Study 4 to allow for a comparison) and found support for more participants perceiving stronger regret for action Jim (61.26%) than for inaction Dave (38.74%), z = 4.95, χ2 (1, N = 493) = 24.99, p < .001, V = 0.23, 95% CI [0.14, 0.31] (see Figure 4 top left plot).

In the long-term condition, we conducted a chi-square goodness of fit test and found support for more participants perceiving stronger regret for inaction Dave (57.61%) than for action Jim (42.39%), z = -3.33, χ2 (1, N = 493) = 11.41, p < .001, V = 0.15, 95% CI [0.06, 0.24] (see Figure 4 top right plot).

Comparing the proportion of participants choosing action Jim in the short term and choosing inaction Dave in the long term (113/133, the total number of participants who showed reversal in answers, 84.96%), versus the proportion of participants choosing inaction Dave in the short term and choosing action Jim in the long term (20/133, 15.04%), we found support for a deviation from 50-50, z = 7.98, χ2 (1, N = 133) = 65.03, p < .001, V = 0.70, 95% CI [0.56, 0.80]. More participants chose action in the short term and inaction in the long term, compared to inaction in the short term and action in the long term. We also conducted a McNemar test, and found support for the association between temporal distance and action-inaction regret, OR = 0.18, 95% CI [0.10, 0.29], p < .001.

Examining responsibility in the short-term condition, we conducted a chi-square goodness of fit test and found support for more participants perceiving stronger responsibility for action Jim (73.83%) than for inaction Dave (26.17%), z = 10.54, χ2 (1, N = 493) = 112.02, p < .001, V = 0.48, 95% CI [0.40, 0.55] (see Figure 4 bottom left plot).

Examining responsibility in the long-term condition, we conducted a chi-square goodness of fit test and found support for more participants perceiving stronger responsibility for action Jim (60.65%) than for inaction Dave (39.35%) , z = 4.68, χ2 (1, N = 493) = 22.36, p < .001, V = 0.21, 95% CI [0.13, 0.29] (see Figure 4 bottom right plot).

Comparing the proportion of participants choosing action Jim in the short term and choosing inaction Dave in the long term (94/123, 76.42%), versus the proportion of participants choosing inaction Dave in the short term and choosing action Jim in the long term (29/123, 23.58%), we found support for a deviation from 50-50, z = 5.77, χ2 (1, N = 123) = 34.35, p < .001, V = 0.53, 95% CI [0.37, 0.67]. More participants chose stronger responsibility for action in the short term and for inaction in the long term, compared to stronger responsibility for inaction in the short term and for action in the long term. We also conducted a McNemar test, and found support for the association between temporal distance and action-inaction responsibility, OR = 0.31, 95% CI [0.20, 0.47], p < .001. Temporal distance had an impact on the choice distribution. The difference in the proportion between action and inaction was weaker in the long-term compared to that in the short-term. The effect was in the same direction as that in regret, but did not lead to a complete reversal of perceptions.

We summarized descriptive statistics of all studies in Table 5. We summarized the comparison of all effects of original studies versus our replications in Table 6, with an interpretation of the results based on the LeBel et al. (2019) replication results evaluation criteria.

We successfully replicated and found support for the original findings in Studies 1, 3, and 4 (short-term regret and temporal effect) with smaller effect sizes. For our replication of Study 4’s long-term regret, the CIs of the replication did not cover the original effect size, and we failed to find support for the effect (which was the case in both the original and the replication).

We failed to successfully replicate Study 5. We failed to find support for an action-effect in the short-term, and failed to find support for an inaction-effect in the long-term.

Regarding our responsibility extensions, we conclude stronger responsibility for action over inaction across all studies, for both short-term and long-term.

We conducted a pre-registered replication of the temporal pattern of action-effect by Gilovich and Medvec (1994), with a more diverse (Buhrmester et al., 2011) and high-powered sample. We successfully replicated Study 1, which focused on general regrets, as well as Studies 3 and 4, which were scenario studies asking participants to compare regret for action versus inaction. More participants reported stronger regret for action in the short-run, but stronger regret for inaction in the long run.

In Study 5, we failed to find support for an action-effect in the short-term, failed to replicate findings for the long-term, and the proposed association between temporal distance and action-effect.

Why did the replication of Study 5 fail while the replication of the other studies succeeded?

Previous studies have shown MTurk to be a reliable platform for the study of action and inaction, and judgment and decision-making more broadly (e.g. G. Feldman, 2020; G. Feldman & Albarracín, 2017). Our design and the other successful replications of Studies 1, 3, and 4 address concerns regarding sample characteristics or time, given that they were conducted using the same sample. Therefore, we believe the more plausible explanations are the differences in methods and the likelihood of a false positive. We note that the sample size in the original Study 5 was 32, with a much higher likelihood of a false-positive. Our sample was substantially larger and well-powered, yet we were unable to detect the inaction-effect in the long run.

Why would the method used in Study 5 result in different findings? We believe this might have to do with the ways action and inaction are conceptualized in the different studies. In Study 5 action refers to “something they did” versus inaction as “something they did not do”. In Study 3 and Study 4, action is conceptualized as a switching behavior, a change to the status quo, versus inaction, which is sticking with the status quo. Unfortunately, these issues seem to be widespread in this literature, with recent reviews alerting that action and inaction are often ill-defined terms and in urgent need of clarifications (G. Feldman et al., 2021).

Our findings are consistent with a large body of literature showing support for an action effect in high-risk recent situations that result in negative outcomes. The typical action-effect scenarios refer to changing, switching, or deviating from a set reference point (e.g., past behavior, status quo) (e.g. G. Feldman & Albarracín, 2017; Gleicher et al., 1990; Landman, 1987), with norm theory (Kahneman & Miller, 1986) arguing that is likely due to action being perceived as more abnormal than inaction (G. Feldman, 2020), and that exceptionality tends to elicit higher regret than normality (Kutscher & Feldman, 2019). The meaning of action and inaction in Studies 3 and 4 was closer to the typical action-effect and norm theory scenarios (Kahneman & Miller, 1986). However, in Study 5, the meaning of action was far broader with no clear reference to a norm or a reference point. Therefore, the differences between action as in doing and inaction as in not doing seem less clear and with no clear indication of what to compare to.

Beyond differences in meaning, another possible explanation is regarding the differences between perceptions of others’ emotions in Studies 3 and 4 and the evaluation of personal actual experiences of emotions elicited in Study 5. Perceptions of emotions in others tend to be less accurate and differ from actual personal experiences, especially if there is no personal relevance to the situation evaluated in the presented scenario. The failed replication of Study 5 seems consistent with most studies that directly ask participants about their personal experiences (Bonnefon & Zhang, 2008; J. Feldman et al., 1999,²; Towers et al., 2016), but inconsistent with Zeelenberg et al. (1998) studies, in which they found support for temporal pattern of regret in real life experiences for interpersonal regrets (they did not test other kinds of regrets).

Another possible explanation for our different findings in Study 5 is a change we made to the original study’s design. We required participants to describe their regrets, whereas Gilovich and Medvec (1994) only required participants to recall their regrets without writing those down. We, however, find this explanation unlikely. The target’s concern was that participants may be unwilling to describe their very embarrassing and shameful regrets, yet taking a closer look at the responses (found in our dataset), we found that many participants described highly personal, shameful, and somewhat tragic events, including events involving death, major career failures, major educational failures, major relationship failures, etc. However, we cannot completely rule out the possibility of such differences having an impact on the findings, and future research can further test this possibility by asking half of the participants to describe a major regret and asking other participants to simply think about a major regret.

Another possible explanation raised in the peer-review was that Study 5 may not have been suitable for our online MTurk target sample, given concerns of attentiveness and seriousness. As we discussed earlier, MTurk with CloudResearch/TurkPrime provides high-quality responses, in which participants are as if not more attentive than on other platforms (Eyal et al., 2021). The successful replications of the other studies in our unified design address concerns of attentiveness. We also addressed this concern by conducting additional exploratory analyses (see the Exploratory Analyses of Study 5 section in the Supplementary) excluding responses in which participants seem to have misunderstood or confused action and inaction, or participants reporting lifetime mistakes in past week mistakes questions (or vice versa), and non-regret/non-responsibility responses. We found that only a very low % of participants (ranging from 1.68% for lifetime regret, to 7.28% for past week responsibility) misclassified responses. Our results of the exploratory analyses with those participants excluded were consistent with results we reported above.

Studies 3 and 4 were also better controlled with specific scenarios, whereas Study 5 did not restrict the range of elicited regrets, which included many different domains in life, such as education, work, relationships, finance, etc.

What is puzzling about the above explanations is that although we failed to find support for lifetime recalls in Study 5, we did find support for the effects in Study 1. Both Study 1 and Study 5 asked participants about “things that they did” versus “things that they didn’t do”. Yet, one key difference was that Study 1 elicited general regrets and asked participants to evaluate those together rather than contrasting one most regrettable action against one most regrettable inaction. Therefore, it is possible that specificity is a moderating factor of the effect. The autobiographical memory framework by Davison and Feeney (2008) suggested that regret is about remembering past events with different levels of specificity and generality. They found that general regrets were more likely to be for inactions over action, yet specific regrets were more likely to be for actions than inactions.

The above proposed reasons for the discrepancies between studies in the same article are speculative, and we conclude that more work is needed to examine any of the proposed moderators with direct testing.

Regret and responsibility are often positively related (e.g. Zeelenberg et al., 2000, 2002), -yet are distinct constructs. We found consistent support for stronger responsibility for action compared to inaction for the recent past, and the finding for responsibility in the recent past generally aligned with that of regret. However, while there was some support for differences between recent past and distant past for responsibility, the effect seems much weaker than for regret, and we did not find a full reversal toward stronger responsibility for inaction in the distant past. Instead, consistently across different designs, we also found support for stronger responsibility for action for the distant past events. Responsibility seems to be more strongly associated with morality, causality, and agency (Connolly et al., 1997). Changes in perceived responsibility intensity perhaps fluctuate less over time compared to regret, which is a counterfactual emotion that may fluctuate over time more as people may feel more confident that they would have made it if they had taken actions, thereby regretting inaction more in the long-run (Gilovich et al., 1993; Gilovich & Medvec, 1995). That said, we note this explanation is speculative and more work comparing mechanisms of regret and responsibility in action-inaction studies is needed.

These findings may hold important implications for the link between the action-effect and omission-bias. Omission bias extends the action-effect to reflect action-inaction asymmetries regarding responsibility and blame. More work is needed on the potential moderating effect of time with regret and responsibility examined together for both action-effect and omission-bias scenarios.

Our findings for responsibility in Study 3’s within-subject design and Study 4’s between-subject design were slightly different, with stronger effects for the within-subject design. There are quite a few judgment and decision-making effects that are stronger with within-subject design compared to between-subject design (Charness et al., 2012), and more work is needed to contrast the two regarding action-effect and omission-bias.

We faced some challenges with the studies eliciting life events. A small number of participants (1.68% to 7.28%) wrote inaction events in the action description box or vice versa, with some participants reporting they did not experience any action or inaction regret or responsibility in the past week. In an online study we cannot rule out the possibility that some participants may have copy-pasted, and based on our experience there are indications that MTurk participants typically dislike writing tasks. However, we tend to think that the likelihood of this being an issue in our design is very low, as we adopted numerous quality control methods, aligned expectations in advance about the task, and only asked for brief descriptions in one or two sentences. We checked all responses, and found that most responses were of high-quality. We also conducted exploratory non-pre-registered analyses excluding possibly irrelevant and incorrect responses (which were only a very small proportion of the entire sample) and the results (reported in the supplementary) were very similar to the results after pre-registered exclusion or full results. Therefore, we believe it is less likely that the null findings in Study 5 are due to this issue and find it more likely that such null findings are due to differences in meanings of action-inaction. To resolve the discrepancy in findings between Study 5 and Studies 1, 3, and 4, future studies can make adaptations to Study 5 by manipulating the definitions of action and inaction (G. Feldman et al., 2021). Also, we cannot completely rule out the possibility that our adjustments of adding brief writing to the Study 5 recall task may have impacted the results, and so future studies may also compare findings of recall tasks that involve versus do not involve writing the recalled memory, as well as test this phenomenon with other non-MTurk/CloudResearch samples to investigate if there are meaningful differences.

We note that a single replication of a single article is insufficient to answer all the questions in the literature with high certainty, and we call for more well-powered pre-registered replications of work in this domain, preferably by third-parties and in the form of Registered Reports.

We reported aggregated tendencies regarding temporal effects related to action and inaction yet there are individual differences factors that may play a role in moderating these effects (e.g. action-state orientation, Diefendorff et al., 2000; regulatory focus, Itzkin et al., 2016).

Many of the findings in this literature were conducted in mostly WEIRD settings (Western, Educated, Industrialized, Rich and Democratic; Henrich et al., 2010), and more research is needed to study these effects in less WEIRD regions, and/or include cultural dimensions as potential moderators of these effects.

In our extension, we found differences in findings regarding regret and responsibility. Studies in the action-inaction literature rarely measure regret and responsibility together, and more work is needed to investigate the associations between the two constructs in the context of action-inaction effects.

We believe that more replications with extensions are needed to better understand the robustness of the findings in this literature and examine new directions, together with meta-analyses of the action-inaction related literature (e.g., action-effect: Yeung & Feldman, 2022; omission bias: Yeung et al., 2022), to examine possible moderating factors such as temporal distance, scenarios versus experience, between-subject versus within-subject study design comparison, and the used meanings of action versus inaction. We require a more comprehensive systematic aggregation of findings and insights to identify boundary conditions.

We conducted a replication and extension of Gilovich and Medvec (1994) revisiting the temporal pattern of regret in action versus inaction and adding extensions examining temporal pattern of responsibility. We found support for the original findings on regret with different designs both examining lifelong experiences in Study 1 and hypothetical scenarios in Studies 3 and 4, though with weaker effects. However, we failed to find support for such a pattern in Study 5, and we discussed possible explanations. We called for better conceptualizations of the terms action and inaction in the literature, with more replications and extensions of classic studies of the action-inaction literature and follow-up meta-analyses to help resolve inconsistencies in findings. We also reported the findings of an extension examining temporal pattern regarding attributions of responsibility, and discussed the regret-responsibility link.

We would like to thank Qinyu Xiao, Adrien Fillon, Dr. Frances Jin. and Prof. Kin Fai Ellick Wong for their helpful comments. We thank the original author Thomas Gilovich for providing the original materials.

Kit wrote the pre-registrations, conducted data analyses, and wrote the manuscript, as part of his mPhil thesis. Gilad Feldman supervised Siu Kit Yeung throughout, conducted the pre-registrations, and ran data collection. Gilad and Kit jointly finalized the manuscript for journal submission.

The author(s) declared no potential conflicts of interest with respect to the authorship and/or publication of this article.

The author(s) received no financial support for the research and/or authorship of this manuscript.

Pre-registration, materials, data, and code are publicly available on: https://osf.io/7m3q2/