We report seven experiments to investigate the effects of control threat manipulations on different measures of illusory pattern perception: magical thinking (Study 1–3), conspiracy beliefs (Study 4), paranormal beliefs (Study 5) and agent detection (Study 6 and 7). Overall we did not find evidence for an effect of control threat on any of our relevant dependent measures. By using Bayesian analyses we obtained positive evidence for the null-hypothesis that an experimentally induced loss of control does not affect illusory pattern perception. Finally, by re-conducting a recent meta-analysis we found strong evidence for publication bias and a relatively small effect size for control-threat manipulations. Together, these results cast doubt on the potential efficacy of experimental autobiographical recall manipulations to manipulate feelings of control.
1. Introduction
Most people are familiar with the feeling of losing control. Some people experience a loss of control when they miss their connecting flight, some after realizing they are going to fail an exam, and others after a loved one ends their relationship. In these situations, people have the feeling that they are unable to control the world around them. Because such experiences can be psychologically stressful, it has been suggested that people have a motivated tendency to counter such feelings through different psychological mechanisms (Landau, Kay, & Whitson, 2015). For instance, people may tend to overestimate the extent to which they can control their lives or their environment (Presson & Benassi, 1996; Whalen, 1998), as becomes evident in research on positive illusions and magical thinking (Taylor & Brown, 1988; van Elk, Rutjens, & van der Pligt, 2015). Furthermore, people who lack control may try to restore their implicit sense of control through attributional biases (Pittman & Pittman, 1979). One important mechanism that has been proposed as a way to cope with uncertainty and lack of control consists of adjusting the perception of the external world – a phenomenon that has been labeled compensatory control (Kay, Gaucher, McGregor, & Nash, 2010; Kay, Shepherd, Blatz, Chua, & Galinsky, 2010; Kay, Whitson, Gaucher, & Galinsky, 2009; Landau et al., 2015).
According to the compensatory control framework, people have a basic need for feeling in control and for living in an orderly and predictable world. When this basic need is violated compensatory strategies come into play: a lack of personal control triggers the search for external systems providing a sense of control (Landau et al., 2015). In other words, when people are deprived of control, they tend to behave in ways aimed to restore their feelings of control and to increase epistemic predictability (i.e., the feeling that one lives in a world that is coherent and predictable). Over the past decade, researchers have started to experimentally manipulate feelings of control by using different experimental paradigms. For instance, researchers have used an autobiographical memory manipulation, by asking participants to recall and describe specific life episodes where they were lacking control (Kay, Gaucher, Napier, Callan, & Laurin, 2008; Rutjens, van der Pligt, & van Harreveld, 2010); participants have been shown pictures related to a loss of control (de Arcos, Verdejo-Garcia, Peralta-Ramirez, Sanchez-Barrera, & Perez-Garcia, 2005); participants have been instructed to complete unsolvable tasks (Reed, Frasquillo, Colkin, Liemann, & Colbert, 2001); or participants performed a task in which they did not have control over actions they were asked to perform (Maier & Seligman, 1976). By using these manipulations it has been found that control threat results in a stronger endorsement of spiritual beliefs (Kay et al., 2009), a stronger belief in paranormal abilities (Greenaway, Louis, & Hornsey, 2013), an enhanced tendency to detect illusory patterns in noise (Whitson & Galinsky, 2008), a stronger attachment towards in-groups (Agroskin & Jonas, 2014; Fritsche et al., 2013; Fritsche, Jonas, & Fankhanel, 2008), a stronger preference for governmental control (Kay et al., 2008), more belief in science (Rutjens et al., 2010), and more prejudice toward outgroup members (Greenaway, Louis, Hornsey, & Jones, 2014). Thus, the empirical evidence for the notion that lack of control triggers compensatory strategies for restoring subjective feelings of control and predictability is quite impressive (for recent review, see: Landau et al., 2015).
In this paper we report 7 experiments that we conducted over the past five years in our lab, aimed at providing a conceptual replication of previous studies, by using similar independent manipulations of control threat (e.g., autobiographical memory task; unsolvable anagram task; cf. Kay et al., 2008; Muraven, Tice, & Baumeister, 1998; Whitson & Galinsky, 2008). We aimed to validate the prediction that a lack of control should result in a motivated tendency to perceive illusory patterns. In this paper we used ‘illusory pattern perception’ as an umbrella term to denote related phenomena, such as ‘magical/optimistic thinking’ (i.e., overestimating the contingency between two events), ‘belief in conspiracy and paranormal beliefs’ (i.e., over-inferring the relation between two unrelated phenomena) and ‘agent detection’ (i.e., inferring the presence of a human-like agent). Of course we are well aware of the fact that conceptual replications can only be considered complementary compared to pre-registered direct replications (for discussion, see: Stroebe & Strack, 2014). Still, one could argue that the compensatory control framework provides a general principle that should lead to novel testable predictions, rather than being limited to a specific set of circumscribed studies. In fact, for each of the studies that we conducted, based on the existing literature we predicted a priori that our control threat manipulation would result in an effect (i.e., reflected in stronger magical thinking, stronger belief in conspiracy beliefs and Psi, and enhanced pattern perception). By opening up our file-drawer we propose that these studies should be taken into account in future research to question the validity of using autobiographical recall tasks to experimentally manipulate feelings of control (i.e., potentially, lab-based procedures for manipulating feelings of control do not yield the expected effects). For each study we also report additional measures that were included as potential moderators of our experimental manipulation and we explicitly state whether the study was conducted as part of a bigger research project and whether additional studies were conducted simultaneously. The different scales and the data from all experiments are included in the Supplementary Online Material. In this manuscript we focus instead on our main manipulation of control-threat on the different dependent measures, as this was our primary effect of interest.
2. Overview of Experiments and Approach
All our studies share the same experimental rationale and approach: we manipulated participants’ feelings of control and we investigated the effects on compensatory control strategies (see Table 1). To manipulate feelings of control we used an autobiographical manipulation (in Study 1–4 and Study 7) in which participants were required to think back about a situation in which they lacked control (compared to a negative situation in which they were in control; Kay et al., 2008; Rutjens et al., 2010). We also manipulated control by asking participants to work on an unsolvable anagram task (Study 5) and by presenting participants with short vignettes describing situations associated with low feelings of control (Study 6). We investigated the effects of our control threat manipulations on magical thinking (Study 1–3), which was operationalized as the perception of an illusory correlation between two objectively unrelated events (van Elk, Rutjens, et al., 2015). We also investigated the effects on general belief in conspiracy theories and in paranormal beliefs as a source of compensatory control (Study 4 and 5). Finally, we tested the effects of control threat on illusory agent detection, by focusing on the number of false positive responses in a perceptual decision making task. For each study we predicted an effect of control threat on the dependent measure.
Overview of the different experiments, the independent manipulations and the dependent measures that were used.
Study . | Manipulation . | Dependent Measure . |
---|---|---|
1 | Memory Recall task Between-subjects | Contingency Learning Task (Medicine – Outcome) |
2 | Memory Recall task Between-subjects | Contingency Learning Task (Medicine – Outcome) |
3 | Memory Recall task Between-subjects | Illusory Contingency Task (Mouse-trap Paradigm) |
4 | Memory Recall task Between-subjects | Conspiracy Belief Questionnaire |
5 | Unsolvable Anagram task Between-subjects | Belief in Precognition |
6 | Reading vignettes Within-subjects | Illusory Pattern Perception |
7 | Memory Recall task Within-subjects | Biological Motion Detection Task |
Study . | Manipulation . | Dependent Measure . |
---|---|---|
1 | Memory Recall task Between-subjects | Contingency Learning Task (Medicine – Outcome) |
2 | Memory Recall task Between-subjects | Contingency Learning Task (Medicine – Outcome) |
3 | Memory Recall task Between-subjects | Illusory Contingency Task (Mouse-trap Paradigm) |
4 | Memory Recall task Between-subjects | Conspiracy Belief Questionnaire |
5 | Unsolvable Anagram task Between-subjects | Belief in Precognition |
6 | Reading vignettes Within-subjects | Illusory Pattern Perception |
7 | Memory Recall task Within-subjects | Biological Motion Detection Task |
All studies were approved by the local ethics committee at the Psychology Department at the University of Amsterdam. Participants provided written informed consent before the start of each experiment. Only Experiment 5 was conducted as an online study and for this experiment we obtained online informed consent, by asking participants to click on a button to accept the conditions for participation. This procedure was also approved by the local ethics committee. At the end of each study participants were debriefed regarding the true purpose of the experiment. All studies were conducted in accordance with the declaration of Helsinki.
Below we also describe the outcomes of a new meta-analysis that we conducted based on existing data (Landau et al., 2015) and including the novel studies that we report here. In contrast to the original meta-analysis on the effects of control-threat manipulations, reporting a weak to modest effect size and no evidence for publication bias (Landau et al., 2015), in our new meta-analysis we find strong evidence for publication bias in the field of compensatory control and a relatively small effect size for control-threat manipulations. These findings further challenge the idea that control threat manipulations can be used successfully to systematically study the effects of lack of feelings of control in the lab.
3. Experiment 1: Control threat & Magical Thinking
3.1. Introduction
A powerful mechanism that provides people with a feeling of control can be found in magical thinking, of which a key feature is the tendency to overestimate the relation between two objectively unrelated events (Vyse, 2013). Examples of magical thinking are the belief that a sun dance will cause the sun to shine afterwards, the belief that hurting a voodoo doll will lead to similar feelings of pain in a real person or the belief in astrology that the constellation of the planets exerts a causal influence on our well being and behavior.
Malinowksi already noted that a key function of magical thinking is to provide people with a sense of control over the environment, especially when that environment is perceived as threatening and unpredictable (Malinowski & Redfield, 1948). Indeed, several studies have related magical thinking to feelings of reduced control. In one study, Israeli citizens filled in questionnaires about magical thinking during the Gulf War (Keinan, 1994). The citizens were divided over two groups: a high stress group located in areas exposed to missile attack and a low stress group located in relatively safer areas. Citizens in the high stress areas were more prone to magical than citizens in low stress areas. In another study, people were primed with superstitious thoughts, which resulted in an increase in reported feelings of personal control (Damisch, Stoberock, & Mussweiler, 2010). Together these studies suggest that reduced feelings of control result in an enhanced propensity to engage in magical thinking.
As indicated, a key feature of magical thinking is an overestimation of the contingency between two objectively related events (van Elk, Rutjens, et al., 2015). People prone to magical thinking tend to perceive more illusory contingencies than non-magical thinkers (Brugger & Graves, 1997). For instance, Brugger and Graves (Brugger & Graves, 1997) showed that participants who scored high on a magical ideation scale tested fewer hypotheses – but retrospectively believed in many more hypotheses – than participants who scored low on that scale. Similarly it has been found that paranormal believers more readily develop illusions of control on a simple contingency learning task in the laboratory (Blanco, Barberia, & Matute, 2015; Matute, Yarritu, & Vadillo, 2011). These studies indicate a close relation between magical thinking and the feeling of control and that people prone to magical thinking tend to perceive more illusory contingencies than skeptics. However, an important limitation of these studies is that they did not experimentally manipulate control. In this regard, earlier studies were problematic because these results might also be explained by confounding factors, such as individual differences in locus of control (Dag, 1999). In our first experiment, we aimed to overcome these limitations by experimentally manipulating the feeling of control and investigating its effect on the perception of illusory contingencies. We used a contingency learning task in which participants were presented with medicines resulting in specific outcomes (De Houwer & Beckers, 2002). By systematically manipulating the contingency between the medicine and the outcome, we could investigate whether control threat specifically results in an over-estimation of the efficacy of the least contingent and most ambiguous medicines (i.e., akin to belief in the efficacy of homeopathy). Following the logic of error management theory (Haselton & Nettle, 2006), people should show a behavioral bias to avoiding the more costly error, and thus in the contingency medicine task participants might show a general bias toward overestimating the efficacy of the ambiguous medicines for healing a patient. This ‘optimism bias’ may be further exaggerated following a control threat manipulation, as participants may be motivated to compensate their lack of control by inferring more order and control.
3.2. Method
3.2.1. Participants
In our first experiment 81 psychology students participated (59 women and 22 men; 18–51 years old; mean age = 22.5; 76 Dutch and 5 other), who received 5 euros or .5 course credits for participation. We excluded one participant from further analysis because this participant failed to describe a situation during the control threat manipulation.1 Both experimental conditions (i.e., control-threat and control-affirmation) contained 40 participants. Prior to the study we performed a power analysis in G*power with an alpha of 0.05 and a desired power of 0.8. Concerning Experiment 1, earlier research (Whitson & Galinsky, 2011) showed an effect size of 0.58, which results in a required sample size of 37 participants per condition. Note that at the time when we conducted this study, the meta-analysis of Landau, Whitson & Kay (2015) was not yet available, and thus we based our power-analysis on previously reported effect sizes. We did not officially pre-register our study at the OSF, but used the local ethics online submission system (of the University of Amsterdam, Department of Psychology) instead, from which the hypotheses and the specified sample sizes can be retrieved.
3.2.2. Experimental Procedure
Upon arrival, we briefly instructed participants about the experiment and they signed an informed consent. In the first part of the experiment, we randomly presented participants with either the control threat or the control affirmation manipulation. To manipulate control threat we used an autobiographical recall task (Kay et al., 2008; Rutjens et al., 2010) where participants in the control threat condition were asked to think about a negative situation in which they had absolutely no control. Conversely, in the control affirmation condition, participants were asked to think about a negative situation in which they were in full control. In both conditions we asked participants to describe the event in about 100 words. This manipulation resulted in a between-subjects design with two conditions (control threat condition and control affirmation condition). We used four items on a slider scale of 0 to 100 to check whether our control threat manipulations were successful (i.e., ‘How much control did you experience in the situation you just described?’; How upsetting was the situation you just described?’; ‘To what extent do you consider yourself the actor or the director of your life?’; ‘To what extent do you believe that you are the one who is in control of your life?’). Similar items have been used as manipulation check in earlier research (Rutjens et al., 2010).
We subsequently presented participants with a task to tap into the development of illusory contingencies as a measure of magical thinking. We used a free response paradigm (Blanco, Matute, & Vadillo, 2011) in which participants were repeatedly presented with scenarios about a virtual patient who was given one of four different medicines. Participants saw either that the virtual patient was healed by the particular medicine or that the patient remained ill (see Figure 1). We used different medicine-healing contingencies (see Figure 1) and after observing 80 medicine-outcome contingencies, participants were asked to indicate on a scale of 0 to 100 to what extent they thought that each medicine was effective in healing the virtual patient. We expected participants in the control threat condition to overestimate the effectiveness of the medicines of which the effectiveness was most ambiguous (i.e., the 40 and 60% medicines), compared to participants in the control affirmation condition.
Stimuli in the Medicine Task used in Experiment 1 and 2. Participants were repeatedly presented with four different types of medicines, which could be followed by two different outcomes (i.e., the patient was healed or remained ill). The repeated presentation of different medicine-outcome contingencies allowed participants to infer the actual contingency for each medicine, which could in turn be affected by the participant’s feeling of control. The contingency table represents the percentage of trials in which a particular medicine was followed by a particular outcome.
Stimuli in the Medicine Task used in Experiment 1 and 2. Participants were repeatedly presented with four different types of medicines, which could be followed by two different outcomes (i.e., the patient was healed or remained ill). The repeated presentation of different medicine-outcome contingencies allowed participants to infer the actual contingency for each medicine, which could in turn be affected by the participant’s feeling of control. The contingency table represents the percentage of trials in which a particular medicine was followed by a particular outcome.
This study was conducted as a lab-based study. Following this study an additional unrelated experimental study was conducted on biological motion perception (van Elk, 2013). In addition to the measures described above, we also included the PANAS scale to measure overall emotion and arousal (Crawford & Henry, 2004), the Tellegen Absorption scale (Tellegen & Atkinson, 1974), the revised paranormal belief scale (Tobacyk & Milford, 1983) and the magical ideation scale (Eckblad & Chapman, 1983). These scales were included for exploratory purposes, were administered only after the main experiment and DVs were recorded, and are reported in the supplementary material online (the main finding is that we replicate previously established correlations between individual differences in paranormal beliefs and illusory pattern perception), while below we restrict our analyses to the main hypotheses.
3.3. Results
The manipulation check items and the dependent measures are presented in Table 2. Our control threat manipulation check indicated that participants in the control threat condition reported lower feelings of control (M = 9.43, SD = 11.01) during the recall task than participants in the control affirmation condition (M = 74.78, SD = 16.54), t(78) = –20.80, p = .000; d = –4.65. Participants in the control-threat condition rated the situation as more negative (M = 92.65, SD = 8.50) compared to participants in the control affirmation condition (M = 82.83, SD = 19.92), t(78) = 2.87, p = .0053, d = .64. Contrary to our predictions, general feelings of personal control did not differ between conditions (t(78) = –1.73, p = .0876, d = –.39; t(78) = –.7, p = .486, d = –.16 for respectively the 2 questions assessing feelings of personal control; see Table 2).
Mean ratings in Experiment 1 for the manipulation check items (upper part of table) and the ratings for the efficacy of the different medicines (lower part of table) for the control threat (left column) and the control affirmation condition (right column). Standard deviations are between brackets.
. | Control Threat (N = 40) . | Control Affirmation (N = 40) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 9.43 (11.01) | 74.78 (16.54) |
Valence of Situation | 92.65 (8.50) | 82.83 (19.92) |
Actor vs. Director | 56.35 (26.21) | 65.65 (21.71) |
Control in Life | 66.45 (19.64) | 69.43 (18.10) |
Efficacy of Medicines | ||
Medicine 1 (20%) | 21.98 (18.88) | 26.58 (23.71) |
Medicine 2 (40%) | 40.75 (20.21) | 40.35 (19.28) |
Medicine 3 (60%) | 60.90 (17.38) | 63.13 (17.81) |
Medicine 4 (80%) | 81.98 (13.57) | 74.05 (20.93) |
. | Control Threat (N = 40) . | Control Affirmation (N = 40) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 9.43 (11.01) | 74.78 (16.54) |
Valence of Situation | 92.65 (8.50) | 82.83 (19.92) |
Actor vs. Director | 56.35 (26.21) | 65.65 (21.71) |
Control in Life | 66.45 (19.64) | 69.43 (18.10) |
Efficacy of Medicines | ||
Medicine 1 (20%) | 21.98 (18.88) | 26.58 (23.71) |
Medicine 2 (40%) | 40.75 (20.21) | 40.35 (19.28) |
Medicine 3 (60%) | 60.90 (17.38) | 63.13 (17.81) |
Medicine 4 (80%) | 81.98 (13.57) | 74.05 (20.93) |
A Mixed ANOVA indicated a significant multivariate effect for medicine F(3,234) = 107.81, p = .000, pη2 = .58. On average, participants were able to correctly identify the effectiveness of each medicine relative to the other medicines (see Table 2). However, the two conditions did not differ significantly with respect to their ratings of the four medicines (F(1,78) = .05, p = .824, pη2 = .00) and the interaction between condition and medicine was not significant (F(3,234) = 1.43, p = .235, pη2 = .02), indicating that the control threat manipulation did not influence the extent to which participants overestimated the effectiveness of the medicines.
4. Experiment 2
4.1. Introduction
A potential limitation of our first experiment is the possibility that we found a ceiling effect in the medicine task. Our results showed that independent of the experimental condition, participants quite accurately assessed the effectiveness of each medicine. We only measured the perceived medicine effectiveness once, at the end of the task, at which point the real contingencies may have been obvious to almost all participants. The control threat manipulation was conducted at the beginning of the experiment and therefore effects on the detection of illusory contingencies may have been most pronounced at the beginning of the medicine task – when the real contingencies were not yet clear to the participants. Therefore, in our second experiment we used the same experimental paradigm as in the first study, but we asked participants to judge the effectiveness of the medicine multiple times during the task instead.
4.2. Method
4.2.1. Participants
In our second experiment 134 psychology students participated (94 women and 40 men; 18–65 years old; mean age = 23.02). We excluded eleven participants from further analyses because these participants failed to describe a situation during the control threat manipulation. After exclusion, the experimental conditions contained 55 participants and the control condition 68.
4.2.2. Experimental Procedure
The procedure was similar to that of our first experiment, except that we asked participants to rate the effectiveness of the medicine four times during the task, instead of only once at the end of the task. Each measurement regarding the effectiveness occurred after 20 medicine-outcome trials. In line with our first experiment, there were 80 medicine-outcome trials in total. Furthermore, in order to boost the control threat manipulations, after 40 trials participants were required to write down three arguments why the future is uncontrollable (in the control threat condition) or controllable (in the control affirmation condition), before rating the efficacy of the medicines. We expected participants in the control threat condition to overestimate the effectiveness of the ambiguous medicines (i.e., 40 and 60% condition), compared to participants in the control affirmation condition, especially at the earlier measurement occasions (i.e., this should be reflected in an interaction between Condition, Time and Medicine). This study was conducted as a lab-based study. In addition to this study participants also conducted an unrelated study on feelings of awe and the effects on the perception of the self, which are reported elsewhere (van Elk, Karinen, Specker, Stamkou, & Baas, 2016). No additional measures were included beyond the items reported above.
4.3. Results
Our control threat manipulation check (which was administered directly after the recall task) indicated that participants in the control threat condition reported lower feelings of control (M = 12.02, SD = 15.58) during the recall task than participants in the control condition (M = 58.46, SD = 32.57), t(121) = –9.71, p < .001, d = 1.76 (see Table 3). The conditions did not differ in valence ratings (t(121) = .99, p = .323, d = –.18; see Table 3). General feelings of personal control did not differ between conditions (t(121) = –.94, p = .35, d = .17; t(121) = –.282, p = .778, d = .05 for respectively the 2 questions assessing feelings of personal control).
Mean ratings in Experiment 2 for the manipulation check items (upper part of table) and the ratings for the efficacy of the different medicines (lower part of table) for the control threat (left column) and the control affirmation condition (right column). Standard deviations are between brackets.
. | Control Threat (N = 55) . | Control Affirmation (N = 68) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 12.02 (15.60) | 58.46 (32.57) |
Valence of Situation | 80.96 (28.40) | 76.01 (26.80) |
Actor vs. Director | 57.25 (26.99) | 61.75 (25.81) |
Control in Life | 63.20 (20.10) | 64.32 (23.34) |
Efficacy of Medicines | ||
Time 1 | ||
Medicine 1 | 25.07 (15.80) | 28.93 (22.59) |
Medicine 2 | 51.54 (18.54) | 52.35 (23.42) |
Medicine 3 | 60.41 (15.28) | 59.71 (23.83) |
Medicine 4 | 71.62 (17.87) | 69.64 (24.00) |
Time 2 | ||
Medicine 1 | 31.84 (16.09) | 26.98 (23.75) |
Medicine 2 | 49.12 (16.54) | 47.18 (22.84) |
Medicine 3 | 57.78 (16.91) | 58.87 (21.52) |
Medicine 4 | 72.15 (16.98) | 71 (20.70) |
Time 3 | ||
Medicine 1 | 29.25 (16.91) | 25.16 (19.54) |
Medicine 2 | 48.57 (14.76) | 43.91 (24.24) |
Medicine 3 | 61.69 (16.32) | 56.47 (23.50) |
Medicine 4 | 74.32 (14.6) | 73.65 (22.59) |
Time 4 | ||
Medicine 1 | 26.63 (16.98) | 26.67 (21.61) |
Medicine 2 | 48.09 (17.21) | 45.51 (21.11) |
Medicine 3 | 57.54 (16.02) | 57.47 (21.18) |
Medicine 4 | 73.56 (14.09) | 68.78 (22.84) |
. | Control Threat (N = 55) . | Control Affirmation (N = 68) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 12.02 (15.60) | 58.46 (32.57) |
Valence of Situation | 80.96 (28.40) | 76.01 (26.80) |
Actor vs. Director | 57.25 (26.99) | 61.75 (25.81) |
Control in Life | 63.20 (20.10) | 64.32 (23.34) |
Efficacy of Medicines | ||
Time 1 | ||
Medicine 1 | 25.07 (15.80) | 28.93 (22.59) |
Medicine 2 | 51.54 (18.54) | 52.35 (23.42) |
Medicine 3 | 60.41 (15.28) | 59.71 (23.83) |
Medicine 4 | 71.62 (17.87) | 69.64 (24.00) |
Time 2 | ||
Medicine 1 | 31.84 (16.09) | 26.98 (23.75) |
Medicine 2 | 49.12 (16.54) | 47.18 (22.84) |
Medicine 3 | 57.78 (16.91) | 58.87 (21.52) |
Medicine 4 | 72.15 (16.98) | 71 (20.70) |
Time 3 | ||
Medicine 1 | 29.25 (16.91) | 25.16 (19.54) |
Medicine 2 | 48.57 (14.76) | 43.91 (24.24) |
Medicine 3 | 61.69 (16.32) | 56.47 (23.50) |
Medicine 4 | 74.32 (14.6) | 73.65 (22.59) |
Time 4 | ||
Medicine 1 | 26.63 (16.98) | 26.67 (21.61) |
Medicine 2 | 48.09 (17.21) | 45.51 (21.11) |
Medicine 3 | 57.54 (16.02) | 57.47 (21.18) |
Medicine 4 | 73.56 (14.09) | 68.78 (22.84) |
We conducted a repeated measures ANOVA with Greenhouse-Geiser correction for the violation of sphericity. We found a significant multivariate effect for medicine (F(3,363) = 231.46, p < .001, pη2 = .64). On average, participants were able to correctly identify the effectiveness of each medicine relative to the other medicines. Participants in the control threat condition rated the effectiveness of medicines higher (52.54) than participants in the control affirmation condition (50.77, F(1,132) = 4.38, p = .038, pη2 = .03). Further analyses showed a significant interaction between condition and time (F(2.756,130) = 2.831, p = .043, pη2 = .02), showing that over time participants in the control threat condition rated the medicine as less effective than people in the control condition. However, these effects are based on sum score of the four medicine ratings. Further inspection shows that these effects do not differ between the four medicines, because both the critical interaction between condition and medicine (F(2.712,130) = .43, p = .712, pη2 = .00) and between condition, medicine and time (F(7.389,130) = 1.124, p = .345, pη2 = .01) failed to reach significance. Finally, post hoc analyses for each medicine rating at each time point showed no significant differences between the two conditions (All F’s < 2.2 and all p’s > 0.05), suggesting that the main effect for condition might be a Type 1 error resulting from noise accumulation within the computed aggregate scores.
5. Experiment 3
5.1. Introduction
In our first two experiments, we looked at the influence of a control threat manipulation on the extent to which people perceived illusory contingencies between four medicines and potential outcomes. The absence of an effect of our control threat manipulation on the perception of illusory contingencies as a measure of magical thinking, may be related to the contingency learning paradigm that was used, in which participants were able to learn the actual contingency between two stimuli. As one of the medicines obviously was effective (i.e., the 80% condition) and the other medicine was clearly ineffective (i.e., the 20% condition), the dependent measure could already have fulfilled in the basic need for epistemic structure and predictability. It could be that our control threat manipulation mainly affects the extent to which people perceive illusory contingencies, because magical thinking could be defined as involving beliefs that are “illusory” according to scientific standards (Lindeman & Svedholm, 2012). Therefore, in our third experiment we investigated whether a threat to personal control influenced the extent to which people perceived truly illusory contingencies (rather than the actual contingencies in the first two studies). To this end we used a paradigm that has been introduced before in the context of illusory contingency detection and the relation with schizotypal personality features (Brugger & Graves, 1997).
In this task participants were presented with a computer game in which they were instructed to move an animal through a maze to a target object. Systematically varying the ‘hidden rule’ that determines whether participants receive the target object, results in participants developing illusory beliefs about the ritual or strategy that needs to be performed in order to get the reward (e.g., by repeatedly moving along a specific path). We hypothesized that a control threat manipulation would lead participants to perceive more illusory contingencies, reflected in a belief in more different rules during the computer game.
5.2. Method
We recruited 71 participants through advertisements on the university campus and on the research recruitment website. This experiment was part of a larger sequence of experiments for which participants were either paid 10 euros or received 1 course credit. We excluded two participants from further analyses because these participants failed to describe a situation during the control threat manipulation. We analyzed the data of 69 participants (46 women and 25 men; 18–41 years old; mean age = 22.7; 68 Dutch and 3 other). The control threat condition contained 35 participants and the control condition 34. Prior to conducting the study we performed a power analysis in G*power with an alpha of 0.05 and a desired power of 0.8. In study 3, based on earlier research (Brugger & Graves, 1997) we expected a slightly larger effect size (0.72; i.e., based on the difference in number of believed but not tested hypotheses between low and high magical ideation participants), which makes the required sample size of at least 25 participants per condition. We did not officially pre-register our study at the OSF, but used the local ethics online submission system (of the University of Amsterdam, Department of Psychology).
The procedure was similar to our first two experiments, except that we measured the perception of illusory contingencies not with a medicine task, but with a mouse-trap task and a rabbit-trap task (see Figure 2), inspired by a previously used paradigm (Brugger & Graves, 1997; Heltzer & Vyse, 1994). Participants either first completed 2 × 20 mousetrap trials, followed by 2 × 20 rabbit-trap trials, or vice versa. We counterbalanced the order in which we presented participants with either the rabbit or the mouse-task. On each trial participants were presented with a 3 × 3 matrix in which the lower-left corner represented an animal (mouse/rabbit) and the upper right corner a trap with food (cheese/carrot). Participants were asked to move the animal towards the food by using the arrow keys on the keyboard. When the animal arrived at the trap, participants either received the feedback that the animal got the food, or that the animal was trapped. Furthermore, the participant’s task was to give the animal as much food as possible, and to find out the rule that determined whether the animal got the food or not. Both tasks used a rule that determined whether the animal would get the food or not. In the mousetrap task, this rule was that the mouse would get the cheese only when trial time exceeded 3.5 seconds. In the rabbit-trap task the rule was that the rabbit had to walk a minimum of 8 steps before moving to the field with the trap. As a consequence, both paradigms encouraged the development of illusory contingencies (e.g., participants believed they had discovered the ‘rule’ by which they could successfully give food to the animal, whereas in fact this rule often turned out to be illusory and/or incorrect).
Stimuli used for the mouse-trap task and the rabbit-trap task. Participants were instructed to move the animal to the food to obtain a reward and unbeknownst to the participant we manipulated the rule by which reaching the upper right corner resulted in reward (food) or ending up in the trap (punishment).
Stimuli used for the mouse-trap task and the rabbit-trap task. Participants were instructed to move the animal to the food to obtain a reward and unbeknownst to the participant we manipulated the rule by which reaching the upper right corner resulted in reward (food) or ending up in the trap (punishment).
After completing a total of 80 trials, we asked participants to indicate which rules they tested during the rabbit- and mousetrap trials (i.e., by presenting them with 13 statements reflecting potential contingencies; i.e., this comprised the number of ‘tested’ hypotheses) and also to point out which rule they in the end believed determined whether the animal got the food (i.e., again presenting them with a list of 13 statements; this comprised the number of ‘believed’ hypotheses). For the analysis we focused on the differences in the total number of believed hypotheses between the control threat and the control affirmation condition. After participation, we thanked and debriefed the participants. Because control threat has been linked to magical thinking (Damisch et al., 2010; Keinan, 1994) we expected that participants in the control threat condition would believe in a higher number of rules (perceive more illusory contingencies) than in the control affirmation condition.
This study was conducted as a lab-based study. Following this study an additional unrelated experimental study was conducted on biological motion perception (van Elk, Rutjens, van der Pligt, & van Harreveld, 2016). In addition to the measures described above, we also included the PANAS scale to measure overall emotion and arousal (Crawford & Henry, 2004), the Tellegen Absorption scale (Tellegen & Atkinson, 1974), the revised paranormal belief scale (Tobacyk & Milford, 1983) and the magical ideation scale (Eckblad & Chapman, 1983). These scales were included for exploratory purposes, were collected after the main experiment and DVs were administered, and are reported in the supplementary material online (the main finding is that we replicate previously established correlations between individual differences in paranormal beliefs and illusory pattern perception), while below we restrict our analyses to the main hypotheses.
5.3. Results
Our control threat manipulation check indicated that participants in the control threat condition reported lower feelings of control (M = 14.57, SD = 22.67) during the recall task than participants in the control affirmation condition (M = 74.94, SD = 23.72), t(67) = –10.81, p < .001, d = –2.60 (see Table 4). There was no difference in the valence ratings between both conditions (t(67) = –.22, p = .827, d = –.05; see Table 4). General feelings of personal control did not significantly differ between conditions (t(67) = 1.675, p = .099; t(67) = 1.136, p = .260 for respectively the 2 questions assessing feelings of personal control), although the mean ratings were in the expected direction (stronger feelings of general control in the control affirmation compared to the control-threat condition).
Mean ratings in Experiment 3 for the manipulation check items (upper part of table) and the ratings for the efficacy of the different medicines (lower part of table) for the control threat (left column) and the control affirmation condition (right column). Standard deviations are between brackets.
. | Control Threat (N = 35) . | Control Affirmation (N = 34) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 14.57 (22.67) | 74.94 (23.72) |
Valence of Situation | 83.17 (23.92) | 84.29 (18.71) |
Actor vs. Director | 56.57 (27.93) | 67.15 (24.36) |
Control in Life | 67.49 (19.31) | 72.56 (17.71) |
Belief in Illusory Contingencies | ||
# Hypotheses tested | 12.49 (4.05) | 12.62 (3.67) |
# Hypotheses believed | 8.14 (4.05) | 10.50 (5.05) |
. | Control Threat (N = 35) . | Control Affirmation (N = 34) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 14.57 (22.67) | 74.94 (23.72) |
Valence of Situation | 83.17 (23.92) | 84.29 (18.71) |
Actor vs. Director | 56.57 (27.93) | 67.15 (24.36) |
Control in Life | 67.49 (19.31) | 72.56 (17.71) |
Belief in Illusory Contingencies | ||
# Hypotheses tested | 12.49 (4.05) | 12.62 (3.67) |
# Hypotheses believed | 8.14 (4.05) | 10.50 (5.05) |
We used a MANOVA to analyze the differences between the two conditions in the number of tested and believed hypothesis. Box’s test of the equality of covariance matrices was not significant (F(10, 21418) = 1.06, p = .39), so the assumption of equal covariance matrices across conditions holds. The correlation between the dependent variables (number of tested and number of believed hypotheses) was r = .07, p = .592, df = 69. The multivariate effect for condition on the number of believed (Threat: M = 8.14, SD = 4.05; Control: M = 10.5, SD = 5.05) and tested hypotheses (Threat: M = 12.49, SD = 4.05; Control: M = 12.62, SD = 3.67) did not reach significance (F(2,66) = 2.70, p = .075, pη2 = .08) – although the pattern was in the opposite direction from our hypothesis (participants tended to believe more hypotheses in the control affirmation compared to the control threat condition). Thus, the number of believed and tested hypotheses did not differ between participants in the control threat and control affirmation condition. We found similar results after looking at both trap tasks (mouse & rabbit) in isolation. The order in which we presented participants with both tasks (mouse – rabbit vs. rabbit – mouse) did not influence the abovementioned results.
6. Experiment 4
6.1. Introduction
In our first three experiments, we manipulated feelings of control and measured the perception of illusory contingencies in different ways. The results showed that a control threat manipulation did not affect the extent to which people perceived and developed illusory contingencies. Previous studies using similar paradigms have indicated that people readily develop illusions of control in a lab-based setting (Langer, 1975; Langer & Roth, 1975; Matute, 1996) and these so-called ‘positive illusions’ have been related to a basic motivational need to maintain self-esteem (Taylor & Brown, 1988). Still, it could be argued that the perception of illusory contingencies in the tasks that we used, does not fulfill the need for having ‘epistemic structuring tendencies’ that help people need to cope when faced with a loss of control (Landau et al., 2015). That is, the illusory contingencies were rather arbitrary and may have been far removed from the compensatory control strategies that people employ in everyday situations and in their personal lives. In addition, the relation between our control-threat manipulation and the dependent measures may have been rather distal and abstract.
It has been argued that specific worldviews and political systems can act as a powerful means to cope with uncertainty and a lack of control (Kay et al., 2008; Kay, Shepherd, et al., 2010; Rutjens et al., 2010). For instance, Whitson and Galinsky showed that people when faced with uncertainty are more willing to engage in conspiracist thinking (Whitson & Galinsky, 2008) – again the idea being that the belief in structure and coherence that conspiracy beliefs provide, helps people to cope with a loss of control. Indeed it has been found that the perceived morality and uncertainty of companies or governments is associated with an increased belief in conspiracy beliefs (Van Prooijen & Jostmann, 2013). Accordingly, based on the existing literature in the fourth experiment we hypothesized that the control threat manipulation that we used in our first three experiments would make participants more willing to believe in conspiracy theories, as measured by using a conspiracy theory questionnaire (Bruder, Haffke, Neave, Nouripanah, & Imhoff, 2013). Conspiracy beliefs could potentially offer a more general and powerful mechanism to cope with a loss of control, than the illusory contingency tasks that we used in our first three studies.
6.2. Method
6.2.1. Participants
In this experiment we tested 236 psychology freshmen, who were obliged to participate in a series of experiments as part of their first year curriculum. We excluded three participants from further analyses because these participants failed to describe a situation during the control threat manipulation. After exclusion, we analyzed the results of 233 participants (176 women and 57 men; 17–39 years old; mean age = 19.7). For the analysis 116 participants were included in the control-threat condition and 117 in the control condition.
6.2.2. Experimental Procedure
The procedure was similar to that of the previously described experiments, except that a different dependent measure was used. The manipulation check items were completed using a 7-point Likert-scale ranging from 1 (not at all) to 7 (very much). To measure belief in conspiracy theories we used the conspiracy belief questionnaire, containing 38 items related to general conspiracy beliefs (e.g., ‘NASA faked the moon landing’; ‘The attack on the WTC was organized by the US government’ etc.) measured on a 1–11 likert scale (Cronbach’s α = .93; Bruder et al., 2013). In line with earlier research (Landau et al., 2015; Whitson & Galinsky, 2008), we expected participants in the control threat condition to show higher scores on the conspiracy questionnaire than participants in the control affirmation condition. As this study was conducted as part of the ‘testing week’ for freshmen psychology students, this study was part of a bigger package of online studies that participants were required to complete. Beyond the measures reported above, we also included an unrelated task consisting of the representativeness heuristic, which is reported elsewhere (Gervais et al., 2017).
6.3. Results
Participants in the control threat condition reported lower feelings of control (M = 1.38, SD = 1.92) than participants in the control condition (M = 5.86, SD = 2.31), t(231) = 9.626, p < .001, d = 1.26. However, generalized feelings of personal control did not differ between conditions (t(231) = .703, p = .483, d = .10 and t(231) = –.042, p = .676, d = .03; see Table 5).
Mean ratings in Experiment 4 for the manipulation check items (upper part of table) and the belief in conspiracy theories (lower part of table) for the control threat (left column) and the control affirmation condition (right column). Standard deviations are between brackets.
. | Control Threat (N = 116) . | Control Affirmation (N = 117) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 3.18 (1.92) | 5.86 (2.31) |
Actor vs. Director | 5.07 (1.54) | 5.21 (1.50) |
Control in Life | 5.15 (.96) | 5.09 (1.21) |
Belief in Conspiracy Theories | 4.87 (1.17) | 5.28 (1.21) |
. | Control Threat (N = 116) . | Control Affirmation (N = 117) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 3.18 (1.92) | 5.86 (2.31) |
Actor vs. Director | 5.07 (1.54) | 5.21 (1.50) |
Control in Life | 5.15 (.96) | 5.09 (1.21) |
Belief in Conspiracy Theories | 4.87 (1.17) | 5.28 (1.21) |
Contrary to our expectations, an independent samples t-test revealed that participants in the control threat condition believed less in conspiracy theories (M = 4.9; SD = 1.2) compared to the control affirmation condition (M = 5.3, SD = 1.2), t(231) = 2.62, p = .009, d = .34. These results imply that people whose feelings of control are threatened tend to believe less in general conspiracy theories. Whitson & Galinsky (2008) found that control threat increased belief in conspiracy theories, but in contrast to our study they measured belief in personal conspiracy theories (e.g., believing that your boss wants to fire you). Instead, in our study we measured general conspiracy beliefs, which could provide a sense of control and predictability for people scoring high on schizotypy and paranormal beliefs (Darwin, Neave, & Holmes, 2011; van Elk, 2015), but might actually be perceived as threatening by the student population in our study. It could be argued that many conspiracy beliefs provide a threat to structure and coherence as they challenge contemporary and more simplistic explanations. Accordingly, conspiracy beliefs could be considered as a poor source of compensatory control, which could potentially explain the opposite effect that we observed in our study (i.e., lower feelings of control result in a reduced belief in conspiracy beliefs).
6.4. Post-hoc analysis: Subjective Reports
In the first four experiments we used a recall task to manipulate feelings of control. In each of these experiments, our first manipulation check item showed that people complied with the instruction: they experienced less personal control during the recall of low control situations than during the recall of high control situations. However, in all four experiments the manipulation did not influence general feelings of being in control of one’s life – contrary to what has been reported in previous studies using similar manipulations (Kay et al., 2008; Rutjens et al., 2010). Therefore, it could well be that the autobiographical recall task does not effectively manipulate feelings of control. The absence of an effect on our dependent measures may thus have been related to the fact that our experimental manipulation did not sufficiently affect general feelings of control.
As a first remedy to address this problem we re-analyzed the data from Experiment 4, by looking carefully at the open responses that participants provided in the autobiographical memory task. First, we subjectively rated each response according to (1) whether the participants indeed had followed the instructions, (2) whether participants had misunderstood the instructions, or (3) whether participants described a nonsense story that was completely unrelated to the instructions. After discussing a few example responses we each rated the individual responses (i.e., to what extent the participant had followed the task instruction; ‘yes’, ‘no’ or ‘partly’) and the inter-rater reliability was Pearson’s r(236) = .883, p < .001. Next, we only included those participants in the analysis for which we both agreed that the response provided truly reflected the task instructions. For this analysis, 72 participants were included in the control condition and 83 participants in the control-threat condition. Our post-hoc analysis confirmed the main analysis: people believed less strongly in conspiracy beliefs in the control-threat condition (M = 4.91, SD = 1.14) compared to the control condition (M = 5.44, SD = 1.14), t(153) = 2.89, p = .004, d = .47.
Thus the findings from our post-hoc analysis confirm the outcomes from the main analysis: people believed less in conspiracy beliefs following a loss of control manipulation. However, the importance and severity of the situation recalled in the control threat task did not affect belief in conspiracy theories.
7. Experiment 5
7.1. Introduction
In four studies using a recall task as manipulation we did not find that control threat resulted in a stronger tendency to perceive illusory contingencies or an increased belief in conspiracy theories. In fact, in two out of four studies (Study 2 and Study 4) our findings were in the opposite direction compared to our hypothesis: participants in the control affirmation condition perceived more illusory contingencies and believed more strongly in conspiracy theories than participants in the control threat condition.
It could be argued that the recall task we used in our studies did not effectively manipulate actual feelings of personal control. In all four studies the task failed to significantly influence general feelings of being in control of one’s life. In order to investigate whether other control threat manipulations are more successful in directly manipulating personal feelings of control, we introduced a different manipulation in Study 5 and 6. In Study 5 we used an unsolvable anagram task to manipulate feelings of control. Similar to Study 4 we investigated whether a loss of control would make people more willing to accept other belief systems providing people with a sense of predictability and control. Previous studies have indicated that a loss of control increases belief in precognition, i.e., the belief that people can accurately predict the future (Greenaway et al., 2013). We actually conducted a quite similar study (in 2013 – at the time we were unaware of the at that time still unpublished work by Greenaway et al., 2013) to investigate whether a loss of control would increase people’s willingness to accept belief in Psi. For this study we used a more diverse pool of participants, by recruiting participants at our university, psychic fairs and through online mailing lists.
7.2. Method
7.2.1. Participants
We recruited 156 participants and we excluded 42 participants from further analyses because these participants failed to complete the experiment. After exclusion, we analyzed the results of 114 participants (75 women and 39 men; 18–63 years old; mean age = 22.6). The control threat condition contained 54 participants and the control affirmation condition 60 participants. We rewarded 4 participants with a 25-euro voucher by organizing a lottery.
7.2.2. Experimental Procedure
The experiment was conducted as an online study, which could be accessed by a link to a webpage. After obtaining online informed consent, in the first part of the experiment, we randomly assigned participants to either the control threat or the control manipulation. To manipulate control threat we used an anagram task in which participants in both conditions were asked to solve six anagrams within 12 minutes (2 minutes for each anagram). In the control condition, all anagrams were solvable, while in the control threat condition only the first three anagrams were solvable. We used four different questions to assess feelings of control which were completed on a 7-point Likert scale ranging from 1 (not at all) to 7 (very much): Feeling of Control during the anagram task (‘I felt in control when solving the anagrams’), Feeling of Control in Life (‘I am able to control my own life’), Self-efficacy (‘I am able to organize my life in the way I want to’) and Agency (‘My life is determined by my own actions’).
To measure belief in precognition we presented participants with a short story about scientific evidence on precognition (Bem, 2011). Next we asked participants to indicate on a 1–7 Likert scale their belief in Precognition (‘It is possible that individuals are affected by events in the future’; ‘There exist psychological processes and means of energy-transfer that cannot be explained by contemporary science’; ‘Psychologists have proven that precognition exists’). In line with earlier research (Greenaway et al., 2013), we expected participants in the control threat condition to show a stronger belief in precognition than participants in the control condition.
In addition to the measures described above we also included the revised paranormal belief scale (Tobacyk & Milford, 1983), which was administered prior the experimental manipulation. The data from this scale is included in the supplementary material online, while below we only report the main analyses to test the effects of our experimental manipulation.
7.3. Results
Participants in the control threat condition reported lower feelings of control (M = 2.94 SD = 2.28) than participants in the control condition (M = 3.62, SD = 2.10), although this result did not reach significance (t(112) = –1.64, p = .104, d = .31). Furthermore, generalized feelings of personal control did not differ between conditions (t(112) = .502, p = .616, d = –.09; t(112) = .378, p = .706, d = –.07; t(112) = .252, p = .802, d = –.05; for respectively the 3 questions assessing feelings of personal control), suggesting that the anagram task did not strongly affect feelings of control.
The different items measuring belief in Psi that were used as dependent variable were all positively correlated (r = .483, p < .001; r = .190, p = .043; r = .206, p = .028, df = 113). An MANOVA revealed that the control threat manipulation did not influence belief in precognition (F(3,109) = 3.109, p = .240, pη2 = .038). These results imply that after experiencing a threat to their feeling of being in control, people do not compensate this control loss by believing in paranormal events such as precognition (see Table 6).
Mean ratings in Experiment 5 for the manipulation check items (upper part of table) and the ratings for belief in precognition (lower part of table) for the control threat (left column) and the control affirmation condition (right column). Standard deviations are between brackets.
. | Control Threat (N = 54) . | Control Affirmation (N = 60) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control Task | 2.94 (2.28) | 3.62 (2.10) |
Feeling of Control in Life | 5.50 (1.19) | 5.38 (1.28) |
Self Efficacy | 5.37 (1.34) | 5.28 (1.21) |
Agency | 5.78 (1.04) | 5.73 (.84) |
Belief in Precognition 1 | 3.48 (1.82) | 3.27 (1.72) |
Belief in Precognition 2 | 3.57 (1.72) | 4.00 (1.67) |
Belief in Precognition 3 | 3.68 (1.60) | 3.77 (1.72) |
. | Control Threat (N = 54) . | Control Affirmation (N = 60) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control Task | 2.94 (2.28) | 3.62 (2.10) |
Feeling of Control in Life | 5.50 (1.19) | 5.38 (1.28) |
Self Efficacy | 5.37 (1.34) | 5.28 (1.21) |
Agency | 5.78 (1.04) | 5.73 (.84) |
Belief in Precognition 1 | 3.48 (1.82) | 3.27 (1.72) |
Belief in Precognition 2 | 3.57 (1.72) | 4.00 (1.67) |
Belief in Precognition 3 | 3.68 (1.60) | 3.77 (1.72) |
Participants in the control affirmation condition on average correctly completed 2.54 anagrams (SD = 1.57; range = 0–6), indicating that the solvable anagram task was quite difficult. As a consequence, participants in the control affirmation condition could actually also have experienced a loss of control because they were unable to solve the task, thereby potentially obscuring any effect of our experimental manipulation. To control for this possible confound, in a post-hoc analysis we selectively analyzed data from participants who correctly completed at least 3 out of the 6 anagrams in both the control threat and the control affirmation condition. By using this criterion we ended up with 54 participants in the control threat condition and 29 in the control condition. The difference in the feeling of control rating became more pronounced, with participants experiencing less control in the control threat condition (M = 2.94, SD = 2.28) than in the control condition (M = 4.17, SD = 1.95), t(81) = 2.460, p = .016, d = .57. Participants did not experience more general control in their lives in the control condition (M = 5.50, SD = 1.19) compared to the control-threat condition (M = 4.93, SD = 1.44), t(81) = –1.926, p = .058, d = –.44. No differences were found for the other control ratings (t(81) = –1.154, p = .252, d = –.27; t(81) = –.704, p = .484, d = –.16 respectively). Nor did we find a difference between conditions for the belief in precognition items (F(3,79) = 2.20, p = .10, pη2 = .08). These findings indicate that even a more effective control manipulation still does not increase belief in precognition.
8. Experiment 6
8.1. Introduction
Contrary to our expectations, our control threat manipulation did not increase belief in precognition – even though we used a different manipulation than in the previous studies to actually reduce participants’ feelings of control. Therefore, in experiment six we continued our search for a stronger and more reliable control threat manipulation. In the social psychological literature an often-used paradigm is presenting participants with short vignettes to assess participants’ implicit attitudes (Burstin, Doughtie, & Raphaeli, 1980), but also to directly simulate participants’ responses to particular scenarios (Robinson & Clore, 2001). For instance, in a recent study participants were presented with short vignettes describing important life events and subsequently they were required to what extent they perceived a special meaning or message in a random picture (Lindeman, Svedholm, Riekki, Raij, & Hari, 2013). By using this methodology it was found that paranormal believers perceived more special messages in the pictures than skeptics – in line with the idea that paranormal believers imbue random stimuli with more meaning and more readily project illusory patterns in random noise (Riekki, Lindeman, Aleneff, Halme, & Nuortimo, 2013; van Elk, 2013).
Based on these findings we hypothesized that the vignette technique could also be used as a tool to experimentally manipulate feelings of control. While the autobiographical memory manipulation that we used in Study 1–4 yielded many different types of responses and scenarios (e.g., as exemplified in our additional analysis described in section 6.3), presenting participants with vignettes provides a better control over the type of situations that participants imagine or recall. A related advantage of using vignettes is that the repeated presentation of different types of vignettes – associated with either low or high feelings of control – allows the use of a within-subjects design, yielding a higher statistical power and hence a more sensitive method to detect an eventual effect (Greenwald, 1976). Thus in Study 6 we instructed participants to imagine themselves as being in short stories described and we constructed different stories associated with low and high feelings of control (see below for examples).
In our final two studies we specifically investigated the effects of control-threat on illusory agent and pattern perception – following earlier suggestions that a lack of control results in perceiving more illusory patterns in the environment (Whitson & Galinsky, 2008). As dependent measure in Experiment 6 we used a modified version of the snowy pictures task (van Harreveld, Rutjens, Schneider, Nohlen, & Keskinis, 2014), by presenting participants with pictures representing visual noise, from which all image details were removed. Participants were instructed to report whether they saw an agent or object in the picture and if so, what they specifically thought was represented in the picture.
8.2. Method
8.2.1. Participants
The sixth experiment employed a within-subjects design, in which participants were repeatedly shown a short vignette describing a situation that was associated with low or high feelings of control. We recruited 36 participants (mean age = 22.1 years; 29 females) through advertisements on the university campus and on the research recruitment website. This experiment was conducted together with another unrelated study for which participants were either paid 10 euros or received 1 course credit in total.
8.2.2. Experimental Procedure
In order to manipulate control in a within-subjects design and on a trial-by-trial basis, we developed a novel task. Based on earlier studies (Lindeman et al., 2013) and inspired by the major life events inventory (Brugha, Bebbington, Tennant, & Hurry, 1985), we constructed 64 short vignettes describing everyday negative situations that were either implying a lack of control (e.g., ‘You bought some item via EBay. Two weeks after you transferred the money, you did not yet receive your parcel. You are concerned whether you will get the item at all.’), or a negative situation in which there was no control-threat (e.g., ‘You had a fight with your best friend and you are somewhat nervous to see him/her again.’). In a separate pre-test we asked 48 participants to rate each vignette for the overall feelings of control one would experience in the scenario described. For each vignette we calculated the average control rating and as expected the low-control scenarios induced lower feelings of control (M = 19.3, SD = 7.2; 1 = no control at all; 100 = very much in control) than the high-control scenarios (M = 57.4, SD = 4.5), t(62) = 25.2, p < .001).
After each vignette, participants were presented with a modified snowy picture (we edited the original Snowy pictures task, in order to remove all object information from the pictures). Participants were asked to indicate (1) their imagined feeling of control in the situation described (on a 5-point scale; 1 = no control, 5 = high control) and (2) to what extent they believed an image was present in the picture (on a 5-point scale; 1 = no picture present; 5 = picture present) and if so, what image they saw. In total participants were presented with 64 vignette-picture pairs that were presented in random order. In line with earlier research (Whitson & Galinsky, 2008), we expected participants to see more patterns in the snowy pictures after low control stories than after high control stories. For the analysis we looked at the average amount of control experienced following low- and high-threat stories and the average ratings regarding the absence/presence of a picture in the noisy stimuli.
In addition to the measures described above, we also included the revised paranormal belief scale (Tobacyk & Milford, 1983) and the creative experience questionnaire as a measure of fantasy-proneness (Merckelbach, Horselenberg, & Muris, 2001), which were administered after the experimental manipulation and relevant dependent variables. The data from these scales is included in the supplementary material online, but below we only report the main analysis to test the direct effects of our experimental manipulation.
8.3. Results
Our control threat manipulation was successful: participants experienced less control when reading the low control (M = 2.60, SD = .46) compared to high control (M = 3.12, SD = .46) stories, t(35) = –7.848, p < .001, d = –1.31. However, a paired sample t-test showed that the control threat manipulation did not influence the extent to which participants perceived patterns in the snowy pictures (t(35) =.736, p = .47, d = .12; see Table 7).
Mean ratings in Experiment 6 for the feeling of control (upper part of table) and the amount of illusory patterns perceived (lower part of table) for the control threat (left column) and the control affirmation condition (right column). Standard deviations are between brackets. Please note that in this study a within-subjects design was employed.
. | Control Threat (N = 35) . | Control Affirmation (N = 35) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 2.60 (.46) | 3.12 (.46) |
Illusory Pattern Perception | 2.44 (.93) | 2.40 (.82) |
. | Control Threat (N = 35) . | Control Affirmation (N = 35) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 2.60 (.46) | 3.12 (.46) |
Illusory Pattern Perception | 2.44 (.93) | 2.40 (.82) |
9. Experiment 7
9.1. Introduction
In the sixth experiment we did not find that a control threat manipulation (by presenting participants with different vignettes) resulted in an enhanced tendency for illusory pattern perception. We used the snowy pictures task as our dependent measure, in line with previous studies showing effects of control manipulations on this task (van Harreveld et al., 2014; Whitson & Galinsky, 2008). However, at the same time the snowy pictures task can be considered a projective task, as it relies entirely on the top-down effects of participants’ expectations on perception – akin to the effects obtained in research using reverse correlation methods (Dotsch & Todorov, 2012). In our study we noted that participants often reported imagery and content related to the scene that preceded the presentation of the snowy picture – indicating that this task may be more sensitive to capture semantic priming effects from the preceding stimuli, rather than the control threat manipulation that we were primarily interested in.
To remedy this problem, in our final study we used a perceptual decision-making task, which allows us to obtain a more objective measure of participants’ tendency for perceiving illusory patterns. Specifically, we used a biological motion detection task that we have used successfully in previous studies on illusory agent detection (van Elk, 2013; van Elk, Rutjens, et al., 2016). In this task participants are presented with short movies representing moving dots; in half of all movies a walking human avatar can be inferred from the motion pattern of the dots; in the other half of all movies no human movement is present. By instructing participants to detect whether an agent is present or not, it can be investigated to what extent participants display a bias toward making false positive responses, that is toward falsely inferring the presence of a human agent. As the control-threat manipulations that we introduced in Experiment 5 and 6 did not turn out to be more effective in manipulating feelings of control (in terms of the absolute size of the subjective control ratings), in our final study we returned to the autobiographical manipulation that we used in Experiment 1–4 and that has been used in previous studies on the effects of lack of control on illusory pattern perception (Whitson & Galinsky, 2008).
9.2. Method
9.2.1. Participants
In Experiment 7 we also used a within-subjects design, by presenting participants with a control-threat and a control-affirmation condition in a randomized order. The rationale for using a within-subjects design was that the perceptual agent detection task yields strong individual differences in the tendency to perceive illusory agents (i.e., as measured by the response bias and the perceptual sensitivity; cf. van Elk, 2013). Accordingly, by using a within-subjects design we could easily control for individual differences in the baseline tendency to report illusory agents – irrespective of the experimental condition (we also included block-order as a variable in our analyses to control for eventual practice- or repetition-related effects). We recruited 30 participants for the experiment (mean age = 22.6 years; 19 females) through advertisements on the university campus and on the research recruitment website and they were either paid 3.50 euros or received 0.5 course credit. Two participants did not provide the control ratings following each scenario and one participant did not provide the valence rating for one condition. These participants were excluded for the manipulation check analysis, but were still included in the analysis of the biological motion detection task.
9.2.2. Experimental Procedure
We manipulated control threat by using a recall task similar to experiment 1–4. However, in the present experiment we did not randomly divide participants in a control threat and control condition, but we tested each participant in both conditions. In the control threat blocks we asked participants to describe a negative situation in which they experienced low feelings of control, while in the control affirmation condition we asked participants to describe a negative situation in which they were in control. Across participants, we counterbalanced the order in which blocks (i.e., control threat vs. control affirmation) were presented.
After the control threat/control affirmation manipulation, participants conducted a biological motion detection task: they were instructed detect the presence of point-like walker embedded in a mask of moving visual distractor stimuli. The point-light walker consisted of 12 moving white dots against a black background, representing the motion of the joints of a human figure walking on a treadmill (for similar stimuli, see: van Elk, 2013). The point-light walker could move in a left- or a rightwards direction and could appear at 5 possible horizontal locations on the screen (–10°, –5°, 0°, 5°, 10°). In half of all stimuli the walker was presented in an unscrambled fashion and in the other half of all stimuli the walker was presented in a scrambled version by randomly presenting the dots on the screen, while keeping the motion information the same. By varying the amount of distractor points (48, 96, 192) three different levels of visual noise were created, thereby making it more difficult to detect the presence or absence of the walker. All stimuli were generated and rendered using the software package PointLightLab (www.pointlightlab.com) and consisted of 2-second movies. After each movie, participants were asked to indicate whether a human agent was present in a visual motion display. In each block a total of 60 stimuli was presented according to the following within-subjects design: Agent (present vs. absent) × Noise level (48, 96, 192 distractor points) and 10 repetitions per stimulus category. Following the notion that lacking control increases illusory pattern perception, we expected participants to make more false alarms (i.e., detecting an agent in random noise stimuli) after describing low control memories than after describing high control memories. The behavioral data was analyzed using a signal detection approach, focusing on the perceptual sensitivity (d’) and the response bias (c) (Stanislaw & Todorov, 1999).
This study was conducted as a lab-based experiment and was conducted as a single study. No additional measures or scales beyond the measures described above were included.
9.3. Results
Our control threat manipulation was successful: in the control-threat condition, participants reported lower feelings of control (M = 1.67, SD = 1.36) than in the control-affirmation condition (M = 5.03, SD = 2.30), t(27) = –7.33, p < .001, d = –1.35. No differences were found in the valence ratings between both conditions (control-threat: 4.72, SD = 1.85; control-affirmation: M = 5.24, SD = 1.75; t(28) = –1.51, p = .142, d = –.29; see Table 8).2
Mean ratings in Experiment 7 for the feeling of control and valence items for the control threat (left column) and the control affirmation condition (right column). Standard deviations are between brackets. Please note that in this study a within-subjects design was employed.
. | Control Threat (N = 28) . | Control Affirmation (N = 28) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 1.69 (1.36) | 5.04 (2.30) |
Valence | 4.72 (1.85) | 5.24 (1.75) |
Illusory Pattern Perception | 2.44 (.15) | 2.40 (.14) |
. | Control Threat (N = 28) . | Control Affirmation (N = 28) . |
---|---|---|
Manipulation Check Items | ||
Feeling of Control | 1.69 (1.36) | 5.04 (2.30) |
Valence | 4.72 (1.85) | 5.24 (1.75) |
Illusory Pattern Perception | 2.44 (.15) | 2.40 (.14) |
Analysis of d’ showed a main effect of Noise, F(2, 58) = 28.79, p < .001, pη2 = .50, indicating that with increased levels of visual noise the perceptual sensitivity decreased (see Figure 3). The control manipulation did not affect d’ and the interaction with Noise was not significant (F(2, 58) = 1.77, p = .180, pη2 = .06). Analysis of the response bias c did not reveal significant effects of the experimental manipulations (condition: F(1, 29) = .919, p =.346, pη2 = .03; condition*noise interaction: F(2, 58) = .725, p = .489, pη2 = .02). These results indicate that lower feelings of control did not increase illusory pattern perception.
Perceptual sensitivity (left) and Response bias (right) in both the control threat and control affirmation condition, for different amounts of visual noise.
Perceptual sensitivity (left) and Response bias (right) in both the control threat and control affirmation condition, for different amounts of visual noise.
10. Bayesian Analysis
Frequentist statistics p values cannot be used to provide evidence in favor of a null hypothesis, so we used a Bayesian information criterion, BIC (Masson, 2011; Wagenmakers, 2007) to investigate whether there was any support for the null hypothesis in our experiments. This Bayesian information criterion can be computed based on the output of the analysis of variance (ANOVA). Table 9 shows the strength of evidence according to different ranges of ρBIC values. In principle, the probability of the null hypothesis given the data (ρBIC(H0|D)), is the inverse of the probability of the alternative hypothesis given the data (ρBIC(Ha|D)).
values for the main hypotheses in our seven experiments.
Study . | Manipulation . | Hypothesis . | ρBICH0D . | ρBICHaD . | Evidence . |
---|---|---|---|---|---|
1 | Recall task | Efficacy Medicine 1 | .85 | .15 | Positive H0 |
Efficacy Medicine 2 | .90 | .10 | Positive H0 | ||
Efficacy Medicine 3 | .88 | .12 | Positive H0 | ||
Efficacy Medicine 4 | .54 | .46 | Weak H0 | ||
2 | Recall task | Efficacy Medicine 1 | .91 | .09 | Positive H0 |
Efficacy Medicine 2 | .89 | .11 | Positive H0 | ||
Efficacy Medicine 3 | .91 | .09 | Positive H0 | ||
Efficacy Medicine 4 | .89 | .11 | Positive H0 | ||
3 | Recall task | Nr. Believed rules | .89 | .11 | Positive H0 |
Nr. Tested rules | .46 | .54 | Weak H0 | ||
4 | Recall task | Conspiracy beliefs | .34 | .66 | Weak Ha |
5 | Unsolvable anagram task | Paranormal beliefs | .89 | .11 | Positive H0 |
6 | Reading threat-related stories | Perceived patterns | .82 | .18 | Positive H0 |
7 | Recall task | Perceptual sensitivity | .87 | .16 | Positive H0 |
Response bias | .77 | .23 | Positive H0 |
Study . | Manipulation . | Hypothesis . | ρBICH0D . | ρBICHaD . | Evidence . |
---|---|---|---|---|---|
1 | Recall task | Efficacy Medicine 1 | .85 | .15 | Positive H0 |
Efficacy Medicine 2 | .90 | .10 | Positive H0 | ||
Efficacy Medicine 3 | .88 | .12 | Positive H0 | ||
Efficacy Medicine 4 | .54 | .46 | Weak H0 | ||
2 | Recall task | Efficacy Medicine 1 | .91 | .09 | Positive H0 |
Efficacy Medicine 2 | .89 | .11 | Positive H0 | ||
Efficacy Medicine 3 | .91 | .09 | Positive H0 | ||
Efficacy Medicine 4 | .89 | .11 | Positive H0 | ||
3 | Recall task | Nr. Believed rules | .89 | .11 | Positive H0 |
Nr. Tested rules | .46 | .54 | Weak H0 | ||
4 | Recall task | Conspiracy beliefs | .34 | .66 | Weak Ha |
5 | Unsolvable anagram task | Paranormal beliefs | .89 | .11 | Positive H0 |
6 | Reading threat-related stories | Perceived patterns | .82 | .18 | Positive H0 |
7 | Recall task | Perceptual sensitivity | .87 | .16 | Positive H0 |
Response bias | .77 | .23 | Positive H0 |
Table 9 shows the values for both the null and alternative hypotheses in our seven experiments. For instance, in experiment 1 we investigated the influence of control threat on the efficacy ratings of different medicines. Considering the first of the four medicines, we used the ANOVA output to compute the posterior probability in favor of the null hypothesis: (ρBIC(H0|D)) = .85. For the second, third, and fourth medicine, the probabilities were .9, .88 and .54 respectively. According to Table 9, these results imply that for the first three medicines there is positive evidence for the null hypothesis that the control threat manipulation does not influence the medicine effectiveness rating. For the fourth medicine, the Bayesian analysis indicates weak evidence in favor of the null hypothesis, yet weaker evidence in favor of the alternative hypothesis (ρBIC(Ha|D)) = .46.
In general, the Bayesian analyses of all experiments represented in Table 9 show similar results. All values – except one (experiment 4) – show evidence in favor of the null hypothesis. Two tests only weakly favor the null hypothesis, while twelve tests positively favor the null hypothesis. The weak evidence in favor of the alternative hypothesis in experiment four was opposite to the expected direction (i.e., participants believed more strongly in conspiracy theories in the control affirmation compared to the control threat condition).
11. Correlation Analysis
Across the different studies we were primarily interested in whether our control manipulation affected the dependent measures. However, it could well be that – rather than experimentally manipulated control – subjective feelings of control are primarily related to compensatory control efforts (Landau et al., 2015). To investigate this possibility, in an exploratory analysis we looked at the relation between the subjective control ratings and our dependent measures. Following the compensatory control framework, we expected that stronger feelings of personal control would be associated with a reduced tendency to engage in compensatory control strategies. Below we will only report the significant correlations between our control and dependent measures.3
In Experiment 2, we found a positive correlation between control in life and the effectiveness of the least effective medicine (at both the first, r = .279, p = .002, df = 122, and second measurement, r = .243, p = .007, df = 122). People who were more in control of their lives tended to rate the least effective medicine as more effective.
In Experiment 3, we found a positive correlation between Feeling of Control and the number of believed hypotheses (r = .292, p = .015, df = 68) and between Actor vs. Director and the number of tested hypotheses (r = .328, p = .006, df = 68), indicating that higher feelings of personal control were associated with the perception of more illusory contingencies.
In Experiment 4, we found a negative correlation between Feeling of Control and conspiracy beliefs (r = –.220, p = .001) and between Control in Life and conspiracy beliefs (r = –.221, p = .001), reflecting that higher overall feelings of control in one’s life are associated with a reduced tendency to believe in conspiracy theories.
Similarly, in Experiment 5 we found a negative relation between Agency (i.e., ‘My life is determined by my own actions’) and belief in precognition items #1 and #2 (r = –.272, p = .003, df = 113; r = –.226, p = .016, df = 113 respectively), reflecting that a stronger feeling of agency over one’s life was associated with reduced belief in precognition.4
12. Meta-Analysis on Control-threat manipulations
Recently a meta-analysis on different studies on compensatory control has been conducted (Landau, Kay & Whitson, 2015). However, upon close inspection of the original meta-analysis it turned out that the standard errors of the individual studies included in the meta-analysis were larger than they should be.5 This problem might be the result of a coding error in the original meta-analysis. If one performs a meta-analysis and mistakenly uses the standard error as a measure of the sampling variance, then the y-axis scale of the funnel plot will be overestimated.
We extracted from the original meta-analysis all relevant statistics in Table 3 and used the R software and specifically the R-package metafor (Viechtbauer, 2010) to reconduct the meta-analysis. In line with the original authors, we first applied a Fisher z transformation to each raw correlation coefficient (Fisher, 1921) and computed its appropriate standard error . Because the standard error of the raw correlation coefficient depends on the correlation itself , not using the Fisher transformation will result in an underestimated standard error for studies with larger effects.
The left panel of Figure 4 shows the funnel plot of the original meta-analysis (figure 2 in Landau et al., 2015) while the right panel shows the funnel plot of our re-analysis. The funnel plots of the original meta-analysis and our re-analysis differ because we adjusted the calculation of the standard errors. In addition, the original funnel plot shows the raw correlation coefficient and its standard error, whereas our funnel plot shows the fisher z transformed correlation and its standard error. Using the raw – instead of the transformed – measures results in smaller standard errors for larger correlations. This effectively reduces funnel plot asymmetry, thereby concealing signs of publication bias.6 In the original meta-analysis, the authors concluded that there was no evidence of missing studies based on their funnel plot analysis. However, our funnel plot shows extreme asymmetry where studies with low sample sizes (high standard errors) show larger effects than studies with high sample sizes, a pattern characteristic of publication bias. This visual conclusion is supported statistically by the results of Egger’s test (Egger, Smith, Schneider, & Minder, 1997; Sterne & Egger, 2001), showing that the standard error significantly moderates the effect size in our meta-analysis (β = 3.52, SE = .652 Z = 5.397, p < .0001). In fact, the standard error explains 64.8% of all heterogeneity in the meta-analysis, making it the most important predictor of effect sizes in the compensatory control field. Our re-analysis of the original meta-analysis results in an effect size estimate of r = .26, p < .0001, 95% CI = [.22–.30]. In line with the original meta-analysis, we applied the trim and fill procedure to take into account any missing studies. As the trim and fill procedure is sensitive to funnel plot asymmetry, the original analysis influenced the overall effect size estimate only slightly. But, as can be seen in the right panel of figure 4, the asymmetric effect size distribution of our funnel plot results in a substantial amount of potential missing studies (the white dots), resulting in an adjusted effect size estimate of r = .186, p < .0001, 95% CI = [.133–.238].
Funnel plots of the original meta-analysis (left) and our updated analysis (right). White dots refer to expected missing studies according to the Trim and Fill analyses.
Funnel plots of the original meta-analysis (left) and our updated analysis (right). White dots refer to expected missing studies according to the Trim and Fill analyses.
13. General Discussion
The aim of our studies was to extend the compensatory control framework, according to which a loss of control results in a motivated tendency to search for other epistemic structuring tendencies that fulfill one’s basic need for control and predictability (Landau et al., 2015). We investigated to what extent a loss of control increases different types of illusory pattern perception, namely: magical thinking, endorsement of conspiracy beliefs, paranormal beliefs and illusory agent perception. We found that our control threat manipulation did not affect any our dependent measures. In fact, in two out of seven experiments the observed effect was in the opposite direction, reflected in a decreased tendency to engage in magical thinking and to believe in conspiracy theories following a control threat manipulation. Furthermore, in all experiments Bayesian statistics provided weak to positive evidence in favor of the null hypothesis that our control threat manipulation did not result in increased illusory pattern perception. Finally, our update of a meta-analysis on compensatory control (Landau et al., 2015) resulted in a smaller effect size estimate, because of a discovered error in the publication bias analysis. Although our meta-analysis shows a significant overall effect size estimate after correcting for missing studies, we advise to interpret this estimate with caution. The original meta-analysis included only 2 unpublished experiments, compared to 53 published experiments. Estimates show that 50% of psychology experiments remain unpublished (Cooper, DeNeve, & Charlton, 1997) and that makes it likely that there are still a lot of unpublished experiments locked up in file drawers. Furthermore we note that meta-analytic techniques can and will never provide the compelling evidence that is required to convince the skeptic of the existence (or non-existence) of an effect, as different meta-analytic procedures can produce dramatically different results and conclusions (Van Elk, Matzke, et al., 2015). The best remedy to establish the robustness of an effect is to conduct a collaborative pre-registered replication study in different countries and settings, and we note that such a project is currently underway for testing the predictions of the compensatory control framework in the context of religion (Hoogeveen, Wagenmakers, Kay & van Elk, submitted).
In all experiments using the autobiographical recall task to manipulate feelings of control, participants complied with the instruction to recall a situation in which they did or did not lack control, as evidenced by the manipulation check item. However, in all experiments the manipulation did not influence general feelings of being in control of one’s life. Therefore, our results cast doubt on the reliability and usability of autobiographical control threat manipulations for effectively manipulating feelings of control. It could be argued that merely asking participants how much control they experience in their lives in general (as was done in the manipulation check items following the experimental manipulations), already suffices for restoring one’s general feelings of control. Although previous studies using similar experimental manipulations have shown that a control threat manipulation also affects general feelings of control in one’s life (Greenaway et al., 2013; Kay et al., 2008; Kay, Shepherd, et al., 2010; Rutjens et al., 2010), our findings cast doubt on the validity of experimental control threat manipulations. The lack of an efficacious control threat manipulation – in turn – could also explain the absence of an effect on our dependent measures related to illusory pattern perception. Participants simply did not compensate by an increased tendency for illusory pattern perception, because in the experimental setting they did not effectively experience a lack of control. In the final two experiments we employed a within-subjects design, including vignettes (Study 6) and a repeated autobiographical recall task (Study 7) to manipulate feelings of control within the same participant. Although within-subjects designs typically have higher statistical power, this comes at the expense of other problems, such as increasing the potential for socially desirable responding and participants guessing the hypothesis (Greenwald, 1976). Especially feelings of control are unlikely to fluctuate rapidly over time and in fact, a prior affirmation of control could cancel out any eventual subsequent effect of a control threat manipulation. A potential remedy to this problem could be to use a longitudinal design, in which participants’ feelings of control are probed or manipulated at different time points that are separated by longer intervals (see for instance: Hoogeveen et al., in prep). Within the present context we note that the absence of an effect of control threat on our DVs in Study 6 and 7 could also be related to a lack in our ability to effectively manipulate people’s feelings of control in an experimental setting.
In our experiments we used a wide range of different dependent measures ranging from the perception of illusory contingencies to conspiracy beliefs and illusory pattern perception. A common denominator of these different measures is the tendency to perceive illusory patterns in random noise or between unrelated events (van Elk, Bekkering, & Friston, 2015). For instance, many conspiracy beliefs are characterized by the idea that unexplained and seemingly random details of a major event (e.g., the time it took the ambulance to take Princess Diana to the hospital) must have a cause at deeper level (e.g., the ambulance was intentionally delayed because of a conspiracy by the secret service; cf. Brotherton & French, 2014). Similarly, paranormal beliefs presume a fundamental relation between two objectively unrelated events, e.g., as in the case of mental healing or thought-transfer. It has been argued that a key feature of magical thinking is indeed the perception of illusory relations in patterns of random noise (Malinowski & Redfield, 1948) and many studies have shown that participants who engage in magical thinking, are also prone to the perception of illusory contingencies and illusory patterns (Blackmore & Moore, 1994; Blanco et al., 2015; Krummenacher, Mohr, Haker, & Brugger, 2010; Riekki et al., 2013; van Elk, 2013). Following the compensatory control framework, it is to be expected that the perception of illusory patterns and contingencies provides a sense of order, meaning and control – especially in case when personal feelings of control are low. Although field studies have provided evidence for the bidirectional relation between loss of control and magical thinking (Dudley, 1999; Keinan, 1994, 2002) – in our experiments we did not find evidence for a causal role of control threat on magical thinking. Instead, in two studies the effects of our experimental manipulation were in the opposite direction, reflected in a reduced tendency to perceive illusory contingencies and a reduced belief in conspiracy beliefs following a control-threat manipulation. These findings are reminiscent of the classical effects of learned helplessness (Abramson, Seligman, & Teasdale, 1978), in which exposure to uncontrollable situations results in passivity and a reduced tendency to develop ‘positive illusions’ (Taylor & Brown, 1988).
We acknowledge that our studies may have been underpowered, as the sample sizes were relatively small compared to today’s standards and also in light of the adjusted effect size estimate following the meta-analysis by Landau et al. (2016). This is a natural consequence of the fact that these studies were conducted well before the meta-analysis came out. At the time we based our power analysis on the available effect sizes that had been reported in the existing literature (i.e., consisting of studies which were admittedly even more strongly underpowered). Even though our sample sizes were limited, the pattern of results and the Bayesian re-analysis all point in the same direction, namely that control threat manipulations did not affect our dependent variables at all.
In sum, experimentally manipulated control did not yield the expected effects and as such our experiments question the validity of and the possibility to use laboratory-based manipulations of control to obtain causal evidence for compensatory control strategies. This concern is further underlined by our exploratory correlation analyses (reported in Section 11). For three studies we found a negative relation between overall feelings of personal control over one’s life and belief in conspiracy theories and paranormal beliefs. Interestingly, the directionality of this relation is exactly in line with the predictions of compensatory control theory (Landau et al., 2015), such that reduced overall feelings of control are associated with an increased tendency to believe in conspiracy theories and paranormal phenomena. These findings corroborate the validity of at least some of our dependent measures, such as conspiracy and paranormal beliefs, as a potential source for epistemic structuring tendencies. It could well be that the measures that were used in the other studies (e.g., contingency learning; illusory agent detection) are not in any meaningful or direct way related to the experience of control – as they rely more strongly on ‘objective’ abilities (or inabilities) to detect a pattern or structure. Thus, future studies could shed further light on the potential boundary conditions of different sources of compensatory control (i.e., potential difference between affirming either pre-existing belief systems or ad-hoc pattern perception). Still, the individual differences analyses suggest that a more fruitful way to address the relation between loss of control and epistemic structuring tendencies is the use of an individual difference approach, by comparing participants who overall experience high or low feelings of control in life. Of course we note that the findings regarding the individual differences need to be interpreted with caution – as the results were exploratory and we only observed a significant correlation for a sub-set of the dependent measures that we used. In addition, correlational findings cannot be used to infer causality and it could well be that a third factor actually underlies the observed relation between feelings of control and conspiracy/paranormal beliefs (e.g., education; socio-economic status; gender). Still, the observed relation between general feelings of control and conspiracy/paranormal beliefs is in line with other studies, also indicating that reduced control is associated with a stronger tendency to belief in astrology or conspiracy theories (Lillqvist & Lindeman, 1998; Newheiser, Farias, & Tausch, 2011).
It has been suggested that belief systems can provide a source of compensatory control – only when the control-providing aspects of the belief framework are emphasized (Landau et al., 2015). For instance, effects of control threat manipulations were found selectively when God was presented as actively intervening in the world, rather than as a Creator (Kay, Shepherd, et al., 2010). Accordingly, it could be argued that the absence of an effect in our experiments is related to the fact that our dependent measures did not fulfill the basic need for personal control and predictability. Still, a similar argument could be made for the measures that have been used in earlier studies on compensatory control, such as illusory pattern perception, conspiracy beliefs (Whitson & Galinsky, 2008) and belief in precognition (Greenaway et al., 2013).
Rather than doubting the validity of our dependent measures, we argue that a more fruitful approach is to seriously question the reliability of previously reported control-threat manipulations. Many ‘early’ studies on compensatory were highly underpowered (especially given the effect sizes reported) and our meta-analysis suggests that the field suffers strongly from publication bias, with many null-results ending up in the file-drawer. Even though the evidence for effects of experimental control manipulations may seem overwhelming – especially in light of the sheer number of studies that has been published and the recent meta-analysis showing a small though consistent effect – it is important to remain critical. The present series of studies is only a first attempt to open the file-drawer in this field and we hope that more replication attempts are to follow. We would especially welcome direct replications of effects that have had a strong theoretical impact. Given the promising findings of the individual difference analysis, we would like to call for more substantial evidence that the compensatory control theory can actually be supported by experimental control manipulations.
Data Accessibility Statement and Open Practices
The data from the studies reported is available on the OSF: DOI https://doi.org/10.17605/OSF.IO/T9VBY.
Next to the measures directly related to assessing the effects of control threat on illusory pattern perception, for each study we also listed any additional measures that were included to assess individual differences (e.g., related to paranormal beliefs or suggestibility). In addition, we also describe whether the study was part of a larger research project, in which multiple studies were combined.
In all studies we applied a simple exclusion criterion according to which participants were excluded who did not provide any description at all (i.e., they left the field blank or only filled in gibberish words). In some cases this resulted in the exclusion of quite a few participants.
We note that – in contrast to the other studies – only 2 manipulation check items were included in this experiment.
We did not correct these analyses for multiple comparisons. The correlation analyses are exploratory and the findings should therefore be considered with caution (see also: General Discussion).
A non-significant correlation was observed between Control in Life and Precognition (r = –.173, p = .066, df = 113), reflecting that higher feelings of control in one’s life were associated with reduced belief in precognition.
For instance, take the study with the largest effect (Waytz et al. (2010), study 3) that has a sample size of 23 and a raw correlation coefficient of r = 0.79. In the left panel of figure 4, this study is located all the way to the right on the x-axis. The standard error of this study appears to be approximately 0.3, but according to the formula it should be .
When detecting publication bias based on funnel plot asymmetry one should look whether studies with smaller sample sizes show larger effects. In case of publication bias studies located to the far right on the x-axis (large effects) should also be located near the bottom of the y-axis (larger standard errors due to the smaller sample sizes). Using the standard error of the raw correlation coefficient results in a smaller standard error for studies with larger effects, locating them higher on the y-axis and thereby concealing funnel plot asymmetry.
Funding Information
This research was supported by a VENI grant no. 016.135.135 to the first author from the Netherlands Organization for Scientific Research (NWO).
Competing Interests
The authors have no competing interests to declare.
Author Contributions
These authors contributed equally to this work.
MvE and PL designed the experiments. PL and research assistants collected the data. PL and MvE analysed the data and the meta-analysis was conducted by PL. MvE wrote the manuscript with input from PL.
Peer review comments
The author(s) of this paper chose the Open Review option, and the peer review comments are available at: http://doi.org/10.1525/collabra.155.pr