For years, adult psychological research has benefitted from web-based data collection. There is growing interest in harnessing this approach to facilitate data collection from children and adolescents to address foundational questions about cognitive development. To date, however, few studies have directly tested whether findings from in-lab developmental psychology tasks can be replicated online, particularly in the domain of value-based learning and decision-making. To address this question, we set up a pipeline for online data collection with children, adolescents, and adults, and conducted a replication of Decker et al. (2016). The original in-lab study employed a sequential decision-making paradigm to examine shifts in value-learning strategies from childhood to adulthood. Here, we used the same paradigm in a sample of 151 children (N = 50; ages 8 - 12 years), adolescents (N = 50; ages 13 - 17 years), and adults (N = 51; ages 18 - 25 years) and replicated the main finding that the use of a “model-based” learning strategy increases with age. In addition, we adapted a new index of abstract reasoning (MaRs-IB; Chierchia et al. 2019) for use online, and replicated a key result from Potter et al. (2017), which found that abstract reasoning ability mediated the relation between age and model-based learning. Our re-analyses of two previous in-lab datasets alongside our analysis of our online dataset revealed few qualitative differences across task administrations. These findings suggest that with appropriate precautions, researchers can effectively examine developmental differences in learning computations through unmoderated, online experiments.

Belief system structure can be investigated by estimating belief systems as networks of interacting political attitudes, but we do not know if these estimates are replicable. In a sample of 31 countries from the World Values Survey (N = 52,826), I find that countries’ belief system networks are relatively replicable in terms of connectivity, proportion of positive edges, some centrality measures (e.g., expected influence), and the estimates of individual edges. Betweenness, closeness, and strength centrality estimates are more unstable. Belief system networks estimated with smaller samples or in countries with more unstable political systems tend to be less replicable than networks estimated with larger samples in stable political systems. Although these analyses are restricted to the items available in the World Values Survey, they show that belief system networks can be replicable, but that this replicability is related to features of the study design and the political system.

The crisis of confidence in psychology has prompted vigorous and persistent debate in the scientific community concerning the veracity of the findings of psychological experiments. This discussion has led to changes in psychology’s approach to research, and several new initiatives have been developed, many with the aim of improving our findings. One key advancement is the marked increase in the number of replication studies conducted. We argue that while it is important to conduct replications as part of regular research protocol, it is neither efficient nor useful to replicate results at random. We recommend adopting a methodical approach toward the selection of replication targets to maximize the impact of the outcomes of those replications, and minimize waste of scarce resources. In the current study, we demonstrate how a Bayesian re–analysis of existing research findings followed by a simple qualitative assessment process can drive the selection of the best candidate article for replication.

Method sections in psychology articles differ in the amount of information they provide, or the level of specificity at which they do so. This can make incremental research (e.g., replication efforts) difficult, because potentially relevant methodological decisions or practices may not be reported. As yet, these unwritten practices have not been systematically studied; the current work represents a first attempt in this direction. For this paper, we interviewed 22 experimental psychologists in the Netherlands to find out about these unwritten aspects of experimenting, as well as their opinions and beliefs regarding these practices. Thematic analysis of the transcripts suggests that (a) experimental psychologists indeed employ a variety of methods that they consider relevant yet do not routinely report in method sections, and (b) these unwritten practices seem to fall into two broad categories: ‘professionalism’ and ‘the production of good data’. We discuss implications for psychological research generally, and the replication debate more specifically.

Worrying about the negative consequences of appearing imperfect to others (i.e., perfectionistic self-presentation) is conceptually related to social anxiety. Mackinnon, Battista, Sherry and Stewart ( 2014 ) tested whether perfectionistic self-presentation could predict social anxiety beyond several important covariates using a 21-day daily measurement approach. We sought to replicate Mackinnon et al.’s ( 2014 ) findings using the same daily diary methodology. Participants included 263 young adults (79.9% women; M age = 21.4) who completed a series of questionnaires once per day for 21 days. Participants completed measures of perfectionistic self-presentation, perfectionism cognitions, social anxiety, depressed mood and socially prescribed perfectionism. Intraclass correlations suggested measures had both within-subjects and between-subjects variability. Confirmatory factor analyses supported the a-priori factor structures at both levels. Using multilevel structural equation modeling, we showed that perfectionistic self-presentation predicted social anxiety even when controlling for socially prescribed perfectionism, depressed mood and perfectionism cognitions at both levels, replicating Mackinnon et al ( 2014 ). Our replication suggests that perfectionistic self-presentation is an important predictor of daily social anxiety. Intervention efforts may wish to target perfectionistic self-presentation in order to better help treat those with social anxiety. Open data/methods: https://osf.io/ty2aj/ .

The Harry Potter series describes the adventures of a boy and his peers in a fictional world at the “Hogwarts School of Witchcraft and Wizardry”. In the series, pupils get appointed to one of four groups (Houses) at the beginning of their education based on their personality traits. The author of the books has constructed an online questionnaire that allows fans to find out their House affiliation. Crysel, Cook, Schember, and Webster ( 2015 ) argued that being sorted into a particular Hogwarts House through the Sorting Hat Quiz is related to empirically established personality traits. We replicated their study while improving on sample size, methods, and analysis. Although our results are similar, effect sizes are small overall, which attenuates the claims by Crysel et al. The effect vanishes when restricting the analysis to participants who desired, but were not sorted into a particular House. On a theoretical level, we extend previous research by also analysing the relation of the Hogwarts Houses to Schwartz’s Basic Human Values but find only moderate or no relations.

Study 1a of Giessner and Schubert ( 2007 ) found a causal effect of vertical spatial cues on power judgments. Recent work showed that this was a false positive ( Klein et al., 2018 ). Here, we test whether another paradigm (i.e., original Study 3a) can be replicated, and develop an adjusted paradigm of original Study 1a to clarify what kind of vertical spatial cues influence power judgments. Our current preregistered Study 1 confirms original Study 3a of Giessner and Schubert ( 2007 ). It shows that information about the power of a leader is represented spatially by placing the leader’s box higher in an organigram. Our current Study 2 distinguishes vertical ranks from magnitude of vertical spatial difference without changes in rank. The original Study 1a and the failed replication manipulated only magnitude while leaving rank equal. We confirm the null finding here. However, we also find that vertical rank order does indeed affect power judgments, again in a preregistered study, and in line with prior work. In sum, building on earlier work and the failed replication, we clarify that vertical rank order, but not magnitude of elevation, are associated with power judgments.

When you lack the facts, how do you decide what is true and what is not? In the absence of knowledge, we sometimes rely on non-probative information. For example, participants judge concretely worded trivia items as more likely to be true than abstractly worded ones (the linguistic truth effect ; Hansen & Wänke, 2010 ). If minor language differences affect truth judgements, ultimately they could influence more consequential political, legal, health, and interpersonal choices. This Registered Report includes two high-powered replication attempts of Experiment 1 from Hansen and Wänke ( 2010 ). Experiment 1a was a dual-site, in-person replication of the linguistic concreteness effect in the original paper-and-pencil format ( n = 253, n = 246 in analyses). Experiment 1b replicated the study with an online sample ( n = 237, n = 220 in analyses). In Experiment 1a, the effect of concreteness on judgements of truth (Cohen’s d z = 0.08; 95% CI: [–0.03, 0.18]) was smaller than that of the original study. Similarly, in Experiment 1b the effect (Cohen’s d z = 0.11; 95% CI [–0.01, 0.22]) was smaller than that of the original study. Collectively, the pattern of results is inconsistent with that of the original study.

The present article reports the results of a meta-analysis of nine student replication projects of Elliot et al.’s ( 2010 ) findings from Experiment 3, that women were more attracted to photographs of men with red borders (total n = 640). The eight student projects were part of the Collaborative Replication and Education Project (CREP; https://osf.io/wfc6u/ ), a research crowdsourcing project for undergraduate students. All replications were reviewed by experts to ensure high quality data, and were pre-registered prior to data collection. Results of this meta-analysis showed no effect of red on attractiveness ratings for either perceived attractiveness (mean ratings difference = –0.07, 95% CI [–0.31, 0.16]) or sexual attractiveness (mean ratings difference = –0.06, 95% CI [–0.36, 0.24]); this null result held with and without Elliot et al.’s ( 2010 ) data included in analyses. Exploratory analyses examining whether being in a relationship moderated the effect of color on attractiveness ratings also produced null results.

We report the first set of results in a multi-year project to assess the robustness – and the factors promoting robustness – of the adult statistical word segmentation literature. This includes eight total experiments replicating six different experiments. The purpose of these replications is to assess the reproducibility of reported experiments, examine the replicability of their results, and provide more accurate effect size estimates. Reproducibility was mixed, with several papers either lacking crucial details or containing errors in the description of method, making it difficult to ascertain what was done. Replicability was also mixed: although in every instance we confirmed above-chance statistical word segmentation, many theoretically important moderations of that learning failed to replicate. Moreover, learning success was generally much lower than in the original studies. In the General Discussion, we consider whether these differences are due to differences in subject populations, low power in the original studies, or some combination of these and other factors. We also consider whether these findings are likely to generalize to the broader statistical word segmentation literature.

Recent research has focused on the potential negative consequences of belonging to the upper class. The present study attempted to directly replicate previous research examining whether upper-class individuals had more positive attitudes toward greed than lower-class individuals, and whether these attitudes mediated the negative association between social class and unethical behaviour. The current research includes two studies with 317 and 320 participants, from Amazon’s Mechanical Turk and Prolific Academic, respectively. We used the same measures and procedures of the original research. The resulting dataset, and analytic code, are hosted on the Open Science Framework ( Clerke et al., 2017 ). Collectively these datasets provide access to data from over 600 participants pertaining to social class, ethical behaviour, and sociodemographic information, such as obtained education and religious and political orientation. As in the original, we found a significant positive correlation between SES and greed in one of two studies, however the size of the effect was smaller. Contrary to the original, we did not find a significant association between SES and the propensity to lie in a hypothetical salary negotiation.

The present study is a pre-registered replication of a study by Specker et al. ( 2018 ) that tests the hypothesis that brightness of colors is associated with positivity. Our results showed an implicit association between brightness and positivity in both Study 1 and Study 2, however, an explicit association between brightness and positivity was only found in Study 2, thereby replicating 3 out of 4 effects. To investigate these effects in more detail, we present a meta-analysis of both the original and the replication study. This indicated a large effect 1.31 [1.12, 1.51]. In addition, we used meta-analysis to assess potential moderators of the effect, in particular stimulus type (chromatic vs. achromatic) and measure type (implicit vs. explicit). This indicated that the effect is stronger when measured implicitly than when measured explicitly and that the effect is stronger when achromatic stimuli are used. In sum, we take these findings to indicate that there is a strong and replicable association between brightness and positivity. These findings offer researchers interested in the effect concrete tools when designing a study investigating the effect with regard to effect size estimates for power analysis as well as stimulus and measurement design.

The results reported by Kidd and Castano ( 2013 ) indicated that reading a short passage of literary fiction improves theory of mind (ToM) relative to reading popular fiction. However, when we entered Kidd and Castano’s results in a p -curve analysis, it turned out that the evidential value of their findings is low. It is good practice to back up a p -curve analysis of a single paper with an adequately powered direct replication of at least one of the studies in the p -curve analysis. Therefore, we conducted a direct replication of the literary fiction condition and the popular fiction condition from Kidd and Castano’s Experiment 5 to scrutinize the effect of reading literary fiction on ToM. The results of this replication were largely consistent with Kidd and Castano’s original findings. Furthermore, we conducted a small-scale meta-analysis on the findings of the present study, those of Kidd and Castano and those reported in other published direct replications. The meta-analytic effect of reading literary fiction on ToM was small and non-significant but there was considerable heterogeneity between the included studies. The results of the present study and of the small-scale meta-analysis are discussed in the light of reading-times exclusion criteria as well as reliability and validity of ToM measures.