Undergraduate Psychology Students’ Perceptions of Open Research: The Relationship Between Statistics Understanding, Attitudes, and Questionable Research Practices Open Access

All materials and data are available on the Open Science Framework: https://osf.io/ume9k/.

Participants were 267 final-year undergraduate psychology students (232 women, 34 men, and 1 nonbinary) with a mean age of 21.63 years (SD = 4.0) from ten UK universities. Participants were recruited through their universities’ student participant pools or by emails from their programme leaders advertising the study. All participants provided informed consent and received participant pool credit or a chance to win a £20 Amazon voucher. The study was approved by the ethics committees at University of the West of England and University of Cambridge.

Participants followed a link to the survey on Qualtrics. They completed basic demographic questions and then answered a series of questionnaires, which were presented in a randomised order. The survey took approximately 15 minutes to complete, and the data were collected between December 2019 and January 2020.

We collected data on student awareness of OR, student self and lecturer perceptions of QRPs, statistics knowledge and confidence, student statistics and overall grades, student endorsement of QRPs, motivations, general and project-related stress in final year, opinions on significant findings in the final year project, and the quality of their supervisor relationship. The full list of questionnaires is available here: https://osf.io/4qpwb, and details of the measures are described below:

Self and Lecturer Perceptions of QRPs. These questionnaires were adapted from Krishna and Peter (2018) to measure participants’ perceptions and their assumptions about their lecturer’s perceptions of QRPs. Self and Lecturer Perceptions were measured by participants indicating whether they perceived various research practices as “sensible” or “problematic”. From the total of 15 research practices used by Krishna and Peter (2018), such as “Falsifying data” and “Reporting effect sizes”, a subset of ten were utilised, excluding the positive or neutral practices since these were not included in the analysis of the original study. One question pertaining to claiming that results were unaffected by demographic variables was unintentionally omitted. Scores were reversed and summed to create a total Self-Perception score and a total Lecturer-Perception score (each ranging from 10 to 70), with higher scores indicating greater endorsement of QRPs as sensible.

OR Awareness. The OR Awareness questionnaire was also adapted from Krishna and Peter (2018) and consisted of a series of 12 concepts related to OR (e.g., “Replication Crisis.”). Participants indicated whether they recalled learning the specific OR principle during their course by ticking a box or leaving it unticked. Responses were summed to create a total OR Awareness score ranging from 0-12, with higher scores indicating higher awareness of OR concepts.

Statistics Knowledge and Confidence. Participants answered six multiple-choice questions designed by the researchers to assess Statistics Knowledge and Statistics Confidence (e.g., “What does a small p-value indicate? Select all answers that you believe to be TRUE.”). Participants chose one or more options from a number of answers (option choices ranging from three to nine). A Statistics Knowledge score was computed as follows: for each answer choice, the response (or lack of response) was coded as correct or incorrect (correct = correctly ticked or correctly left unticked; incorrect = incorrectly ticked or incorrectly left empty). Each question had a different number of options so, to weight all questions equally, an average mark was calculated for each question (as percentage correct) which was then averaged to create a total knowledge score (ranging from 0 to 1), where higher scores indicated higher levels of statistical knowledge. After each knowledge question, participants were also asked to rate their confidence in their response on a 5-point Likert scale (1 = “very unconfident”, 5 = “very confident”). A Statistics Confidence score was computed by summing participant answers to each question (ranging from scores of 6 to 30), with higher scores indicating higher levels of participants’ confidence in their own answers.

Average Grades, Statistics Grades, and Related Information. Participants were asked questions about their Statistics Marks in their compulsory statistics modules and their Overall/Average Marks in their first and second years by indicating the range most appropriate for their marks (e.g., “40-49%”, “80-100%”; note that on the UK grading scale, a 40% is a pass mark or ‘D’, 50-59% is a ‘C’, 60-69% is a ‘B’, 70-79% is an ‘A’, and 80+% is considered exceptional) in their statistics modules and all modules overall. They were also asked whether they had taken any optional statistics modules; a positive response to this question prompted a question about their marks in these modules or additional statistics training. They were also asked about the methodology of their dissertation project (quantitative, qualitative, or mixed methods).

QRP Endorsement. Four scenarios involving instances of QRPs (data peeking, HARKing, p-hacking, and selective reporting) were displayed to participants to assess their Endorsement of QRPs. An example scenario is as follows: “Your ethics approval has taken longer than expected and you only have a limited amount of time for data collection before the final hand in date for your 3^rd-year research project. Therefore, you decide to check data while they are being collected and terminate the data collection as soon as a significant result is found. How would you best categorise this decision?” Participants rated their views on a 7-point Likert scale (1 = “extremely inappropriate”, 7 = “extremely appropriate”) which was then summed to create a total QRP Endorsement score (ranging from 4 to 28), with higher scores indicating greater QRP endorsement.

Student Motivations. The Development-Demonstration Achievement Goal Questionnaire (Korn & Elliott, 2016) was used to assess participants’ motivations about their projects. Participants answered 12 statements about their Motivations for their final-year projects and rated them on a 7-point Likert scale (1 = “not at all true of me”, 7 = “Extremely true of me”). The items included 6 positive motivations, (e.g., “My focus is to develop my knowledge”), and 6 negative motivations, (e.g., “My aim is to avoid showing incompetence”). These were then summed to compute two total motivation scores (Positive Motivation and Negative Motivation), both ranging from 6 to 42. Higher Positive Motivation scores indicate stronger motivations to develop and demonstrate knowledge and competence, whereas higher Negative Motivation scores indicate stronger motivations to avoid demonstrating a lack of knowledge or competence.

General Stress and Project Stress. The General Stress questionnaire consisted of 12 items adapted from stress-related questions on the NEO-PI-R (Costa & McCrae, 2008), Barratt Impulsiveness Scale (BIS-11; Patton et al., 1995), and California Psychological Inventory (Boer et al., 2008) scales. This scale measured participants’ overall levels of stress experienced during their final year (e.g., “get stressed out easily”). Participants responded using a 7-point Likert scale (1 = “strongly applies to me”, 7 = “strongly does not apply to me”). Negative statements were reverse scored, and scores were summed to create a total General Stress score (ranging from 12 to 84), with higher scores indicating higher levels of stress. Project Stress assessed participants’ perceived stress relating to their project on a 5-point Likert scale (1 = “very stressful”, 5 = “not stressful at all”, then reverse scored). This involved four statements, (e.g. “Choosing the most appropriate statistical analysis method to test your hypothesis”), from which a total Project Stress score was then computed (ranging from 4 to 20) with higher scores indicating greater stress associated with aspects of the project work such as statistical decisions.

Hoping for Significance. This scale assessed to what degree participants believed that their project success was dependent upon finding significant results. Participants were presented with 5 statements, (e.g., “If my results are significant, I will get a higher mark (as write-up will be easier/study is stronger”), which they rated on a 7-point Likert scale (1 = “strongly agree”, 7 = “strongly disagree”, then reverse scored). A total Hoping for Significance score was computed from participants’ answers (ranging from 5 to 35), with higher scores indicating stronger desire to obtain significant results.

Supervisor Relationship. Participants rated their perceived Student-Supervisor Relationship by indicating their agreement with 8 statements, e.g. “I am satisfied with the support provided by my supervisor”, on a 7-point Likert scale (1 = “completely disagree”, 7 = “completely agree”, then reverse scored), to create a total Supervisor Relationship score ranging from 8 to 56, with higher scores indicating a better student/supervisor relationship. Data was also collected about whether they had regular meetings with their supervisor, about their supervisor’s academic rank, and if they had received additional supervision, and if so, by whom.

The total number of survey entries was 329. Of these, 13 were test entries, 15 entries contained no data, 7 did not report a valid university name, and 27 entered demographic information but did not complete any full questionnaires. These 62 entries were excluded from all analyses, and the number of remaining participants was 267. Participants who did not complete a given questionnaire were excluded from the analyses for that questionnaire but included for questionnaires they did complete, and no data from partially completed questionnaires were included. Because the questionnaire order was randomised for each participant, slightly different numbers of participants completed each questionnaire.

Internal reliability of the scales used in the survey (excluding Statistics Marks, Average Grades, and OR Awareness, which consisted of only binary response items) was investigated. Since Cronbach’s alpha is best used to assess multi-item rating scales pertaining to a particular construct (Vaske et al., 2017), the internal reliability of Statistics Knowledge was not assessed since this scale does not measure a construct and instead gives a score similar to a percentage on an exam. All questionnaires met the criteria for high internal reliability (i.e. Cronbach’s α greater than 0.7, Peterson, 1994) except for Hoping for Significance at 0.58, but this was still included in the analyses.

Exploratory analyses were conducted to assess the relationships between all variables using Pearson’s r correlations for normally distributed continuous data and Spearman’s 𝞺 for ordinal data such as student grades, which were reported in grade bands rather than numerically. Statistics are presented in terms of both traditional frequentist p-values as well as Bayes factors to evaluate strength of evidence (Dienes, 2011). There were no relevant previous data to guide an informed prior, and thus Bayesian analyses were computed using the default Jeffreys-Zellner-Siow (JZS) prior with a Cauchy distribution (r = 0.707; Rouder et al., 2009) in JASP (JASP Team, 2020). The JZS prior is a non-informative default and objective prior designed to minimise assumptions about the expected effect size. The alpha for our correlations was set at p < 0.001 as a conservative p-value, and effect sizes less than 0.2 were dismissed as noise (see Orben & Lakens, 2020, for a discussion of the “crud” factor). We considered BF₁₀ > 6 as evidence for the alternative hypotheses in comparison to the null hypotheses and BF₁₀ < 0.17 as evidence for the null hypotheses in comparison to the alternative hypotheses (e.g., Etz & Vandekerckhove, 2016). Results are shown in Figure 1, and statistically significant (p < 0.001, r > 0.2, and BF₁₀ > 6) relationships are outlined below; p-values are not reported in text as all p’s < 0.001. In the event of disagreements between the p-values and Bayes factors, we only considered the results meaningful if both statistics exceeded the specified thresholds.

OR Awareness and QRPs. Greater OR awareness was moderately associated with higher statistics confidence (r = 0.37, BF₁₀ = 980,945.88) and average grades (𝞺 = 0.26, BF₁₀ = 444.54). OR awareness also predicted lower levels of student endorsement of QRPs (r = -0.33, BF₁₀ = 32,636.88), less tendency to hope for significance (r = 0.29, BF₁₀ = 2,790.78), and poorer self and lecturer QRP perceptions of QRPs (r = -0.28, BF₁₀ = 1,111.64; r = -0.32, BF₁₀ = 26,378.96). Likewise, higher student endorsement of QRPs was strongly associated with more positive self (r = 0.58, BF₁₀ = 1.06×10¹⁸) and lecturer perceptions (r = 0.56, BF₁₀ = 1.21×10¹⁷) of QRPs and also related to an increased hope for significant findings in their final project (r = 0.28, BF₁₀ = 540.94). There was a strong positive relationship between self and lecturer perceptions of QRPs (r = 0.78, BF₁₀ = 8.26×10⁴²), and higher student self perceptions of QRPs were associated with a higher tendency to hope for significance (r = 0.27, BF₁₀ = 354.70) and negatively related to their positive motivation (r = -0.34, BF₁₀ = 57,603.00).

Bayes factors provided evidence for no relationship between OR awareness and negative motivation (r = 0.04, BF₁₀ = 0.10), general stress (r = -0.08, BF₁₀ = 0.16), and supervisor relationship (r = 0.07, BF₁₀ = 0.14). There was also no relationship between QRP endorsement and statistics grades (𝞺 = -0.01, BF₁₀ = 0.08), negative motivation (r = -0.04, BF₁₀ = 0.10), general stress (r = 0,05, BF₁₀ = 0.11), and project stress (r = 0.09, BF₁₀ = 0.10). Both lecturer perceptions and self perceptions had no relationship with statistics grades (𝞺 = -0.04, BF₁₀ = 0.12; 𝞺 = -0.08, BF₁₀ = 0.13), general stress (r = 0.06, BF₁₀ = 0.13; r = 0.06, BF₁₀ = 0.15), and project stress (r = 0.05, BF₁₀ = 0.12; r = 0.78, BF₁₀ = 0.16).

Statistics Knowledge and Confidence. Statistics knowledge did not have any significant relationships with any other variables, and Bayes factors evidenced no relationship with general stress (r = 0.06, BF₁₀ = 0.13) and supervisor relationship (r = 0.07, BF₁₀ = 0.14). However, statistics confidence was negatively correlated with project stress (r = -0.29, BF₁₀ = 840.47). Bayes factors also provided evidence for no relationship between statistics confidence and negative motivation (r = -0.002, BF₁₀ = 0.09) or supervisor relationship (r = -0.01, BF₁₀ = 0.10).

Student Grades, Motivations, and Stress. Student statistics grades were strongly positively correlated with their average grades (𝞺 = 0.70, BF₁₀ = 6.12×10²⁹). Students’ positive and negative motivations were related to each other (r = 0.54, BF₁₀ = 2.09×10¹⁶), and general stress and project stress were positively associated with each other (r = 0.30, BF₁₀ = 3,294.24). Additionally, lower general stress (r = 0.24, BF₁₀ = 57.57) was correlated with a better supervisor relationship.

Bayes factors provided evidence for no relationship between both statistics/average grades and positive motivation (𝞺 = 0.05, BF₁₀ = 0.10; 𝞺 = 0.003, BF₁₀ = 0.10) as well as negative motivation (𝞺 = -0.04, BF₁₀ = 0.11; 𝞺 = -0.06, BF₁₀ = 0.10). There was also no relationship between statistics grades and general stress (𝞺 = -0.03, BF₁₀ = 0.12), positive motivation and general stress (r = -0.07, BF₁₀ = 0.15), or positive motivation and project stress (r = -0.07, BF₁₀ = 0.13).

The results of Study 1 revealed strong relationships between several variables measuring undergraduate students’ perceptions of QRPs. Our measure examining QRP endorsement had a strong relationship with how students perceive QRPs and how they believe their lecturers to perceive QRPs, such that lower levels of QRP endorsement were associated with believing QRPs to be problematic and believing that one’s lecturers see QRPs as problematic as well. More awareness of OR (i.e., exposure to OR in their studies) was also moderately associated with less favourable perceptions of QRPs in both students and lecturers, decreased endorsement of QRPs, and a reduced likelihood of hoping for significant findings.

There were further relationships between awareness of OR and academic performance such that higher levels of OR awareness had a moderate relationship with higher grades on the course overall and even participants’ confidence in their own statistics aptitude. Higher levels of statistics confidence were associated with reduced project stress, and reduced general stress was related to better student-supervisor relationships. Higher positive motivation was also associated with less favourable views of QRPs within the students as well as their perceptions of their lecturers.

This exploratory study allowed for a broad survey of final-year students’ experiences with research and identified several interesting relationships between awareness of OR, endorsement of QRPs, academic performance, and statistics confidence. In Study 2, we aimed to conceptually replicate these relationships in a broader sample including psychology students on all years of their programme. We also aimed to further interrogate the relationship between these factors and other experiences of learning statistics, including statistics attitudes and anxiety. To do so, we partnered with the multi-year “Study of Open Research Methods” (STORM; Gilligan-Lee et al., under review), which aims to assess the attitudes, understanding, and awareness of OR practices in university students in successive years. In Study 2, we collected data on students’ statistics anxiety and understanding in a single year alongside the STORM questionnaire.

The planned analyses were preregistered (https://osf.io/3yx7j), and all materials and data are available online (https://osf.io/ume9k/).

A series of a priori power analyses were conducted using G*Power 3.0 (Faul et al., 2007) to determine the minimum sample size required. To obtain 80% power with an alpha of 0.05, 90 participants would be required to achieve equivalent effect sizes to Krishna and Peter (2018; f² = 0.19), who investigated similar constructs to our proposed hypotheses. For the planned hierarchical multiple regressions, we followed Tabachnick and Fidell’s (2014) suggestion of N ≥ 50 + 8K, where K is the number of predictors, to yield a required sample of 122. For the correlations, to achieve a small effect size (r = 0.20) with 80% power and our adjusted 0.001 alpha, we required at least 419 participants. We used the same recruitment methods as STORM (Gilligan-Lee et al., under review) had used in prior years, and as STORM typically recruited approximately 900 participants, we anticipated that we would reach our requirement of at least 419 participants.

Participants were recruited from the psychology participant pools at University of the West of England, University of Bristol, Aston University, and University of Surrey and received course credits for completing the study. Participants were also recruited from other UK universities by contacting programme leaders of all UK undergraduate psychology programmes and encouraging them to advertise the study to their students for a chance to win a £20 Amazon voucher for their participation. Given these recruitment methods, we had little control over the number of respondents, and thus we based our stopping rule on date: per our preregistration, we collected data until June 30, 2022. A total of 695 participants from 28 universities completed the questionnaire. The sample consisted of 573 women, 108 men, 11 nonbinary, and 3 people of another gender, with a mean age of 20.87 years (SD = 5.37). There were 334 participants in the first year of their studies, 194 in the second year, 30 in the third year, and 137 did not indicate their year of study. All participants provided informed consent, and the project was approved by the Ethics Committees at University of the West of England.

Participants followed a link to the survey on Qualtrics. They completed basic demographic questions and then a series of questionnaires, which included measures related to students’ perceptions of OR and QRPs as well as their anxiety, attitudes, and understandings related to statistics. First, the scales measuring conceptual perceptions, situational perceptions, awareness and experience with OR (STORM questionnaire measures), self perceptions of QRPs, and general anxiety were presented in a random order, and then the remaining questionnaires were presented in a random order. The questionnaires took approximately 30 minutes to complete. The data were collected between March 2022 and June 2022.

The questionnaires administered in Study 2 consisted of several scales previously administered in Study 1: Self Perceptions of QRPs (Krishna & Peter, 2018), Statistics Knowledge, Statistics Confidence, and Student Grades (Overall Grades and Statistics Grades). We also included four questionnaires on perceptions of OR developed as part of STORM (Gilligan-Lee et al., under review), established questionnaires about statistics anxiety and attitudes (STARS [Cruise et al., 1985]; SATS-28 [Schau et al., 1995]), a questionnaire about data interpretation (from the science section of the American College Testing [ACT]), a questionnaire about generalised anxiety (STAI-T), and questions about students’ attitudes toward learning. The details of these measures are described below

Situational Perceptions of OR. Similar to the QRP Endorsement scale from Study 1, the Situational Perceptions of OR scale (Gilligan-Lee et al., under review) assessed participants’ views about the appropriate decisions in particular research scenarios. An example scenario is as follows: “Sadia completed a study showing that eating cabbage made people less happy. She wrote a paper and made it available online, before it was reviewed by a journal. She also put her raw data online (anonymised) and anyone could access it. A) Sadia should not have shared her paper until an academic journal had reviewed it. B) It is ok for Sadia to make her data available online.” A series of four scenarios was presented, after which participants rated two statements on a 5-point Likert scale (0 = “strongly disagree”, 4 = “strongly agree”). Items articulating a QRP were reverse-scored, and participants’ answers were summed to create a total Situational Perception score (ranging from 0 to 32), with higher scores indicating greater agreement with responsible research practices.

Conceptual Perceptions of OR. A second scale developed by Gilligan-Lee et al. (under review) was used to assess participants’ perceptions of 8 key concepts of OR, including replication, statistical power, preregistration, preprints, open research materials, open education materials, open access journals, and open data. Participants were presented with 16 statements (two for each concept, e.g., “Before they start to collect data, researchers should state their plans for the study in a time-stamped file online”) and rated them on a 5-point Likert scale (-2 = “strongly disagree”, 2 = “strongly agree”). Items articulating a QRP were reverse-scored, and participants’ responses were summed to create a total Conceptual Perception score (ranging from -32 to +32), with higher scores indicating more positive views of OR practices.

Awareness of OR. This scale was similar to the OR Awareness scale administered in Study 1 (Krishna & Peter, 2018) and was developed by Gilligan-Lee et al. (under review). As in Study 1, participants indicated whether they had heard of specific OR practices (previously stated in Conceptual Perceptions of OR) at any time during their degree, but there were only 8 practices rather than 12 as in Study 1. Positive responses were summed to give participants a total OR Awareness score ranging from 0 to 8, with a higher score indicating a greater level of OR awareness.

Experience with OR. Using the same eight practices from the Awareness of OR scale above, participants’ experience with OR was measured using an 8-item scale (Gilligan-Lee et al., under review) where participants indicated whether or not they had used OR practices. Responses were summed for a total OR Experience score ranging from 0 to 8, where a larger score suggested experience having used more OR practices.

Self Perceptions of QRPs. As in Study 1, questions adapted by Krishna and Peter (2018) were used to assess student Self Perceptions of QRPs. One item (claiming results are unaffected by demographic variables) was unintentionally omitted from this scale in Study 1 but was included here. Additionally, in Study 1, participants rated the items on a 1-7 scale to be in line with the other scales, but here, we used a 1-5 scale to mirror the original Krishna and Peter (2018) questionnaire. There were 11 negative practices in which participants rated on a 5-point Likert scale (1 = “sensible”, 5 = “problematic”); as in the original Krishna and Peter (2018) study, the positive and neutral practices (four items) were excluded from the analyses, and scores were reversed. The Self Perception of QRPs score ranged from 11 to 55, with higher scores indicating greater endorsement of QRPs as sensible.

Statistics Anxiety and Attitudes (STARS Section A and B). The Statistics Anxiety Rating Scale (STARS) questionnaire developed by Cruise et al. (1985) was used to measure Statistics Anxiety and Attitudes. This questionnaire consists of 51 items split into two subscales: the first 23 questions measure Anxiety, and the remaining 28 statements measure Attitudes. Both subscales were rated on a 5-point Likert scale (Section A: 1 = “no anxiety”, 5 = “a great deal of anxiety”; Section B: 1 = “strongly disagree”, 5 = “strongly agree”). Some items in this scale were adapted by Terry and Field (2021) from the original items by Cruise et al. (1985) to better represent more modern experiences of students; for example “Asking someone in the computer centre for help in understanding a printout” was changed to “Asking someone in the computer lab for help in understanding statistical output”. Total scores were computed for Statistics Anxiety (STARS-A; ranging from 23 to 115) and Statistics Attitude (STARS-B; ranging from 28 to 140), with higher scores indicating greater anxiety levels or more positive attitudes.

Statistics Attitudes (SATS-28). The Survey of Attitudes Toward Statistics (SATS-28; Schau et al., 1995) was also used to assess student’s attitudes towards statistics. This questionnaire included 28 statements, for example, “I like statistics” and “I am scared by statistics.”, which participants rated on a 7-point Likert scale (1 = “completely disagree”, 7 = “completely agree”, where the negative items were reverse scored). A total Statistics Attitude score ranging from 28 to 196 was computed, with higher scores indicating more positive attitudes towards statistics.

Statistics Knowledge and Confidence. Statistics Knowledge and Statistics Confidence were measured using the questionnaires outlined in Study 1.

Statistics Interpretation. Questions from the science section of the ACT, a test in the US designed to measure student readiness to attend university, was used to assess student interpretation of statistics and science. Participants were asked to read a passage that included two tables of information and a graph and answer a number of multiple-choice questions using the information provided. There were five multiple-choice questions, each with four possible answers where only one was correct. Correct answers were summed to create a total Statistics Interpretation score ranging from 0 to 5, with higher marks indicating better statistics interpretation abilities.

Statistics Grades, Average Grades, and Statistics Competence. Participant grades were also assessed for their compulsory statistics modules and average overall grades during each year of their course (years one to three and also year-four for Scottish students). (Note that this differs from Study 1, in which only final-year students were included; here, all undergraduate levels were eligible to participate, and participants reported on their marks obtained thus far.) Student Grades were measured using the same questions outlined in Study 1. The questions were displayed in relation to the years of study completed and appropriate for the country’s system as indicated by the demographic questions. Participants also answered an additional question that asked them to rate their self-perceived Statistics Competence compared to the rest of their cohort on a Likert scale (1 = “below average”, 7 = “above average”).

Generalised Anxiety. The Trait component of the STAI-T (Spielberger, 1983) was used to assess participants’ trait anxiety to control for Generalised Anxiety in the STARS questionnaire. This is a 20-item scale on which participants rated how much they perceived statements (e.g., “I feel nervous and restless;” “I feel secure”) to be similar to themselves on a 4-point Likert scale (1 = “not at all”, 4 = “very much so”, with the positive items reverse scored), giving a total General Anxiety score ranging from 20 to 80, with higher scores indicating greater levels of anxiety.

Student Attitudes Toward Learning. Questions developed in collaboration with the Framework for Open and Reproducible Research Training Community (FORRT; https://forrt.org) measured Students’ Attitudes Toward Learning about OR. These included a number of questions (e.g., “For me, it is more useful to learn the facts of science than how science is made”), presented on a 5-point scale (strongly disagree to strongly agree). Items demonstrating a lack of interest in learning about OR were reverse scored, and a composite Attitudes Toward Learning Score was computed, with higher scores indicating more positive attitudes towards learning about OR (ranging from 19 to 101).

Data collection and analyses deviated from plans outlined in the preregistration as follows:

1. Data were also collected from the participant pool at Aston University, as well as the other university participant pools mentioned in the preregistration. This has no bearing on the results or interpretations.

2. The preregistered plan was that participants who spent less than 12 minutes completing the questionnaire would be excluded. Participants who completed the full questionnaire in less than 12 minutes were excluded; however, participants who only partially completed the questionnaires were not excluded based on the same time restriction. We were less concerned that participants who did not complete all questionnaires would be speeding through the questionnaires (without engaging with the content) for a participation credit, and examination of their participation durations appeared reasonably in line with the amount of content they completed; therefore, data from incomplete participants’ fully completed questionnaires were included, regardless of their participation duration, as per Study 1.

3. Statistics Knowledge scores were computed differently than outlined in the preregistration but were calculated the same for both Study 1 and 2. This was done to equally weight the questions because they had different numbers of answer options. An average of participants’ percentage of correct answers for each question was used to compute a total Statistics Knowledge score ranging from 0 (where no questions were answered correctly) to 1 (where all questions were answered correctly).

4. Our planned regression models included a third step in which we would examine interactions between OR Awareness, General Anxiety, and our other predictor variables; however, this was based on the assumption that OR Awareness and General Anxiety would significantly predict our outcome variables. In the first regression model, neither predicted our outcome variable, so we did not pursue the third step with the interactions. In the latter two regression models, only OR Awareness predicted our outcome variables, but the interactions between OR Awareness and the other variables were not significant predictors of our outcome variables. Therefore, there were no relevant analyses to include for our planned third step, and this has been omitted from all regression models.

Twenty-five participants were excluded from analyses because they took less than 12 minutes to complete the survey, which was the minimum time cap as outlined in the preregistration. Additionally, 66 participants were excluded for failing to complete any questionnaires, and four participants were excluded as they did not provide a university name to indicate they were a current psychology student. In total, 95 entries were removed, and the number of remaining participants included in the analysis was 619, thus exceeding the planned sample size target. (All participants who completed any given questionnaire were included in the relevant analyses, but no data from partially completed questionnaires were included. The questionnaire order was randomised for each participant, so slightly different numbers of participants completed each questionnaire.)

As outlined in Study 1, the internal reliability of some scales (Statistics Knowledge, Awareness of OR, Experience with OR, Statistics Marks, Average Marks, Statistics Competence, and Statistics Interpretation) was not assessed in cases where they consisted of only binary response items or did not measure a construct but rather provide a score similar to a percentage on an exam. For the remaining questionnaires, all but one questionnaire met the criteria for high internal reliability (i.e. Cronbach’s α greater than 0.7, [Peterson, 1994]; see Table 2). One scale, Situational Perceptions of OR, was below the acceptable threshold (0.56), perhaps because this questionnaire can be considered more similar to an exam score rather than measuring a construct; however, this scale was still utilised in the analyses.

Pearson’s r correlations were used to assess the relationship between normally distributed continuous data, and Spearman’s 𝞺 rank correlations were used for ordinal data including students’ grades, which were reported in grade bands rather than numerically. As in Study 1, both p-values and Bayesian statistics are reported. As per our preregistration, the alpha for correlations was set at p < 0.001 with effect sizes less than 0.2 dismissed as noise, BF₁₀ > 6 were regarded as evidence for the alternative hypotheses in comparison to the null hypotheses, and BF₁₀ < 0.17 were considered as evidence for the null hypothesis in comparison to the alternative hypothesis (e.g. Etz & Vandekerckhove, 2016). Results are displayed in Figure 2. Significant correlations (p < 0.001, , r > 0.2, and BF₁₀ > 6) between variables are outlined below; p-values are not reported in text as all p’s < 0.001. In the event of disagreements between the p-values and Bayes factors, we only considered the results meaningful if both figures exceeded the specified thresholds.

Perceptions and Awareness of OR and QRPs. Conceptual and situational perceptions of OR were strongly positively correlated with each other (r = 0.59, BF₁₀ = 8.96×10⁵⁴), and more positive conceptual perceptions of OR were also moderately related to less of a tendency to endorse QRPs (r = -0.31, BF₁₀ = 1.97×10¹¹). Positive situational perceptions of OR were weakly predicted better statistics attitudes (r = 0.21, BF₁₀ = 25070.62), improved statistics interpretation (r = 0.22, BF₁₀ = 21579.99), and more negative self perceptions of QRPs (r = -0.31, BF₁₀ = 1.97×10¹¹). Student self-endorsement of QRPs was also associated with worse attitudes toward statistics (r = -0.22, BF₁₀ = 34755.63) and learning science (r = -0.22, BF₁₀ = 41966.64).

Bayes Factors provided evidence for no relationship between conceptual OR perceptions and OR awareness (r = 0.06, BF₁₀ = 0.16), general anxiety (r = 0.004, BF₁₀ = 0.05), statistics grades (𝞺 = 0.07, BF₁₀ = 0.13), and average grades(𝞺 = 0.02, BF₁₀ = 0.05), as well as between situational perceptions and general anxiety (r = 0.01, BF₁₀ = 0.05) and average grades(𝞺 = 0.05, BF₁₀ = 0.10), and between self-perceptions of QRPs and experience with OR (r = 0.03, BF₁₀ = 0.07), general anxiety (r = -0.003, BF₁₀ = 0.05), statistics anxiety (r = 0.03, BF₁₀ = 0.07), statistics attitudes (r = -0.06, BF₁₀ = 0.015), statistics confidence (r = -0.05, BF₁₀ = 0.12), statistics grades (𝞺 = -0.07, BF₁₀ = 0.12), average grades (𝞺 = -0.002, BF₁₀ = 0.06), and statistics competence (r = 0.04, BF₁₀ = 0.09). Additionally, no relationship was evidenced between awareness of OR and experience with OR (r = -2.37×10^-4, BF₁₀ = 0.05), general anxiety (r = -0.02, BF₁₀ = 0.06), statistics anxiety (r = -0.001, BF₁₀ = 0.05), statistics attitudes (r = 0.05, BF₁₀ = 0.09; r = 0.02, BF₁₀ = 0.06), statistics interpretation (r = 0.05, BF₁₀ = 0.10), statistics competence (r = -0.01, BF₁₀ = 0.05), and between experience of OR and general anxiety (r = -0.03, BF₁₀ = 0.07), statistics anxiety (r = 0.001, BF₁₀ = 0.06), statistics attitudes (r = -0.04, BF₁₀ = 0.08; r = -0.02, BF₁₀ = 0.06), statistics interpretation (r = -0.01, BF₁₀ = 0.05), and student attitudes of learning (r = 0.04, BF₁₀ = 0.08).

Statistics Anxiety and Attitudes. Statistics anxiety was moderately positively correlated with general anxiety (r = 0.42, BF₁₀ = 3.31×10²²) and was associated with worse statistics attitudes (r = -0.48, BF₁₀ = 1.211×10³¹; r = -0.54; BF₁₀ = 2.04×10⁴⁰), confidence (r = -0.39, BF₁₀ = 8.67×10¹⁷), student self-rated competence (r = -0.44, BF₁₀ = 2.21×10²⁴), and attitudes toward learning (r = -0.21, BF₁₀ = 18447.53). General anxiety also predicted poorer statistics attitudes (r = -0.30, BF₁₀ = 9.97×10⁹), confidence (r = -0.21, BF₁₀ = 15758.86), and student self-rated competence (r = -0.31, BF₁₀ = 1.78×10¹¹). Additionally, better student attitudes of statistics were correlated with higher self-competence rating (r = 0.39, BF₁₀ = 5.69×10¹⁷; r = 0.47, BF₁₀ = 2.21×10²⁴), more positive attitudes toward learning (r = 0.39, BF₁₀ = 6.10×10¹⁷; 0.36, BF₁₀ = 6.63×10¹⁴), more statistics confidence (r = 0.31, BF₁₀ = 1.20×10¹¹; r = 0.34, BF₁₀ = 1.06×10¹³) and a higher aptitude for statistics interpretation (r = 0.25, BF₁₀ = 3.17×10⁶). Bayes Factors provide evidence for no relationship between statistics anxiety and statistics grades (𝞺 = -0.09, BF₁₀ = 0.13) and average grades (𝞺 = -0.02, BF₁₀ = 0.05), and for statistics attitudes and average grades (r = 0.02, BF₁₀ = 0.05; r = 0.01, BF₁₀ = 0.05). General anxiety also was shown to have no relationship with statistics interpretation (r = -0.02, BF₁₀ = 0.06) and statistics grades (r = -0.09, BF₁₀ = 0.16).

Statistics Understanding and Confidence. Statistics knowledge did not have any significant relationships with the other variables, and Bayes factors provide evidence for no relationship with general stress (r = -0.04, BF₁₀ = 0.08), student self-competence (r = 0.05, BF₁₀ = 0.11), and self- and lecturer-perceptions (r = 0.01, BF₁₀ = 0.05; r = 0.02, BF₁₀ = 0.06). Higher levels of statistics confidence were associated with better statistics interpretation (r = 0.22, BF₁₀ = 14235.83), higher statistics grades (𝞺 = 0.24, BF₁₀ = 33329.31), more positive statistics attitudes (r = 0.31, BF₁₀ = 1.20×10¹⁵; r = 0.34; BF₁₀ = 1.06×10¹¹), increased self-competence (r = 0.35, BF₁₀ = 8.28×10¹³), and lower levels of statistics anxiety (r = -0.39, BF₁₀ = 8.67×10¹) and general anxiety (r = -0.21, BF₁₀ = 15758.86).

Student Grades and Competence. Student statistics grades were strongly positively correlated with their average grades (𝞺 = 0.77, BF₁₀ = 2.91×10¹¹²). Bayes factors also provide evidence for no relationship between average grades and statistics competence (𝞺 = 0.03, BF₁₀ = 0.06).

We used hierarchical regressions to investigate whether statistics anxiety, attitudes, knowledge, confidence, and interpretation as well as students’ average and statistics grades predict perceptions of OR and QRPs (as measured by Conceptual Perceptions of OR, Situational Perceptions of OR and Self Perception scales). As per our preregistration, three regression models were used, one for each outcome variable, each model containing the same steps. During step one, Awareness of OR and Generalised Anxiety were entered to control for knowledge of OR and general anxiety (rather than statistics anxiety specifically). Then in step two, the statistics anxiety, attitudes, knowledge, confidence, and interpretation, and student marks scales were entered. For all regression models, our alpha was set at the p < 0.05 level.

Regression Model One: Conceptual Perceptions of OR. All predictors entered can be seen in Table 3. The final model included Statistics Attitudes (STARS-B) and Statistics Interpretation and accounted for 4.8% of the variance in Conceptual Perceptions of OR; see Table 4.

Regression Model Two: Situational Perceptions of Open Research. All predictors entered can be seen in Table 5. In the final model, OR Awareness was included in Step 1 and accounted for 3% of the variability in Situational Perceptions of OR. Between Step 1 and 2, ΔR² = 0.09, and significant predictors included Statistics Knowledge and Statistics Interpretation; see Table 6.

Regression Model Three: Self Perceptions of QRPs. All predictors entered can be seen in Table 7. In the final model, OR Awareness was included in Step 1 and accounted for 1% of the variance in Self Perceptions of QRPs. Between Step 1 and 2, ΔR² = 0.08, and significant predictors included both measures of Statistics Attitudes (SAT-28 and STARS-B); see Table 8.

There was overlap between several of our measures between Study 1 and Study 2, and Study 2 served as a conceptual replication for these measures but with a full undergraduate population rather than only final-year students as in Study 1. We considered the findings from Study 2 to conceptually replicate those from Study 1 if the relationships met our inference criteria and were in the same direction.

The Situational Perceptions of OR and Conceptual Perceptions of OR scales used in Study 2 were similar to the QRP Endorsement scale from Study 1 in that they measured participants’ views on research decisions. (Note that higher scores on Situational/Conceptual Perceptions of OR indicate higher endorsement of OR whereas higher scores on QRP Endorsement from Study 1 indicate the opposite – higher scores indicate higher endorsement of QRPs). As in Study 1, higher endorsement of OR (akin to lower endorsement of QRPs) in research decisions was associated with less favourable views of QRPs. We also found a similar relationship between statistics confidence and grades, such that higher levels of confidence were associated with better academic performance. However, we did not replicate the Study 1 relationships between OR awareness and views on QRPs, potentially because we sampled a wider selection of students in Study 2, many in their first year of study, with limited exposure to OR on their programmes thus far.

In Study 2, we also identified relationships between perceptions of OR, attitudes toward statistics, and statistics interpretation abilities, such that more positive views of OR predicted more positive statistics attitudes and abilities. Furthermore, higher levels of statistics anxiety were associated with worse attitudes toward statistics and lower levels of statistics confidence and competence. Levels of general anxiety were tied to levels of statistics anxiety and overlapped with these other factors. Students’ attitudes toward learning about OR concepts were negatively associated with their perceptions of QRPs and their levels of statistics anxiety but positively associated with their attitudes toward statistics. Finally, we found that statistics attitudes and statistics interpretation predicted participants’ conceptual perceptions of OR. When controlling for OR awareness, statistics knowledge and interpretation predicted participants’ situational perceptions of OR, and statistics attitudes predicted participants’ perceptions of QRPs.

We aimed to investigate how perceptions of OR and QRPs are related to psychology students’ attitudes toward and understanding of statistics and experiences with research. Among final-year psychology students in Study 1, awareness of OR practices was associated with higher levels of confidence and better academic performance. Greater OR awareness also predicted less favourable views or QRPs, and students’ own views on research practices were also aligned with the views they perceived their lecturers to hold. In the expanded sample of all years of psychology students in Study 2, we conceptually replicated some of the relationships in Study 1 and identified additional relationships between perceptions of OR and statistics such that positive views of OR were associated with better attitudes toward statistics and statistics abilities. Regression analyses indicated that conceptual perceptions of OR were predicted by statistics attitudes and interpretation, that situational perceptions of OR were predicted by OR awareness and statistics knowledge and interpretation, and that perceptions of QRPs were predicted by statistics attitudes.

Research has predominantly focused on established academics and the factors that drive them to use QRPs, such as publication bias, an incentive culture focused on statistical significance, quantity over quality, and career incentives (Bruton et al., 2020; Doyle & Cuthill, 2015; Janke et al., 2019; Pennington, 2023; Smaldino & McElreath, 2016). However, students do not encounter these same influences as professional researchers to the same extent because they are largely unaffected by publication pressures and incentives to bolster their careers, and they often have limited research experience (Bruton et al., 2020; Krishna & Peter, 2018). Therefore, other factors may be related to students’ use of QRPs, such as their understanding of OR and attitudes toward statistics, confidence, academic success, and views of their lecturers’ perceptions of research, which we explored here.

Like Krishna and Peter (2018), we found a strong positive relationship between student perceptions and their views of their lecturer’s perceptions of QRPs. Students are first exposed to research during their undergraduate degree, and their individual learning environments are likely to impact their approach to research (Gigerenzer, 2018). Whether they are taught about OR on their course influences their own approach to research (Pownall, Azevedo, et al., 2023), and we found that, in final-year students, OR awareness is related to their perceptions of QRPs, their academic performance, and their statistics confidence. We did not find the same relationship with OR awareness in our expanded sample of all undergraduate students; in Study 2, only 4% of our sample were final-year students, and Bayesian analysis indicated evidence for no relationships between OR awareness and many of our factors of interest. These primarily first- and second-year undergraduate students perhaps did not yet have enough exposure to OR or research in general for this relationship to be apparent.

Stress in students is well documented and linked to mental health problems and decreased academic performance (Bekkouche et al., 2022; Khan et al., 2013), and it is possible that students who are particularly stressed about their project may see QRPs as shortcuts to achieving a successful outcome (e.g., grades). Our findings that believing significant results would be seen as more favourable (Hoping for Significance measure) was associated with more positive views of QRPs is in line with the idea that some students may engage with QRPs in the hopes of achieving significant results (see Pownall, Pennington, et al., 2023).

Student motivations in their studies are not yet well understood, although some research suggests students are particularly goal oriented, seeing their education as a way to reach their aims rather than internally rewarding in itself (Nilsson & Warrén Stomberg, 2008; Ryan & Deci, 2020). Students may still be pressured by academic culture to endorse and use QRPs, but their motivations likely differ; academics may engage in QRPs to aid the publication of their research whereas students may be instead driven by their desires to achieve good grades (Moran et al., 2022). However, we also found that higher student grades were associated with more positive perceptions of OR, so an understanding of responsible research conduct rather than engaging in QRPs may be a better route to academic success. If the pressures associated with a ‘culture of significance’ were no longer reinforced in academia, student perceptions of QRPs and OR may change (Moran et al., 2022; Stürmer et al., 2017). This could also be achieved through the teaching of the replication crisis and the associated scientific reforms of open research (see Pennington, 2023), perhaps as a mandatory component of undergraduate education (see Pownall, Azevedo, et al., 2023 for a review, but also see Thibault et al., 2023 for alternative views on whether open research is a critical part of undergraduate education and the challenges associated with fitting in the required content).

The pressures of academic achievement and how they are related to students’ views of QRPs may also be seen in their approaches to studying research methods and statistics and the anxiety they experience about statistics. Statistics anxiety has many detrimental effects on students, such as loss of academic buoyancy and performance and an increase in their procrastination (Dunn, 2014; Onwuegbuzie, 2004; Paechter et al., 2017; Putwain & Daly, 2013). As such, we predicted that statistics anxiety would be negatively associated with perceptions of OR, since it may increase the likelihood of students feeling pressured during their project and make them more likely to endorse QRPs. In our wider survey of undergraduate psychology students across all years (Study 2), our results show that neither perceptions of OR nor QRPs were associated with statistics anxiety as we had hypothesised, and Bayesian analysis revealed evidence for there being no relationship between statistics anxiety and QRP endorsement.

However, we did find that students’ interest in and openness to exploring OR research concepts (Attitudes Toward Learning) were negatively associated with their perceptions of QRPs and statistics anxiety but positively associated with better attitudes toward statistics. Students’ anxiety and attitudes toward statistics were inversely related to each other such that higher levels of statistics anxiety were associated with worse statistics attitudes, and these measures were also linked with a number of other factors we investigated. Statistics anxiety was negatively associated with statistics confidence and competence, whereas better attitudes toward statistics were related to higher levels of confidence and competence. Statistics attitudes were also moderately negatively associated with perceptions of QRPs and positively correlated with situational perceptions, as well as with statistics interpretation. Because ‘delusions’ and false beliefs of statistics might influence students’ use of QRPs (Gigerenzer, 2018), good statistics knowledge and competence act as a protective factor, and this in turn may foster higher endorsement of OR practices.

Perhaps surprisingly, however, there were no relationships between statistics knowledge and our other variables measures of interest, with Bayesian analyses indicating evidence for there being no relationship between statistics knowledge and OR awareness, statistics competence, and attitudes toward learning. This may be driven by the nature of the questions in the statistics knowledge questionnaire: the questions required a rather nuanced understanding of statistical concepts, such as the meaning of a p-value, which many professional researchers get wrong themselves (Gigerenzer, 2018). Given that the students in the sample may have been taught incorrect ideas about these concepts, their performance on this questionnaire may not be predictive of other factors if strong students were simply responding with the incorrect information they have been taught. Our measure of statistics interpretation may have allowed for a better assessment of students’ statistical prowess, and this measure was associated with students’ perceptions of OR as well as confidence and attitudes toward statistics.

Finally, we examined how our various measures predict views of OR and QRPs when controlling for awareness of OR and general anxiety. Because different psychology programmes contain different amounts of OR-related content, some of our participants may have had very limited exposure to OR concepts. Therefore, we controlled for OR awareness as well as for general anxiety to examine the impacts of statistics attitudes, knowledge, and anxiety-related factors on student perceptions of OR. OR awareness did not predict conceptual perceptions of OR but did predict situational perceptions of OR and perceptions of QRPs. Statistics interpretation was a significant predictor for both conceptual perceptions of OR and situational perceptions of OR, and statistics attitudes predicted both situational perceptions of OR and perceptions of QRPs. Conceptual perceptions of OR and situational perceptions of OR are similar constructs that both tap into how students view various QRP- or OR-related scenarios, where conceptual perceptions are presented in terms of a general concept (e.g., should researchers share their data?) and situational perceptions are presented in terms of a particular scenario (e.g., Elaine has put her data online; should she have done this before her manuscript was reviewed?). The situational perceptions questionnaire could, in effect, be considered more nuanced and require a more advanced understanding of OR, and thus factors such as awareness of OR and statistics knowledge also predicted participants’ situational perceptions of OR.

Our results highlight important relationships between OR and student success and academic experiences. Research is starting to explore the benefits of student education in OR, and competence and experience in academic research can be improved by incorporating OR into students’ teaching (Pennington, 2023; Pownall, Pennington, et al., 2023), and although our work is correlational, our results are in line with related research that has identified a causal link between teaching OR and academic understanding (e.g., Pownall, Azevedo, et al., 2023). Students may benefit from the chance to gain experience in OR methods during their degree courses, by incorporating practice of OR into student assignments and dissertations, for example, by setting write-ups of preregistrations in assessments (Blincoe & Buchert, 2020; Parsons et al., 2022), running replications or encouraging the use of preregistrations within final year projects (Button et al., 2020; Pownall, Pennington, et al., 2023; Strand & Brown, 2019). There are also many resources for educators wanting to embed the teaching of open research within their curriculum, such as FORRT (2023), Pennington (2023), Pennington & Pownall (2024), Button et al. (2020), Chopik et al. (2018), and Pownall et al. (2021).

Due to the high prevalence of QRPs and statistics anxiety among students (Krishna & Peter, 2018; Onwuegbuzie, 2004), our results also have implications for teaching OR and statistics. Our results suggest many links between statistics anxiety and attitudes and other variables, including student confidence and interpretation as well as OR perceptions. Students may benefit from university lecturers fostering positive attitudes towards statistics, decreasing their anxieties and boosting confidence in their abilities (Kvetnaya et al., 2019), but further work is required to confirm the causal link between these factors. Gigerenzer (2018) also argued that statistics teachers should teach a statistical ‘toolbox’ to their students and that null-hypothesis testing can encourage students to see significance as their aim rather than good scientific practice, as we found in Study 1. Perhaps wider teachings of the history and methods of statistics, including both frequentist and Bayesian techniques, could help students understand more deeply the methods and decisions made during analysis. This could steer them away from ritualistic behaviours, especially since an awareness of the problems caused by QRPs and statistics rituals can help students make more informed decisions to not engage in them (Gigerenzer, 2018; Moran et al., 2022).

Our work has identified a host of important relationships between students’ understanding of and attitudes toward statistics and OR, but future research should aim to further disentangle the relationships and possible factors related to both OR and statistics in student populations. It may also be important to investigate student perceptions about the importance of OR and research processes, both in their studies and the scientific process as a whole, to help adequately inform students about the real development of scientific knowledge and assist their understanding of the scientific process, and also establish this across multiple research disciplines, beyond psychology students. Additionally, researchers should explore the effectiveness of different integrations of OR and statistics into degree programmes to determine how best to educate students in these key areas and encourage practices of scientific integrity.

Since the future integrity of science is dependent upon its practicing researchers, investigating students’ OR and QRPs perceptions is important because they will form the next generation of researchers. In this two-study UK-wide project, we found important relationships between factors relating to statistics and perceptions of OR and QRPs amongst psychology undergraduate students. The understanding of OR has clear links to students’ academic experiences, attitudes, and success, and reducing students’ desires to attain significant results in their research might not only improve scientific rigour but also help reduce student stress. Our findings have important implications for teaching OR to students in terms of the potential benefits not only regarding research understanding but also for the sake of students’ wellbeing and the reduction of academic-related stress. Future research should strive to further understand the relationship between statistics and OR and to explore students’ perceptions of the importance of OR and its practices in order to enhance scientific reform.

Conceptualization: K.C., A.L., K.G.-L., L.d.-W., and N.E.S.-S.; Data curation: K.C. and A.L; Formal analysis: K.C. and A.L..; Funding acquisition: K.C.; Investigation: K.C., A.L., K.G.-L., L.d.-W., N.E.S.-S., and C.R.P.; Methodology: K.C., A.L., K.G.-L., L.d.-W., N.E.S.-S., and F.A.; Project administration: K.C., A.L., and K.G.-L.; Supervision: K.C.; Visualization: K.C.; Writing - original draft: K.C. and A.L.; Writing - review & editing: K.C., A.L., K.G.-L., L.d.-W., N.E.S.-S., F.A., and C.R.P.

We thank Caroline Elliott for her work in the early stages of this project, and we thank Andy Eastwood and P Taint for their thoughtful comments.

This work was partially supported by funding from the Psychological Sciences Research Group at the University of the West of England obtained by K.C.

The authors have no competing interest to declare.

All materials, code, and raw data are available via the Open Science Framework: https://osf.io/ume9k/