The number of possible approaches to conducting and analyzing a research study—often referred to as researcher degrees of freedom—has been increasingly under scrutiny as a challenge to the reproducibility of experimental results. Here we focus on the specific instance of time window selection for time series data. As an example, we use data from a visual world eye tracking paradigm in which participants heard a word and were instructed to click on one of four pictures corresponding to the target (e.g., “Click on the hat”). We examined statistical models for a range of start times following the beginning of the carrier phrase, and for each start time a range of window lengths, resulting in 8281 unique time windows. For each time window we ran the same logistic linear mixed effects model, including effects of time, age, noise, and word frequency on an orthogonalized polynomial basis set. Comparing results across these time ranges shows substantial changes in both parameter estimates and p values, even within intuitively “reasonable” boundaries. In some cases varying the window selection in the range of 100–200 ms caused parameter estimates to change from positive to negative. Rather than rush to provide specific recommendations for time window selection (which differs across studies), we advocate for transparency regarding time window selection and awareness of the effects this choice may have on results. Preregistration and multiverse model exploration are two complementary strategies to help mitigate bias introduced by any particular time window choice.

Sexual behaviors and substance use exhibit high rates of co-occurrence and similar patterns of age-related change, with typical initiation in middle adolescence followed by large increases in late adolescence and emerging adulthood. Because adolescent sexual behaviors are associated with negative health consequences including sexually-transmitted infections and substance use, adolescent sexual behaviors are often conceptualized within a health-risk framework. Sexual development, however, is a normative process important for healthy psychosocial adjustment, with the timing (early vs later initiation) and context (casual vs romantic partner) of sexual behaviors influencing their association with health risks. We tested whether seven common sexual behaviors could be conceptualized as markers along a continuum of sexual development (e.g., from kissing and making out to oral, vaginal, and anal sex) and then examined their associations with various measures of substance use in a sample of adolescents and young adults ( N = 626; ages 13-22 years old). We found a 2-parameter logistic item response theory model provided a good fit to the prevalence and covariance among the sexual behaviors, with each behavior holding a different location on a trait representing the progression of sexual development. Each sexual behavior was associated with greater substance use even after adjusting for an early initiation of sexual behavior, current age, biological sex, and socioeconomic status. The results indicate that even normative sexual behaviors have strong and non-specific associations with substance use in adolescence and emerging adulthood.

Coincident with the global rise in concern about the spread of misinformation on social media, there has been influx of behavioral research on so-called “fake news” (fabricated or false news headlines that are presented as if legitimate) and other forms of misinformation. These studies often present participants with news content that varies on relevant dimensions (e.g., true v. false, politically consistent v. inconsistent, etc.) and ask participants to make judgments (e.g., accuracy) or choices (e.g., whether they would share it on social media). This guide is intended to help researchers navigate the unique challenges that come with this type of research. Principle among these issues is that the nature of news content that is being spread on social media (whether it is false, misleading, or true) is a moving target that reflects current affairs in the context of interest. Steps are required if one wishes to present stimuli that allow generalization from the study to the real-world phenomenon of online misinformation. Furthermore, the selection of content to include can be highly consequential for the study’s outcome, and researcher biases can easily result in biases in a stimulus set. As such, we advocate for pretesting materials and, to this end, report our own pretest of 224 recent true and false news headlines, both relating to U.S. political issues and the COVID-19 pandemic. These headlines may be of use in the short term, but, more importantly, the pretest is intended to serve as an example of best practices in a quickly evolving area of research.

Language users encounter words in at least two different modalities. Arguably, the most frequent encounters are in spoken or written form. Previous research has shown that – compared to the spoken modality – written language features more difficult words. An important question is whether input modality has effects on word recognition accuracy. In the present study, we investigated whether input modality (spoken, written, or bimodal) affected word recognition accuracy and whether such a modality effect interacted with word difficulty. Moreover, we tested whether the participants’ reading experience interacted with word difficulty and whether this interaction was influenced by modality. We re-analyzed data from 48 Dutch university students that were collected in the context of a vocabulary test development to assess in which modality test words should be presented. Participants carried out a word recognition task, where non-words and words of varying difficulty were presented in auditory, visual and audio-visual modalities. In addition, they completed a receptive vocabulary and an author recognition test to measure their exposure to literary texts. Our re-analyses showed that word difficulty interacted with reading experience in that frequent readers (i.e., with more exposure to written texts) were more accurate in recognizing difficult words than individuals who read less frequently. However, there was no evidence for an effect of input modality on word recognition accuracy, nor for interactions with word difficulty or reading experience. Thus, in our study, input modality did not influence word recognition accuracy. We discuss the implications of this finding and describe possibilities for future research involving other groups of participants and/or different languages.

COVID-19 has induced stress motivated by individuals’ fears of infection and death, while mitigation efforts have also exacerbated stress by reducing individuals’ income, creating feelings of social isolation, and imposing barriers to obtaining basic resources. In this study, we use a representative sample of 693 adults in Michigan to investigate the association between COVID-19 stress and life satisfaction. We find that COVID-19 stress is significantly negatively related to life satisfaction, and that this association is strongest among those living alone and those living in households with many other adults. Exploratory analyses suggest that shopping challenges are most strongly associated directly with stress and indirectly with life satisfaction, while income and boredom challenges are less important. We conclude by discussing the implications of declines in life satisfaction due to an ongoing and potentially worsening stressful event, and consider some strategies for mitigating this non-trivial form of COVID-19 harm.

Sequential testing enables researchers to monitor and analyze data as it arrives, and decide whether or not to continue data collection depending on the results. Although there are approaches that can mitigate many statistical issues with sequential testing, we suggest that current discussions of the topic are limited by focusing almost entirely on the mathematical underpinnings of analytic approaches. An important but largely neglected assumption of sequential testing is that the data generating process under investigation remains constant across the experimental cycle. Without care, psychological factors may result in violations of this assumption when sequential testing is used: researchers’ behavior may be changed by the observation of incoming data, in turn influencing the process under investigation. We argue for the consideration of an ‘insulated’ sequential testing approach, in which research personnel remain blind to the results of interim analyses. We discuss different ways of achieving this, from automation to collaborative inter-lab approaches. As a practical supplement to the issues we raise, we introduce an evolving resource aimed at helping researchers navigate both the statistical and psychological pitfalls of sequential testing: the Sequential Testing Hub (www.sequentialtesting.com). The site includes a guide for involving an independent analyst in a sequential testing pipeline, an annotated bibliography of relevant articles covering statistical aspects of sequential testing, links to tools and tutorials centered around how to actually implement a sequential analysis in practice, and space for suggestions to help develop this resource further. We aim to show that although unfettered use of sequential testing may raise problems, carefully designed procedures can limit the pitfalls arising from its use, allowing researchers to capitalize on the benefits it provides.

At its best, strategic thinking yields an advantage needed to beat an opponent. At the least, it protects the person from exploitation. In four studies, conducted in two countries, we used a simple number-guessing game, in which one respondent wins by guessing the number chosen by another. We show that people generate numbers nonrandomly, and, on the basis of this finding, we predict and find that nonrandom strategic choice is advantageous to the guesser if the chooser does not randomize either. As expected, respondents in the role of the guesser preferred to play a game in which they were to actively think of a number instead of randomizing if the chooser had to think of a number, too. Guessers did not prefer thinking if the chooser selected a number randomly. Having shown these limitations to strategic reasoning, we close with the observation that successful strategic reasoning may – at times – require the breaking of rules and being the first to do so.

Although romantic relationships have been found to boost well-being, some studies demonstrated that staying single has similar beneficial effects. One cause of such contradiction is probably due to the lack of a quantitative measurement of attitudes toward singlehood. To address this methodological gap, four studies involving 1,276 undergraduate students in Malaysia and India were conducted to develop and examine the psychometric qualities of the Attitudes toward Singlehood Scale (AtSS). Study 1 selected 15 items from the pool and identified a 3-factor solution using exploratory factor analysis. However, a 9-item second-order factor model was found superior in Study 2 using confirmatory factor analysis. The 9-item AtSS demonstrated good internal consistency and test-retest reliability measured two weeks apart as well as construct and criterion validity. Study 3 further supported the superiority of the 9-item second-order factor model with replicated results of Study 2 on a new sample. Measurement invariance test supported scalar invariance across gender while ANCOVA showed female participants displaying higher scores than male counterparts. Study 4 then examined the properties of the 9-item AtSS on a sample of young adults in India. The results are consistent with Study 2 and 3, lending further support to the usability of the AtSS in different cultural contexts. Overall, the consistent findings promote the AtSS as a promising tool for assessing young adults’ attitudes toward singlehood. Implication and suggestions for future studies are also discussed.

Previous research examining transference – which posits people draw on past experiences with close others to inform novel interactions – has found that people attribute qualities to and express preferences for novel targets based on their similarity to significant others. However, classic tasks for testing transference required multiple sessions over many weeks, limiting the test of this process to relatively small, college student samples. The purpose of the current study ( N = 532, M age = 34.81, SD = 9.83, 61.47% Men, 63.35% White) was to create an online version of a transference task administered in one session and replicate the effect of transference with a larger sample of participants across the lifespan, and test whether targets resembling parents and ex-partners were preferable to control targets. The effects of transference and preference were replicated in the online version of the transference task. We also found preliminary evidence that the effects of transference and preference were slightly stronger in older individuals and secure individuals, albeit the effects were small. Results are discussed in the context of how individuals use previous and existing relationships to guide their behavior in new relationships.

Event boundaries are important moments throughout an ongoing activity that influence perception and memory. They allow people to parse continuous activities into meaningful events, encode the temporal sequence of events and bind event information together in episodic memory (DuBrow & Davachi, 2013). Thus, drawing attention to event boundaries may facilitate these important perceptual and encoding processes. In the current study, we used emotionally arousing stimuli to guide attention to event boundaries because this type of stimulus has been shown to influence perception and attention. We evaluated whether accentuating event boundaries with commercials improves memory and whether emotional stimuli further enhance this effect. A total of 97 participants watched a television episode in which we manipulated commercial break locations (boundary, non-boundary, no commercial) and the type of commercial (emotional, neutral) and then completed memory tasks. Overall, placing emotionally arousing commercials at event boundaries increased memory for the temporal order of events, but no other effects of accentuating event boundaries were observed. Thus, drawing attention to event boundaries—via emotionally charged commercials—increases the likelihood that people will perceive the change in events, update their mental model accordingly and better integrate temporal information from the just-encoded event.

Associations between socioeconomic status (SES) and personality traits have important implications for theory and application. Progress in understanding these associations depends on valid measurement, unbiased estimation, and careful assessment of generalizability. In this registered report, we used data from AIID, a large online study, to address three basic questions about personality and SES. First, we evaluated the measurement invariance of a common measure of personality, the Big Five Inventory, across indicators of educational attainment, income, and occupational prestige. Fit indices showed some instances of detectable noninvariance, but with little practical impact on substantive results. Second, we estimated associations between SES and personality. Results showed that personality and SES were largely independent (most rs < .1), in contrast to predictions derived from several previous studies. Third, we tested whether age trends in personality were moderated by SES. Results did not support predictions from social investment theory, but they did suggest that age trends were largely generalizable across SES. We discuss the implications of these findings for developing and validating personality measures for use in diverse samples. We also discuss the implications for theories that propose that the Big Five are responsive to, or partially responsible for, people’s economic and social conditions.

Registered reports are a new publication workflow where the decision to publish is made prior to data collection and analysis and thus cannot be dependent on the outcome of the study. An increasing number of journals have adopted this new mechanism, but previous research suggests that submission rates are still relatively low. We conducted a census of journals publishing registered reports ( N = 278) using independent coders to collect information from submission guidelines, with the goal of documenting journals’ early adoption of registered reports. Our results show that the majority of journals adopting registered reports are in psychology, and it typically takes about a year to publish the first registered report after adopting. Still, many journals have not published their first registered report. There is high variability in impact of journals adopting registered reports. Many journals do not include concrete information about policies that address concerns about registered reports (e.g., exploratory analysis); however, those that do typically allow these practices with some restrictions. Additionally, other open science practices are commonly encouraged or required as part of the registered report process, especially open data and materials. Overall, many journals did not include many of the fields coded by the research team, which could be a barrier to submission for some authors. Though the majority of journals allow authors to be anonymous during the review process, a sizable portion do not, which could also be a barrier to submission. We conclude with future directions and implications for authors of registered reports, journals that have already adopted registered reports, and journals that may consider adopting registered reports in the future.

Research evidence in the social sciences often relies on effect size statistics, which are hard to understand for the public and do not always provide clear information for decision-makers. One area where interpretation of research evidence has profound effects on policy is college admission testing. In this paper, we conducted two experiments testing how different effect size presentations affect validity perception and policy preferences toward standardized admission tests (e.g., ACT, SAT). We found that compared to traditional effect size statistics (e.g., correlation coefficient), participants perceived admission tests to be more predictively valid when the same evidence was presented using an alternative effect size presentation. The perceived validity of the admission test was also positively associated with admission test policies (e.g., test-optional policy) preferences. Our findings show that policy preferences toward admission tests depend on the perception of statistical evidence, which is malleable and depends on how evidence is presented.

Women are more likely than men to be sexualized, objectified and dehumanized. Female sex workers experience stigma and violence associated with these judgements at far higher rates than other women. Here, we use a pre-registered experimental design to consider which aspects of sex work – the level of sexual activity, earned income, or perceived autonomy of the work – drive dehumanization. A first group of participants ( N = 217) rated 80 vignettes of women varying by full-time employment, hobbies and interests on humanness. These ratings were subtracted from the ratings of a second group of participants ( N = 774) who rated these same vignettes which additionally described a part-time job, hobby or activity that varied in sexual activity, income earned and autonomy over one’s actions. We find that women and especially men dehumanize women they believe are engaging in penetrative sex. We also find that women’s autonomy of, but not their income from, their sexual activity increases dehumanization. Our findings suggest that opposition to women’s ability to pursue casual sex and generalizations about the exploitative conditions of sex work may drive the harshest negative prejudice towards female sex workers and, by similar mechanisms, women’s sexuality in general.

Repeated experiences and activities drive personality development. Leisure activities are among the daily routines that may elicit personality change. Yet despite the important role they play in daily life, little is known about their prospective effects on personality traits and vice versa. The objective of this study was to examine the extent to which within-person changes in leisure activities lead to prospective changes in personality traits, and whether changes in personality elicit prospective changes in leisure activities. We applied random-intercept cross-lagged panel models (RI-CLPM) to four waves of 13-year longitudinal data (2005−2017) from the German Socio-Economic Panel Study (SOEP) for the sample as a whole ( N = 55,790) and for three specific age groups (young, middle-aged, and older adults). We examined between-person associations and within-person auto-regressive effects, correlated change and cross-lagged effects for Big Five personality traits (i.e., openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism) with self-reported frequency of leisure activities (i.e., physical activities, socializing, volunteering, political activity, artistic and musical activity, going out) and overall participation in leisure activities. At the between-person level, leisure activities and overall participation were most strongly associated with openness to experience. At the within-person level, we found reciprocal effects of extraversion only with overall participation in leisure activities and socializing. We found unidirectional within-person cross-lagged effects between leisure activities and personality traits and vice versa. Some effects were age-group-specific only. These findings suggest that leisure activities that are associated with certain traits at the between-person level are not necessarily those that trigger change in the respective personality trait. We discuss our findings based on the TESSERA framework for personality development. We conclude that the specificity of an experience or behavior and its corresponding trait is essential for personality development and should be subjected to further research.

In recent years, understanding psychological constructs as network processes has gained considerable traction in the social sciences. In this paper, we propose the aesthetic effects network (AEN) as a novel way to conceptualize aesthetic experience. The AEN represents an associative process where having one association leads to the next association, generating an overall aesthetic experience. In art theory, associations of this kind are referred to as aesthetic effects . The AEN provides an explicit account of a specific cognitive process involved in aesthetic experience. We first outline the AEN and discuss empirical results (Study 1, N =255) to explore what can be gained from this approach. Second, in Study 2 ( N =133, pre-registered) we follow calls in the literature to substantiate network theories by using an experimental manipulation, and found evidence in favor of the AEN over other alternatives. The AEN provides a basis for future studies that can apply a network perspective to different aesthetic experiences and processes. This perspective takes a process-based approach to aesthetic experience, where aesthetic experience is represented as an active interaction between viewer and artwork. If we want to understand how people experience art, it is central to know why people have different experiences with the same artworks, and, also, why people have similar experiences when looking at different artworks. Our proposed network perspective offers a new way to approach and potentially answer these questions.

Recent issues around reproducibility, best practices, and cultural bias impact naturalistic observational approaches as much as experimental approaches, but there has been less focus on this area. Here, we present a new approach that leverages cross-laboratory collaborative, interdisciplinary efforts to examine important psychological questions. We illustrate this approach with a particular project that examines similarities and differences in children’s early experiences with language. This project develops a comprehensive start-to-finish analysis pipeline by developing a flexible and systematic annotation system, and implementing this system across a sampling from a “metacorpus” of audiorecordings of diverse language communities. This resource is publicly available for use, sensitive to cultural differences, and flexible to address a variety of research questions. It is also uniquely suited for use in the development of tools for automated analysis.

Exaggerated claims and low levels of reproducibility are commonplace in psychology and cognitive neuroscience, due to an incentive structure that demands “newsworthy” results. My overall argument here is that in addition to methodological reform, greater modesty is required across all levels - from individual researchers to the systems that govern science (e.g., editors, reviewers, grant panels, hiring committees) - to redirect expectations regarding what psychological and brain science can effectively deliver. Empirical work and the reform agenda should pivot away from making big claims on narrow evidence bases or single tools and focus on the limitations of our individual efforts, as well as how we can work together to build ways of thinking that enable integration and synthesis across multiple modalities and levels of description. I outline why modesty matters for science including the reform agenda, provide some practical steps that we can take to embrace modesty, rebut common misconceptions of what modesty means for science, and present some limitations of the approach. Ultimately, by presenting a more sober view of our capacities and achievements, whilst placing work within a wider context that respects the complexity of the human brain, we will bolster the fidelity of scientific inference and thus help in a small way to generate a firmer footing upon which to build a cumulative science.

The outbreak of a global pandemic such as COVID-19 poses a challenge for societies across the world. Lacking both vaccination and medical treatment, the only way to combat the spread of a virus in its early stages are behavioral measures, particularly physical distancing behavior. The present work proposes three pillars of individuals’ engagement in physical distancing: anxiety, prosociality, and rule compliance. In a large ( N = 1,504), pre-registered study among German adults, we studied both situation-specific tendencies and stable personality traits that are theoretically associated with these pillars in relation to self-reported physical distancing behavior and underlying motives. Results supported the importance of each of the proposed pillars for physical distancing behavior. That is, for each pillar, we found (some) relations of the corresponding tendencies and personality traits with physical distancing (motives) as expected. Overall, the project provides a comprehensive picture of physical distancing behavior during the COVID-19 pandemic.

Self-interest is a central driver of attitudes and behaviors, but people also act against their immediate self-interest through prosocial behaviors, voting incongruously with their finances, or punishing others at personal cost. How much people believe that self-interest causes attitudes and behaviors is important, because this belief may shape regulation, shared narratives, and institutional structures. An influential paper claimed that people overestimate the power of self-interest on others’ attitudes and behavioral intentions (Miller & Ratner, 1998). We present two registered, close, and successful replications (U.S. MTurk, N = 800; U.K. Prolific, N = 799) that compared actual to estimated intentions, with open data and code. Consistent with the original article, participants overestimated the impact of payment on blood donation in Study 1, ds = 0.59 [0.51, 0.66], 0.57 [0.49, 0.64], and overestimated the importance of smoking status for smoking policy preferences in Study 4, ds = 0.75 [0.59, 0.90], 0.84 [0.73, 0.96]. These replications included two extensions: 1) communal orientation as a moderator of overestimation and 2) a more detailed measure of self-interest in Study 4 (ordinal smoking status). Communal orientation did not predict overestimation, and the ordinal smoking measure yielded similar results to the main study. Verifying the overestimation error informs behavioral theories across several fields and has practical implications for institutions that require trust and cooperation. All materials, data, and code are available at osf.io/57mdc/