Financial-Scarcity-Related Cues’ Impact on the Cognitive Performance of the Poor: A Meta-analysis

The procedure and the analysis were preregistered and can be found on the OSF page of the project (https://osf.io/m3sh2/). As for the preregistration, we followed the recommendations of Moreau & Gamble (2022). Our reporting largely adhered to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P, Moher et al., 2015) and to the reporting guidelines of PRISMA (Page et al., 2021). Deviations from the Prisma guidelines are detailed in Table S1 and Table S2, while deviations from the preregistered PRISMA-P protocol are detailed in Table S3.

We included studies from two main sources. First, we scanned the reference list of publications on the topic and requested their authors to recommend potentially relevant studies. Second, we carried out a systematic literature search. The first phase of the systematic review process included three academic databases. As the topic of this meta-analysis lies at the intersection of psychology and economics, we screened a domain-general (Web of Science - Core Collection, Science Citation Index Expanded, Social Sciences Citation Index, Emerging Sources Citation Index), and two domain-specific databases covering psychology (PsycINFO), and economics (Econlit). Additionally, as the investigated topic is relatively novel, we screened databases that potentially contain not (yet) published literature (PsyArXiv, OSF, OSF preprints, and SocialScienceRegistry.org).

In the literature search, we focused on studies that mentioned both the financial and cognitive function-related aspects of the study either in the title or in the abstract or between the keywords. Table 1. contains the used search terms. We did not apply restrictions based on the date of the publications. A literature search was conducted between November 2020 and April 2021. The exact dates of the searches are disclosed in the Supplementary materials in Table S4.

The initial screening of the titles of the candidate studies was reviewed by one team member with the instruction to include any study that could potentially be relevant. Studies passing this stage were evaluated by two team members. First, to select the relevant papers, they screened the abstracts, then the full text of the remaining papers based on the inclusion criteria detailed in the main body of the paper. If at least one team member identified an article as potentially relevant based on the abstract, the paper was included in the full-text review stage. In case of disagreement, the team made the final decision together.

We applied the following eligibility criteria during the screening procedure. First, we considered papers that were written in English. Second, only studies based on primary data were included. Third, the studies needed to have at least one condition where financial-scarcity-related stimulus was presented². We considered the stimuli financial-scarcity-related where the stimuli described or implied (1) a situation with financial scarcity or (2) the decrease of real or hypothetical financial resources. Fourth, we only included studies that compared (1) a control condition to an experimental condition with financial-scarcity-related stimuli or (2) two experimental conditions with different levels of financial scarcity in the stimuli. Fifth, we included only experimental or quasi-experimental studies that enabled us to make causal inferences. Sixth, the unit of the analysis had to be individuals. Seventh, we only included studies that measured one of the following cognitive functions: memory, attention, executive functions (e.g., shifting/inhibition, inhibitory control, cognitive control, monitoring) or higher-order intelligence. Eighth, from each study whereby both individual and aggregated cognitive performance measures were reported, we only included results that were based on the performance from a single cognitive functioning test. Ninth, in cases where accuracy and response times are both reported we only included effect sizes based on accuracy. Tenth, we only included studies that investigated the interaction of a financial indicator and the experimental condition. Finally, we excluded books, book chapters, review articles, encyclopedias, editorial contents, bibliographical items and conference abstracts.

Prior to screening the literature, 7 articles (8 studies) had been selected from other sources. We identified 52605 results during the initial literature search. Removing the duplicates and screening the titles reduced the number of potential articles to 221. Reviewing the abstracts of the papers resulted in 80 remaining articles. After screening the full text of the papers, 2 articles (2 studies) were chosen to be included in the review beyond the expert recommendations. Figure 1 shows a PRISMA flow diagram depicting the process of the literature search.

After we identified the articles of interest, one team member scanned every study for the statistical results. If the manuscript did not contain the necessary statistics, we attempted to conduct reanalyses. In these cases, we downloaded the data if it was accessible or sent out requests to the corresponding authors to share the data or the statistical results with us. The flowchart summarising this communication can be found in the Supplementary materials in Figure S1.

We collected or calculated the statistical results for two different effects, one for each research question. First, to explore whether financial-scarcity-related cues influence the cognitive performance of the poor and the rich differently, we extracted the effect sizes corresponding to the interaction effect of the experimental condition and adjusted household income of the participant, hereafter, absolute income effect. This effect shows how one’s household income moderates the impact of financial-scarcity-related cues on cognitive performance. Second, we aimed to determine if a person’s relative rank in household income distribution within the sample has its own influence. To do this, we first performed a percentile rank transformation on the financial status measures used in the absolute income effect models. We then built the same model as before and extracted the effect sizes corresponding to the interaction effect of the percentile rank income variable and the experimental condition. We will refer to this interaction as the rank income effect.

Finally, to explore further influential contextual factors we originally also planned to conduct moderator analyses. Therefore, we coded several additional variables which potentially influence the impact of scarcity-related stimuli on cognition (see the Moderators subsection within the Results section). However, due to the low number of identified studies, we could not conduct a moderator analysis (see Schwarzer et al., 2015). We provide a detailed description of the collected variables in the Supplementary materials.

The analysis of the data was conducted in R (R Core Team, 2013). The analysis code can be found at https://doi.org/10.17605/osf.io/hxf85. First, we recalculated the effect sizes that we could retrieve from either the original papers or by communicating with the authors. As all selected studies followed the same design, we conducted all analyses with the same analytical specifications. We opted to fit quasibinomial generalised linear models, as the outcome variables didn’t have enough levels for the residuals to follow a normal distribution. Quasibinomial generalised linear models address overdispersion in binary or proportion data by allowing variance to exceed the mean (treating overdispersion as a free parameter). This contrasts with standard binomial models, which assume variance strictly as a function of the mean (fixed dispersion), potentially leading to biased parameter estimates and misleading conclusions when overdispersion is present. (Faraway, 2016). We excluded those data points that had relatively large Cook’s distance and the Cook’s distance was greater than 0.5 (Cook, 2011). The analytic deviations for each study are detailed in the Supplementary materials in Table S5. Next, we transformed the extracted effect sizes into Hedge’s g. Hedge’s g was chosen as it was shown to provide a more accurate estimate of the standardized mean difference than Cohen’s d as the latter tends to underestimate the effect size in small samples (Grissom & Kim, 2005). The effects have been coded in a way that a higher Hedge’s g value indicates that the effect of financial-scarcity-related stimuli on cognitive performance was more severely affected by household income.

Bayesian meta-analytic models were fitted for both interaction effects. Two-level random-effect meta-analyses were used to account for dependencies, treating effect size as a random effect nested within studies. Parameter estimation was performed using the R package brms (Bürkner, 2017). Following the recommendations of Haaf and Rouder (2021), we chose to use Normal(0, 0.3) as the prior for the effect size and Inverse-Gamma(1, 0.3) as the prior for between-study heterogeneity. Posterior median and 95% equal-tailed credible interval were used to summarise model parameters. Bayes factors were calculated using the R package bayestestR (Makowski, Ben-Shachar, & Lüdecke, 2019) to test the null hypothesis that the population effect size lies between -0.1 and 0.1 g (BF_ROPE, see Makowski, Ben-Shachar, Chen, et al., 2019; Williams et al., 2021), as we interpreted this interval as being practically equal to zero (Region Of Practical Equivalence, ROPE; Kruschke, 2018). BF_ROPE values were calculated as odds of the prior distribution of a parameter falling within vs. outside of the ROPE divided by the odds of the posterior distribution of that parameter falling within vs. outside of the ROPE. To draw conclusions based on Bayes factors, we used the interpretation of Jeffreys³ (1961) with a minor adjustment. We interpreted Bayes factors between ⅓ and 3 as inconclusive instead of anecdotal evidence (Keysers et al., 2020).

Note, that we included different types of cognitive performance measures as outcome measures in the same models without making computational distinctions. We opted to do so for two reasons. First, most theoretical accounts agree that financial worries of lower-income individuals lead to a decline in attentional performance which in turn deteriorates higher-level cognitive performance as well (Mani et al., 2013a; Shah et al., 2012; Zhao & Tomm, 2018). Second, due to the low number of identified studies, we would not have the necessary statistical power to conduct multiple analyses - one for each cognitive domain.

To reveal the sensitivity of our analysis to different specifications, first, we conducted a multiverse analysis (Steegen et al., 2016). The multiverse analysis involves “performing all analyses across the whole set … of reasonable scenarios” (Steegen et al., 2016, p. 702). Accordingly, we calculated 12 alternative meta-analytic estimates for both interaction effects using the following analytical specifications. First, following the example of Haaf and Rouder (2021), beyond the original prior, we also fitted the models half (narrow prior) and double (wide prior) width of the original prior for both of the standard deviation parameters. Second, we repeated the calculation of effect sizes using the same exclusion criteria as in our main analysis described above and also without excluding any data points. Third, we found that in the original studies occasionally more predictors were entered into the model than it was justified by the sample size (Green, 1991). Therefore, we conducted reanalyses of the original studies with and without the introduced control predictors. As a result, we ended up with 3 (prior) X 2 (outlier exclusion) X 2 (inclusion of predictors) meta-analytic estimations for both types of interaction effects.

Publication bias in each meta-analytic model was assessed utilising a Bayesian selection model approach. The selection model (Iyengar & Greenhouse, 1988) approach provides a publication bias-corrected estimate by adjusting the weight of each effect size in the analysis based on the corresponding p-values. Specifically, the larger the p-value is, the greater the weight of the effect size is on the final estimate. To conduct the selection model analysis, we used the STAN (Carpenter et al., 2017) code of Jin (2021/2022), which was specifically designed for multilevel models. In this approach, the selection model estimates the publication-bias-adjusted effect size based on the following assumptions: (1) the range of the possible p-values can be divided into intervals in a way, that within an interval all results are published with the same probability; (2) the higher p-values have lower chance to get published; (3) the results were surely published if the p-values are below 0.05. The prior used for the publication probabilities across different p-value intervals in the model was a non-informative two-sided Dirichlet prior with equal concentration parameters set to 1, implying all intervals are equally likely a priori. In the selection models we utilised the same prior distributions for effect size and heterogeneity as we did in the main analysis.

To determine whether the primary models or the selection models provide a better explanation of the data, we applied Bayesian bridge sampling (Meng & Schilling, 2002; Meng & Wong, 1996) using the R package bridgesampling (Gronau et al., 2021). Bridge sampling calculates the marginal likelihood of models, allowing for direct comparison. Subsequently, a Bayes factor was computed to indicate the extent to which one model is likely to be superior.

In the analyses we included data from ten studies (Table 4), which included 14 effect sizes (Table 5). For a detailed breakdown of the results of the screening procedure please see the Screening Results section of the Supplementary materials.

The meta-analytic model estimating the effect size of the interaction between experimental condition and household income on cognitive performance yielded a small, positive effect with the credible interval containing zero, and the Bayes factor indicated moderate evidence for the effect size falling within the range -0.1 to 0.1 g (g = 0.09 [-0.03, 0.21], BF_ROPE = 0.21), with small between-study heterogeneity⁴ (τ = 0.16 [0.09, 0.29]). Figure 2 illustrates this result in a forest plot.

Results of the multiverse analysis are detailed in Table 2 and visualised in Figure 3. The analysis showed that our results are sensitive to the chosen prior, but not to our other methodological choices. The use of wide priors resulted in moderate to strong evidence for the effect lying inside the bounds -0.1 and 0.1 g (BF_ROPE values ranging between 0.08 and 0.11). The estimated effect sizes range between 0.09 g and 0.10 g with credible intervals containing zero in all cases. In the cases where narrow priors were used, BF_ROPE values ranged between 0.4 and 0.52, yielding inconclusive results. Note, that the utilised narrow prior Normal(0, 0.15) and the defined region of practical equivalence are so similar that a high number of studies would have been required to present conclusive evidence. Between-study heterogeneity did not vary considerably across the models (with estimates ranging between 0.14 and 0.19).

The models adjusting for publication bias yielded different results. Namely, they showed strong evidence for the effect lying inside the ROPE. BF_ROPE values range between 0.02 and 0.09. The bias-adjusted main result shows stronger evidence as well (g = 0.07 [-0.02, 0.17], BF_ROPE = 0.05). The adjusted effect size estimates range between 0.06 g and 0.08 g, with all credible intervals including zero. The Bayes factors, which compare the marginal likelihoods of primary and selection models, were inconclusive across all cases, with values ranging from 2.26 to 2.34 favouring the primary models.

To investigate whether an individual’s relative income position within the population can be an influential factor, we repeated the analysis after conducting rank transformation on the household income data. We found inconclusive evidence about the effect size lying in or outside the ROPE (g = 0.22 [-0.05, 0.49], BF_ROPE = 1.92), with medium to high between-study heterogeneity (τ = 0.4 [0.20, 0.73]). Figure 4 illustrates the result of the effect size estimation in a forest plot.

Results of the multiverse analysis are detailed in Table 3 and visualised in Figure 5. Our results proved to be robust against all methodological choices. The estimated effect sizes range between 0.14 g and 0.27 g and in all cases credible intervals include zero. Across models, between-study heterogeneity did not vary considerably (with estimates ranging between 0.37 and 0.44).

Adjusting for publication bias did not change the main result meaningfully (g = 0.23 [-0.08, 0.55], BF_ROPE = 1.57). Selection models yielded similar results for the rest of the multiverse analysis conditions as well. The adjusted effect size estimates range between 0.11 g and 0.32 g. Similarly to the main analysis, BF_ROPE values are inconclusive ranging between 0.84 and 1.85. The Bayes factors, which compare the marginal likelihoods of primary and selection models, showed strong evidence for the main models across all cases, with values ranging from 9,950 to 10,367.

Table 4 lists the studies included in the meta-analyses and the coded study-level moderators, while Table 5 shows the characteristics of each included statistical result. In total 12 different potential moderator variables were coded. A detailed list of these variables is provided in the Moderator Exploration section of the Supplementary materials. However, due to the low number of identified studies, we could not test the moderating effect of the measured cognitive ability, population, or experimental venue in an analysis (see Schwarzer et al., 2015).

Based on our systematic review and meta-analysis, we conclude that the data published in the literature provides moderate evidence against the idea that financial-scarcity-related cues disproportionately impede the performance of those with lower income. However, this conclusion was not robust against the choice of the prior distributions in our analyses, as different priors lead to different conclusions. The use of a wider prior led to strong evidence against the existence of the effect, and a narrower prior yielded inconclusive results. Furthermore, we found no evidence for or against the presence of publication bias. Correction for the bias resulted in strong evidence against the effect.

While the between-study heterogeneity (τ = 0.16 [0.09, 0.29]) can be interpreted as small using the benchmarks of Cohen (Cohen, 1988), even this amount of heterogeneity has the potential to cloak a small effect that is estimated in the models (< 0.1 g). To explain some of this heterogeneity, we aimed to identify the environmental and interpersonal influencing factors of the target effect, but the number of the identified studies was too low to conduct sub-group analyses. As a result, we could not quantify the effect of contextual moderators. The investigation of the influence of relative ranks in the income distribution provided inconclusive evidence with moderate to high between-study heterogeneity.

In sum, our main conclusion is that the evidence available in the literature is extremely limited, and it is not possible to make any strong inference. However, to move forward the research of the investigated phenomenon, in the following paragraphs, we list a number of steps future researchers could take to improve the understanding of the effect of scarcity-related cues on cognitive performance.

Researchers in the field need to make sure that the theory, the experimental design, and the used instrument investigate the same dimension of financial scarcity. Hagernaars and de Vos (Hagenaars & de Vos, 1988) distinguish three possible ways people experience poverty: objective absolute poverty (having less than an objectively defined absolute minimum; e.g., living on less than a dollar a day), objective relative poverty (having less than others in society; e.g., belonging to the bottom 20% of earners in the country of residence), subjective poverty (having less than others in society). It has been shown that there is little overlap in the groups of people defined as poor by the different dimensions of financial scarcity (Bradshaw & Finch, 2003). Consequently, theory, design, and instruments targeting different dimensions could potentially lead to invalid conclusions regarding when and why scarcity-related cues affect performance.

This inconsistency can be observed in the reviewed body of research as well. More than half of the studies (60%) investigating the interaction of socioeconomic status and the experimental condition drew conclusions based on household income only, without the exploration of the impact of relative or subjective poverty dimensions. Such practice can be problematic when these studies use the theoretical framework of Scarcity theory focusing on the effect of subjective poverty on cognitive performance (de Bruijn & Antonides, 2022).

The exploration of contextual factors that moderate the effect should be prioritised, as it would be a prerequisite for understanding when and who would be vulnerable to the detrimental effects of scarcity. To the best of our knowledge, no systematic effort has been made to explore these contextual factors, although there are several accounts in the literature that highlight potentially important influences. For example, Shafir (2017) emphasised the role of the available funds for unforeseen expenses. While this fund can stem from high income, it can also be grounded on financial savings or one’s community buffer (Jachimowicz et al., 2020). Furthermore, the perception of the size of one’s financial funds might be affected by the number of people who are dependent on the person. It has also been demonstrated (Kim & Chatterjee, 2013) that financial socialisation in childhood affects financial management in adulthood. It might be possible that those who experienced childhood poverty not only learned about financial management but also the psychological reactions (e.g., anxiety, anger) of their parents in times of scarcity. Investigating individual differences unrelated to money might be beneficial as well. Exploring the role of resilience (Herrman et al., 2011) for instance, could not only potentially establish a valid approach for helping the poor, the results would also be telling about the mechanism of the effect.

Empirical studies putting Scarcity theory to the test should put a bigger emphasis on the investigation of attentional performance. Although most theoretical accounts agree that financial worries of lower-income individuals lead to a decline in attentional performance which in turn deteriorates higher-level cognitive performance (Mani et al., 2013a; Shah et al., 2012; Zhao & Tomm, 2018), the identified studies mainly investigated cognitive control and fluid intelligence, with no emphasis put on attentional performance.

Within the identified articles all studies adapted the design introduced by Mani and colleagues (2013a) with minor differences. This lack of diversity can enhance some factors while covering up other potentially relevant variables. For example, the lack of a control condition with no financial-scarcity-related stimuli might be the most crucial characteristic of the design. It might be, that less the strength but rather the mere presence of the financial-scarcity-related cue matters, which could lead to the underestimation of the effect using this design. This could also explain why we failed to find supporting evidence for the existence of the effect. There are, however, other aspects of the design that should be varied such as the modality of the experiment, the time passed between the manipulation and the measurement of the outcome variable, and whether participants received payment for participation. The role of experimental venue should be explored as well, as the biggest effect sizes were demonstrated by Mani and colleagues (2013a) who conducted the experiment in a mall where environmental financial-scarcity-related stimuli are plenty, suggesting that such venue might amplify the effect. Also, in the applied design, participants are explicitly encouraged to think about their own finances. This is problematic, as encountering scarcity-related cues in real life (e.g., the sight of an expensive watch or an exam question about compound interests) and asking someone to think about their own finances does not necessarily involve the same mechanisms.

Systematically varying the applied stimuli could lead to the discovery of relevant characteristics of the financial-scarcity-related cues. Several testable attributes of the stimuli can be highlighted which could induce the cognition deteriorating effect. The modality, whether scarcity is hypothetical or real, the degree of the resource shortage, and the degree to which scenarios are articulated are all factors that could be alternated between studies. It would be worth exploring whether barely financial-scarcity-related phrases (e.g., debt, electricity bill, salary reduction) can affect performance, as such stimuli would be much closer to the relevant real-life events of the poor. Similarly, the dimension of financial scarcity can also influence the strength of the effect and who it affects, therefore it should be investigated thoroughly. For instance, a fictional scenario in which the neighbour drives a nicer car might affect those in objective relative poverty, while inducing the thought of emergency spendings will worsen the performance of people living in absolute objective poverty. Understanding the relevant characteristics of the stimuli would enable practitioners to diminish the presence of cues that have a negative impact on the poor’s performance in important life situations such as exams or interviews.

Our results must be treated with a number of limitations in mind. First, there are studies that contain suggestive evidence regarding the effect (Dang et al., 2016; Destin & Svoboda, 2018; Duquennois, 2022) but were not included in the meta-analysis due to lacking causal evidence, not having individual-level outcome data, or not sharing the data or detailed results with us. Second, several of the included studies are conducted on a sample with moderate variation in income which can be a plausible explanation behind the lack of strong observed effects. Third, the number of studies included in our meta-analyses was relatively low, which limited our abilities to investigate our more nuanced research questions with statistical tools. Finally, the ecological validity of the studies that we included in the meta-analysis is relatively low.

Additional information about the procedure, analyses, tables, and figures can be found in the Supplementary materials. Original and extracted data, analysis codes, and further materials are available on the OSF page of the project at https://doi.org/10.17605/osf.io/hxf85.

The datasets acquired from other researchers are not publicly available. However, said datasets are available from the authors upon reasonable request and with permission of the original publishers of the data.

Contributed to conception and design: PS, BS

Contributed to acquisition of data: PS, BS

Contributed to analysis and interpretation of data: PS, BS

Drafted and/or revised the article: PS, BS

Approved the submitted version for publication: PS, BS

We would like to thank Zoltan Kekecs and Tamas Nagy for the useful comments on this research project. We are grateful to Anna Bognár, Marcell Püski and Aikaterini Taka for their help in the screening procedure.

PS was supported by the ÚNKP-21-3 New National Excellence Program of the Ministry for Innovation and Technology from the source of the National Research, Development and Innovation Fund. The work of BS was supported by the Eötvös Loránd University Excellence Fund (EKA).

The authors have no competing interests to declare that are relevant to the content of this article.