Harm prevention (safer gambling) messages are often used as a population-based intervention against gambling-related harm, and independently-designed messages (e.g., “Chances are you’re about to lose”) are increasingly replacing industry-designed slogans (e.g., “Take time to think”). One common type of harm prevention message warns people about the likelihood of losing money in gambling (e.g., “99% of gamblers lose in the long run”), but this intervention can potentially be improved by testing pre-existing messages against novel alternatives. We asked UK-based online gamblers (N=4,025) to rate ten pre-existing and novel messages on 7-point scales relating to one potential negative impact (challenging participants to try to win at gambling), and three potential positive impacts (making participants want to gamble less, and being perceived as relevant to the participant and to gamblers experiencing harm). Participants also completed the Problem Gambling Severity Index (PGSI) to explore potential interaction effects based on levels of harm. Messages were all on average perceived as not challenging participants to try to win; as making them want to gamble less; and as being most relevant to gamblers experiencing harm. Significant differences were observed between messages, with “99% of gamblers lose in the long run” scoring the best overall, and the five pre-existing messages currently used in Australia, Belgium, and the UK scoring the worst. Messages were more likely to be seen as a challenge by participants with higher PGSI scores. Continual message design and evaluation can help improve the effectiveness of harm prevention messages.
Independently-designed harm prevention (“safer gambling”) messages have been implemented in Australia (Butler, 2022; Chapman & Priestly, 2022), and are forthcoming in the UK (Peacock, 2025), as a population-based harm intervention against gambling-related harm. Messaging is often seen as a way to initially warn large segments of the population about gambling’s potential harms (Ray et al., 2024), with more restrictive measures such as self-exclusion being better placed to help those who are already experiencing significant harm. While messaging was for many years controlled by the gambling industry, which chose to focus on slogans emphasizing gamblers’ personal responsibility (e.g., “take time to think” in the UK, or “gamble responsibly” more broadly; Lole et al., 2019; Newall et al., 2023; Rintoul, 2022), this has recently started to change. In Australia, gambling adverts generally rotate between showing one of seven independently-designed messages (e.g., “Chances are you’re about to lose” or “You win some. You lose more”; Chapman & Priestly, 2022). Meanwhile in the UK, the Office of Health Improvement and Disparities was announced in 2025 as the new prevention commissioner to lead on implementing independent prevention-based work (Peacock, 2025). Therefore, additional independent work on messaging appears beneficial in a UK context as a population-wide measure to complement other measures such as affordability checks.
The term “safer gambling” is used by various stakeholders to refer to interventions designed to minimize gambling-related harm. In recent years, it has largely replaced “responsible gambling,” which many stakeholders see as shifting the responsibility for harm prevention from the industry and government toward gamblers themselves (Livingstone, 2024). Despite being commonly used by both the UK regulator (Gambling Commission, 2019) and the government (DCMS, 2023), some stakeholders may also consider “safer gambling” to be overly favorable toward industry. Its use has also been discouraged within recent guidelines for the respectful reporting on gambling (Biggar & Wardle, 2024). Based on these considerations, we will herein use the term “harm prevention” messages.
Although the current Australian messages represent an improvement, further advances are likely still possible. For example, Australian stakeholders may consider these messages as being shown with insufficient prominence (e.g., size, and good use of color to contrast them from surrounding material; similar findings have been observed with other messages; Critchlow et al., 2020; Lole et al., 2019), or to be shown less often than they should. Harm prevention message implementation can also potentially be improved by designing and testing alternative messages. It may be that specific wordings communicate certain ideas most effectively (e.g., the fact that gamblers are likely to lose overall), in which case superior wordings should be used, all else equal. It may also be that different wordings communicate the same idea to a similar degree of effectiveness. It would still be useful in this instance to have more different wordings to draw from, as any specific message is thought to become less effective as it is seen more often (McCullock et al., 2024; Newall, Rockloff, Hing, Thorne, et al., 2023; Ray et al., 2024).
Harm prevention messages can take many forms, including messages informing gamblers about tools such as deposit limits (Heirene & Gainsbury, 2021), fear-inducing messages (Mutti-Packer et al., 2022), or positive-emotion messages that aim to motivate gambling abstinence (Harris et al., 2018; Newall, Rockloff, Hing, Browne, et al., 2023). One common type of harm prevention message warns people about the likelihood of losing money. This type of message includes the two Australian messages quoted above as well as a third Australian message: “What are you prepared to lose today? Set a deposit limit” (Chapman & Priestly, 2022). Two other pre-existing international examples of this type of message are the “Odds are: they win” campaign, which is used by Greater Manchester Combined Authority, a local government in England (Mills et al., 2023), and the Belgium message, “What does gambling cost you? Stop in time” (De Jans et al., 2023). This is a logical type of message, as many gamblers are motivated by the chance of winning money (Tabri et al., 2022), and yet almost all regular gamblers lose (Alexander, 2020), with losses often leading to harms across several domains (Langham et al., 2016; Muggleton et al., 2021). As so many pre-existing harm prevention messages warn people about the likelihood of losing money, the remainder of the present work will focus on testing different wordings of this same core idea.
Other wordings have been proposed in the previous literature, but which cannot as yet be directly compared due to the lack of controlled tests. “Over 99% of bettors lose in the long run” is one such message (Rockloff et al., 2024), which was initially created based on the testimony provided by a gambling executive to a UK parliamentary committee, who said: “I am not going to sugar-coat it: 99% of the customers who play on our sites will lose, so you’re probably losing more if you play more” (Alexander, 2020). In the UK, GambleAware (2024) recently proposed three messages to replace the industry’s current “Take time to think” message, of which “Gambling comes at a cost” appears relevant to this type of harm prevention message, given that it implies an expectation of losses (GambleAware, 2024). Another relevant message is, “Whether you win or lose, the betting companies are always the real winners” which was shown to be more relevant to people who gamble extensively than, for example, severe warning messages about gambling harms (Davies et al., 2022). Since many gamblers both underestimate their losses (Heirene et al., 2022), and have a tendency to selectively remember wins (Toneatto et al., 1997), this message could be more personally-relevant to gamblers than alternative messages.
Finally, novel relevant messages can also be conceived. One approach of generating new messages is to modify and potentially improve upon an existing message. “Odds are: they win” for example, could be modified based on insights that gambling information can be made more meaningful by focusing on gamblers’ personal losses (Newall et al., 2020) by reframing it as “Odds are: you lose.” This may well be more comprehensible to many less-engaged gamblers, who may not intuitively understand that the “they” in the existing message refers to the gambling industry, and the implications this has for the gambler’s likelihood of losing. Another approach is to generate messages based on other aspects of successful gambling information provision. Longer-form educational videos which expose the persuasive aspects of gambling advertising appear promising at countering gambling’s appeal, although their length might reduce their practical applicability (Torrance et al., 2025). However, since gambling adverts misleadingly present gamblers as solely winning (Lopez-Gonzalez et al., 2018), one potential message based on these longer-form videos that appears worth testing is, “Gambling adverts show people winning, but most gamblers lose overall.”
The current Australian messages were formed via a mixed-methods package of studies, including a Delphi expert consensus study, and qualitative interviews with gamblers (Chapman & Priestly, 2022). However, the final seven messages were also chosen from an initial list of ten potential messages via a quantitative self-report rating study. This is a commonly-used methodology in harm prevention message development, as it can allow for a number of messages to be cost-effectively compared across multiple relevant dimensions (Chapman & Priestly, 2022; Davies et al., 2022; Newall et al., 2024; Rockloff et al., 2024). Although self-reports may not necessarily lead to behavioral changes, this methodology does find support in the more established tobacco warning literature, where it has been suggested that a message’s perceived effectiveness at changing behavior predicts that message’s actual effectiveness (Noar et al., 2020). In the present work, perceived message effectiveness is proxied by the self-report item, “This message makes me want to gamble less”.
However, self-report rating studies could provide an inaccurate picture of potential messages by failing to include a rating scale for some relevant message aspect. Recently, it has been suggested based on qualitative interviews, that messages warning gamblers about the likelihood of losing money risk presenting gambling as an inviting challenge to try to win (GambleAware, 2024). But previous self-report rating studies did not include any items relevant to this feature (Chapman & Priestly, 2022; Davies et al., 2022; Newall et al., 2024; Rockloff et al., 2024). The present work will therefore include the item, “This message challenges me to try to win at gambling”, to provide a first indication of whether tested messages could, as an unintended negative consequence, backfire and actually prompt some people to gamble more.
Another issue is that prior self-report rating studies have each probed slightly different measures, which could lead to different outcomes and which could miss some relevant features. For example, one study asked whether a given message “suggests that people should cut back on their gambling [emphasis added]” (Newall et al., 2024), while another asked whether it “suggests that I should cut back my gambling activity [emphasis added]” (Chapman & Priestly, 2022). Notwithstanding the minor changes in wording, one study asked for a message’s relevance to other people in general, whereas the other study asked for a message’s relevance to the participant personally. This is an important distinction as, for example, people often think that marketing communications have more effect on other people than on themselves (Perloff, 1999). The inclusion of two related items, asking for both a message’s relevance to the participant personally and to people experiencing gambling harms in general, would therefore best probe the extent to which different gamblers think that messages are relevant to themselves versus to other people. Therefore, the present study uses a message’s perceived relevance to participants and to people experiencing gambling harms as the final two dependent measures.
Messaging is an intervention thought best suited to no- and low-risk gamblers, as messages can be seen frequently, and yet do not restrict behavior (Ray et al., 2024). However, it is still important to test messages across the full spectrum of gambling-related harm. Some messages may appear surprisingly effective among high-risk gamblers, which could unlock important theoretical or applied insights. Perhaps more likely, some messages may have counterproductive effects among high-risk gamblers, which is important for policy stakeholders to know when deciding on which messages to implement. The present therefore tested for these possibilities by requiring participants to complete the Problem Gambling Severity Index (“PGSI”; Ferris & Wynne, 2001).
The ten messages highlighted here were tested in a preregistered study, and compared for their effectiveness across positive and negative impacts on gambling attitudes and intentions. Testing a combination of pre-existing, altered and novel messages in a controlled study with UK gamblers and measuring the impacts of these messages, as well as their interaction with PGSI levels, enables an exploration of their relative potential effectiveness within a UK context.
Pre-Registered Research Questions (RQs)
The null hypothesis will be tested that for each rated dimension that there are no significant differences between each message for participants overall
For each message and rated dimension, an interaction model will be run to see if the message’s rating depends on gamblers’ Problem Gambling Severity Index (PGSI) levels
Participants will rate each message as being more relevant to gamblers at risk of harm than they are to themselves, and we will explore this difference’s robustness to different PGSI levels
Method
Materials, data, and analyses can be accessed at https://osf.io/bd2jk/, while the preregistration is available at https://osf.io/b3hfr. Ethical approval was granted by the University of Bristol’s School of Psychological Science Research Ethics Committee (#22746).
Participants
Participants were recruited through the online crowdsourcing platform Prolific. Eligible participants were required to be at least 18 years old, resident in the UK, and have reported engaging in at least one online gambling format beyond the lottery. Participants were compensated £1 (mean duration before exclusions = 6.1 min, equivalent to £9.9 per hour). We aimed to recruit an initial sample of 4,240 participants with a final target sample of 4,000 after data-quality exclusions. This sample size was chosen heuristically, based on our understanding that messaging interventions can have quite small effect sizes, and based on considerations regarding our access to research funds. We preregistered to first exclude the 5% fastest participants, and also exclude an anticipated 1% reporting that they were not paying attention at the end of the study. We used a fixed percentage for consistent exclusion rates rather than response-time thresholds based on the mean and standard deviation, as response times are always positively-skewed.
The study initially captured data from N=4,245 participants. Applying our preregistered criteria, we excluded eight responses for self-reported non-attention and 212 participants for completion times that were under 2.6 minutes (the 5% fastest). No participants were excluded due to missing data. Our final dataset comprised of N=4,025 participants. Demographic information and self-reported gambling engagement were automatically collected by Prolific and are displayed in Tables 1 and 2, respectively. Although participants were all required to have engaged in one non-lottery gambling format, the lottery was still the single most engaged-with gambling format (64%), which follows the pattern shown in the overall population. Bingo was the second-most engaged-with gambling format (58%), and this is generally seen to be, like the lottery, as a high-engagement gambling format with relatively weak links to harm (Allami et al., 2021). Constrastingly, slots were the third-most engaged-with gambling format (51%), and this is commonly seen as being a gambling format with relatively strong links to harm (either online or as part of land-based electronic gaming machines; Allami et al., 2021).
Demographic category | N (%) |
Sexa | |
Female | 2,034 (51%) |
Male | 1,991 (49%) |
Age: mean (SD) | 40 (12.4) |
Ethnicity | |
White | 3,352 (83%) |
Black | 273 (7%) |
Asian | 241 (6%) |
Mixed | 124 (3%) |
Other | 33 (1%) |
Not available | 2 (0%) |
Employment status | |
Full-time | 2,398 (60%) |
Part-time | 668 (17%) |
Not in paid work | 349 (9%) |
Unemployed and job seeking | 163 (4%) |
Starting soon | 26 (1%) |
Other | 106 (3%) |
Not available | 315 (8%) |
Demographic category | N (%) |
Sexa | |
Female | 2,034 (51%) |
Male | 1,991 (49%) |
Age: mean (SD) | 40 (12.4) |
Ethnicity | |
White | 3,352 (83%) |
Black | 273 (7%) |
Asian | 241 (6%) |
Mixed | 124 (3%) |
Other | 33 (1%) |
Not available | 2 (0%) |
Employment status | |
Full-time | 2,398 (60%) |
Part-time | 668 (17%) |
Not in paid work | 349 (9%) |
Unemployed and job seeking | 163 (4%) |
Starting soon | 26 (1%) |
Other | 106 (3%) |
Not available | 315 (8%) |
a) Prolific does not provide additional options for sex (e.g., non-binary)
Gambling formatb | N (%) |
Lottery | 2,571 (64%) |
Bingo | 2,328 (58%) |
Slots | 2,058 (51%) |
Roulette | 1,552 (39%) |
Blackjack | 1,550 (38%) |
Poker | 1,467 (36%) |
Race/sports betting | 1,260 (31%) |
Virtual sports betting | 1,151 (29%) |
Video poker | 461 (12%) |
Pachinko | 154 (4%) |
Baccarat | 135 (3%) |
Craps | 133 (3%) |
PGSI: mean (SD) | 3.1 (4.7) |
PGSI categoriesc | |
No-risk (0) | 1,558 (39%) |
Low-risk (1 or 2) | 1,070 (27%) |
Moderate-risk (3 to 7) | 854 (21%) |
High-risk (8+) | 543 (13%) |
Gambling formatb | N (%) |
Lottery | 2,571 (64%) |
Bingo | 2,328 (58%) |
Slots | 2,058 (51%) |
Roulette | 1,552 (39%) |
Blackjack | 1,550 (38%) |
Poker | 1,467 (36%) |
Race/sports betting | 1,260 (31%) |
Virtual sports betting | 1,151 (29%) |
Video poker | 461 (12%) |
Pachinko | 154 (4%) |
Baccarat | 135 (3%) |
Craps | 133 (3%) |
PGSI: mean (SD) | 3.1 (4.7) |
PGSI categoriesc | |
No-risk (0) | 1,558 (39%) |
Low-risk (1 or 2) | 1,070 (27%) |
Moderate-risk (3 to 7) | 854 (21%) |
High-risk (8+) | 543 (13%) |
b) Participants could choose more than one format
c) The lowest and highest PGSI risk categories that previously labeled individuals as ‘non-problem’ and ‘problem gambler’ will herein be referred to as ‘no-risk’ and ‘high-risk,’ respectively, to remove stigmatizing language.
Participants completed the Problem Gambling Severity Index (PGSI), a widely-used gambling screening measure (Ferris & Wynne, 2001), with results showing the distribution of PGSI scores across the original four groupings shown in Table 2. The distribution of PGSI scores confirm a widespread trend of online samples having higher average risk levels than population base-rates, which allowed us to efficiently sample from the important group of high-risk gamblers (Russell et al., 2021).
Procedure
After receiving an information sheet and providing informed consent, participants were presented with the ten harm prevention messages shown in Table 3 in a randomized sequence. Two existing messages were changed by replacing “bettors” or “betting” with the synonymous words “gamblers” or “gambling,” which made the language more in keeping with the other messages. We also chose to delete the word “over” from the rest of the message “99% of gamblers lose in the long run,” as larger percentages may be perceived as less believable, and this is also more in line with what was actually said by the gambling industry executive (Alexander, 2020).
Message | Abbreviation | Source (and any edits) |
99% of gamblers lose in the long run | Lose Long Run | Rockloff et al., 2024 (edited to delete “over”, and replace “bettors” with “gamblers” |
Chances are you’re about to lose | Chances Are | Chapman & Priestly, 2022 |
Gambling adverts show people winning, but most gamblers lose overall | Adverts Show | Novel |
Gambling comes at a cost | Comes At Cost | GambleAware, 2024 |
Odds are: they win | They Win | Mills et al., 2023 |
Odds are: you lose | You Lose | Novel |
What are you prepared to lose today? Set a deposit limit | Prepared To Lose | Chapman & Priestly, 2022 |
What does gambling cost you? Stop in time | Cost You | De Jans et al., 2023 |
Whether you win or lose, the gambling companies are always the real winners | Real Winners | Davies et al., 2022 (edited to replace “betting companies” with “gambling companies”) |
You win some. You lose more | You Win Some | Chapman & Priestly, 2022 |
Message | Abbreviation | Source (and any edits) |
99% of gamblers lose in the long run | Lose Long Run | Rockloff et al., 2024 (edited to delete “over”, and replace “bettors” with “gamblers” |
Chances are you’re about to lose | Chances Are | Chapman & Priestly, 2022 |
Gambling adverts show people winning, but most gamblers lose overall | Adverts Show | Novel |
Gambling comes at a cost | Comes At Cost | GambleAware, 2024 |
Odds are: they win | They Win | Mills et al., 2023 |
Odds are: you lose | You Lose | Novel |
What are you prepared to lose today? Set a deposit limit | Prepared To Lose | Chapman & Priestly, 2022 |
What does gambling cost you? Stop in time | Cost You | De Jans et al., 2023 |
Whether you win or lose, the gambling companies are always the real winners | Real Winners | Davies et al., 2022 (edited to replace “betting companies” with “gambling companies”) |
You win some. You lose more | You Win Some | Chapman & Priestly, 2022 |
Participants were asked to evaluate each message based on agreement with the four statements, which formed the study’s dependent measures and were also presented in a random order. Two statements were newly introduced:
“This message challenges me to try to win at gambling” and
“This message makes me want to gamble less”
The other two statements were derived from discrepancies observed between Chapman and Priestly (2022) and Newall, Torrance et al. (2024):
“This message is relevant to me” and
“This message is relevant to people experiencing gambling harms”
Responses were recorded on a seven-point Likert scale, ranging from “strongly disagree” to “strongly agree.” Subsequently, participants completed the PGSI. To help improve data quality, a self-reported carelessness check was implemented: “In your honest opinion, should we use your data in our analyses in this study? (Do not worry, this will not affect your payment, you will receive the payment code either way.)” (Brühlmann et al., 2020). Only data from participants who responded”yes” to this question were retained for analysis. After exclusions, participants completed the task in 6.3 minutes on average (SD=8.0).
Analysis
All analyses were conducted using R 4.4.1 (R Core Team, 2023). Mixed effect models were estimated using the function lmer from the package lme4 (Bates et al., 2015). Model estimation was achieved using Restricted Maximum Likelihood and p-values were calculated using Kenward-Roger’s method for estimating degrees of freedom (McNeish, 2017) using the package lmerTest (Kuznetsova et al., 2017), as they have been shown to produce acceptable Type I errors (Luke, 2017). To further reduce Type I errors, all multiple comparisons were adjusted using the Sidák method (Gramm et al., 2007).
For RQ1, we ran four separate mixed effects models, one for each statement. There was a single fixed predictor (message), which was a factor identifying each of the 10 messages, and one random intercept for each participant. Due to the four statements being analyzed concurrently, we used a base confidence level (alpha) of p=.01 to reduce the potential for Type I error inflation. To identify differences between the responses given to each message, we present the results based on a compact-letter-display (CLD) analysis which identifies groupings of mean responses that are not significantly different (nsd) from each other using pairwise comparisons across the 10 messages, corrected for multiple comparisons using Sidák’s method, which equals to a pre-adjusted alpha threshold of p=.00022 for significant results.
For RQ2, we added centered PGSI scores as a covariate (with interaction) to each of the four models above, and we tested the effects of PGSI scores via a significant main effect and interaction with the fixed factor (message). To evaluate how PGSI affected each message individually, we tested the slope of PGSI score interaction for each message against zero, and present the results from a CLD analysis identifying different groupings of the slope of the PGSI interaction which are not different from each other using pairwise comparisons across the 10 messages, both corrected for multiple comparisons using the Sidák method.
For RQ3, we calculated a new dependent variable (“DV”) as the difference between the response given to the statement “relevant to people experiencing gambling harms” minus the answer to the statement “relevant to me.” Positive values of this difference indicate that participants rated the message as more relevant to people experiencing gambling harms than to themselves, and vice-versa. We ran a similar model as above, using this difference as the DV, with a fixed factor for message, PGSI score, their interaction, and a random effect for each participant. Results were evaluated by comparing the difference against zero for each message, as well as the slope of the PGSI score interaction against zero for each message, both corrected for multiple comparisons using the Sidák method. All analyses were preregistered, apart from the comparisons with the indifference mid-point, as well as the estimated marginal means shown in Figure 3C, which were exploratory.
For all PGSI-related analyses, PGSI was used as a continuous variable, which avoids the loss of information which occurs when dichotomizing a variable into sub-groups (e.g., the high-risk gambling group with PGSI scores of 8+). Furthermore, the analysis used is consistent with the public health conceptualization of harm as a continuous spectrum.
Results
Research Question 1: Differences between each message
Outcomes for Research Question 1’s four dependent measures are shown in Figure 1. In Figure 1A, which shows the outcomes for the “challenges me” measure, it can be seen that participants did not on average find that the messages challenged them to try to win at gambling (all message averages were significantly below the indifference mid-point of 4, all ps<.001). However, there was significant variation between the messages (𝜒2(9)=1683, p<.001, pseudo-R2=0.43), with They Win presenting the greatest challenge (M=3.4), with the remaining messages being placed in four descending groups via the CLD. By contrast, the novel framing You Lose was in the third-lowest grouping (M=2.8). Cost You and Comes At Cost presented the joint-lowest challenge to gamble (both M=2.5). The difference from greatest to lowest, a measure of unstandardized effect size, was 0.92, almost one full point change in the response scale (i.e., between ‘disagree’ and ‘somewhat disagree’).
Model-predicted marginal means. Error bars represent 99% confidence intervals around the means. Horizontal brackets identify the groups of messages that were not significantly different (nsd) from each other at padj=.01 (using Sidák’s method for multiple comparisons, equivalent to p=.00022 before adjustments).
Model-predicted marginal means. Error bars represent 99% confidence intervals around the means. Horizontal brackets identify the groups of messages that were not significantly different (nsd) from each other at padj=.01 (using Sidák’s method for multiple comparisons, equivalent to p=.00022 before adjustments).
Figure 1B shows outcomes for the “makes me want to gamble less” measure. On average, participants thought that each message would make them want to gamble less (all message averages were significantly above the indifference mid-point of 4, all ps<.001). But the ranking of the messages were significantly different with this measure (𝜒2(9)=2381, p<.001, pseudo-R2=0.44), with Cost You (M=4.4) and Comes At Cost (M=4.8) ranking less well here. One similarity is that They Win was again the worst message on this measure (M=4.3), with the novel You Lose framing again performing better, being placed in the CLD’s second-highest grouping (M=5.0). Lose Long Run was the single best message on this measure (M=5.2). The difference from worst to best was 0.94, again close to one full point change in the response scale (i.e., from ‘neither agree nor disagree’ to ‘somewhat agree’).
Figure 1C shows outcomes for the “relevant to me” measure. Only Real Winners was rated as being on average relevant to the participants themselves (M=4.1, 99% CI = [4.2, 4.0], p=.001). Overall, there was significant variation between the messages (𝜒2(9)=978, p<.001, pseudo-R2=0.66). Five messages were joint-second highest, including Adverts Show (M=4.0) and You Lose (M=4.0). They Win (M=3.8) was in the second-lowest group, therefore again performing worse than You Lose, with Cost You being ranked as the single worst message (M=3.5). The difference from worst to best was 0.62.
Figure 1D shows outcomes for the “relevant to people experiencing gambling harms” measure. On average, participants thought that each message was relevant to this group (all message averages were significantly above the indifference mid-point of 4, all ps<.001, and the highest responses overall). Again, there was significant variation between the messages (𝜒2(9)=1814, p<.001, pseudo-R2=0.36), They Win was seen as being the least relevant to participants (M=5.1), with the novel You Lose framing again performing better and in the CLD’s second-highest group (M=5.8). Lose Long Run was again the single best message on this measure (M=6.0). The difference from worst to best was 0.84.
Research Question 2: Interaction with PGSI
Figure 2 demonstrates outcomes for Research Question 2, exploring whether messages’ ratings across the four dependent measures depended on participants’ PGSI scores. In Figure 2A, which shows interaction effect outcomes for the “challenges me” measure, it can be seen that all individual interaction effects were significantly positive, meaning that all messages were seen as being a greater challenge for participants with higher PGSI scores. However, there were no significant differences between each message in terms of the strength of this interaction. Adding PGSI scores to the model significantly improved its fit in comparison to RQ1 (𝜒2(10)=272, p<.001).
Model-predicted marginal means. Error bars represent 99% confidence intervals around the means. Horizontal brackets identify the groups of messages whose PGSI slopes were not significantly different (nsd) from each other at padj=.01.
Model-predicted marginal means. Error bars represent 99% confidence intervals around the means. Horizontal brackets identify the groups of messages whose PGSI slopes were not significantly different (nsd) from each other at padj=.01.
Figure 2B shows interaction effect outcomes for the “makes me want to gamble less” measure. Adding PGSI scores to the model significantly improved model fit (𝜒2(10)=58, p<.001). The only messages without significant positive interactions were They Win (b=0.01, 99% CI = [0.03, -0.01]) and Adverts Show (b=0.02, 99% CI = [0.04, -0.00]), meaning that, for all other messages, participants with higher PGSI scores reported wanting to gamble less. However, again there was limited between-message difference in the slope of this interaction, with the top nine messages all being grouped together (including the highest Cost You, b=0.03, 99% CI = [0.02, 0.05]), as well as the lowest nine messages (including They Win and Adverts Show). Furthermore, these estimated interactions were all small in magnitude.
Figure 2C shows interaction effect outcomes for the “relevant to me” measure. The model with PGSI scores showed a significant improvement over a model without them (𝜒2(10)=934, p<.001). Here all messages had significant positive interaction effects, showing that they were judged to be more relevant to participants with higher PGSI scores (see OSF results Section 3.4.3 at https://osf.io/zhxg3?view_only=0b2f3a2f01ab419e99af7b628c37c785). However, the interaction effects were largely similar in estimated size, with the CLD grouping the largest seven (including the largest interaction, Lose Long Run, b=0.15, 99% CI = [0.14, 0.17]) and smallest eight messages together (including the smallest interaction, They Win, b=0.13, 99% CI = [0.11, 0.15]).
Figure 2D shows interaction effect outcomes for the “relevant to people experiencing gambling harms” measure. Here, none of the messages had a significant interaction, and none of these were identified as being significantly different (𝜒2(10)=18, p=.05). PGSI score did not appear to matter when it came to a message being considered as relevant to people experiencing gambling harms (see OSF results Section 3.4.4).
Research Question 3: Relevant to Gamblers at Risk of Harm vs. to Themselves
The model fit the data significantly better than the null model (𝜒2(19)=1560, p<.001, R2=0.59). All messages were seen as being on average more relevant to gamblers at risk of harm than to themselves (Figure 3A), which confirms Research Question 3. This is represented by significant positive means for the difference (Gamblers − Me) for each message. The smallest difference is for They Win (M=1.35, 99% CI=[1.25, 1.45]), while the largest differences are for Comes At Cost (M=1.91, 99% CI=[1.81, 2.01]) and Lose Long Run (M=1.98, 99% CI=[1.88, 2.07]).
Model-predicted marginal means for the difference Gamblers at risk - Me. Positive values indicate that participants rated the message as more relevant to gamblers at risk of harm than to themselves, and vice-versa. The vertical line in plot C identifies the mean PGSI. Error bars represent 99% confidence intervals around the means. Horizontal brackets identify the groups of messages whose mean differences and PGSI slopes were not significantly different (nsd) from each other at padj=.01.
Model-predicted marginal means for the difference Gamblers at risk - Me. Positive values indicate that participants rated the message as more relevant to gamblers at risk of harm than to themselves, and vice-versa. The vertical line in plot C identifies the mean PGSI. Error bars represent 99% confidence intervals around the means. Horizontal brackets identify the groups of messages whose mean differences and PGSI slopes were not significantly different (nsd) from each other at padj=.01.
The slopes of the effect of PGSI on the differences for all messages were significantly negative, with the CLD grouping both the top six and bottom six messages together as seen in Figure 3B. In effect, this meant that the overall tendency for participants to view messages as being less relevant to themselves was reversed for participants with very high PGSI scores (equal to or above 17, or about 3% of the participants), who contrastingly saw messages as being more relevant for themselves than for gamblers at risk of harm (Figure 3C), due to higher responses given to the “relevant to me” question as PGSI increased (see OSF results Section 3.4.3).
Exploratory Analysis: Order Effect
As each participant was shown 10 messages in a random order, we also evaluated if there was an order effect of message presentation. We added a new variable identifying the order in which each message was shown (from 1 to 10) as a main effect and interaction to the H1 models above, in an exploratory analysis that was not preregistered.
Overall, there was an order effect for two of the measures. It was significantly positive for “challenges me” (b=0.006, CI=[0.000, 0.012], p=.009), significantly negative for “makes me want to gamble less” (b=-0.013, CI=[-0.019, -0.007], p<.001), and not significant for “relevant to me” (b=-0.002, CI=[-0.006, 0.003], p=.35) or “relevant to people experiencing gambling harms” (b=0.002, CI=[-0.003, 0.007], p=.27). Therefore, responses to the “challenges me” question increased (slightly, given the small magnitude of the coefficient b) as the task progressed and more messages were shown, while responses to the “makes me want to gamble less” question decreased slightly. The combined effect, after 10 questions, was to increase the former by 0.05 and decrease the latter by 0.12, on average across all messages, which could be considered as relatively small effects.
There were also significant interactions between order effect and message for all four measures (all ps < .001), although not all individual slopes for each message were significant, after adjusting for multiple comparisons (see OSF supplement for detailed results). An analysis estimating the responses to each question assuming messages are seen first (as opposed to at the average midpoint after five messages as in the main results), resulted in their slight re-ordering, because of the different ways in which responses changed over time. To summarize, if the overall ranking (see Table 4 below) had been conducted with estimated first responses (1st message shown) instead of their averages, the best and worst messages would not change, but we would see an improvement in Adverts Show moving it from 5th into 3rd place (pushing You Lose and Comes at Cost to 4th and 5th respectively), while Cost You was worse and would be ranked 9th instead of joint 8th. The differences between the worst and best scoring messages for each measure, an indication of unstandardized effect size, increase at the first message shown, from [0.62, 0.94] to [0.82, 1.23], or a larger differentiation between messages.
Overall Rank | Average Rank | Message | Abbreviation |
1 | 2.5 | 99% of gamblers lose in the long run | Lose Long Run |
2 | 3.25 | Whether you win or lose, the gambling companies are always the real winners | Real Winners |
3 | 4 | Odds are: you lose | You Lose |
3 | 4 | Gambling comes at a cost | Comes At Cost |
5 | 4.5 | Gambling adverts show people winning, but most gamblers lose overall | Adverts Show |
6 | 5.75 | Chances are you’re about to lose | Chances Are |
7 | 6.75 | You win some. You lose more | You Win Some |
8 | 7.25 | What are you prepared to lose today? Set a deposit limit | Prepared To Lose |
8 | 7.25 | What does gambling cost you? Stop in time | Cost You |
10 | 9.75 | Odds are: they win | They Win |
Overall Rank | Average Rank | Message | Abbreviation |
1 | 2.5 | 99% of gamblers lose in the long run | Lose Long Run |
2 | 3.25 | Whether you win or lose, the gambling companies are always the real winners | Real Winners |
3 | 4 | Odds are: you lose | You Lose |
3 | 4 | Gambling comes at a cost | Comes At Cost |
5 | 4.5 | Gambling adverts show people winning, but most gamblers lose overall | Adverts Show |
6 | 5.75 | Chances are you’re about to lose | Chances Are |
7 | 6.75 | You win some. You lose more | You Win Some |
8 | 7.25 | What are you prepared to lose today? Set a deposit limit | Prepared To Lose |
8 | 7.25 | What does gambling cost you? Stop in time | Cost You |
10 | 9.75 | Odds are: they win | They Win |
For example, <em>Lose Long Run</em> was ranked 4<sup>th</sup>, 1<sup>st</sup>, 1<sup>st</sup>, and 4<sup>th</sup>, while <em>They Win</em> was ranked 10<sup>th</sup>, 10<sup>th</sup>, 10<sup>th</sup>, 9<sup>th</sup>, across the four measures (with the first measure, “challenges me,” reversed). This analysis was conducted at the midpoint of 10 messages shown. If we estimated the rankings at the first message shown, <em>Adverts Show</em> would be at 3<sup>rd</sup> place (with <em>You Lose</em> at 4<sup>th</sup> and <em>Comes at Cost</em> at 5<sup>th</sup>) while <em>Cost You</em> would be 9<sup>th</sup> instead of joint 8<sup>th</sup>.
Discussion
Messaging is often regarded as a way of raising awareness of gambling harms among large segments of the population (Ray et al., 2024), while more restrictive measures, such as self-exclusion, are generally deemed more appropriate for those already experiencing significant harms. Although harm prevention messages can be designed with multiple potential strategies in mind, the present work provided an in-depth exploration of existing and novel messages that warn gamblers about the likelihood of losing money.
Despite having a similar aim, systematic differences were found between the messages regarding the four dimensions of RQ1. How each message performed on average is summarized here in an exploratory fashion by ranking each message from best to worst on each dimension, and then calculating each message’s multidimensional average rank (reverse scoring the “challenges me” measure). Table 4 shows the outcomes of this ranking, where Lose Long Run and Real Winners were ranked highest, while the five lowest-ranked messages were all messages currently in use in Australia (Prepared To Lose; You Win Some; Chances Are), Belgium (Cost You), and the UK (They Win). Policymakers in these jurisdictions could consider implementing the novel better-ranked messages in future, especially since any positive effects of currently-used messages may weaken with repeated exposure over time (Cacioppo & Petty, 1979; Newall, Rockloff, Hing, Thorne, et al., 2023). One clear improvement appears to be replacing the existing UK message They Win (Mills et al., 2023), which was the lowest-ranked here, with the novel reframing of You Lose which was ranked joint third. This finding conceptually replicates previous work showing that the framing of statistical information matters in gambling (Newall et al., 2020).
Some of the findings do contradict messaging’s purpose as a population-based harm prevention measure (Ray et al., 2024). Only one message was perceived as being relevant to the average participant (“relevant to me”), with RQ2’s analysis showing that messages only began to be perceived as personally relevant as PGSI scores increased. Therefore, these message’s effects might be enhanced if they are shown in, for example, online gambling platforms and land-based gambling venues, where people who are already experiencing some level of gambling-related harm predominate. The significant interaction effect in RQ3’s analysis showed that only participants with very high PGSI scores (17 or above) saw the messages as being more relevant to themselves than they were to gamblers experiencing harm. This is actually quite a high bar for messages to exceed, and so this shows self-awareness among participants experiencing very high levels of harm. Furthermore, the significant interaction effect involving PGSI for the “challenges me” measure suggests that these messages may also counterproductively be seen as a greater challenge by participants with higher PGSI scores. Overall, these findings underscore to us also that messaging should only be used as one of a number of different interventions directed at attempting to prevent gambling-related harm.
These findings are subject to various limitations. Crowdsourced samples do not represent gamblers as a whole, for which representative surveys are more appropriate (Pickering & Blaszczynski, 2021). The present work focused on gamblers, so future work should see whether these and other messages can potentially prevent non-gamblers from experiencing harm by either taking up gambling or from another person’s gambling. Self-report studies are limited by the dependent measures that are selected by researchers, and different outcomes might have occurred with the measures used in other studies (Chapman & Priestly, 2022). Randomized controlled trials (RCTs) with behavioral outcomes are more ecologically-valid than self-report studies, but must be adequately powered given that effect sizes tend to become even smaller in naturalistic environments (Auer & Griffiths, 2015; Heirene & Gainsbury, 2021). Furthermore, RCTs require active collaboration with a gambling operator, which limits the number of researchers with access to this methodology (Cassidy et al., 2013).
These findings are also limited to the 10 messages tested here, and so input from multiple stakeholders is needed to continue generating and testing as many messages as possible. Message effectiveness may be influenced by past exposure (McCullock et al., 2024), and we note that the lowest-ranked message was the only one in current UK use (by one local government), which might have therefore been the message participants were most likely to have been previously exposed to. We did not measure prior message exposure. There were also significant order effects, with responses changing as participants were exposed to multiple messages during the experiment. While we believe this to be a realistic setting – messages do not exist in a vacuum and campaigns often employ multiple messages to reduce message fatigue (McCullock et al., 2024), some carryover effects were observed across messages. Further work is therefore needed to understand order effects in gambling messages, particularly in naturalistic environments. These ten messages were also not compared to other potential message types (Harris et al., 2018; Mutti-Packer et al., 2022), which future work should explore. Future work should continue to look to other areas, such as public health more generally (Abroms & Maibach, 2008), or fear-based tobacco warnings (Witte & Allen, 2000), as further inspiration for harm prevention message design.
Further considerations are also relevant to policymakers considering harm prevention messages as an intervention. The present results do not compare messaging against a control condition of no messaging, nor to other potential interventions, such as restrictions on advertising or product availability. Therefore, the present findings are perhaps most relevant to policymakers who have already decided to implement or update their harm prevention messages, and now face the decision of selecting specific messages. On the one hand, the relatively small observed differences across many of these messages suggest that this policymaker might be best placed to choose a large number of related messages to rotate between, so as to reduce repetition-based fatigue (McCullock et al., 2024; Newall, Rockloff, Hing, Thorne, et al., 2023; Ray et al., 2024). On the other hand, some clear differences do also emerge and should inform this choice, especially when comparing the current “odds are: they win” message to the novel “odds are: you lose” message. These initial conclusions should furthermore be updated as more evidence emerges.
In conclusion, the present work has contributed to the understanding of harm prevention messages that warn gamblers about the expectation of losing money. This can inform policymakers and other stakeholders with an interest in this population-based intervention.
Acknowledgements
Thanks to Dan Riley for providing helpful input during the conceptualization of this project.
Funding
This research was funded by a Policy Support Fund grant awarded to Dr Philip Newall from the University of Bristol.
Disclosures
Philip Newall is a member of the Advisory Board for Safer Gambling – an advisory group of the Gambling Commission in Great Britain. In the last three years, Philip Newall has contributed to research projects funded by the Academic Forum for the Study of Gambling, Clean Up Gambling, Gambling Research Australia, NSW Responsible Gambling Fund, and the Victorian Responsible Gambling Foundation. Philip Newall has received honoraria for reviewing from the Academic Forum for the Study of Gambling and the Belgium Ministry of Justice, travel and accommodation funding from the Alberta Gambling Research Institute and the Economic and Social Research Institute, and open access fee funding from the Academic Forum for the Study of Gambling and Greo Evidence Insights.
Leonardo Weiss-Cohen has received open access fee funding from Gambling Research Exchange Ontario.
Elena Petrovskaya has received funding from the Academic Forum for the Study of Gambling and is currently funded by GREO.
Jamie Torrance has received, in the last three years: (1) Open access publication funding from Gambling Research Exchange Ontario (GREO), (2) Conference travel and accommodation funding from the Academic Forum for the Study of Gambling (AFSG), (3) A minor exploratory research grant from the ASFG and GREO.
Madison Palmer has no disclosures to make.
In relation to the past three years, L.Y.X. has been providing paid consultancy for Public Group International Ltd (t/a PUBLIC) (Companies House number: 10608507), which has been commissioned by the UK Department for Culture, Media and Sport (DCMS) to conduct independent research on understanding player experiences of loot box protections (October 2024 – Present). L.Y.X. undertook a brief period of voluntary work experience at Wiggin LLP (Solicitors Regulation Authority number: 420659) in London, England in August 2022. L.Y.X. has contributed and continues to contribute to research projects that were enabled by data access provided by the video game industry, specifically Unity Technologies (NYSE:U) (October 2022 – Present). L.Y.X. has been invited to provide advice to the UK Department for Digital, Culture, Media and Sport and its successor (the Department for Culture, Media and Sport; DCMS) on the technical working group for loot boxes and the Video Games Research Framework. L.Y.X. was the (co-)recipient of three Academic Forum for the Study of Gambling (AFSG) Postgraduate Research Support Grants (March 2022, January 2023, & July 2024) and a Minor Exploratory Research Grant (May 2024) that were derived from ‘regulatory settlements applied for socially responsible purposes’ received by the UK Gambling Commission and administered by Gambling Research Exchange Ontario (GREO) and its successor (Greo Evidence Insights; Greo). L.Y.X. has accepted funding to publish academic papers open access from GREO and the AFSG that was received by the UK Gambling Commission as above (October, November, & December 2022, November 2023, & May 2024). L.Y.X. was the recipient of an Elite Research Travel Grant 2024 [EliteForsk-rejsestipendium 2024] awarded by the Agency for Higher Education and Science of the Danish Ministry of Higher Education and Science [Uddannelses- og Forskningsstyrelsen under Uddannelses- og Forskningsministeriet] (February 2024). L.Y.X. has accepted conference travel and attendance grants from the Socio-Legal Studies Association (February 2022 & February 2023); the Current Advances in Gambling Research Conference Organising Committee with support from GREO (February 2022); the International Relations Office of The Jagiellonian University (Uniwersytet Jagielloński), the Polish National Agency for Academic Exchange (NAWA; Narodowa Agencja Wymiany Akademickiej), and the Republic of Poland (Rzeczpospolita Polska) with co-financing from the European Social Fund of the European Commission of the European Union under the Knowledge Education Development Operational Programme (May 2022); the Society for the Study of Addiction (November 2022 & March 2023); the organisers of the 13th Nordic SNSUS (Stiftelsen Nordiska Sällskapet för Upplysning om Spelberoende; the Nordic Society Foundation for Information about Problem Gambling) Conference, which received gambling industry sponsorship (January 2023); the MiSK Foundation (Prince Mohammed bin Salman bin Abdulaziz Foundation) (November 2023); and the UK Gambling Commission (March 2024). L.Y.X. has received honoraria from the Center for Ludomani for contributing parent guides about mobile games for Tjekspillet.dk, which is funded by the Danish Ministry of Health’s gambling addiction pool (Sundhedsministeriets Ludomanipulje) (March & December 2023); from the Fundació Pública TecnoCampus Mataró-Maresme [TecnoCampus Mataró-Maresme Foundation] for a guest lecture (November 2023); and from the YMCA (Young Men’s Christian Association) of Greater Toronto Youth Gambling Awareness Program for a presentation, which is funded by the Government of Ontario, Canada (March 2024). L.Y.X. received royalties by virtue of the copyright subsisting in some of his publications from the Authors’ Licensing and Collecting Society (ALCS) (Companies House number: 01310636) (March 2023 & 2024). A full gifts and hospitality register-equivalent for L.Y.X. is available via: https://sites.google.com/view/leon-xiao/about/gifts-and-hospitality-register. The up-to-date version of L.Y.X.’s full conflict-of-interest statement beyond the last three years is available via: https://sites.google.com/view/leon-xiao/about/conflict-of-interest.