Comparison of Two Ecological Momentary Intervention Modules for Treatment of Depression on Momentary Positive and Negative Affect

Depression is characterized by at least one of two main symptoms: depressed mood and loss of interest or pleasure (American Psychiatric Association, 2013). Both symptoms illustrate the important role of the affective system in depression (Gross & Jazaieri, 2014; Hofmann et al., 2012). Not surprisingly, a large body of research has investigated the role of positive and negative affect (PA and NA) in depression and how interventions change experienced affect (Aldao et al., 2010; Dunn, 2012; Sin & Lyubomirsky, 2009).

In the past two decades, researchers have increasingly used Ecological Momentary Assessment (EMA, also referred to as Experience Sampling Methodology or ESM) to study how affect fluctuates in daily life in people with depression (for reviews see: aan het Rot et al., 2012; Myin‐Germeys et al., 2018). EMA is a structured self-monitoring data collection technique, which allows repeated, real-time measuring of thoughts, feelings and behaviors during the day (Csikszentmihalyi & Larson, 2014; Shiffman et al., 2008). A typical design consists of up to ten short questionnaires a day for several days or weeks. EMA provides rich and detailed data leading to new insights in psychopathology, and has been suggested as a tool for treatment as well (Myin-Germeys et al., 2016; van Os et al., 2017).

Currently, only two randomized controlled trials (RCTs) that used EMA as an intervention (EMI) strategy for depressive symptoms also investigated concurrent changes in affect (for a review see: Colombo et al., 2019). These interventions combined repeated self-monitoring for several weeks with person-tailored feedback. In the first RCT by Kramer et al. (2014), 102 depressed adults received either six weeks of EMA with weekly PA-focused feedback sessions, six weeks of EMA without feedback, or treatment as usual (TAU). Although depressive symptoms reduced in the EMA-plus-feedback group, no significant differences in affect were found during the intervention period (Hartmann et al., 2015). In the second RCT by van Roekel et al. (2017), 69 young adults with elevated anhedonia scores received an EMI for two months. After the first month the participants were randomized into one of three groups 1) EMA, 2) EMA and lifestyle advice, or 3) EMA, lifestyle advice and a tandem skydive. Contrary to the results from the first RCT, the intervention groups showed higher PA compared to controls, while no group differences were found for depression.

The most obvious source of these mixed effects is the difference in study population. A second likely source is that the content of the EMI questionnaires and feedback differed between the RCTs. Besides differences in number of PA and NA questions the first RCT included questions about activities and events, whereas the second RCT added questions regarding stress, worrying and discomfort, which could have created differences in focus between the two interventions. A third possible source of the mixed findings are the different measurement schedules. The EMI in the first RCT consisted of ten measurements a day for three consecutive days each week, at semi-random intervals, compared to three times each day at fixed intervals in the second RCT. The potential influence of number of measurements is illustrated by Conner and Reid (2012), who found that a positive effect of measuring happiness once a day dissipated with increasing numbers of measurements per day in individuals with depressive symptoms. Together, these differences in population, content and methods make direct comparisons of these interventions impossible. To understand the role of content on affect, we need to look at EMI’s with identical methods and study populations, where only the content differs.

The recent ZELF-i¹ RCT (Bastiaansen et al., 2018) was set up to investigate the effects of two different EMI modules as an add-on to regular depression treatment. Participants who were randomized into one of the two intervention modules engaged in similar procedures, but with a different focus of the questionnaires and the feedback. Both groups engaged in 28 days of EMA (five times per day) and received weekly descriptive feedback reports, next to or preceding the start of TAU. The first part of each EMA questionnaire measured momentary affect and was identical between groups, whereas the second part and the weekly feedback reports had a different focus. One module focused on activities and PA (“Do-module”), whereas the other focused on thoughts and NA (“Think-module”). This design allowed us to directly compare the effects of a stronger focus on activities and PA versus thoughts and NA.

The focus on activities and PA in the Do-module incorporates some of the core aspects of behavioral activation (Cuijpers et al., 2007; Dimidjian et al., 2011; Kanter et al., 2010). Classical behavioral activation therapy aims to counter the passivity and NA symptoms in depression by teaching patients to avoid behaviors that decrease mood and engage in behaviors that improve mood (Hopko et al., 2003). One prominent tool to achieve these goals is for patients to monitor activities and rate the experienced pleasure, which is incorporated in the Do-module. Furthermore, both the broaden and build theory (Fredrickson, 2004) and evidence from positive psychology interventions (Sin & Lyubomirsky, 2009) suggest that repeated focus on what goes well can result in a positive spiral of activities and PA. Similarly, a study by Dainer-Best et al. (2018) shows that repeated focus on positive cues (e.g. achievements) can lead to increased positive self-referential cognition, which in turn is thought to help reduce depressive symptoms. The feedback reports in the Do-module sustain this focus by showing the experienced PA, but not NA, in relation to the reported activities.

In the Think-module, participants monitor their thoughts and feelings and receive feedback on how these thoughts were associated with minor daily events and experienced NA. This could enable them to identify and target maladaptive thinking patterns, a key component of cognitive therapy (Beck, 2005; Cuijpers et al., 2016). The feedback reports in the Think-module support this process by showing the experienced NA, but not PA, in relation to thoughts and daily events.

Regardless of focus, both modules could influence affect through various memory processes involved in having participants reflect on the past three hours five times a day. Directly, by automatically triggering emotions associated with recalled events (see for example Holmes & Mathews, 2010 for a review), but possibly also indirectly through increased memory specificity. Depressive symptoms have repeatedly been associated with reduced memory specificity (Williams et al., 2007), and several studies have shown promising results of various memory training interventions on depressive symptomatology (see for example Hitchcock et al., 2016; Raes et al., 2009; or Watkins et al., 2009). These interventions typically entail some form of autobiographical memory recall, adding specificity by focusing on experienced emotions or context, which is comparable to the EMI questionnaires.

Despite the potential of both modules to positively influence affect, repeated self-monitoring could also have negative effects. Both modules might lead to an increase in self-focused attention, which has been associated with NA (Mor & Winquist, 2002). Furthermore, the focus on activities and PA in the Do-module could lead to a realization of all that does not go well, and consequently decrease PA. In the Think-module, focusing on thought patterns could invite rumination, which has repeatedly been associated with NA (Thomsen, 2006). Although it is conceivable that a difference in focus will lead to different effects on momentary affect, the conflicting ideas and theories in the literature preclude clear hypotheses regarding the direction of effects. We will therefore use two-sided tests to investigate whether there is a differential effect of module type (Do vs. Think) on momentary PA or NA. Given that all participants received TAU next to the intervention, we further hypothesize that, on average, both groups will show an increase in PA and decrease in NA.

For this study we included the 110 participants who were randomized into the treatment arms of the ZELF-i RCT (ZELF-i, Dutch Trial Register, NTR5707, http://www.trialregister.nl). The total sample consisted of 161 patients who were assigned to treatment for unipolar depression in one of five study locations. Exclusion criteria were: required crisis intervention, presence of psychotic or manic symptoms and incapability of following research procedures due to inadequate Dutch language proficiency or significant auditory or visual impairments. All participants provided written informed consent before enrolment.

During clinical intake patients were informed about the study and screened for eligibility. Interested eligible patients were subsequently contacted by a researcher for an on-site study intake during which randomization was performed and the EMI program tested, explained and trained once. Randomization was stratified based on current psychotherapy and use of antidepressant medication. A full description of the design, procedures and primary outcomes of the ZELF-i trial are published elsewhere (For the protocol paper see Bastiaansen et al., 2018, for outcomes see 2020). The ZELF-i study was approved by the Medical Ethical Committee of the University Medical Center Groningen (no. 2015/530).

Both ZELF-i modules consisted of 28 consecutive days of EMA in which participants filled out brief questionnaires on their own smartphones five times a day. For each measurement a link to the questionnaire, hosted on a secured website for routine outcome monitoring (RoQua, www.roqua.nl), was sent by text message after which participants had 30 minutes to complete the survey. The measurements were set at fixed moments during waking hours with an interval of 3 hours, programmed individually to optimally fit the participants daily routine (e.g., during breaks at work, see Figure 1a for an example). Each measurement comprised a momentary part, a module-specific retrospective part (past three hours) and a module-specific prospective (next three hours) part (see Figure 1b). In both modules momentary well-being (“How do you feel right now”), momentary affect (PA/NA, see ‘measures’) and momentary physical state (i.e., hunger, tiredness, physical discomfort and impact of drugs/alcohol) were measured first (for the full item list see Bastiaansen et al., 2018).

Participants in the Do-module then retrospectively recorded experienced pleasure, motivation, physical activity, busyness and time spent at home, in pleasant social contexts and outdoors. Participants additionally recorded which activities they performed. A list of common activity categories was provided, with the option of adding an extra activity using a text box. For each performed activity, participants selected the best fitting category. Next, participants prospectively recorded anticipatory pleasure and motivation.

Participants in the Think-module retrospectively recorded the amount of focus on feelings and amount of brooding. Participants then recorded which negative events had taken place. A list of possible negative event categories was provided, with the option of adding an extra event using a text box. For each event participants selected the best fitting category. Next, participants indicated how unpleasant the events were taken together and whether they experienced negative thoughts about themselves or their situation. This procedure was repeated for positive events and positive thoughts. Lastly, participants prospectively recorded worrying.

Personalized feedback reports were automatically generated by RoQua and then emailed as a pdf by a research assistant to the participant after each week of EMA, with each successive report containing richer information. The Do-module reports comprised various graphs showing PA patterns and associations between PA and activities, whereas graphs in the Think-module focused on NA over time and associations with thinking patterns.

Treatment as usual was provided for all participants when available, irrespective of participation in the ZELF-i study. Most participants started a form of psychotherapy during the intervention period. This psychotherapy most often consisted of some form of cognitive behavioral treatment, in combination with a diverse number of other treatments (For further details see Bastiaansen et al., 2020).

Momentary affect: For momentary affect, participants scored emotional adjectives on visual analogue scales ranging from “not at all” (0) to “very much” (100). The adjectives used were balanced on both the valence dimension (i.e., positive and negative) and arousal dimension (high/low) of emotional experience (Watson & Tellegen, 1985). PA was measured with the adjectives cheerful (e.g. “I feel cheerful”), relaxed, satisfied, energetic, enthusiastic and calm. NA was measured with the adjectives irritated, down, indifferent, anxious, listless and stressed. The order of the items was fixed. Similar to other studies in the field (Hartmann et al., 2015) a PA and NA scale score were calculated by averaging the positive and negative adjectives, respectively, resulting in a score between 0 (not at all) and 100 (very much) per person per measurement. Multilevel reliability analyses following Nezlek (2017) showed moderate reliability for the PA items (α = 0.78) and fair reliability for the NA items (α = 0.58). For each participant the first five measurements (1 day) were excluded to reduce the effect of a potential initial elevation bias (Shrout et al., 2018), resulting in a maximum of (27 * 5) 135 measurements. Furthermore, all measurements that were not completed within a half-hour window were discarded.

To reduce experimenter bias, all analyses and data handling procedures were preregistered before any analyses were performed (https://osf.io/4e52q/), but after data collection was completed. Blinded analyses were not possible due to the researchers’ knowledge of the unequal number of dropouts between the groups. Furthermore, the preregistered exploratory analysis on the number of feedback reports read was not possible because there were no substantive subgroups of participants that did not read the feedback reports (Table S1). All other analyses were otherwise performed as preregistered. All analysis codes, outcomes and supplements can be found online (https://osf.io/bg7pr/).

For both PA and NA, we applied an identical multilevel model to accommodate the nested structure of the data, with measurements (level 1) nested within individuals (level 2). We used R (R Core Team, 2019) and the lme4 (Bates et al., 2015) and lmerTest (Kuznetsova et al., 2017) packages to perform a linear mixed model (LMM) analysis of the relationships between group and momentary affect over time. The model (Figure S1) included group and time as fixed effects. To test our hypothesis on differential effects over time the interaction term group * time was included. As depressive symptoms do not necessarily change linearly over time (see for example Dinga et al., 2018), we estimated whether model fit improved with a quadratic effect for time (Table S2). This quadratic effect is technically an interaction of time with time, which indicates whether the effect of time on depressive symptoms differs over the period under inspection (e.g. stronger changes shortly after intervention start than later on). The model included random intercepts and slopes for the linear time variable, effectively allowing participants to vary in their experienced affect at the start (T0) and in trajectories of affect change over time. The model was fitted using restricted maximum likelihood estimation and correlated random slopes and intercepts. As indication of the random effects the variance of both intercept and slope are reported.

Baseline characteristics of the two groups are shown in Table 1. The groups were very similar with respect to gender, age, educational level and depression level at intake. In total, 110 participants completed 10,173 valid measurements, with an average of 92.5 measurements per participant. After controlling for dropouts (Do: 7, Think: 13), by-group boxplots of the percentage of completed measurements showed a similar spread for participants in the Do-module (median % complete = 80.4, IQR = 66.9 – 89.5) and Think-module (median % complete = 80.4, IQR = 65.7 – 90.0; Figures S2 & S3). The percentage of completed measurements at each time point showed a small decrease over time (Figure S4). After removing the first five measurements (one day) of each participant, we analyzed the data of 107 participants, who completed 9,713 measurements in total (3 participants dropped out of the study on the first day).

All mixed model assumptions were satisfied (Figures S5 to S8 and Table S3). The results of the LMM analyses for PA and NA are presented in Table 2 and Table 3, respectively (see tables S4 and S5 for results under maximum likelihood). The fixed effects show the results for the typical participant and are represented by the four regression lines in Figure 2. Contrary to our primary hypothesis, no significant (p > .19) differences were found between the two modules on the development of PA and NA over time. Overall, PA showed an initially decreasing trend, which leveled off towards the end of the intervention period. NA did not change significantly over time.

The random effects, representing the differences between participants across groups, indicated a large variance in both PA and NA starting levels (S² PA intercept = 145; S² NA intercept = 212) and linear changes over time (S² PA slope = 0.38; S² NA slope = .47). This heterogeneity is illustrated by the large spread of the individual predicted regression lines in Figure 3. The lines were created by using the model estimates described in Tables 2 and 3 and adding the individual model residuals. The intraclass correlation (ICC) indicating the proportion of variance accounted for by participants (level 2) was .51 for PA and .63 for NA. This effectively means that more than half of the residual variance is explained by between-person differences.

The effect size f² was .14 for both PA and NA, indicating that the full models explain 14% of the variance in momentary affect relative to the remaining unexplained variance, which is a small to medium effect according to Lorah (2018).

To investigate the influence of the first five measurements and dropouts, both PA and NA models were re-run twice; once with the first five measurements included and once with the dropouts removed. No large differences were found with the original models, for details see tables S6 to S9. Upon reviewer suggestion, we additionally included covariates to account for within-week and within-day (time of day) trends of PA and NA. Again no large differences were found with the original models; details can be found in tables S10 to S13.

In this study we compared the effects of two EMI modules on momentary affect in 110 individuals with depression. The modules employed similar methods but differed in the content of the EMA questionnaires and the weekly feedback reports; the Do-module focused on activities and PA, whereas the Think-module focused on thoughts and NA. The results do not support a differential effect on momentary affect between the modules over time. Furthermore and contrary to our hypothesis of an average increase in PA and decrease in NA, PA showed an initial decline which leveled off towards the end of the intervention period and NA showed no significant change at all over the course of the intervention.

The lack of a differential effect over time could mean that EMI content does not have a substantial influence on momentary affect. It is possible that completing EMA questionnaires leads to a generic introspective process regardless of the specific questionnaire contents. Another possibility is that the content does matter, but that both the Do- and Think-module lead to similar outcomes on affect. This would correspond to the findings in the literature that both cognitive therapy and behavioral activation are effective in decreasing depressive symptoms, although the working mechanisms differ (Cuijpers et al., 2013; Sturmey, 2009). Regardless of the underlying mechanism, in practice both possibilities mean that the specific content of the EMI does not influence momentary affect. This point is also supported by Addington et al. (2019), who found similar changes in positive and negative emotion for both an intervention group and an emotion reporting control group. One implication of this conclusion would be that EMI’s with different contents are more comparable than we anticipated, and that the diverging results from the previous RCTs using EMI’s likely stem from other factors than the content of the questionnaires and feedback reports. Another implication would be that there is no content specific measurement reactivity in terms of affect level (see Widdershoven et al., 2019 for EMA reactive effects on emotion differentiation). This means that a focus on thoughts and NA compared to activities and PA may not lead to added adverse effects on mood, which is an often voiced concern when using EMA in both research and clinical practice. This is surprising given evidence from meta-analyses that focusing on negative self-aspects is associated with higher NA than focusing on positive self-aspects (Mor & Winquist, 2002) and that positive psychological interventions enhance well-being and decrease depression (Sin & Lyubomirsky, 2009).

Because all participants started treatment during or shortly after the EMI, we expected a general increase in PA and decrease in NA. Instead, NA did not change over the course of the intervention and PA did not improve, even showing a small initial drop. A possible explanation for the decrease in PA is the measurement frequency. A study by Conner & Reid (2012) showed that people with depressive symptoms showed decreased happiness over time when they reported momentary happiness six times per day, while this effect was reversed for participants who reported only once a day. Hartmann et al. (2015) also found no positive effect of an EMI for depression on PA, with ten measurements a day. Conversely van Roekel et al. (2017) found an increase in PA using three measurements a day and Addington et al. (2019) found increased positive emotions and decreased negative emotions using one measurement a day. Together with our findings, this suggests that the measurement frequency might play a crucial role in the effects of EMI’s on momentary affect. For example, it is possible that reflecting on your mood a few times a day improves affect by providing insight into what goes well. Reflecting more frequently on the other hand could act as constant reminder of the depressive symptoms and subsequently reduce affect.

Although the focus of the intervention modules differed, there was overlap in the first part of each EMA questionnaire; both the Do- and Think-module measured PA and NA. Less overlap between the modules might have led to a larger and therefore more detectable effect of content on affect. That said, it was precisely this overlap in the first half of the questionnaires that enabled us to compare the modules on momentary affect. In order to compare EMI’s with larger content differences one would have to find other suitable measures to assess the influence on affect. Ideally one would add a neutral EMA control group to disentangle the content-specific effects from more general effects of self-monitoring. However, whether it is possible to create such a truly neutral control condition is subject of debate, since monitoring of affect can already be regarded as an intervention in itself (See for example Arslan et al., 2020; or Bos et al., 2019). Hence, the lack of such a group is an inevitable limitation to this design.

Another characteristic of our study design was that our participants started TAU during or shortly after the intervention. Patient expectations, placebo effects and varying wait-list periods are among the many processes other than a differential effect of the EMI that could have influenced momentary affect (Posternak & Miller, 2001; Rutherford et al., 2012). Our design provided the benefit of testing the intervention directly in a clinical setting, but the heterogeneity of this setting may have masked possible subtle effects of the intervention content. Closely related is the argument that, given the large between-person heterogeneity of affect trajectories in our sample, which module works best may well be different for each participant. Theoretically, it is possible that exactly because content matters, but differently for each person, we did not find differential effects across groups.

To our knowledge, this is the first study that directly compares the effects of EMI content on momentary affect. Our results show that focusing on NA and thoughts or PA and activities in self-monitoring was not associated with differential effects on momentary affect. This suggests that the effects of EMI’s can be compared even when content differs and that focusing on thoughts and NA does not necessarily lead to adverse effects compared to focusing on PA and activities. Future research should also investigate the effects of number of measurements in EMI’s, as they might have a mediating role.

Jojanneke A. Bastiaansen and Albertine J. Oldehinkel set up the ZELF-i study, JAB and Daan A. Ornée participated in data collection (For a detailed description see Bastiaansen et al., 2018). For this article, DAO cleaned and analyzed the data, and wrote the pre-registration and manuscript with regular supervision and feedback by JAB and AJO. All authors contributed to and approved the final manuscript.

We are grateful to everyone who participated in, worked on and contributed to the ZELF-i study. For this research article we would like to thank Eveline Snippe for advice and support in performing the analyses.

Research reported in this publication was supported by a pilot grant (2015–05) from the Gratama foundation to JAB and with part of the subsidy from the Foundation VCVGZ (in Dutch: Stichting tot Steun VCVGZ) awarded to JAB and dr. H. Riese (grant number 239). The funding bodies have no role in the design of the study, data collection, analysis, or interpretation of data, nor in writing the manuscript.

No competing interests exist.

A full description of the design, procedures and primary outcomes of the ZELF-i trial are published elsewhere (Bastiaansen et al., 2018, 2020). Analysis scripts, pre-registration and materials are available online (https://osf.io/ha7z8/). Due to privacy restrictions, the participant data of this study is not publicly available. For research purposes this data can be made available after submission of a research protocol and analysis plan, upon completion of a signed data transfer agreement. For more information, search www.groningendatacatalogus.nl for the ZELF-I study or contact the authors.