Validation of the Moral Foundation Vignettes in Latin America: The Scope of Moral Foundations Through the Lens of an Instrument

Making fun of somebody with a disability, defying your boss, and urinating on someone’s grave all seem morally wrong, albeit for different reasons. Poking fun at someone might cause emotional harm, whereas defying your boss is typically not a matter of harm. According to Moral Foundations Theory (MFT), moral judgment is sensitive to distinct domains of concerns that differentially inform judgments of wrongness. These distinct domains, or foundations, are characterized by their association with psychological adaptations that are culturally widespread and present in socially relevant normative judgment (Graham et al., 2013). The core formulation of MFT identifies five domains of moral concern: Care/Harm, Fairness/Reciprocity, Authority/Respect, Ingroup/Loyalty, and Purity/Sanctity (Graham et al., 2011, 2013). In turn, Graham et al. (2009) grouped these foundations into two superordinate categories: individualizing (Harm and Fairness) and binding foundations (Loyalty, Authority, Purity). Developments of the theory propose refinements of the foundations–such as distinguishing emotional from physical harm (Clifford et al., 2015)—as well as additional foundations, including liberty (Iyer et al., 2012), self-control (Hofmann et al., 2012), equality and proportionality (Atari et al., 2023). While MFT has been criticized on both conceptual (Gray & Keeney, 2015a, 2015b) and methodological grounds (Iurino & Saucier, 2020), the theory has also been used widely to explain moral judgment in several areas, from politics to education (for review, see Graham et al., 2018).

Several instruments have been developed to measure moral judgment within the framework of MFT. The most widely used is the Moral Foundations Questionnaire (Graham et al., 2009, 2011), validated in several languages and recently extended to include new items and divisions among existing foundations (Atari et al., 2023). This tool relies on ratings of highly abstract moral principles along the dimensions of relevance to moral decision making (e.g. “Whether or not someone’s action showed love for his or her country” is relevant to decide if something is right or wrong) and moral judgment (e.g. agreement with statements such as “It is more important to be a team player than to express oneself”). Thus, responses to the MFQ create a snapshot of people’s moral theories (Graham et al., 2009; Haidt, 2001) which might diverge from how people actually make moral judgments (Clifford et al., 2015). Other instruments that have been developed to capture moral judgment in a more concrete level of representation, relying on images (Crone et al., 2018), Twitter posts (Hoover et al., 2020), or descriptions of concrete behaviors (Chadwick et al., 2006; Clifford et al., 2015).

The Moral Foundations Vignettes (MFV, Clifford et al., 2015) is a psychometric instrument designed to measure assessments of brief descriptions of concrete behaviors grouped according to MFT. The MFV was recently validated in Portuguese (Marques et al., 2020) and has the potential to allow rich cross-cultural comparisons that can be both theory driven and also empirically flexible to capture local variation (Barrett, 2020; Hruschka, 2020). It consists of 90 vignettes depicting moral violations across six foundations: Care (27 vignettes); Fairness (12); Authority (14); Loyalty (16); Purity (10) and Liberty (11) (a shortened version that exhibits good psychometric properties was recently developed; see Crone et al., 2021). All vignettes have the same structure: the respondent is asked to consider a third-party moral violation from their point of view (e.g. You see [a woman spanking her child with a spatula for getting bad grades in school]) and judge how morally wrong it is on a scale from 1 (not all wrong) to 7 (extremely wrong). The validated version of the MFV has been used to show relationships between purity violations and the intensity of moral judgment (Wagemans et al., 2018), partisan preferences and loyalty (Clifford, 2017), perceived social distance and moral valence (Dehghani et al., 2016). The MFV has also been used to test the primacy of care and fairness in the MFT (Isler et al., 2021).

Several challenges have been raised against MFT. Some concern the underlying pluralism of MFT (Beal, 2020; Fitouchi et al., 2023) and the adaptations underlying the evolution of distinct moral foundations (Curry et al., 2021). Other challenges concern the validity of the instruments developed within the framework of MFT. Generally speaking, validity challenges come in either of two varieties. One can challenge whether the instrument actually measures what the theory purports to explain. Also, one can challenge whether the instrument has adequate scope in light of the underlying theory (Borsboom, 2006).

Both challenges have been applied to the MFT and its corresponding measurement tools. On the first issue, for example, there are questions of whether the MFQ accurately captures the underlying complexity of moral judgments for different groups, given that its structure tends to vary significantly across samples. For example, Doğruyol et al. (2019) used secondary data from non-validated versions of the MFQ to show that the five-factor solution implied by the core version of MFT is stable for 30 WEIRD and non-WEIRD countries. However, Iurino and Saucier (2020) found weak evidence of measurement invariance for the (validated) short-form MFQ across 27 countries (spanning 5 continents). The source of these differences is debatable: it could be a matter of theory (e.g., foundations are represented differently in different cultural contexts) or it could be an issue with the MFQ as an instrument (e.g., the MFQ is highly sensitive and fails to represent the theory cross-culturally). On the second issue, instrument scope, there is no standard to evaluate whether different validated versions of an instrument are equivalent in terms of appropriately measuring the same construct (Iliescu, 2017). Does lack of equivalence suggest a limitation of the theory or a validation failure? Theory limitations can then be adjusted by using a bottom-up approach, where the use of an instrument suggest adjustments to the way the construct is conceived. For example, Atari et al. (2020) failed to establish the five factor solution for the MFQ with an Iranian sample. The results of their analysis suggested that loyalty was more central to overall moral concerns for this sample. Taken together, these data-driven results led the authors to propose a new moral foundation, only for Iran (Qeirat values), as opposed to the top-down approach favored by other proponents of MFT.

In consequence, although the main objective of the present study is to validate a Spanish version of the MFV, we believe this validation effort touches on several concerns about the relationship between theories of moral cognition and the instruments used to measure such theories. In addition to these overarching concerns about the relationship between methodology and theory, we also wanted to explore local characteristics of how moral judgment varies cross-culturally. We suspected that we would observe greater variability among vignettes within the Binding foundations (Loyalty, Authority, Purity) compared to Individualizing foundations for three reasons: (1) Clifford et al. (2015) explicitly selected loyalty violations on the basis of three requirements that result in a narrow characterization of loyalty compared to the original MFT; (2) Purity and Authority items on the MFQ exhibit greater variability in internal reliability compared to other foundations, which suggests that these foundations might be understood differently by participants in different contexts, and; (3) in the Portugese validation of the MFV (Marques et al., 2020), the largest number of vignettes discarded to maintain structural validity were, proportionally, from the purity and authority foundations.

Following recent guidelines on scale development and validation (Clark & Watson, 2019; Flake et al., 2017), the current research is presented in three phases: Substantive (translation and initial item adjustments), Structural validation (factor analyses, reliability, [Studies 1 and 2]) and external validation (Convergent and discriminant validity, invariance testing [Study 3]). All participants were recruited through Netquest (polling agency). Research was approved by the IRB board of the Universidad de los Andes. Analyses were performed with R version 4.2.1 (R Core Team, 2022).

The data, materials, and supplementary information associated with this research are available at https://osf.io/mjtxw/.

The full set of MFV items was translated into Spanish by two independent bilingual speakers (native speakers of Colombian Spanish). Minor differences in translations (mainly the use of equivalent nouns) were discussed and solved by the translators. Next, two of the authors reviewed each item and proposed culturally adapted alternatives to Colombian Spanish, when necessary, keeping the meaning and the magnitude of the offense as close as possible to the original items. This adapted set of items was back translated by an independent translator and then passed onto the other two authors, who evaluated whether the original sense of the items was preserved. For this step, we took into account modifications suggested by Marques et al (2020) in order make easier any possible comparisons between Brazil and other Latin-American countries. This last round of revisions resulted in the final Colombian Spanish version used in Study 1, which was submitted to two native speakers of Peruvian and Argentinian Spanish to adjust any other cultural differences that could improve cultural appropriateness of the items for these countries.

The final set of items items used in all countries can be found at the OSF page for the project. Most vocabulary changes centered on the object of the transgression. For example, the item You see a US swimmer cheering as a Chinese foe beats his teammate to win the gold was translated into Colombian Spanish as You see a Colombian cyclist celebrating as his teammate is beaten by a Chinese opponent given the salience of cycling in Colombia as a sport where individual athletes are identifiable. Following the same logic, we used surfing in Peru and tennis in Argentina. These changes made the transgression more natural to the local variant of Spanish and did not change the intended meaning.

More significant changes were introduced in two of the loyalty items (You see a coach celebrating with the opposing team’s players who just won the game and You see a former US General saying publicly he would never buy any American product). The former was deemed too abstract in the cultural adjustment review, and thus was replaced by introducing the names of actual soccer teams for each country. The latter was considered an unrealistic moral transgression by experts and it was replaced by an explicit negative comparison of a local product by a relevant political public figure. Average number of character and words did not differ significantly from the original version (US: average of 15 words and range 13-18, average of 83 characters and range of 69-91, Spanish: average of 17 words and range of 10-25 words, average of 89 characters, and range of 60-130).

The objective of this study is to examine the structural validity of the MFV with a Colombian sample as proposed by Clifford et al (2015), which established a six-factor solution according to the MFT taxonomy (care, fairness, authority, purity, loyalty and liberty).

618 people took part in this study (52% female, 48% male) with ages ranging between 18 and 76 (M = 41.30, SD = 14.54) and contacted through a polling service. 32% of the sample reported having a bachelor’s degree, 27% reported having a technical degree, 18% reported having achieved a high school diploma, 16% reported having a master’s degree, and less than 1% (2 participants) reported having doctorates.

All participants completed the following self-reported measures.

Moral Foundation Vignettes. We used the 90 vignettes recommended by Clifford et al. (2015) plus 5 neutral scenarios also used in the original study as controls. For each vignette, participants had to judge how morally wrong was the action of the person [¿Qué tan moralmente incorrecta considera que fue la acción de la persona?] using a scale from 1 (not wrong at all [nada mala en absoluto]) to 5 (extremely wrong [extremadamente mala]). Immediately after, participants had to report why they thought the action was morally wrong [¿Por qué cree que esta acción estuvo mal?] and response choices corresponded with each of the moral foundations (e.g. it violates norms of purity (e.g., degrading or disgusting acts) [Viola las normas de pureza (por ejemplo, actos degradantes o repugnantes)]). The format for both questions corresponds exactly to Clifford et al. (2015).

Sociodemographic characteristics. Participants reported their age, gender, education level, religion and political preference. The latter was asked with the question Which of the following best describes your overall political preference? With responses ranging from 1 (Strongly liberal) to 6 (Strongly conservative).

The survey was administered through Qualtrics. Item presentation was fully randomized for the MFV, which was always followed by questions on sociodemographic information, in a fixed order.

Table 1 presents the summary of classification and average wrongness judgement by item and foundation. Overall average wrongness was 3.46 (SD = 0.66) and the average successful classification rate (e.g. classifying an infraction under the intended foundation) was 64%. By foundation, loyalty violations had the lowest perceived wrongness (M = 3.33, SD = .32) and also the lowest classification success rate (57%) in contrast with justice and liberty violations, that had the highest average perceived wrongness (M = 3.82, SD = .54) and classification rate (74%), respectively. Additionally, three vignettes (two of the purity foundation¹ and one from the care foundation²) were classified as “not being morally wrong” at least 50% of the time. Overall, 29 of the 90 original items did not meet the inclusion criteria set in the original development of the scale (60% classification accuracy and less than 20% classification in other categories).

We proceeded to run exploratory factor analyses with and without the problematic items. KMO values ranged between .8 and .96 (overall of .95) and the items were strongly correlated (Bartlett test χ2 = 26782, p < 0.001). Parallel analyses suggested five factors while the original test comprised six factors and a recent validation (Marques et al, 2021) found seven factors (splitting care into physical and emotional), so we fitted rotated (Promax) and non-rotated solutions with five, six and seven factors. We also fitted an additional rotated eight-factor solution to discriminate between animal and human care but did not consider it further as it exhibited poor fit (See supplementary material).

These analyses suggested retaining the seven-factor rotated solution with the full data set. This solution exhibited good fit indices (RMSEA = 0.028 [0.027, 0.03], TLI = .90). The full factor loadings table is presented in the supplemental materials, where only factor loadings greater than .35 are displayed. Items for liberty, loyalty and purity clearly load together (factors 3,4 and 5). The other factors present a mix of themes. Factor 1 contains mostly care items, but also includes four fairness items and items from purity and loyalty. Factor 2 comprises mostly items from the authority foundation but also includes several fairness items. Factors 6 and 7 refer exclusively to emotional care and physical care, respectively. Overall, fairness items are interspersed among several factors and factor loadings tend to be around .4, indicating a good load. There are however some vignettes that failed to adequately load onto any factor (class, brother, judge, date, attractive, disfigured). All of these also failed to display adequate classification rates (except class from the authority foundation).

We ran confirmatory factor analyses comparing the six- and seven-factor solutions (see Table 2). The best models are the ones with seven factors (where care is split into emotional and physical) and the model with the best fit is the seven-factor solution fitted to the items retained in the prior process, likely because of estimating fewer parameters. We also included alternative models including modification indices where covariances were estimated for items within foundations, but these did not significantly improve the overall fit and are not reported here.

The final set of items exhibit excellent internal consistency: Care (α =.92, ω = .93), Emotional Care (α =.85, ω =.88), Physical Care (α =.92, ω = .9), Fairness (α =.84, ω =.87), Authority (α =.87, ω =.88), Loyalty (α = .81, ω = .83), Purity (α = .76, ω = .82) and Liberty (α = .88, ω = .90). With this set of items, we estimated correlations between scores derived from the average of each foundation per participant and reported political ideology (highest numbers indicate conservatism). The highest correlations were with purity (.13), authority (.10) and loyalty (.11), compared with Care (.03), Fairness (.08) and Liberty (-.03), all significant at p < 0.001. These correlations are similar in direction to those found by Clifford et al (2015) but overall smaller in magnitude.

The final set of items validated items includes 60 vignettes: Care (18), Justice (8), Liberty (10), Authority (10), Purity (6) and Loyalty (8). This is similar to the validated Brazilian Portuguese version (68 items) but somewhat shorter than the original US version (90 items). The items in the Purity and Loyalty foundations are particularly problematic, where 40% and 50% of the items were discarded. While we are confident that this is a tool that can be used to adequately probe moral judgment with Colombian participants and to make meaningful comparisons with other validated versions, there are reasons to doubt the generality of the Purity and Loyalty vignettes (Murray et al., 2024). This pattern could be completely idiosyncratic to Colombian participants and not generalize to Spanish speakers from other countries. In Study 2 we pursue this question while also including alternative measures of moral judgment to assess the convergent validity of the MFV.

The objective of this Study was to further examine the structural validity of the MFV with a larger sample of participants from Perú, Argentina and Colombia. It also assesses convergent validity by examining how the MFV is associated with other measures of moral judgment, the Morality as Cooperation Questionnaire (MAC-Q) and Moral Foundations Questionnaire (MFQ).

1033 participants were recruited from Peru (400, 48% female, 52% male), Argentina (400, 49% female, 51% male) and Colombia (233, 49% female, 51% male) using the same polling firm. None of the participants took part in Study 1 and data from one participant from Peru was excluded for incomplete responding. Colombian participants were contacted in two waves. 73 participants had to be contacted again due to a coding error. Education levels were similar to participants in Study 1, with most participants reporting having at least a college degree (26% Colombia, 24% Peru and 25% Argentina) or a high school degree (17% Colombia, 19% Peru and 37% Argentina).

Moral Foundation Vignettes. Participants were given the version of the MFV described in Study 1 with versions adapted to their countries. Colombian participants only completed the moral judgement rating questions but not the classification task. Only one neutral item was included in this version of the MFV.

Moral Foundation Questionnaire. The translation of the MFQ 30 into Spanish (Gudino & Fernández-Cárdenas, 2015) was used for all participants. No further adjustment was deemed necessary, since the level of abstraction inherent to this test made the language easily understood by any native Spanish speaker. The MFQ consists of two dimensions, relevance (When you decide whether something is right or wrong, to what extent are the following considerations relevant to your thinking? [Cuando usted decide que algo está bien o mal, ¿qué tan relevantes son las siguientes consideraciones en su desición?]) and moral judgment (Please read the following sentences and indicate your agreement or disagreement [Por favor, lea las siguientes afirmaciones e indique qué tan de acuerdo está con ellas]. Both questions require responses from 1 (not at all relevant/strongly disagree [sin relación alguna/totalmente en desacuerdo]) to 6 (extremely relevant/strongly agree [extremadamente relacionado/muy de acuerdo]) and both subscales require evaluating three statements corresponding to five foundations (Care, Fairness, Authority, Loyalty and Purity) for a total of 30 items.

Morality as Cooperation Questionnaire. This instrument was recently developed to measure morality in the framework of the Morality as Cooperation theory (Curry et al., 2019, 2021). The theory and instrument posit that moral judgement and behavior consist of adaptations to distinct cooperation problems that arise during human evolutionary history and can be characterized as distinct game-theoretic scenarios. These cooperation areas (identified with the labels of Family, Heroism, Group, Reciprocity, Deference, Fairness, Property) define seven first-order moral domains that form the basic concerns of moral reasoning. In turn, the MAC-Q implies a seven -factor model of morality that partially overlaps with the structure of the MFQ and the MFV (Curry et al., 2021). It also has two subscales, relevance and judgement and three items per element for a total 21 items per dimension and 42 overall. The relevance question asks participants to rate “to what extent are the following considerations relevant to your thinking? (0–100; not at all relevant, not very relevant, slightly relevant, somewhat relevant, very relevant, extremely relevant)” and the judgement items ask participants to rate “To what extent do you agree with the following statements? (0–100; strongly disagree, disagree, neither agree or disagree, agree, strongly agree)”. Since this scale has not been validated in Spanish, we followed a simplified procedure of translation, back translation and cultural adjustment. The final version can be found in the supplemental materials.

The sociodemographic questions were the same as in Study 1. The survey was administered through Qualtrics. Instrument and item order, within instrument, was fully randomized. Participants always saw the sociodemographic questions last.

Overall results are summarized in Table 3. The average successful classification rate was 64% for Peru and 63% for Argentina. Within-foundation classification rates range from 38% to 84% in Peru and from 26% to 88% in Argentina, exhibiting a similar pattern to Colombian responses. Loyalty violations had the lowest perceived wrongness (Peru M = 3.38, SD = .46; Argentina M = 2.99, SD = .47) and also the lowest classification success rate (Peru 60%; Argentina 54%) in contrast with liberty violations, that had the highest average successful classification rates for both countries (Peru 68%; Argentina 77%). Fairness violations had the highest average wrongness in Peru (M = 3.75, SD = .43) compared with Care infractions in Argentina (M = 3.66, SD = .61). The judged wrongness and classification rates pattern are a mirror image of the Study 1 results. The number of items that failed to meet the inclusion criteria was lower for Peru (26) and higher in Argentina (33). Proportionally, most of these vignettes came from the Loyalty foundation (Peru 7, Argentina 11) and, unlike Colombia, from the Care foundation (Peru 9, Argentina 7).

We ran exploratory factor analysis per country following the same logic of Study 1. Correlations between items were generally good (KMO Peru = .92, Argentina = .93) and parallel analyses suggested between 5 and 7 factors. Given this and the findings from Study 1, we fitted three factor analyses per country with a Promax rotation. For both countries, the seven-factor solution performed better with adequate fit indices (Peru RMSEA = 0.034 [0.031, 0.035], TLI = .885; Argentina RMSEA = 0.033 [0.031, 0.035], TLI = .88).

The composition of the factors however does not match predictions of the MFT and offers a more disordered pictured compared with the Study 1 data. For Peru, the most clearly defined factors are factor 4 (Liberty) and 6 (Care). The rest of the factors exhibit a mix of items from different foundations: Loyalty in factors 5 and 7, authority and fairness items in factor 1, Care and Purity in Factor 2, and Care, Fairness and Purity in factor 3. For Argentina we found that items related to Liberty, Loyalty, Purity and Care loaded onto their own factor neatly (factors 2,3,6 and 7 respectively) while vignettes of Authority and Fairness were mixed in Factor 1 and Care items loaded onto factors 3 and 4 without a clear logic. The most consistent result from the exploratory factor analysis in both Study 1 and 2 is that Liberty items tend to cluster clearly together as well as Care items. Items that loaded weakly on any factor (<.3) matched the items that did not meet the classification thresholds, so no other items were flagged at this stage.

We conducted confirmatory factor analysis with six and seven factors with the hypothesized structure by country excluding items identified in the prior analysis. Summary of the fit indices for these models can be seen in Table 4. For both countries, a likelihood ratio test revealed no significant differences in fit between the six and seven factor solution, even though the seven-factor model performs slightly better in most indicators. We decided to retain the seven-factor solution, in line with Study 1 results and Marques et al (2020).

The MFQ and the MAC-Q were also analyzed with a series of exploratory and confirmatory factor analyses (see supplemental materials). Solutions for both the MFQ consistently point out to a two-factor solution that roughly coincided with distinction between individualizing and binding foundations. The MAC-Q differs between countries ranging from four to seven factor solutions. We also estimated scores for the MFV, MFQ and MAQ-Q by averaging ratings by participant and by foundation/moral element. For the MFV, this was restricted only to the items that were retained in the previous analyses per country. We then calculated correlations between all subscales scores and also included our measure of ideology. Results are presented in Table 5.