How do we make decisions about allocating scarce resources to others when we face multiple alternatives? While people usually evaluate multiple allocations jointly or non-independently, distributional preferences have been analyzed through behavioral models that assume independent evaluation. Here, we explore distributional preferences that can be identified when we analyze allocation decisions with a flexible model class that can factor in joint evaluation. In Study 1, we recruited 3,006 Japanese crowd workers and provided them with 13,041 theoretically designed resource allocation problems. Using artificial neural networks, we discovered two types of allocation problems among 13,041 problems where the choices made by the participants could only be predicted with a model that factored in joint evaluation of two options. For example, one of these problems showed that the participants became sensitive to differences in their self-reward between two allocations which pitted the participants against disadvantageous equity, which could be naturally supported by our intuition but could not have been discovered unless joint evaluation was considered. In preregistered Study 2, we recruited 185 Japanese participants to conduct a conceptual replication of the machine-discovered distributional preferences. We had participants evaluate two allocation options on the same (i.e., jointly) or different (i.e., separately) screens, confirming that the distributional preferences discovered in Study 1 (i.e., joint-evaluation situation) were more often observed in the former joint-screen situation. Our findings showcase the usefulness of a prediction-oriented machine learning approach to the exploration of novel behavioral theories in social decision-making.

Sharing resources, such as food, land, vaccines, and graphical processing units, with others is a fundamental process in human society. Given the importance of distributive justice, people’s allocation behavior in relation to its fairness has been a central concern of humanities and social sciences (Deutsch, 1985; Frohlich & Oppenheimer, 1992; Harsanyi, 1975; Konow, 2003; Rawls, 1971). Research has discovered several factors that explain and predict individuals’ allocation behavior. In addition to self-interest, for example, individuals have been shown to consider inequality or inequity between themselves and others (Adams, 1965; Bolton & Ockenfels, 2000; Fehr & Schmidt, 1999), the welfare of the less fortunate person among allocation recipients (Charness & Rabin, 2002; Gates et al., 2020; Kameda et al., 2016; Mitchell et al., 1993), and the social efficiency (i.e., total surplus; Charness & Rabin, 2002; Engelmann & Strobel, 2004; Fisman et al., 2015) of allocation. The discovery of these social preferences has been significant for explaining and predicting how individuals allocate scarce resources.

Among other things, inequality (inequity) aversion refers to the idea that individuals have a preference for equal outcomes. Research has empirically shown that people are averse to unequal allocations and are sometimes willing to incur costs to ensure fairness (Bolton & Ockenfels, 2000; Fehr & Schmidt, 1999; for a review, see Dhami, 2016). One of the most significant findings in the research on inequality aversion from a psychological standpoint is the identification of two distinct types of inequality aversion: advantageous inequality aversion and disadvantageous inequality aversion (Fehr & Schmidt, 1999; see also Leventhal et al., 1969; Walster et al., 1973). Advantageous inequality aversion refers to the idea that individuals have a disutility from inequality, when they have more material benefit than others (i.e., guilt). On the other hand, disadvantageous inequality aversion refers to the idea that individuals have a disutility from inequality, when they have less benefit than others (i.e., envy). These concepts of inequality aversion are paramount in explaining human behavior in crucial economic games such as the ultimatum game (Güth et al., 1982) and the dictator game (Forsythe et al., 1994; Kahneman et al., 1986), where self-interest alone is inadequate to account for observed human behavior. More recent research has identified mechanism underlying inequality aversion. To name a few, neuroeconomic research has revealed that the brain’s reward circuitry is associated with inequality aversion, providing neural correlation evidence that this preference is integrated during value computation (Tricomi et al., 2010). Furthermore, studies have suggested that advantageous and disadvantageous inequality aversions have separate neural bases (Morishima et al., 2012) and distinct developmental trajectories (Blake & McAuliffe, 2011), with advantageous inequality aversion developing more gradually with age compared to disadvantageous inequality aversion. Cross-cultural research has identified both cultural universality and differences of inequality-averse behavior. For instance, studies have demonstrated that disadvantageous inequality aversion is more commonly observed across cultures in comparison to advantageous inequality aversion (Blake et al., 2015). The cultural differences in social preferences have also been investigated in previous studies (see Henrich et al., 2001).

Despite the extensive exploration of new behavioral patterns including the above discussion on inequality aversion, allocation behavior is often studied with choice models that do not factor in the joint evaluation in decision-making. Joint evaluation refers to the process by which options are evaluated non-independently (cf. Tversky & Simonson, 1993). For example, in the case of allocation choices between two options, joint evaluation implies that the contents of allocation option A (i.e., self-reward and other recipients’ reward) influence the evaluation of allocation option B. Established behavioral models, such as the inequity-aversion model (Fehr & Schmidt, 1999), can model inequity between self and others in option A but are not designed to factor in any information regarding option B when factoring in inequity in option A. As choice models do not usually violate the independence axiom and the transitivity of decision-making (Regenwetter et al., 2011), capturing joint evaluation in allocation choices is impossible.

Nonetheless, a consideration of joint evaluation in allocation choices is of theoretical and practical importance. For example, in the evaluation of the policy of allocating scarce resources, joint evaluation indicates that the policy of allocating the same resources can be evaluated differently depending on other policies presented with it. Theoretically, this can lead to irrational or cyclical preferences (e.g., one prefers policy x to y and y to z but prefers z to x) at the individual level, not to mention at the collective or aggregate level. Moreover, among decisions individuals make, the one on resource allocation considerably impacts other individuals’ welfare. Thus, joint evaluation in allocation choices can be detrimental to society. However, if individuals understand how joint evaluation affects distributional preferences, desirable choice architectures that promote better allocation decisions may be designed.

Despite the ample evidence of joint evaluation in risk, consumer, and other types of decision-making (Bettman et al., 1998; Stewart et al., 2003; Trueblood et al., 2013), joint evaluation in allocation choices has not been studied extensively to our knowledge, with a few exceptions (e.g., Bazerman et al., 1992). One reason may be that a systematic examination of the effect of joint evaluation in allocation choices has been difficult. For example, to identify distributional preferences caused by a combination of two allocation options, statistical interaction patterns arising from all the contents of the two options (i.e., the amounts received by self and others) should be predicted. To predict this is a challenge because individuals exhibit distributional preferences in addition to concerns about self-interest. Hypothesis-driven methods for finding distributional preferences caused by joint evaluation would allow researchers to exploit their expertise and knowledge from previous literature. However, systematic exploration is difficult when the number of combinations in allocation options is enormous. Thus, to investigate the impact of joint evaluation on allocation behavior, we need to employ a systematic and automatic search method with a more flexible model class than the one used before.

Recent research on decision-making, including morality, ethics, risk, information-search, and economic games, has progressed using machine learning techniques for discovery of novel behavioral theories (Agrawal et al., 2020; Fudenberg & Liang, 2019; Peterson et al., 2021; Peysakhovich & Naecker, 2017; Plonsky et al., 2019; Plonsky & Erev, 2021; Skirzyński et al., 2021). Such machine learning methods are also applicable to allocation choices to discover distributional preferences caused by joint evaluation. Relevant to our study is an approach that uses artificial neural networks that facilitate psychological interpretation. Peterson et al. (2021) used their artificial neural networks to predict participants’ risky choices (see also Plonsky & Erev, 2021). One of their neural networks that evaluated two lottery options separately did not allow the content of one lottery (i.e., outcomes and their probabilities) to influence the valuation of the other. The other neural network evaluated the contents of two lotteries jointly; thereafter, each lottery’s outcomes and probabilities were jointly input into the neural network, unlike in the first one. Thus, the latter neural network could model the influence of one lottery on the valuation of another lottery. They showed that risky decision-making is evaluated jointly through this comparative analysis. Critically, the neural network that evaluates two options separately cannot predict choices for a problem, which is a set of two lotteries, in which the participants’ behavior is affected by their joint evaluation. In our allocation-decision scenario, allocation problems, represented by a set of two allocations, sometimes cause distributional preferences affected by joint evaluations. These can be discovered through the comparison of the predictions of the two neural networks for each allocation choice problem.

In Study 1 of the present research, we employed this approach to explore distributional preferences modulated by joint evaluation in allocation choices. For the application of neural networks, a large dataset containing various choice problems must be collected (Peterson et al., 2021). Our study collected choice data from approximately 3,000 participants for more than 13,000 allocation choice problems that were theoretically designed based on essential distributional preferences (Charness & Rabin, 2002; Engelmann & Strobel, 2004; Fehr & Schmidt, 1999; Frohlich & Oppenheimer, 1992; Konow, 2003; see the Supplemental Online Material). After discovering the choice problems that led to distributional preferences caused by joint evaluation in Study 1, we investigated whether we could manipulate the presence and absence of the machine-discovered distributional preferences in an intervention (Study 2). We manipulated participants’ behavior by having participants evaluate two allocation options either on the same screen (i.e., jointly) or on different ones (i.e., separately). Such interventions explore the “evaluation mode” (i.e., joint evaluation vs. separate evaluation), which has been investigated in several existing theoretical studies (Bazerman et al., 1992; Hsee, 1996; Hsee et al., 1999). We hypothesized that having individuals evaluate allocation options separately will lead their evaluation to be more like the separate-evaluation neural network than the joint one.

Method

Participants

We recruited convenience samples of 3,006 participants (1,733 women, 1,218 men, 14 non-binaries, and 41 who preferred not to report their gender; mean age = 39.37, standard deviation = 11.44) from a Japanese crowdsourcing platform. We did not conduct a formal power analysis for Study 1. We determined the sample size to obtain an average of more than 20 choices for each allocation problem. The study was approved by the ethics committee of the Graduate School of Arts and Letters, Tohoku University (no. 20211028). Informed consent was obtained from all participants before the experiment.

Task and Choice Problem

Figure 1a illustrates one of the allocation problems we studied. The participants were asked to choose one of two allocations and determine the monetary payoff between themselves and others. Previous studies (Charness & Rabin, 2002; Engelmann & Strobel, 2004; Fehr & Schmidt, 1999; Frohlich & Oppenheimer, 1992; Konow, 2003) have identified important social preferences that are considered while making allocation choices. Based on these studies, we developed 13,041 allocation choice problems, which varied across three key social preferences: (1) the total amount of allocation (sum of self-reward and the other two recipients’ rewards), (2) inequity of allocation (Gini coefficient), and (3) the minimum amount of allocation (as defined by percentage in the total amount of allocation). Each participant chose one of the two allocations for 100 choice problems randomly selected from 13,041 ones. For more detailed information, see the section “Design of Choice Problems” in the Supplemental Online Material.

Figure 1.
Task Design and Two Types of Neural Networks Used in Study 1

Note. An example choice problem (a) consisted of two allocation options, one of which the participants chose. They were informed that the other two recipients of the allocation would not be their acquaintances but would be recruited via the Internet. A randomly selected participant received an additional bonus based on their choice for a randomly selected choice problem in the task (i.e., the decisions were incentivized; participants were also notified that if their particular choice was implemented, others would receive their allocations. See the Supplemental Online Material). The display order of the two options was randomized across participants. At that time, 1,000 yen was approximately equivalent to $8.8. Panel (b) is the architecture of the separate evaluation (SE) model. Each option’s input was evaluated independently and transformed into the probability of choosing option A using a softmax function. Example inputs from Panel (a) are shown. Panel (c) is the architecture of the joint-evaluation (JE) model. All inputs were evaluated jointly, and the neural network directly outputs choice probability.

Figure 1.
Task Design and Two Types of Neural Networks Used in Study 1

Note. An example choice problem (a) consisted of two allocation options, one of which the participants chose. They were informed that the other two recipients of the allocation would not be their acquaintances but would be recruited via the Internet. A randomly selected participant received an additional bonus based on their choice for a randomly selected choice problem in the task (i.e., the decisions were incentivized; participants were also notified that if their particular choice was implemented, others would receive their allocations. See the Supplemental Online Material). The display order of the two options was randomized across participants. At that time, 1,000 yen was approximately equivalent to $8.8. Panel (b) is the architecture of the separate evaluation (SE) model. Each option’s input was evaluated independently and transformed into the probability of choosing option A using a softmax function. Example inputs from Panel (a) are shown. Panel (c) is the architecture of the joint-evaluation (JE) model. All inputs were evaluated jointly, and the neural network directly outputs choice probability.

Close modal

Procedure

Before the monetary allocation task, participants were informed that (1) the other two recipients of the allocation would not be their acquaintances but would be recruited via the Internet; (2) that they and the recipients would remain mutually anonymous; (3) that the other recipients would perform a different task from the monetary allocation to receive the assigned reward. After completing allocation choices for 100 problems, participants answered the questions related to their age, gender, and occupation.

In addition to a fixed reward of 300 yen, a randomly selected participant received an additional bonus based on their choice for a randomly selected choice problem in the task. The additional reward was paid after the completion of Study 1 (i.e., no deception was used). To make the allocation more natural for participants, we presented on the screen other recipients’ occupations (e.g., farmer), randomly selected from about 500 occupations listed on a website maintained by the Japanese government (https://shigoto.mhlw.go.jp). See the Supplemental Online Material for more details.

Data Analysis

To identify choice problems that led to distributional preferences modulated by joint evaluation, we compared the predictions of two neural networks. One neural network separately evaluated the two options (Figure 1b), as in traditional choice models. The other neural network jointly evaluated the options (Figure 1c). We call the first model the separate-evaluation model (SE model) and the second one the joint-evaluation model (JE model). Importantly, the SE model could not exploit the information of a combination of two allocation options for choice prediction and thus could not predict choices strongly modulated by joint evaluation. Based on previous research (Peterson et al., 2021; Plonsky et al., 2019), we aggregated the mean choice rate for option A for each of the 13,041 choice problems, and the neural network models predicted that mean aggregated choice rate. Throughout our analysis, the six input features (money for self, other X, and other Y from each option) were z-score normalized. Consequently, their mean and standard deviation became zero and one, respectively.

To identify allocation problems where the choices of participants were unpredictable using the SE model but predictable using the JE model, we used a method proposed by Agrawal et al. (2020). This involved comparing the predictions of a less constrained flexible neural network (the JE model in this study) with those of a more constrained model (the SE model in this study), based on Equation 2 and 3 in Agrawal et al. (2020). We searched for allocation problems across the 13,041 problems where there were largest differences in prediction between the JE model and SE model. Note that we searched for allocation problems where the difference in prediction between the JE and SE models was the largest, rather than problems where the difference in predictive accuracy was the largest. The reason for this is that Agrawal et al. (2020) demonstrated that comparing predictive accuracy (or prediction errors) can result in a large difference that reflects noise (as per Equation 1 in Agrawal et al., 2020). For example, the choices of the participants for some allocation problems among the 13,041 could have reflected noise. In that case, comparing predictive accuracy would not make sense. Although we reported the average predictive accuracy for all the 13,041 problems (see the evaluation section below), this was not for the purpose of searching for specific allocation problems that cause distributional preferences modulated by joint evaluation.

Importantly, our main goal was not to claim that the JE model is superior to the SE model in terms of predictive accuracy. Rather, we used the SE model to highlight allocation patterns that cannot be accounted for by models based on separate evaluations, which constitute most of the decision models in the past (Peterson et al., 2021).

Model architecture

We implemented the SE model using a three-node (or three-neuron) input layer, a 20-node hidden layer with sigmoid activation functions, and a softmax output layer (Table S3). The JE model was implemented using a six-node input layer, 10-node hidden layer, 8-node hidden layer, and output layer, all with sigmoid activation functions (Table S4). To be clear, the SE model produced a valuation of each choice option, which was then passed through a softmax, whereas the JE model produced a probability of choosing one of the options (over the other). For more detailed specifications of the neural network and results of hyperparameter search, see the Supplemental Online Material.

Evaluation

Our neural network models predicted the mean choice rate for option A for each of the 13,041 choice problems. Based on prior work that employed a similar approach (Agrawal et al., 2020; Peterson et al., 2021), our primary analysis compared the models in terms of predictive accuracy using mean squared error (MSE). To calculate the MSE, we took the mean of the squared differences between the actual mean choice rate for option A and the predicted choice rate for option A, for each model across the 13,041 choice problems. Other evaluation metrics were reported in the Supplemental Online Material (see the section “Evaluation Metrics Other than Mean Squared Error”).To ensure each allocation problem was included in the test set once, we employed a 10-fold cross-validation for each of the SE and JE neural networks. This involved randomly dividing the 13,041 allocation problems into 10 subsets. Each subset was used as a test set once. During each iteration, one subset (10% of the 13,041 problems) was held out as the test set, while the remaining nine subsets (90% of the 13,041 problems) were used as training data. To ensure that the result does not depend on how the problems are partitioned into 10 subsets, we performed the 10-fold cross-validation 10 times, with different subsets composition and averaged each result. We separated the data based on the choice problems because we aimed to examine which choice problems would cause distributional preferences that could only be captured by the JE model. Thus, the test-train split based on the choice problems was sufficient for our goal. However, we nonetheless reported the predictive accuracy of each model using the test-train split based on participants (i.e., generalizing from one set of participants to a new group of participants). See the section “Predictive Accuracy of Joint and Separate Models When the Data Were Separated by Participants, Not by Choice Problems” in the Supplemental Online Material.

Transparency and openness

Study 1 has not been pre-registered. Data, analysis code, and research materials are available at https://doi.org/10.17605/OSF.IO/WFXJD. Data were analyzed using PyTorch (Paszke et al., 2019), version 1.10.047, and we used NN-SVG (LeNail, 2019) for the illustration of neural networks.

Disclosure statement

All studies, measures, manipulations, and data/participant exclusions are reported in the manuscript or the Supplemental Online Material.

Results

First, we checked the predictive accuracy of the SE and JE models for the 13,041 choice problems. We compared the models in terms of predictive accuracy using a mean squared error (MSE). To calculate the MSE, we took the mean of the squared differences between the actual choice rate for option A and the predicted choice rate for option A for each model. Results showed that the JE model predicted participants’ choices better than the SE model (MSE of 0.0087 for the JE model and 0.0157 for the SE model), demonstrating that participants’ evaluations of one allocation option were influenced by the contents of the other allocation option. We also confirmed that the advantage of the JE model over the SE model did not change even when the JE model contained fewer parameters than the SE model (see the section “Analysis by a JE Model with Fewer Parameters” in the Supplemental Online Material). To understand how accurate these predictive estimates were, we investigated the predictive performance of the inequity-aversion model, an established allocation model (see the section “Analysis by the Inequity-aversion Model” in the Supplemental Online Material for more details regarding the model specifications). The inequity-aversion model displayed an MSE of 0.0537, which was poorer than the predictive estimates exhibited by the SE model (MSE of 0.0157). This outcome indicated that the SE model could capture more behavioral patterns than the inequity-aversion model, even though it used the same model architecture as the inequity-aversion model in terms of the application of the softmax function to transform values into choice probabilities. Also, refer to the section “Analysis by the Independent Evaluation Model” in the Supplemental Online Material for results from an additional modeling approach. This approach processes each of the three monetary outcomes (i.e., self, other X, and other Y) independently using neural networks. Thus, it does not model inequity between self and others as the SE model does.

Next, we investigated allocation problems in which participants’ choices were unpredictable using the SE model but predictable using the JE model from the 13,041 choice problems. For this, we investigated the top 50 largest discrepancies in prediction between the two models (cf. Agrawal et al., 2020). Through this investigation, we discovered two types of allocation problems, which we will refer to as Types A and B, respectively. Type A problems (Figure 2a) consist of two non-advantageous options, one of which is slightly more self-rewarding than the other. In Type B problems (Figure 2b), one allocation option dominated the other (i.e., each reward for one allocation was more than or equal to the corresponding reward for the other). These two types consistently exhibit a significant disparity between the models, accounting for 46 out of the 50 choice problems that demonstrate the largest prediction difference between the two models, with 17 classified as Type A and 29 as Type B (for a complete list of the 50 allocation problems, see Supplementary Table S1).

Figure 2.
Largest Differences between the JE and SE Models

Note. In Type A (a), choice problems consist of two non-advantageous options, one of which is slightly more self-rewarding than the other. In Type B (b), choice problems consist of an allocation option dominating the other one. Our analysis of the top 50 choice problems with the largest difference in predictive choice rate for option A between the joint-evaluation (JE) and separate-evaluation (SE) models also revealed the same two types of choice problems (see Supplementary Table S1). “JE left” and “SE left” indicate the predictions by the neural networks of choosing the left option. “Participant left” indicates the ratio of participants who chose the left option in the choice problem. Note that the predictions of the JE model were not always closer to the participants’ choices than those of the SE model (e.g., nos. 8526 and 1000). This is not an error but a property of our methodology. See the “Data Analysis” section for more details. In the experiment, the presentation order (i.e., left, or right) of each option was randomized across participants.

Figure 2.
Largest Differences between the JE and SE Models

Note. In Type A (a), choice problems consist of two non-advantageous options, one of which is slightly more self-rewarding than the other. In Type B (b), choice problems consist of an allocation option dominating the other one. Our analysis of the top 50 choice problems with the largest difference in predictive choice rate for option A between the joint-evaluation (JE) and separate-evaluation (SE) models also revealed the same two types of choice problems (see Supplementary Table S1). “JE left” and “SE left” indicate the predictions by the neural networks of choosing the left option. “Participant left” indicates the ratio of participants who chose the left option in the choice problem. Note that the predictions of the JE model were not always closer to the participants’ choices than those of the SE model (e.g., nos. 8526 and 1000). This is not an error but a property of our methodology. See the “Data Analysis” section for more details. In the experiment, the presentation order (i.e., left, or right) of each option was randomized across participants.

Close modal

For Type A allocation problems (Figure 2a), the SE model could not predict participants’ preference for the slightly self-rewarding option. Notably, this sensitivity to the slight difference of self-rewards between options was much less clearly observed when choice problems consisted of two advantageous options (i.e., participants’ reward was greater than that of other recipients). In other words, the differences in predictions between the SE and JE models were much smaller when participants were in advantageous situations than when they were not. See Supplementary Figure S6 for example allocation problems and the models’ predictions. For Type B allocation problems (Figure 2b), the SE model predicted more choices for the dominated option than the JE model. These results showed that specific and non-random combinations of options caused allocation behavior that can only be modeled when joint evaluation is considered.

Study 1 used a machine learning approach to reveal how distributional preferences are modulated by the joint evaluation in allocation choices. In the 13,041 choice problems, our artificial neural networks discovered two types of allocation problems that induce allocation behavior specifically caused by joint evaluation. Type A influenced the participants’ concerns about their self-reward, and Type B indicated that participants were sensitive to whether one option dominated another.

Study 1 suggests that changing evaluation mode of allocation problems may change individuals’ choices. If individuals evaluate allocation options separately, as the SE model did in Study 1, their evaluations would be more similar to those made by the SE model than the JE model. For example, for Type A problems (Figure 2a), the JE model predicted more selfish allocations than the SE model. This result indicates that the participants will evaluate selfish allocations higher if they evaluate allocation options jointly rather than separately. We could also apply this reasoning to Type B problems. We preregistered our hypotheses at https://doi.org/10.17605/OSF.IO/XK6SB, which are included below, and tested them in Study 2.

Method

Participants

We recruited a convenience sample of 200 participants from a Japanese crowdsourcing platform. Our final sample size resulted in 185 participants (85 women, 93 men, 2 non-binaries, and 5 who preferred not to report their gender; mean age = 42.63, standard deviation = 10.45) after excluding those who did not meet the participation criteria (e.g., participants who used a smartphone to respond were excluded, see our preregistration for more details) and those about whom there were missing data owing to network errors. As per our preregistration, we adopted the Sequential Bayes factor with maximal n design to collect data (Schönbrodt & Wagenmakers, 2018) (see the Supplemental Online Material). The study was approved by the ethics committee of the Graduate School of Arts and Letters, Tohoku University (no. 20220423). Informed consent was obtained from all participants before the experiment.

Task and Choice Problem

Participants in Study 2 (N = 185) evaluated allocation options by indicating how preferable each option was on a scale of 1–7 (1 = “not at all preferable,” 7 = “absolutely preferable”) instead of making binary choices as in Study 1. This change was due to two conditions used in Study 2 (Figure 3). In the JE condition, two allocation options were presented on the same screen, as in Study 1. In the SE condition, the same options were presented on separate screens, and participants evaluated them separately— the SE condition could not be implemented with binary choices. Each participant evaluated allocation options in both the JE and SE conditions, with the condition order being randomized across participants. In Study 2, participants were presented with a total of 32 allocation problems, which were selected from the 13,041 problems (Supplementary Table S2). Among these 32 problems, 8 were categorized as Type A and another 8 were categorized as Type B. The remaining 16 problems were categorized neither Type A or Type B. We preregistered the contents of these 32 choice problems specifically as shown in Supplementary Table S2.

Figure 3.
Conditions and an Example of Data Structure in Study 2

Note. In the joint-evaluation (JE) condition (a), two allocation options were presented on the same screen as in Study 1. In the separate-evaluation (SE) condition (a), the same options were presented on a separate screen. We asked participants to rate options on a scale of 1–7 (1 = “not at all preferable,” 7 = “absolutely preferable”). Sample evaluations are marked with gray circles. An example-data structure for Type A is provided (b). We calculated the evaluation difference between two allocation options for each problem in the SE and JE conditions. For Type A, the evaluation difference was calculated as the evaluation of higher self-reward options minus that of smaller self-reward options (as per our preregistration). Regarding the effect of conditions, we calculated the effect of the conditions for a choice problem as the evaluation difference. We had eight choice problems for each of Type A and Type B. Accordingly, for each type, a participant provided eight effects of conditions, which were averaged to calculate the average effect of the conditions. Instead of using willingness to pay (WTP), as was done in previous research (e.g., Hsee et al., 1999), we adopted a rating method in the current study.

Figure 3.
Conditions and an Example of Data Structure in Study 2

Note. In the joint-evaluation (JE) condition (a), two allocation options were presented on the same screen as in Study 1. In the separate-evaluation (SE) condition (a), the same options were presented on a separate screen. We asked participants to rate options on a scale of 1–7 (1 = “not at all preferable,” 7 = “absolutely preferable”). Sample evaluations are marked with gray circles. An example-data structure for Type A is provided (b). We calculated the evaluation difference between two allocation options for each problem in the SE and JE conditions. For Type A, the evaluation difference was calculated as the evaluation of higher self-reward options minus that of smaller self-reward options (as per our preregistration). Regarding the effect of conditions, we calculated the effect of the conditions for a choice problem as the evaluation difference. We had eight choice problems for each of Type A and Type B. Accordingly, for each type, a participant provided eight effects of conditions, which were averaged to calculate the average effect of the conditions. Instead of using willingness to pay (WTP), as was done in previous research (e.g., Hsee et al., 1999), we adopted a rating method in the current study.

Close modal

Procedure

In the JE condition, participants evaluated 64 allocation options, each of which was presented as 32 choice problems (Figure 3a, left). In the SE condition, participants evaluated 48 unique options on different screens (Figure 3a, right). Note that although there were 64 choice problems employed, some of these problems contained identical options. Therefore, participants evaluated only 48 unique options. After completing both the JE and SE conditions, the participants provided the details of their age and gender.

Besides a fixed reward of 300 yen, a randomly selected participant received an additional bonus based on their evaluation during the task (see the Supplemental Online Material). The additional reward was paid after the completion of Study 2.

Preregistration

Preregistration is at https://doi.org/10.17605/OSF.IO/XK6SB. The date of the preregistration was May 9, 2022. For Type A and Type B problems, we had two hypotheses for Study 2, as follows:

Type A: When two resource allocations are both highly disadvantageous (Figure 2a; for the specific eight choice problems, see Type A choice problems listed in Supplementary Table S2), participants prefer a relatively prosocial one in the SE condition more than in the JE condition. Put differently, participants place importance on self-reward in the JE condition more than in the SE condition.

Type B: When one resource allocation dominates the other (Figure 2b; for the specific eight choice problems, see Type B choice problems listed in Supplementary Table S2), participants prefer the dominated one in the SE condition more than in the JE condition. An option is defined as a dominating option if each reward for the option is more than or equal to the corresponding reward for another option.

For statistical analyses regarding the above hypotheses, a choice problem consisting of allocation options X and Y is considered. Participants will evaluate options X and Y both in the SE and JE conditions, resulting in four evaluations (1: evaluation of option X in the JE condition, 2: likewise in the SE condition, 3: evaluation of option Y in the JE condition, and 4: likewise in the SE condition). We will calculate the evaluation difference between option X and option Y in the SE and JE conditions (see also Figure 3b). Each participant will have two outcome measures for each problems (i.e., evaluation difference in the JE and SE conditions).

Regarding the above hypotheses, we proposed two additional hypotheses:

The magnitude of the “effect of condition” in Figure 3b shown by the participants for each type of allocation problems will be meaningfully correlated.

This hypothesis will test commonalities among joint evaluations for the different types of allocation problems. If a participant who exhibited a strong effect of condition for one type of allocation problems also showed it for another type of allocation problems, these joint evaluations may share some common mechanisms.

Our last preregistered hypothesis was as follows:

Participants who report that evaluating the allocation option in the JE condition is easier than in the SE condition are influenced by the evaluation modes more.

The above hypothesis will examine the possibility that the easier the participants find evaluations in the JE condition, the extent of joint evaluation (the effect of condition in Figure 3b) shown by participants will be stronger.

To test the second and third hypotheses, we will calculate the average effect of the conditions (JE vs. SE) shown by each participant. If a participant’s evaluation differences of a choice problem are three in the JE condition and zero in the SE condition, the effect of the conditions shown by that participant for the choice problem will be calculated as the difference of three and zero (see also Figure 3b). There will be eight differences (i.e., the eight choice problems) for each type, which will be averaged to calculate the average effect of the conditions for each participant.

Data Analysis

According to our preregistration, we used mixed-effects linear regressions with a fixed effect of condition (JE vs. SE), varying intercepts for participants and allocation options, and varying slopes for participants. For correlation analysis, we estimated Pearson’s correlation coefficient with a uniform prior using JASP (jasp-stats.org).

Transparency and openness

The design and analysis of Study 2 have been pre-registered and are available at https://doi.org/10.17605/OSF.IO/XK6SB. Data, analysis code, and research materials are available at https://doi.org/10.17605/OSF.IO/WFXJD. Data were analyzed using brms (Bürkner, 2017), version 2.17.0 and JASP (jasp-stats.org), version 0.16.2. We preregistered four types of allocation problems in which participants’ choices were unpredictable using the SE model but predictable using the JE model the SE model. However, we removed two of these types of allocation problems from our final analysis based on the feedback obtained during the review process.

Results

Preregistered Analyses

As illustrated in Figure 3, we investigated whether differences in the evaluations of two allocation options of each choice problem differed between JE and SE conditions. As shown in Figure 4a, for Type A and B choice problems, participants evaluated allocation options differently between JE and SE conditions (Bayes Factor [BF] > 100, β = 0.13, 95% credible interval [CI; 0.06, 0.19] for Type A choice problems; BF > 100, β = -0.19, 95% CI [-0.26, -0.13] for Type B choice problems). In Type A choice problems, participants placed importance on self-reward in the JE condition more than in the SE condition. In Type B choice problems, participants preferred the dominating option in the JE condition more than in the SE condition. These results demonstrated that for certain types of choice problems when individuals evaluated allocation options separately (as the SE model did in Study 1), their evaluations of each option were more similar to those made by the SE model than to those made by the JE model.

Figure 4.
Analyses of Study 2

Note. Panel (a) shows evaluation differences (see Figure 3b) across the two types of joint evaluations. The evaluation difference was calculated within each type of allocation problems as follows: Type A, the evaluation of higher self-reward options minus that of smaller self-reward options; and Type B, the evaluation of dominated options minus that of dominating options. Error bars represent 95% confidence intervals (CIs). Panel (b) shows the relationship between Type A joint evaluation and Type B joint evaluation. Each point represents an individual participant (N = 185). The correlation coefficient in parenthesis (r = -0.24) indicates a result when a point colored in gray is removed as an outlier (a non-preregistered analysis), which resulted in a Bayes factor of 15.67, 95% CI (-0.37, -0.09).

Figure 4.
Analyses of Study 2

Note. Panel (a) shows evaluation differences (see Figure 3b) across the two types of joint evaluations. The evaluation difference was calculated within each type of allocation problems as follows: Type A, the evaluation of higher self-reward options minus that of smaller self-reward options; and Type B, the evaluation of dominated options minus that of dominating options. Error bars represent 95% confidence intervals (CIs). Panel (b) shows the relationship between Type A joint evaluation and Type B joint evaluation. Each point represents an individual participant (N = 185). The correlation coefficient in parenthesis (r = -0.24) indicates a result when a point colored in gray is removed as an outlier (a non-preregistered analysis), which resulted in a Bayes factor of 15.67, 95% CI (-0.37, -0.09).

Close modal

Next, we tested if a participant who was influenced by one type of joint evaluation was also susceptible to another. The result supported our prediction. Participants who were more likely to change their behavior between joint and separate evaluation in Type A allocation problems were also more likely to do so in Type B (Figure 4b, BF > 100, Pearson correlation coefficient r = -0.38, 95% CI [-0.50, -0.25]), suggesting that these two joint evaluations might have shared some common mechanisms.

Finally, we tested our third hypothesis. The participants indicated which condition was easier to make evaluations on a scale of 1–7 (1 = “when two options were presented jointly,” 7 = “when each option was presented one at a time”). Contrary to our prediction, there was no meaningful relationship between participants’ self-report easiness of evaluations in the JE condition and the extent to which they were influenced by joint evaluation (BF = 0.34, r = -0.12, 95% CI [-0.26, 0.02] for Type A choice problems; BF = 0.17, r = 0.08, 95% CI [-0.07, 0.22] for Type B choice problems). These results suggest that there may not be a simple relationship between the ease of evaluation subjectively perceived by the participants and the susceptibility to evaluation mode.

This article presents a machine-learning approach to investigate the influence of joint evaluation in monetary allocation problems. Using a large-scale experiment, we showed that a machine-learning approach could discover distributional preferences modulated by joint evaluation in allocation choices. An additional preregistered experiment conceptually replicated the machine-discovered allocation behavior caused by joint evaluation. This replication involved a causal intervention for individual participants.

Previous research has identified several patterns of allocation choices, including inequity-aversion (Fehr & Schmidt, 1999), efficiency concern (Engelmann & Strobel, 2004), and maximin concern (Charness & Rabin, 2002; Kameda et al., 2016). However, to our knowledge, most research has assumed that individuals evaluate each option independently. Thus, how joint evaluation can influence allocation choices has not been studied systematically. To overcome difficulties in the discovery of the influence of joint evaluation, we analyzed participants’ choices using artificial neural networks. The results revealed that employing a model that can incorporate joint evaluation is important for predicting individuals’ allocation choices. More importantly, we specifically clarified when and how joint evaluation influences participants’ allocation behavior by identifying the two types of allocation problems (Figure 2), which contributes to theoretical progress in decision-making research.

In particular, joint evaluation in Type A allocation problems (Figure 2a) suggests that individuals become sensitive to minor differences in their self-reward in two allocations when they are not in advantageous situations. First, this type of joint evaluation highlights an important practical implication. Namely, excessive inequality in society or allocation policy can be detrimental to society because individuals in disadvantageous situations are less likely to care about socially good outcomes, which highlights the importance of social equality (Wilkinson, 2007) from the point of decision-making with joint evaluation. Second, in addition to the practical implication, our findings also contribute to the theory of joint evaluation as well. Previous studies already showed that people become sensitive to small differences in a certain attribute in a JE situation, which was considered one of the mechanisms behind joint evaluations (Hsee et al., 1999). Our results additionally clarified whether this known pattern depends on decision-makers’ social situations. That is, we did not observe sensitivity to small differences (in self-reward) when participants were in advantageous situations (i.e., their reward was greater than that of others) and only discovered this phenomenon when participants were in non-advantageous situations (as illustrated in Figure 2a). Future research may perform a confirmatory experiment, similar to Study 2, to directly demonstrate that the difference between a JE and SE situation would matter less in advantageous situations than in disadvantageous situations.

The relationship between self-interest and inequality aversion (i.e., fairness) has been emphasized in previous research on equity in social psychology. Prior research suggests that individuals have a psychological drive to create fair relationships with others and receive compensation based on their own or others’ efforts (Adams, 1963, 1965; Adams & Rosenbaum, 1962; Homans, 1961). Critically, people experience distress when fairness is violated (Leventhal et al., 1969; Walster et al., 1973), indicating that distress is one of the primary psychological drivers for pursuing fairness. Moreover, research has indicated that individuals are more compelled to seek fair allocation when they are disadvantaged, as opposed to benefiting from an unfair allocation (Andrews, 1967; Pritchard, 1969). This indicates that individuals experience greater distress when they are being exploited. These psychological mechanisms may explain why individuals are particularly sensitive to self-interest in situations where they are disadvantaged, as demonstrated in the current study’s joint evaluation.

This study addresses the classical issue of inequality by examining the evaluation mode of decision-making. As mentioned earlier, research has shown that individuals tend to pursue the restoration of equality, especially when they are at a disadvantage. The current work sheds further light on the allocation problems under which psychological distress or resentment that arise from unfair treatment could trigger joint-evaluation allocation decisions.

It is worth noting that the current study only examined the simpler concept of equality, which did not involve input from each recipient. In contrast, equity research in social psychology primarily focused on determining fair output based on each person’s input, which can be considered a more complex social situation. Thus, future research should aim to further clarify how joint evaluation affects allocation behavior by employing richer allocation situations that involve input from each participant.

On the other hand, joint evaluation in Type B allocation problems indicates that participants chose a dominating allocation and avoided a dominated one. The SE model was unable to predict participants’ likely choices for a dominating allocation. In the choice problem 8526, shown in Figure 2b, option B dominates option A, while the extent of disadvantageous inequity is higher in option B than option A. The SE model can model social preferences, such as inequity. However, it cannot take into account the dominating/dominated relationship since it processes each option separately. Conversely, the architecture of the JE model enables us to model the simultaneous effects of social preferences and the dominating/dominated effect. In conclusion, our experiment highlights the importance of employing models that can simultaneously take multiple psychological effects into account regarding allocation choices.

While our intuition naturally fits with the discovered joint evaluation types or distributional preferences once we knew them, models that do not factor in joint evaluation cannot discover the same phenomena, demonstrating the benefit of using machine learning. Critically, we can also use the machine learning method to confirm whether our intuition or manually conceived behavioral patterns would be supported by the formal algorithm. In sum, the current paper demonstrates that such human-machine collaboration for scientific discovery can accelerate psychological science.

Our results also showed that participants who were more likely to change their behavior between joint and separate evaluation in Type A allocation problems were also more likely to do so in Type B, as seen in Figure 4b. This systematic individual difference raises questions about psychological and cognitive mechanisms underlying joint evaluation. For example, individual differences in information processing during decision-making may be relevant to the difference in joint evaluation. This prediction may be tested using a process tracing technique like eye-tracking technique (Holmqvist et al., 2011) to measure how people allocate attention to each option when they decide (Krajbich et al., 2010). Specifically, it may be possible that people who frequently alternate their attention between each option during evaluation tend to make allocation behavior that is modulated more strongly by joint evaluation. Alternatively, but not mutually exclusive, differences in individual cognitive resources would also explain the individual difference in joint evaluation. If joint evaluation is more cognitively demanding compared to separate evaluation, people who have more cognitive resources at their disposal may be more likely to make joint evaluation. To test this conjecture, it would be reasonable to intervene in the availability of people’s cognitive resources by imposing a cognitive load on them (Sweller, 1988). Taken together, the relationship between Types A and B demonstrated in the current study may contribute to literature by suggesting potential psychological mechanisms regarding joint and separate evaluation that will be testable in future research.

The present study utilized artificial neural networks to compare joint evaluation and separate evaluation. Although only the JE model was capable of modeling joint evaluation, both the JE and SE models could model social preferences such as inequality aversion as the influence of monetary allocations to self and others interacted within either neural network (Figures 1b and 1c). Therefore, the machine learning method allowed us to model how individuals value social relationships and evaluate social situations. Given the flexibility of the machine learning approach, using a similar technique to model richer social interactions is feasible. For instance, it can be essential to consider whether recipients of an allocation are in-group or out-group members (Brewer, 1979; Tajfel et al., 1971) for a decision-maker in allocation decisions. Incorporating such information about group membership into a machine learning model can be done by including a variable that indicates whether each recipient, other than the decision maker, is an in-group or an out-group member. For the JE model in this study, incorporating recipients’ identity would add two input variables that indicate the identity of recipients X and Y, in addition to the six monetary inputs already received by the model. Importantly, this new JE model can take into account a potentially complex interaction between in-group favoritism and various types of social preferences, such as inequality-aversion, in a data-driven way, which can be challenging for researchers to hypothesize through intuition. Additionally, the model is flexible enough to accommodate other relevant variables that affect in-group favoritism by incorporating additional input nodes. Moreover, to provide a comprehensive analysis, it is essential not only to allow flexible modeling of interaction effects between critical features, such as in-group favoritism and social preferences, but also to compare various models with distinct architectural features. This comparison of models, similar to the present study comparing the JE and SE models, helps in better understanding of psychological mechanism. Specifically, comparing models that allow for the interaction between certain features with those that do not is useful. If the former flexible model performs better in predicting human choices than the latter one, it implies the need to consider interaction effects to develop more accurate models of human psychology.

We acknowledge constraints on generality regarding the current article. First, because we analyzed the mean choice rates of participants, and given that different participants faced different choice problems in Study 1, we could not confirm whether our inference about joint evaluation would also be consistent at the individual level. It is important to consider individual heterogeneity when one attempts to generalize insights at the group and individual levels (Fisher et al., 2018; Shadish et al., 2002), not to mention the generalization to real-world settings. However, this issue has been resolved in Study 2 with individual-level analyses. We demonstrated that participants changed their choices between the JE and SE conditions in both Type A and Type B allocation problems. Second, we needed to recruit crowd workers in Study 1 to obtain large samples for our machine learning approach. In Study 2, we used the same crowd-sourcing website to recruit participants with similar characteristics as the participants in Study 1. Although crowd workers are generally more diverse populations than student samples, and social preferences have been exhibited by people across various populations, we should be careful to generalize the current findings because we do not fully know how joint evaluation influences people’s choices in social allocation.

We also acknowledge that the incentive compatibility in this study was not strong. Only one participant was paid according to an allocation choice during the experiment both in Study 1 and Study 2. Despite this limitation, we conducted an additional analysis to confirm that the participants’ behavior aligned with previous research in terms of pursuing self-interest. The result of this analysis is presented in the section “Incentive compatibility” in the Supplemental Online Material.

It would be reasonable to include a separate evaluation condition in Study 1 to identify allocation problems that cause participants to behave differently in separate and joint evaluations. However, we believe that using only a joint evaluation situation in Study 1 and comparing the JE model to the SE model can also be a reasonable approach to identify allocation patterns that result in different behaviors between separate and joint evaluations. The SE model can accurately predict participants’ behavior as long as their evaluation of an allocation problem is not influenced by joint evaluation. On the other hand, if joint evaluation specifically modulates participants’ behavior, the SE model cannot predict their behavior, and only the JE model can do so. This difference in algorithmic capability allows us to identify allocation problems that cause participants to behave differently between separate and joint evaluations, even though we only used a joint evaluation situation in Study 1. In other words, we investigated what would happen if we used a misspecified model (i.e., the SE model) instead of a more plausibly specified model (the JE model) and shed light on allocation problems that can only be captured by the correctly specified model.

Our manuscript did not demonstrate that only machine learning, specifically neural networks, can discover the distributional preferences modulated by joint evaluation. We acknowledge that non-machine learning models can be sufficient when a researcher can predict how and when joint evaluation modulates people’s distributional preferences based on past empirical and theoretical findings. However, our aim in the present study was to explore allocation problems that are evaluated jointly without relying too heavily on previous behavioral theories. Our rationale for doing so is that we suspected there may be some behavioral patterns that have not been captured by previous theories. To accomplish this, we utilized neural networks that are more flexible than traditional choice models to explore the 13,041 allocation problems.

Regarding the results showing that the JE model predicts participants’ behavior with higher accuracy than the SE model, we acknowledge the possibility of researcher bias. In other words, depending on the architecture of the neural network and various parameter settings, there remains a potential for the JE and SE models to demonstrate similar levels of performance. In order to address this concern, we have provided the results of an automated hyperparameter search in the section titled “Automated Hyperparameter Search” in the Supplemental Online Material. Moreover, it is important to be cautious when interpreting our results, as the use of softmax function in building our neural networks made the results non-linear and potentially complicated the comparison between the two models.

It should also be noted that the high predictive accuracy exhibited by the JE model does not imply that predicting people’s distributional preferences in general is an easy task. The 13,041 choice problems used in the current study were densely created, and similar problems can be included in both the test and training sets when we evaluate the models using cross-validation, leading to overoptimistic performance estimates. This does not negatively affect the contributions made by the current study because our focus was not on evaluating the absolute predictive performance of the JE or SE models. However, future studies that focus purely on predicting allocation behavior may need to consider these limitations.

Lastly, obtaining a deeper understanding of the mechanism behind joint evaluation is critical to addressing the concern about generalizability (Shadish et al., 2002; Yarkoni, 2020) and to integrating our findings with past studies (Huber et al., 1982; Simonson, 1989; Tversky, 1972). To better understand the mechanism, one promising way might be to interpret the learned weights of neural networks (Tsang et al., 2017; but see also Rudin, 2019) given that joint evaluation can be represented by interaction effects among inputs of a neural network (Figure 1c). While we primally focused on the discovery and replication of joint evaluation in social decision-making, we believe the current findings would contribute to the further understanding of the mechanism behind decision-making with joint evaluation.

The current study shows that individuals engage in joint evaluation instead of separate evaluation, which is consistent with previous research (Bettman et al., 1998; Stewart et al., 2003; Trueblood et al., 2013). However, our study took it a step further by addressing a larger scale problem space of 13,041 allocation problems compared to previous research (cf. Almaatouq et al., 2024). Our approach allowed for a more comprehensive investigation of allocation problems explained by models assuming joint evaluation versus those explained by models assuming separate evaluation. This extensive investigation would not be possible with manual examination of only a few questions by researchers. We believe that the present work will serve as a reasonable example of the effective use of machine learning models in social psychology.

Contributed to conception and design: AU, HT

Contributed to acquisition of data: AU

Contributed to analysis and interpretation of data: AU, HT

Drafted and/or revised the article: AU, HT

Approved the submitted version for publication: AU, HT

This study was supported by JSPS KAKENHI grants (nos. 21J00403 and 20H01563) and a Research Grant from Yoshida Hideo Memorial Foundation.

The authors declare that there’s no financial/personal interest or belief that could affect their objectivity.

Deidentified data for all studies along with a codebook, the materials, and data-analysis scripts are posted at https://doi.org/10.17605/OSF.IO/WFXJD

Adams, J. S. (1963). Towards an understanding of inequity. The Journal of Abnormal and Social Psychology, 67(5), 422–436. https://doi.org/10.1037/h0040968
Adams, J. S. (1965). Inequity in social exchange. In L. Berkowitz (Ed.), Advances in experimental social psychology (pp. 267–299). Elsevier.
Adams, J. S., & Rosenbaum, W. B. (1962). The relationship of worker productivity to cognitive dissonance about wage inequities. Journal of Applied Psychology, 46(3), 161–164. https://doi.org/10.1037/h0047751
Agrawal, M., Peterson, J. C., & Griffiths, T. L. (2020). Scaling up psychology via Scientific Regret Minimization. Proceedings of the National Academy of Sciences of the United States of America, 117(16), 8825–8835. https://doi.org/10.1073/pnas.1915841117
Almaatouq, A., Griffiths, T. L., Suchow, J. W., Whiting, M. E., Evans, J., & Watts, D. J. (2024). Beyond playing 20 questions with nature: Integrative experiment design in the social and behavioral sciences. Behavioral and Brain Sciences, 47, e33. https://doi.org/10.1017/S0140525X22002874
Andrews, I. R. (1967). Wage inequity and job performance: An experimental study. Journal of Applied Psychology, 51(1), 39–45. https://doi.org/10.1037/h0024242
Bazerman, M. H., Loewenstein, G. F., & White, S. B. (1992). Reversals of preference in allocation decisions: Judging an alternative versus choosing among alternatives. Administrative Science Quarterly, 37(2), 220–240. https://doi.org/10.2307/2393222
Bettman, J. R., Luce, M., & Payne, J. (1998). Constructive consumer choice processes. Journal of Consumer Research, 25(3), 187–217. https://doi.org/10.1086/209535
Blake, P. R., & McAuliffe, K. (2011). “I had so much it didn’t seem fair”: Eight-year-olds reject two forms of inequity. Cognition, 120(2), 215–224. https://doi.org/10.1016/j.cognition.2011.04.006
Blake, P. R., McAuliffe, K., Corbit, J., Callaghan, T. C., Barry, O., Bowie, A., Kleutsch, L., Kramer, K. L., Ross, E., Vongsachang, H., Wrangham, R., & Warneken, F. (2015). The ontogeny of fairness in seven societies. Nature, 528(7581), 258–261. https://doi.org/10.1038/nature15703
Bolton, G. E., & Ockenfels, A. (2000). ERC: A theory of equity, reciprocity, and competition. American Economic Review, 90(1), 166–193. https://doi.org/10.1257/aer.90.1.166
Brewer, M. B. (1979). In-group bias in the minimal intergroup situation: A cognitive-motivational analysis. Psychological Bulletin, 86(2), 307–324. https://doi.org/10.1037/0033-2909.86.2.307
Bürkner, P.-C. (2017). brms: an R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01
Charness, G., & Rabin, M. (2002). Understanding social preferences with simple tests. Quarterly Journal of Economics, 117(3), 817–869. https://doi.org/10.1162/003355302760193904
Deutsch, M. (1985). Distributive justice: A social-psychological perspective. Yale University Press.
Dhami, S. (2016). The foundations of behavioral economic analysis. Oxford University Press.
Engelmann, D., & Strobel, M. (2004). Inequality aversion, efficiency, and maximin preferences in simple distribution experiments. American Economic Review, 94(4), 857–869. https://doi.org/10.1257/0002828042002741
Fehr, E., & Schmidt, K. M. (1999). A theory of fairness, competition, and cooperation. Quarterly Journal of Economics, 114(3), 817–868. https://doi.org/10.1162/003355399556151
Fisher, A. J., Medaglia, J. D., & Jeronimus, B. F. (2018). Lack of group-to-individual generalizability is a threat to human subjects research. Proceedings of the National Academy of Sciences of the United States of America, 115(27), E6106–E6115. https://doi.org/10.1073/pnas.1711978115
Fisman, R., Jakiela, P., Kariv, S., & Markovits, D. (2015). The distributional preferences of an elite. Science, 349(6254), aab0096. https://doi.org/10.1126/science.aab0096
Forsythe, R., Horowitz, J. L., Savin, N. E., & Sefton, M. (1994). Fairness in simple bargaining experiments. Games and Economic Behavior, 6(3), 347–369. https://doi.org/10.1006/game.1994.1021
Frohlich, N., & Oppenheimer, J. A. (1992). Choosing justice: An experimental approach to ethical theory. University of California Press. https://doi.org/10.1525/9780520914490
Fudenberg, D., & Liang, A. (2019). Predicting and understanding initial play. American Economic Review, 109(12), 4112–4141. https://doi.org/10.1257/aer.20180654
Gates, V., Griffiths, T. L., & Dragan, A. D. (2020). How to Be Helpful to Multiple People at Once. Cogn Sci, 44, e12841. https://doi.org/10.1111/cogs.12841
Güth, W., Schmittberger, R., Schwarze, B. (1982). An experimental analysis of ultimatum bargaining. Journal of Economic Behavior Organization, 3(4), 367–388. https://doi.org/10.1016/0167-2681(82)90011-7
Harsanyi, J. C. (1975). Can the maximin principle serve as a basis for morality? A critique of John Rawls’s theory. American Political Science Review, 69(2), 594–606. https://doi.org/10.2307/1959090
Henrich, J., Boyd, R., Bowles, S., Camerer, C., Fehr, E., Gintis, H., McElreath, R. (2001). In search of homo economicus: behavioral experiments in 15 small-scale societies. American Economic Review, 91(2), 73–78. https://doi.org/10.1257/aer.91.2.73
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford University Press.
Homans, G. C. (1961). Social behavior: Its elementary forms. Harcourt, Brace World.
Hsee, C. K. (1996). The evaluability hypothesis: An explanation for preference reversals between joint and separate evaluations of alternatives. Organizational Behavior and Human Decision Processes, 67(3), 247–257. https://doi.org/10.1006/obhd.1996.0077
Hsee, C. K., Loewenstein, G. F., Blount, S., Bazerman, M. H. (1999). Preference reversals between joint and separate evaluations of options: A review and theoretical analysis. Psychological Bulletin, 125(5), 576–590. https://doi.org/10.1037/0033-2909.125.5.576
Huber, J., Payne, J. W., Puto, C. (1982). Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9(1), 90–98. https://doi.org/10.1086/208899
Kahneman, D., Knetsch, J. L., Thaler, R. H. (1986). Fairness and the assumptions of economics. Journal of Business, S285–S300. https://doi.org/10.1086/296367
Kameda, T., Inukai, K., Higuchi, S., Ogawa, A., Kim, H., Matsuda, T., Sakagami, M. (2016). Rawlsian maximin rule operates as a common cognitive anchor in distributive justice and risky decisions. Proceedings of the National Academy of Sciences of the United States of America, 113(42), 11817–11822. https://doi.org/10.1073/pnas.1602641113
Konow, J. (2003). Which is the fairest one of all? A positive analysis of justice theories. Journal of Economic Literature, 41(4), 1188–1239. https://doi.org/10.1257/002205103771800013
Krajbich, I., Armel, C., Rangel, A. (2010). Visual fixations and the computation and comparison of value in simple choice. Nature Neuroscience, 13(10), 1292–1298. https://doi.org/10.1038/nn.2635
LeNail, A. (2019). NN-SVG: Publication-ready neural network architecture schematics. Journal of Open Source Software, 4(33), 747. https://doi.org/10.21105/joss.00747
Leventhal, G. S., Allen, J., Kemelgor, B. (1969). Reducing inequity by reallocating rewards. Psychonomic Science, 14, 295–296. https://doi.org/10.3758/BF03329132
Mitchell, G., Tetlock, P. E., Mellers, B. A., Ordóñez, L. D. (1993). Judgments of social justice: Compromises between equality and efficiency. Journal of Personality and Social Psychology, 65(4), 629–639. https://doi.org/10.1037/0022-3514.65.4.629
Morishima, Y., Schunk, D., Bruhin, A., Ruff, C. C., Fehr, E. (2012). Linking brain structure and activation in temporoparietal junction to explain the neurobiology of human altruism. Neuron, 75(1), 73–79. https://doi.org/10.1016/j.neuron.2012.05.021
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., … Bai, J. (2019). PyTorch: An imperative style, high-performance deep learning library. arXiv. https://arxiv.org/abs/1912.01703
Peterson, J. C., Bourgin, D. D., Agrawal, M., Reichman, D., Griffiths, T. L. (2021). Using large-scale experiments and machine learning to discover theories of human decision-making. Science, 372(6547), 1209–1214. https://doi.org/10.1126/science.abe2629
Peysakhovich, A., Naecker, J. (2017). Using methods from machine learning to evaluate behavioral models of choice under risk and ambiguity. Journal of Economic Behavior and Organization, 133, 373–384. https://doi.org/10.1016/j.jebo.2016.08.017
Plonsky, O., Apel, R., Ert, E., Tennenholtz, M., Bourgin, D., Peterson, J. C., … Erev, I. (2019). Predicting human decisions with behavioral theories and machine learning. arXiv. https://arxiv.org/abs/1904.06866
Plonsky, O., Erev, I. (2021). To predict human choice, consider the context. Trends in Cognitive Sciences, 25(10), 819–820. https://doi.org/10.1016/j.tics.2021.07.007
Pritchard, R. D. (1969). Equity theory: A review and critique. Organizational Behavior Human Performance, 4(2), 176–211. https://doi.org/10.1016/0030-5073(69)90005-1
Rawls, J. (1971). A theory of justice. Harvard University Press. https://doi.org/10.4159/9780674042605
Regenwetter, M., Dana, J., Davis-Stober, C. P. (2011). Transitivity of preferences. Psychological Review, 118(1), 42–56. https://doi.org/10.1037/a0021150
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215. https://doi.org/10.1038/s42256-019-0048-x
Schönbrodt, F. D., Wagenmakers, E. J. (2018). Bayes factor design analysis: Planning for compelling evidence. Psychonomic Bulletin and Review, 25(1), 128–142. https://doi.org/10.3758/s13423-017-1230-y
Shadish, W. R., Cook, T. D., Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton, Mifflin, and Company.
Simonson, I. (1989). Choice based on reasons: The case of attraction and compromise effects. Journal of Consumer Research, 16(2), 158–174. https://doi.org/10.1086/209205
Skirzyński, J., Becker, F., Lieder, F. (2021). Automatic discovery of interpretable planning strategies. Machine Learning, 110(9), 2641–2683. https://doi.org/10.1007/s10994-021-05963-2
Stewart, N., Chater, N., Stott, H. P., Reimers, S. (2003). Prospect relativity: How choice options influence decision under risk. Journal of Experimental Psychology: General, 132(1), 23–46. https://doi.org/10.1037/0096-3445.132.1.23
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4
Tajfel, H., Billig, M. G., Bundy, R. P., Flament, C. (1971). Social categorization and intergroup behaviour. European Journal of Social Psychology, 1(2), 149–178. https://doi.org/10.1002/ejsp.2420010202
Tricomi, E., Rangel, A., Camerer, C. F., O’Doherty, J. P. (2010). Neural evidence for inequality-averse social preferences. Nature, 463(7284), 1089–1091. https://doi.org/10.1038/nature08785
Trueblood, J. S., Brown, S. D., Heathcote, A., Busemeyer, J. R. (2013). Not just for consumers: Context effects are fundamental to decision making. Psychological Science, 24(6), 901–908. https://doi.org/10.1177/0956797612464241
Tsang, S., Cheng, D., Liu, Y. (2017). Detecting statistical interactions from neural network weights. In arXiv. https://arxiv.org/abs/1705.04977
Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79(4), 281–299. https://doi.org/10.1037/h0032955
Tversky, A., Simonson, I. (1993). Context-dependent preferences. Management Science, 39(10), 1179–1189. https://doi.org/10.1287/mnsc.39.10.1179
Walster, E., Berscheid, E., Walster, G. W. (1973). New directions in equity research. Journal of Personality and Social Psychology, 25(2), 151–176. https://doi.org/10.1037/h0033967
Wilkinson, R. G. (2007). The impact of inequality: How to make sick societies healthier. New Press.
Yarkoni, T. (2020). The generalizability crisis. Behavioral and Brain Sciences, 45, e1. https://doi.org/10.1017/S0140525X20001685
This is an open access article distributed under the terms of the Creative Commons Attribution License (4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Supplementary Material