The Science Behind the Magic? The Relation of the Harry Potter “Sorting Hat Quiz” to Personality and Human Values

Harry Potter fans can be assigned to one of these Houses by filling out an eight-item questionnaire (the Sorting Hat Quiz) available on the Pottermore website (https://www.pottermore.com). Crysel et al. (2015) asked Pottermore users to complete the Sorting Hat Quiz as well as to answer questions regarding their personality traits such as the Big Five (McCrae & John, 1992) and the Dark Triad (Jones & Paulhus, 2013). The authors report significant associations between personality characteristics and House affiliation in line with the depictions in the book; for instance, they found that Slytherin House members scored higher on Dark Triad traits. It is important to note that the Sorting Hat Quiz was not developed with any scientific purpose in mind, and includes items of ambiguous content such as “Dawn or Dusk?” and “Heads or Tails?”. Thus, on the face of it, the validity of the Sorting Hat Quiz seems problematic. However, we do not wish to question J.K. Rowling, as albeit opaque to us, she might have had deep reasons for including such questions; moreover, being a proprietary test, the scoring rules are not public. This viewpoint may be substantiated by the results of Crysel et al. (2015), who found that the Sorting Hat Quiz is related to established personality traits. We refer back to this tension between face validity and predictability in the discussion.

Crysel et al. (2015) were the first to study the relation between Harry Potter Houses and established personality traits. However, their study has several limitations which attenuate the strength of the authors’ conclusions. First, the overall sample size collected was low (N = 132), with certain Houses being represented by a remarkably small number of respondents (e.g., n = 23 for Gryffindor). This could have led to overestimated effect sizes, since the only way to get a significant result is by having an estimate that “overcomes” a large standard error (e.g., Yarkoni, 2009). Two main concerns were present with regards to Crysel et al.’s selection of measures. Firstly, personality scales were deemed as unreliable, with Cronbach’s alpha as low as .48 (Agreeableness) and .42 (Openness). Thus, selecting short measures of personality could pose some problems in the latter analysis. Low reliability poses problems when performing ANOVA (in terms of Type I and II errors) using observed scores (Bobko, Roth & Bobko, 2001), as measurement error—which increases as reliability decreases—directly disattenuates the squared sum of errors involved in the F-statistic computation (which would be underestimated; Liu, & Salvendy, 2009). Ultimately, this situation would also affect the classical significance test involved. Second, the choice of short versions for personality measurements may have undermined internal consistency (see Table 1 in Crysel et al., 2015) and given a less precise picture of participants’ personalities. Finally, we believe that the analysis proposed by Crysel et al. (2015) did not adequately reflect the research questions they aimed to test, and under certain circumstances could have led to wrong conclusions (see Statistical Analysis section).

In the current study, we aim to address these limitations by conducting a replication of Crysel et al. (2015). Specifically, we recruited a considerably larger sample of participants, employed measures with stronger psychometric properties, and utilized a Bayesian framework to more adequately translate theoretical predictions into statistical hypotheses (Etz, Haaf, Rouder, & Vandekerckhove, 2018; Klugkist & Hoijtink, 2007).

On a theoretical level, we stipulate that the decisions made by the Sorting Hat are more strongly based on an individual’s values rather than on personality traits. This hypothesis is based on human values (Table 2) being more closely aligned with House descriptions as reflected in the Harry Potter books. According to Schwartz (2012), the relative importance of multiple values one holds guides actions, distinguishing the existing values by the theoretical model from one another by the goal or the motivation they express (Table 2). Therefore, we aim to investigate whether identification processes could relate to the theory of Basic Human Values (Schwartz, 1992; Schwartz & Bilsky, 1990). Additionally, values are thought to be universal across cultures, and the theory of Human Values may be a useful framework for understanding a person’s identification with fictional elements. To our knowledge, the relation between Human Values and the identification with fictional elements has not been explored in the literature before.

We report how we attained our sample size, all data exclusions (if any), all manipulations, and all measures in the study (as suggested by Simmons, Nelson, & Simonsohn, 2012). Statistical analyses were performed using the R environment for statistical computing (R Development Core team, 2015). All materials including data and code can be found online at https://osf.io/rtf74/. This research was conducted in accordance to the Declaration of Helsinki, with ethical approval granted by the Department of Psychology, Centre for Croatian Studies, University of Zagreb, prior to data collection. All participants provided informed consent for survey procedure, and were informed about data handling procedures prior to their participation.

We recruited participants through social media (Facebook, Twitter, and Reddit), private contacts, and student e-mail groups. Of the 988 participants who took part in our study, 91 participants were excluded for not meeting the age criterion of 18 years, and one for indicating an age of 122. We removed 49 of the remaining participants as the Sorting Hat assigned them to multiple Houses, resulting in a total sample size of 847 (Table 3). The median age of the remaining participants was 23 with a standard deviation of 4.36. The majority of participants were women (702).

Prior to the start of this study, each participant was requested to, if they have not done so before, complete the Pottermore Sorting Hat Quiz. The items and scoring key of the Sorting Hat Quiz are not publicly available, and our multiple attempts to obtain access have not been successful. In general, the Sorting Hat Quiz uses a database of 28 questions and eight questions are displayed to each test taker. The questions have a range of possible answers and the questions seem to not always be related to a particular trait of the House members, although most have a clear relation to a certain part of the Harry Potter series and Houses described in it.

Subsequently, participants were asked for the assigned Hogwarts House membership and whether it constituted their desired House. We had no means to check whether the person was providing rightful information with regards to the House assigned by the Pottermore quiz, as we had no access to the quiz itself and thus had to trust individuals to provide sincere responses. Additionally, participants were asked to declare which House they would like to have been sorted in. Demographic data collected included age, gender, country, education level, occupation, and native language, as well as data about participants’ experience with the Harry Potter franchise (i.e., the books and movies they have read or watched).

Building upon the hypotheses proposed by Crysel et al. (2015), we decided to reproduce their research design—with a few modifications. In particular, while they used the Ten-Item Personality Inventory (TIPI; Gosling, Rentfrow, & Swann, 2003) to measure the Big Five traits (Extraversion, Conscientiousness, Agreeableness, Emotional Stability, and Openness to Experience), we chose the 50-item International Personality Item Pool (IPIP 50; Goldberg et al., 2006). The IPIP 50 encompasses ten items per trait, and it provides higher reliability than other alternatives (Ypofanti et al., 2015). The items present individual descriptions measured with a 5-point Likert scale (1 = very inaccurate to 5 = very accurate).

The Dark Triad encapsulates three personality traits often deemed negative (Jones & Paulhus, 2013): Psychopathy (lack of emotional warmth for others or empathy paired with sensation seeking, risk-taking behaviour, and lack of guilt); Machiavellianism (tendency to manipulate others for own gain); and Narcissism (exaggerated sense of grandiosity, importance, entitlement, and need to be admired). To improve reliability, we again replaced the original questionnaire with an alternative. While the original authors selected the Dark Triad Dirty Dozen, we selected the Short Dark Triad (SD3, Jones & Paulhus, 2013). It evaluates the three personality traits with nine items per trait on a five-point Likert-scale (1 = strongly disagree to 5 = strongly agree). Total scores for each IPIP-50 or SD3 subscale are obtained by summing the respective item scores.

We measured participants’ values via the Portrait Values Questionnaire (PVQ-RR; Schwartz et al., 2012). The PVQ-RR uses 57 items to describe an individual’s goal held as important in their life, and the respondent is prompted to indicate to what extent he or she is similar to this fictional individual. The items are measured using a 6-point Likert-scale (1 = not like me at all to 6 = very much like me). The scores are obtained as a 10-value model following the instructions provided by Schwartz (2016), namely reversing negative items as well as obtaining mean-centred PVQ values’ scores.

Crysel et al. (2015) hypothesized that Gryffindor would score higher on Extraversion; Ravenclaw on Intellect; Hufflepuff on Agreeableness, Emotional Stability, and Conscientiousness; and Slytherin on Machiavellianism, Narcissism, and Psychopathy. They used ANOVAs with Helmert contrasts to compare, for example, the mean score of Gryffindor against the mean score of all the other Houses. However, it is possible for Gryffindor to have significantly higher scores than the mean of the other Houses even if Ravenclaw and Slytherin have higher scores than Gryffindor. For this situation to happen, the remaining House (e.g., Hufflepuff) would only need to score substantially lower on this variable (see Appendix 1 for a simulated example). Thus, this statistical approach could provide misleading results under specific (but not unlikely) circumstances.

We decided to employ an alternative statistical approach that would allow us to test what we believe are the original authors’ (and our) specific research questions: Does Gryffindor score highest on Extraversion? Ravenclaw highest on Intellect? Hufflepuff highest on Agreeableness, Emotional Stability, and Conscientiousness? Slytherin highest on Machiavellianism, Narcissism, and Psychopathy?

We use Bayesian order-constrained inference to assess the evidence for these hypotheses. Bayesian statistics entails quantifying one’s uncertainty about the world using probability; for details, see Appendix 2. For a gentle introduction which incidentally uses Harry Potter as an example, see Etz and Vandekerckhove (2018).

Our statistical approach is as follows: for each hypothesis of interest, we compared three main models that reflected three different hypotheses regarding Houses’ means on the trait of interest. First, we specified a restricted model (M_r) that reflected our substantive hypothesis. In other words, in this model one of the House means is restricted to be higher than the alternative Houses’ means. This definition reflects our hypothesis that members of a specific House are expected to score the highest when compared with members of other Houses. Second, we estimated a full model (M_f) where the means were allowed to vary freely. In other words, this model reflects a lack of knowledge regarding which House (if any) will score higher or lower. Finally, we estimated a model (M₀) in which all means were constrained to be equal. This model reflects the hypothesis that House membership is not associated with personality.

For each of our hypotheses, we tested which of the three models was supported by the data using Bayes factors. The Bayes factor formalizes how well one model (e.g., the model reflecting our hypothesis; M_r) predicts the observed data relative to another model (e.g., the model implying all Houses means to be equal; M₀). Consequently, a higher Bayes factor when comparing M_r against M₀ would indicate that data are supporting our hypothesis compared to the hypothesis that all Houses have the same mean. To compare all three models at once, we transform the Bayes factors into posterior model probabilities. Interpreting these posterior model probabilities requires care. First, we have assigned uniform priors to models, and readers may prefer different priors. Second, by reporting posterior model probabilities, we have effectively reduced the number of possible models to three. We are thus in the “small world of statistics” (McElreath, 2015): probability statements are conditional on the set of models (see also Morey, Romeijn, & Rouder, 2013). From a pragmatic perspective, however, we believe that the three models we focus on here are fairly exhaustive: they include a sceptic’s view, predicting that all means are equal (M₀); the most flexible (M_f) model; and a substantively motivated model (M_r). We complete these analyses by estimating the variance explained for each case (Marsman, Waldorp, Dablander, & Wagenmakers, 2019). Prior information is needed in Bayesian inference. We used “default priors” for all our analyses (Rouder, Morey, Speckman, & Province, 2012); further details concerning priors and sensitivity analyses are presented in Appendix 2.

Descriptive information regarding each House’s raw scores on personality measures is visualised in Figure 1. Overall, individuals scored higher in Intellect and Agreeableness and lower on Dark Triad traits (especially Narcissism and Psychopathy), irrespective of House membership.

Table 4 displays the descriptive statistics, the correlations among the personality traits, and Cronbach’s α, an estimate of internal consistency (Cronbach, 1951; Cronbach & Shavelson, 2004). In contrast with Crysel et al., (2015), all measures showed excellent internal consistency indices, improving scale reliability.

Table 5 displays the posterior model probabilities for each hypothesis. We find that, except for the hypothesis for Emotional Stability, all our hypotheses are supported: the models in which Gryffindor scores the highest on Extraversion, Ravenclaw scores the highest on Intellect, Hufflepuff scores the highest on Agreeableness and Conscientiousness, and Slytherin scores the highest on Machiavellianism, Narcissism, and Psychopathy have the highest posterior probability, respectively.

Figure 2 shows the posterior distribution over the mean for each dependent variable and House, together with the 87% credible interval of the posterior predictive distribution. This distribution quantifies the uncertainty about a yet unseen data point for a particular House and dependent variable, given the data we have observed. Note that there is considerable uncertainty, indicating that, if one’s (somewhat quixotic) goal is to predict a person’s personality score, House is not of particular predictive utility.

Figure 3 shows that the proportion of variance explained in each case varied across personality measures: 10% (Agreeableness), 2.9% (Conscientiousness), 0.05% (Emotional Stability), 1.4% (Extraversion), 3.8% (Intellect), 11.3% (Machiavellianism), 6.6% (Narcissism), and 6.5% (Psychopathy).

In our study, similar as in Crysel et al. (2015), most people who were sorted into a particular House also desired that House (i.e., 60.1% of Gryffindors wanted Gryffindor, 68.5% of Hufflepuffs wanted Hufflepuff, 62.4% of Ravenclaws wanted Ravenclaw, and 75.7% of Slytherins wanted Slytherin). Conversely, most people who desired a particular House were sorted into that House (see Table 3).

Following Crysel et al. (2015), we studied what would happen if individuals would have been sorted to their desired House rather than the actual House sorting decided by the Pottermore quiz. We found a similar pattern of results for sorted and desired House, with the exception of Conscientiousness (p(M_r|y) = 0.77) and Emotional Stability (p(M_r|y) = 0.02). Thus, the desire to be sorted into a particular House—even if not sorted into that House—has very similar effects compared to being sorted into that House. What about people who have been sorted into a House that they did not desire to be sorted into? If the same pattern of results emerges, we could conclude that Crysel et al. (2015) and our finding are not due to self-selection—that is, answering questions in a self-serving way—but that the Pottermore quiz is related to personality. We tested this by restricting our analysis to participants who were sorted into the Houses they did not desire (which resulted in 92 Gryffindors, 90 Hufflepuffs, 55 Ravenclaws, and 58 Slytherins). For all but Agreeableness (p(M_r|y) = 0.90), the evidence was either equivocal or in favour of the null model (e.g., p(M₀|y) = 0.90 for Psychopathy). These results suggest that Crysel et al.’s (2015) and our findings are confounded by the desire to be sorted in a particular House; that is, the association between House and personality seems not to be driven by the particular House one is sorted into, but which particular House one desires to be sorted into.

We conduct a similar analysis as above for the Human Values data; Figure 4 shows the raw data. Table 6 displays the posterior model probabilities. Note that the second column indicates our confirmatory hypotheses and that our approach can be easily generalized to multiple order restrictions. For example, we predicted that both Gryffindor and Ravenclaw should score higher on the value “Achievement” than Hufflepuff and Slytherin. Such an order-constrained model is then compared against a null model (assuming no differences across Houses) and an unrestricted model (allowing the House means to vary freely). Our hypotheses for Conformity, Power, and Tradition were strongly supported and received the highest possible support. However—and similar to the personality measures—the variance explained by assuming four distinct Houses differed across values, from 1% (Hedonism) to 9.9% (Power). In particular, Figure 5 shows that the proportion of variance explained in each case varied across Human Values, but was generally low: 4.3% (Achievement), 2.7% (Benevolence), 5.1% (Conformity), 1% (Hedonism), 9.9% (Power), 1% (Security), 3.4% (Self-Enhancement), 4.6% (Stimulation), 2.8% (Tradition), 5.6% (Universalism). The variance explained by the Human Values is slightly lower (on average) than the variance explained by the personality measures.

Identification with fictional elements influences an individual’s perception of their own personality characteristics and values (Cohen, 2001). Individuals seek to establish connections between fictional elements and themselves, adapting their own views depending on the characteristics and groups identities that represent the main and side characters. Against this background, Crysel et al. (2015) hypothesized that readers and viewers of the Harry Potter series would engage with elements from this work of fiction—the Hogwarts House system—that reflected distinct characteristics which could be linked to individual personality traits (i.e., the Big Five and Dark Triad traits). In other words, Crysel and colleagues (2015) investigated the possibility that Harry Potter fans identify themselves with House-specific personality traits that are congruent with their own personality traits.

We aimed to replicate Crysel et al. (2015) and to gain a more nuanced understanding of identification processes through Houses. To achieve this, we used an alternative, larger sample, and investigated whether similar processes could also be related to the theory of Basic Human Values (Schwartz, 1992). We also employed a statistical framework that allowed us to more directly map our substantive into statistical hypotheses.

Our results partially support the original claims by Crysel et al. (2015), presenting evidence for the association between respondents’ personality and the House they identified with. We also demonstrated that the proportion of variance explained was too weak in many cases to merit specific original claims made by Crysel et al. (2015). It is doubtful that the “distinct Houses appear[s] to correspond to established psychological constructs” (Crysel et al., 2015, p. 178). Both Figure 3 and the central 87% of the posterior predictions displayed as error bars in Figure 2 show that predictive uncertainty remains high. For applied contexts, this means that being a member of a particular Hogwarts House does not reliably predict the relative positioning on personality measures.¹ Therefore, even though identification processes could be linked to personality traits to a minor extent, not much is to be learned about the personality of somebody who has been sorted into a particular Hogwarts House. An exception is Slytherin, which leads to a larger explained variance for Agreeableness and the Dark Triad.

On a theoretical level, we hypothesized that the Sorting Hat does not assign Houses based on personality but on values. While we also found some support for our confirmatory hypotheses concerning the theory of Basic Human Values (Schwartz & Boehnke, 2004), the overall effect sizes were again small. Similar to the results concerning personality, the variance explained differed across Human Values. Future research may look into these differences.

We echo the conclusion that an undesired House assignation had little effect on how individuals view themselves; that is, we agree with the original authors that “these findings suggest that participants who wish to be included in their respective Houses largely embodied the same traits associated with those Houses” (2015, p. 178). However, when considering House members who did not want to belong to this House (i.e., individuals who identify themselves with alternative Houses), the observed effect sizes diminished or disappeared. It seems that the desire to be assigned to a specific House acts as a confounder, inducing a relationship between the House assignment based on the Pottermore quiz and personality measures. Thus, under further scrutiny, it seems that not the House assignment per se, but the desire to be sorted into a particular House drives the association between Houses and personality.

The research presented here has various limitations. First, due to copyright restrictions, we did not have access to the individual response patterns but only to the self-reported House assignment. It could be speculated that relations between established psychological constructs and the Hogwarts Houses would be substantiated when looking at individual response patterns instead. However, glancing at the actual questions of the Sorting Hat Quiz, this is not immediately apparent. It features simplistic questions such as “Black or White?” and “Moon or Stars?”, and complex items as “If you were attending Hogwarts, which pet would you choose to take with you?”, with 15 different options to choose from. However, since the scoring rules are not public, it may well be that some of these opaquer questions are added for mere “show”, without actually being counted towards House assignment. If they do count, however, then this might seriously threaten the validity of the measure. Based on our results, it might well be that the Sorting Hat Quiz is valid only for people who have a strong desire to be sorted into a particular House, and thus are able to “game” the Quiz to result in their desired House.

A second limitation is that we analysed the mean responses instead of using latent variable techniques. It is possible that using structural equation modelling would have provided us with a more detailed picture. As we make our data publicly available (https://osf.io/rtf74/) we encourage interested researchers to engage in any additional analysis they find interesting. Lastly, future research should reproduce these results in alternative samples (specifically such that include participants not familiar with the Harry Potter series), controlling for previous knowledge of the fiction works, and potential age-related effects.

We have made all our materials, including data and analysis code publicly available at the Open Science Framework (https://osf.io/rtf74/).

The authors would like to thank Sanja Budimir for her guidance in the initial phases of this project, Jonas Haslbeck and Peter Edelsbrunner for comments on an earlier version of this manuscript, and Alexandra Sarafoglou for her comments on the final version of the manuscript.

The authors have no competing interests to declare.

• Contributed to conception and design: LJ, HJ, and EGG

• Contributed to acquisition of data: LJ, HJ, and EGG

• Contributed to analysis and interpretation of data: EGG and FD

• Drafted and/or revised the article: LJ, HJ, EGG, and FD

• Approved the submitted version for publication: LJ, HJ, EGG, and FD