What People Say and How People Play: Mixed Methods and Multiple Measures of Fishing Behavior with Implications for External Validity

Does resource extraction during experimental economic games reflect resource use and behavior in other contexts? This question about external validity is central to determining the inferences that can be drawn from experimental evidence to broader sets of circumstances. Building on previous studies of external validity, which often raise concerns about the generalizability of experimental economic games to other contexts, we instead highlight parallelism between behavior inside and outside of games and its implications for applied research. Here we present the results from a framed multiphase common-pool resource game played with fishermen from a fishing community in Baja California Sur to explore the associations between game play and both self-reported and observed behavioral measures. Participants played an experimental economic game that incorporated both fisheries-specific resource dynamics and the current policy context of local fisheries management. Game administration followed two years of interviews, participant observation, and household surveys in the community. In addition to providing an in-depth understanding of the broader social-ecological context, this allowed us to compare similar game, self-reported, and observed behaviors for the same individuals. Overall, we found little evidence for external validity in terms of parallelism. None of the tests for association between fishing behaviors in the game and survey met the study-wide significance threshold. Only one association—between game behavior and conservation values—was significant at the single-test level. Based on our results, we caution against relying solely on experimental economic games as proxies for behaviors regarding resource use when developing policies. Collecting multiple or iterative measures of behavior using mixed methods is a necessary precaution, especially if the goal is to make specific policy recommendations.