Try, try again: Lessons learned from success and failure in participatory modeling

Participatory Modeling (PM) is becoming increasingly common in environmental planning and conservation due to advances in cyberinfrastructure and to greater recognition of the importance of engaging a diverse array of stakeholders in decision making. We initiated this reflective article at the first of a series of workshops on PM, sponsored by the National Socio-Environmental Synthesis Center (SESYNC). The goal of the workshop series is to organize and consolidate scholarship around the practice of PM. PM encompasses the use of a broad range of modeling approaches in various forms of collaboration among practitioners, academics, and other stakeholders who engage in a purposeful learning process that elicits and formalizes the implicit and explicit knowledge of participants to support decision-making and action. Our knowledge community has learned important lessons from their experiences, successes, and failures. Given our experience, we firmly believe that PM has the potential to help communities guide themselves toward more positive futures. We offer these lessons learned in the hope that they will guide practitioners as they assist communities with using meaningful, appropriate, and illuminating modeling tools in participatory processes.

The reflections in this paper emerge from the authors’ over 200 years of cumulative and diverse experience conducting PM processes with communities concerned with different issues. Some examples of our work can be found in Table 1, at www.participatorymodeling.org, and in Gray et al., 2018.

Our collective interdisciplinary background allowed for reflection on a rich range of collaborative modeling approaches, incorporating insights and experiences with environmental modeling, spatial analysis, urban and regional planning, psychology, anthropology, computer science, social science, and economics. The reflections in this paper are also informed by prior summary and review articles. These prior articles, covering multiple participatory studies, describe commonalities, articulate different overarching structures for the PM processes, describe lessons learned, and propose principles for effective PM (e.g., Voinov and Brown Gaddis, 2008; Voinov and Bousquet, 2010; Oteros-Rozas et al., 2015; Voinov et al., 2016; Gray et al., 2018). This paper focuses less on summarizing previous PM projects, and more on what we have learned about the challenges of implementing PM, especially in creating meaningful stakeholder and modeler partnerships within the complex social and political contexts where PM is most needed.

These reflections and lessons learned were generated and organized as follows. During our first workshop (February 2016), each participant wrote a summary of the key lessons that they had learned in their PM research and practice. We used inductive logic (Charmaz and Belgrave, 2007) to identify patterns arising from compiled data and grouped them into the three key themes described below. We built on this synthesis through several iterations, drawing from diverse literature to provide the foundation for this paper.

We have not previously shared these lessons widely because their articulation is frequently extraneous to traditional academic scholarship. We are often discouraged from publishing about failures (Becu et al., 2007). Our intent herein is to share our experience beyond advancing the technological dimension of modeling. We seek to encourage, and maybe even inspire others to embrace the uncertainty and messiness inherent to PM through lessons we have learned via many trials, successes, and even more errors, very much like any modeling process (Railsback and Grimm, 2012). It is our hope that sharing these lessons will help other practitioners skip some of the more painful learning steps we ourselves have worked through, and more effectively build the powerful collective knowledge and social capital that can emerge from PM processes.

Figure 1 illustrates a common evolution of assumptions and practices in PM, where researchers, who are eager to put their skills and knowledge to good use in supporting decisions, move from a strictly technical perspective towards full embrace of the partnership perspective. Each puzzle piece depicted in Figure 1 is necessary but not sufficient in describing how best to achieve innovative solution-building and action. For example, a common initial assumption is that providing decision-makers with the “right” information (e.g., more accurate/precise data, an understanding of interaction effects) and the “right” tools (e.g., models that accurately represent processes, interactions, and provide meaningful results), is sufficient for them to solve their problems. This is not enough to lead to improved decisions, actions, and effective solutions, possibly because decision-makers distrust modeling processes and tools that exclude their involvement.

Faced with falling short of their goals, researchers add increasingly more nuance to their approaches, moving next to an emphasis on communicating in the “right” ways (e.g., clearly translating and presenting information and models to both decision-makers and stakeholders), and when that too fails, moving towards more inclusion and participation. Researchers may assume that engaging selected stakeholders in data collection, analysis, and modeling will facilitate ownership and make stakeholders feel empowered and informed. When anticipated outcomes do not occur, researchers then turn to the need for diversity to enhance innovation and representation. Conflict may arise in engaging multiple stakeholders. In response, while keeping in mind previous assumptions and responses, the researchers focus on social processes, towards an understanding that attending to emotions, transparency of models and assumptions, power dynamics, and tradeoffs are key elements of effective PM.

This evolution takes place in individuals as well as across whole fields where initial technocratic approaches are enhanced by and eventually embrace more inclusive approaches. We have seen similar evolutions in risk management (Fischhoff, 1995), in probabilistic decision analysis (Spetzler et al., 2016), and in stakeholder engagement (Sterling et al., 2017). Recognition of greater complexity leads to consideration of processes as well as outcomes; more inclusive public input and consultation approaches, including possibilities for civic engagement and co-management (Leong et al., 2009, 2012); and a “commitment to action” based on the entire process (Leong et al., 2011; Spetzler et al., 2016).

We have grouped our reflections into three main categories regarding lessons learned: a) how modelers need to engage in PM, b) how to adapt to the social and political context of collaboration, and c) how to set up the PM process itself. Our reflections may be of most interest to modelers and researchers but we also share these lessons learned for facilitators and communities who may be interested in working in this arena.

“Participatory modelers,” in the context of this paper, are individuals who facilitate, organize, and develop models for PM projects. A diverse set of skills, including prior content knowledge, facilitation skills, and technical modeling skills are needed to ensure that the PM process and resulting models are useful and helpful to the participating communities. It is a rare individual who possesses all of these skills; it is often better to consider a collaborative team as “the” participatory modeler (Prell et al., 2007). Here we report the skills that we have found to be most important for the team:

1. Content knowledge coupled with openness to learn from local experts: A participatory modeler needs to be able to speak intelligently with stakeholders about the problem being modeled, and thus needs to have or develop background knowledge regarding the subject matter(s) related to the site, problem, or issue to be modeled. In addition to academic knowledge, important content knowledge comes from learning directly from the stakeholders about their unique experience with the complex problem to be addressed, and about their concerns, needs, and aspirations. When modelers come in as “outside experts,” all too often the result is alienation rather than collaboration (Hoch et al., 2015; Zellner et al., In Press). Recognizing the value and importance of local content knowledge means that modelers focus less on their own expertise and more on learning from local stakeholders. As an example, in the context of a collaborative stormwater management park project in a low-income community (Zellner et al., 2018), project leaders developed an ice-breaker activity to explain how to use causal loop diagramming (CLD) to lay out the complex interactions between flooding and economic development. The issue of traffic congestion was chosen to illustrate how one could draw the links among factors that reinforce congestion or alleviate it. As project leaders developed the exercise with key stakeholders in preparation for a larger community workshop, they learned from participants that traffic congestion was not an issue in their community, so they could not relate to the topic or to the CLD activity. The team then switched to a housekeeping and chores problem that most people could relate to, and which proved to be very effective in illustrating how community members could use CLD to link the problem they were facing with their collective aspirations. This unlocked and completely transformed the planning process for the project. The community’s CLD was built on local knowledge, beyond what the modelers could contribute.

2. Skills to liaise effectively and build relationships with stakeholders: Participatory modelers need skills in identifying and selecting stakeholders (Reed et al., 2009, 2013a), and in building relationships and, consequently, credibility with stakeholders. These include strong listening skills accompanied by humility and patience. A good listener learns more than an expounder, and deep listening (not interrupting, allowing for people to express their ideas in their preferred way) and patience are critical in developing trusting relationships. Furthermore, the recognition that stakeholders’ expertise is equally valid to the modelers’ provides a solid foundation for mutual trust and respect, with the understanding that modelers are, themselves, another type of stakeholder (Barreteau et al., 2003; Zellner, 2008).

3. Facilitation skills to elicit effective stakeholder engagement: Effective stakeholder engagement includes ensuring stakeholders communicate with each other. Often it means managing conflict. The facilitation process (Kaner, 2014) elicits participants’ needs, priorities, and understanding of the system, and fosters empathy among people who may hold different worldviews (Zellner, 2008; Hovmand, 2014). This may require opportunities for participants to “step into other people’s shoes”. For example, we saw increased understanding of a complex system and empathy with other stakeholders when we coached stakeholder groups to discuss mental model representations by other groups and to brainstorm solutions that would be acceptable to these groups (Singer et al., 2017). Professional facilitation can be useful in such cases, but just as with “outside experts” with content knowledge, it may not always benefit the engagement process. Community facilitators can be effective partners in the participatory modeling team (Hovmand, 2014).

4. Ability to recognize, extract, and communicate a conceptual model from discussions with and among stakeholder groups: It is not a simple task to identify common themes from group discussions and translate them into a conceptual model of the system. It is equally difficult to make that conceptual model transparent and tractable in a way in which stakeholders recognize their inputs and gain intellectual ownership of the model. The modeling techniques need to fit the problem characteristics, and the model needs to be understandable and usable for the people who contributed to building it (Borshchev and Filippov, 2004; Martin and Schlüter, 2015). Modelers must strive to ensure their own views and favored methods do not drive the model development. As one example, an initial effort to develop an extension to a National Park in Thailand failed when modelers focused on using a spatial distribution model to demarcate boundaries. Applying different modeling techniques (including role-playing games and agent-based modeling) helped move negotiations to a more successful, innovative, and integrative discussion of solutions based on resource access and multi-functional space (Barnaud et al., 2013).

5. Ability to recognize how one’s own intuitions can serve as biases in the PM process: Reflective thinking and self-awareness can help us assess the limits of our expertise and recognize the subjectivity in our own knowledge and judgements (Glynn, 2017). Such awareness generally improves the outcomes of our decisions (Kruger and Dunning, 1999; Wu and Dunning, 2017). Reflective thinking can help us recognize our biases, beliefs, heuristics, and values that often drive our efforts and the decisions that we make (Voinov et al., 2014; Glynn et al., 2017). For example, one of the authors collaboratively developed a flooding model to help communities design green infrastructure scenarios, where the modelers had represented flooding as a specific depth of accumulated water (Zellner et al., In Press). In collaborative design workshops with stakeholders, participants noted that there were, in fact, different definitions of flooding, depending on who and what areas were affected. The modeling team then included in the PM interfaces an ability for users to define the depth of water that constituted flooding for them (Zellner, pers. obs.). Another author realized that her own bias towards an older, male participant in a leadership role caused her to interpret his emphasis on responsibility and single-point decision making as authoritarian and power hungry. However, in the context of the stakeholder group—a group of wildland firefighters—it was a sign of caution and a risk management approach. Without this realization, important insights would have been lacking from the PM process.

6. Ability to measure the success of the PM process: Building a model collectively is not the final goal of a PM process. Rather, the social learning that model development and use supports, and the actions that learning informs are most important. PM efforts must be evaluated, both formatively and summatively, and results shared (Hoch et al., 2015; Milz et al., 2017; Radinsky et al., 2017). Also, it is critical that modelers participate in measuring and communicating how PM helped communities to make better decisions.

Stakeholder groups are not monolithic; they have varying organizational structures, multiple and competing objectives, and they often evolve over time. Individual stakeholders cannot represent an entire group; stakeholders and stakeholder groups are dynamic and cross-scalar. Multiple manifestations of a “community” exist within any particular geographic region. Communities can be defined, for instance, by political, religious, genealogical/familial/clan, or by resource management oversight characteristics. Setting manageable boundaries for selection of groups to participate in a modeling initiative, and identifying in which stages different stakeholders should participate, is crucial (Prell et al., 2007; Reed et al., 2013a, 2013b, 2017). Some stakeholders need to be engaged from the very beginning (Reed et al., 2009; Cormier-Salem, 2014; Sterling et al., 2017). Yet, trying to engage every stakeholder may mean spending a lot of time reaching consensus but not creating a lot of clarity on action (Büscher and de Beer, 2011; Sterling et al., 2017). Experience, context, and clear stakeholder engagement strategies help to overcome these challenges (Sterling et al., 2017).

Navigating power asymmetries is a tricky ethical and logistical issue. On the one hand, when significant power differentials exist, inviting all stakeholders (a “neutral posture”) may simply exacerbate initial power asymmetries if less powerful participants are less able to contribute (Kritek, 2002). On the other hand, excluding or separating the more powerful participants (a “non-neutral posture”) to empower the weaker actors engenders questions regarding how people are chosen (Barnaud and Van Paassen, 2013) and potentially weakens the usefulness of the results of the PM process.

Understanding the social and political context within which one is operating is key. For instance, some stakeholders may have specific expectations about roles and relationships based on previous experiences. Others may have little or no experience in working with modelers on planning and decision-making, and may need extra assistance in learning how to participate effectively. Still others may have engagement fatigue. Governance structures differ around the world, and those structures and norms may change the real and perceived benefits of participating in a PM process. Some groups may be uncomfortable with group decision-making, preferring to defer to figures of authority. And, at times, groups are set on the decisions that they want to make, and, while they may be open to a collaborative engagement, they are resistant to new ideas. Our lessons from and about operating within a variety of social and political contexts include:

1. In some cultures, knowledge is not shared freely: Researchers from a western science perspective often assume that people share knowledge, and share it in the same way across cultures. In some cultures, knowledge is considered to be freely available to all. In other cultures, knowledge is held more closely, and may not be shared with others until they have earned the right to learn something (van Kerkhoff, 2013; Matsui, 2015). Knowledge ownership has implications for who participates in various aspects of the process, for how information circulates, what information is available for use in the modeling processes, and if and how that information is used. Knowledge that is not represented in the PM process has little to no chance of informing the resulting decisions. Explicit criteria for a culturally appropriate process of knowledge sharing and use can help ensure these concerns are considered (Conley and Moote, 2003), even if not represented directly in models. As an example, when working in the Pacific, researchers often ask participants to specify who should have access to knowledge used in the model. This could include restricting to those who provide the information and the modelers only, adding other knowledge holders in a particular domain (for instance other fisherpeople), adding other community members but no one beyond the community, and so on (Sterling pers. obs.).

2. Community co-leads are crucial partners in navigating social and political context: Frequently, it helps to identify a small group of advisors/co-leads from the focal community who can help you navigate the local social and political setting (Hovmand, 2014; Zellner et al., In Press). Selecting appropriate co-leads is a challenge of its own, and it is important to explore systematically who would best fill these roles, and who has appropriate knowledge and trust of the community. As an example, while working in Solomon Islands, one of the authors found that some people who have knowledge that might be of most interest to researchers may be reluctant to engage in a leadership role. In contrast, some of those who were interested in assuming that role in the process lacked the trust of the community (Sterling pers. obs.). It took deep knowledge of the community members and their relationships to determine whom to engage.

3. Economic or policy gains may not be the (only) benefits participants care about: Participatory modelers and conveners may assume that their goals for the process are shared by those who engage with them. More specifically, they may assume that reaching a policy decision, implementing an action, and delivering economic gains to the community are important to everyone. This is not universally the case. In our work, we have found that some participants may be guided more by social norms, such as wanting to fit in or enhance their stature in a group or a community. Others may be motivated by a desire to contribute in a positive way to another community or to the next generation or to leave behind a meaningful legacy. Some may desire to preserve the present or the past, to act against the potential disappearance of a culture or way of life, or against the potential extinction of a species, or against the degradation of an environment or ecosystem (Sterling et al., 2017). Some might participate to be at the table, gain first-hand information, or try to influence participants towards their point of view. Some participants may be motivated by all of these factors. It is important for participatory modelers to recognize that the PM process is embedded in this social and political context, and that focusing too narrowly on economic or policy questions may shortchange the stakeholder community.

4. Many people care deeply about their social standing, which can affect their participation: PM is a process that naturally engenders group dynamics. Group dynamics (such as “groupthink,” posturing and differentiation, attention seeking, and following a confident leader) can sometimes impede critical thinking and independent perspectives that are essential to improving knowledge and management of an issue (Forsyth, 2014). In situations where group dynamics are very strong and or are poorly managed, the social standing of individuals who think differently from the group may be threatened by speaking counter to the group, and they may reduce their participation. Alternatively, people might only share what they think the researcher or leader wants to know. For other individuals, social standing may be increased by being a strong voice in the group. Unmanaged concerns about social standing and power imbalances within a group may lead to a less representative process or model than intended (Hoch et al., 2015; Zellner et al., In Press).

5. Methods used, and the way in which they are used, can significantly influence how engaged stakeholders feel: Not everyone likes structure, or “being taught.” An over-reliance on capturing input in writing (e.g., on flip charts) and on categorization can hamper data collection in societies where storytelling, narrative, and oral information exchange is the norm. Similarly, sharing information using graphs and pie charts can be off-putting for some stakeholders while being very helpful for others. Visual depictions of future scenarios accompanied by vignettes or narratives can be a powerful tool for eliciting discussion, but may also lead people to think of the particular scenarios described as the most likely or only possible futures (Spiegelhalter et al., 2011). Open and active engagement in collaborative design is crucial to tailoring the PM into a process that the community owns, masters, legitimizes, and ultimately uses to guide their actions (Zellner et al., In Press). In Solomon Islands, for instance, use of art-based facilitation elicited responses from community members who might not otherwise have shared perspectives (McCarter et al., 2018).

6. Conscious and unconscious biases can affect processes, interactions, and models: PM processes can threaten existing social and political order, and the existing social and political order can threaten the integrity and usefulness of PM. Not everyone is open to learning that their preferred decision may fail. The reluctance can affect the models themselves: people may select data to fit their pre-existing mental model (Tversky and Kahneman, 1974) or cultural model or decision, or to seek confirmation of their hypotheses (Anderson et al., 1980; Nickerson, 1998; Sébastien and Bauler, 2013; Glynn et al., 2017). This reluctance can affect the acceptance and implementation of PM results: people may show more distrust when outputs do not match their expectations (Hoch et al., 2015). Many innate prioritizations unconsciously affect the way we make decisions, conduct science, and manage ourselves, our communities, and our environments and resources. Recognizing these prioritizations improves PM (Hämäläinen and Alaja, 2008; Glynn, 2014, 2017; Hämäläinen, 2015; Morewedge et al., 2015; Voinov et al., 2016; Glynn et al., 2017).

Managing the participation process in PM is as important as managing the model building process. Models within a collaborative process can function as “boundary objects” (Star and Griesemer, 1989; Harvey and Chrisman, 1998), providing a means to bridge ideas across disciplines and participants’ perspectives, and thus promoting learning through collaboration (Akkerman and Bakker, 2011; Zellner et al., 2012). PM processes help stakeholders bring to light assumptions, causes, solutions, and values of which they may have only subconscious awareness. This provides opportunities for users to make ideas visible and open for discussion, negotiation, and revision, and supports constructive discourse. PM models also allow individual and collective cognition to be externalized and made explicit, by mentally offloading difficult tasks into an environment (e.g., computer screens and notebooks) where thinking can be organized and discussed (Bart, 1995; Zellner, 2008). Furthermore, because modeling forces us to explicitly formalize diverse knowledges, ideally individuals coming from different backgrounds should be able to communicate in this shared workspace (van der Leeuw, 2004).

Robust, meaningful, and impactful PM is a huge effort, especially if compared to a more traditional top-down or authoritative decision-making process. It takes time, funding, and iterative engagement to build the relationships and trust among the participants, tools, and process (particularly how to navigate conflict and tradeoffs), in addition to building the models themselves. That said, once the iterative process is established, further collective innovation and action may come more easily and quickly. Participants become used to thinking collaboratively with modeling tools. Slowing down to engage in such processes paradoxically gets communities to where they want to be, faster (Zellner and Campbell, In Press).

Previous experiences with collaborative processes shape participant expectations of PM, however. Community trust in such processes can be eroded if conveners of a past collaborative process did not take their input seriously, or if members are afraid to voice their opinions, perspectives, and knowledge in public. Some processes become driven more by the modelers and by the needs of a quantitative model than the interests and needs of the community. This can establish an expectation that the modelers are “in charge” and lead to less innovation and participation from the community in model development and use.

Lessons we have learned about how to manage these issues include:

1. PM is not always the right approach: Evaluating the effectiveness of PM is inherently complex, and there is no universal agreement about its benefits. Perhaps especially because we are participatory modelers, it is crucial to consider explicitly whether the benefits of PM are relevant and sufficiently valuable for the problems being considered to be worth the time and effort PM requires. For example, in cases of emergency one might want to follow a top-down approach rather than having extensive discussions. If the social and political context will not allow true decision-making authority to go to a participatory group, then it might not be a wise use of time to engage in PM.

2. Participants’ needs, rights, and relationships must be respected and protected: Free, prior, and informed consent (FPIC) is a principle mandated by international human rights standards that is fundamental to ethical engagement (FAO, 2016). This principle ensures that there is no coercion to stakeholder engagement, that stakeholders have the opportunity to consent or deny engagement prior to commencement of activities, and that they are informed regarding the details of the engagement, including but not limited to scale, scope, purpose, implications, and potential impacts including risks. However, obtaining FPIC can be challenging. Sometimes the process of informed consent raises suspicion, making participants wary of signing documents and prompting questions about risks involved in participation. In the initial stages of collaboration, it is necessary to build trust and explain how informed consent means participants should have the freedom to act and speak without concern about what others outside of the process might think of them. It is important to be aware of power dynamics that favor consent and to allow for critical inquiry and dissent (Forsyth, 2014).

3. The most informative results arise from fearless and playful collaborative modeling: The first model is rarely the best model, and one of the underlying motivations for PM is the belief that we get better models, and better and more useful results by engaging the full knowledge of the community. It takes time to build the collaboration and confidence in model conclusions and model limitations (Zellner and Campbell, 2015). Confidence increases as understanding increases, and open, iterative modeling where all questions are explored with an open mind increases that understanding. Processes that create an environment where mistakes are acceptable and even desirable for learning and innovation help build confidence. Art and improvisation exercises can be especially useful to build and connect different pieces, overcome the fear of trial-and-error, create relationships among people who do not know each other well, or strengthen relationships among those who do not trust each other (Pink, 2006).

4. Stakeholder expectations may exceed the scope of the PM process: Communication is key among researchers and between researchers and other stakeholders and participants. It is particularly important to understand and communicate clearly about the process, what the outcomes will be, what products will result from the process, and how they will be shared or used. When stakeholders have expectations that cannot be met, they may disengage from the process, and may be more reluctant to engage in future collaborative processes. For instance, when working with herders in Kenya, Reid et al. (2016) suggested that external organizers be clear regarding their ability to meet a request. Especially important is addressing those situations where researchers do not have expertise and therefore need to reach out to others for relevant resources or to co-generate new knowledge. It is useful to iteratively state, revise, and refine expectations throughout the process, to ensure everyone knows what they are doing and why. This can help maintain a clear definition of the kind of outcomes that are expected, and increases the likelihood of acceptance of the outcomes.

5. Conflict management may be necessary: Processes that focus on positions in negotiations often mask the underlying needs people have but cannot articulate (Susskind et al., 1999). A good process will help people identify their motivations, which in some conflict situations might be related to feelings of being disrespected or undervalued, or to disparities in social power among participants (Sturm and Antonakis, 2015; Simpson et al., 2015). Listening to how stakeholders talk about an issue or learning about their relationships with each other can help identify other factors that may be driving disagreements, such as history, identity conflicts, or deep value differences (Madden and McQuinn, 2014). Respectful listening to, and acknowledgment of, opposing points of view can be helpful in resolving differences and conflicts and in fostering innovation, especially with power dynamics in mind (Barnaud and Van Paassen, 2013; Minter et al., 2014). Learning and trust-building, for both scientists and communities, is a critical outcome of collaborative processes, including PM (Susskind et al., 1999; Conley and Wondolleck and Yaffee, 2000; Moote, 2003; Gray et al., 2017).

6. Exploration and compromise may be more valuable than consensus: PM generally involves multiple sources of knowledge stemming from diverse knowledge systems, where differing opinions and proposed strategies are common. One knowledge system should not be used to validate or invalidate information from another system. Rather, credible, salient, and legitimate knowledge (as defined by Cash et al., 2003) from within a knowledge system can be brought to the table and the congruences and differences explored (Tengö et al., 2014). This often involves balancing rigor and flexibility in what constitutes “evidence” (Sterling et al., 2017). Rather than attempting to minimize differences when they arise, it is important to recognize how differences shed light on diverse parts of a system or on the varying tradeoffs across solutions (Zellner et al., In Press). Attempts to reach general agreement/consensus across differences can, at times, completely invalidate a set of values or knowledge. True differences can be overcome by focusing on mutually valued goals, or by working with these differences to assess what participants give up and how to compensate for it.

PM is a powerful approach for addressing complex social and environmental problems. While it holds great promise, it can also come at a high cost: it is difficult to gather adequate skills, funds, and participants’ time, and it takes time to gradually build strong relationships between scientists, community partners, and public agencies. It may require that actors with decision-making responsibilities cede some of their authority to a group and process over which they have little control. Based on the lessons presented above, we conclude our reflection with some cross-cutting recommendations.

Members of a project team should reflect on their abilities with regards to our lessons learned on the role of participatory modelers. All participatory modelers need to be aware of how to support relationship building among participants, and design and use tools accordingly. If critical skills are missing, the project team should first build its own capacity and may consider recruiting project partners with the needed expertise. Successful PM efforts create an inclusive environment that supports participants with differing values, ideas, and priorities. Sharing (time, viewpoints, stories, common challenges, food, small slices of life, etc.) helps to develop these relationships and understanding. Stakeholders know more about the problem, obstacles and opportunities, and the community than do modelers from outside the community. Provide time for people to understand the purpose of the modeling, but more importantly, for participants to share their wants and needs. Modelers are a support to them, to help them achieve their goals and aspirations. Find ways to “check in” regularly and in different ways to see how learning is progressing in individuals and across the collaboration, and how stakeholders are perceiving the collaboration and the issues in general (Bennett, 2016). This includes structured observations of the dialogue and deliberations occurring around the PM tools, and what actions participants have collectively agreed to support. Power dynamics frequently influence how people from different backgrounds interact. Try to understand the motivations underlying participants’ behavior, and explicitly acknowledge and legitimize those motivations. Note that researchers are all part of the process, not outsiders taking objective notes from afar, so modelers should also expose their motivations and reflect on their biases.

Whatever tools and techniques are used, make sure to work with a subset of the stakeholders to design the PM with them. Not only will the PM process (tools, location and time, facilitation and deliberation setups) thus be comfortable to users, but also the non-modeler participants become advocates and facilitators of these tools, building trust in the process, and easing the transition towards self-reliance and appropriate tool use to inform decision-making. Provide feedback to stakeholders on model development and, to groups who were not involved in its development, on prior model use and outcomes. Recruit and, if necessary, train different kinds of facilitators (technical/modeling, community) to help manage the social interactions among community participants, between modelers and participants, and between users and the tools developed and used (Hovmand, 2014).

Consider how to deal with dissonance between what participants initially expect and what the model suggests; if the gap is too wide, it can be difficult for participants to embrace the results and make effective decisions. This discomfort may be compounded by the fact that uncertainty is inherent in any complex problem that is the focus of the PM activity, and is often reflected in the modeling process and outcomes. Participants may equate such uncertainty to ignorance, and thus distrust the modeling effort and its insights, instead of incorporating the uncertainty into the design of robust policies that can be effective in an uncertain world (Zellner 2008). Scaffolding between expectations and results needs to be built into the PM process, in order to address confirmatory bias (Hoch et al., 2015; Zellner et al., In Press).

Keep models simple, relevant, and tractable, distilled to key decision points of importance to the participants (Zellner et al., In Press). The purpose of PM is for stakeholders to participate in the analysis of the complex issues they face, to gain insights relative to their roles in problems and in their solutions, to harness their collective creativity in designing solutions, and to examine and deliberate about the various tradeoffs with appropriate information. If models become intractable or irrelevant “black boxes” that do not adequately represent and make it easy for stakeholders to examine assumptions, goals, and values, then that possibility is lost.

Finally, PM involves the community in understanding and modeling what are typically complex human, social, and natural systems. Although a project may “wrap up,” achieving its decision-specific goals, the community and those complex systems continue to exist and evolve. A valuable result of PM would be to foster ongoing dialogue and collaborative analysis that is adaptive to the inevitable surprises brought about by complex problems and systems. Theoretical frameworks, case studies, and individual experiences with PM and other forms of collaborative governance are signaling what some have identified as a “new governance era” (e.g., Leong et al., 2011). Yet, these approaches are not fully integrated into agency policies and practices (Leong et al., 2011; Zellner and Campbell, 2015) despite notable international efforts and commitments such as the Local Agenda 21 coming out of the UN Rio Declaration on Environment and Development, the European Union’s (EU) Aarhus Convention on Access to Information, Public Participation in Decision-Making and Access to Justice in Environmental Matters, and the EU’s Water Framework Directive. Having overcome the initial hurdles of setting up a PM process, it is worth investing in collaborative stakeholder networks to facilitate ongoing learning and exchanges from peer to peer, outside of PM efforts. Long-term relationships with stakeholders can carry across many complex projects, transferring the ways of thinking across projects and building the capacity to effectively deal with such complexity in different cases.

Our hope is that these lessons are useful to new practitioners who venture on this path, with a realistic view of what PM entails, recommendations of steps to take to ensure a productive process, and the enthusiasm and confidence in this approach as a support for a new form of collaborative governance to grapple with difficult and persistent problems.

Co-authors of this paper are U.S. Government employees, therefore: Any use of trade, product, or firm names in this publication is for descriptive purposes only and does not imply endorsement by the U.S. Government. We thank Nadav Gazit for comments, technical edits, and graphic design, Don Rosenberry and Collin Lawrence for U.S. Geological Survey technical reviews, and Amanda Sigouin for editing support. We appreciate the helpful comments from two outside reviewers and the thoughtful guidance from the editors.

This work was supported by the National Socio-Environmental Synthesis Center (SESYNC) under funding received from the National Science Foundation DBI-1052875. The material is also based upon work supported by the National Science Foundation (NSF) under Grants No. EF-1427091, 1427453, and 1444184. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. In addition, this research was conducted by the Assessing Biocultural Indicators Working Group supported in part by SNAPP: Science for Nature and People Partnership, a collaboration of The Nature Conservancy, the Wildlife Conservation Society and the National Center for Ecological Analysis and Synthesis (NCEAS) at the University of California, Santa Barbara.

The authors have no competing interests to declare.

Contributed to conception and design: EJS, MZ, KEJ, KL; Contributed to acquisition of data: All authors; Contributed to analysis and interpretation of data: EJS, MZ, KEJ, KL, PDG; Drafted and/or revised the article: All authors; Approved the submitted version for publication: EJS, MZ.