Designing and evaluating analytic-deliberative engagement processes for natural resources management

The need to involve experts, scientists, stakeholders, Indigenous peoples and the general public in decision-making is widely accepted in the field of natural resource management (Chilvers and Kearnes, 2015; Conroy and Peterson, 2013a; Reed, 2008; Talley et al., 2016). This is particularly the case when issues involve technological complexity, risks and/or a diversity of multi-layered, complex and conflicting values, knowledge systems and worldviews (Webler and Tuler, 2018). Current approaches for participatory engagement processes in natural resources decision-making include mediation (Striegnitz, 2006), citizens juries (Fish et al., 2013), multi-criteria decision analysis (Marttunen et al., 2015; Mavrommati et al., 2017) and cooperative discourse (Renn, 2015). The broad premise behind these and other participatory approaches is that by providing a more comprehensive understanding of the diversity of values, knowledge and interests of different actors, the quality of political choices can be enhanced (Carr, 2015; Renn and Schweizer, 2012). Participatory engagement is also deemed effective in increasing trust, credibility, legitimacy and acceptance of the decision-making process (Chilvers and Kearnes, 2015) and can enhance implementation and compliance (Schenk et al., 2007).

One notable trend is the increasing prominence of participatory engagement methods that exploit the best of deliberation (i.e., process of discussion, debate and rational argumentation and negotiation between participants) and structured analysis (i.e., assessment of options based on best available empirical information and participants views as well as clear, reproducible and agreed upon procedures). Known as analytic-deliberative processes (Mavrommati et al., 2017; Rauschmayer and Wittmer, 2006; US NRC, 1996; Webler et al., 2014), these methods attempt, through interactions between the different actors involved, to integrate technical and expert-based knowledge within the framework of stakeholder and public engagement and deliberation. Analytic-deliberation was first introduced in the risk field by the United States National Research Council in its influential report Understanding Risk (US NRC, 1996), and then progressively applied in various sectors including health (Abelson et al., 2003), environmental, natural resources management (Mendoza and Martins, 2006; Mendoza and Prabhu, 2006; Sheppard, 2005), urban planning (Davies, 2001) and conflict resolution (Rauschmayer and Wittmer, 2006).

There are two main “normative schools of thought” influencing and shaping the discussions on stakeholder engagement: participatory and deliberative theories (Papadopoulos and Warin, 2007). Research for the former derives from the assumption that decision-making will always benefit from public involvement, notwithstanding its format. It mainly focuses on issues of accessibility and representation and on identifying obstacles to participation. In contrast, deliberative theory grants more importance to the process where participants’ views and beliefs evolve through debate and interaction. Deliberationists favour questions related to the qualitative components of discussions that precedes decisions. The concepts of participation and deliberation are not mutually inclusive, meaning that deliberative processes are not necessarily participatory, and vice versa. For instance, a deliberative process can be carried out behind closed doors with the involvement of a limited number of decision-makers and/or experts. However, complex environmental problems, because of their associated risks, uncertainties and complexity, often require the input from, and deliberation among, various actors such as scientists, policy-makers, stakeholders, Indigenous peoples and the public (Renn, 2004). We therefore mainly focus here on engagement processes that combine participation and analytic-deliberation.

Research providing insights into the design of effective analytic-deliberative engagement processes originate from many strands of the natural resources management literature, including strategic and cumulative environmental assessment (Noble et al., 2014), strategic land use planning (Gunton, 2017), risk decision-making (Webler and Tuler, 2018; Webler et al., 2014) and social acceptance of renewable energy (Cleland et al., 2016; Devine Wright, 2010; Fast et al., 2016). Nevertheless, the question of what constitutes successful participatory engagement, let alone successful participatory analytic-deliberative processes, remains contested in the literature: “despite hundreds of books and articles on the subject and countless ventures into the participatory arena, we still know precious little about what kinds of participatory programs are likely to be successful in any given situation, or even what success really means and how it might be measured” (Kasperson, 2006). In response, theoretical frameworks and guidelines have been proposed to inform the design and evaluation of participatory engagement process in natural resources management (e.g., Reed, 2008; Wittmer et al., 2006). Yet while participatory engagement is increasingly applied in practice, practitioners often fail to meaningfully engage or even consider these theoretical frameworks and guidelines (Webler and Tuler, 2018). Such oversight can lead to situations where principles of good practice are violated, thereby generating suboptimal or even detrimental outcomes (e.g., worsen conflict, reduce legitimacy, delay decision-making; National Research Council, 2008).

A proposed explanation for this observed discrepancy between theory and practice is that frameworks and guidelines are overly complex and inflexible, making them inaccessible for natural resources managers and practitioners to apply in practice (Talley et al., 2016). Starting from this premise, we address two specific objectives. First, building on the literature indicated above, we propose a simple yet comprehensive framework for the design and evaluation of analytic-deliberative processes. Second, we trial and evaluate this framework in the context of an analytic-deliberative engagement process involving stakeholders and Indigenous peoples across the Canadian province of British Columbia on topics relating to forest carbon mitigation. Drawing insights from this empirical case study, we provide recommendations for the design and implementation of similar analytic-deliberative engagement process for natural resources decision-making.

The design and assessment of analytic-deliberative processes is complex since it brings together technical and scientific knowledge (analytic) as well as the type and extent of the interactions between participants (deliberative). While there exist disagreements about their design and evaluation, recent scholarship has converged on the importance of recurrent criteria (Table S1 contains an overview of eight such attempts). We group these criteria here into four broad categories: (1) representation, (2) deliberation, (3) knowledge and analysis, and (4) outcomes.

Representation addresses issues associated with access to the process and the degree of representativeness of the participant pool, referring to “getting the right participation” (US NRC, 1996). Fair representation is perceived as a prerequisite not only for effective democracy, but also for coherent decision-making, especially when perceptions and environmental values vary greatly between groups, as is the case in natural resources management. However, while there is clear consensus on the need to achieve fair and representative stakeholders and public involvement, what is often observed in practice is that “stakeholder involvement programs do much better on ‘interested’ than ‘affected’ parties” (Kasperson, 2006). This criticism originates from the fact that “the inarticulate, the forgotten, and the marginal peoples” are in many cases excluded from deliberative processes, often because they do not know yet how and to what extent they are affected by a project or because they lack the political or economic capabilities to participate effectively (Kasperson, 2006).

Decisions about who should be involved in an engagement process should be done early and carefully by applying systematic and reflective recruitment procedures such as stakeholder analysis (Talley et al., 2016). That said, not all analytic-deliberative processes require the same degree of participant representativeness (Conroy and Peterson, 2013b; Gregory et al., 2012). Insights from the risk management literature suggest that the objective, nature and complexity of the management problem can inform who should be involved in a participatory engagement process (Aven and Renn, 2010; Renn, 2015). In general, simple problems with low uncertainties, no controversial value and clear consequences only require the involvement of decision-makers. In contrast, highly complex or ambiguous problems that involve normative values (e.g., human health, environmental protection, ethics) and/or where uncertainties cannot be reduced sufficiently should not be solved by decision-makers and technical experts alone, but should also consider stakeholders, Indigenous peoples and the public’s concerns and perceived trade-offs.

The second category comprises all the elements related to deliberation, including its rules, legitimacy and fairness, the capacity of the participants to frame how the process is carried out, and the social dynamics and interactions between the participants. When carried out effectively, deliberative activities are expected to facilitate respectful and constructive exchanges between participants. In some contexts, however, the types and nature of the interactions between participants, organizers and experts taking parts in analytic-deliberative activities can lead to power inequalities (Webler and Tuler, 2018). In particular, debates that are directed in a top-down fashion without possible (or very few and superficial) amendments from participants restrict the capacity of participants to fully contribute to, and influence deliberation (Irwin, 2001). Power structures can be further exacerbated by strategic and manipulative behaviours (Chilvers, 2007). Bickerstaff and Walker (2005) outline three main classes of strategies that, when (sometimes unintentionally) used by participants, enhance power inequalities: (1) teleological action, when participants employ argumentation or other similar tactics to convince or gain the upper hand; (2) normatively regulated behaviour, when participants use values or norms shared by some participants to gain more power (e.g., through regrouping and working together); and (3) dramaturgical behaviour, when participants hide part of their position or objective to reduce or avoid negative perceptions by other participants.

In practice, deliberation is often exercised with the aim to reach consensus on the best option or policy in order to provide decision-makers with a justification for their decision. Under such circumstances, deliberation represents a dispute resolution approach that focuses on “‘solving’ a problem, finding the ‘right answer’, or making decisions about a problem” (Mendoza and Prabhu, 2005; see also Chilvers and Kearnes, 2015; Talley et al., 2016). This approach, referred to as “closing down”, can limit the richness, originality, representativeness and diversity of discussions on alternatives and values, focusing only on a few options where consensus can be struck and ignoring minority views. In contrast, various authors recommend ensuring that deliberation is “as open a process as possible, avoiding premature closure of particular options or framings, while striking a balance with providing enough information to make meaningful engagement possible” (Chilvers and Kearnes, 2015; Pidgeon et al., 2014, see also Bellamy et al., 2016; Blue, 2015; Chilvers, 2007; Stirling, 2008). Opening up deliberation emphasizes the diversity of values, objectives and interests rather than consensus building. While closure is at some point necessary in order to make a decision, an open debate and framing potentially leads to “‘plural and conditional’ policy advice” and thereby more progressive decision-making (Stirling, 2008). Such a strategy is sometimes more realistic and offers a more democratic, productive, transparent and accountable process where all stakeholders’ values and perspectives are considered (Gregory et al., 2012).

Knowledge and analysis refers to what type of information and knowledge is considered (“getting the right science”) and how analysis is carried out (“getting the science right”; US NRC, 1996). There is no doubt that any analytic-deliberative process should be based on the best available scientific and technical knowledge. However, focusing only on technical knowledge overlooks other important expertise and does not necessarily lead to better environmental practices (Jasanoff, 2010). Engagement processes are sometimes criticized in the literature for not successfully grasping the complexity and diversity of existing knowledge (Webler and Tuler, 2018). This is due in part because some practitioners, decision-makers and experts are still not convinced by the utility and relevance of considering lay knowledge or expertise (Chilvers and Kearnes, 2015; Kahan et al., 2012; Petts and Brooks, 2006), which can lead to situation where the public is consulted, but cannot substantially review and criticize decisions or influence policy. There is thus a need for the scientific community to reflect on their work and take a new approach to knowledge and analysis “without merely perpetuating the lay-expert dichotomy” (Eden, 1998). Instead, all forms of knowledge, values and worldviews should be taken into account (Aven and Renn, 2010).

Analysis within analytic-deliberative processes can have very different functions and formats, ranging from a focus on a single evaluation criterion to the consideration of several objectives, indicators or conflicting scientific conclusions (Rauschmayer and Wittmer, 2006). There are generally two main approaches to identifying decision criteria: (1) value-focused, and (2) alternative-focused (Keeney, 1992). Most of the time, criteria elicitation in analytic-deliberative processes is alternative-driven, which puts the emphasis on potential alternatives and then derives criteria allowing for their evaluation. While such approach might be most effective for reaching rapid consensus, it also limits deliberation, closing down and “anchor[ing] the thought process, stifling creativity and innovation” (Ibid, p. 48). In contrast, under value-focused thinking, stakeholders are asked to identify what really matters for them in the decision-making-process prior to looking at the alternatives being evaluated. Many authors argue that, when possible (i.e., if not limited by time or resource constraints), value-focused thinking should be prioritized (Belton and Stewart, 2002; Gregory et al., 2012; Talley et al., 2016; Webler et al., 2014). Such an approach expends deliberation beyond a limited focus on technical issues, creates a positive environment for deliberation and dialogue on disagreements, facilitates subsequent value trade-offs and ensures that all the needed information, knowledge and consequences of the proposed alternatives are taken into consideration. Furthermore, focusing on values at the outset can help demonstrate to participants that their disagreements, values and objectives are not as fundamentally mismatched as believed, thereby reducing conflict.

Traditional multi-criteria decision analysis typically involves complex techniques with highly sophisticated mathematical calculations and algorithms (Kumar et al., 2017). While such approaches are suitable in circumstances requiring specialized analysis, in the context of analytic-deliberation they are frequently characterized as too complicated, “rigid and highly algorithmic” (Mendoza and Martins, 2006), “technocratic” and as “a ‘black box’ to the public” (Sheppard, 2005). Complex quantitative methods to elicit preferences can lead to participants’ frustration and disengagement, especially with “local communities and stakeholders who often are not familiar with, or do not have the experience and technical knowledge concerning, models” (Mendoza and Prabhu, 2006).

Research shows both the importance and challenges related to evaluating the outcomes of deliberative processes (Bickerstaff and Walker, 2005; Talley et al., 2016), including the legitimacy and accountability of decision-making (as opposed to the deliberative process) and the capacity and effectiveness of the process and participants in influencing final decisions and generating improved policies.

Fiorino (1990) identifies three main arguments to justify why the public should be involved in defining and evaluating environmental risks. First, the substantive argument is that the general public generates a valuable contribution by considering values and objectives that may otherwise be overlooked by experts. Second, the normative argument is that citizens should have the chance to participate and influence decisions on issues that may affect themselves and their community. Third, the instrumental argument is that public participation can lead to better reasoned, rational and publicly acceptable decisions, thereby ensuring greater public support for environmental programs. For instance, community engagement is crucial for the energy sector where changes in technologies with lower carbon footprints largely depend on public acceptability (Kasperson and Ram, 2012; Spence and Pidgeon, 2009). Analytic-deliberative processes, because they typically consider multiple criteria, also have the potential to identify policy solutions that are both cost-effective and less likely to engender decision failure costs (i.e., costs associated with sub-optimal decisions; Rauschmayer and Wittmer, 2006).

Yet the implementation of analytic-deliberative and other engagement processes can become cost intensive and generate time delays, particularly when they require the engagement of multiple stakeholders, public and experts. Because managerial concerns about process costs and timeliness can act as deterrent for decision-makers to initiate engagement processes (Moynihan, 2003), the costs associated with different methods should be considered in light of the potential costs associated with the problem at stake.

While analytic-deliberative should ultimately enhance policy outcomes, allow the public and stakeholders to influence decision-making or produce plans and/or reports that reflect participants’ values, criticisms indicate that they sometimes fail to do so (Webler and Tuler, 2018). These unfulfilled promises are at times the results of managers, decision-makers or bureaucrats perceiving engagement processes as “outreach efforts” (Kasperson, 2006) and “symbolic politics” (Papadopoulos and Warin, 2007) that provide legitimacy to technical decisions already agreed upon by decision-makers (Chilvers and Kearnes, 2015). Because of this figurative status, engagement efforts can be carried out at the end of the decision-making process rather than at the outset – too late and marginal to have any significant impact on the resulting decisions.

Within the outcomes category, we further identify the importance of social learning as it relates to public engagement (see Rodela et al. (2012) for a review). It is believed that contexts such as analytic-deliberation—that allows for open discussions and exchange between various participants with differing perspectives and worldviews—provide the greatest opportunities for social learning (Huitema et al., 2010). However, the concept of social learning is also criticized. For instance, Crona and Parker (2012) point out that there is no consensus on its definition, its process and how it is evaluated, no clarity on how both social interaction and social environment influence learning, and that little attention is given on how learning is impacted by power and conflict dynamics. For clarity we define social learning here as “a change in understanding that goes beyond the individual to become situated within wider social units or communities of practice through social interactions between actors within social networks” (Reed et al., 2010). Huitema et al. (2010) further identify three levels of social learning: (1) relational (i.e., improved trust in, and understanding of, others), cognitive (i.e., enhanced technical/factual knowledge) and normative (i.e., change in norms, values and belief systems) learning.

The potential for analytic-deliberative approaches to contribute to improving forest management decisions is significant (Acosta and Corral, 2017; Mårald et al., 2015), particularly since conventional methods employed in forest-related decision-making face various limitations (Sheppard, 2005; Sheppard and Meitner, 2005). Notably, public consultation efforts sometimes fail to provide participants with useful and trusted information and knowledge (Tindall et al., 2010). These processes have also been shown to discourage discussion and fail to provide a suitable, safe and respectful setting for the public to present their concerns and perceptions (Scott, 2011). For instance, participants of a land use and natural resource planning in western Canada explained that such processes were “particularly hostile and divisive, often breaking down in to shouting matches” (Booth and Halseth, 2011). Conventional consultation processes are also criticized for being overly technical, for instance by focusing on complicated concepts without providing the necessary background as a basis of understanding, or by using overly complex rating systems (Mendoza and Martins, 2006; Sheppard, 2005). The prominent role played by scientific and technical knowledge, which is often “privileged at the management table and for both structural and cultural reasons,” can result in traditional and lay knowledge being disregarded, underrepresented and devalued (Hamersley Chambers and Beckley, 2003).

In the past two decades, the role of forests for mitigating climate change has become a major focus for international organizations (FAO 2016; IPCC, 2014), and both national and regional science and governance bodies. A central, policy-relevant aspect of the technical complexity referred to above concerns forest carbon science and accounting. Most often however, discussion and consideration of carbon-accounting has been reserved for scientific experts (Pearse and Böhm, 2015). Considering the shortcomings of conventional public consultation in forest management, there is a need for participatory engagement processes on forest carbon mitigation that (1) are structured to enable the systematic appraisal and analysis of multiple perspectives and trade-offs (Ananda and Herath, 2009), (2) allow for deliberation between participants (Gregory et al., 2012), and (3) combine technical and scientific information with public knowledge and values (Mendoza and Prabhu, 2006; Sheppard and Meitner, 2005). Analytic-deliberative methods, when well designed and implemented, offer such opportunities.

The Canadian province of British Columbia’s 55 million hectares of forests, 95% of which is publicly owned, offer important climate change mitigation potential (BC MFLNRO, 2013). However, since 2003 British Columbia’s forests have emitted more greenhouse gases than they have sequestered (i.e., they are a net source of carbon) (BC MOE, 2016), mainly because of increased wildfires and insect outbreaks (Kurz et al., 2008). Until 2016, British Columbia still lacked a set of concrete forest carbon management policies (Hoberg et al., 2016). In 2016 and even more so since then the government has begun to invest in efforts to increase forest carbon sequestration and reduce emissions, for instance with the Forest Carbon Initiative (Government of BC, 2017) and the Forest Enhancement Society (FESBC, 2018). Given the province’s plans to invest in forest carbon mitigation on public land in the coming few years (BC MRLNRO, 2016), there is a need to understand how the public and stakeholders perceive different mitigation options.

The engagement process trialled and evaluated in this study draws on two specific analytic-deliberative methods. The first method, multi-criteria decision analysis (MCDA), typically involves the assessment of diverse alternatives with evaluation criteria and preference elicitation (Kumar et al., 2017; Marttunen et al., 2015; Mavrommati et al., 2017). Belton and Stewart (2002) define MCDA as “an umbrella term to describe a collection of formal approaches which seek to take explicit account of multiple criteria in helping individuals or groups explore decisions that matter”. The second method is “structured decision making” (SDM), which “combines analytical methods drawn from decision analysis and applied ecology with insights into human judgment and behaviour from cognitive psychology, group dynamics, and negotiation theory and practice” (Gregory et al., 2012). The steps of SDM are very similar to MCDA. However, SDM mainly emphasizes on structuring the problem analysis and decision-making, whereas MCDA often focuses on identifying a prescriptive solution that relies on complex and rigid quantitative assessment methodologies (e.g., analytic hierarchy process, outranking models; Kumar et al., 2017).

The engagement process was designed to allow stakeholders and Indigenous peoples to disclose their relative preference for, and perceived acceptability, credibility and effectiveness of, mitigation alternatives for British Columbia’s forests. The engagement process included two series of workshops (Table 1), conducted in four locations (two each in British Columbia’s coastal and interior forest regions). Prior to engagement, a stakeholder analysis was carried out to ensure that concerned parties were invited (Billgren and Holmén, 2008; Colvin et al., 2016; Reed et al., 2009). Prior to both series of workshops, participants were provided with illustrated documents providing relevant technical information (e.g., overview of forest carbon balance and potential mitigation opportunities, description of the evaluated strategies; Text S1 and S2).

The first series of workshops aimed at identifying key objectives to be considered when evaluating forest carbon mitigation strategies in British Columbia. Five 3.5-hour workshops were held (two in the same location because of higher demand) with a total of 76 participants (Table 1). Participants were asked to identify what they viewed as the most important objectives (defined as “what really matters”; see Gregory et al., 2012) through small group and plenary deliberation. A final aggregated list of 11 objectives combining results from all the workshops was generated by the research team.

The goal of the 2^nd series of workshops was to elicit the preferences of participants in relation to six predefined mitigation strategies as measured against the objectives identified in the first series of workshops. Four full day participatory workshops with a total of 72 participants (44 of which also participated in the first series) were held in the same four locations previously visited (Table 1). During the workshop, participants evaluated each of the six strategies against the objective through rating using an 11-point scale (from “very good” to “very poor”) and subsequently discussed their results in plenary discussion. The group discussions were moderated and noted by team members at each table. Five of the mitigation strategies were selected from a parallel study that modelled the climate change mitigation potential and financial and socioeconomic impacts of the strategies for the years 2017–2050 (Xu et al., 2018). Information about the strategies and the modelling results was provided to participants prior to workshops (Text S2). At least one co-author of this parallel study attended each workshop as an expert to present the strategies and answer any questions. A sixth strategy, forest rehabilitation, was also later identified by the research team and included because of its importance in the BC government’s proposed climate change policy. Unfortunately, the decision to include the sixth strategy was made after completion of the parallel study and no modelling result was available.

A comprehensive analysis of the results of the engagement process are presented elsewhere (Peterson St-Laurent et al., 2018). Here we briefly highlight the main outcomes of the evaluation of the different forest carbon mitigation strategies carried out by stakeholders and Indigenous peoples during the process.

The final list of 11 objectives identified by participants during the first series of workshops was grouped under four categories: (1) biophysical, (2) procedural, (3) social, and (4) economic. On average, participants ascribed a higher level of importance to biophysical and procedural objectives than to economic and social objectives (Figure S1). The evaluation of the six forest carbon mitigation strategies indicated that conservation-focused strategies (old growth conservation and reduced harvest) generally performed better against biophysical objectives and in the coastal region, whereas improved forest management strategies (e.g., bioenergy and higher utilization) scored higher against economic and social objectives and in BC’s interior (Table S2). Conserving old growth forests and rehabilitation obtained the highest aggregated score (i.e., average score of the strategies on performance against each of the objectives) in both regions. All strategies received positive average levels of support (+60%) in post-workshop individual surveys (Figure S2), indicating widespread willingness to consider new forest carbon management strategies.

These quantitative results were supported by important arguments made during small group and plenary discussions, which highlighted participants’ concerns as well as the diversity of perspectives and values on the potential role of forests in mitigating climate change as well as the different mitigation strategies. The main issues discussed were associated with scientific uncertainties (e.g., reservations about climate modelling and forest carbon accounting), the potential impacts of climate change (e.g., increase in natural disturbances) and the current capacity (or lack thereof) of the forest industry in British Columbia. Overall, certain groups of stakeholders (e.g., NGOs, carbon offset companies) firmly indicated that old growth forests should be conserved, and that this represents the best forest carbon mitigation strategies, whereas others (e.g., forest industry, bioenergy companies) would prefer a more intensive management of BC’s forests involving harvesting and the use of harvested wood products. However, notwithstanding apparent differences in preferences and priorities, results from the engagement process show that compromises (e.g., flexible and geographically-differentiated implementation of a mix of mitigation strategies) are possible.

We collected three types of data during both series of workshops to inform the evaluation of the analytic-deliberative process (Table 1): (1) post-engagement evaluative survey (2) voluntary verbal or email comments from participants, and (3) observation of the process by researchers at the workshops. First, workshop participants completed a survey at the end of each workshop during both series. The surveys comprised 18 questions designed to inquire on participants general opinion on the process and quantify the performance of the workshops against the evaluation criteria. We used an additional follow-up online survey after the first series of workshops to inquire about participants’ overall level of agreement with the aggregated list of objectives generated by our team. The survey questions were mainly used as self-reflection tools to evaluate the engagement process. When appropriate we report below participants answers to specific survey questions. Second, participants were encouraged to provide comments regarding their experience. These comments were noted by the moderators throughout (during plenary and group discussions, coffee break) and after the workshops. Some participants also provided comments by email. Third, researchers and moderators observed and actively interacted with participants during small group discussions, during coffee breaks and at the end of the workshops. Each researcher and moderator took notes that were used to supplement quantitative date collected with surveys and help identify recurrent issues discussed by participants (e.g., uncertainties associated with modelling results, preference for certain strategies).

In the next sections, we use the four evaluation criteria outlined in section 2 to evaluate the analytic-deliberative process for forest carbon management that we carried out in British Columbia. Based on this evaluation we provide recommendations in the form of a framework that can be used for the design, implementation and evaluation of similar processes for natural resource management decisions (Figure 1).

Because of the highly technical nature of forest carbon management, and to allow for effective deliberation, we decided to limit the number of participants per workshop to less than 25, and thus purposefully restrained participation to specific stakeholder groups and Indigenous peoples. That being said, we strongly believe that a comprehensive public engagement process, including the broader public, should be conducted to inform future forest carbon management policy. In response to the common criticism that “contentious groups” that might prevent consensus are often “purposefully avoided” in participatory processes (Hamersley Chambers and Beckley, 2003), we invited representatives with diverse and diverging perspectives (e.g., representatives of environmental NGOs and forest industry). Yet despite our efforts to ensure broad representation, a majority of participants during the first (63%) and second (78%) series of workshops indicated in the post-workshop survey that some important groups were missing, most often the general public (e.g., citizens, teachers) and Indigenous peoples.

Figure 2 shows over-representation of some groups (i.e., NGOs, forest professional and forest industry) at certain workshops. While we attempted to avoid this situation by inviting an equal number of individuals from each group identified during the stakeholder analysis, the availability and interest to participate fluctuated greatly by region and sector. To offset potential inequality in representation we separated participants into smaller groups with only one representative of each sector, thereby preventing over-representation during discussions and ensuring that each participant had equal chance to contribute. since the workshops were not designed around voting, having more participation from one group (e.g., NGO) did not mean that other views (e.g., bioenergy companies) were comparatively less important.

We also had low or no participation of Indigenous peoples in some regions, pointing to another limitation of our study. Cullen et al. (2010) explain that a single engagement process that seeks to group all interested parties together often leads to poor engagement of Indigenous peoples who have special legal rights and title to the land and do not perceive themselves as simply another stakeholder. As a consequence, a new engagement model – a two tier stakeholder process – involving a discussion table with both stakeholders and Indigenous people and another one involving government-to-government negotiation (i.e., Indigenous people and provincial government) is being explored in British Columbia (ibid). In this context, a separate process involving only different groups/communities of Indigenous peoples would have been better adapted to their special circumstances, thereby potentially generating more interest and participation. We considered such approach when designing the process, but resource limitations did not allow for two separate processes.

At the beginning of every workshop, we informed participants of the format and goals of the engagement process. Moderators then asked participants to reach agreement on the set of rules that would govern the discussion and whether they wanted to modify the format of the workshops. At every stage of the process, we encouraged participants to provide feedback and share any apprehensions. Results from the survey (Figure 3) indicate that most respondents deemed that the deliberation was respectful, effective and inclusive of all points of views. Various respondents noted that the format (i.e., alternation between small group and plenary discussions) allowed participants to present and defend their points of view. Comments from participants also indicated that neutral moderation during small group discussions was effective at “opening up” deliberation.

We avoided a rigid and narrow framing in effort to foster open deliberation during discussion. In order to open-up deliberation, interactions during the workshops concentrated on exploring agreements and divergences between participants’ opinions. While participants also quantitatively assessed the mitigation strategies against the objectives during small group discussion, moderators encouraged each participant to explain and justify their positions. Scoring of the strategies’ performance was mainly used to stimulate discussions. When participants could not reach agreement, they were allowed to provide individual scores or to forgo the quantitative assessment. By focusing on deliberation rather than quantitative assessment, we prioritized participants’ identification and in-depth understanding (and sometimes transformation) of their values, and how they relate to the issue under consideration and to other participants’ views.

A limitation of our engagement process was the lack of flexibility associated with the six forest mitigation strategies that were evaluated during the process. Best practices recommend that participants identify and/or influence the selection of the different options evaluated during analytic-deliberative processes (Gregory et al., 2012). Our reliance on the modelling results from a parallel study did not allow for the participants to modify and/or recommend other mitigation strategies, which may have limited their capacity to explore diverse scenarios. While participants qualified the strategies evaluated during the workshops as realistic and diverse, they also pointed out that the strategies were somewhat conservative and failed to provide a full range of options for implementation (i.e., from small to large shift from business as usual). For instance, one of the strategies, “reduced harvest,” involved a reduction in annual harvest volumes of only 2%. Analysis of a greater range of reductions (2% to 30%) could have provided more information and flexibility to the participants in their evaluation of the different strategies. Furthermore, the modelling results provided to the participants during the second series of workshops only addressed six biophysical and socioeconomic indicators (i.e., greenhouse gas emissions, economic and employment implications). Participants may have had more informed and diverse discussions of the strategies and their expected performances if we had provided them with data for more objectives of interest (e.g., impact of strategies on biodiversity, old growth forests, tourism and water quality).

A majority of participants agreed that the information and knowledge provided was helpful (Figure 3). Yet a relatively large proportion of people did not agree or strongly agree that the information and knowledge provided helped generate new ideas (both series of workshops) or stimulate discussion (1^st series of workshops). This finding, combined with observations, comments and discussions during the workshops, also indicated hesitation and sometimes distrust towards the information provided, or a feeling that the information was incomplete, particularly the data originating from the biophysical and economic modelling. Participants often tended to emphasize values-based judgements instead of the information provided, leading to conclusions during group evaluation that (sometimes) diverged from the modelling projections. This observation was most noticeable during the evaluation of controversial strategies (i.e., old growth conservation) on which participants already had clearly formed opinions.

This result underscores two challenges of engagement processes more broadly. First, forest management is a controversial subject that involves multiple actors with diverging agendas (Cashore et al., 2001; Pralle, 2006). Since participatory processes are not immune from interest-based manipulation (Chilvers, 2007), particularly when problems at stake are divisive, it is possible that some participants may try to advance political agendas.

Second, while many participants commented that they appreciated the analysis-based information, they did not necessarily take it into account when they made judgements on different policy options. Other studies have similarly shown that the provision of scientific information, even if clear and simplified, does not eliminate intractability, public division and ideologically motivated reasoning (Kahan, 2012; Parkhurst, 2016). Rather than scientific illiteracy and incomprehension, it is often the “cultural meanings that divide citizens of opposing world-views” (Kahan et al., 2012).

While computational models are increasingly used to inform environmental policy and decision-making, they still face distrust from the public and even decision-makers because they are viewed as back boxes that are complex, uncertain, rigid and difficult to understand, validate or communicate (Gilbert et al., 2018; Winickoff et al., 2015). In a recent piece, Pfenninger (2017) highlights the important effect of framing in increasing public trust in scientists and the models they develop, urging modelers to be more transparent and open. Furthermore, forest carbon management is an extremely complex issue with several uncertainties that warrants caution (Newell et al., 2013), including issues of non-permanence (i.e., sequestered forest carbon is compromised by logging or natural disturbances; Parker et al., 2014) and leakage (i.e., displacement of the mitigated emissions outside of a jurisdiction or project boundaries; Murray et al., 2006). These uncertainties, combined with the already contentious status of forest management in the province, could have led some participants to display scepticism and doubts about the information provided. An earlier and more hands-on involvement of participants in the modelling assumptions could have increased trust and acceptability of the technical data.

All participants at the workshops were considered experts in their respective field(s) (e.g., conservation, forestry, forest carbon offsets, tourisms) and brought valuable experience and perspective. In this context, we kept technical presentations as concise as possible, and emphasized the expertise and worldviews of participants. To allow the inclusion of other forms of knowledge, a large amount of time during small group and plenary discussions was devoted to dialogues where participants were encouraged to share their personal knowledge and worldviews, ask questions, provide comments on the computational modelling methodology, assumptions and results and suggest other sources of information.

At least one forest carbon management expert attended each workshop during the 2^nd series to present the results from the technical modelling and answer questions. These experts were independent from, and had no role in planning the engagement process, but were members of the larger research project. These technical experts were available to answer questions, but they did not participate directly in the process and did not explicitly seek to influence deliberation.

The analysis (i.e., strategy evaluation) was structured around clear criteria (i.e., objectives) identified by the participants. As a consequence, we decided to use a simple rating technique that is intuitive, straightforward, comprehensible and that encourages deliberation and emphasizes structuring the process rather than seeking quantifiably-supported solutions. The point here is not to argue against the use of technical and quantitative approaches, which are very useful in some context (e.g., technical decision with few experts and decision makers). However, as previously discussed, participatory processes should be designed in a transparent and iterative manner enabling participation and continuous feedback between all phases, which is unlikely with highly technical, inflexible and arithmetical approaches.

The limited capacity of engagement processes to influence technological choice and decision-making over environmental issues is often criticized in the literature (Stilgoe et al., 2013). Since our engagement process was not led by decision-makers (i.e., government) one could reasonably argue that it will have no significant impact. However, reports from the results were provided to government entities (e.g., the provincial Ministry of Forests, Lands, Natural Resource Operations and Rural Development) and representatives of the government attended certain workshops as neutral observers. So while the engagement process arguably did not have a direct impact on decision-making, it may influence decision-makers as they design and implement policies.

Because of rapidly evolving technical and ecological conditions—including the increasingly noticeable impacts of climate change—decision-making over forest carbon management and other complex natural resources management problems should be responsive to stakeholder and public values and evolving circumstances. Building on the concept of responsiveness, analytic-deliberative processes should aim to produce outcomes and policy recommendations that recognize knowledge and technical insufficiencies and accommodate new circumstances as they arise (Pellizzoni, 2004; Stilgoe et al., 2013).

Even if consensus was not sought after, a large majority of respondents agreed with the result of the workshops and believed that participation helped foster agreement (Figure 3), suggesting that our approach focused on opening-up deliberation rather than requiring consensus around a policy solution can still generate a substantial degree of agreement around complex environmental issues. In terms of cost, the methodology was designed and implemented based on limited resources. The success of the process in gathering diverse groups of stakeholders and Indigenous peoples, and in having them reach substantial consensus, confirms the cost-effectiveness of the method employed.

A majority of respondents agreed that their participation allowed them to foster new and enhance existing relationships as well as to better understand other participants’ values and perspectives (Figure 3; relational learning). In the controversial and historically divisive context of forest management in British Columbia, building relationships, familiarity and mutual understanding of both agreements and disagreements can help participants begin to collaborate (see Baird et al., 2014). Furthermore, we designed the format of our workshops (small, diverse groups of actors) with the goal of having individuals from different sectors who are sometimes ‘adversaries’ in their day-to-day interactions become familiar with one another. While such approach may not always lead to consensus, a better awareness and understanding between participants can lead to more productive interactions.

Most participants indicated, particularly during the 2^nd series of workshops, that they learned about forest carbon management, the potential impacts of climate change on forests and how forests can contribute to greenhouse gases mitigation (Figure 4; cognitive learning). Discussions with participants allowed us to confirm two main sources of learning: 1) technical information provided prior and during the workshops, and 2) discussions with other participants. The hope is that this new shared knowledge “becomes situated within wider social units or communities of practice within society” (Reed et al., 2010), allowing for enhanced participation of stakeholders and Indigenous peoples in future policy-making processes.

As much as 76% of respondents indicated a change in levels of support (i.e., either more or less supportive) for at least one of the six forest mitigation strategies after participation in the 2^nd series of workshops. Most participants who changed their levels of support became more supportive (Figure 5), indicating that analytic-deliberative process can indeed increase overall support for forest mitigation strategies. While one could argue that changes in support for a strategy is not a direct measure of changes in values and normative learning, it still allows us to get a sense of potential changes in perception. For instance, one respondent noted that his perceived importance of mitigating climate change in the forest sector increased, leading to greater support for intensive forest management strategies that he “would never have otherwise supported”. Discussions with participants allowed us to identify three important factors that led to these changes in support: 1) better understanding of the strategies, 2) discussions during the workshops and 3) the technical information provided.

This study provides support for the claim that a well-designed and implemented analytic-deliberative engagement process can effectively structure decision insights, be inclusive and value-driven rather than technocratic, and potentially increase knowledge of, and support for, potential policy recommendations. However, these findings are somewhat limited by a relatively small participant sample size and the fact that the workshops did not always involve all interested and/or affected parties (e.g., general public, Indigenous peoples). These commonly experienced challenges (Webler and Tuler, 2018) underline the need for a broader and more comprehensive engagement of the general public as well as separate engagement process with Indigenous peoples based on culturally sensitive and appropriate research protocols and methodologies (Smith, 2012). In addition, involving a greater diversity of participants earlier in the process (specifically at the stage of selecting the mitigation strategies and evaluation criteria) would likely have increased trust and acceptability of the technical data, and the results of the engagement process overall.

By identifying recommendations for designing similar analytic-deliberative processes (Figure 2), we answer the call in natural resources management generally, and forest management in particular, for “more transparent, simple, and easily accessible” participatory processes (Mendoza and Martins, 2006). These recommendations highlight the importance of involving multiple actors, views and worldviews. We also emphasize the importance of inclusive deliberation that is based on the best available science, but also on other forms of expertise, including lay and traditional knowledge. Perhaps most importantly, our recommendations are consistent with others who call for opening-up analysis, deliberation and appraisal (Chilvers and Kearnes, 2015; Gregory et al., 2012; Stirling, 2008). This means acknowledging that a one-size-fits-all solution does not always exist, but rather that plural and conditional policy options are often more advised in the context of complex environmental issues.

We have conducted our study following the BREB guidelines (Behavioural Research Ethics Board) at the University of British Columbia, Canada (ID H15-01354). It is BREB’s position that a breach of confidentiality of study participants has taken place when there is a failure to conform to the commitment that the researchers have made to the study participants when some or all the data has entered the public domain (i.e., the data has become available to any person who is not authorized to view or access the data). Thus, we shall not publicly disclose any research data. However, if readers are interested in requesting data from this publication, they can contact the lead authors, and data will be available upon request. They must sign a non-disclosure agreement and comply with the BREB guidelines for further use of the data. Readers may contact [email protected] if they want to request data.

We would like to thank Michelle Connolly, Caren Dymond, Robert Kozak, Tony C. Lemprière, Sara Muir-Owen, Dennis Paradine, Byron Smiley, Carolyn Smyth, Werner A. Kurz, and Zach Xu for their support and roles played in organizing and carrying-out the workshops. We are particularly grateful to Robin Gregory for reviewing and providing insightful comments on previous versions of the manuscript.

This research was funded by the Pacific Institute for Climate Solutions (PICS) Forest Carbon Management Project. The work of G.P.S. was financially supported by the Social Sciences and Humanities Research Council of Canada, the University of British Columbia and PICS.

The authors have no competing interests to declare.

• Contributed to conception and design: GPSL, GH

• Contributed to acquisition of data: GPSL, GH, SS

• Contributed to analysis and interpretation of data: GPSL, GH, SH

• Drafted and/or revised the article: GPSL, GH, SS, SH

• Approved the submitted version for publication: GPSL, GH, SS, SH