Linking knowledge to action in ocean ecosystem management: The Ocean Modeling Forum Open Access

Managing the Earth’s marine resources is an increasingly complex task. Demands on ocean resources mount as a growing human population identifies more uses for the services and benefits provided by the world’s oceans, and builds the capacity to access them (Crowder et al., 2006; Halpern et al., 2015). Supported by technological advances, the areal extent of human activities and impacts has expanded steadily over the past half century (Merrie et al., 2014). In addition, the number of countries engaging in the extraction of ocean resources is increasing. This increase is especially prevalent in previously inaccessible waters outside of national jurisdiction, further adding to the complexity of the ocean management landscape (Merrie et al., 2014). Considerable effort is being devoted to devising management programs and strategies that are sufficiently lithe and responsive to these increasing demands (Foley et al., 2010; Ban et al., 2014; Mengerink et al., 2014; Lewison et al., 2015). In addition, the increasing emphasis on ecosystem-based management (EBM) of the oceans broadens the decision landscape beyond traditional ocean resource management to incorporate multiple ecosystem stressors and endpoints ranging from toxic pollutants to food web dynamics to human wellbeing (National Oceanic and Atmospheric Administration [NOAA], 2006; Ehler and Douvere, 2009; European Commission, 2009; Council on Environmental Quality, 2010; National Ocean Council, 2013; International Council for the Exploration of the Sea [ICES], 2014). This structural and thematic complexity requires a suite of technical tools to support decision-making, including holistic, system-wide understanding of impacts across multiple objectives, and evaluation of tradeoffs among objectives (Levin et al., 2009; Lewison et al., 2015).

The complexity of decisions in ocean ecosystem management has been met with the development of increasingly complex analytical tools, and models are often required to describe system dynamics and to evaluate the consequences of potential management actions (Levin et al., 2009; Punt et al., 2016a). End-to-end models have been developed to address this challenge (Collie et al., 2009; Fulton et al., 2011; Moloney et al., 2011; Plagányi et al., 2014), although these models are often data-hungry and computationally intensive, creating inefficiencies and long waiting periods for model development of this type (Ascough et al., 2008; Kaplan et al., 2018).

A challenge remains, in successfully linking sophisticated analytical tools to the complex decision landscape, in part owing to major gaps that exist between the communities of technical tool developers and decision makers (McNie, 2007; Cook et al., 2013), including potentially different perspectives on the usefulness of technical information in decision-making (Kunseler et al., 2015). The challenge to link knowledge to action for effective natural resource management is now well recognized (Knight et al., 2008; Weaver et al., 2014; Clark et al., 2016b; Beier et al., 2017; Wall et al., 2017). Knowledge that is useful to management is salient (relevant to management needs), credible (authoritative and trusted), and legitimate (developed within a process that accounts for all stakeholder perspectives) (Cook et al., 2013; Kunseler et al., 2015; van Voorn et al., 2016). Recognition of the need to make science usable has led to efforts to improve science communication (Fischhoff and Scheufele, 2014, and references therein; Young et al., 2014), to explicitly engage in the process of knowledge co-production or collection (Jacobs et al., 2016; Reid et al., 2016), and to increase support for interdisciplinary research groups (Levin et al., 2016), but few of those efforts translate to management action. Meaningfully linking scientific findings to policy may be done most successfully by science-policy intermediaries, boundary organizations, or “knowledge brokers” (Cook et al., 2013; Bednarek et al., 2016, 2018; Clark et al., 2016b).

One approach to addressing the challenges posed by modeling complex systems with high degrees of uncertainty is to compare across multiple models that differ in their level of complexity, structure, assumptions, and parameterization (Verburg et al., 2008; Steinacher et al., 2010; Fulton et al., 2011). This approach was the foundation of the successful Energy Modeling Forum, formed at Stanford University in 1976, which has a focus on investigating the structure and function of multiple models relevant to individual energy policy topics (Huntington et al., 1982). Multi-model approaches offer many advantages, including the ability to compare the consequences of hypotheses, the ability to identify areas of agreement and divergence among model outputs, collaboration within the modeling community, and continued model development (Weller et al., 2013). They also take different forms, each intended to improve forecasts, reduce the uncertainty associated with a single model approach, and/or broaden the scope of evaluation beyond that available for a single model. Multi-model approaches include ensemble modeling (Tebaldi and Knutti, 2007; Reichler and Kim, 2008; Ruiz et al., 2014; Krishnamurti et al., 2016; Shin et al., 2017) and model coupling (Reichler and Kim, 2008; Bever and MacWilliams, 2013; Boomer et al., 2013; Krishnamurti et al., 2016), approaches that have also been applied in evaluating the impacts of climate change on marine ecosystems and fisheries worldwide (Monier et al., 2013; Niiranen et al., 2013; Henson et al., 2017).

To address the challenge of linking model-based advice to action, the Ocean Modeling Forum (OMF) was developed to address specific ocean-ecosystem management issues using multi-model approaches. The OMF brings together models and modelers, decision makers and/or managers, subject and system experts, and other experts and stakeholders, in ad hoc working groups of 15–20 individuals to jointly develop usable model-based advice in support of specific ocean management issues. The OMF framework has been tested on two inaugural case studies, focused on social-ecological system impacts of forage fish harvest in the California Current and Northeast Pacific ecosystems: Pacific sardine (Sardinops sagax), and Pacific herring (Clupea pallasii). These test cases have led to the development of guidance for generating actionable, model-based knowledge. Based on these test cases, we identify and discuss below five primary conditions or characteristics of model-based knowledge for effectively linking it to management of the world’s oceans: the knowledge is aspirational, applicable, parsimonious, co-produced, and amplifying.

Here we describe characteristics that make model-based knowledge “ready for action,” i.e., useful in a decision space. We use the word “knowledge” not only to represent specific model results and outputs, but also to generally reflect the scholarship generated through the process described herein of developing model-based advice. We suggest that this knowledge has various characteristics that reflect the attitudes, motivations, and approaches of the groups involved in its production. However, we argue that not only do the attitudes, motivations, and approaches of the people involved in knowledge production matter, but the knowledge, advice, and outputs produced also need to share the characteristics we identify and describe below.

Aspirational means of lofty purpose, or reaching for the sky, such as aspirational goals. Ocean managers and policymakers have increasingly signaled their lofty desire to pursue ecosystem-based management of ocean resources over the last decades, in Europe (ICES, 2014) and the United States (Witherell, 2003; Council on Environmental Quality, 2010; Pacific Fishery Management Council, 2014a). While there is widespread agreement on the desire and need for EBM, the implementation and operationalization of EBM and of ecosystem-based fisheries management (EBFM) are both complex and challenging (NOAA, 2006; ICES, 2014; Link and Browman, 2017). Frameworks for organizing EBM and EBFM, such as the Integrated Ecosystem Assessment (IEA) approach (Levin et al., 2009), have been developed and are being deployed globally (NOAA, 2006; Levin et al., 2013; ICES, 2014; Harvey et al., 2017a; Levin et al., 2018). Yet, very few concrete fishery management actions or harvest rules in place explicitly account for impacts on ecosystem components (Skern-Mauritzen et al., 2016; Harvey et al., 2017b), such as dependent predator species, or dependent human communities. In most cases, explicit EBFM remains aspirational. Here, we argue that knowledge to inform EBFM must also be aspirational, by encompassing ecosystem components and processes.

The Pacific Fishery Management Council (PFMC), the agency responsible for management of fisheries in federal waters along the US West Coast, has aspirational objectives for the management of Pacific sardines (and other coastal pelagic species) that include providing adequate forage for dependent species (Pacific Fishery Management Council, 2016a). Achieving this objective requires understanding which species are dependent on forage species, and what abundance of forage is adequate to support their populations. It also requires understanding how current management strategies affect the abundance of forage fish available to dependent species in space and time, and how forage-dependent species respond to different levels of forage biomass available. Evaluating such impacts of management strategies requires incorporating details related to population dynamics of the predators and the prey, including other available forage species, migration/movement, interannual and seasonal climate variation, and oceanographic influences (Sydeman et al., 2017).

To address the aspirational goals of the PFMC, the OMF Pacific sardine working group evaluated the impacts on sardine-dependent predators of the harvest rule implemented in regulation to set catch limits for the US fishery using multiple models that, taken together, allow consideration of such ecosystem impacts. The current harvest control rules take into account environmental influences via a temperature-dependent exploitation rate parameter. To the conceptual model for Pacific sardine used in the previous evaluations of harvest control rules (Pacific Fishery Management Council, 2013), the OMF working group added several ecosystem-level components that could be added to any future control rule revision, including food web interactions, spatial distribution and movement of predators and prey, seasonal and interannual climate variation, and bottom-up forcing using a suite of models of varying levels of complexity (Figure 1).

For model results to be truly useful in decision-making, they need to be applicable, that is, what they produce should match the expectations of the management decision process, which sits within a larger governance system. Decision-making processes that influence the management of natural resources have their own cultures, processes, and timelines. Governance systems can be generally classified along a gradient from top-down control to community-based, and each governance archetype is associated with a general approach to using science and scientific advice (Acheson, 2006). Models most likely to be used are typically initially designed – or adjusted – to address the specific management question at hand. Modeling results are more likely to be actionable if they provide outputs that can be readily applied to decisions (Wall et al., 2017). Examples of this dependence on context include requests for specific research initiatives within US Fishery Ecosystem Plans (Dereynier, 2014), and application of social science within reviews of fishing quota programs (Gulf of Mexico Fishery Management Council, 2013; Pacific Fishery Management Council and National Marine Fisheries Service, 2010). This context can only be navigated by direct collaboration between modelers, decision makers, and stakeholders.

Collaborative efforts to address specific management actions should provide the opportunity to weigh tradeoffs among ecosystem goals, and model outputs that are situated in a tradeoff framework will be more applicable to decisions than those offering only one way forward (Link, 2010). An example is the use of a management strategy evaluation framework, which involves using a stakeholder process to identify objectives and potential management strategies. This approach determines which models and other tools are available to evaluate the alternative strategies, the outcomes of which can be compared for their performance against stated objectives, so long as standards for model performance against stated objectives are identified (Punt, 2017). Where objectives have been identified, scenarios can be designed to generate outputs that are linked to these objectives so that the multi-objective performance of alternative management actions can be compared (e.g., Fulton et al., 2014; Punt et al., 2016a). Thus, the knowledge is applicable to decisions related to achieving objectives.

To achieve its management objective of providing adequate forage for dependent species, the PFMC has the power to set limits on the U.S. harvest of Pacific sardines. The decision to be made, therefore, is how much sardine to harvest in U.S. waters. The ecological impacts of this decision relate to the long-term sustainability of sardines and the predators that depend on them. The applicability of models relevant to this problem was guided by a committee of the PFMC (PFMC, 2013), whose members reviewed the models existing before the OMF case study, found them inadequate, and provided guidance on specifications for new models that would be required for policy applications.

To support management objectives under the current harvest control rule, the OMF focused on existing harvest levels by evaluating impacts on predators under ranges of sardine biomass estimated using a model of intermediate complexity for ecosystem assessment (MICE; Plaganyi et al., 2014). The MICE projects population dynamics of: Pacific sardine, Northern anchovy Engraulis mordax, and “other forage”; predators brown pelican Pelecanus occidentalis and California sea lion Zalophus californianus; and three fisheries (Canada, United States, and Mexico) in a spatially-, temporally- and age-explicit framework with environmental stochasticity driving sardine recruitment (Punt et al., 2016b). The models explored impacts of current sardine management on forage fish predators using model-predicted sardine biomass potentially resulting from the current harvest control rules. The final harvest limit for the US fishery for sardine is the minimum of the acceptable biological catch (the exploitation rate corresponding to maximum sustainable yield, H_MSY, multiplied by the estimate of the biomass of sardine aged 1 year and older, reduced by a buffer to account for scientific uncertainty) and the harvest guideline (HG). The latter is calculated using the formula:

where B¹⁺ is the estimate of the biomass of sardine aged 1 year and older, and 0.87 is a factor to account for the assumed proportion of sardine in US waters. The value of H_MSY depends on sea surface temperature collected by the CalCOFI program (Punt et al., 2016b).

To complement the MICE, we applied a multi-model approach (Kaplan et al., 2018) to evaluate the broader ecosystem-level impacts of the range of sardine abundances projected by the MICE under the current control rule (Table 1). This multi-model approach (Figure 2) involved application of a simple food web model (Koehn et al., 2016) as well as a full ecosystem Atlantis model (Kaplan et al., 2017). The food web model allowed calculation of indicators of prey importance in the food web and predator dependence upon those prey groups. The multi-model approach allowed consideration of how a broader set of ecological and economic objectives responds to alternative levels of sardine abundance (Figure 2), and of how model parameterization and structural uncertainty influence results. In the case of Atlantis and the MICE, economic performance was measured simply, in terms of sardine catch.

Improved socio-economic metrics can also provide context for the ecological and fishery outputs from the Atlantis model or the MICE, as part of this multi-model approach (Figure 3). One such suite of metrics, coastal community vulnerability indices, are generalized socioeconomic vulnerability metrics for human communities that rely on the marine ecosystem. These metrics have had broad application for defining the risks to human communities from exogenous shocks such as disasters (hurricanes, oil spills, fishery collapses) and have been employed in the US Northeast and Southeast (Jacob et al., 2012; Jepson and Colburn, 2013), Alaska (Himes-Cornell and Kasperski, 2015), at the US national level (http://www.st.nmfs.noaa.gov/humandimensions/social-indicators/map), and in Mexico (Morzaria-Luna et al., 2014). The Community Social Vulnerability Index is derived from seven community-level social and economic indices of community characteristics: personal disruption, poverty, population composition, housing characteristics, labor force structure, natural resource labor force, and wealth and education. These methods are being applied to fishery management nationally (Gulf of Mexico Fishery Management Council, 2013) and have been identified as applicable tools for the California Current.¹ Thus, one step toward the aspirational goal of assessing social impacts of changing harvests (e.g., as projected by ecosystem models) is to consider those harvests in the context of social and economic vulnerability metrics familiar to fishery decision makers.

Bigger is not always better. The shift from single-species management to ecosystem-based management has been accompanied by a parallel shift in focus from single-species models and assessments to multi-species and ecosystem models. While the advent of supercomputing coupled with the desire to address complex, ecosystem-level questions has afforded increasing opportunity to develop complex ecosystem models – as of the writing of this paper, 28 Atlantis ecosystem models are in use and an additional 16 are in development (E. Fulton, personal communication) – building such models is time-, resource-, and data-intensive. The added complexity in ecosystem-scale models confers many benefits, including the ability to incorporate a broad range of ecosystem components and evaluate impacts of management scenarios across a suite of objectives. However, there are multiple challenges associated with that added complexity, including competing hypotheses about ecosystem dynamics, increasing need for data, and highly uncertain predictions (Fulton et al., 2003). Investing in the development of such models may be appropriate and necessary for the breadth of management questions of the modern era, but parsimonious models are often better suited to addressing specific problems at hand, or providing tactical advice, which may be preferred (Hannah et al., 2010). For ecosystem-level projects, deploying multiple models of varying levels of complexity, each suited to a different purpose and audience, can be useful (Fulton et al., 2015). In addition, models containing high numbers of parameters and outputs, not all of which are relevant to the question at hand, increase parameter uncertainty and may impede the dialogue between resource users, managers and modelers (Collie et al., 2016), a fact that can erode trust in the efficacy of models in multi-stakeholder processes. Given limitations on time and energy, a prudent investment approach may be to couple simpler, more parsimonious models with more complex ones, including using qualitative models and conceptual models as components of complex EBM efforts (Harvey et al., 2016).

In the case of the OMF Pacific sardine working group, the relevant management objective is to “provide adequate forage for dependent species” (Pacific Fishery Management Council, 2016a), and four models were employed to address this objective. The simplest model in terms of its ecosystem components was a single-species model developed to evaluate the impacts of existing sardine harvest rules on fishery- and sardine-specific objectives, such as mean sardine biomass, and probabilities of falling above or below certain biomass thresholds (Figure 1; Hurtado-Ferro and Punt, 2014). This model included the influence of an environmental variable, temperature, on recruitment. The next most complicated model was the static food web model constructed specifically for California Current forage fish with high trophic resolution (Koehn et al., 2016). This model was used to identify predators most likely to be highly dependent upon sardines and therefore potentially impacted by sardine harvest, and to identify forage species in addition to sardine that are important prey in the California Current.

The MICE model described above was intentionally constructed to be a parsimonious approach to addressing the particular questions at hand. This approach is advantageous in evaluating the impacts of management actions because MICE are tactical; i.e., they can be structured to target specific management questions or objectives, they contain parameters that can be adjusted by fitting model outputs to observations, and they are not very intensive computationally. Fast computation times (on the order of hours versus days) mean that MICE are well-suited to exploration of a range of specific uncertainties via scenarios and across multiple replicates associated with stochasticity or process/observation error (Plagányi et al., 2014). In this case the MICE expanded upon the single-species model by incorporating climate, oceanography, predators and other fisheries. Climatic and oceanographic factors were accounted for in the MICE by modelling the slope at the origin of the stock-recruitment relationship as a function of an oceanographic driver. The function of the driver is to generate simulated variation in sardine abundance similar in magnitude to that observed in sardine scales contained in sediment core samples from the southern California Bight, prior to exploitation. Atlantis end-to-end ecosystem model (Kaplan et al., 2017) was conducted, following its initial review by an external panel of the Pacific Fishery Management Council (2014b). This “end-to-end” Atlantis model included food web dynamics, oceanography, biogeochemistry, and fisheries (Figure 1), and the update refined the parameterization to explore sardine ecology and management questions.

For many years, scientists aiming to produce usable knowledge have followed the “loading dock” model, wherein scientific data are produced and results are published in the peer-reviewed literature, where they remain (somewhat) available to anyone interested in applying them to management. The potential consequence of this approach is that decision makers, when faced with a decision and seeking technical input, can find that scientific outputs are mismatched to the decision point. Increasing attention is being paid to replacing these standard practices with a process of co-production of knowledge: the co-identification of research questions and outputs between scientists and decision makers or managers, informed by known decision points and available, relevant scientific tools; and the extended, iterative progress towards shared vision and action (Cash et al., 2006; Clark et al., 2016b; Nel et al., 2016). Research should be designed and conducted in consideration of possible linked management actions to overcome the gaps between knowledge production or research on the one hand, and action or management on the other (Jarvis et al., 2015). In this way, the ensuing technical inquiry can be shaped to target a specific decision: outputs match management-relevant variables, timeframes are relevant for decision landscapes, and points of entry for technical information into the policy process are identified. Likewise, through direct collaboration with scientists, decision makers can bracket their needs by what information can be produced, within what timeframe, and bearing what level of uncertainty.

Knowledge co-production often also includes stakeholders, whose participation can influence the data or knowledge used for technical analyses, and who will weight the potential outcomes during the decision process. Furthermore, as the impacts of decisions on multiple ecosystem components are weighed against each other, expertise across multiple disciplines is required within the group. Natural scientists can be joined, for example, by social scientists to assist in defining and measuring the human dimensions of ecosystems, such as social vulnerability (Figure 3), or by economists to describe and evaluate the influence of supply chains on the system; a multitude of additional examples exist. Often, the use of knowledge brokers or “boundary spanners” can be employed (Bednarek et al., 2016). Knowledge co-production is a vital component of actionable science, though barriers to applying the resulting outputs to actions still exist (Weichselgartner and Kasperson, 2010).

Motivated by the idea that knowledge co-production improves the usefulness of science, the OMF convenes working groups comprised of modelers, managers, empiricists and stakeholders to address single issues of ocean management concern. By including managers or decision makers along with modelers and other experts, the management questions can be framed together, the models can be structured together to ensure that they will address the relevant decision(s), and other stakeholders can influence what information is used in the modeling activities. The working groups are diverse in their membership, and convene groups of individuals who otherwise would not have the opportunity to collaborate.

Quantifying co-production is a challenge. One approach is to measure the diversity of the stakeholders participating in the knowledge production activity. To identify the diversity of the OMF network of collaborators, we collected data from the OMF Pacific herring working group participants on their collaboration networks, as defined by their publications. The Pacific Herring working group of 20 individuals included academic ecologists, food web and stock assessment modelers, anthropologists, political scientists, economists, indigenous tribal elders, First Nations scientists, resource management agency scientists, fishing industry executives, and commercial fishermen.

We conducted a bibliographic network analysis, mapping the 1,039-person co-authorship network based on the publication records (in Web of Science) of working group members, prior to jointly producing manuscripts as part of the working group (Figure 4). Our analysis showed the dominance of individuals publishing in the life sciences, as well as the existence of many interdisciplinary participants publishing in life and social sciences. We also found multiple nodes (individuals) and clusters of individuals that, prior to participation in the OMF, were disconnected from a central, highly connected aggregation of individuals. Those disconnected nodes illustrate the impact of the OMF: the map shows disconnected individuals and their networks as separate from the rest of the OMF network. A shared publication links all isolated nodes into the network (Supplementary Figure S1), and the newly created network includes not only a diverse set of stakeholders, but individuals and their collaborators who were previously not linked, despite having expertise on the same topic, Pacific herring. The new network formed after a joint publication resulted in an increase in the number of authors in the network, from 1,039 to 1,044, and more author-to-author links in the group, from 5,894 to 6,097. In addition, the total number of links (the number of individual manuscript-based links between pairs of authors) increased from 8,049 to 8,259. Thus, the OMF broadened the network, linking previously disconnected individuals, and increased the number of links between individuals and the average between-person links in the network, strengthening ties within the network.

Opportunities exist for knowledge co-production to improve ocean management efforts, as most relevant research, ranging from fisheries management to human sense of place, is currently conducted in isolated silos of disciplinary homogeny versus in cross-disciplinary groups. To illustrate the potential for improvement, we surveyed the members of the OMF Pacific Herring and Pacific Sardine working groups (see Appendix S1 for methods) to identify the ocean management issues that could most benefit from multi-model approaches and knowledge co-production. We then conducted a network analysis of the 1,000 most connected authors from the 500 most recently published papers in the top nine ocean management issues identified (Figure 5). The analysis shows a core of individuals publishing in two or more themes that are disconnected from clusters of other closely-linked individuals publishing in single themes, especially fisheries, eutrophication, and governance. In other words, interdisciplinary (i.e., working across topics) researchers are largely disconnected from individuals working only within a single discipline or topic. Many scientists are not exposed to interdisciplinary research opportunities. Furthermore, within topics, or individual ocean management themes, individual tightly-connected groups are very distant and disconnected from each other. This disconnect may occur between research groups of different disciplines, such as social and natural scientists, or of physicists and environmental scientists, who conduct research in the same field but not often with each other. The existence of clusters, plus individuals who are wholly disconnected from everyone else, represents opportunity for increased co-production of knowledge across disciplinary silos to inform ocean decision-making.

The final characteristic of model-based knowledge that is ready for action is that it amplifies existing empirical information, and information derived from individual models, through cross pollination among multiple modelers and between modelers and other experts. At its foundation, a multi-model approach is an extension of the general scientific strategy of multiple hypotheses (Chamberlin, 1890), and a suite of tools has been developed in response to this strategy for evaluating among competing hypotheses (Burnham and Anderson, 2004; Burnham et al., 2011; Grueber et al., 2011). The power of bringing multiple models together is not only in reducing uncertainty by varying model complexity and breadth, but also in modelers learning from each other through iteration, in more effective engagement with stakeholders, and in being able to address both tactical and strategic concerns (Fulton et al., 2015). The multi-model process provides richness about how the system should be modeled. Dialog among modelers also daylights individual biases and distortions in model building, which leads to being a better modeler.

Additional wisdom is also gained by engaging with experts with extensive experience in the ecosystem, or with the focal species, who serve to represent knowledge about the ecosystem. Models can amplify such system understanding. Developing models through iterative interaction with knowledge holders can improve models by identifying which system dynamics are most likely to impact the issues at hand and which should be prioritized for inclusion in the models, what additional sources of data and analyses can be used to inform the models, how to validate model outputs, etc. (Fulton et al., 2015). Models can amplify ideas and different types of knowledge, including traditional knowledge, arising from other stakeholders and sources. The value of traditional knowledge, including traditional ecological knowledge, in ecological understanding and resource management is widely recognized; combining traditional knowledge with scientific approaches can amplify the power of each individual form of knowledge (Huntington, 2000; Moller et al., 2004), and incorporating traditional knowledge into models can effectively inform conservation and management (Butler et al., 2012; Polfus et al., 2014).

In the OMF Pacific sardine working group, diets of predators in two models, the MICE (Punt et al., 2016b) and the Atlantis ecosystem model (Kaplan et al., 2017), were informed by the highly resolved Ecopath food web model of the California Current ecosystem (Koehn et al., 2016). Data collection and initial model parameterization for the Atlantis model and the food web model were a collaborative effort (with authors shared across papers). This effort did not necessarily force parallel interpretation of data sources or parallel model structure, but provided some efficiency to the efforts and some common language before the models were brought to broader participants. The outputs of the MICE, sardine abundance, were also used to structure the Atlantis model outputs, allowing for an apples-to-apples comparison of model predictions by managers. Ecopath information was directly passed to the MICE, including being used to prioritize the predators in the MICE. The MICE and Ecopath results both led to realization of the importance of anchovy, which was highlighted in an Atlantis manuscript (Kaplan et al., 2017).

The Herring working group also employed models that amplified messages and findings from other models and sources of knowledge. The Herring working group evaluated the performance of alternative management actions against social, ecological and economic objectives in a management strategy evaluation framework, using an operating model of herring population dynamics internally linked to a stock assessment model. This stock assessment model was structured to mimic a separately developed, novel, spatially-structured stock assessment model (Punt et al., 2018). These two models were developed through consultation among modelers within the OMF. Another model evaluated the consequences of harvest under scenarios of socially learned migration behavior in herring, the presence of which was informed by traditional ecological knowledge (MacCall et al., 2018), amplifying existing knowledge about herring not previously incorporated into models to investigate the impacts of management decisions.

Development of the Ecopath, MICE, and Atlantis models involved extensive synthesis of existing empirical and modeling studies, but also additional expert knowledge generated within the Ocean Modeling Forum. For example, improvements during the OMF informed the effort to represent sardine specifically in the California Current Atlantis model, disaggregating them from other forage fish species, as had been the approach in the earlier versions of the model (Horne et al., 2010; Kaplan et al., 2013). This disaggregation made the model more relevant for the present case study, while addressing improvements to ecosystem models that were requested by the Pacific Fishery Management Council (2013) and an external review panel (Pacific Fishery Management Council, 2014b; Kaplan et al., 2017). The representation of sardine included refining the diet information, first taken from individual-based models contributed by OMF members (Rose et al., 2015), then modified based on expert knowledge within the Forum that suggested reliance on filter feeding and higher consumption of phytoplankton by adult sardines (Figure 6), also supported by studies from other ecosystems (van der Lingen et al., 2006, 2009). Another example of including expert knowledge from the OMF involved parameterization of sardine migration patterns in both Atlantis and the MICE based on unpublished modeling of a working group member (summarized in Punt et al., 2016b, their Appendix A.1.1.1 and Figures A.2 and A.3). These examples are consistent with the role of expert knowledge within approaches to ecosystem-based fishery management (Smith et al., 2007).

We have illustrated five characteristics of model-produced knowledge that make such knowledge suitable for use in ocean management and decision-making. What, then, have been the impacts of such model-produced advice? Quantifying impact is one of the greatest challenges facing practitioners working broadly at the science-policy boundary (Lyall et al., 2004; Meagher and Lyall, 2013). While the holy grail for applied scientists has long been direct evidence that their science is being used to improve concrete outcomes, such as (for conservation scientists) more habitat protected, improved water quality, or increased fish biomass, the direct linking of science to such impacts is difficult and rare. In addition to these direct instrumental impacts of research on policy and practice decisions (Meagher and Lyall, 2013), a broader suite of impacts of research on decisions can be measured. These include conceptual impacts, where research changes thinking or raises consciousness, and capacity-building, meaning education, training, or other forms of collaboration. Other processes can also be impacted: attitudes can be changed, especially towards knowledge exchange; and enduring connectivity, especially between researchers and knowledge users, can be established (Meagher and Lyall, 2013).

As illustrated above, the Sardine working group focused on developing parsimonious models (e.g., a MICE) that aspired to address ecosystem and especially food web impacts of sardine harvest, and that were applicable to management (i.e., evaluated impacts of the existing sardine harvest rule on the food web). These models may ultimately have instrumental impacts, i.e., changes to harvest practices to support sardine-dependent predators, but those impacts are likely to occur over longer periods of time. In the meantime, we can track conceptual impacts as the decision-making body considers the models in its deliberations: the subgroup of the PFMC focused on sardines reported its intention to track and use these models to examine the impacts of harvest on the food web (Pacific Fishery Management Council, 2016b), and a modeling update was subsequently requested by PFMC and presented by an OMF working group member (Pacific Fishery Management Council, 2016c).

The Herring working group, multi-stakeholder and interdisciplinary, co-produced knowledge related to the consequences of herring fisheries for social, economic, and ecological objectives, and amplified traditional knowledge in the formulation of models to evaluate potential outcomes of management decisions. This process has impacted attitudes about knowledge exchange, as noted by Harvey Kitka, a member of the Sitka Tribe of Alaska’s Herring Committee and a member of the Herring working group:

Before the Herring working group was formed, exchange of information used in decision-making about herring and herring fisheries was limited between fisheries management agencies and traditional knowledge holders, which was a source of conflict between these groups (Hume, 2015; Welch, 2015; Woolsey, 2015). Testimony from Herring working group members demonstrated that participation in the co-production of model-based knowledge served to change attitudes about the potential for various types of knowledge to be used in models and improved collaborations, opening pathways for enduring connections (J. Cleary, personal communication, S. Dressel, personal communication). Tracking impacts of knowledge and research remains a challenge for those working at the boundary between science and its use, and future efforts should focus on this broader suite of impact types.

Operationalizing ocean management for multiple benefits is challenging and requires a flexible approach (Dickey-Collas, 2014). Ocean resources are often shared among multiple sectors, and governance of large marine ecosystems is complex and heterogeneous (Crowder et al., 2006; Mahon et al., 2010), making the successful application of knowledge in decision spaces difficult. Considerable effort has been applied to developing guidance for improving the use of science in natural resource management and decision-making, including identifying features of actionable science (Cook et al., 2013; Kirchhoff et al., 2013; Clark et al., 2016b), intermediaries for bridging the knowledge-action divide (Bednarek et al., 2016, 2018; Clark et al., 2016a) and developing how-to guides for the co-production of knowledge (Beier et al., 2017). Such thinking can be extended to encompass the challenges and opportunities for linking model-based knowledge to ocean management. The Ocean Modeling Forum has developed one framework, described here, for linking model-based advice to ocean management decision-making; there are likely others (e.g., Lynam et al., 2016), though a full review is beyond the scope of the current paper. Unique to the OMF approach is its working group structure, flexibility to various decision settings (i.e., not designed for application to a single governance system), multi-stakeholder approach, and inclusion of decision makers as working group members. Particularly pressing is the need to connect climate science to local decision makers, and work continues to identify indicators and tools that can help to meet this need (Mauger and Kennard, 2017; Wall et al., 2017). We have determined five elements of actionable knowledge useful for linking model-based outputs to ocean management issues: knowledge is aspirational, parsimonious, applicable, co-produced, and amplifying. Such a framework can also complement large-scale investigations into data gaps and modeling needs, as has been done for terrestrial biodiversity modeling (Urban et al., 2016). Linking science to decision-making is a primary challenge for the conservation and sustainability of ocean resources, and indeed natural resources worldwide. Investing effort in ensuring that marine research is well positioned to have impact, while energy- and time-intensive, should be a primary consideration for ocean scientists.

We thank two anonymous reviewers, C Harvey, and J Samhouri for their insightful and helpful comments on earlier drafts of this manuscript. We acknowledge the contributions by the members of the OMF Pacific sardine and Pacific herring working groups, as well as the guidance of our advisory board members. We particularly appreciate Kai Lee for his vision and support in the genesis of the Ocean Modeling Forum, and its development thus far.

The Ocean Modeling Forum is a program of the University of Washington College of the Environment and in collaboration with The Nature Conservancy of Washington. The Ocean Modeling Forum and TBF are supported by the David & Lucile Packard Foundation and the Pew Charitable Trusts. Atlantis model development benefited greatly from a grant to ICK from the NOAA Ocean Acidification Program and National Centers for Coastal Ocean Science. Contributing support for this work was provided by the National Oceanic and Atmospheric Administration’s Integrated Ecosystem Assessment (NOAA IEA) Program. This paper is NOAA IEA program contribution #2018.

The authors have no competing interests to declare.

• Contributed to conception and design: TBF, PSL, AEP

• Contributed to acquisition of data: TBF, PSL, AEP, IK, KN, AV

• Contributed to analysis and interpretation of data: TBF, PSL, AEP, IK, KN, AV

• Drafted and/or revised the article: TBF, PSL, AEP, IK

• Approved the submitted version for publication: TBF, PSL, AEP, IK, KN, AV