The sustainable use of global marine resources depends upon science-based decision processes and systems. Informing decisions with science is challenging for many reasons, including the nature of science and science-based institutions. The complexity of ecosystem-based management often requires the use of models, and model-based advice can be especially difficult to convert into policies or decisions. Here, we suggest five characteristics of model-based information and advice for successfully informing ocean management decision-making, based on the Ocean Modeling Forum framework. Illustrated by examples from two fisheries case studies, Pacific sardines Sardinops sagax and Pacific herring Clupea pallasii, we argue that actionable model-based output should be aspirational, applicable, parsimonious, co-produced, and amplifying.

Managing the Earth’s marine resources is an increasingly complex task. Demands on ocean resources mount as a growing human population identifies more uses for the services and benefits provided by the world’s oceans, and builds the capacity to access them (Crowder et al., 2006; Halpern et al., 2015). Supported by technological advances, the areal extent of human activities and impacts has expanded steadily over the past half century (Merrie et al., 2014). In addition, the number of countries engaging in the extraction of ocean resources is increasing. This increase is especially prevalent in previously inaccessible waters outside of national jurisdiction, further adding to the complexity of the ocean management landscape (Merrie et al., 2014). Considerable effort is being devoted to devising management programs and strategies that are sufficiently lithe and responsive to these increasing demands (Foley et al., 2010; Ban et al., 2014; Mengerink et al., 2014; Lewison et al., 2015). In addition, the increasing emphasis on ecosystem-based management (EBM) of the oceans broadens the decision landscape beyond traditional ocean resource management to incorporate multiple ecosystem stressors and endpoints ranging from toxic pollutants to food web dynamics to human wellbeing (National Oceanic and Atmospheric Administration [NOAA], 2006; Ehler and Douvere, 2009; European Commission, 2009; Council on Environmental Quality, 2010; National Ocean Council, 2013; International Council for the Exploration of the Sea [ICES], 2014). This structural and thematic complexity requires a suite of technical tools to support decision-making, including holistic, system-wide understanding of impacts across multiple objectives, and evaluation of tradeoffs among objectives (Levin et al., 2009; Lewison et al., 2015).

The complexity of decisions in ocean ecosystem management has been met with the development of increasingly complex analytical tools, and models are often required to describe system dynamics and to evaluate the consequences of potential management actions (Levin et al., 2009; Punt et al., 2016a). End-to-end models have been developed to address this challenge (Collie et al., 2009; Fulton et al., 2011; Moloney et al., 2011; Plagányi et al., 2014), although these models are often data-hungry and computationally intensive, creating inefficiencies and long waiting periods for model development of this type (Ascough et al., 2008; Kaplan et al., 2018).

A challenge remains, in successfully linking sophisticated analytical tools to the complex decision landscape, in part owing to major gaps that exist between the communities of technical tool developers and decision makers (McNie, 2007; Cook et al., 2013), including potentially different perspectives on the usefulness of technical information in decision-making (Kunseler et al., 2015). The challenge to link knowledge to action for effective natural resource management is now well recognized (Knight et al., 2008; Weaver et al., 2014; Clark et al., 2016b; Beier et al., 2017; Wall et al., 2017). Knowledge that is useful to management is salient (relevant to management needs), credible (authoritative and trusted), and legitimate (developed within a process that accounts for all stakeholder perspectives) (Cook et al., 2013; Kunseler et al., 2015; van Voorn et al., 2016). Recognition of the need to make science usable has led to efforts to improve science communication (Fischhoff and Scheufele, 2014, and references therein; Young et al., 2014), to explicitly engage in the process of knowledge co-production or collection (Jacobs et al., 2016; Reid et al., 2016), and to increase support for interdisciplinary research groups (Levin et al., 2016), but few of those efforts translate to management action. Meaningfully linking scientific findings to policy may be done most successfully by science-policy intermediaries, boundary organizations, or “knowledge brokers” (Cook et al., 2013; Bednarek et al., 2016, 2018; Clark et al., 2016b).

One approach to addressing the challenges posed by modeling complex systems with high degrees of uncertainty is to compare across multiple models that differ in their level of complexity, structure, assumptions, and parameterization (Verburg et al., 2008; Steinacher et al., 2010; Fulton et al., 2011). This approach was the foundation of the successful Energy Modeling Forum, formed at Stanford University in 1976, which has a focus on investigating the structure and function of multiple models relevant to individual energy policy topics (Huntington et al., 1982). Multi-model approaches offer many advantages, including the ability to compare the consequences of hypotheses, the ability to identify areas of agreement and divergence among model outputs, collaboration within the modeling community, and continued model development (Weller et al., 2013). They also take different forms, each intended to improve forecasts, reduce the uncertainty associated with a single model approach, and/or broaden the scope of evaluation beyond that available for a single model. Multi-model approaches include ensemble modeling (Tebaldi and Knutti, 2007; Reichler and Kim, 2008; Ruiz et al., 2014; Krishnamurti et al., 2016; Shin et al., 2017) and model coupling (Reichler and Kim, 2008; Bever and MacWilliams, 2013; Boomer et al., 2013; Krishnamurti et al., 2016), approaches that have also been applied in evaluating the impacts of climate change on marine ecosystems and fisheries worldwide (Monier et al., 2013; Niiranen et al., 2013; Henson et al., 2017).

To address the challenge of linking model-based advice to action, the Ocean Modeling Forum (OMF) was developed to address specific ocean-ecosystem management issues using multi-model approaches. The OMF brings together models and modelers, decision makers and/or managers, subject and system experts, and other experts and stakeholders, in ad hoc working groups of 15–20 individuals to jointly develop usable model-based advice in support of specific ocean management issues. The OMF framework has been tested on two inaugural case studies, focused on social-ecological system impacts of forage fish harvest in the California Current and Northeast Pacific ecosystems: Pacific sardine (Sardinops sagax), and Pacific herring (Clupea pallasii). These test cases have led to the development of guidance for generating actionable, model-based knowledge. Based on these test cases, we identify and discuss below five primary conditions or characteristics of model-based knowledge for effectively linking it to management of the world’s oceans: the knowledge is aspirational, applicable, parsimonious, co-produced, and amplifying.

Here we describe characteristics that make model-based knowledge “ready for action,” i.e., useful in a decision space. We use the word “knowledge” not only to represent specific model results and outputs, but also to generally reflect the scholarship generated through the process described herein of developing model-based advice. We suggest that this knowledge has various characteristics that reflect the attitudes, motivations, and approaches of the groups involved in its production. However, we argue that not only do the attitudes, motivations, and approaches of the people involved in knowledge production matter, but the knowledge, advice, and outputs produced also need to share the characteristics we identify and describe below.

### 1. Knowledge is aspirational

Aspirational means of lofty purpose, or reaching for the sky, such as aspirational goals. Ocean managers and policymakers have increasingly signaled their lofty desire to pursue ecosystem-based management of ocean resources over the last decades, in Europe (ICES, 2014) and the United States (Witherell, 2003; Council on Environmental Quality, 2010; Pacific Fishery Management Council, 2014a). While there is widespread agreement on the desire and need for EBM, the implementation and operationalization of EBM and of ecosystem-based fisheries management (EBFM) are both complex and challenging (NOAA, 2006; ICES, 2014; Link and Browman, 2017). Frameworks for organizing EBM and EBFM, such as the Integrated Ecosystem Assessment (IEA) approach (Levin et al., 2009), have been developed and are being deployed globally (NOAA, 2006; Levin et al., 2013; ICES, 2014; Harvey et al., 2017a; Levin et al., 2018). Yet, very few concrete fishery management actions or harvest rules in place explicitly account for impacts on ecosystem components (Skern-Mauritzen et al., 2016; Harvey et al., 2017b), such as dependent predator species, or dependent human communities. In most cases, explicit EBFM remains aspirational. Here, we argue that knowledge to inform EBFM must also be aspirational, by encompassing ecosystem components and processes.

#### Illustration

The Pacific Fishery Management Council (PFMC), the agency responsible for management of fisheries in federal waters along the US West Coast, has aspirational objectives for the management of Pacific sardines (and other coastal pelagic species) that include providing adequate forage for dependent species (Pacific Fishery Management Council, 2016a). Achieving this objective requires understanding which species are dependent on forage species, and what abundance of forage is adequate to support their populations. It also requires understanding how current management strategies affect the abundance of forage fish available to dependent species in space and time, and how forage-dependent species respond to different levels of forage biomass available. Evaluating such impacts of management strategies requires incorporating details related to population dynamics of the predators and the prey, including other available forage species, migration/movement, interannual and seasonal climate variation, and oceanographic influences (Sydeman et al., 2017).

To address the aspirational goals of the PFMC, the OMF Pacific sardine working group evaluated the impacts on sardine-dependent predators of the harvest rule implemented in regulation to set catch limits for the US fishery using multiple models that, taken together, allow consideration of such ecosystem impacts. The current harvest control rules take into account environmental influences via a temperature-dependent exploitation rate parameter. To the conceptual model for Pacific sardine used in the previous evaluations of harvest control rules (Pacific Fishery Management Council, 2013), the OMF working group added several ecosystem-level components that could be added to any future control rule revision, including food web interactions, spatial distribution and movement of predators and prey, seasonal and interannual climate variation, and bottom-up forcing using a suite of models of varying levels of complexity (Figure 1).

Figure 1

Conceptual model of the Pacific sardine system. Ecosystem components (white ovals) and the models of varying complexity (colored boxes) employed by the Ocean Modeling Forum’s Pacific sardine working group, each of which incorporated different parts of the ecosystem. Single species model (SSM), the sardine operating model used to estimate impacts of harvest rules on sardine biomass (Hurtado-Ferro and Punt, 2014); Ecopath, a highly-resolved, forage fish-centric version of a food-web model used to identify highly dependent sardine predators (Koehn et al., 2016); MICE, a model of intermediate complexity for ecosystem assessment built to evaluate the impacts of sardine harvest on predators (Punt et al., 2016b); Atlantis, an end-to-end ecosystem model of the California Current Ecosystem (Kaplan et al., 2017). DOI: https://doi.org/10.1525/elementa.338.f1

Figure 1

Conceptual model of the Pacific sardine system. Ecosystem components (white ovals) and the models of varying complexity (colored boxes) employed by the Ocean Modeling Forum’s Pacific sardine working group, each of which incorporated different parts of the ecosystem. Single species model (SSM), the sardine operating model used to estimate impacts of harvest rules on sardine biomass (Hurtado-Ferro and Punt, 2014); Ecopath, a highly-resolved, forage fish-centric version of a food-web model used to identify highly dependent sardine predators (Koehn et al., 2016); MICE, a model of intermediate complexity for ecosystem assessment built to evaluate the impacts of sardine harvest on predators (Punt et al., 2016b); Atlantis, an end-to-end ecosystem model of the California Current Ecosystem (Kaplan et al., 2017). DOI: https://doi.org/10.1525/elementa.338.f1

Close modal

### 2. Knowledge is applicable

For model results to be truly useful in decision-making, they need to be applicable, that is, what they produce should match the expectations of the management decision process, which sits within a larger governance system. Decision-making processes that influence the management of natural resources have their own cultures, processes, and timelines. Governance systems can be generally classified along a gradient from top-down control to community-based, and each governance archetype is associated with a general approach to using science and scientific advice (Acheson, 2006). Models most likely to be used are typically initially designed – or adjusted – to address the specific management question at hand. Modeling results are more likely to be actionable if they provide outputs that can be readily applied to decisions (Wall et al., 2017). Examples of this dependence on context include requests for specific research initiatives within US Fishery Ecosystem Plans (Dereynier, 2014), and application of social science within reviews of fishing quota programs (Gulf of Mexico Fishery Management Council, 2013; Pacific Fishery Management Council and National Marine Fisheries Service, 2010). This context can only be navigated by direct collaboration between modelers, decision makers, and stakeholders.

Collaborative efforts to address specific management actions should provide the opportunity to weigh tradeoffs among ecosystem goals, and model outputs that are situated in a tradeoff framework will be more applicable to decisions than those offering only one way forward (Link, 2010). An example is the use of a management strategy evaluation framework, which involves using a stakeholder process to identify objectives and potential management strategies. This approach determines which models and other tools are available to evaluate the alternative strategies, the outcomes of which can be compared for their performance against stated objectives, so long as standards for model performance against stated objectives are identified (Punt, 2017). Where objectives have been identified, scenarios can be designed to generate outputs that are linked to these objectives so that the multi-objective performance of alternative management actions can be compared (e.g., Fulton et al., 2014; Punt et al., 2016a). Thus, the knowledge is applicable to decisions related to achieving objectives.

#### Illustration

To achieve its management objective of providing adequate forage for dependent species, the PFMC has the power to set limits on the U.S. harvest of Pacific sardines. The decision to be made, therefore, is how much sardine to harvest in U.S. waters. The ecological impacts of this decision relate to the long-term sustainability of sardines and the predators that depend on them. The applicability of models relevant to this problem was guided by a committee of the PFMC (PFMC, 2013), whose members reviewed the models existing before the OMF case study, found them inadequate, and provided guidance on specifications for new models that would be required for policy applications.

To support management objectives under the current harvest control rule, the OMF focused on existing harvest levels by evaluating impacts on predators under ranges of sardine biomass estimated using a model of intermediate complexity for ecosystem assessment (MICE; Plaganyi et al., 2014). The MICE projects population dynamics of: Pacific sardine, Northern anchovy Engraulis mordax, and “other forage”; predators brown pelican Pelecanus occidentalis and California sea lion Zalophus californianus; and three fisheries (Canada, United States, and Mexico) in a spatially-, temporally- and age-explicit framework with environmental stochasticity driving sardine recruitment (Punt et al., 2016b). The models explored impacts of current sardine management on forage fish predators using model-predicted sardine biomass potentially resulting from the current harvest control rules. The final harvest limit for the US fishery for sardine is the minimum of the acceptable biological catch (the exploitation rate corresponding to maximum sustainable yield, HMSY, multiplied by the estimate of the biomass of sardine aged 1 year and older, reduced by a buffer to account for scientific uncertainty) and the harvest guideline (HG). The latter is calculated using the formula:

$HG= (B1+−150,000)HMSY0.87$

where B1+ is the estimate of the biomass of sardine aged 1 year and older, and 0.87 is a factor to account for the assumed proportion of sardine in US waters. The value of HMSY depends on sea surface temperature collected by the CalCOFI program (Punt et al., 2016b).

To complement the MICE, we applied a multi-model approach (Kaplan et al., 2018) to evaluate the broader ecosystem-level impacts of the range of sardine abundances projected by the MICE under the current control rule (Table 1). This multi-model approach (Figure 2) involved application of a simple food web model (Koehn et al., 2016) as well as a full ecosystem Atlantis model (Kaplan et al., 2017). The food web model allowed calculation of indicators of prey importance in the food web and predator dependence upon those prey groups. The multi-model approach allowed consideration of how a broader set of ecological and economic objectives responds to alternative levels of sardine abundance (Figure 2), and of how model parameterization and structural uncertainty influence results. In the case of Atlantis and the MICE, economic performance was measured simply, in terms of sardine catch.

Table 1

Sardine biomasses resulting from realistic management scenarios using two ecosystem models. DOI: https://doi.org/10.1525/elementa.338.t1

MICEAtlantisc

Sardine biomass categories by range of model predictionsa (tons)Proportion of model runsbSardine biomass (tons)Multiplier of fishing mortality

2.8–121 million 0.42 3.7 million 0.0
1.4–2.8 million 0.16 1.9 million 0.5
628,000–1.4 million 0.12 960,000 1.0
171,000–628,000 0.10 290,000 2.0
26,000–171,000 0.07 50,000 4.0
0–26,000 0.13 3,000 8.0
MICEAtlantisc

Sardine biomass categories by range of model predictionsa (tons)Proportion of model runsbSardine biomass (tons)Multiplier of fishing mortality

2.8–121 million 0.42 3.7 million 0.0
1.4–2.8 million 0.16 1.9 million 0.5
628,000–1.4 million 0.12 960,000 1.0
171,000–628,000 0.10 290,000 2.0
26,000–171,000 0.07 50,000 4.0
0–26,000 0.13 3,000 8.0

a Biomass outputs by the MICE are the results of 100,000 stochastic realizations using the existing harvest rule (Punt et al., 2016b); results are binned into biomass categories.

b Relative proportion or frequency of biomass categories.

c To achieve sardine biomasses that also fell within the MICE biomass categories, the deterministic Atlantis model (Kaplan et al., 2017) was forced by constant sardine fishing rates, yielding biomasses based on the fishing rate multipliers.

Figure 2

Trade-offs among multiple sardine ecosystem objectives explored by multiple models. Kaplan et al. (2018) combined the MICE and Atlantis model results to evaluate effects of declines in sardine abundance on predators and the ecosystem. Here, nodes on the seven axes represent multiple objectives evaluated by the MICE (“Pelican” axis) or the Atlantis model (all other axes). Colored lines are different levels of sardine abundance (see Table 1), which lead to the scores for objectives indicated on each axis. All objectives are standardized to a range of 0–1. Scores for all biomass levels are equal to 1 for Sea lions, Pelagic feeding seabirds, and Piscivores. Scores for all but the two lowest biomass levels are equal to 1 for Pelicans from the MICE. The scores for the two lowest biomass levels are equal to 0 for Sardines. DOI: https://doi.org/10.1525/elementa.338.f2

Figure 2

Trade-offs among multiple sardine ecosystem objectives explored by multiple models. Kaplan et al. (2018) combined the MICE and Atlantis model results to evaluate effects of declines in sardine abundance on predators and the ecosystem. Here, nodes on the seven axes represent multiple objectives evaluated by the MICE (“Pelican” axis) or the Atlantis model (all other axes). Colored lines are different levels of sardine abundance (see Table 1), which lead to the scores for objectives indicated on each axis. All objectives are standardized to a range of 0–1. Scores for all biomass levels are equal to 1 for Sea lions, Pelagic feeding seabirds, and Piscivores. Scores for all but the two lowest biomass levels are equal to 1 for Pelicans from the MICE. The scores for the two lowest biomass levels are equal to 0 for Sardines. DOI: https://doi.org/10.1525/elementa.338.f2

Close modal

Improved socio-economic metrics can also provide context for the ecological and fishery outputs from the Atlantis model or the MICE, as part of this multi-model approach (Figure 3). One such suite of metrics, coastal community vulnerability indices, are generalized socioeconomic vulnerability metrics for human communities that rely on the marine ecosystem. These metrics have had broad application for defining the risks to human communities from exogenous shocks such as disasters (hurricanes, oil spills, fishery collapses) and have been employed in the US Northeast and Southeast (Jacob et al., 2012; Jepson and Colburn, 2013), Alaska (Himes-Cornell and Kasperski, 2015), at the US national level (http://www.st.nmfs.noaa.gov/humandimensions/social-indicators/map), and in Mexico (Morzaria-Luna et al., 2014). The Community Social Vulnerability Index is derived from seven community-level social and economic indices of community characteristics: personal disruption, poverty, population composition, housing characteristics, labor force structure, natural resource labor force, and wealth and education. These methods are being applied to fishery management nationally (Gulf of Mexico Fishery Management Council, 2013) and have been identified as applicable tools for the California Current.1 Thus, one step toward the aspirational goal of assessing social impacts of changing harvests (e.g., as projected by ecosystem models) is to consider those harvests in the context of social and economic vulnerability metrics familiar to fishery decision makers.

Figure 3

Social vulnerability composite score for communities with substantial sardine landings compared to other communities. Solid bars indicate social vulnerability composite scores for communities with substantial sardine landings, based on the outputs from Atlantis and the MICE from the OMF working group. Open bars indicate social vulnerability measures of five other communities, ranging from the community with lowest vulnerability measure (El Granada) to the highest vulnerability measure (Neah Bay). Communities with substantial sardine landings are those that contain ports with annual average sardine landings of >1000 metric tons for 2004–2013 (Pacific Fisheries Information Network, 2016). The ports of San Pedro and Terminal Island are contained within Los Angeles. CDP indicates census designated place. See Appendix S1 for detailed methods. DOI: https://doi.org/10.1525/elementa.338.f3

Figure 3

Social vulnerability composite score for communities with substantial sardine landings compared to other communities. Solid bars indicate social vulnerability composite scores for communities with substantial sardine landings, based on the outputs from Atlantis and the MICE from the OMF working group. Open bars indicate social vulnerability measures of five other communities, ranging from the community with lowest vulnerability measure (El Granada) to the highest vulnerability measure (Neah Bay). Communities with substantial sardine landings are those that contain ports with annual average sardine landings of >1000 metric tons for 2004–2013 (Pacific Fisheries Information Network, 2016). The ports of San Pedro and Terminal Island are contained within Los Angeles. CDP indicates census designated place. See Appendix S1 for detailed methods. DOI: https://doi.org/10.1525/elementa.338.f3

Close modal

### 3. Knowledge is parsimonious

#### Illustration

In the case of the OMF Pacific sardine working group, the relevant management objective is to “provide adequate forage for dependent species” (Pacific Fishery Management Council, 2016a), and four models were employed to address this objective. The simplest model in terms of its ecosystem components was a single-species model developed to evaluate the impacts of existing sardine harvest rules on fishery- and sardine-specific objectives, such as mean sardine biomass, and probabilities of falling above or below certain biomass thresholds (Figure 1; Hurtado-Ferro and Punt, 2014). This model included the influence of an environmental variable, temperature, on recruitment. The next most complicated model was the static food web model constructed specifically for California Current forage fish with high trophic resolution (Koehn et al., 2016). This model was used to identify predators most likely to be highly dependent upon sardines and therefore potentially impacted by sardine harvest, and to identify forage species in addition to sardine that are important prey in the California Current.

The MICE model described above was intentionally constructed to be a parsimonious approach to addressing the particular questions at hand. This approach is advantageous in evaluating the impacts of management actions because MICE are tactical; i.e., they can be structured to target specific management questions or objectives, they contain parameters that can be adjusted by fitting model outputs to observations, and they are not very intensive computationally. Fast computation times (on the order of hours versus days) mean that MICE are well-suited to exploration of a range of specific uncertainties via scenarios and across multiple replicates associated with stochasticity or process/observation error (Plagányi et al., 2014). In this case the MICE expanded upon the single-species model by incorporating climate, oceanography, predators and other fisheries. Climatic and oceanographic factors were accounted for in the MICE by modelling the slope at the origin of the stock-recruitment relationship as a function of an oceanographic driver. The function of the driver is to generate simulated variation in sardine abundance similar in magnitude to that observed in sardine scales contained in sediment core samples from the southern California Bight, prior to exploitation. Atlantis end-to-end ecosystem model (Kaplan et al., 2017) was conducted, following its initial review by an external panel of the Pacific Fishery Management Council (2014b). This “end-to-end” Atlantis model included food web dynamics, oceanography, biogeochemistry, and fisheries (Figure 1), and the update refined the parameterization to explore sardine ecology and management questions.

### 4. Knowledge is co-produced

For many years, scientists aiming to produce usable knowledge have followed the “loading dock” model, wherein scientific data are produced and results are published in the peer-reviewed literature, where they remain (somewhat) available to anyone interested in applying them to management. The potential consequence of this approach is that decision makers, when faced with a decision and seeking technical input, can find that scientific outputs are mismatched to the decision point. Increasing attention is being paid to replacing these standard practices with a process of co-production of knowledge: the co-identification of research questions and outputs between scientists and decision makers or managers, informed by known decision points and available, relevant scientific tools; and the extended, iterative progress towards shared vision and action (Cash et al., 2006; Clark et al., 2016b; Nel et al., 2016). Research should be designed and conducted in consideration of possible linked management actions to overcome the gaps between knowledge production or research on the one hand, and action or management on the other (Jarvis et al., 2015). In this way, the ensuing technical inquiry can be shaped to target a specific decision: outputs match management-relevant variables, timeframes are relevant for decision landscapes, and points of entry for technical information into the policy process are identified. Likewise, through direct collaboration with scientists, decision makers can bracket their needs by what information can be produced, within what timeframe, and bearing what level of uncertainty.

Knowledge co-production often also includes stakeholders, whose participation can influence the data or knowledge used for technical analyses, and who will weight the potential outcomes during the decision process. Furthermore, as the impacts of decisions on multiple ecosystem components are weighed against each other, expertise across multiple disciplines is required within the group. Natural scientists can be joined, for example, by social scientists to assist in defining and measuring the human dimensions of ecosystems, such as social vulnerability (Figure 3), or by economists to describe and evaluate the influence of supply chains on the system; a multitude of additional examples exist. Often, the use of knowledge brokers or “boundary spanners” can be employed (Bednarek et al., 2016). Knowledge co-production is a vital component of actionable science, though barriers to applying the resulting outputs to actions still exist (Weichselgartner and Kasperson, 2010).

#### Illustration

Motivated by the idea that knowledge co-production improves the usefulness of science, the OMF convenes working groups comprised of modelers, managers, empiricists and stakeholders to address single issues of ocean management concern. By including managers or decision makers along with modelers and other experts, the management questions can be framed together, the models can be structured together to ensure that they will address the relevant decision(s), and other stakeholders can influence what information is used in the modeling activities. The working groups are diverse in their membership, and convene groups of individuals who otherwise would not have the opportunity to collaborate.

Quantifying co-production is a challenge. One approach is to measure the diversity of the stakeholders participating in the knowledge production activity. To identify the diversity of the OMF network of collaborators, we collected data from the OMF Pacific herring working group participants on their collaboration networks, as defined by their publications. The Pacific Herring working group of 20 individuals included academic ecologists, food web and stock assessment modelers, anthropologists, political scientists, economists, indigenous tribal elders, First Nations scientists, resource management agency scientists, fishing industry executives, and commercial fishermen.

We conducted a bibliographic network analysis, mapping the 1,039-person co-authorship network based on the publication records (in Web of Science) of working group members, prior to jointly producing manuscripts as part of the working group (Figure 4). Our analysis showed the dominance of individuals publishing in the life sciences, as well as the existence of many interdisciplinary participants publishing in life and social sciences. We also found multiple nodes (individuals) and clusters of individuals that, prior to participation in the OMF, were disconnected from a central, highly connected aggregation of individuals. Those disconnected nodes illustrate the impact of the OMF: the map shows disconnected individuals and their networks as separate from the rest of the OMF network. A shared publication links all isolated nodes into the network (Supplementary Figure S1), and the newly created network includes not only a diverse set of stakeholders, but individuals and their collaborators who were previously not linked, despite having expertise on the same topic, Pacific herring. The new network formed after a joint publication resulted in an increase in the number of authors in the network, from 1,039 to 1,044, and more author-to-author links in the group, from 5,894 to 6,097. In addition, the total number of links (the number of individual manuscript-based links between pairs of authors) increased from 8,049 to 8,259. Thus, the OMF broadened the network, linking previously disconnected individuals, and increased the number of links between individuals and the average between-person links in the network, strengthening ties within the network.

Figure 4

OMF Herring working group network. Network of coauthors based upon all Web of Science-listed publications of the OMF Herring working group members. Each circle represents an individual; circle size indicates number of links (i.e., shared publications), and color represents the research area that is the focus of the publication, based on Web of Science categorizations. Gray circles indicate a publication that encompasses more than three research areas. Distance between circles is relatedness, based upon publications: the more shared publications or co-authors, the closer are two circles. Lines represent shared publications. Clusters are formed by individual people closely related by shared publications. A total of 5,894 links exist in the network. See Appendix S1 for detailed methods. DOI: https://doi.org/10.1525/elementa.338.f4

Figure 4

OMF Herring working group network. Network of coauthors based upon all Web of Science-listed publications of the OMF Herring working group members. Each circle represents an individual; circle size indicates number of links (i.e., shared publications), and color represents the research area that is the focus of the publication, based on Web of Science categorizations. Gray circles indicate a publication that encompasses more than three research areas. Distance between circles is relatedness, based upon publications: the more shared publications or co-authors, the closer are two circles. Lines represent shared publications. Clusters are formed by individual people closely related by shared publications. A total of 5,894 links exist in the network. See Appendix S1 for detailed methods. DOI: https://doi.org/10.1525/elementa.338.f4

Close modal

Opportunities exist for knowledge co-production to improve ocean management efforts, as most relevant research, ranging from fisheries management to human sense of place, is currently conducted in isolated silos of disciplinary homogeny versus in cross-disciplinary groups. To illustrate the potential for improvement, we surveyed the members of the OMF Pacific Herring and Pacific Sardine working groups (see Appendix S1 for methods) to identify the ocean management issues that could most benefit from multi-model approaches and knowledge co-production. We then conducted a network analysis of the 1,000 most connected authors from the 500 most recently published papers in the top nine ocean management issues identified (Figure 5). The analysis shows a core of individuals publishing in two or more themes that are disconnected from clusters of other closely-linked individuals publishing in single themes, especially fisheries, eutrophication, and governance. In other words, interdisciplinary (i.e., working across topics) researchers are largely disconnected from individuals working only within a single discipline or topic. Many scientists are not exposed to interdisciplinary research opportunities. Furthermore, within topics, or individual ocean management themes, individual tightly-connected groups are very distant and disconnected from each other. This disconnect may occur between research groups of different disciplines, such as social and natural scientists, or of physicists and environmental scientists, who conduct research in the same field but not often with each other. The existence of clusters, plus individuals who are wholly disconnected from everyone else, represents opportunity for increased co-production of knowledge across disciplinary silos to inform ocean decision-making.

Figure 5

Network analysis of individuals publishing in the top nine OMF case study opportunities. Each circle represents an individual researcher publishing in one or more topic. Clusters are formed by individual people closely related by shared publications. Circle color indicates publication topic; circle size indicates number of links (shared publications). Distance between circles indicates degree of relatedness, according to the bibliographic records (connecting lines have been omitted for clarity). Clusters of circles are closely-related groups of people, as indicated by their publications. Group A includes climate change, habitat loss, toxics; Group B includes governance, fisheries, eutrophication, cumulative impacts, and marine spatial planning. See Appendix S1 for detailed methods. DOI: https://doi.org/10.1525/elementa.338.f5

Figure 5

Network analysis of individuals publishing in the top nine OMF case study opportunities. Each circle represents an individual researcher publishing in one or more topic. Clusters are formed by individual people closely related by shared publications. Circle color indicates publication topic; circle size indicates number of links (shared publications). Distance between circles indicates degree of relatedness, according to the bibliographic records (connecting lines have been omitted for clarity). Clusters of circles are closely-related groups of people, as indicated by their publications. Group A includes climate change, habitat loss, toxics; Group B includes governance, fisheries, eutrophication, cumulative impacts, and marine spatial planning. See Appendix S1 for detailed methods. DOI: https://doi.org/10.1525/elementa.338.f5

Close modal

### 5. Knowledge is amplifying

The final characteristic of model-based knowledge that is ready for action is that it amplifies existing empirical information, and information derived from individual models, through cross pollination among multiple modelers and between modelers and other experts. At its foundation, a multi-model approach is an extension of the general scientific strategy of multiple hypotheses (Chamberlin, 1890), and a suite of tools has been developed in response to this strategy for evaluating among competing hypotheses (Burnham and Anderson, 2004; Burnham et al., 2011; Grueber et al., 2011). The power of bringing multiple models together is not only in reducing uncertainty by varying model complexity and breadth, but also in modelers learning from each other through iteration, in more effective engagement with stakeholders, and in being able to address both tactical and strategic concerns (Fulton et al., 2015). The multi-model process provides richness about how the system should be modeled. Dialog among modelers also daylights individual biases and distortions in model building, which leads to being a better modeler.

Additional wisdom is also gained by engaging with experts with extensive experience in the ecosystem, or with the focal species, who serve to represent knowledge about the ecosystem. Models can amplify such system understanding. Developing models through iterative interaction with knowledge holders can improve models by identifying which system dynamics are most likely to impact the issues at hand and which should be prioritized for inclusion in the models, what additional sources of data and analyses can be used to inform the models, how to validate model outputs, etc. (Fulton et al., 2015). Models can amplify ideas and different types of knowledge, including traditional knowledge, arising from other stakeholders and sources. The value of traditional knowledge, including traditional ecological knowledge, in ecological understanding and resource management is widely recognized; combining traditional knowledge with scientific approaches can amplify the power of each individual form of knowledge (Huntington, 2000; Moller et al., 2004), and incorporating traditional knowledge into models can effectively inform conservation and management (Butler et al., 2012; Polfus et al., 2014).

#### Illustration

In the OMF Pacific sardine working group, diets of predators in two models, the MICE (Punt et al., 2016b) and the Atlantis ecosystem model (Kaplan et al., 2017), were informed by the highly resolved Ecopath food web model of the California Current ecosystem (Koehn et al., 2016). Data collection and initial model parameterization for the Atlantis model and the food web model were a collaborative effort (with authors shared across papers). This effort did not necessarily force parallel interpretation of data sources or parallel model structure, but provided some efficiency to the efforts and some common language before the models were brought to broader participants. The outputs of the MICE, sardine abundance, were also used to structure the Atlantis model outputs, allowing for an apples-to-apples comparison of model predictions by managers. Ecopath information was directly passed to the MICE, including being used to prioritize the predators in the MICE. The MICE and Ecopath results both led to realization of the importance of anchovy, which was highlighted in an Atlantis manuscript (Kaplan et al., 2017).

The Herring working group also employed models that amplified messages and findings from other models and sources of knowledge. The Herring working group evaluated the performance of alternative management actions against social, ecological and economic objectives in a management strategy evaluation framework, using an operating model of herring population dynamics internally linked to a stock assessment model. This stock assessment model was structured to mimic a separately developed, novel, spatially-structured stock assessment model (Punt et al., 2018). These two models were developed through consultation among modelers within the OMF. Another model evaluated the consequences of harvest under scenarios of socially learned migration behavior in herring, the presence of which was informed by traditional ecological knowledge (MacCall et al., 2018), amplifying existing knowledge about herring not previously incorporated into models to investigate the impacts of management decisions.

Development of the Ecopath, MICE, and Atlantis models involved extensive synthesis of existing empirical and modeling studies, but also additional expert knowledge generated within the Ocean Modeling Forum. For example, improvements during the OMF informed the effort to represent sardine specifically in the California Current Atlantis model, disaggregating them from other forage fish species, as had been the approach in the earlier versions of the model (Horne et al., 2010; Kaplan et al., 2013). This disaggregation made the model more relevant for the present case study, while addressing improvements to ecosystem models that were requested by the Pacific Fishery Management Council (2013) and an external review panel (Pacific Fishery Management Council, 2014b; Kaplan et al., 2017). The representation of sardine included refining the diet information, first taken from individual-based models contributed by OMF members (Rose et al., 2015), then modified based on expert knowledge within the Forum that suggested reliance on filter feeding and higher consumption of phytoplankton by adult sardines (Figure 6), also supported by studies from other ecosystems (van der Lingen et al., 2006, 2009). Another example of including expert knowledge from the OMF involved parameterization of sardine migration patterns in both Atlantis and the MICE based on unpublished modeling of a working group member (summarized in Punt et al., 2016b, their Appendix A.1.1.1 and Figures A.2 and A.3). These examples are consistent with the role of expert knowledge within approaches to ecosystem-based fishery management (Smith et al., 2007).

Figure 6

Diets of juvenile and adult sardine in Atlantis ecosystem model. Discussion during the Ocean Modeling Forum first centered around parameterizing Atlantis using diets from a recent individual based model (Rose et al., 2015; bottom panels), with subsequent addition of phytoplankton to the diets based on expert opinion (top panels). Results are from the first year of a 50-year Atlantis model run. DOI: https://doi.org/10.1525/elementa.338.f6

Figure 6

Diets of juvenile and adult sardine in Atlantis ecosystem model. Discussion during the Ocean Modeling Forum first centered around parameterizing Atlantis using diets from a recent individual based model (Rose et al., 2015; bottom panels), with subsequent addition of phytoplankton to the diets based on expert opinion (top panels). Results are from the first year of a 50-year Atlantis model run. DOI: https://doi.org/10.1525/elementa.338.f6

Close modal

We have illustrated five characteristics of model-produced knowledge that make such knowledge suitable for use in ocean management and decision-making. What, then, have been the impacts of such model-produced advice? Quantifying impact is one of the greatest challenges facing practitioners working broadly at the science-policy boundary (Lyall et al., 2004; Meagher and Lyall, 2013). While the holy grail for applied scientists has long been direct evidence that their science is being used to improve concrete outcomes, such as (for conservation scientists) more habitat protected, improved water quality, or increased fish biomass, the direct linking of science to such impacts is difficult and rare. In addition to these direct instrumental impacts of research on policy and practice decisions (Meagher and Lyall, 2013), a broader suite of impacts of research on decisions can be measured. These include conceptual impacts, where research changes thinking or raises consciousness, and capacity-building, meaning education, training, or other forms of collaboration. Other processes can also be impacted: attitudes can be changed, especially towards knowledge exchange; and enduring connectivity, especially between researchers and knowledge users, can be established (Meagher and Lyall, 2013).

As illustrated above, the Sardine working group focused on developing parsimonious models (e.g., a MICE) that aspired to address ecosystem and especially food web impacts of sardine harvest, and that were applicable to management (i.e., evaluated impacts of the existing sardine harvest rule on the food web). These models may ultimately have instrumental impacts, i.e., changes to harvest practices to support sardine-dependent predators, but those impacts are likely to occur over longer periods of time. In the meantime, we can track conceptual impacts as the decision-making body considers the models in its deliberations: the subgroup of the PFMC focused on sardines reported its intention to track and use these models to examine the impacts of harvest on the food web (Pacific Fishery Management Council, 2016b), and a modeling update was subsequently requested by PFMC and presented by an OMF working group member (Pacific Fishery Management Council, 2016c).

The Herring working group, multi-stakeholder and interdisciplinary, co-produced knowledge related to the consequences of herring fisheries for social, economic, and ecological objectives, and amplified traditional knowledge in the formulation of models to evaluate potential outcomes of management decisions. This process has impacted attitudes about knowledge exchange, as noted by Harvey Kitka, a member of the Sitka Tribe of Alaska’s Herring Committee and a member of the Herring working group:

“This group has been very interesting. I and my group, especially the Herring Committee, we figured for the longest time that if we can get traditional knowledge and science to work together, it’ll probably be the best thing that’s ever happened.”

Before the Herring working group was formed, exchange of information used in decision-making about herring and herring fisheries was limited between fisheries management agencies and traditional knowledge holders, which was a source of conflict between these groups (Hume, 2015; Welch, 2015; Woolsey, 2015). Testimony from Herring working group members demonstrated that participation in the co-production of model-based knowledge served to change attitudes about the potential for various types of knowledge to be used in models and improved collaborations, opening pathways for enduring connections (J. Cleary, personal communication, S. Dressel, personal communication). Tracking impacts of knowledge and research remains a challenge for those working at the boundary between science and its use, and future efforts should focus on this broader suite of impact types.

Operationalizing ocean management for multiple benefits is challenging and requires a flexible approach (Dickey-Collas, 2014). Ocean resources are often shared among multiple sectors, and governance of large marine ecosystems is complex and heterogeneous (Crowder et al., 2006; Mahon et al., 2010), making the successful application of knowledge in decision spaces difficult. Considerable effort has been applied to developing guidance for improving the use of science in natural resource management and decision-making, including identifying features of actionable science (Cook et al., 2013; Kirchhoff et al., 2013; Clark et al., 2016b), intermediaries for bridging the knowledge-action divide (Bednarek et al., 2016, 2018; Clark et al., 2016a) and developing how-to guides for the co-production of knowledge (Beier et al., 2017). Such thinking can be extended to encompass the challenges and opportunities for linking model-based knowledge to ocean management. The Ocean Modeling Forum has developed one framework, described here, for linking model-based advice to ocean management decision-making; there are likely others (e.g., Lynam et al., 2016), though a full review is beyond the scope of the current paper. Unique to the OMF approach is its working group structure, flexibility to various decision settings (i.e., not designed for application to a single governance system), multi-stakeholder approach, and inclusion of decision makers as working group members. Particularly pressing is the need to connect climate science to local decision makers, and work continues to identify indicators and tools that can help to meet this need (Mauger and Kennard, 2017; Wall et al., 2017). We have determined five elements of actionable knowledge useful for linking model-based outputs to ocean management issues: knowledge is aspirational, parsimonious, applicable, co-produced, and amplifying. Such a framework can also complement large-scale investigations into data gaps and modeling needs, as has been done for terrestrial biodiversity modeling (Urban et al., 2016). Linking science to decision-making is a primary challenge for the conservation and sustainability of ocean resources, and indeed natural resources worldwide. Investing effort in ensuring that marine research is well positioned to have impact, while energy- and time-intensive, should be a primary consideration for ocean scientists.

We thank two anonymous reviewers, C Harvey, and J Samhouri for their insightful and helpful comments on earlier drafts of this manuscript. We acknowledge the contributions by the members of the OMF Pacific sardine and Pacific herring working groups, as well as the guidance of our advisory board members. We particularly appreciate Kai Lee for his vision and support in the genesis of the Ocean Modeling Forum, and its development thus far.

The Ocean Modeling Forum is a program of the University of Washington College of the Environment and in collaboration with The Nature Conservancy of Washington. The Ocean Modeling Forum and TBF are supported by the David & Lucile Packard Foundation and the Pew Charitable Trusts. Atlantis model development benefited greatly from a grant to ICK from the NOAA Ocean Acidification Program and National Centers for Coastal Ocean Science. Contributing support for this work was provided by the National Oceanic and Atmospheric Administration’s Integrated Ecosystem Assessment (NOAA IEA) Program. This paper is NOAA IEA program contribution #2018.

The authors have no competing interests to declare.

• Contributed to conception and design: TBF, PSL, AEP

• Contributed to acquisition of data: TBF, PSL, AEP, IK, KN, AV

• Contributed to analysis and interpretation of data: TBF, PSL, AEP, IK, KN, AV

• Drafted and/or revised the article: TBF, PSL, AEP, IK

• Approved the submitted version for publication: TBF, PSL, AEP, IK, KN, AV

1
Acheson
,
JM
.
2006
.
Institutional failure in resource management
.
Annu Rev Anthropol
35
(
1
):
117
134
. DOI:
2
Ascough
,
J
,
Maier
,
H
,
Ravalico
,
J
and
Strudley
,
M
.
2008
.
Future research challenges for incorporation of uncertainty in environmental and ecological decision-making
.
Ecol Model
219
(
3
):
383
399
. DOI:
3
Ban
,
NC
,
Bax
,
NJ
,
Gjerde
,
KM
,
Devillers
,
R
,
Dunn
,
DC
,
Dunstsan
,
PK
,
Hobday
,
AJ
,
Maxwell
,
SM
,
Kaplan
,
DM
,
Pressey
,
RL
,
Ardron
,
JA
,
Game
,
ET
and
Halpin
,
PN
.
2014
.
Systematic conservation planning: A better recipe for managing the high seas for biodiversity conservation and sustainable use
.
Conserv Lett
7
(
1
):
41
54
. DOI:
4
Bednarek
,
AT
,
Shouse
,
B
,
Hudson
,
CG
and
Goldburg
,
R
.
2016
.
Science-policy intermediaries from a practitioner’s perspective: The Lenfest Ocean Program experience
.
Sci Publ Policy
43
(
2
):
291
300
. DOI:
5
Bednarek
,
AT
,
Wyborn
,
C
,
Cvitanovic
,
C
,
Meyer
,
R
,
Colvin
,
RM
,
,
PFE
,
Close
,
SL
,
Curran
,
K
,
Farooque
,
M
,
Goldman
,
E
,
Hart
,
D
,
Mannix
,
H
,
McGreavy
,
B
,
Parris
,
A
,
Posner
,
S
,
Robinson
,
C
,
Ryan
,
M
and
Leith
,
P
.
2018
.
Boundary spanning at the science–policy interface: The practitioners’ perspectives
.
Sustain Sci
13
:
1175
1183
. DOI:
6
Beier
,
P
,
Hansen
,
LJ
,
Helbrecht
,
L
and
Behar
,
D
.
2017
.
A how-to guide for coproduction of actionable science
.
Conserv Lett
10
(
3
):
288
296
. DOI:
7
Bever
,
AJ
and
MacWilliams
,
ML
.
2013
.
Simulating sediment transport processes in San Pablo Bay using coupled hydrodynamic, wave, and sediment transport models
.
Mar Geol
345
:
235
253
. DOI:
8
Boomer
,
KMB
,
Weller
,
DE
,
Jordan
,
TE
,
,
L
,
Liu
,
Z-J
,
Reilly
,
J
,
Shenk
,
G
and
Voinov
,
AA
2013
.
Using multiple watershed models to predict water, nitrogen, and phosphorus discharges to the Patuxent Estuary
.
J Am Water Resour Assoc
49
(
1
):
15
39
. DOI:
9
Burnham
,
KP
and
Anderson
,
DR
.
2004
.
Multimodel inference: Understanding AIC and BIC in model selection
.
Sociol Method Res
33
:
261
304
. DOI:
10
Burnham
,
KP
,
Anderson
,
DR
and
Huyvaert
,
KP
.
2011
.
AIC model selection and multimodel inference in behavioral ecology: Some background, observations, and comparisons
.
Behavioral Ecology and Sociobiology
65
:
23
35
. DOI:
11
Butler
,
J
,
Tawake
,
A
,
Skewes
,
T
,
Tawake
,
L
and
McGrath
,
V
.
2012
.
Integrating traditional ecological knowledge and fisheries management in the Torres Strait, Australia: The catalytic role of turtles and dugong as cultural keystone species
.
Ecol Soc
17
(
4
). DOI:
12
Chamberlin
,
TC
.
1890
.
The method of multiple working hypotheses
.
Science
15
:
92
96
. (reprinted in Science 148: 754–759 [1965]). DOI:
13
Clark
,
WC
,
Tomich
,
TP
,
van Noordwijk
,
M
,
Guston
,
D
,
Catacutan
,
D
,
Dickson
,
NM
and
McNie
,
E
.
2016a
.
Boundary work for sustainable development: Natural resource management at the Consultative Group on International Agricultural Research (CGIAR)
.
113
(
17
):
4615
4622
. DOI:
14
Clark
,
WC
,
van Kerkhoff
,
L
,
Lebel
,
L
and
Gallopin
,
GC
.
2016b
.
Crafting usable knowledge for sustainable development
.
113
(
17
):
4570
4578
. DOI:
15
Collie
,
JS
,
Botsford
,
LW
,
Hastings
,
A
,
Kaplan
,
IC
,
Largier
,
JL
,
Livingston
,
PA
,
Plagányl
,
É
,
Rose
,
KA
,
Wells
,
BK
and
Werner
,
FE
.
2016
.
Ecosystem models for fisheries management: Finding the sweet spot
.
Fish Fish
17
(
1
):
101
125
. DOI:
16
Collie
,
JS
,
Gifford
,
DJ
and
Steele
,
JH
.
2009
.
End-to-end foodweb control of fish production on Georges Bank
.
ICES J Mar Sci
66
(
10
):
2223
2232
. DOI:
17
Cook
,
CN
,
Mascia
,
MB
,
Schwartz
,
MW
,
Possingham
,
HP
and
Fuller
,
RA
.
2013
.
Achieving conservation science that bridges the knowledge–action boundary
.
Conserv Biol
27
(
4
):
669
678
. DOI:
18
Council on Environmental Quality
.
2010
. Final Recommendations of the Interagency Ocean Policy Task Force.
Washington, D. C.
:
Council on Environmental Quality
. Available at: http://www.whitehouse.gov/administration/eop/ceq/initiatives/oceans. Accessed 1 September 2010.
19
Crowder
,
LB
,
Osherenko
,
G
,
Young
,
OR
,
Airamé
,
S
,
Norse
,
EA
,
Baron
,
N
,
Day
,
JC
,
Douvere
,
F
,
Ehler
,
CN
,
Halpern
,
BS
,
Langdon
,
SJ
,
McLeod
,
KL
,
Ogden
,
JC
,
Peach
,
RE
,
Rosenber
,
AA
and
Wilson
,
JA
.
2006
.
Sustainability – Resolving mismatches in US ocean governance
.
Science
313
(
5787
):
617
618
. DOI:
20
Dereynier
,
YL
.
2014
.
US fishery management councils as ecosystem-based management policy takers and policymakers
.
Coast Manage
42
(
6
):
512
530
. DOI:
21
Dickey-Collas
,
M
.
2014
.
Why the complex nature of integrated ecosystem assessments requires a flexible and adaptive approach
.
ICES J Mar Sci
71
(
5
):
1174
1182
. DOI:
22
Ehler
,
CN
and
Douvere
,
F
.
2009
. Marine Spatial Planning: A step-by-step approach toward ecosystem-based management.
Intergovernmental Oceanographic Commission and Man and the Biosphere Programme
53
.
Paris, France
:
UNESCO
.
23
European Commission
.
2009
. An integrated Maritime Policy for the European Union.
Brussels, Belgium
:
European Commission
. DOI:
24
Fischhoff
,
B
and
Scheufele
,
DA
.
2014
.
The science of science communication II
.
111
:
13583
13584
. DOI:
25
Foley
,
NN
,
Halpern
,
BS
,
Micheli
,
R
,
Armsby
,
MH
,
Caldwell
,
MR
,
Crain
,
CM
,
Prahler
,
E
,
Rohr
,
N
,
Sivas
,
D
,
Beck
,
MW
,
Car
,
MH
,
Crowder
,
LB
,
Duffy
,
JE
,
Hacker
,
SD
,
McLeod
,
KL
,
Palumbi
,
SR
,
Peterson
,
CH
,
Regan
,
HM
,
Ruckelshaus
,
MH
,
Sandifer
,
PA
and
Steneck
,
RS
.
2010
.
Guiding ecological principles for marine spatial planning
.
Mar Pol
34
(
5
):
955
966
. DOI:
26
Fulton
,
EA
,
Boschetti
,
F
,
Sporcic
,
M
,
Jones
,
T
,
Little
,
LR
,
Dambacher
,
JM
,
Gray
,
R
,
Scott
,
R
and
Gorton
,
R
.
2015
.
A multi-model approach to engaging stakeholder and modellers in complex environmental problems
.
Environ Sci Policy
48
:
44
56
. DOI:
27
Fulton
,
EA
,
,
JS
,
Kaplan
,
IC
,
Savina-Rolland
,
M
,
Johnson
,
P
,
Ainsworth
,
C
,
Horne
,
P
,
Gorton
,
R
,
Gamble
,
RJ
,
Smith
,
and
Smith
,
DC
.
2011
.
Lessons in modelling and management of marine ecosystems: The Atlantis experience
.
Fish Fish
12
(
2
):
171
188
. DOI:
28
Fulton
,
EA
,
Smith
,
and
Johnson
,
CR
.
2003
.
Effect of complexity on marine ecosystem models
.
Mar Ecol Prog Ser
253
:
1
16
. DOI:
29
Grueber
,
CE
,
Nakagawa
,
S
,
Laws
,
RJ
and
Jamieson
,
IG
.
2011
.
Multimodel inference in ecology and evolution: Challenges and solutions
.
J Evol Biol
24
:
699
711
. DOI:
30
Gulf of Mexico Fishery Management Council
.
2013
. Red Snapper Individual Fishing Quota Program 5-year Review.
Tampa FL
:
Gulf of Mexico Fishery Management Council
. http://archive.gulfcouncil.org/docs//amendments/Red%20Snapper%205-year%20Review%20FINAL.pdf.
31
Halpern
,
BS
,
Frazier
,
M
,
Potapenko
,
J
,
Casey
,
KS
,
Koenig
,
K
,
Longo
,
C
,
Lowndes
,
JS
,
Rockwood
,
RC
,
Selig
,
ER
,
Selkoe
,
KA
and
Walbridge
,
S
.
2015
.
Spatial and temporal changes in cumulative human impacts on the world’s ocean
.
Nat Commun
6
:
7
. DOI:
32
Hannah
,
C
,
Vezina
,
A
and
John
,
MS
.
2010
.
The case for marine ecosystem models of intermediate complexity
.
Prog Oceanogr
84
:
121
128
. DOI:
33
Harvey
,
CJ
,
Garfield
,
N
,
Williams
,
G
,
Andrews
,
K
,
Barceló
,
C
,
Barnas
,
K
,
,
S
,
Brodeur
,
R
,
Burke
,
B
,
Cope
,
J
,
deWitt
,
L
,
Field
,
J
,
Fisher
,
J
,
Greene
,
C
,
Good
,
T
,
Hazen
,
E
,
Holland
,
D
,
Jacox
,
M
,
Kasperski
,
S
,
Kim
,
S
,
Leising
,
A
,
Melin
,
S
,
Morgan
,
C
,
Munsch
,
S
,
Norman
,
K
,
Peterson
,
WT
,
Poe
,
M
,
Samhouri
,
J
,
Schroeder
,
I
,
Sydeman
,
W
,
Thayer
,
J
,
Thompson
,
A
,
Tolimieri
,
N
,
Varney
,
A
,
Wells
,
B
,
Williams
,
T
and
Zamon
,
J
.
2017a
.
Ecosystem Status Report of the California Current for 2017: A Summary of Ecosystem Indicators Compiled by the California Current Integrated Ecosystem Assessment Team (CCIEA)
.
U.S. Department of Commerce, NOAA Technical Memorandum NMFS-NWFSC-139
. DOI:
34
Harvey
,
CJ
,
Kelble
,
CR
and
Schwing
,
FB
.
2017b
.
Implementing “the IEA”: Using integrated ecosystem assessment frameworks, programs, and applications in support of operationalizing ecosystem-based management
.
ICES J Mar Sci
74
:
398
405
. DOI:
35
Harvey
,
CJ
,
Reum
,
JCP
,
Poe
,
MR
,
Williams
,
GD
and
Kim
,
SJ
.
2016
.
Using conceptual models and qualitative network models to advance integrative assessments of marine ecosystems
.
Coast Manage
44
(
5
):
486
503
. DOI:
36
Henson
,
SA
,
Beaulieu
,
C
,
Ilyina
,
T
,
John
,
JG
,
Long
,
M
,
Séférian
,
R
,
Tjiputra
,
J
and
Sarmiento
,
JL
.
2017
.
Rapid emergence of climate change in environmental drivers of marine ecosystems
.
Nat Commun
8
:
9
. DOI:
37
Himes-Cornell
,
A
and
Kasperski
,
S
.
2015
.
Assessing climate change vulnerability in Alaska’s fishing communities
.
Fish Res
162
:
1
11
. DOI:
38
Horne
,
PJ
,
Kaplan
,
IC
,
Marshall
,
KN
,
Levin
,
PS
,
Harvey
,
CJ
,
Hermann
,
AJ
and
Fulton
,
EA
.
2010
. Design and Parameterization of a Spatially Explicit Ecosystem Model of the Central California Current.
NOAA Tech
.
Memo. NMFS-NWFSC-104
,
1
140
.
39
Hume
,
M
.
2015
Apr
1
.
Heiltsuk First Nation claims victory over disputed herring fishery. The Globe and Mail
. https://www.theglobeandmail.com/news/british-columbia/heiltsuk-first-nation-claims-victory-over-disputed-herring-fishery/article23757390/.
40
Huntington
,
HG
,
Weyant
,
JP
and
Sweeney
,
JL
.
1982
.
Modeling for insights, not numbers: The experiences of the Energy Modeling Forum
.
OMEGA The Intl J of Mgmt Sci
10
(
5
):
449
462
. DOI:
41
Huntington
,
HP
.
2000
.
Using traditional ecological knowledge in science: Methods and applications
.
Ecol Appl
10
:
1270
1274
. DOI:
42
,
F
and
Punt
,
AE
.
2014
.
Revised analyses related to Pacific sardine harvest parameters
:
Report to Pacific Fishery Management Council
.
40
.
43
International Council for the Exploration of the Sea
.
2014
.
Implementing the ICES Strategic Plan 2014–2018
.
Copenhagen, Denmark
:
International Council for the Exploration of the Sea
.
44
Jacob
,
S
,
Weeks
,
P
,
Blount
,
B
and
Jepson
,
M
.
2012
.
Development and evaluation of social indicators of vulnerability and resiliency for fishing communities in the Gulf of Mexico
.
Mar Policy
29
(
10
):
16
22
.
45
Jacobs
,
K
,
Level
,
L
,
Buizer
,
J
,
,
L
,
Matson
,
P
,
McCullough
,
E
,
Garden
,
P
,
Saliba
,
G
and
Finan
,
T
.
2016
.
Linking knowledge with action in the pursuit of sustainable water-resources management
.
113
(
17
):
4591
4596
. DOI:
46
Jarvis
,
R
,
Borrelle
,
S
,
Breen
,
BB
and
Towns
,
D
.
2015
.
Conservation, mismatch and the research–implementation gap
.
Pacific Conservation Biology
21
(
2
):
105
107
. DOI:
47
Jepson
,
M
and
Colburn
,
LL
.
2013
. Development of social indicators of fishing community vulnerability and resilience in the US Southeast and Northeast regions.
NOAA Technical Memorandum NMFS-F/SPO-129
.
Department of Commerce
,
.
48
Kaplan
,
IC
,
Francis
,
TB
,
Punt
,
AE
,
Koehn
,
LE
,
Curchitser
,
E
,
,
F
,
Johnson
,
KF
,
Lluch-Cota
,
SE
,
Sydeman
,
WJ
,
Essington
,
TE
,
Taylor
,
N
,
Holsman
,
K
,
MacCall
,
and
Levin
,
PS
.
2018
.
A multi-model approach to understanding the role of Pacific sardine in the California Current food web
.
Mar Ecol Prog Ser
. DOI:
49
Kaplan
,
IC
,
Koehn
,
LE
,
Hodgson
,
EE
,
Marshall
,
KN
and
Essington
,
TE
.
2017
.
Modeling food web effects of low sardine and anchovy abundance in the California Current
.
Ecol Model
359
:
1
24
. DOI:
50
Kirchhoff
,
CJ
,
Lemos
,
MC
and
Dessai
,
S
.
2013
.
Actionable knowledge for environmental decision making: Broadening the usability of climate science
.
Annu Rev Environ Resour
38
(
1
):
393
414
. DOI:
51
Knight
,
AT
,
Cowling
,
RM
,
Rouget
,
M
,
Balmford
,
A
,
Lombard
,
AT
and
Campbell
,
BM
.
2008
.
Knowing but not doing: Selecting priority conservation areas and the research-implementation gap
.
Conserv Biol
22
(
3
):
610
617
. DOI:
52
Koehn
,
LE
,
Essington
,
TE
,
Marshall
,
KN
,
Kaplan
,
IC
,
Sydeman
,
WJ
,
Szoboszlai
,
AI
and
Thayer
,
JA
.
2016
.
Developing a high taxonomic resolution food web model of the California Current ecosystem to assess the trophic position of forage fish and their predators
.
Ecol Model
335
:
87
100
. DOI:
53
Krishnamurti
,
TN
,
Kumar
,
V
,
Simon
,
A
,
Bhardwaj
,
A
,
Ghosh
,
T
and
Ross
,
R
.
2016
.
A review of multimodel superensemble forecasting for weather, seasonal climate, and hurricanes
.
Rev Geophys
54
(
2
):
336
377
. DOI:
54
Kunseler
,
EM
,
Tuinstra
,
W
,
,
E
and
Petersen
,
AC
.
2015
.
The reflective futures practitioner: Balancing salience, credibility and legitimacy in generating foresight knowledge with stakeholders
.
Futures
66
:
1
12
. DOI:
55
Levin
,
PS
,
Breslow
,
SJ
,
Harvey
,
CJ
,
Norman
,
KC
,
Poe
,
MR
,
Williams
,
GD
and
Plummer
,
ML
.
2016
.
Conceptualization of social-ecological systems of the California Current: An examination of interdisciplinary science supporting ecosystem-based management
.
Coast Manage
44
(
5
):
397
408
. DOI:
56
Levin
,
PS
,
Essington
,
TE
,
Marshall
,
KN
,
Koehn
,
LE
,
Anderson
,
LG
,
Bundy
,
A
,
Carothers
,
C
,
Coleman
,
F
,
Gerber
,
LR
,
Grabowski
,
JH
,
Houde
,
E
,
Jensen
,
OP
,
Möllmann
,
C
,
Rose
,
K
,
Sanchirico
,
JN
and
Smith
,
.
2018
.
Building effective fishery ecosystem plans
.
Mar Policy
92
:
48
57
. DOI:
57
Levin
,
PS
,
Fogarty
,
MJ
,
Murawski
,
SA
and
Fluharty
,
D
.
2009
.
Integrated ecosystem assessments: Developing the scientific basis for ecosystem-based management of the ocean
.
PLoS Biol
7
(
1
): e1000014. DOI:
58
Levin
,
PS
,
Kelble
,
CR
,
Shuford
,
RL
,
Ainsworth
,
C
,
deReynier
,
Y
,
Dunsmore
,
R
,
Fogarty
,
MJ
,
Holsman
,
K
,
Howell
,
EA
,
Monaco
,
ME
,
Oakes
,
SA
and
Werner
,
F
2013
.
Guidance for implementation of integrated ecosystem assessments: A US perspective
.
ICES J Mar Sci
71
:
1198
1204
. DOI:
59
Lewison
,
RL
,
Hobday
,
AJ
,
Maxwell
,
S
,
Hazen
,
E
,
Hartog
,
JR
,
Dunn
,
DC
,
Briscoe
,
D
,
Fossette
,
S
,
O’Keefe
,
CE
,
Barnes
,
M
,
Abecassis
,
M
,
,
S
,
Bethoney
,
ND
,
Bailey
,
H
,
Wiley
,
D
,
Andrews
,
S
,
Hazen
,
L
and
Crowder
,
LB
.
2015
.
Dynamic ocean management: Identifying the critical ingredients of dynamic approaches to ocean resource management
.
Bioscience
65
(
5
):
486
498
. DOI:
60
,
JS
and
Browman
,
HI
.
2017
.
Operationalizing and implementing ecosystem-based management
.
ICES J Mar Sci
74
(
1
):
379
381
. DOI:
61
Lyall
,
C
,
Bruce
,
A
,
Firn
,
J
,
Firn
,
M
and
Tait
,
J
.
2004
.
Assessing end-use relevance of public sector research organisations
.
Res Policy
33
:
73
87
. DOI:
62
Lynam
,
CP
,
Uusitalo
,
L
,
Patrício
,
J
,
Piroddi
,
C
,
Queirós
,
AM
,
Teixeira
,
H
,
Rossberg
,
AG
,
Sagarminaga
,
Y
,
Hyder
,
K
,
Niquil
,
N
,
Möllmann
,
C
,
Wilson
,
C
,
Chust
,
G
,
Galparsoro
,
I
,
Forster
,
R
,
Veríssimo
,
H
,
Tedesco
,
L
,
Revilla
,
M
and
Neville
,
S
.
2016
.
Uses of innovative modeling tools within the implementation of the Marine Strategy Framework Directive
.
Front Mar Sci
3
. DOI:
63
MacCall
,
,
Francis
,
TB
,
Punt
,
AE
,
Siple
,
MC
,
Armitage
,
DR
,
Cleary
,
JS
,
Dressel
,
SC
,
Jones
,
RR
,
Kitka
,
H
,
Lee
,
LC
,
Levin
,
PS
,
McIsaac
,
J
,
Okamoto
,
DD
,
Poe
,
M
,
Reifenstuhl
,
S
,
Schmidt
,
JO
,
Shelton
,
AO
,
Silver
,
JJ
,
Thornton
,
TF
,
Voss
,
R
and
Woodruff
,
J
.
2018
.
A heuristic model of socially learned migration behaviour exhibits distinctive spatial and reproductive dynamics
.
ICES Journal of Marine Science
. DOI:
64
Mahon
,
R
,
Fanning
,
L
,
McConney
,
P
and
Pollnac
,
R
.
2010
.
Governance characteristics of large marine ecosystems
.
Mar Pol
34
(
5
):
919
927
. DOI:
65
Mauger
,
GS
and
Kennard
,
HM
.
2017
. Integrating Climate Resilience in Flood Risk Management: A Work plan for the Washington Silver Jackets: A report for FEMA.
University of Washington, Seattle, WA, USA
:
Climate Impacts Group
. DOI:
66
McNie
,
EC
.
2007
.
Reconciling the supply of scientific information with user demands: An analysis of the problem and review of the literature
.
Environ Sci Policy
10
(
1
):
17
38
. DOI:
67
Meagher
,
L
and
Lyall
,
C
.
2013
.
The invisible made visible: Using impact evaluations to illuminate and inform the role of knowledge intermediaries
.
Evid Policy
9
:
409
418
. DOI:
68
Mengerink
,
KJ
,
Van Dover
,
CL
,
Ardron
,
J
,
Baker
,
M
,
Escobar-Briones
,
E
,
Gjerde
,
K
,
Koslow
,
JA
,
Ramirez-Llodra
,
E
,
Lara-Lopez
,
A
,
Squires
,
D
,
Sutton
,
T
,
Sweetman
,
AK
and
Levin
,
LA
.
2014
.
A call for deep-ocean stewardship
.
Science
344
(
6185
):
696
698
. DOI:
69
Merrie
,
A
,
Dunn
,
DC
,
Metian
,
M
,
Boustany
,
AM
,
Takei
,
Y
,
Elferink
,
AO
,
Ota
,
Y
,
Christensen
,
V
,
Halpin
,
PN
and
Osterblom
,
H
.
2014
.
An ocean of surprises – Trends in human use, unexpected dynamics and governance challenges in areas beyond national jurisdiction
.
Global Environ Change
27
:
19
31
. DOI:
70
Moller
,
H
,
Berkes
,
F
,
Lyver
,
P
and
Kislalioglu
,
M
.
2004
.
Combining science and traditional ecological knowledge: Monitoring populations for co-management
.
Ecol Soc
9
(
3
). https://www.jstor.org/stable/26267682. DOI:
71
Moloney
,
CL
,
St John
,
MA
,
Denman
,
KL
,
Karl
,
DM
,
Köster
,
FW
,
Sundby
,
S
and
Wilson
,
RP
.
2011
.
Weaving marine food webs from end to end under global change
.
J Marine Syst
84
(
3
):
106
116
. DOI:
72
Monier
,
E
,
Scott
,
JR
,
Sokolov
,
AP
,
Forest
,
CE
and
Schlosser
,
CA
.
2013
.
An integrated assessment modeling framework for uncertainty studies in global and regional climate change: The MIT IGSM-CAM (version 1.0)
.
Geosci Model Dev
6
(
6
):
2063
2085
. DOI:
73
Morzaria-Luna
,
HN
,
Turk-Boyer
,
P
and
Moreno-Baez
,
M
.
2014
.
Social indicators of vulnerability for fishing communities in the Northern Gulf of California, Mexico: Implications for climate change
.
Mar Pol
45
:
182
193
. DOI:
74
.
2006
.
Evolving an ecosystem approach to science and management through NOAA and its partners
.
. https://sab.noaa.gov/sites/SAB/Reports/EETT/eERRT%20-%20Final%20Report%20to%20NOAA%20Oct%2006.pdf.
75
Nel
,
JL
,
Roux
,
DJ
,
Driver
,
A
,
Hill
,
L
,
Maherry
,
AC
,
,
K
,
Petersen
,
CR
,
,
LB
,
Van Deventer
,
H
and
Reyers
,
B
.
2016
.
Knowledge co-production and boundary work to promote implementation of conservation plans
.
Conserv Biol
30
:
176
188
. DOI:
76
Niiranen
,
S
,
Yletyinen
,
J
,
Tomczak
,
MT
,
Blenckner
,
T
,
Hjerne
,
O
,
MacKenzie
,
BR
,
Muller-Karulis
,
B
,
Neumann
,
T
and
Meier
,
HEM
.
2013
.
Combined effects of global climate change and regional ecosystem drivers on an exploited marine food web
.
Glob Change Biol
19
(
11
):
3327
3342
. DOI:
77
Pacific Fisheries Information Network (PacFIN)
. Data retrieval dated
January
2
,
2016
,
Portland, Oregon
:
Pacific States Marine Fisheries Commission
(www.psmfc.org).
78
Pacific Fishery Management Council
.
2013
. Draft Report of the Pacific Sardine Harvest Parameters Workshop.
Portland Oregon U.S.A.
:
Pacific Fishery Management Council
79
Pacific Fishery Management Council
.
2014a
. Managing our Nation’s Fisheries 3: Advancing Sustainability.
Portland, Oregon, USA
:
Pacific Fishery Management Council
. http://www.managingfisheries.org/s/Complete-PFMC-book-for-web2.pdf.
80
Pacific Fishery Management Council
.
2014b
. Atlantis Model for the California Current: Report of Methodology Review Panel Meeting.
Portland, Oregon, USA
:
Pacific Fishery Management Council
. https://www.nwfsc.noaa.gov/research/divisions/cb/documents/Atlantis_Panel_Report_final.pdf.
81
Pacific Fishery Management Council
.
2016a
. Coastal Pelagic Species Fishery Management Plan.
Portland, Oregon, U.S.A.
:
Pacific Fishery Management Council
82
Pacific Fishery Management Council
.
2016b
. Coastal Pelagic Species Management Team report on Northern Anchovy stock assessment and management measures.
November
2016.
Portland Oregon U.S.A.
:
Pacific Fishery Management Council
83
Pacific Fishery Management Council
.
2016c
. Coastal Pelagic Species Management Team Meeting Minutes.
October
2017.
Portland Oregon U.S.A.
:
Pacific Fishery Management Council
84
Pacific Fishery Management Council and National Marine Fisheries Service
.
2010
. Rationalization of the Pacific Coast Groundfish Limited Entry Trawl Fishery; Final Environmental Impact Statement Including Regulatory Impact Review and Initial Regulatory Flexibility Analysis.
Portland, OR
:
Pacific Fishery Management Council
.
June
85
Plagányi
,
ÉE
,
Punt
,
AE
,
Hillary
,
R
,
Morello
,
EB
,
Thébaud
,
O
,
Hutton
,
T
,
Pillans
,
RD
,
Thorson
,
JT
,
Fulton
,
EA
and
Smith
,
.
2014
.
Multispecies fisheries management and conservation: Tactical applications using models of intermediate complexity
.
Fish Fish
15
(
1
):
1
22
. DOI:
86
Polfus
,
JL
,
Heinemeyer
,
K
,
Hebblewhite
,
M
,
Taku River Tlingit First Nation
.
2014
.
Comparing traditional ecological knowledge and western science woodland caribou habitat models
.
J Wildlife Manage
78
:
112
121
. DOI:
87
Punt
,
AE
.
2017
.
Strategic management decision-making in a complex world: Quantifying, understanding, and using trade-offs
.
ICES J Mar Sci
74
(
2
):
499
510
. DOI:
88
Punt
,
AE
,
Butterworth
,
DS
,
de Moor
,
CL
,
De Oliveira
,
JAA
and
,
M
.
2016a
.
Management strategy evaluation: Best practices
.
Fish Fish
17
:
303
334
. DOI:
89
Punt
,
AE
,
MacCall
,
,
Essington
,
TE
,
Francis
,
TB
,
,
F
,
Johnson
,
KF
,
Kaplan
,
IC
,
Koehn
,
LE
,
Levin
,
PS
and
Sydeman
,
WJ
.
2016b
.
Exploring the implications of the harvest control rule for Pacific sardine, accounting for predator dynamics: A MICE model
.
Ecol Model
337
:
79
95
. DOI:
90
Punt
,
AE
,
Okamoto
,
DK
,
MacCall
,
,
Shelton
,
AE
,
Armitage
,
DR
,
Cleary
,
JS
,
Davies
,
IP
,
Dressel
,
SC
,
Francis
,
TB
,
Levin
,
PS
,
Jones
,
RR
,
Kitka
,
H
,
Lee
,
LC
,
McIsaac
,
JA
,
Poe
,
MR
,
Reifenstuhl
,
S
,
Silver
,
JJ
,
Schmidt
,
JO
,
Thornton
,
TF
,
Voss
,
R
and
Woodruff
,
J
.
2018
.
When are estimates of spawning stock biomass for small pelagic fishes improved by taking spatial structure into account?
Fish Res
206
:
65
78
. DOI:
91
Reichler
,
T
and
Kim
,
J
.
2008
.
How well do coupled models simulate today’s climate?
Bull Amer Meteorol Soc
89
(
3
):
303
311
. DOI:
92
Reid
,
RS
,
Nkedianye
,
D
,
Said
,
MY
,
Kaelo
,
D
,
Neselle
,
M
,
Makui
,
O
,
Onetu
,
L
,
Kiruswa
,
S
,
Kamuaro
,
NO
,
Kristjanson
,
P
,
Ogutu
,
J
,
BurnSilver
,
SB
,
Goldman
,
MJ
,
Boone
,
RB
,
Galvin
,
KA
,
Dickson
,
NM
and
Clark
,
WC
.
2016
.
Evolution of models to support community and policy action with science: Balancing pastoral livelihoods and wildlife conservation in savannas of East Africa
.
113
(
17
):
4579
4584
. DOI:
93
Rose
,
KA
,
Fiechter
,
J
,
Curchitser
,
EN
,
Hedstrom
,
K
,
Bernal
,
M
,
Creekmore
,
S
,
Haynie
,
A
,
Ito
,
S
,
Lluch-Cota
,
S
,
Megrey
,
BA
,
Edwards
,
CA
,
Checkley
,
D
,
Koslow
,
T
,
McClatchie
,
S
,
Werner
,
F
,
MacCall
,
A
and
Agostini
,
V
.
2015
.
Demonstration of a fully-coupled end-to-end model for small pelagic fish using sardine and anchovy in the California Current
.
Prog Oceanog
138
:
348
380
. DOI:
94
Ruiz
,
D
,
Brun
,
C
,
Connor
,
SJ
,
Omumbo
,
JA
,
Lyon
,
B
and
Thomson
,
MC
.
2014
.
Testing a multi-malaria-model ensemble against 30 years of data in the Kenyan highlands
.
Malar J
13
:
14
. DOI:
95
Shin
,
DW
,
Baigorria
,
GA
,
Romero
,
CC
,
Cocke
,
S
,
Oh
,
JH
and
Kim
,
BM
.
2017
.
Assessing crop yield simulations driven by the NARCCAP regional climate models in the southeast United States
.
J Geophys Res-Atmos
122
(
5
):
2549
2558
. DOI:
96
Skern-Mauritzen
,
M
,
Ottersen
,
G
,
Handegard
,
NO
,
Huse
,
G
,
Dingsør Stenseth
,
NC
and
Kjesbu
,
OS
.
2016
.
Ecosystem processes are rarely included in tactical fisheries management
.
Fish Fish
17
(
1
):
165
175
. DOI:
97
Smith
,
,
Fulton
,
EJ
,
Hobday
,
AJ
,
Smith
,
DC
and
Shoulder
,
P
.
2007
.
Scientific tools to support the practical implementation of ecosystem-based fisheries management
.
ICES J Mar Sci
64
(
4
):
633
639
. DOI:
98
Steinacher
,
M
,
Joos
,
F
,
Frolicher
,
T
,
Bopp
,
L
,
,
P
,
Cocco
,
V
,
Doney
,
SC
,
Gehlen
,
M
,
Lindsay
,
K
and
Moore
,
JK
.
2010
.
Projected 21st century decrease in marine productivity: A multi-model analysis
.
Biogeosciences
7
(
3
). DOI:
99
Sydeman
,
WJ
,
Thompson
,
SA
,
Anker-Nilssen
,
T
,
Arimitsu
,
M
,
Bennison
,
A
,
Bertrand
,
S
,
Boersch-Supan
,
P
,
Boyd
,
C
,
Bransome
,
NC
,
Crawford
,
RJM
,
Daunt
,
F
,
Furness
,
RW
,
Gianuca
,
D
,
,
A
,
Koehn
,
L
,
Lang
,
JW
,
Logerwell
,
E
,
Morris
,
TL
,
Phillips
,
EM
,
Provencher
,
J
,
Punt
,
AE
,
Saraux
,
C
,
Shannon
,
L
,
Sherley
,
RB
,
Simeone
,
A
,
Wanless
,
RM
,
Wanless
,
S
and
,
S
.
2017
.
Best practices for assessing forage fish fisheries-seabird resource competition
.
Fish Res
194
(
Supplement C
):
209
221
. DOI:
100
Tebaldi
,
C
and
Knutti
,
R
.
2007
.
The use of the multi-model ensemble in probabilistic climate projections
.
Philos T Roy Soc A
365
(
1857
):
2053
2075
. DOI:
101
Urban
,
MC
,
Bocedi
,
G
,
Hendry
,
AP
,
Mihoub
,
JB
,
Pe’er
,
G
,
Singer
,
A
,
Bridle
,
JR
,
Crozier
,
LG
,
De Meester
,
L
,
Godsoe
,
W
,
Gonzalez
,
A
,
Hellmann
,
JJ
,
Holt
,
RD
,
Huth
,
A
,
Johst
,
K
,
Krug
,
CB
,
,
PW
,
Palmer
,
SC
,
Pantel
,
JH
,
Schmitz
,
A
,
Zollner
,
PA
and
Travis
,
JM
.
2016
.
Improving the forecast for biodiversity under climate change
.
Science
353
(
6304
). DOI:
102
van der Lingen
,
CD
,
Bertrand
,
A
,
Bode
,
A
,
Brodeur
,
R
,
Cubillos
,
L
,
Espinoza
,
P
and
Friedland
,
K
.
2009
. Trophic Dynamics. In:
Checkley
,
DM
,
Alheit
,
J
,
Oozeki
,
Y
and
Roy
,
C
(eds.),
Climate Change and Small Pelagic Fish
,
112
157
.
Cambridge, UK
:
Cambridge University Press
. DOI:
103
van der Lingen
,
CD
,
Hutchings
,
L
and
Field
,
JG
.
2006
.
Comparative trophodynamics of anchovy Engraulis encrasicolus and sardine Sardinops sagax in the southern Benguela: Are species alternations between small pelagic fish trophodynamically mediated?
Afr J Mar Sci
28
(
3–4
):
465
477
. DOI:
104
van Voorn
,
GAK
,
Verburg
,
RW
,
Kunseler
,
EM
,
,
J
and
Janssen
,
PHM
.
2016
.
A checklist for model credibility, salience, and legitimacy to improve information transfer in environmental policy assessments
.
Environ Modell Softw
83
:
224
236
. DOI:
105
Verburg
,
PH
,
Eickhout
,
B
and
van Meijl
,
H
.
2008
.
A multi-scale, multi-model approach for analyzing the future dynamics of European land use
.
Ann Regional Sci
42
(
1
):
57
77
. DOI:
106
Weaver
,
CP
,
Mooney
,
S
,
Allen
,
D
,
Beller-Simms
,
N
,
Fish
,
T
,
Grambsch
,
AE
,
Hohenstein
,
W
,
Jacobs
,
K
,
Kenney
,
MA
,
Lane
,
MA
,
Langner
,
L
,
Larson
,
E
,
McGinnis
,
DL
,
Moss
,
RH
,
Nichols
,
LG
,
Nierenberg
,
C
,
Seyller
,
EA
,
Stern
,
PC
and
Winthrop
,
R
.
2014
.
From global change science to action with social sciences
.
Nat Clim Change
4
(
8
):
656
659
. DOI:
107
Weichselgartner
,
J
and
Kasperson
,
R
.
2010
.
Barriers in the science-policy-practice interface: Toward a knowledge-action-system in global environmental change research
.
Global Environ Change
20
(
2
):
266
277
. DOI:
108
Welch
,
C
.
2015
Feb
12
.
Fighting Over Herring—the Little Fish That Feeds Multitudes
.
National Geographic
. https://news.nationalgeographic.com/news/2015/02/150211-herring-decline-british-columbia-fishery-seabirds-environment/.
109
Weller
,
DE
,
Benham
,
B
,
Friedrichs
,
M
,
Najjar
,
R
,
Paolisso
,
M
,
Pascual
,
P
,
Shenk
,
G
and
Sellner
,
K
.
2013
.
Multiple models for management in the Chesapeake Bay
.
STAC Publication
14-004(14-004),
37
.
110
Witherell
,
D
.
2003
. Managing our nation’s fisheries: Past, present and future. In:
Witherell
,
D
(ed.).
Washington, D. C.
:
Fisheries management in the United States
.
111
Woolsey
,
R
.
2015
.
Feb
25
. Individual, commercial fishing interests clash in board testimony.
:
. https://www.kcaw.org/2015/02/25/individual-commercial-fishing-interests-clash-in-board-testimony/.
112
Young
,
JC
,
Waylen
,
KA
,
Sarkki
,
S
,
Albon
,
S
,
Bainbridge
,
I
,
Balian
,
E
,
Davidson
,
J
,
Edwards
,
D
,
Fairley
,
R
,
Margerison
,
C
,
McCracken
,
D
,
Owen
,
R
,
Quine
,
CP
,
Stewart-Roper
,
C
,
Thompson
,
D
,
Tinch
,
R
,
Van den Hove
,
S
and
Watt
,
A
.
2014
.
Improving the science-policy dialogue to meet the challenges of biodiversity conservation: Having conversations rather than talking at one-another
.
Biodivers Conserv
23
(
2
):
387
404
. DOI:
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.