Principles-focused evaluation for agroecology

This article offers a framework for evaluating agroecological principles, practices, and claims. Advocates of agroecology offer principles for sustainable, productive, and efficient agricultural systems. The evaluation question that immediately and inevitably arises in the face of such claims is: Does agroecology work? Thus do 2 fields of inquiry intersect: evaluation and agroecology. But there are many approaches to and models of evaluation. The challenge is to match the evaluation framework to the nature of agroecology. Because agroecology is based on principles, the appropriate evaluation approach is principles-focused evaluation. This article (1) elucidates principles-focused evaluation and demonstrates its relevance to evaluating agroecological practices and claims, (2) offers a principles-focused evaluation framework for monitoring agroecological transition and intensification within a particular context, and (3) concludes with lessons about principles-focused evaluation from agroecology.


Introduction
How can the entirety of a food system, including production, processing, transportation, financial intermediation, marketing, and consumption, be ecologically sustainable, agriculturally productive, and economically efficient? This is the big picture evaluation question. Advocates of agroecology offer a framework for sustainable, productive, and efficient agricultural systems. The evaluation question that immediately and inevitably arises in the face of such claims is: Does agroecology work? Thus, do 2 fields of inquiry intersect: evaluation and agroecology. This article examines that intersection. In so doing, I'll examine both possibilities and pitfalls. Let me begin with evaluation.
Evaluation I date the beginning of the evaluation profession to 1975 and publication of the 2-volume Handbook of Evaluation Research (Guttentag and Struening, 1975). That was also the era when the first professional evaluation associations were formed. There are now more than 200 national and regional organizations for professional evaluators around the world involving an estimated 75,000 evaluators. The year 2015 was designated as the International Year of Evaluation by the United Nations as part of the transition from the Millennium Development Goals (2000Goals ( -2015 to the Sustainable Development Goals (2015)(2016)(2017)(2018)(2019)(2020)(2021)(2022)(2023)(2024)(2025)(2026)(2027)(2028)(2029)(2030). The global evaluation profession has a number of evaluation journals, handbooks, textbooks, training institutes, annual conferences, webinar series, and online libraries.
Everything is subject to evaluation. Government agencies, international organizations, nongovernmental organizations (nonprofits), private sector initiatives, philanthropic foundations, universities, movements, networks, and interventions of all kinds are subject to evaluation even as they engage in evaluating others. Questions of accountability, effectiveness, efficiency, and impact are classic evaluation questions. The formal logic of evaluation involves 4-steps: 1. define the criteria that will be used to evaluate something, 2. set standards of performance on those criteria, 3. measure the actual performance, and 4. synthesize the results to reach an evaluative judgment.
Standard evaluation requirements include specifying SMART goals (specific, measurable, achievable, relevant, and time-bound) with performance indicators. Programs are expected to have logic models and theories of change. But over the course of 5 decades evaluation models, approaches, framework, methods, and measures have proliferated: impact evaluation, utilization-focused evaluation, theory-driven, cost-benefit, empowerment, participatory, social justice, realist, and culturally responsive, to name but a few examples. Standards have been formulated and adopted specifying what constitutes evaluation quality and checklists are available for what should be included in an evaluation. Principles-focused evaluation (Patton, 2018) is a new direction in evaluation that offers a framework for evaluating principles-based interventions and frameworks like agroecology.
1 Utilization-Focused Evaluation, Minneapolis, MN, USA Given the diversity and complexity of the evaluation profession, the challenge is one of matching: what approach to evaluation is appropriate for a particular situation, purpose, and evaluand (the thing being evaluated in the jargon of the field)? That brings us to the alignment between agroecology and principles-focused evaluation.

Agroecological principles
Agroecology is multifaceted: science, practices, and movement. The science of agroecology consists of knowledge about agroecological systems. Practices specify how to apply the science. The movement advocates values of sustainability, diversity, equity, and inclusion that inform the science and practices while seeking to ensure a supportive policy environment. There are diverse, sometimes conflicting, and still emergent perspectives on what agroecology is. Where agroecology coheres is being grounded in and guided by principles. Discussion and debate about which principles are essential illustrates the critical role of principles in defining agroecology. Principles matter, thus the intensity of the engagement about which principles, worded in what ways, are essential. Evaluative thinking and judgment can contribute to both the principles clarification process and assessing the degree of adherence to principles. Therein lies the challenges and opportunities where evaluation and agroecology intersect.

Principles of agroecology
Compared with conventional agricultural systems, agroecological farms seek to support biodiversity, enhance carbon sequestration, and utilize resources more efficiently by farming according to ecological principles (Ong and Liao, 2020, p. 1). In his seminal textbook and influential website, Steve Gliessman of the University of California, Santa Cruz, identified the comprehensive list of overarching agroecological principles presented in Table 1.
Later, Ernesto Méndez, at the University of Vermont, adapted and used these principles to create a tool for analyzing the potential agroecological impact of agricultural interventions.
Analyzing the potential of agriculture interventions to do harm to food production systems and create opportunities to build resilience within them requires looking at the entire food system. This includes the agroecological, biological, farming and market, social and political systems, as well as the interactions between these components. A deep consideration of these sometimes complex interactions is necessary in order to ensure program interventions in one area of a food system do not have unintended negative effects in other dimensions of the food system and inadvertently increase vulnerability of households or the agroecosystem. The overall goal of this screening tool is to support productivity and income generation for smallholder farmers with a long term vision towards resilience. The tool utilizes a food system perspective and agroecological principles to guide those who design and implement agriculture and food security programs through a series of inquiries into how their interventions may affect the resilience of food systems and of those who depend on them. This application of agroecological principles to context analysis and program design is especially useful in minimizing the depletion of crucial natural capital of smallholder farmers. By identifying opportunities to leverage and maximize available resources in resource scarce environments, programs may enhance the resilience of productive, sustainable agroecosystems and the households, markets, and food systems which rely on them. (Méndez, 2010, p. 2) These agroecological principles include the following: 1. Preservation and enhancement of agroecosystem diversity 2. Conservation and enhancement of soil health and nutrient cycling 3. Supporting ecological pest and disease regulating mechanisms 4. Maximizing renewable energy potential  Gliessman (2014Gliessman ( , 2015. Art. 9(1) page 2 of 6 Patton: Principles-focused evaluation for agroecology 5. Supporting and diversifying livelihoods to manage and mitigate risk exposure 6. Prioritizing and enhancing local food production and food security 7. Reducing dependence on external synthetic inputs 8. Optimizing water use-preserving and regenerating water resources 9. Integrating local and scientific knowledge 10. Strengthening local organizations (Scarborough et al., 2014, p. 1) There is, then, a broad consensus that agroecology is built around a set of principles, derived both from ecological principles, and a strong ethic of valuing farmer and local knowledge. There is also the sense that these principles should provide-not prescriptive advice-but a "design framework" to be optimized and adapted to local contexts (Gemmill-Herren, 2016, p. 5).
Because agroecology supports locally derived solutions, "rather than a set of context-specific technologies, agroecology uses a set of principles that can be locally disseminated and adapted to multiple contexts" (Méndez et al., 2015, p. 3).

Evaluating agroecological principles
As the works of Steve Gliessman, Ernesto Méndez, Barbara Gemmill-Herren, and others illustrate, agroecology has developed as a field, and is still developing, through the identification, articulation, testing, refining, debating, and revising of principles. Undergirding and manifest throughout every aspect of principles development is principlesfocused evaluation. The emergent challenges for evaluating agroecological principles have to do with new units of analysis and broader areas of focus for evaluation, what we call the evaluand, that is, the thing evaluated.
Evaluation origins were in evaluating projects. Such tools as logic models and SMART goals work well for project evaluation. They do not work well, in my judgment, for different kinds of evaluands like agroecological principles and systems transformation initiatives. Projects are closed systems, or at least treated as such in most evaluations, in which boundaries can be established and control can presumably be exercised within those boundaries by both program staff and evaluators. In contrast, complex dynamic interventions and principles-driven strategic initiatives are open systems characterized by volatility, uncertainty, and unpredictability, all of which make control problematic. Treating these complicated and complex evaluations like more simple projects is inappropriate, ineffective, and insufficient. Indeed, it can do harm by misunderstanding, misconceptualizing, and misrepresenting the very nature of complex change and thereby generating results that are inaccurate and irrelevant.
The key point here is that evaluating principles is different from evaluating projects. Principles-driven interventions and initiatives are different from goals-driven programs. Principles-focused evaluation can evaluate processes of implementing principles, outcomes associated with principles, longer term and broader impacts that result from principles-driven programming, and innovative approaches to principles adaptation. Principlesfocused evaluation can serve a variety of purposes: accountability, formative, summative, developmental, and knowledge generating. In all these applications, principles-focused evaluation should be utilizationfocused.

Principles-focused evaluation questions
Three core questions bring the focus to principles-focused evaluation:  (Patton, 2018) These evaluation questions flow from and depend on well-conceptualized, evaluable principles. What constitutes a high-quality principle? The GUIDE framework offers 5 criteria.

GUIDE framework for principles
The GUIDE framework for developing and evaluating principles specifies that principles should provide clear guidance [G], be useful for informing decisions and actions [U], be inspirational [I], be developmental, that is, adaptable to diverse contexts [D], and be evaluable [E]. Table 2 presents the GUIDE criteria.

From principles to practices
As the GUIDE framework illustrates, principles provide overall direction but have to be translated into and interpreted for application within a specific context. That involves converting principles to practices. Moreover, not all principles are adopted or adapted at the same time. The transition to agroecology often involves stages of applying and implementing agroecological principles. Given the importance of the transition process, Table 3 presents a principles-focused evaluation framework for monitoring agroecological transition and intensification within a context.

Lessons about principles-focused evaluation from agroecology
Here are 6 lessons about principles-focused evaluation related to agroecology.

Generating alternative principles can facilitate
dialogue and deliberation about the essential contributions of a new and emergent field of knowledge and practice. In the case of Patton: Principles-focused evaluation for agroecology Art. 9(1) page 3 of 6 agroecology, diverse sets of principles provide evidence about areas of agreement as well as dimensions of difference and debate. Some sets of principles are quite comprehensive and multifaceted while others are short and focused. These differences help define the boundaries of this innovative and still emergent field of inquiry and knowledge. 2. Principles, by their very nature, emphasize the importance of context. Agroecological principles encapsulate interdisciplinary knowledge and sustainable farming practices at a general level, but because agroecological systems vary significantly, the specific practices that follow from these principles within a given context will vary. Evaluation includes examining the translation of principles into practices within a context. 3. Evaluating the principles on shared criteria is how the field advances. Principles-focused evaluation supports evolution and further development of this emergent field of knowledge and practice. Evaluating the principles supports coherence about what is essential as the scientific knowledge base expands and coheres based on research and evaluation. 4. Principles point to practices. Practices operationalize principles. The guidance in the GUIDE framework is guidance toward practices. This is a critical test for principles, that they can point to practices that, when implemented, lead to desired results.  what to exclude (boundary issues in systems approaches). Principles can inform that boundary-setting process.

Summary and conclusion
This article has offered a framework for evaluating agroecological principles, practices, and claims. Advocates of agroecology offer principles for sustainable, productive, and efficient agricultural systems. The evaluation question that immediately and inevitably arises in the face of such claims is: Does agroecology work? But there are many approaches to and models of evaluation. The challenge is to match the evaluation framework to the nature of agroecology. Because agroecology is based on principles, this article has argued that the appropriate evaluation approach is principles-focused evaluation. To support that argument, the article has (1) elucidated principles-focused evaluation and demonstrated its relevance to evaluating agroecological practices and claims, (2) offered a principles-focused evaluation framework for monitoring agroecological transition and intensification within a particular context, and (3) concluded with lessons about principlesfocused evaluation from agroecology.