Measuring agroecology: Introducing a methodological framework and a community of practice approach

Agroecology is increasingly recognized as an effective pathway toward food systems transformation, contributing significantly to addressing converging socioecological crises (International Assessment of Agricultural Knowledge, Science and Technology for Development, 2009; International Panel of Experts on Sustainable Food Systems [IPES-Food], 2016; Food and Agriculture Organization of the United Nations [FAO], 2018a; High Level Panel of Experts [HLPE], 2019; Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services, 2019; Intergovernmental Panel on Climate Change [IPCC], 2019; Herren et al., 2020; Wanger et al., 2020; IPCC, 2022).

However, funding for the development of agroecology in particular territories or associated research does not match this recognition. There have been increasing efforts to track finance flows to agroecology, in order to identify the amount of money that donor countries and other institutions make available for a transition to sustainable food systems, and in order to increase and leverage existing funds (DeLonge et al., 2016; Pimbert and Moeller, 2018; Achterberg and Quiroz, 2020; Biovision and IPES-Food, 2020; DanChurchAid, 2020; Moeller, 2020; Vermeylen and Schutter, 2020; Anderson and Bruil, 2021; Botreau et al., 2021; New Economics Foundation and Croatan Institute, 2021; Olivera and Popusoi, 2021; BothEnds, 2022; Global Alliance for the Future of Food [GAFF], 2022; Greenberg and Muchero, 2022).

As part of these studies, the methodological question arose as to what makes a project more or less agroecological, that is, under which conditions a funding stream can be said to be supporting agroecology. This is particularly important in the light of increasing attempts to water down the radically transformative¹ ambitions of agroecology by co-opting or ill-defining it.

Following Pimbert and Moeller (2018), most of the studies analyzing finance flows adopted a version of Gliessman’s 5 levels of food system change (see Box 1) as a metric to guide assessment. However, it soon became obvious that the framework offered by Gliessman’s 5 levels was not adapted for the requirements of assessing the “agroecologicalness” of particular projects and that a different approach was needed to do so. Illustrating the dynamic evolution of agroecological thinking—transitioning from the “field level” to the “territorial food system” as discussed by Pimbert et al. (2021)—more recent frameworks have used the principles and elements of agroecology for evaluation, reflecting a more nuanced and integrated, albeit analytically more complex approach.² In this article, we introduce a new framework for measuring the degree of agroecological integration in projects and project portfolios, which builds on and further develops existing approaches, and we describe its development. Most of the work has been carried out in the context of the Financing and Investment Working Group of the Agroecology Coalition.

In Section 2, we outline why, and to what ends, a new framework for assessing agroecology was needed. In Section 3, we present the practical background, the process, and actors that were involved in developing a new framework. In Section 4, we outline the conceptual framework that was developed and the way it works and can be used. We outline ways to get involved in the use of the framework in Section 5 and outline our community of practice (CoP) approach, before we conclude this article in Section 6 by summarizing the key distinguishing features of our framework, including but not limited to its contribution in terms of operationalizing agroecology principles in an evaluation framework that can also act as a pedagogical tool. Supplementary material accessible alongside this article presents the full assessment rubric of the framework, as well as a test case assessment to illustrate the functioning of our framework “in action.”

When applied as a metric to guide the assessment of projects and portfolios, the Gliessman framework has considerable shortcomings. It needs to be stated upfront that, of course, Gliessman never intended his 5 levels to guide agroecology assessments, but rather to guide the conversion toward ecological management (Gliessman, 2016). However, even though this was not the author’s intention, Gliessman’s levels can be taken to suggest a stepwise transition process based on a common starting point of external input intensive, monoculture, industrial, or conventional agroecosystems, which does not reflect the reality for many smallholder farmers in the Global South.

Moreover, Gliessman’s levels 1–3 represent progressions within what we can understand as the agronomic or environmental dimension of agroecology, with associated changes applied at the field and farm level only. Only Levels 4 and 5, understood as further progressions toward sustainable food systems, concern the social/economic/political dimension. However, in reality, an agroecological transition is likely to unfold in multiple dimensions at once with parallel processes at work simultaneously. While Gliessman recognizes this complexity in his broader opus, the use of Gliessman’s levels for agroecology assessment purposes, without additional caveats and explanations, risks reducing this complex reality to a linear process, thereby potentially limiting the scope of transformative agroecology.

The levels were developed as a heuristic framework to open conversations about transformation of food systems (Gliessman, 2014, 2016). However, once used as a methodological tool for assessment, its shortcomings needed addressing and different studies had to find new and additional categories to be able to deal with the materials under investigation. For example, the report “Money Flows” (Biovision and IPES-Food, 2020) makes use of the idea of “social enablers” to categorize projects, which address the social dimensions of food systems, yet do not address the environmental dimension or work at the field or farm level. Similarly, a study on EU and Green Climate Fund finance flows (Moeller, 2020) uses additional categories, such as “governance institutions,” when projects support the development of specific governance mechanisms without any activities at the field or farm level.

Conceptually, this remains unsatisfactory, as the “real world” of projects and programs has to be made to fit a theoretical framework that is not adapted for the purpose. Moreover, while mapping the FAO’s 10 elements of agroecology onto the Gliessman levels (as in, e.g., the Agroecology Criteria Tool, which was used by some of the finance flow studies, see Figure 1) has great advantages for communication and awareness raising purposes, it also risks misconstruing the elements as steps in a progression toward an agroecological food system. This would be a misrepresentation, as the elements are intended to be understood as an integrated and nonhierarchical or sequential whole (Barrios et al., 2020; FAO, 2023).

In parallel to these studies on finance flows, the Tool for Agroecology Performance Evaluation (TAPE) was being developed and tested by the FAO and partners. TAPE has been designed for a participatory, on-the-ground evaluation of agroecological performance and does not lend itself very well to the analysis of project portfolios, especially with a high number of projects and when limited information is available on each of the projects.

Given the caveats and limitations of the existing tools and frameworks, there has not been, until recently, an ideal framework for the important methodological step of measuring the degree of agroecological integration in project portfolios in a pragmatic and yet holistic manner.

Conversations about the need for a methodological framework arose in different fora as a CoP formed around the aforementioned studies. This emerging CoP had the aim to promote agroecology, divestment from industrial agriculture, and a rethinking of funding modalities in support of food systems transformation. Within this community, consensus grew that an appropriate methodological framework would need to be:

• conceptually rigorous (based on an understanding of a spectrum of approaches to food system transformation that would be more or less strongly transformative in different aspects or dimensions of food systems);

• adaptable to different (ecological, social, and economic) contexts;

• useful for the analysis of relatively large project portfolios with often only superficial documentation (i.e., not a deep dive into single projects about which detailed information is available);

• user-friendly; and

• aligned with a conception of agroecology as based on the 13 principles identified by the HLPE on Food Security and Nutrition in their report to the Committee on World Food Security.

The desire to build a framework on the 13 principles consolidated by the HLPE (see Table 1) rather than the 10 FAO elements is grounded in the greater nuance the principles provide. As expressed by the HLPE (2019), the 13 principles of agroecology, which were consolidated from the existing literature by the HLPE in an iterative process, are clearly aligned with the 10 FAO elements. Yet, the 13 consolidated principles are slightly more differentiated and “articulated as actionable statements containing normative…and causative…aspects” (Wezel et al., 2020). As such, they have come to be preferred by researchers internationally. Nonetheless, recognizing the alignment with the FAO 10 elements is important, as the latter are so far the only internationally agreed language on the proposed features of agroecology.

Having said that, it is of course also important to recognize that HLPE principles could still be improved upon—for example, water is not given the importance it deserves among the principles and it might make more sense to separate social values from diets rather than combine these in a single principle. However, for the purposes of an assessment framework that needs to have sufficient international weight and buy-in, it was agreed that the set of principles needed to be used without any modification.

In June 2022, a 3-day workshop was hosted by the German Federal Ministry for Economic Cooperation and Development (BMZ) to share experiences and develop a common approach to continue finance analyses in ways that would make the different studies as consistent with one another as possible to create a basis for comparability. At this workshop, 3 key groups came together: The Civil Society Community of Practice on Financing Agroecology, The Financing Agroecology Donor Group, and the Working Group on Financing and Investments of the Agroecology Coalition. Participants included representatives of civil society organizations, farmers organizations, international institutions (FAO, United Nations Environment Programme [UNEP], and International Fund for Agricultural Development [IFAD]), as well as researchers and technical cooperation officials.³ On the first day, we shared experiences, compared needs, and it was decided to further develop a proposal for an assessment framework that had been put together by researchers at the Centre for Agroecology, Water and Resilience (CAWR), in conversation with program officers at the social justice umbrella organization Cooperation Internationale pour le Développement et la Solidarité (CIDSE).⁴ Effective and productive, the following 2 days were spent adapting and refining the proposal and have resulted in the framework presented here.

Among other, the following decisions were made as a frame for the framework:

1. The framework will be based on the 13 principles defined by the HLPE, while recognizing their alignment with the FAO 10 elements.

2. The framework will be translated into an online tool, with an offline (spread sheet-based) version for individual use. An accompanying “user guide” and video tutorial will be developed. The online tool will focus on collating data from different finance flow and portfolio analysis exercises, hosted by the agroecology coalition or another organization.

3. TAPE remains the tool of choice for participatory, farm- or territorial-level assessments of performance; our new framework is particularly well adapted for portfolio analysis. Performance and impact assessment is out of its scope. TAPE can be used alongside our new framework to assess performance on the ground. The 2 tools are meant to complement each other, and together, they can provide evidence on the positive impact of agroecological transformations for sustainability imperatives.⁵

4. Evaluative ratings under each principle (on a 0–2 spectrum, see Section 4) are based on a qualitative assessment by the user and should take into account specificity of context, location, scale, and other particularities.

5. Qualitative assessment will be based on the information that is available (such as project documents). In this context, it is crucial to understand that robust qualitative assessments are not subjective—they have strong justifications and are clear about the reasons for their conclusion.

6. All 13 principles are important, which means that for the calculation of the final score of a project, all principles will be equally weighted.

In the months since the Berlin workshop, based on the approach developed by the participants, a spreadsheet operationalizing the framework was developed by one of the authors (Stephanie Piers de Raveschoot) to provide a preliminary “tool,” which can be easily used and tested. The spreadsheet has then been tested through test assessments of existing projects by 13 organizations. Feedback has been collected and based on this, an online prototype has been developed by Statistics for Sustainable Development for finalization on an open-source platform. At the time of writing, this online prototype has been tested by 11 organizations (the majority of which also was involved in testing the original spreadsheet).

To clarify, the framework exists to evaluate individual projects, or entire portfolios of projects, for their degree of agroecological integration or, we might say, their “agroecologicalness.” Figure 2 shows its basic structure through the example of the HLPE principle of recycling. The entire framework rubric is presented in the Supplementary Material to this article. The framework’s key aspects are introduced in turn.

The framework introduces a set of “red flags” against which projects will initially be screened (Table 2). Red flags specify practices that run counter to agroecological values. Any project raising a red flag will automatically be ranked as zero across all principles and hence does not need to be rated for individual principles, saving time when project portfolios are large. Projects with any “red flags” (one example: promotion of GMOs) do not qualify as agroecological no matter how the rest of their work looks like. Projects that do not raise any red flags will then enter the full assessment.

Red flags exist to make assessment easier, but also in order to counter the cooptation and watering down of agroecology by organizations aligned with the interests of big agribusiness.

Projects are assessed on each of the 13 HLPE principles, on a scale between 0 and 2, where 2 is an aspirational expression of a strong transformative approach, 1 represents partial alignment, and 0 represents no alignment (or business as usual). Decimals can be used if individual users need to express further gradations.

Each of the 13 HLPE principles is expressed in 2 “value statements,” representing the 2 poles (0 and 2) of the spectrum. These statements exist to guide the judgement of the user and facilitate assessment.

Some principles can be out of the scope of individual projects (e.g., a project focused on grain production may not address animal health), and the framework indicates when that might be the case. Principles can thus be rated as nonapplicable (N/A). This mechanism is included to avoid that projects with a narrow focus, but with an important contribution to agroecological transitions, are excluded from the assessment or rated negatively. However, 4 principles are always applicable and cannot be rated N/A. These are co-creation of knowledge, social values and diets, fairness, and participation. These must always be rated and would be rated 0 if the project in question does not address these. This ensures that the project overall score is reduced for projects, which do not address these important agroecological principles.

For each principle, criteria for or examples of good or transformative practice are listed. Developed by a working group at the Berlin workshop and in subsequent weeks, these are meant to be informative and support users in the rating of the principles, but importantly will not automatically determine the rating. These examples give an indication of the kind of practices or approaches that would support a decision for a high rating. The list of criteria is dynamic and can evolve: New examples can be added over time, for example, proposed by users or other practitioners, and a mechanism for doing so is currently being developed by the CoP.⁶

To aid robustness, the framework enables documentation of the criteria that have contributed to the specific ratings of each principle and encourages adding specific notes that justify their selection.

To calculate the percentage of a project’s budget that is considered contributing to the agroecological transformation, we make a simple mathematical calculation using the following formula (see also the Example Assessment in the Supplementary Material to this article for an example case and calculation):

The assessment framework and its associated online tool were created within the context of an emergent CoP that has evolved over time. As stated by Wenger (2011): “Communities of practice are groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.” The intentional coalescence of individuals and organizations who were developing and testing different aspects of evaluating agroecology provided an opportunity to combine the collective intelligence and complementary strengths of participants. For example, a constructive critical review of the Biovision ACT tool by CIDSE and CAWR led to new ideas for how to improve or even to create a new tool. Later in the process, the participation of Stats4SD provided a new technical capacity to adapt the developed framework into a useable tool via an online interface. The emergence of the agroecology coalition provided a platform through which to promote and deploy the framework and its associated tool, which is now being tested for further iterative development. Indeed, the framework has been tested by 13 of the organizations participating in the Berlin workshop—often though not always by individuals who did not participate in the discussions and development of the workshop. Three further organizations also reviewed the framework (without testing it through test assessments). Feedback has been collected in order to improve the tool as well as the conceptual framework underlying it. To date, feedback has included issues of clarification, which will be addressed in a user guide accompanying the tool, as well as small improvements in textual formulation of value statements and examples. It is in this spirit of adaptive learning and of cooperation rather than competition among a diversity of actors that the framework and its associated tool have been able to reach their potential. The ongoing connection through the CoP will facilitate further improvements and new applications as the community evolves and as new communities of practice form and interact.

The online tool will be available for use by organizations who wish to assess, monitor, and track their own investment or project portfolios. It will keep data private when desired but will encourage sharing of (some) data in order to constitute a kind of overall “tracking mechanism,” which aggregates information on investments in agroecology across a number of different organizations. As such, the tool has a very clear objective and is designed with this in mind.

The underlying conceptual framework, on the other hand, may be useful in a greater diversity of contexts. We foresee that it could contribute to the design of projects and programs, as well as funding calls—ensuring that these are as transformatively agroecological as possible. It may also act as a “conversation starter,” which a group of people could use in order to discuss such things as their agroecological initiative’s aims and goals, specific approaches to discrete objectives, or how to improve transformation of food systems holistically in a local context. The framework also has pedagogical value inside and outside of the classroom, wherever agroecology, food systems transformation, or sustainable agriculture is being discussed.

This article is hence also an invitation to form communities of practice on understanding, evaluating, conceptualizing, and creating food systems transformations through agroecology. The framework introduced here provides a useful tool in starting or advancing conversation on agroecological transformation. We would hence hope that other communities of practice would use the framework and share the results of their evaluations, and over time, critique and contribute to improving the framework we have presented.

It is important to underline that we recognize that this framework is just that: a framework. It can be used to guide decisions in either project evaluation or design contexts, but it cannot replace real live thinking humans with varied experience and knowledge—these are needed to make careful judgments and come to appropriate conclusions. The framework will yield best results when used by individuals who are sympathetic and committed to food systems transformation through agroecology. It is also acknowledged that it is relatively coarse-grained and does not go into details of on-the-ground project implementation—the framework does not pretend to be the best tool for all types of evaluations.

However, we believe that the framework is a useful and important step in bringing more clarity to what agroecology is, how agroecology principles can be operationalized in an evaluation framework, and thus, how programs can be analyzed and ultimately encouraged to shift toward an agroecology that reflects all of its core principles. When adopting a principles-based approach to agroecology, there is a risk that projects cherry-pick principles that are relatively easier to implement (e.g., principles in the environmental/agronomic dimension, such as enhancements in biodiversity) without adopting the more challenging political, economic, and social dimensions. Not least to counteract this, our framework provides a conceptualization of agroecology across multiple dimensions (environmental/agronomic, social, cultural, economic, and political) that are all equally important for food systems transformation. It relies on built-in mechanisms to ensure equal prioritization across these dimensions—specifically through equal weighting and always applicable principles. This, alongside the introduction of red flags, is intended to safe-guard against the cooptation of agroecology. Illuminating multiple dimensions is crucial not only for analyzing project portfolios of donor organizations and countries’ investments in agriculture, and monitoring spending, but also for enabling agroecology transformations more widely.

No datasets have been generated as part of this research. The online tool can be freely accessed on https://agroecology-coalition.org/agroecology-finance-assessment-tool/.

The supplemental files for this article can be found as follows:

Table S1: Multidimensional continuum of approaches to sustainable food systems: full framework rubric (.pdf)

Document S2: Example Assessment (.pdf)

The authors would like to thank the work of all additional participants at the Berlin workshop, whose contributions have been a fundamental part of shaping the assessment framework presented here: Anne Maina (African Biodiversity Network), Esther Penunia (Asian Farmers Association), François Delvaux (Cooperation Internationale pour le Développement et la Solidarité [CIDSE]), Ingrid Prem (GIZ), Octavio Sotomayor (CEPAL/ECLAC), Rikke Olivera (IFAD), Siham Drissi (UNEP), and Vincent Dauby (CIDSE). They also thank Alexander Lingenthal (BMZ) for hosting the organization of this workshop and all those who tested and reviewed the framework. This article was improved through helpful comments by 2 anonymous reviewers.

Funding for the Berlin workshop was provided by the German Federal Ministry for Economic Cooperation and Development (BMZ) and the Agroecology Fund.

All authors are invested in furthering agroecological transformations of food systems. Colin Anderson is also an editor of the Special Feature on Principles-based Agroecology to which this article has been submitted. Carlos Barahona and David Mills are commissioned to develop the online version of the Agroecology Funding Assessment Tool. None of the authors gain financially through the publication of this article or the data contained therein.

Contributed to conception and design: NIM, EF, CA.

Contributed to acquisition of data: NIM, MG, EF, CBa, CBr, RC, FL, DM, AMvL, SPdR, AM, GH.

Contributed to analysis and interpretation of data: NIM, MG, EF, CBa, CBr, RC, FL, DM, AMvL, SPdR, AM, GH.

Drafted and/or revised the article: All authors.

Approved the submitted version for publication: All authors.

How to cite this article: Moeller, NI, Geck, M, Anderson, C, Barahona, C, Broudic, C, Cluset, R, Henriques, G, Leippert, F, Mills, D, Minhaj, A, Mueting-van Loon, A, de Raveschoot, SP, Frison, E. 2023. Measuring agroecology: Introducing a methodological framework and a community of practice approach. Elementa: Science of the Anthropocene 11(1). DOI: https://doi.org/10.1525/elementa.2023.00042

Domain Editor-in-Chief: Alastair Iles, University of California Berkeley, Berkeley, CA, USA

Guest Editor: Gabriela Bucini, University of Vermont, Burlington, VT, USA

Knowledge Domain: Sustainability Transitions

Part of an Elementa Special Feature: Principles-Based Approaches in Agroecology