SUSTAINABILITY ASSESSMENT OF AGROECOLOGICAL PRODUCTION UNITS: A COMPARATIVE STUDY OF IDEA AND MESMIS METHODS

Abstract

This article is a theoretical essay in which are compared two methods for assessing the sustainability of agro-ecosystems - IDEA and MESMIS - using the following analysis criteria: sustainability concept; objectives and target audience; flexibility and adaptability; systemic approach; and involvement of stakeholders. The criteria were selected aiming to contribute on future choices for sustainability assessment in agroecological production units. The study concluded that both methods can be employed, depending on purposes, conditions and evaluation context. The MESMIS method is more flexible and participatory, but requires a multidisciplinary trained team to support its application among farmers, having the possibility of facing problems of adherence to the concept of sustainability. The IDEA method has a rigid and coherent structure with a clear concept of sustainability and can be applied by the farmer himself with support from a technician, but it requires prior adjustments to the local context and to the data availability.

Keywords:
Sustainability assessment; Agroecology; MESMIS; IDEA

Resumo

Este artigo é um ensaio teórico em que são comparados dois métodos de avaliação da sustentabilidade de agroecossistemas - IDEA e MESMIS - por meio dos critérios de análise: conceito de sustentabilidade; objetivos e público-alvo; flexibilidade e adaptabilidade; abordagem sistêmica e envolvimento de stakeholders. Os critérios foram selecionados visando contribuir com escolhas futuras para avaliação da sustentabilidade em unidades produtivas agroecológicas. O estudo concluiu que ambos os métodos podem ser empregados, a depender dos propósitos, condições e contexto da avaliação. O MESMIS é mais flexível e participativo, mas requer equipe multidisciplinar capacitada para apoiar a aplicação junto aos agricultores, podendo haver problemas de aderência ao conceito de sustentabilidade. O IDEA tem estrutura rígida e coerente com um claro conceito de sustentabilidade, podendo ser aplicado pelo próprio agricultor com apoio de um técnico, mas requer adaptações prévias ao contexto local e à disponibilidade de dados.

Palavras-chave:
Avaliação de sustentabilidade; Agroecologia; MESMIS; IDEA

Resumem

Este artículo es unensayo teórico enen que son comparados dos métodos para evaluarlasostenibilidad de losagroecosistemas - IDEA y MESMIS - a través de loscriterios de análisis: concepto de sostenibilidad; objetivos y público destinatario; flexibilidad y adaptabilidad; enfoque sistémico; y participación de las partes interesadas. Los criteriosfueronseleccionados paracontribuir alasopciones futuras para laevaluación de lasostenibilidadenunidades de producción agroecológicas. El estudioconcluyó que ambos métodos se pueden utilizar, dependiendo de los propósitos, condiciones y contexto de laevaluación. El MESMIS es más flexible y participativo, pero requiereun equipo multidisciplinario capaz de apoyar a los agricultores enlaaplicacióndel método, que puede causar problemas de adherenciaconel concepto de sostenibilidad. ElIDEAtieneuna estructura rígida conun concepto claro y coherente de lasostenibilidad y puede ser aplicado por elpropio agricultor, conelapoyo de un técnico, pero necesita de adaptaciones al contexto local y ladisponibilidad de losdatos.

Palabras clave:
Evaluación de lasostenibilidad; Agroecología; MESMIS; IDEA

Introduction

The agricultural activity emerged about ten to twelve thousand years ago and, since then, growing and rearing techniques have been developed in order to meet increasing human demand for food, fiber and fuel. However, agricultural production model that has spread from the second half of the twentieth century has caused many harmful social and environmental impacts such as erosion, contamination of soil and water, biodiversity loss, depletion and expulsion of rural populations, besides being associated with increased incidence of several chronic diseases, demonstrating the need to seek a more sustainable agriculture.

In this context, agroecology emerged as a new scientific approach that can support the search for more sustainable alternatives to the hegemonic style of conventional agriculture. This new approach takes a holistic and systemic character that, in addition to treating the environmentally-responsible management of natural resources, aims at contributing to the diversion of the course of societies' social and ecological co-evolution (Caporal, Costabeber, & Paulus, 2006).

Measuring the agro-ecosystems' degree of sustainability is crucial to find the most appropriate solutions for the problems, which has been made through various approaches (Costa, 2010b; Sulvarán, Rieche, & Vargas, 2014).

In this perspective, this article comparatively analyzes two approaches based on a set of relevant criteria for guiding future studies on the sustainability of agroecological production units. The approaches under discussion are IDEA (Indicauters de Durabilité des Exploitations Agricoles) and MESMIS (Marco de Evaluación de Sistemas de Manejo Incorporando Indicadores de Sustentabilidad). Both methods, despite systematic and structural differences, aim at assessing agricultural production units that seek to become more sustainable. The choice for these approaches was due to their applicability in the scale of production units and to the availability of theoretical-methodological and empirical data.

Besides this introductory content, the article explores, in its literature review, aspects related to agriculture and sustainability, systems of agricultural sustainability indicators and agroecology. Then it exposes the methodological procedures adopted, results and concluding remarks.

Bibliographic review

Agriculture and sustainability

Agricultural activity is the primary means of interaction between man and nature, being responsible for the major changes in the environment and, therefore, the greatest environmental impacts (Moura, Almeida, & Miguel, 2004). The way this activity is conducted affects the environment in which it operates on a greater or lesser degree, so seeking sustainability in agriculture is fundamental to the sustainable development of society as a whole (Conway & Barbier, 2013).

All the debate involving the ideal of sustainability confronts visions of various political-ideological and scientific aspects, resulting in different definitions of what would be a sustainable production system. The notion of sustainability in agriculture comprises proposals of simple adjustments in conventional production pattern and more radical positions, which face this notion as a long-term goal, involving changes not only on agricultural production, but in economic, political, socio-cultural and environmental aspects (Marzall, 1999).

The various definitions reflect the discipline and interests of the thinker, the scales of analysis in spatial and temporal terms and the characteristics of the system under study (Costa, 2010a). The three basic dimensions of sustainability - environmental, economic and social - are addressed with greater or lesser emphasis on sustainable agriculture definitions (Yunlong & Smit, 1994). Some of the common attributes found in these definitions are: high efficiency and stability in production; use of organic techniques; food security and self-sufficiency; conservation of biodiversity; preservation of traditional values and knowledge; family agriculture; assistance to the underprivileged; and participation of farmers on agricultural development decisions (Conway & Barbier, 2013).

Overcoming the conceptual controversies, there is broad consensus on the importance of evaluating agricultural systems to make them more sustainable, which is being done through a variety of methods.

Agricultural sustainability's indicators and evaluation approaches

Sustainability indicators represent the most adopted sustainability assessment tool in practical and theoretical terms, either individually or in condensed indexes, or integrated into more complex models (Costa, 2010c). A sustainability indicator is a measure whose interpretation highlights the condition of a system as sustainable or not, according to the standards established for the context under analysis (Marzall, 1999). A sustainability index results from a mathematical manipulation of certain data aiming at simplifying them and can be formed by various types of indicators, including different themes (Verona, 2010). Sustainability indicators and indexes can contribute in the decision making processes aiming at sustainable development. To do so, these instruments should allow: the measurement of different dimensions in order to understand the complexity of social phenomena; society participation in defining the development process; communicate trends; and relate variables, because reality is not linear or one-dimensional (Guimarães & Feichas, 2009).

In the 1990s, from the early work of the United Nations Food and Agriculture Organization (FAO), there was a proliferation of sets of sustainable agriculture indicators (Sanchez & Matos, 2012). However, challenges faced in the selection and design of indicators led to the search and development of approaches that could guide the process of sustainability analysis. These approaches include analytical-conceptual structures from which the indicators and indexes of agricultural sustainability derive from (Rao & Rogers, 2006). They underlie the evaluation - including the steps of selection, design and interpretation - of indicators, as well as the organization of data and the communication of final results (Sanchez & Matos, 2012). The MESMIS and IDEA methods are two major approaches for assessing agricultural sustainability that are applicable to the scale of production units (Tommasino, Ferreira, Marzaroli, & Gutiérrez, 2012).

In the context of the search for more sustainable forms of agriculture, there are various initiatives, among which stands out agroecology, a holistic approach to sustainable rural development.

Agroecology

Agroecology is a scientific approach to support the transition of current models of rural development and conventional farming to sustainable agriculture. This idea refers to a process of continuous and increased evolution which requires changes in attitudes and in values of social actors in relation to the management and conservation of natural resources (Caporal et al., 2006). However, agroecology is not restricted to natural resource management in ecological basis, constituting a strategy for the analysis of social and environmental impacts generated by the conventional agriculture model, as well as for implementation of sustainable rural development programs (Moreira & Carmo, 2004).

Therefore, agroecology is not just a concept, but an orientation, whose contributions go beyond the technological and agronomic aspects of production, incorporating broader and more complex dimensions. Thus, agroecology is as a field of study that aims at the ecological management of natural resources that, through a collective social action of participatory nature, will restructure in a new order the social and ecological progress, halting the degradation and despoliation of nature and society (Guzman & Molina, 1996).

This new field of knowledge, unlike the compartmentalized and Cartesian ways of seeing and studying reality, integrates and articulates knowledge of different sciences as well as popular knowledge. Through an interdisciplinary and holistic approach, agroecology provides insight, analysis and criticism of the current model of agricultural development as well as the design of alternative strategies for achieving more sustainable farming styles (Caporal et al., 2006). Knowledge in areas such as ecology, agronomy, sociology and economics are integrated for a better understanding of the interactions, in agro-ecosystems, among plants, animals, humans and the environment (Dalgaard, Hutchings, & Porter, 2003). The importance of agroecology goes beyond merely physical goals: it seeks for a true revolution of the rural areas priorities, seeking to pay up and enable the variables that lead to the functioning of agro-ecosystem in harmony with social, environmental, economic, political and cultural aspects.

Methodological procedures

In methodological terms, the present study is a theoretical essay exploring concepts, techniques and approaches of two methods for assessing the agro-ecosystems sustainability - IDEA and MESMIS - through a set of analysis criteria. These criteria were selected from the theoretical basis used and from authors' inferences drawn on the characteristics of the methods with greater influence - on appropriateness and applicability - on agroecological production units. Following is an explanation of the relevance of each criterion used in the comparison of methods: the sustainability concept, objectives and target audience, flexibility and adaptability, systematic approach and involvement of stakeholders.

Sustainability Concept

The various notions of what would be sustainability in agriculture stimulated the development of different assessment approaches (Binder, Feola, & Steinberger, 2010; Marie, 2011). The notion of sustainability is reflected in indicators and determines the interpretation of results (Marzall, 1999). Therefore, it is crucial to understand the sustainability concept underlying the assessment method to be adopted in order to ensure consistency with the understanding of sustainability among the groups interested on the assessment.

Objectives and target audience

The objectives of an evaluation method reveal different concerns, leading to the consideration of specific aspects, which does not mean much importance to one aspect or another. Moreover, each method is designed aiming at one or more types of target audience such as farmers, agricultural technicians, government institutions, policy makers and researchers. Marzall and Almeida (2000) emphasize that the applicability of the evaluation method depends on its suitability for the target audience in terms of outcomes, process of reading and interpretation. For example, researchers may need more detailed information involving more complex processes of reading and analysis than technicians or farmers. Therefore, it is necessary to know the goals of the method and the public to whom it is intended in order to make it fit the purpose of the groups involved in the evaluation.

Flexibility and adaptability

Methods for assessing sustainability can provide flexible or rigid structure, allowing or not the adaptation to specificities of the agro-ecosystem under analysis. Some authors argue that we should not adopt global agricultural sustainability indicators applicable to all realities, needing to define them according to the specific conditions of each environment investigated (Deponti, Córdula, & Azambuja, 2002; Marzall, 1999; Reed, Fraser, Morse, & Dougill, 2005). However, this orientation makes it hard to compare different contexts, what can be, in some cases, the purpose of evaluation. Moreover, the use of standardized indicators and criteria at the local level could allow the aggregation of the results to national and global levels, guiding the development of broader policies (Rao & Rogers, 2006).

Systemic approach

Agroecology recognizes the inherent complexity of agro-ecosystems, fully treating all elements that compose these systems in a holistic and systemic approach. In this sense, the sustainability assessment should reflect such complexity by allowing the analysis of economic, social and environmental aspects, as well as relations and trade-offs between these aspects (Binder et al., 2010; Yunlong & Smit, 1994). Therefore, it is crucial to understand how the sustainability assessment method deals with this issue in order to diagnose its applicability in agroecological production units.

Involvement of stakeholders

In the context of this article, stakeholders are groups interested in the agroecological transition, among which we find the farmers themselves, agricultural technicians, extension and research institutions, governmental and non-governmental groups. Binder et al. (2010) find that stakeholders' involvement in the evaluation process may occur in the implementation and monitoring, but also in the establishment of assessment procedures (definitions of sustainability concept, objectives, types of evaluation and representation of the system, selection of indicators and respective methods of measurement and evaluation). Considering the involvement of stakeholders in the establishment of the evaluation procedures, there are two approaches: top-down (guided by experts) and bottom-up (involving different actors, especially the farmers themselves) (Reed et al., 2005; Sanchez & Matos, 2012). Each of these approaches has advantages and disadvantages, therefore the choice of method should consider this point.

After an initial literature review on the main topics related to the evaluation of the sustainability of agroecological production and the definition of analysis criteria, specific data on the IDEA and MESMIS methods were sought. This research was performed on books, articles, theses, dissertations, websites and other documents describing the methodologies or treating their field applications.

The results of the study are presented in two parts, the first containing the description of IDEA and MESMIS methods and the second one containing a comparative analysis between them, seeking to highlight common and diverging points, strengths and weaknesses of the methods, considering each analysis criteria.

Analysis and presentation of results

Description of methods

IDEA Method

The IDEA method (Indicateurs de Durabilité des Exploitations Agricoles or Indicators of Sustainable Farm Development) was created upon request of the General Board for Education and Research of the French Ministry of Agriculture which, since 1996, aims at assessing and diagnosing the sustainability of agricultural systems. It was developed by a multidisciplinary team and had its indicators' robustness, sensitivity and relevance tested, obtaining, over the period 2003 to 2006, successive improvements in its formulation and weighting. The method is based on the possibility of assessing the sustainability of an agricultural system by the quantification of technical, spatial, economic and human characteristics and by practices judged favorable to biophysical and social environments. Its structure is based on three dimensions of sustainability: agro-environmental, economic and socio-territorial (Vilain, 2008).

As shown in Table 1, the dimensions are divided into components on which 41 indicators are distributed. Indicators values are aggregated hierarchically by component and size, so that each dimension is rated on a scale from 0 to 100. Indexes of the dimensions are not aggregated and the degree of sustainability of the plant is given by the smallest ratio obtained between the three dimensions. This approach avoids compensations between dimensions, but assumes tradeoffs among components of the same dimension (Zahm, Viaux, Vilain, Girardin, & Mouchet, 2008). For example, if the diversity component obtains a score equal to zero, but the other two components of the agro-environmental dimension obtain maximum score (33 and 34 respectively), the index of the dimension will be 67. That is, the poor performance of the diversity component was compensated by the excellent performance of other components of the agro-environmental dimension. However, even if the indexes of socio-territorial and economic dimensions are greater than 67, the general index of sustainability of the plant will be 67 because there is no compensation between dimensions.

Table 1:
IDEA Method Structure

• Source: Adapted from Vilain (2008)

Indexes generated on the IDEA application may be presented in radar graphs (Figure 1) or histograms, allowing the performance analysis of a single production unit and the comparison among production units (Vilain, 2008).

Figure 1
Example of IDEA results presentation (radar graph)

IDEA is considered one of the most successful methods on assessing the sustainability of various agro-ecosystems (Elfkih, Guidara, & Mtimet, 2012). It was one of the first to be widely adopted in Europe (Bélanger, Vanasse, Parent, Allard, & Pellerin, 2012). Only in France, where it was created, it has more than 1,500 applications from 1997 to 2007. Zahm et al. (2008) presented and analyzed results of 65 of these applications, representing various cultivation systems. These authors, precursors of the method, discuss how the scientific approach of IDEA supports farmers and policymakers in their search for a more sustainable agriculture and conclude that the method can be used to compare production units with similar contexts in terms of production type, soil and climate.

Despite being a teaching tool, the IDEA has been little used in Brazilian academic channels. Looking ahead, it can become an important tool to assess the points in which agroecological farming systems need public policy actions to become more sustainable.

MESMIS Method

MESMIS - Marco para Evaluación de Sistemas de Manejo de Recursos Naturales Incorporando Indicadores de Sustentabilidad - was created in 1995 by an interdisciplinary and multi-institutional group of Mexico, aiming at translating general principles of sustainability into operational definitions, indicators, and practices in the context of natural resource management in rural communities (López-Ridaura, Masera, & Astier, 2002). It is a method that, on a cyclical, participatory and multi-scalar manner, seeks to identify anthropogenic changes on a system based on sustainability standards. The method is applicable in the agricultural, forestry and / or livestock production systems, and tries to point out, holistically, the limits and possibilities of sustainability of the system under economic, social (including cultural and political) and environmental (Masera, Astier, & López-Ridaura, 2000) prospects.

The method consists of six-steps assessment cycles (Figure 2). Firstly, the characterization of the analyzed system is made through the identification of the aspects of the management system and its socioeconomic and environmental context. Then, there is the analysis of critical points of the agro-ecosystem, in order to identify sustainability's limiting and favorable factors. The third step is the determination of diagnostic criteria associated with the attributes of sustainability (productivity, stability, resilience, reliability, adaptability, equity and self-management). From the diagnostic criteria are determined the sustainability indicators. The fourth step refers to the measurement and monitoring of the indicators over time; in the fifth step there is an integration and presentation of results, and finally, after a critical system analysis, conclusions and recommendations considered important for improving the management system sustainability are made. This last step closes the cycle, while the other begins at the start of the new evaluation system (MESMIS, 2014).

Masera et al. (2000) suggest that qualitative and quantitative indicators reflecting environmental, economic and social dimensions of agro-ecosystems are used. Table 3 provides examples of indicators that can be adopted in the implementation of MESMIS, organized according to their attributes of sustainability and diagnostic criteria. It is interesting to note that the attributes of sustainability involve aspects of different dimensions, highlighting the systemic approach of the method.

The integration and presentation of the MESMIS results can also be done with the use of amoeba or radar type diagrams. These graphical representations provide the concept of distance from a reference value, allowing a simple and comprehensive comparison of advantages and disadvantages of the evaluated system (López-Ridaura, Masera, & Astier, 2000).

MESMIS has been used in various parts of the world, especially in family-based systems with an emphasis on activities with ecological basis (Verona, 2008). Although it has almost the same lifetime of the IDEA, it has a significantly lower number of applications. After fifteen years of MESMIS creation, some of its creators account more than 60 case studies conducted mainly in Latin America but also in Spain, Portugal and the United States (Astier, Barrios-Garcia, Galván-Miyoshi, González-Esquivel, & Masera, 2012).

Figure 2
Assessment cycle of MESMIS

These authors make a critical assessment of MESMIS from the analysis of 25 case studies selected according to the level of detail and the quality of data available. Their goal was to identify strengths and limitations of the method in order to promote improvements in future applications. The conclusion is that MESMIS enables the conduction of agricultural sustainability assessments in different socio-ecological contexts with a long-term, participatory, interdisciplinary and multi-institutional approach. However, it would be necessary to evolve in long-term longitudinal studies, capturing the dynamic properties of the analyzed systems and involving relevant stakeholders in creative, participatory and sustainable processes. In this sense, the authors present a new version of MESMIS, yet to be tested and validated, incorporating tools and interactive techniques in each of the six steps of the evaluation cycle.

In Brazil, MESMIS has been applied, primarily, in family-based production units as an academic exercise, in the form of dissertations, thesis and other research and extension projects supported by funding agencies.

Table 3:
Examples of indicators that can be adopted in the applications MESMIS

• Source: Adapted from Masera et al. (2000)

Comparative analysis of methods

Concept of sustainability

In the IDEA method the sustainability of agro-ecosystems is a function of its economic efficiency (viability), of the living conditions of farmers and their families (livability) and of environmental impacts arising from agricultural practices (environmental reproducibility) (Zahm et al., 2008). These three concepts are strictly related to the method dimensions, in which the indicators are organized: economic, socio-territorial and agri-environmental, respectively.

The MESMIS methodology translates the concept of agricultural sustainability in five attributes: productivity; resilience; reliability; stability; adaptability; equity; and self-management. The relationship between the attributes of sustainability and system indicators is made during each evaluation process (MESMIS, 2014). In this process, farmers build their own concept of sustainability and define their priorities, contributing to the development of new skills and the ability to reason, enhancing decentralization and local development. However, Speelman et al. (2007) realized, in the analysis of 28 MESMIS application, several problems in the association of indicators and sustainability attributes. The authors credited this to the difficult concept of attributes and lack of practice of evaluators in the use of more abstract indicators.

Thus, on the one hand, IDEA adopts a well-defined and rigid concept of sustainability, maintaining enough consistency between this concept and the system's indicators. This, however, does not guarantee that the notion of sustainability underlying the method is shared by stakeholders in the evaluation. On the other hand, MESMIS works with participatory construction of the concept of sustainability, which legitimizes the evaluation process with the target audience, but does not guarantee adherence between the indicators used for evaluating and the attributes of sustainable agriculture.

Objectives and target audience

The MESMIS was developed in Mexico as an initiative of the Rockefeller Foundation with the goal of promoting alternative peasant agro-ecosystems capable of responding to the challenges imposed by global socio-ecological changes. To reach this achievement, functional ecological processes should be created in order to alleviate the poverty of peasant communities of that country (Astier et al., 2012). The IDEA method was developed at the request of the General Board for Education and Research of the Ministry of Agriculture of France, who intended to raise awareness to the notion of sustainability based, encouraging voluntary farmers to reflect on their own practices (Briquel et al., 2001). Both methods have farmers as main target audience, but the realities of the poor Mexican peasants and the French farmers were, and still are, very different, which is reflected in the objectives of the methods. While IDEA aims at generating information for reflection, MESMIS was developed with the broader purpose of investigating and promoting new means of agricultural production, to change the socio-environmental reality of poor peasant communities.

The results of the assessment and the processes of reading and interpretation should be accessible to the target audience. The IDEA method aggregates quantitative indicators into indexes usually presented in graphs of histogram and radar, while MESMIS involves the use of qualitative and quantitative indicators, often presented in radar (or Amoeba) graphs. These graphical representations are appropriate to the target audience of the methods because they facilitate reading and interpretation of indicators, allowing a ready identification of the strengths and weaknesses of the production unit and serving farmers and other stakeholders as guides for policies and corrective actions (Bélanger et al., 2012; Nicholls et al., 2004). It also assists farmers to holistically reflect on their management strategies (Astier, Masera, & Galván-Miyoshi, 2008) and facilitate the comparison of analyzed systems (Acosta-Alba & Van der Werf, 2011).

Additionally, the application of MESMIS generates a list of recommendations built by the multiple stakeholders involved in the assessment, in order to implement improvements to make more sustainable the system under evaluation. These improvements may include actions to be taken not only by farmers but also by other stakeholders. The application of the IDEA does not necessarily involve the establishment of recommendations, so the search for more sustainable practices must start from the reflection on sustainability indexes obtained for each component and dimension.

Operationally, both methods can be used not only to individually evaluate a property over time (longitudinal analysis) but also to compare different properties in the same region and identify more sustainable practices (transversal analysis) (Masera et al., 2000; Zahm et al., 2008).

Flexibility and adaptability

The MESMIS method has a flexible structure, adaptable to different levels of information and technical training (Astier et al., 2008). Acosta-Alba and Van der Werf (2011) consider these the main advantages of the method. The flexibility, combined with a participatory and interdisciplinary approach, allows the process of sustainability assessment to be adapted to the specificities of the analyzed agro-ecosystems (Verona, 2010). The flexibility is given as the evaluation team defines the concept of sustainability, identifies the critical points of the agro-ecosystem, determines the diagnostic criteria and defines the sustainability indicators. Consequently, each assessment process adopts its own indicators. Astier et al. (2008) identified, in practice, a prevalence of indicators associated with the attribute productivity and the environmental aspects, with indicators of social nature being less frequent, which reflects the evaluator teams' interests and training. In this sense, the consistency of the evaluation with the MESMIS method can be questioned, once its flexible nature allows relevant aspects to be neglected (Marie, 2011).

The IDEA method has a rigid structure composed of 41 well-defined indicators in terms of calculation method, evaluation criteria and method of aggregation. Its high standardization favors large-scale application, enabling cross-sectional analyzes with significant samples. On the other hand, it can be a hindering-factor for the application, since the method was designed and its parameters were set to the context of the production units in France, with no planned adaptations. Once these limitations are recognized, the creators of IDEA, Zahm et al. (2008), concluded that it is necessary to make adaptations to local contexts in terms of climate, landscape, and other specificities. However, the authors warn of the importance that the indicators continue to meet their scientific principles of construction.

Although recognizing the need to adapt IDEA to local contexts, the creators of the method do not define clear methodological procedures to guide the adaptation, so different approaches have been adopted raising questions about its consistency with the method. For example, adaptations of IDEA in Tunisia (Elfkih et al., 2012), Lebanon, Algeria and in France (Marie, 2011) disregarded some indicators of the method due to difficulties in measurement or simulation. This approach may be questioned, since relevant aspects were eliminated by difficulties in obtaining data and by the lack of relevance to the context under evaluation.

Thus, IDEA, even requiring adaptations, provides a starting point for structured evaluations on different contexts. These adaptations, however, should be methodologically well justified, since they are not provided by the method. On the other hand, the MESMIS approach is inherently flexible and adaptable, geared precisely to reflect the specificities of contexts under evaluation. Caution should be on the adhesion between the sustainability attributes proposed by the method and the indicators adopted in the assessment and over the balance between environmental, social and economic aspects.

Systemic approach

In IDEA, each indicator is made operational in separate, allowing the adoption of independent and non-cumulative scales (Vilain, 2008). The indicators measures are aggregated into indexes at the level of its respective components, which are aggregated by dimension, generating agri-environmental, economic and socio-territorial indexes.

As for the MESMIS, it proposes a systematic approach on the process of sustainability assessment, promoting the interaction between technical, economic, social and environmental dimensions (Deponti et al., 2002), as shown in Table 3. However, most of the applications of MESMIS did not seek to understand the relationships (synergies and tradeoffs) between attributes or between dimensions (Astier et al., 2012; Speelman et al., 2007). This type of analysis would promote a better understanding of how the agro-ecosystems variables behave with different management strategies. However the capture of this systemic dynamics would need the conduction of longitudinal analyzes, which hardly occurred in studies considered by the authors.

This leads to the conclusion that the IDEA method fails to present a systemic approach, because, even having indicators characterizing different aspects of sustainability, it does not consider their interactions (Binder et al., 2010). Meanwhile MESMIS seeks to promote these interactions, but faces difficulties in operationalizing the systemic approach, since most applications focus on cross-sectional analyzes, making it hard to analyze the dynamics of evaluated systems. The adoption of an integrated and holistic approach in the assessment of agro-ecosystems' sustainability is still rare due to the lack of a tradition in systemic research (Marzall & Almeida, 2000). Some of MESMIS creators propose that future applications of MESMIS seek to establish relations, especially trade-offs between variables and between management alternatives, in order to support decision making. This could be done though longitudinal analyzes and also with the incorporation of interactive tools such as games and simulations of scenarios, in evaluation processes (Astier et al., 2012).

Involvement of stakeholders

Social actors connected to sustainability of agro-ecosystems can be involved in three stages of the evaluation process: the establishment of procedures, the application of these procedures and the monitoring of results (Binder et al., 2010). The IDEA is designed as a tool for self-assessment of sustainability which can be applied by the farmer with the assistance of a technician advisor on a single shift, but the method has also been used by managers, agricultural colleges, and rural extension workers (Zahm et al., 2008). Therefore, stakeholders are engaged only in two stages of the evaluation process - method application and indicators monitoring. As for MESMIS, it stimulates an evaluation process including the various stakeholders, suggesting that the assessment team is composed of representatives of the systems evaluated and of members of educational and research institutions, NGOs, government agencies and policy makers (MESMIS, 2014). In this more participatory approach, stakeholders are involved in the three stages of the evaluation process.

Approaches like MESMIS, involving stakeholders since the establishment of assessment procedures (bottom-up), can accessibly reflect the complexity of the local context to the target audience, besides favoring the formation of shared values and commitments to the implementation of recommendations (Reed et al., 2005). However, the review process takes more time and resources, and hinder comparisons between different contexts (Binder et al., 2010).

Despite the fact that MESMIS stimulates the involvement of different stakeholders in the evaluation process, Speelman et al. (2007) observed, on 28 applications, that stakeholders involved are basically the farmers, with little or no involvement of other groups such as consumers, environmental and agriculture agencies. The authors believe that this fact occurs because most of the analyzed applications have been carried out as an academic exercise. Astier et al. (2012) also realized the need to promote greater involvement of various stakeholders in evaluation processes. To do so, the creators of MESMIS propose the incorporation of a set of new tools and interactive techniques to the cycle of method, aiming at stimulating assessment a greater involvement from stakeholders; facilitating the understanding of agro-ecosystems' complexity; assisting the management of conflicts of interest; and promoting greater commitment to implement actions that lead to higher levels of sustainability. These tools include games, scenarios simulation and exercises of group dynamics.

Assessment approaches which procedures are standardized and established by experts (top-down), as the IDEA, are more scientifically rigorous and favor objectivity, but they can be difficult to use and may impose restrictions in disagreement with the complexity and diversity of evaluated contexts (Reed et al., 2005). According to Binder et al. (2010), the top-down approaches favor the comparison between different systems, but the low involvement of stakeholders in the definition of the evaluation procedures may hinder the implementation of recommendations that could make them more sustainable systems.

Table 2:
Comparison between the IDEA and MESMIS methods

Analysis MESMIS IDEA Criteria
Sustainability Built by each assessment team Preset and based on 3 key concept from 7 agricultural sustainability concepts: viability, livability and attributes: productivity, stabiity, environmental reproducibility. reliability, resilience, adaptability, Warrant of adherennce among equity and self management. indicators and key concepts. There's no guarantee of adherence between sustainability indicators and attributes.
Objectives and Promoting alternative peasant agro- Supplying self assessment tool to target audience ecosystems, involving encourage farmers and students to stakeholders that are relevant in reflect on the notion of agricultural the process of sustainability sustainability. assessment.
Performing longitudinal and transversal analysis in rural properties.
Results of the assessment, reading and interpretation processes acessible to target audience (radar or amoeba graphs).
Flexibility and Flexible and adaptable structure to Rigid and standard structure which adaptability different levels of information, does not predict, but requires technical capacity and local adaptations to local contexts. contexts.
Systemic Proposes analysis of the Adopts agri-environment, socioapproach interactions and trade offs between territorial and economical techical, economical, social and dimensions, without considering environmental dimensions. the interactions and trade offs between them.
Involvement of Reelevant stakeholders involved in Farmer, student and or techinician stakeholders the whole assessment process: involded in the application and in establishment of assessment the monitoring. Assessment procedures (bottom-up approach), procedures preset by specialists application and monitoring. (top down approach).

Considering the comparative analysis developed in this article and summarized in Table 2, we conclude that both MESMIS and IDEA can be used to assess production units that seek sustainability through agroecology. However, specific features of the methods make either of them more suitable for the application depending on the evaluation context and purpose.

Final considerations

Agroecology seeks to restructure social and environmental agro-ecosystems through participatory collective social actions (Guzman & Molina, 1996). Therefore, the process of sustainability assessment is considered a step that contributes to this restructuring, being important that relevant stakeholders are involved in the evaluation and that indicators are suitable to the specificities of the reality to be changed. Also, the complex and multidimensional nature of agricultural sustainability requires assessment approaches for analyzing the economic, social and environmental aspects, as well as the relationships and trade-offs between these aspects (Binder et al., 2010; Yunlong & Smit, 1994). In this sense, the MESMIS, with its proposal of a participatory, flexible and systemic approach, appears to be more appropriate for the assessment of agroecological production units. However, some difficulties were observed when operationalizing these three aspects, which minimizes - but does not eliminate - the advantages of MESMIS if compared to IDEA.

If the purpose of the evaluation is to conduct broader cross-sectional analyzes, involving a large number of production units, the IDEA stands out in relation to MESMIS, because its standardized and well-structured assessment procedures favor large-scale comparisons and its application is faster, cheaper and more simple. It is also important to consider the availability of data for evaluation, because the IDEA requires predetermined data that, if unavailable, may hinder or even prevent its implementation. The MESMIS approach is very flexible in terms of indicators to be monitored, which, on the one hand, solves the issue of data unavailability, but, on the other, can leave gaps in the assessment, compromising the adhesion between the set of indicators and the concept of sustainability. However, as noted by Zahm et al. (2008), although certain principles of sustainability are common to agricultural systems, there is not an evaluation model suitable for all systems. So even the IDEA method requires adaptations to technical, environmental, social, political and economic contexts of the production units to be evaluated, which should be done without losing the scientific principles of indicators construction.

Finally, the comparative analysis conducted in this paper shows that the MESMIS (bottom-up) and IDEA (top-down) approaches have their own advantages and disadvantages, which constitute trade-offs to be considered when deciding which method to adopt. However, as reported by Reed et al. (2005), there is a growing trend in the area of sustainability assessment for a convergence between bottom-up and top-down approaches. According to these authors, the cross-fertilization of ideas between the two approaches has the potential to increase the general knowledge about the social and environmental problems, facilitate empowerment of communities and guide the development of policies for development. Bélanger et al. (2012) have adopted a combination of approaches, which is also advocated by Roy and Chan (2011) and by Tommasino et al. (2012). Thus, depending on the characteristics of the agro-ecosystem under assessment and based on the analysis developed in this article, it's possible to select the method that is most suitable to the context and to the evaluation purposes.

Referências Bibliográficas

• ACOSTA-ALBA, I.; WERF, H. M. G. VAN DER. The Use of Reference Values in Indicator-Based Methods for the Environmental Assessment of Agricultural Systems. Sustainability, v. 3, n. 12, p. 424-442, 2011.
• ASTIER, M.; BARRIOS-GARCIA, L.; GALVÁN-MIYOSHI, Y.; GONZÁLEZ-ESQUIVEL, C. E.; MASERA, O. Assessing the Sustainability of Small Farmer Natural Resource Management Systems. A Critical Analysis of the MESMIS Program (1995-2010). Ecology & Society, v. 17, n. 3, 2012.
• ASTIER, M.; MASERA, O.; GALVÁN-MIYOSHI, Y. Evaluación de Sustentabilidad: Un enfoque dinámico y multidimensional. SEAE/CIGA/ECOSUR/CIEco/UNAM/GIRA/Mundiprensa/FundaciónInstituto de Agricultura Ecológica y Sustentable: Espanha, 2008.
• BÉLANGER, V.; VANASSE, A.; PARENT, D.; ALLARD, G.; PELLERIN, D. Development of agri-environmental indicators to assess dairy farm sustainability in Quebec, Eastern Canada. Ecological Indicators, v. 23, p. 421-430, 2012.
• BINDER, C. R.; FEOLA, G.; STEINBERGER, J. K. Considering the normative, systemic and procedural dimensions in indicator-based sustainability assessments in agriculture. Environmental Impact Assessment Review, v. 30, n. 2, p. 71-81, 2010.
• BRIQUEL, V.; VILAIN, L.; BOURDAIS, J.; et al. La méthode IDEA (indicateurs de durabilitédesexploitationsagricoles): une démarche pédagogique. Ingénieries-EAT, v. 1, n. 25, p. 29-39, 2001.
• CAPORAL, F. R.; COSTABEBER, J. A.; PAULUS, G. Agroecologia: matriz disciplinar para ou novo paradigma para o desenvolvimento rural sustentável. In: TOMMASINO, H; HEGEDUS, P. (Ed.). Extensión: reflexiones para la intervención en el medio urbano y rural.UFSM / Universidad de La República, 2006.
• CONWAY, G. R.; BARBIER, E. B. After the Green Revolution: Sustainable Agriculture for Development. Londres: Earthscan, 2013.
• COSTA, A. Agricultura sustentável II: Avaliação. Revista de Ciências Agrárias, v. 33, n. 2, p. 75-89, 2010a.
• COSTA, A. Agricultura sustentável I: Conceitos., Revista de Ciências Agrárias v. 33, n. 2, p. 61-74, 2010b.
• DALGAARD, T.; HUTCHINGS, N. J.; PORTER, J. R. Agroecology, scaling and interdisciplinarity. Agriculture, Ecosystems&Environment, v. 100, n. 1, p. 39-51, 2003.
• DEPONTI, C. M.; CÓRDULA, E.; AZAMBUJA, J. L. B. Estratégia para construção de indicadores para avaliação da sustentabilidade e monitoramento de sistemas. Agroecologia e Desenvolvimento Rural Sustentável, v. 3, n. 4, p. 44-52, 2002.
• ELFKIH, S.; GUIDARA, I.; MTIMET, N. Are Tunisianorganicolivegrowingfarmssustainable? An Adapted IDEA approach analysis. Spanish Journal of Agricultural Research, v. 10, n. 4, p. 877-889, 2012.
• GUIMARÃES, R. P.; FEICHAS, S. A. Q. Desafios na construção de indicadores de sustentabilidade. Ambiente & Sociedade, v. 12, n. 2, p. 307-323, 2009.
• GUZMAN, G. C.; MOLINA, M. G. Introducción a la Agroecologia como desarrollo rural sostenible. Madri: Ediciones Mundi- Prensa, 1996.
• LÓPEZ-RIDAURA, S.; MASERA, O.; ASTIER, M. Evaluating the Sustainability of integrated Peasantry Systems: The MESMIS Framework. Ileia Newsletter, Dez., p. 28-30, 2000.
• LÓPEZ-RIDAURA, S.; MASERA, O.; ASTIER, M. Evaluating the sustainability of complex socio-environmental systems: The MESMIS framework., Ecological Indicators v. 2, p. 135-148, 2002.
• MARIE, M. Evaluation of small ruminant systems sustainability. From conceptual frameworks to implementation. In: BERNUÉS, A. et al. (Eds.); Economic, social and environmental sustainability in sheep and goat production systems. Zaragoza: CIHEAM/FAO/CITA-DGA, p.61-74, 2011.
• MARZALL, K. Indicadores de sustentabilidade para agroecossiste-mas.1999. 208f. Dissertação (Mestrado em Fitotecnia), UFRGS, Porto Alegre, 1999.
• MARZALL, K.; ALMEIDA, J. Indicadores de Sustentabilidade para Agroecossistemas: Estado da arte, limites e potencialidades de uma nova ferramenta para avaliar o desenvolvimento sustentável. Cadernos de Ciência & Tecnologia, v. 17, n. 1, p. 41-59, 2000.
• MASERA, O.; ASTIER, M.; LÓPEZ-RIDAURA, S. El marco de Evaluación MESMIS. Sustentabilidad y Sistemas Campesinos. México: GIRA-Mundi-Prensa, p.13-44, 2000.
• MESMIS. Marco de Evaluación de Sistemas de Manejo Incorporando Indicadores de Sustentabilidad. Disponível em: <Disponível em: http://mesmis.gira.org.mx/es/static/mesmis_framework?culture=en >. Acesso em: abril, 2014.
  » http://mesmis.gira.org.mx/es/static/mesmis_framework?culture=en
• MOREIRA, R. M.; CARMO, M. S. Agroecologia na construção do desenvolvimento rural sustentável. Agri. São Paulo, v. 51, n. 2, p. 37-56, 2004.
• MOURA, L.; ALMEIDA, J.; MIGUEL, L. Avaliação da sustentabilidade em agroecossistemas: um pouco de pragmatismo. Redes, v. 9, n. 2, p. 133-155, 2004.
• NICHOLLS, C. I.; ALTIERI, M. A.; DEZANET, A.; et al. A rapid, farmer-friendly agroecological method to estimate soil quality and crop health in vineyard systems. Biodynamics, Outono, p. 33-40, 2004.
• RAO, N. H.; ROGERS, P. P. Assessment of agricultural sustainability. Current Science, v. 91, n. 4, p. 439-448, 2006.
• REED, M.; FRASER, E. D. G.; MORSE, S.; DOUGILL, A. Integrating methods for developing sustainability indicators to facilitate learning and action. Ecology and Society, v. 10, n. 1, p. 1-6, 2005.
• ROY, R.; CHAN, N. W. An assessment of agricultural sustainability indicators in Bangladesh: review and synthesis. The Environmentalist, v. 32, n. 1, p. 99-110, 2011.
• SANCHEZ, G. F.; MATOS, M. M. Marcos Metodológicos para Sistematização de Indicadores de Sustentabilidade da Agricultura. Cadernos [SYN]THESIS, v. 5, n. 2, p. 255-267, 2012.
• SPEELMAN, E. N.; LÓPEZ-RIDAURA, S.; COLOMER, N. A.; ASTIER, M. Ten years of sustainability evaluation using the MESMIS framework: Lessons learned from its application in 28 Latin American case studies. International Journal of Sustainable Development & World Ecology, v. 14, p. 345-361, 2007.
• SULVARÁN, J. A. R.; RIECHE, A. K. S.; VARGAS, R. A. D. V. Characterization of Cocoa (Theobroma cacao L.) Farming Systems in the Norte de Santander Department and Assessment of Their. Sustainability Revista Faculdad Nacional de Agronomia Medellín, v. 67, n. 1, p. 7177-7187, 2014.
• TOMMASINO, H.; FERREIRA, R. G.; MARZAROLI, J.; GUTIÉRREZ, R. Indicadores de sustentabilidad para laproducciónlechera familiar enUruguay: análisis de tres casos. AgrocienciaUruguay, v. 16, n. 1, p. 166-176, 2012.
• VERONA, L. A. F. Avaliação de sustentabilidade em agroecossistemas de base familiar e em transição agroecológica na região sul do Rio Grande do Sul. 2008.193f. Tese (Doutorado em Ciências), Univ. Federal de Pelotas, Pelotas. 2008.
• VERONA, L. A. F. A real sustentabilidade dos modelos de produção da agricultura: Indicadores de sustentabilidade na agricultura. Horticultura Brasileira, v. 28, n. 2, p. 52-66, 2010.
• VILAIN, L. La Méthode IDEA: Indicateurs de DurabilitédesExploitations Agricoles. EducagriÉditions, 2008.
• YUNLONG, C.; SMIT, B. Sustainability in agriculture: a general review. Agriculture, Ecosystems & Environment, v. 49, n. 3, p. 299-307, 1994.
• ZAHM, F.; VIAUX, P.; VILAIN, L.; GIRARDIN, P.; MOUCHET, C. Assessing farm sustainability with the IDEA method: from the concept of agriculture sustainability to case studies on farms. Sustainable Development, v. 16, n. 4, p. 271-281, 2008.

Publication Dates

• Publication in this collection
  July-Sep 2015

History

• Received
  22 July 2013
• Accepted
  03 Sept 2014