Table of Contents
- 3.Process for development of indicators
- 4.Using indicators in assessment processes
- 5.Combination with other methods
This text is based on Guenther, D. (2006). Accounting tools, physical analysis tools and indicator sets. SustainabilityA-Test. http://www.ivm.vu.nl/en/projects/Archive/SustainabailityA-test/index.asp (13 December 2013).
Since the UN Conference for Environment and Development (UNCED) 1992 in Rio de Janeiro indicator−based assessments of international, national, regional and/or local sustainability have received an increased amount of attention. They should help to understand, capture and communicate information about the complexity of reality and to resume them to information with high insights about the assessed system. As with the most methods used for an assessment indicators have the general aim to inform the public, experts and decision makers on the actual state of the system of concern and to assist to analysis and decision making.
A huge amount of different indicator sets, differing in their spatial temporal and thematic scopes, have been developed meanwhile by various agencies on all political levels. And the international scientific and policy community is making strong efforts to further develop appropriate indicator−based assessment sets for measuring sustainable development and to implement the collection of SI for assessments.
As “bellwether tests of sustainability” (Sustainable Seattle, 1993) indicators are supposed to provide both, insights to the state of a (eco− and/or socio−) system operationalized in different state characteristics and their linkages and definitions of a “desirable” future state of a system. Therewith indicators should measure the distance between the desired and the current state.
Indicators in general are defined as correlates or proxies depending on a interesting dimension or component of a complex system under consideration which cannot be measured directly (Coenen et al.,2001). The aim is a) to provide insights to the state and the development of the system by reducing the complexity, b) to assess by quantitative and/or qualitative criteria the interested issues and c) to describe a desired future stage of the system and its components.
According to MacLaren (1996: pp186) Sustainability Indicators (SI) in contrast to the already longer existing indicators for particular dimensions of sustainable development (e.g. ecological, economical, and social) should be:
- integrating in the sense of attempting to cover linkages among the environmental, economic and social dimensions of sustainability
- forward−looking to be able to measure progress towards achieving intergenerational equity, a key issue of sustainable development
- distributional, i.e. to be able to measure not only intergenerational aspects but also intragenerational issues in terms of accounting distributions of conditions within populations and spatial units and
- developed with input from multiple stakeholder in order to archive influential, valid and reliable indicators within a value−laden and context−specific concept as sustainability.
The intended purpose is to set priorities and the request to take actions to achieve the intended goals of sustainability. However, as both sustainable development and indicators are complex and value−laden concepts with normative implications, they also provide broad space for discussion and controversy. Hence, indicators are never a complete and entire picture of the reality, which they should represent (Benninghaus, 1994). Due to a variety of reasons they are not coinciding exactly with the systems dimension of concern. There is always a difference between the indicator(s) and concerned dimension. The term “indicator” should be taken therefore literally as it provides just an indication of the systems conditions and problems (MacLaren, 1996). As an indicator can also fail in representing the issue under concern it is supposed to base the assessment always on multiple indicators and to develop indicator sets to address the different aspects of the interesting dimension However, this can lead to a conflict of the aim of reducing complexity, as such indicator set can become very complex itself. One has to find a balance between validation requirements and complexity reduction.
Classification of indicators
A lot of different SI−sets exist for an indicator−based assessment. Depending on their specific focus and understanding of sustainability these SI sets are rather different and can be grouped by different classifications.
First, all SI−sets are based on a specific conceptual framework, which defines the basic assumptions of the assessment, e.g. what are the issues of concern, from which perspective the assessment will be done and how sustainability is understood. These conceptual frameworks are supposed to ensure the comprehensiveness or at least the consistency of the selected indicator set. Each conceptual framework establishes a unique approach of assessing information about the system under consideration (Besleme/Mullin, 1997:43).
MacLaren (1996) provides a comprehensive overview on different conceptual frameworks of indicator development and use. She distinguishes 5 different frameworks, namely:
- domain−based: This framework starts with defining the key dimensions of sustainability and identifies indicators for each of them. According to MacLaren it is most effective in covering all dimensions of sustainability, but fails in linking the indicators with sustainability goals.
- goal−based: This framework starts with the identification of the sustainability goals and creates one or more indicators for each goal or goal combination. One weakness of this approach is that it has difficulties in capturing complex interrelations among the various dimensions of sustainable development, as it is still a fairly simple indicator approach.
- sectoral: Within this approach indicators are developed for each sector for which the administration has typically responsibilities. Therefore the actual coverage of sectors depends on the political scale the indicator set is aimed for. It is suited to assessments of policy performance, but is not very effective in indicating linkages across different sectors.
- issue−based: This framework starts with identifying and listing of key issues of sustainability in the system of concern, e.g. air quality, water resources, employment. To theses key issues one or more indicators are linked.Like the sectoral framework the issue−based framework is rather context specific as the key issues many differ among assessment contexts.
- causal: This framework goes beyond the above one by considering cause and effect relationships. Human activities are seen as affecting environmental conditions as stressors, which in turn has impacts to health, economy, and social conditions. To these impacts policy is responding due to alleviation of the stressor or modifying the environmental conditions directly. Within the framework indicators will be developed for the stressor, conditions and the policy reponses. However the distinction between the different cause effect relations are not always clear.
The most commonly used conceptual frameworks in sustainability assessments are the domain−based and causal frameworks, often applied in combination.
Within the ecological dimension of sustainability the OECD (1993) has developed the Pressure−State−Response framework which groups the information about the underlying system in three interdependent categories. The first category „pressure −indicators“ gives information about man−made environmental stress. The „state indicators“ aim at informing about environmental quality and last the „response indicator“ category indicates how and to which extent the society react to environmental changes. This causal framework within the ecological dimension of sustainability was enhanced in 1996 by the UN−Commission on Sustainable Development (CSD) and the UN Statistics Division (UNSTAT) with the Driving Force – State − Response (DSR) framework. This framework was developed together with a catalogue of 134 indicators for all three dimensisons of sustainable development. This framework again has been enhanced by the EEA with the pressure−category and an impact−category to the Driving Force – Pressure – State – Impact –Response (DPSIR) framework considering all human activities (processes and archetypes) with negative and positive impacts to sustainability. Based on theses three frameworks the most approaches of indicator−sets have been development. However, most often indicator sets for sustainability assessment focus either on the ecological dimension or on combinations of ecological and economic indicators.
The indicator set of CSD and UNSTAT should be the reference set for the national reporting about progress towards sustainability, asked by the CSD (UNCSD, 1996). Hence, the approach has been adopted as well by Eurostat, the European Commission and the EEA for their own indicator−sets (Commission, 2001).
Beside the distinction of indicators according to the underlying conceptual framework, SI−sets, as already mentioned, can differ widely with respect to geographical, temporal and or thematic scopes. According to this second classification SI−sets can be distinguished into hundreds of different groups, from the global to the local level, from society assessment to assessments of individual ways of life.
A third classification of indicator−sets is according to their complexity and the aggregation level. In general, existing indicator−set approaches can be grouped on an analytical level into two different classes:
List of indicators
A list of indicators consists of series of measures that do not stay explicitly in a relation to each other. The single measure can be, but not necessarily, observed and displayed directly on a scale. It is the most widespread approach to capture and measure sustainability in numbers. A series of lists of indicators have been developed by different governmental and non−governmental institutions. These lists are more or less detailed and differ in its geographical and/or temporal scale or thematic approach. Some approaches use one list for each of the three dimensions of sustainability, others link indicators for all three dimensions to only one list.
Examples for lists of indicators are the OECD Environmental Indicators (OECD, 1993), the CSD Indicators of Sustainable Development (UNCSD, 1996), but also the European Environmental Indicators (EEA, 2005) and the majority of Local Agenda 21 Indicator−sets.
A second classe form indices, merely based on lists of indicators. In comparison to a list of indicators, however, indices aggregate the rather complex information of the underlying list into a single value to improve communication and quick reference. This implies a selection, appraisal and weighting of the underlying indicators. For this procedure, however, is rather subjective as a clear and objective basis is not existing (Landis/Sawicki, 1998) and the single variable can't be observed and displayed directly. Furthermore, indices reflect a particular perception of and theory about reality (society and nature) and value judgement about what is important (Innes/Booher, 2000: 176) Hence, by reducing complexity through aggregation a loss in information about the underlying system and on transparency occurs as well.
The aggregation requires also a common denominator or unit. This is normally a dimensionless value between 0 and 1, a percentage figure of change, a monetary or currency unit or physical value like land−use or resources use. The selection of the denominator, however, influences as well the selection and weighting process of the variables. Aggregated measures, like indices, often only have little meaning to the potential users of the indicator set and there is agreement that it is not possible to define only one single index of sustainable development, rather that a substantial number of indicators are needed to capture all complex and important aspects of sustainability. Moreover, an Impact Assessment indicator−sets has to reflect the complexity of interactions of natural and human systems (Bossel, 2001) as well.
Aims and functions of indicators
As with the most methods used for an assessment indicators have the general aim to inform the public, experts and decision makers on the actual state of the system of concern and to assist to analysis and decision making.
The particular aim of indicators, as said already above, is to densify information about complex systems, that are not directly capturable, into signs and measures which allow the description and communication of the underlying issue. In this sense indicators to assess sustainable development, should give information about (1) the stage of the described system and the need to act for sustainability (urgency), (2) whether the system of concern is moving towards more sustainability or rather away from it (direction) and (3) where and what has to be done to improve the current situation and to archive more sustainability (intervention). This reduction and classification of information enable a temporal and spatial comparison of different states and systems and its development (Birkmann, 2001), but also with the danger of undesired lost of information if indicators are not proper selected or developed. Therewith indicators should provide guidelines to management and decision making by focusing on relevant aspects.
However, as Innes and Booher (2000) argue, indicators are not aiming to evaluate the impact of a particular policy directly. Indicators are measures of specific characteristics of an eco−system, society or other systems, which are chosen because these characteristics are seen as important. Other characteristics are not considered, which might be important for a coherent policy evaluation as well. Indicators can show improvements and policy impacts in the observed characteristics of the system, but they can not tell about whether and what policy was the cause. Thus, SI−Sets can support decision−makers in policy development.
Nevertheless, indicators can be used to evaluate policies as well. By using indicator sets not to assess the policy impact on the system but to evaluated policy performance and customer satisfaction with a policy (Osborne/Gabler, 1992), the quality of a specific policy can be derived. In this sense indicator sets are not applied to the actual system of concern, but on the policy aiming to solve the problem at state. Within this approach other types of indicators are needed and it differ from other indicator aims because it emphasises dialog with relevant actors in the design and use of indicators and their interpretation.
Process for development of indicators
To ensure the adequacy of the provided picture of the system, the selection of indicators should be mainly driven by their relevance for the issue to assess. If an issue is identified, the indicator which represents this issue best should be selected. This selection or development process is mostly derived by scientists, a central authority, or another sources external from the system under consideration but with expert knowledge on it. This traditional expert driven approach is working on an intuitive and discipline biased basis. By doing so the selected indicator should provide rather objective and knowledge−based measures of the reality. However, both the content of an issue (problem at state) as well as the indicator which should represent this content might be rather controverse. Even the desirable future state of the system might be controverse as well. Hence, the entire selection process is depending on normative implications and subjective perceptions based on personal norms and values, but also on interests. Therefore it can be judged that there is no value free indicator, but all serious indicator work is subjective and political because indicators mirror the perspectives, norms and values of the actor or group developing them. "Even with a solid scientific approach based on physical facts as well as systems theory and analysis, indicator cannot be defined without a significant amount of subjective choice“ (Bossel, 2001: 9). Thus Innes (1990) give the process of development an important role in usage and application of indicator sets by arguing that if indicators are defined as instruments for managing, this should not only consider its application, but also the development of indicators. However,´little attention is normally paid to the process of indicator production or development.
One of the most important precondition for a influential and effective indicator system is to focus on a clear purpose and to reflect the norms and values of the user, i.e. a community having a stake on the assessed system (Besleme and Mullin, 1997: 50). If this is not respected indicators sets might be of little use and meaning for the intended user. Thus Innes and Booher (2000) ague that indicator sets must be developed with the participation of those who will use and learn from them, if they expected to be useful in their application. This approach of collaborative indicator development and incorporation of stakeholders in the development and selection process is getting increased recognition meanwhile. In particular in North America various experiences in participatory or collaborative development of Sustainability Indicators are existing. In Europe this alternative approach is not that common up to now. Landeiro (2006) ascribe this to different traditions in public participation and participatory decision making. However, with the new European legislation according to public participation and transparency of policy this will change in the next years.
Beside focusing to the target audience and reflecting the purpose of the indicator sets several other important criteria and requirements for the development and selection of sound indicator−sets can be found in the literature. Most of them are rather technical driven. The most commonly mentioned technical requirements for SI−sets are:
- coverage of the major sustainability issues
- adequacy of the provided figure of the assessed issue
- international comparability
- availability of data and
- reliability of direction and target focusing
The actual development process of an indicators set follows several steps from defining the system under consideration and delimiting of the system boundary, via identifying relevant system components important for the problem at state, defining goals and the connected time horizon to reach these goals to selecting adequate indicators representing these issues. The detailed steps depending on the selected conceptual framework, the scale and thematic scope and on various other circumstance. An general overview about indicator development process is give for example by Reed (2002, 2005), Bossel (2001), Bell (1999), Redefing Progress (2002).
Using indicators in assessment processes
In the process of policy development and implementation an indicator−based assessment may support the process on different stages. Within the policy−cycle framework (Brewer/de Leon, 1983) first of all indicator sets can support the initiation phase by identifying the „urgency“ to act. This may be described best with the application of state indicators, according to the DSPIR framework. Here indicators may play the role of initial impact for the exploration of a policy.
Secondly indicator sets can support the policy−estimation phase by providing parameters to other tools, analysing the likelihood that the explored policy proposal will be successful, like models or scenario tools. Parameters link to estimation can be the assessment of risks or costs and benefits of the policy proposal, for which an indicator−based assessment can provide measures.
Finally, in the Policy−evaluation phase indicators can assess selected and implemented policy option, if the target issue of the system is selected properly and stakeholder are involved in the process. However it is not possible to entirely assess policy outcomes and impacts of a specific policy by an indicator based assessment. Impacts of policies are mostly complex as well and have influence to many different components of the human−nature system. Although indicators can measure rather good certain states of system components and therewith also the impacts of a policy to this components, it is not possible to cover all possible components impacted by a policy.
Due to wide variety of different approaches of Sustainability Indicators, the operational aspects of an indicator based assessment differ widely as well. Some raw data for indicators are accessible easily from official statistics, like economic data or many ecological or even some social data. Costs and manpower to invest for the assessment in these cases are relatively low. However, often specific data considered as important for the assessment are rather difficult to obtain, e.g. many social data or other qualitative data and reliable perceptions of stakeholders. Then the efforts to obtain the required information can be rather high, which increases the costs to invest in the assessment. In general: the more complex an indicator set is, the more cost intensive the assessment is. Both for the collection and the aggregation of data. In some cases the investments tends to increase that high, that it will be impossible to develop or incorporate indicators for a certain issue.
A second source of cost is the application of the indicator set. To be valuable for decision maker and other user indicator sets have to be updated frequently. The collection of new data for each indicator in the set might be cost intensive as well. The more complex and the more indicators the set consists of, the more cost intensive the assessment is.
However, due to participation the costs can be reduced. By incorporating stakeholders into the development process of indicators, the involved actors can contribute with their tacit knowledge and/or their data sources which might be inaccessible otherwise. Furthermore stakeholder can provide access to data sources by their own networks and further increase the data access for indicator sets.
The European Sustainable Development Strategy (EU, 1998) defined twelve aspects of sustainable development representing linkages between the three dimensions of the sustainability concept. For an useful sustainability assessment or an impact assessment of a specific policy in particular these cross−cutting aspects are rather important.
In this regard indicator−sets are rather suitable for assessments, as they can address particularly well cross−cutting aspects of Sustainable Development and in general all defined aspects by the EU can be addressed.. However, often indicator−sets concentrate only on specific single aspects without considering cross−cutting aspects. Depending on the approach and the class of indicators, they can more or less assess:
- Inter−generational equity
- Distributional equity
- Global responsibility
- Spatial scale; differ widely in its spatial coverage
Indicator−based approaches to assess Sustainable Development are intended to cover all impacts of sustainability. In fact it seems not possible to easily apply indicator−sets to of them. Impacts which could be quantified in numbers, like the most environmental or economic ones can be cover more or less easily. But for impacts which require not quantification, but rather a qualification indicator−sets have difficulties and deal with high uncertainties. That counts in particular for social impacts of Sustainable Development.
In particular impacts like social cohesion, employment quality, public health, liveable communities or social capital are quite difficult to cover with quantitative indicators.
Nevertheless, in principle it is possible to provide and to assess qualitative data as well. But this needs a translation of this qualitative data to quantitative data or another methodology to deal with qualitative descriptions. Where this is not possible mostly these aspects remain uncovered (see chapter 3.7 Strengths and weaknesses).
Easily to asses are impacts of environmental sustainability, because most of them can be quantified. Also economic impacts can be covered quite good, as economists have developed a broad spectrum of indicators to assess these kind of impacts.
Depending on the selected approach of indicator−sets and the process of their devleopement, an indicator−based assessment can also cover cross−cutting impacts, like intergenerational impacts.
Strengths & weaknesses of indicators
As shown above, sustainability indicator−sets depend strongly on the underlying comprehension of sustainable development. Therefore the quality of sustainability assessment hinges for a large part in first step on the quality of the system description. In a second step it is also depending on the selection and development of the indicators them−self.
Indicators are always subjective concepts, depending on normative implications and perceptions of the actors involved in the selection, development and application of them. There is no indicator set coming from law of nature. All of them are constructed in a more or less subjective social process. This means that for the application of indicators the applicant has to know the underlying axioms of choice and the boundary and description of the system. If not, indicators stick to be arbitrary and meaningless in communication. This holds in particular for the assessment of sustainable development, as an inherently complex and controvers concept.
From this nature of indicators it explains the most important weakness of an indicator−based assessment, namely the underlying subjectivity and therewith the implicit lacking influence. This weak point of an indicator−based assessment is also related to the process of the development of the indicator−set. Is the indicator−set developed according to the described requirements and considering the perspectives, norms and knowledge of the relevant actors, this weakness can be minimized.
A second weakness occurs from the fact that indicators tend to cover only quantitative aspects. Qualitative parameters were tried to commute into quantitative variables. Many aspects of sustainable development, however, could only be described in a qualitative way. These aspects remain often unconsidered.
Whereas ecological and economic parts of the sustainability concept could be quantified, normally social and institutional aspects are only ascertainable in a qualitative way (e.g. social tension or political stability). This excludes these issues and other which are difficult to capture in numbers from the analysis. Consequently a shift in weighting of the different dimension of the concept results from this with respect to comprehensive understanding of sustainability.
Combination with other methods
Because of its variety and general broad usability indicator−sets have got various possible links to other assessment methods.
A clear link exits to stakeholder analysis methods, as indicator−sets, because of their subjectivity, depend strongly on stakeholders. Also there is a required link to participatory tools, as stakeholders have to provide their perceptions and knowledge to a sound indicator−set and therefore give indicator−sets their meaning. Also stakeholder set the reference frame for indicator sets and the desired target state. Finally stakeholders are supposed to use indicator−sets to make them influential. Here the questions of interest and power are playing an important role, which is assessed by stakeholder analysis tools.
A further link can be identified to scenario methods and modelling tools. Indicator−sets provide the data basis for these tools and deliver the required numbers. However, also the other way round there is a link: Depending on the approach models might provide indicators to be incorporate into an indicator set. Further, scenarios can support indicator−based assessments and the development process of indicator−sets in defining the desired future state and delimiting the system boundaries.
For Simulation Tools indicator sets are applied as variables connected to each other by equations. Some authors are describing simulation models as an own class of indicators. Although this might go beyond the aim of indicators, there is a strong link between this kind of tools. Indicator can provide to data required for scenario building which are used in simulations.
Last but not least several tools considered in the Sustainable A−Test projects are based on indicators or are further aggregations of indicator−sets, e.g. Cost−Benefit−Analysis, Life−Cycle Assessments, Material−Flow−Analysis and the Ecological Footprint.
Bell, S. and S. Morse (1999). Sustainability Indicators: Measuring the Immeasurable. London, Earthscan.
Besleme, K. and M. Mullin (1997). "Community Indicators and Healthy Communities." National Civic Review 86(1): 43−52.
Birkmann, J., H. Koitka, V. Kreibich and R. Lienenkamp, Eds. (1999). Indikatoren für eine nachhaltige Raumentwicklung: Methoden und Konzepte der Indikatorenforschung. Dortmund, IRPUD.
Birkmann, J. and O. F. Martinez (2001). "Indicators for Sustainable Development for the Regional and Local Level. Objectives, Opportunities and Local Level." Eureg 9: 23−30.
Bossel, H. (korr 2001). "Assessing Viability and Sustainability: a Systems−based Approach for Deriving Comprehensive Indicator Sets." Conservation Ecology [online] 5(2): 12.
Brewer, G. D. and P. DeLeon (1983). The foundations of policy analysis. Chicago, Ill., Dorsey Press.
Coenen, R., V. Brandl, J. Jörissen, et al. (2001). Das System der Nachhaltigkeitsindikatoren. Forschungswerkstatt Nachhaltigkeit. Wege zur Diagnose und Therapie von Nachhaltigkeitsdefiziten. A. Grunwald, R. Coenen and J. Nitsch. Berlin, Ed. Sigma.
European Commission (2001). A Sustainable Europe for a Better World: A European Union Strategy for Sustainable Development, COM (2001)264 final.
European Commission and Eurostat (2001). Measuring progress towards a more sustainable Europe. Proposed indicators for sustainable development. Luxembourg, Office for Offical Publication of the Europ. Communities.
Innes, J. E. and D. E. Booher (korr 2000). "Indicators for Sustainable Communities: A Strategy Building on Complexity Theory and Distributed Intelligence." Planning theory & practice 1(2): 173 − 186.
Landis, J. and D. Sawicki (1998). "A planner's guide to the places rated almanac." Journal of the American Planning Association 54(3): 336−358.
Maclaren, V. W. (korr 1996). "Urban sustainability reporting." American Planning Association. Journal of the American Planning Association 62(2): 184.
Redefining Progress and Earth Day Network (2002). Sustainability Starts in Your Community. A Community Indicators Guide. Oakland, CA.
Reed, M., E. D. G. Fraser, S. Morse and A. J. Dougill (2005). "Integrating Methods for Developing Sustainability Indicators to Facilitate Learning and Action." Ecology and Society 10(1 [online]): online.
Reed, M. S. and A. J. Dougill (2002). "Participatory selection process for indicators of rangeland condition in the Kalahari." The Geographical Journal 168(3): 224−234.
Sustainable Seattle (1993). Sustainable Seattle Indicators of Sustainable Community: A report to citizens on the long term trends in their community. Seattle.
UNCED (1992). Agenda 21. Report of the United Nations Conference on Environment and Development, held at Rio de Janeiro from 3−14 June, A/CONF.151/26 (Vol. I). New York, United Nations.
UNCSD (1996). Indicators of Sustainable Development: framework and methodolgies. UN−DPCSD, UN.