Neo-liberal State
3.4 Reducing Complexity by Means of Indicators .1 Indicators—A Case Study
3.4.2 Methodology for Indicators
The essential lesson from all the above criticisms is that indicators have a very specific role to play in the statistical information portfolio and that, consequently, a specific methodology has to be applied. The methodology used here needs, however, to be tailored for this particular type of information and its utilisation. Ensuring that indicators are not politically driven, albeit policy-relevant, is ultimately the crucial issue on which the decision between trust and mistrust depends. In this respect, questions of communication and governance must be anchored from the outset in the methodology.
Of course, quantitative indicators are statistics. Not all statistics are, however, indi- cators. In Sect.2.1.5, indicators were defined as follows: an indicator is a summary measure related to a key issue or phenomenon and derived from a series of observed facts. Indicators can be used to reveal relative positions or show positive or negative change. Indicators are usually a direct input into national, EU and global policies.
In strategic policy fields, they are important for setting targets and monitoring their achievement.
Indicators are a special type of statistical information in two respects:
• First, it is a particularly high level of compression, focus and synthesis with regard to the relevant statistical message;
• Second, it is the close connection to a scope, a purpose and the associated user community.
In contrast to the broad and detailed basic statistics as well as the consistent accounts; therefore, indicators have a different quality profile (see Fig.3.2) with a particular emphasis on the criterion of relevance. They have a specific job to do, namely to condense and communicate the informational content contained in statis- tics in such a way that it can be understood and used by the respective target group.
Indicators are pointers that point to particular feature. In contrast to multipurpose basic statistics, indicators are designed (or at least should be designed) in such a way that they serve one (partly very specific) purpose.
It is now crucial to consider the two peculiarities and dimensions of indicators as interrelated and interdependent. A special focus, aggregation and consolidation of various pieces of information into one indicator is only possible if it is known for
what purpose it is planned to use that indicator. Even an already highly condensed aggregate of National Accounts, such as GDP, requires further specifications in order to be customised as an indicator to suit a specific application: shall inflation be filtered out, shall an index for growth be calculated, is it a net amount without capital depreciation, or is it a seasonally adjusted quarterly GDP that is expected?
Depending on the phase in the life cycle of a policy area targeted by the indicator, different characteristics are expected and needed. In a phase of awareness raising for a new phenomenon, it might, for example, be sufficient to work with indicators of lower accuracy and granularity while for the monitoring of target achievement, very high precision and resolution is a necessary feature of provided indicators.
Indicators can reveal, suggest, distort and conceal.70Which of these characteristics is actually built in the design of an indicator or an indicator set will determine their impact on the debate in the ‘bazaar’.71
It follows that it is neither meaningful nor possible to develop indicators external to and isolated from the system that is going to use this information, in a separate area of statistics or solely through collaboration of statistics with science. Rather, a co-construction is required, where different stakeholders contribute and participate.
From a systemic point of view, the observing and observed systems cannot be isolated from each other.
Taking note of this fact leads consequently to different questions, expectations and approaches concerning also an envisaged indicator methodology. As pointed out in the example of Sustainable Development (see Sect.3.5), it is not a question of carrying out a ‘measurement of sustainability’ as a mere academic and analytical undertaking, which is then fed into the political discussion. Rather, the point is—
and this is demonstrated by the current process at the global level—to develop and constantly improve in close connection between the political and statistical worlds the measurement and use of these measurements. The particular challenge is, here, to ensure the quality of indicators (see quality principles and Code of Practice in earlier sections) when they are produced under these circumstances. It must therefore be the goal to ensure both this quality and convince the users of this quality in communication.
Figure3.10outlines such a co-construction between a ‘laboratory’ (collaboration of statistics and science) and a ‘bazaar’ that interact and have mutual relationships.
It is the task of the actors in the laboratory, to reduce the complexity as far as is possible with the technical and methodical tools at their disposal. However, the (pre-)selection of relevant aspects, or the setting of indicator-related targets, or the definition of weighting schemes (as part of composite indicators and rankings) may then, at least to a large extent, belong to the field of politics, i.e. the ‘bazaar’. Thus, a reduction of complexity requires inputs from both sides.
In this co-construction, the laboratory (statistics and science) has two tasks:
firstly, to develop and apply methods of aggregation and synthesis; and secondly,
70See Lehtonen (2015), Ravetz et al. (2018).
71In this sense, indicators are of high relevance for human rights and (global) governance (Merry 2011).
Fig. 3.10 Co-construction of indicators (Radermacher2005)
to develop and apply methods of active engagement with the users. The decisive factor is that these two tasks are thought, conceived and implemented together, by an interdisciplinary cooperation, as explained in Sect.3.2.3.
Methodological development and research in the field of indicators is spread over different disciplines, each of them focusing on their respective field of expertise.
Furthermore, specific methodological approaches and concepts have been devel- oped side by side in single statistical communities for various sets of indicators.
While economics statistics, given that their observation units are often expressed in monetary terms resulting from market transactions, prefer an accounting approach to obtain highly aggregated economic indicators, such as inflation, growth, or pro- ductivity, this is difficult to do similarly in social and environmental statistics, where many essential variables are not monetised on actual markets.72This is one of the rea- sons why statistical methods of synthesis and aggregation come into play.73Mutual fertilisation through methodical cooperation across the disciplines should therefore be supported—not least because of new opportunities arising from new data sources, the interest in successful methodologies in neighbouring areas (e.g. geographical sci- ences and statistics) or new disciplines (e.g. data science) as a source of innovation and efficiency.
In parallel with the statistically/methodologically oriented fields, a growing group of researchers has been dealing with the sociological aspects of co-construction of indicators (general or specific ones) in recent years, pointing out the particular chal- lenges of indicators in terms of communication as boundary objects or concerning
72Even if this might change, following the analysis of Cook (2017).
73For an overview over the statistical methods and approaches, see Maggino’sComplexity in Society:
From Indicators to their Synthesis(Maggino2017).
required governance provision.74 Which function, for instance, should indicators have in the political process? Should they be used to facilitate an exchange of views (‘opening-up’) or should they be used to shorten or close a discussion (‘closing- down’)? What is the relationship between targets, goals and indicators? In which sequence should they be defined and by whom? How much evidence is available from market research for the (correct) effectiveness of indicators? Is this evidence systematically fed into the learning cycles of statistics? To what extent and at what stage of the process is a consultation or even stakeholder participation (i.e. civil soci- ety) taking place? Such questions would require significantly intensified research work, and this in collaboration between statisticians, communication experts and sociologists.
Indicators with a political weight, such as the price index, the number of unem- ployed or the GDP, did not arise overnight. Their strength stems from the fact that they have evolved over years and decades, in a constant interplay between new user demands, conceptual advancements based on scientific work and new data sources, and statistical methods. Therefore, it cannot be expected that, for complex issues (such as Sustainable Development), an indicator or system of indicators can be born with a forced act, even if the political will so demands. It is more reasonable to assume that such a system evolves and learns, that all participants (science, statis- tics, society) contribute to this evolution, and that this long-term and multifaceted process requires order, governance and management.
Indicators with a high political impact, such as the indices of indebtedness of the public sectors in Europe, which are closely linked to the Excessive Deficit Procedure (EDP, introduced with the common currency by the Maastricht Treaty75) belong to an extraordinary category of indicators that deserves a special degree of attention. As history has shown76the fact that these indicators are used for surveillance purposes and that they could result in serious consequences contains the risk of manipulation and interference by politics. The special degree of authority that has been assigned to these indicators therefore requires special governance that must go beyond the usual dose of institutional competencies of statistics.
Indicators with a high potential to influence the markets, such as the inflation rate or the quarterly GDP, deserve special attention with regard to equal access and in particular pre-release access.77 Like many statistical offices, Eurostat has
74See, for example, the following selection: Power (1994), Desrosières (2010), Sangolt (2010a,b), Desrosières (2011), Hammer (2011), Saetnan et al. (2011), Davis et al. (2012a,b), Coyle (2014), Sébastien et al. (2014), Porter (2015), König (2015), Rottenburg et al. (2015), Supiot (2015a,b), Davies (2016), Diaz-Bone and Didier (2016), Cherrier (2017), Cook (2017), Eyraud (2018), König (2018a,b), Ravetz (2018), Ravetz et al. (2018).
75For more details seehttp://ec.europa.eu/eurostat/web/government-finance-statistics/excessive- deficit-procedure.
76See the example of Greek statistics in Sect.3.2.1.1.
77See Eurostat’s policies here (http://ec.europa.eu/eurostat/about/policies/dissemination).
discussed this subject with representatives of the media and has recorded the results in a “Protocol on impartial access to Eurostat data for users”.78
Rankings and indicators with a weighting not based on observations or explicit market values do not fall into the area of official statistics. Nonetheless, it is very useful, through close cooperation between applied research and official statistics, to offer users a range of highly condensed information that combines the best available statistics with the best practices for their compression.