benchmarks that evaluate government organisation’s efforts. They try to establish some form of impact, be it citizens that are ‘happier’ or waiting lists that are shorter because of the introduction of e-government. Only a few benchmarks deal with these kinds of indicators. They are, of course, also the hardest to put into practice and require primary data gathering: interviews with citizens, overall evaluations of organisation’s efforts.
They do measure in a much more direct sense than usage indicators (which represent
‘consumer power’) the actual satisfaction of end users, or more generally the way that things have been improved because of e-government.
3.4.5. Environmental indicators
Examples of environmental indicators
x ICT penetration rates (pc, internet, mobile phone) private households, work, schools
x Indicator that measures ‘fear of invasion of privacy’
x Online shopping rates as an indicator of trust in online environments
x Indicator that measures ‘quality of legislation concerning the information society’
x Telephone tariffs, GSM tariffs, Internet access tariffs
The environmental, or ‘readiness’-indicators do not measure e-government as such, but instead measure some of the preconditions of a successful e-government. They are indicators of the e-society that is the surrounding environment of e-government, and mostly have to do with ICT infrastructure, ICT skills, trust in ICT and the legal environment. ICT infrastructure is one of the basic requirements of online government and can be measured by indicators such as internet penetration rates, broadband penetration, internet access tariffs, amount of public access points, and so on. ICT skills have to do with the way a country’s population is able to handle computers and ICT. A further categorisation here distinguishes ICT skills among citizens, businesses, and civil servants. Another indicator that is sometimes used here concerns the presence of scientific or academic institutions that excel in ICT knowledge.
A third group of environmental indicators indirectly measure trust in online environments by measuring the presence and success of e-business and e-commerce. A final group then, focuses on a country’s legal environment and assesses this in the light of the requirements of the information society, dealing for example with the juridical value of an e-mail and with the issues of online identification, online safety and online privacy.
4. Alternative approaches to e-government benchmarking: qualitative versus
next e-government benchmark. As we have seen in the first section of this chapter, this is the first stage of benchmarking. What has happened with the other two stages of benchmarking: the analysis of processes and the improvement of processes? Recently, this lack of interest in back-office processes of e-government has been acknowledged.
In this section we will discuss two interesting, although very different, initiatives. The first approach is this of Kunstelj and Vintar from the University of Ljubljana (Slovenia) [30]. These authors developed a quantitative approach to measure progress of the back- office integration of e-government projects. The second approach is a study by Millard et al of the Danish Technological Institute and the German Institute für Informationsmanagement of the University of Bremen [31]. This is a qualitative approach. We will briefly discuss these two different approaches and try to learn some lessons about the value of both quantitative and qualitative methods of benchmarking back-office processes.
4.1. Quantitative study of back-office processes
Kunstelj and Vintar draw the conclusion that the slow development of e-government is caused by the lack of attention for back-office reforms [30]. Due to the focus of benchmarking studies on front-office solutions, governments give priority to front- office quick wins over the re-engineering of back-office processes. In an attempt to solve this problem Kunstelj and Vintar propose an alternative method of e-government benchmarking. Their approach has a double goal. First, they want to present a comprehensive method for the evaluation of e-government initiatives that not only focuses on front-office solutions but also takes into account back-office reforms.
Second, their model has to be a source for the development of guidelines for the implementation of e-government.
The model proposes indicators at four levels: the environment, the back-office, the front-office and the impact of e-government. Compared to the benchmarking studies from our analysis, the indicators concerning the back-office processes, in particular, are important. To measure the development of the back-office several indicators are proposed. For example the availability and use of information systems in the back- office and the informatization of databases. Also, a method has been developed to quantify the progress of back-office integration. In order to measure back-office integration the approach focuses on service delivery as well as on process reform. For both these aspects, the level of development and the level of integration is measured in a quantitative way. The level of development of a service or a process refers to the level of informatization. Is a service or process carried out manually or automatically (for example pro-active services or a fully automated process)? The level of integration refers to one way in which services and processes are dispersed or integrated. Are services and processes scattered over a lot of different administrations or are they linked to one another? Each dimension is broken up into several stages with a hierarchic structure. The higher a particular service or process is placed in the hierarchy of stages, the higher its score will be. For example, to measure the level of process development a five-stage model was designed: a process can be carried out manually (0 points), partially electronic (1 point), through a workflow with separate information system (2 points), electronically (3 points) or automatically (4 points).
This method goes further than the first stage of benchmarking we mentioned in the first section of this chapter: the benchmarking of products. The method not only looks at products (front-office solutions) but also at the processes behind these products
K. Snijkers et al. / Critical Review of e-Government Benchmarking Studies 81
(back-office process reforms). Beside the interest for back-office reforms the method also states clearly that one of its objectives is to be a source of guidelines for the development of e-government. So, processes are not only measured, there is also an attempt to learn from these processes and to transfer knowledge about them. However, the authors are not quite clear about the way in which this knowledge has to be transferred.
4.2. Qualitative study of back-office processes
Millard et al start from a similar position as Kunstelj and Vintar. They argue that there has been little attention for back-office reorganization in general, and for the relation between back-office reorganization and the quality of public services in specific [31].
The objective of their study is threefold. First they want to present good practices of back-office reorganization. Second, they try to demonstrate the relation between back- office reorganization and the quality of public services. Third, attention is paid to the transferability of knowledge about good practices.
Although Millard et al start from the same point of departure as Kunstelj and Vintar, they took a different approach. Whereas Kunstelj and Vintar propose a quantitative benchmark, Millard et al propose a qualitative benchmark. They started with a detailed study of 29 cases spread over different countries and different service clusters. Information was collected by web-research, telephone interviews and face-to- face interviews. Findings from these cases are presented in two ways. First, five management issues are identified (for example the management of change or the way in which user needs are met). For each management issue best practices are described.
Second, the cases are presented according to the type of service delivery. In this way, for a certain type of service delivery (for example electronic tax filing) different best practices are elaborated.
The analysis of 29 cases results in the formulation of eight possible back-office reorganization strategies. Millard et al distinguish (1) the digitalization of an unchanged back-office, (2) the reorganization of a back-office, (3) the centralization of the back-office combined with the decentralization of the front-office, (4) a back-office clearing house, (5) generic types of interaction between user and agency, (6) the creation of portals, (7) pro-active services and (8) the creation of user control [31]. The authors describe each strategy and give examples and evidence from their case studies.
Special attention is given to the transfer of knowledge. Because of the qualitative description and analysis of the best practices, practitioners can obtain a detailed and nuanced view on back-office process reorganization. In this way, you do not only identify best practices, but you can also understand them.
Although this benchmarking initiative, at the first glance, seems to be the odd one out in the large pile of benchmarking initiatives, it may very well come the closest to what a real benchmark is supposed to be. According to Waalewijn, Hendriks, and Verzijl the best form of benchmarking is strategic benchmarking. This form of benchmarking focuses on the way in which another company has obtained a certain competitive advantage [4]. So, strategic benchmarking does focus on the products, the processes, and tries to learn from these processes. This is exactly what Millard et al try to do: identify e-government best-practices, try to understand these best-practices and formulate strategies that can be of help for other governments to realize a higher quality of public services.
K. Snijkers et al. / Critical Review of e-Government Benchmarking Studies 82
4.3. Quantitative and qualitative benchmarking: two sides of the same medal
To end this chapter on e-government benchmarking we want to analyze the difference between a quantitative and a qualitative approach of e-government benchmarking. The most important criticism that emerged from our analysis was that most of the benchmarks do not pay attention to back-office reforms. Now, the two benchmarks mentioned above do both specifically pay attention to the back-office of an e- government project. However, the first developed a quantitative approach while the second developed a qualitative approach. What is the best way? Probably a combination of both.
A quantitative approach poses some severe problems. A first problem is the risk for goal-displacement [5]. When a quantitative benchmark has been established, the risk exists that countries do not focus on the best strategy to develop their e- government program, but on the way in which they can rise in a ranking. So, the goal of improving public services is displaced by the goal of doing well in a ranking. A second problem with quantitative benchmarks is their impossibility to give a nuanced view on e-government developments. Because all countries are measured with the same (rather general and abstract) instrument, differences between countries, administrations or services are not taken into account. A third problem is the lack of explanatory power of a quantitative benchmark. A quantitative benchmark can tell you which country is doing better or worse than another, but does not explain the reasons why.
A qualitative benchmark can give an answer to the shortfalls of a quantitative benchmark. In, for example, case studies of different e-government initiatives a lot of attention can be paid to the context and the particularities of a project. A qualitative benchmark has a lot more explanatory power than a quantitative benchmark has.
However, a qualitative benchmark has its own weaknesses. A great advantage of a quantitative benchmark is the pressure it puts on organizations to perform better. This is a very important characteristic, especially in the public sector were there often is no, or little, external pressure on public administrations to perform better. Public sector organizations often have a monopoly and citizens often are compelled to make use of a public service (for example paying income taxes) [32]. As this is the case, there is no battle for customers for a lot of public sector organizations, as there is for private sector organizations. A quantitative benchmark, with a ranking, can function as a substitute of these forms of pressure: it can create competition between different administrations to improve the quality of their service delivery.
As both quantitative and qualitative approaches of e-government benchmarking have their advantages and disadvantages the best way to benchmark e-government practices is a combination of both methods.