6.4 Performance Agreements: Goals and Results
6.4.2 Assessment
In the meantime, the results of the performance agreement for 2007–2009 have attracted some highly critical judgements (Austrian Science Board2007; Gantner 2007; Hagleitner and Loisel2008; Court of Audit2009:3–24). The following is a summary of the main findings from the economic point of view. An assessment in terms of the political logic is to be found inChap. 5.
16 General provisions for the performance agreements are to be found in Art. 13 UG 2002 and URÄG (2009).
17 Pursuant to URÄG (2009) the University Council approves not only the draft of the performance agreement but also that of the annual special funding requirement. The University Council also has the right to issue an opinion on the performance agreement before it is finalised by the Reactor but not on the special funding requirement.
Severe criticism came very quickly from the Austrian Science Board, which complained of a lack of commitment to forward-looking development. According to the Science Board, most of the universities’ development plans were not worthy of the name: on the one hand they were not supported by relevant data with regard to the required resources and on the other hand they were vague,18especially in terms of priority-setting. The Ministry was criticised on the grounds that inade- quate funds were employed, that the whole process resembled the old bud- get allocation model and that the universities had been let off the hook in advance with the assurance that there would only be winners.
It is true that the draft performance agreements are meant to be derived from the university development plans and that the agreements and their funding were designed as ‘steps in the realisation of a binding university concept’. Due to
‘clearly inadequate funding’, however, that did not happen. Only in exceptional cases was the result of the negotiations the sum of the financial requirements for the university’s agreed plans and goals, while the last round of negotiations was no more than a veritable process of haggling over additional resources‘(Gantner 2007:11).
The dearth of information available in advance of the performance agreements also gave cause for complaint. Just as the goals and plans in the universities’ draft performance agreements were to be derived from their respective development plans, the Ministry was expected to have produced an overall strategy for the country’s tertiary education sector or at least for the universities. According to the Austrian Court of Audit, however, that was not the case: ‘The Ministry had no overall strategy for steering the programmes offered by the various universities’
(Court of Audit2009:3).19Similarly: ‘Before the start of the negotiations for the first performance agreement, the universities were largely lacking in information relating to any specific focus in the further development of the tertiary education sector and university research. They accordingly submitted repeated requests to the Ministry for priority-setting with regard to the range of activities expected of the universities’ (Court of Audit2009:Sect. 6.1). Not until the release of the 2008 University Report was it possible to identify the Ministry’s internal priorities for the financial negotiations with the universities.20
With regard to sanctions for failure to achieve the set goals, the authors of the study produced by the St. Gallen Management Center said (Hagleitner and Loisel
18 ‘Far too often the impression cannot be avoided that the universities merely compiled the wishes of the various university units (e.g. departments and faculties).’ What was intended as a sustainable focus-setting and profile-building exercise is revealed as a mixture of more-of-same and wishes for the future’’ (Austrian Science Board2007:4).
19 For examples and key data, see: Mittelstrass (2008); Austrian Science Board (2008a,b).
20 According to the report, the Ministry’s priorities for university funding were the innovative developments already launched, reorganization projects and the priorities listed in the development plans, improved staff-student ratios, implementation of the Bologna Declaration, support for young scientists, and an increase in the number of chairs awarded to women (BMWF 2008:79).
2008:11): ‘For many, the question of consequences for non-achievement is unclear:’ In a contract, provision must normally be made for impairment of per- formance, but that was not done in this case.
As far as the ‘clearly inadequate funding’ is concerned, the Ministry allocated about 3 % (€186 million) of total federal university financing to new and continuing focal fields. The remainder was assigned to general funding for the universities in the form of the basic and formula budgets. Largely as a result of earlier allocations for building projects, plus a reimbursement item for stand-by duties at the medical universities, however, this sum was greatly reduced even before the negotiations started (to€117 million), with the result that the Ministry had to offer the universities about 50 % of the ‘retention for special funding requirements’, i.e. the reserve for the following years, so as to obtain an agreement on the performance agreements, an amount that significantly limited scope for further interventions and assistance during the performance agreement period.
In the 2008 government programme and the Ministry’s presentation for URÄG (2009), the performance agreement was defined as a ‘discourse and funding tool’.
The former has so far been true: ‘Never before have there been such intensive and targeted discussions between the Ministry and the universities on the content of university development’ (Austrian Science Board 2007:5). The first round of performance agreements, which had pilot project character with a focus on the learning effects, did not do justice to the funding aspect, however. Many of the Rectors questioned came to the verdict that ‘the content of the performance agreement had hardly any influence on the allocation of funds’ (Hagleitner and Loisel2008:10). The Rectors themselves naturally preferred to have only a small volume of flexibly allocable federal funds for the performance agreements so as to avoid any unduly big surprises.
The Court of Audit also criticises the fact that, in the performance agreement process, hardly any significance has so far been attached to the cost of per- formance by the universities. ‘The question whether a university could also achieve the agreed performance at a lower cost or how high the costs are relative to those of comparable universities could not be answered on the basis of the available information’ (Court of Audit2009:15, text no. 16.1). The degree of correlation between the development plans and their funding, and also between the draft performance agreements and the results of the funding negotiations has so far been low. This is a consequence of the fact that neither the development plans nor the goals and plans in the draft performance agreements had to be quantified in financial terms by the Ministry. In those cases, both parties benefitted from an improved negotiating position at the internal and external levels.
The above comment made by the Austrian Court of Audit raises the question of the comparability of the universities. In this context, one has to consider the structural and qualitative problems of the universities’ material resources. As far as the structural problem is concerned, it is not logical to measure general univer- sities, technical universities, medical universities and art universities all by the same yardstick; they are too heterogeneous with regard to the needs of research
and teaching, and also—given the differences in size—human resources, buildings, capital goods, etc. Even between comparable types of universities, major differ- ences are to be found in terms of resources. That is the result of long years of lobbying with varying degrees of success by representatives of the individual universities, present realities such as student numbers, personnel structure, history (e.g. new versus old established universities), and the condition and functionality of the buildings.
With all these differences in structure and quality, the Austrian universities were made legally autonomous. For the performance agreements negotiated to date, the Ministry has based its approach on a simple working hypothesis:
according to the Ministry, the 21 universities are fully comparable because they have been able in the past to offer their programmes in their respective fields and with the available resources, and the same can be expected in the future. This simplified assumption, however, does justice neither to the past nor to the future:
on the one hand no attempt has been made to assess the extent to which quality problems and students’ inability to complete their courses in the prescribed number of semesters are due to inadequate funding relative to the universities’
functions; on the other hand expansion of their study programmes and the other future cost increases will have to be covered out of a budget for which only minor nominal annual increases are foreseen.
The funding shortfall is not easy to quantify for the individual universities, and qualitative aspects would have to be taken into consideration. Such a national financial survey is the responsibility of the coordinating ministry. It requires dif- ferentiation by subject with regard to student numbers as it is not possible to operate in terms of square metres per student where universities are using his- torical buildings or facilities that have become dysfunctional with changing requirements in research and teaching.
The performance agreements have so far paid too little consideration to the significant structural and qualitative differences between the various types of universities and their resource situation as the product of historical developments (‘‘backlog’’). Nor has any attempt been made to really address the question of the essential minimum (‘‘basic resources of a university’’).
All in all, financing for the performance agreements reflects the old input- oriented and incrementalist approach that had been considered a thing of the past:
no attempt is made to start with a clean slate; the status quo is treated as a given, with attention focussed primarily on add-ons and budgetary growth.
As stated above, only when basic funding is guaranteed does it make economic sense to discuss and finance new focus-setting plans and related profile-building.
As a consequence of this problem such projects are often not costed correctly nor financed to the extent commissioned. On the contrary, the co-financing offered by the universities signals to the Ministry a corresponding degree of interest on their part. And any additional funding tends to be granted with no consideration for the resulting operating costs, and after the initial start-up there is often no guarantee of continued financing in future budget periods.
The results of the cost/performance calculations could point the way forward, but in their current state of development they are not yet robust enough and they are insufficiently coordinated for purposes of interuniversity comparisons. As there can hardly be any universities with a level of resources that is ‘too good’, it will certainly be necessary to upgrade those that are currently worst provided for in line with a mutually agreed standard, if their performance offerings are to continue to be consumed. The real difficulties in this context doubtless lie in the non-economic sphere.
Performance agreements involve the universities in a huge investigative effort of data capture and the Ministry in a very considerable burden of processing. In the first round of agreements, that included verification of the match between the performance agreements and the development plans for 22 universities and anal- ysis of the above mentioned 1210 (!) goals and plans. In future, the Ministry will be additionally confronted with the financial statements, the intellectual capital statements and the university data that are also a regular reporting requirement of the universities, all of which is essential if interuniversity comparisons are to be made but which at the same time jeopardises them through information overkill. In this context, the Austrian Court of Audit also finds critical words: ‘The need for individual ratios must be examined critically in the light of the cost of data acquisition’ (Court of Audit2009:14, text no. 12.2). Similarly: ‘It was found that in general only very few ratios from the intellectual capital statement surfaced as parameters in the performance agreements’ (Court of Audit 2009:14, text no.
13.1).
The Court of Audit also recommends that the number of university plans be limited, that the plans be prioritised and that the reports to be submitted to the Ministry by the universities be assessed in terms of the content required by the Ministry for its steering and information functions (Court of Audit2009:23–25).
With regard to the performance agreement itself, there is a one-sided distri- bution of future risks to the detriment of the universities during the term of the agreement, and this has received too little public attention. The universities, for example, have to meet the additional expenditures arising from increasing student numbers and their distribution among study programmes that vary in terms of real costs—something they can rarely influence themselves—out of the fixed amount of the performance agreement. Regular difficulties also arise in terms of legislative risk. As in the case of URÄG (2009), additional costs deriving from changes to the legal requirements are reimbursed inadequately or not at all.
Nor do the Rectors know what the rate of inflation will be for the term of the performance agreements. And that is a major factor for developments in costs on the supply side (collective bargaining agreements, lease and rental payments). In addition, to the annual special funding requirements, which can be expected to involve co-financing on the part of the universities, the latter are particularly affected by increasing expenditures deriving from collective bargaining agree- ments for university employees, price increases for services and energy, cost-of- living adjustments for leased buildings, building adaptation works, as well as cost increases for general refurbishments and replacement buildings in the construction
and fitting-out phases. Nor should the additional costs be forgotten that derive from the mandatory structural measures required to satisfy industrial safety regulations.
An assessment of the first performance agreement thus provides an unsatis- factory picture but also reveals a number of points of departure for improvements in the next round.