● This section identified five areas of SME policy that are to be found in most developed countries. It then takes examples of policy evaluations in each of
Table 2.19. Technology assistance to small firms, US Available online http://ideas.repec.org/p/nbr/nberwo/5753.html
Country United States
Time period of study Not specified.
Title of report The Government as Venture Capitalist: The long run impact of the SBIR Program.
Date of report 1999
Author/details Josh Lerner, Journal of Business, Vol. 72, No. 3, pp. 285-318.
Objective of policy Objective of SBIR not specified in article.
Key findings ●Awardees grew significantly faster than a matched sample over a decade.
●Awardees were more likely to attract venture financing.
●Multiple awards did not increase performance.
Sophistication of evaluation Step 5:
●However there is likely to be both selection bias and application bias.
Comments Reservations about the selection bias must raise a question mark over the findings.
Table 2.20. The SBIR program, US
Available online www.springerlink.com/content/nnap7wyb7e1r/?p=ba199ec0f06c48e0af015d1011f3c3ff&pi=38
Country United States
Time period of study 1982-1999
Title of report Purpose and Performance of the SBIR Program.
Date of report 2003
Author/details R.S. Cooper, Small Business Economics, Vol. 20, No. 2, pp. 137-151.
Objective of policy ●The purpose of the program is to strengthen the role of the small innovative firms in federally funded research and development as a base for technological innovation.
●To meet agency needs and contribute to the growth and strength of the nation’s economy.
Key findings SBIR addresses each of the dimensions of the financial gap. It provides:
●Recognition.
●Small amounts of capital.
●Assistance to a wide variety of technologies.
Sophistication of evaluation Step 2:
●Invited opinions of recipients and reports the results of other studies using this methodology, together with that of Lerner [1999] reported in Table 2.19.
Comments
the five areas and presents a summary of both the methods used and the key findings. Its conclusions are the following: sophisticated quantitative evaluations are the exception rather than the norm. In many countries it has been difficult to obtain any [English language] evaluations of policy undertaken by government that are in the public domain. Evaluations
Table 2.21. The UK SMART programme Available online www.berr.gov.uk/files/file22002.pdf
Country United Kingdom
Time period of study 2000-2001 Title of report Evaluation of SMART.
Date of report 2001
Author/details Public and Corporate Economic Consultants.
Objective of policy ●Increase and improve technology use and adaptation and research and development by individuals and SMEs.
●Contribute to a climate which encourages investment in innovative technology by individuals, firms and financial institutions and which stimulates a market in technological advice.
Key findings ●SMART is generally effective in relation to its objectives and provides value for money.
●There is scope to increase benefits by increasing market penetration and improving the integration of the support delivered by the scheme with the wider business support infrastructure.
Sophistication of evaluation Step 6:
●Compared the performance of SMART award winners and unsuccessful applicants after accounting for selection effects, i.e. factors that distinguish successful from unsuccessful programme applicants.
●Included in-depth qualitative interviews with scheme administrators and grant recipients on their view of the impact of the scheme and possible improvements.
Comments Combines quantitative and qualitative evidence. Qualitative stakeholder views on the usefulness of different aspects of the scheme and how its operation might be improved was critical to the recommendations on adjustments to the scheme. However, the recommendation to retain the scheme was based on value for money evidence from the quantitative evaluation.
Table 2.22. Impact of science parks, Greece
Available online www.ingentaconnect.com/content/els/01664972/2002/00000022/00000002/art00087
Country Greece
Time period of study 2000
Title of report Science Parks, a high-tech fantasy? An analysis of the Science Parks of Greece.
Date of report 2002
Author/details Bakouras, Y.L., Mardas, D.C. and Varsakelis, N.V., Technovation, Vol. 22 [2], pp. 123-128.
Objective of policy ●To promote the development of high technology firms.
●To promote this through establishing formal and informal links between NTBFs and Universities.
Key findings ●Greek Science Parks have more small firms than is typical of EU Science Parks.
●The formal links between Science Park Firms and Universities exists only in one Science Park in Greece.
●Informal links do exist.
Sophistication of evaluation Step 2:
●Details are collected from 17 firms on three Greek Science Parks.
No “controls” are used.
Comments
undertaken by outsiders – academics or consultants – are more frequently accessible, but they are still far from commonplace. Even when they are available the level of sophistication is mixed.
● The central finding is that, taken as a whole, sophisticated evaluations of SME support are on balance less likely to provide evidence of policy impact than the evaluations using less sophisticated approaches. The implication of this important finding is that policy makers, in principle, can influence the outcome of an evaluation by their methodological choices.
Table 2.23. Impact of science parks, Sweden Available online www.springerlink.com/content/100338/
Country Sweden
Time period of study 1999-2001
Title of report Science Park Location and New Technology Based Firms in Sweden – implications for Strategy and Performance.
Date of report 2003
Author/details P. Lindelof and H. Loftsen, Small Business Economics, Vol. 20 [3], pp. 245-258.
Objective of policy ●To promote the economic development of high technology firms.
●To promote the development of research.
●To promote firm/university linkage.
Key findings ●No statistically significant differences between Science Park Firms and off-Park Firms.
●The economic performance of the on-Park and off-Park firms do not differ.
●Collaboration is less with on-Park than with off-Park firms.
Sophistication of evaluation Step 5:
●10 established Science Parks. Matches 134 on Park Firms with 139 not on Park.
Comments Sophisticated study.
Table 2.24. University/SME links, New Zealand Available online www.med.govt.nz/templates/MultipageDocumentTOC____13694.aspx
Country New Zealand
Time period of study 2002-2004
Title of report Review of Polytechnic Regional Development Fund.
Date of report 2004
Author/details Ministry of Economic Development.
Objective of policy ●Enable and encourage institutes of technology and polytechnics to collaborate with regional industry.
●To build capability in ITPs to be responsive to regional training needs aligned with regional comparative advantage.
●Improve regional economic development.
Key findings ●NZD 3.8 million was distributed to ITPs.
●ITP’s and industry have significantly increased collaboration following the project.
●Participants expect this to continue beyond the funding stream.
Sophistication of evaluation Step 2:
●ITPs in receipt of funding and those not receiving funding were contacted.
●“Industry Partners” – only 20% of which were SMEs were also contacted.
Comments
● A second important finding is that larger, longer duration programmes are more likely to be assessed using sophisticated methods, than is the case of short duration, small scale programmes. However, this also presents problems since it means that a different “currency” is being used to evaluate programmes of different scales, perhaps to the disadvantage of larger programmes.
Table 2.25. Impact of management training on SMEs, UK Available online www.envplan.com/abstract.cgi?id=c110331
Country United Kingdom
Time period of study 1989-1991
Title of report The impact of government assisted management training and development on SMEs in Britain.
Date of report 1993
Author/details J.N. Marshall, N. Alderman, C. Wong and A. Thwaites, Environment and Planning C, Vol. 11, pp. 331-348.
Objective of policy ●The policy aims at business growth through management training.
●The policy provides financial assistance to firms with 25-500 employees to employ consultants to help them develop their management.
●Firms are provided with up to half the costs, maximum GBP 15 000 of the consultants.
●The employment of consultants would result in improvements in firm organisation and improvement.
Key findings ●There is a major boost to management training, combined with a smaller but continuing increase in the number of firms carrying out management training and development.
●Less evidence that human resource development thereby improves business performance.
●This may be because the evaluation was undertaken “too early”.
Sophistication of evaluation Step 5:
49 Option 3 firms were “matched” with 100 firms similar in terms of size, industry and location.
Comments
Table 2.26. Small firms training loans, UK Available www.dfes.gov.uk/research/programmeofresearch/
projectinformation.cfm?projectid=12837&resultspage=1
Country United Kingdom
Time period of study 1994-1998
Title of report Evaluation of Small Firms Training Loans.
Date of report 1999
Author/details Kevin Maton, UK Research Partnership, DfEE Research Report, No. 97, RR 1999.
Objective of policy ●Small Firm Training Loans (SFTL) are loans offered to small firms (< 50 employees) to invest in training.
●They are financed through banks but include repayment holidays and preferential rates of interest.
Key findings In terms of client satisfaction the scheme scores very highly, with 81% of businesses saying it works well, as a way of funding training for small firms.
Sophistication of evaluation Step 2:
●The study interviewed 92 SFTL clients by telephone and more in-depth interviews with others.
●It also interviewed support agencies.
Comments No control group included.
● We therefore emphasise again the points made above that the choice of evaluation approach needs to be carefully assessed.
● This inevitably raises the question of whether there is merit in undertaking what is referred to in the Six Steps approach as monitoring. Our view is that policy makers need to be aware that there is a risk that low grade evaluations – monitoring – can lead to misleading pictures of programme effectiveness.
Policies and Programmes
© OECD 2007