One of the objectives of many SME and entrepreneurship programmes at regional and local level is to increase incomes and employment in disadvantaged regions and localities. Evaluations of three types of programmes for tackling geographical disadvantage through small firm promotion are reviewed: Distressed urban areas, lagging regions and distressed rural areas.
Distressed urban areas
Tables 3.11 and 3.12 describe sophisticated evaluations of state-led Enterprise Zone initiatives in distressed urban areas in the USA. Table 3.13
Table 3.8. Regional development agency grants, Ireland, Shannon Available online www.envplan.com/abstract.cgi?id=c170303
Country and region Ireland, Shannon Time period of study 1995-1997
Title of report An evaluation of a regional development agency’s grants in terms of deadweight and displacement.
Date of report 1999
Author/details Lenihan, H., Environment and Planning C: Government and Policy, 17.3, 303-318.
Objective of policy Support the growth and competitiveness of new and existing firms in the Shannon regional development agency area by providing investment grants, with particular focus on indigenous firms.
Key findings ●53% of firms reported full deadweight – i.e. the policy was judged to have made no difference to their investment project – and 79% of associated jobs would have occurred anyway.
●Only 4% of sales from assisted firms displaces output and jobs from other Shannon region companies.
Sophistication of evaluation Step 3:
●In-depth face-to-face interviews with managing directors of 73 firms that received grants in 1995.
●Data are then used to obtain estimates of deadweight and displacement in the region.
●Relies on the implicit counterfactual reported by the beneficiary, which may be biased or unreliable.
Comments Provides an in-depth exploration of the extent to which policy succeeds in avoiding two key problems – namely deadweight (supporting projects that would go ahead anyway) and displacement (indirectly reducing activity in other firms by competition for markets and resources). Here displacement is estimated only in the product market.
reviews a major evaluation of Enterprise Zone policies in the UK that included matched firm comparisons, but in a less sophisticated way than the two presented US evaluations. Table 3.14 shows a combined quantitative and qualitative evaluation of a UK programme that aimed to encourage the development of more innovative and specialist business support to enterprise in disadvantaged areas.
Lagging regions
Table 3.15 describes an evaluation of enterprise support in lagging regions of the UK. The support was available for inward investment projects, for local large companies and for new and small firms. However important direct and indirect (through linkages and multipliers) impacts on SMEs and entrepreneurship are picked up.
Table 3.16 focuses on a more sophisticated evaluation of SME-specific financial support in areas of industrial decline in regions of northern Italy.
Rural policy
Table 3.17 shows an evaluation of a rural entrepreneurship initiative.
Although presenting findings on employment impact this is based on analysis of management information rather than an evaluation as such. Table 3.18 sets
Table 3.9. Local innovation system policy, EU regions Available online ftp://ftp.cordis.europa.eu/pub/innovation-policy/studies/
studies_regional_technology_transfer_strategies.pdf Country and region Regions in European Union countries
Time period of study 2000
Title of report Assessment of the Regional Innovation and Technology Transfer Strategies and Infrastructures (RITTS) Scheme: Final Evaluation Report.
Date of report 2000
Author/details Charles, D., Nauwelaers, C., Mouton, B. and Bradley, D.
Objective of policy To upgrade the regional technology and innovation infrastructure in 42 participating European Union regions with reference to the needs for support shown by regional SMEs.
Key findings The programme brought more strategic thinking, created a regional dialogue, helped develop a broader concept for innovation and supported many regions in developing the scene of innovation support infrastructure and to develop actions to rationalise, better define and augment the visibility of the infrastructure.
Sophistication of evaluation Step 3:
●Questionnaires were sent to project managers and members of the steering committee in each region on regional characteristics, programme actions and results.
●A sample of case studies of projects were undertaken to identify a set of good practices through interviews with key policy actors and firms.
Comments The focus was more on the impact on innovation infrastructures and policies than on the impact of these changes on SMEs. It offers a participatory and formative approach but summative evidence is weak.
Table 3.10. Business networking, UK, North East England Available online www.ignite-ne.com/ignite/DTIEvaluations-hvstr.nsf/LookupUNID/
BB1025FD7910342B8025715C0053CE91?OpenDocument Country and region North East England, UK
Time period of study 2003
Title of report Evaluation of the RIF-funded Business to Business Project.
Date of report 2003
Author/details Public and Corporate Economic Consultants.
Objective of policy To support innovative sectoral/geographic networks and promote access to and application of new technologies by SMEs through visits and mentoring of SME networks by larger host companies.
Key findings The objectives of the programme were appropriately specified. The majority of project participants felt their objectives were largely satisfied. Around half the benefits companies ascribed to the programme would not have accrued without participation. No adverse comments were made on the management and administration of the project. There is concern over potential duplication of effort with other programmes.
Sophistication of evaluation The evaluation is both qualitative and quantitative, Step 3.
●The qualitative assessment includes desk based research on project and strategic documentation, face-to-face consultations with project partners and a telephone survey of beneficiaries. The focus is on assessment of the appropriateness of the project rationale, the management and administration of the programme, and its relationship with similar initiatives. Questions are included on the satisfaction of participants with the programme.
●The quantitative assessment included beneficiary self-assessments of impacts on their performance and an assessment of the counterfactual through asking whether participants would have otherwise taken alternative actions to achieve the same ends.
Comments This is an example of a small-scale evaluation at a relatively early stage of the project life cycle.
It demonstrates that useful information can be achieved from relatively low cost evaluation approaches, although sophisticated estimates of impact could not be obtained.
Table 3.11. Enterprise Zone evaluation, US, Indiana Available online http://papers.ssrn.com/sol3/papers.cfm?abstract_id=476122 Country and region United States of America, Indiana
Time period of study 1980-89
Title of report Tax policy and urban development. Evidence from the Indiana enterprise zone program.
Date of report 1994
Author/details L. Papke, Journal of Public Economics, Vol. 54, pp. 37-49.
Objective of policy To induce investment and enhance employment opportunities for residents in 15 distressed urban areas in Indiana state by providing tax preferences to capital and labour.
Key findings The programme permanently increased the value of inventories by about 8% in the zones and reduced the value of machinery and equipment by about 13%. Unemployment claims decline by about 19%.
Sophistication of evaluation Step 6:
●Estimates impact using changes in capital stock and unemployment before and after zone designation.
●External influences on performance are controlled for by using a panel of zones and non-zones.
●The panel approach also controls for non-random selection of areas for zone designation.
Comments Tackles the issue of area selection bias.
out a broad evaluation of the appropriateness of business support arrangements in a largely rural county of the United Kingdom.
Table 3.12. Enterprise Zone evaluation, US, Five States Available online www.sciencedirect.com/science/article/B6V89-40WDSVD-3/2/
4a74e2a35255bf9e1f30f2acb114dc63 Country and region United States of America, five states Time period of study 1981-94
Title of report Enterprise zones and local employment: evidence from the states’ programs.
Date of report 2000
Author/details D. Bondonio and J. Engberg, Regional Science and Urban Economics, Vol. 30, pp. 519-549.
Objective of policy To generate employment through business development in distressed urban areas in five states:
California, Kentucky, New York, Pennsylvania and Virginia.
Key findings The analysed Enterprise Zone programmes do not have a significant impact on local employment. Programme impact does not depend on the monetary amount of the incentives or on specific features of programme design.
Sophistication of evaluation Step 6:
●Compares zone and non-zone performance pre- and post-designation.
●A first method controls for selection bias in the areas receiving designation by including in the regression model an area-specific fixed effect and an area-specific growth rate.
●A second method controls for selection bias by estimating the designation probability of each area based on pre-designation characteristics and controlling for differences in these characteristics by including the predicted probability in the model.
Comments Tackles the issue of area selection bias by including information on the characteristics of the areas selected in econometric models. Uses the diversity of state programmes to estimate the impact of different programme designs.
Table 3.13. Enterprise Zone evaluation, UK
Available online www.communities.gov.uk/archived/general-content/citiesandregions/finalevaluation Country and region United Kingdom
Time period of study 1985-95
Title of report Final Evaluation of Enterprise Zones.
Date of report 1995
Author/details PA Cambridge Economic Consultants.
Objective of policy To overcome economic and environmental problems in zones of urban and industrial economic decline by promoting business development and physical regeneration.
Key findings Approximately 60 000 additional jobs were created in supported businesses at an estimated cost per job year of GBP 1 700 pounds per year (1994/95 prices). The most important benefit for business development was business rates relief.
Sophistication of evaluation Steps 3-5:
●The main estimates were based on recipients’ views of the difference made by the assistance.
●A comparison with matched enterprises outside the zones was used to verify the results.
Comments Provides explicit treatment of displacement and leakages.
Table 3.14. Evaluation of enterprise support in disadvantaged areas, UK Available online www.berr.gov.uk/files/file37787.pdf
Country and region United Kingdom Time period of study 2003-04
Title of report Evaluation: The Phoenix Development Fund.
Date of report 2005
Author/details Peter Ramsden, Freiss Ltd.
Objective of policy To tackle social exclusion by supporting innovative projects providing business support to enterprise in disadvantaged geographical areas and to groups currently underrepresented among business owners.
Key findings The Fund has encouraged new innovative approaches to providing outreach services. Projects were successful at reaching underrepresented groups. The Fund has not succeeded in transforming the mainstream provision of business support.
Sophistication of evaluation Step 3:
●The main impact assessment was through a client survey that measured the support received and beneficiary views on the impact of that support.
●The client survey was accompanied by 24 case studies of supported service providers to understand the delivery methods and services that were developed and their rationales. The focus was on uncovering interesting practice.
Comments There was particular focus on assessing whether the programme encouraged fresh thinking in the types of outreach services provided and the demonstration effects on mainstream providers.
Insufficient information was available on costs of different approaches used by projects to reach a judgement on value for money.
Table 3.15. Regional policy evaluation, UK Available online www.berr.gov.uk/files/file22008.pdf
Country United Kingdom, Assisted Areas
Time period of study 1999-2000
Title of report Evaluation of Regional Selective Assistance 1991-1995: Final Report.
Date of report 2000
Author/details Arup Economics and Planning.
Objective of policy To create and safeguard employment in Assisted Areas in a cost-effective manner.
Key findings Almost 40 000 net discounted permanent job equivalents were supported in Assisted Areas over the period 1991-1995. The policy has operated in a cost effective manner, with a central estimate of GBP 17 500 per net discounted job in 1995 prices.
Sophistication of evaluation Step 3:
●Estimates of additionality and impact are based on recipients’ views of the difference made by assistance.
●Displacement effects are estimated from respondent information on the location of their competitors. Displacement outside of the assisted areas was not considered.
●The timing and duration of jobs created are considered important. The number of job years the support created was estimated and job years closer to the present were assumed to be more valuable through discounting to present value job years.
Comments A survey of recipient firms provided information on expected job life, additionality, displacement and linkages, which was used to adjust gross employment information provided to government by all recipient firms.
Conclusion
T h is s e c t i o n h a s s e t o u t a nu m ber o f su b - n a tio na l S M E a nd entrepreneurship policy evaluations that show the range of evaluation options. It is hoped that the methods outlined and the results demonstrated will build the demand for more and better evaluation. Some of the evaluations reported lack the sophistication required to provide robust estimates of programme impact, although there are also examples of very good quantitative evaluations falling into the categories of Steps 4-6 of the Six Steps
Table 3.16. Regional policy evaluation, Italy Available online http://eur.sagepub.com/cgi/content/abstract/13/3/225 Country and region Italy, EC Objective 2 regions
Time period of study 1984-98
Title of report Do Business Investment Incentives Promote Employment in Declining Areas? Evidence from EU Objective 2 Regions.
Date of report 2006
Author/details D. Bondonio and R. Greenbaum, European Urban and Regional Studies, Vol. 13.3, 225-244.
Objective of policy Increase economic development in regions of industrial decline through packages of SME incentives.
Key findings 24,000 additional jobs between 1995 and 1998 can be attributed to the programme at an average cost of 21,000 euros. Incentives were more successful when rewarding growing industrial sectors.
Sophistication of evaluation Step 6:
●Longitudinal parametric model analysing firm employment data aggregated by province and industrial sector.
●Exploits the fact that firms in non-target areas receive incentives as well as firms in target areas, but that Objective 2 increases the overall programme budget (and hence the number of firms supported) in target areas.
●Trends in supported and non-supported firms are included in target regions and non-target regions.
Comments Tackles area selection bias.
Table 3.17. Rural policy evaluation, Canada, Quebec Available online www.dec-ced.gc.ca/Complements/Publications/Evaluation-EN/eval_entr_rurale_en/css/
eval_entr_rurale_en.htm Country and region Canada, Quebec Time period of study 1999-2000
Title of report Evaluation of the Rural Enterprise Program. A Component of the Bas Saint Laurent and Gaspésie/Îles de la Madeleine Regional Strategy.
Date of report 2000
Author/details Cérac Inc.
Objective of policy To support economic activity in rural communities by financing support to new and small enterprises Key findings 59 grants of between CAD 11,000 and CAD 25,000 were associated with the creation of 1.7 jobs and
protection of 3.0 jobs per project approved.
Sophistication of evaluation Step 1:
●Analysed project take up and associated outputs declared in project applications
Comments No survey of beneficiaries.
approach. Other examples are interesting because of their more qualitative focus on policy options and stakeholder involvement. Ideally, quantitative and qualitative approaches should be brought together in comprehensive programme evaluations where budgets permit and there is an opportunity for significant policy adjustment.
Table 3.18. Rural enterprise support, United Kingdom, Northumberland Available online www.ignite-ne.com/ignite/DTIEvaluations-hvstr.nsf/LookupUNID/
EEF952CCD3B762B68025716B003E72CE?OpenDocument Country and region United Kingdom, Northumberland
Time period of study 2005
Title of report Business Support in Northumberland: A Review.
Date of report 2005
Author/details SQW Ltd.
Objective of policy To support business start-up and development through a range of services provided by the Northumberland Strategic Partnership, mainly operated through Business Link for Northumberland.
Key findings In reorganising services it is important to build on existing strengths and support infrastructures.
However, there is a need to develop more extensive geographical coverage and better support businesses with high value added potential. Two-thirds of respondents believed the support had a significant effect on increased sales and increased productivity. Over 80% found services vital, important or useful.
Sophistication of evaluation The evaluation combines quantitative (Step 3) and qualitative elements.
●There is an assessment of quantifiable outputs through a telephone survey of 60 businesses that received support.
●There is also assessment of qualitative results such as improving access to services and the perceived quality of services, drawn from views of companies, Business Link for Northumberland staff and staff in partner organisations.
Comments The evaluation has a broad focus addressing a package of funding streams and providers. It makes recommendations on how to organise future delivery based on what has worked well. It is largely qualitative in focus.
Policies and Programmes
© OECD 2007