INSTITUTIONAL CONTEXT
9.4 Evaluation and impact of the interdisciplinary EEP
aspects). It is, however, the collective work and results of the students that is valorized, meaning that all team members are assessed based upon the collective quality of the team- work. This leaves the responsibility for the balance of workload and quality of deliver- ables to the team members themselves. Individual assessment is therefore used only for an assessment of the disciplinary skills involved in the EEP and of the students’ ongoing par- ticipation to the learning activities. Moreover, it is the quality of the group work that is evaluated, not the economic potential of their entrepreneurial project. Indeed, a group might do a good job by correctly concluding that the entrepreneurial opportunity they had considered had no or limited potential, given its features, resources requirements, competitive environment or market. The result of the project in itself cannot be the only assessment criterion. It is the ability of the team to reach this conclusion, to argument and to synthesize it that will be assessed, as it reflects their ability to adopt the integrated point of view required in an entrepreneurial context.
The master thesis project students have to complete at the end of the program provides a good illustration of this collective interdisciplinary assessment and its complexity. As each group includes members from different disciplines, they will be mentored by a team of academics including members of the corresponding faculties. These typically involve three thesis directors (instead of one in traditional projects), from three different schools of the university. This multi-headed structure, with which academics are sometimes uncomfortable, can generate negative side effects that might affect the principle of col- laboration underpinning the EEP. Indeed, each director can feel that his or her contribu- tion is diluted or underrepresented in the project, and therefore may not contribute significantly enough to its success. On the contrary, one of the thesis directors can try to bias the project towards his or her own disciplinary interest or assessment criteria, threat- ening the overall balance and coherence of the entrepreneurial project. It is the EEP manager’s responsibility to ensure that those potential pitfalls are avoided across the port- folio of projects that is generated each year.
Detailed and quantifiable assessment criteria are therefore difficult to define and/or implement, and the assessment of each project tends to be consensus based, the program manager balancing the opinions of each of the thesis directors involved. While this approach appears relatively pragmatic, let us stress that it generates some discomfort as students are faced with an assessment process that sometimes appears arbitrary or at least that lacks transparency. This assessment issue is frequently raised in the feedback pro- vided by the students, although it does not affect their overall (positive) perception of the EEP itself. This issue of the evaluation of the impact of the EEP itself, by the students and from an entrepreneurial point of view, is discussed next.
9.4.1 Students’ satisfaction
In order to assess the students’ satisfaction, a first qualitative survey (including open and semi-open questions) was sent to the first three promotions of students, having com- pleted the programs in 2000, 2001 and 2002. Based on this preliminary survey, a ques- tionnaire with 57 closed questions and three open questions (related respectively to the key strengths of the program, its key limitations and some suggestions) was defined and sent by post to the 2003 and 2004 promotions. All the students completed the first survey, and 54 per cent replied to the postal questionnaire.
The surveys provide very encouraging results: they indicate that 98 per cent of the stu- dents are ‘quite satisfied’ or ‘very satisfied’ with the EEP as a whole. The two main posi- tive motivations stated by the students relate, on the one hand, to the interdisciplinary approach of the program and, on the other hand, to the high level of interactivity of the learning process. Those two features of the EEP also appear as the main driver of student satisfaction a posteriori, indicating that their expectations regarding interdisciplinary approach and interactivity have been met.
Compared to traditional courses, the EEP requires additional efforts from these students who have to dedicate two evenings to their entrepreneurship classes during three years and to provide a lot of additional work for their different assignments. This means that these students are probably more motivated, but also probably more demanding than ‘regular’
students. In terms of evaluation, we can assume that the process is more similar to adult education programs, grouping people with different backgrounds and expectations, than to traditional disciplinary student evaluation. However, compared to adult education evalua- tion, the fact that the students have different educational backgrounds is not perceived as a problem, because interdisciplinarity is at the core of the program. As result, students are not dissatisfied because of different initial levels of knowledge among their peers.
Regarding factors that could be improved, students mention, first, the coordination among the teachers and, second, the assessment process. They report that the links between the different classes should be made more explicit, indicating that the integration between the various classes could be improved. They also tend to perceive that the col- lective assessment process lacks transparency and is somewhat unfair. In particular, as mentioned above, a small weight is attached to the individual contributions relative to the collective results of the group. Furthermore, the interdisciplinary dimension of the program makes it difficult to select detailed and explicit assessment criteria and thresh- olds that would cope with the diversity of the student projects in terms of scope, content and disciplinary knowledge mobilized. In particular the respective expectations of the teachers coming from various disciplines, and how they are combined towards a consen- sus, appear sometimes unclear to the students, or at least are not communicated clearly enough. The weakness of the program thus seems to appear in the collaboration princi- ple. However, let us remember that the interdisciplinary dimension is present at the level of the program content, the teaching team, the student groups, the guest speakers and the assessment jury. These multiple sources of interdisciplinarity probably partially explain their relative dissatisfaction.
9.4.2 Interdisciplinary dimension
In parallel with the students survey mentioned above, we have evaluated the EEP, adapt- ing an assessment tool of the interdisciplinary dimension, developed by Rege Colet 158 Handbook of research in entrepreneurship education
(2002). This tool, approved by the original author, evaluates the level of integration, col- laboration and synthesis involved in an interdisciplinary learning process, based on Likert-scale surveys. This approach validates the balance between, on the one hand, the structure of the knowledge contents (integration principle) and, on the other hand, the work processes organizations (collaboration principle).
In this approach, the integration principle is declined along four indicators: the type and level of content integration, the problem-based learning processes and, finally, the assessment. The collaboration principle is declined along four other indicators: teacher cooperation, students’ cooperation, student–teacher interactions and course settings.
Those eight indicators are each covered by several items along which the questionnaire tests the level of agreement of the students (completely agree, and so on). Finally, an inter- disciplinary index is defined as the ratio of the scores along the integration and collabo- ration dimensions. A truly interdisciplinary program should be balanced along those two dimensions, that is, achieve an interdisciplinary index of 1 (Rege Colet, 2002).
This test, conducted with the first-, second- and third-year students’ cohorts in April and December 2004, confirmed that the EEP was quite interdisciplinary, with interdisci- plinary indices of respectively 1.08, 1.17 and 0.99 for the three years of the program. The balance between the structure of the contents and of the work organization of the pro- grams during the three years appears, therefore, well perceived by the students.
The test also provides detailed information regarding the perceived strengths and weak- nesses of the program in terms of coherence of the teaching strategy. These results were consistent with the students’ satisfaction surveys mentioned in the previous section.
Again, room for improvement was identified in terms of collaborations between the teach- ers and in terms of clarity of the student assessment process. As highlighted above, col- laboration between teachers and consensus about the assessment process are contingent to the willingness and ability of teachers and speakers from distinct disciplines to work together, exchange information and experience as well as communicate and act as a team – all things that are not common and do not tend to emerge spontaneously from an acad- emic environment.
Let us stress that both evaluations were built and conducted with the intention to provide the teachers and the program managers with feedback that could be used to val- orize and regulate the quality of this complex interdisciplinary program.
Given that the first students only graduated in 2000, further investigations will be required regarding the assessment of the program in terms of short- and medium-term professional development. In particular, the professional development objective of the EEP to help the students in their process of creation of a new business activity cannot yet be fully tested. We discuss this issue of the entrepreneurial impact of the EEP in the next section.
9.4.3 Entrepreneurial impact
In the early stages of the EEP, at least from a formal point of view, its aim was limited to entrepreneurship in its most restrictive meaning, that is, new firm creation. In this per- spective, such a program could only be targeted to potential firm creators and would be conditioned by a success imperative: the number of new firms created (Schmitt, 2003).
This approach is largely predominant within university entrepreneurship courses and/or programs. Under the simultaneous influence of students and professors, the purpose of Interdisciplinary approaches in entrepreneurship education programs 159
this EEP has actually been broadened beyond firm creation in a restrictive sense, to include intrapreneurship, working for an SME, not-for-profit creation, spin-offs, and so on. Entrepreneurial intentions (Fayolle, 2003; 2005; Fayolle et al., 2005) and perceptions, such as self-efficacy (Ehrlich et al., 2000) can indeed also be demonstrated through the development of new business activities in existing organizations, an involvement in a young SME or a not-for-profit activity. In particular, any young graduate joining a start- up could in a way be considered as entrepreneurial, even if he or she did not create the firm himself or herself.
Among alumni of the EEP, some have launched their own business, developed a not- for-profit association or joined a start-up or a university spin-off.2The entrepreneurial impact of the EEP overall can from that point of view be considered quite positive, when taking into account entrepreneurial activities in their widest sense, beyond the immediate creation or buyout of a new business. There is, however, a significant number of alumni of the EEP that have made ‘traditional’ career choices, joining large corporations or orga- nizations. Those choices correspond in general to their disciplinary affiliation. Over time, we have identified various potential explanations of this apparent misfit, in terms of delayed effects, selection bias, technology intensity or lack of entrepreneurial culture. We discuss these next.
Delayed effects can derive from the fact that the EEP has actually taught students to apprehend an entrepreneurial career with care and caution, highlighting the pitfalls of naïve ‘dot-com type’ projects. Some students with strong entrepreneurial aspirations could therefore decide to first learn about ‘the business’ in a traditional organization before trying to launch their own venture. For those students the entrepreneurial impact of the EEP in terms of future intentions (Kolvereid and Moen, 1997; Noel, 2001) is not yet visible, and can only be observed from a long-term perspective. This ‘cautious patience’ from the students and ‘observation delay’ of the EEP’s impact is consistent with several empirical studies that have shown that a strong functional or sector-based profes- sional experience from an entrepreneur actually improves the subsequent survival prospects and increase the growth potential of his or her venture (Dahlqvist et al., 1999;
Dunkelberg and Cooper, 1982; Hambrick and Mason, 1984; Storey, 1994; Westhead and Birley, 1995). The entrepreneurial impact of an EEP can therefore only be assessed several years after its launch (Block and Stumpf, 1992). Informal studies quoted by Vesper and Gartner (1997) have indicated a strong correlation between the participation in an entre- preneurial course and the likelihood of launching one’s own business in the future. This should also be the case when considering students attending complete EEP rather than only one course.
A selection bias in the students attending the EEP, diverging from the new business cre- ation objective, could be another potential mitigating factor of the entrepreneurial impact of the program. Indeed, anecdotal evidence suggests that some students apply for the program not because they want to become entrepreneurs, but because they perceive that attending this program would boost their perceived value on the job market, either because of its interdisciplinary dimension or because of the positive perception often associated by recruiters with entrepreneurship. These ‘resume-driven’ students would probably never create a new business activity, whatever entrepreneurial training would be offered to them. Dealing with this selection bias would require the EEP managers to be able to better select up front the students with the right entrepreneurial aptitudes.
160 Handbook of research in entrepreneurship education
A survey of the academic research (Brenner et al., 1991; Chen et al., 1998; Chell et al., 1991; Filion, 1997) identifies several factors that can be associated a priori with entrepre- neurial aptitudes: individual characteristics (independence, result-orientation, internal locus of control, flexibility, leadership), motivations (self-realization, search of auton- omy) and external factors (socio-cultural environment, family context, education).
Similarly, typologies of entrepreneurs have been suggested. However, the entrepreneurial process remains multifaceted and contingent, and cannot be reduced to a predefined model that can be used to identify a priori future entrepreneurs. Indeed, some psycho- logical tests or typology that had been developed to ‘spot’ entrepreneurs have been severely criticized (Chell, 1985). If the elements mentioned above can contribute to improve the design and implementation of the selection process (that is, the format and content of the written application and interview), the entrepreneurial aptitudes of a student remain very difficult to assess a priori in a systematic manner. However, even this audience of ‘resume-driven’ students can play a role in the diffusion of an entrepreneur- ial culture within their future organizations or within society at large and therefore con- tribute to the objectives of the EEP. Indeed, the promotion of an entrepreneurial culture should also involve the education of students whose career will have indirect entrepre- neurial features (Saporta and Verstraete, 2000). Such students will be involved with entre- preneurs as managers and consultants, or can contribute to the emergence of a more entrepreneurial environment.
The ‘technology intensity’ of the program could also have mitigated its entrepreneurial impact so far, as technology-oriented start-ups such as university spin-offs were probably overrepresented in the first editions of the program. Indeed, as part of the interdiscipli- nary requirements all project groups had initially to include a student from the engineer- ing faculty. As a consequence most of the master thesis projects had a strong technology orientation. In particular, a large proportion of those projects related to the valorization of intellectual property and technologies developed within the university. This technol- ogy bias excluded de facto a wide range of entrepreneurial opportunities, for example in the retail or service sectors, that could otherwise have been pursued in the context of the EEP. To deal with this ‘technology’ issue, the rules defining the structure of the master thesis groups have since been made more flexible. Students are now allowed to develop their own project even if it involves no or limited technology and therefore offers limited room for an in-depth contribution from an engineering student. Yet all groups must remain interdisciplinary, that is, include students from at least two different schools.
The last element that could explain the limitation of the entrepreneurial impact of the EEP relates to the relatively low level of entrepreneurial culture in the education system as a whole in Europe in general and in Belgium in particular. This issue, however, goes far beyond the scope and reach of the EEP considered here, and should probably be tackled through other ‘mobilization’ EEPs, as defined earlier in this chapter, aimed at introducing a more entrepreneurial culture during the first years of university education or even at an earlier stage.