CHAPTER 8 SURVEY DESIGN AND FRAMEWORK EVALUATION
8.4 SURVEY CONSTRUCTION
In order to evaluate the framework, a self-administered or individually-administered survey was utilised, ergo allowing participants to complete it themselves at a convenient time for them (Sinkowitz-Cochran, 2013). The survey was composed of scaled, closed-ended statements (questions), thus requiring participants to rate their level of agreement to the provided statements (Sinkowitz-Cochran, 2013, De Vos et al., 2011b).Furthermore, a comment section was included in the survey, giving participants a platform to provide any additional commentary they deem fit.
8.4.1 Survey design requirements
Based on the literature covered in chapter 2 (section 2.5.7), a checklist was set out to guide the design of the survey. All the design requirements were adhered to while designing the survey, as is evident by the checklist in table 8.
Table 8: Survey design requirements checklist
Survey design requirement Achieved
1. Format of the survey should be uncluttered and well spread 2. Instructions should be included
3. A thank you message should be included at the end of the survey 4. One should make use of a pre-existing scale for responses 5. Statements should not be vague or ambiguous
6. Statements should not be leading or persuasive
7. Statements should not contain unexplained jargon or acronyms 8. Statements should be as few as possible
9. Statements should as short as possible 10. Statements should not double-barrelled
11. Grouping should be utilised for compartmentalising statements
12. Surveys should maintain anonymity and confidentiality of the participants
13. Statements should be composed in simple and basic English, & pitched at the right level 14. Each statement should be relevant to the validation and verification process
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
8.4.2 Scale of choice
According to multiple sources, a Likert scale is best suited for ranking agreement levels of survey participants (Bryman and Bell, 2016, De Vos et al., 2011b, Fanning, 2005, Kelley et al., 2003, OECD, 2012, Sinkowitz-Cochran, 2013). For the purpose of this survey, a 5 point Likert scale was utilised, as captured in table 9.
Table 9: Likert scale for survey
(1) (2) (3) (4) (5)
Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree
8.4.3 Defining consensus
Although this study did not follow the Delphi method, the Delphi consensus definition was utilised.
Ergo, consensus amongst participants was reached if the minimum average percentage for that statement was greater than 75%. Practically, this was a measure of 3.75 on the 5 point Likert scale (Nordin et al., 2012b). When consensus was achieved, it was accepted that survey statement was verified or validated.
In a case where consensus was not reached for a survey statement, the qualitative feedback (comments) were analysed in order to understand why consensus was not achieved and to address any shortcomings that were mentioned. The shortcomings were discussed and addressed, resulting in the updated framework (presented in section 8.7).
8.4.4 Survey statements design
The survey was designed with a specific goal of evaluating the Lean-Ubuntu implemention framework. In order to achieve this goal, table 10 was created to verify and/or validate the following areas:
• Verify design requirements (DR) of the framework
• Validate problem statement (PS)
• Prove the artefact is valid (address the problem)
• Verify applicability of the artefact
• Verify novelty of the artefact
Table 10 explores each of the aforementioned objectives’ elements and which survey statements were developed in that area (Mangaroo-Pillay, 2020, De Vos et al., 2011b).
Table 10: Development of survey statements
Objective Elements Survey statements
Verify design requirements (DR) of the framework
DR 1 – Simple in structure DR 2 – Visually intuitive DR 3 – Legible
DR 4 – Outline of a philosophy
DR 5 – Change process to be adopted DR 6 –Different parties have tasks to address
DR 7 – Must have stage, phases or levels
DR 8 – Aspects of Lean implementations DR 9 – Aspects of Lean DR 10 – Aspects of Ubuntu DR 11 – Aspects of change model DR 12 – Must not include any barriers to Lean implementation
1 - The framework is simple in design 2 – The framework is visually intuitive 3 – The framework is legible
4 – The framework outlines a philosophy
5 – The framework adopts change processes
6 – The framework has different tasks for different parties (people) 7 – The framework has stages and phases
8 – The framework contains aspects of Lean implementation
9 – The framework contains original aspects of Lean
10 – The framework contains aspects of Ubuntu
11 – The framework incorporates a change model
12 – The framework does not include any barriers to Lean implementation Validate problem
statement (PS)
PS - The research problem to be addressed, is the low Lean
implementation success rate in South Africa due to a lack of culture-specific Lean implementation frameworks
13 – There is a low implementation success rate for Lean in South Africa 14 – Low implementation success rates may be due to a lack
consideration of culture-specific lean implementation
Prove the artefact is valid (address the problem)
PS - The research problem to be addressed, is the low Lean
implementation success rate in South Africa due to a lack of culture-specific Lean implementation frameworks
15 - The Lean-Ubuntu framework considers South African cultural aspects into Lean implementation 16 – The Lean-Ubuntu framework could increase the success rate of Lean implementation in South Africa
Verify applicability of the artefact
To check if this can be implemented in different organisations and industries
17 – This framework could be
implemented in various organisations and industries
Verify novelty of the artefact
To test the originality and novelty of the proposed framework
18 - To the best of your knowledge would you say, the construction and integration of the Lean-Ubuntu framework is an original design (as opposed to just a duplication of other previous work)
8.4.5 Format (Layout) of the survey
The final survey format (available in appendix A) used a structure akin to that of Coetzee (2018) and was composed of the following sections:
• Introduction and informed consent
• Participant data
• The outline of the Lean-Ubuntu Implementation framework video (link)
• Confirmation of the design requirements
• Confirmation that the Lean-Ubuntu implementation framework addresses the research problem
• Closure