1. Review of the topic
4.8 Analysis approach
As discussed, this study is quantitative in nature and was combined with and produced descriptive data (categorical and numerical) that can be used to identify and explain certain variables in relation to each other and to the constructs that were investigated, being project success and project governance. The data that was collected from the online survey and automatically collated by the Typeform platform, was downloaded by the researcher into Microsoft Excel. The downloaded data was in a raw and unformatted form which made it possible for the researcher to follow a structured approach to evaluating and scrutinising the data (Wegner, 2016).
The downloaded data was first evaluated for validity in terms of incomplete responses. All test runs of the survey were highlighted, and 21 respondents were identified as missing one or two answers from their submitted survey. Missing item responses were not estimated statistically but were rather excluded from the study of that particular variable (Craig & McCann, 1978).
45 The data was also analysed per respondent to see if the same response was given to every question, and fortunately, no surveys had to be deleted due to this reason.
Some changes to the respondent’s answers were necessary. The free text field in which the respondent typed in the city in which the ERP project was performed was manually changed per user to ensure consistent spelling for reporting purposes. The question regarding project governance which were based on a 7-point Likert scale had to be converted from text to a numerical value based on the rescaled score shown in Table 4 in section 4.6.2 of this chapter.
Once the data was scrubbed and coded correctly, basic statistical analysis and more complex statistical modelling was conducted within Microsoft Excel using the statistical tools available within the software. The statistical techniques utilised, and steps taken to analyse the data were used in similar studies done to analyse relationships around either organisational or project success, ERP projects, and organisational or project governance, as conducted by Joslin and Müller (2016), Müller and Lecoeuvre (2014), and Wang and Chen (2006).
The first step in the statistical analysis was the compilation of a number of different demographics of the respondents to the survey. The next required parts are summarised into 6 main themes below.
4.8.1 Construct validity
A very important part of the statistical analysis was to test the questions and the three constructs (project success, stakeholder governance, and behaviour governance) for validity.
For the proposes of this study, it was necessary to determine if significant relationships amongst the measured variables exist which is determined via the execution of a variety of exploratory factor analysis techniques.
In order to determine if factor analysis is the correct data reduction tool to use for this research, the KMO index and the Bartlett’s test of sphericity were used. The KMO measures the adequacy of the sample whilst the Bartlett’s test od sphericity is a comparison between the data’s correlation matrix and the identity matrix to ascertain if any variables are not needed. A KMO value in excess of 0.5 and a significance level where p is greater than 0.05 are indicative that a factor analysis is appropriate (Field, 2014).
The exploratory factor analysis done in this research also helped to confirm if that data received from the respondents for project success construct can be used to gage the associated construct as a whole. Factor analysis was not suitable for testing the validity of the
46 project governance constructs due to the scale used to measure the answers of the respondents. The scale used required the respondent to choose between which of two statements they related two more in terms of how their organisation and projects are governed.
Therefore, within one question there were 2 possible constructs, either the shareholder and stakeholder approaches to governance, or the behaviour control or outcomes control approaches to governance. A single question that has 2 or more possible factors within it cannot be run through any exploratory factor analysis tests. Validity for the project governance questions and construct was tested via the Pearson’s correlation coefficient, as well as assumed through the previous and successful use and validity testing of the questions in the published works of Müller and Lecoeuvre (2014), Joslin and Müller (2015), and Joslin and Müller (2016).
In conducting the exploratory factor analysis tests on the project success construct, it is automatically assumed that any of the observed variables or responses could potentially be associated with any factor. Therefore, this analysis also provides a verification that each question does in fact relate to the desired construct as assumed previously in the research (Field, 2014). Through this verification, the researcher is able to reduce the respondent variable to a smaller set of summary variables and values which will deliver a stronger model of the underlying construct, and thus confirm the validity of the construct.
4.8.2 Measuring the reliability of the instrument
The reliability of the responses to the project success questions in the survey will be determined by running a series of Cronbach’s Alpha tests. The Cronbach’s Alpha is used to measure internal consistency. Hair, Black, Babin, and Anderson (2014) explain that reliability of all the constructs can be assumed if the Cronbach’s alpha calculation has a value greater than 0.7. In measuring the internal consistency using Cronbach’s alpha, the scale to be used has varied from study to study. Therefore, for this research, and based on other recent and related studies, the instruction of Tavakoi and Dennick (2011) will be adopted whereby an alpha value of between 0.7 and 0.9 will indicate internal consistency and thus, reliability.
Reliability of the project governance construct could not be determined successfully due to the reasons pointed out in section 4.8.1, however reliability was assumed as the project governance questions and construct were proven reliable through the previous and successful use and reliability / internal consistency testing of the questions in the published works of Müller and Lecoeuvre (2014), Joslin and Müller (2015), and Joslin and Müller (2016).
47
4.8.3 Descriptive statistics
Once the reliability of the survey and the validity of the constructs was established, a descriptive statistical analysis was conducted for each variable within the respondent’s data in order to get an understanding for how respondents were generally answering the questions.
The mean scores were calculated for each question to discover the central tendency of answers (Wegner, 2016). The minimum and maximum scores given per question were calculated to indicate the highest and lowest that were given per question. And the standard deviation calculation would show how dispersed the answers were in general from the mean.
Finally, scores were calculated for each construct by averaging out all the scores of the items within each construct, so that the mean, minimum, maximum, and standard deviation could be assessed for each of the 3 research constructs.
4.8.4 Analysis of variance across demographics
For certain cases, the mean scores per variable and per construct was compared across different demographics groups to ascertain if significantly different answers were received from respondents depending on their demographics. Two-sample pooled variance t-tests were performed on the answers given by male and female respondents as these are two independent and categorical variables (Wegner, 2016). For the t-test, a p-value of less than 0.05 at a 95% confidence interval will indicate that women and men responded significantly differently to each other.
To do the same type of testing for the variables in the study but where there were more than two categorical variables that the respondents were able to select, analysis of variance (ANOVA) tests had to be and were conducted (Wegner, 2016). Therefore, a one factor ANOVA analysis was conducted in order to find out if certain factors, for example, industry, work experience, age, and value of project, have an influence on the responses given in the survey. For this test, a p-value of less than 0,05 at a 95% confidence interval will indicate that a certain factor definitely has a significant influence on how the respondents answered that particular questions that were part of a specific construct.
4.8.5 Correlation analysis
Given the relevant scores received from the respondents to the survey, correlation analysis was conducted in order to ascertain the level of the relationship that may exist between certain variables and constructs (Wegner, 2016). To measure this potential correlation, specific
48 Pearson’s correlation coefficient tests were performed. This is the most relevant measure for this research as the values found for the constructs (as outlined in section 4.8.3 and done via a descriptive statics analysis) were investigated using an interval measurement scale, and that in analysing correlations or associations in data that was obtained using Likert scales, the Pearson’s test of correlation is the most idyllic (Boone & Boone, 2012).
The value extracted from a Pearson’s correlation coefficient test can only be between the values of negative 1 and positive 1. A value close to negative 1 will show that a strong negative relationship exists between two variables, a value close to positive 1 will show that a strong positive relationship exists between two variables, and a value closer to 0 will show a weak relationship exists between the variables (Wegner, 2016). The Pearson’s bivariate correlations were utilised to potentially show that all the survey questions that make up a specific construct have a significant correlation, and that certain constructs are associated with each other, which is evidenced by a confidence level of over 95% (i.e. a p-value of less than 0.05).
4.8.6 Hypotheses testing
Finally, the two key hypotheses of this study, as framed in chapter 2 and 3, set out to determine if a positively correlated relationship exists between a stakeholder-oriented approach to project governance and ERP project success, and if a positively correlated relationship exists between a behaviour controlling approach to project governance and ERP project success.
To prove or disprove these hypotheses, the researcher performed statistical modelling in the form of linear regression to test the correlation amongst the relevant variables. Linear regression modelling is able to measure and provide a value in quantifying the relationship between the constructs, as well as measuring the strength of how well the linear regression model is able to predict the relevant relationship. This allowed the researcher to comprehend the predictive value, at specific significance levels, that a stakeholder-oriented approach to project governance has on ERP project success, and the predictive value that a behaviour controlling approach to project governance has on ERP project success (Wegner, 2016).
If it was found by the researcher during the statistical analysis phase of this study that either of the 2 research hypotheses was proven correct and significant, then the method of project governance described by the hypothesis (ether stakeholder orientation or behaviour control), then an exploratory analysis would be conducted to assess the relationship between these governance variables and each of the 5 sub dimensions making of project success (i.e.
efficient project execution, organisational advances, project impact, future potential, and satisfying stakeholder needs). Through linear regression analysis, these project success sub-
49 dimension variable would be assigned as the dependent variables, with the method of project governance as the independent variable.