Manuscript received June 5, 2010 Manuscript revised June 20, 2010
Comparation of CobiT Maturity Model and Structural Equation
Model for Measuring the Alignment between University
Academic Regulations and Information Technology Goals
Haryanto Tanuwijaya
1and Riyanarto Sarno
2,
1
Study Program of Information System, STIKOM SURABAYA College, Surabaya, 60298 Indonesia 2
Faculty of Information Technology, Institut Teknologi Sepuluh Nopember, Surabaya, 60111 Indonesia
Summary
A University needs to align between Academic Regulations and Information Technology goals in order to improve the academic quality. The university academic regulations include
compliance with external laws, regulations and contracts. Information Technology goals comprises ensure critical and confidential information in withheld from those who should not have access to it. In this regard, the measuring the alignment is important. The measurement is complicated, because it involves many aspects which are sometimes intangible. CobiT is usually used to measure the maturity level of information technology processes; whereas Structural Equation Model is employed to measure the alignment between business and information technology goals.This research proposes the use of CobiT Maturity Model and Structural Equation Model to measure the alignment between the University Academic Regulations and Information Technology goals.
In the CobiT Maturity Model, an auditor develops questions for each Maturity Level from the statement in each CobiT Maturity Level. The auditor collects the evidence by interviewing staffs and observing the related documents. The maturity levels of information technology processes are resulted from the maturity calculation. The maturity levels are used to assess the alignment level.
In Structural Equation Model, the auditor develops questionnaires from the statement in each CobiT Maturity Level. The questionnaires are filled by respondents assisted by the surveyors. The data is processed by using Partial Least Square, and the results are the effects of the processes to the information technology goals, and the effect of information technology goals to business goals. Then, these effects are used to calculate the total effect of the alignment.
Both of the models show that CobiT Maturity Levels and the effect scores of Structural Equation Model have the same list of maturity ranks of the information technology processes. The information technology process with the highest rank is Ensure Systems Security, and the lowest rank is Manage the Physical Environment. The alignment levels from Cobit Maturity Model and the effect score of alignment from Structural Equation Model are relatively the same. Furthermore, Structural Equation Model has more advantages; i.e. easier to understand, more objective in judgment, shorter time of audit, lower cost of audit, and more active participants.
Key words:
Alignment measurement, CobiT Maturity Model, Structural Equation Model
.
1.
Introduction
The alignment of business goals and information technology goals becomes an important strategy of a university due to the high competition. Reference [1] stated that the alignment of information technology strategy had a positive impact on the profitability of the organization through superior strategies to achieve competitive advantage.
Reference [2], [3] claimed that Information Technology (IT) applications provided benefits when they were aligned with business goals. Reference [4] claimed that the contribution of information technology systems in business performance is related to the size of the role of IT to improve business performance. IT support for organizational functions will not guarantee the continuous support in the future. Therefore, business goals and IT goals alignment is needed to enhance the strategic role of IT for strategic business planning [5].
regulations.
The assessment of alignment is complicated, because it involves many aspects, which are sometimes intangible. There have been numerous studies that focus on business and IT alignment [8], and several frameworks have been proposed to assess business strategic regarding to the role of IT. However, they have not yielded empirical evidence and have not provided a roadmap to alignment [9]. Most of similar studies showed that there were no such optimal method to achieve alignment [7]. There was also lack of empirical evidence to support the theory that competitive strategy and IT structure are related [10]. Furthermore, it was difficult to identify how alignment was achieved [11], and difficult to optimize the power of IT for their long-term benefit [9]. The failure of business and IT alignment influences the perception that IT is only a cost center rather than as an enabler for the business values [9], [12]. Reference [2] stated that if IT is not implemented in alignment with business then the investment will not be optimal.
Control Objectives for Information and Related Technology (CobiT) provides standards in a domain framework that consists of a set of IT processes representing activities. IT processes can be controlled and structured to achieve the alignment between business goals and IT goals. CobiT classifies 17 business goals related to the IT activities in organizations. The business goals are categorized into four perspectives of balanced scorecard, which are financial perspective, customer perspective, business processes/internal perspective, and learning & growth perspective [13].
Since researches to compare CobiT Maturity Model and Structural Equation Model have never been conducted, therefore this paper proposes to use the CobiT Maturity Model (CMM) to assess the alignment between the business goals and the IT goals. The levels of the maturity process determine the degree of the alignment. The maturity process is usually carried out by an auditor, who determines the score through subjective judgment. The maturity calculation results the maturity levels of IT processes. These maturity levels are used to assess the alignment level between business goals and IT goals. The highest level of the maturity shows the highest contribution of IT goals to the alignment; whereas the lowest level of the maturity shows the lowest contribution of IT goals to the alignment.
In addition to that, this paper proposes to use Structural Equation Model (SEM) to assess the alignment between the business goals and the IT goals. SEM uses questionnaires to compute the scores of the effects. The score is between 0 and 1, where 0 indicates no effect between the latent variables; and 1 indicates high relation
between the latent variables. The results of SEM indicate the effects among the latent variables. It identified the effect of IT processes to the ITG, and the effect of ITG to the business goals. Then these effects are used to calculate the total effect of the alignment.
In this study, we assessed business goals number 12 covering compliance with external laws, regulations and contracts. Business goals number 12 is very important for a university existence due to obligations to fulfill external laws, regulations and contracts from the government through DGHE and MoE. IT Governance Institute survey shows that business goals number 12 is also one of the ten most important business goals [15]-[16].
2. Theoritical Consideration
2.1 Control Objectives for information and Related
Technology (CobiT)
Control Objective for Information and Related Technology (CobiT) is the information technology governance framework, which applies to management, IT services, control department, audit functions, and more importantly the owners of the business process to ensure the accuracy, integrity, and availability of data and information which are important and sensitive. CobiT essentially is developed to help meet the various needs of management by bridging the information gap between business risks, control, and technical problems. CobiT supports IT governance by providing a framework to establish the alignment of IT with the business. In additon to that, the framework also ensures that IT enables the business, maximizes its benefits, IT risks are managed appropriately, and IT resources are used responsibly [13]-[14].
2.2 Business Goals and Information Technology
Goals Linkage
CobitT framework provides alignment mapping between business goals and IT goals in each perspective [13]. Reference [6] explains that this mapping is important as it can be a reference for companies to translate business requirements to IT availability. Figure 1 depicts the relationship business goals number 12 (BG 12) covering
compliance with external laws, regulations and contracts, with IT goals (ITG) number 2, 19, 20, 21, 22, 26, and 27, in the SEM construct [13].
Fig. 1 Business goals and IT goals linkage. The seven IT goals (ITG) are depitcted in Table 1.
Table 1: Information technologi goals
ITG Numbe
r
Description
2 respond to governance requirements in line with board direction
19 ensure critical and confidential information is withheld from those who should not have access to it
20 ensure automated business transactions and information exchanges can be trusted
21 ensure IT services and infrastructure can properly resist and recover from failures due to error, deliberate attach or disaster magnetic moment 22 ensure minimum business impact in the event of
an IT service disruption or change magnetization 26 maintain the integrity of information and
processing infrastructure
27 ensure IT compliance with laws and regulations
2.3 Alignment of IT Goals and IT Process
After defining the IT goals, the next step was describing IT related processes. Every IT goals has one or more IT processes. On the other hand one IT process can meet several IT goals. In this study, the researchers discuss business goals number 12 linkages with IT goals number 19 which comprises ensure critical and confidential information is withheld from those who should not have access to it. As a preliminary study, the IT goals number 19 was chosen because the academic data of university are so critical and confidential that have a high degree of risk; and therefore should be protected from harassment by disinterested participants. This is in accordance with reference [16] statement that the selection process applied to IT enterprise depends on the importance of business processes, which can be based on risk level.
Figure 2 depicts that there are four IT processes aligned with IT goals (ITG) 19, which are Plan and Organize 6 (PO6), Delivery and Support 5 (DS5), Delivery and Support 11 (DS11), and Delivery and Support 12 (DS12). The IT processes can be seen in Table 2.
Fig. 2 IT goals and IT process alignment.
Table 2 : Information technology process
IT Process Description
PO6 communicate management aims and direction DS5 ensure systems security
DS11 manage data
DS12 manage the physical environment
ITG 2
ITG 22
ITG 26
ITG 27 ITG 20
ITG 21 BG 12
ITG 19
DS12
ITG 2
ITG 26 ITG 20
ITG 21 BG 12
ITG 19
ITG 22
ITG 27
PO6
DS5
2.4 Maturity Level
Maturity Level (ML) was performed on each IT process from level 0 (non-existent) to level 5 (optimized). Assessment was done by the auditor to the staff related to the management process. The questions at each maturity level were generated from the statement in each CobiT maturity level. The answers were converted into score with a value between 0.0 to 1.0. Table 3 and Table 4 are the examples for ML 0 and ML 5 of IT progress PO6 respectively. After the auditor filled in each question with the score, the next step was to calculate the maturity level
score of each IT process. The example of the result of calculation for PO6 maturity level is shown on Table 5.
Table 5 : Maturity Level of PO6
Maturity Level
Compliance Score
Contributio
n Level Score
0 1.00 0.0 0.0
1 0.55 0.3 0.2
2 0.50 0.7 0.4
3 0.48 1.0 0.5
4 0.28 1.3 0.4
5 0.72 1.7 1.2
Maturity Level of PO6 2.6
Table 3 : Maturity Level 0 of PO6
Process Name : Communicate Management Aims
and Direction Do you agree?
Value
Process ID : PO6 Maturity Level : 0
Not at all A little To Some Degree
Completely
No Statement Weight 0.00 0.33 0.66 1.00
1 Management has established a positive IT
control environment 1 √ 1.00
2 There is no recognition of the need to establish a set of policies, plans and procedures, and compliance processes.
1 √ 1.00
Total Weight 2 Compliance 1.00
Table 4 : Maturity Level 5 of PO6
Process Name : Communicate Management Aims and Direction
Do you agree?
Value
Process ID : PO6 Maturity Level : 5
Not at all A little To Some Degree
Completely
No Statement Weight 0.00 0.33 0.66 1.00
1 The information control environment is aligned with the strategic management framework and vision.
1 √ 1.00
2 The information control environment is aligned frequently reviewed, updated and continuously improved.
1 √ 1.00
3 Internal experts are assigned to ensure that industry good practices are being adopted with respect to control guidance and communication techniques
1 √ 1.00
4 External experts are assigned to ensure that industry good practices are being adopted with respect to control guidance and communication techniques
1 √ 0.00
5 Monitoring, self-assessment and compliance checking are pervasive within the organization.
1 √ 0.66
6 Technology is used to maintain policy and awareness knowledge bases and to optimize communication, using office automation and computer-based training tools.
1 √ 0.66
The compliance score in Table 5 was obtained from calculation of each question score. The level score was the result of multiplication of compliace score and contribution. The maturity level of PO6 was the result of addition of level score. Table 5 depicts that the maturity level of PO6 score is 2.6. The highest compliance score of each maturity level is 1.0, then the highest Maturity Level of each IT process is 5.0. It means that the maturity level of IT process PO6 is moderate. Maturity level of the four IT process can be seen in Table 6.
Table 6 : Maturity Level of all of IT Process
IT
Process Description
Maturity Level
PO6 communicate management aims
and direction
2.6
DS5 ensure systems security 3.4
DS11 manage data 0.9
DS12 manage the physical environment 0.8
Table 6 depicted that DS5 had the highest maturity level 3.4, which means that DS5 gave the highest contribution to the IT goals number 19 (ITG 19). Whereas the lowest maturity level 0.8 was DS12, which means that DS12 gave the lowest contribution to the ITG 19.
2.5 Structural Equation Mode
Structural Equation Modeling (SEM) is an integrated approach between factor analysis, structural model and path analysis. In SEM, a researcher can perform three activities simultaneously. The first activity is testing the validity and the reliability of the questionnaires. The second activity is testing the model relationships between latent variables, and the last activity is to obtain a useful model for forecasting.
The steps of SEM in this study are in accordance with reference [17], which starts from developing the concept-based models and theories. The next step is to construct a path diagram which can be seen completely in the Figure 3. The exogenous variables X1, X2, X3 and X4 are for PO6, DS5, DS11 and DS12 respectively. The endogenous variables Y1 and Y2 are for ITG 19 and BG 12 respectively. The exogenous variables X1, X2, X3 and X4 are independent variables that affect the endogenous variable Y1. Y1 is an endogenous intervening variable that affects the endogenous dependent variable Y2. The Maturity Level (ML) 0 to ML 5 are indicators of the exogenous variables. The endogenous variable Y1 has four performance indicators and the endogenous variable Y2 has six performance indicators.
Each indicator has several sub ML indicators obtained from the statements in each Maturity Level. Each statement in ML can be divided into one or more questions, and each question represents one sub maturity level indicators. As stated in the above example, the statements in PO6 maturity level 4 are Management accepts responsibility for communicating internal control policies and has delegated responsibility and allocated sufficient resources to maintain the environment in line with significant changes. These statements can be divided into two statements. The first statement is the management accepts responsibility for communicating internal control policies, and the second statement is the management has delegated responsibility and allocated sufficient resources to maintain the environment in line with significant changes. Each statement is a ML indicator that is used as a question in the questionnaires. Total indicators shown in Figure 3 are 148 indicators. It means there are 148 questions in the questionnaires, which are 30 questions for PO6, 49 questions for DS5, 34 questions for DS11, and 35 questions for DS12.
The first step was to input the data into the Excel worksheet table. Then, the validity and reliability testing was carried out to determine the valid questions for representing the dependent variables in the study. The validity testing was done by using analysis of confirmatory factor to each of the latent variable PO6, DS5, DS11, DS12, ITG 19, and BG 12. The reliability testing was then done to measure the reliability and accuracy of the indicators used. The reliability was measured by using goodness-of-fit composite reliability (CR) with the minimum cut off value 0.7. Based on the valid and reliable model, then each path coefficient was interpreted into Structure Equation (1).
2 1 1 . 2 2
1 4 4 . 1 3 3 . 1 2 2 . 1 1 1 . 1 1
ε
β
ε
γ
γ
γ
γ
+
=
+
+
+
+
=
Y
Y
X
X
X
X
Y
(1)
ML 0 (X4.1)
ML 5 (X4.6) X4.1.1
X4.1.2
X4.6.1 X4.6.9 ML 1
(X4.2) X4.2.1
X4.2.4
ML 2 (X4.3) X4.3.1
X4.3.4 ML 3
(X4.4)
X4.4.7 X4.4.1
DS 12 (X4)
ML 4 (X4.5)
X4.5.1 X4.5.9 BG 12 Performance
(Y2)
Y2.1
ITG 19 (Y1)
H5
PO 6 (X1) ML 0
(X1.1) (X1.3) ML 2
ML 4 (X1.5)
ML 5 (X1.6) ML 3
(X1.4) ML 1
(X1.2)
Y2.3
Y2.5 Y1.1 Y1.2 Y1.3 Y1.4
X1.1.1 X1.1.2
X1.2.1 X1.2.3
X1.3.1 X1.3.3
X1.4.1 X1.4.9
H1
Y2.2
Y2.4
Y2.6 DS 5
(X2)
H2
H3
H4
X1.5.1 X1.5.6
X1.6.1 X1.6.6 ML 0
(X2.1) X2.1.1
X2.1.5
ML 1 (X2.2) X2.2.1
X2.2.6 ML 2 (X2.3) X2.3.1
X2.3.8 X2.4.1 X2.4.7
X2.5.1
X2.5.12 ML 5 (X2.6) X2.6.1
X2.6.11
DS 11 (X3) X3.1.1
X3.1.3
X2.2.1
X3.2.5 ML 2 (X3.3) X3.3.1
X3.3.5 X3.4.1
X3.4.7
X3.5.1 X3.5.6
ML 5 (X3.6) X3.6.1
ML 3 (X2.4)
ML 1 (X3.2)
ML 3 (X3.4)
X3.6.8
ML 4 (X2.5)
ML 4 (X3.5) ML 0
(X3.1)
3. Research Hypothesis
In this study, we attempted to compare the alignment assessments between business regulations and the IT goals using CMM, and SEM. The main hypothesis in this study is the alignment assessments using CMM and SEM are close.
Figure 3 depicts the construct of SEM used in this study. The model depicts that IT process PO6, DS5, DS11, and DS12 are linked to the IT goals (ITG 19). Also the model depicts the link between IT goals (ITG 19) and the business goals (BG 12). The first four hypotheses test the effect between PO6, DS5, DS11, DS12 and ITG 19. The fifth hypothesis tests the effect between ITG 19 and BG 12. Therefore, the five hypotheses are:
Hypothesis 1: IT process PO6 has a significant effect to IT goals (ITG19).
Hypothesis 2: IT process DS5 has a significant effect to IT goals (ITG19).
Hypothesis 3: IT process DS11 has a significant effect to IT goals (ITG19).
Hypothesis 4: IT process DS12 has a significant effect to IT goals (ITG19).
Hypothesis 5: IT goals (ITG 19) has a significant effect to business goals (BG 12).
4. Methodology Consideration
The data collection in this study was carried out by doing a survey conducted by the researchers acted as auditors. The questions in the survey were constructed based on the statements in each CobiT Maturity Level. In the CMM, the auditor filled out the questions according to subjective judgments based on the evidences obtained during the interview and observed on the available documents. Meanwhile, the questionnaires in SEM were filled out by the respondents. The auditor only explained the questionnaires that were not clearly understood. The staffs interviewed in CMM and the respondents filled out the SEM questionnaires had to meet the criteria of RACI chart. COBIT defines one RACI chart for each IT process [13], [18]-[19]. RACI chart describes the duties, which are Responsible, Accountable, Consulted, and Informed.
Reference [18] states the mapping to the decision-making processes for data quality. The first is Responsible, which is for the staff role in which staffs were given the authority to implement a policy. It was actually decided in
a data quality decision area; the second is Accountable, which is for other role of staffs which provided direction and authorized decisions, the third is Consulted which is for the role of staffs in which they provided input and support for decisions; and the fourth is Informed, which is for the staff role to be informed about decisions.
4.1 CobiT Maturity Model
Maturity Model essentially collected the evidence through the interviews to the staffs and observation on the documents related with the IT process. The interviewed staffs that met the RACI chart criteria consisted of one head of university academic division (Responsible), one staff of front office (Accountable), two staffs of back office (Accountable), one Vice Rector in academic (Consulted), and one head of Study Program (Informed).
In collecting the evidence needed by the auditor, there were adjustments to the questions available in ML 0 to ML 5 at PO6, DS5, DS11, and DS12. Due to the abundant evidence, the collection processes were done in several sessions.
Among the evidence obtained, there were pitfalls that caused high risk to the system, such as processes that were not applied consistently, had not been made, and did not have manual guidelines and security. In addition to pitfalls, there were also strengths of the system such as manuals documents that were complete, process application matching with the procedures, and process that had anticipated errors or other risks. Based on the obtained evidence and supported with the auditor’s experience in audit, the auditor judged and scored each question in ML 0 to ML 5.
effect of the whole processes to the university academic regulations.
BG
12
ITG
19
PO6
DS5
DS11
DS12
L
1L
3L
4A
L
L
2Fig. 4 The alignment assessment of university academic regulations and IT goals using CMM.
4
4 3 2
1 L L L
L
AL = + + + (2)
AL L TML
n n
∑
=
× = 4
1
(3)
4.2 Structural Equation Model
The targeted population for this study were all university academic staffs who are in the criteria of RACI chart. They were staffs of Academic Administration Division, Programmers, Head of Study Program, and Vice Rector in Academic.
Out of 19 questionnaires distributed, there were 12 questionnaire (63.1%) returned with the detail answers, i.e. the Head of Academic Administration Division (Responsible), one staff of front office (Accountable), two staffs of back office (Accountable), and two operators (Accountable), two programmers in Application Development Division (Accountable), the Vice Rector in academic (Consulted), and three Heads of Study Programs (Informed), while 7 questionnaires (36.9%) from two programmers and five Heads of Study Programs were not returned.
Reference [20] stated that Partial Least Square (PLS) as a soft modeling and a powerful analysis method which can be implemented to all data scale and it does not require many assumptions and a large samples. Considering the sample size in this study which were only 12 respondents (less than 30 respondents), the data processing of SEM
was carried out by using SEM Partial Least Square (PLS) [20]-[21]. The results are the effects of the IT processes to the ITG, and the effect of ITG to the BG. Then these effects are used to calculate the total effect.
In SEM, the effect score ES1, ES2, ES3, and ES4 are for ML PO6, ML DS5, ML DS11, and ML DS12 respectively.
ES5 is the alignment assessment value between ITG 19 and BG 12, and the total effect score (TES) of this model can be calculated using Equation (4). TES is the total effect of the whole processes to the university academic regulations.
TES will be compared with TML in CMM.
5 4
1
ES
ES
TES
n n
∑
=
×
=
(4)5. Planning and Implementation
5.1 CobiT Maturity Model
An auditor prepared a plan which consisted of several stages before conducting the audit. The first stage of the plan was to ensure the audit permission authority from university management and the second stage was to conduct an interview appointment with the staff who was given authority to implement the policy. In this second stage, the interview was conducted in several sessions due to the many questions available. The next stage was to prepare all of the questions based on the CobiT ML 0 to 5 for PO6, DS5, DS11 and DS12. The total questions in this stage were 148 questions. The following stage after preparing all of the questions was to ask the staff to prepare the whole academic documents related the process to be reviewed during the audit process, and the final stage was to conduct interviews and observe documents to find evidence.
5.2 Structural Equation Model
In Structural Equation Model, there were several stages of planning carried out. The first stage was to ensure the audit permission authority from university management. The second stage was to prepare questionnaire based on the CobiT ML 0 to ML 5 for PO6, DS5, DS11 and DS12 which have totally 148 questions, and the last stage was to conduct a meeting appointment with the respondents who fulfilled the questionnaires.
During the process of filling out the questionnaire, a surveyor clarified the questions arose from respondents regarding the items in the questionnaires. The data was processed using PLS. The results were the effects of the IT processes to the IT goals and the effect of IT goals to the business goals. Then, the total effects were calculated using these effects.
6. Evaluation and Discussion
The result of the alignment assessment using CMM had been discussed above and the result can be seen in Table 6. The ML is divided by 5 in order to compare between the ML and the effect score as the maturity level of CMM is between 0.0 to 5.0 and the effect score of SEM is between 0.0 to 1.0. Therefore the values 0.5, 0.5, 0.2, and 0.2 are for L1, L2, L3, and L4 respectively. The alignment assessment between ITG 19 and BG 12 had calculated using Equation (2) is:
4
.
0
4
2
.
0
2
.
0
5
.
0
5
.
0
+
+
+
=
=
AL
The alignment assessment between IT goals and university academic regulation is 0.4. The ITG 19 have a contribution to BG 12 as many as 0.4. Figure 5 depicts the whole ML in this model. The alignment level is 0.4 which approximately low as effected by the low maturity level of IT process DS 11 and DS 12. This result explain that the IT process DS11 and DS12 should be the essential processes to be improved.
The total maturity level (TML) using (3) is:
6
.
0
)
4
.
0
2
.
0
(
)
4
.
0
2
.
0
(
)
4
.
0
5
.
0
(
)
4
.
0
5
.
0
(
×
+
×
+
×
+
×
=
=
TML
BG 12 ITG 19 PO6
DS5
DS11
DS12
0.5
0.7
0.2
0.4 0.2
Fig. 5 The alignment assessment of university academic regulations and IT goals using CMM.
TML is 0.6, means that the total effects of the whole processes to the university academic regulations as many as 0.6. This TML has proven that the ML of each IT process contribute to the alignment of the university academic regulations and IT goals.
The next discussion focuses on the alignment assessment using SEM. In validity testing, an indicator is valid when the value of loading factor is more than 0.5. Table 7 depicts the result of validity testing on latent variable. While in reliability testing, the whole latent variables have composite reliability (CR) with the value of cut off more than 0.7. Table 8 depicts the result of reliability testing on the whole latent variables.
Table 7 : The result of latent variable validaty testing
IT Process Number of Valid Indicators Number of False Indicators PO6 6
DS5 5 ML 0 (X2.1)
DS11 5 ML 5 (X3.6)
DS12 6
ITG 19 4
BG 12 4 0.155 (Y2.5, Y2.6)
Table 8 : The result of latent variable reliability testing
IT Process Goodness of Fit Composite Reliability
PO 6 0.789
DS 5 0.811
DS11 0.828
DS 12 0.734
ITG 19 0.863
BG 12 0.967
structural equation of BG 12 which was processed by using PLS.
BG 12
ITG 19
PO6
DS5
DS11
DS12
O:0.738 | B: 0.735
O:0.175 | B: 0.164
O:0.620 | B: 0.622
O:0.155 | B: 0.123
O:0.282 | B: 0.286
Fig. 6 The Result of testing using PLS.
Figure 6 describes the structural equation at (5). The residues are close to zero as the out layers have been discarded. The detail of the path coefficient testing in Figure 6 and the Structural Equation (2) can be seen in Table 9. 1 2 4 3 2 1 1
622
.
0
123
.
0
164
.
0
735
.
0
286
.
0
Y
Y
X
X
X
X
Y
=
+
+
+
=
(5)Table 9 : The result of business goals coeffisien model testing
Variable Coefficient Statistics T Description PO 6 Æ ITG 19 0.286 6.511 Significant DS 5 Æ ITG 19 0.735 20.346 Significant DS 11 Æ ITG 19 0.164 2.432 Significant
DS 12 Æ ITG 19 0.123 1.407 Not
Significant ITG 19Æ BG 12 0.622 18.359 Significant
Table 9 explains that PO6, DS5, and DS11 had positive and significant effect on ITG 19. This is shown from the value of T statistics which is more than T table of 1.96. This means that PO6, DS5, and DS11 have direct effects on ITG 19. Each increase in PO6 automatically increases ITG 19 as many as 0.286. While each time DS5 increased, it would automatically increase ITG 19 as many as 0.735. Moreover, the increment of DS 11 will increase ITG as many as 0.164. The result also shows that ITG 19 had positive and significant effect on BG 12. This can be seen from the path coefficient with the positive value of 0.622 with the value of T statistics 18.359 which is more than T
table of 1.96. It can be concluded that ITG 19 affects BG 12 directly as many as 0.622. Which means the increase in ITG 19 automatically increases BG 12 as many as 0.622. Table 9 depicts that only IT process DS12 does not have significant effect on ITG 19. This is shown from the value of T statistics, which is less than T table of 1.96.
Figure 6 also describes that IT process PO6 has indirect effect to BG 12 as many as 0.2. It can be concluded that the increment in IT process PO6 indirectly increases BG 12 as many as 0.2. Figure 5 also depicts that DS5 has 0.5 indirect effect to BG 12, and DS11 has 0.1 indirect effect to BG 12. The IT process DS12 does not have indirect effect to BG 12 due to the value of T statistics which is less than T table of 1.96. The total effect score (TES) IT process to BG 12 had concluded using Equation (4) is:
8
.
0
)
6
.
0
1
.
0
(
)
6
.
0
2
.
0
(
)
6
.
0
7
.
0
(
)
6
.
0
3
.
0
(
×
+
×
+
×
+
×
=
=
TES
Since the maturity level is between 0.0 to 5.0 and the effect score is between 0.0 to 1.0, then the SEM effect score is multiplied by 5 in order to compare between the maturity level and the effect score. The result of measurement by using CMM, then was compared with the result of the SEM calculation which could be seen in Table 10. For the clearer description, the result in Table 10 was illustrated in Spider Web Graphic as seen in Figure 7.
Table 10 : The comparation of CobiT Maturity Model and Structural Equation Model assessment
IT Process CobiT Maturity Model Score
Structural Equation Model Score
PO6 2.6 1.4
DS5 3.4 3.7
DS11 0.9 0.9
DS12 0.8 0.8
Table 10 shows that the alignment assessments that used CMM and SEM result in the same ranks. Both of the models describe the consistent results in which the highest contribution is given by DS5, whereas the lowest contribution is given by DS12. The spider web graphic on Figure 7 shows that the red shading area is the result of the alignment assessment using CMM. The blue shading area is the result of the alignment assessment using SEM. Figure 7 shows that the CMM and SEM have the same forms.
AI6 PO6
1 2 3 4
AI7
DS4
DS5
DS11 DS12
AI6 PO6
1 2 3 4
AI7
DS4
DS5
DS11 DS12
Fig. 7 Spider Web Graphic of alignment assessment using CMM and SEM.
As shown in Table 11, the IT audit using CMM was done by an auditor. However, in several cases, the IT audit using CMM was carried out by more than one auditor. It is compulsory for the auditor to interview and collect evidence from six staffs that were specified by RACI chart. In this study, the time needed for collecting the data and evidence from six staffs was fifty four hours. It means that the average time to collect the data and evidence from each staff was nine hours; whereas in SEM, the time needed to collect the data and evidence from twelve staffs was thirty six hours which was less than that of CMM. It proves that collecting the data and evidence using SEM was more efficient.
.
In CMM the number of participants involved in the interview were six staffs, who were specified by RACI chart; whereas in SEM the participants involved in the process of completing the questionnaires were twelve staffs. In this process the respondents involved were staffs from any levels. This shows that the involvement of participants in SEM was more active than that in CMM, and thus the judgment obtained was more objective. Furthermore, the level of understanding of research instruments in CMM was difficult because the questions could only be comprehended by the auditor, while in SEM the participants could complete the questionnaires by themselves. The participants were only assisted by the surveyor. It proves that the research instruments used in SEM were easier to understand.
Due to the difficulty of the research instruments in CMM, the competence of the assessor needed was relatively high. Therefore, the assessor required for doing the task is an auditor; while in SEM the competence of the assessor needed was less than that of CMM. Therefore, it only required surveyors.
The high competence of the auditors affected the standard of the remuneration; therefore, it increased the cost of IT audit process as well. In addition to the competence, the auditor took more time to obtain more data and evidence to get the more accurate results. This also contributed to the high cost of IT audit processes.
7. Conclusion
In this study we proposed the alignment measurement between university academic regulations and the IT goals by computing the maturity level of IT processes. The computation of the maturity level using CobiT procedure Table 11: The data of carrying out the survey for CobiT Maturity Model and Structural Equation Model
Item CobiT Maturity Model Structural Equation Model
The time needed to collect data and evidence
Fifty four hours used to interview six staffs that meet the criteria of RACI chart.
Thirty six hours used to distribute the questionnaires, clarify the questions arising from respondents, and collect the questionnaires from twelve staffs that meet the criteria of RACI chart
The number of participants Six staffs Twelve out of nineteen staffs
Questionnaires assessor Carried out by an auditor by interviewing six staffs
Carried out by twelve staffs that acted as respondents. The respondents were assisted by surveyor in filling out the questionnaires
Competence required High-competence auditor Surveyors with less competence than the auditor Level of understanding of research
instruments
was called CMM, whereas that using Structural Equation Model was called SEM.
The results of this study proved that the alignment measurement using CMM and SEM gave relatively the same results, which described the same priority list of maturity levels of the IT processes. However, it indicated that an auditor could use SEM to collect the evidence through questionnaires distributed to the respondents. The collection of evidence using SEM was effective and efficient than those using CMM. The results of the alignment measurement using SEM were more objective than those using CMM since the respondents specified by RACI chart were more active in participating the fulfillment of the questionnaires. Moreover, SEM has more advantages in terms of easier to understand, shorter time of audit, and lower cost of audit. Therefore, this conclusion should be able to encourage the practitioners to determine the best model to audit IT in their organizations, and encourage other researchers to explore further findings related to IT audit using SEM.
Acknowledgments
We would like to thank the university and the academic division, where we conducted the research, for the opportunity given to us. We also acknowledge the reviewers for the suggestion to improve the quality of this paper.
References
[1] B.H. Reich, and I. Benbasat, ”Factors that influence the social dimension of alignment between business and information technology objectives,” Management Information Systems Quarterly, vol. 24, no. 1, pp. 81-113, 2000.
[2] R. Hirschheim, and R. Sabherwal, “Detours in the path towards strategic information systems alignment,”
California Management Review, vol.44, no. 1, pp. 87-108, 2001.
[3] G. Premkumar, and W.R. King, “An empirical assessment of information systems planning and the role of information systems in organization,” Journal of Management Information Systems, vol. 9, no. 2, pp. 99-125, 1992.
[4] G.S. Kearns, and R. Sabherwal, “Strategic alignment between business and information technology: a knowledge-based view of behaviors, outcome, and consequences,” Journal of Management Information Systems, vol. 23, no. 3, pp. 129-162, Winter 2006.
[5] A.M. Johnson, and A.L. Lederer, “The effect of communication frequency and channel richness on the convergence between chief executive and chief information officers,” Journal of Management Information Systems, vol. 22, no. 2, pp. 227-252, Fall 2005.
[6] R. Sarno, Audit Sistem Informasi/Teknologi Informasi, Surabaya: ITS Press, July 2009.
[7] Albrecht, R., et al., Research Study Roadmap published by the EDUCAUSE Center for Applied Research,information technology alignment in higher education, 2004, Available:
http://www.educause.edu/ir/library/pdf
[8] Y.E. Chan, S.L. Huff, D.W. Barclay, and D.G. Copeland, “Business orientation, information systems strategic orientation, and strategic alignment,” Information Systems Research, vol. 8, no. 2, pp. 125-150, June 1997.
[9] J. Luftman and T. Brier, “Achieving and sustaining business-IT alignment,” California Management Review,
vol.42, no. 1, pp. 109-122, 1999.
[10]H. Tavakolian, “Linking the information technology structure with organizational competitive strategy: a survey,” Management Information Systems Quarterly, vol. 13, no. 3, pp. 309-317, September 1989.
[11]P. P. Tallon, “A process-oriented perspective on alignment of information technology and business strategy,” Journal of Management Information Systems, vol. 24, no. 3, pp. 227-268, Winter 2007-8.
[12]N. Shin, “The impact of information technology on financial performance: the importance of strategic choice,”
European Journal of Information System, vol. 10, pp. 227-236, 2001.
[13]The IT Governance Institute, COBIT 4.1, ISACA, 2007. [14]R. Sarno, and A. Herdiyanti, “A service portfolio for an
enterprise resource planning,” International Journal of Computer Science and Network Security, vol.10, no..3, pp. 144-156, March, 2001.
[15]Understanding How Business Goals Drive IT Goals, The IT Governance Institute, 2008.
[16]R. Sarno, Strategi Sukses Bisnis dengan Teknologi Informasi, Berbasis Balanced Scorecard & CobiT. Surabaya: ITS Press, 2009.
[17]J.E. Hair, R.E. Anderson., R.L. Thatham, and W.C. Black,
Multivariate Data Analysis, 5th ed., New York: Prentice Hall International Inc, 1998.
[18]K. Wende, “Data governance – defining accountabilities for data quality management,” in Isari, D., D'Atri, A., & Winter, R., Ed., Switzerland: Luiss University Press., 2008.
[19]R. Sarno, and A. Herdiyanti, “Information technology policies for enterprise resource planning practices to improve customer orientation and service,” International Journal of Computer Science and Network Security, vol.10, no..5, pp. 82-94, March, 2001.
[20]H. Wold, “Partial Least Square,” in Encyclopedia Statistical Sciences, vol. 8, S Kotz & N.L. Johnson, Ed., New York: Wiley, 1985, pp. 587-599.
Haryanto Tanuwijaya was born in Samarinda in March 1966. The author’s educational background is S.Kom degree in Information Systems from STIKOM SURABAYA College, Indonesia, 1993 and M.MT degree in Technology Management from Sepuluh Nopember Institute of Technology, Indonesia, 2002. He is now a Doctor candidate in Economic Faculty, Airlangga University, Surabaya. He was the Director of STIKOM SURBAYA College. He is currently a senior lecturer of STIKOM SURABAYA College. He wrote a book entitiled
Information System: Concept, System, and Management
(Yogyakarta, Indonesia : Graha Ilmu, 2005). His research interests include Strategic Planning for Information System, IT Audit, Human Resoruces Information System, Expert System., and Decision Support System.
Riyanarto Sarno was born in Surabaya in August 1959. The author’s educational background is Ir. degree in Electrical Engineering from Bandung Institute of Technology, Indonesia 1983 and Drs. degree in Economics from Padjajaran University 1985. This author earned MSc. and PhD in Computer Science from University of Brunswick Canada in 1988 and 1992, respectively. He was the Chairman of Institute of Technology Development for Community. He is currently the Head of Software Engineering Laboratory and the Dean of Information Technology Faculty, Sepuluh Nopember Institute of Technology. He wrote three books entitled Business Success Strategy utilizing Information Technology (Surabaya, Indonesia : ITS Press, 2009),
