A framework for implementing a scalable business intelligence system 74 As can be observed, the average refresh time spans from 0.4 minutes to 24 minutes.
Furthermore, some refreshes may take longer than usual owing to external circumstances such as the PowerBI service being unavailable, the gateway machine crashing, or the network just being slow, among others.
As a result, those long atypical refresh times may influence the average refresh time.
According to the data, this appears to be the case frequently, especially when the CV value is greater than 100%. This suggests that the real refresh times may deviate significantly from the mean. The data also shows that there are many more successful refreshes than failures i.e., 121 total refresh failures and 1 545 completed refreshes reported. This results in a total of 1 666 refreshes being recorded.
Figure 3-18 Report refresh outcome
The ratio of failed to completed report refreshes is depicted in Figure 3-18. Only 7.26% of
A framework for implementing a scalable business intelligence system 75 for the percentage of report failures and the report data size is 0.01 which is very low, indicating that there is a little to no relationship between the report fail and the data size of the report.
Figure 3-19 Impact of report size on report refresh
In terms of the average refresh time of the report, the data suggests that many small reports take less time to refresh, and the larger the report, the longer it takes to refresh. Reports 0 and 1 in Table 3-3 are two examples. Report 0 is 105 times smaller than Report 1, hence it takes 20 minutes longer to refresh than Report 0.
This is not always the case, as evidenced by Report 26. Report 26 is 36 times larger than Report 0, but it refreshes in nearly half the time. Although Report 26 has a far higher number of failures and a CV value greater than 100%, this may indicate that data size is not the sole factor influencing report refresh time.
Figure 3-20 below depicts a regression analysis of the report's data size and average refresh time. The chart clearly shows that there is a relationship between the data size and the report refresh time. The correlation value between the report's data size and the refresh time is 0.596, which is greater than 0.4, showing that the report's data size impacts how long it takes to refresh to a certain degree.
A framework for implementing a scalable business intelligence system 76
Figure 3-20 Impact of the report size on the refresh time
Other variables that may influence the report refresh time include the type of operations performed in the report. This can range from using JOINS across tables to complicated metrics or importing specialised functionality such as Visio components. As a result, even if a report does not require a large amount of data, the actions that it performs may contribute to the report's refresh time.
Users of the BI system were also asked to rate their satisfaction with the system's quality, specifically the ease of use and timeliness. This data is a good predictor of whether the system's users consider the timeliness to be sufficient. Table 3-4 summarises the survey responses regarding ease of use and timeliness.
Table 3-4 Responses for user satisfaction with system quality
No. Question Response
Q7 Is the BI system user friendly? 5 41.67%
4 33.33%
3 16.67%
2 8.33%
1 0%
Q8 Is the BI system easy to use? 5 25%
4 41.67%
3 33.33%
2 0%
1 0%
Q9 Does the BI system get the information you need in time? 5 33.33%
4 41.67%
3 0%
2 25%
1 0%
A framework for implementing a scalable business intelligence system 77 Q10 Does the BI system provide up-to-date information? 5 25%
4 41.67%
3 25%
2 0%
1 8.33%
A score of 4 or 5 suggests that respondents are usually pleased with the BI system. A 3 implies that the BI system is good but still requires improvement, and anything less than 3 indicates that the user is completely dissatisfied. 75% of respondents believe the BI system is user friendly, however only 66.67% believe it is genuinely easy to use.
These figures indicate that most users are satisfied with the BI system. Only a little more than a third of those polled felt that the ease of use is unsatisfactory. This is an excellent case for implementing user-centred support and maintenance on the BI system, which may increase user satisfaction with the system's quality even further.
Furthermore, Table 3-4 reveals that 75% of respondents believe that the BI system gives information just in time, whereas the other 25% believe that this is not the case. Considering the report refresh times from Table 3-3, external factors such as internal report operations can have an impact on report speed.
Thus, training users on the usage of efficient report operations can also help users recognise that because they build and personalise their own reports, they have an impact on the speed at which the report is generated, as well as the system's quality.
Again, only 66.67% of respondents said that the reports are always up to date, while 33.33%
said they are not. This could be because certain reports have a high frequency of report refresh failures. Upgrading system resources such as memory and processing power may help improve report refreshes, ensuring that reports always have the most recent and up-to-date information.
Figure 3-21 below demonstrates that the average rating for the system's ease of use and speed is greater than 3.5, indicating that most respondents were pleased with the BI system's quality, particularly its timeliness and ease of use.
A framework for implementing a scalable business intelligence system 78
Figure 3-21 Average user rating for system quality
Furthermore, the standard deviation for the various evaluations is low, indicating that the average rating is reliable. Thus, based on the findings, the implemented BI system clearly addresses the system quality success factor, particularly in terms of reliability, timeliness, and usability.
The use of the BI system is the next factor to be validated. As one of the most popular measures used to assess the success of most information systems, the goal of this validation is to guarantee that the BI system is being used.
This is done to validate the implemented BI system to the research objective stated in Section 1.3.2, which specifies that the implemented BI system must:
1. Be used by a diverse range of users.
This will firstly be performed by gathering the number of reports generated over the months by various project teams using the implemented BI system. Furthermore, the amount of data used by each report will be collected to analyse the system's increased data demand.
DATA COLLECTION AND ANALYSIS
The BI system was used to generate many reports. These reports were prepared by various users for a range of purposes. Some reports were built to track data on electricity, cooling, compressed air, and water usage. Other project teams developed reports for budget and mining commodity monitoring, as well as internal data monitoring.
To determine the number of BI reports created, the PowerBI API was used to query the total number of imports (report creations) that were made for each workspace, where each project
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5
Q7 Q8 Q9 Q10
Rating