4. Introduction
5.4 Discussion of Findings
5.4.4 Explanations to Software Quality Assurance (SQA) Challenges in Scrum
As outlined in Table 9, there are various challenges facing the implementation of Scrum methodologies which play a role in improving the quality of Scrum projects.
Table 9: Challenges to the Success of Scrum Applications Question/s Total No of
participants
Responses
Adaptation Insufficient Time Lack of Resources Developers’ Skill Level Communication Structures Improper Testing Task Tracking No Answer
What are the challenges facing Scrum SQA?
14 (100%)
5 (35.7)
4 (28.5%)
3 (21%)
3 (21%)
3 (21%)
2 (21%)
1 (21%)
No of respondents per research item
93 The most cited common challenges to the success of Scrum projects are: difficulties in the way the development team operates, insufficient sprint duration (time), insufficient developers, and not enough skills for developers to deliver on time, communication structures, insufficient testing and inadequate bug tracking.
5.4.4.1 Adaptability (Change Management)
On the aspect of adaptability/change management, one Scrum master highlighted that their team is experiencing issues relating to difficulties in changing the way people work, especially those who have been implementing traditional methodologies, such as waterfall methods, saying “…change from the people and change management has been hard” (GT – R6). On the same point another Scrum master said “… it’s difficult to change the way you do things” (JP – R5). This seems to be the case in many organizations, what makes it more difficult is the lack of openness in the organisation, lack of transparency is also found to be a problem in the successful implementation of these methodologies (ibid).
Further, from the organisational perspective, Scrum teams are also faced with challenges relating to management support, especially during the release management. In the current context, release management refers to a process that “…guides IT efforts from application development through testing and into production, [focusing on] resources on timely delivery of a feature or set of features that the business needs”. This business disruption delays the process resulting in teams not being able to deliver a quality product on time, as release management reduces testing time. On this point one team said “…we have got a major problem when we want to release, [management] want to take a very conservative release approach which takes about four to six weeks roll out and update the product now because of that it means we can only do about four releases a year” (A – R7.2). This becomes a concern to the development teams especially when it comes to quality assurance because delivery time is also one of the measures of a quality product. It makes it difficult for a team to roll out the system for pilot testing - releasing the system to users for about a week before the system goes into production (ibid). Although Scrum methods seem to be flexible, findings suggest that Scrum teams are still facing a challenge caused by leaning towards traditional methodologies, which means often times that the Scrum process is mixed with waterfall methods. In fact, some teams find that they are unable to work with the Scrum practice of a sprint duration of over five weeks, because teams spend over four weeks developing a product, and thus they are resulting in teams not achieving their goals.
94
5.4.4.2 Insufficient Development Time
In addition to traditionalists struggling to adapt to new and better ways of doing things (change), due to insufficient time to better understand and follow the Scrum process, insufficient development time is central to the quality assurance aspect. In essence, insufficient time was cited as the second most common challenge to the implementation of Scrum methodology, and this affects product quality. Time, in particular sprint duration, which in most teams is two weeks, affects the way a Scrum team delivers a quality product, in that it is not always easy to finish agreed requirements on time. On this point, one Scrum master confirmed that “the challenge we have is the time…, we don’t always finish on time [therefore] …we measure our success rate by having something shippable and fully functional” (NP – R4). Although it is in the Agile Manifesto to deliver “…working software over comprehensive documentation”, it must also be remembered that one of the measures of a quality product, time (meeting the delivery time) is also an important aspect.
Apart from teams not having enough time to deliver all the sprint items, the nature of Scrum itself is sometimes a challenge, especially in a corporate environment where one product owner is responsible for more than project. One of the Scrum practices is that teams should attend daily Scrum meeting to maintain the development progress, remove impediments and to ensure that the process is properly followed. However, due to time and the amount of work that Scrum masters deal with, it becomes difficult for one product owner or Scrum master to attend all daily meetings. On this point, one Scrum master highlighted the impact this problem has on the success of a project. Because of the workload in one team, the product owner “…is never in the office he is in meetings the whole time now we need him in the mornings” (GT – R6). Due to this problem, the product backlog could not be managed properly, hence projects failed (ibid). In this instance team members believe that time plays a huge role in developing a quality product, but it is important to “… do just enough things at the right time..., just enough analysis, just enough documentation, …just enough unit testing” these things can make a huge difference.
From the quality assurance perspective, findings show that Scrum teams are still faced with challenges in terms of ensuring quality. The participants made it clear that it is still difficult to follow the Scrum practice and deliver a product on time, due to various reasons such as accountability (which is currently lacking), the chaotic nature of the Scrum process and insufficient resources.
95
5.4.4.3 Lack of Resources (Developers)
Although time is cited as the second most common challenge to the success of Scrum projects, there are various factors such as insufficient resources and developers’ skills which could affect the delivery of a product. In fact, one Scrum master said, “…we also got resources’ challenges, for example people being on leave” (NP – R4), the shortage of developers working in a project seems to be another reason why this participant sometimes fails to deliver requirements on time. Despite the shortage of developers and their skill level, even those that are available have little or no experience in Scrum methodology hence it is not easy to finish all the requirements on time. One team member feels that specialisation (a person who has worked with Scrum in previous projects) also plays a huge role in speeding up the process. This team member said
“…I can say and I think the teams that have worked in a more agile manner are generally teams where we end up having more than one person who knows how something works you’ve shared the effort” (GT – R6.1). On this point, one Scrum master said “…people are not used to it” (GT – R6), because of this reason, “…team members didn’t want to do the Scrum board because they don’t understand it” (ibid). From the quality assurance perspective, findings suggest that apart from insufficient resources, in some teams, Scrum teams are still engaged in the learning process, when it comes to the understanding, application and a proper implementation of Scrum methods. This makes it difficult to be accurate in measuring the success rate of the projects developed under this methodology. Therefore, a question of what quality assurance processes to follow and how to maximize quality remains unclear, since team members are in the process of adapting to Scrum methodology.
The nature of a Scrum process requires transparent communication channels between project stakeholders and a good management support system therefore, complicated communication structures play a negative role in delivering high quality projects.
5.4.4.4 Communications Structures
Participants also mentioned challenges relating to the size of the organisation and to the nature of Scrum methodology which encourages self-organization among team members. In most instances Scrum teams work better in projects where communication is open between team members and the customer. However, this is not always the case in some organizations in fact some companies “…do not have openness, they do not have transparency, they do not have the trust structures in place” (JP – R5). These make Scrum implementation difficult resulting in delays in delivering the project on time. The same participant pointed out that Scrum
96 methodology works best in environments where the communication structure is not complicated but in some instances “… the communication structures are very complicated” ending up affecting the project delivery (ibid). Another Scrum master added that in some companies, communication is “…difficult because there are too many people with input so there is no one owner of the backlog” (GT – R6). If there are too many hierarchies and too many roles to accommodate in an agile environment, it becomes a challenge to implement Scrum without experiencing time delays (ibid). Further, some teams experience issues with unstable requirements, for example where the customer might change the backlog at any time, but not change the release date. On this point, one Scrum master said “We have had a lot of issues even when thing has been approved but they turn around and say actually we didn’t think this, you have got to do it this way instead or actually can’t you just slide this as well” (AP – R4).
Associating these statements with the main research question of this thesis, it is clear that quality assurance processes in Scrum are flexible, but also problematic. The three quotations suggest that processes are not seamless – mostly because of multiple structures and stakeholders with divergent interests that are involved in the same project. In fact, the last quotation reveals a sad state of affairs, showing that even those phases, where informal methods are used, can lead to frustrations.
5.4.4.5 Insufficient Testing
The testing process was also cited as one of the biggest challenges facing Scrum teams. In some instances, the Scrum process itself is questioned regarding the accuracy and the amount of testing conducted in such a short period of time. Although there were few participants who highlighted testing as one of the factors which affect delivery of a quality product, testing is perceived to be a huge concern to those who are experiencing it. In this instance, one team member (QA manager) said “…the biggest challenge I believe is to do proper testing” (GT – R7). On the same point a Scrum master said “from the time perspective the developers don’t have enough time to do all that regression testing” (NP – R3). As mentioned in the preceding passages, insufficient time to do testing is one of the contributing factors to unsuccessful Scrum projects in terms of quality assurance. However, this does not necessary mean that these projects fail completely, but teams often fail to deliver on time. If they do deliver on time, there is the possibility that developers will be working on fixing defects instead of adding new requirements (NP – R4). This participant also mentioned an instance where one of the developers implemented a change, and because of time, there was an assumption that the code had gone for testing, but it had not. In fact the project was delivered went into production
97 and they realized later that that code had never been tested and the team were “lucky that nothing happened” (ibid).