2.4 Agile Software Development
2.4.4 The Migration from XP Methodology to Scrum Methodology
According to Fowler (2005) Scrum attaches importance to the management aspects of software development, dividing development into concise time periods referred to as iterations or sprints. The intention is to establish closer monitoring
and control of the software process by engaging in daily meetings referred to as the daily scrum. As was observed in the multiple case study by de O Melo et al.
(2012), a viable arrangement would be to combine Scrum and XP thereby enabling the attainment of a project management and software engineering focus at the same time. This complementary coupling has been pivotal in establishing Scrum and XP as the 2 most popular agile methodologies (Fitzgerald et al., 2006).
Theoretically, the project management value provided by Scrum and the software engineering value provided by XP would render an integration of these methodologies an optimal one. This strategy of selecting only parts of agile methods that are perceived to be ‘fit for purpose’ within an organisational context is referred to as method engineering (Henderson-Sellers, 2006) or a meta-method approach (Fitzgerald et al., 2006) where a methodology is developed from existing method fragments.
In order to establish the viability of the method engineering approach, Fitzgerald et al. (2006) adopted a case study approach to investigate the customisation of XP and Scrum at the software engineering division of an international organisation. The study involved an interpretive, exploratory strategy that entailed a series of interviews with software engineers who were responsible for the customisation of XP and Scrum. The interviews were designed to obtain feedback from the software engineers on the basis of their continuous monitoring and evaluation regarding this customisation process over a 3-year period. The culmination of the study was a presentation of a tentative framework for software development that consisted of a hybrid arrangement of XP and Scrum based methods, illustrated in Figure 2.10
The framework proposed by Fitzgerald et al. (2006) consists of an invocation of selected methods from XP and Scrum where the objective is to leverage the benefits of the methods in a complementary manner so that the customisation process resonates with the prevailing socio-technical environment in an organisation. As illustrated in Figure 2.10, the specific context of the problem domain necessitated the inclusion of only a subset of the full suite of XP methods.
The omission of certain XP practices such as the presence of an on-site customer, continuous integration of code artefacts and a 40-hour working week was necessitated by the requirements of the project. Also, the practice of pair programming and the use of a metaphor to describe the main system functionality was used sparingly because it was not practically feasible to enforce either practice.
The Scrum contribution is centered on the value provided by the sprints which are used as a node of the development process around which much of the planning and control is invoked. This approach obviated the need for the ‘planning game’ approach advocated in XP. In this way, the customisation process entailed the selection of specific methods from a palette of XP and Scrum methods so that Figure 2.10: An Integration of XP and Scrum Methodology (taken from Fitzgerald et al. (2006))
complementarity of these methods is manifested in the support provided by XP for the technical aspects of the systems development effort while the project planning and tracking support is provided by Scrum. It should be noted that Paulk (2001) warns against the adoption of a fragmented, method based customisation of an agile methodology. However, the successful customisation of agile methodology where the agile methods are chosen in an a la carte manner as suggested by Fitzgerald et al. (2006) is corroborated by Treacy et al. (2008) who used a multiple case study approach involving 3 companies that used customised versions of XP as well as an integration of XP and Scrum methodologies. The findings of this study concur with the claims by Fitzgerald et al. (2006) that XP and Scrum can be modified to match the requirements of the project as well as organisational and management norms and expectations.
The preceding discussion alludes to the positive reports regarding the complementarity between XP and Scrum. While this integration of XP and Scrum has been endorsed by the empirical findings of Fitzgerald et al. and Treacy et al., both these studies are subjected to the limitation that XP and Scrum have not been assessed to ascertain whether they may each be individually superior to the customised versions. Fitzgerald et al. do concede that the problem with agile method engineering is that only those parts of an agile method that are perceived to be ‘fit for purpose’ are included in the customised version, thereby running the risk of compromising the holistic benefits that may be achieved when using the methodology in its original format. These concerns regarding the method engineering of agile methods coupled with the observation that there was a need for better project management guidance in the adopted methodology resulted a gradual decline in the popularity of XP as well as the XP and Scrum combination.
This trend is explored in the succeeding discussion.
According to Dybâ and Dingsoyr (2008), many of the academic studies published until 2005 focused on XP. During this period, the prominence of XP in the academic community was also aligned to its prominence in the practitioner domain, as illustrated in Ambler’s survey (reported in VersonOne, 2011) of 4232 software practitioners who were using agile methodology. A significant outcome of
this study was that the highest adoption rate of all the agile methodologies used, was XP (57%). Vijayasarathy and Turk (2008) reported on the results of a similar study as the one conducted by Ambler. This study was underpinned with a theoretical framework that consisted of Rogers’ Diffusion of Innovation Theory(DOI) (explained in Rogers, 1983). One of the objectives of the study was to survey a sample of early adopters of agile methodology and to establish which agile methods were most popular. Using Rogers’ DOI theory as a theoretical base, the study examined the preferences of early adopters of agile methodology and concluded that there was a strong preference for agile methods that were intrinsic to XP methodology. This included TDD, pair programming and code refactoring.
The early adopters of agile methodology had a preference for the engineering oriented aspects of the methodology (as embodied by XP) rather than the management aspects of software development (as embodied by Scrum). However, the ‘later adopters’ of agile methodology had a preference for the methods that enabled better control and management of the software development process. This trend that represented a need for better control and management of the software process rather than focusing on the operational aspects of the process, manifested in Scrum replacing XP as the more popular agile methodology.
From a practitioner perspective, the results from Ambler’s survey were slightly different to the global survey conducted by VersionOne13 (see VersionOne, 2013) in 2013. The results from the 2013 survey reported Scrum as the most adopted agile methodology (40%), followed by XP (23%) and then a hybrid methodology based on Scrum and XP (14%). The results suggested a trend that Scrum was regarded as the most popular methodology, followed by the Scrum hybrids and then XP. This trend continued in an unabated manner and Scrum has become established as the most highly adopted agile methodology. The preceding assertion resonates with the prediction by Dybâ and Dingsoyr (2008) that Scrum methodology was gaining substantial focus and would soon replace XP as the most popular agile methodology. In the 2015 VersionOne survey, a total of 3880 software
13 VersionOne is a software organisation that conducts global survey on issues pertaining to ASDM in
practitioners were surveyed on their use of agile methodology. The results indicated that 58% of the respondents were using Scrum as the default methodology for software development. The Scrum/XP hybrid was the 2nd most preferred methodology with exclusive use of XP reported at less than 1%.
The Migration from XP to Scrum
A significant outcome of the 2015 survey was an affirmation of the trend regarding the decreasing reliance on exclusive use of XP methodology and an increase in the adoption of Scrum methodology. The popularity of the Scrum/XP hybrid was however an indication that many agile concepts attributed to XP methodology (such as TDD, code refactoring, pair programming and continuous code integration) were perceived by practitioners as being quite useful to the software development process. The significant preference for a Scrum based approach was quite distinctive in this regard. A possible explanation for this trend is that the regular ‘stand up’ meetings intrinsic to Scrum enables more frequent reporting on the status of a project thereby facilitating better project management and ultimately ensuring accountability on the consumption of project resources.
This overwhelming preference for Scrum warrants a closer inspection of the underlying techniques used in the methodology.