One way for an organisation to have some valuable insights regarding itself, its assets and processes is to conduct some appropriate measurements and interpret them correctly. The question of what comprises appropriate measurements is critical. The GQM method helps one to arrive at those measurements, provided there is a set of specific goals for that organisation [2]. To get from each specific goal to a measurement the method defines three levels (Fig.1), each representing a step on the way from the goal to a metric:
– Goal:Conceptual level, where each of the specific goals is represented.
– Question: Operational level, at which questions are posed against the goal in order to assess its achievement and identify some of its aspects through which the progress can be observed and measured.
– Metric:Quantitative level, at which appropriate metrics is selected to answer the questions posed at the operational level in a quantitative manner.
Fig. 1.Levels of the GQM model
Following is an example of the GQM process using a simple example of a web application of a company selling airplane tickets. The process starts with a spe- cific goal. It has to be formulated in a particular way and be associated with an object, which in the context of software development could be an artefact, process or resource. Within the goal a purpose is defined, together with a par- ticular quality issue of the object and a viewpoint from which it is looked at.
The example goal is to improve usability of the ticket booking feature from the point of view of a person booking a ticket (end user). The object of the goal is the ticket booking feature. The purpose is to improve, the quality issue is the usability and the viewpoint is that of a person booking a ticket.
Once the specification of the goal is ready, questions can be posed against the goal and metrics can be proposed to answer the questions. Looking at the example goal above, its purpose and quality issue one can deduce following questions:
– What are the completion rates of the feature?, i.e. what portion of users actually accomplishes booking a ticket.
– What is the average user task time?, i.e. how much time does it normally take a user to accomplish booking a ticket.
– How satisfied are the end users with the feature’s usability?
Each of the questions can have one or several metrics answering it. The whole GQM can be presented in a tabular format [2], as in the Table1. Often, there is a set of specific goals and a GQM is produced for each one of them. Evidently, having these set of specific goal is what makes deriving metrics possible. The question arises then of how the set of specific goals for an organisation is actually produced.
Table 1.Exemplary GQM
Goal Improve the usability of the ticket booking feature from the point of view of a person booking a ticket (end user) Purpose Improve
Quality issue Usability
Object The ticket booking feature Viewpoint End user
Question Q1: How long does it take a user to complete the booking?
Metrics M1: Average task time
M2: Standard deviation from the average
M3: Percentage of cases exceeding the average maximum limit
Question Q2: What are the completion rates of the feature?
Metrics M4: Percentage of successfully completed bookings Question Q3: How satisfied are the users with the feature?
Metrics M5: Average subjective rating by users grading the feature
In fact, the team constructing GQMs needs to accomplish a thorough work of understanding the context of the organisation itself, its mission, strategy, policy and high level goals. The team also needs to understand the local context of the object being measured, which can be a product or some process within a project.
Finally, viewpoints of various stakeholders need to be taken into consideration to serve as another important element formulating the goals. The whole process involves interviewing stakeholders, analysing the environment and collaboration between the corporate management, project and GQM team. The derived goals need to be analysed with regards to their relevancy and relation to the high level goals [2].
Evidently, deriving the specific goals is a lengthy and nontrivial process.
Therefore presenting only the specific goals and their corresponding GQMs is insufficient to understand the greater context and the relationship between spe- cific goals, the environment and strategic high level goals of the organisation. So the question arises, what techniques and approaches can be used to simplify the process of deriving the set of specific goals? Furthermore, can the context and its relationships with the specific goals be represented in a concise and compre- hensive manner?
Yet one more important aspect related to the metrics is related to the actors involved. Does the measurement process involve personnel, devices or software?
Who or what is responsible for accomplishing the goal and gathering the mea- surement data?
The GQM’s tabular format above is in fact only a concise output of the work preceding it, while the context and the actors involved with the system can be conveyed through text or by using other modelling approaches [2]. The KAOS
goal modelling approach provides tools and techniques necessary to reflect the strategic context and actors involved. In the following section, KAOS and its tools are introduced, and the benefits of combining it with GQM is elaborated upon based on an example.