LITERA TURE REVIEW
2.4 EVALUATION TOOLS THAT CAN BE USED IN ASSESSING HEALTH PROGRAMS THROUGH OPERATIONAL RESEARCH
2.4.4 Previous Studies Conducted, Using the LQAS Approach
et al., 2001).
Within each SA, no confidence levels are calculated as the samples of 19 (if 5 SAs are used) or 24 (if 4 SAs are used) are small, and the information is used for decision making rather than as
4~}' •
coverage values. "-;/~-K iI';I
l:,r ~ ";,."...:=-.". .... ~
/I~/ '~(..
,'} ffJ"' iIJ1"D'~ ,L\l
>;;'
j~~C:::~ .~ . .,'''''' ~ ~ ...
i\
,)H Ln:3hARY ;;:,",In LQAS, What a Sample of 19 or 24 Can Tell US ;~~:!:;,'t~
__
~4yjLQAS is meant to assist local managers to monitor the
p~dormarf t~' ofthe
coverage of health services in their catchment areas. The survey points out to managers, areas with obviously low service coverage and areas with obviously high service coverage. Due to resource limitations, managers are interested in finding out where supervision should be focused. Instead of spreading scarce supervision resources equally to all catchment areas, LQAS enables managers to identify low performing areas according to an upper threshold and a lower threshold of performance specified before the survey.Lots which perform above the upper threshold are "acceptable" and attempts can be made to maintain this level of performance, whereas lots performing below the lower threshold are
"rejected" and need focused attention (Valadez et al., 2001).
What a Sample of 19 or 24 Cannot Tell Us
This evaluation tool cannot calculate exact coverage in a supervision area as the sample size is too small. In addition, the LQAS method cannot set priorities among supervision areas that have little difference in coverage among them (Valadez et aI., 2001).
A study conducted in Madras, India with an objective to explore the usefulness of LQAS to identify division in a city that had an immunisation coverage level of 80% for all of the four Expanded Programme of Immunisation (EPI) vaccines. The conclusion was that the study demonstrated the utility of the LQAS technique in identifying unsatisfactory pockets in Madras City when the overall coverage was satisfactory. The technique will have greater application with an increase in the number of large units (cities/districts) having an overall coverage of 90%
or more (Singh et al., 1996).
LQAS was used to evaluate the technical competence of two cohorts of family planning service providers trained with a new six-week curriculum developed by the Kenyan Ministry of Health Family Planning Training Program. This study, using an LQAS methodology helped to identify task categories in which the new curriculum needed strengthening
01
aladez et al., 1997). The WHO EPI compared the LQAS methodology to the 30-cluster sampling methodology more usually advocated by the WHO as a rapid epidemiological assessment method to evaluate immunisation coverage. It showed that data collection took longer to complete in the LQAS survey than the EPI cluster survey. Likewise, travel and cost was higher in the LQAS than EP!.However it may be useful for routine monitoring of immunisation programs in small areas where local staff are used and a very heterogeneous coverage exists in the area being evaluated.
(Sandiford, 1993)
In a study in Mali, the LQAS methodology was used to determine the overall coverage and quality of the data in the HIS, to identify specific health diseases that needed improvements in data collection methods, and to determine particular areas of weakness in data collection (Stewart et al., 2001).
United Nations Children's Fund (UNICEF) has used this method to estimate measles
vaccination, using a good performance to be 80% coverage and a "poor performance" to be 50% coverage to demonstrate the strength and limitation of the LQAS method. The exercise revealed that LQAS is very good at detecting poor performances. Its sensitivity is almost 99%
and its community risk is less than 2%. On the other hand the LQAS method is not specific and its positive predicted value tends to be low in most settings. Thus, the LQAS method is not necessarily good at predicting when a programme is doing a good job. (Singh et al., 1996)
2.4.6 The Use of LQAS in the TDCSP
Due to the fact that it is almost impossible to survey an entire population, survey evaluation methods have to rely on extracting a sample from the entire population to conduct the analysis.
Cluster sampling was proposed as a reliable and cost-efficient way to gather the information needed, and has been the primary sampling method used in KPC surveys over the last 10 years.
This sampling method was selected assuming that the data collected would be used for the purposes of decision-making and program management. The KPC survey was never expected to be a tool to address research issues or gather in-depth social and demographic data, which would require different sampling approaches. (World Vision South Africa, 2000).
During the last 10 years KPC cluster-surveys have considerably improved the ability of Child Survival Projects to identify priorities, define objectives based on data, and measure progress towards these objectives. KPC cluster-surveys were never expected to measure change between two periods of time, or to compare different groups of population in order to demonstrate that a specific intervention was the cause of an observed change. (Valadez, 1991)
LQAS has come to the fore as a method of sampling for surveys, for being able to assess performance in each SA of a project district, and for routine monitoring during child survival activities (Valadez et al., 2001). It was decided that a survey would be undertaken to fill in the data gaps from the KPC in 2000, and to teach the LQAS methodology to a broad range of role- players who would be able to use it in their work in the District. In the process, the indicators for the project would be refmed (World Vision South Africa, 2000).
In addition, by using the LQAS method, it would be possible to compare SAs (municipalities in the case) to decide on health priorities and interventions in municipalities. LQAS could be used
to find Health District area coverage for chosen indicators and to monitor whether the gains made in previous projects are being maintained. (Valadez, 1991)