7.3 Making Site Visits and Collecting and Analyzing Data
7.3.3 Review Local Records and Reports
In addition to interviewing the local staff, you and the staff will need to review available records and reports. Records frequently reveal many is-sues that the staff may not have mentioned.
Look for the following during this review—
• Check whether all records and reports are easily accessible to the person in charge and/or program manager and are well organized.
If records are not accessible, it is unlikely that keeping them up to date is a priority. Poorly organized reports are difficult to use in making the calculations described in previous chapters.
• Confirm that records are complete and that the math is correct.
For self-balancing reports, determine if the report balances correctly.
As appropriate, determine whether the opening balance on the current report equals the ending balance from the previous report.
Guidelines for Conducting Interviews
The following are guidelines for conducting interviews at all levels—
< Gather as much background information as possible before the interview.
< Prepare questions ahead of time.
< Introduce yourself.
< Make eye contact (as appropriate).
< Be polite.
< Gain your interviewee’s trust by being honest and clear about the purpose of the interview.
< Select an informal and nonthreatening environment for the interview.
< Establish a positive attitude; show interest in whatever is said without passing judgment.
< Help your interviewee by restating or clarifying ques-tions, if asked.
< Use language and terms easily understood by the per-son you are interviewing.
< Be a good listener and observe what is being com-municated, both verbally and silently through body language.
< Probe for the why of things, but be sensitive if the person does not want to discuss a particular topic.
< Record the information you are gathering accurately;
do not try to interpret it.
< Use follow-up questions to learn more.
< Follow the interests and energy of the person being interviewed.
< Thank your interviewees for their information and time.
< Send the interviewee a copy of your final report, as appropriate.
Three tips are—
• On consumption records, confirm that dispensing protocols are being followed.
Dispensing protocols determine how many of each type of contraceptive or medication should be given to each customer.
Consumption records often collect information about the number of customers who have visited the clinic. The total quantity dispensed should match the number of customers served.
• Verify the date that reports were submitted.
If you have visited higher-level facilities, compare the completion date with the date the higher level received the report. Confirm that the person in charge knows when the report is due at the higher level. If the facility receives reports from lower levels, check the dates completed and askabout the date received.
• At intermediate facilities, determine how the data is aggregated and whether it is followed correctly.
At the district level, depending on the reporting system (1) clinic reports may be aggregated, added to district data, and submitted as one report to the central level; (2) clinic reports may be
aggregated on one form and district data sent on a separate report;
or (3) district data and each clinic’s separate report may be sent to the central level. Determine if the correct procedure is followed and whether the math is correct. See section 2.4.2 for more
information.
• Assess the stock status for a sample of products.
If consumption data are not available, use issues data and the results of your physical inventory to determine the number of months of stock available for a sample of items. Record this information in your report.
• Review the stockkeeping records.
Check a sample of stock cards to see whether they are complete and mathematically correct. Verify that physical inventories are conducted routinely and the results recorded on the cards. Look at the levels of losses and adjustments, and ask for information
When reviewing records or reports, you may not understand how the form was completed or the origin of a specific number. Ask, “Can you tell me how you got this number?” or, “Can you tell me how you completed this box on the form?” Neither question implies a judgment, but both ques-tions encourage the staff to tell you how the form should be completed.
The general question, “Are you having any difficulty completing any of the forms?” may also encourage staff to tell you about any problems they may be having.
Your review of records and reports may contradict what you were told during your interview. For example, you may have been told that physical inventories are taken monthly, but the stock cards may indicate only an occasional inventory. When you find such discrepancies, ask for clarifica-tion. Do not assume that the person interviewed was dishonest; items such as physical inventories may not always be recorded even though they were conducted. During the interview, the staff member may have thought you were asking about the policy rather than the actions actually taken at the facility.
Be sure to record consumption rates, stockouts and their duration, levels of losses and adjustments, level of knowledge of policies and procedures, and other management issues, so you can include these in your report.
At the conclusion of your site visit, thank the staff for their assistance and cooperation in helping you with your assessment. Report what you ob-served to the supervisor (if he or she did not accompany you), explaining the strengths and weaknesses you observed. You can also use this second interaction with the supervisor as an opportunity to verify your findings.
7.3.4 Additional Considerations for Interviewing and Data Collection at the Central Level
At the central level, in addition to the tasks mentioned above, your as-sessment should include collecting additional information about the whole program. Try to meet with senior members of the staff, such as the director of family planning, preventive health services, HIV/AIDS prevention or reproductive health; the central warehouse manager; and the program’s logistics manager. Include the following questions in your interview—
• What are the program’s goals? What is the level of coverage? Are
• Will management of different programs be integrated (e.g., family planning with other reproductive health services)? What will happen to resources previously assigned to individual programs?
What changes in the number of managers and their duties are expected?
• Will responsibility for logistics be decentralized? To what level?
What problems are anticipated?
• Are new drugs or treatment protocols to be introduced?
• Will national essential drug lists (EDL) be updated in the near future? How will this affect the number of products intended to be available at each level of the system?
• What level of financial support is available for logistics? What level of other resources, particularly human resources, will be available?
• How does the program interact with donors? What donors are involved?
• What logistics data are used to make decisions? What types of decisions are based on these data? What is the level of confidence in the quality of the logistics data that are received?
Meet with donors, as appropriate, particularly the donor sponsoring your assessment, and ask them these questions as well. Your recommendations should reflect the policies of your sponsoring agency and should not com-mit the sponsor beyond agreed-upon goals.
Your draft report should address how your recommendations would sup-port the goals of the program.
7.3.5 Analyzing Strengths and Weaknesses of the System
After you conduct all your site visits, reconvene your working team or teams and collect all data in one location for analysis. Your findings should include both the strengths and weaknesses discovered during your data gathering. It is often true that strengths may also have a component of weakness. For example, the system may collect all essential data (stock on hand, losses and adjustments, and consumption). This is a powerful strength, but, if central-level decision makers do not use the data, it is also a weakness. Your report should provide a balanced discussion of strengths and weaknesses. Program managers will not respond positively if their systems are only criticized.
In analyzing the available data, separate all information not related to logistics. Save important information on other functions, such as quality of care, for the last part of your report. Also, remove mention of problems that are beyond the system’s control. For example, difficult weather condi-tions and political challenges usually cannot be overcome by program-level logistics interventions. These important considerations should be included
Program Goals and Central–Level Interviewing
At the central level, ambitious goals may have been set for progress in family planning. When you conduct your interview at the central level, listen carefully to the types of new activities planned for the next two years. Determine what makes these activities different from previous efforts. Make sure your report describes how your recommendations support these efforts.
When meeting with donors, listen carefully to their level of support for program goals. Remember, donors may not be able to support all program goals. Your report should reflect the gap between available resources and the resources needed, if such a gap exists. Be sure, also, that your recommendations do not contradict donor policies.