This blog post demonstrates the value of piloting and validating instruments within the given context for monitoring a program’s progress.
Three of our PFP staff members arrived in Santo Domingo this week to work with our Resource Mothers. As PFP’s internal evaluator, my goal for this visit was to meet with the Resource Mothers and their coordinator, Jacqueline to discuss how they are currently tracking the progress of their clients and to revise the measures to accurately reflect daily life, to ensure questions asked are clear and to provide a means for PFP staff to track and monitor the program’s progress.
Currently, the Resource Mothers collect information through home visit interviews conducted four times throughout the course of the program: when a client enters the program,when she finds out she is pregnant, when the client gives birth, 6 months after her baby is born, and one year after her baby is born. Questions asked address progress on program objectives such as prenatal visits, social support, attendance at birthing and breastfeeding classes, school and work, nutrition, and postnatal care. Questions were revised in order to create consistency across questionnaires and response choices to track progress over time.
We arrived at the Resource Mothers office to the chatter and warm welcome of 19 Resource Mothers. They greeted us with open arms. We presented the activity of reviewing the revised questionnaires and asked the Resource Mothers to separate into four different groups, one per questionnaire. Each group was asked to read through the questionnaires to ensure that the questions were clear. If the Resource Mothers thought that something was missing from a question or could be phrased in a better way, they were encouraged to make suggestions. Suggestions were reviewed as a large group in order for everyone to provide feedback.
One example of the importance of having open dialogue with the Resource Mothers about the questionnaires arose when we were discussing the question “When do you attend school?”. Several Resource Mothers noted that a response choice was needed (the questionnaire was missing the option of reporting they attend school in the afternoon). This example was one of many questions we reviewed and edited or added to create a more comprehensive, appropriate, and valid set of questions to track client’s progress. The level of engagement and commitment from the Resource Mothers to take the time to ensure that these questions were meaningful will be invaluable to demonstrating how the program works and where improvements can be made. As a researcher and evaluator, this experience of reviewing and piloting these instruments within the community was extremely successful and I am grateful to the Resource Mothers for investing their time to demonstrate how the program that they are a part of works within their community.