Seven grantees, 13 states, three levels of government and one mission: to increase accountability for maternal health in Nigeria, which has one of the world’s highest maternal mortality ratios. This presented a complex, multi-faceted context for establishing a theory of change to guide an evaluation of a grant portfolio in Nigeria.
The MacArthur Foundation commissioned EnCompass, LLC to conduct a learning-focused evaluation that illuminates what is working well to advance maternal health in Nigeria, and what alternative strategies could be considered for increasing government accountability for maternal health. Establishing a theory of change was an important first step for bringing to the fore what grantees were trying to achieve and how they intended to get there. The pathways to maternal health accountability are complex, and this is reflected in the diversity of interventions in which the grantees are involved. Our aim was to develop a framework for the portfolio to which individual grant activities would contribute.
Our key challenge for the evaluation design was how the sum of the parts leads to a unifying theory of change that would resonate with the grantees and enable evaluation of the entire portfolio. To determine this, we needed to design a process that would create a shared understanding of what each grantee was going to do, to what end, and how all the pieces fit together.
Creating an empowering environment
Many evaluators begin with a theory of change in hand and move quickly to data collection. EnCompass, however, collaborates with stakeholders to develop a meaningful theory of change. We spent two days in Nigeria in a participatory process with the grantees and donor, which was crucial for our collaborative work. We started with appreciative visioning, asking participants to envision a future in which their work was wildly successful. We did this by reflecting on the following scenario:
Imagine that we are coming together to celebrate the success of your grant. All we discussed at the evaluation design meeting has been achieved and more. What has happened to make your grant such a success? What are people saying? What are the results?
Appreciative visioning can help people take a step back to focus on the vision and see the commonalities of their work. This manifested in a spectacular way. As participants shared their stories, any initial sense of distance between the individual grants melted: it was easy to see how they were working toward a common goal. The grantees were able to articulate how their joint work contributed to the ultimate goal of keeping Nigerian women from dying.
Tip: It is important to cultivate an atmosphere that empowers participants to lead the process. This creates a more relevant theory of change and fosters enthusiasm for the evaluation.
Trusting the Appreciative Process
Appreciative visioning was the first step in helping participants understand how the pieces fit together for a common purpose. This set the stage for us to develop collaboratively a set of indicators that would enable measurement not only of each grant’s progress, but of the entire portfolio. For those of us who use appreciative inquiry, there wasn’t anything particularly special about this process. The trick is to trust the appreciative process, the participants, and the possibilities.