Above: Batman by mecookie (2012), via Flickr, is licensed under CC BY 2.0.
There is a famous Sidney Harris cartoon depicting two men in front of a blackboard where they have scribbled a formula that covers the entire board. One of them is pointing to a section in the middle of the formula that reads “then a miracle occurs,” implying that whatever result they got at the end of the formula was caused, in part, by an unexplained, miraculous event. The caption reads “I think you need to be more explicit here in step 2.” That man is probably a colleague of mine, working on design, monitoring and evaluation.
It is very common for organizations to start working on a project with a goal and solution in mind (Jana wrote this great blog about how to address this in the development process). In the nonprofit sector, we know that in order to stay afloat and deliver on our mission and vision we need to make our best, most informed bets on how we think we can contribute to the impact we aim for. Good bets are grounded in a thorough understanding of the problem we hope to solve, the stakeholders involved, the context in which we work, and a well-thought-out, explicit explanation of how we think change will occur – all to reduce the risks of placing our bets on miracles.
For example, say you are baking bread. You follow the recipe step by step: you mix all the ingredients, knead the dough, and bake it at the right temperature and for the right amount of time. You are meant to get a nice loaf of bread, right? You did everything you needed to do. But, what happens when you open the oven and find a flatbread cracker? (No shame, it’s happened to the best of us.) Your causal assumption about what happens after you do everything you could, did not hold true. The miracle moment, step two in the cartoon, is that moment between you doing everything you can do and either ending with a bread cracker or a loaf of bread. That portion in between, where things are out of your hands and the miracle occurs, is what we aim to make explicit when we design a Theory of Change (ToC). For, it is rarely a miracle, but rather live active yeast feeding on the sugars of flour which, combined with gluten that is activated by kneading, makes your bread dough rise.
Successful bread baking doesn’t rely simply on you following the instructions, but on that yeast process you trust to work well once you’ve done your part. This is important because in the case of bread, there is enough evidence to support the causal assumption that live, active yeast will make the bread rise so, if your bread is flat, it is likely because the yeast was not active (or some other contextual factor like the altitude at which you are baking). Further, if you’ve made your assumptions explicit and are now aware of the role of yeast, you can do things like monitor if the dough rises after you knead and before it goes into the oven or you can let it rest and knead a second time, therefore increasing the certainty that the yeast is active and that you’ll end up with a loaf of bread. That is, knowing and being explicit about your assumptions can help you do things different to get you closer to your goals.
In social phenomena, however, we don’t always have a ton of evidence to explain how change occurs. There are usually higher levels of uncertainty than in bread baking, so being explicit about the changes or outcomes we expect and our causal assumptions between them can be difficult but, in a context of limited resources, becomes ever more important. This is why we are doing a ToC at Akvo: we want to make the best, most informed bets so that we don’t end up with flatbread. What is the yeast process that occurs between what we do at Akvo and the “equitable, sustainable, global society that works well” that we envision? Understanding the causal assumptions we make about how change occurs can only help us make more strategic decisions about what we do, in how we partner, and on what we build and what we don’t.
Developing Akvo’s Theory of Change. Photo by Henry Jewell. Amsterdam. 21 February, 2017
Alvaro made this cool video and wrote a blog about the ToC workshop some of us participated in but the process started way before that and is still underway. I won’t give away the content of the ToC but want to highlight, from the Akvo Americas perspective, a bit of the insights that inspired us to work on it and how, even if the ToC is not complete yet, it is already informing the way we work.
First, about the insights that inspired us to become involved. Remember my colleague in the cartoon pointing at “step 2” on the formula? Well, when I joined Akvo over a year and a half ago and read our vision and mission statements I felt bit uneasy. Our vision statement says “Akvo envisions an equitable, sustainable, global society that works well. We believe that this will occur faster if country governance and international development are open, transparent, collaborative and effective.” Our mission is “to enable data to be captured, understood and shared for country governance and development activities. To achieve maximum impact as rapidly as possible, we develop open source, online technology and professional services as well as some hardware, while working closely with organizations that share our vision.”
I wondered how helping organizations capture, understand and share data leads to more open, transparent, collaborative and effective country governance and international development? I wondered how more open, transparent, collaborative and effective country governance and international development leads to an equitable, sustainable, global society that works well? What’s the yeast here? To quote the guy at the blackboard, I think we need to be more explicit here.
I wholeheartedly believe in what we do: open source, online technologies and professional services enable people to capture, understand and share data. We are already doing this and have good cases to show that we do it well. But, what happens after we enable people to capture, understand and share data? Do we need to knead the dough twice?
With this in mind, Henry and I interviewed two Akvo team members and two trusted collaborators to get insight on what they think happens next. We asked four questions:
- Do you believe that more openness, transparency, effectiveness and collaboration in country governance and international development leads to more equitable societies? Consider each – openness, transparency, effectiveness and collaboration – separately. If so, how does that process occur? If not, why not?
- What effect can open, online platforms that help capture, understand and share data have in the process of achieving more equitable societies?
- What effect can efforts to build capacity around data use have in the process of achieving more equitable societies?
- What are the three main opportunities and the three main challenges in achieving more openness, transparency, effectiveness and collaboration in country governance and international development?
One of the respondents told me I could write a dissertation on just the first of those questions so I’ll spare the details and note: 1. I think it is a good exercise for everyone at Akvo to think about how they would answer those questions and what it would mean for the work we do. 2. All respondents indicated that there is a large jump from open online platforms that help capture, understand and share data to an equitable society that works well. 3. I was surprised that all four people I spoke with asked what we mean by openness and how it differs from transparency. And 4. Overall, respondents agreed that tools are useful to the extent that they help people in the process of capturing, understanding and sharing data that informs decision-making, in favor of positive changes in society. There are a ton of caveats here but the main one was expressed best by Ben Mann, one our close collaborators:
“I view technology as Batman’s utility belt. You don’t want just his utility belt, you want Batman. So, technology has a role to play but the most important part of it is probably what people do with the tool. The political will and all the resources available – including online platforms – play a role if they can actually achieve what we want to do in international development.”
Discussing the Akvo USA Foundation strategy. Washington DC. 18th April 2017. Photo by Henry Jewell.
This insight leads me to the second point I wanted to make, which is how the ToC work is already guiding our work in the Americas. We know that technology won’t solve development problems on its own. Our tools are Batman’s utility belt but, what we really want is the user of the tools – our partners who work to improve the lives of others – to have all the skills, knowledge and tools to generate and use high quality data. That is Batman. So we place our bets on people and their capacities to transform and use insights from data to change their realities.
And there are several reasons why we place our bets there.
Personally, in a previous professional life, I did research on research impacts and, while there is some debate about definitions, it generally refers to how research findings have an effect on projects and policies. The process is convoluted at best and certainly goes beyond having quality data. The politics and incentives around data use are key determinants. This issue about context is also highlighted in this brand-new study on Avoiding Data Graveyards:
“Technocratic ideals of evidence-informed policymaking and data-driven decision-making are easily undercut by individual prerogatives, organizational imperatives, and ecosystem-wide blind spots.”
I also know this from painful experience after, in yet another professional life, someone tried to discredit the results of my work collecting and analyzing data because the findings did not align with their political agenda. People, incentives, context matter. Not acknowledging and addressing this can make technology and the generation of data useless.
So our strategy in the Americas aims to strengthen the services we offer to help build capacities and work on creating a more fertile context for people to use data insights for the benefit of society. To generate more evidence about how we move from data to impact. This means that we need to be clear on what services – not just tools – we offer, how we conceptualize and measure success, and for a large part, doing all of this in Spanish.
Henry, Emily and I discussed this at length during our team meeting on April 18th and 19th. Based on the ToC, we developed an operational plan for the next 18 months. Four priority areas, with specific activities and output targets that we will review and revise in order to ensure we learn and adapt as necessary.
We look forward to collaborating with the other Akvo hubs to draw on their experience working in different contexts so that we don’t recreate the wheel or place our bets on miracles. We want to make sure we are aligning efforts and understand our context well enough to help Batman bake a delicious loaf of bread.
Ethel Méndez Castillo is a monitoring and evaluation specialist, with a particular focus on the US and Latin America. Follow her @ethelnmc.