One of the key criticisms against the current state of international development cooperation is that it is very hard to see what the results from the invested funds and the work are. Often the results reporting is unclear or not accessible at all. The International Aid Transparency Initiative (IATI) has gone some ways towards addressing this, but IATI has so far primarily been about making data available about funding flows. Today it doesn’t really say anything about results.
Above: Typical results reporting archive, in the basement of a major international development funder. Expensive to produce and of limited use. Photo: Peter van der Linde.We at Akvo, working together with some of our key partners, are not alone in wanting to extend the IATI effort to include results reporting and we have long talked about this as a concept which we call end-to-end transparency. An end-to-end transparency system for international development, as we envisage it, will take many forms. Not all parts of the system will necessarily be using the IATI standard. But we think that IATI should not be limited to the current standard, but be used as a vehicle to extend the standard to encompass results reporting as well. It is an important stepping stone towards effective and useful international development work.
Additionally, we believe that it would be good if as much as possible of the information and data systems we build are based around open systems: open standards, open data and open source software. To best illustrate how we envisage it to work we’ll use some specific examples to show how this can all hang together. Our examples are mostly based around IATI, and we presume that extensions to the standard are in place, for example to support the reporting of progress of projects and programs as well as results. But first a little about traditional results reporting.
How results reporting used to workTraditionally the results from an international development programme would be reported on in a document. Generally a 30-50 page document, often in paper format, read by a few people to quickly ascertain that the money was spent responsibly, and then stored for accounting purposes in the basement of the organisation who provided the money.
The data in the document generally comes from surveys that have been performed in the communities where the work is being done. Typically a so called baseline survey would be created before the start of the programme. Enumerators would go around in the community and ask questions from the survey and fill in the answers on a piece of paper. The papers would be collated and then the information would be entered, by hand, into spreadsheets. These spreadsheets would then be used to keep track of the data. The whole process was slow, cumbersome and had a lot of data entry errors. (Some of our Akvo colleagues in Asia recall seeing trucks literally filled with survey papers being carted off to Delhi for reporting.)
This data would then be integrated with other data, often in expensive geographic information systems (GIS) which require advance training and skills. The GIS system would output maps which would be embedded in the report documents which would be sent by email to the funding organisation. Which would print it, read it and archive it. The information would be anything up to 18 months old when it reached the funder.
So we see that there is quite a lot we can do to improve on results reporting, which we at Akvo are working on with several of our products. But what isn’t often discussed is how to really tie together results reporting with what IATI was originally conceived to resolve: more transparent funding flows. So what would that look like?
End-to-end transparency overviewAn end-to-end transparency system interlinks information from many organisations who are involved in the work with international development cooperation.
Funding organisations, which could be international development agencies such as the Swedish International Development Agency, or a UN organisation, send funds (A) to intermediary organisations, who are tasked to find programs and projects to fund. The intermediary could be an organisation such as the Liberian government, UNICEF, or Flying Doctors AMREF Netherlands. The intermediary passes the funds (B) onto an implementing organisation, such as AMREF Kenya or EQUIP Liberia. (This is simplified a bit. I have seen funding chains seven levels deep. Which is problematic, as each organisation takes a cut from the funding stream to finance its operation.)
Figure 1: End-to-end transparency system overview
The funding and intermediary organisations publish IATI data about the programs and projects, and the disbursements of funds (C). The intermediary organisations may publish IATI data directly, but may also publish program and project data to a project database (D), which then publishes IATI data (G). The intermediary and/or the implementing organisations enter detailed project data and updates about the progress of the project in a project database (D), information and data which is then also published automatically to IATI data.
The IATI data which is published via the IATI Registry and different IATI data aggregators is linked automatically via activity identifiers (H) to project databases.
The intermediary organisations or the implementation organisations, sometimes in collaboration, defines the metrics which will show if a project or programme is successful and create monitoring protocols to measure the impact of the work done by the implementing organisations. Surveys and monitoring programmes are defined and data is then collected (E).
Projects and programmes in the project databases are linked to relevant survey data and monitoring databases (J).
Other data may be produced as well, indirectly related to the projects, or acquired from third parties (F). This could be maps, population data, hydrology data and much more.
All the data repositories: IATI data, project and programme data, survey and monitoring data and other data, are made available to data warehouses (K). The data in the data warehouse is accessible from dashboards and visualisation systems where graphs, maps and other visualisations are created to understand the progress of the work and the impact (L).
Several of the components in figure 1 have been or are planned to have an implementation by Akvo. The IATI aggregator and database is Akvo Openaid. The project database is Akvo RSR. The survey and monitoring database is Akvo FLOW, and finally the data warehouse and the dashboards plus visualisations will be what right now is dubbed “Akvo DASH”, which we are designing at the moment.
How IATI works today – Publishing, aggregation and republishingIn early 2014 more than 220 organisations have published IATI data. These organisations are mostly funding organisations, typically governments, multi-lateral organisations or large NGOs, who have published their spending commitments and disbursements. The published data sets consist of 3300 datasets (XML sources) describing 670,000+ activities. An activity is often a program or project and can vary a lot in size or scope.
The organisations publish this data in IATI XML format (1) and report to the IATI Registry (2) where one can find the published XML data files online.
Figure 2: IATI XML data publishing process
These published data files many have many different uses, but here we’ll use an example of an IATI data aggregator. An IATI data aggregator would typically interrogate the IATI registry (3) to understand what IATI data is available and where it is. It would then fetch this data (4) and incorporate it on a regular schedule into the aggregators own data set (5). The aggregated data set is made available both through a number of tools (6), including a web interface, where a user can search and read the information, and through an Application Programming Interface (API), which can be used to programatically extract data from the data set via other systems. Figure 3: Akvo Openaid Search screenshot
Today our partner Zimmerman & Zimmerman has an aggregator in place with the Open Aid Search site, which is based on the Akvo Openaid software / OIPA (Openaid IATI Parser and API) framework. Other examples are The IATI explorer by Development Gateway and AidView by aidinfolabs. Figure 4: IATI aggregator publishing system
An aggregator system can have many different features that a user can make use of, such as: search, navigation via hierarchies, data export, an API and more.
Full participation in end-to-end transparencyFor an end-to-end transparency system to be truly useful we believe that many more, if not all, organisations involved in the process need to be engaged in the publishing of data and information. For the larger organisations, such as some governments and multilateral organisations this can potentially be achieved through custom software development that integrates with their internal project management and monitoring & evaluation systems. However, the vast majority of organisations involved are often small and/or with a limited capacity to adhere to technical standards such as the IATI XML data standard.
The result is that much of the activity data published according to the IATI standard today is very high level data with little visibility of the actual implementation of the work. A typical example could be a program of water and sanitation improvements in several countries with a budget of many millions with many dozens of partners involved, which is represented by one single IATI data entry.
Figure 5: Openaid.nl screenshot
To illustrate this we will use the Dutch WASH Alliance as an example. Openaid.nl is another Akvo Openaid based system. The IATI data entry on Openaid.nl shows the Dutch Wash Alliance program as one entry with Simavi as the main partner, with €50 million in budget. This doesn’t really show the breadth and the scope of the program, which spans 74 projects with 88 partners.
We believe that this problem can easiest be solved by having online systems that make it easy to publish IATI XML as part of the normal processes that these organisations work with.
Figure 6: IATI data about the Dutch WASH Alliance program, on Openaid.nl
Integrating high level IATI data with progress and results dataWe have together with our partner Zimmerman & Zimmerman created a basic integration between the the Akvo Openaid IATI aggregator and another Akvo system, Akvo Really Simple Reporting (Akvo RSR). The aggregator links IATI activities with a particular activity identifier to the Akvo RSR system, which then via the RSR API fetches detailed project data and presents this in conjunction with the IATI data (14).
Figure 7: Projects visualised as part of an IATI activity in the Openaid.nl website. These are integrated between the systems and the integration uses the IATI activity identifier to fetch the relevant information from Akvo RSR’s API.
Each project that is part of the Dutch WASH Alliance programme is presented with information and a link to a detailed project page in the Akvo RSR system. This way one can drill down to the next level of information and data about where the money has gone and what the results are.
Figure 8: The Dutch WASH Alliance web site in Akvo RSR. Note that the project information is the same as in Openaid.nl above. The link on the project title (LEauCAL-WASH) in Openaid.nl leads to this project.
Figure 9: On the budget page for a project in Akvo RSR one can see the relevant IATI activity identifier (under the grant value from ICCO).
We have been working on the Akvo RSR system for a number of years now. Akvo RSR is a web and mobile phone based system that makes it easy for international development teams to bring complex networks of projects online and instantly share progress with everyone involved and interested. Briefly one could describe Akvo RSR as a project content management system (CMS). By March 2014, Akvo RSR contained over 1600 projects, collaborated on by over 1800 partners, with a total budget of around €800 million.
The Dutch WASH Alliance program in the above example is published in Akvo RSR with breakdown of all the projects, giving a much better overview and deeper understanding of who participates, what the goals are and what the progress is.
Making Akvo RSR IATI compatibleRecently we have been working together with the organisations Cordaid, RAIN Foundation and others to make it possible to load projects into Akvo RSR using IATI XML via an API. During the autumn of 2013 Cordaid loaded or updated 500+ projects through this path into Akvo RSR. We are continuing this work, with the goal of making this IATI XML data import feature something any organisation attached to Akvo RSR can use.
During the spring of 2014 we hope to also enable the publishing of IATI XML data for all the 1600+ projects in Akvo RSR (15). This means that even organisations which use simple web forms to fill in the project data into RSR will automatically publish IATI data as well.
Figure 10: Cordaid project website publishes data from the Akvo RSR system using the Akvo RSR API. This data originated from Cordaid’s project management system’s IATI XML export.
The general process we envisage for a project content management system (CMS) like Akvo RSR is as follows. Figure 11: Project content management system with IATI XML publishing
If your organisation has the technical capability or the systems available to publish IATI XML from your project management systems then you would export the data to the project CMS this way (7). Just like Cordaid has done for publishing projects in RSR.
If you don’t have the scale or capacity required to have an automated system you can manually enter the project data into the web forms system which the CMS probably has for the maintenance and upkeep of the data (9). In this case you may want to enter the transaction data about disbursements from your financial system via some type of simple data file, such as a comma separate values file (CSV file) (8), where the transaction data would be attached to the project data by the project CMS. The project CMS would then automatically make available the IATI XML data via its API (1) and if we also register that this data is available with the IATI registry (2) then we are back at Figure 2 in this document, where aggregators now can use this information.
Other functions of a project CMSWe believe that to provide good information about projects and programs a project CMS would ideally provide a number of features. This would include web pages that describe the project with: budgets, organisations, locations, goals, descriptions, collaborating partners, among other things. All these could be present in the IATI data for a particular activity. Beyond this there should probably be information about the progress of a particular project as well as the results of the completed project.
In Akvo RSR we have the beginnings of a progress reporting function, called Updates. It is currently informal and the structure and content of the updates are decided by the organisation entering the information. However, we have discussions with our partners about making this a more structured way of reporting progress. And we believe this will integrate well with the discussions about bringing progress reporting into the IATI standard.
Figure 12: UNDP updates on the progress of an energy efficiency project in Uzbekistan.
With Akvo RSR we have also provided a number of features which make it possible to follow the projects in more active ways, such as easy to setup web pages (10) which show projects for a particular organisation, widgets such as maps that can be published on other web sites, RSS feeds to allow you to follow Updates to a project without having to visit the individual web page, tools for donations to projects that need money, social media integration and more. Figure 13: Other features in an IATI capable project CMS system
Akvo RSR also has an API where information about a project or an organisation in the system can be retrieved (11), as used by Cordaid for example, as seen in figure 4, and republished on their own website (12).
The overview below has two information flows (1)/(15) and (2/13) which are actually equivalent. They are just separated to make it easier to understand when considering the Project CMS as an “externally maintained system” to an organisation (like Akvo RSR is). I also added an explicit box for mobile phone and tablet apps (16), as we believe that this is going to be of ever increasing importance. ￼ Figure 14: An overview of the IATI system with the IATI Registry, IATI aggregator and database, Project CMS system and how the information flows hang together.
Integrating results information and data into End-to-end transparencyIf all the programs and projects are entered into a project CMS, like Akvo RSR, with interlinking data, such as the originating IATI activity identifier for a particular set of funds, then using an IATI aggregator we can see where the money is being spent, with whom and for what purpose, in a detailed way.
But we would like to know more. What is the result of the program? Did it actually make a difference?
How we are improving on results reporting
To improve on this Akvo has a data collection system called Akvo FLOW, which was originated by Water for People. Akvo FLOW is a mobile-phone and web based system for field surveys.
Akvo FLOW is today being used in 28 countries by 70 organisations to perform baseline surveys and to measure results. The data which is being collected on a mobile phone is uploaded and reviewed by the team to an online data store, sometimes the minute the survey interview is concluded but otherwise often the same day. This means the surveys are done faster, with less data errors and the ability to review the data as it comes in, in near-real time. Which means that errors or problems can be corrected quickly and one can use the information to steer the effort rather than just measuring it. We are currently in the process of understanding how best to integrate this results reporting data into the overall End-to-end transparency framework.
The Liberian government used Akvo FLOW for a baseline survey in 2011, supported by the World Bank’s Water and Sanitation Project and UNICEF. This data served as the basis for the Liberian Water Atlas, which was published during the autumn of 2011 and serves as a policy document for improving the rural drinking water infrastructure.
Figure 15: Akvo FLOW data in map from the Liberian Water Atlas.
Liberia has gone one step further and are now tracking all the water, sanitation and hygiene (WASH) projects in Liberia, involving about 30 organisations, government departments, NGOs and funding partners, in an Akvo RSR based project site.
Figure 16: The Akvo Sites based WASH Liberia website, managed by the National Water Sanitation and Hygiene Promotion Committee. The projects tab uses Akvo RSR Pages.
On the WASH Liberia web site, results data from the field surveys are visible in a way humans can interact with, but still in electronic document format.
We are now looking at how we can integrate this data with the IATI data and project data in Akvo RSR in a sensible way. We expect to do this through a new service we are working on for data warehousing / data mashups and dashboards and data visualisations, which has a working name of Akvo DASH. The resulting system will be able to operate like figure 1 at the beginning of this blog/paper.
Closing thoughtsAt Akvo the goal is to create a really useful set of interlinking services, each of which can be used standalone, or as part of a larger data and information ecosystem. We think it is very important to have a well thought through and comprehensive solution in place that is easy to use and implement.
I believe that we are one of the few organisations who are looking at providing a full end-to-end system, but it is important to understand that we really want to collaborate with others in creating an ecosystem of services which can interchange data with each other. The Akvo system isn’t meant to be the only solution, but meant to be a practical and usable way to get us well on the way towards something useful for all of us. In the end, we at Akvo, working as a non-profit foundation, are here to help. It is less important to us whether you use an Akvo system to create better information and data systems than that you do create good systems. Preferably systems which can be shared with others, systems which interchange data with others in open and responsible ways. Systems which we can collaborate on so that we don’t reinvent the wheel all the time in every organisation.
Thomas Bjelkeman-Pettersson is the CTO for Akvo, based in Stockholm.