Opening up MEL technology
Technology for monitoring, evaluation and learning (MEL): we need to address complexity, interoperability and affordability.
I've been working for ten years at the intersection of monitoring, evaluation and learning (MEL) and technology for development.
I've seen three problems over and again. I want to share some ideas about to how we might address these problems and pose some questions.
When I talk about MEL technology, I mean the tools we use to meet the following needs:
- we want to plan interventions and monitor progress;
- we want to collect data and store it in a secure and responsible way;
- we want to explore that data and see what we can learn;
- and sometimes we want to publish data or report our learning.
First problem. Put yourself in the shoes of someone at an NGO or development agency who is responsible for dealing with MEL technology – it is a very complex space to navigate. How do you know which tools you can use? How do you evaluate them? How do you know when you should create your own? And how do you do that?
Faced with this complexity, I've seen an organisation go through a 12 month evaluation and procurement process to choose an MEL tool – only to end up with inconclusive results. I've seen a leader at another organisation develop a custom-built MEL tool over several years, at great expense, then leave the organisation before it was adopted – leaving the tool without a champion.
My question is: how can we make it easier to navigate the complex space of MEL technology and find appropriate tools?
We facilitated an open-space meeting for the Bond MEL and Tech4Dev groups last Tuesday and one potential solution we agreed to experiment with was to create a map of MEL tech tools that can be used at each stage of the planning cycle. This should be up online in the next couple of weeks, maybe we could develop this into a kind of Trip Advisor for MEL tech, we'll see.
Second problem. By hook or by crook, you've managed to embed your MEL tool in your organisation. But now you discover your sister organisation, with whom you want to share data, is using a different tool for the same task, or an edict has been issued from head office that everyone must now start using the Acme MEL tool instead, or the company who made your tool has stopped supporting it. Or maybe you just have one tool for data collection and another for reporting and visualisation and it's a real headache to move data from one to the other.
Your problem is that you need to move your data, or exchange it with another tool, but this is not easily done because they codify your data in entirely different ways – the tools are not interoperable – they cannot interpret the same data.
We wrestled with Nigerian agricultural survey data, trying to get this into a usable format for a monitoring dashboard, and we've found tools like OpenRefine very useful in what has been a very time-consuming, laborious process. In other projects we've developed Application Programming Interfaces (APIs) to enable different databases to share resources. It's all work for us so I guess I shouldn't complain but I can't shake the feeling that non-technical people should be able to move data between tools without paying for technical help. There must be a better way!
My question is: how can we enable interoperability between tools and promote data sharing capabilities?
One idea is that we could adopt an open standard for MEL data. It should be possible to codify MEL frameworks, indicators and monitoring data in a way that can be commonly interpreted by different tools. This could save a lot of time.
It may be that such a standard already exists. Speaking to a colleague who recently developed the open contracting standard with the World Bank, she reminded me that a standard is only as good as its adoption. So if there is already an open standard for MEL data (and if there is, please tell me about it) then let's deal with the adoption problem.
Third and final problem. MEL technology can be very expensive and therefore out of reach for many poorer organisations.
Linda Raftree and Michael Bamberger's report on tech-enabled MEL talked about the fact that proprietary management information systems often have high costs for training, installation and long-term service agreements. And developing a system in-house can require a large investment.
Why not use open-source tools you might say? The same report argued that many organisations don't have the skills to set up and maintain open-source tools themselves and that, in terms of management information systems at least, open-source tools lack functionality and usability.
This chimes with my experience, so my final question is: how can we make MEL technology attainable and sustainable?
Personally, I think there is so much commonality in the needs for MEL technology between organisations that it should be possible to pool resources to build and maintain a set of excellent open-source tools that are easy enough to setup and maintain for poorer organisations or that can be offered as low-cost hosted services.
At Aptivate we want to this with our Kashana project. The name is derived from the amharic word käshänä, meaning 'do good work'. I like the name because I don't think you work in international development because you want to run complex IT projects or spend days collating spreadsheets of data for a report – you do it because you want to do good work, in every sense of the term.
We are developing a pilot programme for Kashana so that we can test its viability and get user feedback. We are looking for NGOs or development programmes who use a structured MEL framework, such as a log frame. Would you like to be involved? If so, please email kashana (at) aptivate.org