Participatory M&E
IICD’s approach to Monitoring & Evaluation (M&E) in ICT-enabled development has always been guided by the principle of local ownership: our approaches, systems and tools aimed to assist local partner organisations to work with M&E data to learn from their experiences and improve their practice. IICD supported its local partners to monitor and evaluate the social, economic and organisational impact of their ICT-related initiatives and measure how their activities have empowered end-users to achieve their own goals. The approaches were designed to help local actors to understand and drive the change processes, as well as build an experience base on which to build further ICT-enabled initiatives in their sector.
For example, once foundational capacities were developed to build and implement technical solutions, the actual process of deploying and measuring the usefulness of solutions was monitored by collecting user and beneficiary feedback on an ongoing basis, and discussing the results in Focus Groups with the direct users and beneficiaries of the projects. Impact data was analysed, areas of concern were identified and discussed, and solutions to challenges in implementation were devised – all with the implementing organisation, users and beneficiaries together, revealing experiences and lessons with a high degree of relevance to a country’s national context.
While M&E was mainly used as a tool to measure results and provide accountability in the late 1990s, IICD pioneered the focus on M&E for the purpose of learning about ICT integration in development processes. The method of data collection was inspired by early approaches to commercial consumer research, since as in a commercial business, a good way to monitor progress in satisfaction and impact is by asking the target group, the end-users of projects, such as farmers and teachers, rather than conducting ex-post evaluations only.
A key aspect of IICD’s approach to M&E for learning was separating the learning process (informed by M&E data) from financial control and accountability, an approach that helped to overcome reluctance to openly discuss difficulties and challenges. Another key aspect was the use of the system to monitor satisfaction with the services and support provided by IICD to our partner organisations, as well as the services and support provided by our partner organisations to the user and beneficiary groups. These elements of auto-evaluation and downward accountability ensured that M&E not only informed improvements in IICD’s programme management or impact reporting to donors, but benefitted end-users too.
In order to measure impact at the level of these end-users and beneficiaries, IICD’s M&E system looked into five key indicators: awareness, empowerment, economic impact, negative impact and organisational impact, which taken together, gave a good impression of a project’s impact on intended users in education, health, economic development and governance.
IICD partnered with independent local M&E experts and organisations to directly assist implementing organisations with the collection of quantitative data, analysis of the data and presenting the findings in reports and presentations at focus group discussions. In these focus groups, partners and end users jointly reflected on the analysed data: on trends between different years, comparisons between different projects and different groups of users, leading to lively discussion and reflection. Quantitative data is useful, but the ‘why’ behind the data is what’s most important, and the M&E focus group meetings were critical instruments in knowledge sharing and exchange of experience among all the implementing parties.
Basing progress monitoring and understanding impact on such social learning processes not only resulted in the continuous adaptation and enhancement of individual organisations’ ICT solutions, but also resulted in an improved appreciation of which approaches and solutions work best to address a particular sector’s real needs. Where possible, focus group meetings were open to local change agents and decision-makers to attend, providing invaluable practice-based experiences to relevant policy makers and governing institutions on how ICT can best be employed and deployed to enhance sector governance and development.
