Measurement and more in DR-Congo

[Gaston]

Kinshasa. Dusty as usual, but there are some clear stars shining through the dust. It has been more than 15 months since my last visit. There are flatscreen TVs now in the airport. I feel both a little nervous and excited. The friends of RDCCompetence come and pick me up with their usual hospitality. Already on the way to my Christian guesthouse, I hear some of the impressive progress that is happening over the last months. I feel the excitement takes over and have troubles sleeping under my blue mosquito net that night. I am too eager to start working with this outstanding team. 

For 8 days, I worked with the national facilitation team. Main objectives were 1) to provide technical support to the three UNICEF projects that are currently running, 2) provide institutional support to the secretariat of RDCCompetence and 3) develop a strategy for M&E that meets all needs of the partners involved (e.g. Constellation, RDCCompetence, UNICEF and the communities).

Let me focus here on the last point. Why do we measure progress and how can we accurately measure the progress we make at various levels during a CLCP-related project? For us, we identified four major interests:

1. In line with our philosophy, we’d like communities to take ownership of their health as well as measuring the progress of their health;
2. We as facilitators would like to learn and improve and need to measure the results of our own facilitation and the delivery of this;
3. UNICEF has its (global) frameworks of measurement and needs to be accountable for the funds it receives.

How to merge the above three interests in one measurement system of a project?

In short, we proposed the following to UNICEF:

-          We, the Constellation/ RDCCompetence measure the results on our level. This includes indicators such as:

• Number of communities that developed their action plan;
• Number of facilitators trained that apply the SALT approach;
• Number of support teams that increased their capacity to manage community mobilisation efforts;
• Number of concertations (multi-disciplinary community groups) that improve their action based on exchanges with other concertations.
• Etc.

-          The communities are invited to measure for each of their 3 priority practices (that they select themselves from the self-assessment framework):

• 1-2 indicators per practice. They don’t have to be 100% SMART and great. The essence is that the community takes ownership of the measurement progress of their own work, improves, celebrates progress and can share this with others.
• Their own baseline data and desired levels;
• Using their own means of verification.

-          UNICEF in this case conducts a study of 3 questionnaires for health zones, families and mothers at the beginning and end of the project. UNICEF shares the outcomes with the community for two reasons: 1) To stimulate an even deeper discussion at the next self-assessment exercise and 2) To invite the community to improve their action and measurement plan given these external data.

For us, we felt that this struck a good balance, reaps synergies and respects needs and values at various levels. And it simplifies the M&E process to the extent possible.

Attached slideshow gives more depth to these ideas. 

Your suggestions are most welcome. Oh yes, and below for inspiration. RDCCompetence's dream for 2030!

 

 120819%20Measure%20progress%20small.pptx

[Rituu B. Nanda]

Hi Gaston,

One aspect we can incorporate in our CLCP process is baseline and endline evaluation. I say this as I have tried reporting to donors through self assessment framework but it has had its limitations. For instance by the time community facilitators understand SALT and then self assessment it is already 4-5 months and that does not gives the true baseline for comparison. Secondly, we see that over time communities mature in their response and tend to rate themselves at level lower in the self assessment which donors at times find hard to understand. Thirdly, the funding agency would like a more objective evaluation process.

 

Listing outcome indicators is not an easy task. Another thing we can brainstorm on qualitative indicators. for instance the indicator you mention- Number of support teams that increased their capacity to manage community mobilisation efforts, we need to define what we mean by capacity and community mobilisation. Also another place you mention practice SALT- what that means.

 

Just sharing from my limited experience. I highly value your posting and have read it several timees. Thank you.

[Rituu B. Nanda]

Hi Gaston, I facilitate India Civil Society Group . Here is the feedback:

Anand Chaudhuri excellent synergy, keep it simple...

• Katya Burns Community level M&E is extremely important. I have found that empowering community organisations to collect data can work well, and that it is then very important to follow up with building the skills to analyse the data at the community level and then feed the learnings from that process back into programs to continuously improve and update them.

  4 hours ago ·
• 

  Rituu At Constellation Thanks Anand Chaudhuri. thanks Katya Burns! I like what you say Katya, something like action research. That's what the self assessment tool of Community life competence process aims to do.
• 

  Rituu At Constellation I will add you responses to the blog.

   
• 

  Anand Chaudhuri ‎Rituu At Constellation, its all there...the action research you talk of is inbuilt into the current M&E process and as one progresses, gets enhanced (community life competency and depth of M&E in terms of number of indicators) by interlinkage with UNICEF indicators and feedback loop....

  about an hour ago ·

[Autry Haynes]

Dear Gaston, great work (^_^). My first thoughts are to have a community conversation i.e between the four stakeholders to dream about an effective and sustainable M&E for ALL. Having this one dream encapsulates responsibilities of each.The conversation continues with "what do we need to do to achieve our dream of an effective M&E?" This question with candid discussion will likely result in identifying a number of SMART (S: specific; M: measurable; A: Appropriate; R: realistic; T: time bound) ACTIONs Once the ACTIONS are finalized , we decide who of the stakeholders does what, when, how and with what indicators. I like the 2030 dream!

[John Piermont Montilla]

Upon reading the initiatives above, i remembered this quote:

The only man I know who behaves sensibly is my tailor; he takes my measurements anew each time he sees me. The rest go on with their old measurements and expect me to fit them. ~~~ George Bernard Shaw

I have the following questions:

1. the development of indicators. Who develops the indicators to be measured? what is the basis of these indicators (CLC practices, MDGs, national Goals, UNICEF goalsd etc.) where does the community comes in? 
2. the baseline and the benchmarks (current status and desired levels) who sets them?
3. there are four key players, Constellation, RDCCompetence, UNICEF and the communities, I wish to know what each key players take away each time they measure they want to measure.

I wish to see M&E people as one Tailor in the quote above that is built-in in the communities where communities themseves measure their own otcomes and the data can be harvested and interpreted in line with other institution's indicators and  not just many tailors measuring many dimensions of the community that are not unified into one M& E framework. Almost all M & Es I know are too programmatic and scattered, new project new M & E framework after funding communities does not anymore measure their own change and be accountable to their own change.