Presentation at the Evaluation Conclave 2017, Thimphu, Bhutan

[Santanu Pramanik]

It was a memorable experience to visit Bhutan and talk about the novel community engagement approach SALT, related evaluation challenges and the lessons learnt from the field. I would like to thank 3ie for the opportunity. In case you are interested, the slides are given in the link below:

Santanu_Pramanik_EC2017_3iePanel.pdf

Photo credit: Durgadas Menon, 3ie

[Marie Lamboray]

Hi Santanu,

Thank you for sharing the comments/questions. I liked questions 2. I am convinced that there will be contamination/spillover effect/transfer. "AIDSCompetence spread faster than the virus." :) 3km is a very short distance when someone wants to share something he or she is passionate about. At the end of the project, will PHFI ask residents of control villages if they know/use SALT?

Please help me understand this: I know that PHFI did a baseline for the impact evaluation. What is the difference between baseline and initial situation as mentioned in 1. below?

[Rituu B. Nanda]

Santanu, in my implementation and evaluation experience, I have observed that in many cases community has lost the belief in its own capacity we outsiders try to provide solutions as experts to so called marginalized, poor, illiterate people. You have read Marlou's blog where the Kamrup community asked her if she will help them fulfill their dream.

What I find  even more challenging- outsiders do not believe that communities can do things for themselves. 

Thus, in above scenarios a strength-based approach like SALT makes people realize their own strengths as well as strengths of others. Mentioned Tutumani ( on 8th June during SALT visit) our coordinator from Udalguri "I have begun to see my own strengths. At the same time I have also begun to see the strengths in the community and this has improved our relationship." 

[Santanu Pramanik]

Thanks, Rituu and Bobby. By negative opinion regarding community, I mean, in some households we encountered members who do not believe in the role of community in improving their children's health.

[Santanu Pramanik]

The following comments/questions were discussed after the presentations (last 30 min of the session): 

1. One person (seems to be an active member of Community of Evaluators, South Asia- presented mementos to us) feels that this type of intervention is likely to work. He was asking for evidence- I mentioned that this is an ongoing project, we don’t have evidence yet. However, there are some evidences of SALT reducing vaccine hesitancy in DRC. He also asked whether we did any situation analysis before implementing the intervention. I had to confess that we did not. However, districts are selected in such a way that we have districts having varied socio-demographic characteristics (Udalguri being tribal-dominated, Bongaigaon has higher percentage of Muslim population- both known to have poor immunization coverage; whereas Kamrup rural is better off). The goal was to see how SALT works under various context. 
2. One woman from  Azim Premji Philanthropic Initiatives asked couple of questions. In the context of minimizing intervention-control contamination, she asked (my responses in italics)

a)      Why didn’t we randomize at a higher geographical level?

Randomizing higher geographical areas (such as blocks) to intervention and control groups would definitely reduce the chance of contamination, however, it may not guarantee balance between intervention and control group at the individual level (HH and mother/child- which is our unit of analysis) as there exists larger variation among units within a block than that of within a village

 

b)      Instead of minimizing the contamination why didn’t we measure the spillover effect which is a good thing in the context of scaling-up of the intervention?

Since this is one of the first randomized evaluations of SALT, our primary objective was to identify the impact of SALT, if any. For that purpose, our goal was to keep the control group as pure as possible. Measuring the spillover could be our next objective.

3. Another woman (from 3ie, I guess) asked- in the context of evaluation team’s involvement, independent and unbiased feedback provided to the implementation team based on our participation and observation- so at the end what we are evaluating is not SALT, it’s “SALT plus something”. I had to agree that ideally the implementation should happen in such a way as if the evaluation were not in place. We were quite hesitant at the beginning regarding our involvement. However, for the purpose of near-optimal implementation of SALT in the field, we thought of sharing our feedback with the implementation team. Otherwise, we may end up implementing something which is not SALT but “SALT minus something”.

[Bobby Zachariah]

Great presentation Santanu!

How did  the group respond to this presentation?  What critique and suggestions did you receive?

It is indeed an useful summary.

[Rituu B. Nanda]

Thanks Santanu for your presentation. Very clear articulation. Please would you elaborate on the point- negative opinion on the community.