Winners and Runners-Up of the 2019 Award Competition for Evaluation for Transformational Change announced.
The Award Steering Committee has the pleasure to congratulate the winners and runners-up of the 2019 Award Competition with their achievement! The prize winners will receive their prize in Prague on Wednesday 2 October, during a ceremony taking place at the Residence of the Mayor of Prague in the evening. This event is part of the Global Assembly of IDEAS and the Third International Conference and will be open to all participants. Representatives of all three prize winners will be present in Prague.
At a later stage the evaluations will also be made available on the website of the Award, so stay tuned for further news! The considerations of the Judging Panel follow below.
Considerations of the Judging Panel
26 September 2019
The Judging Panel has reviewed all eligible nominations and has identified three winners, one in each category. The considerations of the Judging Panel are reflected below.
Runners-up to the prize winners were identified as well, but the Judging Panel did not find sufficient grounds to identify in which category they should be placed as runners-up, and felt that while second and third placed nominations were expected to appear in the three categories, they felt that five nominations should be recognized as “runner-up” for the first prize, without specifying in what place or for what category. The Judging Panel will reflect on this further and interact with the Steering Committee on the implications this may have for the next Award Competition that will take place in 2021.
Award for credibility
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia
Team leader: Ashish Mukherjee
This evaluation had great strengths in the main elements of credibility – a rigorous methodology combined with impartiality and independence. The methodology was well-documented, matched the characteristics of the programme whilst also taking into account of what had been learned from other related programmes. The Theory of Change, evaluation questions and sampling strategy were convincing and appropriate. Data was collected from a full range of stakeholders including beneficiaries, counterparts and donors; and the resulting evidence was carefully and transparently weighed up. At the same time the evaluators maintained appropriate close-working relationships with the evaluation Commissioners UNICEF; and governance arrangements gave confidence that the overall exercise continued to be independent. The management response and follow-up actions confirmed that the evaluation was taken seriously by Commissioners.
Award for innovation
Mid Term Evaluation of the Global Climate Partnership Foundation
Team leader: Antonia Dickman
This evaluation was innovative in a number of ways. The evaluation recognised the scope and scale of the transformation challenge of climate change and CO2 reduction – something that cannot always be taken for granted. The evaluation also acknowledged that timing is critical in evaluations of transformational policies e.g. that for sustainability, we cannot rely on a one-off ‘snapshot’ judgement but also need to invest in ongoing monitoring. A number of established and ‘frontier’ evaluation methods were deployed that as far as the judges are aware have not been used before in a transformational context. Whilst approaches such as Theory Based evaluation, Contribution Analysis and QCA are beginning to feature in mainstream evaluations, how these approaches have been combined in this global policy evaluation is new. Efforts to clarify causal assumptions and links between micro and meso actions and macro-policy goals are also consistent with state-of-the-art evaluation practice.
Award for influence
Evaluation of Transformational Change in the Climate Investment Funds
Team leader: Sam McPherson
This evaluation is an authoritative and high quality evaluation, using an interesting theory-based evaluation methodology testing hypotheses with a mixed-method theory-based approach using contribution analysis and comparison across cases, exploring the mechanisms how and under what circumstances transformational change takes place and identifying the role that CIF has played in these changes. It was undertaken in parallel with an evidence synthesis on transformational change in CIF. The evaluation was of a very large-scale programme ($8 billion) and found the CIF had positively contributed to transformational change in the climate sector over the last 10 years. The launch of the evaluation was in itself very significant with a very impressive panel during the CIF conference in Ouarzazate, Morocco in January 2019, including ministers from Morocco, Niger and Zambia, and high-level representatives from the World Bank and the UK government, and the evaluation appears to have played a significant role in safeguarding the future of the Fund.
Runners-up in alphabetical order
- Evaluation of the early impacts of the better cotton initiative on smallholder cotton producers in Kurnool District, India
Team leader: Ravinder Kumar
This evaluation stood out for aptly combining a rigorous and sophisticated use of mixed methods with a proactive orientation towards learning, usability and effective communication of findings. The result was a rigorous, credible and pertinent report.
- Evaluation of the Ma’An (Together) towards a Safe School Environment Programme 2009-2016 in Jordan
Team leader: Nadeem Haider
This evaluation was thoroughly researched and is regarded as very influential by stakeholders. It included participatory work with children and specific work on using quantitative data. It was culturally sensitive and balanced by giving special consideration to negative feedback. The result is a well-grounded report that offers pertinent and actionable recommendations following best/good practice.
- Learning from evidence. Lessons and pathways to an inclusive agricultural market for smallholder farmers in Mexico
Team leader: Gabriela Pérez-Yarahuán
This was an interesting evaluation, going beyond the usual evaluation tools to do a proper root cause analysis, undertaking a wider diagnostic of the causes of the problem the programme intended to address, comparing what the programme was doing against that, and then looking at international experience that could be relevant to address the gaps. It took a systemic approach, including how the system could consider climate change and gender. It mostly engaged with the client (Walmart) rather than wider policymakers.
- A multi-dimensional outcome harvest. EU-SRSP’s programme for economic advancement and community empowerment (EU-PEACE) evaluation in Pakistan
Team leader: Jeph Mathias
This evaluation was interesting for its innovative use of outcome harvesting on a multidimensional intervention aimed at systemic change, and for the way remote work and local team was handled, bearing in mind the security challenges. It is an eloquent example of the use of adaptive evaluation processes and tools with an inclusive and transformative ethos.
- Outcome evaluation of the education capacity development partnership fund (CDPF) in Cambodia
Team leader: Frans van Gerwen
This outcome evaluation undertaken by Lattanzio Advisory SpA. and commissioned by UNICEF showed many examples of good evaluation practice. It deployed local field workers; relied on national structures; engaged widely with stakeholders and paid attention to scaling-up and Cambodian government policy. The Theory of Change was credible as were the evaluation questions posed. Data collection was well-planned and well-documented.