Terms of Reference for the Final Evaluation of “Improving access to WASH services and socio-economic well-being of conflict-affected women, men, girls and boys in Iraq”

1. BACKGROUND:

Oxfam is a global movement of people working together to end the injustice of poverty. Together we save, protect and rebuild lives. And we won’t stop until every person on the planet can defeat poverty or any type of inequality or discrimination. We are an international confederation of 20 organisations (affiliates) working together with partners and local communities in the areas of humanitarian, development and campaigning, in more than 90 countries. All our work is led by three core values: Empowerment, Accountability, Inclusiveness.

Iraq’s post-conflict context is characterized by fragility, sectarian divisions and climatic shocks. The 2019 Iraqi Humanitarian Response Plan estimated that 6.7 million people remain in need of some form of humanitarian assistance. While more than 4.3 million people have returned to their communities, approximately 1.6 million remain displaced. Complicating the humanitarian context further, a recent military operation ‘Will of Victory’ by the Iraqi government, launched to clear territory from ISIS groups, has exacerbated the already fragile security scenario, limiting the population’s access to basic public services and humanitarian assistance.

Regarding Water Sanitation and Hygiene (WASH): Since 2016, most of the humanitarian assistance in Iraq has been channeled to large urban areas leaving significant WASH gaps in rural communities. Because rural communities base their livelihood on agriculture businesses, they are more vulnerable than the population residing in urban areas, where industries (refineries) and other activities such markets are present. Limited coping mechanisms and disrupted livelihood capacity, coupled with lack or limited access to health and water services, have created new urban migration patterns as well as new pockets of poverty and deprivation. While communities across Iraq are in need of all forms of support, WASH needs for host communities and returnees in communities in Kirkuk and Diyala governorates are particularly acute due to the severe damage the areas suffered under ISIS. Both Hawija and Khanakin districts were considered strategic locations to ISIS and were heavily damaged under their control. Damage to public WASH infrastructure has created major water access and water quality needs.

Regarding Emergency Food Security and Vulnerable Livelihoods (EFSVL): Out of the 6.7 million people in need in Iraq, an estimated 2.38 million individuals (37%) require livelihood support.  Challenges to secure livelihood opportunities are among the top three needs for both displaced people and those who have returned to their pre-conflict communities. IDPs particularly have difficulties accessing employment and livelihoods opportunities, thus preventing them from obtaining shelter, food, and essential non-food items. The average amount of debt per household is 2.2 million IQD (USD 1,800). Conflict-affected households in Kirkuk reported an average of 2,480,000 IQD of debt (2,066 USD). In Kirkuk and Diyala, there is limited access to food due to damaged infrastructure, logistical issues, security constraints, limited economic resources and disruption in the agricultural sector and the Public Distribution System (PDS).

3. PROJECT INTERVENTION AND TARGET GROUPS:

The project has the following goal, outcomes, outputs and key activities

Goal: ‘Improved access to WASH services and socio-economic well-being of conflict-affected women, men, girls and boys in Diyala and Kirkuk,’

Water Sanitation and Hygiene (WASH)

Outcome 1: Improved use of WASH facilities through safe and equitable access to clean water, sanitation facilities and hygiene promotion for vulnerable IDPs, returnees and host communities (women, men, girls, and boys).

Overall, a total of 25,000 people in Kirkuk, including children, youth, PWD and elderly benefit from Outcome1, which is calculated to be the number of people in the catchment area of the public buildings. Outcome 1 targets Hawija sub District and Abbaasy sub District. In Diyala, a total of 36,000 individuals benefit from Outcome 1 including children, youth, PWD and elderly in Jalawla sub-district and Sadiya sub-district.

Output 1.1:  Humanitarian output. The WASH hardware capacity of the services providers and communities is increased. Safe potable water is provided through the construction/ rehabilitation and extension of water networks and WASH facilities for public institutions (health and education facilities).

Activity 1.1.1: Rehabilitation and construction of water supply systems in targeted communities

Activity 1.1.2: Rehabilitation/construction of water and sanitation infrastructures in public buildings

Output 1.2: Resilience output. The WASH software management capacity of the service providers and communities is maintained in the future (gender-responsive WASH in programming and sustainability).

Activity 1.2.1: Mid-long-term capacity building of Community Committees and district/sub-district water authorities for mid-long-term management of WASH facilities

Activity 1.2.2: Development of eight basic Assets Management Plans (AMP) for water facilities and resources

Activity 1.2.3: Development of eight basic Monitoring and Alerting Platforms (MAP) for targeted communities and water services

Activity 1.2.4: Sustainable hygiene behaviour change^[1] capacity building and campaigns designed to bridge the national priority named Basic Health Service Package (BHSP) - by the MoH for HHs, health facilities (Primary Health Care - PHC - and Community Health Houses - CHH) and schools

Emergency Food Security and Vulnerable Livelihoods (EFSVL)

Outcome 2: Enhanced economic inclusion of conflict affected communities in Kirkuk and Diyala

Output 2.1: Increased population participation in community recovery efforts  

Activity 2.1.2. Development and training of community committees

Output 2.2:  Improved livelihoods restoration. public infrastructure is rehabilitated to facilitate service provision and create labour opportunities.

Activity 2.2.1: Asset replacement

Activity 2.2.2: Provision of Cash for Work for rehabilitation of community livelihood infrastructure  

Output 2.3: Improved vocational and business development capacities

Activity 2.3.1: Conduct vocational and skill training for youth and women groups

The project focuses on 1) meeting the basic needs of the most vulnerable communities, and 2) providing a basis for longer term interventions and sustainability (resilience). Oxfam has implemented the project in Khanaquin District (Diyala governorate) and Hawija district (Kirkuk).

In total, the project has targeted 36,000 beneficiaries in Khanaquin and 25,000 beneficiaries in Hawija. The project targeted conflict affected men, women, boys and girls in Khanaquin District (Diyala governorate) and Hawija district (Kirkuk) including IDPs, returnees and host communities. Specific vulnerability selection criteria were developed for the project during the inception phase and incorporated vulnerability criteria guidelines from WASH and livelihood clusters.

Oxfam, REACH, and FUAD work directly with the beneficiary communities throughout the project cycle in close coordination and involvement of local water authorities and other relevant stakeholders to ensure community ownership and sustainability. Women’s engagement and empowerment were the key areas of intervention in the project theory of change and its needs to be strongly supported.

4. OBJECTIVES OF THE FINAL EVALUATION

This SDC-funded project will end in June 30, 2021 and an external final evaluation is planned to take place on June and July 2021.

The overall objective of the SDC final evaluation is to  systematically review the overall progress of the project towards the project’s outcomes,  assess the learnings from project design and project implementation from our work with partners and communities, and provide recommendations on overall design of the project, modifications, and specific actions that might be taken into consideration in the future during the design of future projects of a similar nature. The scope for examination is determined using OECD-DAC criteria for evaluating humanitarian action. Relevant criteria are associated with a number of key questions (under section 5) that are to be addressed and explored.

Specific Objectives of the Evaluation:

• To assess the design, planning, delivery and management of the project by Oxfam and its partners in accordance with Oxfam Programme Standards and donor requirements.
• To identify and assess key internal and external factors that have contributed, affected, or impeded the achievements of the project, and how Oxfam and the partners have managed these factors.
• Review the project outcome indicators and update the final/endline values in the log frame
• To assess the beneficiary targeting, verification and selection processes/approaches of the project
• To draw key lessons and learning from the project and make recommendations that will help inform Oxfam’s formulation and design of future projects that will benefit the Iraqi displaced, host and returnee communities affected by the armed conflict.
• To assess the relevance, effectiveness, efficiency, impact and sustainability of the project on IDPs, returnees and remainees on Oxfam and its partners.
• Describe and assess efforts of stakeholders in support of the implementation of the project
• Assess the likelihood of continuation and sustainability of project outcomes and benefits after completion of the project
• Examine the extent to which the impact of the project has reached the intended beneficiaries

5. EVALUATION CRITERIA AND KEY QUESTIONS TO BE ADDRESSED IN THE EVALUATION:

The project evaluation should be organized taking into consideration of Oxfam’s key evaluation criteria, and program quality standards. The evaluation will also consider key sector and sub-sector indicators for assessing the performance. The following provides a guide to the questions to be addressed by this evaluation, under each of the criteria below:

Relevance and Appropriateness:

• Have Oxfam and its partners selected relevant operational areas for their work?
• Have Oxfam and its partners targeted the most vulnerable people, including vulnerable men, women, boys and girls?
• To what degree is the intervention addressing the WASH and livelihoods needs of the targeted women, men, boys and girls and contributing to reduced vulnerability?
• Was project design consistent with the Do No Harm principle? and with a Safe Programming approach?
• What was the level and quality of participation of the beneficiaries and the partners in project design?
• How responsive were project activities to the needs of targeted women, men, boys and girls in targeted communities (if possible, please distinguish between displaced/returnee and host communities) given their circumstances and priorities?
• To what extent have the key contextual changes, threats and opportunities that arose during implementation been influencing and informing project implementation?
• How and to what extent were monitoring and evaluation findings used to inform decision-making and the improvement of project implementation by Oxfam and partners?
• Was the project adapted appropriately to changing context and needs in a timely manner? What changes took place in the implementation of the project differed from the original design?

Efficiency:

• Was the project implemented based on the best use of existing resources/capacity, e.g. the capacity of the partners and the internal capacity and expertise of Oxfam itself? What key limitations existed on this front? What could Oxfam’s future projects do to increasingly develop and invest in existing resources?
• How cost-effective was the intervention? What cost-effective alternatives could have been used?
• Were the resources (financial, human, etc) for running all the activities available, adequate and was this the best use of resources to achieve the results?

Effectiveness:

• Has the project proceeded well towards the achievement of its outcomes?
• Is there anything about this project that has had any impact on the beneficiaries (e.g. predictability of aid, etc.)? (positive or negative)
• Has this project had any impact on the partners? (positive or negative)
• What were the main challenges of the project and how well have they been addressed? Is there anything in this project that made these challenges more/less manageable?
• How were beneficiaries selected and were they informed of the selection criteria?
• How effective were the selection criteria in reaching out to the most vulnerable populations?
• How inclusive and culturally sensitive were the activities carried out in terms of the approach, quality of participation, information and its dissemination?
• To what extent has gender been mainstreamed into the project? How could gender be better mainstreamed in a future project?
• What unintended consequences (if any), whether positive or negative, did the intervention have on women, men, boys and girls so far (IDPs/returnees/host communities)?
• What system and mechanism are in place (at Oxfam and partners) to ensure accountability to the beneficiaries and how well have they been working throughout the project?
• Have the ways of working with the partners been contributing to building the capacity of the partners?
• Were there any risks inherent to the duration of the project, either in the course of or towards the end/after the end of the intervention?
• What factors have contributed to achieving or not achieving intended outcomes and set objectives of the project? Identify any exceptional experiences/achievements of the project substantiated with evidences, such, but not limited to case studies, best practices, etc

Impact:

• What the positive and negative, intended and unintended, changes produced by the project?
• Have these changes contributed to any identified changes according to aspirations of project?
• Analyse the contribution of the project to any observed impact (intended, unintended, positive, negative) and analyse what other actors and factors contributed to the impact. What real difference has the intervention made to the beneficiaries?
• How many people have been affected? - Are the local communities and stakeholders more resilient than before?
• How likely is it that any positive changes may be sustained in the short- and medium-term?

Sustainability:

• To which extent did the planning and implementation of the interventions take longer-term and integrated problems into account?
• How did the project consider adequately to the context and the nexus of between humanitarian action, recovery and development?
• Did the project plan and implement an adequate transition and exit strategy that ensures longer-term positive effects and reduces risk of dependency?
• How likely will critical services and effects be sustained beyond the duration of the project?
• How well are the project’s outputs linked to more long-term focused objectives?
• Assess capacity of key actors to contribute to sustaining the positive changes according to the developed ToC.
• What were/are the major factors which influenced the achievement or non-achievement of sustainability of the project?
• To what extent did the benefits of the project continue after donor funding ceased? What were the major factors which influenced the achievement or non-achievement of sustainability of the project?

6. METHODOLOGY:

Oxfam recommends mixed methods (a combination of both quantitative and qualitative methods) that will improve the evaluation findings by ensuring that the limitations of one type of data are balanced by the strength of another. Therefore, the data collection for this evaluation will contain various tools and methodologies (both quantitative and qualitative approaches). Regarding the quantitative approach, the Oxfam MEAL team will conduct the data collection activities of the endline HH survey in the selected intervention areas to find the endline  values of outcome level indicators , which will be compared later against the baseline survey values which was already done at the beginning of the project. Therefore, the consultant is not responsible to design the tool for endline HH survey, as well as to conduct the HH survey data collection in the field. Rather, the consultant is responsible to analyse the data, and write the report, which will be integrated with other information/data to be collected through other qualitative tools, such as FGDs, KIIs, case studies and desk review. Quantitative data using structured questionnaire can be used to see how the project’s intervention bring change to the life of conflict affected communities. The endline survey findings will be also integrated with the evaluation findings according to the key evaluation criteria (relevance, effectiveness, efficiency, sustainability and impact).   

Regarding the qualitative approach, the consultant is expected to develop a detailed methodology (to be endorsed by Oxfam) for this evaluation based on the expected deliverables & OECD-DAC criteria. The consultant will use qualitative data collection methods, such as, focus group discussions/FGD/ with different groups (men, women), key informant interview/KII/, with local government authorities, Oxfam staff, community leaders, partners and other stakeholders, case studies, etc).

The consultant(s) will also do desk study/ literature review, which includes review of different documents, such as baseline and other surveys, project proposals, progress reports, etc and analysing secondary data. The evaluation methodology should make use of baseline and other data already generated by Oxfam’s monitoring and evaluation team. The information and findings of the desk review should be integrated with the data and findings from primary data collection and analysis, and not placed in a separate section.

We anticipate that this evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s)/firms to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders that include:

• Targeted beneficiaries
• Partners and actors directly involved in the project at different levels:
• The local implementing partners
• Community leaders (if applicable) and representative bodies of the affected population.
• Local authorities.
• Oxfam staff involved the implementation of the project (e.g. PMs, Technical Coordinators and field team.)

Data collection tools and appropriate sampling methodology should be prepared by the consultant and shared with Oxfam MEAL team for comment and approval.  And the data collection tools proposed by the consultant should able to respond the key evaluation questions mentioned above.

The evaluators/external consultants should consider the following methodological topics at minimum:

• The evaluator’s understanding of the evaluation questions
• Description of the phases related to the evaluation approach proposed.
• Sources of information for primary and secondary data collections.
• Sampling strategies, including area and population group represented in the sample, sampling procedures and sample size.
• Instruments to be used for data collection.
• Different types of data analysis that will be carried out.
• Reference indicators and benchmarks for each evaluation question (if relevant).

^[1] Sustainability for behaviour change as per “Social norms are socially accepted or agreed values, beliefs, attitudes and behaviours – reflecting what a person considers right and expected behaviour. This is related to how people think others expect them to behave, and what most other people do." (IDS, 2015)

. REQURIED EXPERTISE FOR THE EVALUATION CONSULTANT(S):

This final evaluation should be led by a person (or persons) or firm have following:

Mandatory qualification:

• A minimum of 10 years of experience in humanitarian and development interventions in conducting evaluations for international humanitarian organisations and donor agencies
• Demonstrated experience in the monitoring and evaluation of WASH and Livelihoods, with preferably some experience in gender and protection in emergencies programming or programmatic knowledge of these two sectors
• Experience in the use of quantitative and participatory qualitative methods of data collection
• Strong facilitation and English writing skills are also required
• Knowledge of ICT tools for mobile data collection (Mainly Mobenzi, Survey CTO)
• Commitment to safeguarding policy.
• Commitment to feminist principles in Evaluation.
• Knowledge of the context in Iraq or in middle east and familiarity or direct experience working in Iraq or in the Middle East broadly

Following are not mandatory but desirable: 

• Familiarity or fluency in Arabic is an advantage

Experience of working modalities of Oxfam, is preferred

8- TIMEFRAME, KEY ACTIVITIES AND EXPECTED LEVEL OF EFFORT

Timeframe: The evaluation is to be carried out on June and July 2021, with the final report submitted to Oxfam as per the timeline below. The exact dates of the evaluation are to be confirmed with the selected consultant(s) or firm. Corona Virus may impact on the evaluation process and final date of submission the report.

Total expected level of effort: 25 working days

Note: Following are the suggested number of days. Actual days will be agreed with the selected consultant(s)/firm as per workplan to be delivered during the time frame.

• Review essential documents of the project, including but not limited to the original project proposal, interim or on-going internal reports, and evaluations and lessons learned exercises undertaken thus far and review the key questions suggested and if necessary propose adjustment (3 to 4 days) – work to be done from consultant’s/firm’s home location, all documents will be shared by Oxfam via email);
• Develop and submit the draft inception report with a detailed evaluation plan (to be endorsed by Oxfam) (2 to 3 day);
• Developing and translation of evaluation tools (Tools must be developed in English and translated in Arabic to administer in the field. (2 to 3 days)
• Primary data collection (quantitative and qualitative) (8 to 10 days in Iraq- Diyala and Kirkuk Governorate);
• Data analysis and preparation of draft evaluation report (7 to 10 days);
• Meeting (online) to share and validate the findings from the evaluation (1 day). Selected Oxfam and partner staff will participate in this meeting.
• Finalize the final evaluation report and send it to Oxfam (2 to 3 days). Oxfam will then prepare a management response to be annexed to the evaluation report.

Working closely:

The consultant(s) /firm will report directly to the Country MEAL Coordinator and will work closely with Programme Managers and field teams. The Country MEAL Coordinator will coordinate and facilitate the evaluation process with the selected firm/consultant(s) in collaboration with the Program Managers and MEAL team.

9. EXPECTED OUTPUTS:

1. Develop a detailed evaluation plan (to be submitted after the document review but before the data collection - to be endorsed by Oxfam), outlining the proposed methodology.
2. Draft evaluation report in English to be presented to Oxfam (no more than 40 pages excluding annexes, including executive summary not exceeding 3 pages);
3. Meeting to present the draft findings of the evaluation, and to give feedback to Oxfam staff and the partners.
4. A draft copy of the evaluation report within one week (7 days) of the meeting with Oxfam and partner staff. Feedback from Oxfam will be provided within one (1) week after the submission of the draft report. The consultant/firm will review, then submit a second draft, which will also be subject to Oxfam’s feedback within 2 days. The final report will be produced in 3 working days of submission of the comments. It will include changes/modifications, agreed between Oxfam in Iraq and the consultant/firm. Some minimum guidelines on the evaluation report:
   • The report should systematically answer the key questions posed.
   • It should fairly and clearly represent the views of the different actors/stakeholders.
   • It should give the conclusions of the evaluator, in a way that is clear and substantiated by the available evidence.

Please note that the consultant should submit the final evaluation report with Oxfam before/on 31, July 2021

10. Payment and instructions for interested consultants/firms

Overall expression of interest/EOIs/ will be evaluated based on technical and financial proposals demonstrating value of money and strong technical description.  

Payment will be done in two instalments, 30% upon contract signature, 70% upon Oxfam’s approval of the final evaluation report.

What costs to include in the offer: Consultants/firms should include the following costs in their offer’s budget: daily rate, cost of international travel (home location of consultant/firm to Erbil and back, applies to international consultants only).

What costs not to include in the offer: Oxfam will pay for and procure the following for the evaluation and therefore the following costs should NOT be included in the offer: pick-up/drop-off of consultant(s) from airport, travel inside Iraq for data collection and meetings, interpretation services during meetings with beneficiaries/the partner, printing/photocopying costs.

Note that payment will be made based on the budget in the offer (not based on actual expenses incurred by the consultant). No receipts will be requested from the consultant towards the end of the evaluation.

11. CODES OF BEHAVIOUR:

The evaluation process will be directed by Oxfam’s guidelines for the ethical conduct of evaluations and research, guiding the evaluation team through careful consideration of the key ethical implications at every stage of the evaluation. These guidelines are available at this link: http://policy-practice.oxfam.org.uk/publications/undertaking-research-with-ethics-253032 

Selected consultant or firm will be asked to sign and adhere with Oxfam’s Code of Conduct.

12. SHARING AND USING FINDINGS:

The Oxfam International’s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations. The Policy is available at this link: http://www.oxfam.org/sites/www.oxfam.org/files/oxfam-program-evaluation-policy-dec10.pdf

13. DISCLOSURE:

Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of the Oxfam.

14.Bid Evaluation Criteria

15. EXPRESSIONS OF INTEREST (EOI):

Interested Individuals, organizations or research/consultancy firms, with the experience and skills descried in the ToR should submit Expression of Interest/EOI/. The EOI must include

1. A cover letter of no more than 2 pages introducing the evaluator/organisation and how the skills and competencies described above are met, with concrete examples. Please also use this cover letter to indicate the consultants’ availability for the proposed period. Note: the evaluation should start as early as possible in June 2021 to conclude in July 2021
2. Technical Proposal: The technical proposal should include an interpretation of the objectives of the consultancy, detailed methodology and an elaborate work plan. Organization/individual capacity statement, past experience and activities related to evaluation, CVs of the consulting team and their roles in the achievement of the assignment; names, addresses and telephone numbers of three professional referees.
3. Financial Proposal: A one-page budget of the offer, covering all major anticipated costs; (proposed budget should not be included in the technical proposal).
4. Two to three samples of evaluation reports for evaluations conducted in WASH, and/or livelihoods or in other sectors

How to Apply

Please submit the EOI and other documents by 12:00 am Iraq time by 24 June 2021 to  irqconsultancy@oxfam.org.uk with “Improving access to WASH services and socio-economic well-being of conflict-affected women, men, girls and boys in Iraq, funded by SDC’’ in the subject line. No EOIs will be accepted after deadline.

All questions or clarifications of a technical nature are to be sent to: irqconsultancy@oxfam.org.uk  

Interested applicants can collect the Terms of Reference/ToR/ from OXFAM in IRAQ Country Office, starting from June 13, 2021. Or Oxfam Logistics/HR team can send the ToR for interested applicants via email.  Applicants are required to carefully read through the ToR, make a note of the planned time table, and submit their responses by 24 June 2021 ( 5:00 Pm).  No questions or clarification will be entertained after 24 June 2021 COB.

This invitation to tender has been issued for the sole purpose of obtaining offers for the provision of the services requested against the TOR. OXFAM reserves the right not to enter into or award a contract as result of this invitation to tender. OXFAM also reserves the right to terminate any contract issued as a result of this invitation to tender as set out in the contract terms and conditions.

OXFAM in Iraq office is located in Erbil, Ankawa