FAQ

About MOPAN

  1. What is MOPAN?
  2. What does MOPAN do?
  3. Which donor countries are members of MOPAN?
  4. When and why was MOPAN established?
  5. Who runs MOPAN?
  6. What has MOPAN accomplished since its creation in 2002?

Assessments in 2014

  1. Which multilateral organisations are assessed in 2014?
  2. In which countries are the assessments taking place in 2014?
  3. What is the timing of the 2014 MOPAN assessment process?

Previous Surveys

  1. Which multilateral organisations have been assessed so far, in which countries?

The MOPAN’s assessment approach

  1. What is the MOPAN “Common Approach”?
  2. Does the MOPAN Common Approach assess the achievement of development results?
  3. Why was MOPAN’s “Common Approach” developed?
  4. What data is collected by MOPAN assessments?
  5. What are the data sources used for MOPAN assessments?
  6. What are the strengths of MOPAN’s assessment approach?
  7. What are the limitations of MOPAN’s assessment approach?
  8. Donors are pressing for ‘Delivering as One’ countries so why assess United Nations (UN) agencies individually?
  9. What is the link between MOPAN assessments and the performance frameworks being developed between some MOPAN members and UN agencies?
  10. How will MOPAN’s assessment approach evolve in the future?

Findings

  1. How are findings from MOPAN assessments used?
  2. What is the ‘Country Dialogue Process’?
  3. What does MOPAN hope to achieve from the Country Dialogue Process?
  4. Does MOPAN rank multilateral organisations?
  5. Why does MOPAN not compare multilateral organisations?
  6. Where can I get a copy of MOPAN’s Assessment Reports?

 

 

About MOPAN

  1. What is MOPAN?

The Multilateral Organisation Performance Assessment Network (MOPAN) is a network of 19 donor countries with a common interest in assessing the organisational effectiveness of the major multilateral organisations they fund.

  1. What does MOPAN do?

MOPAN members share an interest in assessing the effectiveness of the major multilateral organisations they fund. Members are committed to a joint approach to assessment and focus on whether multilateral organisations have in place four strategic dimensions of organisational effectiveness—strategic, operational, relationship and knowledge management. The assessments also examine evidence of progress towards outcomes and relevance to country stakeholders. More information about the assessment methodology can be found at www.mopanonline.org/commonapproach/

Each year, MOPAN assesses a select number of multilateral organisations in several countries, gathering, over time, a mix of qualitative and quantitative data on each organisation’s effectiveness. In 2014, MOPAN is assessing UN Women, the United Nations Population Fund (UNFPA), the UN Refugee Agency (UNHCR) and the Food and Agriculture Organization (FAO).

  1. Which donor countries are members of MOPAN?

Members in 2014: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Ireland, Japan, Luxembourg, The Netherlands, Norway, Republic of Korea, Spain, Sweden, Switzerland, the United Kingdom and the United States of America.

  1. When and why was MOPAN established?

MOPAN was established in 2002 to harmonise donor approaches to assessing the organisational effectiveness of multilateral organisations.

  1. Who runs MOPAN?

MOPAN is run by a Steering Committee comprising the 19 MOPAN member countries with a rotating Chair. MOPAN is supported by a Secretariat which is, since 2013, hosted by the OECD. The MOPAN Secretariat is the focal point for members and all actors involved in the assessment process (mopan@oecd.org).

  1. What has MOPAN accomplished since its creation in 2002?

2014

In 2014, UN Women, UNHCR, FAO and UNFPA are assessed by MOPAN in six developing countries MOPAN is assessing both organisational effectiveness and results achievement.

2013

In 2013, the Asian Development Bank, IFAD, WHO and the World Food Programme are assessed by MOPAN in six developing countries. The expanded methodological framework, assessing both organisational effectiveness and results achievement, was applied to all four organisations.

2012

In 2012, an expanded methodological framework which assesses organisational effectiveness as well as the results achieved by multilateral organisations was piloted with four of the assessed organisations (African Development Bank, UNDP, UNICEF and the World Bank). Consultations with staff of the organisations being assessed was also introduced as an additional data source.

The inclusion of global funds was tested by assessing GAVI in addition to the World Bank, the African Development Bank, UNDP, UNICEF, and UNAIDS.

Details about the assessment methodology used by MOPAN can be found in Volume 2 of all Assessment Reports.

2011

In 2011, five organisations were assessed. The methodology was broadened and tested to include humanitarian organisations (UNCHR, UNRWA) as well as organisations with a normative role (UNEP).

2009- 2010

Four organisations were assessed each year, using the MOPAN “Common Approach”. Based on recommendations from multilateral organisations, in 2010, the initial 5-point rating scale was replaced by a 6-point scale. Furthermore, a document review process was introduced as an additional data source to triangulate the information gathered from the perception-based survey.

2003-2008

In 2003 to 2008, MOPAN conducted an Annual Survey, each year assessing the organisational effectiveness of three multilateral organisations in eight to ten developing countries. The survey was completed through MOPAN member embassies and country offices and provided periodic assessments of the partnership behaviour of multilateral organisations at country level.

In 2008, the methodology of the assessments was thoroughly reviewed and adjusted, resulting in the adoption of the MOPAN “Common Approach” in 2009.

The Common Approach is deeper and broader in scope than the Annual Survey. Organised around the widely recognised balanced scorecard approach it takes a more systematic look at organisational effectiveness, focusing on four key areas – strategic management, operational management, relationship management and knowledge management. It is based on stakeholder perceptions of the organisations’ performance in these areas incorporating not only the views of multilateral donors, i.e. MOPAN members at both headquarters and country level, but also those of national partners of the assessed multilateral organisations. The MOPAN Common Approach was established with a threefold purpose - to support (1) the aid effectiveness agenda with the aim to reduce the number of bilateral assessments, (2) accountability and (3) dialogue with the organisations and learning.

Assessments in 201

1. Which multilateral organisations are assessed in 2014?

In 2014, four multilateral organisations are being assessed in six developing countries. These are UN Women, the United Nations Population Fund (UNFPA), the UN Refugee Agency (UNHCR) and the Food and Agriculture Organization (FAO).

2. In which countries are the assessments taking place in 2014?

The countries for the assessment include: Bangladesh, Cambodia, Democratic Republic of Congo, Ecuador, Kenya, Tanzania.

3.What is the timing of the 2014 MOPAN assessment process?

Data will be collected through a survey, consultations and a document review process between February and May 2014. Data analysis will take place from May to June. The Assessment Reports (a “Synthesis Report” and a “Technical Report”) for each organisation will be drafted, reviewed, discussed and finalised in the following months.

Final Assessment Reports for each organisation are scheduled to be published in December 2014. A dialogue process on the findings may take place at country level in late 2014.

Previous Surveys

1. Which multilateral organisations have been assessed so far, in which countries?

 

MO assessed

Countries were the assessments were carried out

2003

WB, UNICEF, WHO

Africa: Uganda, Ghana, Mozambique, Malawi

Asia: Bangladesh, India, Vietnam

LAC: Nicaragua

2004

UNDP, FAO, AfDB

Africa: Benin, Burkina Faso, Uganda, Rwanda, Kenya, Mali

Asia: Nepal, Pakistan, Sri Lanka

LAC: Guatemala

2005

WB, UNFPA, UNAIDS

Africa: Tanzania, Ethiopia, Zambia

Asia: Bangladesh, Vietnam

EE: Albania, Bosnia& Herzegovina

LAC: Bolivia, Nicaragua

2006

UNICEF, ILO, ADB

Africa: Burkina Faso, Kenya, Uganda, Mozambique

Asia: Pakistan, Sri Lanka, Indonesia, Nepal

LAC: Colombia, Guatemala

2007

UNICEF, ILO, ADB

Africa: Benin, Egypt, Senegal, Ethiopia, Mali, Zambia

Asia: Bangladesh,

LAC: Bolivia, Nicaragua

EE: Serbia

2008

EC, UNFPA, WB

Africa: Burkina Faso, Sudan, Tanzania

Asia: Bangladesh, Cambodia, Nepal, Vietnam

EE: Albania, Bosnia & Herzegovina

LAC: Bolivia

2009

AfDB, UNDP, UNICEF, WB

Africa: Ethiopia, Mozambique, Senegal, Uganda

LAC: Guatemala, Peru

Asia: Pakistan, Thailand

EE: Serbia

2010

IFAD, UNFPA, WHO, ADB

Africa: Benin, Kenya, Rwanda

Asia: Indonesia, Vietnam, Afghanistan, Sri Lanka, Indonesia, Zambia

LAC: Nicaragua, Colombia

2011

FAO, IDB, UNEP, UNHCR, UNRWA

Africa: Tanzania, Burundi

Asia: Syria, Palestinian Territories, Jordan, Bangladesh, Lebanon, Nepal

LAC: Bolivia, Ecuador, Brazil, Peru

2012

IBRD, GAVI, AfDB, UNDP, UNICEF, UNAIDS

Africa: Cambodia, Democratic Republic of Congo, Ghana, Zimbabwe, Nigeria, Niger, Morocco

LAC: Honduras

Asia: Philippines

2013

ADB, IFAD, WHO, WFP

Africa: Ethiopia, Mozambique

LAC: Guatemala

Asia: Indonesia, Pakistan, Vietnam

2014

FAO, UNHCR, UN Women, UNFPA

Africa: Tanzania, Kenya, DRC

LAC:Ecuador

Asia: Bangladesh, Cambodia

 

The MOPAN’s assessment approach

1. What is the MOPAN “Common Approach”?

The “Common Approach” is the name of the approach that MOPAN uses to assess the effectiveness of multilateral organisations. It explores whether these organisations have the systems, processes and behaviours needed to be effective over time, and the results achieved by multilateral organisations, particularly at country level.

The MOPAN assessment approach:

  • Generates relevant, credible and robust information on multilateral organisations' effectiveness that MOPAN members can use to fulfil their responsibilities and obligations as bilateral donors and, based on which, MOPAN members, multilateral organisations and direct partners can discuss multilateral effectiveness in order to build better understanding and improve performance.
  • Supports dialogue between MOPAN members, multilateral organisations and their partners, with a specific focus on improving organisational effectiveness over time, both at country and HQ level.

2. Does the MOPAN Common Approach assess the achievement of development results?

Before 2012, the Common Approach did not examine development effectiveness or the achievement of development results. Rather, it examined whether the multilateral organisations had in place the necessary behaviours, systems and processes to help achieve those results. In 2012, MOPAN piloted an expanded methodological framework which provided an assessment of organisational effectiveness as well as the results achieved by multilateral organisations. The assessment of results focuses on the degree to which progress is being made towards the organisation’s stated objectives and will analyse the relevance of its programming. It was piloted with AfDB, UNDP, UNICEF and the World Bank and has been included as part of the assessments as of 2013. This may be considered as a first tentative step in the long-term process of developing an approach to ultimately assess development effectiveness.

Final reports on the findings of MOPAN’s 2013 assessments of ADB, IFAD, WHO and WFP are now available online, under publications along with the executive summaries, main findings of the reports and responses from the organisations.

3. Why was MOPAN’s  “Common Approach” developed?

The MOPAN Common Approach was developed for several reasons:

  • There are growing demands internationally to better understand how public funds are used for international aid purposes. This also applies to multilateral assistance, which is one reason that multilateral organisations are increasingly focusing on effectiveness and results.
  • Currently, there is no widely accepted, coherent approach to assessing organisational effectiveness across multilateral organisations. Many international donor agencies have developed their own approaches to assess the effectiveness of the multilateral organisations they fund, but they have done so in isolation and without a ‘common approach’. The Common Approach was developed in 2008-2009 to address the recognised need for a common comprehensive multilateral organisation assessment system.
  • In line with the principles for effective aid, MOPAN members recognised the need to harmonise their work to avoid duplication, increase the amount and scope of information on the effectiveness of their individual organisations, and reduce the transaction costs associated with running their own evaluations. The Common Approach is derived from, and meant to replace, seven existing bilateral assessment tools.

4. What data is collected by MOPAN assessments?

The survey is built around four strategic dimensions of organisational effectiveness, namely:

  1. Strategic management
  2. Operational management
  3. Relationship management
  4. Knowledge management

In addition, the development results achieved by multilateral organisations are assessed.

Organisations are assessed against a set of key performance indicators (KPIs), which are in turn comprised of micro-indicators.

Data are collected through a survey (tailored to the operations of the different types of multilateral organisations), consultations and a document review process.

5. What are the data sources used for MOPAN assessments?

Data are collected from several sources:

              i.        MOPAN member officials at headquarter level, including missions, delegations and representations, and at country level in the selected survey countries

             ii.        MOPAN member officials at country level (in the selected survey countries)

            iii.        Multilateral organisations’ direct partners at country level, including governmental representatives, private sector, non-governmental and other civil society organisations etc.

These three groups are asked to complete the survey. Findings from the survey are complemented with:

            iv.        .A review of documents published by multilateral organisations, comprising publicly available documents and publications—including strategies and plans, human resource documents, reports and system descriptions

             v.        Interviews with staff of the multilateral organisation (primarily at headquarters), who are consulted on the topics of the assessment.

6. What are the strengths of MOPAN’s assessment approach?

The MOPAN “Common Approach” has many strengths, including:

  • It is based on the core elements of existing bilateral assessment tools
  • It seeks information from two different perspectives: MOPAN members (both at headquarters and country level) and direct partners of multilateral organisations, in line with the commitments made by donors to the Paris Declaration on Aid Effectiveness and the Accra Agenda for Action regarding harmonisation, partner voice and mutual accountability
  • It uses a mix of quantitative and qualitative information (through closed and open-ended questions) which strengthens the accuracy of measurement and provides a basis for discussion about improving the organisations’ effectiveness
  • It triangulates the data through complementing perceptual data from the survey with a document review and consultations, thus adding additional data sources and increasing the validity of the assessment
  • It compares findings with other sources where possible, for example results of the surveys on monitoring the Paris Declaration
  • It is customised to take into account the differences between different types of multilateral organisations

7. What are the limitations of MOPAN’s assessment approach?

The MOPAN “Common Approach”:

  • Allows MOPAN members and the multilateral organisations assessed to choose the most relevant individuals to complete the survey, but there is no way to know whether the most qualified ones complete the survey
  • Triangulates the data collected with consultations and a document review but while a document review can comment on the contents of a document, it cannot assess the extent to which the spirit of that document has been implemented within the organisation (unless implementation is documented elsewhere.
  • Produces numerical scores which suggest a high degree of precision but these scores only provide a picture of effectiveness in priority areas—they do not provide deep insight into the different dimensions of organisational effectiveness because the approach is based on a perceptions survey and not on an analysis of behaviours, systems and procedures

8. Donors are pressing for ‘Delivering as One’ countries so why assess United Nations (UN) agencies individually?

Each UN agency has a different remit and mandate which make it possible to assess them individually under this common assessment approach. MOPAN’s assessment approach allows members to obtain information that is not otherwise available. MOPAN members are also actively following progress on the operationalisation of UN reform initiatives.

9. What is the link between MOPAN assessments and the performance frameworks being developed between some MOPAN members and UN agencies?

The performance frameworks set the medium-term strategic direction for the partnership between a MOPAN member and a UN multilateral organisation. They link MOPAN members’ core funding to the attainment of the performance targets included in the UN agencies’ strategic plan.

Performance frameworks differ from the MOPAN assessment, which is not directly linked to funding, and are geared towards providing insights on how targets could be set and achieved.

10. How will MOPAN’s assessment approach evolve in the future?

MOPAN has agreed to review its approach and methodology in order to become even more relevant to its members. Starting in 2015, MOPAN will be implementing a revised approach (“MOPAN 3.0”) to assessing multilateral organisations, including a strengthened methodology for assessing contributions to development effectiveness.

Findings

1. How are findings from MOPAN assessments used?

It is imperative for donor countries that multilateral organisations deliver effectively and are ready to meet the challenges they face in delivering aid. MOPAN assessment findings highlight areas of strength and pinpoint areas for improvement. The findings are discussed internally in each multilateral organisation and also among MOPAN members —dialogue is a key feature of the Network.

Each MOPAN member decides how best to use the findings inside their own organisation. In general, they are used:

  • To build better understanding of multilateral organisations’ effectiveness
  • To support discussion between developing country partner governments, bilateral donors and multilateral organisations, as part of on-going processes to strengthen mutual accountability at country level (discussions could include exploring how bilateral partners can better support multilateral organisations in-country)
  • As input to policy development
  • To support the steering of, and participation in, the governance of the multilateral organisations (for example, at a board or governing body meeting)
  • To strengthen relationships with multilateral organisations at country level
  • As input to wider debates about multilateral organisations’ effectiveness.

2.What is the ‘Country Dialogue Process’?

After publication of the Assessment Reports, the country-specific findings of the MOPAN assessment are presented and discussed with key stakeholders at country level in a meeting, a workshop in an existing forum. This dialogue process with the assessed organisations, their partners and MOPAN members present in the country is initiated and led by the MOPAN members who have taken on the role of Country Leads.

3. What does MOPAN hope to achieve from the Country Dialogue Process?

The dialogue process provides the opportunity for MOPAN members in-country to present and discuss the key findings of the MOPAN assessment in each country with relevant development actors. The findings of the assessment provide an evidence base, which can be used to promote dialogue between MOPAN members, multilateral organisations and their direct partners, with the aim of improving multilateral organisations’ effectiveness. MOPAN also aims to build consensus about the actions required at the country level to improve multilateral organisations’ effectiveness. The meetings or workshops with the organisations and direct partners at country level play a key role in achieving these aims.

4. Does MOPAN rank multilateral organisations?

Due to the different remits of the organisations, MOPAN does not rank the performance of individual multilateral organisations. Rather, it uses assessment findings to encourage discussion among MOPAN members and multilateral organisations about lessons learned and ways to further build capacity to be effective. MOPAN aims to assess each multilateral organisation on its own merit, and MOPAN assessments should therefore not be used as a tool to compare one organisation to another.

5. Why does MOPAN not compare multilateral organisations?

It is not possible to compare multilateral organisations to one another—their mandates and structures vary too much in scope and nature. However, because the MOPAN assessment is repeated at intervals of time it can help determine whether the multilateral organisation’s effectiveness is changing over time.

6. Where can I get a copy of MOPAN’s Assessment Reports?

The Assessment Reports for each organisation assessed are available on MOPAN’s website (www.mopanonline.org) under Publications and also on most MOPAN members’ websites.