About MOPAN

1. What does MOPAN stand for?
2. What does MOPAN do?
3. Which donor countries are members of MOPAN?
4. When and why was MOPAN established?
5. Who runs MOPAN?
6. Who was in charge of the Secretariat in previous years?
7. What has MOPAN accomplished since its creation in 2002?

The 2011 Common Approach Survey

1. Which multilateral organisations will be assessed in 2011?
2. In which countries will the assessment take place in 2011?
3. What is the timing of the 2011 Common Approach?

Previous Surveys

1. Which multilateral organisations were assessed before 2011?
2. In which countries were assessments held before 2011?

The Common Approach Assessment

1. What is the Common Approach assessment?
2. Does the Common Approach assess development effectiveness?
3. How does MOPAN support other measures and work being handled by other parties to gain a comprehensive picture of multilateral organisations’ effectiveness?
4. Why was the Common Approach developed?
5. What data is collected in the Common Approach?
6. What are the data sources used in the Common Approach?
7. How is the Common Approach broader than a perceptions-based survey?
8. How does the document review add value?
9. What are the strengths of the MOPAN Common Approach?
10. What are the limitations of the MOPAN Common Approach?
11. Donors are pressing for ‘Delivering as One’ countries so why assess United Nations (UN) agencies individually?
12. What is the link between the MOPAN Common Approach and the performance frameworks being developed between some MOPAN members and UN agencies?

Findings

1. How are the MOPAN Common Approach findings used?
2. What is the ‘Country Dialogue Process’?
3. What does MOPAN hope to achieve from the Country Dialogue Process?
4. In terms of ranking, is there a top performer every year?
5. Why does the MOPAN Common Approach not compare multilateral organisations?
6. Where can I get a copy of the MOPAN 2010 Institutional Reports?

About MOPAN

What does MOPAN stand for?

MOPAN stands for Multilateral Organisation Performance Assessment Network—a network of 16 donor countries with a common interest in assessing the organisational effectiveness of the major multilateral organisations they fund.

What does MOPAN do?

MOPAN is a network of 16 donor countries. Members share a common interest in assessing the organisational effectiveness of the major multilateral organisations they fund. Members are committed to a joint approach to assessment and focus on whether multilateral organisations have in place four strategic dimensions of organisational effectiveness—strategic, operational, relationship and knowledge management.

Every year, MOPAN assesses a select number of multilateral organisations in several countries, gathering, over time, a mix of qualitative and quantitative data on each organisation’s effectiveness.

Which donor countries are members of MOPAN?

Members in 2011: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Ireland, The Netherlands, Norway, Republic of Korea, Spain, Sweden, Switzerland and the United Kingdom.

When and why was MOPAN established?

MOPAN was established in 2002 to harmonise donor approaches to assessing the organisational effectiveness of multilateral organisations.

Who runs MOPAN?

A Secretariat runs MOPAN. The Secretariat is the focal point for members and other interested parties on all MOPAN activities. It coordinates the administrative and other work undertaken by and on behalf of MOPAN members. The Secretariat is managed by members on a rotational basis and is supported by a Technical Working Group, a Communications Working Group, a Contractual Working Group and a Task Team for dealing with the future development of the network. In 2011 the Secretariat is being led by Germany.  

Who was in charge of the Secretariat in previous years?

2010: Finland

2009: Denmark

2008: United Kingdom

2007: Austria

2006: The Netherlands

2005: Switzerland

2004: Canada

2003: Norway and Sweden

What has MOPAN accomplished since its creation in 2002?

Until 2008, MOPAN’s main focus was on conducting an annual ‘perception-based’ survey. The survey was completed through MOPAN member embassies and country offices and captured data on organisational effectiveness of multilateral organisations at country level.

The first survey was conducted in 2003. Each year from 2004 to 2008, three multilateral organisations in each of eight to ten developing countries were surveyed. In 2009, four multilateral organisations in nine developing countries were assessed, and in 2010, four multilateral organisations were reviewed in ten developing countries. Findings are available in the publications section of the MOPAN website (www.mopanonline.org) and on member websites.

In 2011, five multilateral organisations in twelve developing countries will be assessed. In 2012 MOPAN may extend its approach to test it with a global fund.

The 2011 Common Approach Survey

Which multilateral organisations will be assessed in 2011?

Inter-American Development Bank (IDB), Food and Agriculture Organisation (FAO), United Nations Environment Program (UNEP), United Nations High Commissioner for Refugees (UNHCR), and the United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA).

In which countries/territories will the assessment take place in 2011?

Bangladesh, Bolivia, Brazil, Burundi, Ecuador, Jordan, Lebanon, Nepal, Palestinian Territories, Peru, Syria and Tanzania.

What is the timing of the 2011 Common Approach?

Data collection will begin at the end of March 2011. Final Institutional Reports for each organisation are scheduled to be published in December 2011.

Previous Surveys

Which multilateral organisations were assessed before 2011?

 In which countries were assessments held before 2011?

The Common Approach Assessment

What is the Common Approach assessment?

The MOPAN Common Approach is an annual assessment of the effectiveness of multilateral organisations. It explores whether these organisations have the systems, processes and behaviours needed to be effective over time, particularly at country level.

The MOPAN Common Approach:

• Generates relevant, credible and robust information MOPAN members can use to meet their domestic accountability requirements, and fulfil their responsibilities and obligations as bilateral donors
• Provides an evidence base for MOPAN members, multilateral organisations and direct partners to discuss organisational effectiveness and in doing so, build better understanding and improve organisational effectiveness and learning over time
• Supports dialogue between MOPAN members, multilateral organisations and their partners, with a specific focus on improving organisational effectiveness over time, both at country and HQ level.

Does the Common Approach assess development effectiveness?

No. The Common Approach does not examine development effectiveness or the achievement of development results, but rather the multilateral organisation’s behaviours, systems and processes that help ensure those results. Nor does the Common Approach scrutinise administrative economy or efficiency.

How does MOPAN support other measures and work being handled by other parties to gain a comprehensive picture of multilateral organisations’ effectiveness?

MOPAN intends to collaborate in an initiative by the Organisation for Economic Cooperation and Development and the Development Assistance Committee’s Network on Development Evaluation (EvalNet). This initiative aims to augment current information systems on multilateral organisations’ effectiveness—such as MOPAN and the Peer Reviews of the evaluation function—with information on development effectiveness.

Why was the Common Approach developed?

The Common Approach was developed for several reasons:

• There are growing demands internationally to better understand how public funds are used for international aid purposes. This also applies to multilateral assistance, which is one reason that multilateral organisations are increasingly focusing on effectiveness and results.
• Currently, there is no widely accepted, coherent approach to assessing organisational effectiveness across multilateral organisations. Many international donor agencies have developed their own approaches to assess the effectiveness of the multilateral organisations they fund, but they have done so in isolation and without a ‘common approach’. The Common Approach was developed to address the recognised need for a common comprehensive multilateral organisation assessment system. 
• In line with the Paris Principles, MOPAN members recognised the need to harmonise their work to avoid duplication, increase the amount and scope of information on the effectiveness of their individual organisations, and reduce the transaction costs associated with running their own evaluations. The Common Approach is derived from, and meant to replace, seven existing bilateral assessment tools. It is also meant to forestall the development of other assessment approaches.
• After a few years of conducting the MOPAN annual survey, members agreed that the initial approach of a ‘perceptions-based’ survey needed to be broadened and deepened (in particular, to include the views of direct partners) to provide them with more robust findings.

What data is collected in the Common Approach?

Data is collected through a survey (tailored to the activities of different types of multilateral organisations) and a review of documents published by the multilateral organisations.

The survey is built around four dimensions of organisational effectiveness, namely:

1. Strategic management (corporate governance, corporate and other strategies)
2. Operational management (use of performance information, financial resources management, human resources management, portfolio management)
3. Relationship management (ownership, alignment, harmonisation)
4. Knowledge management (performance monitoring and evaluation, performance reporting, application of lessons learned).

Each dimension is broken down into nineteen key performance indicators (KPIs), which are in turn broken down into micro-indicators (MIs). The survey links to, and flows directly from, the MIs.

What are the data sources used by the Common Approach?

Donors (MOPAN member representatives) at headquarters and at country level, ministry and Non-Government Organisation representatives in the selected developing countries complete the survey on the multilateral organisations assessed in those countries.

External and internal assessments (for example Common Performance Assessment System (COMPAS) reports and Survey Reports on Monitoring the Paris Declaration) are also used where possible. In 2011 it will be supplemented with a review of documents publicly available from the multilateral organisations surveyed. 

How is the Common Approach broader than a perceptions-based survey?

The Common Approach uses a sampling approach for its survey where respondents are specifically selected from the population surveyed. While the survey is perception-based, the sampling assembles a group of individuals with known or demonstrable experience and expertise related to the multilateral organisations being assessed (i.e. it is based on “expert sampling”). 

How does the document review add value?

In 2011 additional data will be collected through a document review to bring a better balance between factual- and perception-data to the findings. The document review will:

• Inform the assessment of the micro-indicators to be analysed through a qualitative analysis of each multilateral organisation’s documentation
• Allow for triangulation of findings
• Provide additional contextual information on the multilateral organisations in the four dimensions of organisational effectiveness assessed in the MOPAN Common Approach
• Help illustrate the level of transparency on the multilateral organisation in question.

The review will use publicly available documents. It will be conducted on all indicators where a document review is considered relevant.

What are the strengths of the MOPAN Common Approach?

The MOPAN Common Approach has many strengths, including:

• It is based on the core elements of existing bilateral assessment tools
• It seeks information from two different perspectives: MOPAN members (both at headquarters and country level) and direct partners of multilateral organisations (in line with the commitments made by donors to the Paris Declaration on Aid Effectiveness and the Accra Agenda for Action regarding harmonisation, partner voice and mutual accountability)
• It uses a mix of quantitative and qualitative information (through closed and open-ended questions) which strengthens the accuracy of measurement and provides a basis for discussion about improving the organisations’ effectiveness
• It compares findings with other sources where possible (for example results of the surveys on monitoring the Paris Declaration)
• It is customised to take into account the differences between different types of multilateral organisations
• It is based on a web tool which improves the efficiency of data collection and data analysis, and reduces the burden on survey respondents.

What are the limitations of the MOPAN Common Approach?

The MOPAN Common Approach:

• Allows network members and the multilateral organisations assessed to choose the most relevant individuals to complete the survey, but there is no way to know whether the most qualified ones complete the survey
• Produces numerical scores with a high degree of precision but these scores only provide a picture of effectiveness in priority areas—they do not provide deep insight into the different dimensions of organisational effectiveness because the approach is based on a perceptions survey and not on an analysis of behaviours, systems and procedures
• Findings are based, for the most part, on perception data, however, this will be triangulated with other findings where possible.

Donors are pressing for ‘Delivering as One’ countries so why assess United Nations (UN) agencies individually?

Each UN agency has a different remit and mandate which make it possible to assess them individually under this common assessment approach. The MOPAN Common Approach allows members to obtain information that is not otherwise available. MOPAN members are also actively following progress on the operationalisation of UN reform initiatives.

What is the link between the MOPAN Common Approach and the performance frameworks being developed between some MOPAN members and UN agencies?

The performance frameworks set the medium-term strategic direction for the partnership between a MOPAN member and a UN multilateral organisation. They link the MOPAN members’ core funding to the attainment of the performance targets included in the UN agencies’ strategic plan.

Performance frameworks differ from the MOPAN assessment, which is not directly linked to funding, and are geared towards providing insights on how targets could be set and achieved.

Findings

How are the MOPAN Common Approach findings used?

It is imperative for donor countries that multilateral organisations deliver effectively and are ready to meet the challenges they face in delivering aid. MOPAN Common Approach findings highlight areas of strength and pinpoint areas for improvement. The findings are discussed internally in each multilateral organisation and also among members of the network—dialogue is a key feature of MOPAN’s strategic approach.

Each MOPAN member decides how best to use the findings inside their own organisation. In general, they are used:

• To build better understanding of multilateral organisations’ effectiveness
• To support discussion between developing country partner governments, bilateral donors and multilateral organisations, as part of ongoing processes to strengthen mutual accountability at country level (discussions could include exploring how bilateral partners can better support multilateral organisations in-country)
• As input to policy development
• To support the steering of, and participation in, the governance of the multilateral organisations (for example, at a board or governing body meeting)
• To strengthen relationships with multilateral organisations at country level
• As input to wider debates about multilateral organisations’ effectiveness.

What is the ‘Country Dialogue Process’?

The Country Dialogue Process is the process during which the findings of the MOPAN Common Approach assessment are presented and discussed at the country level. The dialogue process itself can take the form of a meeting, a workshop or there may be an existing forum, which could be used to present and discuss the country-specific findings. Country leads have the flexibility to use whatever format they deem most appropriate for their specific country context.

What does MOPAN hope to achieve from the Country Dialogue Process?

The dialogue process provides the opportunity for MOPAN members in-country to present and discuss the key findings of the MOPAN country data summaries with relevant development actors. The findings of the MOPAN Common Approach assessment provide an evidence base, which can be used to promote dialogue between MOPAN members, multilateral organisations and their direct partners, with the aim of improving multilateral organisations’ effectiveness. MOPAN also aims to build consensus about the actions required at the country level to improve multilateral organisations’ effectiveness. It is hoped that the in-country meetings or workshops will play a key role in achieving these aims.

In terms of ranking, is there a top performer every year?

Due to the different remits of the organisations each year, MOPAN does not rank the performance of individual multilateral organisations. Rather, it uses the findings of its work to encourage discussion among MOPAN members and multilateral organisations about lessons learned and ways to further build their capacity to be effective. The Common Approach is meant to assess each multilateral organisation on its own merit, and therefore should not be used as a tool to compare one organisation to another.

Why does the MOPAN Common Approach not compare multilateral organisations?

It is not possible to compare multilateral organisations to one another—their mandates and structures vary too much in scope and nature. However, because the MOPAN assessment is repeated at intervals of time it can help determine whether the multilateral organisation’s effectiveness in the areas covered by the survey is changing over time.

Where can I get a copy of the MOPAN 2010 Institutional Reports?

The Institutional Report for each organisation assessed in 2010 and those of previous years are available on MOPAN’s website (www.mopanonline.org) and also on most MOPAN members’ websites.

July 2011