FAQ
About MOPAN
- What does MOPAN stand for?
- What does MOPAN do?
- Which donor countries are members of MOPAN?
- When and why was MOPAN established?
- Who runs MOPAN?
- Who was in charge of the Secretariat in previous years?
- What has MOPAN accomplished since its creation in 2002?
The 2013 Common Approach
- Which multilateral organisations will be assessed in 2012?
- In which countries will the assessment take place in 2012?
- What is the timing of the 2012 Common Approach?
Previous Surveys
- Which multilateral organisations were assessed before 2012?
- In which countries were assessments held before 2012?
The Common Approach Assessment
- What is the Common Approach assessment?
- Does the Common Approach assess development effectiveness?
- How does MOPAN support other measures and work being handled by other parties to gain a comprehensive picture of multilateral organisations’ effectiveness?
- Why was the Common Approach developed?
- What data is collected in the Common Approach?
- What are the data sources used in the Common Approach?
- How is the Common Approach broader than a perceptions-based survey?
- How does the document review add value?
- What are the strengths of the MOPAN Common Approach?
- What are the limitations of the MOPAN Common Approach?
- Donors are pressing for ‘Delivering as One’ countries so why assess United Nations (UN) agencies individually?
- What is the link between the MOPAN Common Approach and the performance frameworks being developed between some MOPAN members and UN agencies?
Findings
- How are the MOPAN Common Approach findings used?
- What is the ‘Country Dialogue Process’?
- What does MOPAN hope to achieve from the Country Dialogue Process?
- In terms of ranking, is there a top performer every year?
- Why does the MOPAN Common Approach not compare multilateral organisations?
- Where can I get a copy of the MOPAN 2012 Institutional Reports?
About MOPAN
What does MOPAN stand for?
MOPAN stands for Multilateral Organisation Performance Assessment Network—a network of 17 donor countries with a common interest in assessing the organisational effectiveness of the major multilateral organisations they fund.
What does MOPAN do?
MOPAN is a network of 17 donor countries. Members share a common interest in assessing the organisational effectiveness of the major multilateral organisations they fund. Members are committed to a joint approach to assessment and focus on whether multilateral organisations have in place four strategic dimensions of organisational effectiveness—strategic, operational, relationship and knowledge management.
Every year, MOPAN assesses a select number of multilateral organisations in several countries, gathering, over time, a mix of qualitative and quantitative data on each organisation’s effectiveness.
Every year, MOPAN assesses a select number of multilateral organisations in several developing countries, gathering, over time, a mix of qualitative and quantitative data on each organisation’s effectiveness.
In 2013, MOPAN will continue an expanded assessment framework which will provide an assessment of organisational effectiveness as well as the results achieved by multilateral organisations.
Which donor countries are members of MOPAN?
Members in 2012: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Ireland, The Netherlands, Norway, Republic of Korea, Spain, Sweden, Switzerland, the United Kingdom and the United States of America.
When and why was MOPAN established?
MOPAN was established in 2002 to harmonise donor approaches to assessing the organisational effectiveness of multilateral organisations.
Who runs MOPAN?
MOPAN is run by a Secretariat. The MOPAN Secretariat is the focal point for members and all actors involved in the assessment process as well as other interested parties on all MOPAN activities. It is responsible for implementing, coordinating and delivering the MOPAN Common Approach 2013 and is supported in doing so by collaboration with two working groups comprised of different MOPAN members: a Technical Working Group and a Strategic Working Group (SWG)
To further optimise its procedures and organisational structures, from 2013, MOPAN will establish a permanent Secretariat hosted by the Organisation for Economic Cooperation and Development (OECD)
Who was in charge of the Secretariat in previous years?
2013: Australia
2012: Ireland, jointly with Germany
2011: Germany; from September 2011 onwards, Secretariat jointly led with Ireland
2010: Finland
2009: Denmark
2008: United Kingdom
2007: Austria
2006: The Netherlands
2005: Switzerland
2004: Canada
2003: Norway and Sweden
What has MOPAN accomplished since its creation in 2002?
2003 to 2008:
MOPAN has conducted an Annual Survey, each year assessing the organisational effectiveness of three multilateral organisations in eight to ten developing countries. The survey was completed through MOPAN member embassies and country offices and provided periodic assessments of the partnership behaviour of multilateral organisations at country level.
In 2008, the methodology of the assessments by MOPAN was thoroughly reviewed and adjusted, resulting in the adoption of the MOPAN Common Approach in 2009.
The MOPAN Common Approach is deeper and broader in scope than the Annual Survey. Organised around the widely recognised balanced scorecard approach it takes a more systematic look at organisational effectiveness, focusing on four key areas – strategic management, operational management, relationship management and knowledge management. It is based on stakeholder perceptions of the organisations’ performance in these areas incorporating not only the views of multilateral donors, i.e. MOPAN members at both headquarters and country level, but also those of national partners of the assessed multilateral organisations. The MOPAN Common Approach was established with a threefold purpose - to support (1) the aid effectiveness agenda with the aim to reduce the number of bilateral assessments, (2) accountability and (3) dialogue with the organisations and learning.
Since 2009 MOPAN has continuously aimed to further extend the assessment’s scope and reach and to further develop the Common Approach methodologically.
2009 to 2010:
Four organisations were assessed each year, using the Common Approach. Based on recommendations from multilateral organisations, in 2010, the initial 5-point rating scale was replaced by a 6-point scale. Furthermore, a document review process was introduced as an additional data source to triangulate the information gathered from the perception-based survey.
2011:
In 2011, five organisations were assessed. The methodology was broadened and tested to include humanitarian organisations (UNCHR, UNRWA) as well as organisations with a normative role (UNEP).
2012:
In 2012, the number of organisations under review has increased to six, the inclusion of global funds into the MOPAN assessments is being tested by assessing GAVI. Consultations with staff of the organisations under review are introduced as an additional data source.
An expanded methodological framework which will provide an assessment of organisational effectiveness as well as the results achieved by multilateral organisations is piloted with four of the six organisations.
Findings are available on this website under Publications. Details on the methodology used by MOPAN can be found in Vol. 2 of the Institutional Reports for each year.
The Common Approach
Which multilateral organisations will be assessed in 2013 in which countries?
In 2013, four multilateral organisations will be assessed in six developing countries. These are, namely, the Asian Development Bank (ADB); International Fund for Agricultural Development (IFAD); World Health Organisation (WHO) and UNited Nations World Food Programme (WFP) and the countries for the assessment include: Ethiopia; Mozambique; Guatemala; Indonesia; Pakistan and Vietnam.
What is the timing of the 2013 Common Approach?
Data will be collected through a survey, consultations and a document review process between February and May 2013. Data analysis will take place from May to June. Institutional Reports for each organisation will be drafted, reviewed, discussed and finalised in the following months.
Final Institutional Reports for each organisation are scheduled to be published in December 2013.
A dialogue process on the findings at country level will take place at the beginning of 2014.
Previous Surveys
Which Multilateral Organisations were assessed before 2013?
World Bank (WB), African Development Bank (AfDB), Asian Development Bank (ADB), Inter-American Development Bank (IDB), World Health Organization (WHO), United Nations Development Programme (UNDP), United Nations Food and Agriculture Organization (FAO), United Nations Population Fund (UNFPA), United Nations Joint Programme on HIV/ AIDS, United Nations International Children’s Emergency Fund (UNICEF), International Labour Organization (ILO), European Commission (EC), International Fund for Agricultural Development (IFAD), United Nations Environment Programme (UNEP), United Nations High Commissioner for Refugees (UNHCR), United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA), Global Alliance for Vaccines and Immunization (GAVI).
In which countries were assessments held before 2013?
|
2012 |
Cambodia, The Democratic Republic of Congo, Ghana, Honduras, Morocco, Niger, Nigeria, Philippines, Zimbabwe |
|
2011 |
Bangladesh, Bolivia, Brazil, Burundi, Ecuador, Jordan, Lebanon, Nepal, Palestinian Territories, Peru, Syria, Tanzania |
|
2010 |
Afghanistan, Benin, Colombia, Indonesia, Kenya, Nicaragua, Rwanda, Sri Lanka, Vietnam, Zambia |
|
2009 |
Ethiopia, Guatemala, Mozambique, Pakistan, Peru, Senegal, Serbia, Thailand, Uganda |
|
2008 |
Albania, Bangladesh, Bosnia & Herzegovina, Bolivia, Burkina Faso, Cambodia, Nepal, Tanzania, Sudan, Vietnam |
|
2007 |
Benin, Bangladesh, Bolivia, Mali, Nicaragua, Serbia, Zambia |
|
2006 |
Burkina Faso, Colombia, Guatemala, Indonesia, Kenya, Mozambique, Nepal, Pakistan, Sri Lanka, Uganda |
|
2005 |
Albania, Bangladesh, Bolivia, Bosnia & Herzegovina, Ethiopia, Nicaragua, Tanzania, Vietnam, Zambia |
|
2004 |
Benin, Burkina Faso, Guatemala, Kenya, Mali, Nepal, Rwanda, Sri Lanka, Uganda |
|
2003 |
Bangladesh, Ghana, India, Nicaragua, Malawi, Mozambique, Vietnam |
The Common Approach Assessment
What is the Common Approach assessment?
The MOPAN Common Approach is an annual assessment of the effectiveness of multilateral organisations. It explores whether these organisations have the systems, processes and behaviours needed to be effective over time, particularly at country level.
The MOPAN Common Approach:
- Generates relevant, credible and robust information on multilateral organisations' effectiveness that MOPAN members can use to fulfil their responsibilities and obligations as bilateral donors and, based on which, MOPAN members, multilateral organisations and direct partners can discuss multilateral effectiveness in order to build better understanding and imrpove performance.
- Supports dialogue between MOPAN members, multilateral organisations and their partners, with a specific focus on improving organisational effectiveness over time, both at country and HQ level.
Does the Common Approach assess development effectiveness?
Before 2012, the Common Approach has not examined development effectiveness or the achievement of development results. Rather, it examined whether the multilateral organisations had in place the necessary behaviours, systems and processes to help achieve those results. In 2012, MOPAN piloted an expanded methodological framework which provided an assessment of organisational effectiveness as well as the results achieved by multilateral organisations. The assessment of results focuses on the degree to which progress is being made towards the organisation’s stated objectives and will analyse the relevance of its programming. It was piloted with AfDB, UNDP, UNICEF and the World Bank, all previously assessed in 2009 and will be included as part of the 2013 assessment across all four multilateral organisations to be assessed. This may be considered as a first tentative step in the long-term process of developing an approach to ultimately assess development effectiveness.
Final reports on the findings of MOPAN’s 2013 Common Approach assessment of each multilateral organisation are expected to be released in December 2013, along with responses from the organisations.
How does MOPAN support other measures and work being handled by other parties to gain a comprehensive picture of multilateral organisations’ effectiveness?
MOPAN intends to collaborate in an initiative by the Organisation for Economic Cooperation and Development and the Development Assistance Committee’s Network on Development Evaluation (EvalNet). This initiative aims to augment current information systems on multilateral organisations’ effectiveness—such as MOPAN and the Peer Reviews of the evaluation function—with information on development effectiveness.
Why was the Common Approach developed?
The Common Approach was developed for several reasons:
- There are growing demands internationally to better understand how public funds are used for international aid purposes. This also applies to multilateral assistance, which is one reason that multilateral organisations are increasingly focusing on effectiveness and results.
- Currently, there is no widely accepted, coherent approach to assessing organisational effectiveness across multilateral organisations. Many international donor agencies have developed their own approaches to assess the effectiveness of the multilateral organisations they fund, but they have done so in isolation and without a ‘common approach’. The Common Approach was developed to address the recognised need for a common comprehensive multilateral organisation assessment system.
- In line with the Paris Principles, MOPAN members recognised the need to harmonise their work to avoid duplication, increase the amount and scope of information on the effectiveness of their individual organisations, and reduce the transaction costs associated with running their own evaluations. The Common Approach is derived from, and meant to replace, seven existing bilateral assessment tools. It is also meant to forestall the development of other assessment approaches.
- After a few years of conducting the MOPAN annual survey, members agreed that the initial approach of a ‘perceptions-based’ survey needed to be broadened and deepened (in particular, to include the views of direct partners) to provide them with more robust findings.
What data is collected in the Common Approach?
The survey is built around four strategic dimensions of organisational effectiveness, namely:
- Strategic management
- Operational management
- Relationship management
- Knowledge management
These areas of organisational effectiveness are broken down into nineteen key performance indicators, which are in turn comprised of micro-indicators.
To inform the assessment of key development results achieved by multilateral organisations, data on four additional key performance indicators will be collected which will cover the extent of the multilateral organisations’
- Progress towards its institutional/organisation-wide objectives
- Contributions to relevant MDGs
- Contributions to country-level goals and priorities
- Relevance of its objectives and program of work to major stakeholders.
Data will be collected through a survey (tailored to the operations of the different types of multilateral organisations), consultations and a document review process.
What are the data sources used by the Common Approach?
Data will be collected from five sources:
1. Donors, i.e. MOPAN member officials at headquarter level, including missions, delegations and representations, and at country level in the selected survey countries
2. Donors, i.e. MOPAN member officials at country level (in the selected survey countries)
3. Multilateral organisations’ direct partners at country level, including governmental representatives, private sector, non-governmental and other civil society organisations etc.
These three groups will complete the survey. Findings from the survey are complemented with:
4.A review of documents published by multilateral organisations, comprising publicly available documents and publications—including strategies and plans, human resource documents, reports and system descriptions
5.Staff of the multilateral organisation (primarily at headquarters), who will be consulted on the topics of the assessment.
How is the Common Approach broader than a perceptions-based survey?
The Common Approach uses a sampling approach for its survey where respondents are specifically selected from the population surveyed. While the survey is perception-based, the sampling assembles a group of individuals with known or demonstrable experience and expertise related to the multilateral organisations being assessed (i.e. it is based on “expert sampling”).
How does the document review add value?
The document review:
- Informs the assessment of the micro-indicators to be analysed through a qualitative analysis of each multilateral organisation’s documentation
- Allows for triangulation of findings
- Provides additional contextual information on the multilateral organisations in the four dimensions of organisational effectiveness assessed in the MOPAN Common Approach
- Helps illustrate the level of transparency on the multilateral organisation in question.
The review will use publicly available documents. It will be conducted on all indicators where a document review is considered relevant.
What are the strengths of the MOPAN Common Approach?
The MOPAN Common Approach has many strengths, including:
- It is based on the core elements of existing bilateral assessment tools
- It seeks information from two different perspectives: MOPAN members (both at headquarters and country level) and direct partners of multilateral organisations (in line with the commitments made by donors to the Paris Declaration on Aid Effectiveness and the Accra Agenda for Action regarding harmonisation, partner voice and mutual accountability)
- It uses a mix of quantitative and qualitative information (through closed and open-ended questions) which strengthens the accuracy of measurement and provides a basis for discussion about improving the organisations’ effectiveness
- It triangulates the data through complementing perceptual data from the survey with a document review and consultations, thus adding additional data sources and increasing the validity of the assessment
- It compares findings with other sources where possible (for example results of the surveys on monitoring the Paris Declaration)
- It is customised to take into account the differences between different types of multilateral organisations
What are the limitations of the MOPAN Common Approach?
The MOPAN Common Approach:
- Allows network members and the multilateral organisations assessed to choose the most relevant individuals to complete the survey, but there is no way to know whether the most qualified ones complete the survey
- Triangulates the data collected with consultations and a document review but while a document review can comment on the contents of a document, it cannot assess the extent to which the spirit of that document has been implemented within the organisation (unless implementation is documented elsewhere.
- Produces numerical scores which suggest a high degree of precision but these scores only provide a picture of effectiveness in priority areas—they do not provide deep insight into the different dimensions of organisational effectiveness because the approach is based on a perceptions survey and not on an analysis of behaviours, systems and procedures
Donors are pressing for ‘Delivering as One’ countries so why assess United Nations (UN) agencies individually?
Each UN agency has a different remit and mandate which make it possible to assess them individually under this common assessment approach. The MOPAN Common Approach allows members to obtain information that is not otherwise available. MOPAN members are also actively following progress on the operationalisation of UN reform initiatives.
What is the link between the MOPAN Common Approach and the performance frameworks being developed between some MOPAN members and UN agencies?
The performance frameworks set the medium-term strategic direction for the partnership between a MOPAN member and a UN multilateral organisation. They link the MOPAN members’ core funding to the attainment of the performance targets included in the UN agencies’ strategic plan.
Performance frameworks differ from the MOPAN assessment, which is not directly linked to funding, and are geared towards providing insights on how targets could be set and achieved.
Findings
How are the MOPAN Common Approach findings used?
It is imperative for donor countries that multilateral organisations deliver effectively and are ready to meet the challenges they face in delivering aid. MOPAN Common Approach findings highlight areas of strength and pinpoint areas for improvement. The findings are discussed internally in each multilateral organisation and also among members of the network—dialogue is a key feature of MOPAN’s strategic approach.
Each MOPAN member decides how best to use the findings inside their own organisation. In general, they are used:
- To build better understanding of multilateral organisations’ effectiveness
- To support discussion between developing country partner governments, bilateral donors and multilateral organisations, as part of ongoing processes to strengthen mutual accountability at country level (discussions could include exploring how bilateral partners can better support multilateral organisations in-country)
- As input to policy development
- To support the steering of, and participation in, the governance of the multilateral organisations (for example, at a board or governing body meeting)
- To strengthen relationships with multilateral organisations at country level
- As input to wider debates about multilateral organisations’ effectiveness.
What is the ‘Country Dialogue Process’?
After publication of the Institutional Reports the country-specific findings of the MOPAN assessment are presented and discussed with key stakeholders at country level in a meeting, a workshop or there may be an existing forum, which could be used to this effect. This dialogue process with the assessed organisations, their partners and MOPAN members present in the country is initiated and led by the MOPAN Country Leads. They have the flexibility to use whatever format they deem most appropriate for their specific country context.
What does MOPAN hope to achieve from the Country Dialogue Process?
The dialogue process provides the opportunity for MOPAN members in-country to present and discuss the key findings of the MOPAN country data summaries with relevant development actors. The findings of the MOPAN Common Approach assessment provide an evidence base, which can be used to promote dialogue between MOPAN members, multilateral organisations and their direct partners, with the aim of improving multilateral organisations’ effectiveness. MOPAN also aims to build consensus about the actions required at the country level to improve multilateral organisations’ effectiveness. The meetings or workshops with the organisations and direct partners at country level play a key role in achieving these aims.
In terms of ranking, is there a top performer every year?
Due to the different remits of the organisations each year, MOPAN does not rank the performance of individual multilateral organisations. Rather, it uses the findings of its work to encourage discussion among MOPAN members and multilateral organisations about lessons learned and ways to further build their capacity to be effective. The Common Approach is meant to assess each multilateral organisation on its own merit, and therefore should not be used as a tool to compare one organisation to another.
Why does the MOPAN Common Approach not compare multilateral organisations?
It is not possible to compare multilateral organisations to one another—their mandates and structures vary too much in scope and nature. However, because the MOPAN assessment is repeated at intervals of time it can help determine whether the multilateral organisation’s effectiveness in the areas covered by the survey is changing over time.
Where can I get a copy of the MOPAN 2012 Institutional Reports?
The Institutional Report for each organisation assessed in 2012 and those of previous years are available on MOPAN’s website (www.mopanonline.org) under Publications and also on most MOPAN members’ websites.
December 2012