Introduction
The purpose of this toolkit is to make lessons learned from the ‘SDG 16 National Monitoring Initiative’ available to all interested actors. Between 2017 and 2021, the initiative supported more than ten countries in monitoring SDG 16 using a three-step methodology. Initiated by UNDP, the pilot initiative later became part of the Global Alliance for Reporting Progress on Peaceful, Just and Inclusive Societies and was taken forward in partnership with OHCHR, UNESCO, UNHCR, UNODC, UN Women, the UN Global Compact, TAP Network, and law firm White & Case. The toolkit offers examples, lessons and tools around the three-step methodology as well as case studies, FAQs and contacts.
Background
When the SDG 16 National Monitoring Initiative launched in 2017, guidance on how to monitor SDG 16 indicators was scarce: Many SDG 16 indicators were not fully developed yet (so-called ‘Tier III’ indicators) and UN custodian agencies were only beginning to develop the required guidance with the UN Statistical Commission.
The initiative was thus developed with the aim to ‘boost commitment to monitoring, and provide assistance with some standardization of an effective, multi-stakeholder monitoring of SDG16 at the national level, while respecting national policy space’.
The basis of the initiative was its three-step methodology to help national stakeholders monitor SDG16: (1) Data & Indicators, (2) Stakeholder Engagement, and (3) Scorecards. The methodology proved useful for countries preparing for a future or building on a past Voluntary National Report (VNR), but it was considered equally (or even more) important for countries that were not yet planning a VNR.
The initiative evolved in three phases:
- Start-Up (2017): UNDP, in partnership with the Open Government Partnership (OGP) and the Community of Democracies (DoC), engaged pilot teams in seven countries - El Salvador, Georgia, Indonesia, Mexico, South Africa, Tunisia, and Uruguay. Financial support was received from USAID. Mexico hosted a meeting of all pilot counties to exchange experiences and a first pilot report was published.
- Scale-Up (2018/2019): Nine additional countries joined - Argentina, Burkina Faso, Cape Verde, Central African Republic (CAR), Colombia, Lebanon, Moldova, Mongolia, and Timor Leste. It provided countries with technical support and facilitated peer exchange among and between the first and the second cohort of countries. The offer included technical webinars, connecting countries with similar needs, facilitating participation in key global meetings and an in-person workshop among Francophone countries hosted by Tunisia.
- Global Alliance (2020/2021): The initiative was integrated in the Global Alliance for Reporting Progress on Peaceful, Just and Inclusive Societies to offer pilot countries the tools and expertise of five UN agencies (OHCHR, UNESCO, UNHCR, UNODC, UN Women)and other members of the Global Alliance (e.g. UN Global Compact, TAP Network, law firm White & Case). Financial support was received from the UK. While the Global Alliance members were pooling their tools and experts, the Covid-19 pandemic hit and shifted priorities across all countries and involved organisations. Despite the challenges, three pilot countries were technically and financially supported throughout - Tunisia, Mexico, Moldova.
After four years of piloting, the initiative’s Steering Group considered it timely to take stock and make the initiative’s methodology and learning accessible to a broader audience. The objective of this toolkit is thus to
- Provide guidance that can be accessed and used by any interested actor (learning)
- Signpost how the three-step methodology is linked with and can complement other relevant UN support (coherence)
- Enable countries to connect with previous pilot countries or the UN agencies that have advised the initiative for peer-to-peer learning and expert advice (sustainability).
The toolkit seeks to be useful for a broad target audience, including
- Government focal points for national SDG 16 coordination (e.g. development planning commission; ministry of Justice);
- National Statistical Offices;
- UNCT and UN agency focal points for SDG monitoring support;
- National human rights institutions, supreme audit institutions, parliaments and other accountability institutions;
- Civil society and other non-state actors;
- Other development partners including donors.
More than five years into the SDGs, the situation has greatly improved. Overall, it has become clear that measuring progress on SDG 16 is politically, technically and financially possible.
- Technical clarity
The UN’s Statistical Commission, more specifically its Inter-Agency and Expert Group on Sustainable Development Goals Indicators (IAEG-SDGs), has made enormous progress in developing the global SDG 16 indicators. Each SDG indicator has a designated UN ‘custodian’ agency which is responsible, inter alia, for methodological development and ongoing refinement of the indicator; collecting data from national statistical systems and UN regional commissions; providing metadata for the indicator; and contributing to statistical capacity building. SDG 16 has no ‘Tier III’ indicators anymore which means all indicators are now conceptually clear, have an internationally established methodology and data is being collected. See an overview of SDG 16 indicator tier classification and custodian agencies here. - Practical and political feasibility
The annual VNR analyses by law firm and Global Alliance member White & Case indicate that reporting on SDG 16 is becoming more feasible and politically accepted or even expected: While in 2016, only 10 of the 22 (45%) VNR countries reported or made some reference to SDG 16 indicators, in 2020, it was 41 out of 46 (89%) countries that included reporting on SDGs 16. In general, the 2019 and 2020 VNRs provide more data analysis and include a broader list of data sources, including on SDG 16, and countries reporting for the second time in 2019 and 2020 provided a more comprehensive review. - SDG 16 as an enabler
SDG 16 is not only an important goal in itself, it also contains key principles, such as accountability, transparency, participation and inclusion, that – if applied - facilitate progress on all SDGs. The VNR analyses by White & Case indicate that an increasing number of countries are trying to follow these principles and leverage this enabling effect: For example, the degree to which governments involve civil society in the preparation of their VNR has increased: While in 2016, only five countries confirmed to have prepared their VNR report in collaboration with civil society with most others focusing on awareness raising and communications efforts, the engagement of civil society and other stakeholders is more prominent in 2019 and 2020 VNR. For example, in 2020, 43 out of 45 reporting Member States stated that their VNRs were the result of an inclusive and participatory process with various stakeholders. This is despite the COVID 19 pandemic making this process more challenging in 2020. The 2019 and 2020 VNR reports show important contributions from civil society, including some drawing on consultations with persons with disabilities and the LGBTI community. In addition, most 2020 VNRs discussed human rights throughout and some explicitly linked the VNR to human rights reporting, including Universal Periodic Review (UPR) to SDG 16.
The VNR analyses by White & Case reveal remaining gaps around monitoring and reporting on SDG 16, especially the lack of reliable and disaggregated data. This seems especially due to lack of resources for regular data collection, limited capacities and resources within the national statistical system and restricted access to official data kept by government institutions. These challenges have been further exacerbated by the COVID-19 pandemic , which has disrupted routine operations for data production throughout the global statistical and data system, with delays in planned censuses, surveys and other data programmes.
Looking ahead, the 2019 Global Alliance SDG 16+ report , updated in 2021 , recommends that to enhance a country’s ability to gather, analyse and use data on SDG 16+ targets and indicators, the following is required:
• political and financial investments in SDG 16+ data and statistics , including development of methodologies and standards to ensure the measurability and comparability of reported data beyond existing SDG 16 indicators such as the Praia Group on Governance Statistics
• Data disaggregation to ensure no one is left behind;
• Partnerships between national statistical systems and regional and local governments, civil society organizations, youth organizations and academia as well as the private sector and international organizations in data collection, analysis and policy making.
• The initiative’s three-step methodology has proven a useful tool for countries to monitor and report on their SDG16 commitments: Pilot countries found the methodology helped organize the process of establishing a national SDG16 monitoring system into a coherent and clear sequence of activities. Already in its first year, some pilot countries mentioned in their VNRs that the initiative helped them report on SDG 16, e.g. El Salvador 2017 and Uruguay 2017.
• Despite progress on monitoring SDG 16, the three-step methodology is still relevant: Regarding Data & Indicators (Step 1), global indicators have now been fully developed and more and more data is being collected (see above) but the challenge to align national indicators with global SDG 16 indicators at different levels and across different processes persists. In terms of Stakeholder Engagement (Step 2), coordination, consultation and collaboration to monitor progress is now more accepted and also explicitly recommended by the UN Secretary General’s voluntary common reporting guidelines on VNRs but is still not undertaken systematically across countries. The COVID-19 pandemic has led to practical challenges and, in some countries, dwindling political will for stakeholder engagement. Scorecards (Step 3) and other ways of visualizing data and institutionalizing monitoring are critical to ensure accountability and have become much more common but remain unsystematic, especially beyond VNRs, partly due to lack of technical and financial resources.
• The three ‘steps’ or ‘phases’ of the pilot methodology are not necessarily sequential. Which ‘step’ a country starts with depends on where they are in the process of monitoring SDG 16 or where there is specific technical or political demand. The second step, stakeholder engagement, should ideally take place at the same time (and support) the first and the third step. More importantly, the pilot experience shows that, no matter the sequence, it is critical to work on all three steps at some point as they are mutually reinforcing : For example, stakeholder engagement makes data identification and scorecards more effective and relevant; scorecards facilitate stakeholder engagement and help refine data identification and indicator refinement; and data identification and indicator refinement can be key to trigger stakeholder engagement and the development of scorecards.
• The ultimate goal of monitoring is policy development and implementation. When designing indicator frameworks and associated data collection strategies, and when filling out national scorecards, stakeholders should not lose sight of the end goal: SDG16 data should trigger action by policymakers and, eventually, tangible improvements in people’s lives. The ‘making use of monitoring data to design and implement policies’ could thus be considered a fourth step or the overall objective of the initiative’s methodology.
On piloting more broadly…
• The use of a pilot methodology such as this one is most effective when embedded in ongoing national processes (e.g. drafting, implementation and monitoring of a National Development Plan or relevant sectoral strategies, or the preparation of a VNR) and owned by key national partners , who are involved and invested to continue the monitoring process. Who those partners are and which responsibilities and capacities they have varies across countries. For example, National Statistical Offices (NSO) are critical but other institutions, e.g. those that hold administrative data, are important too. These actors tend to have established working relationships and need to find the methodology useful to advance their joint work.
• Peer-to-peer exchanges among pilot countries are important. Opportunities for pilot countries to meet, either in person (e.g. in Mexico in 2017 and in Tunisia for Francophone countries in 2019) or virtually (e.g. though technical webinars in 2018 or in initial ‘virtual coffee chats’ in 2021), provided pilot countries with valuable opportunities to meet and learn from each other and to “import” relevant approaches to their national settings.
• Financial sustainability is required. The initiative made it clear that even a small amount of catalytic funding of USD 20-50,000 and technical support can go a long way and kick-start systematic monitoring of SDG 16. The sustainability of the national SDG16 monitoring architecture established by this pilot initiative, however, requires strong multi-year financial commitments from national sources. In line with the principles of national ownership and data sovereignty promoted by the 2030 Agenda, national resources will need to be mobilized to ensure that the necessary investments in national SDG16 data production are made early on. Private sector actors could also be approached in this regard, in view of their vested interest in supporting efforts to improve the governance climate in which they do business, as long as data protection and privacy are safeguarded.
Methodology
When the SDG 16 National Monitoring Initiative launched in 2017, guidance on how to monitor SDG 16 indicators was scarce: Many SDG 16 indicators were not fully developed yet (so-called ‘Tier III’ indicators) and UN custodian agencies were only beginning to develop the required guidance with the UN Statistical Commission.
The basis of the initiative was its methodology to help national stakeholders monitor SDG16 in three (reinforcing but not necessarily sequential) steps:
- Step 1: Definition of indicators and baseline data collection. In consultation with national statistical offices (NSOs) and drawing from international SDG 16 data platforms and national (official and non-official) data sources.
- Step 2: Multi-stakeholder consultations and review of progress. Joint review by government and civil society of the proposed indicator framework and of indicator results, and joint formulation of broad policy recommendations.
- Step 3: Periodic scorecards. Periodic tracking of progress using the selected indicators, identifying and addressing data gaps, and formulating specific policy recommendations for each target.
The methodology was adapted to suit each country’s context, needs and experience with governance monitoring.
- For instance, Indonesia and Tunisia built on their previous experiences with illustrative measuring of governance, initiated in the run-up to the adoption of SDG 16 (2014-15). While the methodology proved useful for countries preparing for a future or building on a past Voluntary National Report (e.g. Mexico, El Salvador, Uruguay),it was considered equally (or even more) important for countries that were not yet planning a VNR. Some of the countries applied or complemented the methodology at the local level (Tunisia, Argentina) or combined it with the monitoring of other national plans or strategies. Some are considering the application of the three-step methodology for the monitoring of other SDG indicators (Senegal). Draft ToR to apply the methodology can be found here.
The following guidance, lessons learned, examples and tools seek to illustrate the possible use of the pilot methodology. Some of the examples are not the direct result of use of the pilot methodology but were still considered useful to illustrate the relevance of the three steps. It is also important to note that the initiative has not been formally evaluated so the information should thus be considered as experiences rather than validated ‘best practice’.
Country Case-Studies
- Examples of country-level experiences in implementing the methodology, showing how the three steps come together in practice
- Contact information for each case-study
More Information
About the SDG 16 National Monitoring Initiative…
What is the SDG 16 National Monitoring Initiative?
The SDG 16 National Monitoring Initiative (NMI) was a pilot initiative to build and strengthen inclusive and robust SDG 16 monitoring frameworks at the national level. Between 2017 and 2021, the initiative supported more than ten countries in monitoring SDG 16 using a three-step methodology. Conceived by UNDP, the pilot initiative later became part of the Global Alliance for Reporting Progress on Peaceful, Just and Inclusive Societies. More information on the initiative, see here. More information on the methodology see here.
Which pilot countries have participated in the initiative?
Start-up Phase: El Salvador, Georgia, Indonesia, South Africa, Tunisia and Uruguay plus Mexico
Scale up Phase: Argentina, Colombia, Burkina Faso, Cape Verde, Central African Republic, Lebanon, Mongolia, Moldova, and Timor Leste.
Over the years, other countries (‘self-starters’) have sought advice on the methodology and have drawn on it using their own resources, e.g. Uganda and Senegal.
How can I find out more about efforts in specific pilot countries?
This toolkit includes examples, lessons learned and tools from different countries along the different steps of the pilot methodology. Full case studies are available for Mexico, Tunisia, Moldova and Senegal.
About the three-step methodology of the SDG 16 National Monitoring Initiative…
What is the 3-step methodology of the SDG 16 National Monitoring Initiative?
The three step methodology was conceived by UNDP in 2017: (1) refine indicators and collect baseline data on SDG 16; (2) engage stakeholders, including civil society, private sector, academia and others in monitoring of SDGs 16; and (3) establish scorecards to ensure periodic tracking of progress, identifying and addressing data gaps, and formulating specific policy recommendations for each target. The methodology proved useful for countries preparing for a future or building on a past Voluntary National Report (VNR), but it was considered equally (or even more) important for countries that were not yet planning a VNR.
More information on the methodology see here.
Should the three steps be applied sequentially?
The three ‘steps’ or ‘phases’ of the pilot methodology do not necessarily build on each other sequentially. Which ‘step’ a country starts with depends on where they are in the process of monitoring SDG 16 or where there is specific technical or political demand. The second step, stakeholder engagement, should ideally take place at the same time (and support) the first and the third step. More importantly, the pilot experience shows that, no matter the sequence, it is critical to work on all three steps at some point as they are mutually reinforcing: For example, stakeholder engagement makes data identification and scorecards more effective and relevant; scorecards facilitate stakeholder engagement and help refine data identification and indicator refinement; and data identification and indicator refinement can be key to trigger stakeholder engagement and the development of scorecards.
Is the three-step methodology still relevant now that SDG monitoring and reporting has advanced?
Despite progress on monitoring SDG 16, the three-step methodology is still relevant: Regarding Data & Indicators (Step 1), global indicators have now been fully developed and more and more data is being collected (see above) but the challenge to align national indicators with global SDG 16 indicators at different levels and across different processes persists. In terms of Stakeholder Engagement (Step 2), coordination, consultation and collaboration to monitor progress is now more accepted and also explicitly recommended by the UN Secretary General’s voluntary common reporting guidelines on VNRs but is still not undertaken systematically across countries. The COVID-19 pandemic has led to practical challenges and, in some countries, dwindling political will for stakeholder engagement. Scorecards (Step 3) and other ways of visualizing data and institutionalizing monitoring has become much more common but remain unsystematic, especially beyond VNRs, partly due to lack of technical and financial resources.
Who can use the 3-step methodology and for what purpose?
The three-step methodology has been designed to be used primarily by UN Member States, which are the main parties responsible for implementation of the 2030 Agenda for Sustainable Development. The implementation of the 2030 Agenda and its SDGs, including SDG 16, requires continuous monitoring of progress, based on reliable and transparent data and sources of information. This methodology helps countries to use a structured framework based on key principles of the 2030 Agenda such as accountability, transparency, participation and inclusion. The methodology has proved useful for countries preparing for a future or building on a past Voluntary National Report (VNR) at the annual High-level Political Forum on Sustainable Development (HLPF) but it is considered equally (or even more) important for countries that are not yet planning a VNR to ensure that some form of monitoring and reporting on SDG 16 is taking place. The ultimate goal of the methodology is to help national decision-makers develop and implement policies on issues addressed by SDG 16 and to tangible improve people’s lives.
Key government actors who will find the methodology useful include focal points for national SDG 16 coordination (e.g. development planning commission, Ministry of Justice) and National Statistical Offices.
The methodology is also a useful tool for
- UN Country Teams and UN agency focal points for SDG monitoring to support Member States in their efforts to implement and monitor SDG 16.
- National Statistical Offices (NSOs)
- National human rights institutions, supreme audit institutions, parliaments and other accountability institutions;
- Civil society and other non-state actors;
- Other development partners including donors
Have all pilot countries gone through all three steps?
Some countries have gone through all steps, with different degrees of intensity or progress. Several started with Step 1, some with Step 2 and are progressing on the respective other two steps. Especially in 2020, the COVID-19 pandemic impacted the activities planned by the pilot countries in relation to different steps, particularly Step 1 (e.g. difficulties with data collection) and Step 2 (e.g. challenges to engage stakeholders in the usual ways).
How can I get further guidance on the steps of the methodology?
Please see themethodology sectionof this toolkit.
About SDG 16…
What are the official global SDGs 16 indicators and targets?
Sustainable Development Goal 16 includes 12 targets and 24 indicators.
Where can I find information about my country reporting on SDGs 16?
The UN Statistical Commission’s SDG Global Database provides access to data on more than 210 SDG indicators for countries across the globe by indicator, country, region or time period. It includes data on SDGs 16 and the possibility to create country profiles.
Some UN Regional Economic Commissions are developing regional databases that allow for country profile and sometimes include more information beyond the SDGs:
About SDG data and indicators…
What are global, proxy and national indicators?
Global SDG indicators: Those officially adopted by the United Nations Statistical Commission and classified in three tiers, depending on their state of development. For a global indicator, a country cannot change the language and must follow the metadata provided to produce data for the indicator. Differences caused by e.g. different methodologies or definitions, invalid or inconsistent entries can cause confusion and can also damage the credibility of statistical agencies.
Proxy indicators: These are alternative national or global measures. Proxy indicators are sometimes used to complement or supplement the global SDG indicators in countries’ VNR reporting. Some proxy indicators are internationally comparable, some are not. In any case, proxy indicators cannot be used for official reporting on the SDGs to the UN Statistical Division.
National indicators: These are developed by governments and often draw on the M&E system of an existing National Development Plan (NDP) or other policy documents. National indicators and their tracking methodologies are often specific to one country and use data from the national statistical system or by non-official data producers such as civil society, research institutions or the private sector (see below). Unlike global indicators, they can be difficult to compare across countries. While they are sometimes used for countries’ VNR reporting, they cannot be used for official reporting on the SDGs to the UN Statistical Division.
What are common types of data?
Administrative data: Administrative data is information generated by line ministries and regulatory authorities of the Government, e.g. civil registration systems, national population registers and other administrative records systems.
Statistical survey data: Statistical surveys are used to collect direct quantitative and qualitative information on population subsets. Household surveys are especially important for many of the SDG 16 indicators if they gather popular perceptions and experiences around governance issues
Census data: A census is a complete enumeration of all members of the population of a country or territory, unlike statistical surveys, where only selected members of the population are surveyed.
Data from opinion and perception polls: Perception and opinion polls aim at polling a representative sample of individuals for their personal views on a given issue. Polls are often led by private polling companies, academia and civil society.
Expert surveys: These are perception surveys of experts. They are often used to get data for governance indicators since they are a cheaper and more efficient data source than the above and can be used to measure more sensitive governance performance issues. However, they are sometimes challenged as being too subjective or because they cannot be disaggregated.
What is official and non-official data?
Official data: Data produced by a country’s national statistical system (NSS) which is coordinated by the county’s national statistical office (NSO). The national statistical system is often defined by legislation and includes mostly the statistical units of government and other state entities. It can include non-governmental entities if it is officially recognized as such. Another way how non-governmental data can become ‘official’ is through partnership or collaboration of the NSS with a non-governmental entity (see: “How to use official and non-official data sources?”).
Non-official data: Data produced by entities that are not part of the national statistical system, i.e. mostly by the private sector, NGOs, academia or international organizations.
About the Global Alliance for Reporting Progress on Peaceful, Just and Inclusive Societies…
What is the ‘Global Alliance’?
The Global Alliance for Reporting Progress on Peaceful, Just and Inclusive Societies was a coordinating platform for UN Member States, private sector, civil society and UN Agencies to strengthen evidence-based reporting, monitoring and measurement of SDG16.
It consisted of a Steering Committee composed of a group of States (Cabo Verde, Italy, Mexico, Norway, Sierra Leone (last Co-chair), Tunisia, Qatar and United Kingdom (last Co-chair), civil society (Pathfinders, WFUNA, TAP Network) and private sector (Deloitte, Lexis-Nexus, White & Case) and co-facilitated by a Secretariat composed of UN agency members (UNDP, UNODC, UN Women, UNESCO,OHCHR, UNHCR), and UN Global Compact.
A key resource on SDG 16 reporting are the annual VNR analyses by Global Alliance member White & Case, see 2016, 2017, 2018, 2019, 2020.In early 2020, the National Monitoring Initiative was integrated into the Global Alliance to offer pilot countries the tools and expertise of its six UN agencies and other members. A Steering Group provided guidance to pilot countries and was comprised of representatives from UNDP, UNESCO, UNHCR, UNODC, UN Women, OHCHR, UN Global Compact, TAP Network, White & Case).
The Global Alliance concluded its activities at the end of 2021.
To learn more about work in any of the former pilot countries or to seek guidance from any of the former Global Alliance Steering Group members (see below), please contact:contact@sdg16hub.org.
(UNDP)
|
Argentina |
Nora Luzi |
|
Burkina Faso |
Losseni Cisse, Isabelle Tschan |
|
Colombia |
Jairo Matallana |
|
Cape Verde |
Adelaide Ribeiro, Elisabete Mendes |
|
CAR |
Youssoufa Silla, Antje Kraft |
|
El Salvador |
Laura Rivera |
|
Georgia |
Nino Kakubava, Salome Odishari |
|
Indonesia |
Siprianus Soro |
|
Lebanon |
Fadi Abilmona |
|
Mexico |
Carlos Cortes-Zea |
|
Moldova |
Dumitru Vasilescu |
|
Mongolia |
Barkhas Losolsuren, Munkhtuul Batbaatar |
|
South Africa |
Bongani Matomela |
|
Timor Leste |
Bruno Lencastre |
|
Tunisia |
Eduardo Lopez-Mancisidor |
|
Uruguay |
Virginia Varela |
Global contacts (Global Alliance Steering Group members)
|
OHCHR |
Marc Cebreros |
|
UNDP |
Mariana Neves |
|
UNESCO |
Jaco du Toit |
|
UNHCR |
Jason Pronyk |
|
UNODC |
Angela Me |
|
UN Women |
Chiao-Ling Chien |
|
UN Global Compact |
Michelle Breslauer |
|
TAP Network |
John Romano |
|
White & Case |
Laura Garr |