Humanitarian action provides a lifeline to people during some of the hardest moments of their lives. And for those who have experienced the worst that conflict and disaster can inflict, it can provide a flickering reminder of humanity. People can die even in the most effective humanitarian responses, including those who risk their lives to deliver them. For many others, humanitarian assistance becomes a mainstay, shaping their lives and opportunities over decades. With such high stakes, humanitarians have an obligation to learn and improve. This applies equally to the largest international agencies and the smallest local civil society actors.

ALNAP’s State of the Humanitarian System report (SOHS) supports this learning by monitoring changes in the shape and size and performance of the international humanitarian system. The SOHS is a unique, independent longitudinal study that gathers and synthesises evidence to form a picture of the system and assess how well it meets the needs of people affected by crises. There have been four previous editions of the report, beginning with a pilot study in 2010. This fifth edition covers the period from January 2018 to December 2021 inclusive.

The last edition of the SOHS, published at the start of 2018, charted a three-year period during which the humanitarian system made a set of high-profile commitments to change; the World Humanitarian Summit and Grand Bargain on humanitarian financing followed a suite of global summits on climate change, disaster risk reduction and sustainable development. In the four years covered by this 2022 edition of the SOHS, these high-level intentions were stress-tested against the realities of implementation as they faced both the constraints of the humanitarian system and the challenges of turbulent crises – not least those stemming from the COVID-19 pandemic. 

Objectives and approach

This report explores how organisations managed to deliver against the promise of humanitarian action during this period. We aim to answer the following questions, focusing on the evidence in the study period and drawing on previous editions of the SOHS to take a long view:

  • What was the context and need for humanitarian action?
  • What was the size and shape of the humanitarian system?
  • How has the humanitarian system performed?

These objectives are consistent with previous editions of the SOHS, which allows us to make comparisons over time. Our approach, however, has evolved in three important ways:

  1. Looking outside the formal system: The international humanitarian system is just one of many sources of support for people and communities in crisis. This report seeks to better recognise this wider array of resources, dedicating a section to them in the SOHS for the first time (‘Focus on: Support beyond the system’ section). Chapter 10 also Chapter 9 addresses the effectiveness of the international humanitarian system in supporting local and nationally led action. 
  2. Deepening the participation of affected people in SOHS research: In assessing humanitarian action, the SOHS has always relied on interviews and focus group discussions with recipients of assistance. This time, for the first time, we consulted aid recipients on the design of the research itself, asking people in three different response contexts what should be included in the SOHS assessment. On the basis of their views, we adjusted our preliminary research questions and data collection plans, which led to greater emphasis in this edition on their priority issues: targeting, corruption, do-no-harm and accountability to affected populations.
  3. Presenting analysis in a policy- and practice-relevant format: Previous editions of the SOHS have been organised around the evaluation criteria set out by the Development Assistance Committee of the Organisation for Economic Co-operation and Development (OECD DAC) (see Box A). These criteria provide a useful framework but a narrow perspective on humanitarian performance. Some stakeholders for the SOHS report, such as local NGOs and affected communities, suggested that the system was assessing itself on its own terms, while others, such as headquarters-based humanitarians, felt the criteria did not fully capture the issues that were most pressing for them. In response to this feedback and in line with other humanitarian evaluations – which remain a key source of evidence for the SOHS – we present this edition’s findings as answers to a core set of policy and practical questions. A longitudinal overview of system performance against the DAC criteria is also provided in the conclusion of the report.

Scope and method

Definitions

Humanitarian action is the principled provision of assistance and protection in order to save lives, prevent and reduce suffering and preserve people’s dignity in crises arising from armed conflict, hazards and other causes2. Humanitarian action is international when these activities involve resources (financial, technical or in-kind) provided by sources in one country to respond to a crisis in another. International humanitarian action excludes responses that are fully resourced within the country experiencing the crisis, which fall within the domain of domestic crisis management.

The SOHS study team adopts a working definition of the international humanitarian system as:

"The network of interconnected institutional and operational entities through which humanitarian action is undertaken when local and national resources are, on their own, insufficient to meet the needs of a population in crisis."

These entities are operationally or financially related to each other and share common overarching goals, norms and principles. The international humanitarian system is international in the sense that it is cross-border, and humanitarian in the sense that at least one actor involved in its funding or delivery self-identifies with the goals, norms and principles of humanitarianism. These entities may be funded by governments as well as private individuals, and include national and international NGOs conducting humanitarian activities; UN humanitarian agencies; the International Red Cross and Red Crescent Movement; host government agencies and authorities; regional intergovernmental agencies; multilateral agencies; government aid agencies; and other offices that provide humanitarian funding and coordination.

Figure 1: Inside and outside the international humanitarian system – the entities involved in humanitarian action


The international humanitarian system is comprised of entities that accept international funding and identify with humanitarian norms or principles. They operate in a wider context of other sources of support for crisis-affected people.


Source: ALNAP.
Note: The size of the circles in this visualisation are not to scale and are therefore not representative of each entity’s role or importance in the system.

Research components and methods

Findings are drawn from 10 research components, using a combination of primary data collection and secondary data synthesis. Data collection across the components was integrated using a shared research framework, which can be found in Annex 3.

  Primary data collection and analysis
Country-level research: Focus group discussions (FGDs) and key informant interviews (KIIs), along with gathering relevant context-specific documentation and observations, were conducted in Bangladesh (Cox’s Bazar), Democratic Republic of the Congo (DRC), Ethiopia, Lebanon, , Venezuela and Yemen.
Country studies on localisation:

Two country-level studies, commissioned by ALNAP, on localisation in Turkey and Somalia, featuring surveys and in-depth interviews with local and international actors.
Aid recipient survey: A survey of 5,487 aid recipients in six crisis contexts, using SMS and computer-assisted telephone interviewing (CATI)to elicit their assessment of humanitarian performance.
Practitioner and host government survey: A global web-based survey, with 436 completed responses, to elicit the perceptions of humanitarian practitioners and host-government representatives on humanitarian performance.
Key informant interviews:

Interviews with humanitarian practitioners and thought leaders to assess performance, identify important trends and address key evidence gaps.
Organisational mapping and analysis: Data collected from individual organisations and through desk-based review to estimate the number of humanitarian staff and organisations worldwide.
Thematic research:

Two original studies, commissioned by ALNAP, to assess the state of evidence on mortality in humanitarian settings and to understand the impact of innovation funding.

 

  Synthesis of secondary data
Evaluation synthesis: A synthesis of findings from humanitarian evaluations published between 2018 and 2022 in the ALNAP HELP library (add link). Over 500 humanitarian evaluations were assessed for inclusion with over 130 evaluations chosen for more in-depth analysis.
Financial analysis:

ALNAP worked with experts in humanitarian financing to analyse data and produce statistics on humanitarian financing.
Literature review:

ALNAP reviewed over 250 research reports and academic work published within the study period on a set of 15 themes related to humanitarian policy and practice.


Using approaches common to mixed method studies,3 the SOHS study team identified general trends and findings through a consideration of frequency, quality and triangulation across research components. To rigorously synthesise such a large volume of variable data, the research components shared a coding framework and the analysis process used iterative hypothesis testing and iteration. Emerging data was shared and gaps identified in routine meetings with the component leads – enabling hypotheses to be developed and tested, and further data collection to be targeted to confirm or disconfirm these. The full methodology for each component and the synthesis is in Annex 3.

Report limitations and pervasive data gaps

Identifying general trends and findings for so many humanitarian responses over a four-year period is inherently challenging, particularly given the fact that no sample can truly be considered ‘representative’ of the entirety of such a vast system. And even when using a shared indicator framework, it is difficult to avoid the problem of data comparability that is common to mixed method approaches.4

The foreword to the 2010 SOHS pilot study notes that, ‘Almost as important as what the report says, is what it does not say’.5 Persistent and pervasive data gaps continue to limit this report’s ability to provide clear, definitive assessments on key performance issues – such as how many people humanitarian assistance reaches, whether humanitarian action saves lives and protects people, and how cost-effective responses are. For this edition of the report ALNAP went to new lengths to locate or generate this data, but it is clear that addressing these gaps requires system-wide resources and effort beyond what can be achieved for a single research project – even one as long running and large in scope as the SOHS.

There have been repeated calls for the humanitarian system to improve its evidence base, in each edition of the SOHS and by many others the system.6 The stretch on limited humanitarian funding described in this report is likely to mean a continued deprioritisation of knowledge production, monitoring and evaluation and data quality and accessibility. This is at the system’s own peril. Better evidence could not only guide more effective improvements to performance, but also help to demonstrate the system’s value in the context of global economic strains and rising costs of crises.

Report structure

The report begins with a framing chapter, which looks at how the context for humanitarian action has changed over the period studied. The analysis of the system’s changing shape and performance is then presented in three parts:

Part I: What is the system?
Describes the size, shape and resources of the humanitarian system, and gives an overview of support available to crisis-affected people outside this system. It examines the financial resources available for meeting humanitarian need over the study period, and whether these were sufficient.

Part II: What is it achieving?
Assesses the humanitarian system on how it is meeting a core set of expectations for its performance: whether it reaches the ‘right’ people, whether it offers the right forms of support, whether it works and whether it inadvertently causes harm.

Part III: How is it working?
Summarises the evidence on how the system is improving its ways of working, including its relationships with others. It covers how humanitarian actors treat crisis-affected people, how internationals relate to local actors, how the system connects with development and peace actors, how it upholds humanitarian principles, and how well it uses resources.

A series of focus studies are included throughout the report. These look at major global challenges to the humanitarian system – including conflict, hunger, protracted crises, forced displacement and links to wider crisis support – and summarise how the system has responded. The studies draw on evidence from across the report, additional research and original country case studies conducted for the SOHS by local researchers.

The conclusion summarises changes in the system’s performance against the OECD DAC criteria and how this has shifted since previous reports. It draws out implications from the report’s findings for the system’s ability to meet future crises and challenges.

Box A: The SOHS performance criteria, adapted for use from the OECD-DAC criteria used in humanitarian settings (ALNAP/Beck, 2006)* 

Sufficiency:* The degree to which the resources available to the international humanitarian system are sufficient to cover humanitarian needs. 

Coverage:** The degree to which action by the international humanitarian system reaches all people in need. 

Relevance and appropriateness:** The degree to which the assistance and protection that the international humanitarian system provides addresses the most important needs of recipients (as judged both by humanitarian professionals and by crisis-affected people themselves). 

Accountability and participation:* The degree to which actors within the international humanitarian system can be held to account by crisis-affected people, and the degree to which crisis-affected people are able to influence decisions related to assistance and protection. 

Effectiveness:** The degree to which humanitarian operations meet their stated objectives, in a timely manner and at an acceptable level of quality. 

Efficiency:** The degree to which humanitarian outputs are produced for the lowest possible amount of inputs. 

Coherence:** The degree to which actors in the international humanitarian system act in compliance with humanitarian principles and IHL, and the degree to which they are able to influence states and non-state armed groups to respect humanitarian principles and conform to IHL. 

Complementarity:* The degree to which the international humanitarian system recognises and supports the capacities of national and local actors, in particular governments and civil society organisations. 

Connectedness:** The degree to which the international humanitarian system articulates with development, resilience, risk reduction and peacebuilding. 

Impact:** The degree to which humanitarian action produces (intentionally or unintentionally) positive longer-term outcomes for the people and societies receiving support. 

Criteria with two **  are original DAC evaluation criteria included in the ALNAP guidance (Beck, 2006). Criteria listed with one * are issues traditionally assessed under other DAC criteria which, for the purposes of the SOHS, we have pulled out to examine in more detail. 

2

ALNAP, The State of the Humanitarian System 2015 (London: ALNAP, 2015) www.alnap.org/help-library/the-state-of-the-humanitarian-system-report-2015; IASC, Human Rights Up Front: An Overview (Geneva: IASC, 2015). https://www.alnap.org/help-library/human-rights-up-front-an-overview. While ‘principled’ refers primarily to the International Committee of the Red Cross (ICRC) humanitarian principles, the efforts of local and national actors can also be considered principled by appeal to culturally specific norms around assisting those in need.

3

Mieke Heyvaert, Bea Maes and Patrick Onghena, ‘Mixed methods research synthesis: Definition, framework, and potential’, Quality & Quantity 47, no. 2 (February 2013): 659–76 https://www.alnap.org/help-library/mixed-methods-research-synthesis-definition-framework-and-potential.; Robert K. Yin, Case Study Research: Design and Methods, 5th ed. (New York: SAGE Publications, 2013). https://www.alnap.org/help-library/case-study-research-design-and-methods

4

 Heyvaert, Maes and Onghena, ‘Mixed Methods Research Synthesis’. https://www.alnap.org/help-library/mixed-methods-research-synthesis-definition-framework-and-potential

5

ALNAP, The State of the Humanitarian System: Assessing Performance and Progress (London: ALNAP, 2010), 5. www.alnap.org/help-library/the-state-of-the-humanitarian-system-assessing-performance-and-progress-alnap-pilot

6

e.g., Fred Carden, Teresa Hanley and Anna Paterson, From Knowing to Doing: Evidence Use in the Humanitarian Sector (London: ELRHA, 2021). https://www.alnap.org/help-library/from-knowing-to-doing