In brief

As the system’s estimates of the number of people needing humanitarian assistance have grown, so have its investments in building longer-term efficiency into humanitarian response. Examples range from significant improvements to funding mechanisms driven by the Grand Bargain to investments in multi-agency and digital cash payment systems. Increased use of more cost-effective modalities and working through local and national actors may also have contributed to better efficiency over 2020 and 2021. There were efficiency gains through enhanced coordination and multi-agency response consortiums, and agencies engaged more in internal-focused innovations and private sector partnerships to drive better performance. 

Evidence points to an overall improvement in efficiency since the 2018 SOHS, despite humanitarian actors facing numerous external challenges – including higher rates of inflation, increased access constraints and disrupted supply chains due to COVID-19. A notable exception to this is the continued under-investment in preparedness and early action, for which there is limited but compelling evidence that suggests acting before a crisis delivers greater cost-effectiveness.

This reflects a clear trend, running back to the 2012 SOHS, of the system focusing on internal organisational processes to gain efficiency, rather than improving its understanding and use of the most cost-effective programme modalities. While this approach has delivered consistent incremental improvements over time, it has not delivered significant changes in system-wide efficiency, or a good understanding of how to design programmes to reduce the most need for the largest number of vulnerable people with available resources.

Introduction

In the face of rising humanitarian need and competing priorities for government budgets, limited humanitarian resources must be spent in the best way possible. However, persistent lack of relevant, publicly accessible data makes it hard to understand the efficiency of humanitarian action. Figures such as cost-per-output or cost-per-aid recipient are available, but do not reflect the reality that those who are in greatest need are often the most expensive to reach. More useful data, such as a weighted cost-per-outcome, or the return on investment from mechanisms that support more effective humanitarian response, is either held privately by individual agencies, or not measured at all. 

Three key trends over 2018–2021 shaped considerations of how the system uses its resources. Firstly, as detailed in Chapter 3, humanitarian funding increased but fell short of requirements. Understanding how humanitarian agencies converted this increased but over-stretched in funding into performance gains is a central question for the system’s efficiency. Secondly, the implementation of the 2016 Grand Bargain agreement targeted inefficiencies in funding mechanisms and the more technical aspects of humanitarian delivery. Finally, various internally grown and externally imposed changes brought new approaches to efficiency, including increased engagement with digital innovation, more experience with consortium-based approaches and the impact of the COVID-19 pandemic on supply chains.

While it is important not to over-emphasise issues of efficiency, the way in which the humanitarian system uses its limited resources is still of critical importance. Against this broader context, this chapter reviews how well the system used its resources over 2018–2021 by addressing three main areas: (1) how well resources were used to improve system performance, (2) improvements to the efficiency and quality of humanitarian funding, and (3) operational efficiency.

Box K: Defining efficiency in humanitarian interventions

For comparability, this edition of the SOHS adopts the definition of efficiency used in the 2018 edition, drawing from the same version of the DAC evaluation criteria.672

Efficiency measures how economically inputs (usually financial, human, technical and material resources) were converted to outputs. This generally requires comparing alternative approaches to achieving an output, to see whether the most efficient approach has been used.673 Efficiency as an umbrella concept covers several different ways of thinking about how resources are used most economically:

  • Cost-efficiency considers cost per output. For example, dollar amount per shelter constructed or cost per child reached in activity sessions. Cost-per-output is the most common approach to evaluating efficiency in the system and is the measure for which most data is available. 
  • Cost-effectiveness is a more challenging measure in the humanitarian system because it examines the cost of achieving specific outcomes, for which, as noted in Chapter 7, limited data is available in several sectors. Evaluations agree that the information required to support cost-effectiveness analysis has not been consistently collected or reported in this period.674
  • Value for money is the most complicated measure of efficient use of financial resources to assess at a system-level. The value of different modalities of support could be compared across a range of factors – such as dignity, quality, timeliness or environmental impact – depending on what is conceived as being of ‘value’ by the person or entity defining the term. Given the conceptual variations associated with this measure, this chapter focuses predominantly on cost-efficiency and cost-effectiveness.

How well does the system convert inputs into impacts for affected populations?

Limits of evidence

The efficiency of humanitarian aid is a highly contested and sensitive performance criterion. It is also poorly evidenced. Evaluations still frequently assess efficiency through the perception of humanitarian practitioners, rather than actual cost measures. Agencies have been slow to achieve internal transparency on how they spend their funding and, when they do so, these figures are rarely comparable to other agencies or made publicly available. 

Donors do not consistently require outcome reporting by agencies, and agencies often lack the time, resources and skills to reliably measure outcomes. While a drive for efficiency was noted as one of the stumbling blocks to meaningful community engagement in Chapter 8, there are some signs that cost-efficiency and cost-effectiveness considerations are becoming less important to some donors, or at least held in balance with other priorities. Interviews with donor agencies indicated no significant changes to how they approached cost-effectiveness over the study period. Some donors expressed an interest in having better data but noted that this was difficult to combine with an overall push for less onerous assessment and reporting burdens on agencies. One major humanitarian donor reported that, while they value financial efficiency to ensure maximum support to crisis-affected people, cost-efficiency is only one criterion they consider among issues of safe and accountable quality programming: ‘Cost effectiveness is the last criterion. We first want to make sure that whatever modality we’re choosing, whether it be cash or something else, is feasible and is going to be the most appropriate and effective to meet the goals and objectives of the programme.’ 

Donors also sometimes use cost-effectiveness data in ways that discourage agencies from providing it. One agency key informant discussed specific examples of donors suggesting funding cuts on the basis of cost-effectiveness data – in one case, based on poor cost-efficiency of a programme, and in another, based on improvements in cost-efficiency, which led the donor to conclude the agency could achieve the same with less support. Similar concerns of efficiency data being used to shift funding priorities were reported by other research published during the study period.675

There were positive efforts over the study period, particularly driven by INGOs, to improve the methodologies for assessing cost-effectiveness and the comparability of data across agencies. For example, IRC worked with USAID and the CALP Network to develop better standards for assessing the cost-efficiency and cost-effectiveness of cash programming, while the Norwegian Refugee Council developed a protocol for cost harmonisation across agencies and donors. A coalition of INGOs developed and began applying a common tool (the ‘Dioptra tool’) for analysing cost-effectiveness, to generate cross-organisational cost-effectiveness data (see Box L).

One way to understand the efficiency of the humanitarian system at large is to look at whether it can deliver the same assistance at lower cost over time, thereby reaching more people. HRPs have begun to report on the number of people reached by assistance, showing an increase that has outpaced funding provided, as described in Chapter 5. However, these figures have only been available at the global level since 2019 and the evolving methodology hinders meaningfully comparison between years. 

The figures on funding requirements/requested per person targeted are slightly more comparable and reflect the system’s own estimates of how much it needs in order to reach affected people. As we saw in Chapter 3, these too have dropped, from $205 at the end of the previous SOHS study period in 2017 to $178 per person in 2021 (Figure 22). The reasons for this fall are, however, unclear: whether humanitarian assistance is becoming more efficient to deliver, or whether other factors affect the amount of funds requested per person. The figures all have to treated with caution – they are the system’s collective estimates and the methodology behind them is not clear or consistent. 

Requirements per targeted person across the largest HRPs in 2021, do not follow a discernible pattern (Figure 39). While the most expensive contexts were countries featuring conflict and significant access constraints – Libya, Iraq, Syria – one of the contexts with the most severe access issues observed in this edition of the SOHS, Ethiopia, also had one of the lowest requirements per targeted person ratios. This indicates that multiple factors influence the cost of humanitarian response including access, market conditions and the type of needs – but differences in arriving at the estimated costs may be an equally significant factor.

Figure 39: Requirements per person targeted and funding per person reached in UN-coordinated appeals by country, 2021


Requirements per targeted person across the largest humanitarian response plans in 2021, do not follow a discernible pattern. While the three most expensive crises by far in 2021 (Libya, Iraq and Syria) were all contexts of active conflict, multiple factors influence the estimated cost of humanitarian response including access, market conditions, the type of needs and methodology used.


Source: Development Initiatives based on UNOCHA HPC API data.

Notes: Data in the table includes HRPs only, given those more consistenly report on people targeted and expected people reached. El Salvador, Honduras and Guatemala have not been included as their expected data on people reached was not final.

Context and modality

While humanitarian practitioners have strong opinions on what is most cost-effective, the lack of data and evidence makes it difficult to draw firm conclusions about the best programme modalities or operational approaches, and the complexity of humanitarian settings means that cost-effectiveness is highly context dependent. 

Cash assistance

Cash and voucher assistance (CVA) often appears to be the most cost-efficient way to support people in crisis. Cost-efficiency comparisons between CVA and in-kind food distributions – which compare the cost of delivering cash against the same financial value of in-kind food – generally find in favour of CVA. In Kenya, for example, the cost of delivering $1 in cash was $1.18, compared to $1.94 for $1-worth of food.676 Practitioners also generally view cash assistance as being cheaper to deliver. However, there are measurement challenges with certain aspects of CVA delivery, such as the budget transparency required to meaningfully compare the costs of single-payer systems and multi-agency platforms. As one practitioner in Lebanon put it: ‘When we hear about the UN talking about the common cash approach and how [they] want to work more together to achieve better value for money? I think my perspective would be, but can you actually demonstrate value for money when there’s so little information provided on your budgets?’

Being cheaper does not necessarily make a modality more cost-effective if it does not deliver the same level or quality of outcomes. While the evidence for the effectiveness of CVA is strong (see Chapter 7), cost effectiveness comparisons are generally mixed. Two significant factors affecting the cost-effectiveness of CVA are the scale of an intervention and market conditions.677 

Scale – both in terms of number of households reached and programme duration – is key. Cash transfer programmes take time and resources to establish in new contexts, but once the initial foundations are in place, adding new recipients or providing additional payments over time requires less input. As described in (see Box L), the NGO partners using the Dioptra tool found that, to be cost-effective, cash programmes require a minimum of 1,000 recipients, while a study by the Cash Consortium for Iraq found that a shift from one-off to multi-month transfers brought significant cost savings.678 Biometric identification (iris scanning), mobile money and digital payment technology were useful in providing faster and less labour-intensive transfer programmes, and thus improving efficiency but come with data ethics considerations.679

Market conditions were even more significant to the cost-effectiveness of CVA. For example, in Lebanon, inflation soared over the study period and gross domestic product contracted by over 50% between 2019 and 2021,680 which had a significant impact on the scale of need and on the costs incurred by agencies to address this through CVA due to rising market prices and the increase in bank fees applied to cash withdrawals.681

Box L: The Dioptra tool and consistent humanitarian efficiency data

Authors: Lucian Lee, Advisor for Systematic Cost Analysis, International Rescue Committee; David Leege, Senior Director, Impact, Learning, Knowledge and Accountability, CARE; Tanaka Nyamadzawo, Aid Transparency Advisor, Danish Refugee Council; Marco Scagliusi, Senior Monitoring, Evaluation, & Learning Advisor, Mercy Corps; Purti Sharma, Senior Economist, Save the Children UK; Caitlin Tulloch, Director for Best Use of Resources, International Rescue Committee

Since 2018, a coalition of NGOs has developed and applied a standardised tool for frontline programme managers called Dioptra (formerly known as ‘SCAN’) to conduct cost-efficiency analysis of humanitarian programmes. Dioptra is a web-based software which applies a standard methodology using an organisation’s existing accounting data to identify the costs and results of individual activities. To date, five NGOs – CARE, the Danish Refugee Council, IRC, Mercy Corps and Save the Children – are using Dioptra and have collectively conducted several hundred efficiency analyses, providing the opportunity to compare the efficiency of different programme modalities with multi-agency data. Here we highlight some of the key findings.

Cost of cash assistance 

Dioptra analyses show that the costs of delivering cash assistance in humanitarian contexts vary widely, from $0.20 to nearly $3 for every $1 of assistance provided to people in need (including indirect and shared costs). Reasons for this high variation include differences in the cash transfer provided (delivering lower cash amounts appeared more costly per dollar transferred, because delivery costs were divided among fewer dollars transferred to clients), and scale (programmes which reached more households had significantly lower delivery costs per dollar transferred). Most of the ‘returns to scale’ are achieved once programmes reach 1,000 households (see below), and beyond that point increases in scale do not significantly reduce delivery costs. This suggests a minimum scale for cash-based programmes so that a greater proportion of resources go to those in need.

Trends in the cost-efficiency of basic needs cash programmes

These examples suggest the importance of scale (and thus sufficiency of resources) and the need for context-sensitive efficiency benchmarks. For example, using CVA as a global benchmark for the delivery cost per dollar transferred could lead to under-resourcing of critical elements of programme quality in some contexts.

Multi-year funding

As part of the Quality Funding Caucus of the Grand Bargain 2.0, consortium members have also been conducting Dioptra analyses to assess how longer-term funding enables greater programmatic efficiency. As one example of this kind of analysis, IRC looked for two programmes delivering the same activity and implemented in the same country, differing only in the duration of their funding. Analysis showed that, for every €1 of WASH non-food items delivered, the delivery costs for distribution through a 24-month programme were almost half that of distribution through a 12-month programme.

This efficiency gain did not appear to be driven by differences in the dollar value of NFIs between the two awards. When efficiency is considered using a metric which does not depend on the value of NFIs, such as the delivery cost per household reached, the longer-term programme was still considerably more efficient (€13.41 in delivery costs versus €78.36). Danish Refugee Council another consortium member that participated in the Quality Funding analysis, looked at multi-purpose cash assistance programmes in their Colombia office and observed efficiency gains between the shorter- and longer-term programmes analysed.

Similar to the findings on cash assistance, this difference appears to be driven by scale: the longer-term grant enabled a much larger scale of NFI procurement and distribution. This is consistent with previous evidence, which suggests that longer-term programmes unlock efficiencies by allowing for investments in infrastructure that enable large-scale delivery and provide useful evidence of the programmatic impacts of changes to funding processes.

Logistics and supply chains

Challenges of inflation and pandemic-related supply chain issues were not limited to cash. Country-level interviews reported that hyperinflation in Venezuela made project planning virtually impossible, and inflation resulting from the de facto blockade in Ethiopia meant NGOs were unable to meet their proposal estimates. Rising fuel prices had widespread impacts across many humanitarian contexts on the cost of delivering food and other in-kind support. In a global survey of humanitarian logistics and procurement professionals, 75% reported rising local transport costs and 69% reported rising costs for international transport over 2021682– notably before the Russia-Ukraine conflict sent global fuel prices soaring in early 2022.

The bureaucratic and access constraints described in Chapters 4 and 11 also had considerable impacts on the cost of humanitarian operations. In Venezuela and Bangladesh, NGOs described a huge increase in administration and bureaucracy associated with obtaining approvals and permissions for access, visas, bank accounts and international transfers. In Yemen, agencies used less cost-effective distributed supply chains to maintain services in the face of airstrikes on infrastructure and access restrictions. 

Although the impact of COVID-19 on global supply chains presented challenges to efficiency, agencies seemed largely able to rise to this challenge – including through the use of pooled or collaborative mechanisms. WFP created a Common Services structure for the COVID-19 response, supporting both UN agencies and INGOs with logistics capacity when commercial capacity was not available.683 DG-ECHO created the EU Air Bridge, with the aim of complementing WFP’s Global Common Services by supporting humanitarian airlifts on demand and strengthening the pooled resources through projects at international and country level.684

Funding mechanisms

The study period of 2018–2021 witnessed significant progress in the efficiency of funding mechanisms, primarily driven by the reform efforts of the Grand Bargain signatories. While the annual Grand Bargain reporting process frequently noted that implementation was slower than desired, with several commitments remaining out of reach at the end of the first five years,706 comparison with previous SOHS reports indicates that the achievements of the past four years are significant compared to a decade ago. 

For example, the 2012 SOHS noted the development of more ‘complex proposal and reporting tools’ by donors, which created additional burdens on NGOs and UN agencies; in contrast, by 2021, 30 signatories had signed up to the ‘8+3’ template685 developed through the Grand Bargain, a standardised format for use in project-level reporting. While the template is not in widespread use offers existence as a viable alternative to individual donor reporting is an improvement. 

Even donor assessments, an area considered ‘most disappointing’686 in the Grand Bargain reviews, have seen some improvements. While a 2020 study for the Grand Bargain found that donor assessments had doubled between 2016 and 2020, this was driven primarily by two donors rather than reflecting a general trend. Instead, there was a more common move across donors towards using Multilateral Organisation Performance Assessment Network (MOPAN) assessments and longer-term partnership frameworks and agreements – which, depending on their set-up, can reduce the costs of frequent assessments. There were also efforts to harmonise assessments and due diligence practices by UN agencies with their NGO partners and to share information on partner capacity through the UN Partners Portal, which at the end of 2021 had over 20,000 NGOs registered and five UN agencies participating.687

Quality funding in terms of multi-year or flexible funding can in theory increase efficiencies by avoiding repeated funding applications, hefty reporting, lengthy negotiations to alter programming in the face of changing contexts, and by enabling activities that cease to be useful to be replaced with those that better meet needs instead of being completed based on contractual obligations. But progress on flexible and multi-year funding was mixed, despite it also being frequently cited by evaluations as a factor in the timeliness, coverage and effectiveness of humanitarian response, as well as becoming an increasing priority in the localisation of humanitarian action (see Chapter 9), and for addressing needs in protracted crises. 

Donors reported a significant increase in multi-year funding, including, importantly, USAID and DG-ECHO as two of the largest humanitarian donors. In 2021 USAID increased its multi-year funding by over 60% and ECHO increased the percentage of humanitarian funding provided as multi-year from 3% to 9%.688 In volume, several key multilateral agencies – WFP, UNHCR and UNICEF – saw an increase in multi-year funding over the course of the Grand Bargain.689 At the same time, and as with funding to local actors (see Chapter 9), these volume increases have occurred against the backdrop of a rise in overall humanitarian funding, meaning that the proportion of annual funding to these agencies has either stayed the same or declined.690 COVID-19 also had varying impacts across donors. One donor reported that the pandemic left them less agile to respond to new crises, had created such uncertainties that they moved away from multi-year funding to more annualised approaches to allocation and had drained their contingency funds. Finally, despite years of discussion and implementation, there remain different opinions between agencies and donors on what qualifies as efficient multi-year funding. Interviewees claimed that multi-year grants that require annual approvals fail to provide the predictability and reduced paperwork that make truly multi-year funding more efficient.

Donors reported a significant increase in multi-year funding, including, importantly, USAID and DG-ECHO as two of the largest humanitarian donors.

Flexible funding – both unearmarked and softly earmarked – was more volatile. Overall, donors shifted to more flexible funding in 2020 to respond to COVID-19, but by 2021 much of this flexibility had receded. Unearmarked funding rose to $3.4 billion in 2020, before falling in 2021 to $2.7 billion, , well below 2018 levels. 

These generally static global figures mask significant variations in flexible funding from agency to agency: three UN agencies – OCHA, UNHCR and UNRWA – reported receiving more than 30% of their funding as unearmarked or softly earmarked in 2021, while WHO, after seeing a sharp increase in unearmarked funding to respond to COVID-19 in 2020, had their unearmarked funding drop from $714 million in 2020 to $83 million in 2021.. To improve their access to flexible funding, several agencies created or attempted to expand their own internal funding mechanisms for surge and early response. 

Figure 40: Unearmarked funding to UN agencies, 2018–2021


Between 2018 and 2021 unearmarked ‘flexible’ funding to UN agencies was volatile. Although donors shifted to flexible funding in 2020 as a response to COVID-19 (with funding rising to $3.4bn), this quickly receded in 2021, decreasing to $2.7bn, and accounting for only 13% of total humanitarian funding to UN agencies.


Source: Development Initiatives based on data provided bilaterally by UN agencies.

Notes: The calculations comprise earmarked and unearmarked humanitarian and humanitarian-related contributions given to the Food and Agriculture Organisation (FAO), International Organisation for Migration (IOM), UN Development Programme (UNDP), UN OCHA, UN Hight Commissioner for Refugees (UNHCR), UN International Children’s Emergency Fund (UNICEF), UN Relief and Works Agency for Palestinian Refugees in the Near East (UNRWA), the World Food Programme (WFP), and the World Health Organisation (WHO). 2020 data for IOM and 2020 and 2021 data for UNDP is an estimation and not available at the time of writing. Data is in constant 2020 prices.

One area that continues to see little improvement since the first SOHS in 2010 is the ‘pass through’ or layers through which humanitarian funding is provided. If anything, the inefficiencies of these layers became more apparent during the COVID-19 pandemic, as frontline agencies waited months to see resources trickle through and the UN agencies receiving the bulk of COVID-19 response funding were poor at providing transparency on its use. One senior humanitarian practitioner described the experience: 

We came to find out that a lot of them did spend these first major tranches of resources on PPE. It would have been really helpful to know that up front … It should never be that it takes months for frontline actors who have public information and are very aware of how much money has gone to the big UN agencies to find out how it’s been spent, right?

Multi-layered funding can be inefficient not only because of the additional transaction costs of passing funds through multiple agencies to reach the frontline, but also due to the poor quality of the funding received. Efficiency gains achieved through multi-year or unearmarked funding for UN agencies was rarely passed on to INGOs or L/NNGOs over the study period, meaning that funding for many frontline actors remained unpredictable and heavily restricted. While donors demonstrated more interest in reducing layers of funding, for many, existing regulatory frameworks prevented more direct funding of national actors. This was frustrating for national actors but also, as indicated by evaluations, a major driver of inefficiency for donors.691

The greatest success in reducing transactional layers was through an increase in resources provided through pooled funding mechanisms such as the CERF and country-based pooled funds. During the COVID-19 pandemic, when UN agencies proved slow in passing funds through to local NGOs,692 the CERF began making direct allocations to national actors for the first time. While this was done with the intention of improving timeliness, key informants reported that it also reduced transaction costs.

Are there system-wide mechanisms to improve efficiency?

Coordination and multi-agency collaboration

Since 2005 the term ‘coordination’ has become synonymous with the formal UN cluster coordination system activated in countries with a UN HRP. Previous SOHS reports have noted the contributions that cluster coordination, and other forms of multi-agency collaboration, make to humanitarian effectiveness. This largely remains true, although for this study period there was considerably more evidence available on non-cluster forms of coordination and collaboration, such as multi-agency consortiums, in comparison to findings on the cluster system. 

Strong coordination among agencies was credited with improved effectiveness in protection, education and food security. Area-based programming also led to more comprehensive meeting of needs in some responses. This appears to be an area of upward trajectory: when asked ‘Where has the humanitarian system showed the greatest improvement over the past three years?’ in the SOHS practitioner survey, coordination was the second most common answer.

When coordination was poor, the impacts on performance were clear. Specifically, coordination between the humanitarian cluster system and WHO’s incident management system was a consistent problem, affecting multiple health responses, from cholera in Yemen to Ebola in DRC and the COVID-19 pandemic.693 

After more than 15 years of the sector-based coordination system, its limitations were beginning to show, prompting efforts to rethink and trial new models.694 While sector-based coordination proved to be widely credited with improving effectiveness and efficiency compared to pre-2005 humanitarian responses, it has been most successful at coordinating actors in the system whose internal structures and approaches are organised by sector. But sector-based coordination has also been critiqued for its ‘siloing’ of aid,695 and has been limited in its ability to increase the leadership of national actors and to address cross-sector issues, such as CVA and services to IDPs.696 Cash Working Groups (CWGs) are used unevenly across responses and were found to duplicate or compete with clusters due to the increasing use of multi-purpose cash.697 

Efforts are under way to address these limitations. On CVA, during the study period the IASC developed a new model for cash coordination, approved by IASC Principals in March 2022. As discussed in Chapter 5, the Secretary-General’s High-Level Panel on Internal Displacement call for actions to better address the needs of IDPs included a review of the cluster coordination system and its appropriateness for ensuring national leadership and addressing risk and longer-term vulnerability for displaced populations.698

Over the study period, questions of whether coordination is worth the cost also persisted. Perceptions, upon which assessments of this aspect of efficiency tend to be based, were mixed in interviews and the evaluation synthesis. Practitioners in the SOHS survey were largely positive, with 45% answering that coordination benefits outweighed the costs, while country-level practitioners felt that responses without coordination were less efficient. Multi-agency consortiums and informal collaborations delivered key efficiencies in reducing duplication of effort and supporting greater harmonisation – but only if they featured strategic selection of partners, appropriate human resource capacity, joint objectives, values and common ways of working, and streamlined decision-making and contractual processes.699 In their absence, consortia and multi-agency collaborations were deemed to be inefficient and costly.700

It is also increasingly important to ask, ‘Efficient for whom?’ when it comes to coordination mechanisms. Coordination mechanisms can be efficient sources of information and coordinated decision-making by large agencies but require an investment of time by local actors that many – from DRC, Somalia and Turkey – deemed to be inefficient due to their relative lack of influence. For example, one Somali local actor reflected, ‘In front of powerful actors, you think you are better off just commenting on things and giving information, answering questions etc but you don’t challenge, you don’t lead and you don’t object strongly.’701

Box M: The impact of innovation on humanitarian performance

Authors: Lydia Tanner, Ian Gray, Alice Obrecht

A decade ago, innovation was widely embraced in the humanitarian system as an answer to many of its performance challenges. Investing in a formal innovation function similar to those in the private and development sectors promised to deliver greater efficiency and effectiveness and transform the way the humanitarian system did business. Since then, repeated SOHS editions have noted increased investment and activity in innovation, while also stressing that widespread impacts were yet to be seen. Original research for the 2022 SOHS on innovation identified four areas of activity with different performance trajectories.

Areas of innovation

When evaluations note the contribution of innovation to an agency’s performance, they tend to describe innovations that are internally developed for the purposes of advancing that particular agency’s operating capacity. These operational innovations are generally implemented in larger agencies and funded through core funds and overheads. They include the adoption of project management information systems and related digital data-gathering technologies, which has moved many humanitarian organisations over the past decade away from inefficient spreadsheets and paper-based systems. While getting comparable or collated data on the impacts of these innovations is impossible due to their individualised nature, evaluations often report efficiency gains from the introduction of technologies.

The second area is programme and enabling innovations. Hundreds of examples here, contribute to the effectiveness of many sector interventions. In WASH, LifeStraws have simplified water filtration. In nutrition, community-based management of acute malnutrition has scaled to improve the management of acute malnutrition. In education, several edtech solutions, such as Can’t wait to Learn, have enabled children who are not accessing formal education due to displacement, violence and conflict to learn to read, write and count. In communication with communities (CwC), innovations with complaints boxes, radio and SMS have improved channels of engagement for some crisis-affected communities, though (as noted in Chapter 8) large challenges in implementation of effective community engagement strategies persist. In voucher, food and NFI distributions, biometric cards have reduced the time people spend queuing for assistance. These programmatic innovations are sometimes funded by internal resources, but their number has also grown due to support from innovation funds. Many of these innovations are not only effective, but also have comparable cost-effectiveness over alternatives. However, the track record for scaling programmatic innovations in the humanitarian system remains weak, leaving this potential for cost savings unlocked.

Third are the ‘humanitarian to humanitarian’ (or H2H) innovations, which are being developed and delivered by organisations providing services to humanitarian agencies. The emergence of the H2H Network in 2016 reflects the role of these innovative small organisations. Examples include Translators Without Borders (now CLEAR global), which aims to improve the use of language in CwC; ACAPS, which provides shared context analysis; and Ground Truth Solutions, which streamlines community feedback. There is some evidence that these innovations improve effectiveness by providing performance enhancing support that is best done as a shared service, and by addressing humanitarian needs that have largely been ignored before, such as contextualised and accessible information. 

Finally, there are many examples of businesses, social enterprises, CSOs and universities that have traditionally worked in the development sector, social impact sector or the private sector, but are seeding and delivering innovations for humanitarian action. These innovations from non-traditional and local actors include scalable technologies such as Dimagi, Ushahidi and Mpesa, as well as smaller-scale entrepreneurial activities – from refugees in Kakuma Camp and Syria addressing problems in supply chains to local innovators addressing natural hazard risks through the bottle-net lifejacket in the Philippines or community-driven innovations supported by the Response Innovation Lab and through the Start Network’s DEPP Innovation Hubs. Non-traditional donors are increasingly funding these innovations – for example, Audacious, which has funded the Humanitarian Open Street Map, and the MacArthur Foundation’s, which has funded Sesame Street Works.

Scoring innovation 

The experience of the past decade of innovation raises important questions for its future. While agencies have partially fulfilled its promise, using new technologies and approaches to rise to the challenge of longer and more frequent crises, other system-wide solutions have failed or stalled. 

Under-investment, lack of realistic expectations for how long scaling takes, and poor prioritisation in the mechanisms needed to support system-wide innovation present significant barriers.702 The majority of innovation grants remain small, between $10,000 and $100,000 and, in the dataset collected for this edition of the SOHS, only one funder (of eight) awarded grants of over $900,000, with just 3 innovators (of 577) receiving over $1 million and 1 receiving over $2 million.703 Without the funding needed to have impact at scale, most innovators are left tinkering at the edges. 

This may also explain why the system as a whole is slow to adopt innovations, even those with proven track records – CVA being a prime example. Despite repeated evidence for the effectiveness and efficiency of CVA in many contexts, and despite yearly increases in CVA, this still comprises a smaller proportion of overall humanitarian assistance than could reasonably be expected. By one estimate, CVA would amount to between 37% and 43% of the share of total international humanitarian assistance if it were used as a default in the 70% to 80% of contexts where it would be appropriate.704 At present, it amounts to 20%, indicating that the system remains slow to scaling effective ways of working. 

Funding alone is insufficient to address scale: the system has not invested in the mechanisms needed to support innovators, broker partnerships and develop new business models, especially at the local level. Many of the innovations that are sector ‘agnostic’ have struggled to find a sustainable business model, with some closing despite having a significant impact, such as Frontline SMS, the SMS messaging platform used by humanitarian organisations globally the SMS.705

Finally, understanding the impacts of innovation is challenging. It is hard to collate monitoring and evaluation (M&E) data on impact and the ‘tail’ for innovations’ impacts tends to be much longer than the timeframes used to evaluate humanitarian grant funding. A study for this edition of the SOHS on the M&E evidence from eight humanitarian innovation funders found that evidence of impact was only available for 16% of funded projects, and that most funders were unable to collect data on the outcomes of an innovation after grant funding had ended. While there are many examples of the impact individual innovations (such as digital data collection, cash-based assistance, community-based management of acute malnutrition, and blockchain at the largest scale), there is still a significant data gap on what impact innovations are collectively having on humanitarian effectiveness and efficiency.

672

The DAC criteria were revised in 2018/19 with a new conceptualisation of efficiency that includes economic, operational and timeliness. ALNAP is in the process of updating its own evaluation criteria guidance, to be produced in 2023.

673

T. Beck, ‘Evaluating Humanitarian Action Using the OECD-DAC Criteria’ (London: ODI/ALNAP, 2006), 44. https://www.alnap.org/help-library/evaluating-humanitarian-action-using-the-oecd-dac-criteria

674

ADE, Response to the Rohingya Refugee Crisis https://www.alnap.org/help-library/evaluation-of-the-european-union%E2%80%99s-humanitarian-response-to-the-rohingya-refugee-crisis; P. Breard, ‘Evidence Summary on COVID-19 and Food Security’ (United Nations Evaluation Group (UNEG), 2021) https://www.alnap.org/help-library/evidence-summary-on-covid-19-and-food-security; Betts et al., WFP Regional Response https://www.alnap.org/help-library/evaluation-of-the-wfp-regional-response-to-the-syrian-crisis-2015-2018; Global Affairs Canada, ‘Evaluation of the International Humanitarian Assistance Program 2011/12 to 2017/18’. https://www.alnap.org/help-library/evaluation-of-the-international-humanitarian-assistance-program-20112012-to-20172018

675

L. Weingärtner, T. Pforr, and E. Wilkinson, ‘The Evidence Base on Anticipatory Action’ (Rome: WFP, 2020). https://www.alnap.org/help-library/the-evidence-base-on-anticipatory-action

676

B. Mwongela, ‘Kenya, General Food Distribution Cash Modality Scale up for the Refugees and Host Community in Kakuma and Dadaab Camp: An Evaluation’ (Rome: WFP, 2018) https://www.alnap.org/help-library/kenya-general-food-distribution-cash-modality-scale-up-for-the-refugees-and-host

677

J. Hoddinott, S. Sandström, and J. Upton, ‘The Impact of Cash and Food Transfers: Evidence from a Randomized Intervention in Niger’, American Journal of Agricultural Economics 100, no. 4 (July 2018): 1032–49; https://www.alnap.org/help-library/the-impact-of-cash-and-food-transfers-evidence-from-a-randomized-intervention-in-niger Betts et al., WFP Regional Response https://www.alnap.org/help-library/evaluation-of-the-wfp-regional-response-to-the-syrian-crisis-2015-2018; Daniels, Anderson and Yusuf Ali, 2017 Somalia Response. https://www.alnap.org/help-library/evaluation-of-the-2017-somalia-humanitarian-cash-based-response

678

I. Betzler and O. Westerman, ‘Evidencing the Value for Money of the CCI’s Cash and Legal Programmes’ (Mercy Corps, Oxfam, Norwegian Refugee Council (NRC), International Rescue Committee, Danish Refugee Council (DRC), 2018). https://www.alnap.org/help-library/evidencing-the-value-for-money-of-the-cci%E2%80%99s-cash-and-legal-programmes

679

Again, though, it is important to distinguish between efficiency from the agency perspective and from the end user’s: if an individual is unable to travel to collect their transfer and biometric identification is used, they cannot send a family member in their place (Daniels, Anderson and Yusuf Ali, 2017 Somalia Response. https://www.alnap.org/help-library/evaluation-of-the-2017-somalia-humanitarian-cash-based-response

680

World Bank Group, Lebanon Economic Monitor, Fall 2021: The Great Denial (Lebanon: World Bank Group, 2022). https://www.alnap.org/help-library/lebanon-economic-monitor-fall-2021-the-great-denial

681

Key informant interview in Lebanon.

682

CHORD, ‘The State of Logistics and Supply Chain in the Humanitarian Context. Global Survey Findings’ (Center For Humanitarian Logistics and Regional Development (CHORD), 2022). https://www.alnap.org/help-library/the-state-of-logistics-and-supply-chain-in-the-humanitarian-context-global-survey

683

WFP, ‘WFP Common Services,’ 2021. https://www.alnap.org/help-library/wfp-common-services-situation-report-31-january-2021

684

WFP, ‘WFP COVID-19 Common Services Situation Report’ (Rome: WFP, 2021). https://www.alnap.org/help-library/wfp-common-services-situation-report-31-january-2021

685

The 8+3 Template, 2019. https://interagencystandingcommittee.org/system/files/83_template_final.pdf

686

Victoria Metcalfe-Hough, Collaborative Advocacy between Humanitarian and Human Rights Actors: Opportunities and Challenges (London: HPG/ODI, 2021). https://www.alnap.org/help-library/collaborative-advocacy-between-humanitarian-and-human-rights-actors-opportunities-and

687

Similar to many areas of the Grand Bargain, while streamlined assessment and partnership arrangements are assumed to support efficiency gains, there has been no quantitative study confirming their contributions to overall reductions in operating costs.

688

 Victoria Metcalfe-Hough et al., The Grand Bargain in 2021: An Independent Review (London: HPG/ODI, 2022). https://www.alnap.org/help-library/the-grand-bargain-in-2021an-independent-review

689

Metcalfe-Hough et al., Grand Bargain in 2021. https://www.alnap.org/help-library/the-grand-bargain-in-2021an-independent-review

690

Ibid. https://www.alnap.org/help-library/the-grand-bargain-in-2021an-independent-review

691

e.g., Murray, Response to the Refugee Crisis in Turkey. https://www.alnap.org/help-library/evaluation-of-the-european-union%E2%80%99s-humanitarian-response-to-the-refugee-crisis-in

692

Jeremy Konyndyk, Patrick Saez, and Rose Worden, ‘Humanitarian Financing Is Failing the COVID-19 Frontlines’ (Washington DC: Center for Global Development, 2020). https://www.alnap.org/help-library/humanitarian-financing-is-failing-the-covid-19-frontlines

693

Paul Spiegel et al., ‘Responding to epidemics in large-scale humanitarian crises: A case study of the cholera response in Yemen, 2016–2018’, BMJ Global Health 4, no. 4 (July 2019): e001709. https://www.alnap.org/help-library/responding-to-epidemics-in-large-scale-humanitarian-crises-a-case-study-of-the-cholera

694

Konyndyk, Saez and Worden, ‘Inclusive Coordination’. https://www.alnap.org/help-library/inclusive-coordination-building-an-area-based-humanitarian-coordination-model

695

P. Knox Clarke and L. Campbell, ‘Improving Humanitarian Coordination: Executive Summary and Recommendations’ (London: ODI/ALNAP, 2016) https://www.alnap.org/help-library/improving-humanitarian-coordination-executive-summary-and-recommendations; Konyndyk, Saez and Worden, ‘Inclusive Coordination’. https://www.alnap.org/help-library/inclusive-coordination-building-an-area-based-humanitarian-coordination-model

696

J. Jodar et al., ‘The State of the World’s Cash 2020 – Executive summary’, 2020. https://www.alnap.org/help-library/the-state-of-the-world%E2%80%99s-cash-2020

697

Daniels, Anderson and Yusuf Ali, 2017 Somalia Response. https://www.alnap.org/help-library/evaluation-of-the-2017-somalia-humanitarian-cash-based-response

698

UN, ‘Shining a Light on Internal Displacement A Vision for the Future.’ (New York: United Nations, 2021). https://www.alnap.org/help-library/shining-a-light-on-internal-displacement-a-vision-for-the-future

699

Pham et al., DEPP Evaluation https://www.alnap.org/help-library/depp-evaluation-summative-phase-report.; J. Baker et al., ‘Evaluation of the European Union’s Humanitarian Interventions in India and Nepal, 2013-2017’ (Brussels: European Commission ECHO, 2018), 2013–17. https://www.alnap.org/help-library/evaluation-of-the-european-union%E2%80%99s-humanitarian-response-to-the-rohingya-refugee-crisis

700

Pham et al., DEPP Evaluation. https://www.alnap.org/help-library/depp-evaluation-summative-phase-report

701

This view, however, was not consistent across all local actors, as some found the opportunities for information sharing and contribution valuable.

702

GAHI, ‘Lessons Learned. December 2019’ (Amstelveen: KPMG, 2019). https://www.alnap.org/help-library/the-global-alliance-of-humanitarian-innovation-gahi-lessons-learned-exercise-2019

703

 C. Komuhangi et al., ‘The State of the Humanitarian System: Innovation report.’ (ALNAP, 2022)

704

J. Steets et al., ‘Drivers and Inhibitors of Change in the Humanitarian System’, (GPPI, 2016). https://www.alnap.org/help-library/drivers-and-inhibitors-of-change-in-the-humanitarian-system-a-political-economy

705

**To read more about why Frontline SMS closed see: **

706

Metcalfe-Hough et al., Grand Bargain at Five. www.alnap.org/help-library/the-grand-bargainat- five-years-an-independent-review.