likelihood and potential consequences. Members of the public use weather warnings as one source of information to determine their behavioural response. Lindell and Perry (2012) describe how the decision-making process follows a series of ideal stages:

Often the decision will be to look for further information to increase understanding of any of these factors (Wood et al. 2017). More generally, a number of factors related to the hazard, the situation and other respondents' characteristics (see Sect. 3.2.4) determine whether respondents are able to assess all of the decision-making steps described above.

Responding agencies may conduct risk assessments as part of their decision. Risk assessments can be quantitative (e.g. dollar losses or fatalities), or qualitative (e.g. low, medium, high), and use various tools. The result may fall into the categories of 'acceptable risks', where the risk is seen to be minimal and no mitigation measures are required; 'tolerable risks', where it is determined that the benefts of living with the risk outweigh the potential cost, and some mitigation may be required; and 'intolerable risks', where the risk is seen as being high and mitigation actions are required (e.g. Standards New Zealand 2009).

# *3.2.3 Which Information Sources Do Receivers Use?*

Identifying the information sources receivers use is important for understanding the relative value of each source, and the contributions of those sources to the provided information and messages (Lazo et al. 2009). Information sources vary between users, according to their needs. They may include offcial sources, private weather services, environmental cues (such as seeing storm clouds), social cues and/or culturally indigenous cues. The information often travels through multiple channels, including the traditional and social media (such as tabloid newspapers), social connections (such as from friends, family, neighbours, colleagues and education facilities) and/or response agencies (including emergency services, local government, infrastructure companies and emergency management). Each of these channels, and the way the information travels through secondary sources, can have its own challenges.

Offcial sources of weather information tend to be National Meteorological and Hydrological Services (NMHS). Depending on national legislation, many NMHS are entitled alerting authorities, listed in the WMO Register of Alerting Authorities

(https://alertingauthority.wmo.int/). Some receivers of information (e.g. airline companies) pay for a subscription service to receive more detailed or specifc data to meet their needs.

Offcial information can be disseminated and often altered through other channels (such as TV stations/media or emergency managers) to provide additional information. In fact, most people access weather forecasts from the media and via the private sector, rather than directly from offcial information sources (Lazo et al. 2009, Hayes et al. 2014). As a result, most people use multiple sources of information when making response decisions, such as evacuating from a hurricane (e.g. Dow & Cutter 1998), to meet their individual needs. The majority of private sector weather forecasts are directly or indirectly based on offcial weather services (Lazo et al. 2009), including using public-sector global observations and models, and atmospheric research (Thorpe & Rogers 2018). The private sector is often highly interested in incorporating the 'authoritative voice', such as issued warnings, into their products and services (e.g. Google Crisis Response/Public Alerts – https://crisisresponse.google/products/public-alerts/) (Kaltenberger et al. 2020), for consistency, and to add further information or features to make their products more user-oriented. An increasing interest in impact-based forecasts and warnings by the private sector is expected due to the focus on user needs (WMO 2015; Thorpe & Rogers 2018).

Environmental and social cues are key sources of information that can infuence the receiver's perceptions and actions (Lindell & Perry 2012). Environmental cues, such as a gathering storm, a funnel cloud, a roaring sound or heavy rain, are often considered when determining a response, such as during the Joplin, Missouri and US tornado warnings (Kuligowski et al. 2014). However, environmental cues can be hidden by low light (at night) and muffed by other noise (e.g. fash fooding muffed by heavy rain). Some cues are subtle or ambiguous, whereas others are much more obvious. Perceptions of environmental cues can also be altered depending on the time of day and other activities that are prioritised by the recipient (Ruin et al. 2014). Social cues stem from observing what other people are doing (e.g. packing a car to evacuate) and receiving information from community networks, such as family/ friends and neighbours. This can be via a range of channels, including social media, phone calls, text messages and face-to-face. The availability of assistance, including the provision of transport and shelters, also provides cues (Lindell & Perry 2012).

In addition to the above sources, responding agencies sometimes have their own information sources. These can include trained storm spotters, staff members collecting impact or hazard data, crowdsourced reports, video cameras in public areas (e.g. traffc cameras) and their own monitoring equipment.

Finally, personal experience and cultural knowledge including, for many people, indigenous knowledge are used as information sources or a lens through which to interpret cues, prepare and respond. In many areas of the world, such knowledge is drawn upon to interpret environmental cues, such as the behaviour of wildlife, to help forecast future weather and seasonal events (e.g. Pareek & Trivedi 2011). Traditional knowledge is also a source of information to inform mitigation plans, and preparedness and response activities, such as terracing hillsides to prevent runoff and erosion and building sturdy houses that can withstand strong wind. The relevance of such knowledge is often lost when people migrate, especially into urban environments, but can be replaced if communities make efforts to build a new shared experience, e.g. by marking the heights of historic foods or the limit of inundation by tsunamis, by acknowledging anniversaries of disasters and by ensuring that schools teach their children about the hazards and responses appropriate to where they live and go to school.

# *3.2.4 Behavioural Infuences on Decision-Making*

Why do some people not respond in the way that they were directed to by a weather warning? There are many things that infuence people's behavioural response to a warning. Several theoretical models describe these infuences, such as the Protective Action Decision Model (PADM; Lindell & Perry 2012). Other theories and models are described in the best practice guidelines for risk communication and behaviour by the NOAA Social Science Committee (2016).

Lindell and Perry (2012) describe the factors infuencing the 'pre-decisional processes' of the receiver (Fig. 3.2). They include the receiver's characteristics, such as their age, gender, primary language, mental models (general understandings and misconceptions), economic resources, social resources and physical abilities. As discussed in Sect. 3.2.3, behavioural responses to warnings may also be infuenced

**Fig. 3.2** Factors infuencing pre-decisional processes towards protective actions. (From Potter (2018), based on the Protective Action Decision Model (Lindell & Perry 2012))

by indigenous knowledge and cultural norms, environmental and social cues and information sources (Shaw 2009). Message characteristics (particularly content and format, e.g. Wood et al. 2015; Potter 2018) and channel access and preference (see Sect. 3.3.3) are further infuences, with relevant channel characteristics including the level of detailed information, precision of the message, the frequency that the message is sent out, the equipment requirements and how much it interrupts the receiver's activities.

The pre-decisional processes inform the decision that is being made according to the level of exposure the receiver had to the cues or warnings, the attention they paid (including intrusiveness of the alert) and their level of understanding of the message – including weather literacy (Fleischhut et al. 2020) and understanding of forecast uncertainty (Joslyn & Savelli 2010; Morss et al. 2010).

This demonstrates how using simple language, avoiding jargon and using a range of languages, when possible, infuence the response. The infuences mentioned here can take place very quickly and without much conscious thought. Theory from cognitive science suggests that in situations of high stress, people may make decisions using a faster decision pathway that is rather emotion-driven, while in less stressful situations, they are more likely to base their decisions on information (Weyrich et al. 2020b). In such cases, more information is not necessarily better. Moreover, which decision-making pathway people utilise may depend on the context. At the same time, cognitive theory has been hard to test in the feld, because of the ethical challenges of submitting people to actually dangerous conditions.

A high level of preparedness can reduce the shock and facilitate an informed response to a warning. This may be achieved through education, drills and exercises and early alerts. Once a likely future hazardous event has been identifed in forecasts, a range of communication formats can be used to build a general expectation that action will need to be taken.

Once the pre-decisional process is complete, the receiver's core perceptions provide another flter through which the information is assessed. The receiver's perception of the threat (or risk) has a large infuence on their decision to respond (e.g. Kox & Thieken 2017). Risk perception refers to the judgments people make when they are asked to characterize and evaluate hazardous situations. This is infuenced by numerous factors such as a respondent's prior experience, familiarity with the hazard, expected/perceived severity of consequences, perception of benefts and control over the risk source, etc. (for a review of risk perception literature, see Slovic et al. 2004, 2007). Cultural and social factors also strongly affect how people live with and perceive risks (for a review of cultural theories of risk, see Thompson et al. 1990). Yet, research shows mixed results as to whether a higher risk perception relates to a higher level of preparedness or response behaviour (e.g. Mileti & Sorensen 1990, Potter et al. 2018, Wachinger et al. 2013).

The receiver's perceptions of the source of the information (such as the agency issuing the warning) also infuence their response. In particular, the perceived trustworthiness, expertise and protection responsibility of the source are taken into account (Arlikatti et al. 2007). On the one hand, institutional trust, for example, in NMHS issuing weather warnings, and credibility of the source, increases the likelihood for protective action (Potter et al. 2018, Ripberger et al. 2015). On the other hand, self-effcacy, including the belief in their own ability to inform and protect themselves against severe weather, is also an important driver for protective action (Kox & Thieken 2017).

A receiver's perception of protective actions can infuence their decision to respond (e.g. Demuth et al. 2016; Johnston et al. 2005). An action perceived as effective in reducing the risk has a higher likelihood of being acted on, as do actions that are considered to be affordable and achievable and take little time and effort and that don't require coordination with other people. However, much of the research that has been conducted requires more testing in a warning context, rather than a general preparedness context, and in a 'real event' context, as opposed to a hypothetical context.

Following the pre-decisional processes and the triggering of the core perceptions, the decision to respond is undertaken (Lindell & Perry 2012). Once the risk has been identifed, people look for information about the risk, involving thinking about personal impacts and length of time until the impacts are likely to occur. Options to increase their safety include drawing on their memory and past personal experiences as well as those they have seen in the media or read about, and education initiatives (Demuth et al. 2016; Sutton & Woods 2016). The receiver may seek further information, observe social cues and/or use guidance messages from warnings. Alternative actions are compared with continued normal activities. If it is determined that multiple actions need to be taken, the order of these needs to be assessed. A response plan may be determined only if there is time and a goodenough knowledge to do so. The plan may indeed range from vague to detailed, potentially including factors such as evacuation route, destination and means of travel. These decision-making processes can cause delays to taking action as further information is sought, which is referred to as 'milling' (Wood et al. 2017). This highlights the importance of including 'what to do' guidance messaging in the warning message, and hyperlinks to detailed information to help people determine their actions. Once an action plan has been developed, the situation may hinder its implementation. For example, roads could be blocked, children and pets need to be found, or there may be a lack of access to transport, prompting alternative actions to be assessed, decided on and implemented.

# **3.3 Capabilities of the Warner**

# *3.3.1 Who Issues Warnings?*

While many countries have legal limitations on who can issue a public warning, in reality, both professional responders and the public have access to a variety of sources of warnings. Government agencies dominate as sources of information, while police and/or fre and rescue often have the fnal authority to order people out of an area of immediate danger. For infrastructure operators, however, while the advice may come from a government agency, it will often be an in-house emergency manager who will issue the emergency response order to the operations department. More generally it is a variety of public, private and non-governmental organisations that issue warnings to the public, in a wide range of formats and through a variety of media. Some of these organisations may simply repeat the message issued by the originator, but more often than not, there will be some degree of translation and reformatting. Indeed, a compelling presenter, even if they have minimal knowledge of how the warning was created, can have a huge infuence on the level of response – and may become, in themselves, the basis for the trust of the audience.

Underlying observations and model predictions are generally provided by NMHS. This information is then used by themselves and private sector meteorologists to produce forecasts and warnings (Thorpe 2016; Pettifer 2015). These different – public and private – sources do not always provide consistent warning information: through the use of different colour-coding and warning thresholds, by interpreting model outputs differently, or by conveying different overall messages, they create perceived or real inconsistencies in the warning information received by the public (Weyrich 2019). National weather services and emergency managers have suspected that these conficting cues exist and negatively infuence public behaviour; however, more research is needed. Existing empirical evidence clearly shows that contradictory visual and textual information have negative effects on public behaviours (Lindell & Brooks 2013; Williams & Eosco 2021).

The challenge of consistent warning information grows with the number of agencies involved in the warning decision process. Inter-agency coordination is needed to ensure the delivery of consistent messages by multiple agencies within impactbased forecasting and warning systems (Potter et al. 2021). On a regional scale, cross-border high-impact weather situations have a greater potential for conficting warning information from multiple-channel sourcing information from different NMHSs. This is not just because of differently designed warning systems but also because of a lack of standard operating procedures (SOPs) in exchanging information among neighbouring NMHSs (Kaltenberger et al. 2020). Regional programmes, such as Meteoalarm in Europe (https://meteoalarm.eu) and WMO GMAS-Asia (https://gmas.asia), aim to foster regional cooperation and information exchange that improves cross-border warning information consistency. Such consistency is of fundamental importance both to global re-users and to global responders such as the UN and international NGOs.

# *3.3.2 Types of Warnings*

Warnings may be classifed in many ways, including how they are produced, their message structure or the mode of delivery. For speed of response, automated warning is optimal, e.g. a fre alarm based on a smoke sensor or a food siren that sounds when an upstream river gauge registers above a critical threshold. Such systems depend on recipients being familiar with what the alert means and what they should do to respond. In the past, warnings were often produced using bespoke templates and with highly compressed language to minimise the number of characters or words to be transmitted. Warnings of this type remain in use, such as in maritime and aviation safety. They typically use fxed hazard thresholds related to impacts that are meaningful to a specifc audience. Free-form warnings give more options for the warner to include information specifc to an event or for multiple audiences, but also increase the risk of misunderstanding or lack of clarity.

In order for everyone to have a common understanding of the severity of the hazards that are forecast, are observable or have occurred in the recent or distant past, they are often given a label. This understanding can have many uses, one of which is to help raise awareness in the public about an impending event so as to prompt preparedness actions. Some systems are numerical and continuous (e.g. earthquake magnitudes), some are divided into categories (such as the Saffr-Simpson hurricane wind scale), others have a small number of levels (such as meteorological 'outlook, watch, warning' systems), and some are binary (e.g. fre alarms). When designing a warning or category system for a hazard, one of the many decisions that needs to be made is what it will be based on (the 'foundation'; Potter et al. 2014). This determines the trigger for issuing a warning. Options range from hazard through to guidance on response, i.e. what people should do. Each has its own benefts and challenges.

At one end of the spectrum is basing the foundation on the hazard. This means that warnings are triggered by the severity, extent, duration or magnitude of the peril, regardless of whether people will be exposed to it. Examples include expected wind speed, rainfall intensity and earthquake magnitudes (such as the Richter scale, moment magnitude and local magnitude), where the scales have fxed thresholds based only on their intensity. Volcanic alert-level systems primarily use the magnitude or severity of volcanic activity as a foundation for warnings (e.g. New Zealand's, Potter et al. 2014; and the US system, Gardner & Guffanti 2006). The benefts of these systems are that scientists can issue information quickly, based on their understanding and data; little coordination is needed with agencies who hold information on impacts, vulnerability, exposure and mitigation procedures. The main challenge is that phenomena-based systems may not initiate the most appropriate or timely responses by members of the public. Including impact information and guidance on response can improve effectiveness (e.g. WMO 2015).

Further along the spectrum, the severity thresholds chosen for each warning or category level may be fxed according to the severity of damage it would cause to people or property if they were exposed to the hazard. These systems assume someone or something could always be exposed; i.e. the thresholds do not vary over space or time. Examples of this type of system include the enhanced Fujita scale, Saffr-Simpson hurricane wind scale and the modifed Mercalli intensity (earthquake shaking intensity) scale. Once the thresholds vary over space and/or time (e.g. climatology-based thresholds), the system is heading towards being an impactbased warning system. This can take account of the dynamic situation, including antecedent ground conditions (e.g. prior rainfall causing wet catchments and therefore accelerated fooding), variable populations (such as rush hour) and specifc impacts such as airport closures (WMO 2015). These systems require collaboration between agencies who hold the information about meteorology, society and impacts, which has the potential to cause delays in issuing warnings (Potter et al. 2021). Impact-based forecasts and warnings are thought to increase the level of understanding about the situation by the public, and raise risk perceptions, but there are mixed results as to their effectiveness in prompting a behavioural response (e.g. Weyrich et al. 2018; Potter et al. 2018). Determining which impacts or consequences to base the warning on becomes important, whether it is for safety of life, injuries and well-being, damage, disruption, or economic or environmental impacts. Risk modelling can help with mitigation and hazard management (Crawford et al. 2018) and may become increasingly utilised in real-time situations.

Personalised warning messages that include information about local disruptions and impacts may help to prompt effective responses and would generally be issued by partner agencies who hold the roles and responsibilities to issue them (WMO 2015). These 'impact-oriented' warnings require substantial improvements in impact data collection and storage (Kaltenberger et al. 2020). Finally, some warning systems are based on the action required by the population at risk. Higher levels may include evacuations of large areas, and lower levels may promote increased awareness. Examples include New Zealand's COVID-19 pandemic alert-level system, Japan's volcanic alert-level system, fre alarms or tsunami sirens requiring an evacuation and the seatbelt sign in aircraft. These systems tend to promote compliance and require receivers to understand the actions relating to the levels or alerts. Further investigation into these types of systems would be benefcial to identify how underlying observational data support the decision-making to trigger a warning.

# *3.3.3 Communication Channels*

Getting the warning to the receiver is essential if it is to have any value. The diverse types of dissemination channels – print, mechanical, electronic and face-to-face – have very different characteristics, for instance with respect to the dissemination rate or precision (Lindell & Perry 2012). A wide range of channels should be used according to the needs of receivers, including local or national TV (including cable), radio (including specifc weather radio channels), newspapers, friends/family, coworkers, neighbours and smart/cell phones. Increasingly, web pages and mobile applications are important sources of weather information, including cell broadcast alerts and social media (Hayes et al. 2014). A siren can make people instantly aware of a threat but has a limited reach and provides no further information. Mobile phone applications can provide instant messaging with greater information content, but the recipient needs to have a (functioning) mobile phone and to consult it to receive the message. Newspapers are slow, but can provide detailed information and context, and are ideal for early alerts. Television can strongly engage the viewer, but with limited airtime, and may miss out important information. A trusted neighbour or offcial is a compelling source in an emergency but is very resource intensive and time-consuming.

Different channels support a different range of warning formats. For instance, some channels are restricted to text (SMS on cell phone), audio (radio) or video (regular TV), while others can contain a variety (websites, smartphone apps, weather TV). A mixture of formats can reinforce the message if consistent and if they are quickly and easily accessible. For instance, the Met Offce weather app provides colour-coded warnings identifying the type of hazard on a map of the UK. Each warning can be expanded to a higher-resolution map, identifying affected towns and cities, supported by a brief explanation of the hazard and its source, what the likely impacts will be and the status of the warning (including when it was last updated and why). Links are provided to guidance on how to respond, and to additional information including a more detailed context and a list of the administrative authorities covered. Supporting material is also provided in the form of the TV weather forecast video, which includes the same information communicated verbally and visually by a presenter. Care must be taken when using visual formats as many people have diffculty reading maps, colours should be distinguishable by those who are colour blind, and colour scales and icons are not universally recognised (but see Guemil (2021) for an initiative towards an internationally accepted set of emergency icons). Consideration needs to be given to how many and which languages will be used, and to reaching the visually impaired and those with severely limited language skills.

Channels differ in the extent that they reproduce the authoritative warning or add or provide their own or other independent warning information, but it is essential to acknowledge that only in an ideal world should they all tell the same story, but in reality inconsistencies persist. Thus it is essential for agencies to recognise and to live with inconsistencies. Where the information source is known and trusted, the dissemination channel should provide source attribution and branding, as these will speed up recognition by the user, and help to distinguish the message from other potentially conficting or misleading messages. Indeed, as the information landscape becomes more crowded, an increasingly important role of the information provider must be to monitor these multiple sources and try -to the extent possible- to issue corrective statements to counteract false information before it spreads.

New technologies, based on smartphones, are not only a channel to disseminate information. They can also be used to test the effectiveness of different communication strategies, such as by disseminating different types of messages (e.g. impact and non-impact based) to the receivers and by enabling verifcation – including via surveys – of which types of messages lead to adaptive behaviours (Weyrich et al. 2020a). They can also open a window for two-way communication through social media and for collecting data through crowdsourcing, for example.

Sights and sounds that indicate hazard onset are important inputs for people's emergency response (Lindell & Perry, 2012), but may come too late. For example, the presence or absence of thunder and lightning during a storm may infuence responses by providing evidence supporting the imminent threat. Where the threat is not yet suffciently close, CCTV, webcams and social media videos can provide equivalent cues at a distance. Using two-way communication such as this can be a valuable component of a warning service. Not only can the local information provided through social media be used to reinforce the warning in places not yet reached by the storm, but it can also help the forecaster to keep abreast of an evolving situation and to fne-tune their forecast for use in updated warnings.

Many challenges currently hinder the incorporation of social media and crowdsourced data in warning practices. For example, agencies are afraid that social media will produce harmful and inaccurate information and that it can be diffcult to evaluate the credibility and validity of user-generated content (Weyrich et al. 2020a,b, Goolsby 2009, Kaplan & Haenlein 2010). Also, the enormous amount of information can be overwhelming, and ethical concerns can further discourage agencies from fully exploring their potential. These issues include the potential for breaches of privacy, even from anonymised datasets, the lack of consent involved and the possibility of misuse by commercial entities interested in surveillance (Maxmen 2019). Other problems arise from unequal access to social media. For example, in 2016–2017 nearly 1.3 million households had no Internet connection in Australia, and lower digital inclusion was observed in already vulnerable groups, including the unemployed, migrants and the elderly (Howarth 2018). As a result, gaps between privileged and marginalised people may grow wider. It must also be acknowledged that the private weather sector may view these social dilemmas and challenges differently, because of their interest in the commercial aspects of warning dissemination.

Opportunities for warning agencies to use these newer channels include a better understanding of public debates about warning-related issues, monitoring dangerous situations and interacting with receivers, promoting crowdsourcing and other collaborations as well as extending the reach of organisational information and improving transparency, visibility and reputation. In the future, artifcial intelligence may increasingly be used to monitor these channels and to sift out the critical pieces of information that indicate a change to a warning is needed.

This plethora of media and formats can seem daunting when designing a warning system to a restricted budget. The international standard Common Alerting Protocol (CAP) provides an increasingly valuable tool for minimising the overheads of using multiple channels (FEMA 2020). Provided a warning is produced in the XML-based CAP format (OASIS-Open 2010), standard software is available to convert it for delivery through a wide and increasingly varied set of channels, including a degree of automatic tailoring to the needs of specifc user communities. As an internationally supported standard, training, support and implementation guidelines are available to facilitate the use of CAP in new and existing warning systems (e.g. WMO 2013).

The following points highlight where CAP-enabled alerting can provide beneft:

# *3.3.4 Infuences on the Warner*

People who issue warnings are infuenced by many factors, both individually as a person and corporately as a member of an organisation. These range from the quality and quantity of available information to their role and responsibility in providing science advice (legislative), to managing perceived risks from over-warning and warning fatigue by the public, through to personal and group psychological biases. Warners should be familiar with the nature of their audiences, including the way in which warnings will reach them. While helpful in general, this can easily lead to biased warnings and ultimately to a degraded response. Many procedures (e.g. tick boxes, thresholds and templates) are in place to help overcome biases and other infuencing factors, and to retain consistency over time and between different warners and agencies (Fig. 3.3).

The legal, institutional and political context can also infuence the warner and his/her behaviours and attitudes (see Chap. 2). For instance, in some countries there may be an increase in legal conficts related to the dissemination, use and interpretation of risk, forecasts and warning information (Altamura et al. 2011) and, more generally, in disaster risk management (Lauta 2014). This can lead to defensive, self-protective behaviours of warners to avoid personal blame and liability. For example, following the trial and prosecution of scientists and offcials in Italy following the communication of risk information and a subsequent impactful earthquake in the town of L'Aquila, the number of false alarms was observed to increase in Italy (Altamura et al. 2011), with consequent impacts on weather-related decisionmaking. On the other hand, a US National Weather Service policy decision to reduce false alarms for tornadoes by switching from county-based to storm-based warning areas resulted in a dramatic reduction in the size of warning areas (Sutter & Erickson 2010). The institutional framework can also weaken the warning process, for

**Fig. 3.3** Reaching a plethora of receivers with different needs from a single information source through multiple channels using the Common Alerting Protocol. (Source: Eliot Christian, based on ITU 2019)

instance by failing to defne stakeholders' distinct roles and responsibilities (Thorpe 2016). This can impede collaboration and cooperation between the actors involved in the warning chain. Institutional priorities may also determine whether warnings are primarily aimed at reducing impacts on individuals or on society as a whole (Potter et al. 2021).

Technical and fnancial constraints mean that a warner is almost certainly working under time pressure and with access to information limited by the speed of a desktop computer. They usually have access to only a subset of the observational data used in the forecast, and to a limited range of forecast products, especially if the forecast has come from a global forecasting centre. They may be trying to provide direct information by phone to key individual responders while preparing their warning for the public and a script for a live broadcast. They should also be monitoring the current state of the hazard and information circulating in the media, including social media. Under these pressures, the warner's attitudes and behaviours will be infuenced by a lack of resources, inadequate communication protocols and ineffective engagement with the media and the private sector (UNISDR 2013, NOAA 2016, Handmer & Dovers 2007).

Timeliness is of particular importance to the user since response actions take time and must be completed ahead of the arrival of the hazard. When forecast information is produced across several organisations, such as a weather service, a food agency and an emergency response agency, there may be a substantial time delay before the warner even receives the information. This can be exacerbated with impact-based forecasts and warnings, which may require information from several agencies to be collated (Potter et al. 2021).

Behaviour may also be affected by organisational competitiveness, whether for proft, funding, prestige or power (such as with other ministries, businesses or nations), by inappropriate organisational or individual target setting (e.g. concentrating on volume or accuracy over user value) or by excessive protectionism (including requiring that everything is produced in-house) or outsourcing (so that internal roles are devalued). Misinformation on external media can be particularly challenging; interventions to correct false information carry the risk that both the false information and the correction distort the original message.

Warners, and experts in general, tend to perceive risk differently from each other, and from lay people (Bostrom 1997). While experts tend to use formal defnitions of risk and emphasise the magnitude of the hazard and its likelihood, members of the public may include other factors, such as the familiarity of the threat and dread (Fischhoff et al. 1978). The social and institutional contexts of risk are important, such as motives and values. Experts also translate verbal probabilities (such as 'likely') to numerical probabilities (such as 70%) differently from non-experts, highlighting the importance of using probability translation tables (Doyle et al. 2011) as encouraged by the WMO (WMO 2008). Interpretations of probabilities also differ between experts and non-experts, and even within expert groups (Kox et al. 2015). The decision-making that goes into assessing risk inevitably requires a series of judgements to be made, to refne the circumstances or scope of the situation.

Many psychological biases can infuence judgements at an individual or group level. In a setting of scientists/warners making decisions, these can include the desire to conform to the group (Asch 1952), the infuence of a minority (Crano & Chen 1998), the groupthink phenomenon (Janis 1982), obedience to authority (Milgram 1974) and potentially the presence of an audience affecting performance (Dashiell 1930). The ways in which questions requiring judgement are asked can infuence the outcome, with care needing to be taken to not bias the result (Potter 2014). Another infuence on warner decision-making is avoidance of potential false alarms. While warners are often very concerned about the 'cry wolf' effect of false alarms, several studies have shown that their effect can be minimised by careful messaging before, during and after events. For example, Kox and Lüder et al. (2018) show that emergency managers state their discomfort about warning fatigue and high false alarm rates but indicate that they would respond to the warning as common practice. In general, policymakers are much more concerned about misses than false alarms. Various alternatives exist to support decision-making, such as a structured approach (e.g. Bayesian), or a more naturalistic approach (Doyle & Johnston 2011).

# **3.4 The Bridge Between Warner and Receiver**

# *3.4.1 Building a Relationship*

Those issuing warnings must understand the information needs of the receivers. This is achieved by building a relationship between the receiver, the warner and any other stakeholders involved in the process. This relationship can take many forms, depending on the laws and culture of the country, the governance structure of the warning organisation and the nature of the receiver – whether an individual, an organisation or a whole community. Relationships take time to build, require active management to thrive, depend on fexibility and compromise on all sides and ultimately need to commit to solutions that are benefcial to all parties. Relationships are equally necessary whether the warner is in a public or private sector organisation, and whether the receiver is a member of the public, a global business or a nongovernmental relief organisation. However, the nature of the relationship will be different in each case. Sustaining the relationship requires continued and regular two-way interaction, as personnel change and technical capabilities evolve, including periodic training and exercising.

The level of engagement underpinning these relationships ranges from one-way informing (e.g. a radio announcement about a hazard/event) to 'empowering' the end users to make the decision (as described by the International Association for Public Participation; https://cdn.ymaws.com/www.iap2.org/resource/resmgr/pillars/Spectrum\_8.5x11\_Print.pdf). There is a vast literature maintaining that effective warning communication is achieved through a two-way exchange of information between parties and that involving stakeholders in the communication, planning and discussion stages can increase their commitment, and ultimately foster the adoption of effective behaviours (for a review, see Macintyre et al. 2019). Indeed, this relationship and, in general, the whole warning communication process are circular and two-way as it also depends very much on the environmental, social and technical context, as well as the degree of partnership among the involved stakeholders. Twoway exchange of information also includes co-design and co-production of information, in which the receiver's role shifts from pure user to collaborator and partner (Kox, Kempf et al. 2018).

There are multiple successful examples of co-design of people-centred early warning systems and citizen science engagement processes (Preuner et al. 2017, Scolobig et al. 2016). These show how it is essential to adopt a warning value chain approach to engage with multiple stakeholders and to co-design warning system options. They also provide evidence that the inclusion of different stakeholder perspectives on the technical, social, economic, legal and institutional characteristics of a warning system helps to address decision stalemates and conficts, e.g. about funding allocation or priorities.

Yet, despite many authors highlighting that communication should be a two-way exchange of information, there is an operationalisation gap in many countries. The current literature and established practices suggest that most warning communication with the public remains unidirectional, from decision-maker to an uninvolved public, rather than a dialogue. Without doubt, there are multiple barriers to true two-way communication (see also Sect. 3.3.3). For example, given the large number of sectors and groups receiving warnings (as described in Sect. 3.2.1), it is diffcult for agencies issuing warnings to build relationships and establish multidirectional engagement with all of them. Where stakeholder engagement is used in the codesign process, it can be diffcult to involve some vulnerable groups such as women with young children, housebound elderly or those with chronic illness, and even when present, for these groups to feel equally engaged, especially if they lack good language skills. A combination of group meetings and individual interviews is needed, but community acceptance of such distributed inputs requires a high level of transparency in presenting and interpreting the evidence. It especially requires dedication of time and resources to develop successful processes. Co-production is also not suited for all kinds of warning communication. While it has proven successful for planning and evaluation, e.g. co-design of warning system options and evaluation of warning communication effectiveness, it is more challenging for fast two-way communication. This is currently the preserve of a small but rapidly growing group of receivers, such as storm spotters and organised groups of amateur observers (see, e.g. Elmore et al. 2014). Combining the concept of trained storm spotters or 'trusted spotters' with quality management of the data received was rated best practice by the European Meteorological Society (Krennert et al. 2018). Where funding is available for a tailored service to a specifc business or sector, the opportunities for building a relationship are much greater than for a general public service. Yet, they are often missed, with available resources focused on technical capability at the expense of achieving a mutual understanding of the problem and its solution. With the growth of web-based warning services, tailoring has begun to be feasible for a much wider range of receivers. Tailoring by location has been in use in the USA for many years, frstly on NOAA Weather Radio and more recently on cell phones (Wireless Emergency Alerts), targeting messages to individual cities or counties (NOAA 2021). The ability to set locations of interest is also well established, for instance in the Red Cross warnings app (e.g. British Red Cross 2021). However, the potential for much greater use of targeting is shown by its increasingly sophisticated use in online advertising, for instance by Google and Facebook. Such targeting lies not only in selecting the information of interest but also in presenting it according to the selected or revealed preferences of the receiver. Ethical issues surround the use of this technology, but as it gains wide acceptance through online platforms like Google, there will undoubtedly be increasing application in the communication of tailored forecasts and warnings.

An effective long-term partnership provides an environment within which to build up many of the conditions for successful use of warnings, including education and training between events, shared experience and interpretation of historical events, mutual understanding of forecast capability and trust in the reliability of the warning system, all of which infuence the way a warning is both produced and received (Parker et al. 2009).

Two examples follow that exemplify current good practices, via the establishment of strategic partnerships aimed at strengthening the relationships between the warner and the receiver.

#### **Box 3.2 The Global Weather Enterprise** Sally Potter

The Global Weather Enterprise (GWE) 'is an enabling environment fostering global engagement between public, private, and academic sectors that share the common goal of providing accurate and reliable weather information and services that save lives, protect infrastructure, and enhance economic output' (World Bank 2019: 3) and is described by Thorpe and Rogers (2018). It focuses on increasing value through the full chain from scientifc research and observations to models, forecasts, products and services. It is an initiative aiming to acknowledge and address obstacles in the collaboration of these three sectors (particularly public and private), including funding pressures on the public sector, growth of the private sector capabilities and international fnancing structures. It encourages a fair market for weather forecasts and services, and clear governance of taxpayer-funded open-access data in relation to commercialisation.

Indeed, business can derive economic value from weather knowledge – estimated at \$US13 billion in a report by the US National Weather Service (NOAA/NWS 2017). Increasing the recognition of the value of underpinning publicly funded weather services in leveraging the private sector may help ease the funding pressures on the NMHSs (Thorpe & Rogers 2018). Additionally, it has been suggested that the public sector could make more use of private sector data, with the recognition that open-access data and longterm reliability of data be considered (Hayes et al. 2014). This includes exploring potential business relations with the private sector as part of pilot projects (World Bank 2019). The GWE also calls for clear roles and responsibilities, particularly around data ownership, as the private sector moves towards the provision of data services and away from infrastructure (Thorpe & Rogers 2018). At the same time, there are also opportunities and good practices related to the role played by the private sector that can be exemplary to improve warning communication. For a public audience, the ability to respond online to social media posts and media stories, and provide information through crowdsourcing mobile applications, enables a higher level of engagement than the traditional one-way communication of hazard information.

# **Box 3.3 NOAA's National Weather Service Strategic Goal: Building a Weather-Ready Nation**

Douglas Hilderbrand

In 2011, a series of extreme events across the USA, including tornadoes, foods, hurricanes and wildfres that killed over 1000 people, became the driving force behind a profound movement to improve the country's 'weather services'. The National Oceanic and Atmospheric Administration (NOAA) and its daughter agency, the National Weather Service, started a long-term, strategic goal to improve forecast accuracy, communication and delivery of information to the public. The strategic goal to build a 'Weather-Ready Nation' (WRN) galvanised the operational and research arms of NOAA to improve not only forecasts and warnings but also their value through better societal responses (Hilderbrand, 2014). Internally, WRN became the impetus to measure value not just from a forecast accuracy perspective but also by a societal outcome perspective. This change in mindset has resulted in four focus areas:

Some WRN successes include:

• Creation of storm surge inundation maps that better communicate where and how much storm surge can be expected.

**Fig. 3.4** Components of the NWS Weather-Ready Nation initiative © NOAA 2021

Beyond the changes made internally, mentioned above, WRN also became a commitment to collaborate at the community level (Fig. 3.4). Government could not achieve a Weather-Ready Nation alone, but rather needed to embrace external partnerships at the federal, state and local levels, and across industry, non-profts and other community organisations. To show its commitment to partnership and give others 'partial ownership' of WRN, NOAA launched the Weather-Ready Nation Ambassador programme in 2014. As of August 2020, NOAA has recognised over 11,000 organisations as ambassadors, sharing the goal of making communities ready, responsive and resilient to extreme weather, water and climate events. WRN Ambassadors are as diverse as community needs – from global corporations to small non-profts. With these WRN Ambassadors acting as force multipliers, weather safety and life-saving forecast/warning information can reach many more people in communities across the USA. The wide range of skill sets collectively across these ambassador organisations allow for innovative collaborations with NOAA and even other ambassadors.

Looking ahead, WRN continues to build momentum on tough challenges such as quantifying and communicating forecast certainty, folding probabilistic forecasting into the decision-making process and fnding new ways to inspire the public to take appropriate actions so that when extreme weather threatens, communities will be ready.

# *3.4.2 What Works?*

The content and format of a message differs depending on varying user needs, as does the channel through which they would like to receive it, and the timing and frequency of updates. As described in Sect. 3.1.1., extensive user engagement is required to understand these needs in advance, to ensure the information is received in a useful and usable way (e.g. Becker et al. 2019, Kox, Kempf et al. 2018).

The warning message directly infuences people's warning response. Message content and style are thus important factors in determining whether people take selfprotective behaviour or not. In order to be effective at inducing such behaviour, a message should contain the information elements of hazard (nature and magnitude), location (area affected by the hazard), time (occurrence time or time to impact), guidance (action recommendations) and source (Mileti & Sorensen 1990). Previous research also shows that, to be effective, warnings should describe the exact nature of the threat (including potential impacts), provide a source of confrmation and be personally relevant (Weyrich et al. 2018, Lindell & Perry 2012, Mileti 1999). Technology developments are increasingly allowing the inclusion of links to further information, facilitating faster information seeking and response decision-making. The order of these elements and the length of the message can themselves infuence responses, with social science research showing that relative to shorter messages, longer messages can reduce people's inclination to search for further information, which then shortens the delay before responding (Wood et al. 2017). In addition, each of the information elements should be addressed by the fve stylistic dimensions of a warning message, which are specifcity, consistency, accuracy, certainty and clarity (Mileti & Sorensen 1990).

Familiarity and education have a key role to play. When a response is practised frequently, it becomes an almost automatic reaction. Fire alarms are a good example. Practising fre evacuations is mandatory for organisations in many countries, to ensure that employees not only recognise the warning but know what to do without thinking about it. Most hazard warnings are more complex than that but achieving an initial reaction through familiarity is still important. Beyond that, the response will be informed by knowledge, which requires education. The best time to learn is at school, but education about hazards and their impacts, warnings and their capabilities should be continuous and focused especially on those lacking familiarity with the area they live or work in. Education may take many forms, ranging from talks and workshops, through paper exercises and online games, to drills and

full-scale exercises (ITU 2019). These build on one another and can be targeted at different stakeholders. Opportunities for knowledge reinforcement ahead of periods of increased risk should be taken, whether provided by changes in the seasons or by long-range weather forecasts. A critical issue is to mainstream warning-related learning units designed for different age groups in different subjects in order to guarantee that risk education is part of the curriculum.

Weather forecasts and warnings are inherently uncertain. Yet, reducing uncertainty is not the same as managing the effects of scientifc uncertainty and communicating it (Brashers 2001). Notably, reducing and communicating uncertainty require completely different types of knowledge and expertise, ranging from natural to social sciences. Research shows that people understand that there is uncertainty inherent in weather forecasts, as well as that different scientifc factors shape this uncertainty (Joslyn & Savelli 2010). With respect to communication, some researchers argue that communicating scientifc uncertainty (e.g. in probabilistic forecasts) leads to better outcomes (Fischhoff & Davis 2014), as it will increase understanding (Stirling 2010). LeClerc and Joslyn (2012) found that providing weather forecasts (of high consequences but low probability) with uncertainty information enhances the chances that users take precautionary action. However, conclusions about how scientifc uncertainties should be communicated (quantitative vs. qualitative, graphs/ maps vs. text, etc.) are not yet clear and depend on the user, the context and the type of information.

Technological improvements continually reduce the level of scientifc uncertainty in weather forecasts and warnings. Over the last decades, messages have been improved as warnings are made with greater accuracy, geographic precision and lead time. However, the information is not always clear, specifc or consistent. Information may be more or less available, information from different sources may be inconsistent or contradictory, and information can increase or decrease the perception of uncertainty (Brashers 2001). For example, many people do not understand the standard phenomenon-based warnings and have diffculties translating a 'heavy' rainfall warning (e.g. indicating 100 mm of rain) into effective impacts. Communicating specifc impacts, for instance on road and rail transport, and possibilities of delays, could improve warning effectiveness (Weyrich et al. 2018). Moreover, it is important that a warning message is consistent within itself and across different messages (Mileti & Fitzpatrick 1992). This means that the underlying meaning of a message and potentially the colours and terminology used are similar or uniform, including from different providers at a given point in time (Williams & Eosco 2021).

Unlike in some weather forecasts, including probability of rain forecasts, there is almost no room for the communication of uncertainties within most current public warning systems. Warnings are issued by a forecaster when expected weather events reach a subjective level of certainty. The inclusion of probabilistic information could enable the forecaster to better communicate the varying degrees of certainty associated with each warning situation. However, the price of this type of probabilistic information can be the risk of misinterpretation or lack of understanding within the target audience (Kox, Kempf et al. 2018). One means of dealing with these complications that has gained credibility in climate forecasting is to develop storylines based on forecast scenarios (Shepherd 2019). For instance, during December 2011, an extratropical cyclone was forecast to move up the English Channel, producing extensive snow in central England (Mylne 2012). However, within the range of ensemble predictions, there was a low probability that the cyclone would turn north, moving the area affected by snow and allowing damaging winds to affect the extreme south-east of England. Rather than limit his presentation to the most likely forecast, or attempt to present probabilities, the TV forecaster chose to convey the uncertainty by presenting these two scenarios as alternative storylines of likelihood and impact.

Communication of uncertainty is further complicated because it is multifaceted: not just scientifc but also social, legal, institutional and political uncertainties affect how a warning is perceived and acted upon. Moreover, perceptions of uncertainty vary between people and social groups: one person may have an amount of information that other people would deem suffcient to make a decision, yet she/he may still feel uncertain on what to do (Brashers 2001).

While the general inclusion of probabilistic information in public warnings remains challenging and contentious, there is no doubt that it is an important component of the communication with many professional responders. In some cases, they have suffcient evidence to calibrate their actions directly on the probability of a specifc threshold being crossed. In these cases, a role of the relationship is to identify these thresholds and the nature of the uncertainties, so that the warner can ensure that the information reaching the receiver contains the required level of probabilistic detail, and that it is unbiased.

#### **Box 3.4 Probabilistic Weather Information for Emergency Managers in Germany** Nadine Fleischhut

Although probabilistic forecasts are key to informing decisions under uncertainty, probabilistic weather forecasts are still rarely communicated to lay audiences due to fears that they are diffcult to communicate clearly and that lay people may be reluctant to use them. Yet there is growing evidence that people understand probabilistic information if it is presented transparently (Hoffrage et al. 2000), that it can improve decisions (e.g. Joslyn & LeClerc 2013) and that it is preferred by the public (Morss et al. 2008). In contrast, deterministic warnings can hinder informed decisions, since forecasters must issue warnings without knowing the needs of their users, who are left to guess the uncertainty of the forecasts (e.g. Fleischhut et al. 2020, Joslyn & Savelli 2010). Probabilistic forecasts, however, enable everyone to apply the decision thresholds that ft their needs.

In Fundel et al. (2019), we described how we evaluated the usefulness of probabilistic forecasts for emergency managers' decisions in Germany. As a large and diverse group, their required lead times and probability thresholds differ considerably (Kox et al. 2015), likely refecting varying institutional constraints and capacities (Demeritt et al. 2010).

Using a new approach, we designed FeWIS Pro: fve uncertainty representations implemented in parallel within the fre brigade weather information system (FeWIS) of Germany's national meteorological service. The representations display forecasts for wind, rain and thunderstorms (48-hour lead time) as probabilities for binary events (exceeding warning thresholds) or as probability thresholds for continuous variables (e.g. for wind or precipitation). All representations were designed based on evidence from risk communication research in a range of felds, including medicine. The approach made it possible to observe and quantify emergency managers' preferences under real operational constraints and over a longer period of time.

We analysed which representations emergency managers preferred for two severe storms (Xavier and Herwart) in Germany in October 2017 (see Fig. 3.5). In general, emergency managers used probabilistic forecasts frequently, indicating that they found the forecasts informative and useful. During both storms, the most frequently consulted representation was a map displaying probabilities for exceeding warning thresholds. The map, which provides a clear overview of the areas most likely to be affected by the storms, may be useful for emergency managers coordinating emergency services, vehicles and personnel.

**Fig. 3.5** Search for information before, during and after storms Xavier and Herwart. Plots show how often each representation was selected by time, from 48 hours before to 24 hours after the storm. Dotted lines mark when the frst weather watch was issued; solid lines mark the frst weather warning. Red lines along the x-axis indicate when the storms passed through Germany. For each storm, the analysis includes the behaviour of emergency managers who should expect to be affected frst by the storm, defned here as all users for which the 90 percentile of the forecast was ≥110 km/h during the frst third of the storm (*N* = 93 collective users for Xavier; *N* = 114 for Herwart). Overall, they selected representations 439 times during Xavier and 722 times during Herwart. For details, see Fundel et al. (2019)

The other four representations presented timelines for a selected area. Of these, the representation of probabilities of exceeding warning thresholds was used most frequently during Xavier. Box plots showing the likely range of wind speeds were used most frequently during Herwart; they were only slightly less popular during Xavier. Quantile information (e.g. box plots) was consulted frequently shortly before – and even more so during – a storm.

Emergency managers' focus on warning thresholds may refect that their main duties are reactive and that warnings trigger a range of decisions, such as the declaration of an emergency. Representations displaying probabilities of exceeding warning thresholds may thus be particularly useful during an early weather watch. In contrast, quantile plots such as box plots make it easier to evaluate how wind speeds might develop; they may therefore help emergency managers maintain the ability to respond during an event and prepare for daily operations afterwards (see also Kox, Lüder et al. 2018).

FeWIS Pro was developed as part of the interdisciplinary research project WEXICOM, funded by the Hans Ertel Centre for Weather Research. WEXICOM aims to improve the communication, understanding and use of uncertainty in weather and impact forecasts. The work and results reported here have previously been published in more detail in Fundel et al. (2019).

# *3.4.3 Measuring Success*

The question of what makes a warning successful is one of the key aspects identifed in the HIWeather context (Zhang et al. 2019). Agreeing how this will be assessed is critical to the relationship between warner and receiver. Transparent sharing of evidence and its interpretation is essential. Yet often the producer assumes that it is suffcient to verify the observable components of the information content, using statistically correct methods, and is then surprised if the receiver has a quite different perception of the value of the warning service. While the roots of successful warnings lie in having accurate meteorological information, its value depends on applications of the social, behavioural and economic sciences (Zhang et al. 2019), thus becoming an interdisciplinary matter.

To be successful, the warning must enable its recipients to make the right decisions to protect themselves and their communities (Golding et al. 2019; Taylor et al. 2018). In order to achieve an effect in the sense of risk-reducing behaviour through warning communication, warnings must generate attention (wake-up signal), indicate potential impact and give guidance for adequate response.

Recipients may differ in their needs and requirements for weather information and warnings, subject to their responsibilities and competencies. Thus, what constitutes useful or 'good' information varies according to the areas of activity an end user represents. In an essay on the goodness of weather forecasts, Murphy (1993) mentions three general types of forecast goodness: consistency, quality and value. In order to be consistent, the forecast (oral or written) should be the best possible estimate or assessment of the weather situation by the forecaster. The forecast may be inconsistent if it contains (more or less) spatial or temporal specifcity, or if the uncertainty in the judgement of the forecaster is not accurately refected in the corresponding forecast, either in words or numbers. Quality refers to the degree of agreement (or similarity) between the forecast and observed events, expressed in terms of distortion, accuracy or skill. Finally, the value refers to an increase in beneft to a forecast user as a result of using the forecast. End users of forecasts and warnings place particular emphasis on the value aspect of a forecast's goodness (Kox, Kempf et al. 2018). Economic perspectives are most commonly used to defne the value of weather forecasts and warnings (e.g. Lazo et al. 2009), but weather forecasts do not have an intrinsic value in an economic sense, they rather have a specifc value for a user when he/she takes measures, and these measures avoid or reduce damage costs (Murphy 1993, 1994, Mylne 2002). Kox and Thieken (2017) add that a purely economic perspective does not apply to situations where monetary damage costs are diffcult to allocate, such as loss of life or social or political prestige. In other situations, people may want to act, but may not be able to do so due to professional constraints or limited resources.

Fire brigades are a good example of such an end-user group operating outside of a simple cost-loss analysis, while road maintenance services responsible for salting roads have clearer cost calculations (Kox, Kempf et al. 2018). Accordingly, the enduser perspective may vary on what is understood as the goodness of a forecast or the success of a warning. A warning message that is of high value to one user may be useless to another. In this context, the ability to tailor warnings to individual needs (Joslyn & Savelli 2010) and to provide access to additional meteorological information is of importance to achieve high value for all end users.

Keeping all that in mind, the measurement of success is a diffcult task. It gets further complicated by the possibility of 'grades of success' (Golding et al. 2019), for instance in near-miss situations, when the spatial or temporal extent is only slightly in error (Sharpe 2016), or in situations where no damage occurs due to successful warning. For example, in the case of road icing, observations of road state will not show that the road is covered in ice if it has been treated in advance, though the temperature may still show the road surface to be below freezing. In this instance the meteorological trigger for the hazard (sub-zero temperature) is verifable, but the hazard itself (slippery road) is not. Here, the absence of such impacts might be an indicator of success especially if other impacts of the hazard are observed.

Continuous evaluation of warnings is essential if the benefts of improvement are to be identifed and any degradation is to be arrested quickly. Changes in technology and external conditions can otherwise lead to a warning system rapidly losing its impact. Regular surveys of a range of users, especially following the issue of warnings, should be followed up with one-to-one interviews to identify negative issues before they lead to a loss of trust, and to reinforce positive changes.

To improve weather warnings in all three dimensions of forecast goodness, a broad range of challenges need to be addressed. A strong collaboration between the main users and the national weather services in the form of an ongoing dialogue and discussion of critical needs is important for the success of weather warnings (Kox, Kempf et al. 2018). This dialogue should be particularly active following any unsuccessful or only partially successful warning, to bring together the perceptions of the two sides and to conclude a joint evaluation. In parallel, it is also important to explore new options (e.g. for warning communication) from the perspectives of different stakeholders and to test these options theoretically and sometimes practically, to determine what is critical to improve the process from warning to decision (World Bank 2019).

Relationships are not static, and it is essential that the success of a particular relationship is reviewed frequently. With rapid population growth and urbanisation occurring in much of the world, vulnerabilities are changing, while in a warming climate, the hazards to which audiences are exposed are changing, too. And with new technical capabilities, both the forecast quality and the ability to communicate them are shifting. Among all these external changes, the lifestyles and expectations of warners and receivers, and the structures and personnel of the partner organisations, are also evolving. For a relationship to survive and thrive in this dynamic environment, it requires active management – including an open channel for feedback on any aspect of the warning service. However, it is also important to periodically review the health of the partnership: is the relationship still the right one? Does it have the right membership? Are the outcomes improving? Is the cost affordable? All of these questions should be addressed periodically – perhaps every 5 years – and if the answers are 'no', the relationship needs overhauling. Below is an example of rapid changes in the development of an early warning system.

# **3.5 Example**

**Box 3.5 Community-Based Early Warning System in Nepal** Dharam Uprety and Bikram Rana

Based on work undertaken in 2018.

Extreme and regular fooding in Nepal results in signifcant loss of life, property and livelihoods (Practical Action 2009, 2020; Bhandari et al. 2018). In response, several food early warning systems (EWSs) have been established, in an attempt to reduce the number of people affected and killed by foods. However, there are still challenges in these systems, especially in communicating food warning to the most vulnerable and ensuring they have the skills and resources to be able to respond:

"If we can get information before the foods come, it will save our lives. We may not be able to rescue everything, but our children and families will be saved."

The roll-out of food EWSs began about 20 years ago and was initially focussed on manual observation towers. These towers provided peace of mind for people in their immediate vicinity but had numerous limitations such as maintaining observers 24 hours each day as well as communicating warnings during heavy rainfall events. Their success created the momentum to work with the national Department of Hydrology and Meteorology (DHM) to explore more comprehensive systems, with expansion into real-time river level monitoring, automation of gauge stations and subsequently the adoption of technologies including forecasting and SMS messaging. The evolution of the system is indicated in Fig. 3.6.

The evolution of the EWS has been marked by parallel improvements in the local area. For example, social capital has been enhanced between the upstream and downstream communities, with a substantial number of households informing their neighbours immediately after receiving food warning, and this is the primary warning source for many households. The increased warning time gives people more time to respond; human capital has been built as individuals have adapted from 'fright and fight' to learning what to do, including moving vulnerable assets to high ground or stored on upper foors prior to evacuating, with these responses reinforced by annual drills. This additional time has enhanced learning, and community members have constructed simple mitigation measures, such as waterproof storage facilities in their houses for grains and less movable assets, improving their physical

**Fig. 3.6** Development and use of technology for food early warning systems in Nepal

(continued)

capital. As local communities have experienced the positive benefts of the EWS, some people have started to express a willingness to pay for the EWS services, offering a boost in fnancial capital and perhaps a long-term and sustainable mechanism to cover the EWS operational costs long term. This has been further enhanced with the two major telecommunication companies in Nepal (NTC and NCELL) joining hands with DHM to send free mass SMS warnings to all mobile users living in the food plains of major river basins during the monsoon:

"If we can make early warning effective and effcient, the necessity for rescue can be reduced – if we can manage better with what we have I don't think we need 'big' technologies, we don't need more resources and we don't need extra personnel."

However, the EWS is far from comprehensive and many challenges remain. Firstly, climate change is a huge uncertainty in the Himalayas with implications for the appropriateness and robustness of the evolving system. Secondly, while progress has clearly been made in the monitoring and forecasting, integration of sociocultural aspects needs to be strengthened that can make early warning information accessible to the most vulnerable. Thirdly, engagement and ownership with local communities must be maintained to ensure the system is co-designed to deliver information tailored to meet their diverse needs for rapid dissemination and timely protective action. Finally, for food risk communication to bridge 'the last mile' in terms of reaching the most vulnerable in the community, it must take account of their distinct social, economic and political experiences in both the content and the delivery of the information.

# **3.6 Summary**

We conclude the chapter with a checklist of aspects of the warning that need to be considered to improve the relationship between warner and decision-maker/receiver when designing or upgrading a warning system:

# **References**

content/uploads/Risk-Communication-and-Behavior-Best-Practices-and-Research-Findings-July-2016.pdf (Accessed 2/9/2021)

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 4 Connecting Forecast and Warning: A Partnership Between Communicators and Scientists**

**Cheryl L. Anderson, Jane Rovins, David M. Johnston, Will Lang, Brian Golding, Brian Mills, Rainer Kaltenberger, Julia Chasco, Thomas C. Pagano, Ross Middleham, and John Nairn**

**Abstract** In this chapter, we examine the ways that warning providers connect and collaborate with knowledge sources to produce effective warnings. We frst look at the range of actors who produce warnings in the public and private sectors, the sources of information they draw on to comprehend the nature of the hazard, its impacts and the implications for those exposed and the process of drawing that information together to produce a warning. We consider the wide range of experts

C. L. Anderson (\*) · J. Rovins

D. M. Johnston Massey University, Palmerston North, New Zealand

WMO/WWRP HIWeather project, Geneva, Switzerland

W. Lang · R. Middleham Met Offce, Exeter, UK

B. Golding Met Offce, Exeter, UK

WMO/WWRP HIWeather project, Geneva, Switzerland

B. Mills

University of Waterloo, Environment and Climate Change Canada, Waterloo, Canada

WMO/WWRP HIWeather project, Geneva, Switzerland

R. Kaltenberger Zentralanstalt für Meteorologie und Geodynamik, Vienna, Austria

WMO/WWRP HIWeather project, Geneva, Switzerland

J. Chasco National Meteorological Service of Argentina, Buenos Aires, Argentina

WMO/WWRP HIWeather project, Geneva, Switzerland

T. C. Pagano · J. Nairn Bureau of Meteorology, Melbourne, Australia

© The Author(s) 2022 87 B. Golding (ed.), *Towards the "Perfect" Weather Warning*, https://doi.org/10.1007/978-3-030-98989-7\_4

Massey University, Palmerston North, New Zealand

who connect hazard data with impact data to create tools for assessing the impacts of predicted hazards on people, buildings, infrastructure and business. Then we look at the diverse ways in which these tools need to take account of the way their outputs will feed into warnings and of the nature of partnerships that can facilitate this. The chapter includes examples of impact prediction in sport, health impacts of wildfres in Australia, a framework for impact prediction in New Zealand, and communication of impacts through social media in the UK.

**Keywords** Warning producer · Impact · Communication · Social media · Trust · Information broker · Tailored warning · Evaluation

# **4.1 Introduction**

This chapter examines the ways that warning providers connect and collaborate with impact experts to improve and communicate warnings. We frst look at the role of the warner, then the impact forecaster and fnally the linkages between them. In doing so, we shall see that:

# *4.1.1 Warnings*

Warnings are produced by a wide range of actors in the public and private sectors, based on information from weather and hazard forecasts, on science related to weather or hazards and on estimates of the anticipated impact of the hazard, produced using a variety of tools and technologies. The intent is to provide enough lead time to reduce the risk from the hazard and thus prevent a disaster. Expert risk information must be presented in a form that enables the creation of a convincing warning narrative that ultimately supports decision-makers and encourages action. Research in the last decade has demonstrated that strengthening warnings with impact information signifcantly reduces the disaster (WMO 2015, Casteel 2016, Anderson-Berry et al. 2018, Potter et al. 2018).

In producing a warning, the warner is as much an artist as a scientist, crafting a persuasive story out of a selection of uncertain facts, using their experience of context and precedent and ftting the result into a variety of formats to be delivered

**Fig. 4.1** Interactions from the Warner to Impact Expert embedded in complex systems. (Source: Adapted from Ruti et al. 2020; Beaven et al. 2016; Pielke Jr. 2007)

through different media, all while under considerable time pressure. Warning information must reach decision-makers in broad, varied disciplines and felds of service and expertise. The available information is constrained by forecasting capabilities, which vary with lead time and location and which may be more or less relevant to the end user. Temporal scales of the potential threat and the time taken to issue warnings add further complexity. Partnerships that help the warner to have confdence in the sources of their information are critical.

The "bridges" connecting the information provider to user may be complex because of their situation within complicated, embedded systems and should therefore be designed prior to hazard occurrence (see Fig. 4.1). Research shows that intermediaries, which may be organisations or individuals within an organisation, can aid in connecting people throughout the warning chain.

# **4.2 The Warner and Warning Information**

# *4.2.1 The Warner*

The warner of each potential threat will vary with the type of threat and the roles that defne positions in organisations and governments, including authority to provide warnings and the systems in which the threat is evaluated. Warnings may be categorised by the type of threat (hazard type and complexity, science and technology that provides analyses), by role (forecaster/nowcaster, modeller, public information offcer/risk communicator, emergency manager), by discipline (hydrology, meteorology, physical science, social science), by authority (Meteorological Service, emergency management, government entity) and by geography (local, regional, national or transboundary scale of the threat). The ways in which the warning is issued depend on all of these factors with additional consideration for the means of warning – offcial press release, television/radio announcements, sirens, SMS/DM mobile device alerts and social media. These researchers, technicians and operators, information systems and technologies are integrated into early warning systems (EWS) which aid in reducing disaster risks (Tan et al. 2020a). Experience in the UK demonstrates the importance of institutional trust in warnings, which is enhanced with impact-based warnings (Taylor et al. 2019).

Warnings may be entirely automated without any human input if a warning system has suffcient information and analytical skill to make warnings reliable and if they can be communicated appropriately. Some situations – such as very short notice warnings – are perhaps better suited to the automated approach, but many of today's weather and natural hazard warning systems require a mixed approach, with the human adding to the automated system, either "in the loop" or "over the loop" (Pagano et al. 2016). Issuing warnings of this type requires expertise in science and the art of communication in equal measures. It is not a purely *mechanistic* process, which can be easily automated, and there will always remain an element of subjectivity, but it should at least be a *methodical* process. The methods adopted will vary depending on circumstances, but all should look to ensure a balance between the scientifc, the practical and the useful and should ensure a level of consistency by limiting differences of opinion, biases and risk appetites.

Invariably, no one person has all of the expertise, information or experience across these felds, which is why warning creation needs to be a collaborative and multi-disciplinary process. The resulting diversity of perspectives, experiences and insights is both a strength and a challenge of this collaborative approach. The warner operates within guidelines of EWS design, which may be an automated system managing big datasets and information sources, or a human forecaster analysing hydrometeorological conditions, and/or may further contain interpretation of the nature of the impact. The means of communication and the target audience must also be considered, together with wider aspects of decision-making from the individual to broader governance systems (Tan et al. 2020b).

# *4.2.2 Warning Content*

The warning must focus on what the warner is trying to achieve with the warning. Information is needed both as warning content and for decisions on the importance and timing of the warning. The warner must trust their information sources (offcial and unoffcial), be able to select key aspects of information received and determine ways to interact with information to produce the warning in different situations and contexts.

"An effective warning…specifes the exact nature of the threat" (Casteel 2016). The content should be clear and understandable. The more that the warning includes information about the hazard impact, the more actionable it is and the more effective

the warnings become (Lazo 2020; Potter 2018). "The increased specifcity provided by the "hazard" portion…should therefore enhance personal relevance and potentially increase the likelihood that the recipient takes protective action" (Casteel 2016).

Even though technological advances have improved the information available, it must be interpreted for sector-specifc use. Disaster managers and emergency responders need to know the nature, severity and geographical extent of the disaster, the timing (start and end) and how soon evacuations must occur. The agriculture sector needs to know the likelihood of a threat such as fooding, its geographical extent (how many crops will be affected) and scale (one district or several, crossjurisdictional boundaries) to effectively enact readiness measures that protect crops, livestock and agricultural livelihoods. The energy sector requires knowledge of the timing and type of event to ensure that energy can be supplied to users, including emergency responders and at-risk populations.

The information required by the warner evolves as the threat approaches. At the early warning stage, there are likely to be few sources, though they may reach the warner through multiple routes. At this stage there is also more time available to refer back to experts for clarifcation and to carefully craft a convincing narrative for the receiver. As deadlines for specifc actions approach, the message needs to be oriented to the impacts that are relevant to those actions, with emphasis on the level of confdence and on alternative outcomes. Once the threat is imminent, there is no longer time for careful analysis, but details of changes in track, intensity, timing and associated impacts may be critical to responders' actions and safety. At this stage the warner will look for multiple data sources, up and down and outside the warning chain, to maintain situational awareness: of the hazard, of responses to warnings and, once it arrives, of the actual impacts. These will all inform warning updates.

# *4.2.3 Warning Creation*

In preparing a warning, the warner aims to create a compelling narrative that will convince the receiver to take notice and then to take action appropriate to their situation. Selecting the warning level is a critical part of the process. It requires careful interpretation of the available information, especially the predicted impact and the level of confdence, as well as the context of the warning, for instance, if those affected are already dealing with the impacts of another hazard. The content of the warning is selected and presented with all this in mind. Central will be the expected impact on the receiver. In support will be suffcient information on the causal hazard including both the prediction and its confdence, supported, when available, by evidence (e.g. links to CCTV at upwind/upstream locations). Where specifc vulnerabilities are relevant, these need also to be included, together with the level of confdence. Where there are signifcant uncertainties in the impacts, the range of outcomes may be usefully represented by two or more scenarios or storylines, while stressing the need for preparation ahead of the situation becoming clearer.

Taking a typical impact-based weather warning system as an example, it is useful to subdivide the warning process into distinct components.

*Weather assessment* → *Risk assessment* → *Change assessment* → *Utility assessment*

Most studies have focused on the weather and risk assessment aspects, but equal emphasis should be given to consideration of the latter components which are related to decision-making and communication.

The range of sources of information for the weather assessment can be very large, typically greater than can realistically be absorbed and processed by even the most experienced meteorologist. For this reason, the information must be fltered, either by limiting the sources used (procedurally or technically) or through intermediary systems which can sift and extract signals from, and summarise, the information.

A range of inputs may be used for the impact/risk assessment:

One of the most diffcult challenges for the warner is to acknowledge that their assessment has changed suffciently to change the warning. Once a hazard "story" has been defned, it possesses inertia; it can be diffcult to accept that it no longer refects the best interpretation, even in the face of new, conficting data. This is the psychological phenomenon known as "anchoring", which, among other things, is why forecasts and warnings can be most prone to change following handovers between shifts.

Even when the warner feels that other criteria for issuing a warning may have been met, they must consider whether the warning information will be useful to end users. This consideration acts as another flter, with the warner playing an editorial role to determine what user(s) need to know. For example, matrix impact-based warning systems should arguably result in far more long-period, low-probability warnings than they do.

Warnings are not issued in isolation. They exist in relation to other warnings, to additional communications and of course to the previous and future versions of warnings for the same event. While a warning to take protective action now for high weather impacts in the next 6 hours may be usable by some recipients, it gives little time for preparation. In this sense, no warning should come as a surprise! So the warner should use a succession of communications: advisories, watches, warnings, press releases, blogs, tweets and other advice to manage uncertainties and expectations well before the event, such that the fnal "take action now" message is expected when it comes. Best practice is to think strategically over the whole period from

initial indications of severe weather up to the event itself and plan the issue of warnings and other communications to best inform users and allow them to prepare while avoiding overcommitting resources should the worst conditions *not* occur.

The key to preparing high impact weather warnings is the development of expertise: in the needs of the warning's users, in the reliability of the various sources of guidance (both meteorological and socio-economic) and in the behaviour of the weather system causing the hazard. Expert warning forecasters assimilate the incoming data, generating a conceptual model of the situation and enabling a level of situational awareness suffcient to anticipate events (Klein 1989). Anticipation enables the expert warner to flter the voluminous information and focus on the most relevant aspects for use in decision-making. Decisions are always subject to judgement in the face of uncertainty. Given the uncertainty and the impact of warning decisions, there is never enough information.

Looking forward, research is focusing on using machine learning systems to undertake the process of fltering, so as to identify the key areas of risk uncertainty requiring human judgement. One way of facilitating this is for hazard and impact predictions to be formulated into a frst-guess warning, combining the probabilistic and severity elements. In some situations, this might provide the route towards fully automated warnings, but more generally it should be accompanied by tools for the warner to probe the individual components, assess the sensitivity of the outputs and amend the resulting warning.

# *4.2.4 Tailored Warnings*

"Forecasts will occasionally take into account some societal factors (e.g., extending a warning's timing to cover when schools are releasing students), but often do not directly account for human factors related to decision-making prior to, and during life-threatening extreme events" (Uccellini & Ten Hoeve 2019). Engaging with social scientists to work with specifc groups of users in the design of tailored warnings can lead to better warning responses. Research shows that tailoring warnings to the needs of recipients increases their effectiveness. However, this beneft, in better warning response, has to be set against the cost, complexity and potential for inconsistency of doing so. The beneft is not restricted to economics but may include a variety of non-economic benefts and ethical issues of human rights as well. We consider several aspects of tailoring here. The frst few options relate to selection of data, while the latter ones relate to presentation.

**Selecting the Best Forecast for the User** When working with a decision-maker to improve their access to predictions of hazards, the basis for selecting the source of that information will include several factors, of which accuracy or skill is likely to be an important one. On the other hand, a provider of information, trying to optimise the value to users of the information they provide, has to choose which improvements to their prediction system to implement. In both cases it may appear to be benefcial to focus the evaluation on the conditions of interest to the user, i.e.

the hazard itself or perhaps the extreme weather conditions that give rise to the hazard – both things that occur rarely.

If the exercise is being undertaken following a disaster, it may even be felt that the most important consideration should be that the disaster would have been predicted with the new system or data source. This approach can be very dangerous, typically leading to over-prediction and loss of confdence in the warning system. While there are unbiased methods of assessing extreme values of weather variables and of assessing the skill in predicting occurrence of an event, in both cases they are not immediately intuitive and so are not the most widely used evaluation approaches. In addition, the small number of data points available for extreme events means that the error bars in the score are likely to be very broad.

Even if different information sources are reliably identifed as giving the best guidance for different phenomena, the risk of inconsistent predictions is considerable. For instance, if one information source gives better hurricane track predictions and another gives better hurricane intensities, it would be foolish to rely entirely on the track from one and the intensity from the other. For the warner this means that decisions on the scenario for the warning should be based on as much information as possible about the current situation, about the evolution predicted by each source and about the performance of that source in similar situations. It is essential that this information includes full probability distributions and that biases have been removed.

**Selecting the Information to Communicate** Having obtained a skilful forecast source, there are a myriad of products that could be extracted. Information for inclusion in warnings is unlikely to be the same as that used for routine forecasting. For instance, extracting the probability of key thresholds being passed enables the user to focus immediately on their specifc concerns. Having said that, it is advisable to standardise if possible, so that users receiving warnings that contain different thresholds do not perceive an inconsistency. Other means of tailoring the information include recalibration, bias correction, and derivation of user-relevant variables (both physical and socio-economic). Thus a food forecast may be presented as fow in the river, as water level above the river bed, as a map of food depth, or more specifcally in terms of the depth on roads to the Highways Department, the extent of residential property fooding to the public, and the probability of reaching a critical depth at an electricity substation to a power company. In the pressured environment of an emergency response team, the more precise and actionable statements are likely to produce more effective responses with less room for error. It is recommended that warnings of the highest identifed risks should use tailored prediction products incorporating the probability of specifc hazard thresholds being crossed and information about exposure and vulnerability of communities in the threatened areas.

**Geographical Tailoring** When considering tailoring of information, making allowance for geography is perhaps the most obvious and most necessary. For instance, when forecasting wintry weather in complex terrain, low-lying valleys may have rain, while higher up the slopes, snow is accumulating. The meteorologist may use the height of the snow line, but to communicate this may require particular locations to be identifed. Proximity to rivers is a key driver of risk from fooding,

but few people know the distance to their nearest river and city dwellers may not realise that a river exists, let alone that they could be fooded by it. Warnings that refer to settlements at risk or escape routes that are safe or threatened may use names that are only current in the immediate vicinity, so that visitors are unaware of their meaning. For instance, reference to numbered highways or to numbered junctions on a highway will be incomprehensible to a portion of the travelling public. Maps can help in avoiding these diffculties, provided they are clearly located relative to major towns, highways and other widely known features.

**Tailoring the Communication** This area offers the greatest opportunity for tailoring with the minimum risk of confusion. For instance, warnings should be disseminated in multiple languages, according to the make-up of the population, and through different media (newspaper, TV, radio, Internet, mobile app, social media, etc.) according to accessibility by the population. It is crucial to use geographical names that are generally understood. Maps can be powerful tools for communicating the proximity of a warning area to dispersed communities – but only to those that are able to read them. Colours can provide powerful support to communication - the green-amber-red sequence of traffc lights is understood in many cultures, but not all, and care must be taken to cater for those with colour blindness. The use of cartoon characters to communicate has been very successful in some cultures, but not all. Since users will often seek confrmation from friends and family before responding, it is important that different means of communicating the information are consistent.

**Other Areas for Tailoring** Many aspects of warning design affect how particular groups receive, interpret and respond to information. Cultural cues can be important, e.g. the colour red has particular and conficting cultural meanings. Similarly, the idioms used in the language can be as important as the words. Phrases such as "snowing handkerchiefs" or "raining cats and dogs" are meaningful to some and not to others. Gender is of particular importance in most countries. However, when considering tailoring for women, it is necessary to consider the route by which the warning will reach them. A direct route, e.g. by social media, will require different tailoring from an indirect one, e.g. through a village chief or street warden. Another potential area of tailoring comes from study of the psychological response to challenging situations. Some people typically respond positively, seeking to turn it to their advantage, while others are followers of the crowd, and yet others will fght against change. In the West, marketing companies have learnt to target these groups differently, and it is likely that similarly targeted warning messages may be effective, though research has yet to demonstrate this.

**Tailoring for Specialists** Where there is an emergency manager for a large infrastructure facility that will affect thousands of people, the case for tailoring very specifcally to that role's needs is very strong. It is essential that the response is preplanned and that it is carried out quickly and effectively when the warning threshold is reached. This may involve simplifying the warning down to a simple code word, which is learnt and practised by each provider and user. The same holds for organisers of large public events such as pop festivals. Tailoring for major public

facilities such as schools or hospitals is more complex, but equally important. Candidate specialist users for tailored warnings include power generators and suppliers, water suppliers, dam operators, telecommunications operators, road transport, rail transport, air transport, marine transport, food retailers, education, emergency responders, health services, waste disposal, public event organisers, major employers and businesses. Such users should not be using generic public warnings to take decisions. They should have carried out a risk assessment for their business, which identifes the hazards they are exposed to and the level of risk for each. They should also have a risk response plan including trigger points at which action must be taken, together with the information needed, both to identify the trigger and to inform the action. The receipt of a tailored warning should be the primary trigger for preparatory actions ahead of weather-related hazards. Activation of response plans should be tied to the receipt of a warning.

**Co-Design in Generic Warnings** Currently a high degree of tailoring cannot be justifed for public warnings. The alternative is to co-design a compromise generic warning system that meets most needs and to use education to embed its characteristics in the users' cultures. Such co-design activities must be very carefully planned to ensure an adequate voice for all sections of the community. Evidence also suggests that a feedback loop is required in which community representatives frst identify what they feel are the problems with current capability and then criticise successive sets of upgrade options in an iterative fashion. Not only does this help to produce effective warnings, but it also builds a sense of ownership in the community that helps with the adoption and use of the fnal product. Looking to the future, social profling in combination with machine learning techniques, e.g. as used for selecting online advertisements, has the potential to enable individual tailoring of weather information based on individual risk profles. However, the implications of getting it wrong mean that warnings are likely to adopt such techniques more slowly than other environmental forecasting services.

# *4.2.5 Evaluating the Warning*

While evaluation is important throughout the entire warning value chain, it is particularly critical where risk information is translated into actionable messages for those at risk and those responsible for mitigating and managing hazard-related impacts (e.g. emergency managers). NMHSs have a long history of using statistical methods to verify weather forecasts (Ebert et al. 2015), but relatively less experience in evaluating the use and effcacy of their products and services.

While it would be desirable to demonstrate beneft by observing a decreasing trend in metrics of death, distress and damage as warnings improve, it is rarely possible to do this. Since the objective of warning is to help the recipient make better decisions, surveys of people's actual receipt and reaction to warnings are probably the best available tool. A baseline is required, so surveys should be designed and established before the introduction or upgrade of a warning system and continued

after it is complete, using the same format throughout. If surveys have been part of a co-design process, it may be appropriate to continue these, bearing in mind the difference between anticipated response and actual response.

By defnition, evaluations are comparative: over time; between places, jurisdictions and populations; or between other contrasting features or situations. For example, the introduction of a warning service or modifcation to add impact information should be evaluated using assessments before and after service implementation. Traditionally, one-off evaluations often take place long "after-the-fact" making it diffcult to collect and interpret the information. Ideally, evaluation should be undertaken continuously throughout the life cycle of any signifcant change, permitting course corrections as the service is developed and introduced.

A useful approach is to treat weather warnings as a form of programme intervention, not unlike a campaign to increase vaccination uptake or use of masks in disease prevention. Whether explicitly or implicitly defned, weather warnings are provided to infuence awareness, risk perception, behavioural intent, decisions, behaviours and, ultimately, outcomes—all of which can potentially be measured. The theory of change is a methodology for planning and evaluating social change programmes (see, e.g. Taplin & Clark 2012) that is now widely used in international development and is very relevant to the challenge of evaluating warnings. It approaches a social intervention of any kind by frst determining the desired outcomes and then associating measurable success indicators with each. It involves documenting the actors and processes through which a service is expected to affect outcomes, together with any intermediary factors (e.g. awareness and comprehension of warning information, trust, beliefs, etc.). The analysis may draw on personal experience, expert opinion or evidence and models obtained from social science research (e.g. Theory of Planned Behaviour, Ajzen 1991; Risk Information Seeking and Processing, Griffn et al. 1999).

The process of confrming a theory of change naturally leads to working hypotheses that may be examined and tested using qualitative and quantitative research. Each approach has strengths and weaknesses, so it is benefcial to adopt multiple lines of inquiry and triangulation over the course of an evaluation. For instance, observational feld research, focus group sessions and mental modelling interviews (Morgan et al. 2002) are often helpful in documenting change theories and underlying constructs among those who are developing, providing and utilising warning services. Surveys, however, may be better suited to assess the representativeness of such fndings across groups of actors (e.g. emergency managers, Hoss et al. 2018) and the effect of intermediary factors (e.g. trust, perceived threat) on behavioural intent and recalled responses and outcomes, particularly following memorable severe events (e.g. Winter Storm Doris, Taylor et al. 2019).

Experimental research using hypothetical or simulated scenarios allows for selective control of variables that might infuence protective decisions and so is particularly useful in comparing multiple formats and content options prior to implementation (e.g. Casteel 2018, Potter et al. 2018). The disadvantage of such fexibility is that hypothetical situations may not adequately capture the context and responses that only fully materialise during actual threat events (Weyrich et al. 2020a). More generally there is a question as to how well stated intent and recalled

responses correspond to actual behaviour (Weyrich et al. 2020b). Both limitations can be partially alleviated through the application of experience-based sampling techniques (Hektner et al. 2007) that attempt to measure warning-related variables in near real time. Finally, it is also possible to understand warning effcacy through analyses of behavioural outcomes (e.g. injuries, damage) using cohort or casecontrol observational study designs (e.g. fall-related and motor vehicle collision injuries, Mills et al. 2020).

# **4.3 Capabilities of the Impact Forecast**

The impact forecast enhances the underlying hazard forecast, incorporating information on vulnerability and exposure to estimate the impact of the hazard. Impact experts provide critical information to "core partners responsible for public safety". Impact-based decision support services help to better understand and utilise forecasts and warnings when dealing with extreme events (Lazo et al. 2020).

Currently impact information is often generated by the warner based on their experience of previous events and is thus limited by the experience of each warner or their understanding of their relevance to the current situation. Sometimes these analogues may be documented and semi-quantitative (e.g. US water supply forecasts put the current forecast on a scatterplot relative to past years).

"A growing number of experts are suggesting that standard warning information should be augmented with additional information about these factors" (Weyrich et al. 2018). Their expertise is often applied offine to develop tools that either enable the forecaster to convert a hazard forecast into an impact forecast or enable the decision-maker or warner to convert the decision threshold into a hazard threshold. For instance, people are increasingly making real-time forecasts of hurricane damages, particularly in the USA (e.g. this hurricane is expected to cause \$750 million in damages if it follows the expected track).

Since they are often not involved in the real-time issue of warnings, the relationship of an impact expert may be more academic and detached than that of some of the other actors in the chain. On the other hand, their studies likely include analysis of events that caused major social and economic loss. By developing warnings within specifc hazard early warning systems, the warnings for a single event may link to consequences of actions and decisions and will be better able to deal with potential impacts of cascading events where multiple responses from different sectors will be necessary.

# *4.3.1 Sources of Impact Information*

A fundamental limitation to our ability to estimate impacts comes from the lack of routine collection of data on weather-related socio-economic impacts. Chapter 5 will cover this in more detail, but most available data are highly aggregated – national scale census, production, health, infrastructure performance, etc. More local data are typically not available except to specifcally accredited researchers. For health data, this is because of patient confdentiality, while for infrastructure and business performance, it is to preserve commercial confdentiality. The result is that models generated using these data sources cannot be replicated or inter-compared, while those from open sources are mostly too coarse to be useful.

Attempts have been made to overcome this barrier using media reports to catalogue impacts. This approach is used in the International Disasters Database (EM-DAT) (https://www.emdat.be) coordinated by the Centre for Research on the Epidemiology of Disasters (CRED) within the Université Catholique de Louvain in Brussels and the United Nations to categorise and identify disasters globally. While originally dependent on manual interpretation, recent research has demonstrated the use of automated methods for classifying reports. According to a recent survey, 19% of NMHSs in Europe are collecting media reports to an in-house impact database, 9% are storing impact observations from storm spotter organisations and 13% are collecting other types of human impact observations (Kaltenberger et al. 2020). Among other sources, media reports of impacts of severe weather are systematically monitored, quality checked and fed into the European Severe Weather Database (ESWD, e.g. Dotzek et al. 2009). Some NMHSs and DRM organisations in least developed countries are also using media reports to gather impact information for use in establishing impact-based warning services.

Another approach that is likely to grow in the future uses automatic data collection from the Internet of Things. For instance, autonomous vehicles carry sensors for the weather, but also record information about speed, traffc density, etc. Taken together such data could provide a major step forward, both for training impact models and for evaluating forecasts and warnings of highway conditions.

# *4.3.2 Capabilities of Different Impact Estimation Methods*

We can identify some key aspects of impact estimation tools that affect performance. The strongest evidence comes from repeatable laboratory testing and is often used as the basis of impact estimation for engineered structures. Certainly, it is important to know the failure modes and thresholds of the materials of concern. However, reproducing conditions in the real world is very demanding, e.g. ageing of materials, combinations of wind and rain, the complex motions of the sea against a barrier, etc. In designing a structure, the remaining unknowns are often dealt with by adding a safety factor. An appropriate way of dealing with this needs to be included in any failure prediction tool. Examples of this approach are wind impacts on concrete bridges, food impacts on retaining walls and wind impacts on moving vehicles.

Where there is a clearly identifable set of processes leading to failure, it may be possible to model these and to calibrate the model parameters using experimental evidence. For instance, the ways in which food water damage a building are well established for particular construction methods, so a relation between food depth

and cost of recovery can be developed (Penning-Rowsell et al. 2013). In a similar way, the response of networks can be modelled. So, for instance, if a road is closed by a fallen tree or accident, or if a telecommunications link is broken, the resulting impact on road or communication traffc can be modelled as a function of the expected loadings for the time, day and season. This approach can also be used for the spread of vector-borne diseases if the behaviour of the vector (which is typically weather-dependent) can be modelled. These approaches have several limitations. Models are unlikely to be complete, so missing processes may, on occasion, be signifcant. The models and their parameters are typically validated for a limited range of conditions – which may not include extremes. To minimise the risk of misleading information, models should be run with hazard inputs sampled from across the uncertainty range – preferably from an ensemble prediction system – and using a range of parameter settings consistent with the validation data. The resulting probability distribution should then be interpreted for use in the warning, e.g. by extracting the most likely, the probable worst case or the probability of exceeding a particular damage threshold.

In most cases, however, the processes are hidden or too complex for modelling. In that case, prediction tools must rely on the statistical analysis of historical data to extract relevant relationships. This approach is most developed in the feld of epidemiology (Armitage, Berry & Matthews 2002), but similar tools apply in many other impact areas including in the atmospheric sciences (Wilks 2006). Traditional approaches have been based on ftting an appropriate statistical distribution to data by selecting the parameters of the distribution that optimise the ft. Increasingly, these approaches are being replaced by machine learning techniques such as neural networks. In order to extract a useful relationship, data should be pre-processed to remove the infuence of extraneous factors, such as time of day, day of the week, holiday periods, policy changes, etc., and to remove trends. It is also essential that all factors that may be expected to infuence the impact data are represented. For instance, if a stormy period is being compared with a non-stormy period to study the relationship between weather and accidents, the different mix of people travelling – perhaps less elderly or less women – could bias the results unless allowed for in the analysis. Like the process models, the resulting statistical models should not be used in parameter ranges that are rare or missing in the training data. Standard statistical techniques can be used to assess the uncertainty in the association, and this information should always be incorporated in any predictive model so as to avoid overconfdence.

The normal statistical approach is to look for a repeatable association between the hazard and its impact. We might call this the forward model. However, where there is a unique decision to be made at a specifc threshold, it may be more appropriate to predict the probability that this threshold will be exceeded. This involves less complex statistical analysis and provides the probabilistic information directly. However, the infuence of confounding factors, trends, etc. can still produce misleading results.

All statistical models must be evaluated using a dataset that is uncorrelated with that used for training the model. It must also be large enough to provide statistical signifcance in the parameter ranges of importance for hazard impacts.

# *4.3.3 Sector-Specifc Impact Tools*

Risks vary across sectors and policy areas, such as health, environment, water/ power supply, transport, technology, security, insurance and fnance, so this is the frst aspect that must be considered before developing warnings (Eiser et al. 2012). In some sectors, advances in data gathering, modelling and computing have increased the ability to provide critical data in their decision-making timeframes (Ruti et al. 2020, Yu et al. 2018). The health sector will be concerned with impacts of death and injury, need and capacity for hospital admissions and services and use impact assessment tools such as epidemiology, transmission, and exposure. The energy sector may be concerned with circuit failures and loss of service to critical users and will rely on detailed engineering modelling. The water sector will have numerous types of threats from lack of supply for drinking and for critical infrastructure support, threats from drainage overfow and contamination and additional health threats; therefore, the water sector must be engaged in detailed modelling of infrastructure. Emergency management is concerned with threats to all critical infrastructure, lifelines and services, such that problems with transport, power, water, energy, agriculture, environment and fnancial protection must be factored into the types of threats, but also impacts that may result in cascading events and multiple types of emergencies.

# **4.4 Structures that Facilitate Warning Information**

The concept of a bridge between warner and impact expert, across which information fows back-and-forth, refects a much more complex reality of multiple connections between different types of warners and numerous experts using multiple communication tools for interaction. It is important to have "an integrated warning system that is built on social science research and ensures full communication between all actors throughout the entire emergency management process" (Lazo et al. 2020). In recent research, the development of mitigation actions emerges from inputs of forecasts and warnings through impact-based decision support services, which feed into reducing asset damage, service interruptions and human health.

# *4.4.1 Relationship Between Information Provider and Warner*

The relationship between information provider and warner is critical, and it is important to understand the research structures and methods of working that facilitate the applicability and application of research. Frameworks that link members through early warning systems, disaster management systems and earth system science have established structures and relationships that move from the development of science, forecasts and models to effective communication with emergency managers and decision-makers, including the general public (Beaven et al. 2016). Information needs to be shared across group boundaries, specifcally by knowledge brokers (Ali et al. 2019). Boundary organisations and individuals that link research with communication and knowledge application are key facilitators of these relationships (Pielke, Jr. 2007).

Coordinated structures, such as the Natural Hazards Research Platform in New Zealand (Beaven et al. 2016) and the Natural Hazards Partnership in the UK (Hemingway & Gunawan 2018), provide mechanisms to improve tools and models and to evaluate warning systems and improve capabilities. Such structures enable discussion of caveats and uncertainties that may prevent the warner from using the information incautiously or out of context.

Within relationships, tensions between the policy and science domain create a hybrid zone in the "bridge". Science becomes "applied science" as information turns into action. "Development of…impact-focused information and advice is supported by coordinated access to cutting-edge science and natural hazard impact research" (Hemingway & Gunawan 2018). Policy relevance requires interdisciplinarity and will likely be time-sensitive, driving a move to shorter-term actions. The general public and non-experts require simplifed information, but this should not compromise the understanding of uncertain, complex information (Beaven et al. 2016).

# *4.4.2 Communicating Impacts and their Uncertainty*

The objective of the interaction between the warner and the impact specialist is to provide the warner with the means to incorporate relevant impact information in the warning. Typically, this is achieved by providing a model or tool. There are several dangers that must be recognised by those involved if the exercise is to be successful in making the warning more effective. Great care must be taken to identify the impacts that matter to the decision-maker and to avoid simply predicting the impacts for which there are good data or simple models. It is also important to avoid generating complex sets of output that overwhelm the warner with data. Since the warner is aiming to produce a narrative that will help the receiver to act, it may be helpful to consider producing storylines (Shepherd, 2019) of hazard impacts that describe one or more impacts together with their uncertainty. Such an approach could be especially suitable when the impact specialist provides real-time interpretation as part of the warning chain.

One of the challenges faced by intermediaries between warning and impact information is the discussion of uncertainty. An important aspect of building trusted relationships is that there should be discussions of how accurate the forecasts are at different time and space scales. When these conversations are combined with hazard impact and sector impact models, knowledge of the certainty of each of these models and the ways in which the impacts interact will be critical. "The noncommunication of these is problematic as interdependencies between them, especially for multi-model approaches and cascading hazards, can result in much larger deep uncertainties" (Doyle et al. 2018). It is important for uncertainty to be communicated effectively to best inform decision-makers and to ensure action is taken that best protects the community.

The full range of uncertainties throughout the warning process must be allowed for (from defning the problem, computational issues, initial conditions, verifcation and beyond). Scientists must set realistic expectations concerning uncertainties, recognise cultural differences between disciplines, and ensure that engagement develops mutual understandings of the issue and supports decision-makers. "When visualizing uncertainty, the focus must be on the data and uncertainty relevant to the decision" (Doyle et al. 2018).

Communication of uncertainty increases levels of trust (Joslyn & LeClerc 2015). The message should be precise about the sources of uncertainty involved and how to effectively present disagreements between experts in a way that does not minimise the message or credibility. It is also important that the impacts are wellunderstood by communicators and that they include in their warnings "decision-relevant time frames, including information on *when the uncertainty may be reduced*" (Doyle et al. 2018). Developing partnerships and communicating uncertainty prior to the need to issue the warning increase trust and confdence.

# *4.4.3 Exchanging Information About Tools*

A general principle across the whole warning chain is that users have greater confdence in warnings if they understand how they were produced. The greatest challenge to achieving that lies in impact prediction, which is often hidden in statistical "black boxes". It is therefore an essential part of any partnership between impact modeller and warner to convey the basis of the model, the predictors used and the uncertainty bounds in the predictions. The warner should have access to routine verifcation and be able to query unexpected results. These requirements place demands on the information produced during tool development and handover and on the availability of ongoing expert support. They also require that users, impact scientists, IT developers and warners are all involved throughout the development process. When a new or upgraded tool is handed over, users should be inducted into its use through presentations and supported hands-on practice. Detailed instructions should be provided, describing its operation, including operating limits and how to deal with any failure modes. It should also be accompanied by a comprehensive test report, together with datasets and any ancillary software required to reproduce the test results. The test report should clearly state the ranges of input data that have been validated and any caveats about tool outputs, including situations where performance will be below the norm, and should identify the sources and magnitudes of uncertainty in the results. In order to guard against overconfdent messaging, uncertainty ranges should be provided as standard outputs from the tool. Metrics used in the evaluation should be clearly described, together with the reasons for using them and their limitations.

These technical aspects of handover are important to ensure that the warner does not inadvertently produce misleading information. However, they also contribute to helping the warner to have enough confdence to use and accept the information that is generated. To fully achieve this acceptance, warners should be involved throughout the development process, to ensure that the tool is designed to produce the information that they feel is required, that the developers test the tool in circumstances identifed as important by the warners and that performance can be challenged by those who will use it. Ideally the relationship between developer and warner should be personal, but if not, regular contact throughout the development process will help ensure that the tool contributes to better warnings once it is handed over.

# *4.4.4 Challenges of Evaluating Tools*

In order to ensure that warning information is used and useful, it is important to conduct evaluations. The data and models need to provide actionable results, and the results of the models should be validated and verifed. Evaluations of each aspect of the system can be complicated, as the warning may be based on integrated, ensemble models with impact scenarios and simulations that are then communicated using various infrastructure and tools throughout the early warning system. Studies have been conducted to determine decisions that are made from warnings, using the determinant that protective action occurred as a measure of success (Gutter et al. 2018). Each of these stages will need evaluation, but fnally the decisionmaking processes must be considered and whether or not action was taken.

# **4.5 Examples**

#### **Box 4.1 A Structure for Warner and Impact Expert Interaction in New Zealand** Cheryl L. Anderson