**Open Access** This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

# Forging a Dialogue Between the Energy Industry and the Meteorological Community

*Alberto Troccoli, Marta Bruno Soares, Laurent Dubus, Sue Haupt, Mohammed Sadeck Boulahya, and Stephen Dorling*

A. Troccoli (\*) World Energy & Meteorology Council, c/o University of East Anglia, Norwich, UK

M.B. Soares University of Leeds, Leeds, UK

L. Dubus Electricite de France (EDF), Paris, France

S. Haupt National Center for Atmospheric Research (NCAR) and WEMC, Boulder, CO, USA

M.S. Boulahya WEMC, Toulouse, France

S. Dorling University of East Anglia (UEA) and WEMC, Norwich, UK

© The Author(s) 2018 65 A. Troccoli (ed.), *Weather & Climate Services for the Energy Industry*, https://doi.org/10.1007/978-3-319-68418-5\_5

**Abstract** The interplay between energy and meteorology (based on its broad meaning of weather, water and climate) has been steadily growing. For this relationship to continue flourishing, a formal structure for stakeholders to interact effectively is required. The process of formation of the World Energy & Meteorology Council (WEMC), an organisation aimed at promoting and strengthening such a relationship, is discussed in this chapter. Such a process involves building many diverse relationships, something which has been happening over several years, alongside the adoption of more formal practices such as stakeholder surveys. While the focus of this chapter is clearly on WEMC, this process could be used as a stimulus for analogous activities in the broader energy and meteorology area, specifically those at the national and regional levels, as well as similar activities straddling diverse disciplines, such as those promoted by the Global Framework for Climate Services (GFCS).

**Keywords** Meteorology • Climate services • Energy • Partnerships • Survey • Capacity building • Communication • Outreach • Stakeholder engagement • Education • Associations

#### Introduction to the World Energy & Meteorology Council

The World Energy & Meteorology Council (WEMC) is a non-profit organisation devoted to promoting and enhancing the interaction between the energy industry and the weather, climate and broader environmental sciences community. Its primary goal is to support improved sustainability, resilience and efficiency of energy systems under ever-changing weather and climate.

Formally established in 2015 as a Company Limited by Guarantee in the UK, WEMC has taken shape over several years. The initial seeds were sown with the 2008 NATO Advanced Workshop, *Weather/Climate Risk Management for the Energy Sector*, which was attended by nearly 30 very active participants who subsequently produced a report published in an international journal (Troccoli et al. 2010) as well as a book that has attracted the attention of thousands of practitioners (Troccoli 2010) and paved the way for more organised interactions between many stakeholders including but not limited to hydrometeorological science and energy sector communities. This start-up workshop was then followed by the more formal and substantial International Conference on Energy & Meteorology (ICEM) series. Four ICEMs have been successfully held thus far—2011 in Australia, 2013 in France, 2015 in USA and 2017 in Italy—with a fifth one to be held in May 2018 in China, where increasing emphasis will be placed on developing world activities and requirements (plans for subsequent conferences are underway). During this period, the WEMC concept has developed substantially thanks to countless discussions amongst experts, the creation of new connections, the burgeoning of the literature in the area of weather, water and climate services as well as work in related international activities such as the International Energy Agency (IEA) Tasks 36 for wind energy, 46 for solar energy and so on. Moreover, the UN-led Global Framework for Climate Services (GFCS) has officially elected energy as a new (fifth) priority area in 2017 and developed a roadmap for the implementation of climate services for the energy industry (WMO 2017, also Ebinger and Vergara 2011).

All of these activities have pointed to the growing interest in strengthening the relationship between energy and meteorology to ultimately help achieve the goals of sustainability, resilience and efficiency of energy systems. In a practical way, WEMC has been acting as the implementing agent for the ideas and recommendations emerging from the ICEMs and beyond (e.g. the GFCS-Energy Exemplar, the World Economic Forum, the United Nations Framework Convention on Climate Change [UNFCCC] Conference of the Parties [COP] 21–Paris Agreement in December 2015). With an average of 200 attendees at each ICEM, an edited book (Troccoli et al. 2014), two special issues in international journals—one in Solar Energy following ICEM 2011 (Troccoli 2013) and one in MetZet following ICEM 2015 (Troccoli and Schroedter-Homscheidt 2017)—and abundant discussions during ICEMs and at related events (e.g. Troccoli et al. 2013), there is a wide-ranging set of issues that naturally feeds into the WEMC concept and plan of action. The plan of action includes (1) the organisation of a series of institutional workshop presentations, webinars and capacity building activities (including internships); (2) the assessments about the significance of future climate projections on energy resource and their implications for energy system investments; (3) the documentation on meteorological and energy data/metadata quality to assist the energy sector to easily access and make optimal use of these data; (4) the formulation and implementation of projects and programmes. These are just some examples amongst the many topics germane to the energy and meteorology intersection. A more comprehensive list of suggested planned activities is presented below, resulting from the WEMC survey.

#### *Rationale for Creating the Organisation*

The major ongoing transformation of energy systems worldwide is highlighting the intimate interplay between energy on the one side and weather, climate and water on the other (Green et al. 2016). Although this connection is self-evident in the case of renewable energy (RE), weather and climate information is also critical to a much wider range of energy industry activities, from managing of energy supply from broader energy sources (e.g. offshore oil operations), to the understanding and estimation of energy demand, to the assessment of meteorological impacts on energy extraction, transportation, transmission and distribution (see also Chap. 2).

Given this context, WEMC seeks to substantially contribute to increasing the productivity, resilience and efficiency of energy systems under the influence of ever changing weather and climate as well as to achieving more affordable and available energy, and thereby, foster sustainable and resilient energy systems.

#### *Aims of the Organisation*

WEMC aims to enhance productivity and policy formulation for the energy industry through a close collaboration between the energy sector and the weather, water and climate community and to achieve improved adoption of weather, water, climate and other environmental information by the energy industry towards more efficient, proactive and sustainable risk management practices. WEMC also aims to assist the energy industry in meeting the demand for energy while being mindful that there is a need for reducing harmful impacts on the natural environment, in line with international protocols such as the COP21 Paris Agreement.

These aims are tackled through the identification of top-class expertise in energy and meteorological sciences as well as through mobilisation of resources, with fund raising and facilitation of focussed programmes executed by an appropriate mix of energy industry, private service providers, government and international institutions and scientific organisations. Such activities are typically in line with, and in support of, existing relevant national and international efforts.

Ultimately, WEMC aims to create knowledge, critical thinking, experimental tools and funding platforms at the global level for a highly effective use of meteorological information within the energy sector, particularly through creative public–private–academic partnerships. For instance, WEMC supports the energy industry in more effectively utilising meteorological products and facilitating the integration of more appropriate services in a changing climate. Specifically, one key area where the energy sector can benefit from the interaction with the weather and climate community, and that WEMC is contributing to, is the use of meteorological forecasts for grid integration and related tasks such as dynamic line rating. A number of transmission system operators (TSOs) and market operators are already using weather and/or production forecasts for the integration of renewable electricity into the grid. However, this is particularly the case in countries where either RE penetration is important (typically larger than 10%) or use of RE forecasts is mandated (or both). In countries where RE production is still marginal, use of forecasts is not considered a priority. There are also indications that countries/organisations that have taken early action in RE forecasting, rather than to react to a sudden increase in RE production, integrate RE into the grid in a more managed and effective way. It is critical therefore to inform TSOs about the latest developments in meteorological forecasting and the benefits of using this information. Thus, by producing easily accessible, jargon-free and succinct publications, WEMC aims to assist, as in this specific case, with the adoption of meteorological forecasting tools to improve the integration of RE into the grid, and at the same time help control the cost of electricity for consumers.

#### *Structure of WEMC*

WEMC is structured around four programmes:

These programmes, which represent a mix of activities ranging from communication of technical information (programme 1) to a continuous engagement with an ever-widening base of stakeholders as well as the strengthening of advisory bodies at the strategic (Advisory Board) and the technical (Technical Advisory Group) levels (programme 2); to the assistance in the uptake of meteorological tools for more efficient, resilient and sustainable energy systems (programme 3); to capacity building activities such as the ICEMs (programme 4), have clearly been constructed to target the goals of WEMC. Top level international experts are leading the efforts in each of these programmes, with the assistance of the WEMC secretariat, which although currently small and with limited resources, is in a growing phase. The WEMC secretariat is sited on the campus of the University of East Anglia (Norwich, UK).

Sustained input from international experts, particularly the ICEM Organising Committee and the WEMC Technical Advisory Group, is key to the success of the WEMC initiative, in a similar manner to the way that related international activities function (e.g. the above-mentioned IEA tasks). Indeed, assistance from a wide base of experts is essential to tackle prioritised activities aimed at improving the interaction between energy and meteorology (e.g. data exchange and their standardisation).

WEMC heavily relies on the expert guidance of its Advisory Board, which is drawn from as diverse a group of people as possible to cover the main international players in the energy sector and the meteorology community, both in the developed and developing countries and including related associations (e.g. the USA-based Utility Variable Integration Group, UVIG).

Also integral to the structure of WEMC are Special Interest Groups (SIGs). The growing WEMC membership is leading the work of the SIGs. The plan is to constitute several such SIGs, one for each identified main critical activity, with around a dozen experts participating in each. Initially WEMC will focus on three initial SIGs, one on Data Sharing and Standards, the second on Grid Integration and the third on Education. This choice has been informed by members and the interests of other experts' interest, as identified through the survey presented in the next section. WEMC will provide communication tools and assistance to facilitate the work of these groups. It will also horizon scan for funding opportunities and pursue the most relevant and promising ones so as to provide financial support to the activities of specific working groups (e.g. to hold physical meetings). These SIGs are tasked to produce output readily usable by the energy industry such as the aforementioned guidance document on grid integration (Fig. 5.1).

**Fig. 5.1** The World Energy & Meteorology Council (WEMC) organigram

Defining Priorities for WEMC: The Users' Survey

#### *Rationale for Undertaking a Survey*

Given the aim and nature of the WEMC activities, it was fundamental to define the next steps for the organisation based on the needs and requirements of the potential future users of those services. A survey was conducted with the target of engaging with as many potential users and organisations around the world as possible.

#### *Methodology and Implementation of the WEMC Survey*

The method selected to engage with potential users of WEMC was an online survey as it enabled the collation of the widest possible number of responses from around the world as possible in a relatively short amount of time (May 2011). The survey, developed using the software Survey Monkey, included four main sections1:

– *Welcoming page*—This introductory page described WEMC and asked respondents to select their sector of activity: energy; weather and climate; or other.

Given the importance of involving people from the energy and weather community worldwide, the survey was disseminated to participants at the 3rd ICEM in Boulder USA in 20152 as well as circulated to targeted mailing lists such as the Energy-L3 and Climate-L,4 mailing lists for energy policy issues and climate-related news, respectively. The survey was officially launched in June 2015 and closed in January 2016. The sections below describe some of the main findings from the survey and how it helped inform future activities within WEMC.

#### Results from the WEMC Survey

A total of 147 responses were received between June 2015 and January 2016. Almost half of the respondents worked in the energy sector (47%, *n* = 69), followed by those working on weather and climate related activities (33%, *n* = 39) and other sectors (20%, *n* = 29). The type of organisations also varied with private companies representing the large majority in the energy sector whilst research institutes were the most represented in the weather and climate sectors (Fig. 5.2).

The participants' organisations also varied in size and between the sectors of analysis. The energy sector showed the highest number of large companies (with more than 5000 employees) followed by smaller organisations (with up to 100 employees). Conversely, the weather and climate sector showed the highest number of smaller companies (up to 100 employees) followed by those with between 1000 and 5000 employees (Fig. 5.3). Organisations in the 'other' sector had a fairly similar distribution

**Fig. 5.2** Type of organisations per sector of activity

**Fig. 5.3** Size of the responding organisations per sector

with regard to the size of the companies who responded to the survey (Fig. 5.3).

The respondents from the energy sector were mainly based in Europe (France, Germany, Denmark, Spain) as well as the USA; whilst those working in the weather and climate sector were mostly based in the USA followed by France and the UK (Fig. 5.4).

With regard to the scope of the organisations' activities, these were predominantly worldwide for both the energy and the weather and climate

**Fig. 5.4** Countries where survey respondents are based. Countries selected by only one respondent were excluded from this chart. These included Brazil, Austria, Vietnam, Costa Rica, Namibia, South Africa, New Zealand, Mexico, Zambia, Greece, Indonesia, Argentina, Malaysia, Bosnia and Herzegovina, India, Finland and Guatemala, Ghana, Morocco, Chad, United Arab Emirates and Mauritania

sector followed by those operating across Europe and in specific countries worldwide for the energy sector and those operating in specific European countries for the weather and climate sector (Fig. 5.5). Those in the other sectors operated across the range of geographical areas as identified in Fig. 5.5.

#### *Activities Across Sectors*

The main activities pursued in the energy sector were distribution/transmission, technology development, and power development (Fig. 5.6), although approximately half of the organisations in the energy sector (*n* = 25) worked in two or more activities.

Similarly, more than half of the organisations (*n* = 27) operated in two or more areas within the renewables (i.e. solar, wind and hydroelectric power) which was the sub-sector most strongly represented amongst the surveyed organisations (Fig. 5.7). Given renewables are, amongst the energy systems, the most impacted by weather and climate events, this result was not unexpected; however, this also reflects the backgrounds and interests of the respondents.

Another interesting aspect was the fact that 75% of these organisations (*n* = 35) were involved in the energy and meteorology nexus (i.e. working in areas linking energy and meteorology). The organisations working with

**Fig. 5.6** Scope of responding organisations' activities in the energy sector (total per cent of *n* = 47; note that this was a multi-answer question)

**Fig. 5.7** Area of the energy sector in which the organisations operate (note that this was a multi-answer question)

weather and climate were fairly homogenous with regard to the provision of different weather and climate information (Fig. 5.8).

The remainder of the organisations (*n* = 29) operating in other sectors was mainly constituted by those working in academic research, government and public administration, forestry, media, biodiversity and ecology and coastal activities.

**Fig. 5.8** Scope of activities in the responding organisations operating in meteorology and climate (note that this was a multi-answer question)

**Fig. 5.9** Number of organisations surveyed interested in the energy and meteorology nexus

#### *Nexus Between Energy and Meteorology*

Survey participants were also asked about the interest of their organisation in the energy and meteorology nexus. The large majority of respondents confirmed the interest in this nexus with approximately 75%, 62% and 87% of the respondents in the energy, weather and climate and other sectors agreeing, respectively (Fig. 5.9).

#### *Future WEMC Projects and Initiatives*

Survey participants were provided with a list of options of potential policy and services initiatives that could be pursued by WEMC (Fig. 5.10). Of those, the three main aspects that respondents (across all sectors of analysis) preferred to have available from WEMC were the 'Development of codes, standards and guidelines for meteorological information'; 'Position papers' and 'Reports on resilience and sustainable energy systems and links to emission reduction requirements'. Conversely, the least preferred option was 'Recommendations on data/metadata quality to assist the energy sector' (Fig. 5.10).

It should be noted, however, that the range of scores between the most and least popular suggested policy/services initiatives is relatively small, and their differences are likely within the sample error. In addition, given this is an evolving area we expect these responses to vary over time. Therefore, it is not straightforward to clearly pinpoint which activities are deemed as critical to the extent that they should be prioritised.

Participants were also asked about their preferences regarding research and technology transfer initiatives that should be pursued by WEMC. The 'Development of methodologies for analysing the linkages between

**Fig. 5.10** Preferences from survey respondents regarding policy/services initiatives to be pursued by WEMC (based on rating average of ranked preferences)

**Fig. 5.11** Preferences from survey respondents regarding research and technology transfer initiatives to be pursued by WEMC (based on rating average of ranked preferences)

energy, water and food' ranked the highest of all the options given (Fig. 5.11). This was followed by 'Information and tools for assessing energy risks and vulnerabilities' and 'Climate information and tools to support decision-making based on energy source and need'. The least preferred option was 'Assessing the impact of high penetration of variable energy generation and their optimal integration into the grid' (Fig. 5.11).

Finally, participants were asked about their preferences regarding outreach and training activities. The main priority for respondents is the creation of a 'WEMC mailing list and newsletter' (Fig. 5.12). Following from that, respondents were also interested in 'Targeted schools on particular topics within the energy and meteorology nexus' and 'A series of online webinars on the energy and meteorology nexus'. The least preferred option was 'Creation of a database of organisations, projects, events, and best practices to support potential collaborations' (Fig. 5.12).

#### *Paying for WEMC Services*

Participants were also asked about their willingness to pay for WEMC services. Of those who responded to this question (*n* = 88), 47% agreed that they would be willing to pay for those services whilst 53% disagreed. Those who indicated they would be willing to pay were then asked about

**Fig. 5.12** Preferences from survey respondents regarding outreach and training activities to be pursued by WEMC (based on rating average of ranked preferences)

**Fig. 5.13** Respondents' willingness to pay for WEMC services, per year

how much they would pay per year for the WEMC services. Around 72% of the respondents would be willing to pay less than 500 US dollars per year, whilst only 16% would pay between 500 and 1000 US dollars, 9% between 1000 and 5000 US dollars and only 3% said they would be willing to pay between 5000 and 15,000 US dollars (Fig. 5.13).

Of the total number of respondents, 46% agreed to continue being involved and updated regarding future WEMC activities (including those both willing and not willing to pay for the services). These mainly included respondents from Europe (56%) followed by those from the USA (21%), Africa (14%), South America and the Caribbean (6%) and Asia (3%).

#### Next Steps for WEMC

The results of the survey have been very useful in shaping and guiding WEMC activities thus far, and for the immediate future. Specifically, the results of the survey have been useful to highlight:

It is also important to note that the results of the survey were written up nearly a year after it was closed. Having had this additional period to see the evolution of the sector, particularly in terms of stakeholders' requirements, it is apparent that some of the priorities have somewhat shifted since then. For instance, the optimal integration of RE into the grid was given as a low priority by the respondents (Fig. 5.11), while there is evidence (as highlighted earlier in this chapter and also through the work of, e.g., the UVIG5) that this is now higher in experts' agendas. This apparent shift may also be indicative of the fact that our survey sample, although reasonably large, was not robust enough to clearly discriminate amongst priorities areas. It may also simply be a reflection of the fact that preferences regarding policy/services initiatives (Fig. 5.10) cannot capture the nuances in the response choices available. In all, stakeholder consultations, through surveys similar to the one presented here or via other processes (e.g. workshops), will need to be an integral component of WEMC activities so that new ideas, needs and other information from stakeholders are taken into consideration in a timely fashion.

Partnerships with analogous organisations (e.g. the International Solar Energy Society, ISES) are also key to the success of WEMC, and these are being actively pursued, for instance, via the co-organisation of webinars, a communication tool that is proving very popular or the involvement of key people (e.g. ISES president) in the WEMC Advisory Board.

Another important aspect of the WEMC activities is to provide a blueprint, and ideally an international reference, for national and regional activities in the area of energy and meteorology. A few such activities have already been initiated, with the USA (e.g. American Meteorological Society's annual Conference on Weather, Climate, Water and the New Energy Economy, which started in 2009) and the EU (the Energy Meteorology session at the European Meteorological Society annual conference, which also started in 2009) leading the way. More recently, a meeting on energy and meteorology was held in China in 2016 for the first time, for which ICEM was taken as a reference for the organisation of the event (Dr Rong Zhu, China Meteorological Administration, personal communication).

Overall, what is clear is that a continuous, adaptable and proactive interaction will be required in order to make WEMC's activities valuable to a wide range of stakeholders in this relatively fast evolving interdisciplinary area. Further discussion about next steps in the area of energy and meteorology is presented in the final chapter of this book.

#### Notes

#### References

water and the environment. *Sustainability Science*. https://doi.org/10.1007/ s11625-016-0402-4.

May, T. (2011). *Social research*. Maidenhead, UK: McGraw-Hill Education.

**Open Access** This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

# Weather, Climate and the Nature of Predictability

#### *David J. Brayshaw*

**Abstract** The prediction and simulation of future weather and climate is a key ingredient in good weather risk management. This chapter briefly reviews the nature and underlying sources of predictability on timescales from hours-ahead to centuries-ahead. The traditional distinction between 'weather' and 'climate' predictions is described, and the role of recent scientific developments in driving a convergence of these two classic problems is highlighted. The chapter concludes by outlining and comparing the two main strategies used for creating weather and climate predictions, and discussing the challenges of using predictions in quantitative applications.

**Keywords** Weather prediction • Climate prediction • Predictability • Chaos • Modelling

#### Introduction

A long-standing challenge for meteorology and climate science has been to develop techniques capable of producing predictions and simulations of the weather and climate across a range of timescales. Although these

National Centre for Atmospheric Science, Reading, UK

D.J. Brayshaw (\*)

Department of Meteorology, University of Reading, Reading, UK

**Fig. 6.1** Weather and climate timescales, forecasting tools and datasets

timescales should be viewed as a continuum, it is nevertheless helpful to identify a number of discrete timescales, ranging from very near term 'nowcasting' to climate change projections over centuries and millennia, as shown in Fig. 6.1.

This chapter briefly reviews how the nature of predictability differs across prediction timescales, identifies the major strategies used to create predictions and discusses some of the general challenges with using predictions in quantitative applications.

#### The Nature of Predictability

To understand weather and climate forecasting, it is important to understand the character of the physical system one is seeking to predict. The atmosphere (and the climate system more broadly) can be viewed as an example of a chaotic system (e.g., Lorenz 1963), associated with two distinct 'types' of predictability (Lorenz 1975, see also Schneider and Griffies 1999). These types—and their relationships to different timescales of weather and climate—are discussed herein.

Due to the complexity of the atmosphere (or climate) system, it is helpful to discuss predictability with reference to an analogous but simpler chaotic system. The Lorenz model contains three inter-dependent variables (U, X and Y) evolving deterministically over time. Each of the three dimensions can be understood as representing a meteorological quantity in analogy (e.g., eastward wind, northward wind and temperature). A typical atmosphere-only climate model will, however, have in excess of ~106 dimensions: one each for six key meteorological properties (wind in the horizontal and vertical, temperature, water vapour and surface pressure) at each point on a 3-dimensional grid (perhaps 192 × 120 × 30). In some cases, the time evolution of the equations may also include a stochastic (random) component rather than being purely deterministic. The equations of the Lorenz model may be written in finite difference form:

$$X\_{\boldsymbol{r}} = X\_{\boldsymbol{r}-1} + \left[\alpha \left(Y\_{\boldsymbol{r}-1} - X\_{\boldsymbol{r}-1}\right)\right] \delta t$$

$$Y\_{\boldsymbol{r}} = Y\_{\boldsymbol{r}-1} + \left[X\_{\boldsymbol{r}-1} \left(\beta - U\_{\boldsymbol{r}-1}\right) - Y\_{\boldsymbol{r}-1} + \epsilon\right] \delta t$$

$$U\_{\boldsymbol{r}} = U\_{\boldsymbol{r}-1} + \left[X\_{\boldsymbol{r}-1}Y\_{\boldsymbol{r}-1} - \gamma U\_{\boldsymbol{r}-1}\right] \delta t$$

where *α*, *β* and *γ* are constant parameters, subscripts denote time-steps, and *δt* is the interval between adjacent time-steps. Following Palmer (1999), *ϵ* is used to denote a small external forcing (for the initial discussion it is assumed that *ϵ* = 0).

The time evolution of the Lorenz system can be represented as a trajectory (or path) in phase space.1 Figure 6.2a shows a short section of a trajectory as an example: from an initial state near (*U*, *X*, *Y*) = (33, 15, 18), the model evolves to a state (24,-12,-18) over a 'time' interval ∑*δt* = 0.7. If the model is allowed to evolve for a longer period to produce a more extended trajectory (referred to as an attractor), a fuller view of the system's properties emerges (Fig. 6.2b). The attractor clearly shows two lobes, with the system preferring to occupy states in one or the other of the lobes.

**Fig. 6.2** The Lorenz model and initial condition problems, using *α* = 10, *β* = 28, *γ* = 8/3 and *ϵ* = 0. See text for discussion

Figure 6.2 can be used to illustrate an example of an *initial condition problem* or 'predictability of the first kind'. Consider at some time *t1* that observations of the system are taken, and found to be *U*1 = 30 and *X*1 = 4. Assume that each of these measurements is subject to an observational error, ∆*U*1 = ∆*X*1 = 1, and *Y*1 is unobservable in practice.2 The systems current position in phase space is, therefore, not known exactly, but can be constrained to a relatively small region of phase space indicated by the black box in Fig. 6.2b. A prediction of the system's state at some future snapshot in time, *t*2 = *t*1 + ∆*t* is sought—that is, we wish to predict the exact values of U2 and X2. This is analogous to weather forecasting: we 'know' the weather today and wish to predict the weather tomorrow.

There are many possible phase space trajectories that are consistent with the available knowledge of the initial conditions at time *t*1. A selection of these are shown in Fig. 6.2c: some lead to the left lobe (black lines), whereas others remain in the right lobe B (grey lines). In consequence, a relatively small error in estimating the starting state (∆*U*1 = ∆*X*1 = 1) grows rapidly to a large error in the prediction (∆*U*<sup>2</sup> ~ ∆*X*2 ~ 30). The rate of error growth is, however, very dependent on the initial state and the forecast time horizon considered; in this example there is very low predictability but, if the initial conditions correspond to some other regions of the attractor, there may be much more predictability (i.e., smaller errors), at least over short time horizons (i.e., small ∆*t*).

The evolution, shape and position of a trajectory are also sensitive to the model's parameters (*α*, *β* and *γ*), typically referred to as boundary conditions. Figure 6.3 shows the original attractor from the previous figure (in black) and a new attractor (in grey)—the only difference is a small change in one of the boundary conditions, *ϵ*. Clearly, one can detect a change in the probability distribution of the observable quantity, U, as indicated by the relative frequency distributions in the bottom panel in Fig. 6.3. This is an example of 'predictability of the second kind', which concerns the ability to predict changes in the attractor in response to changes in *external boundary conditions*. 3 Clearly, if the response is large, then it can be more readily detected against the 'internal' variability corresponding to trajectories moving within a single attractor.

Traditional climate change simulations studying the equilibrium climate under a future greenhouse gas concentration scenario can be viewed

**Fig. 6.3** The Lorenz model and the long-term equilibrium climate change problem. The black and grey curves show two simulations with different boundary conditions (parameters as in Fig. 6.2, but with *ϵ* = 10 for the grey curve). See text for discussion

as an example of second kind predictability: it is a boundary condition prediction problem where one seeks to understand how the statistics of climate differ between two different sets of boundary conditions (e.g., Meehl et al. 2007).

If numerical weather prediction (NWP) (days-ahead) and long-term climate change simulations (decades-ahead) can be considered as examples of initial condition and boundary condition problems, then it is clear that much lies between these two extremes. It is therefore helpful to consider the timescales involved in the system one is seeking to predict.

The climate system in general contains many different components, varying on a wide range of timescales (Fig. 6.4). At forecast lead times of 1–2 days, it is typically sufficient to focus on the evolution of the faster components alone (e.g., troposphere and land-surface temperature) as the slower components (e.g., ocean temperature, ice sheets) change little during the lifetime of the prediction. Indeed, at very short lead times (minutes to hours) many aspects of the large-scale flow in the troposphere may even be considered fixed. Conversely, at longer forecast lead times, the evolution of slower components become significant (e.g., ocean circulation, land-surface moisture, ice sheets, sea ice and snow cover). For a prediction of tomorrow's weather in London, it may, therefore, be sufficient

**Fig. 6.4** Indicative timescales of selected components in the climate system

to neglect small changes in the temperature of the North Atlantic ocean's surface, but the same cannot be said for predicting the seasonal-average temperature over Europe several months in advance.

Weather and climate predictions (as outlined in Fig. 6.1)—particularly those in the range of several days to a few decades—are therefore a mixture of initial and boundary condition problems, and have been the subject of much research in recent years. This can be illustrated by considering, for example, a seasonal forecast. In such a forecast, the state of the troposphere and land-surface temperature change much more quickly (~days) than the forecast horizon (~months). From the perspective of these components, the problem is therefore boundary condition prediction and predictability of the second kind (i.e., the intention is to predict the statistical properties of the troposphere rather than estimate its state at a specific snapshot in time). However, for the ocean surface, soil moisture, snow cover and stratosphere—which only change slowly over the timescale of the forecast—the challenge is to determine the specific evolution, therefore concerns predictability of the first kind where initial conditions play a key role.

#### Prediction Strategies

There are two broad categories of predictive models used in weather and climate forecasting: statistical and dynamical. The primary characteristics of each are outlined below.

#### *Statistical Models*

Statistical predictions come in many forms and are in widespread use throughout academia and industry for many problems. Conceptually, these models are simple: historical observations are interrogated to find relationships between a predictand and a set of potential predictors. These historical relationships are then assumed to remain fixed into the future and are used to create a prediction. Common examples include single- or multi-variate autoregressive models (or more sophisticated versions such as ARMA and GARCH), artificial neural networks, support vector machines and 'analogue'-based techniques.

 It must, however, be recognised that there are limitations and dangers associated with this approach. The process of statistical modelling is essentially adhoc: the predictors to be used are not necessarily known a priori (unless informed by some prior physical or dynamical process understanding), and must, therefore, be established afresh for each new predictand. The ability to identify statistical relationships between predictors and predictand is also constrained by the quantity and quality of the available historic data: the records must be sufficiently long and homogeneous to robustly establish statistical relationships between variables. Finally, statistical models trained on historic data may ultimately be a rather poor guide for a climate system subject to changing boundary conditions (e.g., greenhouse gas concentrations). This is perhaps particularly the case for longer range forecasts where many plausible future states will simply not have been recorded in historic observations

#### *Dynamical Models*

In contrast to statistical models, dynamical prediction models numerically simulate the behaviour of the system itself and are the basis of the weather forecasts provided by most operational weather services. In NWP, the atmosphere is represented by fundamental physical equations based on the laws of motion, thermodynamics and conservation of mass. The equations are discretised in space and time (i.e., the atmosphere is divided into grid boxes) and solved iteratively in each grid box, advancing time-step by time-step. Additional physical and dynamical processes such as clouds, precipitation and radiation are represented through 'parameterization schemes' at the grid box level.

As discussed in section 'The Nature of Predictability', initial condition errors in NWP can grow rapidly over a short period (Fig. 6.2c) and 'ensemble forecasting' is widely used. An NWP ensemble consists of a large set of *N* individual 'member' weather forecast simulations, each starting from a slightly different set of initial or boundary conditions (all of which are consistent with the observations, subject to observational error). Typically, the set of initial conditions chosen will seek to maximise the difference between the ensemble members at the targeted forecast time—that is, to capture the widest possible range of uncertainty associated with errors in the initial conditions. Such forecasts, although still 'predictions of the first kind', provide *N* different potential realisations of the future weather state and must be interpreted probabilistically (typically *N* ~ 50 in many present NWP systems) with the 'spread' of outcomes in a good forecast system providing an indication of the predictability available from the particular set of initial conditions used.

Beyond a few days to a couple of weeks, additional climate system components must also be included in addition to the atmosphere (see Fig. 6.4 and section 'The Nature of Predictability'), such as the stratosphere, oceans and sea ice. The resulting dynamical models draw strongly on the heritage of General Circulation Models (GCMs). First developed in the 1960s–1980s—see, for example, Smagorinsky et al. 1965 for a very early example—and continually developed since, GCMs are identical in concept to NWP models—insofar as they represent a physical model of the system one is attempting to predict or understand—but as they include more physical processes and must be run over longer timescales, they typically use much coarser resolution grid boxes than NWP. As in NWP, the use of ensembles in climate model simulations is common. In this case, however, the ensemble is typically used to sample several different sources of uncertainty4: natural climate variability (Deser et al. 2012), initial condition uncertainty (Scaife et al. 2014), parametric uncertainty (Stainforth et al. 2005) or model structural uncertainty (Taylor et al. 2012).

The power of dynamical models to simulate weather and climate is considerable. The skill of NWP models at lead times of several days ahead has increased continuously over recent decades, and GCMs are now used to produce very sophisticated realisations of physical phenomena affecting a wide range of industrial sectors. There are, however, limitations. NWP and GCM models are computationally expensive compared to statistical models, leading to a three-way trade-off between resolution (grid size), physical complexity (number of processes modelled) and computational feasibility (number of model years simulated and ensemble size). All dynamical models are subject to biases and/or growth of prediction error from many sources, such as deficiencies in model formulation (numerical approximations, missing processes), parametric uncertainty (ill-constrained properties in parameterisation schemes) and initial conditions. Before predictions are used, care should be taken to establish whether dynamical models produce a reliable representation of any particular phenomena of interest

#### Summary and Discussion

There are good reasons to believe that predictability exists in weather and climate forecasting across a range of timescales from hours to decades and beyond. This predictability may take one of two forms: either a prediction of the specific evolution of the weather (an initial condition problem) or else a prediction of the statistical properties of the climate (a boundary condition problem). Weather and climate forecasts in the intermediate range (several days to decades) typically incorporate some aspects of both forms of predictability, and a probabilistic approach to the resulting forecast is essential.

Both the statistical and dynamical approaches discussed above have great power in terms of achieving predictive skill. It is, however, emphasised that the two approaches should be seen as being complementary toolkits rather than competing philosophical strategies. Statistical methods are often used to 'calibrate' dynamical model output (reduce bias when compared against point observations) and configure dynamical models forecasts (e.g., by statistically identifying key boundary conditions such as sea-surface temperature patterns). Conversely, dynamical models enable deeper process understanding (helping to identify robust predictors for statistical models) and—with care—can be used to extend datasets by providing plausible artificially generated climate data (statistical robustness and rare events, including effects of a changing climate).

#### Notes

#### References

**Open Access** This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

### Short-Range Forecasting for Energy

#### *Sue Ellen Haupt*

**Abstract** Short-range forecasts for periods on the order of hours to days and up to two weeks ahead are necessary to smoothly run transmission and distribution systems, plan maintenance, protect infrastructure and allocate units. In particular, forecasting the renewable energy resources on a day-to-day basis enables integration of increasing capacities of these variable resources. This chapter describes the basics of this short-range forecasting, beginning with the observation-based "nowcasting" of the first 15 minutes and ranging up to two weeks using numerical weather prediction. We discuss how blending multiple forecasts can increase accuracy and how probabilistic forecasts are being used to quantify the forecast uncertainty.

**Keywords** Wind power forecasts • Solar power forecasts • Renewable energy • Nowcasting • Numerical weather prediction • DICast • Forecast blending • Analog ensemble • Probabilistic forecasts

© The Author(s) 2018 97 A. Troccoli (ed.), *Weather & Climate Services for the Energy Industry*, https://doi.org/10.1007/978-3-319-68418-5\_7

S.E. Haupt (\*) National Center for Atmospheric Research (NCAR), Boulder, CO, USA

#### The Need for Short-Range Forecasts

Utilities and Independent Transmission Operators (ITOs) depend on accurate forecasts for the next hours to several weeks. They often need to plan operation and maintenance outages weeks to months ahead. Weather events may make it difficult to maintain power lines, wind turbines and other infrastructure. If they can plan around events such as lightning storms and high wind events, the safety of the workers and efficiency of the maintenance can be greatly improved.

Weather also impacts day ahead planning of how to commit units. Although weather impacts all types of power production, it actually also drives the renewable units. It is important to be able to forecast the wind, solar and hydro power available the next day, or often, over the next several days too. The marginal cost to run these renewable resources is quite low and it is economically advantageous to allocate as much power from those units as possible. But overallocation of those units when the wind, irradiance or water power is not available could lead to using much more expensive reserve units in real time. Thus it is critical to produce high quality wind and solar power forecasts.

For timescales less than a day, it is important to know the most recent update to the forecast in order to balance the load in close to real time. Sharp up or down ramps in renewable energy must be balanced with the reserve units if the grid operators were not expecting them.

#### Overview of Scales

To forecast across these scales from minutes to a few weeks, meteorologists must combine forecast methods that are most appropriate to each scale. Figure 7.1 illustrates the big picture of the types of systems that may be useful for such forecasts. Figure 7.1 focuses on the short time range where the stakeholders require optimal accuracy. That figure demonstrates that observation-based nowcasting, which refers to the first few hours of the forecast, provides a much more accurate forecast in the short range, but its skill drops off rapidly. Numerical weather prediction (NWP) becomes more important at about three hours and provides value out to about two weeks. Because NWP run at high resolution over a sizable domain requires on the order of hours to run on supercomputers and often requires spin-up time, it is not available for real-time use in the shortest ranges. Modern methods of forecasting renewable energy output employing postprocessing methods to blend disparate models or ensem-

**Fig. 7.1** Blending of NWP models with observation-based nowcasting enables optimization of the short-range forecast

bles are shown to greatly improve the forecast skill (Giebel and Kariniotakis 2007; Monteiro et al. 2009; Mahoney et al. 2012; Ahlstrom et al. 2013; Orwig et al. 2014; Tuohy et al. 2015). Figure 7.1 indicates that this blending can provide value beyond that of the input models.

#### Nowcasting

The shortest time frames of the first several hours benefit from forecasting based on data sensors that determine the current situation of the variable of interest. For instance, having *in situ* measurements of wind speed or solar radiation in the field allow us to train statistical learning or artificial intelligence models to recognize current conditions and likely changes. Over the first 15–45 minutes, it is often difficult to beat a persistence forecast. If we know the wind now, the best forecast in the first few minutes is that there is no change. For solar radiation, we often use a "smart persistence" where we expect the cloud cover to stay the same, but recognize that the solar angle will change. For solar energy, sky imaging traces cloud conditions in real time, which allows us to use motion vectors derived from image processing succeeding images, or if there is a co-located wind profiler, one can project the cloud motion for the next few steps (Kleissl 2010; Chu et al. 2013; Nguyen and Kleissl 2014; Peng et al. 2015). Cloud imager-based forecasts are effective for about 15–30 minutes.

Statistical learning methods have been shown to be effective from about 30 minutes to around three hours. Such methods may consist of using techniques such as artificial neural networks (ANN—Mellit 2008; Wang et al. 2012), autoregressive models (Hassanzadeh et al. 2010; Yang et al. 2012), Markov process models (Morf 2014) or support vector machines (Sharma et al. 2011; Bouzerdoum et al. 2013) to recognize patterns in the changes of wind speed or solar radiation. Blended techniques, such as that of Pedro and Coimbra (2012), that use a genetic algorithm to optimize an ANN have also been effective. In addition, new methods blend weather observations with irradiance observations and use clustering techniques to identify regimes and then train ANN for the individual regimes (Kazor and Hering 2015), which was shown to improve upon non-regime-dependent ANN and upon smart persistence (McCandless et al. 2016a). Recent work has added satellite data to this type of forecasting (McCandless et al. 2016b). Some methods also predict the variability of the resource (McCandless et al. 2015).

In the time range from about an hour out to six hours, cloud motion vectors derived from satellite data are often used to forecast solar irradiance (Miller et al. 2013). When a satellite observes the cloud cover and the forecast system then advects it using motion vectors derived from successive images or uses observed or modeled data to advect those clouds, the derived liquid water path can be used to provide good estimates of irradiance attenuation. Satellite-based methods depend on the satellite data being received and processed before being used in the motion vector models, which take on the order of 30 minutes after observation. They also do not account for cloud development or dissipation, so the forecasts of individual clouds are only accurate for a limited period of time, beginning to degrade after the first couple hours.

Wind energy forecasting at these scales of the first few hours can be improved by other methods of remote sensing. Mahoney et al. (2012) describe the Variational Doppler Radar Analysis System (VDRAS) that assimilates radar data into a cloud-resolving model to better predict winds. Because that model does not include the full physics, it can be updated at frequencies as high as every 15 minutes. That work showed that in case studies, the winds could be well predicted for the first two hours and could identify weather ramps as they approach. This method, however, relies on having radars sited in close proximity to the wind farms.

#### Numerical Weather Prediction

Beyond about three to six hours, the workhorse of forecasting is NWP. NWP consists of the integration of the nonlinear partial differential equations governing atmospheric flow and includes appropriate models for the physics of clouds, radiation, turbulence, land surface conditions and more (Warner 2011). As computer power has advanced, so has our capability to provide higher resolution simulations in closer to real time. The national centers now run very short-range simulations at about three kilometers horizontal resolution over limited regions (such as over the USA) as often as hourly. Global models have necessarily coarser resolution and run less frequently. As of mid-2017, the US Global Forecast System is run every six hours at 13 kilometers resolution with hourly output for the first five days and at 70 kilometers out to 16 days. The European Center for Medium Forecast is run twice a day at nine kilometers resolution for 16 days with output at three hours temporal resolution. The national centers are continually updating their model resolutions, lengths of simulations and frequency of the runs as computer power is upgraded. In addition, they are including ensemble runs, which provide probabilistic information as well as improving upon the deterministic forecast. As mentioned in the introduction and elaborated later, the sensitivity to initial conditions is what partly limits predictability, which necessitates running ensembles of models.

It is important to assimilate observations of weather data from the global networks to provide the best possible initial condition to the runs. To improve forecasts at specific points, such as at a wind farm, it is advantageous to also assimilate specialized data (such as wind speed measurements) at that farm. Mahoney et al. (2012) and Wilczak et al. (2015) provide evidence that assimilating local wind farm data can improve the NWP forecasts. In a case study, Cheng et al. (2017) show that real-time four-dimensional data assimilation can reduce the mean absolute error in the forecast by 30–40% in the first three hours. Kosovic et al. (2017) indicate that such local data assimilation can significantly improve forecasts over the national models for the first 15 hours. Versions of the models are now being built to specifically improve upon predicting variables of particular importance to energy, such as wind (Wilczak et al. 2015), solar (Jimenez et al. 2016) and hydro (Gochis et al. 2014), although the use of national hydrological models is still in its infancy.

#### Blending the Forecasts and Predicting Power

Modern forecasting includes postprocessing the NWP output and forecast blending to improve upon the results. At a basic level, a multivariate statistical regression model known as model output statistics (MOS) is applied to remove biases (Glahn and Lowry 1972). More complex methods (such as ANN, autoregressive models and others) are also used to provide nonlinear corrections to models (Myers et al. 2011; Myers and Linden 2011; Giebel and Kariniotakis 2007; Pelland et al. 2013). Ensemble MOS (Wilks and Hamill 2007) not only corrects the individual models but also optimizes weights for blending forecasts from multiple models. It is typical to blend multiple models together using some statistical learning technique to produce a better forecast than any single model could produce consistently, often with a 10–15% improvement over the best model forecast (Mahoney et al. 2012; Myers et al. 2011). Figure 7.2 displays an example of wind speed forecasts using National Center for Atmospheric Research (NCAR's) Dynamic Integrated foreCast (DICast®) system to blend multiple models, with the blended forecast showing a lower average root mean square error than any input model.

#### Probabilistic Forecasts and the Analog Ensemble

Utilities and ITOs are also requesting probabilistic information to plan reserves, alleviating transmission bottlenecks, and better planning for renewable operations. To this end, the meteorology sector has been providing probabilistic forecasts. All major national forecasting centers now run ensemble forecasts. The rate of spread of the members of the ensemble quantifies the uncertainty. Various postprocessing methods are used to remove bias, sharpen the probability density function and calibrate the spread, or reliability, of the forecast.

Another interesting postprocessing method is the analog ensemble (AnEn) technique. In this case, however, rather than running multiple models, the historical output from a single, often high resolution, model is used to generate the ensemble. A search of the historical forecast records is made to identify the forecasts that are most similar to the current forecast. We then compare the forecasts to the corresponding observations. Those observations then become the analog ensemble. This method is effective at both improving on the deterministic forecast and using the multiple analogs to form an ensemble that can be used to quantify the

uncertainty of the forecast (Delle Monache et al. 2013; Haupt and Delle Monache 2014; Alessandrini et al. 2015).

Accuracy of forecasts has been improving steadily with some areas now seeing single digit errors in terms of percentage of capacity at a wind farm (Orwig et al. 2014; Haupt and Mahoney 2015). These improvements have stemmed from including observations in the immediate vicinity of the resource, both in the nowcasting and assimilated into the NWP models, as well as better methods of blending multiple models for the appropriate timescales. Solar power predictions have not been a focus for very long, but rapid improvement is also happening here (Lorenz et al. 2009; 2014; Tuohy et al. 2015; Haupt et al. 2017).

#### References

**Open Access** This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

# Medium- and Extended-Range Ensemble Weather Forecasting

#### *David Richardson*

**Abstract** The chapter provides an overview of ensemble weather forecasting for the medium- and extended-range (days to weeks ahead). It reviews the methods used to account for uncertainties in the initial conditions and in the forecast models themselves. The chapter explores the challenges of making useful forecasts for the sub-seasonal timescale, beyond the typical limit for skilful day-to-day forecasts, and considers some of the sources of predictability such as the Madden-Julian oscillation (MJO) that make this possible. It then introduces some of the ensemble-based forecast products and concludes with a case study for a European heat wave that demonstrates how ensemble weather forecasts can be used to guide decision making for weather-dependent activities.

**Keywords** Ensemble • Weather forecast • Uncertainty • Predictability • Medium-range • Extended-range • Sub-seasonal • Madden-Julian oscillation

© The Author(s) 2018 109 A. Troccoli (ed.), *Weather & Climate Services for the Energy Industry*, https://doi.org/10.1007/978-3-319-68418-5\_8

D. Richardson (\*)

European Centre for Medium-Range Weather Forecasts (ECMWF), Reading, UK

#### Preamble

Operational weather forecasts for the medium- and extended-range (days to weeks ahead) are generally based on the output from global Numerical Weather Prediction (NWP) or General Circulation Model (GCM) ensemble forecasts.

#### Initial Condition Uncertainties

The main aim of the ensemble approach is to account for uncertainty in the initial atmospheric conditions. Typically this is done by adding small perturbations to a single "best-estimate" analysis of the current state of the atmosphere. The analysis is generated through assimilation of observations into the NWP model. The size of the perturbations is constrained to be consistent with the known analysis errors. A number of different methodologies are used to generate perturbations that are physically realistic and that will grow to represent the range of possible future states consistent with the initial uncertainty.

The error-breeding method (Toth and Kalnay 1993, 1997) uses a cycling approach where the differences between short-range forecasts are re-scaled to form the initial perturbations for the next forecast. A number of generalisations of the original breeding method have been developed, designed to improve the representation of the analysis uncertainty at each initial time. These include the Ensemble Transform Kalman Filter (ETKF, Bishop et al. 2001), used at the Met Office (Bowler et al. 2008, 2009), and the Ensemble Transform with Rescaling (ETR, Wei et al. 2008) used at the United States National Centers for Environmental Prediction (NCEP).

The Singular Vector (SV) method (Buizza and Palmer 1995) computes new perturbations at each analysis time. The method identifies the fastest growing perturbations over a given time period (e.g. 48 hours). A linear combination of these SVs, scaled to have amplitudes consistent with the analysis error, is added to the best-estimate analysis to make the starting conditions for each ensemble member. The SV method is used operationally at the European Centre for Medium-Range Weather Forecasts (ECMWF) (Leutbecher and Palmer 2008) and the Japan Meteorological Agency (JMA) (Yamaguchi and Majumdar 2010).

Other methods more directly address the observation uncertainty by perturbing the observed values themselves. At ECMWF, perturbations are also provided from an Ensemble of Data Assimilations (EDA) (Buizza et al. 2008). Each EDA member is an independent data assimilation, using the same set of observations, but introducing perturbations to these observations consistent with the known observation errors. The Meteorological Service of Canada (MSC) uses perturbed observations and an ensemble approach, the ensemble Kalman filter (EnKF, Houtekamer and Mitchell 2005; Houtekamer et al. 2009, 2014), to provide an ensemble of initial conditions. It should be noted that for both the EDA and EnKF, it is necessary to take account of model uncertainties (see below) as well as the observation uncertainties to generate appropriate initial perturbations.

#### Model Uncertainties

Global NWP ensemble forecasts typically run with a grid spacing of a few tens of kilometres. Many important physical processes (that affect, e.g., clouds and precipitation) work on much smaller spatial scales than can be resolved directly. These processes are represented in the NWP models by "parametrization schemes" that describe the aggregate effect of the smaller-scale unresolved processes on the larger resolved scales.

The finite resolution of the NWP model and the approximations made in the parametrisation schemes are sources of model uncertainty. Most global NWP ensembles also now include a representation of these uncertainties in the model formulation. A range of methods has been developed and sometimes a combination of methods is used in a single ensemble system to account for different aspects of model uncertainty.

One approach is to use a number of different parametrisation schemes within the ensemble. For example, there are various ways to parametrise convective processes and an ensemble can be generated by running some members using one convection parametrisation scheme, while other members use a different convection scheme (Charron et al. 2010). An alternative is to use a single parametrisation, but to perturb some of the key parameters in the scheme (Bowler et al. 2008). Other schemes represent the uncertainty from the sub-grid scale by stochastically perturbing the tendencies from the parametrisation schemes, as, for example, in the Stochastically Perturbed Parametrisation Tendency scheme (SPPT, Leutbecher et al. 2017; Buizza et al. 1999). Backscatter schemes are designed to simulate the transfer of energy from the unresolved sub-grid scales to the larger scales that are resolved by the model (Shutts 2005; Berner et al. 2009).

#### Operational Global Medium-Range Ensembles

A number of meteorological centres produce operational medium-range ensemble forecasts. Initial condition and model uncertainties are represented using many of the above methods, with different centres adopting different approaches. The TIGGE (The International Grand Global Ensemble) project provides access to regular global ensemble predictions from ten of the leading global NWP centres to support research and has facilitated a comprehensive evaluation of the global ensembles produced by different NWP centres (Swinbank et al. 2016; Bougeault et al. 2010; and references therein).

Comparison of the forecasts from the TIGGE centres confirms that while the different centres each have their strengths and weaknesses, the different perturbation methodologies all have merit. It is more important that an NWP system produces an ensemble that accounts for both initial condition and model uncertainties than the precise methodology used to produce the perturbations. However, it is also important to carry out proper and comprehensive evaluation to ensure that the perturbations are consistent with the uncertainties of the system. Figure 8.1 shows an example of the skill of operational ensemble forecasts from five global centres in predicting the large-scale weather patterns over the extra-tropical northern hemisphere up to two weeks ahead (the temperature at 850 hPa is a good indicator of whether a location is under the influence of a warm or cool circulation pattern). Skill is measured using the Continuous Ranked Probability Skill Score (CRPSS), a standard measure for assessing the usefulness of probabilistic forecasts, which can also be interpreted as an indication of the potential economic value of the forecast systems (Palmer and Richardson 2014). CRPSS ranges from a maximum value of 1 (perfect knowledge of what the weather will be) to zero (only the climatological information is known). This evaluation shows that all the forecasting systems have positive skill in forecasting day-to-day changes in the weather for up to two weeks ahead.

#### Extended-Range Ensembles

The medium-range ensembles described above typically produce forecasts for one to two weeks ahead. This is usually considered the limit for dayto-day predictability, as the influence of the atmospheric initial conditions is much reduced at longer range.

However, there are important sources of predictability that do give the potential to make useful forecasts for the sub-seasonal scale (one or two months ahead). One important phenomenon is the Madden-Julian oscillation (MJO), a feature of the tropical atmosphere that also impacts on the weather in the extra-tropics (Lin et al. 2009; Cassou 2008). The MJO evolves over a period of 40–60 days, and so is a potential source of forecast skill for several weeks ahead. Initial conditions in the stratosphere can affect the circulation in the troposphere over the following month, providing another source for sub-seasonal predictability (Baldwin and Dunkerton 2001). Other potential sources of predictability at this timescale include the land surface (Koster et al. 2010) and snow cover (Jeong et al. 2013) conditions at the start of the forecast.

There have been significant improvements in sub-seasonal forecasts in recent years, with large improvements in skill for predicting the MJO (Fig. 8.2), as well as its influence on other regions, including Europe (Vitart 2014). There is now a growing interest in developing applications to exploit these forecasts as well as to improve the forecasts themselves.

Following the success of TIGGE, a new sub-seasonal to seasonal prediction project (S2S) has been initiated by the World Weather Research Programme (WWRP) and World Climate Research Programme (WCRP). The main goal of this five-year project is to improve forecast skill and understanding of the sub-seasonal to seasonal timescale and to promote its uptake by operational centres and its exploitation by the applications community (Vitart et al. 2012).

The S2S database includes near real-time ensemble forecasts for up to 60 days ahead, from 11 forecasting centres: Australia's Bureau of Meteorology (BOM); the China Meteorological Administration (CMA); ECMWF; Environment and Climate Change Canada (ECCC); Italy's Institute of Atmospheric Sciences and Climate (CNR-ISAC); the Hydrometeorological Centre of Russia (HMCR); the Japan Meteorological Agency (JMA); the Korea Meteorological Administration (KMA); Météo-France; the US National Centers for Environmental Prediction (NCEP); and the UK Met Office (Vitart et al. 2017).

These models are generally different from the NWP models used to produce medium-range forecasts at the same centres. Most are coupled to an ocean model, as it is important to take account of the evolution of the sea-surface temperature and its interaction with the atmosphere over the longer time periods of these forecasts. For the same reason, some systems also include an active sea ice model. Some centres, such as ECMWF, that

**Fig. 8.2** Forecast lead-time (in days) when a correlation-based measure of accuracy of the prediction of the Madden-Julian Oscillation (MJO) reaches 0.6 correlation (orange bars) and 0.5 correlation (yellow bars) (1.0 would indicate a perfect forecast). The black lines indicate the 95% confidence interval of the time when the 0.6 correlation is reached. Results are based on the re-forecast from 1999 to 2010 from all the models, verified against ERA-Interim analyses. Correlations of 0.5 and 0.6 are often used as indication of useful forecast skill (Vitart 2014)

already include an ocean model in the medium-range ensemble, use the same ensemble system for both the medium-range and sub-seasonal forecasts, hence providing a seamless set of predictions covering the timescales from days to months ahead. While the skill in predicting the day-to-day weather may be limited to around two weeks in general (as seen in Fig. 8.1), many of the S2S models demonstrate substantial skill out to three or even four weeks ahead for the MJO (Fig. 8.2); this can lead to enhanced predictability, for example, over Europe, in certain situations.

Another important aspect of the sub-seasonal forecasts is the need to account for model errors. Systematic model errors (biases) can accumulate during the forecast, and while they are often small enough to be neglected for medium-range forecasts, they become too large to be ignored at the longer sub-seasonal range. Hence, an additional set of historical ensemble integrations is generated by making forecasts from start dates covering the last 15–20 years. These re-forecasts (or hindcasts) are used to estimate the model climate. This can then be used to remove the model biases from the real-time forecasts in a statistical post-processing step (Vitart 2014).

#### Ensemble Weather Forecast Products

Ensembles are designed to take account of the uncertainties in the initial conditions and in the NWP model used to make the forecast. The set of ensemble forecasts provides a direct quantitative indication of the range of possible future weather scenarios that may occur. Most ensembles are constructed so that each member of the ensemble is equally likely. The proportion of ensemble members forecasting a specific weather event gives an indication of the probability for it to occur. Grouping the ensemble into a small number of clusters can be valuable for those cases when there are distinct alternative scenarios within the ensemble (Ferranti et al. 2015).

The forecast values of weather variables (temperature, wind, rainfall, etc.) are typically generated on a 10–50-kilometre spatial grid for mediumand extended-range ensembles. These values are not directly comparable to the measurements recorded at specific locations since they represent the average for the area covered by a grid-box. Statistical post-processing can substantially improve the forecasts by tailoring (down-scaling) the gridbox forecasts to smaller areas or individual sites, and also accounting for the finite ensemble size (Hemri et al. 2014). Re-forecasts are useful for calibrating medium-range forecasts as well as for the sub-seasonal range, and re-forecast datasets are increasingly becoming part of the mediumrange forecast configurations. This can be especially important for severe weather forecasting by providing information about how the model performed for severe events in the re-forecast period.

The Extreme Forecast Index (EFI) was developed at ECMWF to highlight potential anomalous weather events, by comparing the real-time forecast to the re-forecast model climate distribution (Lalaurette 2003).

Beyond a few days ahead it is no longer possible to predict the day-today changes in the weather at specific locations. However, by considering the average conditions over a period of time, it is possible to give skilful forecasts for longer lead times (Buizza and Leutbecher 2015). Subseasonal forecasts typically predict average conditions for each week of the coming month. At this longer forecast range, the prediction of changes in large-scale weather patterns is important, for example, to give an early indication of the onset of heat waves or cold spells.

In summer 2015, a heat wave affected large parts of Europe—temperature records were broken in many places, including in Germany, France and Spain. Early indications of widespread warmer than normal conditions during the first week of July can be seen in the ECMWF forecast from 16 June (Fig. 8.3). The signal becomes noticeably stronger in the forecast

**Fig. 8.3** ECMWF forecasts for the heat wave over Europe in July 2015. Lower panel shows the 2-metre temperature anomaly forecasts for the 7-day period 29 June to 5 July initialised on 18 June (left) and 22 June (right). Areas where the forecast distribution is significantly different from climatology are shaded. Upper panel shows the evolution of the ensemble forecasts for the temperature in Paris at 12 UTC on 1 July; the dates on the horizontal axis indicate the start time of each forecast. The box-and-whisker plots show the 1st, 10th, 25th, 75th, 90th and 99th percentile of the forecast, while black dot shows the median of the distribution. The temperature distribution of the model climate (generated from reforecasts for late June and early July for the last 20 years) is shown in red (the dotted line highlights the climate median). Magnusson et al. 2015.

from 22 June. While these successive forecasts give a good indication of the general situation, it is also interesting to consider the forecast for a specific time and location. The top panel of Fig. 8.3 shows a sequence of forecasts for the temperature in Paris at 12 UTC on 1 July (this was the second warmest day on record for Paris, with temperatures reaching to almost 40°C later in the afternoon). Each forecast is represented as a boxand-whisker plot that summarises the information in the ensemble. The forecasts from early June indicate a range of possible outcomes, similar to the model climate distribution (shown in red), with no clear signal for temperature to be warmer or cooler than normal. This is not surprising: as noted in the previous paragraphs, we should not expect skill in predicting the precise temperature at a specific place and time out to a month ahead. By mid-June there is a noticeable shift in the forecasts: while not certain, the ensemble indicates that high temperatures are much more likely to be above normal than below normal, and there is a significant chance that the temperature in Paris could reach more than 30°C. The risk of extreme temperatures increases in consecutive forecasts, and by 26 June the outcome is almost certain (see Magnusson et al. 2015 for more details).

This example shows how ensemble weather forecasts can be used to guide decision making for weather-dependent activities in the days and weeks ahead. For the coming days, detailed information can be obtained on the weather at a particular place and time. Looking further ahead, these details are less predictable, but the forecasts can give an indication of the likely general weather situation, and what alternative scenarios may be. While the weather is not always predictable out to a month ahead, there are particular situations where the predictability is enhanced—on such occasions the ensemble will show higher confidence and a more limited range of alternatives. In all cases, the ensemble approach provides important information about possible alternative scenarios, and enables users to make appropriate decisions, taking account of the confidence and risks quantified by the ensemble.

#### References

**Open Access** This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

### Seasonal-to-Decadal Climate Forecasting

#### *Emma Suckling*

**Abstract** Forecasting climate over the near-term, from a season to decades ahead, has the potential to inform decision-making within the energy sector in a number of ways: from energy trading to scheduling maintenance and resource management. Recent advances in forecasting at these timescales have led to promising levels of skill in predicting the large-scale drivers of seasonal and multi-annual climate variability as well as the consequent local climate impacts of relevance for the energy sector (e.g. seasonal temperatures and wind speeds). This chapter discusses the unique aspects of near-term prediction, how it differs from the task of weather prediction and long-term climate projections, the sources of predictability on these timescales, as well as some of the current climate forecasting tools and products aiming to provide value to the energy sector.

**Keywords** Seasonal-to-decadal • Forecasting • Climate • Predictability • Skill

E. Suckling (\*)

© The Author(s) 2018 123 A. Troccoli (ed.), *Weather & Climate Services for the Energy Industry*, https://doi.org/10.1007/978-3-319-68418-5\_9

NCAS-Climate, Department of Meteorology, University of Reading, Reading, UK

#### Introduction to Climate Forecasting

Climate prediction over the near-term, from seasons to multiple decades ahead, has received much attention over recent years for its potential to inform decisions in areas such as risk management and adaptation planning (Smith et al. 2012; Kirtman et al. 2013; Doblas-Reyes et al. 2013b; Meehl et al. 2014). In theory, local and regional scale forecasts on seasonalto-decadal timescales could be beneficial to the energy sector, for example in terms of understanding vulnerabilities under different energy mixes, planning future wind farm sites or developing resource management strategies. In practice, such forecasts must demonstrate that they are reliable and to add value to the practices currently adopted for decision-making.

Prediction on seasonal-to-decadal timescales occupies a middle ground between weather forecasting, whose goal is to provide a snapshot of atmospheric conditions at a particular point in time for a few days ahead, and climate projection, which aims to estimate the response to external forcings such as from greenhouse gases and aerosols. The goal of seasonal-todecadal prediction is generally to provide a statistical summary, or probability distributions, of conditions over the coming months and years given some knowledge of the current climate state, or phase of climate variability. The feasibility of predictions on these timescales arises from components of the climate system that evolve at a slower rate than the atmosphere, such as the ocean and land surface, and the interactions between them (Palmer and Hagedorn 2006; Meehl et al. 2009; Boer 2011). Sources of potential prediction skill include both externally forced low-frequency variability due to anthropogenic factors (such as greenhouse gas and aerosol concentrations and land use changes), as well as natural variations in solar activity and volcanic aerosol, and low-frequency variations within the climate system (such as large-scale modes of variability in the atmosphere and oceans).

#### Sources of Predictability

On seasonal timescales, the main source of predictability is the coupled ocean-atmosphere El-Niño Southern Oscillation (ENSO) phenomenon (Trenberth et al. 2000; Alexander et al. 2002; Wu et al. 2009), which has been a major factor in the success of seasonal forecasting using both dynamical and statistical models (van Oldenborgh et al. 2005; Coelho et al. 2006; Weisheimer et al. 2009; Wu et al. 2009). This mode of variability in the tropical Pacific is known to influence local-scale seasonal climate in many remote locations over the globe. The Madden Julian oscillation (MJO), in the tropical West Pacific and Indian Oceans, is characterised by an eastward progression of atmospheric convection (Madden and Julian 1971). Studies suggest evidence of teleconnections between the MJO and variability in the extratropics in the Pacific basin (Kim et al. 2006) and Atlantic (Cassou 2008) on monthly timescales. Seasonal predictability is also thought to arise from interactions between the troposphere and stratosphere, associated with the quasi-biennial oscillation (QBO) (Baldwin et al. 2001) and sudden stratospheric warming (SSW) (Marshall and Scaife 2010) which can have an important influence on winter conditions, particularly over Europe (Ineson and Scaife 2009). The North Atlantic Oscillation (NAO) is another source of low-frequency variability often attributed to stratosphere-troposphere coupling (Scaife et al. 2005) and is known to influence winter temperatures and rainfall over Northern Europe and Central Asia (Ineson and Scaife 2009; Matthes et al. 2006). Recent studies have reported skill at predicting winter NAO from seasonal forecasts using dynamical models (Scaife et al. 2014). Evidence of teleconnections between winter NAO and European climate in the following spring has also been suggested based on statistical analyses using observations (Herceg-Bulić and Kucharski 2013). The NAO has also been implicated as a predictor of Northern Hemisphere temperature variability at multidecadal timescales (Li et al. 2013).

On timescales of years to decades ahead, a major source of predictability is likely to arise from slowly evolving (multidecadal) variations in sea surface temperatures (SSTs) in the North Atlantic, referred to as Atlantic Multidecadal Variability (AMV). North Atlantic SST fluctuations are linked to variability in the Atlantic Meridional Overturning Circulation (AMOC), which may vary naturally or through external influences, such as volcanoes or greenhouse gases. It has been suggested that AMV and AMOC could be potentially predictable several years ahead (Griffes and Bryan 1997; Dunstone and Smith 2010) and some evidence suggests that associated changes in climate over Europe, America and the African Sahel, as well as in the strength and position of the Atlantic storm track (Knight et al. 2006; Sutton and Hodson 2007; Sutton and Dong 2012) could also be predictable. Pacific decadal variability (PDV) is also associated with climate impacts over America, Asia, Africa and Australia on multidecadal timescales (Power et al. 1999; Deser et al. 2004); however, the processes involved are currently not well understood and evidence of potential predictability is weaker than for AMV. The Indian Ocean Dipole, characterised as a fluctuation in SSTs, has recently been associated with changes in winds and rainfall across Africa and India on interannual timescales (Webster et al. 1999).

In addition to large-scale modes of internally generated natural variability, the evolution of the climate system is affected by external factors, including a response to anthropogenic activity as well as natural factors such as solar activity and volcanic eruptions. Solar activity is somewhat predictable in terms of sunspot number and solar radiative output, which varies with a period of approximately 11 years. The influence of solar activity on global climate is fairly small and studies suggest global temperatures may vary by about 0.1°C due to solar activity (Lean and Rind 2008), as well as influence stratospheric temperatures and induce small changes in tropical atmospheric circulation (see e.g. Gray et al. 2010, Gray et al. 2016, Lockwood et al. 2010). Volcanic activity cannot be predicted in advance, but large volcanic eruptions have a significant impact on forecast skill once they have occurred. Aerosols emitted into the stratosphere affect global and local temperatures over the timescale of a year or two, as well as the hydrological cycle and atmospheric circulation. Volcanic activity can also impact ocean circulation and heat content on timescales of years or decades, which has important implications for decadal forecast skill (Marshall et al. 2009).

#### The Probabilistic Nature of Climate Forecasting

Climate forecasts are necessarily formulated in a probabilistic way (usually as ensembles of deterministic possible outcomes), owing to the inherently uncertain nature of climate prediction. Both the chaotic nature of the climate system (in which small errors in estimating the initial climate state grow with time in any forecast) and inadequacies of the forecast systems themselves (due to approximations in the formulation of the models and missing processes and feedbacks) contribute to forecast uncertainties, which play an important role in the interpretation and use of such forecasts for decision-making. The relative contributions of different sources of forecast uncertainty depend on the timescale, region and variable of interest, but can be difficult to disentangle (see Fig. 9.1). However, at shorter timescales, from months to years ahead, internal variability (from mechanisms such as ENSO or MJO) is typically a major source of uncertainty, accounting for the largest fraction of the variance (e.g. in global or regional temperatures) in studies based on model predictions (Hawkins and Sutton 2009). Over the near-term, up to a few decades ahead, scenario uncertainty (which is assessed through a series of possible pathways that include estimates of future greenhouse gas emissions, land use changes and socio-economic factors) is typically not a dominant source of uncer-

**Fig. 9.1** Relative contributions to the fraction of total variance from each source of uncertainty in projections of decadal mean surface air temperature in a) global mean and b) Europe mean. Green regions represent scenario uncertainty, blue regions represent model uncertainty, and orange regions represent the internal variability component. The importance of model uncertainty is clearly visible for all policy-relevant timescales. As the size of the region is reduced, the relative importance of internal variability increases. Scenario uncertainty only becomes important at multidecadal lead times (Hawkins and Sutton 2009, see also Kirtman et al. 2013).

tainty at these timescales and is shown to account for only a small fraction of variance in model studies. Model inadequacy accounts for a large source of uncertainty, particularly in the near-term; therefore, the evaluation and comparison of a variety of forecast approaches and models are crucial both in terms of understanding sources of predictability and skill and in terms of designing and calibrating any prediction system for operational use.

Recent progress in the development of seasonal-to-decadal prediction systems includes the use of empirical methods, dynamical models or a combination of both. Empirical methods are based on observations and exploit statistical relationships to represent physical processes. Empirical models include simple approaches such as characterising the historical climatology, or persisting current conditions, as well as more sophisticated methods such as linear regression models (e.g. Eden et al. 2015; Suckling et al. 2016), constructed analogues or neural networks (see, e.g. van den Dool 2007). However, such models rely on an adequate database of historical observations on which to train the model, which is not always available. Dynamical models approximate solutions to the fundamental physical equations that characterise the Earth system. Seasonalto-decadal predictions based on dynamical models typically involve combining a boundary condition problem (i.e. simulating the response to forcings and the feedbacks between them) with an initial condition problem, in which the current state of the atmosphere, ocean, cryosphere and land surface is estimated by initialising the model to a state close to observations through data assimilation (see, e.g. Smith et al. 2012). The aim of initialisation is to narrow uncertainty in the predictions by taking into account the phase of internal climate variability (Doblas-Reyes et al. 2013a). The process of initialisation is not trivial, however, and constraining a model with observations generally causes initialisation shocks, which impact the forecast skill. Furthermore, systematic biases cause a model to drift away from the observations over time towards its preferred climatology. Several recent studies have adopted different methodologies for attempting to overcome such biases (Balmaseda et al. 2009); however, at present the common approach to dealing with model bias and drift is to remove any systematic errors through post-processing. Postprocessing refers to approaches that are used to transform raw model output into forecast products and includes calibration (e.g. bias adjustment or downscaling) and combination (in which information from different sources and models are combined to form a single forecast) (see, e.g. Doblas-Reyes et al. 2013b).

#### Assessing the Quality of Climate Forecasts

The quality of seasonal-to-decadal prediction systems is typically assessed by comparing predictions from different models against each other and against observations over a historical hindcast period (Goddard et al. 2013). Hindcasts (sometimes referred to as reforecasts) are essentially retrospective forecasts generated using today's state-of-the-art models, based on knowledge of the historical climate drivers. Several different attributes of the forecast ensemble are often quantified, including model bias and ensemble spread, as well as the correspondence between forecast and outcome pairs using a variety of statistical metrics (Jolliffe and Stephenson 2003). Such metrics include deterministic skill scores, which consider the ensemble mean properties of a set of predictions, and probabilistic skill scores (Bröcker and Smith 2007) that quantify the quality of the full distribution of ensemble members relative to a reference forecast system (such as climatology, persistence or another forecast system) (Suckling and Smith 2013). Reliability measures the correspondence between the predicted probabilities and observed frequencies of a particular set of events. The evaluation of any forecast system in this way is crucial, not only for the development and improvement of systems for operational use, but also in understanding when forecasts are likely to provide reliable information (e.g. Weisheimer and Palmer 2014).

#### Climate Forecast Tools for the Energy Sector

Both empirical approaches and the use of initialised dynamical prediction systems have been relatively successful for seasonal forecasting, leading to the availability of a number of operational products including the North American Multi-Model Ensemble (NMME)1 and the EUROSIP multimodel seasonal forecasting system.2 Initiatives such as the Advancing Renewable Energy with Climate Services (ARECS) and European Climatic Energy Mixes (ECEM)3 projects aim to develop state-of-the art tools and forecasts that are relevant to the energy sector (see Fig. 9.2).

The field of decadal forecasting is still relatively new, but is rapidly developing. Currently decadal predictions are not widely used as operational products; however, model intercomparison projects such as CMIP5 (Taylor et al. 2012) have advanced the science base for decadal prediction using dynamical model and projects such as the Decadal Forecast Exchange4 and the World Climate Research Programme (WRCP) Grand Challenge on Near-Term Climate Prediction (https://www.wcrp-climate.org/

**Fig. 9.2** Example of the information available from the ECEM Demonstrator tool (http://ecem.climate.copernicus. eu). Historical monthly mean wind speed for November 1979 over Europe. Essential climate variables and energy impact indicators are available on a range of timescales, including a historical reanalysis, seasonal forecasts and climate projections

grand-challenges/gc-near-term-climate-prediction) aim to facilitate development of decadal forecast approaches towards operational use.

#### Concluding Remarks

The recent interest in seasonal-to-decadal forecasting has led to significant improvements in forecast skill, as well as a better understanding of the sources of forecast skill and climate predictability. At seasonal timescales, ENSO is reliably predicted several months ahead by the latest empirical and dynamical models, and ENSO teleconnections are also well predicted (Wang et al. 2009). Recent analyses of the reliability of seasonal forecasts have indicated that temperature predictions in the east and west coast of North America and parts of China and East Asia are reliable, particularly in winter (DJF), while predictions over South America, Southern Africa and Australia are reliable in austral winter (JJA) (see Fig. 9.3) (Weisheimer and Palmer 2014). Predictions in other regions and seasons may also be useful as decision-relevant tools; however, there is often a lot of diversity between different models, making it difficult to make statements about the broad level skill in those regions. Predictions of precipitation are generally less reliable, however. On the other hand, near surface winds are strongly constrained by the ocean in the tropics and are therefore relatively predictable. Furthermore, recent indications of skill in predicting the NAO during winter show promising signals of predictability of winter winds and temperatures in the extratropics, particularly over Europe (Scaife et al. 2014).

On decadal timescales, empirical methods have been demonstrated to represent temperatures well, both for externally forced signals (Lean and Rind 2008; Suckling et al. 2016) and for idealised studies of internal variability (Hawkins et al. 2011). There is also evidence of predictability in north Atlantic SSTs related to skill in predictions of the AMOC (Pohlmann et al. 2013); however, evaluations are currently complicated by a lack of observations over an extended period. The initialisation of model predictions has provided some evidence of improvements in AMV and other large-scale models of variability (Doblas-Reyes et al. 2013a), as well as in predictions of, for example, Atlantic hurricane number more than a year in advance (Smith et al. 2010).

The future direction of climate forecasting for the energy sector will see continued efforts in the development of climate prediction systems, including a new generation of high-resolution global and regional climate models under projects such as PRIMAVERA,5 as well as improved understanding of the mechanisms that drive local variability and impacts and the

**Fig. 9.3** Example of communicating seasonal forecasts skill information. Reliability of the European Centre for Medium-Range Weather Forecasts (ECMWF) Seasonal Forecast System 4 for predictions of 2m temperature during (**a**) cold DJF, (**b**) warm DJF, (**c**) cold JJA and (**d**) warm JJA (Weisheimer and Palmer 2014)

conditions which lead to greater predictability and forecast skill. Initiatives such as the Copernicus Climate Change Service (C3S—climate.copernicus.eu) and its related projects are already working towards tailoring stateof-the-art climate forecasts for the energy sector, including the delivery of energy-relevant climate variables and energy-impact indicators from a season to several decades ahead.

#### Notes

#### References

Winter since 1900. *Journal of Climate, 17*, 3109–3124. https://doi. org/10.1175/1520-0442(2004)017<3109:PICVLB>2.0.CO;2.

A. Nauels, Y. Xia, V. Bex, & P. M. Midgley (Eds.), *Climate change 2013: The physical science basis. Contribution of working group I to the fifth assessment report of the intergovernmental panel on climate change* (pp. 953–1028). Cambridge; New York: Cambridge University Press. https://doi.org/ 10.1017/CB09781107415324.023.

**Open Access** This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

### Regional Climate Projections

#### *Robert Vautard*

**Abstract** When designing adaptation and mitigation measures of climate change for the coming decades and up to the middle of the century, policymakers and industries must rely upon climate information that is at an appropriate scale to evaluate impacts, vulnerabilities and risks due to changes in climate. It is, therefore, essential that the quantitative information on the climate and its impacts is reliable. Reliable quantitative information about climate change impacts must also be available. This includes estimations of uncertainty bounds. In the current state of knowledge, technology and structure of scientific communities, climate change impact studies are achieved from a suite of models: global earth system models, with a generally low-resolution (100–300 km), regional limited-area climate models with a higher resolution (10–50 km), which take their boundary conditions from global models and impact models calculating how changes in weather, ocean and biogeochemical cycles affect the system to be adapted.

**Keywords** Downscaling • Uncertainty • Extreme events • Climate projections • Scenarios

R. Vautard (\*)

Laboratoire des Sciences du Climat et de l'Environnement, Institut Pierre-Simon Laplace, Paris, France

<sup>©</sup> The Author(s) 2018 139

A. Troccoli (ed.), *Weather & Climate Services for the Energy Industry*, https://doi.org/10.1007/978-3-319-68418-5\_10

#### **Regional climate projections**

**Fig. 10.1** Schematic of the modelling chain used to calculate the impacts of climate change. In this illustration, the impacts can be the river discharge or hydropower potential

#### Introduction

Regional climate projections are the key chain elements that provide information at a scale allowing impacts calculations, and the assessment of adaptation measures. Regional climate projections also allow an improved simulation of physical and biogeochemical processes, and of extreme events at the scales relevant for impacts studies (see schematic of Fig. 10.1).

Uncertainty in climate projections cannot be directly estimated by comparing them against observations, as in the case of weather forecasts, due to the long timescales involved. This is why uncertainty is generally estimated from the spread of ensembles of simulations. The larger the spread, the larger the uncertainty. This emphasizes the need to estimate impacts of climate change using ensembles of models. This section describes (1) the specific nature of climate projections as compared to forecasts previously presented, their main underlying assumptions, their uncertainty sources and how the science community is organized; (2) how regional climate projections add value to the global climate projections and (3) how they can be used in the energy sector.

### What Are Climate Projection and How Do They Differ from Weather Forecasts and Decadal Predictions?

Climate projections aim at predicting future climatologies, that is, the statistics of weather, of the state of the ocean, cryosphere, vegetation and atmospheric chemical composition. They are based on a number of assumptions about human evolution (population, economy, land use, land management, technologies and climate policies) and our understanding of the earth systems response to this evolution. Projections are considered as a "boundary value problem", where climate responds to external drivers. The principle is very different from that of weather forecasts, which is an initial value problem whose aim is to provide a snapshot of the state of the atmosphere at particular points in space and time (see also Chap. 6). Climate projections also differ from weather forecasts in that they use fully fledged earth system models describing the evolution of as many as possible variables concerning all compartments of the earth envelope, for example the ocean, cryosphere, as well as the atmosphere. Weather forecasts, when, for instance, limited to a few weeks, do not need to include changes in the oceans as well as other slowly evolving components of the earth system.

Climate projections also aim at explaining the evolution of past climate, from the instrumental period to paleoclimatic periods. A correct simulation of past periods, including the last century, millennium and beyond (Braconnot et al. 2012) provides some confidence to the models ability to simulate climate change, even though comparison with paleo observations often faces numerous scientific challenges. However, as demonstrated in the fifth assessment report of the IPCC WGI (Intergovernmental Panel on Climate Change Working Group I) (IPCC WGI 2013), climate evolutions along the twentieth century, and in particular the late warming, attributed to human influence on atmospheric composition, are fairly well reproduced: the amplitude of the warming is reproduced as well as the long-term modulations, thought to be due to aerosols.

Climate projections used for adaptation are usually initialized from an equilibrium state in a period when human influence is assumed to be minor relative to external natural forcing (solar and volcanic), in the middle of the nineteenth century. Since the memory of initial conditions disappears quickly, it is therefore not expected that projections synchronously represent the actual chaotic short-term observed evolutions across the instrumental period. Instead, one expects climate projections to represent possible trajectories of the weather, and the probability distribution functions (PDFs) of these fluctuations.

However, models have biases difficult to fully control, because of accumulation of approximations, which are due to low-resolution and insufficiently well-represented physical or biogeochemical processes such as convection or land-atmosphere interactions. In models, all these processes are in a balance that is typically shifted as compared to the real world. However, it is often assumed that such biases do not hinder a correct simulation of the response to external forcing. These potential error sources are often called "structural uncertainties".

Two other major sources of uncertainties are also present: the evolution of societies, climate policies and their effect on resulting atmospheric composition (greenhouse gases, aerosols), and on land use. Strong mitigation scenarios, including potential use of negative emission technologies, are likely to significantly alter land use (Smith et al. 2016) and therefore impact climate through an alternative pathway from atmospheric composition. Climate evolutions also undergo a natural internal variability at a timescale of a decade to several decades. Multidecadal modes and oscillations have long been identified (Ghil and Vautard 1991), with yet no full understanding. Such modes can induce temporary warming hiatus such as the one witnessed in the last decade or so. It is considered that the latter uncertainty source dominates for the few coming decades, while uncertainty on scenarios is the dominant driver for the second half of the century.

The global climate projections production that feed the IPCC assessment reports are internationally coordinated by the Climate Model Intercomparison Project (CMIP) (Taylor et al. 2012) supported by the World Climate Research Program. Modelling centres from around the world carry out a core set of simulation using the same natural forcings and socio-economic scenario assumptions. This makes simulations comparable and allows investigation of spread across the different models and some sources of uncertainty to be estimated.

International coordination not only takes place for climate simulations but also for data dissemination through the distributed Earth System Grid Federation (ESGF), with common standards, vocabulary and quality control. The ESGF data nodes1 , where climate simulations are available from, are part of a gigantic internationally coordinated climate data repository, with a number of functionalities (dynamic catalogue, quick look visualizations etc.). This coordinated effort allows for a systematic retrieval and use of model ensembles in order to account for climate projection uncertainties. The archive is now open to data from a number of projects, namely not just CMIP.

#### Regional Climate Projections

After a few pioneering projects such as PRUDENCE (Christensen and Christensen 2007), ENSEMBLES (Hewitt and Griggs 2004) or the North American Regional Climate Change Assessment Program (NARCCAP) (Mearns et al. 2012), regional climate projections were only recently coordinated at an international scale within the framework of the COordinated Regional Downscaling Experiment (CORDEX) (Giorgi et al. 2009). This framework aims at providing downscaled global climate projections through a coordinated approach over several regions of the world, using refined, but scale-limited models which take global model output as boundary conditions. A number of regions of the globe are now modelled using this approach, including Europe (Jacob et al. 2014). In Europe, these coordinated projections were made both at a low (50 km) and high (12 km) resolutions.

Regional climate projections are issued from limited-area regional climate models (RCMs) (Giorgi and Mearns 1999) which describe weather in a limited area (typically a region of around 25 thousand of kilometres square) with a higher resolution (typically 10–50 km) than global climate models (GCMs). These models are often only atmospheric models, forced by sea-surface temperatures and lateral boundary values of temperature, humidity and wind prescribed to be the values obtained from GCMs. Therefore, regional models solve the atmospheric equations with constraints on the boundaries. Recently, regional earth system models are also currently being developed, using ocean coupled with the atmosphere, which is a central development of the MED-CORDEX2 project (Ruti et al. 2015).

Just as is true for GCMs, RCMs have inherent climate biases that can be quite large when underpinning processes are poorly constrained by observations. Such is, for instance, the case of heat waves in Europe (Vautard et al. 2013). These biases add to those already present in boundary conditions provided by the GCMs. In particular, providing climate projections at a higher resolution does not annihilate potential biases in large-scale dynamical structures (jet streams, weather patterns), inherited from GCMs.

The expectation, however, is that RCMs better simulate atmospheric flows, and therefore weather, in the vicinity of marked topography or coastal areas, and other small-scale phenomena, compared to GCMs. Heavy precipitations are usually small-scale phenomena, and recent studies show a clear improvement in the statistics of such phenomena (Prein et al. 2016; Giorgi et al. 2016), in particular over mountainous areas, when using high-resolution simulations (e.g. 12 km as in EURO-CORDEX). This has a clear benefit for use in climate projections for studying applications such as winter tourism or mountain ecosystem impacts. By contrast, no added value of higher resolution was found for larger-scale phenomena such as heat waves (Vautard et al. 2013).

As for GCMs, ensembles of RCMs are assumed to provide a picture of uncertainties in projections. In order to cover a wider range of possibilities, a full GCM-RCM matrix should in principle be explored, each RCM downscaling all GCMs. In practice, while some strategies to explore all combinations have been developed (Mearns et al. 2012), these have not been implemented in practice. As of 2016, the EURO-CORDEX matrix largely remains to be filled for the high-resolution simulations with a 12 × 12 km2 grid.

Regional climate projections have climatological biases that usually need to be accounted for in impact studies. Bias adjustment is a statistical process that modifies model simulation values in order to adjust distributions to observed values. It uses a range of methods from simple mean bias removal to quantile mapping, in order to adjust the full distribution. More sophisticated schemes account both for past climate corrections and the evolution of distributions as simulated in models, such as the Cumulative Distribution Function transform (CDFt) method (Vrac et al. 2012; Vrac et al. 2016). It is an assumption that bias adjustment does not deteriorate the climate change signal, or even improves projections of changes. However, it is difficult to prove this in practice. It is therefore recommended that climate projections be processed by several bias adjustment methods.

The following example (see data description below) uses the CDFt method to bias adjust the number of rain days according to recent improvements (Vrac et al. 2016) using ten EURO-CORDEX simulations (available from the ESGF archive). Figure 10.2 shows an example of multi-model mean change in daily precipitation amount between the last 30 years of the twenty-first century and the corresponding period of the twentieth century. As an example, a clear, robust signal of precipitation increase is found in Northern Europe, and of precipitation decrease in Southern Europe, where more than eight models agree on the sign of the change.

#### The Use of Climate Projections for the Energy Sector

Regional climate projections can be used in several ways to help the energy industry and policymakers. Energy demand, renewable energy resources, risks from extreme events, cooling water for thermoelectric generation and other operation conditions all depend on weather and climate. The exposure to weather and climate variability and change will increase in the future decades owing to the tremendous energy transition that is

**Fig. 10.2** Mean changes in daily precipitation amounts estimated from ten EURO-CORDEX high-resolution model simulations (Jacob et al. 2014), in the RCP8.5 scenario. Changes are measured as differences of mean values calculated over the last 30 years of the twenty-first and twentieth centuries, averaged over the ten projections. Change values, represented by coloured areas, are only displayed when nine or ten models agree on the sign of change. When not, the area is coloured with grey

required to almost completely decarbonize the electricity generation by 2050 in order to reach ambitious climate targets (IPCC WGII 2014).

Both hydropower and thermoelectric generation could be at risk with climate change (van Vliet et al. 2016). In Europe, climate change may be expected to induce rather general decreases in wind power (Tobin et al. 2016), with a higher and more significant signal in Southern Europe (see Fig. 10.3, drawn from the results of the FP7 IMPACT2C project, WP6).

**Fig. 10.3**Changes in mean wind power capacity factor, assuming the installed national wind farms fleets as of 2013, under a 2°C (cyan) and 3°C (salmon) global warming. Ensembles consist of 5 simulations out of the 9 RCP4.5-RCP8.5 simulations that reach 2°C or 3°C global warming, using the methodology defined in Vautard et al. (2014). The coloured wide bars indicate the model ensemble mean. The thin bars indicate the 95% level confidence interval as computed using the Wilcoxon-Mann-Whitney test. Model individual changes are represented by differing symbols: symbols are red when changes are significant at the 95% level using the Wilcoxon-Mann-Whitney test (adapted from Tobin et al. 2016 and the results from the FP7 IMPACT2C project, WP6)

Solar photovoltaic (PV) power may also be expected to decrease in Europe as a result of the analysis of an ensemble of regional climate projections (Jerez et al. 2015).

Climate projections, generally available at the daily timescale, and in some cases at sub-daily timescale, can also be used as "weather generators". Bias-adjusted time series obtained from ensembles of simulations are usually 10–30 times longer than observed time series, due to the number of models available and do not suffer from homogeneity problems. They can generate statistics dedicated to the user's problem, such as the risk of typical extreme events. In some cases, infrastructures are built to cope with reference events, such as, for instance, extreme cold spells (e.g. winter of 1963) that occurred in the past. The risk of such events in current and future climates can be evaluated from the climate projection ensembles time series, although this is still a developing science. Reference events and their odds in the current and future climate can provide concrete information for adaptation.

For the needs of the energy sector, dedicated regional climate projections data sets have been developed and made available through prototype "climate services", such as, for Europe, CLIM4ENERGY3 and European Climatic Energy Mixes (ECEM)4 Copernicus Climate Change Service projects. Other data sets for other regions (e.g. NARCCAP data for North America5) are also available to be used for application in the energy sector. The value of such datasets for adaptation needs is still to be assessed by users as their use is relatively new.

#### Notes

#### References

**Open Access** This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

# The Nature of Weather and Climate Impacts in the Energy Sector

#### *David J. Brayshaw*

**Abstract** The power sector's meteorological information needs are diverse and cover many different distinct applications and users. Recognising this diversity, it is important to understand the general nature of how weather and climate influence the energy sector and the implications they have for quantitative impact modelling. Using conceptual examples and illustrations from recent research, this chapter argues that the traditional 'transfer function' approach that is common to many industrial applications of weather and climate science—whereby weather can be directly mapped to an energy impact—is inadequate for many important power system applications (such as price forecasting and system operations and planning). The chapter concludes by arguing that a deeper understanding of how meteorological impacts in the energy sector are modelled is required.

**Keywords** Climate variability • Energy sector • Power • Impact modelling • Weather impact • Meteorological information • Power demand

National Centre for Atmospheric Science, Reading, UK

© The Author(s) 2018 151 A. Troccoli (ed.), *Weather & Climate Services for the Energy Industry*, https://doi.org/10.1007/978-3-319-68418-5\_11

D.J. Brayshaw (\*)

Department of Meteorology, University of Reading, Reading, UK

#### Weather and Climate Impacts in the Energy Sector

The power sector's meteorological information needs are diverse. On the one hand, Transmission System Operators (TSOs) may be concerned with detailed geographical forecasts of wind power and demand at relatively short lead times (hours or days ahead) for the operational management of the power grid. This contrasts, for example, with long-term investors in infrastructure and system planners who require a longer view of system resilience (years to decades), and energy traders or maintenance planners seeking to position themselves for the coming weeks or seasons.

A common theme, however, is the need for a series of conversions to transform meteorological information into an actionable decision. The three steps in Fig. 11.1 can typically be recognised.

Chapter 6 discussed the first step in this process at length, and understanding user needs and preferences is discussed elsewhere in other chapters of this book (Chaps. 1, 3, 4 and 5). Here, the focus is on the general process of modelling energy system impacts using meteorological data from numerical simulations, illustrated with selected examples (i.e., Impact Simulation). It is, however, noted that user preferences—once elicited and expressed quantitatively—can be thought of as a conversion of a physical impact (in terms of MWh, prices, loss of load) into a 'utility' (a numerical expression of the user's preferences). To some extent, they can therefore be considered as direct extensions of the impact models discussed below.

It is helpful to identify three distinct levels of complexity in weatherand climate-impact modelling, as illustrated in Fig. 11.2.

**Fig. 11.1** The process of converting meteorological data into actionable information

**Fig. 11.3** Simple examples of idealised transfer functions used to convert meteorological quantities into estimates of power system properties: (**a**) an idealised wind power curve based on Brayshaw et al. (2011); (**b**) a simplified demand model based on Bloomfield et al. (2016). In each example here, the transfer function is shown to depend only on a single meteorological variable for simplicity but in general they may incorporate many input variables. Additional dependencies may be meteorological (e.g., wind direction for wind power, cloud cover for demand) or non-meteorological (e.g., day-of-week for demand), and include stochastic 'noise' to simulate the error and uncertainty in the transfer function

The simplest level, commonly referred to as 'point forecasting', may be defined as the response of a *single energy system component* to a set of meteorological drivers for which a *transfer function* can be written. Typical examples might include predicting wind power output for a particular turbine, farm or country, or forecasting power demand over a particular geographical region (Fig. 11.3). The key aspect is that it is possible to write (or otherwise estimate in at least an approximate form) a function, *f*, which converts a set of meteorological variables, {*m*}, into the energy system property of interest *E*:

$$E = f\left(\left\{m\right\}\right)$$

The transfer function may be either physically or empirically derived, may be non-linear, many-to-one or probabilistic. Typical examples include electricity demand models (Thornton et al. 2016; Taylor and Buizza 2003), wind power production models (Dunning et al. 2015; Cannon et al. 2015) and damage models (McColl et al. 2012).

A more complex form of impact occurs when the simultaneous influence of meteorology on several different components of an energy system becomes an important part of the impact. In this case, a transfer function exists for each component, but the impact is perceived through a combination of those components (often referred to as a 'compound impact'). An example is the residual power load of national power systems (i.e., demand net renewables), which depends on demand, solar and wind, each of which has a different sensitivity to weather.

In the simplest case, the system can be considered as a set of noninteracting energy system components,{*E*}, and may be written:

$$S = L\left(\left\{E\right\}\right)$$

where *L* is a mapping of the set of energy system components {E} to a particular system-wide property of interest, *S*.

An example1 of this is the 'merit order' model2 of UK wholesale power price, exploring the extent to which month-ahead forecasts could be beneficial to energy-trading and risk management (Fig. 11.4a; see Lynch et al. 2014; Lynch 2016). In this example, daily ensemble forecasts of UK wind power and national total power demand are created from the European Centre for Medium-Range Weather Forecasts (ECMWF) system for several weeks in advance, and the 'residual demand' calculated.3 The residual demand is assumed to be met by a mixture of coal and gas generation, preferentially utilising the cheapest marginal cost generators first (i.e., those bidding to produce power at the lowest price), with the wholesale power price being determined by the most expensive generation unit required to operate (Fig. 11.4b). The use of sub-seasonal weather forecasts three to four weeks ahead was shown to offer an improvement over standard industry practice for some—though not all—trading applications. This work therefore emphasised both the potential benefits of longer-range meteorological forecasts for energy, but also the need for careful evaluation of the forecast's performance *in the context for which it is being used.*

In both 'point impact' and 'compound impact' problems, the meteorological state is assumed to map directly to that of the impacted system (via a transfer function or set of transfer functions) and, although the mapping may be complicated, it is only dependent on the current meteorological state. This assumption does not hold, however, in many energy system planning and operations problems (e.g., 'optimal power flow' or 'unit commitment'). In problems of this type, there are potentially complex connections in time and space between different energy system components and to forecast the state of the impacted system accurately requires

**Fig. 11.4** An illustration of energy price forecasting using meteorological inputs following Lynch (2016) and Lynch et al. (2014). (**a**) A flow chart illustrating the process through which the forecast is made and evaluated. (**b**) A schematic of the 'merit order model'. In (**b**), the red curve indicates the relationship between supply and price (more expensive power stations are willing to produce as price rises, hence a positive relationship between volume and price). The blue curve indicates the relationship between demand and price (the demand for power decreases with price, but here is assumed to be perfectly price-insensitive). The intersection of the two curves sets the wholesale price and volume of power produced by the market. The qualitative shape of the supply curve produced by the two-generation type model (as fitted by Lynch (2016) to observed price data using an Ensemble Kalman filter) is indicated in (**b**). Lynch (2016) went on to demonstrate that the ECMWF-forecast based process outlined in (**a**) was able to significantly outperform equivalent forecasts using purely historical weather observations for each of wind power, demand and price (evaluated over the period December 2010— February 2014, at a 99% statistical confidence level). ECMWF stands for European Centre for Medium-Range Weather Forecasts

knowledge of both the power system's initial state and the meteorological evolution between the forecast's initialisation and its target lead time.