1065 Application  A high-level schematic diagram of our proposed hybrid AI-
assisted visual analysis framework

# HAiVA: Hybrid AI-assisted Visual Analysis Framework to Study the Effects of
Cloud Properties on Climate Patterns

Subhashis Hazarika1 e-mail<EMAIL_ADDRESS>Haruki Hirasawa2 Sookyung Kim1
Kalai Ramea1 Salva R. Cachay4 Peetak Mitra3 Dipti Hingmire2 Hansi Singh2 Phil
J. Rasch5 1Palo Alto Research Center, 2University of Victoria, 3Excarta,
4University of California, San Dieago , 5University of Washington

###### Abstract

Clouds have a significant impact on the Earth’s climate system. They play a
vital role in modulating Earth’s radiation budget and driving regional changes
in temperature and precipitation. This makes clouds ideal for climate
intervention techniques like Marine Cloud Brightening (MCB) which refers to
modification in cloud reflectivity, thereby cooling the surrounding region.
However, to avoid unintended effects of MCB, we need a better understanding of
the complex cloud to climate response function. Designing and testing such
interventions scenarios with conventional Earth System Models is
computationally expensive. Therefore, we propose a hybrid AI-assisted visual
analysis framework to drive such scientific studies and facilitate interactive
“what-if” investigation of different MCB intervention scenarios to assess
their intended and unintended impacts on climate patterns. We work with a team
of climate scientists to develop a suite of hybrid AI models emulating cloud-
climate response function and design a tightly coupled frontend interactive
visual analysis system to perform different MCB intervention experiments.

Introduction

Clouds play an important role in modulating Earth’s climate by regulating the
amount of solar energy that reaches the surface and the amount of energy that
is radiated back into space. They affect the coupled circulation of atmosphere
and ocean, driving regional changes in temperature and precipitation [34].
However, the climate response to changing clouds is one of the largest
uncertainties in our Earth System Models (ESM) [7, 8], particularly when
producing long-term climate projections. This limitation becomes apparent when
analyzing potential climate change impacts due to large changes in cloud
properties, e.g., the presence of greenhouse gases (loss of clouds) or climate
intervention techniques (higher cloud reflectivity) [26, 19, 38]. Climate
intervention techniques like Marine Cloud Brightening (MCB) [36, 16, 17],
which involves increasing the reflectivity of marine clouds (i.e. “brighten”
them) by introducing additional sea-spray particles near the clouds, require a
thorough understanding of how cloud properties impact large-scale global
circulation and regional climate patterns. This is also critical to assess and
avoid any unintended effects of MCB on overall climate patterns. Such studies
need to be fine-tuned spatio-temporally, by running thousands of ESM ensembles
to find optimal cloud brightening strategies. However, current ESM simulations
are computationally expensive, requiring tens of thousands of core-hours, thus
making it an impractical approach to explore a wide range of possible MCB
interventions and their potential impacts. To facilitate climate scientists in
this process, we have developed a hybrid AI-assisted visual analysis framework
to study cloud to climate responses and interactively explore different
intervention scenarios for MCB and evaluate their potential impacts across
sensitive geospatial sites around the globe.

Our proposed framework comprises of a suite of hybrid AI emulator models
trained for different time-lagged duration and a frontend interactive visual
analysis (VA) system which uses these models in the backend to analyze the
impact of cloud properties on climate patterns. The formulation of the hybrid
AI modeling approach is guided by Fluctuation-Dissipation Theorem (FDT) [15],
a fundamental principle in statistical physics which states that the forced
response of a system mirrors its internal fluctuations. Therefore, we perform
physics-aware preprocessing of ESM simulation data to remove any seasonal
cycles and secular trends, thus extracting month-to-month internal climate
variability (i.e. climate anomalies). The skeleton of our hybrid model is a
Multilayer Perceptron (MLP) architecture, that is designed to handle geodesic
data via a spherical graph sampling method. The model takes as input 6
different cloud radiative fields and predicts sea-level pressure,
precipitation, and air temperature as outputs. To ensure physically consistent
predictions, we include soft physics constraints based on radiative forcing
budget formulations during the training process.

The frontend interactive VA system is developed following the requirements and
guidelines expressed by our team of climate scientists. The multi-panel design
lets the scientists explore different facets of the ESM data and corresponding
model-driven analyses of cloud-climate properties. They can interactively run
the hybrid AI models in the backend, visualize the input and the output
anomaly fields using popular geospatial projections, as well as explore their
multivariate relationships across different spatial regions using interactive
parallel coordinate plots (PCP). To replicate the key steps involved in MCB
intervention studies we incorporated several user-input controls in our VA
system. Users can select the time-duration, spatial region(s), radiation
field(s) to perturb and the extent of perturbation for each MCB intervention
scenario analyzed using our system. Before running the backend hybrid models
with the new MCB input fields, scientists can validate any out-of-distribution
cases by visualizing the distribution-shift of the new radiation fields.
Another important aspect of this analysis is to track the potential impact of
an MCB intervention on the risk of climate tipping point at key climate-
sensitive geospatial site across the globe. We assess surface climate changes
that indicate the tendency of MCB impacts on seven different tipping points
for each MCB intervention performed in our VA system. Finally, during the
analysis task, the scientists can progressively save interesting MCB
intervention settings in a tabular form which can be later exported as
excel/csv files for further downstream experiments and expensive simulation
executions. Thus, our hybrid AI-assisted VA system serves as an efficient
scientific analysis workflow for climate scientists trying to study complex
cloud-climate relationships and hypothesize the impact of different MCB-based
climate intervention scenarios. We validated the predicted climate responses
of our hybrid model for a subset of controlled intervention scenarios against
baseline ESM simulations as well as performed qualitative evaluation of the VA
system with climate scientists and experts.

Figure 1: Complete layout of our proposed hybrid AI-assisted visual analysis
system. (C1, C2, C3) User input control panel to set analysis parameters,
including MCB intervention experiments. (V1) Input cloud radiation anomaly
visualization panel. (V2) Output climate anomaly visualization panel. (V3)
Distribution-shift visualization. (V4, V5) Tipping point risk assessment
panel.

##  1 Related Work

Visual analytics is an interdisciplinary field that combines the expressive
power of data visualization techniques with the modeling power of statistical
and machine learning methods to create efficient analysis workflows. Endert et
al. [6], in their 2018 paper, provides a comprehensive survey of how this
synergy between machine learning and data visualization have grown in the
visualization research community. Machine learning models like SVM (Support
Vector Machine) [39], LDA (Latent Dirichlet Allocation) [18, 24], KNN (K
Nearest Neighbor) [27], Bayes’ rule [9], learning-from-crowds model [23], and
online metric learning [21] have been extensively utilized by visualization
researchers to enhance the analysis experience in their VA systems. Deep
learning models like CNN (Convolutional Neural Network) [40] and Word2Vec [41,
25] models have also been utilized to design interactive VA systems for
different domain problems. Among the recent works of utilizing such system for
scientific domains, graph neural network based surrogate was designed to
explore the parameter space of expensive ocean simulation models [33]. A
multilayer perceptron surrogate model was used to design interactive VA system
for a computational biology domain [10]. Our proposed framework follows
similar VA system design philosophy to facilitate cloud-climate relationship
analysis.

##  2 Climate Science Background

Marine Cloud Brightening (MCB): Climate intervention techniques essentially
aim at reducing climate change impacts by manipulating Earth’s environment
[26, 19, 38]. MCB is a notable type of Solar Radiation Management (SRM) [31,
29] strategy, a category of proposed climate intervention techniques to
reflect sunlight back into space to cool the Earth. MCB involves spraying of
sea-salt aerosols into lower atmosphere marine clouds to increase their albedo
(cloud reflectivity), thereby cooling surrounding regions [36, 16, 17]. Such
intervention techniques require extensive research and in-depth understanding
of their intended as well as unintended impacts on the Earth system. The
scientific community is particularly interested in understanding the impact of
such interventions on different climate tipping points identified for several
sensitive geo-spatial sites around the world [37]. Tipping points are critical
thresholds in our climate system, that when crossed can lead to large and
often irreversible changes with severe impacts on human society [28]. For
e.g., West-Antarctic ice sheet collapse, West African monsoon shifts, and
coral reef collapses.

The effects of MCB are localized, making the possibility space of MCB
intervention scenarios vast, both in terms of forcing strength, spatial
patterns and temporal duration of interventions. This makes MCB an ideal
candidate to perform a wide range of “what-if” analyses to develop a thorough
understanding of both the positive and negative impacts of different
scenarios. However, the large number of possible scenarios also make it a
computationally prohibitive task to design such an exploration tool because it
would require running computationally expensive ESM simulations multiple
times. ESMs are comprehensive dynamical models of the coupled atmospheric-
ocean-land-ice system [7, 8] and can take thousands of core-hours to run few
instances, severely constraining thorough MCB impact assessment. Past
evaluations of MCB have only considered a small number of controlled
intervention scenarios [30, 13, 35, 14].

Fluctuation-Dissipation Theorem (FDT): A challenge towards modeling such a
climate response emulator is that there are only a handful of MCB forcing ESM
simulations that have been conducted, meaning we cannot train on any existing
large repository of ESM responses to MCB forcing data. Thus, we formulate our
modeling approach based on the fundamental principles of FDT [15, 20], a
theorem in statistical mechanics that posits that the response of a dynamical
system to a perturbation can be inferred from the time-lagged correlation
statistics of natural internal fluctuations in the system. Since climate
system is hypothesized to be such a dynamical system, FDT has been used to
estimate the linear response of climate to solar radiation perturbation [3]
and regional ocean heat convergence [22]. As such, we perform a domain-aware
data pre-processing to extract natural internal fluctuations from ESM data and
train a suite of AI models at different time-lagged duration. Applying the
concepts of FDT, we time-integrate the outcome from these models to estimate
the climate impact of the external radiative forcings, thus emulating an MCB
intervention scenario.

##  3 Requirement Analysis

Throughout the course of this project, spanning over a year, we had regular
weekly interactions with our team of climate scientists to understand their
analysis needs and modeling requirements. These also helped us formulate the
problem to best capture the physics of a cloud-climate response function. Here
are some of the main requirements set-forth for our visual analysis framework:

  * R1

Ability to perform MCB intervention scenarios directly for the VA system and
record/save the settings for interesting scenarios to execute downstream
experiments and simulations.

  * R2

Ability to monitor any out-of-distribution instances when running the hybrid
models with new perturbed input forcing fields.

  * R3

Ability to assess the potential impact of an intervention on the risk of
climate tipping points for selected geo-spatial sites.

  * R4

Ability to execute the individual time-lagged AI models separately (outside
the intervention scenarios) and analyze the cloud to climate response
properties, including the various inter-variable relationships.

##  4 Hybrid AI Model

Here we briefly elaborate on the key hybrid modeling steps and some of their
underlying rationale. Additional details are provided as supplementary
material.

Data: Our training data consists ESM data from a state-of-the-art ESM, the
Community Earth System Model 2 [4]. As FDT requires a large amount of climate
noise (i.e. internal variability) for training, we use the CESM2 Large
Ensemble (CESM2-LE) data [32]. Specifically, we use the 50-member ensemble of
historical simulations with smoothed biomass burning emission at nominal 1
degree spatial resolution and at a monthly temporal resolution, accounting for
nearly 100,000 months of data.

Preprocessing: To capture the internal fluctuations of the system, we perform
specific physics-aware preprocessing steps. We de-seasonalize the data to
remove any trends due to annual seasonality by substracting climatologoical
seasonal cycles from monthy data. We de-trend by fitting a third degree
polynomial for each month of the year and subtract it from the data to remove
secular trend in data over time. Finally we remove rolling average by
calculating anomaly at each grid point relative to a running mean that is
computed over a 30-year window for that grid point and month. The resulting
data fields are generally referred to as climate anomalies in the climate
science community. Our model takes as input 6 different radiation anomalies
which include both shortwave and longwave cloud radiative effects at both top
of the atmosphere (TOA) as well as surface-level along with clear-sky
radiation fields. The model predicts 3 different climate anomalies (sea-level
pressure, precipitation, and temperature) as output.

Spherical Grid Sampling: ESM data is typically stored on a regular 2-D
latitude-longitude grid format, which have non-uniform area coverage over the
globe. To incorporate the inherent rotational symmetry of the Earth we utilize
a geodesy-aware spherical sampling approach [5] to convert the 2-D regular
grid to a spherical 1-D icosahedral mesh. The vertices of the icosahedron are
equally spaced points over the sphere that circumscribes it. This sampling
technique also naturally provides a hierarchical multi-resolution structure,
which lets us filter the spatial data at different resolution levels. We
utilize this feature to offer a scalable and interactive visual analysis
experience for the end-user. We trained our models at the highest resolution
level (L5: 10242 grid points with $\sim$100km physical resolution) but during
interactive visual analysis phase, depending on the computational resource of
the analysis machine, users can select any desired low-resolution levels for
exploration.

Multilayer Perceptron (MLP): The architecture of our main machine learning
model is an MLP network which takes in the full icosahedral (L5) grid points
and considers the input and output anomalies as respective channels. Our MLP
architecture has four hidden layers, each containing 1024 neurons. Layer
normalization [2] and Gaussian Error Linear Units (Gelu) activation [11] were
employed in each layer. The learning rate was initially set to 2x$10^{-4}$ and
exponentially decayed at a rate of 1x$10^{-6}$ per epoch. The model have 108M
trainable parameters, and was trained for 15 epochs on the historical CESM2-LE
dataset. For training we used a single NVIDIA Tesla V100-SXM2 GPU using 16GB
VRAM. We trained multiple such models for different monthly time-lagged
duration. As explained in Section 2, following the modeling principles of FDT,
we can use these time-lagged model outputs to generate a time-aggregrated
result which emulates the forcing effect of an MCB intervention for long time
period.

Physical Constraints: To accelerate training and prevent the model from
learning physically irrelevant statistical relationships, we impose a set of
four soft physics-informed constraints: non-negative precipitation,
atmospheric moisture budget, atmospheric mass budget, atmospheric energy
budget. These constraints are implemented as additional weighted terms along
with the original mean-squared error (MSE) loss function, thus, penalizing
large violations of natural conservation laws during model training.

##  5 Visual Analysis System

Figure 2: (a) Example of multivariate relationship exploration for different
spatial regions using interactive PCP to filter lat-lon ranges. (b) Freeform
selection of arbitrary spatial sites to perform MCB intervention experiments
using our system. (c) A concise view of the results of MCB intervention over
SEP region for 10 year duration leading to strong cooling effect in the
Pacific.

Our frontend VA system adopts a multi-panel inter-linked layout design as
shown in Figure 1. Here we elaborate on the different interactions and
visualization features incorporated in our system to meet the analysis
requirements set-forth by the climate scientists.

Interaction Panel: All the major user interaction settings and parameters
needed to conduct detailed analysis are put together in an interaction panel
on the left of the system. Figure 1(C1, C2, C3) highlight the general analysis
settings, backend hybrid model settings, and MCB intervention experiment
settings respectively. “General Settings” in C1 lets the users select the
desired timestep from the loaded dataset, specific projection scheme for
visualizations (we include 22 popular geospatial scheme options), the
resolution level to use for visualization (5 levels of resolution), and the
colormaps to use for the input and output variable visualizations (i.e, in V1
and V2 respectively). “Model Settings” in C2 lets the user select which time-
lagged model to use in the backend, directly run the AI emulator model and
also clear any previous prediction data from memory. C1 and C2, therefore
addresses most of the R4 requirements. To start performing MCB intervention
experiments and conduct different “what-if” analyses users have to first
activate the C3 panel. This lets us select the time duration for intervention,
which spatial site to perform intervention over, which input cloud radiation
anomaly fields to perturb (can select multiple fields), and corresponding
sliders to set the extent of perturbation. For MCB site selection, currently
we offer the option to select from 3 specific sites (Southeast Pacific,
Southeast Atlantic, and Northeast Pacific) that the scientists are interested
in and have conducted past studies over. There is also a “freeform” spatial
site selection option which uses interactive lasso-based tools to select any
arbitrary spatial region. Therefore, C3 supports most of the requirements set-
forth in requirement R1.

Cloud Radiation and Climate Field Visualization: The 6 input cloud radiation
anomaly fields and the 3 output climate anomaly fields are visualized in the
“Input view” (Fig 1 V1) and “Output view” (Fig 1 V2) panels respectively.
Users can select the specific variables to visualize from the dropdown box in
both the panels. V1 has an additional toggle switch to view the corresponding
perturbed radiation anomaly fields, useful during MCB intervention
experiments. V2 has four additional options to select which set of output
fields to visualize. “Current” refers to the model output for the currently
selected timestep (set in C1), “Before MCB” and “After MCB” show the time
aggregated outputs without intervention and with intervention respectively.
“Diff” refers to the difference in the outputs from “Before MCB” and “After
MCB”, which is useful to track the overall impact of the intervention
experiment. Both V1 and V2 has additional “Multivariate View” tabs to help
study the multivariate relationships among the respective anomaly fields.
Parallel coordinate plots (PCP) are provided to interactively explore the
multivariate relationships across the spatial domain. The latitude and
longitude information is also plotted along two separate axes to let the users
interactively filter out different spatial regions, as illustrated in Figure
2a. Moreover, there are additional lasso-based interaction features
incorporated in V1 to select arbitrary MCB sites for intervention as shown in
Figure 2b. To activate this feature users have to select the “freeform” option
in “Select Zone” dropdown in panel C3. Therefore, most of the requirements
associated with both R1 and R4 are addressed by the V1 and V2 panels.

Distribution-shift Visualization: Performing MCB intervention experiments
involve providing a new perturbed input field to the hybrid AI models to make
new predictions. Despite the fact that we have incorporated the physics behind
cloud-climate response function during overall emulator modeling, there can be
a possibility that the model may not behave well when extreme out-of-
distribution radiation fields are provided as input. To track such unusual
extrapolated MCB scenarios we provide distribution-shift visualization as
highlighted in Figure 1(V3). For every MCB intervention experiment, we plot
the perturbed input anomaly fields (marked in red dots) against the
distribution of the original radiation fields (blue density plot) from the
training dataset. To generate this distribution-shift visualization, we
project the spatial fields to lower dimensions using Principle Component
Analysis (PCA). The density plots are created using kernel density estimate
(KDE) on the PCA space, thus capture the high-dimensional distribution shape.
V3, therefore directly addresses the requirement in R2.

Tipping Point Risk View: Given the complex climate teleconnection within the
Earth system, it is important that for any climate intervention scenario we
assess their potential impact on sensitive geospatial sites across the globe
[28]. We look at climate anomaly metrics that are indicators of potential
changes to climate tipping point risk for 7 sites for every MCB experiment
performed in our system via the panels V4 and V5 as shown in Figure 1. V4
shows the world map marking those 7 sites. The size of the dots represent the
average spatial radius of the region. Each site has different metrics for
tipping point risk assessment based on the 3 climate variables that our model
predicts. If because of the MCB experiment, some of these conditions are met,
we mark that site in red indicating a risk of reaching climate tipping point
because of the performed MCB intervention, else color green is used to
indicate no such risk for the site. We display the full condition for each
site as a pop-up on mouse-hover. The detail percentage increase/decrease of
the 3 climate anomaly fields for the sites are shown on the right as bar-
charts in V5. This panel addresses the specific requirement in R3.

MCB Intervention Records: The goal of this framework is to create a scientific
analysis workflow to study cloud-climate response and hypothesize different
MCB intervention scenarios (R1). As such, we allow the users to progressively
store interesting MCB intervention settings like MCB site location,
perturbation parameters and duration in a tabular form (“MCB Record” tab in
Figure 1 V4). Users can include additional comments/observations as notes for
each entry, which can be later be exported as excel/csv files for further
downstream experiments and expensive simulation execution.

##  6 Discussion

Validation and Case Studies: To validate that we can plausibly project the
climate response to MCB-like perturbations, we compared responses from a novel
set of fully coupled CESM2 simulations [12] to those of time-integrated
responses of our hybrid model for radiative flux anomalies computed from fixed
sea-surface temperature MCB simulations. Three previously studied spatial
regions were included in these validation case studies: Northeast Pacific
(NEP), Southeast Pacific (SEP), and Southeast Atlantic (SEA). Scientists found
that our hybrid AI emulator was able to reproduce the patterns of climate
response to MCB with strong correlation scores for all the three anomalies.
They were also able to successfully observe key remote teleconnected responses
to the MCB forcings, specifically La Niña-like temperature signals, with
strong cooling in the tropical Pacific and warming in the midlatitudes east of
Asia and Australia, as well as cooling over low-latitude land regions. Figure
2c shows a concise view of one such study over the SEP regions for a 10 year
duration using the VA interface, showing a strong cooling effect around the
surrounding region. We provide a supplementary video to demonstrate such MCB
intervention cases in detail and highlight all the interaction features. The
system also highlighted key precipitation changes like drying in northeast
Brazil, central Africa, and southern North America, and wetting in the Sahel,
southeast Asia, Australia and central America. Our scientists plan to report
such interesting observations, indicating the ability of hybrid AI models to
correctly project climate responses to MCB forcing in a climate science
journal in near future.

Expert Feedback: Our team of scientists comprises of two climate science
professors and two postdoctoral researchers. They feel that the final VA
system meets all the requirements to conduct interactive MCB-type climate
intervention scenario testing. They found the user interface to be intuitive
enough to replicate intervention studies. Outside this group of experts, we
presented our work to several climate stakeholders and scientific program
officers. They strongly feel that such an interactive VA system can not only
assist core climate science studies of future intervention techniques but can
also serve as a tool for public and government engagement in several climate
change initiatives. Concrete examples were drawn on how it can be used by
policy makers to make climate impact-aware geo-political decisions.

Implementation: The VA system was developed using Plotly’s Dash platform,
which utilizes Flask as the backend engine to run the trained hybrid models,
React.js for user-input controls and Plotly.js for rendering visualization.
Additional details and in-depth explanation for different stages of our work
can be found in our parent project’s open-source documentation page [1].

##  7 Conclusion and Future Work

In this paper, we have proposed a hybrid AI-assisted visual analysis framework
to study the complex cloud-climate response function and facilitate rapid
exploration of a wide range of MCB intervention scenarios along with their
potential climate change impacts. To the best of our knowledge, this is the
first interactive VA system to study any form of climate intervention
technique, an active area of climate science research. In future, we would
like to update our backend models to get better predictions for high-latitude
regions. This will help evaluate key crysopheric tipping points such as
Eurasian and North American permafrost loss. We would also like to incorporate
predictive uncertainty quantification to our hybrid model outcomes.

###### Acknowledgements.

This works is part of the AIBEDO project, funded under the DARPA AI-assisted
Climate Tipping-point Modeling (ACTM) program under award DARPA-PA-21-04-02.

## References

  * [1] AiBEDO.  https://aibedo.readthedocs.io/en/latest/.
  * [2] J. L. Ba, J. R. Kiros, and G. E. Hinton.  Layer normalization.  arXiv preprint arXiv:1607.06450, 2016.
  * [3] I. Cionni, G. Visconti, and F. Sassi.  Fluctuation dissipation theorem in a general circulation model: FDT IN A GENERAL CIRCULATION MODEL.  Geophysical Research Letters, 31(9):n/a–n/a, May 2004.
  * [4] G. Danabasoglu.  The Community Earth System Model Version 2 (CESM2).  Journal of Advances in Modeling Earth Systems, 12(2), Feb. 2020\.
  * [5] M. Defferrard, M. Milani, F. Gusset, and N. Perraudin.  Deepsphere: a graph-based spherical cnn.  arXiv preprint arXiv:2012.15000, 2020.
  * [6] A. Endert, W. Ribarsky, C. Turkay, B. W. Wong, I. Nabney, I. D. Blanco, and F. Rossi.  The state of the art in integrating machine learning into visual analytics.  Computer Graphics Forum, 36(8):458–486, 2017.
  * [7] G. M. Flato.  Earth system models: an overview.  Wiley Interdisciplinary Reviews: Climate Change, 2(6):783–800, 2011\.
  * [8] F. Giorgi and X.-J. GAO.  Regional earth system modeling: review and future directions.  Atmospheric and Oceanic Science Letters, 11(2):189–197, 2018.
  * [9] J. R. Goodall, E. D. Ragan, C. A. Steed, J. W. Reed, G. D. Richardson, K. M. Huffer, R. A. Bridges, and J. A. Laska.  Situ: Identifying and explaining suspicious behavior in networks.  IEEE Transactions on Visualization and Computer Graphics, 25(1):204–214, 2019.
  * [10] S. Hazarika, H. Li, K.-C. Wang, H.-W. Shen, and C.-S. Chou.  Nnva: Neural network assisted visual analysis of yeast cell polarization simulation.  IEEE Transactions on Visualization and Computer Graphics, 26(1):34–44, 2020.
  * [11] D. Hendrycks and K. Gimpel.  Gaussian error linear units (gelus).  arXiv preprint arXiv:1606.08415, 2016.
  * [12] H. Hirasawa, D. Hingmire, H. Singh, P. J. Rasch, S. Kim, S. Hazarika, P. Mitra, and K. Ramea.  Impact of Regional Marine Cloud Brightening Interventions on Climate Tipping Points.  In Preparation, 2022.
  * [13] A. Jones, J. Haywood, and O. Boucher.  Climate impacts of geoengineering marine stratocumulus clouds.  Journal of Geophysical Research: Atmospheres, 114(D10):2008JD011450, May 2009.
  * [14] B. Kravitz, A. Robock, S. Tilmes, O. Boucher, J. M. English, P. J. Irvine, A. Jones, M. G. Lawrence, M. MacCracken, H. Muri, et al.  The geoengineering model intercomparison project phase 6 (geomip6): Simulation design and preliminary results.  Geoscientific Model Development, 8(10):3379–3392, 2015.
  * [15] R. Kubo.  The fluctuation-dissipation theorem.  Reports on Progress in Physics, 29(1):255, jan 1966.
  * [16] J. Latham, K. Bower, T. Choularton, H. Coe, P. Connolly, G. Cooper, T. Craft, J. Foster, A. Gadian, L. Galbraith, et al.  Marine cloud brightening.  Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 370(1974):4217–4262, 2012.
  * [17] J. Latham, B. Parkes, A. Gadian, and S. Salter.  Weakening of hurricanes via marine cloud brightening (mcb).  Atmospheric Science Letters, 13(4):231–237, 2012.
  * [18] H. Lee, J. Kihm, J. Choo, J. Stasko, and H. Park.  ivisclustering: An interactive visual document clustering via topic modeling.  In Computer graphics forum, volume 31, pages 1155–1164. Wiley Online Library, 2012.
  * [19] J. Lee, Y.-h. Huang, J. H. Cheung, Z. Chen, and W. S. Shaw.  A systematic review of the safety climate intervention literature: Past trends and future directions.  Journal of occupational health psychology, 24(1):66, 2019.
  * [20] C. E. Leith.  Climate response and fluctuation dissipation.  Journal of Atmospheric Sciences, 32(10):2022–2026, 1975.
  * [21] Y. Liang, X. Wang, S.-H. Zhang, S.-M. Hu, and S. Liu.  Photorecomposer: Interactive photo recomposition by cropping.  IEEE Transactions on Visualization and Computer Graphics, 24(10):2728–2742, 2018.
  * [22] F. Liu, J. Lu, O. Garuba, L. R. Leung, Y. Luo, and X. Wan.  Sensitivity of Surface Temperature to Oceanic Forcing via q-Flux Green’s Function Experiments. Part I: Linear Response Function.  Journal of Climate, 31(9):3625–3641, May 2018.
  * [23] S. Liu, C. Chen, Y. Lu, F. Ouyang, and B. Wang.  An interactive method to improve crowdsourced annotations.  IEEE Transactions on Visualization and Computer Graphics, 25(1):235–245, 2019.
  * [24] S. Liu, M. X. Zhou, S. Pan, Y. Song, W. Qian, W. Cai, and X. Lian.  Tiara: Interactive, topic-based visual text summarization and analysis.  ACM Transactions on Intelligent Systems and Technology (TIST), 3(2):25, 2012.
  * [25] Z. Lu, M. Fan, Y. Wang, J. Zhao, M. Annett, and D. Wigdor.  Inkplanner: Supporting prewriting via intelligent visual diagramming.  IEEE Transactions on Visualization and Computer Graphics, 25(1):277–287, 2019.
  * [26] A. Y. Mah, D. A. Chapman, E. M. Markowitz, and B. Lickel.  Coping with climate change: Three insights for research, intervention, and communication to promote adaptive coping to climate change.  Journal of anxiety disorders, 75:102282, 2020.
  * [27] G. E. Marai, C. Ma, A. Burks, F. Pellolio, G. M. Canahuate, D. M. Vock, A. S. Mohamed, and C. D. Fuller.  Precision risk analysis of cancer therapy with interactive nomograms and survival plots.  IEEE Transactions on Visualization and Computer Graphics, 2018.
  * [28] D. I. A. McKay, A. Staal, J. F. Abrams, R. Winkelmann, B. Sakschewski, S. Loriani, I. Fetzer, S. E. Cornell, J. Rockström, and T. M. Lenton.  Exceeding 1.5°C global warming could trigger multiple climate tipping points.  Science, 377:6611, 2022.
  * [29] S. Nicholson, S. Jinnah, and A. Gillespie.  Solar radiation management: a proposal for immediate polycentric governance.  Climate Policy, 18(3):322–334, 2018.
  * [30] P. J. Rasch, J. Latham, and C.-C. J. Chen.  Geoengineering by cloud seeding: influence on sea ice and climate system.  Environmental Research Letters, 4(4):045112, Oct. 2009.
  * [31] A. Robock.  Benefits and risks of stratospheric solar radiation management for climate intervention (geoengineering).  Bridge, 50(1):59–67, 2020.
  * [32] K. B. Rodgers, S.-S. Lee, N. Rosenbloom, A. Timmermann, G. Danabasoglu, C. Deser, J. Edwards, J.-E. Kim, I. R. Simpson, K. Stein, M. F. Stuecker, R. Yamaguchi, T. Bódai, E.-S. Chung, L. Huang, W. M. Kim, J.-F. Lamarque, D. L. Lombardozzi, W. R. Wieder, and S. G. Yeager.  Ubiquity of human-induced changes in climate variability.  Earth System Dynamics, 12(4):1393–1411, Dec. 2021.
  * [33] N. Shi, J. Xu, S. W. Wurster, H. Guo, J. Woodring, L. P. Van Roekel, and H.-W. Shen.  Gnn-surrogate: A hierarchical and adaptive graph neural network for parameter space exploration of unstructured-mesh ocean simulations.  IEEE Transactions on Visualization and Computer Graphics, 2022.
  * [34] G. L. Stephens.  Cloud Feedbacks in the Climate System: A Critical Review.  Journal of Climate, 18(2):237–273, Jan. 2005.
  * [35] C. W. Stjern, H. Muri, L. Ahlm, O. Boucher, J. N. Cole, D. Ji, A. Jones, J. Haywood, B. Kravitz, A. Lenton, et al.  Response to marine cloud brightening in a multi-model ensemble.  Atmospheric Chemistry and Physics, 18(2):621–634, 2018.
  * [36] C. W. Stjern, H. Muri, L. Ahlm, O. Boucher, J. N. S. Cole, D. Ji, A. Jones, J. Haywood, B. Kravitz, A. Lenton, J. C. Moore, U. Niemeier, S. J. Phipps, H. Schmidt, S. Watanabe, and J. E. Kristjánsson.  Response to marine cloud brightening in a multi-model ensemble.  Atmospheric Chemistry and Physics, 18(2):621–634, Jan. 2018.
  * [37] United Nations Environment Programme.  One Atmosphere: An Independent Expert Review on Solar Radiation Modification Research and Deployment.  Feb. 2023.
  * [38] N. C. Valentini and M. E. Rudisill.  Goal orientation and mastery climate: a review of contemporary research and insights to intervention.  Estudos de Psicologia (Campinas), 23:159–171, 2006.
  * [39] C. Xie, W. Xu, and K. Mueller.  A visual analytics framework for the detection of anomalous call stack trees in high performance computing applications.  IEEE Transactions on Visualization and Computer Graphics, 25(1):215–224, 2019.
  * [40] X. Xie, X. Cai, J. Zhou, N. Cao, and Y. Wu.  A semantic-based method for visualizing large image collections.  IEEE Transactions on Visualization and Computer Graphics, 2018.
  * [41] Z. Zhou, L. Meng, C. Tang, Y. Zhao, Z. Guo, M. Hu, and W. Chen.  Visual abstraction of large scale geospatial origin-destination movement data.  IEEE Transactions on Visualization and Computer Graphics, 25(1):43–53, 2019.