# I Know About “Up”! Enhancing Spatial Reasoning in Visual Language Models
Through 3D Reconstruction

Hao Zhou
Guangdong Polytechnic Normal University
<EMAIL_ADDRESS>
&Zaiqiao Meng
University of Glasgow
<EMAIL_ADDRESS>
&Yifang Chen
Guangdong Polytechnic Normal University
<EMAIL_ADDRESS>
Corresponding Author.

###### Abstract

Visual Language Models (VLMs) are essential for various tasks, particularly
the visual reasoning tasks, due to their robust multi-modal information
integration, visual reasoning capabilities, and contextual awareness. However,
existing VLMs’ visual spatial reasoning capabilities are often inadequate,
struggling even with basic tasks such as distinguishing left from right. To
address this, we propose the ZeroVLM111Code:
https://github.com/zhouhao028/Iknow_up model, designed to enhance the visual
spatial reasoning abilities of VLMs. ZeroVLM employs Zero-1-to-3, a 3D
reconstruction model for obtaining different views of the input images and
incorporates a view prompt to further improve visual spatial reasoning.
Experimental results on four visual spatial reasoning datasets show that our
ZeroVLM achieves up to 19.48% accuracy improvement, which indicates the
effectiveness of 3D reconstruction and view prompt of our ZeroVLM.

I Know About “Up”! Enhancing Spatial Reasoning in Visual Language Models
Through 3D Reconstruction

Hao Zhou Guangdong Polytechnic Normal University<EMAIL_ADDRESS>Zaiqiao Meng††thanks: Corresponding Author. University of Glasgow
<EMAIL_ADDRESS>Yifang Chen Guangdong Polytechnic Normal University
<EMAIL_ADDRESS>

##  1 Introduction

Visual Language Models (VLMs), such as LLaVA (Liu et al., 2024), MiniGPT-4
(Zhu et al., 2023) and InternGPT (Liu et al., 2023d), are a class of deep
neural models adept at simultaneously processing and understanding both visual
and linguistic information. These VLMs often consist of an image encoder, an
embedding projector to align image and text representations and a text decoder
to process the projected image embedding and text representations, enabling
joint understanding and reasoning between these modalities (Li et al., 2022).
By using the language generation power of underlining text decoders, these
VLMs showcase remarkable interaction capabilities in various applications,
such as referring expression comprehension (Li et al., 2019b; Yu et al.,
2016), visual question answering (Goyal et al., 2017; Zellers et al., 2019;
Hudson and Manning, 2019), visual language reasoning (Liu et al., 2022a), and
entailment (Liu et al., 2021; Xie et al., 2019).

Figure 1: An example of the VQA task, where humans can easily recognize
positions under different views, but the vanilla LLaVA (Liu et al., 2024) can
only predict correctly from certain views. By performing 3D reconstruction to
obtain different views of the image, we can improve LLaVA’s predictive
accuracy.

However, many visual language tasks, such as visual question answering (VQA)
(Redmon et al., 2016; Long et al., 2015; Zhou et al., 2017; Dai et al., 2018;
Hu et al., 2019) and image segmentation (Huang et al., 2023), require the
ability to recognize spatial information from images. For instance, in visual
question answering, a system must understand the spatial relationships between
objects to correctly answer questions about an image (Dai et al., 2018; Hu et
al., 2019). Visual spatial reasoning in VLMs requires multi-modal
understanding (Khattak et al., 2023), cross-modal mapping (Wang et al., 2023),
inference of visual spatial relations (Liu et al., 2023a), and integration of
context comprehension (Zhao et al., 2023). This is because VLMs must
comprehend intricate spatial relationships within images, which are
hierarchical and multifaceted, involving various relationships among multiple
objects (such as containment, proximity, overlap, etc.) (Chen et al., 2024).
Enhancing visual spatial reasoning capabilities of models can significantly
elevate their performance in image comprehension and processing (Yang et al.,
2024).

Presently, numerous studies explore various benchmarks and methods to enhance
the visual spatial reasoning capabilities of VLMs to improve their
performance, exemplified by visual spatial reasoning (Liu et al., 2023a) and
Whatsup_vlm (Kamath et al., 2023). These endeavors employ diverse
methodologies (Su et al., 2019; Qi et al., 2020; Kiela et al., 2019) to
augment the visual spatial reasoning abilities of models. However, despite
these efforts yielding improvements in model performance to some extent,
substantial challenges persist in comprehending complex scenes and intricate
spatial relationships. Existing methodologies predominantly rely on 2D
information, impeding the comprehensive capture of 3D spatial relationships
among objects. For example, Figure 1 illustrates an example of the VQA task.
While humans could easily recognize positions under different views of the
objects, LLaVA can only predict correctly from certain views due to the lack
of image views during their pretraining.

To tackle this challenge, we propose a novel model, called ZeroVLM, to solve
the visual spatial reasoning task through the 3D reconstruction technique,
entailing the reconstruction of images in 3D and capturing them from various
views. In particular, our ZeroVLM utilize the Zero-1-to-3 (Liu et al., 2023c)
model for constructing 3D views from a single 2D image within our tested
datasets. This allows VLMs to access richer spatial information through 3D
transformation, thereby enhancing their visual spatial reasoning capabilities.
We conducted comprehensive experiments across four visual spatial reasoning
datasets, and the results show that all the tested VLMs have notably improved
spatial reasoning capabilities through our ZeroVLM.

Our contributions are summarized as follows:

  * •

We proposed the ZeroVLM model, a new model for visual spatial reasoning tasks,
which can generate images of different views based on the original image to
enhance spatial reasoning ability.

  * •

Our ZeroVLM utilises a 3D reconstruction model to obtain different views from
the input image, which enhances spatial relationships at the image level to
improve the visual spatial reasoning capabilities of VLMs.

  * •

We validate the effectiveness of our ZeroVLM through visual spatial reasoning
experiments conducted on four different datasets.

(a) Single-View

(a)

(b) Multi-View

(b)

Figure 2: Single-view images were generated using Zero-1-to-3 to produce left-
view, right-view, and random-view images. Multi-view images were created by
combining these different single-view images in various configurations.

##  2 Related Work

Vision Language Models (VLMs). VLMs amalgamate computer vision and natural
language processing technologies to comprehend and generate correlations
between images and natural language (Dosovitskiy et al., 2020). VLMs can
accept image input to generate corresponding response in text (Menon and
Vondrick, 2022). In terms of training architecture, recent works include joint
pretraining architectures (e.g. OFA Wang et al. (2022) and VLMo (Bao et al.,
2021)), which train image and text data jointly, and image-to-text mapping
architectures (e.g., PaLI (Chen et al., 2022b)), which map an image encoder to
a well-pretrained text encoder. The latter approach has gained more
popularity. VLMs exhibit cross-modal understanding capabilities, enabling them
to extract information from images and translate it into natural language, or
retrieve information from natural language and generate corresponding images
(Chen et al., 2022a). This wide range of capabilities broadens the potential
of VLMs (e.g., GlaMM (Rasheed et al., 2023) and MiniGPT-4 (Zhu et al., 2023))
in understanding and generating associations between images and language, such
as Image Captioning (Zhang et al., 2021), Visual Question Answering (Yang et
al., 2022), and Multi-modal Translation (Li et al., 2019a). In this study, we
aim to explore the visual-spatial recognition capabilities of these VLMs.

Visual Spatial Reasoning. Visual spatial reasoning refers to the cognitive
ability to understand and manipulate the spatial relationships of objects or
elements in a given environment (Liu et al., 2023a). This reasoning ability
involves not only recognizing the properties of individual entities, but also
understanding the complex relationships and structures between them. Recent
works have aimed to benchmark the problem of visual spatial reasoning, notably
VSR (Liu et al., 2023a) and Whatsup_vlm (Kamath et al., 2023). VSR (Liu et
al., 2023a) evaluates the ability of VLMs using text-image pairs to describe
various visual spatial relationships, while Whatsup_vlm (Kamath et al., 2023)
assesses the visual spatial reasoning of VLMs through specific prepositions
and perspectives. Although these methods provide valuable insights into the
visual spatial reasoning capabilities of VLMs, they remain somewhat limited.
In our work, we employ 3D reconstruction to more comprehensively test the
visual spatial reasoning ability of VLMs.

3D Reconstruction. Visual spatial reasoning encompasses the cognitive ability
to comprehend and manipulate visual spatial information, while 3D
reconstruction involves generating a three-dimensional representation of an
object or scene from two-dimensional images or other sensor data. For
instance, MonoScene (Cao and De Charette, 2022) infers dense geometric
structures and semantic information of a scene from a single monocular RGB
image, offering an efficient and innovative approach to complete 3D semantic
scenes. VoxFormer (Li et al., 2023) proposes a transformer-based (Vaswani et
al., 2017) semantic scene completion framework, addressing camera-based 3D
semantic scene completion by introducing sparse voxel queries (Guchait et al.,
2021) and masked autoencoder design (He et al., 2022). Zero-1-to-3 (Liu et
al., 2023c) is a 3D reconstruction model that excels in zero-shot
generalization on out-of-distribution datasets and unprocessed images by using
a single image as input without requiring additional 3D or depth information.
We chose this model for its simplicity and effectiveness compared to MonoScene
(Cao and De Charette, 2022) and VoxFormer (Li et al., 2023).

Figure 3: An overview of our proposed ZeroVLM model. Our ZeroVLM first uses
Zero-1-to-3 to perform 3D reconstruction to obtain different views of the
input image, and then it stitches the original images with these different
views to obtain the stitched image, which is the input of a VLM for answer
prediction.

##  3 Preliminaries

In this section, we will formally define the task of visual spatial reasoning
and introduce the Zero-1-to-3 model (Liu et al., 2023c), which is one of the
components utilized by our model.

###  3.1 Visual Spatial Reasoning Task

Visual spatial reasoning is critically important in the fields of computer
vision, artificial intelligence, and machine learning (Zakari et al., 2022;
Rajabi and Kosecka, 2023; Liu et al., 2022b). The visual spatial reasoning
task involves inferring and understanding objects, scenes, or spatial
relationships, addressing location, orientation, and spatial arrangement for
accurate object localization and recognition. This task is normally formulated
as a visual question answering task (Liu et al., 2021; Kamath et al., 2023) by
asking and answering questions about objects on images and through context in
text. Formally, we denote an image and its associated question as i and $q$,
respectively. The questions in this task typically ask to identify the
relationships among multiple objects or to locate specified objects within the
image. Figure 1 shows an example of this task. The goal of this task is to
answer this question by accurately comprehending the visual spatial relations
and discerning the visual spatial position of target objects relative to
others.

###  3.2 Zero-1-to-3

Zero-1-to-3 (Liu et al., 2023c) is a model designed for visual reconstruction
tasks. It works by identifying objects in input image i and adjusting the
camera perspective of these objects. Given a image $i\in\mathbb{R}^{H\times
W\times 3}$ and a relative camera rotation matrix $R\in\mathbb{R}^{3\times 3}$
and a translation vector $T\in\mathbb{R}^{3}$, Zero-1-to-3 leans a model $f$
such that its output is a new perspective image under the specified camera
transformation:

$\hat{i}_{R,T}=f(i,R,T)$ (1)

where $\hat{i}_{R,T}$ represents the generated new perspective image. When we
input original image i and hope to generate a 3D reconstructed view from the
left at an angle of 45° through Zero-1-to-3, we can achieve this effect by
setting $R$ to the rotation matrix of 45° around the Y axis and setting $T$ to
the vector translated to the left by a certain distance. In order to
synthesize new views under such partial constraints, Zero-1-to-3 uses a large-
scale diffusion model to integrate the geometric prior knowledge of the input
image. The conditional diffusion model embedded in Zero-1-to-3 (Yang et al.,
2023; Podell et al., 2023) learns to control the relative camera viewpoint
using synthetic datasets, which helps generate new images with specified
camera transformations, such as fixed camera viewpoints (such as left or right
view) or randomly generated camera viewpoints. Despite being trained on
synthetic datasets, Zero-1-to-3 still demonstrates strong zero-shot
generalization capabilities to out-of-distribution datasets and real-world
images. Figure 2 shows the different views generated by Zero-1-to-3 based on
the input image.

##  4 Methodologies

###  4.1 Overview of ZeroVLM

To address the visual spatial reasoning task, we propose ZeroVLM, a model that
leverages Zero-1-to-3 to infer various views of input images and employs a VLM
to generate answers using the combined multiview images and our specially
designed view prompts. The overview of our model is illustrated in Figure 3.
In particular, ZeroVLM is a novel visual-language model that combines large
language models (such as LLaMA (Touvron et al., 2023)) with high-level vision
models, aiming to enhance visual-spatial reasoning capabilities by leveraging
3D reconstruction techniques. Given an image i, ZeroVLM uses Zero-1-to-3 to
infer its various views $\hat{i}_{R,T}$ and performs image stitching from
multiple views, thereby providing richer spatial information. The 3D
reconstruction feature provided by Zero-1-to-3 can enable ZeroVLM to better
understand and infer spatial relationships by viewing objects from different
angles. At the same time, a specialized view prompt is designed into ZeroVLM
to further enhance its visual spatial reasoning capabilities. This prompt
helps guide the model to focus on relevant spatial views and improve its
interpretation accuracy. The architecture of our ZeroVLM is shown in Figure 3.

###  4.2 Data Augmentation by 3D Reconstruction

In our work, we use Zero-1-to-3 for 3D reconstruction on the dataset to
generate $\hat{i}_{R,T}$ from different viewpoints, e.g. the left, right and
random views. Our investigation not only explores whether the creation of
single-view images can enhance the visual spatial reasoning capabilities of
VLMs but also examines whether multi-view images can help this improvement,
where multi-view images are synthesized from different $\hat{i}_{R,T}$.
Because multi-view images can provide richer spatial information, the model
can get the opportunity to observe the same scene from different angles,
thereby capturing a more comprehensive spatial layout and the relationship
between objects, and different views can provide redundant information, so
that the model can still make accurate judgments when facing noise or partial
information loss. Therefore, in our work, after inferring different views of
the input images, we further construct multi-view images from this single
views by stitching them, and test their effectiveness against the single view
images. Figure 2 shows an example of the multi-view image constructed by
stitching multiple single view images generated through Zero-1-to-3.

By synthesizing these various multi-view images, we aim to determine their
effectiveness in improving the spatial reasoning abilities of VLMs.

###  4.3 View Prompt

In our work, we first test whether 3D reconstruction of images can improve the
accuracy of VLM’s visual spatial ability at the image level. To further
explore whether the context prompt can enhance VLM’s visual spatial reasoning
ability, we introduced a special prompt called view prompt in the experiment.
This view prompt varies depending on the input image and its views. Figure 4
shows two view prompt examples over a single view and multiple views of the
input.

We designed a variety of view prompts based on the content of different view
images to guide VLM to better understand and reason about the view spatial
relationship between the target object and other objects in the image. We
first used Zero-1-to-3 (Liu et al., 2023c) to perform 3D reconstruction of VSR
(Liu et al., 2023a) dataset and What’sUp (Kamath et al., 2023) dataset from
different viewpoints. Then during the inference of VLM, we use a prompt
consisting of the question, the view prompt, and the stitched view image, to
generate the answer. Figure 5 provides a detailed description of the process.

Figure 4: These view prompts are manually constructed by us. View prompt
comparison between single-view images and multi-view images. {question} is the
corresponding question in the prompt. Figure 5: For both single-view and
multi-view datasets, we employed a view prompt. The aim was to explore whether
perspective language models could enhance their visual spatial reasoning
abilities through improvements at the textual level.

##  5 Experiments

###  5.1 Experimental Setup

Datasets. We selected two datasets for investigation: the Visual Spatial
Reasoning (VSR) (Liu et al., 2023a) dataset and the What’sUp (Kamath et al.,
2023) dataset. The statistics of the two datasets are summarized in Table 1.
The VSR dataset focuses on a wide range of spatial relations and linguistic
phenomena, emphasizing various visual spatial relations and describing these
relations. The What’sUp dataset focuses on household items and uses different
prepositions to describe spatial relations. Both datasets cover a variety of
visual spatial relations (such as “below”, “in front of”, “beside”, etc.).

DataSet | Train | Development | Test | Total
---|---|---|---|---
VSR (Random Split) | 7680 | 1097 | 2195 | 10972
VSR (Zero shot) | 4713 | 231 | 616 | 5560
What’sUp (SubSet-A) | 200 | 110 | 111 | 421
What’sUp (SubSet-B) | 200 | 108 | 100 | 408

Table 1: The number of images in various datasets.

Baselines. In our research, we employ LLaVA and MiniGPT-4 as the primary VLMs.
LLaVA utilizes multi-modal language-image instruction data for instruction
adjustment, employing CLIP-ViT-L-336px (Liu et al., 2023b) as the visual
encoder and MLP projection as the visual language cross-modal connector,
achieving comprehensive understanding of both visual and language inputs. The
architecture of MiniGPT-4 involves aligning a frozen visual encoder with a
frozen high-level language model, e.g. Vicuna (Chiang et al., 2023), through
projection layers, ensuring correct alignment of visual features with the
high-level large language model. MiniGPT-4 can exhibit advanced multi-modal
capabilities similar to those of GPT-4 (Achiam et al., 2023).

Backbones. We use LLaVA and MiniGPT-4 as the backbone VLMs of our ZeroVLM in
all the experiments. In particular, we denote ZeroVLM (L) as our model based
on the LLaVA (Liu et al., 2024) model, which can process image inputs and
improve efficiency and performance through joint learning of image and text
instructions, while ZeroVLM (M) as our model based on the MiniGPT-4 (Zhu et
al., 2023)) model, which combines the powerful generation ability of language
models with visual information capabilities.

Task Setting. We classify visual spatial relationship questions for VLMs into
two types using our dataset split, which is applied consistently across all
four datasets. The first type disrupts the visual spatial relationship between
the target object and another object (for example, the correct image
description is “The apple is to the left of the banana, ” while our
description is “The apple is above (below, to the right of, in front of, etc.)
the banana”). The second type describes visual spatial relationships involving
objects not present in the image but related to the target object (for
example, the correct image description is “The apple is to the left of the
banana, ” but our description is “The watermelon (an object that does not
exist besides the apple) is above the banana”). These classifications are
employed to evaluate the visual spatial reasoning abilities of VLMs and
highlight the differences between our dataset and the original datasets.

Evaluation Metric. For evaluation, we judge the accuracy of the visual spatial
reasoning ability of ZeroVLM (L) and ZeroVLM (M) based on the answers answered
by ZeroVLM (L) and ZeroVLM (M). The accuracy of ZeroVLM (L) and ZeroVLM (M)
responses determines their accuracy in identifying these relations. Since both
datasets involve binary classification tasks, we use accuracy as the
evaluation metric.

Implementation Details. We obtain new datasets through 3D reconstruction,
consisting of single-view and multi-view datasets. In the single-view dataset,
“Origin” refers to the original, unprocessed data, “Left” refers to the data
processed by 3D reconstruction from the left viewpoint, and “Right” refers to
the data processed by 3D reconstruction from the right viewpoint. “Random”
refers to the data processed from a random viewpoint. In the multi-view
dataset, there are two types of “Multi-view”, one is multi-view without
original images, and the other is multi-view with original images. Multi-view
without original images is composed of single-view images, while multi-view
with original images is composed of original images plus different single-view
images. Our goal is to explore whether the visual spatial reasoning ability of
VLMs increases or decreases with the addition or omission of original images.

Model | VSR (Random Split) | VSR (Zero Shot) | What’sUp (A) | What’sUp (B)
---|---|---|---|---
Human | 95.4 | 95.4 | 100 | 100
CLIP (frozen) | 56.0 | 54.50 | - | -
CLIP (FT) | 65.10 | - | - | -
VisualBERT | 55.20 | 51.00 | - | -
ViLT | 69.30 | 63.00 | - | -
LXMERT | 70.10 | 61.20 | - | -
ZeroVLM (M) | 53.80 | 52.43 | 55.76 | 53.84
ZeroVLM (L) | 70.29 | 70.94 | 71.74 | 80.76

Table 2: Accuracy performance of the compared models. Missing values indicate
that ZeroVLM (L) and ZeroVLM (M) were not tested on the visual spatial
reasoning on the VSR and What’sUp datasets. Bests are in bold, and second
bests are underlined.

L-VR-VRa-VM-V$40$$45$$50$$55$$60$$65$$70$$75$$80$$85$$90$$95$$100$Accuracy
(%)(a) VSR (Random-Split)ZeroVLM (L)ZeroVLM (M)

L-VR-VRa-VM-V$40$$45$$50$$55$$60$$65$$70$$75$$80$$85$$90$$95$$100$Accuracy
(%)(b) VSR (Zero-Shot)ZeroVLM (L)ZeroVLM (M)

L-VR-VRa-VM-V$40$$45$$50$$55$$60$$65$$70$$75$$80$$85$$90$$95$$100$Accuracy
(%)(c) What’sUp (SubSet-A)ZeroVLM (L)ZeroVLM (M)

L-VR-VRa-VM-V$40$$45$$50$$55$$60$$65$$70$$75$$80$$85$$90$$95$$100$Accuracy
(%)(d) What’sUp (SubSet-B)ZeroVLM (L)ZeroVLM (M)

Figure 6: L-V represents the left-view images, R-V represents the right-view
images, Ra-V represents the random-view images, and M-V represents the multi-
view images (excluding the original image). Note that here we use ZeroVLM (L)
and ZeroVLM (M) for fair comparison.

###  5.2 Experimental Result

Overall performance. Our experimental aims to comprehensively evaluate the
visual spatial reasoning capabilities of VLMs ZeroVLM (L) and ZeroVLM (M) on
various datasets. We seek to understand how different datasets (including
original, single-view, and multi-view datasets) affect the performance of
these models in visual-spatial reasoning tasks. We first test ZeroVLM (L) and
ZeroVLM (M) on the original dataset (without 3D reconstruction) and compare
the results with those of VSR (Liu et al., 2023a). The result of this
comparison is reported in Table 2. Our experimental results show a baseline
comparison of ZeroVLM (L) and ZeroVLM (M) testing the original dataset with
VSR (Liu et al., 2023a), highlighting the basic performance of ZeroVLM (L) and
ZeroVLM (M).

Additionally, we tested single-view 3D reconstructions from the VSR and
What’sUp datasets, and multi-view datasets derived from these single views. We
also assessed their visual spatial reasoning accuracy on single-view and
multi-view datasets using different view prompts for ZeroVLM.

Single View vs Multiple View. Our study investigated the impact of single-view
versus multi-view 3D reconstruction on the visual spatial reasoning
capabilities of VLMs. We assessed two VLMs: ZeroVLM (L) and ZeroVLM (M), using
both single-view and multi-view datasets. The results, shown in Figure 6,
reveal that ZeroVLM excels in single-view conditions with an average accuracy
of 79.28%, but its performance drops to 58.42% in multi-view conditions.
ZeroVLM (M) performs less well overall, with average accuracy of 56.95% for
single-view and 56.23% for multi-view. Overall, ZeroVLM (L) outperforms
ZeroVLM (M) in visual spatial reasoning tasks, particularly in single-view
scenarios. While ZeroVLM (M) possesses unique advantages in handling multi-
modalities, there is still room for improvement in specific visual spatial
reasoning tasks (Battaglia et al., 2018).

View Type | View | View Prompt | VSR (Random Split) | VSR (Zero Shot) | What’sUp (A) | What’sUp (B)
---|---|---|---|---|---|---
| Origin | $\times$ | 70.29 | 70.94 | 71.14 | 80.76
| L-V | $\times$ | 81.80 | 82.35 | 82.69 | 71.15
| R-V | $\times$ | 78.60 | 79.41 | 80.76 | 78.84
Single-View | Ra-V | $\times$ | 80.60 | 83.33 | 78.84 | 73.07
| L-V | $\checkmark$ | 87.80 | 87.25 | 88.46 | 75.21
| R-V | $\checkmark$ | 86.40 | 88.23 | 90.38 | 83.71
| Ra-V | $\checkmark$ | 87.00 | 89.21 | 86.53 | 78.33
| M-V | $\times$ | 55.60 | 60.78 | 59.61 | 57.69
Multi-View | Origin + L-V | $\checkmark$ | 64.60 | 50.98 | 63.46 | 65.23
| Origin + L-V + R-V | $\checkmark$ | 67.40 | 61.76 | 69.23 | 68.38
| Origin + M-V | $\checkmark$ | 72.20 | 59.80 | 65.38 | 63.21

Table 3: Performance comparison of ZeroVLM (L) under different view
combinations. L-V represents the left-view images, R-V represents the right-
view images, Ra-V represents the random- view images, and M-V represents the
multi-view images (excl. the original image). Bests are in bold, and second
bests are underlined.

Effect on different view prompts. The primary goal of our concluding study was
to explore the potential enhancement of visual spatial reasoning abilities in
VLMs by employing different view prompts. We sought to investigate if
leveraging contextual connections through textual cues could significantly
improve the performance of VLMs in understanding and reasoning about visual
spatial relationships. To achieve this, we applied various view prompts with
ZeroVLM and assessed its visual spatial reasoning abilities across both
single-view and multi-view datasets. These view prompts were designed to
provide contextual cues that could aid the model in interpreting visual
spatial information more effectively. The results of this comprehensive
assessment are presented in Table 3. The experiment demonstrated that the use
of different prompts can indeed enhance the visual spatial reasoning
capabilities of VLMs. Overall, our study concludes that employing a view
prompt can significantly enhance the visual spatial reasoning abilities of
VLMs.

Case study. Our study aims to investigate the impact of 3D reconstruction on
the visual spatial reasoning capabilities of VLM. We assess whether 3D
reconstruction (Zero-1-to-3) enhances these capabilities from the original
dataset to the single-view dataset and compare the performance of ZeroVLM (L)
and ZeroVLM (M) using single-view images after 3D reconstruction. Comparative
analysis focuses on the visual spatial reasoning capabilities of ZeroVLM on
both datasets. Figure 7 illustrates the enhancement by ZeroVLM using the
single-view dataset. Figure 8 presents the performance comparison between
ZeroVLM (L) and ZeroVLM (M). Results show that ZeroVLM (L) has improved visual
spatial reasoning with the single-view dataset after 3D reconstruction and
outperforms ZeroVLM (M). This suggests that 3D reconstruction provides useful
spatial information aiding visual reasoning.

Figure 7: Different results were obtained by ZeroVLM when evaluating datasets
with and without 3D reconstruction. View prompts are omitted to save space.
Figure 8: Different results were obtained when inputting the 3D reconstructed
datasets into ZeroVLM (L) and ZeroVLM (M). View prompts are omitted to save
space.

##  6 Conclusion

This study presents a novel model, called ZeroVLM, for enhancing the visual
spatial reasoning capability of VLMs through 3D reconstruction. In particular,
ZeroVLM use the Zero-1-to-3 to generate different views over different angles
of the input image, and generate answers with a specially designed view
prompt. We validate the effectiveness of our VLMs by comparative experiments
from various views, including single and multiple viewpoints, and evaluated
the performance of LLaVA and MiniGPT-4 under our ZeroVLM. The experimental
results suggest that 3D reconstruction with a view prompt from a single
perspective can effectively optimize the model’s visual spatial reasoning
ability. Future research could also develop models that dynamically adjust
viewpoints based on task requirements and integrate additional modal
information, such as video and audio, to enhance multi-modal processing
capability.

##  7 Limitations

Although ZeroVLM demonstrates significant performance improvements, we still
identify the following limitations of our work: (1) Data dependency: ZeroVLM
used specific visual spatial reasoning datasets in our experiments. Although
these datasets cover a variety of scenarios, they may not fully represent all
the complex spatial relationships in the real world; (2) Diversity of
datasets: Although we used multiple datasets for testing, these datasets may
not fully cover all possible application scenarios. Therefore, the model may
need further training and adjustment to ensure its generality and robustness
when processing different types of images and tasks; (3) The generalization
ability of the model: Although the Zero-1-to-3 model performs well in zero-
shot generalization, in some extreme cases, the model may still not accurately
capture the spatial relationship. Therefore, considering other methods for the
3D reconstruction module is one of our future work directions.

##  8 Potential risks

The development and deployment of VLMs for visual spatial reasoning present
several potential risks. One significant risk is the dependency on specific
datasets for training, which may not encompass the full diversity of spatial
relationships encountered in real-world scenarios. This can lead to models
that perform well in controlled environments but fail to generalize
effectively. Additionally, the computational resources required for 3D
reconstruction and multi-view generation are substantial, posing scalability
and real-time application challenges. Another concern is the potential for
bias in the datasets, which can result in models that unfairly represent or
perform poorly on underrepresented spatial arrangements or object types.
Furthermore, privacy and security issues arise when handling large amounts of
visual data, necessitating strict adherence to data protection guidelines.
Lastly, there are ethical considerations regarding the misuse of enhanced
spatial reasoning capabilities for surveillance or other invasive
applications, underscoring the need for transparency and accountability in the
deployment of these technologies. Addressing these risks is essential to
ensure the effective and responsible use of VLMs in visual spatial reasoning.

## References

  * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_.
  * Bao et al. (2021) Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, and Furu Wei. 2021.  Vlmo: Unified vision-language pre-training with mixture-of-modality-experts.  _arXiv preprint arXiv:2111.02358_.
  * Battaglia et al. (2018) Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinícius Flores Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Çaglar Gülçehre, H. Francis Song, Andrew J. Ballard, Justin Gilmer, George E. Dahl, Ashish Vaswani, Kelsey R. Allen, Charlie Nash, Victoria Langston, Chris Dyer, Nicolas Manfred Otto Heess, Daan Wierstra, Pushmeet Kohli, Matthew M. Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. 2018.  Relational inductive biases, deep learning, and graph networks.  _ArXiv_.
  * Cao and De Charette (2022) Anh-Quan Cao and Raoul De Charette. 2022.  Monoscene: Monocular 3d semantic scene completion.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Chen et al. (2024) Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Danny Driess, Pete Florence, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024.  Spatialvlm: Endowing vision-language models with spatial reasoning capabilities.  _arXiv preprint arXiv:2401.12168_.
  * Chen et al. (2022a) Feilong Chen, Xiuyi Chen, Shuang Xu, and Bo Xu. 2022a.  Improving cross-modal understanding in visual dialog via contrastive learning.  In _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_.
  * Chen et al. (2022b) Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. 2022b.  Pali: A jointly-scaled multilingual language-image model.  _arXiv preprint arXiv:2209.06794_.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
  * Dai et al. (2018) Bo Dai, Sanja Fidler, and Dahua Lin. 2018.  A neural compositional paradigm for image captioning.  _Advances in Neural Information Processing Systems_.
  * Dosovitskiy et al. (2020) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_.
  * Goyal et al. (2017) Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017.  Making the v in vqa matter: Elevating the role of image understanding in visual question answering.  In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Guchait et al. (2021) Monoranjan Guchait, Arnab Roy, and Seema Sharma. 2021.  Probing mild-tempered neutralino dark matter through top-squark production at the lhc.  _Physical Review D_.
  * He et al. (2022) Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022.  Masked autoencoders are scalable vision learners.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_.
  * Hu et al. (2019) Ronghang Hu, Anna Rohrbach, Trevor Darrell, and Kate Saenko. 2019.  Language-conditioned graph networks for relational reasoning.  In _Proceedings of the IEEE/CVF international conference on computer vision_.
  * Huang et al. (2023) You Huang, Hao Yang, Ke Sun, Shengchuan Zhang, Liujuan Cao, Guannan Jiang, and Rongrong Ji. 2023.  Interformer: Real-time interactive image segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_.
  * Hudson and Manning (2019) Drew A Hudson and Christopher D Manning. 2019.  Gqa: A new dataset for real-world visual reasoning and compositional question answering.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_.
  * Kamath et al. (2023) Amita Kamath, Jack Hessel, and Kai-Wei Chang. 2023.  What’s" up" with vision-language models? investigating their struggle with spatial reasoning.  _arXiv preprint arXiv:2310.19785_.
  * Khattak et al. (2023) Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. 2023.  Maple: Multi-modal prompt learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Kiela et al. (2019) Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, Ethan Perez, and Davide Testuggine. 2019.  Supervised multimodal bitransformers for classifying images and text.  _arXiv preprint arXiv:1909.02950_.
  * Li et al. (2022) Feng Li, Hao Zhang, Yi-Fan Zhang, Shilong Liu, Jian Guo, Lionel M Ni, PengChuan Zhang, and Lei Zhang. 2022.  Vision-language intelligence: Tasks, representation learning, and large models.  _arXiv preprint arXiv:2203.01922_.
  * Li et al. (2019a) Gen Li, Nan Duan, Yuejian Fang, Daxin Jiang, and Ming Zhou. 2019a.  Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training.  _ArXiv_.
  * Li et al. (2019b) Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019b.  Visualbert: A simple and performant baseline for vision and language.  _arXiv preprint arXiv:1908.03557_.
  * Li et al. (2023) Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and Anima Anandkumar. 2023.  Voxformer: Sparse voxel transformer for camera-based 3d semantic scene completion.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_.
  * Liu et al. (2021) Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021.  Visually grounded reasoning across languages and cultures.  _arXiv preprint arXiv:2109.13238_.
  * Liu et al. (2022a) Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. 2022a.  Deplot: One-shot visual language reasoning by plot-to-table translation.  _arXiv preprint arXiv:2212.10505_.
  * Liu et al. (2023a) Fangyu Liu, Guy Emerson, and Nigel Collier. 2023a.  Visual spatial reasoning.  _Transactions of the Association for Computational Linguistics_.
  * Liu et al. (2023b) Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023b.  Improved baselines with visual instruction tuning.  _arXiv preprint arXiv:2310.03744_.
  * Liu et al. (2024) Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024.  Visual instruction tuning.  _Advances in neural information processing systems_.
  * Liu et al. (2023c) Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. 2023c.  Zero-1-to-3: Zero-shot one image to 3d object.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_.
  * Liu et al. (2022b) Yang Liu, Yu-Shen Wei, Hong Yan, Guan-Bin Li, and Liang Lin. 2022b.  Causal reasoning meets visual representation learning: A prospective study.  _Machine Intelligence Research_.
  * Liu et al. (2023d) Zhaoyang Liu, Yinan He, Wenhai Wang, Weiyun Wang, Yi Wang, Shoufa Chen, Qinglong Zhang, Yang Yang, Qingyun Li, Jiashuo Yu, et al. 2023d.  Internchat: Solving vision-centric tasks by interacting with chatbots beyond language.  _arXiv preprint arXiv:2305.05662_.
  * Long et al. (2015) Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015.  Fully convolutional networks for semantic segmentation.  In _2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Menon and Vondrick (2022) Sachit Menon and Carl Vondrick. 2022.  Visual classification via description from large language models.  _arXiv preprint arXiv:2210.07183_.
  * Podell et al. (2023) Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023.  Sdxl: Improving latent diffusion models for high-resolution image synthesis.  _arXiv preprint arXiv:2307.01952_.
  * Qi et al. (2020) Di Qi, Lin Su, Jia Song, Edward Cui, Taroon Bharti, and Arun Sacheti. 2020.  Imagebert: Cross-modal pre-training with large-scale weak-supervised image-text data.  _arXiv preprint arXiv:2001.07966_.
  * Rajabi and Kosecka (2023) Navid Rajabi and Jana Kosecka. 2023.  Towards grounded visual spatial reasoning in multi-modal vision language models.  _arXiv preprint arXiv:2308.09778_.
  * Rasheed et al. (2023) Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Erix Xing, Ming-Hsuan Yang, and Fahad S Khan. 2023.  Glamm: Pixel grounding large multimodal model.  _arXiv preprint arXiv:2311.03356_.
  * Redmon et al. (2016) Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016.  You only look once: Unified, real-time object detection.  In _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Su et al. (2019) Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019.  Vl-bert: Pre-training of generic visual-linguistic representations.  _arXiv preprint arXiv:1908.08530_.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_.
  * Wang et al. (2023) Junyang Wang, Ming Yan, Yi Zhang, and Jitao Sang. 2023.  From association to generation: Text-only captioning by unsupervised cross-modal mapping.  _arXiv preprint arXiv:2304.13273_.
  * Wang et al. (2022) Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022.  Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework.  In _International Conference on Machine Learning_ , pages 23318–23340. PMLR.
  * Xie et al. (2019) Ning Xie, Farley Lai, Derek Doran, and Asim Kadav. 2019.  Visual entailment: A novel task for fine-grained image understanding.  _arXiv preprint arXiv:1901.06706_.
  * Yang et al. (2024) Cheng Yang, Rui Xu, Ye Guo, Peixiang Huang, Yiru Chen, Wenkui Ding, Zhongyuan Wang, and Hong Zhou. 2024.  Improving vision-and-language reasoning via spatial relations modeling.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_.
  * Yang et al. (2022) Jinyu Yang, Jiali Duan, Son Tran, Yi Xu, Sampath Chanda, Liqun Chen, Belinda Zeng, Trishul Chilimbi, and Junzhou Huang. 2022.  Vision-language pre-training with triple contrastive learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Yang et al. (2023) Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. 2023.  Diffusion models: A comprehensive survey of methods and applications.  _ACM Computing Surveys_.
  * Yu et al. (2016) Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. 2016.  Modeling context in referring expressions.  In _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14_.
  * Zakari et al. (2022) Rufai Yusuf Zakari, Jim Wilson Owusu, Hailin Wang, Ke Qin, Zaharaddeen Karami Lawal, and Yuezhou Dong. 2022.  Vqa and visual reasoning: An overview of recent datasets, methods and challenges.  _arXiv preprint arXiv:2212.13296_.
  * Zellers et al. (2019) Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019.  From recognition to cognition: Visual commonsense reasoning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_.
  * Zhang et al. (2021) Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. 2021.  Vinvl: Revisiting visual representations in vision-language models.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_.
  * Zhao et al. (2023) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023.  A survey of large language models.  _arXiv preprint arXiv:2303.18223_.
  * Zhou et al. (2017) Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2017.  Scene parsing through ade20k dataset.  _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Zhu et al. (2023) Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023.  Minigpt-4: Enhancing vision-language understanding with advanced large language models.  _arXiv preprint arXiv:2304.10592_.

##  Appendix A Appendix

This section contains additional results. Figure 9 illustrates the differences
between LLaVA and ZeroVLM (L). In our study, we prepare the required questions
in JSON file format and modify ZeroVLM (L) to recognize and accept JSON file
inputs. We enable ZeroVLM (L) to identify the corresponding questions by
matching the input image names with the image names in the JSON file.Figure 10
illustrates the single-view and multi-view datasets obtained after 3D
reconstruction of the VSR dataset and the What’sUp dataset.

Figure 9: The primary difference between LLaVA and ZeroVLM (L) lies in their
input handling. LLaVA processes a single image as input, whereas ZeroVLM (L)
is designed to handle multiple images. Due to the limited expressive capacity
of individual images, we restrict the input for ZeroVLM (L) to two images.
Figure 10: The 3D reconstructed single-view and multi-view versions of each
dataset are presented.


#  Optimal Transmission Power Scheduling for Networked Control System under
DoS Attack

Siyi Wang, Yulong Gao and Sandra Hirche *This work was funded by the German
Research Foundation (DFG) under the grant number 315177489 as part of the SPP
1914 (CPN), by the Federal Ministry of Education and Research of Germany in
the programme of “Souver¨an. Digital. Vernetzt. ”, and by the joint project
6G-life (Project ID: 16KISK002, 16KISK001K).Siyi Wang and Sandra Hirche are
with the Chair of Information-oriented Control (ITR), Technical University of
Munich, 80333, Munich, Germany. Email<EMAIL_ADDRESS>Gao is
with the Department of Electrical and Electronic Engineering, Imperial College
London, SW7 2AZ, London, UK. Email<EMAIL_ADDRESS>
###### Abstract

Designing networked control systems that are reliable and resilient against
adversarial threats, is essential for ensuring the security of cyber-physical
systems. This paper addresses the communication-control co-design problem for
networked control systems under denial-of-service (DoS) attacks. In the
wireless channel, a transmission power scheduler periodically determines the
power level for sensory data transmission. Yet DoS attacks render data packets
unavailable by disrupting the communication channel. This paper co-designs the
control and power scheduling laws in the presence of DoS attacks and aims to
minimize the sum of regulation control performance and transmission power
consumption. Both finite- and infinite-horizon discounted cost criteria are
addressed, respectively. By delving into the information structure between the
controller and the power scheduler under attack, the original co-design
problem is divided into two subproblems that can be solved individually
without compromising optimality. The optimal control is shown to be certainty
equivalent, and the optimal transmission power scheduling is solved using a
dynamic programming approach. Moreover, in the infinite-horizon scenario, we
analyze the performance of the designed scheduling policy and develop an upper
bound of the total costs. Finally, a numerical example is provided to
demonstrate the theoretical results.

###### Index Terms:

SINR-based communication model, transmission power schedule, DoS attack,
infinite-horizon discounted cost

##  I Introduction

Cyber-physical systems are systems that integrate sensors, controllers, and
actuators to collaborate over a communication network for regulating and
optimizing the behavior of a dynamic system [1, 2]. There are many
applications such as robotics [3], smart grids [4], and intelligent vehicle
[5]. These networked systems generally assume that data are measured and
transmitted periodically to update control signals [6]. However, transmitting
sensory data over a communication network is costly in general due to limited
battery energy and communication channel bandwidth. This fact motivates us to
co-design the control and communication strategies to ensure that the valuable
sensory data is efficiently transmitted to the controller, thereby improving
overall system performance.

Due to increased connectivity, these cyber-physical systems are suffering from
cyber threats. Two most common cyber attacks are deception attacks and denial-
of-service (DoS) attacks [7]. Deception attacks degenerate the system
performance by maliciously modifying the information contained in transmitted
data [8, 9, 10]. DoS attacks render data packets unavailable by jamming
communication channels [11, 12]. It is more destructive compared to deception
attacks when facing threats from large-scale and persistent attacks.
Considering the detrimental impact of cyberattacks on control performance,
proactive risk management strategies are essential to ensure the resilience of
networked control systems.

Transmission power scheduling means that the transmission device dynamically
adjusts power levels according to the changing network conditions to fulfill
requirements of cyber-physical systems [13, 14]. Compared to the traditional
event-based triggering, power scheduling allows for continuous optimization of
energy usage, which is crucial for battery-powered devices and energy-
constrained environments. Increasing transmission power levels typically
enhances communication link reliability. As a consequence, the power scheduler
envisions a tradeoff between energy usage and improved control performance by
reliable transmission. To address this issue, the authors in [14] investigate
the optimal power scheduling in remote estimation. Moreover, [13, 15]
investigate the co-design of power scheduling and control strategies for
networked control. However, none of the above works take cyber attacks into
consideration. Some works, e.g., [16, 17], investigate the power allocation
under the DoS attack to minimize the long-term mean square error covariance
and propose a variance-based scheduling mechanism. However, variance-based
scheduling does not utilize real-time innovations when making decisions and
thus is generally outperformed by state-based scheduling [18]. In real-world
applications such as financial investment, current rewards are typically
valued more than future rewards. Thus, this article aims to obtain a co-design
strategy that minimizes the expectation of a discounted accumulated cost.

In this work, we propose a framework to jointly co-design the control law and
power scheduler for the networked control system under DoS attack. We consider
a signal-to-interference-plus-noise ratio (SINR)-based network model [17],
where the transmission success probability is affected by the transmission
power level chosen by the scheduler and the energy imposed by the attacker.
The contribution of this work is summarized as follows. For both cases, the
optimal co-design of control and scheduling is shown to be separable, given
that the knowledge about the attack energy is symmetric between the controller
and the scheduler. The optimal control law is shown to be certainty
equivalent. By applying the dynamic programming approach to the remaining
sequential decision problem, the condition for optimal power scheduling is
derived. Further, in the infinite-horizon case, we provide an upper bound on
the total cost achieved by the designed power scheduler.

The remainder of this article is structured as follows: Section II introduces
preliminaries. Section III and Section IV present the main result on optimal
co-design of control and transmission power scheduling in finite-horizon and
infinite-horizon cases, respectively. Section V presents numerical
simulations. Section VI concludes this work.

Notations. Denote by $\mathbb{R}$ and $\mathbb{R}^{n}$ the set of real numbers
and the set of the $n$-tuples of real numbers, respectively. Denote by
$\mathbb{N}$ the set of nonnegative integers. For $x,y\in\mathbb{N}$ and
$x\leq y$, the set $\mathbb{N}_{[x,y]}$ denotes $\\{z\in\mathbb{N}|x\leq z\leq
y\\}$. Denote $x_{0:k}$ as the history of state $x_{t}$ during time
$t\in\mathbb{N}_{[0,k]}$. For a random variable $X$, $X\sim\mathcal{D}_{X}$
implies that $X$ is distributed according to the distribution
$\mathcal{D}_{X}$. Denote $\mathbf{E}[\cdot]$, $\mathbf{E}[\cdot\mid\cdot]$
and ${\rm{cov}}[\cdot]$ as the expectation, the conditional expectation and
the covariance of the random variable, respectively. If the random variable
$x$ follows a normal distribution with the mean of $a$ and the covariance of
$\Sigma$, we write $x\sim\mathcal{N}(a,\Sigma)$. Denote by
$\mu_{\Sigma}(X)=\frac{1}{(2\pi)^{\frac{n}{2}}|\Sigma|^{\frac{1}{2}}}\exp(-\frac{(X-a)^{\top}\Sigma^{-1}(X-a)}{2})$
the probability density function of a $n$-dimension random vector
$X\sim\mathcal{N}(a,\Sigma)$. Denote by
$Q_{f}(x)=\frac{1}{\sqrt{2\pi}}\int_{x}^{\infty}e^{-x^{2}/2}$ the tail
function of the standard Gaussian distribution. Denote the spectral radius of
$A$ as $\rho(A)$.

##  II Preliminaries

We consider the energy-constrained feedback control system over the
communication network, as illustrated in Fig. 1. The scheduler periodically
determines the power of transmitting sensory data, which affects packets’
transmission success probability. The controller generates the control signal
based on the remote estimate.

Figure 1: Networked control system with transmission power scheduler.

###  II-A System model

The discrete-time stochastic dynamical system to be controlled is described as

$\displaystyle x_{k+1}$ $\displaystyle=$ $\displaystyle Ax_{k}+Bu_{k}+w_{k},$
(1)

where $x_{k}\in\mathbb{R}^{n}$ and $u_{k}\in\mathbb{R}^{m}$ are the state
vector and the control force, respectively. The system matrices are given by
$A\in\mathbb{R}^{n\times n}$, $B\in\mathbb{R}^{n\times m}$, where the pair
$(A,B)$ is controllable. The process noise
$w_{k}\in\mathbb{R}^{n}\sim\mathcal{N}(0,W)$ is assumed to be independent
identically distributed (i.i.d.) Gaussian processes with zero mean and
positive semi-definite variance $W$. The initial state
$x_{0}\sim\mathcal{N}(\bar{x}_{0},X_{0})$ is a random vector that is
statistically independent of $w_{k}$ for all $k$.

###  II-B Network model

The sensor located at the plant side periodically accesses the system states,
as in Fig. 1. The power-constrained scheduler determines the power
$p_{k}=\pi(\mathcal{I}_{k}^{p})\in\mathcal{E}$ used to send out packets at
time $k$, where $\pi$ and $\mathcal{I}_{k}^{p}$ denote the power scheduling
law and the locally available information set of the power scheduler,
respectively. Moreover, denote $\mathcal{E}=[0,p_{\max}]$ as the admissible
transmission power set with $p_{\max}$ being the maximum transmission power.

Consider an additive white Gaussian noise (AWGN) channel using quadrature
amplitude modulation [17]. In the presence of a DoS interference attacker, the
communication channel is modeled as ${\rm SINR}=p_{k}/(a_{k}+\sigma^{2})$,
where $\sigma^{2}$ is the additive white Gaussian noise power, and $a_{k}$ is
the interference power from the attacker [19]. The following assumptions are
for the attack energy.

###### Assumption 1

  1. 1.

The attack energy $a_{k}\in\mathbb{R}$, for $k\in\mathbb{N}$, is a i.i.d.
random process with a continuous distribution $\mathcal{D}_{a}$.

  2. 2.

The attack energy $a_{k}\in\mathcal{S}$ with $\mathcal{S}:=[0,a_{\max}]$ and
$a_{\max}$ being a positive scalar.

  3. 3.

The random variable $a_{k}$, $k\in\mathbb{N}$, is independent of process noise
$w_{k}$, for all $k$, and the initial state $x_{0}$.

The attack energy $a_{k}$ can be interpreted as a channel fading parameter
that encompasses unpredictable variations in the wireless channel [13]. The
subsequent sections will discuss control and power scheduling co-design,
considering known and unknown attack energy scenarios, respectively.

Denote the transmission success as a random binary process
$\delta_{k}\in\\{0,1\\}$, where $\delta_{k}=1$ denotes the packet is
transmitted successfully, and $\delta_{k}=0$ otherwise. The packet dropout
probability is affected by the transmission power and attack energy:

$\displaystyle
q_{k}=\Pr(\delta_{k}=0|p_{k},a_{k})=2Q_{f}\bigg{(}\sqrt{\frac{\alpha
p_{k}}{a_{k}+\sigma^{2}}}\bigg{)},$ (2)

where $\alpha$ is a communication channel parameter and $Q_{f}(\cdot)$ is the
tail function of the standard Gaussian distribution. Moreover,
$q_{k}\in\mathcal{Q}:=[2Q_{f}\big{(}\sqrt{\alpha
p_{\max}/\sigma^{2}}\big{)},1]$. Note that a higher transmission power
indicates a lower packet dropout probability and vice versa. Assume that the
network induces a one-step delay. At time $k$, the packet arrives at the
remote side is $z_{k}=x_{k-1}$ if $\delta_{k-1}=1$, and $z_{k}=\emptyset$
otherwise, with $z_{0}=\emptyset$. The remote estimator is given as

$\displaystyle\hat{x}_{k}$ $\displaystyle=$
$\displaystyle\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}]=A\mathbf{E}[x_{k-1}|\mathcal{I}_{k}^{c}]+Bu_{k-1}$
(3)

with the initial value $\hat{x}_{0}=\bar{x}_{0}$, and $\mathcal{I}_{k}^{c}$
denotes the remote information set. The control signal $u_{k}$ is generated
according to control law $u_{k}=f(\mathcal{I}_{k}^{c})$, where
$f\in\mathcal{F}$ with $\mathcal{F}$ being the admissible control law set.
Then, the remote information set available for the controller is defined as
$\mathcal{I}_{k}^{c}=\\{z_{0:k},\delta_{0:k-1},a_{0:k}\\}$ with the initial
value $\mathcal{I}_{0}^{c}=\\{a_{0}\\}$. Moreover, assume that the
transmission success index $\delta_{k}$ will be returned to the local side
with a one-step delay. Thus, the local information set available for the power
scheduler by time $k$ is
$\mathcal{I}_{k}^{p}=\\{x_{0:k},\delta_{0:k-1},a_{0:k}\\}$ with the initial
value $\mathcal{I}_{0}^{p}=\\{a_{0}\\}$.

This article aims to co-design the power scheduler and control law to optimize
control performance with limited transmission energy. More specifically, we
consider both finite- and infinite-horizon problems.

###### Problem 1

(Finite-horizon problem) Find the optimal power scheduler $\pi^{*}$ and the
control law $f^{*}$ by solving the following finite-horizon optimization
problem:

$\displaystyle\min_{f,\pi}$ $\displaystyle\Psi(f,\pi)=J_{c}(f,\pi)+\lambda
J_{p}(\pi),$ (4)

where the scalar $\lambda>0$ denotes the tradeoff multiplier. The control
performance $J_{c}(f,\pi)$ is defined as

$\displaystyle J_{c}(f,\pi)=\mathbf{E}\big{[}\sum_{k=0}^{T-1}$
$\displaystyle\gamma^{k}(x_{k}^{\top}Qx_{k}+u_{k}^{\top}Ru_{k})+\gamma^{T}x_{T}^{\top}Q_{N}x_{T}\big{]},$
(5)

where $\gamma\in(0,1]$ is the discount factor. The matrices $Q$, $Q_{N}$ are
semi-definite positive and $R$ is definite positive, respectively. Assume that
the pair $(A,Q^{\frac{1}{2}})$ is detectable, with
$Q=(Q^{\frac{1}{2}})^{\top}Q^{\frac{1}{2}}$. Moreover, the total transmission
energy consumption $J_{p}(\pi)$ is measured by
$J_{p}(\pi)=\sum_{k=0}^{T-1}\gamma^{k}p_{k}$.

###### Problem 2

(Infinite-horizon problem) Find the optimal power scheduler $\pi^{*}$ and the
control law $f^{*}$ by solving the following infinite-horizon optimization
problem:

$\displaystyle\min_{f,\pi}$
$\displaystyle\bar{\Psi}(f,\pi)=\bar{J}_{c}(f,\pi)+\lambda\bar{J}_{p}(\pi)$
(6)

with $\bar{J}_{c}(f,\pi)$ being the infinite-horizon LQG function:

$\displaystyle\bar{J}_{c}(f,\pi)=\mathbf{E}\big{[}\sum_{k=0}^{\infty}$
$\displaystyle\gamma^{k}(x_{k}^{\top}Qx_{k}+u_{k}^{\top}Ru_{k})\big{]},$ (7)

where $\gamma\in(0,1)$, and the transmission energy consumption is
$\bar{J}_{p}(\pi)=\sum_{k=0}^{\infty}\gamma^{k}p_{k}$.

The discount factor $\gamma$ reflects how immediate and future costs are
weighted. A higher discount factor places more weight on future costs and vice
versa.

##  III Finite-horizon case

In this section, we develop the solution to Problem 1. In the following, we
will decompose the co-design optimization problem and design the optimal
control law.

###  III-A Optimal control

In networked control systems, the controller may induce a dual effect: 1)
stabilizing the dynamic system and 2) probing the system to reduce
uncertainties. This leads to a nonclassical information structure between the
power scheduler and the controller. In other words, the information available
to two distributed decision makers might be asymmetric. We provide the
following notion to facilitate the search for the structural results of the
optimization problem.

###### Definition 1

(Dominating policy) Denote $\mathcal{U}$ as the set of all admissible policy
pairs $(f,\pi)$. Consider the cost function $\Psi$ defined in the
corresponding problem. A set of policy pairs
$\mathcal{U}^{\prime}\subset\mathcal{U}$ is called a dominating class of
policies, if for any feasible $(f,\pi)\in\mathcal{U}$, there exists a feasible
$(f^{\prime},\pi^{\prime})\in\mathcal{U}^{\prime}$, such that
$\Psi(f^{\prime},\pi^{\prime})\leq\Psi(f,\pi)$.

The following lemma decomposes the original co-design optimization problem,
assuming that the attack energy is known, and investigates which class of
policies is dominating for problem (4).

###### Lemma 1

Consider an admissible scheduling policy set $\Pi$, in which function only
depends on random variables $\\{x_{0},w_{0:k-1},a_{0:k}\\}$. Then the set
$\mathcal{U}^{\text{CE}}=\\{f^{\ast},\pi)|\pi\in\Pi\\}$ is a dominating class
of policies, where $f^{\ast}$ is the certainty equivalence controller:

$\displaystyle
u_{k}^{\ast}=f^{\ast}(\mathcal{I}_{k}^{c})=-L_{k}\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}]$
(8)

with $L_{k}=\gamma(R+\gamma B^{\top}P_{k+1}B)^{-1}B^{\top}P_{k+1}A$, and
$P_{k+1}$ is solved from algebraic Riccati equation [20]:

$\displaystyle P_{k}$ $\displaystyle=$ $\displaystyle Q+\gamma
A^{\top}P_{k+1}A$ (9) $\displaystyle-\gamma^{2}A^{\top}P_{k+1}B(R+\gamma
B^{\top}P_{k+1}B)^{-1}B^{\top}P_{k+1}A$

with $P_{N}=Q_{N}$.

Proof. Assume that there exists a triggering law
$\tilde{\pi}=\\{\tilde{\pi}_{1},\tilde{\pi}_{2},\dots\\}$ being the function
of random variables $\\{x_{0},w_{0:k-1},a_{0:k}\\}$. Note that the information
pattern of the power scheduler and the controller are nested, i.e.,
$\mathcal{I}_{k}^{c}\subset\mathcal{I}_{k}^{p}$. In addition, we can see that
$\\{x_{0},w_{0:k-1}\\}$ can be fully recovered from $x_{0:k}$ and $u_{0:k-1}$,
which is inferred from $\mathcal{I}_{k}^{p}$ accessible to the local
scheduler. Therefore, there exists a policy pair $(f,\tilde{\pi})$ producing
identical decision variables as $(f,\pi)$ almost surely, i.e.,
$\tilde{\pi}_{k}(x_{0},w_{0:k-1},a_{0:k})=\pi(\mathcal{I}_{k}^{p})$ holds
almost surely. In other words, there exists a policy pair $(f,\tilde{\pi})$
that achieves the same cost as $(f,\pi)$. Denote the remote estimation error
as $e_{k}=x_{k}-\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}]$. According to (3), we
have
$\hat{x}_{k+1}=A\hat{x}_{k}+Bu_{k}+\delta_{k}w_{k}+(1-\delta_{k})A\mathbf{E}[e_{k}|\mathcal{I}_{k}^{c},\delta_{k}=0]$.
Under the control law $f$ that is symmetric with respect to innovation
$w_{k}$, we have $\mathbf{E}[e_{k}|\mathcal{I}_{k}^{c},\delta_{k}=0]=0$, see
[21]. Then the remote estimation error $e_{k}$ evolves as

$\displaystyle e_{k+1}=\delta_{k}w_{k}+(1-\delta_{k})(Ae_{k}+w_{k})$ (10)

with the initial value $e_{0}=x_{0}-\hat{x}_{0}$. Note that $e_{k}$ comprises
primitive random variables $w_{0:k-1}$ and $e_{0}$. Substituting the algebraic
Riccati equation into the cost function (4), we have

$\displaystyle\Psi(f,\pi)$ $\displaystyle=$
$\displaystyle\mathbf{E}\big{[}x_{0}^{\top}P_{0}x_{0}+\sum_{k=0}^{T-1}\big{(}\gamma^{k+1}w_{k}^{\top}P_{k}w_{k}$
(11)
$\displaystyle+\gamma^{k}(u_{k}+L_{k}x_{k})^{\top}\Lambda_{k}(u_{k}+L_{k}x_{k})+\gamma^{k}\lambda
p_{k}\big{)}\big{]}$

with $\Lambda_{k}=R+\gamma B^{\top}P_{k+1}B$. Note that the first, the second,
and the last terms of (11) are independent of the control policy $f$.
Substituting $x_{k}$ with $\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}]+e_{k}$, we
have

$\displaystyle\mathbf{E}[(u_{k}+L_{k}x_{k})^{\top}\Lambda_{k}(u_{k}+L_{k}x_{k})]$
(12) $\displaystyle=$
$\displaystyle\mathbf{E}[(u_{k}+L_{k}\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}])^{\top}\Lambda_{k}(u_{k}+L_{k}\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}])$
$\displaystyle
2(u_{k}+L_{k}\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}])^{\top}\Lambda_{k}L_{k}e_{k}+(L_{k}e_{k})^{\top}\Lambda_{k}L_{k}e_{k}]$
$\displaystyle=$
$\displaystyle\mathbf{E}[(u_{k}+L_{k}\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}])^{\top}\Lambda_{k}(u_{k}+L_{k}\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}])$
$\displaystyle+(L_{k}e_{k})^{\top}\Lambda_{k}L_{k}e_{k}],$

where the second equality is from the tower property of conditional
expectation, i.e.,
$\mathbf{E}[(u_{k}+L_{k}\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}])^{\top}\Lambda_{k}L_{k}e_{k}]=\mathbf{E}[(u_{k}+L_{k}\mathbf{E}[x_{k}|\mathcal{I}_{k}^{c}])^{\top}\Lambda_{k}L_{k}\mathbf{E}[e_{k}|\mathcal{I}_{k}^{c}]]$
and $\mathbf{E}[e_{k}|\mathcal{I}_{k}^{c}]=0$. Then the optimal controller
minimizing (11) results in (8). Moreover, we have
$\Psi(f,\pi)=\Psi(f,\tilde{\pi})\geq\min_{f\in\mathcal{F}}\Psi(f,\tilde{\pi})=\Psi(f^{\ast},\tilde{\pi})=\Psi(f^{\ast},\pi^{\prime})$,
where $\pi^{\prime}$ depends on $\mathcal{I}_{k}^{p}$, and the first and the
last equalities follow from
$\tilde{\pi}_{k}(x_{0},w_{0:k-1},a_{0:k})=\pi(\mathcal{I}_{k}^{p})$. Namely,
for any feasible $(f,\pi)\in\mathcal{U}$, the set
$(f^{\ast},\pi^{\prime})\in\mathcal{U}$ is a dominating class of policy pairs
such that $\Psi(f,\pi)\geq\Psi(f^{\ast},\pi^{\prime})$. $\blacksquare$

Lemma 1 implies that the original co-design problem can be decomposed into two
subproblems, i.e., optimal power scheduling design that depends on primitive
random variables and the optimal control law design, without loss of
optimality.

###  III-B Optimal power scheduling

The following theorem will develop the optimal power scheduler in the finite
horizon scenario.

###### Theorem 1

Consider the optimization problem (4) for system (1). Fix the optimal control
law as the certainty equivalence controller (8). Let

$\displaystyle q_{k}^{\ast}$ $\displaystyle=$
$\displaystyle\mathop{\operatorname{argmin}}\limits_{q_{k}\in\mathcal{Q}}\\{g(e_{k},a_{k},q_{k})+q_{k}\iota_{k}\\},$
(13)

where the stage cost is

$\displaystyle g(e_{k},a_{k},q_{k})=\lambda p(q_{k},a_{k})+\gamma
q_{k}e_{k}^{\top}A^{\top}\Sigma_{k+1}Ae_{k}$ (14)

with $\Sigma_{k}=L_{k}^{\top}(R+\gamma B^{\top}P_{k+1}B)L_{k}$ and
$p(q_{k},a_{k})=\Big{(}Q_{f}^{-1}\Big{(}\frac{q_{k}}{2}\Big{)}\Big{)}^{2}\frac{a_{k}+\sigma^{2}}{\alpha}.$
Moreover,
$\iota_{k}=\mathbf{E}[V_{k+1}(\mathcal{I}_{k+1}^{p})|\mathcal{I}_{k}^{p},\delta_{k}=0]-\mathbf{E}[V_{k+1}(\mathcal{I}_{k+1}^{p})|\mathcal{I}_{k}^{p},\delta_{k}=1]$
and

$\displaystyle
V_{k}(\mathcal{I}_{k}^{p})=\min_{q_{k}\in\mathcal{Q}}\mathbf{E}[\gamma^{k}g(e_{k},a_{k},q_{k})+V_{k+1}(\mathcal{I}_{k+1}^{p})|\mathcal{I}_{k}^{p}],$
(15)

for $k\in\mathbb{N}_{[0,T-1]}$ with the initial condition
$V_{T}(\mathcal{I}_{T}^{p})=0$. Then the optimal power scheduler
$\pi^{*}(\mathcal{I}_{k}^{p})$ is given by

$\displaystyle p_{k}=\pi^{*}(\mathcal{I}_{k}^{p})=p(q_{k}^{\ast},a_{k}).$ (16)

Proof. Substituting the optimal controller (8) into (11) yields

$\displaystyle\Psi(f^{\ast},\pi)$ $\displaystyle=$
$\displaystyle\mathbf{E}\big{[}x_{0}^{\top}P_{0}x_{0}+\sum_{k=0}^{T-1}\big{(}\gamma^{k+1}w_{k}^{\top}P_{k}w_{k}$
(17) $\displaystyle\hskip
30.00005pt+\gamma^{k}e_{k}^{\top}\Sigma_{k}e_{k}+\lambda\gamma^{k}p_{k}\big{)}\big{]}.$

Taking the conditional expectation of $e_{k+1}^{\top}\Sigma_{k+1}e_{k+1}$ with
respect to $\mathcal{I}_{k}^{p}$, and by (10), we have

$\displaystyle\mathbf{E}[e_{k+1}^{\top}\Sigma_{k+1}e_{k+1}|\mathcal{I}_{k}^{p}]$
(18) $\displaystyle=$
$\displaystyle\mathbf{E}\Big{[}(1-\delta_{k})(Ae_{k}+w_{k})^{\top}\Sigma_{k+1}(Ae_{k}+w_{k})$
$\displaystyle+\delta_{k}w_{k}^{\top}\Sigma_{k+1}w_{k}\big{|}\mathcal{I}_{k}^{p}\Big{]}$
$\displaystyle=$
$\displaystyle\mathbf{E}\Big{[}(1-\delta_{k})e_{k}^{\top}A^{\top}\Sigma_{k+1}Ae_{k}+w_{k}^{\top}\Sigma_{k+1}w_{k}\big{|}\mathcal{I}_{k}^{p}\Big{]}$
$\displaystyle=$
$\displaystyle\mathbf{E}\Big{[}q_{k}e_{k}^{\top}A^{\top}\Sigma_{k+1}Ae_{k}\big{|}\mathcal{I}_{k}^{p}\Big{]}+\text{tr}(\Sigma_{k+1}W),$

where the second equality establishes as
$\mathbf{E}[w_{k}|\mathcal{I}_{k}^{p}]=0$ and $w_{k}$ is independent of
$e_{k}$, the last equality follows from
$\mathbf{E}[\mathbf{E}[(1-\delta_{k})|\mathcal{I}_{k}^{p}]\mathcal{I}_{k}^{p}]=\mathbf{E}[q_{k}|\mathcal{I}_{k}^{p}]$.
Then the original optimization problem (4) is reduced to

$\displaystyle\min_{\pi\in\Pi}\sum_{k=0}^{T}\mathbf{E}\Big{[}\gamma^{k}g(e_{k},a_{k},q_{k})\big{|}\mathcal{I}_{k}^{p}\Big{]}.$
(19)

We omit the remaining terms of (17) and the last term of (18) as they are
independent of $\pi$. Note that $e_{k}$ can be fully recovered by the local
power scheduler as $\delta_{k-1}\in\mathcal{I}_{k}^{p}$. Let us apply the
dynamic programming approach to (19). Synthesizing the optimal scheduler boils
down to characterize the value function $V_{k}(\mathcal{I}_{0}^{p})$, which is
defined as in (15). Moreover, we have

$\displaystyle\mathbf{E}[V_{k+1}(\mathcal{I}_{k+1}^{p})|\mathcal{I}_{k}^{p}]=\mathbf{E}\big{[}q_{k}\mathbf{E}[V_{k+1}(\mathcal{I}_{k+1}^{p})|\mathcal{I}_{k}^{p},\delta_{k}=0]$
$\displaystyle\hskip
60.00009pt+(1-q_{k})\mathbf{E}[V_{k+1}(\mathcal{I}_{k+1}^{p})|\mathcal{I}_{k}^{p},\delta_{k}=1]\big{]},$
(20)

which follows from the law of total expectation. Substituting (III-B) into
(15) yields (13), which gives the optimal power scheduler in (16).
$\blacksquare$

###### Remark 1

Theorem 1 characterizes the optimal power scheduler for the finite-horizon
problem (4). Note that it is difficult to compute $\iota_{k}$ in (13), in
particular for a long horizon $T$. Alternatively, there is a practical greedy
way to approximate $q^{*}_{k}$ as follows

$\displaystyle q_{k}^{\ast}$ $\displaystyle=$
$\displaystyle\mathop{\operatorname{argmin}}\limits_{q_{k}\in\mathcal{Q}}g(e_{k},a_{k},q_{k})$
(21)

with $g(e_{k},a_{k},q_{k})$ defined in (14). The greedy scheduling policy
solved from (21) is suboptimal yet has a rather low computation complexity.

Theorem 1 assumes that the attack energy $a_{k}$ is known, the following
proposition considers the case when attack energy $a_{k}$ is unknown.

###### Proposition 1

Consider the optimization problem (4) for system (1). If the attack energy
$a_{k}$ is unknown, consider an admissible scheduling policy set
$\Pi^{\prime}$, which only depends on random variables
$\\{x_{0},w_{0:k-1}\\}$. Then the set
$\mathcal{U}^{\text{CE}^{\prime}}=\\{f^{\ast},\pi)|\pi\in\Pi^{\prime}\\}$ with
$f^{\ast}$ given by (8) is a dominating class of policies.
Proof. Note that removing the attack energy $a_{k}$ from the information sets
$\mathcal{I}_{k}^{p}$ and $\mathcal{I}_{k}^{c}$ does not alter the information
structure between the local power scheduler and the remote controller. It
follows the proof of Lemma 1. Then the set $\mathcal{U}^{\text{CE}^{\prime}}$
constituted by a power scheduling $\pi\in\Pi^{\prime}$ and the certainty
equivalence controller (8) is a class of dominating policies. $\blacksquare$

Next, we provide an approximation to the optimal scheduler when the attack
energy $a_{k}$ is unknown. It follows from the proof of Theorem 1. Then, we
consider the optimisation problem (19). Taking the expectation of $p_{k}$ over
the distribution $\mathcal{D}_{a}$ yields

$\displaystyle\mathbf{E}_{a_{k}\sim\mathcal{D}_{a}}[p(q_{k},a_{k})]=\int_{a_{k}\in\mathcal{S}}\Big{(}Q_{f}^{-1}\Big{(}\frac{q_{k}}{2}\Big{)}\Big{)}^{2}\frac{a_{k}+\sigma^{2}}{\alpha}{\rm
d}a_{k}.$ (22)

Then we replace $p_{k}$ with
$\mathbf{E}_{a_{k}\sim\mathcal{D}_{a}}[p(q_{k},a_{k})]$ in scheduling policy
(13) and (14). Then, an approximation-based greedy scheduling is

$\displaystyle
q_{k}^{\ast}=\mathop{\operatorname{argmin}}\limits_{q_{k}\in\mathcal{Q}}\\{\mathop{\mathbf{E}}\limits_{a_{k}\sim\mathcal{D}_{a}}[\lambda
p(q_{k},a_{k})]+\gamma q_{k}e_{k}^{\top}A^{\top}\Sigma_{k+1}Ae_{k}\\}.$ (23)

##  IV Infinite-horizon case

In this section, we consider the infinite-horizon problem, i.e., Problem 2.
Since the pair $(\sqrt{\gamma}A,B)$ is controllable, the algebraic Riccati
equation

$\displaystyle P=Q+\gamma A^{\top}PA-\gamma^{2}A^{\top}PB(R+\gamma
B^{\top}PB)^{-1}B^{\top}PA$ (24)

has a unique and positive semi-definite solution $P$. Accordingly, matrices
$L_{k}$ $\Lambda_{k}$, $\Sigma_{k}$ are written as $L$, $\Lambda$ and
$\Sigma$. The following assumption is essential for characterizing the optimal
stationary scheduler.

###### Assumption 2

The minimum achievable packet dropout probability under the attack energy $a$,
i.e., $q_{m}(a)=2Q_{f}\bigg{(}\sqrt{\frac{\alpha
p_{\max}}{a+\sigma^{2}}}\bigg{)}$ satisfies
$\mathbf{E}_{a\sim\mathcal{D}_{a}}[q_{m}(a)]<\frac{1}{\rho(A)^{2}}.$

Similar to Lemma 1, we decompose the optimization problem (6) into two
subproblems as follows.

###### Lemma 2

The set $\mathcal{U}^{\text{CE}}=\\{f^{\ast},\pi)|\pi\in\Pi\\}$ is a
dominating class of policies, where $f^{\ast}$ is given by the certainty
equivalence controller:

$\displaystyle
u_{k}^{\ast}=f^{\ast}(\mathcal{I}_{k}^{c})=-L\mathbf{E}[x_{k}\mid\mathcal{I}_{k}^{c}]$
(25)

with $L=\gamma(R+\gamma B^{\top}PB)^{-1}B^{\top}PA$.

Proof. It follows from the proof of Lemma 1. $\blacksquare$

Next we consider how to synthesize the optimal power scheduler. From (17), the
cost function under the optimal control policy (25) is rewritten as

$\displaystyle\bar{\Psi}(f^{\ast},\pi)=\mathbf{E}\big{[}\sum_{k=0}^{\infty}\gamma^{k+1}w_{k}^{\top}Pw_{k}+\gamma^{k}e_{k}^{\top}\Sigma
e_{k}+\lambda\gamma^{k}p_{k}\big{]}.$ (26)

Accordingly, the optimization problem (6) is reduced to

$\displaystyle\min_{\pi\in\Pi}\sum_{k=0}^{\infty}\mathbf{E}\Big{[}\gamma^{k}g(e_{k},a_{k},q_{k})|\mathcal{I}_{k}^{p}\Big{]}$
(27)

with the per-stage cost $g$ defined in (14), $\Sigma=L^{\top}(R+\gamma
B^{\top}PB)L$ and initial value $(e_{0},a_{0}):=(e,a)$. We next formulate the
optimization problem (27) as a MDP with the control model
$(\mathcal{X},\mathcal{Q},\mathcal{P},g)$, where
$\mathcal{X}:=\mathbb{R}^{n}\times\mathcal{S}$ denotes the state space,
$\mathcal{Q}$ denotes the action space, $\mathcal{P}$ denotes the Borel
measurable transition kernel defined on $(\mathcal{X},\mathcal{Q})$. Note that
$w_{k}$ obeys a distribution of $\mathcal{N}(\mathbf{0},W)$. Denote
$e^{+},a^{+}$ as the next states of $e$ and $a$, respectively. Denote
$\mathcal{P}$ as the transition probability from $(e,a,q)$ to $(e^{+},a^{+})$.
By (10), we have

$\displaystyle\mathcal{P}(e^{+},a^{+}|e,a,q)$ (28) $\displaystyle=$
$\displaystyle[(1-q)\mu_{w}(e^{+})+q\mu_{w}(e^{+}-Ae)]\mathcal{D}_{a}(a^{+}),$

where $\mu_{w}$ is the probability density function of the random variable
$w\sim\mathcal{N}(\mathbf{0},W)$. Note that (28) holds as the process noise
$w$ is independent of attack energy $a\in\mathcal{D}_{a}$ and $q$ is the
current decision.

According to the optimization problem (27), define an associated $n$-stage
cost under the policy $\pi\in\Pi$:

$\displaystyle
J_{n}(e,a,\pi):=\mathbf{E}^{\pi}\big{[}\sum_{t=0}^{n-1}\gamma^{t}g(e_{t},a_{t},q_{t})\big{]}$
(29)

with the initial value $(e,a)\in\mathcal{X}$ and $n\geq 1$. The decision
variables $q_{t}$ for $t\in\mathbb{N}_{[0,n-1]}$ is chosen according to policy
$\pi$. Let $G(e,a)$ be a class of nonnegative and lower semi-continuous
(l.s.c.) functions on $(e,a)\in\mathcal{X}$. Define the value iteration
sequence $V_{n}(e,a)\in G(e,a)$, for $n\geq 1$ and $V_{0}(\cdot,\cdot)=0$:

$\displaystyle V_{n}(e,a):$ $\displaystyle=$
$\displaystyle\min_{q\in\mathcal{Q}(e,a)}\big{\\{}g(e,a,q)$ (30)
$\displaystyle\hskip
10.00002pt+\gamma\mathbf{E}[V_{n-1}(e^{+},a^{+}|e,a,q)]\big{\\}},$

where $\mathcal{Q}(e,a)$ denotes a non-empty set associated to $(e,a)$, for
any $(e,a)\in\mathcal{X}$. Then, we have $V_{n}(e,a)=\inf_{\pi}J_{n}(e,a,\pi)$
given initial value $(e,a)\in\mathcal{X}$, for $n\geq 1$.

The following proposition shows that there exists an optimal stationary policy
for the optimization problem (27).

###### Proposition 2

Let Assumption 2 hold, we have the following claims:

  1. 1.

$\lim_{n\rightarrow\infty}V_{n}=V^{\ast}$ with
$V^{\ast}=\inf_{\pi}\lim_{n\rightarrow\infty}J_{n}(e,a,\pi)$;

  2. 2.

$V^{\ast}$ satisfies the Bellman optimality equation:

$\displaystyle
V^{\ast}(e,a)=\min_{q\in\mathcal{Q}(e,a)}\Big{[}g(e,a,q)+\gamma\mathbf{E}[V^{\ast}(e^{+},a^{+})|e,a,q]\Big{]}.$
(31)

  3. 3.

There exist a optimal stationary policy
$\pi^{\ast}\in\Pi:\mathcal{X}\rightarrow\mathcal{Q}$ minimizing the right-hand
side of (31) for all $(e,a)\in\mathcal{X}$, i.e.,
$V^{\ast}(e,a)=g(e,a,\pi^{\ast})+\gamma\mathbf{E}[V^{\ast}(e^{+},a^{+})|e,a,\pi^{\ast}]$.

Proof. According to [22], we need to verify the following conditions:

  1. 1.

$g(e,a,q)$ is nonnegative, l.s.c. and inf-compact on
$\mathcal{X}\times\mathcal{Q}$;

  2. 2.

the transition law $\mathcal{P}$ is weakly continuous;

  3. 3.

the multifunction $\pi(e,a)\rightarrow\mathcal{Q}(e,a)$ is l.s.c.;

  4. 4.

there exists a policy $\bar{\pi}$ such that
$J_{\infty}(e,a,\bar{\pi}):=\sum_{k=0}^{\infty}\mathbf{E}^{\bar{\pi}}\Big{[}\gamma^{k}g(e_{k},a_{k},\bar{q})\Big{]}<\infty$
for each $(e,a)\in\mathcal{X}$.

The first condition holds by the definition of $g(e,a,q)$, as in (14). It is
inf-compact on $\mathcal{X}\times\mathcal{Q}$ as the set
$\\{q\in\mathcal{Q}(e,a)|g(e,a,q)\leq r\\}$ is compact. The second condition
holds as $\mathcal{P}(e,a,q)$ is continuous on
$(e,a,q)\in\mathcal{X}\times\mathcal{A}$. Then for any continuous and bounded
function $V^{\ast}(e,a)$ on $(e,a,q)$, the map
$(e,a,q)\rightarrow\inf_{\mathcal{X}}J^{\ast}(e,a)\mathcal{P}(e^{+},a^{+}|e,a,q){\rm
d}e^{+}{\rm d}a^{+}$ is continuous on
$(e,a,q)\in\mathcal{X}\times\mathcal{A}$. The third condition holds as
$\mathcal{X}\times\mathcal{Q}$ is convex, see [23].

To verify the last condition, we choose
$\bar{\pi}(\mathcal{I}_{k}^{p})=p_{\max}$ for all $k$. Then,
$q(a_{k},p_{\max})=2Q_{f}\Big{(}\sqrt{\frac{\alpha
p_{\max}}{a_{k}+\sigma^{2}}}\Big{)}$. The expected dropout probability under
$\bar{\pi}$ is

$\displaystyle\bar{q}=\mathbf{E}_{a\sim\mathcal{D}_{a}}[q(a,p_{\max})].$ (32)

The next is to calculate $J_{\infty}(e,a,\bar{\pi})$. Denote
$X_{k}^{e}:=\mathbf{E}[e_{k}^{\top}e_{k}]$. By the expression of estimation
error as in (10), we have

$\displaystyle X_{k}^{e}=\left\\{\begin{array}[]{ll}{\rm{tr}}(X_{0})&{\rm
if}~{}k=0\\\ {\rm{tr}}(W)&{\rm if}~{}k>0~{}\&~{}\delta_{k-1}=1\\\
A^{\top}X_{k-1}^{e}A+W&{\rm
if}~{}k>0~{}\&~{}\delta_{k-1}=0.\end{array}\right.$ (36)

Accordingly, define
$\theta_{t}^{w}:={\rm{tr}}\big{(}\sum_{r=0}^{t-1}(A^{r})^{\top}\Sigma
A^{r}W\big{)}$, $\theta_{t}^{x}:={\rm{tr}}\big{(}(A^{t})^{\top}\Sigma
A^{t}X_{0}\big{)}+\theta_{t}^{w},\quad{\rm for}~{}t\geq 1.$ When $k=0$,
$\mathbf{E}^{\bar{\pi}}[\gamma^{k}e_{k}^{\top}\Sigma e_{k}]={\rm{tr}}(\Sigma
X_{0})$. When $k=1$,

$\displaystyle\mathbf{E}^{\bar{\pi}}[\gamma^{k}e_{k}^{\top}\Sigma e_{k}]$
$\displaystyle=$
$\displaystyle\gamma(\bar{q}\theta_{1}^{x}+(1-\bar{q})\theta_{1}^{w})$
$\displaystyle=$ $\displaystyle\gamma{\rm{tr}}(\Sigma W+\bar{q}A^{\top}\Sigma
AX_{0}).$

When $k=2$,

$\displaystyle\mathbf{E}^{\bar{\pi}}[\gamma^{k}e_{k}^{\top}\Sigma e_{k}]$
$\displaystyle=$
$\displaystyle\gamma^{2}\big{(}(1-\bar{q})\theta_{1}^{w}+\bar{q}(1-\bar{q})\theta_{2}^{w}+\bar{q}^{2}\theta_{2}^{x}\big{)}$
$\displaystyle=$ $\displaystyle\gamma^{2}{\rm{tr}}(\Sigma
W+\bar{q}A^{\top}\Sigma AW+\bar{q}^{2}(A^{2})^{\top}\Sigma A^{2}X_{0}).$

By induction, for all $k\geq 1$, we have

$\displaystyle\mathbf{E}^{\bar{\pi}}[\gamma^{k}e_{k}^{\top}\Sigma e_{k}]$ (37)
$\displaystyle=$
$\displaystyle\gamma^{k}{\rm{tr}}\Big{(}\sum_{r=0}^{k-1}\bar{q}^{k}(A^{k})^{\top}\Sigma
A^{k}W+\bar{q}^{k}(A^{k})^{\top}\Sigma A^{k}X_{0}\Big{)}.$

Sum (37) over all $k\geq 0$, we obtain

$\displaystyle\mathbf{E}^{\bar{\pi}}\bigg{[}\sum_{k=0}^{\infty}\gamma^{k}e_{k}^{\top}\Sigma
e_{k}\bigg{]}=\frac{\gamma}{1-\gamma}{\rm{tr}}\bigg{(}\sum_{r=0}^{\infty}(\gamma\bar{q})^{r}(A^{r})^{\top}\Sigma
A^{r}W\bigg{)}$ $\displaystyle\hskip
90.00014pt+{\rm{tr}}\bigg{(}\sum_{r=0}^{\infty}(\gamma\bar{q})^{r}(A^{r})^{\top}\Sigma
A^{r}X_{0}\bigg{)}.$ (38)

Note that when $\gamma\bar{q}(\rho(A))^{2}<1$,
$\sum_{k=0}^{\infty}\mathbf{E}^{\bar{\pi}}[\gamma^{k}e_{k}^{\top}\Sigma
e_{k}]<\infty$. Then Assumption 2 is sufficient for its establishment.
Moreover, we have

$\displaystyle\mathbf{E}^{\bar{\pi}}\bigg{[}\sum_{k=0}^{\infty}\gamma^{k}e_{k}^{\top}\Sigma
e_{k}\bigg{]}-\mathbf{E}^{\bar{\pi}}\bigg{[}\sum_{k=0}^{\infty}\gamma^{k+1}q_{k}e_{k}^{\top}A^{\top}\Sigma
Ae_{k}\bigg{]}$ (39) $\displaystyle=$ $\displaystyle{\rm{tr}}(\Sigma
X_{0})+\frac{\gamma}{1-\gamma}{\rm{tr}}(\Sigma W).$

The total transmission cost under power scheduling law $\bar{\pi}$ is written
as

$\displaystyle
J_{p}^{\bar{\pi}}=\sum_{k=0}^{\infty}\gamma^{k}p_{\max}=\frac{p_{\max}}{1-\gamma}.$
(40)

Combine (IV), (39) and (40), we obtain that

$J_{\infty}(e,a,\bar{\pi})=\mathbf{E}^{\bar{\pi}}\bigg{[}\sum_{k=0}^{\infty}\gamma^{k+1}q_{k}e_{k}^{\top}A^{\top}\Sigma
Ae_{k}\bigg{]}+\lambda J_{p}^{\bar{\pi}}<\infty.$

Conditions 1-4) are verified. According to [22], claims 1-3) holds.
$\blacksquare$

The following theorem develops the optimal scheduler and the cost achieved by
the designed scheduling policy.

###### Theorem 2

Consider the optimization problem (6) for system (1). Fix the optimal control
law as the certainty equivalence controller (25). Let Assumption 2 hold and
let

$\displaystyle
q^{\ast}(e,a)=\mathop{\operatorname{argmin}}_{q\in\mathcal{Q}}\big{\\{}g(e,a,q)+\gamma\mathbf{E}[V^{\ast}(e^{+},a^{+})|e,a,q]\big{\\}},$
(41)

where $V^{\ast}$ is solved from (31).

  1. 1.

The optimal power scheduler $\pi^{*}$ is given by

$\displaystyle
p=\pi^{*}(q^{\ast}(e,a),a)=\Big{(}Q_{f}^{-1}\Big{(}\frac{q^{\ast}(e,a)}{2}\Big{)}\Big{)}^{2}\frac{a+\sigma^{2}}{\alpha}.$
(42)

  2. 2.

Let $\Theta$ be the solution of Laypunov equation:

$\displaystyle\gamma\bar{q}A^{\top}\Theta A+\Sigma=\Theta.$ (43)

where $\bar{q}$ is given by (32). The total cost achieved by the optimal power
scheduler and the optimal control is upper-bounded by

$\displaystyle\bar{\Psi}(f^{\ast},\pi^{\ast})$ $\displaystyle\leq$
$\displaystyle\frac{\gamma{\rm{tr}}(PW+\Theta W)+\lambda
p_{\max}}{1-\gamma}+{\rm{tr}}(\Theta X_{0}).$ (44)

Proof. The first claim follows from Proposition 2. By Assumption 2, the
solution of (43) is written as

$\displaystyle\Theta=\sum_{k=0}^{\infty}(\gamma\bar{q})^{k}(A^{k})^{\top}\Sigma
A^{k}.$ (45)

Substituting (45) into (IV), we have

$\displaystyle\mathbf{E}\bigg{[}\sum_{k=0}^{\infty}\gamma^{k}e_{k}^{\top}\Sigma
e_{k}\bigg{]}$ $\displaystyle=$
$\displaystyle\frac{\gamma}{1-\gamma}{\rm{tr}}(\Theta W)+{\rm{tr}}(\Theta
X_{0}).$ (46)

Substitute (46) and (40) into (26), we have

$\displaystyle\bar{\Psi}(f^{\ast},\pi^{*})$ $\displaystyle\leq$
$\displaystyle\mathbf{E}\bigg{[}\sum_{k=0}^{\infty}\gamma^{k+1}w_{k}^{\top}Pw_{k}\bigg{]}$
$\displaystyle+\frac{\gamma{\rm{tr}}(\Theta W)}{1-\gamma}+{\rm{tr}}(\Theta
X_{0})+\frac{\lambda p_{\max}}{1-\gamma}$ $\displaystyle=$
$\displaystyle\frac{\gamma{\rm{tr}}(PW+\Theta W)+\lambda
p_{\max}}{1-\gamma}+{\rm{tr}}(\Theta X_{0}).$

Thus, we obtain the second conclusion. $\blacksquare$

The optimization problem 2 considers discounted criteria, the minimum expected
cost (44) is affected by the attack energy distribution $\mathcal{D}_{a}$, the
discount factor $\gamma$, the covariance of state initial value $X_{0}$. This
aligns with the result of the discounted optimization problem, see [22].

##  V Simulation

In this section, we provide numerical examples to illustrate our results.
Consider a second-order system with system dynamics $A={\rm
diag}\\{1.3,-1.1\\}$, $B=[0.1~{}0.1]^{\top}$, process noise covariance
$W=0.001$. The weighting matrices in the LQG function (5) are chosen as
$Q=R={\rm diag}\\{1,1\\}$. The discount factor is chosen as $\gamma=0.9$. The
covariance of the initial value is chosen as $X_{0}={\rm
diag}\\{0.01,0.01\\}$.

Figure 2: From top to button: attack energy and transmission power;
transmission success index under greedy scheduler (21). Figure 3: From top to
button: system state; control signal under the greedy scheduler (21). Figure
4: Tradeoff between mean square error and average transmission power archived
by the greedy scheduler (21), the approximation-based greedy scheduler (22),
and constant-power schedulers.  Figure 5: The total cost achieved by the
greedy scheduler (21), the approximation-based greedy scheduler (22). The
theoretical and empirical upper bounds of the total cost achieved by the
designed schedulers. The theoretical lower bound of the total cost achieved by
the best constant-power scheduler.

The communication channel parameters are chosen as $\sigma^{2}=1$, $\alpha=3$,
see [17]. The attack energy is chosen as a uniformly distributed random
variable between $[0,1]$.

We choose the time horizon as $T=100$ and the tradeoff multiplier as
$\lambda=1$. The top subfigure of Fig. 2 depicts the energy of the attack and
the transmission power determined by scheduler (21) during
$k\in\mathbb{N}_{[0,T]}$. The button subfigure of Fig. 2 depicts the
transmission success index during $k\in\mathbb{N}_{[0,T]}$. It can be observed
from Fig. 3 that transmission success is affected by both the transmission
power and attack energy. When addressing the resource utilization efficiency
under attacks, a higher attack energy generally leads a higher transmission
power. Fig. 3 depicts the system state trajectory, its estimate, and control
signal. It can be observed from Fig. 3 that the remote estimator effectively
tracks the system dynamics. Fig. 4 depicts the mean square error under
different average transmission power achieved by the greedy schedulers (21)
and (22) and the scheduler with constant transmission power $p_{k}=3$ for
$k\in\mathbb{N}_{[0,T]}$. The mean square error is measured by
$\frac{1}{T}\sum_{k=0}^{T}e_{k}^{\top}A^{\top}\Sigma Ae_{k}$ and the average
transmission power is measured by $\frac{1}{T}\sum_{k=0}^{T}p_{k}$. It can be
observed that using the same average transmission power, greedy schedulers
(21) and (22) achieves smaller mean square error than constant-power
schedulers.

Fig. 5 depicts the total regulation and transmission costs under different
multipliers under the greedy scheduler (21), the approximation-based greedy
scheduler (22) assuming attack energy is unknown, and constant-power
schedulers. We choose tradeoff multiples $\lambda\in[0.01,1]$. Monte Carlo
simulation runs $20000$ trials. As in (17), only last-second terms depend on
the designed control and scheduling laws. We choose
$\sum_{k=0}^{T}\big{(}\gamma^{k}e_{k}^{\top}\Sigma
e_{k}+\lambda\gamma^{k}p_{k}\big{)}$ as the empirical total regulation and
transmission cost. For the empirical results, shaded areas represent $\pm$ one
standard deviation over 20000 trials. Accordingly, the theoretical upper bound
of the total cost, i.e., the cost achieved by a transmission scheduler with
maximum constant power, is $\frac{\gamma{\rm{tr}}(\Theta W)+\lambda
p_{\max}}{1-\gamma}+{\rm{tr}}(\Theta X_{0})$. The empirical upper bound is
calculated by implementing the constant-power scheduler $p_{k}=3$ for all $k$.
It can be observed from Fig. 5 that the theoretical upper bound of the total
cost approaches the empirical one. As a comparison, we also depict the lower
bound of the total cost by a constant-power scheduler, i.e., the best
performance it can achieve, which is given by

$\displaystyle\min_{q\in\mathcal{Q}}\bigg{\\{}\frac{\gamma{\rm{tr}}(\tilde{\Theta}W)}{1-\gamma}+{\rm{tr}}(\tilde{\Theta}X_{0})+\frac{\lambda\mathbf{E}_{a\sim\mathcal{D}_{a}}[p(q,a)]}{1-\gamma}\bigg{\\}},$
$\displaystyle{\rm s.t.}\ \gamma
qA^{\top}\tilde{\Theta}A+\Sigma=\tilde{\Theta}.$

Moreover, it can be observed that greedy schedulers (21) and (22) both
outperform the schedulers with constant power, which includes the one that
achieves the minimum cost among all constant-power schedulers.

##  VI Conclusion

In this article, we have studied the optimal co-design of control law and
transmission power scheduler that minimizes the regulation and transmission
costs for networked control systems under DoS attacks. Given the
acknowledgment signal from the remote controller, the information structure
between the controller and the power scheduler is nested. Therefore, we showed
that the original co-design can be decomposed into the optimal control design,
yielding a certainty equivalence controller, and the optimal power scheduling
design, which is tracked by dynamic programming approaches. Expressions of the
optimal power scheduling were provided in both finite- and infinite-horizon
cases. To ease the computational complexity in finite-horizon dynamic
programming, an alternative greedy scheduler was developed for implementation.
Additionally, in the infinite-horizon case, we provided the upper bound of the
total regulation and transmission cost under the proposed scheduler.

## References

  * [1] W. Zhang, M. S. Branicky, and S. M. Phillips, “Stability of networked control systems,” IEEE control systems magazine, vol. 21, no. 1, pp. 84–99, 2001.
  * [2] Y. Jiang, S. Wu, R. Ma, M. Liu, H. Luo, and O. Kaynak, “Monitoring and defense of industrial cyber-physical systems under typical attacks: From a systems and control perspective,” IEEE Transactions on Industrial Cyber-Physical Systems, 2023.
  * [3] Z. Yan, N. Jouandeau, and A. A. Cherif, “A survey and analysis of multi-robot coordination,” International Journal of Advanced Robotic Systems, vol. 10, no. 12, p. 399, 2013.
  * [4] A. K. Singh, R. Singh, and B. C. Pal, “Stability analysis of networked control in smart grids,” IEEE Transactions on Smart Grid, vol. 6, no. 1, pp. 381–390, 2014.
  * [5] S. E. Li, Y. Zheng, K. Li, L.-Y. Wang, and H. Zhang, “Platoon control of connected vehicles from a networked control perspective: Literature review, component modeling, and controller synthesis,” IEEE Transactions on Vehicular Technology, 2017.
  * [6] A. Molin and S. Hirche, “Suboptimal event-triggered control for networked control systems,” ZAMM - Journal of Applied Mathematics and Mechanics, vol. 94, no. 4, pp. 277–289, 2014.
  * [7] S. Amin, A. A. Cárdenas, and S. S. Sastry, “Safe and secure networked control systems under denial-of-service attacks,” in Hybrid Systems: Computation and Control: 12th International Conference, HSCC 2009, San Francisco, CA, USA, April 13-15, 2009. Proceedings 12, pp. 31–45, Springer, 2009.
  * [8] A. Teixeira, D. Pérez, H. Sandberg, and K. H. Johansson, “Attack models and scenarios for networked control systems,” in Proceedings of the 1st international conference on High Confidence Networked Systems, pp. 55–64, 2012.
  * [9] H. Zhu, L. Xu, Z. Bao, Y. Liu, L. Yin, W. Yao, C. Wu, and L. Wu, “Secure control against multiplicative and additive false data injection attacks,” IEEE Transactions on Industrial Cyber-Physical Systems, 2023.
  * [10] L. Xu, H. Zhu, K. Guo, Y. Gao, and C. Wu, “Output-based secure control under false data injection attacks,” IEEE Transactions on Industrial Cyber-Physical Systems, 2024.
  * [11] Y. Dai, M. Li, K. Zhang, and Y. Shi, “Robust and resilient distributed mpc for cyber-physical systems against dos attacks,” IEEE Transactions on Industrial Cyber-Physical Systems, 2023.
  * [12] S. Feng, A. Cetinkaya, H. Ishii, P. Tesi, and C. De Persis, “Networked control under dos attacks: Tradeoffs between resilience and data rate,” IEEE Transactions on Automatic Control, vol. 66, no. 1, pp. 460–467, 2020.
  * [13] K. Gatsis, A. Ribeiro, and G. J. Pappas, “Optimal power management in wireless control systems,” IEEE Transactions on Automatic Control, vol. 59, no. 6, pp. 1495–1510, 2014.
  * [14] X. Ren, J. Wu, K. H. Johansson, G. Shi, and L. Shi, “Infinite horizon optimal transmission power control for remote state estimation over fading channels,” IEEE Transactions on Automatic Control, vol. 63, no. 1, pp. 85–100, 2017.
  * [15] T. Soleymani, J. S. Baras, S. Hirche, and K. H. Johansson, “Feedback control over noisy channels: Characterization of a general equilibrium,” IEEE Transactions on Automatic Control, 2021.
  * [16] Y. Li, L. Shi, P. Cheng, J. Chen, and D. E. Quevedo, “Jamming attacks on remote state estimation in cyber-physical systems: A game-theoretic approach,” IEEE Transactions on Automatic Control, vol. 60, no. 10, pp. 2831–2836, 2015.
  * [17] Y. Li, D. E. Quevedo, S. Dey, and L. Shi, “Sinr-based dos attack on remote state estimation: A game-theoretic approach,” IEEE Transactions on Control of Network Systems, vol. 4, no. 3, pp. 632–642, 2016.
  * [18] J. Wu, Y. Yuan, H. Zhang, and L. Shi, “How can online schedules improve communication and estimation tradeoff?,” IEEE Transactions on Signal Processing, vol. 61, no. 7, pp. 1625–1631, 2013.
  * [19] J. G. Proakis, Digital communications.  McGraw-Hill, Higher Education, 2008.
  * [20] K. J. Ȧström, Introduction to stochastic control theory.  Dover Publications, 2006.
  * [21] T. Soleymani, J. S. Baras, S. Hirche, and K. H. Johansson, “Value of information in feedback control: Global optimality,” IEEE Transactions on Automatic Control, 2022.
  * [22] O. Hernández-Lerma and M. Muñoz de Ozak, “Discrete-time markov control processes with discounted unbounded costs: optimality criteria,” Kybernetika, vol. 28, no. 3, pp. 191–212, 1992.
  * [23] O. Hernández-Lerma and W. J. Runggaldier, “Monotone approximations for convex stochastic control problems,” Journal of Mathematical Systems, Estimation, and Control, vol. 4, no. 4, pp. 99–140, 1994.