# Interpretable cancer cell detection with phonon microscopy using multi-task
conditional neural networks for inter-batch calibration

Yijie Zheng, Rafael Fuentes-Dominguez, Matt Clark, George S.D. Gordon,
Fernando Perez-Cota
Optics and Photonics Research Group, University of Nottingham, NG7 2RD
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Advances in artificial intelligence (AI) show great potential in revealing
underlying information from phonon microscopy (high-frequency ultrasound) data
to identify cancerous cells. However, this technology suffers from the ‘batch
effect’ that comes from unavoidable technical variations between each
experiment, creating confounding variables that the AI model may inadvertently
learn. We therefore present a multi-task conditional neural network framework
to simultaneously achieve inter-batch calibration, by removing confounding
variables, and accurate cell classification of time-resolved phonon-derived
signals. We validate our approach by training and validating on different
experimental batches, achieving a balanced precision of 89.22% and an average
cross-validated precision of 89.07% for classifying background, healthy and
cancerous regions. Classification can be performed in $\sim$0.5 seconds with
only simple prior batch information required for multiple batch corrections.
Further, we extend our model to reconstruct denoised signals, enabling
physical interpretation of salient features indicating disease state including
sound velocity, sound attenuation and cell-adhesion to substrate.

###### Index Terms:

Batch effect, multi-task learning, Neural networks, Signal classification,
Physical interpretability

##  I Introduction

Mechanical cancer information such as elasticity and stiffness are closely
associated with cell characteristics, which could play a crucial role in early
and accurate diagnosis in disease such as cancer[1, 2, 3]. Recent developments
in label-free phonon microscopy [4, 5, 6, 7, 8, 9, 10] employing time-resolved
phonon-derived signals at high frequencies ($\sim$ 5 GHz) have demonstrated
its ability to image high-resolution ($\sim$ 490–740 nm axial resolution)
fixed and living cells and obtain their corresponding elastic properties with
high fidelity [2, 11].

In conventional low-frequency medical ultrasound, AI has been demonstrated to
reveal hidden information and extract discriminative features from ultrasound
signals/images without time-consuming manual inspection, providing improved
quantitative results to enhance early detection and treatment of cancer [12,
13, 14, 15]. Similar for high-frequency ultrasound in phonon microscopy, a
previous study has shown convolutional neural networks (CNN) are capable of
learning signal features within one experiment (i.e. intra-batch) and
performing 93% classification between cancerous and normal cells [4].

However, a type of ‘batch effect’ problem arises when classifying signals from
different experiments within the same dataset making inter-batch
classification challenging. This effect can be attributed to various technical
factors, including the experimental dates, temperature, humidity and also the
nanofabricated tolerances. For practical reasons, cancer cells and healthy
cells are measured in separate experiments, and so the combined effect of
these technical factors is to create a confounding variable that is correlated
with the disease status of the cells. This, in turn, causes AI models (e.g.
neural networks) to overfit and prevents them from learning true features that
distinguish between cancerous and normal cells.

Several methods have been developed to address the batch effect problem. In
the field of microarray gene expression data, ComBat [16] and limma [17] have
been employed in normalizing batch-related information in scRNA-seq datasets
but are not compatible with data that shows random variations, such as uptake
rates for vaccinations. Other conventional pre-processing methods such as
nested mixed effect models [18, 19] modeled the shared random effect from each
batch to remove the batch effects. More generalizable models such as mutual
nearest neighbors (MNNs) [20] and canonical correlation analysis (CCA) [21]
have been proposed to establish connections between two datasets, with fastMNN
[22], Scanorama [23], BBKNN [24] and Seurat 3 [25] explored as dimensionality
reduction methods. However, These methods calibrate a single batch of data at
a time, aligning it with the corresponding reference batched data, which can
be computationally demanding, particularly when dealing with multiple batches
of data. To incorporate various batches of data, scVI [26] and BERMUDA [27]
explicitly analyze cells from all batches simultaneously and jointly remove
the batch effect. Nevertheless, these approaches require prior processing of
the batch information, limiting their application to samples where batch
metadata is not available a priori.

Neural networks (NN) have been applied to batch alignment problems as an
alternative to traditional statistical methods. The DESC technique [28] aims
to iteratively eliminate the batch effect by learning the cluster-specific
gene expression representations. This is an unsupervised learning model which
makes it of low scalability and accessibility to effectiveness evaluation.
Supervised learning models such as scGen [29] use variational autoencoders
that are capable of calibrating inter-batch information across scRNA-seq
datasets, but its adaptability to other types of datasets has not been
investigated. More recently, scGCN [30] used a graph neural network-based
model, which has been demonstrated to remove the batch effect for single-cell
datasets and transfer labels across studies.

Nevertheless, raw time-resolved phonon-derived signal traces contain
significantly more information than a single value for stiffness or elasticity
and represent depth variations in sound velocity, elasticity, absorption, etc.
Existing methods to remove the ‘batch effect’ of such time-series data present
unique challenges. Specifically, for high-frequency time-resolved phonon-
derived signals, the batch effect is not random but rather systematically
associated with the experimental procedures. This means conventional methods,
such as mixed-effects models, which are typically designed to address random
batch effects that are assumed to be independent of the specific experimental
conditions, may fail to appropriately capture and correct for the non-random
correlations. Further, time-series data is considered as structured data with
high temporal dependencies, whereas conventional statistical and AI methods
are expected to handle observation-independent data, which is difficult to
align with the sequential nature of time-series data. Finally, current methods
have not been extensively employed in real-world applications within the
domain of data classification.

In this paper, we present a new method that effectively learns the features of
time-resolved phonon-derived signals and also conditions the batch features to
simultaneously produce reliable classification results and inter-batch
calibration. This concept is derived from a theoretical framework of causal
inference [31], where in our scenario, batch features are the common causal
influence of cell features and cell classification results. Our new method
aims to remove the confounding.

Specifically, we present a multi-task conditional neural network framework
that generates a batch-corrected latent space feature vector. We first
demonstrate our framework enables a more robust representative model to
calibrate the inter-batch effect across different experiments with a 20%
increase in classification precision over the baseline model. We then validate
our framework by classifying inter-batch cell signals between cancerous,
normal cells and background with an average precision of 89.22% and cross-
validated precision of 89.07%. Finally, we explore the interpretability of the
latent space feature vector by reconstructing the original signal from feature
vectors at various conditions. We highlighted several advantages of this
neural network-based framework: Firstly, it only requires $\sim$ 0.5 s to
achieve both the inter-batch calibration and classification with high
adaptability to multiple batches. Second, only very limited prior batch
information (i.e. the date) is required to train this framework. Third, the
latent space feature vector enables the reconstruction of de-noised signals,
facilitating physical interpretation of the model and application of
explainable AI techniques.

##  II METHODS

###  II-A Batched data information

The raw time-resolved phonon-derived signals (i.e. main dataset) presented in
this paper was collected by phonon microscopy [4], containing 300k time-series
signal from 80 different breast cells. To perform the inter-batch calibration
and classification, we defined 8 batches of data based on the experiment days
and table I shows the detailed information of each batch. Specifically, there
are 10 cells (37.21k individual signal input) in each batch, where each batch
was performed on different, discontinuous days across several months. Normal
cells (MCF 10a cell type) were measured and recorded with odd Batch IDs while
cancerous cells (MDA-MB-231 cell type) were collected and recorded with even
Batch IDs respectively. The baseline model introduced in [4] performing intra-
batch classification was trained on 60% of data in each batch, which was
confounded by batch features. To establish that the batch effect is
successfully corrected, we defined Batch 1,2, 5 and 6 as the training and
validation set with a ratio of 4:1, and Batch 3, 4, 7 and 8 as the test set.
The validation set is expected to provide unbiased evaluations and stopping
criteria and the test set aims to examine the generalization performance of
the model on unseen data.

TABLE I: Raw phonon-derived data information presented in each batch

|  |  |  |  | Data split for classification
---|---|---|---|---|---
Batch ID |  Experiment day | Cell type | # cell |  # Individual signal input | Intra-batch | Inter-batch
|  |  |  |  | Train / Val |
1 | Day 1 |  Normal (MCF 10a) | 10 | 37.21 k | Test | Train / Val
|  |  |  |  | Train / Val |
2 | Day 2 |  Cancer (MDA-MB-231) | 10 | 37.21 k | Test | Train / Val
|  |  |  |  | Train / Val |
3 | Day 3 |  Normal (MCF 10a) | 10 | 37.21 k | Test | Test
|  |  |  |  | Train / Val |
4 | Day 4 |  Cancer (MDA-MB-231) | 10 | 37.21 k | Test | Test
|  |  |  |  | Train / Val |
5 | Day 5 |  Normal (MCF 10a) | 10 | 37.21 k | Test | Train / Val
|  |  |  |  | Train / Val |
6 | Day 6 |  Cancer (MDA-MB-231) | 10 | 37.21 k | Test | Train / Val
|  |  |  |  | Train / Val |
7 | Day 7 |  Normal (MCF 10a) | 10 | 37.21 k | Test | Test
|  |  |  |  | Train / Val |
8 | Day 8 |  Cancer (MDA-MB-231) | 10 | 37.21 k | Test | Test

Figure 1: Multi-task conditional neural network (a) Conditional Encoder,
trained with a reference dataset as a conditional reference operator. (b)
Multi-task variational encoder, mapping the input data into a shared latent
space with variational distribution. Also, two classifiers are trained
separately where Classifier 1 aims to distinguish the normal, cancerous cells
and the background, while Classifier 2 aims to prevent the model from learning
the batch IDs of the data. (c) Denoising Decoder to reconstruct the input
signal from the shared latent space. (d) Cluster visualization of shared
latent space.

###  II-B Theoretical confounding considerations

In theory, this ‘batch effect’ problem can be referred to a typical example of
confounding model [31], where batch features are the common confounding
variables that associate with both cell features and the disease status of the
cells. Therefore, instead of identifying the direct correlation between cell
features with the disease detection, the results are confounded by the
presence of batch features.

To avoid this issue, we adapt a technique that has been mathematically proven
to remove confounding variables based on Bayesian network analysis by
conditioning on and marginalizing over the confounding variable [32]. First,
to force marginalization over the conditioning, a multi-task output is used in
which the cell classification is maximized simultaneously with minimization of
the batch ID detection. This procedure randomly allocates input phonon signal
traces to a Batch ID, which enforces the condition that a correctly
marginalized model will be unable to predict batch ID. Second, we enforce
conditioning on batch features by selecting pairs of phonon signal traces: one
from a cell area and one from a randomly selected background area.

###  II-C Network architectures

Following these principles, we construct a multi-tasking conditional neural
network model shown in Figure 1 that is able to simultaneously learn batch and
cell features, condition the cell features on the batch features, and then
learn the correct marginalization over these conditions.

Specifically, the conditional encoder (Figure 1(a)) is used to learn batch
features, which are used to condition a variational encoder (VE) that learns
the cell features. The conditional encoder includes four Conv2D and MaxPooling
layers respectively with Leaky Relu activation function defined, exploiting
efficient correlations for time series data. Figure 1(b) shows the
architecture of multi-task variational encoder with two classifiers, where the
encoder which has the same structure as that of conditional encoder shown in
1(a), with a sampling layer to map the input data into a shared latent space.
The rationale for using a variational model is to provide the latent space
with multivariate Gaussian distributed multi-dimensional parameters ($\mu$ and
$\sigma$) with higher orthogonality and generalizations compared to a simple
encoder. Additionally, it enables both signal reconstruction and further
generation of diverse samples through its probabilistic modelling capacity.
Classifier 1 shown in Figure 1(b) aims to classify the given signal into one
of 3 classes (i.e. normal, cancer and background) representing integer values
0 to 2 as the ground truth, inclusively. In terms of Classifier 2 shown in
Figure 1(b), we firstly represent each signal in integer values 1 to 8 based
on its batch ID as the ground truth for test sets, and then shuffled the
values to perform random batch IDs as the ground truth for train and
validation sets. The shuffle process aims to remove the effect of confounding
variables (i.e. batch features) by avoiding the model learning to distinguish
the batch ID itself through assigning randomized ID values. This in turn,
gives the model emphasis on learning the features to distinguish the normal,
cancer and background of the cell.

The connectivity is performed by concatenating the output of each hidden layer
of the conditional encoder and the corresponding output of each hidden layer
of the multi-task variational encoder. This aims to provide both conditioning
on and marginalizing over the confounding variable in order to remove the
confounding effect from the batch features. The whole model was trained
iteratively with the total loss function defined as the sum of two balanced
categorical cross entropy losses performed in two classifiers respectively,
giving more importance to underrepresented classes. It takes the model 145 min
and 2000 epochs to converge using Tensorflow 2.0 running on an NVIDIA GeForce
RTX 3090 GPU. The Adam optimizer was used with a learning rate of 0.001 in a
decay rate of 1e-5.

###  II-D Latent space analysis

The interpretation of the framework can be represented by the latent space,
which contains the latent features learned by the neural networks. We first
perform the cluster visualization by plotting the Uniform Manifold
Approximation and Projection (UMAP) map [33], a low-dimensional structural
preserved graph, from which we expect to visualize the projected clusters from
different categories as shown in Figure 1(d).

###  II-E Signal reconstruction

Having validated the accuracy of our classifier model, we now wish to exploit
the encoder to help identify and interpret features of the input signals that
denote cancer. This could help to identify physical properties indicative of
cancer and thus represent a form of explainable AI.

Specifically, beginning with the trained latent space, we train a denoising
decoder model as shown in Figure 1 (c). This decoder model aims to reconstruct
the original signal from the low-dimensional latent space that contains the
features crucial for accurate calibration and classification. The model is
expected to produce denoised signals that preserve the critical components
contributing to the classification while filtering out unimportant and batch-
specific features.

###  II-F Physical features extraction for model interpretation

To extract physical parameters from the phonon-derived signal, we have used a
simple model which considers the cell to be homogeneous. The model is based on
time-resolved Brillouin scattering where the signal is of sinusoidal nature
due to optical interference and its decay exponential due to sound
attenuation.

$signal=A\cos(2\pi fz+\phi)e^{(-\alpha_{0}z)}$ (1)

where $f$ is the Brillouin frequency, $\nu$ is the sound attenuation
coefficient, $\phi$ is the signal phase, $A$ the initial amplitude and $z$ is
the distance from the transducer. The Brillouin frequency is then a function
of the sound velocity $v$:

$f=2n\nu/\lambda$ (2)

where $n$ is the refractive index of the cell and $\lambda$ the optical probe
wavelength. The sound velocity is an elastic property of the material which is
related to the density and longitudinal modulus of the cell. The experimental
signal is then fitted to equation 2 to the minimum possible error resulting in
the estimation of the sound velocity, the sound attenuation coefficient and
the signal’s phase.

###  II-G Evaluation metrics

We define several evaluation metrics in order to quantitatively access the
performance of classification using this multi-task conditional neural
network, which can be visualized through a confusion matrix. Here, the
confusion matrix shows the frequency map of multi-class predictions, where
each cell in the matrix contains the count or frequency of instances belonging
to a specific actual class and predicted class pair. For each actual class
(i.e. each row), we define the frequency value of correct predictions at the
$i^{th}$ class (where both the actual and predicted labels are within the same
class) as $C_{i}$ among normal ($i=1$), cancerous ($i=2$) and background
($i=3$), and false predictions as $F_{i}$.

To assess the overall accuracy of our model, we employ a balanced precision
metric [34], which is well-suited for our imbalanced datasets. This metric
computes the weighted average precision across the whole test set, taking into
account the relative sizes of different classes:

$\displaystyle\mathrm{balanced\ precision}=$
$\displaystyle{W_{1}}\times\frac{C_{1}}{C_{1}+F_{1}}+{W_{2}}\times\frac{C_{2}}{C_{2}+F_{2}}$
(3) $\displaystyle+{W_{3}}\times\frac{C_{3}}{C_{3}+F_{3}}$
$W_{i}=\frac{\frac{1}{y_{i}}}{{\frac{1}{y_{1}}+\frac{1}{y_{2}}+\frac{1}{y_{3}}}}$
(4)

where, ${W_{i}}$ represents the weight assigned to the $i^{th}$ class and
$y_{i}$ represents the number of test instances belonging to the $i^{th}$
class. This means classes with fewer instances (smaller $y_{i}$ values) will
have a higher weight, while classes with more instances (larger $y_{i}$
values) will have a lower weight.

We next evaluate the model’s predictive capability for individual batches and
gauge any variations in its performance by calculating the average precision
for each cell from different batches:

$\mathrm{Precision}=\frac{1}{N}\times\sum_{n=1}^{N}\mathrm{balanced\
precision}$ (5)

where, for each batch, $N$ represents the total number of time-series signals
associated with each cell, which corresponds to the number of classified
results per cell (3721 in this paper). Therefore, this metric contains a
vector with a length of 10 for each batch where each element in the vector
corresponds to the average balanced precision of a specific cell.

Additionally, we measure the sensitivity and specificity of the model to
evaluate its ability to accurately differentiate between cancerous and normal
cells. High sensitivity is expected to minimize false negatives and ensure the
test is effective in catching cancer cases early while high specificity
ensures that individuals without cancer are not subjected to unnecessary
diagnostic procedures or treatments.

##  III RESULTS

###  III-A Validation of inter-batch calibration

We assess the effectiveness of calibrating the inter-batch effect by examining
the clustering plot of the latent space, which represents the critical
features utilized for classification. Figure 2(a) shows the UMAP plot of the
raw data before the calibration, and (b) to (f) show the UMAP plot of the
latent space original and calibrated signal by our multi-task conditional NN,
with colors representing different batch IDs. The raw data before calibration
exhibit more diffused clustering, especially with Batch 5 and Batch 7
distinctly separated from the others. This explains why features obtained from
specific batches for classification may not adequately represent the rest
batches of data, resulting in poor generalisability for separating different
classes, as shown in Figure 2(g), where the UMAP plot is colored by different
classes. In contrast, after calibration, the clustering results shown in
Figure 2(f) show that data from different batches is more contiguously
distributed, with 3 classes highly separable in Figure 2(l). This demonstrates
effective batch calibration because different batches now map to the same
regions in the clustering map.

We next compared our model with other neural network models, namely the
baseline model using CNN presented in [4], variational encoder (VE) [29]
commonly used for normalization and calibration, conditional NN-only model
without Classifier 2, and multi-task NN only model without background
reference input. Figure 2(b)-(c) show a more condensed distribution in
clustering maps for different Batch IDs compared to the result before
calibration. However, Batch 1 and Batch 5 clusters are still dispersed far
apart, making it challenging to separate different cell classes easily as
shown in Figure 2(h)-(i). Figure 2(d)-(e) show a further compact distribution
in clustering map by using either conditional NN only or multi-task NN only
model respectively. This leads to improved separation between different cell
classes, as shown in Figure 2(j)-(k). Considering this, the combination of
both models(i.e. our multi-task conditional NN) results in the most
continuously distributed UMAP plot where batch IDs may not be distinctly
separated, but different classes can still be distinguished effectively as
shown in Figure 2(l).

Therefore, it demonstrates that both conditioning and marginalization to
control the confounding variables are theoretically necessary to remove
confounding variables. However, the multi-task conditional model which
enforces both conditioning and marginalization shows its optimal capability in
mitigating confounding problems and achieving accurate cell classification.

Figure 2: UMAP clustering plot (a) before and (b)-(f) after calibration with
various calibration models, colored by 8 different batch IDs. Corresponding
UMAP plot (g) - (l), colored by 3 different classes (i.e. normal, cancer, and
background).

###  III-B Validation of inter-batch classification

We evaluate the classification performance in comparison with various models
using Batch 1,2, 5 and 6 as the training sets, as shown in Figure 3. We first
generate the confusion matrix shown in Figure 3(a) for different models and
then calculate the balanced precision by assigning individual weights to each
class with a ratio of 0.343:0.48:0.177. The baseline model demonstrates a 100%
capability in distinguishing the background class from the others. However, it
exhibits limited ability to differentiate between cancerous and normal cells,
as the majority (81%) of predictions that were expected to be normal cells are
instead assigned to the cancerous classes. The commonly used calibration model
(i.e. variational encoder) shows improved behavior in classifying the
cancerous and normal cells, achieving a balanced precision of 73.90%, which is
13% greater than the baseline model. Both the multi-task-only model and the
conditional-only model achieve significantly higher balanced precision of
79.66% and 80.30%, respectively. Further, the combined model (the multi-task
conditional model) achieves the highest balanced precision of 89.22%,
indicating that it correctly classifies the largest portions of cell classes.

To compare the model robustness between batches, we next calculate the average
precision for each individual cell within a batch, which means averaging
across full prediction results (i.e. 3721 points) per cell within each batch.
We then represent these average precision results in each batch using boxplots
in Figure 3(b), where the blue box extends from the Q1 to Q3 quartile of the
data distribution, with the orange line indicating the median (Q2) data value.
The multi-task conditional NN classifier improves the median average precision
value across the four tested batches of data, especially in Batch 4, where the
median precision increases from 0.77 to 0.99. Also, it shows the lowest
standard deviation of the average precision values, with less than 0.2 across
all four batches, indicating the generalization and robustness ability of this
model.

Additionally, we plot the predicted and target cell example images colored by
class ID from four tested batches in Figure 3(c), where green and blue
represent the cancerous and normal pixels respectively, with grey representing
the background (water). Our multi-task conditional model visually shows the
best predictions, especially for predicting normal cells, where both baseline
model and VE classifier has limited capacity to distinguish the normal from
cancerous classes. This further demonstrates that this model has the highest
specificity of 0.7172 as shown in Figure 3(e), which means it can correctly
ensure the cell is not cancer. However, all five models show a superior high
sensitivity with the highest of 0.9992 in Figure 3(f) from the multi-task
conditional NN model, which means the model is effective at detecting
individuals who actually have the disease, minimizing the chances of false
negatives.

To assess the model’s generalizability, we conducted a 6-fold cross-
validation, evaluating performance by assigning various data batches as
training sets to prevent overfitting to specific subsets. The figure 3(d)
shows the cross-validated average confusion matrix, with an average balanced
precision of 89.07% and a standard variation of 0.99%, indicating a
commendable generalization ability of our model. Additionally, enlarging the
size of the latent space from 128 to 1024 resulted in an increased balanced
precision of 91.09%. However, this leads to trade-off considerations, as it
leads to longer training times and higher memory usage.

We finally compared the computational resources of these models which are
presented in Figure 3(g) and (h): converge and prediction time, trainable
parameters comparison. Although the combinational model requires the longest
training time (145 min) and the largest trainable parameters (46 million)
compared to the other models, it only takes less than 0.5s to achieve the
prediction, which is desirable for this task.

Figure 3: Inter-batch classification results of various classifiers evaluated
by metrics (a) confusion matrix, (b) average precision distribution of each
tested batch, (c) example cell imaging, (d) confusion matrix after 6-fold
cross-validation, (e) sensitivity, (f) specificity. Computational resources
comparison between various classifiers: (g) converge time and (h) Trainable
parameters

###  III-C Denoised signal reconstruction from latent space for explainability

Figure 4(a) shows the denoised signal reconstructed from the decoder model,
with the latent space size of 128 and 1024 respectively. In general, the
larger latent space will contain more comprehensive features, which produce a
slightly higher classification precision of 89.9% compared to that of 82.2%
(Figure 3(a)) but requires twice as many trainable parameters and double the
training time. Also, the reconstructed signals from both latent space sizes
exhibit visually similar behavior. This indicates that the size of 128 already
contains the most important features, and further increases in size may not
result in significant improvements. This provides an initial interpretation of
this model by filtering out the unimportant components.

The interpretability becomes more evident when employing a basic physical
model introduced in [4]. The frequency of the signal correlates to the sound
velocity, its decay rate to the sound attenuation coefficient, and its phase,
among other factors, to the cell’s adhesion characteristics to the substrate.
These fitted parameters can be visualized as images to highlight important
structural features of cells that may be associated with the disease. Is
possible to see that the contrast before and after denoising changes (see
Figure 4(b)), reveals features that were previously unclear. Specifically, the
de-noised phase component provides a much clearer definition of the cell’s
boundary and improves contrast for spatial structures within the nucleus.
Visualizing these physical parameters on a scatter plot (see Figure 4(c)),
allows us to discern differences between these characteristics across
different classes. Before denoising, the scatter plot appears diffuse with
significant overlap between pixel class clusters. Following denoising, the
clusters are much more tightly packed with less overlap. To quantify this
enhancement, we trained different conventional classification models, namely
Linear, SVM, Kernel, Discriminant and Naive Bayes classifiers from physical
features that have been extracted (velocity, attenuation and phase). Table II
shows the weighted precision of different models trained from 2D/3D features
(where 2D includes velocity and attenuation features and 3D includes velocity,
attenuation and phase features). Compared to 2D classification, where both
original and reconstruction signals show relatively low precision, 3D
classification shows higher precision, especially from the reconstructed
signals, where the weighted precision results are over 90% from all models
(details in Supplementary Document). This means the phase feature contributes
significantly to the classification task but the inherent noise of this
feature in the raw data is too high to allow interpretation.

Figure 4: Explainability of latent space for signal denoising. (a) Denoised
signal reconstructed from latent space with dimension 128 and 1024. (b)
Feature (Frequency, Attenuation and Phase) map of normal and cancerous cell
examples from the reconstructed and original signal. (c) Feature (Frequency,
Attenuation and Phase) cluster from the reconstructed and original signal,
where blue represents background, red represents cancer and green represents
normal. TABLE II: Weighted precision of classifications different conventional
classification models trained from physical features

|  | Conventional classification models
---|---|---
Feature size | Data Input | Linear | SVM | Kernel | Discriminant | Naive Bayes
| Original | 0.5581 | 0.6695 | 0.5864 | 0.6595 | 0.6827
2D | Reconstructed | 0.4701 | 0.6696 | 0.6293 | 0.6429 | 0.6893
| Original | 0.6826 | 0.7774 | 0.7176 | 0.7608 | 0.7773
3D | Reconstructed | 0.9000 | 0.9202 | 0.9335 | 0.9069 | 0.9035

##  IV DISCUSSION

We addressed the ‘batch effect’ problem that arises during the inter-batch
classification of time-resolved phonon-derived signals, where the baseline
model and AI-based Variational Encoder (VE) models have shown limited capacity
to distinguish the cancerous and normal cells due to confounding factors. To
overcome this, we developed a neural network-based approach: multi-task
conditional NN, incorporating a conditional reference operator at the input
and two separate classifiers trained simultaneously to learn the
distinguishing features of healthy and cancer while being unable to determine
batch ID. Using this approach, we successfully achieved removing batch
confounding variables by adding conditioning and marginalization, in
accordance with established results from Bayesian network theory. Further, we
show that having removed this confounding effect we can reconstruct denoised
versions of the original signals with interpretable physical features useful
for analysis of tissue disease states.

There are several major advantages to our neural network approach over
existing approaches. First, the prediction time is very fast (typically
$\sim$0.5s) even when dealing with multiple batches of data. Also, the use of
background conditioning means it only needs to be trained once per
application, regardless of changes in experimental equipment or days. This
further makes it a feasible approach for future data expansion and other
applications with larger datasets. Second, the only information required for
training the model is a batch identifier that can be used to achieve
randomization, though more advanced models could potentially incorporate other
experimental variables to examine the influence of factors such as
temperature. This is important in practical clinical scenarios where recording
batch information can be challenging and the limited information will lead to
poor calibration and classification performance. Finally, the model generates
an interpretable low-dimensional shared latent space which can be represented
in cluster visualization and used for effective signal denoising. Further
analysis can explore the correlation between denoised signal characteristics
(e.g. its frequency, attenuation, phase, etc.) and different classes of cells.
In particular, we find that the phase information of our time-domain signals
differs significantly between healthy and cancer cells. This difference is not
visible in the raw data but becomes clear in reconstructed denoised versions.
This may imply that cell adhesion to the substrate, one factor that influences
such a phase shift, differs significantly between healthy and cancer cells.
There have been a small number of studies [35] previously that have suggested
this but our analysis suggests this could be an important feature that is
detectable using our techniques. Our AI approach enables this to be used as an
explainable future that may contribute to future applications in biomedical
science or medical diagnostics.

Several improvements should be considered when further exploring this
approach. Firstly, the dataset used in this paper is collected over 8
experimental days, with 8 batches of data in total, containing a relatively
limited diversity of the samples with a total number of 80 cells ($\sim$
300,000 input time traces). Although our approach has been validated with high
calibration and classification accuracy, assessing the model’s robustness and
generalizability requires testing it with a larger and more diverse dataset.
This can be achieved by incorporating additional data collected on various
experimental days, under different settings, and involving diverse cell types.
Secondly, to implement our approach, it is necessary to know one pixel of the
background for each cell as the reference operator before training the multi-
task conditional model. This can be solved by developing a reliable
experimental background extractor that can consistently generate the time
trace of one pixel of background every time during the measurement.

This AI-driven approach shows great promise for future explainable signal
analysis in the area of medical healthcare. For example, It could be applied
to explore drugs that modulate elasticity and examine their effects on key
physical parameters over time. Also, it could facilitate a better
understanding of complex relationships within biological systems (e.g.
substrate adhesion), enabling the prediction of dynamic changes over time by
employing this time-series forecasting AI model, paving the way for innovative
treatments and therapies.

## References

  * [1] Joanna Zemła, Joanna Danilkiewicz, Barbara Orzechowska, Joanna Pabijan, Sara Seweryn, and Małgorzata Lekka.  Atomic force microscopy as a tool for assessing the cellular elasticity and adhesiveness to identify cancer cells and tissues.  In Seminars in cell & developmental biology, volume 73, pages 115–124. Elsevier, 2018.
  * [2] Kozaburo Hayashi and Mayumi Iwata.  Stiffness of cancer cells measured with an afm indentation method.  Journal of the mechanical behavior of biomedical materials, 49:105–111, 2015.
  * [3] Martha E Grady, Russell J Composto, and David M Eckmann.  Cell elasticity with altered cytoskeletal architectures across multiple cell types.  Journal of the mechanical behavior of biomedical materials, 61:197–207, 2016.
  * [4] Fernando Pérez-Cota, Giovanna Martínez-Arellano, Salvatore La Cavera III, William Hardiman, Luke Thornton, Rafael Fuentes-Domínguez, Richard J Smith, Alan McIntyre, and Matt Clark.  Classification of cancer cells at the sub-cellular level by phonon microscopy using deep learning.  Scientific Reports, 13(1):16228, 2023.
  * [5] Rafael Fuentes-Domínguez, Mengting Yao, William Hardiman, Kerry Setchfield, Fernando Pérez-Cota, Richard J Smith, Matt Clark, et al.  Parallel imaging with phonon microscopy using a multi-core fibre bundle detection.  Photoacoustics, 31:100493, 2023.
  * [6] Fernando Pérez-Cota, Richard J Smith, Hany M Elsheikha, and Matt Clark.  New insights into the mechanical properties of acanthamoeba castellanii cysts as revealed by phonon microscopy.  Biomedical optics express, 10(5):2399–2408, 2019.
  * [7] Fernando Pérez-Cota, Richard J Smith, Emilia Moradi, Leonel Marques, Kevin F Webb, and Matt Clark.  High resolution 3d imaging of living cells with sub-optical wavelength phonons.  Scientific Reports, 6(1):39326, 2016.
  * [8] Richard J Smith, Fernando Perez Cota, Leonel Marques, Xuesheng Chen, Ahmet Arca, Kevin Webb, Jonathon Aylott, Micheal G Somekh, and Matt Clark.  Optically excited nanoscale ultrasonic transducers.  The Journal of the Acoustical Society of America, 137(1):219–227, 2015.
  * [9] Robert Prevedel, Alba Diz-Muñoz, Giancarlo Ruocco, and Giuseppe Antonacci.  Brillouin microscopy: an emerging tool for mechanobiology.  Nature methods, 16(10):969–977, 2019.
  * [10] Charles W Ballmann, Jonathan V Thompson, Andrew J Traverso, Zhaokai Meng, Marlan O Scully, and Vladislav V Yakovlev.  Stimulated brillouin scattering microscopic imaging.  Scientific Reports, 5(1):18139, 2015.
  * [11] AJ Rice, E Cortes, D Lachowski, BCH Cheung, SA Karim, JP Morton, and A Del Rio Hernandez.  Matrix stiffness induces epithelial–mesenchymal transition and promotes chemoresistance in pancreatic cancer cells.  Oncogenesis, 6(7):e352–e352, 2017.
  * [12] Yiming Gao, Krzysztof J Geras, Alana A Lewin, and Linda Moy.  New frontiers: an update on computer-aided diagnosis for breast imaging in the age of artificial intelligence.  AJR. American journal of roentgenology, 212(2):300, 2019.
  * [13] Krzysztof J Geras, Ritse M Mann, and Linda Moy.  Artificial intelligence for mammography and digital breast tomosynthesis: current concepts and future perspectives.  Radiology, 293(2):246–259, 2019.
  * [14] Tomoyuki Fujioka, Mio Mori, Kazunori Kubota, Jun Oyama, Emi Yamaga, Yuka Yashima, Leona Katsuta, Kyoko Nomura, Miyako Nara, Goshi Oda, et al.  The utility of deep learning in breast ultrasonic imaging: a review.  Diagnostics, 10(12):1055, 2020.
  * [15] Saliha Zahoor, Ikram U Lali, Muhammad Attique Khan, Kashif Javed, and Waqar Mehmood.  Breast cancer detection and classification using traditional computer vision techniques: a comprehensive review.  Current medical imaging, 16(10):1187–1200, 2020.
  * [16] W Evan Johnson, Cheng Li, and Ariel Rabinovic.  Adjusting batch effects in microarray expression data using empirical bayes methods.  Biostatistics, 8(1):118–127, 2007.
  * [17] Gordon K Smyth and Terry Speed.  Normalization of cdna microarray data.  Methods, 31(4):265–273, 2003.
  * [18] Po-Yuan Tung, John D Blischak, Chiaowen Joyce Hsiao, David A Knowles, Jonathan E Burnett, Jonathan K Pritchard, and Yoav Gilad.  Batch effects and the effective design of single-cell gene expression studies.  Scientific reports, 7(1):39921, 2017.
  * [19] Douglas M Bates.  lme4: Mixed-effects modeling with r, 2010.
  * [20] Laleh Haghverdi, Aaron TL Lun, Michael D Morgan, and John C Marioni.  Batch effects in single-cell rna-sequencing data are corrected by matching mutual nearest neighbors.  Nature biotechnology, 36(5):421–427, 2018.
  * [21] David R Hardoon, Sandor Szedmak, and John Shawe-Taylor.  Canonical correlation analysis: An overview with application to learning methods.  Neural computation, 16(12):2639–2664, 2004.
  * [22] A Lun.  Further mnn algorithm development.  GitHub repository, 2019.
  * [23] Brian Hie, Bryan Bryson, and Bonnie Berger.  Efficient integration of heterogeneous single-cell transcriptomes using scanorama.  Nature biotechnology, 37(6):685–691, 2019.
  * [24] Krzysztof Polański, Matthew D Young, Zhichao Miao, Kerstin B Meyer, Sarah A Teichmann, and Jong-Eun Park.  Bbknn: fast batch alignment of single cell transcriptomes.  Bioinformatics, 36(3):964–965, 2020.
  * [25] Tim Stuart, Andrew Butler, Paul Hoffman, Christoph Hafemeister, Efthymia Papalexi, William M Mauck, Yuhan Hao, Marlon Stoeckius, Peter Smibert, and Rahul Satija.  Comprehensive integration of single-cell data.  Cell, 177(7):1888–1902, 2019.
  * [26] Romain Lopez, Jeffrey Regier, Michael B Cole, Michael I Jordan, and Nir Yosef.  Deep generative modeling for single-cell transcriptomics.  Nature methods, 15(12):1053–1058, 2018.
  * [27] Tongxin Wang, Travis S Johnson, Wei Shao, Zixiao Lu, Bryan R Helm, Jie Zhang, and Kun Huang.  Bermuda: a novel deep transfer learning method for single-cell rna sequencing batch correction reveals hidden high-resolution cellular subtypes.  Genome biology, 20(1):1–15, 2019.
  * [28] Xiangjie Li, Kui Wang, Yafei Lyu, Huize Pan, Jingxiao Zhang, Dwight Stambolian, Katalin Susztak, Muredach P Reilly, Gang Hu, and Mingyao Li.  Deep learning enables accurate clustering with batch effect removal in single-cell rna-seq analysis.  Nature communications, 11(1):2338, 2020.
  * [29] Mohammad Lotfollahi, F Alexander Wolf, and Fabian J Theis.  scgen predicts single-cell perturbation responses.  Nature methods, 16(8):715–721, 2019.
  * [30] Qianqian Song, Jing Su, and Wei Zhang.  scgcn is a graph convolutional networks algorithm for knowledge transfer in single cell omics.  Nature communications, 12(1):3826, 2021.
  * [31] Judea Pearl.  Causal inference in statistics: An overview.  2009\.
  * [32] KJ Jager, C Zoccali, A Macleod, and FW Dekker.  Confounding: what it is and how to deal with it.  Kidney international, 73(3):256–260, 2008.
  * [33] Leland McInnes, John Healy, and James Melville.  Umap: Uniform manifold approximation and projection for dimension reduction.  arXiv preprint arXiv:1802.03426, 2018.
  * [34] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  Journal of Machine Learning Research, 12:2825–2830, 2011.
  * [35] James Varani, Paul E McKeever, Vishva Dixit, Thomas E Carey, and Suzanne EG Flgiel.  Tumor type-specific differences in cell-substrate adhesion among human tumor cell lines.  International journal of cancer, 39(3):397–403, 1987.


# Quantum confinement in $\alpha$-Grushin planes

###### Abstract

We consider here a family of singular Laplace-Beltrami operators, focussing
our attention on the problem of so-called quantum confinement on the half-
plane equipped with Riemannian metrics of Grushin type degenerate at the
boundary. By introducing a costant-fiber direct integral scheme we are able to
rigorously characterize the presence or absence of self-adjointness of these
operators. We also compare our technique and results with the already studied
problem of quantum confinement on the half-cylinder.

Eugenio Pozzoli111Inria, Sorbonne Université, Université de Paris, CNRS,
Laboratoire Jacques- Louis Lions, Paris, France (e-mail:
<EMAIL_ADDRESS>
## Introduction

The quantum evolution on a Riemannian oriented manifold $(M,g,\omega)$ can be
studied via the Schrödinger equation

$i\dfrac{d}{dt}\psi=-\Delta_{g,\omega}\psi+V\psi,\qquad\psi(t)\in
L^{2}(M,\omega).$ (1)

Several objects are involved in (1): $g$ is a Riemannian metric on $M$, that
is, a symmetric, positive definite $(0,2)$-tensor, $\omega$ is a volume form
on $M$, that is, a non vanishing top differential form on $M$, $V$ is a real-
valued function on $M$ (the potential) and
$\Delta_{g,\omega}:=\text{div}_{\omega}\circ\text{grad}_{g}$ is a Laplace
operator which acts on the Hilbert space $L^{2}(M,\omega)$ with domain
$C_{c}^{\infty}(M)$, the compactly supported smooth functions on $M$, and
which depends on both the metric and the volume.
Here we consider a geometric Laplacian, that is called the Laplace-Beltrami
operator of $(M,g)$, which corresponds to the choice of the Riemannian volume
form $\omega_{g}$. In this case, one can write
$\Delta_{g,\omega_{g}}=:\Delta_{g}$. For a general introduction to the
Laplace-Beltrami operator and geometric analysis see, e.g., [Jos17].
If the Schrödinger operator $H:=-\Delta_{g}+V$ is essentially self-adjoint on
its domain $C_{c}^{\infty}(M)$, then the evolution is uniquely described by
the propagator $e^{-it\overline{H}}$, $t\in\mathbb{R}$, where $\overline{H}$
is the closure of $H$, and also the unique self-adjoint extension of $H$.
If $H$ is not essentially self-adjoint on $C_{c}^{\infty}(M)$, it admits self-
adjoint extensions since it is a real operator, i.e., it commutes with complex
conjugation ([RS75, Theorem X.3]). In this case the extension is not unique
and thus the quantum dynamics are not uniquely determined by $\Delta_{g}$ and
$V$. In some sense, there is no natural quantum evolution. In what follows we
will restrict ourselves to the case $V=0$.
For example, one can consider the usual euclidean Laplacian
$\Delta=\sum_{j=1}^{n}\dfrac{\partial^{2}}{\partial x_{j}^{2}}$ on an open
domain $A\subset\mathbb{R}^{n}$, without potential. Here, $\Delta$ is never
essentially self-adjoint on $C_{c}^{\infty}(A)$, basically because it induces
a Sobolev $H_{0}^{2}$ norm on the function space where it is defined. Its
self-adjoint extensions correspond to the choice of suitable boundary
conditions for the functions in the domain of $\Delta$.
The situation becomes more complex for domains $A$ of $\mathbb{R}^{n}$
equipped with non a non euclidean metric $g$. In this case, the geometric
Laplacian $\Delta_{g}$ can be essentially self-adjoint on $C_{c}^{\infty}(M)$.
This is basically due to some singularity of the Riemannian structure $(A,g)$
at the boundary $\partial A$ of $A$, which behaves like a confining _effective
potential_. When this occurs, we say that the particle is naturally _confined_
in $A$ during its quantum dynamics, since there exists a unique evolution
$e^{-it\overline{\Delta_{g}}}$ on $L^{2}(A,\omega_{g})$ without need of
specifying boundary conditions. This is also called geometric confinement
since it is only due to the geometric Laplacian $\Delta_{g}$ and not to a
potential $V$.
So, the essential self-adjointness of $\Delta_{g}$ on $C_{c}^{\infty}(M)$ is
usually called _geometric quantum confinement_ on $M$: no quantum information
escapes from $M$. For further details, see for example [BP16, BL13, FPR20,
GMP19].

##  1 Grushin manifolds

Here we are interested in studying the presence or absence of geometric
quantum confinement in the family of Riemannian manifolds
$M_{\alpha}:=(M,g_{\alpha})$, $\alpha\in\mathbb{R}$,

$M:=\\{(x,y)\in\mathbb{R}^{2}|x>0\\}=\mathbb{R}^{+}\times\mathbb{R},\quad
g_{\alpha}:=dx^{2}+\dfrac{1}{x^{2\alpha}}dy^{2}=\begin{pmatrix}1&0\\\
0&1/x^{2\alpha}\end{pmatrix},$

that is, the half-plane equipped with a family of Riemannian metrics
$g_{\alpha}$.

The curvature of $M_{\alpha}$ is given by
$K_{\alpha}(x)=-\dfrac{\alpha(\alpha+1)}{x^{2}}$, thus the Riemannian
structure is degenerate at the boundary $\partial M$, that is, the $y$-axis,
for every $\alpha$ except for $\alpha=0,1$.
The Riemannian volume form of $M_{\alpha}$ reads

$\omega_{\alpha}=\sqrt{\text{det}g_{\alpha}}dx\wedge
dy=\dfrac{1}{x^{\alpha}}dx\wedge dy,$

and the Laplace-Beltrami operator of $M_{\alpha}$ is

$\Delta_{\alpha}=\dfrac{\partial^{2}}{\partial
x^{2}}+x^{2\alpha}\dfrac{\partial^{2}}{\partial
y^{2}}-\dfrac{\alpha}{x}\dfrac{\partial}{\partial x},$ (2)

acting on the Hilbert space
$L^{2}\Big{(}\mathbb{R}^{+}\times\mathbb{R},\dfrac{1}{x^{\alpha}}dxdy\Big{)}$,
with domain $C_{c}^{\infty}(\mathbb{R}^{+}\times\mathbb{R})$, the smooth
functions on the half-plane compactly supported away from the $y$-axis. In the
work [GMP19] the essential self-adjointness of the operators (2) is studied
depending on $\alpha\in[0,\infty)$. These operators where first studied as
examples of hypo-elliptic operators, and these manifolds find applications in
sub-Riemannian geometry since they provide examples of quantum confinement in
almost-Riemannian structure. For further details on almost-Riemannian and,
more generally, sub-Riemannian structures, see, e.g., [Mon02, ABB20].

Let us briefly recall how the self-adjointness of the operators (2) can be
analyse.
First of all, the operator $\Delta_{\alpha}$ is unitarily equivalent to

$A_{\alpha}:=\dfrac{\partial^{2}}{\partial
x^{2}}-\xi^{2}x^{2\alpha}-\dfrac{\alpha(2+\alpha)}{4x^{2}}$ (3)

via two unitary tranformations:

$U_{\alpha}:L^{2}\Big{(}\mathbb{R}^{+}\times\mathbb{R},\dfrac{1}{x^{\alpha}}dxdy\Big{)}\Rightarrow
L^{2}(\mathbb{R}^{+}\times\mathbb{R},dxdy),\quad\psi\mapsto\dfrac{1}{x^{\alpha/2}}\psi,$

and

$\mathcal{F}_{2}:L^{2}(\mathbb{R}^{+}\times\mathbb{R},dxdy)\Rightarrow
L^{2}(\mathbb{R}^{+}\times\mathbb{R},dxd\xi),$

which is the Fourier tranform in the $y$ variable, that is,

$\mathcal{F}_{2}(\psi)(x,\xi):=\dfrac{1}{\sqrt{2\pi}}\int_{\mathbb{R}}\psi(x,y)e^{-i\xi
y}dy.$

The domain of $A_{\alpha}$ is
$\mathcal{D}(A_{\alpha})=\mathcal{F}_{2}C_{c}^{\infty}(\mathbb{R}^{+}\times\mathbb{R})$,
that is, if $\psi\in\mathcal{D}(A_{\alpha})$, then $\psi(\cdot,\xi)$ is
compactly supported on $\mathbb{R}^{+}$, for every $\xi$, while
$\psi(x,\cdot)$ is a Schwartz function on $\mathbb{R}$, for every $x$. This
transformation is convenient since it allow us to decompose the action of the
operator $A_{\alpha}$ in fiber actions.

##  2 Constant-fiber direct integral scheme

More precisely, we identify the Hilbert spaces

$L^{2}(\mathbb{R}^{+}\times\mathbb{R},dxd\xi)\cong
L^{2}(\mathbb{R},d\xi;L^{2}(\mathbb{R}^{+},dx))=:\int_{\mathbb{R}}^{\oplus}L^{2}(\mathbb{R}^{+},dx)d\xi=:\mathcal{H}$

thus we think of $L^{2}(\mathbb{R}^{+}\times\mathbb{R},dxd\xi)$ as
$L^{2}(\mathbb{R}^{+},dx)$-valued square-integrable functions of
$\xi\in\mathbb{R}$.

Consider, for any $\xi$, the fiber operator

$A_{\alpha}(\xi):=\dfrac{d^{2}}{dx^{2}}-\xi^{2}x^{2\alpha}-\dfrac{\alpha(2+\alpha)}{4x^{2}}$

which acts on the fiber space $L^{2}(\mathbb{R}^{+},dx)$ with domain
$C_{c}^{\infty}(\mathbb{R}^{+})$. Then, let us define the operator

$B_{\alpha}:=\int_{\mathbb{R}}^{\oplus}\overline{A_{\alpha}(\xi)}d\xi,$ (4)

whose action is defined by
$(B_{\alpha}\psi)(\xi):=\overline{A_{\alpha}(\xi)}\psi(\xi)$, for any
$\psi\in\mathcal{D}(B_{\alpha})$, where the domain is

$\mathcal{D}(B_{\alpha}):=\\{\psi\in\mathcal{H}|\psi(\xi)\in\mathcal{D}(\overline{A_{\alpha}(\xi)})\text{
for a.e. $\xi$, and
}\int_{\mathbb{R}}\|\overline{A_{\alpha}(\xi)}\psi(\xi)\|_{L^{2}(\mathbb{R}^{+},dx)}^{2}d\xi<\infty\\}.$

Having introduced this new operator (4), let us describe some of its
properties and explain its relation with (3).

######  Theorem 1.

  1. .

  2. 1.

$B_{\alpha}$ is a closed symmetric extension of $A_{\alpha}$.

  3. 2.

If $A_{\alpha}(\xi)$ is essentially self-adjoint for a.e. $\xi\in\mathbb{R}$,
then $B_{\alpha}$ is self-adjoint.

  4. 3.

If $B_{\alpha}$ is self-adjoint, then $A_{\alpha}(\xi)$ is essentially self-
adjoint for a.e. $\xi\in\mathbb{R}$.

  5. 4.

If $B_{\alpha}$ is self-adjoint, then $A_{\alpha}^{*}\subset B_{\alpha}$.

Property $2$ is well known, [RS78, Theorem XIII.85(i)]. The rest of Theorem 1
is proved in [GMP19].
Finally one can deduce if $A_{\alpha}$ is essentially self-adjoint or not via
the self-adjointness of the fiber operators $\overline{A_{\alpha}(\xi)}$.

######  Theorem 2.

$A_{\alpha}$ is essentially self-adjoint if and only if $A_{\alpha}(\xi)$ is
essentially self-adjoint for a.e. $\xi\in\mathbb{R}$.

###### Proof.

We use the properties established in Theorem 1. Assume that there exists a set
$E\subset\mathbb{R}$ of strictly positive measure such that if $\xi\in E$ then
$A_{\alpha}(\xi)$ is not essentially self-adjoint. Then, $B_{\alpha}$ is not
self-adjoint. Moreover, one always has that $\overline{A_{\alpha}}\subset
B_{\alpha}$. Now, if $A_{\alpha}$ was essentially self-adjoint, it could not
be $\overline{A_{\alpha}}=B_{\alpha}$, because this would violate the lack of
self-adjointness of $B_{\alpha}$. But it could not happen either that
$B_{\alpha}$ is a proper extension of $\overline{A_{\alpha}}$, because self-
adjoint operators are maximally symmetric. Therefore, $A_{\alpha}$ is not
essentially self-adjoint.

Conversely, assume that $A_{\alpha}(\xi)$ is essentially self-adjoint for a.e.
$\xi\in\mathbb{R}$. Then $B_{\alpha}$ is self-adjoint, and hence
$A_{\alpha}^{*}\subset B_{\alpha}$. By taking the adjoint, we obtain
$\overline{A_{\alpha}}=A_{\alpha}^{**}\supset B_{\alpha}^{*}=B_{\alpha}$,
which, together with $\overline{A_{\alpha}}\subset B_{\alpha}$, implies
$\overline{A_{\alpha}}=B_{\alpha}$. The conclusion is that $A_{\alpha}$ is
essentially self-adjoint. ∎

##  3 The compactification of $M_{\alpha}$

In [BP16] quantum confinement has been studied for the cylindric version of
$M_{\alpha}$, that is,

$\widetilde{M}:=\\{(x,y)\in\mathbb{R}\times S^{1}|x>0\\}=\mathbb{R}^{+}\times
S^{1},\qquad\widetilde{M}_{\alpha}=(\widetilde{M},g_{\alpha}).$

In this case, the Hilbert space is
$l^{2}(\mathbb{Z};L^{2}(\mathbb{R}^{+},dx))$ and the inquired operator is

$\widetilde{A_{\alpha}}=\dfrac{\partial^{2}}{\partial
x^{2}}-k^{2}x^{2\alpha}-\dfrac{\alpha(2+\alpha)}{4x^{2}},\qquad
k\in\mathbb{Z},$

with domain $\mathcal{F}_{2}C_{c}^{\infty}(\mathbb{R}^{+}\times S^{1})$, where

$\mathcal{F}_{2}(\psi)(x,k):=\dfrac{1}{\sqrt{2\pi}}\int_{S^{1}}\psi(x,\theta)e^{-ik\theta}d\theta.$

Let us introduce the fiber operators

$\widetilde{A_{\alpha}(k)}:=\dfrac{d^{2}}{dx^{2}}-k^{2}x^{2\alpha}-\dfrac{\alpha(2+\alpha)}{4x^{2}},\qquad
k\in\mathbb{Z},$

as an operator on $L^{2}(\mathbb{R}^{+},dx)$ with domain
$C_{c}^{\infty}(\mathbb{R}^{+})$.

Hence the constant fiber direct integral reduces to a direct sum
$\widetilde{B_{\alpha}}:=\bigoplus_{k\in\mathbb{Z}}\overline{\widetilde{A_{\alpha}(k)}}$

basically due to the discreteness of the spectrum of
$\dfrac{\partial^{2}}{\partial\theta^{2}}$ as an operator on $L^{2}(S^{1})$,
with domain

$\mathcal{D}(B_{\alpha}):=\\{(\psi_{k})_{k\in\mathbb{Z}}\in
l^{2}(\mathbb{Z};L^{2}(\mathbb{R}^{+},dx))|\psi_{k}\in\mathcal{D}(\overline{A_{\alpha}(k)})\text{
for every k and
}\sum_{k\in\mathbb{Z}}\|\overline{A_{\alpha}(k)}\psi_{k}\|_{L^{2}(\mathbb{R}^{+},dx)}^{2}<\infty\\}.$

Actually in this compactified case one does not need all the previous
construction of constant fiber direct integral. Indeed it suffices to define
$\widetilde{C_{\alpha}}:=\bigoplus_{k\in\mathbb{Z}}\widetilde{A_{\alpha}(k)}$
as an operator on $l^{2}(\mathbb{Z};L^{2}(\mathbb{R}^{+},dx))$ with domain all
the sequences $L^{2}(\mathbb{R}^{+},dx)$-valued which are zero except for a
finite number of components. Then one has that
$\overline{\widetilde{C_{\alpha}}}=\overline{\widetilde{A_{\alpha}}}$ and the
essential self-adjointness of $\widetilde{C_{\alpha}}$ is controlled via the
following lemma [BP16, Proposition 2.3].

######  Lemma 1.

The deficency indices of $\widetilde{C_{\alpha}}$ are
$n_{\pm}(\widetilde{C_{\alpha}})=\sum_{k\in\mathbb{Z}}n_{\pm}(\widetilde{A_{\alpha}(k)})$.

######  Remark 1.

Notice that this construction does not make sense for $M_{\alpha}$, since the
domain of $B_{\alpha}$ made by sequences which are zero except for a finite
numbers of components would contain just the function identically zero. Thus,
we strongly need the constant fiber direct integral construction in the non-
compact case.

Owning to the analysis on the fiber operators $\widetilde{A_{\alpha}(k)}$ via
limit-point limit-circle method (see, e.g., [RS75, Appendix to X.1]), the fact
that $\overline{\widetilde{C_{\alpha}}}=\overline{\widetilde{A_{\alpha}}}$ and
using Lemma 1, the description of quantum confinement on
$\widetilde{M}_{\alpha}$ can be sum up in the following

######  Theorem 3.

$\widetilde{A_{\alpha}}$ is essentially self-adjoint if and only if
$\alpha\in(-\infty,-3]\cup[1,\infty)$.
When $\alpha\in(-3,1)$, the lack of self-adjointness is due to:

  * •

all the Fourier $k$’s components if $\alpha\in(-1,1)$;

  * •

the only $0^{\text{th}}$ Fourier component if $\alpha\in(-3,-1]$.

In some sense, for $\alpha\in(-3,-1]$, the only information which escapes from
$\mathbb{R}^{+}\times S^{1}$ is the average over $S^{1}$ of the wave function,
that is, $\psi_{0}(x)=\dfrac{1}{2\pi}\int_{S^{1}}\psi(x,\theta)d\theta.$ For a
proof of Theorem 3, see [BP16, Theorem 1.6].

##  4 Quantum confinement on $M_{\alpha}$

We now develop explicitly the analysis of self-adjointness via limit-point
limit-circle method on the fiber space $L^{2}(\mathbb{R}^{+},dx)$ and then we
integrate the result using Theorem 2 to obtain the characterization of quantum
confinement on $M_{\alpha}$ depending on $\alpha$. We shall see how the
constant fiber direct integral construction differs from a direct sum,
obtaining indeed different regimes of $\alpha$, with respect to Theorem 3.

######  Theorem 4.

$A_{\alpha}$ is essentially self-adjoint if and only if
$\alpha\in(-\infty,-1)\cup[1,\infty)$.
When $\alpha\in[-1,1)$, the lack of self-adjointness is due to:

  * •

all the Fourier modes $\xi\in\mathbb{R}$ if $\alpha\in(-1,1)$;

  * •

the Fourier modes $\xi\in(-1,1)$ if $\alpha=-1$.

It is worth mentioning that, when $\alpha\in(-3,-1)$, the only Fourier mode
which is not essentially self-adjoint corresponds to $\xi=0$, but this does
not affect the self-adjointness of $B_{\alpha}$, because $\\{0\\}$ has zero
measure and hence it does not affect the direct integral operator. We remark
that the self-adjointness of $A_{\alpha}$, when $\alpha\geq 0$, was already
classified in [GMP19, Corollary 3.8]. Here we study also the case $\alpha<0$.

###### Proof.

Thanks to Theorem 2, $A_{\alpha}$ is essentially self-adjoint if and only if
$A_{\alpha}(\xi)$ is essentially self-adjoint as an operator on
$L^{2}(\mathbb{R}^{+},dx)$, for a.e. $\xi\in\mathbb{R}$. Now the self-
adjointness of each $A_{\alpha}(\xi)$, $\xi\in\mathbb{R}$, can be studied by
computing its deficency indices. Notice that $-A_{\alpha}(\xi)$ is a Sturm-
Liouville operator (see, e.g., [Sch12, Section 15.1]), i.e., it has the form

$A_{\alpha}(\xi)=\dfrac{d^{2}}{dx^{2}}+V_{\alpha,\xi},\qquad
V_{\alpha,\xi}:=-\xi^{2}x^{2\alpha}-\dfrac{\alpha(2+\alpha)}{4x^{2}}.$

Thus, by Weyl Theorem [RS75, Theorem X.7], we can compute its deficency
indices by checking if $V_{\alpha,\xi}$ is limit-point or limit-circle in $0$
and $\infty$.
Hence, we have to study the behaviour of the two linearly independent
solutions of

$\psi^{\prime\prime}(x)-\xi^{2}x^{2\alpha}\psi(x)-\dfrac{\alpha(2+\alpha)}{4x^{2}}\psi(x)=0$
(5)

near $0$ and $\infty$, that is, checking whether or not they are
$L^{2}((0,1),dx)$ and $L^{2}((1,\infty),dx)$.

By applying a unitary transformation
$U_{\alpha}:L^{2}(\mathbb{R}^{+},dx)\Rightarrow
L^{2}\Big{(}\mathbb{R}^{+},\dfrac{1}{x^{\alpha}}dx\Big{)}$,
$\psi\mapsto\dfrac{1}{x^{-\alpha/2}}\psi,$ this is equivalent to studying
whether or not the two linearly independent solutions of

$\psi^{\prime\prime}(x)-\xi^{2}x^{2\alpha}\psi(x)-\dfrac{\alpha}{x}\psi^{\prime}(x)=0$
(6)

are $L^{2}\Big{(}(0,1),\dfrac{1}{x^{\alpha}}dx\Big{)}$ and
$L^{2}\Big{(}(1,\infty),\dfrac{1}{x^{\alpha}}dx\Big{)}$.
Let us start with $\xi=0$ and $\alpha\neq-1$, then the solutions to (6) are
$\psi_{1}=1$ and $\psi_{2}(x)=x^{1+\alpha}.$ They are both in
$L^{2}\Big{(}(0,1),\dfrac{1}{x^{\alpha}}dx\Big{)}$ if and only if
$\alpha\in(-3,1)$. Moreover, $\psi_{1},\psi_{2}\notin
L^{2}\Big{(}(1,\infty),\dfrac{1}{x^{\alpha}}dx\Big{)}$ for every
$\alpha\in\mathbb{R}$. Analogously, for $\xi=0$ and $\alpha=-1$ the solutions
to (6) are $\psi_{1}=1,\psi_{2}(x)=log(x)$, and both of them belong to
$L^{2}\Big{(}(0,1),xdx\Big{)}$, while no one of them belongs to
$L^{2}\Big{(}(1,\infty),xdx\Big{)}$. Thus,

  * •

if $\alpha\in(-3,1)$, then $V_{\alpha,0}$ is limit-circle in $0$ and limit-
point in $\infty$;

  * •

if $\alpha\notin(-3,1)$, then $V_{\alpha,0}$ is limit-point in $0$ and
$\infty$.

Consider now $\xi\neq 0$, and as before let us start with $\alpha\neq-1$. Then
the solutions to (6) are $\psi_{1}(x)=exp\Big{(}\dfrac{\xi
x^{1+\alpha}}{1+\alpha}\Big{)},\psi_{2}(x)=exp\Big{(}-\dfrac{\xi
x^{1+\alpha}}{1+\alpha}\Big{)}$. If $\alpha>-1$, $\psi_{1}$ and $\psi_{2}$ are
bounded near $0$, and hence $0$ is limit-circle when $\alpha<1$. One of them
grows exponentially at $\infty$, and hence $\infty$ is always limit-point for
$\alpha>-1$.
If $\alpha<-1$, $\psi_{1}$ and $\psi_{2}$ converge to $1$ as
$x\Rightarrow\infty$ and the measure $\dfrac{1}{x^{\alpha}}dx$ explodes, and
hence $\infty$ is always limit-point when $\alpha<-1$, while one of them
diverges exponentially in $0$, and hence $0$ is always limit-point when
$\alpha<-1$.
If $\alpha=-1$, the solutions of (6) are $\psi_{1}(x)=x^{\xi}$,
$\psi_{2}(x)=x^{-\xi}$. One of them diverges when $x\Rightarrow\infty$ and the
measure $xdx$ diverges, hence $\infty$ is limit-point, while
$\psi_{1},\psi_{2}\in L^{2}((0,1),xdx)$ if and only if $\xi\in(-1,1)$, that
is, $0$ is limit-circle if $\xi\in(-1,1)$ and limit-point if
$\xi\notin(-1,1)$.
Summing up, and using that $A_{\alpha}(\xi)$ is essentially self-adjoint if
and only if $0$ and $\infty$ are limit-point for $V_{\alpha,\xi}$, that is,
the Weyl Theorem, we obtain

  * •

if $\alpha\geq 1$, then $A_{\alpha}(\xi)$ is essentially self-adjoint, for
every $\xi\in\mathbb{R}$;

  * •

if $-1<\alpha<1$, then $A_{\alpha}(\xi)$ is not essentially self-adjoint, for
every $\xi\in\mathbb{R}$;

  * •

if $\alpha=-1$, then $A_{\alpha}(\xi)$ is not essentially self-adjoint for
$\xi\in(-1,1)$ and it is essentially self-adjoint if $\xi\notin(-1,1)$;

  * •

if $-3<\alpha<-1$, then $A_{\alpha}(0)$ is not essentially self-adjoint and
$A_{\alpha}(\xi)$ is essentially self-adjoint for every $\xi\neq 0$;

  * •

if $\alpha\leq-3$, then $A_{\alpha}(\xi)$ is essentially self-adjoint for
every $\xi\in\mathbb{R}$.

∎

Acknowledgement The project leading to this publication has received funding
from the European Union’s Horizon 2020 research and innovation programme under
the Marie Sklodowska-Curie grant agreement no. 765267 (QuSCo). Also it was
supported by the ANR projects SRGI ANR-15-CE40-0018 and Quaco
ANR-17-CE40-0007-01.

## References

  * [ABB20] Andrei Agrachev, Davide Barilari, and Ugo Boscain, _A comprehensive introduction to sub-Riemannian geometry_ , Cambridge Studies in Advanced Mathematics, vol. 181, Cambridge University Press, Cambridge, 2020, From the Hamiltonian viewpoint, With an appendix by Igor Zelenko. MR 3971262
  * [BL13] Ugo Boscain and Camille Laurent, _The Laplace-Beltrami operator in almost-Riemannian geometry_ , Ann. Inst. Fourier (Grenoble) 63 (2013), no. 5, 1739–1770.
  * [BP16] Ugo Boscain and Dario Prandi, _Self-adjoint extensions and stochastic completeness of the Laplace-Beltrami operator on conic and anticonic surfaces_ , J. Differential Equations 260 (2016), no. 4, 3234–3269.
  * [FPR20] Valentina Franceschi, Dario Prandi, and Luca Rizzi, _On the essential self-adjointness of singular sub-Laplacians_ , Potential Anal. 53 (2020), no. 1, 89–112. MR 4117982
  * [GMP19] Matteo Gallone, Alessandro Michelangeli, and Eugenio Pozzoli, _On geometric quantum confinement in Grushin-type manifolds_ , Z. Angew. Math. Phys. 70 (2019), no. 6, Paper No. 158, 17. MR 4019735
  * [Jos17] Jürgen Jost, _Riemannian geometry and geometric analysis_ , seventh ed., Universitext, Springer, Cham, 2017.
  * [Mon02] Richard Montgomery, _A tour of subriemannian geometries, their geodesics and applications_ , Mathematical Surveys and Monographs, vol. 91, American Mathematical Society, Providence, RI, 2002.
  * [RS75] Michael Reed and Barry Simon, _Methods of modern mathematical physics. II. Fourier analysis, self-adjointness_ , Academic Press [Harcourt Brace Jovanovich, Publishers], New York-London, 1975. MR 0493420 (58 #12429b)
  * [RS78] , _Methods of modern mathematical physics. IV. Analysis of operators_ , Academic Press [Harcourt Brace Jovanovich, Publishers], New York-London, 1978.
  * [Sch12] Konrad Schmüdgen, _Unbounded self-adjoint operators on Hilbert space_ , Graduate Texts in Mathematics, vol. 265, Springer, Dordrecht, 2012\. MR 2953553