# Towards Generalizable Referring Image Segmentation via
Target Prompt and Visual Coherence

Yajie Liu1, Pu Ge2, Haoxiang Ma1, Shichao Fan3, Qingjie Liu1,2, Di Huang1,2,
Yunhong Wang1,2 1 School of Computer Science and Engineering, Beihang
University, Beijing, China.2 Hangzhou Innovation Institute, Beihang
University, Beijing, China3 School of Mechanical Engineering and Automation,
Beihang University, Beijing, China

###### Abstract

Referring image segmentation (RIS) aims to segment objects in an image
conditioning on free-from text descriptions. Despite the overwhelming
progress, it still remains challenging for current approaches to perform well
on cases with various text expressions or with unseen visual entities,
limiting its further application. In this paper, we present a novel RIS
approach, which substantially improves the generalization ability by
addressing the two dilemmas mentioned above. Specially, to deal with
unconstrained texts, we propose to boost a given expression with an explicit
and crucial prompt, which complements the expression in a unified context,
facilitating target capturing in the presence of linguistic style changes.
Furthermore, we introduce a multi-modal fusion aggregation module with visual
guidance from a powerful pretrained model to leverage spatial relations and
pixel coherences to handle the incomplete target masks and false positive
irregular clumps which often appear on unseen visual entities. Extensive
experiments are conducted in the zero-shot cross-dataset settings and the
proposed approach achieves consistent gains compared to the state-of-the-art,
e.g., 4.15%, 5.45%, and 4.64% mIoU increase on RefCOCO, RefCOCO+ and ReferIt
respectively, demonstrating its effectiveness. Additionally, the results on
GraspNet-RIS show that our approach also generalizes well to new scenarios
with large domain shifts.

##  I INTRODUCTION

Referring image segmentation (RIS) aims to segment objects in an image
described by a language expression. It has a broad range of applications such
as language-based human-robot interaction [1], image editing [2], etc. In
contrast to the conventional semantic/instance segmentation [3] that segments
objectives in a fixed set of categories, the targets of RIS are given by free-
from expressions with much richer vocabularies and syntactic varieties. An
algorithm for RIS needs to infer the target from the given expression and
segment the corresponding visual entities in the image. Existing state-of-the-
art methods have achieved remarkable performance on multiple benchmarks.
However, there remain many challenges that need to be solved to achieve the
intention of this task.

Figure 1: Predictions from the model pretrained on RefCOCO released by LAVT.
The first row shows the predicted masks for the same object described by
different text descriptions; The second row shows the results for GraspNet,
examples of incomplete target masks and false positive clumps.

Firstly,  _can a well-trained model robustly generalize to unseen language
expressions with varied syntaxes or descriptions of different characteristics
of the object_? Regrettably, the answer is no. As an example demonstrated in
Fig. 1, we describe the elephant tail by two different expressions, then feed
them into the model pretrained on RefCOCO [4] (72.73 oIoU on the validation
set) released by LAVT [5], which we denote as LMODEL. Surprisingly, the two
predicted masks are completely different as shown in the first row of Fig. 1.
Moreover, the performance of LMODEL on RefCOCO+ [4] decreases significantly,
i.e., 72.73 oIoU on RefCOCO _v.s._ 56.86 oIoU on RefCOCO+, even though they
share same image domain and object categories.

Another challenge is,  _can a well-trained model effectively generalize to
unseen visual entities including images with domain shift and objects of
unseen categories_? We observe that the LMODEL exhibits poor performance on
ReferIt [6] (21.22 oIoU) which comprises quite different images and categories
compared to RefCOCO. Moreover, we test the model on GraspNet [7], a general
object grasping dataset in robotics, as shown in the second row of Fig. 1.
There are two typical defects in the predictions: incomplete target masks and
false positive clumps, hindering the generalization of RIS in practical
applications. As discussed above, existing state-of-the-art RIS methods
struggle to “understand” the various free-form text expressions and perform
poorly on unseen visual entities, limiting further application. Collecting
data for each domain is extremely expensive which demands exhaustive
annotations of precise referring expressions and pixel-wise masks. Moreover,
it is impossible to cover all potential textual expressions for visual
entities, even within a fixed set of categories.

In this paper, we aim to enhance the generalization of RIS by addressing the
two challenges mentioned above. Firstly, in order to generalize well to the
unconstrained textual descriptions, it is essential to robustly capture the
true target without biasing towards disturbing items in the expression. To
achieve this, we propose to boost a given expression with an explicit and
crucial prompt. The prompt complements the expression in a unified context
with the core of the target noun extracted from the expression. Based on the
unified form, the model can effectively capture the target, which can reduce
spurious visual-text correlations and promote knowledge learning across
instances with identical prompts. Additionally, the model can learn
characteristics relevant to the target in the descriptions as “residuals” with
the explicit target prompt.

The reason behind the second challenge lies in the pixel-word alignment in
existing methods. They connect textual features with visual features at each
pixel respectively to learn effective cross-modal alignments. However, this
inevitably leads to inconsistent predictions within an object, especially in
unseen visual entities as shown in Fig. 1. To handle this issue, we introduce
a Multi-modal Fusion Aggregation (MFA) module to leverage the visual
coherences based on the well-established generalization capabilities of
aggregation modules in matching literature [8]. Furthermore, we incorporate
the visual knowledge from a powerful pretrained vision model inspired by [9]
to guide the MFA and the decoding of cross-modal features.

To validate the effectiveness of our method, we propose a zero-shot cross-
dataset evaluation protocol similar to the open-vocabulary settings [10].
Comprehensive results demonstrate that our method consistently enhances the
generalization of RIS across all settings. And we produce the GraspNet-RIS by
generating text descriptions for GraspNet using the multi-modal LLM Shikra
[11]. The test results on it show our approach also generalizes well to
practical applications.

Our main contributions can be summarised as follows:

  * •

We introduce a novel task of zero-shot cross-dataset RIS to assess the
generalization of the model to various unconstrained expressions and unseen
visual entities, including images with domain shift and unseen categories.

  * •

We boost the model’s generalization to various text descriptions by
supplementing the given expression with an explicit target prompt in a unified
context. We introduce a multi-modal fusion aggregation module along with
visual knowledge guidance to deal with the incomplete target masks and false
positive clumps that often appear on unseen visual entities.

  * •

We conduct comprehensive experiments and our method achieves consistent gains
compared to the state-of-the-art RIS methods, demonstrating its effectiveness.

##  II RELATED WORK

###  II-A Referring image segmentation

Referring image segmentation has attracted growing attention in the research
community with numerous studies yielding significant results [12], [13], [14],
[15]. Most research works focus on getting effective cross-modal fusion
representations. These efforts have evolved from basic fusion approaches, such
as concatenation [16], to more advanced techniques, such as the utilization of
transformer blocks [17], from decoder-only late fusion to early fusion in the
encoder [5]. Especially, [5] leverages a hierarchical vision Transformer to
jointly embed language and vision features to facilitate cross-modal
alignments. [18] combines sentence-mask alignment with word-pixel alignment to
hierarchically align pixels and masks with linguistic cues. However, the test
results above indicate their generalization abilities do not meet
expectations.

Recently, several studies [19], [20] utilize a Transformer-based encoder-
decoder architecture [21] and transform RIS into the predictions of target
point sets. Nevertheless, achieving competitive results with this architecture
demands a substantial amount of pre-training data due to its complexity. In
contrast to these works, we aim to enhance the generalization of RIS without
using additional training data.

###  II-B Generalization of Referring Image Segmentation

To the best of our knowledge, few works have studied the generalization of
RIS. [22] observes that a model trained on one dataset exhibits poor
performance when applied to a different image domain. They focus on zero-shot
referring expression comprehension using large-scale pretrained vision and
language models such as CLIP [23], ALIGN [24] to avoid the domain shift in
supervised learning. Similarly, [25] transfers from the pretrained knowledge
of CLIP to zero-shot RIS. They exploit an unsupervised mask generator to
generate mask proposals and recognize the target mask by computing the
similarity map using CLIP. Nonetheless, a substantial gap remains between
zero-shot performance and supervised RIS, 24.88 oIoU on the RefCOCO validation
set in [25]  _v.s._ 72.73 oIoU in [5].

CGformer [26] evaluates the generalization of RIS in zero-shot cross-class
settings across the RefCOCO series datasets. In contrast, we propose a more
challenging task, where the model not only deals with unseen classes but also
has to tackle various text expressions and address the domain gap among
different datasets.

Figure 2: The overall pipeline of our proposed method. Given a text
description, we complement the expression with target prompt in a unified
context, then feed it into text encoder. The visual encoding process is the
same as LAVT. We introduce MFA and visual knowledge from a pretrained model to
guide the multi-model fusion and decoding process. The gray parts are
identical to those in LAVT. CMA: cross-modal alignment. MFA: multi-modal
fusion aggregation.

##  III PROPOSED APPROACH

In this paper, we propose to enhance the generalization of RIS from two
perspectives. Firstly, we complement the given expression with an explicit
target prompt in a unified form to address the challenge of various free-from
text descriptions. Secondly, a multi-modal fusion aggregation module along
with visual guidance from a powerful pretrained model is introduced. It
exploits visual coherences to tackle incomplete target masks and noisy clumps
that often appear on unseen visual entities which result in poor performance.
We apply the aforementioned improvements to LAVT [5], constructing a strong
baseline for the generalization of RIS. The details of the overall method are
illustrated in Fig. 2.

###  III-A Problem Formulation

We briefly review LAVT, the first model that conducts cross-modal feature
fusion in the early encoding stage, achieving excellent results in multiple
benchmarks. Given an image $I$ and the referring expression $E_{r}$, LAVT
forwards $E_{r}$ to the text encoder BERT [27] to get the textual feature
$F_{t}\in\mathbb{R}^{C_{t}\times T}$, where $C_{t}$ and $T$ denote the number
of channels for text features and number of tokens, respectively. To align the
given expression to visual pixels, LAVT jointly embeds the image features
$V_{i}\in\mathbb{R}^{C_{i}\times H_{i}\times W_{i}}$, $i\in\\{1,2,3,4\\}$ from
the visual encoder Swin-Transformer [28] with the guidance of pixel-word
attention module (PWAM). Based on the cross-modal features, a lightweight mask
decoder is employed to obtain the final mask. We propose an improved LAVT that
robustly generalizes to unseen referring expressions and visual entities.

###  III-B Target Prompt

The text expression provides a distinguishing description of the targets in an
image that needs to be segmented. As demonstrated in the realistic human-robot
interaction collaboration experiment presented in [29], referring expressions
are unconstrained and vary significantly among human users. Even a simple
object, such as a tomato can, is described in many different ways as “the can
in the middle”, “a tomato can”, “the second can from the right on the top
row”, etc. Without providing direct and explicit guidance, identifying the
target robustly in various referring expressions is an impossible task for a
text encoder which is primarily supervised to extract textual representations
aligned with visual features with limited training data.

To address the above issues, we propose to guide the text encoder with an
explicit and crucial prompt. We first exploit a dependency parsing tool [30]
to parse the given expression $E_{r}$ into a tree structure and identify the
root noun of the sentence as the target $TN(E_{r})$. After capturing the
target in the referring expression, we adopt it to boost the text expression
in a unified context to get the final referring expression  _$E_{f}$_. The
unified context for the prompt is designed with the following form inspired by
[31]:

$E_{f}=[E_{r}][C]_{1}[C]_{2}...[C]_{L}[TN(E_{r})]$ (1)

where each $[C]_{l}~{}(l\in{1,...,L}$) is a token in the unified context. $L$
is the number of context tokens.

Manual Prompt We customize the prompt context for RIS with the form of “It is
a [$TN(E_{r})$]”, which is shared among all referring expressions. Given an
expression  _$E_{r}$_ , we obtain the referring expression  _$E_{f}$_ via:

$E_{f}=[E_{r}].\ It\ is\ a\ [TN(E_{r})]$ (2)

It should be noted that the unified context can be formed in various ways,
e.g., “It describes a [$TN(E_{r})$]”,“It is one of a [$TN(E_{r})$]”, etc. The
learnable prompt is not employed here because the context embeddings need to
be learned from the training data, which may hinder the generalization of the
method.

Based on the unified context, the model can effectively capture the target
within the expression regardless of its linguistic style, thereby reducing
spurious visual-text correlations caused by biasing towards disturbing items
in the expression. Moreover, it can facilitate knowledge learning across
instances with identical prompts, i.e., the same $TN(E_{r})$. By forwarding
_$E_{f}$_ to text encoder $\varphi_{t}$, we obtain the textual feature
$F_{t}$:

$F_{t}=\varphi_{t}(E_{f})$ (3)

###  III-C Multi-modal Fusion Aggregation and Visual Guiding

In order to align the visual pixels with text features to distinguish referred
object from the entire image, pixel-word attention is employed to enhance
pixel-word alignment in recent RIS methods [5], [18]. Given the visual feature
$V_{i}\in\mathbb{R}^{C_{i}\times H_{i}\times W_{i}}$ and textual feature
$F_{t}\in\mathbb{R}^{C_{t}\times T}$, a cross-attention layer is utilized to
fuse multi-modal features computed as:

$Attn_{i}=softmax(W_{iv}(V_{i})W_{t}(F_{t}))$ (4)
${V_{i}}^{{}^{\prime}}=Attn_{i}\odot W_{im}(V_{i})$ (5)

where $\odot$ denotes element-wise multiplication and $W_{iv}$, $W_{t}$,
$W_{im}$ are projection layers. ${V_{i}}^{{}^{\prime}}$ is the visual feature
modulated by the pixel-word attention, i.e., the multi-modal fusion feature,
which is the input of the mask decoder.

The defect of pixel-word alignment is that it considers the similarity between
each pixel embedding and text feature independently, ignoring the semantic
consistency within an object. As a result, there would be irregular false
positive clumps or missing part of target pixels on the predictions,
especially in unseen data without supervision, leading to poor generalization.
To solve the above issues, we introduce a multi-modal fusion aggregation (MFA)
module into cross-attention layer to capture the spatial relations and harness
the favorable generalization capabilities of aggregation module. Concretely,
we feed the pixel-word attention $Attn$ into the MFA module, as shown in Fig.
3. Notably, we only introduce MFA into the final cross-attention module whose
fused features are fed into the decoder for computational efficiency.

The aggregation module needs to have semi-global receptive field to capture
the spatial relations among local regions. Similar to the cost aggregation in
[9], the introduced MFA contains two consecutive Swin-Transformer blocks. Each
block models the spatial relations within a local window via self-attention.
As the text features are same for each pixel, visual similar regions have
similar multi-modal fusion features. Based on this insight, we incorporate
visual guidance from a powerful pretrained vision model to guide the
aggregation of multi-modal features with the visual spatial relations.
Notably, different from [9] which leverages the aggregation in the final cost
of visual features and textual features, we introduce the aggregation into the
multi-modal fusion during encoding. Similarly, we also guide the decoding
process of multi-modal fusion features with visual knowledge.

Given the pretrained vision model, we extract the visual embeddings $P_{i}$,
$i\in\\{1,2,3,4\\}$. MFA can be computed as:

$V_{aggre}=swin\\_blocks(Attn_{i};W_{i}(P_{i}))\odot W_{im}(V_{i})$ (6)

where $W_{i}$ denotes a linear projection layer, [;] denotes concatenation
along the channel dimension. The decoding process can be described as:

$\begin{cases}\,Y_{4}=V_{aggre}\\\
\,Y_{i}=\rho_{i}([Up(Y_{i}+1);V_{i}^{{}^{\prime}};W_{p}(sg(P_{i}))]),i=3,2,1\end{cases}$
(7)

where $\rho_{i}$ is a projection function similar to [5]. We apply group
normalization [32] instead of batch normalization [33] in $\rho_{i}$ to better
capture channel features from multiple sources. $Up$ represents upsampling via
bilinear interpolation. $W_{p}$ denotes a linear projection layer. $sg$ is the
stop gradient operator. The final feature map $Y_{1}$ is fed into the
classification head (foreground and background) to obtain the prediction mask.

Figure 3: Pipeline of the multi-modal fusion aggregation module (MFA). We omit
LayerNorm in swin_blocks for a clearer view.

##  IV EXPERIMENTS

###  IV-A Datasets and Evaluation Protocol

Datasets. There are four widely-used datasets for referring image
segmentation: RefCOCO [4], RefCOCO+ [4], RefCOCOg [34],[35] and ReferIt [6].
The RefCOCO series datasets are collected on top of Microsoft COCO [36], which
comprises 80 common categories. The referring expressions for RefCOCO and
RefCOCO+ are both collected using the ReferitGame [6]. The difference is that
RefCOCO+ focuses solely on appearance and is disallowed from using location
words, while no restrictions are placed on RefCOCO expressions. RefCOCOg
expressions are collected on Amazon Mechanical Turk and the descriptions are
longer and more complex (8.4 words on average  _vs_. 3.61 words of RefCOCO and
3.53 words of RefCOCO+). ReferIt is collected from the ImageCLEF IAPR dataset
[37], which contains 238 different categories. Obviously, there exists a
significant domain gap in terms of images between ReferIt and the other three
datasets. Furthermore, it contains a considerable number of categories that do
not appear in the RefCOCO series datasets.

To evaluate the generalization of RIS in the context of human-robot
interaction, we generate referring expressions for a subset of images from
GraspNet [7] using Shikra [11]. We collect the first frame for every scene
from the subsets train_1, test_seen, test_similar, and test_novel.
Subsequently, we select objects with clear class definitions and typical
appearance, resulting in 50 categories. For instance, we exclude the
_072-k_toy_airplane_ which looks completely different from a conventional
airplane. We generate the text expressions as follows:

  * •

Select objects belong to chosen categories and calculate their bounding boxes
based on the instance masks.

  * •

Feed the image and each bounding box to Shikra with three different prompts,
resulting in three referring expressions for each object.

  * •

Pick the most precise and diverse description and make adjustments to ensure
it accurately corresponds to the object if necessary.

In total, we generate 497 referring expressions for 105 images from GraspNet,
which we denote as GraspNet-RIS.

TABLE I: Zero-shot cross-dataset performance comparison with state-of-the-art methods on public datasets. The models are all trained on one dataset and evaluated on the other datasets. The test results of the trained dataset are in gray. Train Dataset | Method | RefCOCO | RefCOCO+ | RefCOCOg | ReferIt
---|---|---|---|---|---
val | testA | testB | val | testA | testB | val | test | val | test
RefCOCO | oIoU | VLT [17] | 63.15 | 66.57 | 59.75 | 45.65 | 50.21 | 39.14 | 49.81 | 49.52 | 18.56 | 28.52
LAVT [5] | 72.73 | 75.82 | 68.79 | 56.86 | 62.29 | 48.14 | 59.76 | 58.65 | 21.22 | 35.35
Ours | 73.43 | 76.40 | 72.05 | 57.68 | 63.71 | 49.76 | 60.32 | 61.70 | 22.97 | 37.13
mIoU | VLT [17] | 66.15 | 68.95 | 63.16 | 50.37 | 55.38 | 43.35 | 53.58 | 53.30 | 18.06 | 28.94
LAVT [5] | 74.46 | 76.89 | 70.94 | 60.64 | 66.19 | 51.87 | 63.45 | 63.33 | 23.52 | 39.33
Ours | 75.03 | 77.20 | 70.10 | 61.60 | 67.09 | 54.06 | 64.32 | 64.70 | 25.75 | 40.60
RefCOCO+ | oIoU | VLT [17] | 46.35 | 51.74 | 40.73 | 50.93 | 54.90 | 45.25 | 48.93 | 50.13 | 14.66 | 22.40
LAVT [5] | 61.01 | 66.98 | 55.40 | 62.14 | 68.38 | 55.1 | 59.06 | 60.86 | 17.80 | 29.46
Ours | 66.03 | 70.03 | 58.91 | 63.35 | 70.16 | 56.70 | 61.95 | 64.55 | 20.46 | 32.62
mIoU | VLT [17] | 51.67 | 56.70 | 45.39 | 55.42 | 59.37 | 49.44 | 52.86 | 53.50 | 13.87 | 22.18
LAVT [5] | 64.89 | 70.06 | 59.53 | 65.81 | 70.97 | 59.23 | 63.20 | 64.20 | 19.02 | 32.11
Ours | 68.83 | 72.73 | 62.71 | 66.24 | 72.53 | 60.75 | 66.19 | 66.90 | 22.48 | 36.16
ReferIt | oIoU | LAVT [5] | 52.93 | 57.01 | 48.68 | 37.20 | 41.13 | 32.35 | 39.94 | 38.17 | 74.60 | 71.94
Ours | 54.03 | 58.41 | 49.92 | 39.55 | 44.20 | 34.03 | 42.36 | 42.14 | 75.61 | 73.78
mIoU | LAVT [5] | 54.19 | 57.83 | 49.80 | 36.99 | 41.66 | 31.39 | 40.12 | 38.95 | 66.48 | 63.01
Ours | 55.84 | 59.67 | 51.30 | 40.73 | 45.86 | 34.13 | 43.64 | 42.79 | 68.40 | 65.27
RefCOCOg | oIoU | VLT [17] | 56.91 | 59.42 | 55.94 | 43.34 | 46.01 | 39.21 | 50.77 | 51.58 | 12.87 | 23.48
CRIS [38] | 65.77 | 68.73 | 62.37 | 53.75 | 57.43 | 46.42 | 57.79 | 59.06 | 19.59 | 31.95
ETRIS [39] | 65.29 | 68.60 | 60.28 | 53.43 | 59.38 | 45.07 | 59.45 | 59.66 | 16.40 | 24.40
CGFormer [26] | 65.73 | 68.73 | 64.80 | 57.73 | 61.38 | 51.76 | 63.60 | 64.49 | 15.89 | 28.60
3-13[1pt/1pt] | LAVT [5] | 65.20 | 67.43 | 63.07 | 53.88 | 56.52 | 47.45 | 62.09 | 60.50 | 17.89 | 29.09
|  | Ours | 69.18 | 70.96 | 66.65 | 57.64 | 60.74 | 51.13 | 63.52 | 64.00 | 19.88 | 32.89
| mIoU | VLT [17] | 59.66 | 61.37 | 59.06 | 46.85 | 50.18 | 43.99 | 54.45 | 54.61 | 14.65 | 25.58
| CRIS [38] | 68.42 | 70.41 | 65.82 | 58.46 | 62.00 | 53.26 | 61.30 | 61.52 | 21.39 | 33.92
| ETRIS [39] | 65.63 | 68.32 | 62.20 | 55.93 | 61.39 | 49.39 | 60.28 | 60.46 | 16.63 | 25.82
| CGFormer [26] | 67.57 | 70.09 | 67.03 | 60.56 | 63.98 | 56.20 | 65.63 | 66.04 | 19.44 | 33.57
3-13[1pt/1pt] | LAVT [5] | 66.20 | 67.73 | 64.85 | 55.88 | 58.25 | 50.47 | 63.62 | 63.66 | 18.97 | 31.97
|  | Ours | 70.61 | 72.13 | 68.48 | 60.95 | 64.00 | 55.99 | 66.49 | 66.45 | 22.72 | 37.49

Evaluation Protocol. We introduce a zero-shot cross-dataset evaluation
protocol to assess the generalization of RIS. The setting among RefCOCO series
datasets can evaluate the generalization capabilities to varied textual
descriptions since they share image domain and object categories. To further
evaluate the generalization to unseen visual entities, we train on one of the
RefCOCO series datasets and test on ReferIt, and vice versa. Notably, due to
the small size of the generated GraspNet-RIS, it is exclusively utilized for
testing.

We adopt the common metrics of overall intersection-over-union (oIoU), mean
intersection-over-union (mIoU) and precision at 0.5, 0.7, and 0.9 threshold
values in the experiments.

###  IV-B Implementation Details

We maintain the same visual encoder and text encoder as in [5]. And we follow
its training and testing settings without any additional data augmentations or
tricks. We adopt “It is a [] ” as the unified form of the prompt. The MFA is
performed on the last PWAM module in LAVT. The aggregation block uses a window
size of 5 and the same number of heads as the visual features. The number of
output channels in Equation (6) is identical to the input channels, whereas in
Equation (7), it is one-eighth of the input channels. We extract vision
features from a Swin-Transformer pretrained on ImageNet22k, which is frozen
during training111Using the models with better pixel coherence and stronger
generalization can bring more improvements. We adopt it for a fair comparison
since it is identical to the initial visual encoder..

Figure 4: Visualization of predicted masks on examples of the zero-shot
generalization from the model with one suit of parameters trained on RefCOCOg.

###  IV-C Comparison in Zero-Shot Cross-Dataset Setting

We compare our method with the current best-performing methods VLT [17], LAVT
[5], ETRIS [39], CRIS [38] and CGformer [26]. We use the pretrained model
released by VLT and LAVT to report the results under the evaluation protocol.
In addition, we train LAVT on ReferIt using the official code and compare it
with our method in all settings since we share the same model and training
configurations. We train CRIS, ETRIS, and CGFromer on RefCOCOg using their
official code and evaluate the zero-shot performance on other datasets. Our
visual encoder and text encoder are identical to those in LAVT and CGformer,
and ETRIS adopts more stronger generalizable encoders, CLIP-ViT-B16. Notably,
we exclude the multi-scale loss in CGformer and adopt the same loss as LAVT
and ours for a fair comparison. All the models are trained for 50 epochs with
batch size 32.

Tabel I illustrates the comparison results on the public benchmarks. Our
method consistently improves LAVT across all settings. We concretely analyze
the comparison under the setting that the methods are all trained on RefCOCOg
and evaluated on other datasets in the following. Compared to LAVT, our method
improves the average zero-shot mIoU by 4.15%, 5.45%, and 4.64% on RefCOCO,
RefCOCO+ and ReferIt respectively. CRIS, a clip-based contrastive learning
method, exhibits strong generalization despite the superior supervised
performance. Compared to it, our method achieves average improvements of
2.19%, 2.41%, 2.45% mIoU on RefCOCO, RefCOCO+ and ReferIt respectively. ETRIS
freezes the CLIP encoders during training, introducing a few additional
learnable parameters to better transfer the CLIP knowledge into RIS. Our
method surpasses it by 5.02%, 4.74%, 8.88% in terms of average mIoU on
RefCOCO, RefCOCO+ and ReferIt respectively.

TABLE II: Zero-shot performance on GraspNet-RIS.

Train Dataset | Method | GraspNet-RIS
---|---|---
mIoU | oIoU |<EMAIL_ADDRESS>|<EMAIL_ADDRESS>|<EMAIL_ADDRESS>
RefCOCO | LAVT | 35.17 | 31.51 | 37.10 | 28.43 | 10.69
Ours | 39.27 | 34.46 | 41.94 | 30.65 | 12.30
RefCOCO+ | LAVT | 37.43 | 32.18 | 39.92 | 29.03 | 13.51
Ours | 38.70 | 32.41 | 42.54 | 30.04 | 11.29
RefCOCOg | LAVT | 36.90 | 35.10 | 39.72 | 33.87 | 14.31
Ours | 44.53 | 37.06 | 48.19 | 37.30 | 15.52
ReferIt | LAVT | 38.47 | 33.10 | 40.12 | 26.41 | 7.06
Ours | 43.63 | 37.36 | 47.58 | 35.48 | 8.06

And our method outperforms the current best method in generalization,
CGformer, by 3.6% and 4.14% in terms of average mIoU and oIoU on the most
challenging zero-shot evaluation on ReferIt. We exceed CGformer by an average
of 2.18% mIoU on RefCOCO whose expressions are more varied and unconstrained.
The example visualizations of the zero-shot predictions from the model with
one suit of parameters trained on RefCOCOg are shown in Fig. 4.

Tabel II shows the zero-shot performance on GraspNet-RIS of models trained on
four public datasets. Our method achieves clear improvements in all settings,
e.g., our method notably outperforms LAVT by 5.16% mIoU and 4.26% oIoU when
trained on ReferIt. It demonstrates that our method also generalizes well to
scenarios with large domain shifts in practical applications.

###  IV-D Ablation Study

In this section, we conduct ablation studies to evaluate the effects of core
components of the proposed model under the setting that the model is trained
on RefCOCOg and evaluated on other datasets.

TABLE III: Ablation results of the zero-shot performance on RefCOCO, RefCOCO+
and ReferIt. The model is trained on RefCOCOg. TP: Target Prompt. MFA: Multi-
modal fusion aggregation. VG: Visual guidance.

Test Dataset | Method | mIoU | oIoU |<EMAIL_ADDRESS>|<EMAIL_ADDRESS>|<EMAIL_ADDRESS>
---|---|---|---|---|---|---
RefCOCO | val | baseline | 66.20 | 65.20 | 74.51 | 64.39 | 25.97
\+ TP | 68.98 | 67.26 | 76.89 | 68.41 | 34.19
\+ MFA&VG | 70.61 | 69.18 | 78.52 | 70.68 | 35.21
testA | baseline | 67.73 | 67.43 | 77.96 | 67.19 | 24.25
\+ TP | 71.27 | 69.98 | 80.56 | 72.55 | 35.04
\+ MFA&VG | 72.13 | 70.96 | 81.24 | 73.75 | 35.99
testB | baseline | 64.85 | 63.07 | 71.82 | 59.98 | 27.42
+TP | 67.10 | 65.06 | 73.70 | 63.61 | 34.86
+MFA&VG | 68.48 | 66.65 | 75.31 | 65.69 | 37.57
RefCOCO+ | val | baseline | 55.88 | 53.88 | 61.99 | 52.59 | 20.19
\+ TP | 59.38 | 56.33 | 65.63 | 57.70 | 28.68
\+ MFA&VG | 60.95 | 57.64 | 67.25 | 59.52 | 30.16
testA | baseline | 58.25 | 56.52 | 65.68 | 56.23 | 20.31
\+ TP | 63.43 | 59.77 | 70.82 | 63.22 | 30.42
\+ MFA&VG | 64.00 | 60.74 | 71.38 | 64.01 | 30.89
testB | baseline | 50.47 | 47.45 | 54.12 | 44.16 | 20.37
\+ TP | 54.48 | 50.30 | 58.54 | 50.42 | 27.59
\+ MFA&VG | 55.99 | 51.13 | 60.26 | 52.08 | 29.56
ReferIt | val | baseline | 18.97 | 17.89 | 19.09 | 13.30 | 3.29
\+ TP | 19.68 | 16.29 | 19.73 | 13.73 | 2.98
\+ MFA&VG | 22.72 | 19.88 | 22.70 | 15.62 | 3.49
test | baseline | 31.97 | 29.09 | 34.43 | 25.27 | 4.95
\+ TP | 33.97 | 29.09 | 37.39 | 26.68 | 4.97
\+ MFA&VG | 37.49 | 32.89 | 41.04 | 29.93 | 5.56

In Tabel III, the first row of each section shows the baseline for comparison,
which is the results of the model released by LAVT. Boosting the referring
expression with the target prompt achieves a significant improvement on
RefCOCO series datasets, with an average increase of 2.85% mIoU, 8.82%
<EMAIL_ADDRESS>on RefCOCO, and 4.23% mIoU, 8.61%<EMAIL_ADDRESS>on RefCOCO+, which
demonstrates that the target prompt significantly enhances the model’s ability
to generalize to various unconstrained textual descriptions.

Introducing MFA and visual guidance further improves the zero-shot
performance. The model attains noticeable improvements over the baseline with
wide average margins of 4.15% mIoU, 10.38%<EMAIL_ADDRESS>on RefCOCO, and of 5.45%
mIoU, and 9.91%<EMAIL_ADDRESS>on RefCOCO+. The zero-shot performance on ReferIt is
relatively poor since it contains numerous unseen visual entities. As
expected, MFA and visual guidance provide more benefits to ReferIt, while
target prompt yields greater improvements on RefCOCO and RefCOCO+. The model
with MFA and visual guidance achieves improvements of 3.75% mIoU and 5.52%
mIoU on the validation and test subsets of ReferIt respectively.

##  V CONCLUSION

In this paper, we propose to improve the generalization of RIS from two
perspectives. To robustly generalize to various unconstrained textual
descriptions, we boost a given expression with an explicit target prompt in a
unified context to facilitate the target capturing. To deal with the
incomplete targets and false-positive clumps of predictions in unseen visual
entities, we introduce a multi-modal fusion aggregation module along with
visual guidance from a powerful pretrained model to leverage the spatial
relations and visual coherences. Extensive experiments conducted under the
zero-shot cross-dataset evaluation protocol demonstrate the effectiveness of
the proposed approach.

## References

  * [1] X. Wang, Q. Huang, A. Celikyilmaz, J. Gao, D. Shen, Y.-F. Wang, W. Y. Wang, and L. Zhang, “Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 6629–6638.
  * [2] J. Chen, Y. Shen, J. Gao, J. Liu, and X. Liu, “Language-based image editing with recurrent attentive models,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 8721–8729.
  * [3] B. Zhou, H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba, “Semantic understanding of scenes through the ade20k dataset,” _International Journal of Computer Vision_ , vol. 127, pp. 302–321, 2019.
  * [4] L. Yu, P. Poirson, S. Yang, A. C. Berg, and T. L. Berg, “Modeling context in referring expressions,” in _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14_. Springer, 2016, pp. 69–85.
  * [5] Z. Yang, J. Wang, Y. Tang, K. Chen, H. Zhao, and P. H. Torr, “Lavt: Language-aware vision transformer for referring image segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 18 155–18 165.
  * [6] S. KAZEMZADE, V. Ordonez, and M. MATTENV, “Referring to objects in photographs of natural scenes,” in _Empirical Methods in Natural Language Processing_ , vol. 28, 2014, pp. 787–789.
  * [7] H.-S. Fang, C. Wang, M. Gou, and C. Lu, “Graspnet-1billion: A large-scale benchmark for general object grasping,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 11 444–11 453.
  * [8] B. Liu, H. Yu, and G. Qi, “Graftnet: Towards domain generalized stereo matching with a broad-spectrum and task-oriented feature,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 13 012–13 021.
  * [9] S. Cho, H. Shin, S. Hong, S. An, S. Lee, A. Arnab, P. H. Seo, and S. Kim, “Cat-seg: Cost aggregation for open-vocabulary semantic segmentation,” _arXiv preprint arXiv:2303.11797_ , 2023.
  * [10] M. Xu, Z. Zhang, F. Wei, Y. Lin, Y. Cao, H. Hu, and X. Bai, “A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model,” in _European Conference on Computer Vision_. Springer, 2022, pp. 736–753.
  * [11] K. Chen, Z. Zhang, W. Zeng, R. Zhang, F. Zhu, and R. Zhao, “Shikra: Unleashing multimodal llm’s referential dialogue magic,” _arXiv preprint arXiv:2306.15195_ , 2023.
  * [12] G. Feng, Z. Hu, L. Zhang, and H. Lu, “Encoder fusion network with co-attention embedding for referring image segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 15 506–15 515.
  * [13] B. Chen, Z. Hu, Z. Ji, J. Bai, and W. Zuo, “Position-aware contrastive alignment for referring image segmentation,” _arXiv preprint arXiv:2212.13419_ , 2022.
  * [14] Q. Sima, S. Tan, H. Liu, F. Sun, W. Xu, and L. Fu, “Embodied referring expression for manipulation question answering in interactive environment,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023, pp. 7635–7641.
  * [15] D.-J. Chen, H.-Y. Hsieh, and T.-L. Liu, “Referring image segmentation via language-driven attention,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021, pp. 13 997–14 003.
  * [16] A. Kamath, M. Singh, Y. LeCun, G. Synnaeve, I. Misra, and N. Carion, “Mdetr-modulated detection for end-to-end multi-modal understanding,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 1780–1790.
  * [17] H. Ding, C. Liu, S. Wang, and X. Jiang, “Vision-language transformer and query generation for referring segmentation,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 16 321–16 330.
  * [18] Z. Zhang, Y. Zhu, J. Liu, X. Liang, and W. Ke, “Coupalign: Coupling word-pixel with sentence-mask alignments for referring image segmentation,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 14 729–14 742, 2022.
  * [19] J. Liu, H. Ding, Z. Cai, Y. Zhang, R. K. Satzoda, V. Mahadevan, and R. Manmatha, “Polyformer: Referring image segmentation as sequential polygon generation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 18 653–18 663.
  * [20] C. Zhu, Y. Zhou, Y. Shen, G. Luo, X. Pan, M. Lin, C. Chen, L. Cao, X. Sun, and R. Ji, “Seqtr: A simple yet universal network for visual grounding,” in _European Conference on Computer Vision_. Springer, 2022, pp. 598–615.
  * [21] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in Neural Information Processing Systems_ , vol. 30, 2017.
  * [22] S. Subramanian, W. Merrill, T. Darrell, M. Gardner, S. Singh, and A. Rohrbach, “Reclip: A strong zero-shot baseline for referring expression comprehension,” in _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 2022, pp. 5198–5215.
  * [23] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark _et al._ , “Learning transferable visual models from natural language supervision,” in _ICML_. PMLR, 2021, pp. 8748–8763.
  * [24] C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. Le, Y.-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in _ICML_. PMLR, 2021, pp. 4904–4916.
  * [25] S. Yu, P. H. Seo, and J. Son, “Zero-shot referring image segmentation with global-local context features,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 19 456–19 465.
  * [26] J. Tang, G. Zheng, C. Shi, and S. Yang, “Contrastive grouping with transformer for referring image segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 23 570–23 580.
  * [27] J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in _Proceedings of NAACL-HLT_ , 2019, pp. 4171–4186.
  * [28] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 10 012–10 022.
  * [29] M. Shridhar and D. Hsu, “Interactive visual grounding of referring expressions for human-robot interaction,” _Robotics: Science and Systems XIV_ , 2018.
  * [30] M. Honnibal and M. Johnson, “An improved non-monotonic transition system for dependency parsing,” in _Empirical Methods in Natural Language Processing_ , 2015, pp. 1373–1378.
  * [31] K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Learning to prompt for vision-language models,” _International Journal of Computer Vision_ , vol. 130, no. 9, pp. 2337–2348, 2022.
  * [32] Y. Wu and K. He, “Group normalization,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 3–19.
  * [33] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in _ICML_. pmlr, 2015, pp. 448–456.
  * [34] J. Mao, J. Huang, A. Toshev, O. Camburu, A. L. Yuille, and K. Murphy, “Generation and comprehension of unambiguous object descriptions,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 11–20.
  * [35] V. K. Nagaraja, V. I. Morariu, and L. S. Davis, “Modeling context between objects for referring expression understanding,” in _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14_. Springer, 2016, pp. 792–807.
  * [36] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_. Springer, 2014, pp. 740–755.
  * [37] M. Grubinger, P. Clough, H. Müller, and T. Deselaers, “The iapr tc-12 benchmark: A new evaluation resource for visual information systems,” in _OntoImage 2006 Workshop on Language Resources for Content-based Image Retrieval during LREC 2006 Final Programme_ , vol. 2, 2006.
  * [38] Z. Wang, Y. Lu, Q. Li, X. Tao, Y. Guo, M. Gong, and T. Liu, “Cris: Clip-driven referring image segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 11 686–11 695.
  * [39] Z. Xu, Z. Chen, Y. Zhang, Y. Song, X. Wan, and G. Li, “Bridging vision and language encoders: Parameter-efficient tuning for referring image segmentation,” _arXiv preprint arXiv:2307.11545_ , 2023.


# Macroscopic quantum-mechanical scattering

M. Apostol
Department of Theoretical Physics, Institute of Atomic Physics,
Magurele-Bucharest MG-6, POBox MG-35, Romania
email<EMAIL_ADDRESS>
###### Abstract

In certain conditions a macroscopic quantum-mechanical scattering may occur,
which may lead to a coherent cross-section on a macroscopic scale in a
monocrystal. The conditions are satisfied by neutrinos, but not satisfied by
other projectiles, with a higher cross-section. This may explain Weber-type
experiments of neutrino detection by a perfect, stiff sapphire monocrystal.
The occurrence of coherence domains for quantum-mechanical scattering and
classical diffraction is analyzed, and the force exerted upon a macroscopic
target is estimated. It is concluded that neutrinos exhibit a distinctive
feature in this respect, due precisely to their very small cross-section.

Key words: neutrino scattering; coherent scattering; macroscopic coherence;
neutrino detection

Introduction. In two papers published in 1985 and 1988 Weber claimed that
neutrinos (antineutrinos), from various sources like tritium, nuclear reactors
and Sun, could be detected by their coherent scattering by a perfect, stiff,
sapphire monocrystal with a high Debye temperature (mounted on a torsion
balance and equilibrated by a lead dummy).[1, 2] The coherent cross-section
would be $\sigma=N^{2}\sigma_{0}$, where $N$ is the number of unit cells in
the target and $\sigma_{0}$ is the cross-section of a single unit cell
(particle, _e.g._ , atomic nucleus). Such a highly enhanced cross-section
$\sim N^{2}$ would give rise to a measurable force upon a torsion balance.
Weber’s claims have been criticized both on theoretical and experimental
grounds, the main objection being that the form-factor would reduce
appreciably the cross-section, and, on the other hand, such a coherence effect
is not observed in $X-$, gamma rays or neutron scattering[3]-[12] (see also
Refs. [13, 14]). A discussion of the theoretical objections and negative
experiments was given by Nicolescu, who presented a positive experiment;[15]
indeed, an experiment by Cruceru et al exists, which confirms Weber’s
prediction for solar neutrinos.[15]-[17] The problem is still controversial.
We show in this paper that a coherent scattering of neutrinos may appear in
the conditions formulated by Weber and company, as a consequence of a quantum-
mechanical treatment of the crystal as a whole (a macroscopic quantum-
mechanical scattering). This is a distinctive condition of the neutrino
scattering, which is not fulfilled by other projectiles, with a higher cross-
section. The main reason for such a behaviour is precisely the extremely small
cross-section $\sigma_{0}$ (of the order $10^{-44}cm^{2}$) of the neutrinos.

For $\sigma_{0}=10^{-44}cm^{2}$ and $N=10^{22}$ (less than $0.1\,mol$) the
coherent cross-section is $\sigma=1cm^{2}$. For a neutrino flux density
$\Phi=10^{12}/cm^{2}\cdot s$ the time between two collisions is
$\tau=1/\Phi\sigma=10^{-12}s$. On the other hand, an atom in thermal
equilibrium at room temperature has a velocity of the order $v=10^{4}cm/s$. In
an elementary act of collision the atom is perturbed from its equilibrium
state and receives a momentum of the order $p=E/c$, on average, where $E$ is
the energy of the neutrino projectile (and $c=3\times 10^{10}cm/s$ is the
speed of light). For $E=1MeV$ the momentum transfer is of the order $p=5\times
10^{-17}g\cdot cm/s$. Consequently, the energy perturbation of the atom is of
the order $\Delta E=p^{2}/M=2.5\times 10^{-11}erg$ for $M=10^{5}m_{e}$, where
$m_{e}\simeq 10^{-27}g$ is the electron mass ($vp=5\times 10^{-13}erg\ll
p^{2}/2M$). The time needed for this atom to recover its equilibrium is of the
order $\Delta t_{eq}=\hbar/\Delta E\simeq 4\times 10^{-17}s$ (where
$\hbar\simeq 10^{-27}erg\cdot s$ is Planck’s constant). We can see that
$\tau\gg\Delta t_{eq}$. It follows that the atoms recover quickly their
equilibrium state between two sucessive collisions, and the incident neutrino
beam sees the crystal as a whole. Therefore, we need to adopt a quantum-
mechanical treatment for the entire crystal. It is worth noting that if the
cross-section increases to, say, $\sigma_{0}=10^{-24}cm^{2}$, as for $X$-,
gamma rays or neutrons, the "collision" time decreases to $\tau=10^{-32}s$,
which is much shorter than the equilibrium time $\Delta t_{eq}$, all the other
conditions remaining the same. In that case the incident projectile beam sees
the crystal as consisting of distinct, independent atoms, such that a coherent
scattering ($\sigma=N^{2}\sigma_{0}$) for the entire crystal is not possible.
The particularity of a coherent scattering suffered by neutrinos in the whole
crystal resides precisely in their extremely small cross-section $\sigma_{0}$.
On the other hand, a single-particle cross-section $\sigma_{0}=10^{-24}cm^{2}$
increases considerably the total cross-section, such that we need to
reconsider the scattering in this case.

Macroscopic quantum-mechanical scattering. Let us assume a macroscopic target
consisting of $N\gg 1$ identical "atoms" (atomic nuclei, molecules, unit cells
in a crystal). The interaction with an incident beam of particles can be
written as

$H=a^{3}h(\xi)\sum_{i=1}^{N}\delta(\boldsymbol{r}-\boldsymbol{r}_{i})\,\,\,,$
(1)

where $a$ is the range of the single-particle interaction $h(\xi)$, $\xi$
denotes the internal coordinates of the atoms and $\boldsymbol{r}_{i}$ are the
atomic positions. The time between two successive collisions is
$\tau=1/\Phi\sigma$, where $\Phi$ is the incident flux density and $\sigma$ is
the total cross-section. In an elementary act of collision an atom receives a
momentum of the order of the momentum $p$ of the incident particle. The atom
has a thermal velocity $v\simeq\sqrt{T/M}$, where $T$ is the temperature and
$M$ is the mass of the atom. The atomic motion is perturbed by an energy of
the order $\Delta E=p^{2}/M+vp$, so it needs a time $\Delta
t_{eq}\simeq\hbar/\Delta E$ to recover its equilibrium. Let us assume

$\tau>\Delta t_{eq}\,\,;$ (2)

then, the incident particles see the macroscopic target as a whole, and we
need to work with the wavefunction of the whole, macroscopic, target.

A perfect monocrystal suffers two kinds of motion. One kind is the motion of
the crystal as a whole, where all the laticial positions of the atoms
$\boldsymbol{r}_{i}^{0}$ move by the same distance. The wavefunction
corresponding to this motion is

$\Phi_{\boldsymbol{K}}(\xi;\boldsymbol{r})=\sum_{i=1}^{N}e^{i\boldsymbol{K}\boldsymbol{r}_{i}^{0}}\varphi(\xi;\boldsymbol{r}-\boldsymbol{r}_{i}^{0})\,\,\,,$
(3)

where $\boldsymbol{K}$ is the quasi-wavevector of the crystal (quasi-momentum
$\hbar\boldsymbol{K}$) and
$\varphi(\xi;\boldsymbol{r}-\boldsymbol{r}_{i}^{0})$ are (orthonormal)
wavefuctions localized on the positions $\boldsymbol{r}_{i}^{0}$. We can see
that the wavefunction $\Phi_{\boldsymbol{K}}$ has the translational symmetry
of the crystal. We call $\Phi_{\boldsymbol{K}}$ a coherent wavefunction. The
other type of motion of the crystal is the thermal motion with atomic
displacements $\boldsymbol{v}_{i}$
($\boldsymbol{r}_{i}=\boldsymbol{r}_{i}^{0}+\boldsymbol{v}_{i})$; the
corresponding wavefunction is

$\psi(\xi;\boldsymbol{r})=\sum_{i=1}^{N}e^{i\chi_{i}}\varphi(\xi;\boldsymbol{r}-\boldsymbol{r}_{i}^{0}-\boldsymbol{v}_{i})\,\,\,,$
(4)

where $e^{i\chi_{i}}$ are random phase factors; we call $\psi$ an incoherent
wavefunction. Both these wavefunctions are normalized to $N$; they are
orthogonal to each other. The wavefunction of the crystal is

$\varPsi_{\boldsymbol{K}}=\sqrt{1-g^{2}}\Phi_{\boldsymbol{K}}+g\psi\,\,\,,$
(5)

where $g$ is a weight coefficient. This coefficient is proportional to the
square root of the relative number of vibration states of the crystal,
properly normalized. This relative number of phonon states is proportional to

$I=\frac{1}{\omega_{D}^{3}}\int_{0}^{\omega_{D}}d\omega\frac{\omega^{2}}{e^{\hbar\omega/T}-1}\,\,\,,$
(6)

where $\omega_{D}$ is the Debye frequency and $\omega$ is the phonon
frequency; the Debye temperature is $\Theta=\hbar\omega_{D}$. For $T/\Theta\ll
1$ the integral is $I\simeq 2.4(T/\Theta)^{3}$, while for $T/\Theta\gg 1$ the
integral is $I\simeq\frac{1}{2}T/\Theta$. A normalized expression for the
relative number of states is $4.8(T/\Theta)^{2}$ for $T/\Theta\ll 1$ and $1$
for $T/\Theta\gg 1$. An interpolation formula is $I\simeq
4.8(T/\omega_{D})^{2}/\left[1+4.8(T/\omega_{D})^{2}\right]$, such that we can
take for the weight coefficient

$g\simeq\frac{2.2(T/\Theta)}{\left[1+4.8(T/\Theta)^{2}\right]^{1/2}}\,\,.$ (7)

We can see that for high Debye temperatures the main contribution to
$\varPsi_{\boldsymbol{K}}$ comes from the coherent wavefunction ($g\ll 1$),
while for low Debye temperatures the main contribution comes from the
incoherent field ($g\rightarrow 1$).

Let us assume a wavefunction $\frac{1}{\sqrt{V}}e^{i\boldsymbol{kr}}$ for an
incident particle, and an initial ($i$) wavefunction
$\frac{1}{\sqrt{V}}e^{i\boldsymbol{kr}}\varPsi_{\boldsymbol{K}}$, where $V$
denotes the volume. The normalization to unity of this wavefunction requires a
factor $\sqrt{a^{3}}$ in the wavefuctions $\varphi$, such that the scalar
product is
$\left\langle\varphi(\xi;\boldsymbol{r}_{j}-\boldsymbol{r}_{i}),\varphi(\xi;\boldsymbol{r}_{j}-\boldsymbol{r}_{i})\right\rangle^{2}=\delta_{ij}$.
The matrix elements of the interaction between the wavefunctions
$\Phi_{\boldsymbol{K}}$ and $\psi$ are zero. The matrix elements of the
interaction between two wavefunctions $\Phi_{\boldsymbol{K}}$ and
$\Phi_{\boldsymbol{K}^{\prime}}$ (coherent scattering) lead to the momentum
conservation

$H_{fi}\sim\sum_{i=1}^{N}e^{i(\boldsymbol{K}-\boldsymbol{K}^{\prime})\boldsymbol{r}_{i}^{0}}e^{i(\boldsymbol{k}-\boldsymbol{k}^{\prime})\boldsymbol{r}_{i}^{0}}=N\delta_{\boldsymbol{K}^{\prime}+\boldsymbol{k}^{\prime},\boldsymbol{K}+\boldsymbol{k}}\,\,\,,$
(8)

where $\boldsymbol{k}^{\prime},\,\boldsymbol{K}^{\prime}$ are the wavevectors
of the final state ($f$). We can see that the difference in momentum of the
incident particle is taken by the crystal, which moves as a whole. According
to equation (8) the coherent cross-section is

$\sigma_{coh}=N^{2}\sigma_{0}\,\,\,,$ (9)

where $\sigma_{0}$ is the single-particle cross-section. A similar calculation
for the incoherent matrix elements (wavefunctions $\psi$) leads to

$H_{fi}\sim\sum_{i=1}^{N}e^{i(\boldsymbol{k}-\boldsymbol{k}^{\prime})\boldsymbol{r}_{i}^{0}}\left(1+i(\boldsymbol{k}-\boldsymbol{k}^{\prime})\boldsymbol{v}_{i}+...\right)\,\,\,,$
(10)

where

$\boldsymbol{v}_{i}=\frac{1}{\sqrt{N}}\sum_{\boldsymbol{q}}e^{i\boldsymbol{q}\boldsymbol{r}_{\boldsymbol{i}}^{\boldsymbol{0}}}\boldsymbol{v}_{\boldsymbol{q}}$
(11)

is the phonon field. The first term in equation (10) corresponds to a
displacement of the crystal as a whole, so it is already included in the
coherent scattering. We are left with

$H_{fi}\sim\sqrt{N}\left(\boldsymbol{q}\boldsymbol{v}_{\boldsymbol{q}}\right)\delta_{\boldsymbol{k}^{\prime},\boldsymbol{k}+\boldsymbol{q}}\,\,.$
(12)

We can see that the difference in momentum of the incident particle is taken
by phonons; the incoherent scattering excites phonons. In addition, the
incoherent cross-section is proportional to $N$. For the cross-section we need
to average $(\boldsymbol{q}\boldsymbol{v}_{\boldsymbol{q}})^{2}$ over the
thermal states. The maximum value of this average is of the order
$T/Mc_{s}^{2}$ (at room temperature), where $c_{s}$ is the mean phonon
velocity (of the order $10^{6}cm/s$). This thermal factor is reminiscent of
the Debye-Waller factor (and the diffuse scattering). The total incoherent
cross-section can be written as

$\sigma_{incoh}=N\sigma_{0}^{ph}\,\,\,,$ (13)

where the single-particle cross-section $\sigma_{0}^{ph}$, arising from
phonons, is smaller than $\sigma_{0}$ by the thermal factor.

The total cross-section of the crystal is

$\sigma=\sqrt{1-g^{2}}\sigma_{coh}+g^{2}\sigma_{incoh}\,\,.$ (14)

The incoherent cross-section is extremely small in comparison with the
coherent cross-section. For atoms placed randomly (like in amorphous solids,
liquids, etc) the coherent wavefunction $\Phi_{\boldsymbol{K}}$ is absent and
the weight coefficient is $g=1$.

Neutrino scattering. We adopt $N\simeq 10^{22}$ for the number of unit cells
in the sapphire monocrystal ($\simeq 24g$, density $4g/cm^{3}$) used in
Weber’s experiments,[1, 2] and other similar experiments[15]-[17] (the volume
of the unit cell of sapphire is large, $\simeq 10^{3}\textrm{\r{A}}^{3}$).
Making use of $\sigma_{0}=10^{-44}cm^{2}$ we get a coherent cross-section
$\sigma_{coh}\simeq 1cm^{2}$. For a Debye temperature $\Theta=10^{3}K$ the
weight factor at room temperature is $\sqrt{1-g^{2}}\simeq 0.7$. The total
coherent cross-section is $\sigma\simeq 0.7cm^{2}$. We note that this cross-
section is smaller than the cross-sectional area of the crystal. For a flux
density $\Phi=10^{12}/cm^{2}\cdot s$ the collision time is
$\tau=1/\Phi\sigma_{coh}\simeq 10^{-12}s$. At room temperature the thermal
velocity of an atom is $v\simeq 10^{4}cm/s$. For a neutrino energy $E=1MeV$
the momentum is $p=E/c\simeq 5\cdot 10^{-17}g\cdot cm/s$. The perturbation
energy is $\Delta E=p^{2}/M\simeq 2.5\times 10^{-11}erg$ (for an atomic mass
$M=10^{5}m_{e}$, where $m_{e}$ is the electron mass); the contribution $\Delta
E=vp\simeq 5\times 10^{-13}erg$ is much smaller. The equilibrium time is of
the order $\Delta t_{eq}=\hbar/\Delta E\simeq 4\times 10^{-17}s$. Since
$\tau\gg\Delta t_{eq}$ the macroscopic quantum-scattering described above
applies.

The force acting upon the target in the forward direction is $F=\Phi\sigma
p\simeq 3.5\times 10^{-5}\,dyn$. This is a measurable force. We note that it
is sensitive to the values of the input parameters. For instance, a Debye
temperature $\Theta=100K$ leads to a weight coefficient $\sqrt{1-g^{2}}\simeq
0.15$ and a weaker force by a factor $\simeq 5$. Also, for an amorphous solid,
although the conditions of a quantum-mechanical scattering may be fulfilled,
the force is extremely weak, as a consequence of the very small incoherent
cross-section.

For solar neutrinos ($E\simeq 300keV$) the single-particle cross-section may
preserve its value, but the flux density is smaller ($\Phi\simeq
10^{11}/cm^{2}\cdot s$). The conditions of a coherent scattering are
preserved, but the force may be diminished by a factor $\simeq 10^{-1}$
($\simeq 10^{-6}dyn$). Also, a decrease may appear for tritium neutrinos
($E=10keV,$ $\Phi\simeq 10^{14}/cm^{2}\cdot s$), though a higher $\sigma_{0}$
or a slightly greater number of unit cells $N$ may compensate the decrease
(while preserving the conditions of coherent scattering). We note that a large
number of atoms in the unit cell, as for a sapphire crystal, may increase the
single-particle cross-section $\sigma_{0}$. We conclude that Weber-type
experiments could exhibit a measurable force acting upon a sapphire crystal.

Other projectiles. Coherence domains. We adopt the value
$\sigma_{0}=10^{-24}cm^{2}$ for other types of projectiles (like $X$-, gamma
rays or neutrons). A coherent cross-section would be much larger than the
cross-sectional area of the crystal. The crystal responds to this unphysical
situation by developing coherence domains. Let us assume that $n_{d}$
uncorrelated domains exist in the crystal, each with $N_{d}$ unit cells (as a
mean size), such that $n_{d}=N/N_{d}$. By a formal analogy with the high-
purity crystals we use the fraction $f=1/N_{d}$. This fraction varies between
$f=1/N$, when we have only one domain, _i.e._ the whole target, and $f=1$,
when the whole target is fragmented in "atomic" domains.

The scattering amplitude can be written as

$S=\sum_{a=1}^{n_{d}}e^{i\chi_{a}}\left(H_{fi}\right)_{a}\,\,\,,$ (15)

where $e^{i\chi_{a}}$ are random phase factors. By averaging the squared
scattering amplitude over the phase factors, we get $\mid
S\mid^{2}=\sum_{a}\left|\left(H_{fi}\right)_{a}\right|^{2}$, such that the
cross-section becomes

$\overline{\sigma}=n_{d}\sigma_{d}\,\,\,,$ (16)

where $\sigma_{d}$ is the cross-section of a domain.

According to this equation, the coherent cross-section
$\sigma_{coh}=\sigma_{0}N^{2}$ is reduced by the coherence domains to

$\overline{\sigma}=\sigma_{0}n_{d}N_{d}^{2}=\frac{\sigma_{0}N}{f}\,\,;$ (17)

we can see that this formula gives the total coherent cross-section
($\sigma_{0}N^{2}$) for $f=1/N$ and an incoherent cross-section
($\sigma_{0}N$) for $f=1$. In this latter case $\sigma_{0}$ should be replaced
by $\sigma_{0}^{ph}$ (a similar procedure leaves the incoherent cross-section
arising from phonons unchanged,
$\overline{\sigma}_{incoh}=n_{d}N_{d}\sigma_{0}^{ph}=N\sigma_{0}^{ph}=\sigma_{incoh}$).

In order to have a quantum-mechanical scattering the conditions
$\overline{\tau}>\Delta t_{eq}$ and $\overline{\sigma}<A$ should be satisfied,
where $A$ is the cross-sectional area of the target; these conditions lead to

$f>\frac{\hbar\Phi}{\Delta E}\sigma_{0}N\,\,,\,\,f>\frac{\sigma_{0}N}{A}$ (18)

and a number of unit cells $N_{d}=1/f<A/\sigma_{0}N$ in each domain. For
$\sigma_{0}=10^{-24}cm^{2}$ this number is too small for any macroscopic
target ($N_{d}<10^{2}A$, $N=10^{22}$); the domains are not well defined, such
that $f$ approaches unity and the scattering tends to an incoherent
scattering. We conclude that the quantum-mechanical scattering cannot appear
for large single-particle cross-sections, like $\sigma_{0}=10^{-24}cm^{2}$. We
note that for neutrinos ($\sigma_{0}=10^{-44}cm^{2}$) $f>10^{-22}/A$,
$N_{d}<10^{22}A$ and we may have one domain in the whole target. The coherent
scattering occurs for neutrinos in a crystal precisely due to the small
neutrino cross-section$\sigma_{0}.$ The above considerations apply also to a
polycrystalline target, where $f$ is limited, in addition, by the size of the
crystallites and, consequently, the cross-section is much diminished.

Classical scattering. If inequation (2) is not satisfied (_i.e._ , if
$\tau<\Delta t_{eq}$), the incident particles see the target "atoms" (atomic
nuclei, molecules, unit cells) as independent scatterers. We call this
scattering a classical scattering. We can see that this condition implies low
energies. The initial wavefunction is
$\frac{1}{\sqrt{V}}e^{i\boldsymbol{k}\boldsymbol{R}}\frac{1}{\sqrt{V}}e^{i\boldsymbol{K}\boldsymbol{R}_{c}}$
(up to wavefunctions corresponding to the internal degrees of freedom), where
$\boldsymbol{k}$ is the wavevector of the incident particle, $\boldsymbol{K}$
is the wavevector of the center of mass, $\boldsymbol{R}_{c}$ is the position
of the center of mass and $\boldsymbol{R}=\boldsymbol{r}+\boldsymbol{R}_{c}$.
The matrix elements of the interaction given by equation (1),

$H_{fi}\sim\sum_{i=1}^{N}e^{i(\boldsymbol{k}-\boldsymbol{k}^{\prime})\boldsymbol{r}_{i}}\delta_{\boldsymbol{K}^{\prime}+\boldsymbol{k}^{\prime},\boldsymbol{K}+\boldsymbol{k}}\,\,\,,$
(19)

includes the form-factor

$F(\boldsymbol{k}-\boldsymbol{k}^{\prime})=\sum_{i=1}^{N}e^{i(\boldsymbol{k}-\boldsymbol{k}^{\prime})\boldsymbol{r}_{i}}\,\,\,,$
(20)

where $\boldsymbol{k}^{\prime}$ is the wavevector of the scattered particle
and $\boldsymbol{K}^{\prime}$ is the final wavevector of the center of mass.
We can see that the total momentum, including the momentum of the center of
mass, is conserved. For a crystal
$F(\boldsymbol{k}-\boldsymbol{k}^{\prime})=N\delta_{\boldsymbol{k}^{\prime},\boldsymbol{k}+\boldsymbol{g}}$,
where $\boldsymbol{g}$ is a reciprocal vector of the lattice. For an amorphous
target $\boldsymbol{g}=0$. It follows that we have diffraction peaks. For the
cross-section of a peak we get $d\sigma_{\boldsymbol{g}}\sim
N^{2}do_{\boldsymbol{g}}$, where the solid angle $o_{\boldsymbol{g}}$ extends
to the range $\Delta o_{\boldsymbol{g}}\simeq N^{-2/3}(2\pi/dk^{\prime})^{2}$,
where $d$ is the mean distance between unit cells (scatterers). It follows
$\sigma_{\boldsymbol{g}}\sim N^{4/3}/(dk^{\prime})^{2}$. On the other hand the
number of peaks is $\simeq(dk^{\prime})^{2}\gg 1$, such that the total cross-
section is

$\sigma=N^{4/3}\sigma_{0}\,\,\,,$ (21)

where $\sigma_{0}$ is the single-particle cross-section. For one peak $\sigma$
should be divided by the number of peaks. As it is well known, this cross-
section is affected by the Debye-Waller factor and diffuse scattering.
According to equation (15) for $n_{d}=N/N_{d}=fN$ domains the cross-section is

$\overline{\sigma}=n_{d}N_{d}^{4/3}\sigma_{0}=\frac{\sigma_{0}N}{f^{1/3}}\,\,.$
(22)

For $f=1/N$ we recover the total cross-section $N^{4/3}\sigma_{0}$ of one
domain, while for $f=1$ the cross-section reduces to $N\sigma_{0}$ of an
incoherent scattering.

The conditions $\overline{\tau}=1/\Phi\overline{\sigma}<\Delta t_{eq}$ and
$\overline{\sigma}<A$ lead to

$\frac{\sigma_{0}N}{A}<f^{1/3}<\frac{\hbar\Phi}{\Delta E}\sigma_{0}N\,\,\,,$
(23)

which implies $N_{d}=1/f<(A/\sigma_{0}N)^{3}$ ($\overline{\sigma}<A$). For
$\sigma_{0}=10^{-24}cm^{2}$ and $N=10^{22}$ the number of unit cells
$N_{d}<10^{6}A^{3}$ in a domain may indicate well-defined domains for
macroscopic targets. The force is bounded from above according to the
inequality $F<\Phi Ap$. The inequations (23) imply $\Delta E<\hbar\Phi A$,
_i.e._ $\Delta E<10^{-15}A\,erg$ ($\Phi=10^{12}/cm^{2}\cdot s$). Therefore, we
may set $\Delta E=vp$ and $p<\frac{\hbar\Phi}{v}A$. Consequently, this upper
bound is given by $F<\frac{\hbar}{v}(\Phi A)^{2}\simeq 10^{-7}A^{2}dyn$
($v=10^{4}cm/s$). For any reasonably large area $A$ and flux density $\Phi$ it
is difficult to satisfy these conditions ($p<\hbar\Phi A/v$) and to measure
such a force in current experimental situations. For higher energies the
energy transfer is higher and we need to apply inequations (18); the large
cross-section in this case leads to an incoherent scattering.

Concluding remarks. A quantum-mechanical scattering is identified in certain
conditions in macroscopic targets, which may lead to a coherent cross-section
in high-purity, stiff monocrystals. This coherent scattering may explain the
Weber-type experiments of neutrino detection by using sapphire monocrystals.
The coherent-scattering conditions are not fulfilled by other types of
projectiles, with a higher single-particle cross-sections (like $X$-, gamma or
neutrons). In these cases a classical diffraction may occur in crystals, which
generates a weak force, at the limit of detection.

Acknowledgments. The author is indebted to the members of the Laboratory of
Theoretical Physics at Magurele-Bucharest for many fruitful discussions. This
work has been supported by the Scientific Research Agency of the Romanian
Government through Grant PN 19060101/2019

Statement: The author declares no conflict of interest.

## References

  * [1] J. Weber, "Method for observation of neutrinos and antineutrinos", Phys. Rev. C31 1468-1475 (1985).
  * [2] J. Weber, "Apparent observation of abnormally large coherent scattering cross sections using keV and MeV range antineutrinos, and solar neutrinos", Phys. Rev. D38 32-39 (1988).
  * [3] G. F. Bertsch and Sam M. Austin, "Comment on "Method for observation of neutrinos and antineutrinos"", Phys. Rev. C34 361 (1986).
  * [4] R. Casella, "On the theory of coherent neutrino scattering from crystals", Nuovo Cimento 94 42-50 (1986).
  * [5] T. H. Ho, "Comments on the "Method for observation of neutrinos and antineutrinos""Phys. Lett. B166 295- (1986).
  * [6] M. N. Butler, "Comment on "Method for observation of neutrinos and antineutrinos"", Phys. Rev. C35 1164-1165 (1987).
  * [7] T. Y. Aharanov, F. T. Avignon III, A. Casher and S. Nussinov, "Constraints on anomalous scattering of neutrinos from crystals", Phys. Rev. Lett. 58 1173-1175 (1987).
  * [8] H. J. Lipkin, "$N$ dependence of coherent radiation from crystals", Phys. Rev. Lett. 58 1176-1179 (1987).
  * [9] J. D. Fronson and B. C. Jacobs, "Null result for enhanced neutrino scattering in crystals", Phys. Rev. A46 2235-2239 (1992).
  * [10] Y. Su, B. R. Heckel, E. G. Adelberger, J. H. Gundlach, M. Harris, G. L. Smith and H. E. Swanson, "New tests of the universality of free fall", Phys. Rev. D50 3614-3636 (1994).
  * [11] J. Luo, X. Chen, J. G. Li, S. H. Fan and F. G. Li, "Test of abnormaly large coherent scattering cross section using solar-neutrino", Chin. Phys. Lett. 12 136-139 (1995).
  * [12] G. Q. Feng, S. Q. Yang, L. C. Tu and J. Luo, "Improvement of test of solar neutrino coherent scattering with torsion pendullum", Chin. Phys. Lett. 23 2052-2055 (2006).
  * [13] D. Akimov et al, COHERENT Collaboration, "Observation of coherent elastic neutrino-nucleus scattering", Science 357 1123-1126 (2017).
  * [14] E. Akhmedov, G. Arcadi, M. Lindner and S. Vogl, "Coherent scattering and macroscopic coherence: implications for neutrino, dark matter and axion detection", JHEP (2018), doi: 10.10007/JHEP10 (2018)045.
  * [15] G. Nicolescu, "A heuristic approach to the detection of solar neutrinos", J. Phys. G: Nucl. Part. Phys. 40 055201 (2013).
  * [16] I. Cruceru, G. Nicolescu and O. G. Duliu, "A dedicated torsion balance to detect neutrinos by coherent scattering on high Debye temperature monocrystals", Int. J. Mod. Phys. A23 1719-1728 (2008).
  * [17] M. Cruceru, G. Nicolescu, O. G. Duliu and I. Cruceru, "Detection of solar neutrinos by coherent scattering in high-Debye temperature monocrystals", Int. J. Mod. Phys. A26 2773-2782 (2011).