# Free Form Medical Visual Question Answering in Radiology

Abhishek Narayanan  Rushabh Musthyala  Rahul Sankar  Anirudh Prasad Nistala
Pranav Singh  Jacopo Cirrone

###### Abstract

Visual Question Answering (VQA) in the medical domain presents a unique,
interdisciplinary challenge, combining fields such as Computer Vision, Natural
Language Processing, and Knowledge Representation. Despite its importance,
research in medical VQA has been scant, only gaining momentum since 2018.
Addressing this gap, our research delves into the effective representation of
radiology images and the joint learning of multimodal representations,
surpassing existing methods. We innovatively augment the SLAKE dataset,
enabling our model to respond to a more diverse array of questions, not
limited to the immediate content of radiology or pathology images. Our model
achieves a top-1 accuracy of 79.55% with a less complex architecture,
demonstrating comparable performance to current state-of-the-art models. This
research not only advances medical VQA but also opens avenues for practical
applications in diagnostic settings.

Machine Learning, ICML

1Department of Computer Science, Courant Institute of Mathematical Sciences,
New York University, New York, USA
2Center for Data Science, New York University, New York, USA
3Colton Center for Autoimmunity, NYU Grossman School of Medicine, New York,
USA
4Corresponding Author (E-mail<EMAIL_ADDRESS>

##  1 Introduction

Recent advancements in Visual Question Answering (VQA) in the medical and
healthcare sectors have garnered significant interest, building upon extensive
research in general, free-form, and open-ended VQA (Lin et al., 2023). Unlike
traditional AI agents in medicine, often constrained to specific organs or
diseases, a medical VQA system should adeptly handle natural language
questions, comprehend medical imagery, and provide diagnostically accurate and
reliable responses.

Nevertheless, medical VQA faces unique challenges compared to its generic
counterpart. For example, while large-scale annotated datasets like VQA (Al-
Sadi et al., 2019) exist for general VQA, medical VQA datasets are smaller,
requiring costly expert annotation and specialized medical knowledge.
Synthetically generating question-image pairs is typically inappropriate due
to the need for clinical relevance and domain-specific expertise.
Additionally, generic VQA models struggle to adapt to medical images. These
models require further specialization and the ability to focus on finer
details, such as microscopic lesions, crucial for diagnosis. The unrestricted
and frequently highly technical nature of the input questions, which may
contain medical terminology not adequately represented by generic language
models trained on expansive databases like Wikipedia, further increases the
complexity of medical VQA.

Medical VQA holds immense potential in healthcare, offering valuable support
where clinician availability is limited. Given the vast number of queries and
the operational scale, it is often challenging for clinicians to address each
query promptly. This can lead to delays in addressing critical health
inquiries, potentially slowing down the diagnosis of severe conditions with
significant consequences. Furthermore, search engine responses, while
abundant, tend to be generic, error-prone, irrelevant, and sometimes
misleading. This underscores the necessity for an AI system capable of
analyzing medical images and providing specific answers to related questions.
Such a system could also assist clinicians by offering a secondary opinion on
interpreting complex images.

In our work, we introduce an enhanced radiology dataset used for the
pretraining of domain-specific visual encoders. Our experiments with various
deep learning models focus on efficient image and text representation
learning. We demonstrate that intra-domain transfer learning is more effective
than inter-domain transfer learning for medical VQA tasks. Our proposed method
not only matches benchmark accuracy but also has a simpler architectural
design.

##  2 Literature Survey

###  2.1 Existing Datasets

To evaluate the performance of VQA models in the medical field, various
datasets have been created. In this study, we specifically focus on the
radiology sector, thereby concentrating our review on datasets relevant to
radiology imagery.

A notable example for benchmarking medical VQA models is the VQA-Med
dataset(Ben Abacha et al., 2021). This dataset is particularly rich in content
related to radiology images and reports. It comprises 4,500 radiology images
paired with 4,500 question-and-answer combinations for training. Additionally,
it includes sets of 500 images and 500 corresponding question-answer pairs
each for both validation and testing purposes.

In addition to the VQA-Med dataset, there are other notable datasets in the
medical VQA field. The VQA-RAD dataset(Lau et al., 2018), for instance,
includes 315 radiology images accompanied by 3,515 question-answer pairs.
Another significant resource is the ChestX-ray8 dataset(Wang et al., 2017b),
which boasts over 100,000 chest X-ray images paired with associated textual
reports. This dataset has been instrumental not only for VQA but also for
various other medical image analysis tasks. Moreover, the SLAKE dataset(Liu et
al., 2021d) contributes to the diversity of resources. It is a bilingual VQA
dataset containing 642 radiology images from various body parts, along with
more than 15,000 question-answer pairs.

###  2.2 Related Work

In their survey, (Lin et al., 2021) analyzed 46 existing medical VQA works, 39
of which are variations of a common underlying structure, as shown in Fig 1.
This structure is known as the joint embedding framework, a baseline model
frequently used for comparison. Based on general VQA, this framework has an
image vectorizer, a question vectorizer, a fusion algorithm that combines
features from both modes, and an answer generator that can be used as either a
classifier or a generative model. The survey shows that a lot of different
methods (Gong et al., 2021; Gupta et al., 2021a; Sharma et al., 2021b; Liu et
al., 2021b) use CNN models trained on ImageNet data (Deng et al., 2009),
mainly ResNet (He et al., 2016), to do tasks using datasets like VQA-RAD and
SLAKE (Liu et al., 2021d; Lau et al., 2018) that are important to our study.
These models use the pretrained weights for either initial weight setting or
end-to-end fine-tuning. Despite its theoretical viability, using ImageNet,
which has a data distribution vastly different from radiology, might not yield
optimal results. Nevertheless, this practice is widespread, primarily due to
the scarcity of large annotated medical datasets suitable for supervised
pretraining of the image model.

Figure 1: Joint embedding framework for Medical VQA (Lin et al., 2021)

For the text encoder component, language models often employed include
variations of recurrent neural networks, GloVe (Li et al., 2021) , and other
word embedding methodologies(He et al., 2020; Li et al., 2021; Liu et al.,
2021c). While models pre-trained on general domain data exhibit reasonable
performance, there has been limited advancement in enhancing the text encoding
channel. However, recent approaches(Jung et al., 2020; Chen et al., 2020) have
begun integrating models like BioBERT(Lee et al., 2020), which are pre-trained
on medical datasets. These integrations have not only surpassed previous
benchmarks but also highlighted the promising potential of research in this
area.

In the realm of fusion approaches for combining image and text modalities,
some studies have implemented straightforward techniques like element-wise
product or feature concatenation (Allaouzi & Ahmed, 2018; Ambati & Dudyala,
2018; Gupta et al., 2021b) , drawing inspiration from generic VQA. These
methods are somewhat effective, but not entirely optimal, as they often fail
to adequately capture the interaction between the two modalities, particularly
in pinpointing the image regions targeted by the question. While multimodal
pooling has shown effectiveness in enhancing accuracy in generic VQA, it does
come with increased computational demands. Despite this, only a few existing
approaches have adopted these pooling methods(Yu et al., 2017, 2018) or have
proposed unique fusion techniques(Sharma et al., 2021a; Vu et al., 2020),
indicating potential areas for further advancement. Attention-based methods,
as cited in(Kim et al., 2018; Lu et al., 2016; Vaswani et al., 2017), have
significantly improved upon baseline models in generic VQA. However, their
adoption in medical VQA is limited, as these complex architectures often rely
on the extensive data available in the general domain, a luxury not typically
available in medical datasets.

Recently, a limited number of approaches have begun to investigate the
pretraining of image models to create more effective image representations.
This exploration utilizes supervised or semi-supervised learning on
alternative medical data sources(Liu et al., 2021a). Eslami et al. (Eslami et
al., 2021) delved into contrastive multi-modal learning, employing embeddings
generated through contrastive language-image pretraining (CLiP) (Radford et
al., 2021). However, their methodology did not extend to the language model.
Their proposed model is notably complex, incorporating an additional
autoencoder (AE) alongside CLiP for image encoding. Despite the complexity,
these recent developments in the field have shown encouraging results, which
our proposed model seeks to build upon and refine.

##  3 Proposed Methodology

###  3.1 Baseline models

For our experimental comparison, we have chosen the previously mentioned joint
embedding framework as our baseline model. This framework employs VGGNet
(Simonyan & Zisserman, 2014) as the backbone for the image encoder and a
bidirectional LSTM for the text encoder. The integration of these two
modalities is achieved through an element-wise product prior to
classification. In our approach, we conceptualize VQA as a classification task
rather than a language generation problem. While this method is advantageous
for generating short answers, it tends to be less effective for responses that
require longer phrases or sentences. However, to maintain simplicity in model
evaluation and ensure consistent comparisons against state-of-the-art models
on the SLAKE benchmark dataset, we have structured the architecture as a
classifier model.

The traditional transfer learning approach, as applied in the context
described above, encounters specific challenges within the medical domain. In
the baseline model, the image and text encoders are initialized using weights
from models pre-trained on datasets like ImageNet. This strategy is based on
the assumption that there could be underlying, shared knowledge beneficial to
the target domain. This hypothesis persists despite the apparent disconnect in
data distribution between general datasets like ImageNet and the more
specialized fields of radiology and medicine.

He et al. (He et al., 2018) highlighted that although an ImageNet pre-trained
model can accelerate convergence, it may not enhance performance. Particularly
in larger medical datasets, the advantage of an ImageNet pre-trained model
over simpler models is negligible(Raghu et al., 2019). Conversely, Zhang et
al.(Zhang et al., 2023) demonstrated that an inappropriate selection of a
foundational dataset or model for pre-training could lead to worse performance
than foregoing transfer learning altogether. With these insights, our
experiments aim to investigate the efficacy of transfer learning from models
specifically trained on radiology image datasets, such as a DenseNet(Huang et
al., 2018) model pre-trained on the RadImageNet dataset(Mei et al., 2022), and
medical texts, like BioBERT(Lee et al., 2020). This exploration will help us
assess the relative benefits and drawbacks of cross-domain versus intra-domain
transfer for our particular case. We are also looking into self-supervised
learning with CASS(Singh & Cirrone, 2023). In this case, we train the CNN-
Visual Transformer (ViT) cross-architecture model on a larger set of x-ray
images before making it work better for our task.

###  3.2 Data Augmentation

A significant challenge in medical VQA, as reported in numerous studies, is
the limited size of available datasets. For perspective, the general VQA
dataset(vis, ) contains over 200,000 images, whereas medical-specific datasets
like VQA-RAD(Lau et al., 2018) and SLAKE(Liu et al., 2021d) only have a few
hundred images. Addressing this issue, Kovaleva et al.(Kovaleva et al., 2019)
utilized MIMIC-CXR reports and images to create conversational-style question-
answer pairs, employing the Chexpert (Irvin et al., 2019) to generate QA pairs
for each image. Building on this methodology, we aim to generate additional QA
pairs from existing radiology datasets available online. We applied this
strategy to datasets such as Chest X-rays (Indiana University) (Raddar, 2020),
COVID CXR-2(kag, a), RSNA Pneumonia Detection(kag, b), and the NIH Chest
X-rays dataset(of Health Chest X-Ray Dataset, 2018). This approach yielded a
combined dataset of approximately 20,000 QA pairs, nearly triple the size of
the SLAKE dataset. Our comprehensive dataset covers a broader spectrum of
questions, having been trained on a diverse collection of images and reports.
It encompasses over 12 diseases and includes various body parts, scan
orientations, and comments on different images. We created this dataset with
the intention of expanding the range of questions and answers that a trained
model can effectively handle. Additionally, the increased size of our dataset
mitigates the risk of overfitting.

###  3.3 Image Encoder Pre-training

A notable concern highlighted in existing literature pertains to the quality
of image encodings in multi-modal representations. General-purpose VQA methods
often benefit from leveraging weights of image models pre-trained on the
expansive ImageNet dataset(Deng et al., 2009), which houses over a million
images. However, the type of images in ImageNet significantly differs from
those in radiology, leading to less optimal pre-trained ImageNet weights for
encoding radiology images. To address this, we opted to use an image encoder
pre-trained specifically on radiology images. Theoretically, this approach
should yield more accurate image encodings and, consequently, enhance overall
performance. We utilized the DenseNet and ElasticNet models from
TorchXRayVision (Cohen et al., 2022) , a Python library featuring CNN models
pre-trained on a blend of radiology datasets including RSNA Pneumonia
Detection(kag, b), NIH Chest X-rays dataset(of Health Chest X-Ray Dataset,
2018), PadChest(Bustos et al., 2020), CheXpert(Irvin et al., 2019) , and
MIMIC(Johnson et al., 2019). Additionally, we experimented with pre-training
the CASS architecture(Singh & Cirrone, 2023) using images from our larger
combined dataset in a self-supervised manner.

###  3.4 Joint learning of effective multi-modal representations with cross-
modal supervision

In the field of generic VQA, recent studies(Shen et al., 2021) have shown
significant success using Contrastive Language-Image Pre-training (CLIP) for
learning cross-modal supervision with extensive image-text pairs. However, the
application of CLIP has been mainly focused on general-domain multi-modal
challenges. Drawing inspiration from these developments, PubMedClip(Eslami et
al., 2021) utilized a variant of CLIP(Radford et al., 2021), fine-tuned with
medical image-text pairs from the ROCO dataset(Pelka et al., 2018), achieving
state-of-the-art results on the VQA-RAD(Lau et al., 2018) and SLAKE (Liu et
al., 2021d) datasets. Nevertheless, PubMedClip has two main limitations:
firstly, it functions solely as a pre-trained visual encoder, not exploiting
CLIP’s full multi-modal potential for encoding both images and text. Secondly,
it employs CLIP’s image and text encoders pre-trained on general-domain data,
which are not specifically related to radiology. To address these issues, we
propose the use of MedCLiP(Wang et al., 2022), trained on the MedPix
dataset(med, ), which includes MRI, X-Ray, and CT Scan data. MedCLiP utilizes
ClinicalBERT(Huang et al., 2019) for text encoding and ResNet50 for image
encoding. In our pipeline, MedCLiP is integrated into both the image and text
embedding channels, initially freezing its weights for feature extraction.
Subsequently, we fine-tune the derived embeddings for downstream VQA tasks.

##  4 Data And Experiment Setup

In our research, we employ the SLAKE dataset(Liu et al., 2021d) for fine-
tuning and comprehensive evaluation of our proposed models, benchmarking them
against existing standards. SLAKE is a robust dataset featuring 642 radiology
images sourced from three open-source datasets(Simpson et al., 2019; Wang et
al., 2017a; Kavur et al., 2021), encompassing a range of modalities (CT, MRI,
X-Ray) and body parts (head, neck, chest, abdomen, and pelvic cavity). It
includes 14,028 question-answer pairs in English and Chinese, curated from
experienced doctors who selected or modified pre-defined questions. These
questions are categorized by type and balanced to mitigate statistical bias.
The dataset is divided into training (70%), validation (15%), and test (15%)
sets at the image level for each body part-modality category (e.g., headCT,
chestXRay, etc.), yielding 450, 96, and 96 images for training, validation,
and testing, respectively. Our study focuses exclusively on the English
question-answer subset (7,000 pairs) to align with the benchmark. The
experimental pipeline is outlined in Fig 1. We benchmark against the accuracy
of PubMedClip (Eslami et al., 2021) on SLAKE. For training in all experiments,
we use AdaDelta optimization. These experiments are conducted on 2 V100 GPUs
over 150 epochs, with a batch size of 32.

##  5 Results

We reproduced the results of (Eslami et al., 2021) on the SLAKE dataset and
obtained an accuracy (top-1) of 79.45%. We also implemented and tested the
aforementioned baseline model with the VGGNet+LSTM backbone, which performs
reasonably well, producing a test accuracy of 75% on the SLAKE dataset but
leaving out scope for tremendous improvement in the state-of-the-art. By
bridging the gaps we identified in existing work, we hypothesize that the
proposed approach should perform equally well or outperform the state-of-the-
art PubMedClip model on the aforementioned datasets. Fig. 2. shows the output
of various models tested as part of the PubMedClip paper implementation vs.
our baseline.

Figure 2: Predictions of proposed model vs that of PubMedClip (Eslami et al.,
2021)

Table 1. portrays the accuracy metric for the various models tested by the
authors of (Eslami et al., 2021) vs. our initial baseline. The learning curves
of the models that were trained have been portrayed in Fig. 3

Figure 3: Training and validation learning curves for our experiments Table 1:  Comparison of our models with existing approaches wrt accuracy Image Encoder |  Language Encoder |  Validation Accuracy |  Test Accuracy
---|---|---|---
ElasticNet AE (Pretrained) (Zou & Hastie, 2005) |  BiLSTM |  68.85% |  64.43%
DenseNet (Pretrained) (Huang et al., 2018) |  BiLSTM |  81.29% |  77.09%
VGG16 (Simonyan & Zisserman, 2014) |  BiLSTM |  76.44% |  75.0%
MedCLIP (Wang et al., 2022) |  BiLSTM |  80.53% |  79.55%
MedCLIP (Wang et al., 2022) |  MedCLIP (Wang et al., 2022) |  61.25% |  58.34%
CASS-ViT (Singh & Cirrone, 2023) |  BioClinicalBERT (Alsentzer et al., 2019) |  56.41% |  56.73%
PubMedCLIP ViT + AE (Eslami et al., 2021) |  GloVe + LSTM |  N/A |  80.1%
MedCLIP (Wang et al., 2022) |  BioClinicalBERT (Alsentzer et al., 2019) |  59.34% |  59.29%

As expected, from Section 3.1, we see a boost in performance when using the
radiology-pretrained DenseNet as opposed to a CNN model that was pretrained
using ImageNet. From Fig. 4 we can see that by virtue of making a bigger
training dataset, we have enhanced the answering capability of our VQA model.
Trained on only the SLAKE dataset, a VQA model would not be able to diagnose
the patient on the right with calcified granuloma. However, since we have
incorporated data from multiple sources into our training step, our model is
now able to successfully make the diagnosis.

Figure 4: Performance on questions not in the SLAKE dataset

##  6 Discussion

In this study, we introduce an augmented dataset specifically designed for
pre-training visual encoders for medical images. Our experimentation involved
pre-training various image models on an extensive radiology dataset while
employing domain-specific language models for encoding questions. This
approach highlights the superior efficacy of intra-domain transfer learning in
medical visual question answering, in contrast to the inter-domain transfer
learning prevalent in previous methodologies.

We also proposed the use of MedCLiP for the simultaneous development of
effective multi-modal representations through cross-modal supervision. Our
results on the SLAKE dataset show that this approach is on par with state-of-
the-art models yet benefits from a simpler architecture and reduced model
complexity. Our VQA model, incorporating MedCLiP as the image encoder,
surpasses the baseline by 4.55%. However, the performance significantly
diminishes when CLiP embeddings are employed for both image and question
encoders. This drop is likely due to the disparity in distributions between
the clinical captions used in training and the question-style text in our
dataset.

#### Limitations

Despite these advancements, our approach does face limitations. One key issue
is the lack of explainability and interpretability in the model, which are
crucial for its practical deployment and ensuring the reliability of its
predictions before making clinical decisions. Additionally, since the problem
is framed as a classification task, the model’s ability to predict answers is
confined to a fixed vocabulary, limiting its overall scope and applicability.

## Acknowledgments

We are grateful to New York University’s High Performance Computing team for
providing us with the necessary computing support and resources to train our
models.

## References

  * kag (a) Covidx cxr-2, a.  URL https://www.kaggle.com/datasets/andyczhao/covidx-cxr2.
  * kag (b) Rsna pneumonia detection challenge, b.  URL https://www.kaggle.com/competitions/rsna-pneumonia-detection-challenge/data.
  * (3) Medpix.  URL https://medpix.nlm.nih.gov/home.
  * (4) Vqa.  URL https://visualqa.org/index.html.
  * Al-Sadi et al. (2019) Al-Sadi, A., Talafha, B., Al-Ayyoub, M., Jararweh, Y., and Costen, F.  Just at imageclef 2019 visual question answering in the medical domain.  In _CLEF (working notes)_ , 2019.
  * Allaouzi & Ahmed (2018) Allaouzi, I. and Ahmed, M. B.  Deep neural networks and decision tree classifier for visual question answering in the medical domain.  In _CLEF (Working Notes)_ , 2018.
  * Alsentzer et al. (2019) Alsentzer, E., Murphy, J. R., Boag, W., Weng, W., Jin, D., Naumann, T., and McDermott, M. B. A.  Publicly available clinical BERT embeddings.  _CoRR_ , abs/1904.03323, 2019.  URL http://arxiv.org/abs/1904.03323.
  * Ambati & Dudyala (2018) Ambati, R. and Dudyala, C. R.  A sequence-to-sequence model approach for imageclef 2018 medical domain visual question answering.  In _2018 15th IEEE India Council International Conference (INDICON)_ , pp. 1–6. IEEE, 2018.
  * Ben Abacha et al. (2021) Ben Abacha, A., Sarrouti, M., Demner-Fushman, D., Hasan, S. A., and Müller, H.  Overview of the vqa-med task at imageclef 2021: Visual question answering and generation in the medical domain.  In _Proceedings of the CLEF 2021 Conference and Labs of the Evaluation Forum-working notes_. 21-24 September 2021, 2021.
  * Bustos et al. (2020) Bustos, A., Pertusa, A., Salinas, J.-M., and de la Iglesia-Vayá, M.  PadChest: A large chest x-ray image dataset with multi-label annotated reports.  _Medical Image Analysis_ , 66:101797, dec 2020.  doi: 10.1016/j.media.2020.101797.  URL https://doi.org/10.1016%2Fj.media.2020.101797.
  * Chen et al. (2020) Chen, G., Gong, H., and Li, G.  Hcp-mic at vqa-med 2020: Effective visual representation for medical visual question answering.  In _CLEF (Working Notes)_ , 2020.
  * Cohen et al. (2022) Cohen, J. P., Viviano, J. D., Bertin, P., Morrison, P., Torabian, P., Guarrera, M., Lungren, M. P., Chaudhari, A., Brooks, R., Hashir, M., and Bertrand, H.  TorchXRayVision: A library of chest X-ray datasets and models.  In _Medical Imaging with Deep Learning_ , 2022.  URL https://github.com/mlmed/torchxrayvision.
  * Deng et al. (2009) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pp. 248–255. Ieee, 2009.
  * Eslami et al. (2021) Eslami, S., de Melo, G., and Meinel, C.  Does clip benefit visual question answering in the medical domain as much as it does in the general domain?, 2021.
  * Gong et al. (2021) Gong, H., Chen, G., Liu, S., Yu, Y., and Li, G.  Cross-modal self-attention with multi-task pre-training for medical visual question answering.  In _Proceedings of the 2021 international conference on multimedia retrieval_ , pp. 456–460, 2021.
  * Gupta et al. (2021a) Gupta, D., Suman, S., and Ekbal, A.  Hierarchical deep multi-modal network for medical visual question answering.  _Expert Systems with Applications_ , 164:113993, 2021a.
  * Gupta et al. (2021b) Gupta, D., Suman, S., and Ekbal, A.  Hierarchical deep multi-modal network for medical visual question answering.  _Expert Systems with Applications_ , 164:113993, 2021b.
  * He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * He et al. (2018) He, K., Girshick, R., and Dollár, P.  Rethinking imagenet pre-training, 2018.
  * He et al. (2020) He, X., Zhang, Y., Mou, L., Xing, E., and Xie, P.  Pathvqa: 30000+ questions for medical visual question answering.  _arXiv preprint arXiv:2003.10286_ , 2020.
  * Huang et al. (2018) Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K. Q.  Densely connected convolutional networks, 2018.
  * Huang et al. (2019) Huang, K., Altosaar, J., and Ranganath, R.  Clinicalbert: Modeling clinical notes and predicting hospital readmission.  _arXiv preprint arXiv:1904.05342_ , 2019.
  * Irvin et al. (2019) Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R. L., Shpanskaya, K. S., Seekins, J., Mong, D. A., Halabi, S. S., Sandberg, J. K., Jones, R., Larson, D. B., Langlotz, C. P., Patel, B. N., Lungren, M. P., and Ng, A. Y.  Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison.  _CoRR_ , abs/1901.07031, 2019.  URL http://arxiv.org/abs/1901.07031.
  * Johnson et al. (2019) Johnson, A. E. W., Pollard, T. J., Greenbaum, N. R., Lungren, M. P., ying Deng, C., Peng, Y., Lu, Z., Mark, R. G., Berkowitz, S. J., and Horng, S.  Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs, 2019.
  * Jung et al. (2020) Jung, B., Gu, L., and Harada, T.  bumjun_jung at vqa-med 2020: Vqa model based on feature extraction and multi-modal feature fusion.  In _CLEF (Working Notes)_ , 2020.
  * Kavur et al. (2021) Kavur, A. E., Gezer, N. S., Barış, M., Aslan, S., Conze, P.-H., Groza, V., Pham, D. D., Chatterjee, S., Ernst, P., Özkan, S., Baydar, B., Lachinov, D., Han, S., Pauli, J., Isensee, F., Perkonigg, M., Sathish, R., Rajan, R., Sheet, D., Dovletov, G., Speck, O., Nürnberger, A., Maier-Hein, K. H., Akar, G. B., Ünal, G., Dicle, O., and Selver, M. A.  CHAOS challenge - combined (CT-MR) healthy abdominal organ segmentation.  _Medical Image Analysis_ , 69:101950, apr 2021.  doi: 10.1016/j.media.2020.101950.  URL https://doi.org/10.1016%2Fj.media.2020.101950.
  * Kim et al. (2018) Kim, J.-H., Jun, J., and Zhang, B.-T.  Bilinear attention networks.  _Advances in neural information processing systems_ , 31, 2018.
  * Kovaleva et al. (2019) Kovaleva, O., Shivade, C. P., Kashyap, S., Kanjaria, K., Coy, A., Ballah, D., Guo, Y., Wu, J. T., Karargyris, A., Beymer, D. J., Rumshisky, A., and Mukherjee, V. V.  Visual dialog for radiology: Data curation and firststeps.  In _ViGIL@NeurIPS_ , 2019.
  * Lau et al. (2018) Lau, J. J., Gayen, S., Ben Abacha, A., and Demner-Fushman, D.  A dataset of clinically generated visual questions and answers about radiology images.  _Scientific data_ , 5(1):1–10, 2018.
  * Lee et al. (2020) Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., and Kang, J.  Biobert: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_ , 36(4):1234–1240, 2020.
  * Li et al. (2021) Li, Y., Yang, Z., and Hao, T.  Tam at vqa-med 2021: A hybrid model with feature extraction and fusion for medical visual question answering.  In _CLEF (Working Notes)_ , pp. 1295–1304, 2021.
  * Lin et al. (2021) Lin, Z., Zhang, D., Tac, Q., Shi, D., Haffari, G., Wu, Q., He, M., and Ge, Z.  Medical visual question answering: A survey.  _arXiv preprint arXiv:2111.10056_ , 2021.
  * Lin et al. (2023) Lin, Z., Zhang, D., Tao, Q., Shi, D., Haffari, G., Wu, Q., He, M., and Ge, Z.  Medical visual question answering: A survey.  _Artificial Intelligence in Medicine_ , pp. 102611, 2023.
  * Liu et al. (2021a) Liu, B., Zhan, L.-M., and Wu, X.-M.  Contrastive pre-training and representation distillation for medical visual question answering based on radiology images.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part II 24_ , pp. 210–220. Springer, 2021a.
  * Liu et al. (2021b) Liu, B., Zhan, L.-M., and Wu, X.-M.  Contrastive pre-training and representation distillation for medical visual question answering based on radiology images.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part II 24_ , pp. 210–220. Springer, 2021b.
  * Liu et al. (2021c) Liu, B., Zhan, L.-M., and Wu, X.-M.  Contrastive pre-training and representation distillation for medical visual question answering based on radiology images.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part II 24_ , pp. 210–220. Springer, 2021c.
  * Liu et al. (2021d) Liu, B., Zhan, L.-M., Xu, L., Ma, L., Yang, Y., and Wu, X.-M.  Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering.  In _2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI)_ , pp. 1650–1654. IEEE, 2021d.
  * Lu et al. (2016) Lu, J., Yang, J., Batra, D., and Parikh, D.  Hierarchical question-image co-attention for visual question answering.  _Advances in neural information processing systems_ , 29, 2016.
  * Mei et al. (2022) Mei, X., Liu, Z., Robson, P. M., Marinelli, B., Huang, M., Doshi, A., Jacobi, A., Cao, C., Link, K. E., Yang, T., Wang, Y., Greenspan, H., Deyer, T., Fayad, Z. A., and Yang, Y.  Radimagenet: An open radiologic deep learning research dataset for effective transfer learning.  _Radiology: Artificial Intelligence_ , 4(5):e210315, 2022.  doi: 10.1148/ryai.210315.  URL https://doi.org/10.1148/ryai.210315.
  * of Health Chest X-Ray Dataset (2018) of Health Chest X-Ray Dataset, N. I.  Nih chest x-rays, Feb 2018.  URL https://www.kaggle.com/datasets/nih-chest-xrays/data.
  * Pelka et al. (2018) Pelka, O., Koitka, S., Rückert, J., Nensa, F., and Friedrich, C. M.  Radiology objects in context (roco): A multimodal image dataset.  In Stoyanov, D., Taylor, Z., Balocco, S., Sznitman, R., Martel, A., Maier-Hein, L., Duong, L., Zahnd, G., Demirci, S., Albarqouni, S., Lee, S.-L., Moriconi, S., Cheplygina, V., Mateus, D., Trucco, E., Granger, E., and Jannin, P. (eds.), _Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis_ , pp. 180–189, Cham, 2018. Springer International Publishing.  ISBN 978-3-030-01364-6.
  * Raddar (2020) Raddar.  Chest x-rays (indiana university), Feb 2020.  URL https://www.kaggle.com/datasets/raddar/chest-xrays-indiana-university.
  * Radford et al. (2021) Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I.  Learning transferable visual models from natural language supervision, 2021.
  * Raghu et al. (2019) Raghu, M., Zhang, C., Kleinberg, J., and Bengio, S.  Transfusion: Understanding transfer learning for medical imaging, 2019.
  * Sharma et al. (2021a) Sharma, D., Purushotham, S., and Reddy, C. K.  Medfusenet: An attention-based multimodal deep learning model for visual question answering in the medical domain.  _Scientific Reports_ , 11(1):19826, 2021a.
  * Sharma et al. (2021b) Sharma, H., Drukker, L., Papageorghiou, A. T., and Noble, J. A.  Multi-modal learning from video, eye tracking, and pupillometry for operator skill characterization in clinical fetal ultrasound.  In _2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI)_ , pp. 1646–1649. IEEE, 2021b.
  * Shen et al. (2021) Shen, S., Li, L. H., Tan, H., Bansal, M., Rohrbach, A., Chang, K., Yao, Z., and Keutzer, K.  How much can CLIP benefit vision-and-language tasks?  _CoRR_ , abs/2107.06383, 2021.  URL https://arxiv.org/abs/2107.06383.
  * Simonyan & Zisserman (2014) Simonyan, K. and Zisserman, A.  Very deep convolutional networks for large-scale image recognition.  _arXiv 1409.1556_ , 09 2014.
  * Simpson et al. (2019) Simpson, A. L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., van Ginneken, B., Kopp-Schneider, A., Landman, B. A., Litjens, G., Menze, B., Ronneberger, O., Summers, R. M., Bilic, P., Christ, P. F., Do, R. K. G., Gollub, M., Golia-Pernicka, J., Heckers, S. H., Jarnagin, W. R., McHugo, M. K., Napel, S., Vorontsov, E., Maier-Hein, L., and Cardoso, M. J.  A large annotated medical image dataset for the development and evaluation of segmentation algorithms, 2019.
  * Singh & Cirrone (2023) Singh, P. and Cirrone, J.  Efficient representation learning for healthcare with cross-architectural self-supervision.  In Deshpande, K., Fiterau, M., Joshi, S., Lipton, Z., Ranganath, R., Urteaga, I., and Yeung, S. (eds.), _Proceedings of the 8th Machine Learning for Healthcare Conference_ , volume 219 of _Proceedings of Machine Learning Research_ , pp. 691–711. PMLR, 11–12 Aug 2023.  URL https://proceedings.mlr.press/v219/singh23a.html.
  * Vaswani et al. (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I.  Attention is all you need.  _Advances in neural information processing systems_ , 30, 2017.
  * Vu et al. (2020) Vu, M. H., Löfstedt, T., Nyholm, T., and Sznitman, R.  A question-centric model for visual question answering in medical imaging.  _IEEE transactions on medical imaging_ , 39(9):2856–2868, 2020.
  * Wang et al. (2017a) Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R. M.  ChestX-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases.  In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, jul 2017a.  doi: 10.1109/cvpr.2017.369.  URL https://doi.org/10.1109%2Fcvpr.2017.369.
  * Wang et al. (2017b) Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R. M.  Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 2097–2106, 2017b.
  * Wang et al. (2022) Wang, Z., Wu, Z., Agarwal, D., and Sun, J.  Medclip: Contrastive learning from unpaired medical images and text.  _arXiv preprint arXiv:2210.10163_ , 2022.
  * Yu et al. (2017) Yu, Z., Yu, J., Fan, J., and Tao, D.  Multi-modal factorized bilinear pooling with co-attention learning for visual question answering.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 1821–1830, 2017.
  * Yu et al. (2018) Yu, Z., Yu, J., Xiang, C., Fan, J., and Tao, D.  Beyond bilinear: Generalized multimodal factorized high-order pooling for visual question answering.  _IEEE transactions on neural networks and learning systems_ , 29(12):5947–5959, 2018.
  * Zhang et al. (2023) Zhang, W., Deng, L., Zhang, L., and Wu, D.  A survey on negative transfer.  _IEEE/CAA Journal of Automatica Sinica_ , 10(2):305–329, feb 2023.  doi: 10.1109/jas.2022.106004.  URL https://doi.org/10.1109%2Fjas.2022.106004.
  * Zou & Hastie (2005) Zou, H. and Hastie, T.  Regularization and variable selection via the elastic net.  _Journal of the Royal Statistical Society Series B: Statistical Methodology_ , 67(2):301–320, 2005.


# Selecting the Number of Sensor Scans in Surveillance Operations

Graham V. Weinberg
<EMAIL_ADDRESS>
###### Abstract

Searching for concealed threats within a surveillance region is an important
role for military sensors. One prominent case is the search for a submarine by
a helicopter deploying a dipping sonar in anti-submarine warfare. Another is
the utilisation of an uncrewed aerial vehicle for remote detection of land
mines. These platforms will deploy an appropriate sensor to search within a
given region for a potential threat. The sensors employed may not provide
continuous surveillance, but may have active periods where the inherent sensor
is scanning for threats. The reasons for disjoint activation periods may be
both functional as well as tactical. Therefore, given a specified search
pattern under which the surveillance platform operates, the question of the
number of sensor scans required becomes relevant for search mission pre-
planning. Hence it is not only important to develop a modelling framework to
assess performance but it is of relevance to consider whether a minimum number
of sensor scans can be determined to guarantee a desired level of performance.
Consequently this paper introduces a framework for this and illustrates how it
can be utilised in practice.

##  I Introduction

Successful detection of low-signature threats from remote surveillance
platforms is an important capability for the modern fighting force. This is
particularly problematic since there is a juxtaposition between improvements
in both stealth and sensing technology. Nonetheless there is a need to provide
a performance prediction framework to assess the success in detecting relevant
threats with sensor systems. The case of detection of a submarine from a
helicopter utilising a dipping sonar is a good example of the complexities in
successful detection [1, 2, 3]. Dipping sonars are omnidirectional
transceivers which are deployed for a short period of time in anti-submarine
warfare (ASW) operations. The sea environment has a serious effect on the
relative success of detection, since sonar pings are attenuated by
temperature, sea-state, salinity and bathymetry [4]. In addition to this, the
sound speed profile varies with range and so while a submarine is already a
low acoustic-signature threat, the environment itself may impede successful
detection [4].

In a similar vein, an uncrewed aerial vehicle (UAV) may be equipped with land
mine detection functionality, and so is capable of remote detection of land-
mines [5, 6]. In the case of acoustic sensors the environment will also have a
negative impact on successful detection but not as severe as in the undersea
environment. When utilising optical or radar sensors, environmental
characteristics will also contribute to detection performance degradation.

Hence the approach to be adopted in this study is to assume that in a given
operational environment there is a known detection range, such that if a
threat falls within this range it will be sensed by the platform. This is
known as adopting a “cookie-cutter” model for target detection [7]. Doing this
allows approximate rules of thumb to be established to assess operational
capability. For a given operational environment, and with a platform employing
a specific sensor, one may apply well-known engineering functions to determine
maximal detection ranges. In the case of radar, this is the radar-range
equation, while for acoustic sensors it is the well-known sonar equation [4,
8]. Based upon sensor attributes and environmental factors one may then
extract the detection range and use this as a basis for performance prediction
and assessment.

The objective of this paper is to characterise the threat detection problem in
terms of sensor scans and utilising a maximal detection range assumption. In
particular, supposing that a surveillance platform is deploying a sensor,
which is switched on for fixed periods to scan for threats, a model is
developed to determine the probability of detection. The principle of fixed
time scans has its origins in the ASW aspect of this study. However, if a UAV
is deploying a radar then it can become the target of a radio frequency seeker
and so short periodic scans may sustain the platform’s survivability. Once an
expression for the probability of detection is produced, one may proceed to
develop a rule of thumb to determine the minimum number of sensor scans
required to guarantee a desired level of performance.

It is important to note that this study does not consider the problem from the
perspective of determining an optimal sensor platform trajectory but instead
is concerned with performance assessment with a given trajectory. From an ASW
perspective other studies have focused on dipping pattern analysis [2] and
search region coverage [7]. This study’s contribution is to show how a simple
mathematical model may be developed such that it can be used to provide
insights on sensor performance in a number of relevant applications.

The developments in this study will then be illustrated with a simple example
from ASW operations, to demonstrate how it can be applied to a specific sensor
on a given platform.

##  II Surveillance Performance Model

It is assumed that a threat is located somewhere within a bounded surveillance
region, and that this region is viewed as a two-dimensional plane for
modelling simplicity. Suppose that the surveillance path within this region,
traversed by the platform deploying the sensor, is parameteristed through
$\\{(x(t),y(t)),t\in{\cal T}\\}$ which represents a curve in two-dimensional
space, and that the sensor is deployed at times
$\tau_{1},\tau_{2},\ldots,\tau_{N}$, with ${\cal T}$ an interval of time.
Hence the total number of surveillance scans is $N$. For brevity define
$x_{j}=x(\tau_{j})$ and $y_{j}=y(\tau_{j})$ for $j\in\\{1,2,\ldots,N\\}$.

Suppose that at scan time $\tau_{j}$ the threat will be detected by the sensor
if it lies within a maximal detection range, given by $\epsilon>0$. Then if
the threat is located at coordinates $(x_{0},y_{0})$ it will be detected by
the sensor if

$d_{j}=d((x_{j},y_{j}),(x_{0},y_{0}))=\sqrt{(x_{j}-x_{0})^{2}+(y_{j}-y_{0})^{2}}<\epsilon.$
(1)

Expression (1) is the Euclidean distance between the sensor and threat
location. The threat location $(x_{0},y_{0})$ may be modelled through a
statistical distribution, such as assuming that it is uniform in both
coordinates. This will be clarified in the analysis to follow.

Given that there are $N$ scans the probability that the threat is not detected
is exactly the probability that $d_{j}\geq\epsilon$ for each
$j\in\\{1,2,\ldots,N\\}$. If it is assumed that each scan is independent then

${\rm{I\ \kern-5.39993ptP}}({\mbox{Threat not
detected}})=\prod_{j=1}^{N}{\rm{I\ \kern-5.39993ptP}}(d_{j}\geq\epsilon).$ (2)

Hence by complementation it follows that

${\rm{I\ \kern-5.39993ptP}}({\mbox{Threat detected}})=1-\prod_{j=1}^{N}{\rm{I\
\kern-5.39993ptP}}(d_{j}\geq\epsilon).$ (3)

Under the assumption that the sensor deployment times are deterministic it is
not difficult to express the probability in (3) in terms of the conditional
distribution of the threat location. However, (3) may also be evaluated
through Monte Carlo methods in specific examples.

It is now worth providing a specific example of (3). Towards this end, suppose
that the surveillance region is a square defined through the Cartesian product
$[-40,40]\times[-40,40]$ and that the threat’s location is assumed to be
uniformly distributed in this region. The units here are arbitrary but can be
specialised in terms of a specific application. The surveillance path is taken
to be a lemniscate of Booth given by the polar equation $r=20\cos(\theta)$,
where the angle $\theta$ corresponds to uniformly spaced scan points. Hence if
there are $N$ surveillance scans then $\theta_{j}=\frac{2\pi}{N}j$ for
$j\in\\{1,2,\ldots,N\\}$. Figure 1 provides a polar plot of this path. The
selection of this path has been motivated by the fact that it has the
potential to provide better regional coverage than a circular path. Figure 2
plots the probability of detection as a function of $\epsilon$ for the latter
ranging up to 30. Expression (3) has been evaluated using Monte Carlo
simulation with $10^{5}$ runs for each point probability. The figure shows the
probability of detecting the threat, as a function of the number of scans and
the maximal detection range $\epsilon$. As can be observed, increasing the
number of scans will improve performance as expected, and as the sensor’s
maximal detection range increases there is also a corresponding improvement in
performance.

For an explicit example, if the sensor has a range of 10 units then at least
15 scans is required to guarantee a probability of detection of at least 50%.
More than 25 scans would be required to guarantee a probability of detection
of at least 70%. If the sensor had a range of 20 units then only 5 sensor
scans would be required to achieve a probability of detection of approximately
60%. Additionally, 15 scans would be sufficient to ensure that this
probability increases to at least 90%.

Figure 1: Leminscate surveillance path polar plot. Figure 2: Detection
performance as a function sensor effective range $\epsilon$ for various number
of scans $N$.

Although plots such as that in Figure 2 may be used to assess performance of
particular systems it is worth considering whether an approximate guide to the
number of surveillance scans can be produced. In effect one would like to
detemine a suitable minimum $N$ so that (3) is at a desired level. In order to
derive an appropriate rule of thumb for the number of required sensor scans,
suppose that the surveillance region is a square in two-dimensional space
given by the Cartesian product $[-\delta,\delta]\times[-\delta,\delta]$ for
some known $\delta>0$. Assume that the surveillance path is contained within
this region such that for any scan point $(x_{j},y_{j})$ the circle with
radius $\epsilon$ and centre at this point lies within the surveillance region
boundary. Then the probability that the threat is detected at the $j$th scan
is the probability that it lies within the circle, centre $(x_{j},y_{j})$ with
radius $\epsilon$. From a geometric perspective this is exactly the area of
the circle to that of the surveillance region. Therefore

${\rm{I\ \kern-5.39993ptP}}((x_{0},y_{0})\mbox{ lies in the
circle})=\frac{\pi\epsilon^{2}}{4\delta^{2}}.$ (4)

This is independent of the scan position provided the underlying circle lies
within the surveillance region. The complement of expression (4) may now be
applied to (3) and consequently it follows that

${\rm{I\ \kern-5.39993ptP}}({\mbox{Threat
detected}})=1-\left(1-\frac{\pi\epsilon^{2}}{4\delta^{2}}\right)^{N}.$ (5)

The validity of (5) is dependent on the assumption that the series of circles
do not overlap. In order to ensure this, is is necessary to suppose that the
ratio of $\epsilon$ to $\delta$ is sufficiently small. As an example, suppose
that the square of this ratio is 0.05, then Figure 3 plots (5) as a function
of $N$. From this figure one may determine roughly the number of sensor scans
for a desired probability of detection, regardless of the path over which the
surveillance platform operates. However, results derived from this curve will
be somewhat conservative. As an example, the figure suggests that for a
probability of detection of at least 0.7 it is necessary to have at least 30
sensor scans.

Figure 3: Approximate detection performance, based upon (5), where $N$ is the
number of sensor scans.

##  III Application

In this final section an application of the results of the preceding section
is considered. In particular, the problem of ASW with a dipping sonar is
examined, and the following is based upon results discussed in [9]. For
simplicity, suppose that a helicopter is deploying an omnidirectional dipping
sonar, operating in a narrowband mode within a search region which is a square
of length 400 nautical miles (nmi). The location of the submarine is
distributed uniformly within this square. As in the previous analysis, assume
that the helicopter is travelling on a lemniscate path with polar equation
$r=16\cos(\theta)$. The equation’s parameter 16 has been selected based upon
the operational range of the MH-60R Seahawk multimission naval helicopter
whose maximum range is approximately 450 nmi [10]. Using calculus it can be
shown that the path length of the given lemniscate is $\pi/2\times
16^{2}\approx 402$ nmi so that the helicopter should be able to traverse the
full lemniscate path in one mission.

From the active sonar equation, the submarine will be detected if the signal
excess (SE) exceeds zero. Under the assumption that the transmission loss (TL)
from source to target is the same from target to source, the sonar equation is

$SE=SL-2TL+TS-RL-DT,$ (6)

where SL is the source level, TS is the target strength, RL is the
reverberation loss and DT is the detection threshold [11]. This equation is
constructed from the logarithm of the signal to noise ratio, and all its
parameters are in dB. Suppose that the environment has a constant sound speed
profile ($c=3355$ miles per hour) and that RL is a constant multiple of the
SL. The simplified model for TL is given by

$TL=66+10\log_{10}(R)+2\alpha R,$ (7)

where $R$ is the distance to the target and the absorption coefficient
$\alpha$ is given by

$\alpha=0.1f^{2}/(1+f^{2})+40f^{2}/(4100+f^{2})+2.75\times
10^{-4}f^{2}+0.003,$ (8)

where $f$ is the ping frequency.

The model for TS, based upon a cylindrical approximation for the shape of a
submarine, is given by

$TS=(aL^{2}/2\lambda)(\sin(\beta)/\beta)^{2}\cos(\psi)^{2},$ (9)

where $a$ is the radius of a cylinder of length $L$, $\lambda$ is the
wavelength, $\psi$ is the aspect angle off-beam and $\beta=2\pi/\lambda
L\sin(\psi)$.

Finally the DT is given by

$DT=10\log_{10}(d/2T),$ (10)

where $d$ is the detection index and $T$ is the duration of the waveform.

Expressions (7) - (10) may be applied to (6) to produce range-dependent SE
values. In the following $f=10$ Hz with $d=25$ and $T=100$ s. This choice for
$d$ ensures that the corresponding false alarm probability is $10^{-4}$. For
the cylindrical model of the threat, $L=300$ ft, $a=15$ ft and $\psi=\pi/4$.
The SL is taken to be 250 dB and the RL is one tenth of this. This selection
of SL has been partially motivated by discussions in [12] for the MH-60R
helicopter, while the RL has been an arbitrary selection. Figure 4 plots (6)
as a function of the range to the threat. This is then adapted to produce
Figure 5, which graphs (3) for five cases of $N$ as considered previously. The
figure shows that for small SE more scans are required to ensure adequate
probability of detection, while for threats with a stronger SE one may utilise
less scans to achieve likely detection. As tangible examples from Figure 4 at
a range of 140 nmi the SE is approximately 0 dB. Based upon Figure 5 it is
clear that $N=20$ scans is sufficient to guarantee a probability of detection
of at least 70%, while to increase this to at least 80% one must increase the
number of scans to at least 25. By contrast, at a range of 50 nmi the SE is
approximately 60 dB and detection is guaranteed (with a probability of
approximately 90%) with only $N=5$ scans.

Figure 4: A plot of the signal excess (SE) as a function of range, for the
model parameterisations utilised in the study. Detection will occur in cases
where the SE exceeds zero. Figure 5: Probability of detection as a function of
SE for the parameterisations employed in the previous figure. A selection of
number of sensor scans $N$ is shown to illustrate the variation of performance
with this parameter.

##  IV Conclusions and Further Work

The purpose of this study was to introduce a modelling framework within which
one may assess the probability of detection of a threat using a sensor system
which scans at given times. The analysis provided a graphical means of
assessing performance as a function of number of scans and maximal detection
range. An approximate rule of thumb provided a conservative estimate of the
number of scans independent of the surveillance platform’s trajectory. Finally
a specific ASW model was used to illustrate how the techniques could be
combined with more detailed expressions for maximal detection ranges.

Further work could focus on producing a more accurate rule of thumb for
approximate performance, as well as the application of the results to more
comprehensive models in sonar system performance. In addition to this one
could explore the problem of UAV-based detection of land mines.

## References

  * [1] Danskin, J. M. A Helicopter Versus Submarine Search Game. Operations Research 1968, 16, 509–517.
  * [2] Yoash, R. B., Atkinson, M. P., Kress, M. Where to Dip? Search Pattern for an Antisubmarine Helicopter Using a Dipping Sensor. https://faculty.nps.edu/mpatkins/docs/2_ASW_Dipping_2018.pdf, 2018.
  * [3] Young, P. J. Intercept Considerations for Devising a Dipping Sonar Search Strategy to Locate an Approaching Submarine. Proceedings of the 2022 Winter Simulation Conference, 2022, 2034–2045.
  * [4] Urick, R. J. Principles of Underwater Sound. McGraw-Hill, 1975.
  * [5] Sabatier, J. M. Advances in Acoustic Landmine Detection. Battlefield Acoustic Sensing for ISR Applications, 2006. Meeting Proceedings RTO-MP-SET-107, available from: http://www.rto.nato.int/abstracts.asp.
  * [6] Kovacs, Z., Ember, I. Landmine Detection with Drones. Land Forces Academy Review, 2022, 105, 84–92.
  * [7] Craparo, E., Karatas, M. Optimal Source Placement for Point Coverage in Active Multistatic Sonar Networks. Naval Research Logistics, 2020, 67, 63–74.
  * [8] Stimpson, G. W. Introduction to Airborne Radar. Scitech, 1998.
  * [9] Tyler, G. D. The Emergence of Low-Frequency Active Acoustics as a Critical Antisubmarine Warfare Technology Johns Hopkins APL Technical Digest 1992, 1, 145–159.
  * [10] https://www.airforce-technology.com/projects/mh60r-seahawk/
  * [11] Ainslie, M. A. Principles of Sonar Performance Modelling. Springer-Verlag, Heidelberg, 2010.
  * [12] Donaldson, P. Sounding Out. Defence Helicopter 2017, 36, 24–27.