# Automatic Detection and Annotation of Sperm Whale Codas

Guy Gubnitsky  University of Haifa, Department of Marine Technology, Haifa,
3498838, Israel  Project CETI, New York, NY 10003, USA  Yaly Mevorach
University of Haifa, Morris Kahn Marine Research Station, Department of Marine
Biology, 3498838, Israel  Project CETI, New York, NY 10003, USA  Shane Gero
Carleton University, Department of Biology, Ottawa, Ontario, K1S 5B6, Canada
The Dominica Sperm Whale Project, Roseau, Dominica  Project CETI, New York, NY
10003, USA  David F. Gruber  City University of New York, Baruch College, New
York, NY, 10010, USA  Project CETI, New York, NY 10003, USA  Roee Diamant
University of Haifa, Department of Marine Technology, Haifa, 3498838, Israel
University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb,
Unska 3, Croatia<EMAIL_ADDRESS>Project CETI, New York, NY 10003,
USA

###### Abstract

A key technology in sperm whale (Physeter macrocephalus) monitoring is the
identification of sperm whale communication signals, known as codas. In this
paper we present the first automatic coda detector and annotator. The main
innovation in our detector is graph-based clustering, which utilizes the
expected similarity between the clicks that make up the coda. Results show
detection and accurate annotation at low signal-to-noise ratios, separation
between codas and echolocation clicks, and discrimination between codas from
simultaneously emitting whales. Using this automatic annotator, insights into
the characterization of sperm whale communication are presented. The results
include new types of coda signals, analyzes of the distribution of coda types
among different whales and for different years, and evidence for
synchronization between communicating whales in terms of coda type and coda
transmission time. These results indicate a high degree of complexity in the
communication system of this cetacean species. To ensure traceability, we
share the implementation code of our coda detector.

###### keywords:

Animal language, Sperm whale vocalization, Detection of Coda, Coda annotation,
Echolocation clicks, Graph based clustering, Generalized Gaussian Mixture
Model (GGMM)

## Introduction

In recent years, considerable efforts has been made to record and capture the
bioacoustics of sperm whale (Physeter macrocephalus). For example, Project
CETI (Cetacean Translation Initiative), founded in 2020, is an
interdisciplinary research initiative that uses advanced machine learning and
cutting-edge robotics to better understand sperm whale communication [1]. The
backbone of this effort is custom-built passive bioacoustic arrays covering a
20×20 kilometer area where these whale families reside (collecting over 30
TB/month), in conjunction with robotic acoustic and video tags on the whales,
underwater gliders and aerial drones to augment the rich contextual
communication data. This makes it possible to train a complete generative
language model for cetacean communication [2].

More broadly, in the face of biodiversity loss and the climate crisis, armed
with rapidly miniaturizing hardware to record wildlife and wild spaces,
scientists are challenged not by collecting increasing large, and continuous
amounts of data; but by devoting the time and development of the software
tools to curate and interpret it (see [3]). Automated species detection and
annotation of vocalizations is a critical step to characterizing and
monitoring our changing ecosystem, and this is even more true in the worlds
oceans. Further, it allows for rapid assessment of previously data depauperate
areas. In this case, given the importance of culture in conservation [4, 5],
and that the cultural population structure of sperm whale clans is defined by
acoustic repertoire [6, 7], an automatic detection and annotation of sperm
whales codas will allow for the definition of clans present in previously
unrecorded waters using passive acoustics, and thus provide not only a tool
for understanding this species’ communication system, but also for their
applied conservation and management.

###  0.1 Current Approaches

Sperm whales are among the most acoustically active toothed whales, making
them attractive species for passive acoustic monitoring (PAM). Many methods
have been proposed to detect and classify sperm whale clicks [8, 9, 10, 11,
12, 13]. However, existing techniques focus on the detection and
classification of echolocation clicks, which whales use to navigate the
darkness of the deep sea and hunt squid prey. However, when individuals
communicate with each other, sperm whales use an acoustically distinct type of
click [14, 15], which are produced in short sequences with stereotyped rhythm
and tempo called ’codas’ [16, 17, 18]. Codas last less than two seconds and
have been characterized by the varying number of constituent clicks and the
intervals between them. Codas are typically generated in multiparty
conversations that can last from 10 seconds to over half an hour.

Coda recognition is usually done by automatic click detection and subsequent
manual annotation to distinguish coda clicks from echolocation clicks, to
separate vocalizers and to ’link’ clicks to the same coda. Existing manual
annotation systems include custom Matlab scripts [19, 20], the Rainbow Click
software developed by the International Fund for Animal Welfare [17, 21] and
more recently CodaSorter [22, 7].However, while all of these programs provided
a user interface for manual annotation of codas by users based on feature
extractions from the clicks (e.g. interpulse interval, angle of arrival, and
spectra content of the clicks), none of these programs provided automatic
detection of codas, classification of echolocation and coda clicks, or the
ability to automatically separate sources and automatically annotate
individual clicks as belonging to the same coda. While Beslin et al. [13]
performed classification between echolocation and coda clicks, this solution
omits codas as "bad" echolocation. As a result, there are no automated
approaches for detecting coda clicks yet. This may be due in part to the
difficulty of separating sources between vocalizers, as codas from multiple
whales often overlap during exchanges [17, 23].

In this work an automated detector for codas is offered. The detector is
intended for both presence detection and characterization of codas, while
separating overlapping sources. The detector is designed for a wide dynamic
range to enable recordings from both an acoustic tag attached to a sperm whale
(near field) and an acoustic recorder deployed from a boat or mooring (far
field). The complexity of the detector is low, so that real-time operation is
possible. Offline analysis of data from acoustic tags that were not previously
processed revealed new insights into the synchronization of coda
characterization between pairs of communicating whales as well as new types of
coda signals.

###  0.2 Preliminaries: characteristics of the Coda signal

The codas of sperm whales consist of patterns of broadband, impulsive signals
called clicks, which are characterized by a multi-pulsed structure [15]. The
first impulse is generated by forcing air through the whale’s phonic lips. The
generated sound is reflected by the air sacs at the anterior and posterior
ends of the spermaceti organ. The result is a series of pulses of decreasing
amplitude that follow each other at equal time intervals; a product of the
transit time between the nasal air sacs in both directions. The two most
important features for distinguishing between coda and echolocation clicks are
the decay rate between pulses within the two click types, which is probably
the result of different signaling pathways in the whale’s nose [24], and their
rhythmic structure. The echolocation clicks are produced in long series with a
periodic pattern with characteristic inter-click intervals (ICIs) which are a
function of the whale’s search range [14]. In contrast, codas are sequences of
clicks with stereotyped rhythm and tempo [16], which are thought to serve
communication during exchanges between multiple individuals [17], and may
serve social recognition and identity [21, 25, 6].

One challenge in recognizing codas is managing distortions caused by multipath
interference. These lead to high variability in coda features, making it
difficult to distinguish between a coda, a sequence of echolocation clicks or
other interfering transients such as snapping shrimps or shipping cavitation
noises. These challenges are highlighted by the example in Fig. 1. The figure
shows recordings of two codas from a Dtag111A Dtag is an acoustic tag that is
attached to the skin of the whale with a suction cup to measure acoustic
recordings as well as depth and temperature information [26] attached to a
sperm whale. The bottom panel shows a coda originating from the tagged whale
(the focal whale) and the top panel shows a coda received from a distant
whale. The codas are represented by super-resolution spectrograms [27]. While
the coda of the focal whale shows a stable tempo-spectral structure, we
observe that the coda received from the distant whale is distorted.

Figure 1: Illustration of a scenario with a focal and a non-focal whale
emitting 5-click codas. An example of a super-resolution spectrogram of
individual clicks from a real tag recording of the codas of the focal and non-
focal whale are shown in the top right and bottom right panels, respectively.
A clear distortion of the structure of the clicks can be seen in the coda
received from a distance. However, the similarity between the clicks is
retained.

While the coda’s clicks may be distorted [28, 29], it is clear from Fig. 1
that this distortion is typical for all clicks. This is due to the fact that
the same acoustic channel is used for all coda clicks. Such similarity is also
evident in the clicks’ inter-pulse intervals (IPI), which are strongly
correlated with the size of the whale’s spermaceti organ [14, 30]. This
observation motivates us to build our coda detector on a similarity metric to
identify sequences of statistically related clicks. The result is a clustering
scheme that, given a set of detected clicks (e.g., the method in [31]),
separates groups of clicks with similar structure based on a likelihood
metric. To distinguish between echolocation clicks and codas, we constrain the
clustering solution to signals with a large multipulse structure and a
resonant frequency below a threshold.

##  1 Results

###  1.1 Description of the Testbed

Fewer than 600 individuals have been identified in the sperm whale community
in the Eastern Caribbean [32, 33]. These are divided into at least two, but
perhaps three cultural clans [22]. Matrilineal social units of sperm whales
associate with other units of whales which share a similar dialect of codas,
creating a culture-based population structure known as clans. In the Eastern
Caribbean, there are more members of the EC1 clan ($>$ 200 whales) than
members of the EC2 clan ($<$ 200 whales) [32]. Known social units of female
and immature sperm whales [34] were located and tracked in an area covering
approximately 2000 $\mathrm{km}^{2}$ along the entire west coast of the island
of Dominica (N15.30 W61.40).

Our dataset includes both near-field data from 42 tags deployed on 25
different individuals in 11 different social units, with photo-identification
to match the tagged whale, and far-field data by hydrophone deployed from a
boat. Preliminary manual annotations for near-far data between 2014-2016
yielded a set of 3948 codas, while a set of 4930 codas was obtained for far-
field data between 2005-2012. Additional annotations were supported by our
automatic detector to analyze 14.9 GB of near-field data collected in 2018,
yielding a further set of 843 codas. Detector-assisted annotations also
enabled the processing of 25 GB of far-field data, yielding 727 codas. For the
assessment of false alarms, noise segments were used, which were divided into
two categories: 1) four hours of raw acoustic signals that were manually
verified to include ambient noise, ship noise, whistles and clicks from other
marine mammals, and 2) three hours of echolocation clicks from several sperm
whales, which were used to investigate the detector’s ability to distinguish
between codas and echolocation clicks.

###  1.2 Detection and Annotation Results

(a) Noise including echolocation clicks.

(b) Noise do not include echolocation clicks.

Figure 2: ROC curve, evaluated for near- and far-field data. The solid lines
refer to the unconstrained detection mode, while the lines for the constrained
mode are dashed. False alarm rate evaluated for noise data including a) sperm
whale echolocation clicks, b) ambient noise without echolocation clicks (only
far-field data).

Figure 3: Cumulative distribution function (CDF) of the ratio of the number of
clicks detected in the coda in the near- and far-field datasets using
constrained detection mode.

The detection performance is analyzed in Fig. 2 in terms of the Receiver
Operating Characteristic (ROC) to investigate the trade-off between the
detection rate (Pd) and the false alarm rate (FAR). The former is defined by
the rate of true-positive cases verified by the manual annotations, and the
latter by the rate of false-positive cases per minute. Pd and FAR were
calculated based on manual annotations. As explained in Section 3, the
detector applies a maximum likelihood approach, which is based on a
distribution analysis of legacy database. To avoid overfitting, these
distributions were evaluated from our near-field database from 2014-2016 and
our far-field database from 2005-2012, while the performance is shown for the
most recent data collection, i.e. for the near-field from 2018 and the far-
field from 2022-2023.

The performance is compared for noise with echolocation clicks (Fig. 2a) and
for noise without echolocation clicks (Fig. 2b). The detection results are
shown for both far-field and near-field data222The results for noise without
echolocation clicks are for far-field only, as the data from tags almost
always contained echolocation clicks., and for constrained and unconstrained
detection (see details in Sections 3.3.2 and 3.3.5). The results show that the
FAR reduces by an order of magnitude when using constraints, at the cost of a
slight decrease in the detection rate. For the echolocation noise (Fig. 2a),
no significant difference in performance is observed between the near-field
and far-field datasets. From this, we conclude that our detector can handle a
low signal-to-noise ratio (SNR) well. Comparing the results for the two noise
categories, we find that the FAR is twice as high for detection without
constraints when the noise contains echolocation clicks, which is the harder
case. Since the multipulse and resonant frequency constraints also allow the
separation of noise transients, this difference is magnified when considering
the performance differences between the constraint detectors.

In Table. 1, we examine the effects of each constraint for both cases of noise
categories for the far-field dataset. Two detection thresholds are selected by
a trade-off of 1) Pd=0.18 and FAR=0.12 per min and 2) Pd=0.4 and FAR=0.33 per
min. To investigate the effects of the clustering constraints, we show the
results without constraints (None), with the multipulse constraint only
($\mathcal{P}$), with the resonant frequency constraint only ($f_{r}$) and
with both constraints ($\mathcal{P}$ & $f_{r}$). In both cases of noise
categories, we find that the multipulse constraint is most dominant in
distinguishing coda clicks from echolocation clicks and noise transients. The
resonant frequency constraint also contributes, but less. We explain this
result by the effect of the channel impulse response, which distorts the
spectrum of the far-field signal and allows less separation by spectra
analysis. For the more common case of ”general noise” in far-field data, we
observe a very low FAR for constraint detection.

Table 1: False alarm rate (FAR) per minute. Results are given for far-field data with detection thresholds set according to detection rate of {0.18,0.4} and a false alarm rate of {0.12,0.33}. The results are analyzed for two noise categories: without echolocation clicks (general noise) and with echolocation clicks (echolocation clicks). | General Noise | Echolocation Clicks
---|---|---
Constraint applied | $\mathcal{P}$ | $f_{r}$ |  $\mathcal{P}$ & $f_{r}$ | None | $\mathcal{P}$ | $f_{r}$ |  $\mathcal{P}$ & $f_{r}$ | None
Target: Pd=0.18 & FAR=0.12 | 0.0097 | 0.024 |  0 | 0.034 | 0.035 | 0.082 |  0.023 | 0.12
Target: Pd=0.4 & FAR=0.33 | 0.0097 | 0.275 |  0 | 0.33 | 0.199 | 0.987 |  0.152 | 1.5

Next, we examine the accuracy in the automatic annotation of all clicks within
the coda. This attribute is important to define the type of coda, which in
turn is determined by the number of clicks and the rhythm and tempo of the ICI
pattern [23, 18]. Especially in the presence of echolocation clicks or in the
frequent case of overlapping codas, a true coda click can easily be
disqualified as an echolocation. Fig. 3 shows the cumulative distribution
function (CDF) of the ratio between the number of identified clicks and the
actual clicks in the coda. We compare the performance for the far-field and
near-field datasets. Identification of all clicks in the coda is achieved in
about 70% of cases, while in another 10-20% of cases 80% of clicks are
identified. The results show no significant difference between the near-field
and far-field datasets. Although the accuracy level in detecting the coda
clicks is promising, we note that misidentification of a single click may lead
to misclassification of the coda type. We leave the improvement of this result
to future work.

###  1.3 Communication Characterization: findings from the coda annotator

The ability of the developed automatic detector and annotator to provide large
datasets of coda was used to investigate the characteristics of coda signals
towards the understanding of sperm whale communication. To eliminate bias in
our analysis, we filter our coda database to use only codas from the near
field, which allows determination of the identity of the focal whale. Further,
since no classifier is currently proposed to distinguish between codas of
individual whales, the following analysis considers dyadic social interactions
by codas involving only two whales. To that end, we only consider coda pairs
(see definition bellow) between a focal and a non-focal whale. These cases are
identified by considering pairs of codas, detected within a buffer of 7 sec
and where there is a significant gap in both codas’ amplitude. Other codas
that contained a single coda with no detected response or codas of more than
two whales were discarded. An example of a dyadic interaction of 3 coda pairs
is shown in Fig. 4 by the red arrow marks. This process resulted in 635 coda
pairs used for the below analysis.

Our main findings from the operation of the automatic coda annotator are
evidence for synchronization between the focal and non-focal whales. This was
established by examining the time delay between successive codas and between
the clicks of the two codas. This type of correspondence arguably corresponds
to the synchronization required for a stable telecommunication session. In
particular, we note that the structure of the coda can be considered as a
communication modulation signal. The time interval between codas can then be
referred to as the communication baud rate, while variations between the click
delays (ICI) of the codas can reflect the information encoding to obtain the
channel’s capacity outage. Another finding is the discovery of two new coda
structures. This was revealed by comparing the extracted coda features with
those of codas available in the literature. To describe these two findings,
the following definitions are used, and examples of each of these structures
are given in Fig. 4.

  * •

Coda Type: Categorical coda representation obtained by clustering codas based
on their ICI vectors [21].

  * •

Dyadic coda exchange: Period in which codas are made by a pair of whales.

  * •

Inter-coda Interval: A time interval, $\Delta_{CI}$, between a consecutive
pair of calls of a particular whale. As marked in Fig. 4, the inter-coda
interval for a given whale is measured by the time delay between the last
click of a coda and the first click of a subsequent coda of the same whale.

  * •

Coda pair: Two consecutive codas originating from different whales in a dyadic
exchange within a buffer of 7 sec, such that the first click of one coda is
received before the last click of the second coda.

  * •

Inter-coda Break: The time interval, $\Delta_{CB}$, between a coda pair. As
marked in Fig. 4, the Inter-coda Break is defined by the time delay between
the last click of one coda and the start of the first click of the
interlocutor’s next coda.

Figure 4: An example of 13 seconds from a near-field recording showing a
dyadic coda exchange between a focal and a non-focal whale. The Inter Coda
Interval of the focal and non-focal whale are indicated by black solid and
dashed arrows, respectively. The Inter Coda Break is indicated by red solid
arrows.

####  1.3.1 Modulation Signal: Analysis of Coda Type Distribution

A coda type is characterized by the number of clicks it consists of (between 3
and 10) and by the sequence of time delays between the individual clicks.
Examples of this are a sequence of 5 clicks in which the first three clicks
are evenly spaced and at a different distance from each other than the last
two clicks (type 1+1+3), or a sequence of 8 clicks with a gradually increasing
or decreasing distance (type 8i or 8D, respectively). To date, 25 coda types
have been identified [21]. These were determined by a clustering result
applied to annotated ICI vectors of codas [21, 35].

Figure 5: Analysis of the distribution of coda types for all dyadic coda
exchanges found. The left panel shows the distribution matrix of all pairs of
coda types, with each bin representing a particular coda type from the focal
whale (signal) and the subsequent coda type of the non-focal whale (response).
The right panel is zoomed in on the most active coda types. The bins’ color
represents the probability of occurrence on a logarithmic scale.

The analysis in Fig. 5 shows the distribution of codas for all considered coda
pairs found during the periods of dyadic coda exchange. The results are
presented as a matrix for the distribution of a ’signal and response’ (S&R)
session between the focal and non-focal whale. Without loss of generality, the
coda type of the focal whale in the matrix’s columns is considered here as
’signal’ (S) and the coda type of the non-focal whale (hereafter referred to
as the Interlocutor) in the matrix’s columns is considered the ’response’ (R).
The bin’s color corresponds to the probability (on a logarithmic scale) that a
particular focal’s coda type is observed before a particular interlocutor’s
coda type. The results show that the ’1+1+3’ type is the most typical coda
type for both the signaler and the responder. Furthermore, the variation in
the coda types is not large and the highest concentration is between types
’1+1+3’ and itself, between types ’1+1+3’ and ’4R2’ and between type ’5R1’ and
itself.

####  1.3.2 The Baud-rate: Time Delay Dependencies in Whale Codas

(a) Probability density of $\Delta_{\mathrm{CB}}-\Delta_{\mathrm{CI}}$ for
three focal whales, ’Atwood’,’Fork’ and ’Pinchy’.

(b) Probability density of $\Delta_{\mathrm{CB}}$ as a function of coda type.

Figure 6: Distribution functions for the delay between the end of the ’S’-coda
and the beginning of the ’R’-coda, $\Delta_{\mathrm{CB}}$.

The changes in the structure of the identified codas arguably resembles that
of encoding of information via a modulation signal. Here we use the analogy of
a baud rate to correspond to the inter-coda Break, $\Delta_{\mathrm{CB}}$, as
defined in 1.3.The probability density function (PDF) of
$\Delta_{\mathrm{CB}}$ is shown in Fig. 6 for three focal whales named
’Atwood’ (whale #5586), ’Fork’ (whale #5151) and ’Pinchy’ (whale #5560). Fig.
6a shows the PDF of $\Delta_{\mathrm{CB}}-\Delta_{\mathrm{CI}}$ separately for
each whale. An observable shift in the expectation value of the three PDFs
reflects differences in the duration of S&R sessions between whales. The
longer tail towards the negative side of the distribution shows that sometimes
whales skip a coda overlap and leave a gap of approximately similar to the
exchanges Inter-coda Interval. This is further explored in Fig. 6b, where we
show the PDF of $\Delta_{\mathrm{CB}}$ for the three most frequent S&R coda
types, combined for the three focal whales. The seemingly similar PDFs in the
latter figure show that the observed bias in Fig. 6a is not affected by the
coda type.

Taken together, these results suggest that the structure of the exchange in
terms of the Inter-coda Interval can be maintained despite the variation in
ICI within codas, or coda types being exchanged.

####  1.3.3 Capacity Outage: Variation in Information Gain Encoded in Coda
Types

The analogy with the characteristics of telecommunication performance is
further explored by the concept of capacity outage, which is represented here
by the change in the structure of the codas, which arguably corresponds to
symbol modulation and thus to information gain. The change in the coda’s
structure is analyzed by variations in the coda ICI relative to the typical
ICI pattern for the specific coda type. This variation is measured by the
metric

$\displaystyle\Delta_{\mathrm{ICI}}=||\frac{1}{L-1}\left(\sum\limits_{l=2}^{L}\tau_{l}(t)-\bar{\tau}_{l}(t)\right)||_{2}\;,$
(1)

where $\tau_{l}(t)$ and $\bar{\tau}_{l}(t)$ are the ICI delays between the
$l$th and $l-1$th coda’s click of the measured and typical coda type $t$, $L$
is the number of clicks comprising the coda, $T$ is the number of coda types,
and $||\cdot||_{2}$ denotes the $l2$-norm operator.

Figure 7: PDF of $\Delta_{\mathrm{ICI}}$ as a function of coda type. The
number of detected Codas per type is given in the legend in brackets.

The PDF of $\Delta_{\mathrm{ICI}}$ for four common S&R coda types is examined
in Fig. 7. The values of $\Delta_{\mathrm{ICI}}$ for the coda types 5R1 and
5R2 show small fluctuations which, according to our communication analogy,
reflect a low degree of modulation leading to a low level of information
transfer. In contrast, the PDF of $\Delta_{\mathrm{ICI}}$ for coda type 1+1+3
shows higher ICI variation with greater diversity, while the highest diversity
is shown in $\Delta_{\mathrm{ICI}}$ for coda type 6i. This can be explained by
the clicks structure of these coda types, where a greater ICI variation is
observed the more complex the coda type is. That is, assuming that some
information is encoded in the ICI variation (e.g. social identity information,
[25]), the information capacity depends on the coda type. In other words, the
more complex the modulation symbol (coda type) is, the more information (ICI
variation) it can transport, to use the analogy with telecommunication once
again.

(a) PDF of $\Delta_{\mathrm{ICI}}$ for the focal and non focal whales.

(b) PDF of $\Delta_{\mathrm{CI}}$ for the focal and non focal whales.

Figure 8: Differences between variations of Codas generated by the focal and
non focal whales

The above observation is next investigated for various focal whales. The PDFs
of $\Delta_{\mathrm{ICI}}$ for the focal whales ’Atwood’, ’Fork’ and ’Pinchy’
and their corresponding interlocutor are shown in Fig. 8a. The similar PDFs
for the ICI of the focal and interlocutor whales reflect synchronization
between the ’S’-coda and the ’R’-coda. Our results here, and in Fig. 5,
suggest that whales often match coda type (and therefore
$\Delta_{\mathrm{ICI}}$) during dyadic coda exchanges (as suggested by [17,
23]. In fact, Sharma et al. [23], suggests that whales in dyadic coda
exchanges, like those analysed here, are able to precisely match variation in
both rhythm (relative timing of clicks in a coda) and tempo (overall
duration), both which drive variation in ICI, and further can do so by
matching tempo (total duration of a coda) even when not matching the same coda
type (rhythm). Here, we suggest that this synchronization of ICI variation may
also reflect the dependence on the identity of the emitting whale. As is
supported by the differences between the PDFs for ’Atwood’ (upper panel),
’Fork’ (middle panel) and ’Pinchy’ (lower panel) and their interlocutor are
matched to the identity of the whale.

The above interpretation is supported by the results in Fig. 8b, where the PDF
of the Inter-coda Interval, $\Delta_{\mathrm{CI}}$, is compared for the focal
and the interlocutor, again for the whales ’Atwood’, ’Fork’ and ’Pinchy’.
Again, a synchronization of $\Delta_{\mathrm{CI}}$ is observed between the
signaller and the responder, a synchronization that changes depending on the
identity of the different focal whales. This suggests that the whales are able
to vary their Inter-coda Interval depending on who they are interacting with.

###  1.4 Identification of new coda types

Coda types are a categorical representation of continuous ICI multivariate
space. Previously, discrete coda types have been defined based on various
clustering methods (see [20, 6, 36, 21, 7]). During these processes, some of
the codas, whose ICI series are outliers or located in sparse areas between
dense clusters, to be labelled as ’contamination’ or ’noise’ (rather than
being ’forced’ into categories) and thus only define more conservative, dense
clusters in which all codas in a discrete ’type’ are highly similar to each
other. An example of this is shown in Fig. 9, where we visualize the three
most dominant principal components, PC1, PC2 and PC3, for 427 codas with 6
clicks obtained by performing a principal component analysis (PCA) on our
data. This PCA analysis allows the identification of clusters in the dataset;
corresponding to different coda types. In the top right panel of Fig. 9 we
show the analysis for the preexisting annotated dataset. We observe two
prominent clusters labeled ’6i’ and ’6R’, while other scattered points are
labeled as ’noise’ simply because they did not meet thresholds of density or
cluster shape based on the quantitative methods applied to define categories.
By using our automatic detector for the near-field recordings from 2018, which
were not previously annotated due to the large amount of data and the time
required by manual methods, 843 codas were rapidly annotated. This then allows
us to observe two more distinct clusters in the feature space, apart from the
previous 6R and 6i coda types. This can be seen in the bottom right panel of
Fig. 9. We define two more descending class, coda types as the new types ’1+5’
and ’1+1+4’ (rhythm plots are shown in the upper left and lower left panels of
Fig. 9). We note that the codas within each of the new clusters have highly
similar rhythm structure with some within type tempo variation; but that the
two types have distinct rhythms and tempos.

Figure 9: Right panels: Distribution of the three dominant PCA components
(PC1, PC2 and PC3) of 6 click codas out of 427 from our legacy database. The
upper right panel shows the distribution of the annotated database. The bottom
right panel shows the distribution of the unannotated dataset obtained with
our automatic annotator and verified manually. Left panels: rhythm plots of
ICIs from a few examples of the two new coda types.

##  2 Discussion

Our approach, arguably the first automatic detector and annotator for sperm
whale codas, offers robust operation with few parameters to set, allowing
analysis of data obtained by both acoustic tags and boat-mounted hydrophones
located far from the whale. The ROC performance of the detector shows that it
is robust even at low SNR values and in high dynamic range, that it can handle
the noise transients commonly found in the Dominica Island, and that it is
able to distinguish between echolocation clicks and coda. As shown in Section
3 below, our annotator can also separate between overlapping codas. When the
analyzed buffer is small, there are limitations when echolocation clicks from
different whales overlap, as they may resemble a coda structure. Future
developments will therefore focus on the inclusion of blind source separation
to mitigate cases of interfering clicks, as well as a temporal likelihood
prediction model to allow generalization to a larger number of coda types.

The detector provides automatic analysis of large datasets of raw acoustic
data, thereby significantly reducing the processing time. The results of the
automatic annotation were demonstrated to explore the characteristics of the
codas. In particular, the distribution of coda types, the delays between
successive codas and the ICI variation for different coda types and between
different focal and non-focal whale pairs. An analogy was made for the
resemblance of coda exchange to a telecommunications session, where the
synchronization between the focal whale and its interlocutor in terms of
delays between successive codas was compared with the baud rate of the
communication, and the synchronization in terms of ICI variation was
attributed to capacity outage. The large amount of codas collected with the
new detector allowed a statistical analysis of coda features to comment on the
diversity of codas. This led to the discovery of two new coda types.

Significantly, our results suggest that some of the within coda type
variation, the structure of coda exchanges in terms of the inter-coda-
intervals and inter-coda-breaks, vary based on the identity of interlocutors.
While [21] suggested that within coda type ICI variation of the 5R1 coda could
be used to potentially recognize individuals within a small number of social
units, these new findings suggest identity cues at the level of the structure
of the coda exchange. This will require deeper investigation in relation to if
these patterns are upheld depending on which whale vocalizes first (vocal
asymmetries), such that whale B may match whale A’s inter-coda-interval or
inter-coda breaks when A vocalizes first, but A will match B if B vocalizes
first. But also, how these patterns may perhaps reflect vocal cues of social
dominance (see [37, 38] or correlate with social or kin based relationships.
Interestingly, Fork and Pinchy belong to the same social unit, while Atwood
lives in a different one; and their distributions of $\Delta_{\mathrm{CI}}$
are more similar to one another than to Atwood’s distribution; so these
patterns may reflect unit-level, rather than individual level patterns.

Looking towards the future plans for the detector, we intend a real-time
implementation within the framework of Project CETI. This will include online
detection from an offshore moorings to support the tagging work and create a
live map of the whales’ positions. The detector will also be implemented in
real-time on board a sea glider and in surface vessels searching the area for
sperm whales.

##  3 Methods

###  3.1 Method of Data Collection

Data collection was conducted in two ways: 1) acoustic tags attached to sperm
whales (near-field), and 2) acoustic recorders deployed from a boat (far-
field). In the first method, sound and movement tags (Dtag generation 3 [26])
worn by the animals were deployed between 2014 and 2018 as a part of The
Dominica Sperm Whale Project. These tags have a two-channel audio sampling at
120 kHz with a resolution of 16 bit, providing a flat ($\pm$2 dB) frequency
response between 0.4 kHz and 45 kHz. Pressure and acceleration were also
sampled at a rate of 500 Hz and a resolution of 16 bit, decimated to 25 Hz for
offline analysis. Tagging was carried out from an 11 m rigid-hulled inflatable
boat (RHIB). A 9 m hand-held carbon fiber pole was used to attach individual
tags to a whale with four suction cups.

Data collection in the far field was carried out with a C75 Cetacean Research
hydrophone deployed from the RIB at a depth of 15 m. The hydrophone has a flat
($\pm$3 dB) frequency response between 10 Hz and 170 kHz. A MicPre-6 || data
acquisition device from ’Sound Devices’ was used to sample the signal at 96
kHz and 16-bit resolution.

All whales were identified from photographs of the trailing edge of their
fluke ( [39]). The identifications were used to ensure that only recordings
from one of the two sympatric clans (EC1 – the Eastern Caribbean Clan) were
included in the analysis to control for any differences in repertoire between
vocal clans [35].

###  3.2 Permitting and Animal Care Approval

Data from The Dominica Sperm Whale Project were collected under scientific
research permits from the Fisheries Division of the Government of Dominica.
The field protocols for approaching, photographing, non-invasively tagging,
and recording sperm whales were approved by either the University Committee on
Laboratory Animals of Dalhousie University, Canada; the Animal Welfare and
Ethics Committee of the University of St Andrews, Scotland; or Aarhus
University, Denmark; and sometimes several or all of these across years. Data
collected as a part of Project CETI were also collected under a scientific
research permit from the Fisheries Division of the Government of Dominica. The
field protocols for recording sperm whales for Project CETI were approved by
The Institutional Animal Care & Use Committee at Harvard University. All
methods are reported in accordance with ARRIVE guidelines relevant to wild
whales and our analyses.

###  3.3 Coda Detection

Figure 10: A block diagram for the Coda detector.

A block diagram of the coda detector and annotator is shown in Fig. 10. The
input to the detector is a buffer of $7\leavevmode\nobreak\ \mathrm{sec}$
(configurable). First, a transient detector identifies a few regions- of
interest (ROIs). For each ROI, three features are extracted: 1) the shape of
the transient, 2) the intensity of the transient, and 3) a measure of the
multipulse structure, if it exists. Clustering is then performed based on
these features to group the ROIs based on their likelihood similarity. The
clustering is performed iteratively until a stopping criteria is met. The
details of the individual components are described below. A MATLAB
implementation code for our scheme is available in [40].

####  3.3.1 Click Identification

Transient ROIs are determined by selecting identified peaks based on the
expected attributes of valid coda clicks. The process starts with a bandpass
filter for the desired frequency range of 2 kHz-24 kHz [10]. To enhance
transients, following prior works for sperm whale click detection [41, 10],
the Teager-Kaiser energy operator (TKEO) [42],

$\displaystyle z_{n}=x_{n}^{2}-x_{n-1}\cdot x_{n+1}\quad,n=1,\ldots,N\;.$ (2)

is executed over the sampled signal $x_{n},/n=0,\ldots,N-1$. Peaks in $z_{n}$
from (2) are identified as local energy maxima that fulfill a minimum distance
between the peaks and a minimum threshold for the SNR. The former is set
according to the maximum IPI of known codas (a value of 8 ms is used in the
results). We limit the number of identified transients to an expected maximum
of 20 peaks. An example of the transient identification process is shown in
Fig. 10.

####  3.3.2 Clustering transients into candidate codas

The clustering of transient ROIs into groups of possible codas is based on the
observation that codas follow known ICI patterns [21] and on the assumption
that the clicks forming a coda share statistical similarities. There are two
explanations for the latter assumption: 1) each of the coda clicks is
generated by the emitting whale in the same way, and 2) due to the short
duration of the coda ($0.1-2sec$) [43, 44, 21]), the channel’s impulse
response from the whale to the receiver is expected to be roughly time
invariant, and thus the clicks of the same coda share the same channel
distortion. An example of this is shown in Fig. 1, where different spectral
shapes are observed for the far-field and near-field codas.

The clustering is formalized as the following optimization problem

$\displaystyle\hat{K},\hat{\bf{C}}=$
$\displaystyle\mathrm{argmax}_{K,\bf{c}_{k},\
k=1,\ldots,K}\frac{1}{K}+\sum_{k=1}^{K}{\cal J}(K,\bf{c}_{k})_{k=1,\ldots,K}$
(3a) $\displaystyle\;\mathrm{s.t.}\;\;\mathcal{J}_{(\mathbf{c}_{k})}>\rho_{d}$
(3b) $\displaystyle\;\qquad c_{l}\perp c_{k},\leavevmode\nobreak\ \forall
l\neq k\;,$ (3c)
$\displaystyle\;\qquad\mathcal{P}(\mathbf{c}_{k})>\mathcal{P}_{\mathrm{min}}\;,$
(3d) $\displaystyle\;\qquad f_{r}(\mathbf{c}_{k})<f_{r}^{\mathrm{max}}\;.$
(3e)

where $\mathbf{c}_{k},\ k=1\ldots_{K}$ are M-dimensional binary assignment
vectors such that

$\mathbf{c}_{k}=[\omega_{k,1},\ldots,\omega_{k,M}]\;,\quad\omega_{k,l}\in\\{0,1\\},\
l=1,\ldots,M\;.$ (4)

The utility function $\mathcal{J}_{(\mathbf{c}_{k})}$ is defined by

$\mathcal{J}_{(\mathbf{c}_{k})}=\mathcal{L}^{\mathrm{s}}(\mathbf{c}_{k})+\alpha_{1}\mathcal{L}^{\mathrm{t}}(\mathbf{c}_{k})-\alpha_{2}F(\mathbf{c}_{k})\;,$
(5)

where $\mathcal{L}^{\mathrm{s}}(\mathbf{c}_{k})$ and
$\mathcal{L}^{\mathrm{t}}(\mathbf{c}_{k})$ are the structural and temporal
similarity likelihoods for a clustering vector $\mathbf{c}_{k}$, respectively,
$F(\mathbf{c}_{k})$ is a penalty factor for low rank clusters such that

$F(\mathbf{c}_{k})=\exp{({\mathbf{c}_{k}\mathbf{c}_{k}^{\mathrm{T}}})^{-1}}\;,$
(6)

and $\alpha_{1}$ and $\alpha_{2}$ are normalized weighting factors. Note that
$\alpha_{1}$ is a limiting factor intended to restrict the selection of
clusters that define ’sub-codas’. For the case shown in Fig. 10 for example, a
large $\alpha_{1}$ will favor the choice of the entire structure 4+1 over the
sub-structure of 4 clicks with identical ICI, although both represent valid
coda structures. On the other hand, a small $\alpha_{1}$ allows the
identification of new, unknown or rare coda types.

Four constraints are used. In (3b) a utility function above a detection
threshold $\rho_{d}$ is searched for. In (3c) we restrict the solution to a
unique clustering. In (3d), we expect a valid coda click to contain at least
$\mathcal{P}_{\mathrm{min}}$ multipulses and limit the number of multipulses
in a click, averaged over $\mathbf{c}_{k}$, to $\mathcal{P}(\mathbf{c}_{k})$.
While both coda and echolocation clicks consist of repetitive pulses of
decreasing intensity, codas are characterized by a much lower decay rate [14].
This is possible thanks to the whale’s ability to use the right nasal passage
as an adaptive acoustic valve to direct the signal towards the junk in the
case of echolocation clicks, or back in the spermaceti in the case of coda
signals [45]. The low decay rate of the intermediate pulses is reflected in a
larger number of pulse repetitions, resulting in a more pronounced periodic
structure of the overall signal. A method for estimating the number of
multipulses is presented in Section 3.3.5. Our fourth condition in (3e)
considers that the resonant frequency of the coda click, averaged over
$\mathbf{c}_{k}$, $f_{r}(\mathbf{c}_{k})$, should be below a threshold
$f_{r}^{\mathrm{max}}$, and following [14], we use $f_{r}^{\mathrm{max}}=12$
kHz. The resonant frequency is calculated using a super-resolution spectrogram
in [27].

Due to the requirement for unique clustering, the optimal solution of (3) is
an NP-hard problem with similarities to the graph coloring problem [46]. In
the case of many ROI candidates, a suboptimal solution is offered based on the
following iterative procedure:

$\displaystyle\hat{c}_{k}=$
$\displaystyle\mathrm{argmax}_{c_{k}}\mathcal{J}_{(\mathbf{c}_{k})}$ (7a)
$\displaystyle\;\mathrm{s.t.}\;\;\mathcal{J}_{(\mathbf{c}_{k})}>\rho_{d}$ (7b)
$\displaystyle\;\qquad\mathcal{P}(\mathbf{c}_{k})>\mathcal{P}_{\mathrm{min}}\;,$
(7c) $\displaystyle\;\qquad f_{r}(\mathbf{c}_{k})<f_{r}^{\mathrm{max}}\;.$
(7d)

In each iteration, cluster candidates $\mathbf{c}_{k}$ that do not meet the
orthogonality condition,

$\begin{array}[]{rrclcl}c_{l}\cdot\hat{c}_{k}^{T}=0,\leavevmode\nobreak\
\forall l\neq k.\end{array}$ (8)

are omitted. The iterations stop when no more valid clusters are found.

####  3.3.3 Structural likelihood analysis

To represent the similarity between a transient pair, each detected transient
is treated as a node in an undirected graph. Let $s_{i,j}$ denote the
similarity between a node pair $i$ and $j$, where $s_{i,i}=0$. These
similarities combine an affinity matrix

$S_{M\times M}=\left[{\begin{array}[]{cccc}s_{11}&s_{12}&\cdots&s_{1M}\\\
s_{21}&s_{22}&\cdots&s_{2M}\\\ \vdots&\vdots&\ddots&\vdots\\\
s_{M1}&s_{M2}&\cdots&s_{MM}\\\ \end{array}}\right]\;.$

For a candidate cluster, $\mathbf{c}_{k}$, the structural likelihood is
defined as the average similarity between the cluster’s nodes

$\mathcal{L}^{\mathrm{s}}(\mathbf{c}_{k})=\frac{\mathbf{c}_{k}S\mathbf{c}_{k}^{\mathrm{T}}}{0.5(\mathbf{c}_{k}\mathbf{c}_{k}^{\mathrm{T}}-1)\mathbf{c}_{k}\mathbf{c}_{k}^{\mathrm{T}}}\;.$
(9)

Three similarity measures are used: 1) waveform-based similarity,
$s_{i,j}^{\mathrm{shape}}$, 2) IPI-based similarity, $s_{i,j}^{\mathrm{IPI}}$
and 3) intensity-based similarity, $s_{i,j}^{\mathrm{I}}$. The first one is
calculated by the cross-correlation between the pair of clicks $y_{i}(t)$ and
$y_{j}(t)$. Formally,

$s_{i,j}^{\mathrm{shape}}=\frac{\int y_{i}(t)\cdot y_{j}(t)dt}{\sqrt{\int
y_{i}(t).^{2}dt}\cdot\sqrt{\int y_{j}(t).^{2}dt}}\;.$ (10)

The second similarity metric requires the evaluation of the click’s IPI e.g.,
using the method in [41], and is defined by the normalized difference

$s_{i,j}^{\mathrm{IPI}}=1-\frac{|\mathrm{IPI}_{i}-\mathrm{IPI}_{j}|}{\mathrm{max}(\mathrm{IPI}_{i},\mathrm{IPI}_{j})}\;.$
(11)

Since the IPI is a function of the spermaceti size [47], the measure
$s_{i,j}^{\mathrm{IPI}}$ allows the separation between overlapping clicks of
whales of different sizes. Similarly, the intensity-based similarity is
defined by the difference

$s_{i,j}^{\mathrm{I}}=1-\frac{|\mathrm{I}_{i}-\mathrm{I}_{j}|}{\mathrm{max}(\mathrm{I}_{i},\mathrm{I}_{j})}\;,$
(12)

where $\mathrm{I}_{i}$ is the RMS amplitude of the $i$th click in the cluster.
Since underwater acoustic propagation is strongly dependent on distance,
$s_{i,j}^{\mathrm{I}}$ allows the discrimination of clicks emitted by whales
at different distances from the receiver. The combined similarity measure
$s_{i,j}$ is defined by the normalized weighted sum

$s_{i,j}=\rho^{\mathrm{corr}}s_{i,j}^{\mathrm{corr}}+\rho^{\mathrm{IPI}}s_{i,j}^{\mathrm{IPI}}+\rho^{\mathrm{I}}s_{i,j}^{\mathrm{I}}\;,$
(13)

where $\rho^{\mathrm{corr}}+\rho^{\mathrm{IPI}}+\rho^{\mathrm{I}}=1$.

####  3.3.4 Temporal likelihood analysis

Let $\mathbf{H}$ be a database of ICI patterns of $L_{W}$ previously collected
codas, each including exactly $W+1$ clicks. We use $\mathbf{H}$ to evaluate
the likelihood of a possible cluster $c_{k}$ to represent a coda signal.
Features of $\mathbf{H}$ are extracted by principle component analysis (PCA)
to form a $Q\times W\leavevmode\nobreak\ (Q<L_{W})$ representative matrix

$\displaystyle\mathbf{H}_{\mathrm{R}}=\mathbf{\overline{\mathbf{H}}}\cdot{\overline{\mathbf{U}}}\;,$
(14)

where $\overline{\mathbf{H}}$ is an unbiased version of ${\mathbf{H}}$ in
which the sampled mean of each column of ${\mathbf{H}}$ is shifted to zero,
and $\overline{\mathbf{U}}\in\mathrm{I\\!R}^{W\times{Q}}$ is a matrix
representation of the $Q$ eigenvectors corresponding to the largest $Q$
eigenvalues of ${\mathbf{H}}$. Matrix ${\overline{\mathbf{U}}}$ is used also
to extract the same $Q$ features from the ICI pattern,
$\mathbf{ICI}_{(\mathbf{c}_{k})}$, of the examined potential $c_{k}$ cluster,
denoted by

$\displaystyle\mathbf{O}_{(\mathbf{c}_{k})}=\mathbf{ICI}_{(\mathbf{c}_{k})}\cdot{\overline{\mathbf{U}}}\;.$
(15)

To evaluate the probability of $\mathbf{O}_{(\mathbf{c}_{k})}$ to be a valid
coda representation, we calculate the temporal likelihood

$\mathcal{L}^{\mathrm{t}}_{(\mathbf{c}_{k})}=\sum_{g=1}^{G_{W}}P_{r}(\mathbf{O}_{(\mathbf{c}_{k})}\implies
g|\Theta_{W})\;,$ (16)

where the $\mathbf{O}_{(\mathbf{c}_{k})}\implies g$ term implies that
$\mathbf{O}_{(\mathbf{c}_{k})}$ is a feature representation of coda type $g$,
$G_{W}$ is the number of coda types available in $\mathbf{H}$, and
$\Theta_{W}$ is the set of parameters for the distribution of the coda
features in $\mathbf{H}_{\mathrm{R}}$.

To calculate (16), we evaluate $\Theta_{W}$ from ${\mathbf{H}}_{R}$. We
estimate the distribution of each coda type $g$ separately. Since the coda
type is associated with the number of clicks, in the following we drop the
sub-index $W$. Denote $h_{g}\in\mathbb{R}^{1\times Q}$ as a subset of
$\mathbf{H}_{\mathrm{R}}$ corresponding to coda type $g$, and let $\Theta_{g}$
be its distribution parameters. We model $h_{g}$ as the mixture model

$p(h_{g}|\Theta_{g})=\sum_{k=1}^{K_{g}}\phi_{k}\cdot
f_{k}(h_{g}|\theta_{g,k})$ (17)

of $K_{g}$ clusters with prior

$\sum_{k=1}^{K_{g}}\phi_{k}=1$

and the generalized Gaussian distribution333The General Gaussian distribution
is chosen by its flexibility to represent multiple distribution types through
its shape parameter $\beta$ [48] [49],

$f_{k}(h_{g}|\theta_{g,k})=\frac{\Gamma(\frac{Q}{2})}{\pi^{\frac{Q}{2}}\Gamma(\frac{Q}{2\beta_{g,k}})2^{\frac{Q}{2\beta_{g,k}}}}\frac{\beta_{g,k}}{m^{\frac{Q}{2}}|\Sigma_{g,k}|^{\frac{1}{2}}}\cdot\exp\\{-\frac{1}{2m^{\beta_{g,k}}}(h_{g}-\mathbf{\mu}_{g,k})^{T}\Sigma_{g,k}^{-1}(h_{g}-\mathbf{\mu}_{g,k})\\}\;.$
(18)

where $m\in\mathbb{R}^{1\times Q}$ is a vector of scale parameters,
$\theta_{g,k}=\\{\mathbf{\mu}_{g,k},\Sigma_{g,k},\beta_{g,k}\\}$ is the
distribution parameter set with the expectation vector
$\mathbf{\mu}_{g,k}\in\mathbb{R}^{1\times Q}$, the shape vector
$\beta_{g,k}\in\mathbb{R}^{1\times Q}$ and the symmetric covariance matrix
$\Sigma_{g,k}\in\mathbb{R}^{Q\times Q}$.

For $L_{g}$ being the number of codas of type $g$ in $\mathbf{H}$ and
$h_{g}^{l}$ being the corresponding PCA-induced feature vector, we find the
distribution parameters $\hat{\Theta}_{g}=\\{\phi_{k},m,\theta_{g,k}\\}$ by
the maximum likelihood

$\displaystyle\hat{\Theta}_{g}=\operatorname*{arg\,max}_{\Theta_{g}}$
$\displaystyle\sum_{l=1}^{L_{g}}\log\left(p(h_{g}^{l}|\Theta_{g})\right)\;,$
(19)

To solve (19) we use a combination of the expectation maximization (EM)
approach with the Riemannian averaged fixed-point (RA-FA) learning method
according to the steps presented in [50] and [51], respectively.

Note that the above method requires the estimation of the number of clusters,
$K_{g}$, in (17). To estimate $K_{g}$, we consider the Bayesian information
criterion (BIC) [52], which is effective when the dimensionality is small and
the sample size $L_{g}$ is larger than the distribution parameters. Applied to
the above case

$\hat{K}_{g}=\underset{K_{g}}{\mathrm{argmin}}(\mathrm{BIC})\;,$ (20)

where

$\mathrm{BIC}=\mathrm{ln}(L_{g})\left(\sum\limits_{k=1}^{K_{g}}(|\theta_{g,k}|)-1\right)-2\cdot\mathrm{ln}\left(\sum_{l=1}^{L_{g}}\log\left(p(h_{g}^{l}|\hat{\Theta}_{g})\right)\right)\;.$
(21)

Finally, plugging (17) and (19) into (16), the temporal likelihood is
calculated by

$\mathcal{L}^{\mathrm{t}}_{(\mathbf{c}_{k})}=\sum_{g=1}^{G_{W}}p(\mathbf{O}_{(\mathbf{c}_{k})}|\hat{\Theta}_{g})\;.$
(22)

Figure 11: Left: distribution function from our database for the two
significant eigenvectors, PC1 and PC2, for codas with 5 clicks. Right:
projection map representing the temporal likelihood that an observed cluster
of transients is associated with a valid coda.

Fig. 11 shows a distribution function from our database for codas of 5 clicks
and its projection into the temporal likelihood by the above procedure. Each
point in the projection map represents the probability that a potential
cluster of ICIs represented by the principal component represents a valid
coda. We observe five non-overlapping clusters, each representing a different
type of coda: 2+3, 5R1, 5R2, 5R3 and 1+1+3. By (20), the first four codas are
represented by $\hat{K}_{g}=1$, while for the 1+1+3 coda type we estimated
$\hat{K}_{g}=3$.

####  3.3.5 Interpulse analysis

Next, we discuss the process of calculating the average number of multipulses
$\mathcal{P}(\mathbf{c}_{k})$ for (3d) to separate codas from echolocation
clicks. To identify interpulses, we use phase-slope analysis. Based on the
transient nature of pulses, an interpulse is determined at a positive zero
crossing of the phase slope function (PSF) [12]

$\displaystyle\mathrm{PSF}_{n}=-\frac{1}{J}\sum_{\omega_{1}}^{\omega_{J}}\frac{X_{R}(\omega)Y_{R}(\omega)+X_{I}(\omega)Y_{I}(\omega)}{X_{R}^{2}(\omega)+Y_{R}^{2}(\omega)}\;,$
(23)

where $X(\omega)=X_{R}(\omega)+jX_{I}(\omega),$ and $\leavevmode\nobreak\
\leavevmode\nobreak\ Y(\omega)=Y_{R}(\omega)+jY_{I}(\omega)\;,$ are the
Fourier transforms of $z_{n}$ from (2) and $nz_{n}$, respectively, and $R$ and
$I$ stand for the real and imaginary parts, respectively.

The number of identified interpulses $\mathcal{P}(\mathbf{c}_{k})$, is
calculated by the number of positive zero crossings in the PSF across the ROI
of the cluster $c_{k}$. Fig. 12 shows an example of the calculated PSF of code
clicks with high and low SNR as well as of echolocation click. The examples
illustrate the robustness of the method to weak signals, with more interpulses
observed for coda clicks even at low SNR. We note that more pulses are
expected for an echolocation click in the case of on-axis, i.e., when the
whale’s head is pointed at the receiver. However, this is a rare scenario.

(a)

(b)

(c)

Figure 12: (a) Tag measurement of a coda click, (b) Remote measurement of a
coda click, (a) Remote measurement of echolocation click. The corresponding
estimated PSF from (23) is marked in dashed red lines. Each positive zero
crossing of the PSF indicate the presence of a pulse.

###  3.4 Examples of Detection and Annotation

(a) Wide dynamic range: example of near-field annotation for two codas, one
with high SNR from the focal whale (red markers) and one with low SNR from a
distant non-focal whale (yellow markers). A zoom in of the low SNR coda is
shown in the top left box.

(b) Cocktail-party: example of far-field annotation of three overlapping
codas, each marked with a different color.

(c) High interference: example of far-field annotation of a coda signal in the
presence of echolocation clicks of similar intensity. Red crosses mark the
detected coda clicks.

Figure 13: Examples for the operation of the coda detector and annotator in
three challenging scenarios.

Examples of the operation of the detector and annotator are shown in Fig. 13.
Fig. 13a shows the ability to detect both focal and non-focal whales in a wide
dynamic range, where the intensity of the coda signal of the focal whale (red
markers) is $200$ times stronger than that of the non-focal whale (yellow
markers). An example of the annotator’s ability to separate overlapping codas
can be seen in Fig. 13b, where three overlapping codas with similar SNR can be
seen. Fig. 13a shows an example of the separation of a coda signal from an
echolocation signal with similar acoustic intensity.

## References

  * [1] Andreas, J. _et al._ Towards understanding the communication in sperm whales.  _iScience_ 25, 104393, DOI: 10.1016/j.isci.2022.104393 (2022).
  * [2] Goldwasser, S., Gruber, D., Kalai, A. T. & Paradise, O.  A theory of unsupervised translation motivated by understanding animal communication.  In _Thirty-seventh Conference on Neural Information Processing Systems_ (2023).
  * [3] Müller, J. _et al._ Soundscapes and deep learning enable tracking biodiversity recovery in tropical forests.  _Nature communications_ 14, 6191 (2023).
  * [4] Brakes, P. _et al._ Animal cultures matter for conservation.  _Science_ 363, 1032–1034 (2019).
  * [5] Brakes, P. _et al._ A deepening understanding of animal culture suggests lessons for conservation.  _Proceedings of the Royal Society B_ 288, 20202718 (2021).
  * [6] Rendell, L. E. & Whitehead, H.  Vocal clans in sperm whales (Physeter macrocephalus).  _Proc. Royal Soc. B_ 270, 225–231, DOI: 10.1098/rspb.2002.2239 (2003).
  * [7] Hersh, T. A., Gero, S., Rendell, L. & Whitehead, H.  Using identity calls to detect structure in acoustic datasets.  _Methods in Ecology and Evolution_ 12, 1668–1678 (2021).
  * [8] Sánchez-García, A., Bueno-Crespo, A. & Sancho-Gómez, J.  An efficient statistics-based method for the automated detection of sperm whale clicks.  _Applied acoustics_ 71, 451–459 (2010).
  * [9] Roch, M. A., Soldevilla, M. S., Hoenigman, R., Wiggins, S. M. & Hildebrand, J. A.  Comparison of machine learning techniques for the classification of echolocation clicks from three species of odontocetes.  _Canadian Acoustics_ 36, 41–47 (2008).
  * [10] Lohrasbipeydeh, H., Dakin, D. T., Gulliver, T. A., Amindavar, H. & Zielinski, A.  Adaptive energy-based acoustic sperm whale echolocation click detection.  _IEEE Journal of Oceanic Engineering_ 40, 957–968 (2014).
  * [11] Kandia, V. & Stylianou, Y.  Detection of sperm whale clicks based on the teager–kaiser energy operator.  _Applied Acoustics_ 67, 1144–1163 (2006).
  * [12] Kandia, V. & Stylianou, Y.  A phase based detector of whale clicks.  In _2008 New Trends for Environmental Monitoring Using Passive Systems_ , 1–6 (IEEE, 2008).
  * [13] Beslin, W. A., Whitehead, H. & Gero, S.  Automatic acoustic estimation of sperm whale size distributions achieved through machine recognition of on-axis clicks.  _The Journal of the Acoustical Society of America_ 144, 3485–3495 (2018).
  * [14] Madsen, P. _et al._ Sperm whale sound production studied with ultrasound time/depth-recording tags.  _Journal of Experimental Biology_ 205, 1899–1906 (2002).
  * [15] Møhl, B., Wahlberg, M., Madsen, P. T., Heerfordt, A. & Lund, A.  The monopulsed nature of sperm whale clicks.  _J. Acoust. Soc. Am._ 114, 1143–1154 (2003).
  * [16] Watkins, W. A. & Schevill, W. E.  Sperm whale codas.  _The Journal of the Acoustical Society of America_ 62, 1485–1490 (1977).
  * [17] Schulz, T. M., Whitehead, H., Gero, S. & Rendell, L.  Overlapping and matching of codas in vocal interactions between sperm whales: insights into communication function.  _Animal Behaviour_ 76, 1977–1988 (2008).
  * [18] Weilgart, L. & Whitehead, H.  Coda communication by sperm whales (physeter macrocephalus) off the galapagos islands.  _Canadian Journal of Zoology_ 71, 744–752 (1993).
  * [19] Rendell, L. & Whitehead, H.  Do sperm whales share coda vocalizations? insights into coda usage from acoustic size measurement.  _Animal Behaviour_ 67, 865–874 (2004).
  * [20] Rendell, L. & Whitehead, H.  Comparing repertoires of sperm whale codas: a multiple methods approach.  _Bioacoustics_ 14, 61–81 (2003).
  * [21] Gero, S., Whitehead, H. & Rendell, L.  Individual, unit and vocal clan level identity cues in sperm whale codas.  _Royal Society Open Science_ 3, 150372 (2016).
  * [22] Vachon, F., Hersh, T. A., Rendell, L., Gero, S. & Whitehead, H.  Ocean nomads or island specialists? culturally driven habitat partitioning contrasts in scale between geographically isolated sperm whale populations.  _Royal Society Open Science_ 9, 211737 (2022).
  * [23] Sharma, P. _et al._ Contextual and combinatorial structure in sperm whale vocalisations.  _bioRxiv_ 2023–12 (2023).
  * [24] Huggenberger, S., André, M. & Oelschläger, H. H. A.  The nose of the sperm whale: overviews of functional design, structural homologies and evolution.  _Journal of the Marine Biological Association of the United Kingdom_ 96, 783–806, DOI: 10.1017/S0025315414001118 (2016).
  * [25] Hersh, T. A. _et al._ Evidence from sperm whale clans of symbolic marking in non-human cultures.  _Proc. Natl. Acad. Sci._ 119, e2201692119, DOI: 10.1073/pnas.2201692119 (2022).
  * [26] Johnson, M. P. & Tyack, P. L.  A digital acoustic recording tag for measuring the response of wild marine mammals to sound.  _IEEE Journal of Oceanic Engineering_ 28, 3–12 (2003).
  * [27] Moca, V. V., Bârzan, H., Nagy-Dăbâcan, A. & Muresan, R. C.  Time-frequency super-resolution with superlets.  _Nature communications_ 12, 337 (2021).
  * [28] Schulz, T. M., Whitehead, H. & Rendell, L.  Off-axis effects on the multi-pulse structure of sperm whale coda clicks.  _The Journal of the Acoustical Society of America_ 125, 1768–1773 (2009).
  * [29] Jacobs, E. R. _et al._ The active space of sperm whale codas: inter-click information for intra-unit communication.  _Journal of Experimental Biology_ jeb–246442 (2024).
  * [30] Møhl, B., Larsen, E. & Amundin, M.  Sperm whale size determination: Outlines of an acoustic approach.  _Mammals in the Seas. Fisheries Series No. 5, Mammals of the Seas_ 3, 327–331 (1981).
  * [31] Gubnitsky, G. & Diamant, R.  Detecting the presence of sperm whales echolocation clicks in noisy environments.  _arXiv preprint arXiv:2401.00900_ (2023).
  * [32] Vachon, F., Rendell, L., Gero, S. & Whitehead, H.  Abundance estimate of eastern caribbean sperm whales using large scale regional surveys.  _Marine Mammal Science_ DOI: https://doi.org/10.1111/mms.13116.
  * [33] Gero, S., Gordon, J., Carlson, C., Evans, P. & Whitehead, H.  Population estimate and inter-island movement of sperm whales, physeter macrocephalus, in the eastern caribbean sea.  _J. Cetacean Res. Manage._ 9, 143–150 (2007).
  * [34] Gero, S. _et al._ Behavior and social structure of the sperm whales of dominica, west indies.  _Marine Mammal Science_ 30, 905–922 (2014).
  * [35] Gero, S., Bøttcher, A., Whitehead, H. & Madsen, P. T.  Socially segregated, sympatric sperm whale clans in the atlantic ocean.  _R Soc Open Sci_ 3, 160061 (2016).
  * [36] Schulz, T. M., Whitehead, H., Gero, S. & Rendell, L.  Individual vocal production in a sperm whale (physeter macrocephalus) social unit.  _Marine Mammal Science_ 27, 149–166 (2011).
  * [37] Cheng, J. T., Tracy, J. L., Ho, S. & Henrich, J.  Listen, follow me: Dynamic vocal signals of dominance predict emergent social rank in humans.  _Journal of Experimental Psychology: General_ 145, 536 (2016).
  * [38] Kavanagh, E. _et al._ Dominance style is a key predictor of vocal use and evolution across nonhuman primates.  _Royal Society open science_ 8, 210873 (2021).
  * [39] Arnbom, T.  Individual photographic identification : a key to the social organization of sperm whales (1987).
  * [40] Gubnisky, G.  Link to our matlab implementation code. (2024).  https://drive.google.com/drive/folders/1RvymG3zkM7nFbWIEiSsBm5CGgpNy9xDw.
  * [41] Gubnitsky, G. & Diamant, R.  Inter-pulse estimation for sperm whale click detection.  In _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 1–5 (IEEE, 2023).
  * [42] Kaiser, J. F.  On a simple algorithm to calculate the’energy’of a signal.  In _International conference on acoustics, speech, and signal processing_ , 381–384 (IEEE, 1990).
  * [43] Pavan, G. _et al._ Time patterns of sperm whale codas recorded in the mediterranean sea 1985–1996.  _The Journal of the Acoustical Society of America_ 107, 3487–3495 (2000).
  * [44] Amano, M., Kourogi, A., Aoki, K., Yoshioka, M. & Mori, K.  Differences in sperm whale codas between two waters off japan: possible geographic separation of vocal clans.  _Journal of Mammalogy_ 95, 169–175 (2014).
  * [45] Huggenberger, S., André, M. & Oelschläger, H. H.  An acoustic valve within the nose of sperm whales p hyseter macrocephalus.  _Mammal review_ 44, 81–87 (2014).
  * [46] Garey, M. R.  Computers and intractability: A guide to the theory of np-completeness, freeman.  _Fundamental_ (1997).
  * [47] Gordon, J. C.  Evaluation of a method for determining the length of sperm whales (physeter catodon) from their vocalizations.  _Journal of Zoology_ 224, 301–314 (1991).
  * [48] Diamant, R., Tan, H.-P. & Lampe, L.  Los and nlos classification for underwater acoustic localization.  _IEEE Transactions on mobile Computing_ 13, 311–323 (2012).
  * [49] Kotz, S.  Multivariate distributions at a cross road.  In _A Modern Course on Statistical Distributions in Scientific Work: Volume 1—Models and Structures Proceedings of the NATO Advanced Study Institute held at the University of Calgagry, Calgary, Alberta, Canada July 29–August 10, 1974_ , 247–270 (Springer, 1975).
  * [50] Boukouvalas, Z., Said, S., Bombrun, L., Berthoumieu, Y. & Adalı, T.  A new riemannian averaged fixed-point algorithm for mggd parameter estimation.  _IEEE Signal Processing Letters_ 22, 2314–2318 (2015).
  * [51] Najar, F., Bourouis, S., Zaguia, A., Bouguila, N. & Belghith, S.  Unsupervised human action categorization using a riemannian averaged fixed-point learning of multivariate ggmm.  In _Image Analysis and Recognition: 15th International Conference, ICIAR 2018, Póvoa de Varzim, Portugal, June 27–29, 2018, Proceedings 15_ , 408–415 (Springer, 2018).
  * [52] Schwarz, G.  Estimating the dimension of a model.  _The annals of statistics_ 461–464 (1978).

## Acknowledgements

This study was funded by Project CETI via grants from Dalio Philanthropies and
Ocean X; Sea Grape Foundation; Virgin Unite, Rosamund Zander/Hansjorg Wyss,
Chris Anderson/Jacqueline Novogratz through The Audacious Project: a
collaborative funding initiative housed at TED. We thank the Chief Fisheries
Officers and the Dominica Fisheries Division officers for research permits and
their collaboration in data collection; all the crews of R/V Balaena and The
DSWP team for data collection, curation, and annotation of audio recordings
across 15 years; as well as Dive Dominica, Al Dive, and W.E.T. Dominica for
logistical support while in Dominica. We are grateful to Kristian Beedholm for
CodaSorter, as well as Mark Johnson and Peter L. Tyack for their in-kind
contribution of Dtags and associated code during DSWP field research in
2014-2018.

## Availability of data and materials

Code availability of the implementation is available in the link
https://drive.google.com/drive/folders/1RvymG3zkM7nFbWIEiSsBm5CGgpNy9xDw [40].

## Author contributions statement

G.G conceived the detector and annotator design and processed the data. Y.M.
assisted in CETI data collection and manual annotation. S.G conducted and
funded DSWP fieldwork, collected, annotated, and curated DSWP dataset,
assisted in CETI data collection, edited the manuscript, and assisted in
deriving conclusions. D.G provided funding and management and assisted in
deriving conclusions. R.D conceived the detector and annotator design,
assisted in CETI data collection and was in charge of manuscript write up.

## Additional information

The authors declare no conflicts of interest.