# Contrastive Learning of Asset Embeddings
from Financial Time Series

Rian Dolphin  0000-0002-5607-9948 School of Computer Science, University
College DublinDublinIreland<EMAIL_ADDRESS>,  Barry Smyth
0000-0003-0962-3362 School of Computer Science, University College
DublinDublinIreland<EMAIL_ADDRESS>and  Ruihai Dong  0000-0002-2509-1370
School of Computer Science, University College DublinDublinIreland
<EMAIL_ADDRESS>
(2024)

###### Abstract.

Representation learning has emerged as a powerful paradigm for extracting
valuable latent features from complex, high-dimensional data. In financial
domains, learning informative representations for assets can be used for tasks
like sector classification, and risk management. However, the complex and
stochastic nature of financial markets poses unique challenges. We propose a
novel contrastive learning framework to generate asset embeddings from
financial time series data. Our approach leverages the similarity of asset
returns over many subwindows to generate informative positive and negative
samples, using a statistical sampling strategy based on hypothesis testing to
address the noisy nature of financial data. We explore various contrastive
loss functions that capture the relationships between assets in different ways
to learn a discriminative representation space. Experiments on real-world
datasets demonstrate the effectiveness of the learned asset embeddings on
benchmark industry classification and portfolio optimization tasks. In each
case our novel approaches significantly outperform existing baselines
highlighting the potential for contrastive learning to capture meaningful and
actionable relationships in financial data.

Representation Learning, Asset Embeddings, Financial Markets, Artificial
Intelligence, Machine Learning

††copyright: acmlicensed††journalyear: 2024††doi: XXXXXXX.XXXXXXX††conference:
International Conference on AI in Finance; November 14–16, 2024; Brooklyn,
NY††isbn: 978-1-4503-XXXX-X/18/06††ccs: Computing methodologies Machine
learning††ccs: Computing methodologies Learning latent representations

##  1\. Introduction

Representation learning has emerged as a powerful paradigm for extracting
valuable latent features from complex and high-dimensional data across various
domains, such as computer vision (Chen et al., 2020), natural language
processing (Mikolov et al., 2013), and graph mining (Kipf and Welling, 2016).
In the field of finance, learning informative representations of financial
assets is of use for a wide range of applications, including portfolio
optimization, risk management, and sector classification. However, the complex
and stochastic nature of financial markets poses unique challenges for
representation learning techniques, as the underlying relationships between
assets are often non-linear, time-varying, and influenced by a multitude of
factors (Cont, 2001).

Traditional approaches to modeling financial assets rely on handcrafted
features or statistical measures, such as historical returns, volatility, and
correlation (Markowitz, 1952). While these methods have been widely used in
practice, they often struggle to capture the intricate dependencies and
dynamics present in market data (Cont, 2001). Moreover, the increasing
availability of high-frequency trading data and the growing complexity of
financial instruments call for more sophisticated and data-driven approaches
to representation learning in finance (Sezer et al., 2020).

Recent advancements in deep learning and self-supervised learning have shown
promising results in learning meaningful representations from raw data in
various domains (Chen et al., 2020; Devlin et al., 2018). In particular,
contrastive learning has emerged as a powerful framework for learning
representations by maximizing the similarity between positive pairs of samples
while minimizing the similarity between negative pairs (Oord et al., 2018).
Contrastive learning has been successfully applied to learn representations
from images (Chen et al., 2020), text (Logeswaran and Lee, 2018), and graphs
(You et al., 2020), demonstrating its effectiveness in capturing the
underlying structure and relationships in complex data.

In this paper, we propose a novel contrastive learning framework for learning
asset embeddings from financial time series data. Our approach aims to capture
the complex relationships and similarities between assets by leveraging their
returns similarity over rolling subwindows through time. We introduce a
statistical sampling strategy based on a hypothesis test of proportions to
generate informative positive and negative samples for the contrastive
learning process. This sampling strategy addresses the challenges associated
with the noisy and stochastic nature of financial data, enabling the learning
of more robust and meaningful asset embeddings.

We explore various contrastive loss functions that weight the relationships
between the anchor asset and the positive and negative samples in different
ways. These loss functions are designed to pull the embeddings of similar
assets closer together while pushing the embeddings of dissimilar assets
apart, thereby learning a discriminative and informative representation space.
We evaluate the effectiveness of our learned asset embeddings on two
downstream financial tasks: industry sector classification and a naïve risk
hedging experiment. Our experiments on real-world financial datasets
demonstrate that our approach achieves state-of-the-art performance in these
tasks, outperforming traditional methods and showcasing the practical value of
our contrastive learning framework.

The main contributions of this paper can be summarized as follows:

  * •

We propose a novel contrastive learning framework for learning asset
embeddings from financial time series data, leveraging their returns
similarity over rolling subwindows.

  * •

We introduce a statistical sampling strategy based on a hypothesis test of
proportions to generate informative positive and negative samples for the
contrastive learning process, addressing the challenges associated with the
noisy and stochastic nature of financial data.

  * •

We explore various contrastive loss functions that capture the relationships
between the anchor asset and the positive and negative samples in different
ways, learning a discriminative and informative representation space.

  * •

We conduct rigorous experiments on real-world financial datasets and
demonstrate the effectiveness of our learned asset embeddings in downstream
tasks such as industry sector classification and risk hedging, outperforming
traditional methods and showcasing the practical value of our approach.

The remainder of this paper is organized as follows. Section 2 provides an
overview of related work on representation learning in finance and contrastive
learning. Section 3 describes our proposed contrastive learning framework,
including the statistical sampling strategy and the explored contrastive loss
functions. Section 4 presents the experimental setup and results on industry
sector classification and risk hedging tasks. Finally, Section 5 concludes the
paper and discusses future research directions.

##  2\. Related Work

Representation learning has emerged as a powerful paradigm for extracting
meaningful features from complex and high-dimensional data across various
domains, such as computer vision (Chen et al., 2020), natural language
processing (Mikolov et al., 2013), and graph mining (Kipf and Welling, 2016).
In the field of finance, learning informative representations of financial
assets is of paramount importance for a wide range of applications, including
portfolio optimization, risk management, and sector classification (Dolphin et
al., 2022b).

Traditional approaches to representation learning in finance have relied on
extracting handcrafted features from financial time series data, such as
statistical measures and technical indicators (Tsay, 2005; Murphy, 1999).
Econometric models, like the Capital Asset Pricing Model (CAPM) (Sharpe, 1964)
and the Arbitrage Pricing Theory (Ross, 1976), have also been used to capture
the relationship between asset returns and various risk factors.
Dimensionality reduction techniques, such as factor analysis, have also been
applied to financial time series data to extract low-dimensional
representations (Tsay, 2005).

In recent years, deep learning techniques have gained significant attention in
the financial domain. Recurrent Neural Networks (RNNs), particularly Long
Short-Term Memory (LSTM) networks (Hochreiter and Schmidhuber, 1997), have
been widely employed to model the temporal dependencies and capture the long-
term patterns in financial time series data (Bao et al., 2017). Furthermore,
complex deep learning techniques that have become popular in NLP and computer
vision, like transformers and diffusion models, have also been applied to
financial market data (Voigt et al., 2024; Daiya et al., 2024).

Self-supervised learning techniques have also emerged as a promising approach
for learning meaningful representations from financial time series data. These
methods aim to learn representations by solving pretext tasks that do not
require explicit labels. For example, Sarmah et al. (2022) proposed a
framework for learning asset embeddings by applying node2vec (Grover and
Leskovec, 2016) to a graph derived from returns correlations, while Dolphin et
al. (2022a, 2023) also leverage pairwise similarity between returns time
series to learn asset embeddings. Beyond time series, Gabaix et al. (2023) and
Satone et al. (2021) introduced approaches for learning representations from
holdings data, for assets and funds respectively. Furthermore, Ito et al.
(2020) leverage pre-trained language models to generate representations from
public textual filings like 10-K reports.

Contrastive learning, a popular approach within self-supervised learning, has
shown remarkable success in learning meaningful representations from complex
data in various domains. In computer vision, techniques like SimCLR (Chen et
al., 2020) have achieved state-of-the-art performance on image classification
tasks by learning representations that maximize the similarity between
positive pairs of images while minimizing the similarity between negative
pairs. In NLP, contrastive learning has been applied to learn word embeddings
(Mikolov et al., 2013) and sentence representations (Logeswaran and Lee,
2018).

Despite the success of contrastive learning in other domains, its application
to financial time series data remains relatively unexplored. The complex and
stochastic nature of financial markets poses unique challenges for
representation learning techniques, as the underlying relationships between
assets are often non-linear, time-varying, and influenced by a multitude of
factors (Cont, 2001).

In this paper, we propose a novel contrastive learning framework for learning
asset embeddings from financial time series data. Our approach aims to capture
the complex relationships and similarities between assets by leveraging the
co-occurrence of assets in their returns similarity. We introduce a
statistical sampling strategy based on a hypothesis test of proportions to
generate informative positive and negative samples for the contrastive
learning process, addressing the challenges associated with the noisy and
stochastic nature of financial data. We explore various contrastive loss
functions that capture the relationships between the anchor asset and the
positive and negative samples in different ways, learning a discriminative and
informative representation space.

Our work contributes to the growing body of research on representation
learning in finance and advances the application of contrastive learning
techniques to financial time series data. By learning meaningful and
informative asset embeddings, our approach has the potential to improve the
performance of downstream financial tasks and provide new insights into the
complex dynamics of financial markets.

##  3\. Methodology

In this section, we present our proposed contrastive learning framework for
learning asset embeddings from financial time series data. We first introduce
the problem setting and notation, followed by a description of our statistical
sampling strategy for generating positive and negative samples. We then
present the various contrastive loss functions explored in our framework and
discuss the training and optimization procedure.

###  3.1. Problem Setting and Notation

Let $\mathcal{A}=\\{a_{1},a_{2},\dots,a_{N}\\}$ be a set of $N$ financial
assets, where each asset $a_{i}$ is associated with a time series of daily
returns
$\mathbf{r}_{a_{i}}=\\{r_{1}^{a_{i}},r_{2}^{a_{i}},\dots,r_{T}^{a_{i}}\\}$
over the same period of $T$ time steps. The return $r_{t}^{a_{i}}$ of asset
$a_{i}$ at time $t$ is calculated as:

(1) $r_{t}^{a_{i}}=\frac{p_{t}^{a_{i}}-p_{t-1}^{a_{i}}}{p_{t-1}^{a_{i}}},$

where $p_{t}^{a_{i}}$ denotes the price of asset $a_{i}$ at time $t$.

Our goal is to learn an embedding function
$f:\mathcal{A}\rightarrow\mathbb{R}^{d}$ that maps each asset $a_{i}$ to a
$d$-dimensional embedding vector $\mathbf{e}_{i}\in\mathbb{R}^{d}$, such that
the embeddings capture the underlying relationships and similarities between
the assets. We denote the embedding matrix as
$\mathbf{E}\in\mathbb{R}^{N\times d}$, where each row corresponds to the
embedding vector of an asset.

###  3.2. Generating Positive and Negative Samples

A key component of our contrastive learning framework is the generation of
positive and negative sampling distributions for each asset. We propose a
statistical sampling strategy based on the co-occurrence of assets in their
returns similarity over a sliding window.

Given an asset $a_{i}$, we consider a sliding window of length $w$ and stride
$s$ over its returns time series. For each window, we compute the pairwise
similarities between the returns subsequence of $a_{i}$ and the returns
subsequences of all other assets within the same window. We use a similarity
function $sim(\cdot,\cdot)$ to measure the similarity between two returns
subsequences. The choice of the similarity function can be adapted to the
specific characteristics of the financial time series data, such as Pearson
correlation, dynamic time warping (Berndt and Clifford, 1994), or domain-
specific measures (Dolphin et al., 2021).

Figure 1. Observed distribution of $p$-values for proportion test.

####  3.2.1. Co-occurrence Matrix

We then construct a co-occurrence count matrix
$\mathbf{C}\in\mathbb{N}^{N\times N}$, where each entry $\mathbf{C}_{i,j}$
represents the number of sliding windows in which asset $a_{j}$ appears among
the top-$k$ most similar assets to $a_{i}$ across all sliding windows.
Formally,

(2)
$\mathbf{C}_{i,j}=\sum_{t=1}^{\left\lfloor\frac{T-w}{s}\right\rfloor+1}\mathds{1}\left[j\in\mathrm{topk}\left(\mathbf{s}_{a_{i},t},k\right)\right],$

where

  * •

$\mathbf{s}_{a_{i},t}$ is the vector of similarities between asset $a_{i}$ to
all other assets, within the sliding window of length $w$ starting at time
$t_{s}=(t-1)\cdot s+1$, and is given by

(3) $\hskip
28.45274pt\mathbf{s}_{a_{i},t}=\Big{[}sim\left(\mathbf{r}_{a_{i}}[t_{s}:t_{s}+w],\mathbf{r}_{a_{j}}[t_{s}:t_{s}+w]\right)\\\
\bigm{|}j\in\\{1,\dots,N\\},j\neq i\Big{]}$

The introduction of $t_{s}$ for indexing is to ensure that the sliding window
moves by the specified stride $s$ at each step.

  * •

$\mathrm{topk}(\mathbf{x},k)$ returns the indices of the top-$k$ largest
elements in a vector $\mathbf{x}$.

  * •

$\mathds{1}[\cdot]$ is the indicator function.

  * •

$\mathbf{r}_{a_{i}}[q:q+r]$ is the subsequence taken from $\mathbf{r}_{a_{i}}$
of length $r$ starting at index $q$

In this way, $\mathbf{C}_{i,j}$ denotes the number of sub windows where asset
$a_{j}$ was among the top-$k$ most similar assets to $a_{i}$ over all sliding
windows considered.

####  3.2.2. Sampling Strategy

To generate positive and negative samples for an asset $a_{i}$, we propose a
hypothesis testing approach based on the co-occurrence matrix $\mathbf{C}$. We
consider the null hypothesis $H_{0}:p_{i,j}\leq p_{0}$, where $p_{i,j}$ is the
probability of asset $a_{j}$ co-occurring with asset $a_{i}$, and $p_{0}$ is
the expected co-occurrence probability under the assumption of equal
likelihood, i.e., $p_{0}=\frac{1}{N}$.

We compute the p-value for each pair of assets $(a_{i},a_{j})$ using the test
statistic:

(4) $z_{i,j}=\frac{\hat{p}_{i,j}-p_{0}}{\sqrt{\frac{p_{0}(1-p_{0})}{n_{i}}}},$

where $\hat{p}_{i,j}=\frac{\mathbf{C}_{i,j}}{n_{i}}$ is the empirical co-
occurrence probability, and $n_{i}=\sum_{j=1}^{N}\mathbf{C}_{i,j}$ is the
total number of co-occurrences for asset $a_{i}$. The p-value is then given by
$\mathtt{p}(a_{i},a_{j})=1-\Phi(z_{i,j})$ where $\Phi(\cdot)$ is the normal
cumulative distribution function. The observed distribution of p-values is
shown in Figure 1. We see that the tails of the distribution, indicating asset
co-occurrence significantly below and above random chance, are highly
populated.

To generate positive samples for asset $a_{i}$, we only sample assets $a_{j}$
that have a p-value below a threshold $\alpha_{p}$, indicating a significantly
higher observed co-occurrence count than expected by random chance. The
sampling distribution is constructed such that the selection probability is
inversely proportional to p-value. Formally, the positive sampling
distribution is defined as:

(5)
$\mathbf{P}_{i,j}=\begin{cases}\frac{\displaystyle\mathbf{C}_{i,j}}{\displaystyle\sum_{\\{l~{}:~{}\texttt{p}(a_{i},a_{l})<\alpha^{+}\\}}\mathbf{C}_{i,l}}&\text{if
}\texttt{p}(a_{i},a_{j})<\alpha^{+}\\\\[30.0pt] 0&\text{if
}\texttt{p}(a_{i},a_{j})>\alpha^{+}\end{cases}$

where $\mathbf{P}_{i,j}$ is the probability that $a_{j}$ is drawn as a
positive sample for $a_{i}$. From this distribution we can then draw a set of
positive samples for asset $a_{i}$, which we denote $\mathcal{P}_{i}$.

For negative samples, we select assets $a_{j}$ that have a p-value above a
threshold $\alpha_{n}$, indicating a significantly lower co-occurrence count
than expected by random chance. In a similar way to $\mathbf{P}$ we define the
negative sampling distribution as:

(6)
$\mathbf{N}_{i,j}=\begin{cases}\frac{\displaystyle\max_{\begin{subarray}{c}1\leq
m\leq N\\\ m\neq
i\end{subarray}}\mathbf{C}_{i,m}-\mathbf{C}_{i,j}}{\displaystyle\sum_{\\{l~{}:~{}\texttt{p}(a_{i},a_{l})>\alpha^{-}\\}}\mathbf{C}_{i,l}}&\text{if
}\texttt{p}(a_{i},a_{j})>\alpha^{-}\\\\[30.0pt] 0&\text{if
}\texttt{p}(a_{i},a_{j})<\alpha^{-}\end{cases}$

In this case, assets with higher p-values (lower co-occurrence) are sampled
more often. $\mathbf{N}_{i,j}$ denotes the probability that $a_{j}$ is drawn
as a positive sample for $a_{i}$ and the set of negative samples for asset
$a_{i}$ is given by $\mathcal{N}_{i}$.

Figure 2. Example sampling regions based on the computed test statistic. The
negative sampling region is much larger than the positive sampling region to
promote diverse negatives and prevent degenerate solutions.

To create a diverse set of negative samples, necessary in contrastive learning
to prevent a degenerate solution (Robinson et al., 2020), we set $\alpha_{n}$
to be more conservative than $\alpha_{p}$, allowing for the inclusion of
moderately dissimilar assets in the negative sets. See Figure 2 for an
illustration of the respective sampling regions. The red region encompassing
$z$ values of $0$ indicates that assets co-occurring at the level of random
chance have non-zero probability in the negative sampling distribution.

###  3.3. Contrastive Loss Functions

We explore three loss functions to learn the asset embeddings by maximizing
the similarity between an anchor asset and its positive samples while
minimizing the similarity between the anchor asset and its negative samples.
The first does this at the level of individual positive and negative samples,
while the second aggregates positive and negative sample embeddings before
computing loss.

####  3.3.1. Individual Sigmoid Loss

The individual sigmoid loss considers each positive and negative sample
independently and aims to maximize the similarity between the anchor asset and
each positive sample while minimizing the similarity between the anchor asset
and each negative sample. The loss function is defined as:

(7)
$\mathcal{L}_{\text{ind}}(a_{i},\mathcal{P}_{i},\mathcal{N}_{i})=-\frac{1}{|\mathcal{P}_{i}|}\sum_{a_{j}\in\mathcal{P}_{i}}\log\sigma(\mathbf{e}_{i}^{\top}\mathbf{e}_{j})\\\
-\frac{1}{|\mathcal{N}_{i}|}\sum_{a_{j}\in\mathcal{N}_{i}}\log(1-\sigma(\mathbf{e}_{i}^{\top}\mathbf{e}_{j})),$

where $\sigma(\cdot)$ is the sigmoid function, and $\mathbf{e}_{i}$ and
$\mathbf{e}_{j}$ are the embedding vectors of assets $a_{i}$ and $a_{j}$,
respectively.

####  3.3.2. Aggregate Sigmoid Loss

The aggregate sigmoid loss considers the average embedding of the positive
samples and the average embedding of the negative samples. The loss function
is defined as:

(8)
$\mathcal{L}_{\text{agg}}(a_{i},\mathcal{P}_{i},\mathcal{N}_{i})=-\log\sigma\left(\mathbf{e}_{i}^{\top}\frac{1}{|\mathcal{P}_{i}|}\sum_{a_{j}\in\mathcal{P}_{i}}\mathbf{e}_{j}\right)\\\
-\log\left(1-\sigma\left(\mathbf{e}_{i}^{\top}\frac{1}{|\mathcal{N}_{i}|}\sum_{a_{j}\in\mathcal{N}_{i}}\mathbf{e}_{j}\right)\right).$

####  3.3.3. Hybrid Sigmoid-Softmax Loss

The hybrid sigmoid-softmax loss combines the individual sigmoid loss for
positive samples with a softmax loss that contrasts the similarity between the
anchor asset and the average positive embedding against the similarities
between the anchor asset and each individual negative sample. The loss
function is defined as:

(9)
$\mathcal{L}_{\text{hybrid}}(a_{i},\mathcal{P}_{i},\mathcal{N}_{i})=-\frac{1}{|\mathcal{P}_{i}|}\sum_{a_{j}\in\mathcal{P}_{i}}\log\sigma(\mathbf{e}_{i}^{\top}\mathbf{e}_{j})\\\
-\log\frac{\exp\left(\mathbf{e}_{i}^{\top}\frac{1}{|\mathcal{P}_{i}|}\sum_{a_{j}\in\mathcal{P}_{i}}\mathbf{e}_{j}\right)}{\exp\left(\mathbf{e}_{i}^{\top}\frac{1}{|\mathcal{P}_{i}|}\sum_{a_{j}\in\mathcal{P}_{i}}\mathbf{e}_{j}\right)+\sum_{a_{j}\in\mathcal{N}_{i}}\exp(\mathbf{e}_{i}^{\top}\mathbf{e}_{j})}.$

###  3.4. Training and Optimization

We train the asset embeddings using mini-batch stochastic gradient descent
with the Adam optimizer (Kingma and Ba, 2014). In each training iteration, we
sample a batch of anchor assets and their corresponding positive and negative
samples, and compute the contrastive loss for each anchor asset using one of
the loss functions described above. The gradients are then computed and used
to update the embedding matrix $\mathbf{E}$.

To prevent overfitting and encourage the learned embeddings to be uniformly
distributed on the unit hypersphere, we apply a custom regularization to the
embedding vectors after each update step:

(10)
$\mathcal{L}_{\text{reg}}=\lambda\sum_{i=1}^{N}\left(\left\lVert\mathbf{e}_{i}\right\rVert_{2}^{2}-1\right)^{2}$

where $\lambda=0.001$ is the regularization parameter.

We also employ a learning rate scheduler that reduces the learning rate by a
factor of 0.8 when the positive sample portion of the loss plateaus for a
specified number of epochs. This allows the model to fine-tune the embeddings
in the later stages of training and converge to a better optimum. The training
process is summarized in Algorithm 1.

Input: Asset returns $\\{\mathbf{r}_{a_{i}}\\}_{i=1}^{N}$, embedding dimension
$d$, batch size $B$, number of epochs $E$, learning rate $\eta$, positive
threshold $\alpha_{p}$, negative threshold $\alpha_{n}$

Output: Learned asset embeddings $\mathbf{E}$

1 Initialize the embedding matrix $\mathbf{E}\in\mathbb{R}^{N\times d}$
randomly;

2 Compute the co-occurrence matrix $\mathbf{C}$ based on the asset returns
$\\{\mathbf{r}_{a_{i}}\\}_{i=1}^{N}$;

3 for  _$epoch=1,2,\dots,E$_ do

4 for  _$batch=1,2,\dots,\lceil\frac{N}{B}\rceil$_ do

5 Sample a batch of anchor assets $\\{a_{i}\\}_{i=1}^{B}$;

6 for  _$i=1,2,\dots,B$_ do

7 Generate positive samples $\mathcal{P}_{i}$ based on $\mathbf{C}$ and
$\alpha_{p}$;

8 Generate negative samples $\mathcal{N}_{i}$ based on $\mathbf{C}$ and
$\alpha_{n}$;

9 Compute the contrastive loss
$\mathcal{L}(a_{i},\mathcal{P}_{i},\mathcal{N}_{i})$;

10

11 end for

12 Compute the average contrastive loss over the batch;

13 Update the embedding matrix $\mathbf{E}$ using Adam optimizer with learning
rate $\eta$;

14 Apply $\ell_{2}$ normalization to the embedding vectors;

15

16 end for

17 Adjust the learning rate based on the validation loss;

18

19 end for

Algorithm 1 Training procedure for learning asset embeddings using contrastive
learning

##  4\. Experimental Results

In this section, we present the results obtained when using our contrastive
learning approach on two important tasks: industry sector classification and
portfolio optimization. We compare our approach with traditional methods to
demonstrate that our learned embeddings offer significant performance
improvements which suggests that they are better able to capture meaningful
relationships between financial assets than alternative approaches.

###  4.1. Dataset and Experimental Setup

For this evaluation we use a publicly available, real-world financial dataset
consisting of daily returns for a universe of 611 U.S. stocks from 2000 to
2018111This dataset has been collected by the authors and can be found at
https://github.com/rian-dolphin/stock-embeddings.. This dataset includes the
industry sector labels for each stock, which are used for evaluating the
sector classification task. The sector labels are based on the Global Industry
Classification Standard (GICS) (S&P and MSCI, 2020), which categorizes
companies into 11 sectors based on their principal business activities.

For the contrastive learning framework, we set the embedding dimension to
$d=16$, the sliding window length to $w=22$ (corresponding to the number of
trading days in a month), the stride to $s=5$ (trading days in a week), and
the number of top-$k$ similar assets to $k=5$. We use the Pearson correlation
coefficient as the similarity measure between returns subsequences. The
positive and negative thresholds are set to $\alpha_{p}=0.05$ and
$\alpha_{n}=0.3$, respectively and we train the embeddings for 30 epochs using
the Adam optimizer with a learning rate of 0.001 and a batch size of 128.

###  4.2. Industry Sector Classification

TSC Baseline | F1 | Accuracy
---|---|---
Catch22 | 0.31 | 35%
Contractable BOSS | 0.37 | 39%
RBOSS | 0.45 | 42%
Shapelet | 0.45 | 42%
Shapelet Transform | 0.40 | 46%
WEASEL | 0.47 | 47%
MUSE | 0.51 | 54%
TS Forest Classifier | 0.53 | 55%
Canonical Interval Forest | 0.52 | 56%
Arsenal | 0.53 | 58%
Dolphin et al. (2022b)* | 0.60 | 60%
Sarmah et al. (2022)* | 0.61 | 61%
Dolphin et al. (2023)* | 0.66 | 65%
Proposed Contrastive |  |
Individual Sigmoid | 0.68 | 68%
Aggregate Sigmoid | 0.49 | 49%
Sigmoid-Softmax | 0.69 | 69%
Table 1. Sector classification results for the proposed contrastive loss
functions and baselines. * indicates a best-effort implementation by the
authors.

To evaluate the quality of the learned asset embeddings, we first consider the
task of industry sector classification. This task involves assigning companies
into industry sectors (Technology, Health Care, etc.) and was chosen for two
reasons. Firstly, it is frequently used as a benchmarking evaluation task
(Sarmah et al., 2022; Dolphin et al., 2023). Secondly, it is crucial to real-
world financial workflows like indentifying peers and competitors within a
given industry, and deciding on the constituent assets of sector-specific
Exchange Traded Funds (ETFs). In fact, approximately 30% of publications in
the top-three finance journals rely on industry sector classification schemes
(Weiner, 2005), further indicating their importance.

We use the learned embeddings as input features to a support vector machine
(SVM) classifier and perform 5-fold cross-validation. Table 1 presents the
sector classification results, in terms of the average F1-score and accuracy
across the folds, for our proposed contrastive learning approach with
different loss functions, along with several baseline methods. The baseline
methods include traditional time series classification methods as well as
state-of-the-art task specific baselines (Sarmah et al., 2022; Dolphin et al.,
2022b).

The results show that our contrastive learning approach using the sigmoid-
softmax loss achieves the best performance across all metrics, with an
F1-score of 0.69 and an accuracy of 69%. This demonstrates the effectiveness
of our learned embeddings in capturing the sector-level similarities and
relationships between assets, despite only using returns data in the learning
process. The individual sigmoid loss also performs well despite it’s
simplicity, with an F1-score of 0.68 and an accuracy of 68%. Interestingly,
the aggregate sigmoid loss performs much worse than the other contrastive loss
functions, suggesting that preserving the individual characteristics of
positive and negative samples in the loss function is important for learning
high-quality asset embeddings.

The proposed contrastive learning approach outperforms traditional time series
baseline methods, as well as recent approaches for learning asset embedding
from returns time series. This demonstrates the effectiveness of our framework
in capturing meaningful relationships between financial assets.

Figure 3. 2D visualization of the learned asset embeddings using t-SNE. Assets
from the same industry sector tend to cluster together in the embedding space.

Figure 3 visualizes the learned asset embeddings in two dimensions using t-SNE
(Van der Maaten and Hinton, 2008), a dimensionality reduction technique that
preserves the local structure of high-dimensional data. The visualization
shows that assets from the same industry sector tend to cluster together in
the embedding space, further validating the quality of our learned embeddings.

####  4.2.1. Exploring the Learned Embedding Space

Table 2. Top 5 nearest neighbors for Pfizer Inc. (PFE) in the learned
embedding space.
Pfizer has ground truth sector and industry labels of Health Care and Major Pharmaceuticals. Rank | Company | Ticker | Sector | Industry | Similarity
---|---|---|---|---|---
1 | Eli Lilly and Company | LLY | Health Care | Major Pharmaceuticals | 0.9778
2 | Baxter International Inc. | BAX | Health Care | Medical/Dental Instruments | 0.9718
3 | Merck & Company, Inc. | MRK | Health Care | Major Pharmaceuticals | 0.9706
4 | Medtronic plc | MDT | Health Care | Biotechnology: Electromedica | 0.9620
5 | Johnson & Johnson | JNJ | Health Care | Major Pharmaceuticals | 0.9620

To gain further insight into the information captured by the learned asset
embeddings, we explore the nearest neighbors of specific assets in the
embedding space. Table 2 presents the five nearest neighbors of Pfizer Inc.
(PFE), a major pharmaceutical company, based on cosine similarity in the
embedding space. We observe that the nearest neighbors are all classified as
healthcare companies in the traditional sector labels, with the three of the
top five belonging to the same finer grained industry classification, “Major
Pharmaceuticals”, as Pfizer. This example demonstrates that the embeddings
have effectively captured the sector and industry-level relationships between
assets.

Interestingly, the embeddings also capture nuanced relationships that may be
missed by traditional sector classification schemes. Table 3 shows the five
assets with the highest cosine similarity to Procter & Gamble (PG), a company
in the Basic Industries sector and the “Package Goods/Cosmetics” industry. In
this case, none of the nearest neighbours are from the same high-level Basic
Industries sector, and they also span different industries such as
“Beverages”, “Packaged Foods”, and “Specialty Chemicals”. This suggests that
the embeddings have learned similarities between assets that go beyond the
rigid boundaries of traditional industry classifications.

Table 3. High similarity “mismatches” for Procter & Gamble (PG) in the learned
embedding space.
Procter & Gamble has ground truth sector and industry labels of Basic Industries and Package Goods/Cosmetics. Rank | Company | Ticker | Sector | Industry | Similarity
---|---|---|---|---|---
1 | Colgate-Palmolive Company | CL | Consumer Non-Durables | Package Goods/Cosmetics | 0.9907
2 | PepsiCo, Inc. | PEP | Consumer Non-Durables | Beverages (Production/Distri) | 0.9899
3 | Conagra Brands, Inc. | CAG | Consumer Non-Durables | Packaged Foods | 0.9876
4 | Clorox Company | CLX | Consumer Durables | Specialty Chemicals | 0.9832
5 | Kimberly-Clark Corporation | KMB | Consumer Durables | Containers/Packaging | 0.9825

These examples demonstrate that our contrastive learning approach has captured
meaningful relationships between assets that extend beyond traditional sector
and industry classifications. By learning from the raw time series data in an
entirely self-supervised manner, the embeddings are able to uncover nuanced
similarities that may be overlooked by rigid classification schemes. This
highlights the potential of our approach to provide new insights into the
relationships between financial assets and to enable more data-driven decision
making in various financial applications.

###  4.3. Portfolio Optimization

To further demonstrate the practical value of our learned asset embeddings, we
consider a naïve hedging task designed to proxy portfolio optimization. We
construct a simple long-only portfolio for each asset, where the asset is the
target, and a single other asset is selected as the hedge based on the
similarity information encoded in the embeddings (or the baseline similarity
methods). For this experiment, we learn embeddings and compute similarity
based on returns data from 2000-2012 and evaluate on the remaining period from
2013-2018.

For each target asset, we select the hedge asset randomly from the 25 most
dissimilar assets (lowest cosine similarity in the embedding space) and repeat
the experiment 100 times for robustness and to allow for statistical
significance testing. With the selected hedge asset, we construct a two-asset
portfolio with equal weights and evaluate its out-of-sample realized
volatility over the test period. The hypothesis is that if the embedding space
is learning information valuable for portfolio optimization applications, then
we should see the embedding space yielding successful hedge assets that result
in lower out-of-sample volatility.

We compare our approach with baseline similarity methods. For example, the
most natural choice is to follow modern portfolio theory (Markowitz, 1952) and
select the hedge asset based on the lowest Pearson correlation with the target
asset. We also compare against tailored similarity metrics (Chun and Ko, 2020)
and recent embedding-based approaches (Dolphin et al., 2022b, 2023; Sarmah et
al., 2022). Table 4 reports the average realized out-of-sample volatility
across all portfolios for our contrastive learning approach with different
loss functions and the baseline methods. The baseline methods are split
between traditional similarity metrics and task-specific representation
learning frameworks.

Table 4. Portfolio hedging experiment results along with Tukey HSD test indicating significantly lower volatility than Pearson baseline at $\alpha=0.01$. * indicates the authors best effort implementation. Method | Avg Volatility | Significant
---|---|---
Pearson | 23.8% | -
Spearman | 24.0% | ✗
Chun and Ko (2020)* | 23.9% | ✗
Dolphin et al. (2022b)* | 21.3% | ✓
Sarmah et al. (2022)* | 22.0% | ✓
Dolphin et al. (2023)* | 22.5% | ✓
Individual Sigmoid |  19.1% | ✓
Aggregate Sigmoid | 20.3% | ✓
Sigmoid-Softmax | 26.1% | ✗

The results show that our contrastive learning approach with the individual
sigmoid loss achieves the lowest average volatility of 19.1%, outperforming
the baseline method based on Pearson correlation with statistical
significance. This indicates that the similarity information captured by our
learned embeddings is effective in identifying good hedging pairs and
constructing portfolios with lower risk.

The aggregate sigmoid loss also performs well, with an average volatility of
20.3%, demonstrating its ability to capture overall characteristics of
positive and negative samples for portfolio optimization. However, the hybrid
sigmoid-softmax loss, despite its strong performance in sector classification,
yields a higher average volatility compared to the baseline method, suggesting
that the optimal choice of loss function may depend on the specific downstream
task.

Figure 4 presents the distribution of realized volatility for the portfolios
constructed using the proposed embeddings and the Pearson correlation
baseline. The portfolios based on our learned embeddings exhibit a lower
median volatility and a tighter distribution compared to the baseline, further
highlighting the effectiveness of our approach in capturing meaningful
relationships between assets for portfolio optimization.

Figure 4. Distribution of realized volatility for portfolios constructed using
our individual sigmoid loss approach and the Pearson correlation baseline.

Overall, the experimental results on industry sector classification and
portfolio optimization demonstrate the effectiveness of our contrastive
learning framework in learning meaningful and informative asset embeddings
from financial time series data. The superior performance compared to
traditional methods and the practical value in downstream financial tasks
highlight the potential of our approach in capturing complex relationships and
similarities between financial assets.

##  5\. Discussion & Conclusion

In this paper, we proposed a novel contrastive learning and sampling framework
for learning embedding representations of financial assets from returns time
series data. Our approach leverages the co-occurrence of assets in their
returns similarity to generate informative positive and negative samples for
the contrastive learning process. We introduced a statistical sampling
strategy based on a hypothesis test of proportions to address the challenges
associated with the noisy and stochastic nature of financial data. We explored
various contrastive loss functions that capture the relationships between the
anchor asset and the positive and negative samples in different ways, learning
a discriminative and informative representation space.

Our experimental results on real-world financial datasets demonstrate the
effectiveness of our learned asset embeddings in downstream tasks such as
industry sector classification and portfolio optimization. The superior
performance compared to popular time series classification techniques and
tailored state-of-the-art approaches highlights the potential of our approach
in capturing complex relationships and similarities between financial assets.
The statistically significant benefits of our learned embeddings in the
hedging experiment further emphasizes the potential of our framework to add
value to real-world financial applications.

While our work focuses on financial time series data, the proposed techniques
are also applicable to a wide range of non-financial time series. The ability
to learn meaningful representations from raw time series data without relying
on handcrafted features or domain-specific knowledge makes our approach highly
adaptable to various application scenarios. Exploring the effectiveness of our
contrastive learning framework in these non-financial domains is a promising
avenue for future research.

One potential limitation of our approach is the scalability of the pairwise
similarity computation over multiple windows, particularly for datasets with a
large number of time series or very long time series. As the number of assets
and the length of the time series grow, the computational complexity of
calculating pairwise similarities across multiple windows can become
intractable. However, this issue can be mitigated to some extent by adjusting
the window size and stride parameters, effectively reducing the number of
comparisons required. Furthermore, recent research has shown that this type of
time series similarity calculation can scale without issue due to (i) the
ability to leverage hardware and parallelize calculations and (ii) new
algorithms that make the time complexity constant in subsequence length. For
example, a recent paper demonstrates how to use GPUs to compute more than ten
quadrillion pairwise comparisons for time series motifs and joins (Zhu et al.,
2018). By implementing these advances in the similarity computation, we can
overcome this limitation and ensure the scalability of our approach to larger
datasets.

In conclusion, our work introduces a novel contrastive learning framework for
learning asset embeddings from financial time series data, demonstrating its
effectiveness in capturing meaningful relationships and similarities between
assets. The proposed statistical sampling strategy and the exploration of
various contrastive loss functions contribute to the robustness and
discriminative power of the learned embeddings. The superior performance in
industry sector classification and portfolio optimization highlights the
practical value of our approach in real-world financial applications. We
believe that our work opens up new avenues for representation learning in
finance and has the potential to be extended to non-financial time series
domains, paving the way for more accurate and data-driven decision-making in
various fields.

###### Acknowledgements.

This publication has emanated from research conducted with the financial
support of Science Foundation Ireland under Grant number 18/CRT/6183 and
12/RC/2289_P2.

## References

  * (1)
  * Bao et al. (2017) Wei Bao, Jun Yue, and Yulei Rao. 2017.  A deep learning framework for financial time series using stacked autoencoders and long-short term memory.  _PloS one_ 12, 7 (2017), e0180944.
  * Berndt and Clifford (1994) Donald J Berndt and James Clifford. 1994.  Using dynamic time warping to find patterns in time series. In _Proceedings of the 3rd international conference on knowledge discovery and data mining_. 359–370.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations. In _International conference on machine learning_. PMLR, 1597–1607.
  * Chun and Ko (2020) Se-Hak Chun and Young-Woong Ko. 2020.  Geometric Case Based Reasoning for Stock Market Prediction.  _Sustainability_ 12, 17 (9 2020), 7124.  Issue 17. https://doi.org/10.3390/su12177124
  * Cont (2001) Rama Cont. 2001.  Empirical properties of asset returns: stylized facts and statistical issues.  _Quantitative finance_ 1, 2 (2001), 223.
  * Daiya et al. (2024) Divyanshu Daiya, Monika Yadav, and Harshit Singh Rao. 2024.  Diffstock: Probabilistic relational stock market predictions using diffusion models. In _ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 7335–7339.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Dolphin et al. (2022a) Rian Dolphin, Barry Smyth, and Ruihai Dong. 2022a.  A Machine Learning Approach to Industry Classification in Financial Markets. In _Irish Conference on Artificial Intelligence and Cognitive Science_. Springer, 81–94.
  * Dolphin et al. (2022b) Rian Dolphin, Barry Smyth, and Ruihai Dong. 2022b.  Stock Embeddings: Learning Distributed Representations for Financial Assets.  _arXiv preprint arXiv:2202.08968_ (2022).
  * Dolphin et al. (2023) Rian Dolphin, Barry Smyth, and Ruihai Dong. 2023.  A Case-Based Reasoning Approach to Company Sector Classification Using a Novel Time-Series Case Representation. In _International Conference on Case-Based Reasoning_. Springer, 375–390.
  * Dolphin et al. (2021) Rian Dolphin, Barry Smyth, Yang Xu, and Ruihai Dong. 2021.  Measuring Financial Time Series Similarity With a View to Identifying Profitable Stock Market Opportunities. In _International Conference on Case-Based Reasoning_. Springer, 64–78.
  * Gabaix et al. (2023) Xavier Gabaix, Ralph SJ Koijen, and Motohiro Yogo. 2023.  Asset Embeddings.  _Available at SSRN_ (2023).
  * Grover and Leskovec (2016) Aditya Grover and Jure Leskovec. 2016.  node2vec: Scalable feature learning for networks. In _Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining_. 855–864.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ 9, 8 (1997), 1735–1780.
  * Ito et al. (2020) Tomoki Ito, Jose Camacho Collados, Hiroki Sakaji, and Steven Schockaert. 2020.  Learning company embeddings from annual reports for fine-grained industry characterization.  (2020).
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ (2014).
  * Kipf and Welling (2016) Thomas N Kipf and Max Welling. 2016.  Semi-supervised classification with graph convolutional networks.  _arXiv preprint arXiv:1609.02907_ (2016).
  * Logeswaran and Lee (2018) Lajanugen Logeswaran and Honglak Lee. 2018.  An efficient framework for learning sentence representations.  _arXiv preprint arXiv:1803.02893_ (2018).
  * Markowitz (1952) Harry Markowitz. 1952.  Portfolio Selection.  (1952).
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_ (2013).
  * Murphy (1999) John J Murphy. 1999.  _Technical analysis of the financial markets: A comprehensive guide to trading methods and applications_.  Penguin.
  * Oord et al. (2018) Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018.  Representation learning with contrastive predictive coding.  _arXiv preprint arXiv:1807.03748_ (2018).
  * Robinson et al. (2020) Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. 2020.  Contrastive learning with hard negative samples.  _arXiv preprint arXiv:2010.04592_ (2020).
  * Ross (1976) Stephen Ross. 1976.  The arbitrage pricing theory.  _Journal of Economic Theory_ 13, 3 (1976), 341–360.
  * Sarmah et al. (2022) Bhaskarjit Sarmah, Nayana Nair, Dhagash Mehta, and Stefano Pasquali. 2022.  Learning Embedded Representation of the Stock Correlation Matrix using Graph Machine Learning.  _arXiv preprint arXiv:2207.07183_ (2022).
  * Satone et al. (2021) Vipul Satone, Dhruv Desai, and Dhagash Mehta. 2021.  Fund2Vec: mutual funds similarity using graph learning. In _Proceedings of the Second ACM International Conference on AI in Finance_. 1–8.
  * Sezer et al. (2020) Omer Berat Sezer, Mehmet Ugur Gudelek, and Ahmet Murat Ozbayoglu. 2020.  Financial time series forecasting with deep learning: A systematic literature review: 2005–2019.  _Applied Soft Computing_ 90 (2020), 106181.
  * Sharpe (1964) William F Sharpe. 1964.  Capital asset prices: A theory of market equilibrium under conditions of risk.  _The journal of finance_ 19, 3 (1964), 425–442.
  * S&P and MSCI (2020) S&P and MSCI. 2020.  Global Industry Classification Standard (GICS) Methodology.  (2020).
  * Tsay (2005) Ruey S Tsay. 2005.  _Analysis of financial time series_.  John wiley & sons.
  * Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008.  Visualizing data using t-SNE.  _Journal of machine learning research_ 9, 11 (2008).
  * Voigt et al. (2024) Frederic Voigt, Kai Von Luck, and Peer Stelldinger. 2024.  Assessment of the Applicability of Large Language Models for Quantitative Stock Price Prediction. In _Proceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments_. 293–302.
  * Weiner (2005) Christian Weiner. 2005.  The impact of industry classification schemes on financial research.  _Available at SSRN 871173_ (2005).
  * You et al. (2020) Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020.  Graph contrastive learning with augmentations.  _Advances in neural information processing systems_ 33 (2020), 5812–5823.
  * Zhu et al. (2018) Yan Zhu, Zachary Zimmerman, Nader Shakibay Senobari, Chin-Chia Michael Yeh, Gareth Funning, Abdullah Mueen, Philip Brisk, and Eamonn Keogh. 2018.  Exploiting a novel algorithm and GPUs to break the ten quadrillion pairwise comparisons barrier for time series motifs and joins.  _Knowledge and Information Systems_ 54 (2018), 203–236.