# DualFocus: A Unified Framework for Integrating Positive and Negative
Descriptors in Text-based Person Retrieval

Yuchuan Deng  Sichuan UniversityChengduChina<EMAIL_ADDRESS>,
Zhanpeng Hu  Sichuan UniversityChengduChina<EMAIL_ADDRESS>,  Jiakun Han
Sichuan UniversityChengduChina<EMAIL_ADDRESS>,  Chuang Deng  Sichuan
UniversityChengduChina<EMAIL_ADDRESS>and  Qijun Zhao  Sichuan
UniversityChengduChina<EMAIL_ADDRESS>
###### Abstract.

Text-based person retrieval (TPR) aims to retrieve images of a person from an
extensive array of candidates based on a given textual description. The core
challenge lies in mapping visual and textual data into a unified latent space.
While existing TPR methods concentrate on recognizing explicit and positive
characteristics, they often neglect the critical influence of negative
descriptors, resulting in potential false positives that fulfill positive
criteria but could be excluded by negative descriptors. To alleviate these
issues, we introduce DualFocus, a unified framework for integrating positive
and negative descriptors to enhance the interpretative accuracy of vision-
language foundational models regarding textual queries. DualFocus employs Dual
(Positive/Negative) Attribute Prompt Learning (DAPL), which integrates Dual
Image-Attribute Contrastive (DIAC) Learning and Sensitive Image-Attributes
Matching (SIAM) Learning. This way DualFocus enhances the detection of unseen
attributes, thereby boosting retrieval precision. To further achieve a balance
between coarse and fine-grained alignment of visual and textual embeddings, we
propose the Dynamic Tokenwise Similarity (DTS) loss, which refines the
representation of both matching and non-matching descriptions, thereby
enhancing the matching process through a detailed and adaptable similarity
assessment. By focusing on token-level comparisons, DualFocus significantly
outperforms existing techniques in both precision and robustness. The
experiment results highlight DualFocus’s superior performance on CUHK-PEDES,
ICFG-PEDES, and RSTPReid.

Text-based Person Retrieval, Cross-modal Alignment

††conference: ; XXXX; XX††ccs: Text-based Person Retrieval, Cross-modal
Alignment

##  1\. Introduction

Figure 1. Illustration of the effect of negative descriptors. The green box
signifies a triumphant retrieval, whereas the red box delineates a failed
retrieval. Negative descriptors play a crucial role by providing extra
information that narrows down the search parameters, allowing for a more
precise identification of an individual.

Text-based Person Retrieval (TPR) (Li et al., 2017) is a task at the
intersection of multimedia and multimodal retrieval, aiming to identify the
most closely matching target pedestrian image from a query text. This
technology finds application across a broad spectrum of domains, including
social media analytics, image retrieval, and security surveillance. Central to
TPR is the challenge of navigating through extensive visual databases to
locate images of individuals described by textual cues, necessitating a
sophisticated amalgamation of visual and textual information.

In recent advancements, prior works (Yan et al., 2022; Zheng et al., 2020;
Jiang and Ye, 2023; Li et al., 2023b; Fujii and Tarashima, 2023; Bai et al.,
2023; Yang et al., 2023), have been achieved in enhancing the precision and
efficiency of image searches and person recognition, leveraging the
capabilities of Vision-Language Foundation Models (VLFM) such as CLIP (Radford
et al., 2021) and ALIGN (Li et al., 2021). Traditionally, as illustrated in
Figure 1, TPR approaches have primarily concentrated on harnessing explicit,
positively defined characteristics, like ’wearing a grey jacket’. This focus
often overlooks negative descriptors—attributes identified as absent. This
oversight, for example, ignoring phrases like ’not wearing glasses’, allows
the system to keep candidates that meet positive criteria but overlooks
filtering out those that should be excluded based on negative descriptors. We
suggest that the inadequate management of negative descriptions in TPR may
arise from the inherent difficulties VLFM faces in handling fine-grained tasks
like attribute detection (Yao et al., 2021).

To address the identified challenge effectively, we introduce DualFocus: an
innovative and comprehensive framework specifically crafted to highlight the
paramount importance of incorporating both positive and negative descriptors.
Central to our framework is the development of a Dual Attribute Prompt
Learning (DAPL) mechanism. This mechanism is enriched by two novel components:
Dual Image-Attribute Contrastive Learning and Sensitive Attributes Prompt
Matching Learning. Together, these components are ingeniously designed to
amplify the detection capabilities for previously unseen attributes, while
ensuring a meticulous alignment between textual descriptions and corresponding
image patch embeddings. Further augmenting our framework’s efficacy, we unveil
the Dynamic Tokenwise Similarity (DTS) loss. This innovative loss function is
a cornerstone in our strategy, aimed at refining and improving the congruence
between textual narratives and visual representations. By employing DTS loss,
DualFocus makes significant strides towards achieving a more nuanced and
precise alignment of textual descriptions with their visual counterparts,
thereby setting a new benchmark in the domain of text-based person retrieval.
This groundbreaking strategy meticulously aligns matching and mismatching
descriptions at the token level, offering a nuanced and dynamic evaluation of
similarity. Our findings demonstrate that by aligning fine-grained attributes
in a unified embedding space and employing a balanced positive-negative
granularity loss, we significantly enhance the DualFocus model’s performance
in accurately matching complex attributes.

Our contributions can be summarized as follows:

  * •

Dual Attribute Prompt Learning (DAPL) Approach: Our innovative approach
harnesses both positive and negative descriptors to enhance the detection of
previously unseen attributes. We incorporate Dual Image-Attribute Contrastive
(DIAC) Learning and Sensitive Image-Attributes Matching (SIAM) Learning to
achieve precision in aligning textual descriptions with corresponding image
patches. This strategy markedly differentiates our work from traditional
attribute recognition techniques, providing a more nuanced and accurate
framework for attribute detection.

  * •

DualFocus Framework: Building on the DAPL Approach, the DualFocus framework
enhances CLIP’s ability to interpret complex textual prompts through a
sophisticated fine-tuning protocol. It intricately balances the consideration
of positive and negative descriptors, fostering a nuanced understanding of
textual queries and elevating the standard for text-based person search.

  * •

Dynamic Tokenwise Similarity (DTS) Loss: Departing from traditional loss
functions, the DTS Loss introduces a token-level similarity assessment. This
dynamic evaluation mechanism fine-tunes the model’s sensitivity to attribute
complexity and descriptive depth, significantly improving the alignment of
attributes and the interpretation of subtle nuances across a wide spectrum.

##  2\. Related work

###  2.1. Text-to-image Person Retrieval

The landscape of Text-based Person Retrieval (TPR) has dramatically evolved,
shifting from its initial emphasis on global feature extraction towards a
sophisticated paradigm that accentuates local feature alignment and the
incorporation of auxiliary tasks. In the domain of global matching, strategies
(Chen et al., 2018; Wang et al., 2020; Han et al., 2021; Jiang and Ye, 2023)
have been developed to master a comprehensive similarity metric between images
and textual descriptions, aiming to foster improved alignment, which
collectively advocate for a unified understanding across diverse content
types. The emergence of local matching techniques, including contributions
from (Chen et al., 2022; Shao et al., 2022; Wang et al., 2020, 2022a; Yan et
al., 2022), represents a pivotal advancement. These methods, by focusing on
the alignment of local attributes, address the shortcomings of global
matching, delivering heightened accuracy albeit at the cost of increased
computational requirements. Furthermore, the pursuit of auxiliary tasks as a
mechanism for implicit alignment opens new avenues for research. Studies such
as (Jiang and Ye, 2023; Shu et al., 2022; Wu et al., 2021; Ma et al., 2023;
Fujii and Tarashima, 2023; Bai et al., 2023), leverage vision-language
pretraining models to forge superior representations, marking a strategic
pivot towards the integrated utilization of different learning paradigms. A
particularly innovative stride is made by (Yang et al., 2023), which posits
the concept of attribute prompt learning. This approach capitalizes on
attribute prompts for meticulous image-attribute alignment, concurrently
bolstering text matching learning. Such reciprocal enhancement between text
matching and attribute prompt learning enables a deeper engagement with fine-
grained details. Shifting from traditional methods focused on feature
alignment or auxiliary tasks, our innovative strategy employs dual attribute
prompt learning. This approach, integrating both positive and negative
descriptors, allows for a nuanced interpretation of textual queries.

###  2.2. Vision-Language Pre-training

Recent advancements in Vision-Language Pretrained Models (VLP) have harnessed
vast datasets of image-text pairs to explore complex semantic interplays
between visual and textual data. Central to these innovations is the ”pre-
training and fine-tuning” paradigm (Caron et al., 2021; Chen et al., 2020; He
et al., 2020). Through this paradigm, models are meticulously refined across a
spectrum of tasks, including image-text contrastive learning (Li et al., 2021;
Radford et al., 2021; Yu et al., 2022), masked language modeling (Li et al.,
2023a; Yu et al., 2022; Yao et al., 2021), and image caption generation (Dai
et al., 2023; Li et al., 2023a). The result is the generation of contextually
nuanced and semantically rich representations. These refined representations
foster a deeper integration of visual and textual elements, markedly enhancing
the efficacy of VLP models in downstream tasks.

##  3\. Method

###  3.1. DualFocus Architecture

####  3.1.1. The Architecture Overview

Figure 2 outlines our DualFocus framework. This framework is built with five
encoders: one Image Encoder ($E_{I}$), three Text Encoders ($E_{T}$) with
consistent weights, and two Cross Encoders ($E_{C}$) (Jiang and Ye, 2023). To
elucidate the components of our model, we define our training dataset as
$D=\\{\mathbf{I}_{i},\mathbf{T}_{i}\\}_{i=1}^{N}$, comprising $N$ image-text
pairs, where $\mathbf{I}_{i}$ is an image of a person and $\mathbf{T}_{i}$ its
textual description. Following (Yang et al., 2023), we perform text analysis
on each $\mathbf{T}_{i}$ to extract and map attributes $\mathbf{A}_{i}$ from a
predefined attribute set $\mathcal{A}$. This mapping generates positive
prompts $\mathbf{A}^{p}_{i}$ for detected attributes and negative prompts
$\mathbf{A}^{n}_{i}$ for absent ones. These prompts serve as inputs for our
image-attribute streams, which calculate similarity scores between the image
embeddings and both positive and negative attribute prompts, enhancing
attribute detection accuracy. Further details on these processes are provided
in the following sections.

Figure 2. Overview of th propsed DualFocus framework. It consists of five
encoders: one Image Encoder ($E_{I}$), three Text Encoders ($E_{T}$), and two
Cross Encoders ($E_{C}$). The Image Encoder generates embeddings from visual
inputs, while the Text Encoders produce embeddings from textual data. The
Cross Encoders integrate these embeddings to enhance cross-modal prediction
tasks. Text in grey boxes indicates tasks related to learning processes.
Training strategies include Dual Image-Attribute Contrastive Learning (DIAC),
which distinguishes images based on attributes; Sensitive Image-Attributes
Matching Learning (SIAM), aligning attributes in text with images; Dynamic
Tokenwise Similarity Loss (DTS), adjusting token similarity measures for
accuracy; Masked Positive Attribute Language Modeling (MPAM), predicting
masked positive attributes in context; and Masked Language Modeling (MLM),
improving understanding by predicting missing words in sentences.Identity
loss(ID), which ensures the preservation and recognition of individual
characteristics across different modalities.

####  3.1.2. Visual Representation Extraction.

Given an input image $\mathbf{I}_{i}$, we initially divide it into a series of
equally-sized, non-overlapping patches. These patches are then fed into an
image encoder $E_{I}$, producing a set of high-dimensional embeddings
$\\{\mathbf{v}_{k}^{i}\\}_{k=1,2,\cdots,n_{v}}$. To capture the global
features of the entire image, a learnable $\mathbf{v}_{cls}^{i}$ token is
incorporated. Consequently, the final visual representations are formulated as
$\mathbf{F}_{I}^{i}=\\{\mathbf{v}_{cls}^{i},\mathbf{v}_{1}^{i},\mathbf{v}_{2}^{i},\ldots,\mathbf{v}_{n_{v}}^{i}\\}$.

####  3.1.3. Textual Representation Extraction.

For the input text $\mathbf{T}_{i}$, we employ a Text Encoder $E_{T}$ to map
the textual information into a corresponding high-dimensional embedding space.
Specifically, we process the text through lower-case byte pair encoding (BPE)
(Sennrich et al., 2015) with a vocabulary size of $49,152$. The resultant
sequence of textual tokens is augmented with $[SOS]$ and $[EOS]$ tokens to
denote the beginning and end of the sequence, respectively. Consequently, the
final textual representations are formulated as
$\mathbf{F}_{T}^{i}=\\{\mathbf{t}_{sos}^{i},\mathbf{t}_{1}^{i},\mathbf{t}_{2}^{i},\cdots,\mathbf{t}_{n_{t}}^{i},\mathbf{t}_{eos}^{i}\\}$.
Similarly, attribute prompts are embedded as $\mathbf{F}_{A^{p}}^{i}$ for
positive prompts and $\mathbf{F}_{A^{n}}^{i}$ for negative prompts.

###  3.2. Dynamic Tokenwise Similarity Loss

CLIP’s (Radford et al., 2021) reliance on global representations limits
attribute precision within images. To address this, we combine token-level
similarity computations (Yao et al., 2021) with SDM (Jiang and Ye, 2023) loss
to form the Dynamic Tokenwise Similarity (DTS) loss, enhancing fine-grained
correspondences between image and text.

Given a batch of image features and text features
$\\{\left(\mathbf{F}_{I}^{i},\mathbf{F}_{T}^{j}\right),y_{i,j}\\}_{j=1}^{B}$,
where $B$ is the batch size and $y_{i,j}$ is a binary label indicating whether
the pair $\left(\mathbf{F}_{I}^{i},\mathbf{F}_{T}^{j}\right)$ shares the same
identity ($y_{i,j}=1$ for a match, and $y_{i,j}=0$ otherwise). For each visual
token in $\mathbf{F}_{I}^{i}$, we compute its similarity with all tokens in
$\mathbf{F}_{T}^{j}$, selecting the highest as the token-wise maximum
similarity $\xi_{i,j}^{I}$, for each image-text pair as follows:

(1)
$\xi_{i,j}^{I}(\mathbf{F}_{I}^{i},\mathbf{F}_{T}^{j})=\frac{1}{n^{i}_{v}}\sum_{k=1}^{n^{i}_{v}}\operatorname{sim}(\mathbf{v}_{k}^{i},\mathbf{t}_{m_{k}^{I}}^{j}),$

where $\operatorname{sim}(\cdot)$ denotes the cosine similarity function. The
index $m_{k}^{I}$, which corresponds to the maximal similarity for each visual
token, is determined by:

(2) $m_{k}^{I}=\arg\max_{0\leq
r<n^{j}_{t}}\operatorname{sim}(\mathbf{v}_{k}^{i},\mathbf{t}_{r}^{j}).$

This token-level comparison allows for capturing and contrasting features with
greater detail. Non-padded tokens from image and text samples are represented
as $\mathbf{F}_{I}^{i}$ and $\mathbf{F}_{T}^{j}$, respectively.

The probability of image-to-text matching $p^{i2t}_{i,j}$ is evaluated using a
softmax function, defined as:

(3)
$p^{i2t}_{i,j}(\mathbf{F}_{I}^{i},\mathbf{F}_{T}^{j})=\frac{\exp\left(\xi_{i,j}^{I}(\mathbf{F}_{I}^{i},\mathbf{F}_{T}^{j})/\tau\right)}{\sum_{k=1}^{B}\exp\left(\xi_{i,k}^{I}(\mathbf{F}_{I}^{i},\mathbf{F}_{T}^{k})/\tau\right)},$

where $\tau$ is a temperature hyperparameter which controls the probability
distribution peaks. Then the DTS loss from images to text in a mini-batch is
computed by:

(4)
$\mathcal{L}_{i2t}=\frac{1}{B}\sum_{i=1}^{B}\sum_{j=1}^{B}p_{i,j}^{i2t}\log\left(\frac{p_{i,j}^{i2t}}{q_{i,j}^{i2t}+\epsilon}\right),$

where $\epsilon$ is a small constant added to prevent numerical instability.
And $q_{i,j}^{i2t}=y_{i,j}/\sum_{k=1}^{B}y_{i,k}$ represents the true matching
probability. Symmetrically, the DTS loss from text to image,
$\mathcal{L}_{t2i}$, mirrors this by exchanging the roles of image and text
features. And the bi-directional DTS loss is denoted as:

(5) $\mathcal{L}_{dts}=\mathcal{L}_{i2t}+\mathcal{L}_{t2i}.$

###  3.3. Dual Attribute Prompt Learning (DAPL)

####  3.3.1. Motivation

To enhance the precision of attribute detection within images, we unveil the
Dual Attribute Prompt Learning (DAPL) approach. Diverging from the APL (Yang
et al., 2023), DAPL distinctively leverages both positive and negative
attribute prompts. This innovation is primarily achieved through the
integration of Dual Image-Attribute Contrastive Learning and Sensitive
Attributes Prompt Matching Learning, significantly bolstering the detection of
a wide array of attributes.

####  3.3.2. Dual Image-Attribute Contrastive Learning

Dual Image-Attribute Contrastive (DIAC) Learning employs both positive and
negative prompts for contrastive learning at the token-level. Specifically,
for each image $\mathbf{I}_{i}$ within an image-attribute batch, we compute
similarity scores for both types of prompts against $\mathbf{F}_{I}^{i}$, as
described in Eq. 4. For instance, the similarity score for a positive prompt
is calculated using the following methodology:

(6) $S^{i2a}_{p}=p^{i2t}_{i,j}(\mathbf{F}_{I}^{i},\mathbf{F}_{A_{p}}^{i}).$

Similarly, similarity scores $S^{a2i}_{p}$, $S^{i2a}_{n}$ and $S^{a2i}_{n}$
are computed. Following (Yang et al., 2023), for positive attribute prompts,
the Positive Image-Attribute Contrastive Learning (PIAC) Loss is given by:

(7) $\mathcal{L}_{piac}=-\frac{1}{2B}\sum_{A_{i}^{p}\in\mathcal{A}}(\log
S^{i2a}_{p}+\log S^{a2i}_{p})$

In a parallel manner, the Negative Image-Attribute Contrastive Learning (NIAC)
Loss, $\mathcal{L}_{niac}$, is calculated, leading to the formulation of the
Dual Image-Attribute Contrastive Learning (DIAC) Loss as:

(8) $\mathcal{L}_{diac}=\frac{1}{2}(\mathcal{L}_{piac}-\mathcal{L}_{niac})$

####  3.3.3. Sensitive Image-Attributes Matching Learning

The similarity scores play a pivotal role in determining the attribute
probabilities $\mathbf{p}^{a}_{i,j}$, effectively accommodating both positive
and negative attribute prompts. To maintain balanced sensitivity across
attributes, we incorporate a dynamic adjustment factor
$\gamma_{a}^{i}=\frac{\operatorname{Count}(A_{p}^{i})}{\operatorname{Count}(A_{n}^{i})}$.
This factor adjusts the weighting between positive and negative examples for
each attribute $\mathbf{A}_{i}$, ensuring an equitable evaluation.

The attribute probabilities are calculated as:

(9)
$\mathbf{p}^{a}_{i,j}=\frac{1}{2}\left(\operatorname{softmax}\left(\gamma_{a}^{i}S^{i2a}_{p}-\frac{1}{\gamma_{a}^{i}}S^{i2a}_{n}\right)+\operatorname{softmax}\left(\gamma_{a}^{i}S^{a2i}_{p}-\frac{1}{\gamma_{a}^{i}}S^{a2i}_{n}\right)\right)$

To refine the matching of sensitive attributes, we employ a specific loss
function:

(10)
$\mathcal{L}_{siam}=-\frac{1}{B}\sum_{\mathbf{A}_{i}\in\mathcal{A}}\sum_{j=1}^{B}\left(y_{i,j}\log(\mathbf{p}^{a}_{i,j})+(1-y_{i,j})\log(1-\mathbf{p}^{a}_{i,j})\right)$

####  3.3.4. Masked Positive Attribute Language Modeling

Masked Attribute Language Modeling (MAM) (Yang et al., 2023) is designed to
predict masked words utilizing clues from matched pairs of images and
attribute prompts. Our implementation incorporates the Implicit Relation
Reasoning Loss function (Jiang and Ye, 2023) to refine the prediction of
masked attributes. Specifically, we only focus on using masked positive
attribute prompts, and the loss function is represented as
$\mathcal{L}_{mapm}$.

To calculate the overall performance of the Dual Attribute Prediction and
Learning (DAPL) model, we integrate the contributions from various individual
loss functions in a balanced manner:

(11)
$\mathcal{L}_{dapl}=\frac{1}{3}(\mathcal{L}_{diac}+\mathcal{L}_{siam}+\mathcal{L}_{mapm})$

###  3.4. Overall Loss Fuction

The overall loss function for training our proposed method is defined as:

(12)
$\mathcal{L}=\lambda_{dts}\mathcal{L}_{dts}+\lambda_{mlm}\mathcal{L}_{mlm}+\lambda_{id}\mathcal{L}_{id}+\lambda_{dapl}\mathcal{L}_{DAPL},$

where $\mathcal{L}_{id}$ represents the Identity Loss (Zheng et al., 2020),
and $\mathcal{L}_{mlm}$ denotes the Implicit Relation Reasoning Loss (Jiang
and Ye, 2023) that utilizes masked text as input. The coefficients
$\lambda_{dts}$, $\lambda_{mlm}$, $\lambda_{id}$, and $\lambda_{dapl}$ are
hyperparameters designed to balance the contributions of each component loss
to the overall loss function.

##  4\. Experiments

Table 1. Performance (%) comparisons with SOTA methods on CUHK-PEDES. Results
are ordered based on the Rank-1 accuracy. In the ”Type” column, ”G” and ”L”
represent global matching and local matching methods, respectively. ”-”
indicates that the original paper did not use that specific metric to evaluate
its models.

Method | Type | Ref | Image Enc. | Text Enc. | Rank-1 | Rank-5 | Rank-10 | mAP | mINP
---|---|---|---|---|---|---|---|---|---
DSSL (Zhu et al., 2021) | L | MM21 | RN50 | BERT | 59.98 | 80.41 | 87.56 | - | -
SSAN (Ding et al., 2021) | L | arXiv21 | RN50 | LSTM | 61.37 | 80.15 | 86.73 | - | -
LBUL (Wang et al., 2022b) | L | MM22 | RN50 | BERT | 64.04 | 82.66 | 87.22 | - | -
Han et al. (Han et al., 2021) | G | BMVC21 | CLIP-RN101 | CLIP-Xformer | 64.08 | 81.73 | 88.19 | 60.08 | -
CAIBC (Wang et al., 2022a) | L | MM22 | RN50 | BERT | 64.43 | 82.87 | 88.37 | - | -
AXM-Net (Farooq et al., 2022) | L | MM22 | RN50 | BERT | 64.44 | 80.52 | 86.77 | 58.73 | -
IVT (Shu et al., 2022) | G | ECCVW22 | ViT-Base | BERT | 65.59 | 83.11 | 89.21 | - | -
BEAT (Ma et al., 2023) | L | MM23 | RN101 | BERT | 65.61 | 83.45 | 89.57 | - | -
CFine (Yan et al., 2022) | L | arXiv22 | CLIP-ViT | BERT | 69.57 | 85.93 | 91.15 | - | -
IRRA (Jiang and Ye, 2023) | G | CVPR23 | CLIP-ViT | CLIP-Xformer | 73.38 | 89.93 | 93.71 | 66.13 | 50.24
BiLMa (Fujii and Tarashima, 2023) | G | ICCVW23 | CLIP-ViT | CLIP-Xformer | 74.03 | 89.59 | 93.62 | 66.57 | -
DECL (Li et al., 2023b) | G | MM23 | CLIP-ViT | CLIP-Xformer | 75.02 | 90.89 | 94.52 | - | -
RaSa (Bai et al., 2023) | G | IJCAI23 | ALBEF-ViT | ALBEF-BERT | 76.51 | 90.29 | 94.25 | 69.38 | -
APTM (Yang et al., 2023) | L | MM23 | ALBEF-ViT | ALBEF-BERT | 76.53 | 90.04 | 94.15 | 66.91 | -
DualFocus (Ours) | L | - | CLIP-ViT | CLIP-Xformer | 77.43 | 90.73 | 94.20 | 68.35 | 53.56

###  4.1. Experimental Setup

####  4.1.1. Datasets

We evaluate our method on three text-based person retrieval datasets: (1)
CUHK-PEDES (Li et al., 2017) with $34,054$ images and $68,108$ descriptions
for training, plus validation and test sets each with over 3,000 images and
6,000 descriptions, covering $1,000$ identities. (2) ICFG-PEDES (Ding et al.,
2021), featuring 34,674 image-text pairs for training across 3,102 identities,
and nearly 20,000 pairs for testing with descriptions averaging 37 words from
a 5,554-word vocabulary. (3) RSTPReid (Zhu et al., 2021) offers comprehensive
coverage with 5 images and 2 descriptions per identity, segmented into sets
with 3,701, 200, and 200 identities, respectively.

####  4.1.2. Evaluation Metrics

TPR (Text-based Person Retrieval) system effectiveness is gauged using the
Rank-k metric (common $k$ values: $1$, $5$, $10$), which assesses the
likelihood of a correct match in the top-$k$ results of a text query.
Additionally, mean average precision (mAP) and mean Inverse Negative Penalty
(mINP) (Ye et al., 2021) are crucial for comprehensive evaluation, reflecting
accuracy and relevance for various queries. mAP quantifies the precision of
relevant image retrievals, and mINP averages the inverse penalties for
relevant pairs, with higher scores in these metrics indicating better
performance.

####  4.1.3. Implementation Details

We adopt the CLIP-ViT-B/16 model as our pre-trained image encoder and the CLIP
text transformer as our text encoder (Radford et al., 2021). Additionally, we
integrate a cross encoder as proposed by (Jiang and Ye, 2023). Following
(Jiang and Ye, 2023; Li et al., 2023b), we input images are initially resized
to $384\times 128$ pixels and undergo a series of augmentations, including
random horizontal flipping, random cropping with padding, random erasing, and
normalization to enhance the robustness of the model. The maximum token
sequence length, $n_{t}$, is configured to $77$. For optimization, the Adam
algorithm (Kingma and Ba, 2015) is employed over $50$ epochs, starting with a
learning rate of $1\times 10^{-5}$, and incorporating cosine decay to adjust
the learning rate over time. The first $5$ epochs serve as a warm-up period,
during which the learning rate linearly increases from $1\times 10^{-6}$ to
$1\times 10^{-5}$. For modules initialized without pre-trained weights, the
learning rate is set at $5\times 10^{-5}$ from the outset. In our (DTS) loss
calculation, we set the temperature parameter $\tau$ to $0.02$. The
regularization parameters $\lambda_{dts}$ and $\lambda_{dapl}$ parameters
adjusted to $2$ and $0.8$, respectively. The parameters $\lambda_{mlm}$ and
$\lambda_{id}$ are both configured with a value of 1. Our models are
implemented using PyTorch (Imambi et al., 2021) and trained on 4 NVIDIA RTX
4090 24G GPUs.

###  4.2. Comparison with State-of-the-Arts

####  4.2.1. Performance Comparisons on CUHK-PEDES

As illustrated in Table 1, our DualFocus method outperforms a range of state-
of-the-art (SOTA) techniques, particularly in Rank-1 accuracy where it
achieves 77.43%, and mean Average Precision (mAP) at 68.35%. These figures
signify a substantial leap forward in retrieval accuracy, showcasing
DualFocus’s adeptness at matching intricate person descriptions with
corresponding images. This is largely attributed to its local matching
capability, stemming from a detailed image-text alignment strategy. Notably,
unlike APTM (Yang et al., 2023), DualFocus was not pre-trained on the text-to-
person synthetic dataset MALS, emphasizing its inherent efficiency. Beyond
achieving high Rank-1 accuracy, DualFocus excels across a spectrum of metrics,
marked by a remarkable mean Inverse Negative Penalty (mINP) of 53.56. This
extensive performance is a testament to DualFocus’s prowess in person
description retrieval, marking a notable milestone in the domain.

####  4.2.2. Performance Comparisons on ICFG-PEDES

Table 2. Performance (%) comparisons with SOTA methods on ICFG-PEDES dataset.

Method | Type | Rank-1 | Rank-5 | Rank-10 | mAP | mINP
---|---|---|---|---|---|---
Dual Path (Zheng et al., 2020) | G | 38.99 | 59.44 | 68.41 | - | -
ViTAA (Wang et al., 2020) | L | 50.98 | 68.79 | 75.78 | - | -
SSAN (Ding et al., 2021) | L | 54.23 | 72.63 | 79.53 | - | -
IVT (Shu et al., 2022) | G | 56.04 | 73.60 | 80.22 | - | -
BEAT (Ma et al., 2023) | L | 58.52 | 75.92 | 81.96 | - | -
CFine (Yan et al., 2022) | L | 60.83 | 76.55 | 82.42 | - | -
IRRA (Jiang and Ye, 2023) | G | 63.46 | 80.25 | 85.82 | 38.06 | 7.93
BEAT (Ma et al., 2023) | L | 58.25 | 75.92 | 81.96 | - | -
BiLMa (Fujii and Tarashima, 2023) | G | 63.83 | 80.15 | 85.74 | 38.26 | -
DECL (Li et al., 2023b) | G | 64.88 | 81.34 | 86.72 | - | -
RaSa (Bai et al., 2023) | G | 65.28 | 80.40 | 85.12 | 41.29 | -
APTM (Yang et al., 2023) | L | 68.51 | 82.09 | 87.56 | 41.22 | -
DualFocus (Ours) | L | 67.87 | 81.93 | 87.13 | 40.13 | 9.14

As shown in Table 2, despite strong performance across several datasets,
DualFocus achieves a slightly lower than expected Rank-1 accuracy of 67.87% on
the ICFG-PEDES dataset. This figure is slightly below the leading APTM (Yang
et al., 2023) method, which records a Rank-1 accuracy of 68.51%, prompting a
deeper analysis of the dataset’s unique challenges. However, DualFocus still
performs well in Rank-5 and Rank-10 accuracy metrics and achieves a
significant mINP of 9.14%, demonstrating its robustness in retrieving relevant
queries from a large pool. A key feature of the ICFG-PEDES dataset is its
privacy-preserving anonymization of faces, which removes crucial semantic
details vital for accurate person identification. This reduction significantly
affects DualFocus’s performance, underscoring the importance of facial and
accessory cues in retrieval tasks. The impact of this anonymization reveals
the limitations of current models, including DualFocus, in handling datasets
where key identifying features are obscured, highlighting areas for future
improvement in model adaptability.

####  4.2.3. Performance Comparisons on RSTPReid

Table 3. Performance (%) comparisons with SOTA methods on RSTPReid dataset.

Method | Type | Rank-1 | Rank-5 | Rank-10 | mAP | mINP
---|---|---|---|---|---|---
DSSL (Zhu et al., 2021) | G | 39.05 | 62.60 | 73.95 | - | -
SSAN (Ding et al., 2021) | L | 43.50 | 67.80 | 77.15 | - | -
LBUL (Wang et al., 2022b) | L | 45.55 | 68.20 | 77.85 | - | -
IVT (Shu et al., 2022) | G | 46.70 | 70.00 | 78.80 | - | -
CFine (Yan et al., 2022) | L | 50.55 | 72.50 | 81.60 | - | -
IRRA (Jiang and Ye, 2023) | G | 60.20 | 81.30 | 88.20 | 47.17 | 25.28
RaSa (Bai et al., 2023) | G | 66.90 | 86.50 | 91.35 | 52.31 | -
DECL (Li et al., 2023b) | G | 61.35 | 83.95 | 90.45 | - | -
BEAT (Ma et al., 2023) | L | 48.10 | 73.10 | 81.30 | - | -
BiLMa (Fujii and Tarashima, 2023) | G | 61.20 | 81.50 | 88.80 | 48.51 | -
DECL (Li et al., 2023b) | G | 66.90 | 86.50 | 91.35 | 52.31 | -
APTM (Yang et al., 2023) | L | 67.50 | 85.70 | 91.45 | 52.56 | -
DualFocus (Ours) | L | 69.12 | 86.68 | 92.31 | 52.55 | 27.87

Results in Table 3 highlight DualFocus’s effectiveness on the RSTPReid
dataset, achieving a Rank-1 accuracy of 69.12%, outperforming the APTM method.
It also excels in Rank-5 and Rank-10 accuracies at 86.68% and 92.31%,
respectively, and records a competitive mAP of 52.55% along with the highest
mINP of 27.87%, confirming its consistent retrieval quality.

This performance demonstrates DualFocus’s ability to handle the complexities
of the RSTPReid dataset, utilizing CLIP-based visual-textual integrations to
effectively recognize and differentiate individuals against variable
backgrounds and orientations.

Overall, DualFocus consistently delivers excellent results across all
benchmark datasets, proving the robustness and generalizability of our method.

###  4.3. Hyperparameters Analysis

####  4.3.1. The impact of the $\lambda_{dts}$.

Figure 3. The impact of $\lambda_{dts}$ on R@1 on CUHK-PEDES.

As shown in Figure 3, setting $\lambda_{dts}=2$ in our analysis of the CUHK-
PEDES dataset achieves the highest Rank-1 accuracy. This optimal value
indicates that the Dynamic Tokenwise Similarity (DTS) loss effectively
balances the model’s focus between global features and localized, part-based
features. This balance is crucial as it allows the model to accurately compute
token-wise similarities, aligning detailed visual cues with textual
descriptors at a granular level. This precision is key in person re-
identification tasks, where subtle visual differences significantly impact the
matching accuracy of person images with their corresponding descriptions.

####  4.3.2. The impact of the $\lambda_{dpal}$.

Figure 4. The impact of $\lambda_{dpal}$ on R@1,5,10 on CUHK-PEDES, ICFG-
PEDES, and RSTPReid.

As shown in Figure 4, optimizing $\lambda_{dpal}$ to 0.8 markedly improves
person re-identification accuracies (Rank-1, Rank-5, and Rank-10),
demonstrating the effectiveness of dual attribute prompt learning in refining
attribute recognition. This adjustment enhances the model’s ability to discern
nuanced attributes, thereby improving precision across diverse scenarios.

The impact of $\lambda_{dpal}$ across datasets highlights its crucial role in
adapting to complex real-world conditions, boosting the model’s versatility
and robustness. Fine-tuning $\lambda_{dpal}$ along with $\lambda_{dts}$ is
essential for enhancing the performance of advanced recognition models,
leading to better accuracy and adaptability in person re-identification tasks.

###  4.4. Ablation Study

Table 4. Ablation experimental results (%) on the effectiveness of each component in DualFocus. The ”Baseline” represents the IRRA (Jiang and Ye, 2023) model. No. | Methods | Components | CUHK-PEDES | ICFG-PEDES | RSTPReid
---|---|---|---|---|---
$\mathcal{L}_{dts}$ | $DAPL$ | Rank-1 | Rank-5 | Rank-10 | Rank-1 | Rank-5 | Rank-10 | Rank-1 | Rank-5 | Rank-10
| $\mathcal{L}_{diac}$ | $\mathcal{L}_{siam}$
1 | Baseline |  |  |  | 73.38 | 89.93 | 93.71 | 63.46 | 80.25 | 85.82 | 60.20 | 81.30 | 88.20
2 | +$\mathcal{L}_{dts}$ | ✓ |  |  | 74.09 | 90.85 | 93.89 | 65.24 | 80.63 | 86.01 | 65.73 | 75.32 | 87.28
3 | +$\mathcal{L}_{diac}$ |  | ✓ |  | 73.77 | 89.72 | 92.98 | 62.56 | 79.35 | 84.71 | 59.24 | 82.26 | 87.55
4 | +$\mathcal{L}_{siam}$ |  |  | ✓ | 74.13 | 88.81 | 93.31 | 64.92 | 81.04 | 85.18 | 68.27 | 82.90 | 86.59
5 | +DAPL |  | ✓ | ✓ | 75.20 | 89.32 | 93.81 | 66.76 | 81.31 | 85.68 | 67.07 | 83.11 | 86.98
6 | DualFoucs | ✓ | ✓ | ✓ | 77.43 | 90.73 | 94.20 | 67.87 | 82.06 | 87.33 | 69.12 | 86.68 | 92.31
Table 5. Performance (%) comparisons of SOTA (State-of-the-Art) Methods with Negative Descriptors on the CUHK-PEDES, ICFG-PEDES, and RSTPReid Datasets. Methods | CUHK-PEDES | ICFG-PEDES | RSTPReid
---|---|---|---
Rank-1 | Rank-5 | Rank-10 | mAP | Rank-1 | Rank-5 | Rank-10 | mAP | Rank-1 | Rank-5 | Rank-10 | mAP
IRRA (Jiang and Ye, 2023) | 76.82 | 82.63 | 93.88 | 67.89 | 67.04 | 84.28 | 87.66 | 38.83 | 65.88 | 82.29 | 93.88 | 50.28
DECL (Li et al., 2023b) | 77.06 | 84.27 | 94.16 | 68.43 | 68.73 | 84.42 | 88.58 | 39.42 | 64.16 | 84.45 | 92.01 | 53.81
RaSa (Bai et al., 2023) | 78.64 | 85.42 | 94.67 | 69.28 | 69.27 | 84.35 | 89.44 | 40.21 | 67.17 | 85.45 | 91.19 | 54.97
DualFocus | 80.05 | 87.65 | 96.48 | 70.02 | 70.42 | 84.68 | 89.59 | 41.37 | 69.84 | 87.96 | 94.62 | 56.98

####  4.4.1. Effectiveness of the Dynamic Tokenwise Similarity (DTS) Loss

Incorporating the Dynamic Tokenwise Similarity (DTS) loss function into the
baseline model, identified as IRRA, results in significant improvements in
Rank-1 accuracy across all datasets. The model’s performance increases to
74.09% on CUHK-PEDES, 65.24% on ICFG-PEDES, and 65.73% on RSTPReid. This
improvement underscores the DTS loss’s capability in dynamically selecting
challenging samples, thereby enhancing the model’s discriminative power and
fine-grained matching precision over the Static Discriminative Margin (SDM)
loss (Jiang and Ye, 2023).

####  4.4.2. Effectiveness of Dual Attribute Prompts Learning (DAPL)

The DAPL component, which synergistically combines $\mathcal{L}_{diac}$ and
$\mathcal{L}_{siam}$, further elevates the model’s performance.
$\mathcal{L}_{diac}$ maintains consistency across different representations of
similar entities, whereas $\mathcal{L}_{siam}$ focuses on refining the model’s
sensitivity to variable data interactions, thereby reducing intra-class
variations. Independently, each contributes significantly; however, their
integration under DAPL leads to enhanced model performance with Rank-1
accuracies rising to 75.20% on CUHK-PEDES, 66.76% on ICFG-PEDES, and 67.07% on
RSTPReid. These results highlight DAPL’s crucial role in balancing input data
aspects during training and refining the model’s focus, thereby improving its
understanding of complex visual-textual relationships.

####  4.4.3. Combined Effectiveness of DTS and DAPL

The integration of both DTS and DAPL within the DualFocus configuration
achieves the highest performance metrics across all evaluated datasets,
indicating the synergistic effect of these components. This potent combination
attains Rank-1 accuracies of 77.43% on CUHK-PEDES, 67.87% on ICFG-PEDES, and
69.12% on RSTPReid. The combined implementation not only reinforces the
model’s discriminative power but also significantly enhances its ability to
accurately match textual descriptions with corresponding images, thereby
setting a new benchmark in person description retrieval tasks.

####  4.4.4. Summary

The ablation study clearly demonstrates that both the DTS and DAPL components
are integral to the DualFocus framework, significantly contributing to its
enhanced performance. The study also reveals that while each component
independently improves the model’s accuracy, their integration yields the most
substantial performance gains, thereby validating the proposed framework’s
effectiveness in leveraging complex visual-textual interplays for person
description retrieval.

###  4.5. Analysis of Sensitivity to Negative Descriptors

In this section, we evaluated the robustness of our DualFocus model against
negative descriptors in textual queries. We augmented the textual queries by
introducing two negative descriptors, derived from the predefined attribute
table (Yang et al., 2023), to assess the models’ ability to effectively
exclude irrelevant information.

The empirical results, outlined in Table 5, show that DualFocus consistently
outperforms competing methods in both Rank-1 accuracy and mean Average
Precision (mAP) across all datasets. Specifically, DualFocus achieved Rank-1
accuracies of 80.05%, 70.42%, and 69.84% on each respective dataset. These
results highlight the model’s ability to maintain high precision and accuracy
by effectively managing misleading descriptors.

The integration of two negative descriptors from a predefined attribute table
rigorously tests the models’ ability to discern relevant from non-relevant
features. DualFocus’s superior performance is enabled by its innovative
architecture that combines Dynamic Tokenwise Similarity (DTS) and Dual
Attribute Prompt Learning (DAPL), enhancing its handling of complex visual-
textual interactions. This makes DualFocus highly effective in scenarios with
imprecise or incorrect textual queries, maintaining high accuracy despite
potential distractions.

###  4.6. Comparisons on the domain generalization task

Table 6. Cross-Domain performance (%) between CUHK-PEDES and ICFG-PEDES. Here “C” denotes CUHK-PEDES, while “I” represents ICFG-PEDES. Method | C $\rightarrow$ I | I $\rightarrow$ C
---|---|---
| R@1 | R@5 | R@10 | R@1 | R@5 | R@10
SSAN (Ding et al., 2021) | 29.24 | 49.00 | 58.53 | 21.07 | 38.94 | 48.54
IRRA (Jiang and Ye, 2023) | 41.67 | 61.06 | 69.24 | 30.36 | 52.86 | 65.51
DCEL (Li et al., 2023b) | 43.31 | 62.29 | 70.31 | 32.35 | 54.86 | 65.51
DualFocus | 50.47 | 68.62 | 74.60 | 45.34 | 62.67 | 75.43

We conducted a rigorous evaluation of our DualFocus framework on the CUHK-
PEDES and ICFG-PEDES datasets to assess its domain generalization
capabilities. The results, detailed in Table 6, show that DualFocus
substantially outperforms contemporary models like SSAN (Ding et al., 2021),
IRRA (Jiang and Ye, 2023), and DCEL (Li et al., 2023b). For instance,
DualFocus achieved a Rank-1 accuracy of 50.47% when trained on CUHK-PEDES and
tested on ICFG-PEDES, notably higher than its competitors. This performance
highlights DualFocus’s strength in handling domain variations and its ability
to generalize well across different data distributions, owing to its
innovative architecture that effectively integrates visual and textual data,
enhancing adaptability and robustness in real-world scenarios.

##  5\. Conclusion

In this paper, we introduce DualFocus, a novel framework designed to enhance
text-based person retrieval (TPR) systems by integrating both positive and
negative descriptors. Through the Dual Attribute Prompt Learning (DAPL) and
the Dynamic Tokenwise Similarity (DTS) loss, DualFocus achieves precise
attribute alignment and fine-grained attribute matching, significantly
boosting system performance. This advancement not only improves the accuracy
and efficiency of traditional TPR systems but also propels the development of
multimodal retrieval technologies, demonstrating the potential of combining
linguistic and visual data to tackle complex retrieval challenges. In future
work, addressing the generation of negative descriptors remains a crucial
challenge for the DualFocus model. Currently, relying on a predefined list of
attributes to generate negative examples during training limits the model’s
ability to capture the diversity and complexity of natural language
descriptions and real-world scenarios. This constraint impacts the diversity
and practical applicability of DualFocus.

## References

  * (1)
  * Bai et al. (2023) Yang Bai, Min Cao, Daming Gao, Ziqiang Cao, Chen Chen, Zhenfeng Fan, Liqiang Nie, and Min Zhang. 2023.  RaSa: relation and sensitivity aware representation learning for text-based person search. In _Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence_. 555–563.
  * Caron et al. (2021) Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. 2021.  Emerging properties in self-supervised vision transformers. In _Proceedings of the IEEE/CVF international conference on computer vision_. 9650–9660.
  * Chen et al. (2018) Dapeng Chen, Hongsheng Li, Xihui Liu, Yantao Shen, Jing Shao, Zejian Yuan, and Xiaogang Wang. 2018.  Improving deep visual representation for person re-identification by global and local image-language association. In _ECCV_. 54–70.
  * Chen et al. (2020) Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. 2020.  Improved baselines with momentum contrastive learning.  _arXiv preprint arXiv:2003.04297_ (2020).
  * Chen et al. (2022) Yuhao Chen, Guoqing Zhang, Yujiang Lu, Zhenxing Wang, and Yuhui Zheng. 2022.  TIPCB: A simple but effective part-based convolutional baseline for text-based person search.  _Neurocomputing_ 494 (2022), 171–181.
  * Dai et al. (2023) Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023.  InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning.  arXiv:2305.06500 [cs.CV]
  * Ding et al. (2021) Zefeng Ding, Changxing Ding, Zhiyin Shao, and Dacheng Tao. 2021.  Semantically self-aligned network for text-to-image part-aware person re-identification.  _arXiv preprint arXiv:2107.12666_ (2021).
  * Farooq et al. (2022) Ammarah Farooq, Muhammad Awais, Josef Kittler, and Syed Safwan Khalid. 2022.  AXM-Net: Implicit Cross-Modal Feature Alignment for Person Re-identification. In _AAAI_ , Vol. 36. 4477–4485.
  * Fujii and Tarashima (2023) Takuro Fujii and Shuhei Tarashima. 2023.  BiLMa: Bidirectional Local-Matching for Text-based Person Re-identification. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 2786–2790.
  * Han et al. (2021) Xiao Han, Sen He, Li Zhang, and Tao Xiang. 2021.  Text-based person search with limited data.  _arXiv preprint arXiv:2110.10807_ (2021).
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020.  Momentum contrast for unsupervised visual representation learning. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 9729–9738.
  * Imambi et al. (2021) Sagar Imambi, Kolla Bhanu Prakash, and GR Kanagachidambaresan. 2021.  PyTorch.  _Programming with TensorFlow: Solution for Edge Computing Applications_ (2021), 87–104.
  * Jiang and Ye (2023) Ding Jiang and Mang Ye. 2023.  Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 2787–2797.
  * Kingma and Ba (2015) Diederik P Kingma and Jimmy Ba. 2015.  Adam: A Method for Stochastic Optimization. In _ICLR (Poster)_.
  * Li et al. (2023a) Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023a.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  _arXiv preprint arXiv:2301.12597_ (2023).
  * Li et al. (2021) Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021.  Align before fuse: Vision and language representation learning with momentum distillation.  _Advances in neural information processing systems_ 34 (2021), 9694–9705.
  * Li et al. (2017) Shuang Li, Tong Xiao, Hongsheng Li, Bolei Zhou, Dayu Yue, and Xiaogang Wang. 2017.  Person search with natural language description. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1970–1979.
  * Li et al. (2023b) Shenshen Li, Xing Xu, Yang Yang, Fumin Shen, Yijun Mo, Yujie Li, and Heng Tao Shen. 2023b.  DCEL: Deep Cross-modal Evidential Learning for Text-Based Person Retrieval. In _Proceedings of the 31st ACM International Conference on Multimedia_. 6292–6300.
  * Ma et al. (2023) Yiwei Ma, Xiaoshuai Sun, Jiayi Ji, Guannan Jiang, Weilin Zhuang, and Rongrong Ji. 2023.  Beat: Bi-directional One-to-Many Embedding Alignment for Text-based Person Retrieval. In _Proceedings of the 31st ACM International Conference on Multimedia_. 4157–4168.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021\.  Learning transferable visual models from natural language supervision. In _International conference on machine learning_. PMLR, 8748–8763.
  * Sennrich et al. (2015) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015.  Neural machine translation of rare words with subword units.  _arXiv preprint arXiv:1508.07909_ (2015).
  * Shao et al. (2022) Zhiyin Shao, Xinyu Zhang, Meng Fang, Zhifeng Lin, Jian Wang, and Changxing Ding. 2022.  Learning granularity-unified representations for text-to-image person re-identification. In _Proceedings of the 30th ACM International Conference on Multimedia_. 5566–5574.
  * Shu et al. (2022) Xiujun Shu, Wei Wen, Haoqian Wu, Keyu Chen, Yiran Song, Ruizhi Qiao, Bo Ren, and Xiao Wang. 2022.  See finer, see more: Implicit modality alignment for text-based person retrieval. In _European Conference on Computer Vision_. Springer, 624–641.
  * Wang et al. (2020) Zhe Wang, Zhiyuan Fang, Jun Wang, and Yezhou Yang. 2020.  Vitaa: Visual-textual attributes alignment in person search by natural language. In _ECCV_. 402–420.
  * Wang et al. (2022a) Zijie Wang, Aichun Zhu, Jingyi Xue, Xili Wan, Chao Liu, Tian Wang, and Yifeng Li. 2022a.  Caibc: Capturing all-round information beyond color for text-based person retrieval. In _Proceedings of the 30th ACM International Conference on Multimedia_. 5314–5322.
  * Wang et al. (2022b) Zijie Wang, Aichun Zhu, Jingyi Xue, Xili Wan, Chao Liu, Tian Wang, and Yifeng Li. 2022b.  Look before you leap: Improving text-based person retrieval by learning a consistent cross-modal common manifold. In _Proceedings of the 30th ACM International Conference on Multimedia_. 1984–1992.
  * Wu et al. (2021) Yushuang Wu, Zizheng Yan, Xiaoguang Han, Guanbin Li, Changqing Zou, and Shuguang Cui. 2021.  LapsCore: language-guided person search via color reasoning. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 1624–1633.
  * Yan et al. (2022) Shuanglin Yan, Neng Dong, Liyan Zhang, and Jinhui Tang. 2022.  Clip-driven fine-grained text-image person re-identification.  _arXiv preprint arXiv:2210.10276_ (2022).
  * Yang et al. (2023) Shuyu Yang, Yinan Zhou, Yaxiong Wang, Yujiao Wu, Li Zhu, and Zhedong Zheng. 2023.  Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark. In _Proceedings of the 2023 ACM on Multimedia Conference_.
  * Yao et al. (2021) Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. 2021.  Filip: Fine-grained interactive language-image pre-training.  _arXiv preprint arXiv:2111.07783_ (2021).
  * Ye et al. (2021) Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. 2021.  Deep learning for person re-identification: A survey and outlook.  _IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)_ (2021).
  * Yu et al. (2022) Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. 2022.  Coca: Contrastive captioners are image-text foundation models.  _arXiv preprint arXiv:2205.01917_ (2022).
  * Zheng et al. (2020) Zhedong Zheng, Liang Zheng, Michael Garrett, Yi Yang, Mingliang Xu, and Yi-Dong Shen. 2020.  Dual-path convolutional image-text embeddings with instance loss.  _ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)_ 16, 2 (2020), 1–23.
  * Zhu et al. (2021) Aichun Zhu, Zijie Wang, Yifeng Li, Xili Wan, Jing Jin, Tian Wang, Fangqiang Hu, and Gang Hua. 2021.  Dssl: Deep surroundings-person separation learning for text-based person retrieval. In _Proceedings of the 29th ACM International Conference on Multimedia_. 209–217.