#  Kun: Answer Polishment for Chinese Self-Alignment
with Instruction Back-Translation

1Tianyu Zheng11footnotemark: 1 , 1Shuyue Guo11footnotemark: 1 ,
5Xingwei Qu, 5Jiawei Guo, 5Weixu Zhang, 1Xinrun Du,
4 5Chenghua Lin, 1 5Wenhao Huang22footnotemark: 2 , 2 5Wenhu
Chen22footnotemark: 2 , 3 5Jie Fu22footnotemark: 2 , 1 2 5Ge
Zhang11footnotemark: 1 22footnotemark: 2
101.AI, 2University of Waterloo,
3HKUST, 4University of Manchester, 5Multimodal Art Projection Research
Community

###### Abstract

In this paper, we introduce Kun††The dataset is named Kun as Chinese
pronunciation of Humpback Li et al. (2023c)., a novel approach for creating
high-quality instruction-tuning datasets for large language models (LLMs)
without relying on manual annotations. Adapting a self-training algorithm
based on instruction back-translation and answer polishment, Kun leverages
unlabelled data from diverse sources such as Wudao, Wanjuan, and SkyPile to
generate a substantial dataset of over a million Chinese instructional data
points. This approach significantly deviates from traditional methods by using
a self-curation process to refine and select the most effective instruction-
output pairs. Our experiments with the 6B-parameter Yi model across various
benchmarks demonstrate Kun’s robustness and scalability. Our method’s core
contributions lie in its algorithmic advancement, which enhances data
retention and clarity, and its innovative data generation approach that
substantially reduces the reliance on costly and time-consuming manual
annotations. This methodology presents a scalable and efficient solution for
improving the instruction-following capabilities of LLMs, with significant
implications for their application across diverse fields. The code and dataset
can be found at https://github.com/Zheng0428/COIG-Kun

††footnotetext: * These authors contributed equally.††footnotetext: †
Corresponding Authors. Figure 1: Overview of Answer Polishment. Initially, the
Yi base model is fine-tuned using quality seed instruction data to create a
label and a primary chat model. The label model then annotates a large amount
of primary data, turning it into labeled data. This is filtered and refined by
rules and the primary chat model, producing the final dataset. This dataset is
used to further train the primary chat model, resulting in an highly efficient
final chat model.

##  1 Introduction

The development of large language models (LLMs) relies on human-annotated
datasets, a method facing scalability issues due to the significant resources
required. Our study introduces Kun, a novel solution utilizing unlabelled data
to create a high-quality instruction-tuning dataset. This method diverges from
manual annotations, employing a self-training algorithm that includes a unique
process called AP (Answer Polishment),

AP is central to Kun’s strategy. It addresses a critical challenge in the
Humpback (Li et al., 2023c) method, where raw data, once labeled, are directly
used in instruction datasets. The unscreened raw data often mismatches between
instructions and responses, as raw data may not inherently align with the
instructional context. AP refines this raw data, ensuring a tighter
correlation between the instructions and responses through a back-translation
process. This leads to a dataset where each instruction-output pair is more
coherent and contextually relevant. Unlike methods dependent on LLMs (Peng et
al., 2023; Taori et al., 2023a; Zheng et al., 2023), Kun offers an independent
and scalable approach to instruction-based training.

We opt for the 6B-parameter Yi model due to its open-source nature and
dependable performance††https://github.com/01-ai/Yi. Its efficacy is tested
and proven across diverse dataset sizes, including widely recognized
benchmarks like C-EVAL and CMMLU. A comprehensive human evaluation uses 500
prompts from ShareGPT-zh, covering various tasks. Responses generated by our
model are compared with those from other models, showcasing the superiority of
our _Kun-52k_ variant. Additionally, we evaluate the quality of our dataset,
which includes 1,000 instruction-output pairs each from sources like Wudao
(Xue et al., 2022), Wanjuan (He et al., 2023), and SkyPile (Wei et al., 2023).
This evaluation, focusing on clarity, feasibility, practicality, and
alignment, ensures the high quality of our dataset.

The key contributions of our work are:

  * •

Algorithmic Advancement: AP in Kun enhances data retention and resolves
ambiguities, leading to an expanded pool of high-quality data for fine-tuning.

  * •

Data Creation: Over a million diverse Chinese instructional data points are
produced from sources like Wudao, Wanjuan, and SkyPile, surpassing traditional
crowdsourced annotations in quality and reducing reliance on manual
annotation.

##  2 Related Work

Instruction Tuning

Instruction tuning is widely recognized as a key technique for activating LLMs
to adhere to human conversational norms. (Mishra et al., 2022; Wang et al.,
2022b, 2023b). Instruction tuning empowers various domain-specific or task-
specific LLMs, including natural language generation evaluation (Jiang et al.,
2023), math (Yue et al., 2023; Xu et al., 2023a; Azerbayev et al., 2023), code
Luo et al. (2023), music Li et al. (2023a); Deng et al. (2023), and medicine
Wang et al. (2023a). Instruction tuning not only tailors the models’ task-
specific responsiveness but also bolsters their cross-task generalization
capabilities, thus enhancing performance across various dynamic application
contexts (Wei et al., 2021; Sanh et al., 2022; Mishra et al., 2022; Wang et
al., 2022b). Recent studies have broadened the scope of instruction tuning to
encompass a wider array of general tasks, notably incorporating input from
users of language models (Ouyang et al., 2022; Peng et al., 2023).

However, the open-source community is still lacking high-quality Chinese
instruction tuning corpora. Current datasets, like Firefly (Yang, 2023), COIG
(Zhang et al., 2023), BELLE (Ji et al., 2023), MOSS (Sun et al., 2023), and
OL-CC (OL-CC, 2023), face issues such as limited scope, poor quality,
commercial restrictions, or insufficient coverage. This gap hampers the
advancement of LLMs in effectively processing and executing complex Chinese
instructions, highlighting the critical need for more diverse and superior-
quality datasets.

Self-Improvement of LLMs In fine-tuning LLMs, the availability of extensive,
high-quality instructional data is crucial. Presently, the generation of such
data mainly relies on human manual annotation, a labor-intensive method that
lacks scalability for future data augmentation. An alternative approach
involves deriving instructional data from more advanced LLMs (Taori et al.,
2023b), exemplified by extracting authentic instruction-response sets from
dialogues within the GPT series models (Wang et al., 2022a). A more refined
technique utilizes the Self-Instruction framework, autonomously generating
additional instructional data from initial seed data. Combined with the Ada-
Instruct or the Evol-Instruct framework, this approach can transform basic
instructions into complex ones, specifically tailored for distinct tasks (Cui
and Wang, 2023; Luo et al., 2023).

Nevertheless, these instruction generation methodologies invariably require a
robust teacher, whether a human annotator or a superior LLM. The ultimate
potential of the model is limited by the teacher’s expertise or resource
expenditure (Li et al., 2023c). To overcome this limitation, The SPIN (Chen et
al., 2024) framework incorporates a self-play mechanism, enabling the Language
Model to enhance its capabilities by engaging in battles against its
instances. It generates training data from previous iterations, refining its
strategy by distinguishing between responses generated autonomously and those
derived from human-annotated data. This gradual process elevates the LLM from
a nascent model to a robust one. Considering the vast amount of knowledge
present in web text, Humpback introduces a technique based on Instruction
Backtranslation. This method allows a base model to independently utilize vast
amounts of unlabeled data to generate a high-quality instruction tuning
dataset. However, empirical findings indicate that the effectiveness of this
method is still constrained by the seed model’s performance and its ability to
discern truly high-quality data.

Figure 2: Instructions spanning over 20 years, with 56% from the last five
years.

##  3 Method

Our training approach requires a basic model, high-quality seed data, and a
large amount of primary data. This primary data mainly includes a large volume
of unlabeled data, unlike labeled data, which comes with predefined
instructions and is ready for training. The main source of this data is web
corpora, which contain various documents on human-interest topics, written by
individuals and are crucial for several reasons:

  * •

Content Diversity: They encompass various subjects, mirroring the extensive
scope of human interests.

  * •

Absence of Predefined Outputs: Unlike labeled data, these documents lack
predetermined outcomes or objectives.

  * •

Potential for High-Quality Data: Certain subsets of these documents may be
particularly suitable for generating text based on specific user instructions.

With these attributes, we can efficiently extract and use high-quality data
from these rich corpora, adding specific instructions. Our innovative method
improves language model training through a new data augmentation technique.
Based on self-alignment principles, it involves carefully choosing and
refining instructional data to fine-tune language models, as shown in Figure
1. This method allows us to gather a large amount of quality labeled data at a
low cost, avoiding the expensive and time-consuming process of manual
labeling.

Our method consists of two main steps:

  * •

Supervised Fine-Tuning (SFT) with High-Quality Seed Data: This involves using
SFT on the base model with high-quality seed data to create two models - the
label model for annotating primary data and the primary chat for improving
data quality.

  * •

Quality Assessment and Refinement in Primary Chat: The primary chat assesses
and refines the label model’s output. This repeated process produces a lot of
high-quality data, essential for the primary chat’s further training. It leads
to a high-performance final chat model, trained extensively with superior
data.

A more comprehensive explanation of each step is provided subsequently.

Figure 3: Length distribution of instructions and outputs based on Yi-6B model
Figure 4: The top 10 categories in each of these three areas:Academic
Disciplines, Industry Sectors, Text Type

###  3.1 Initialization

Seed data. We use 26,000 carefully chosen pairs of instructions and their
corresponding outputs as our seed data. Each pair is hand-annotated and
undergoes strict quality control to ensure high accuracy and consistency.

Primary Data. The primary data originates from three significant Chinese
datasets: WuDao, Wanjuan, and SkyPile. These datasets are distinguished by
their extensive scale, diverse content, and comprehensive nature. Such
characteristics make them ideal for mining golden texts aligned with specific
user instructions. To facilitate this, we preprocess these datasets to isolate
self-contained segments, denoted as ${y}_{i}$.

Supervised Fine-Tuning. We utilize high-quality seed data to execute SFT on
the foundational model, yielding two distinct models: the label model and the
primary chat model.

  * •

Primary Chat Model: This model is fine-tuned utilizing the (instruction-
output) pairs (${x}_{i}$, ${y}_{i}$) from the seed data. This process creates
a forward model, $M_{xy}:=p(y\mid x)$, with $x$ and $y$ maintaining their
respective meanings as instructions and outputs.

  * •

Label Model: Conversely, this model undergoes fine-tuning with the (output-
instruction) pairs (${y}_{i}$, ${x}_{i}$) derived from the seed data, leading
to the formation of a backward model, denoted as $M_{yx}:=p(x\mid y)$. In this
context, $x$ signifies the instruction, while $y$ denotes the corresponding
output.

###  3.2 Generating Candidate Instructions with the Label Model

For each example ${y}_{i}$ in the unlabeled set, we utilize the backward model
to infer a candidate instruction, denoted as $\hat{x}_{i}$. This procedure
generates a collection of potential enhanced pairing data, represented as
($\hat{x}_{i}$, ${y}_{i}$). During annotation, initial filtering is based on
perplexity (ppl) and length, excluding any data exceeding 512 tokens. We also
discard data unsuitable for instructional use, like purely descriptive
statements, while retaining useful data, such as commands and questions. We
apply a specific prompt in this selection, keeping only data that triggers a
positive response. The final labeled dataset contains instruction and output
components, with instructions from the label model and outputs from primary
data, known as candidate labeled data. Figure 5 shows the filter prompt used
in this stage.

Figure 5: Filter prompt we use to screen out unsuitable content for
instructions.

###  3.3 Refinement of Candidate Labeled Data Using the Primary Chat Model

Given the extensive volume of primary data, not all candidate labeled data
derived from it are of uniformly high quality. Our objective is to curate
high-quality labeled data, necessitating rigorous filtration of the candidate
labeled data.

Our experiments tested two different filtering methods, each with its
strengths and weaknesses:

  * •

Comprehensive Scoring of Labeled Data: This method evaluates the full labeled
data set based on a combined score, including instructions and outputs.

  * •

Focused Scoring of Instruction Component: This technique only assesses the
instruction part (output from the label model). High-scoring instructions are
chosen, and then the output part of these selected data is refined."

Our analysis shows that the second method is more effective than the first. In
the first method, good outputs are often discarded because of poor
instructions from the label model, and the reverse is also true, causing
unnecessary exclusions. Moreover, this approach sometimes retains data with
one poor quality component because the other component is high quality,
leading to uneven data quality and negatively impacting further training.

In contrast, the second method only scores the instruction component,
effectively removing irrelevant data and favoring quality instructions, which
improves data retention. Yet, it doesn’t assess the output, sometimes leading
to suitable instructions paired with unsuitable outputs. To address this, we
use the primary chat model to evaluate and refine the instructions and
outputs, ensuring they align well. This approach produces high-quality labeled
data. The score and refine prompts we used in this process are shown in Figure
6.

Figure 6: The top section comprises score prompt used to assess the quality of
labeled data instructions. The bottom section features refine prompt for
refining the output part of the labeled data.

Utilizing the substantial volume of top-quality labeled data from these
procedures, we further train the main chat model, achieving a high-performance
final model, as shown in Experiments.

Source | Instruction Quality | Output Quality
---|---|---
Clarity% | Feasibility% | Practicality% | Excellent% | Pass% | Fail%
Wudao | 96.67 | 96.40 | 96.87 | 69.50 | 20.03 | 10.47
Wanjuan | 98.27 | 97.63 | 96.57 | 85.63 | 11.13 | 3.24
Skypile | 98.90 | 98.37 | 95.40 | 42.73 | 40.43 | 16.84
ALL | 97.94 | 97.47 | 96.28 | 66.00 | 23.87 | 10.13

Table 1: Manual Quality Analysis of Synthetic Data Generated by Kun.

##  4 Experiments

In this section, we comprehensively detail the experimental procedures and
methodologies employed in our study.

###  4.1 Experimental Setup

We first detail the experimental setup used in our study, covering the base
model selection, fine-tuning process, baseline comparisons, and the evaluation
methods.

####  4.1.1 Base Model & Finetuning

In our experiments, we utilize the Yi model with 6B
parameters††https://huggingface.co/01-ai/Yi-6B, developed by 01.AI, as our
foundational language model for fine-tuning. Renowned for its proficiency in
both English and Chinese, the Yi series has shown impressive results on global
benchmarks like the AlpacaEval Leaderboard (Dubois et al., 2023; Li et al.,
2023d) and SuperCLUE (Xu et al., 2023b).

The fine-tuning process is carried out using varying sizes of our high-
quality, instructionally curated dataset. This phase is executed on a
computing setup with 32 Nvidia A800 GPUs, amounting to a total of 192 GPU
hours. We adopt a learning rate of 2e-5 and a batch size of 16, aiming for an
optimal balance between computational efficiency and model performance. The
generation temperature is set to 0 to ensure precise output generation.

####  4.1.2 Baselines

For the Kun dataset, we annotated command data from three sources: Wudao,
Wanjuan, and Skypile. This process has enriched the dataset, now made open-
source and available for community
use††https://huggingface.co/datasets/m-a-p/COIG-Kun. Quantitative details of
this augmented dataset are provided in Figure 7.In evaluating the performance
of Kun, our study contrasts it with data curated from four prominent Chinese
open-sourced datasets. These datasets are Firefly (Yang, 2023), COIG (Zhang et
al., 2023; BAAI, 2023a, b), OL-CC (OL-CC, 2023), and BELLE (Ji et al., 2023),
each unique in its composition and focus, providing a comprehensive basis for
comparison.

####  4.1.3 Evaluation

Human Evaluation. To assess the general quality of model responses, we conduct
human evaluations using a test set of 500 prompts sampled from ShareGPT-zh.
These prompts, derived from actual user inquiries, encompass a diverse array
of tasks, such as creative writing, information seeking, providing guidance,
logical reasoning, storytelling, problem-solving, etc.

For the evaluation, responses generated by different models for each prompt
are presented side-by-side. Human evaluators are asked to choose their
preferred answer, providing a direct comparison of model performance. In
total, eight models were compared, including three variants of the Kun model
(fine-tuned with different sizes of our curated dataset), four baseline models
(fine-tuned with public datasets), and one mixed model (fine-tuned with a
combination of Kun and baseline datasets). This resulted in 28 pairwise
comparisons across all test prompts. For this evaluation, we engage a team of
experienced crowdsource annotators, ensuring a balanced and unbiased
assessment. Detailed examples that show the comparison process can be found in
Figure 12.

Standard Benchmarks. In addition to human evaluations, the models are also
assessed using two standard benchmarks for Chinese LLM evaluation: C-EVAL
(Huang et al., 2023) and CMMLU (Li et al., 2023b).

These evaluation methods, comprising both human judgment and standardized
benchmarks, offer a multifaceted perspective on the capabilities of the Kun
model, enabling a thorough comparison with existing models and datasets.

Figure 7: Distribution of Data Sources

Figure 8: Human preference evaluation, comparing _Kun-52k_ with other models.

Figure 9: Human preference evaluation, comparing the mix model with other
models.

###  4.2 Augmentation Data Statistics

In this section, we delve into the detailed statistical analysis and diversity
assessment of our augmented dataset, as well as the rigorous quality
evaluation conducted using human annotators. By exploring the comprehensive
scale, varied nature, and assessed quality of the instruction-output pairs, we
aim to highlight the robustness and reliability of the data curated for our
study.

####  4.2.1 Statistics and Diversity

Our work involve the purification of approximately 370,000 high-quality
instruction-output pairs from the Wudao, Wanjuan, and Skypile datasets. This
substantial dataset provide a diverse and rich foundation for our study.

We analyze a 10% subset of instructions from the past 20 years, revealing
significant temporal diversity with 56% of instructions from the recent three
years (Figure 2). The variation in instruction and output lengths, analyzed
using the Yi-6B model, is shown in Figure 3, reflecting content complexity.

To assess instruction diversity, we categorize them into 24 academic
disciplines, 16 industry sectors, and 15 text types as per
Wikipedia††https://www.wikipedia.org using the
Qwen-72B-Chat††https://huggingface.co/Qwen/Qwen-72B-Chat (Bai et al., 2023).
Repeated for accuracy, this categorization highlights the data’s range, as
shown in Figure 4, where the top 10 categories in each area signify its broad
scope.

####  4.2.2 Quality Evaluation

A critical aspect of our dataset curation process is the rigorous data quality
assessment. We conduct a comprehensive quality evaluation of the instruction-
output pairs to achieve this. For augmented data curated from each source
(Wudao, Wanjuan, and Skypile), we randomly select 1,000 instruction-output
pairs, resulting in 3,000 pairs subjected to an independent quality
assessment.

Instruction Quality. For instruction quality, a team of 24 professional
annotators with a bachelor’s degree or higher evaluates each instruction
across three key dimensions: clarity, feasibility, and practicality. Each
aspect is assessed with a simple yes/no answer, providing a straightforward
yet effective measure of instruction quality. The evaluation criteria are as
follows:

  * •

Clarity: Evaluators determine whether the instruction was unambiguous and
coherent, encompassing necessary information without any vague terms or
explanations.

  * •

Feasibility: Evaluators assess whether the instruction was valid and
answerable within the context and scope of the model’s capabilities.

  * •

Practicality: Evaluators judge the relevance of the instruction in everyday
scenarios.

Output Quality. The quality of the outputs is evaluated based on their
alignment with the instructions. Evaluators are asked to rate each output as
_Excellent_ , _Pass_ , or _Fail_ , based on how well it met the requirements
and intent of the instruction.

To ensure objectivity and reliability, three different evaluators evaluate
each instruction-output pair. We conducted a detailed comparison of the
options assessed by different evaluators, and the results demonstrate a
significant degree of consistency in their judgments. Further details on
evaluating identical ratings are presented in Figure 10 and Table 2. Examples
that demonstrate the process of assessing can be found in Figure 12

Figure 10: Distribution of evaluation identical ratings for instruction-
response from three evaluators.

###  4.3 Result Analysis

Source | Consistency%
---|---
Clarity | 96.87
Feasibility | 97.73
Practicality | 97.43
ALL | 92.13

Table 2: The proportion of identical evaluations from three assessors on a
single dimension. All: The proportion of consistent assessments across all
three dimensions within the same item.

As indicated in Table 1, the instruction quality across all sources is
consistently high, suggesting effective formulation and clarity in conveying
their purposes. However, the output quality varies more noticeably among the
sources. While some sources like Wanjuan exhibite a high percentage of
"Excellent" outputs, others such as Skypile demonstrate a more diverse
distribution of output quality.

This section presents the analysis of our experiment results, encompassing
human evaluation outcomes and performance on standard benchmarks.

####  4.3.1 Human Evaluation

Model | C-EVAL | CMMLU
---|---|---
Perplexity (ppl) | Generation (gen) | Perplexity (ppl) | Generation (gen)
Kun-9k | 72.95 | 72.96 | 73.53 | 24.53
Kun-26k | 73.73 | 73.26 | 75.75 | 25.23
Kun-52k | 73.36 | 72.72 | 75.48 | 25.50
Kun-100k | 72.95 | 72.96 | 73.53 | 24.53
Kun-200k | 73.36 | 72.07 | 74.81 | 31.81
Kun-360k | 73.25 | 71.88 | 74.14 | 46.58
Kun-52k + COIG-one-38k | 73.61 | 73.38 | 75.70 | 34.84
COIG-38k | 74.09 | 72.95 | 75.92 | 38.40
Belle-52k | 72.31 | 72.56 | 74.73 | 44.56
Firefly-38k | 73.71 | 73.63 | 76.29 | 41.73
OL-CC-10k | 72.00 | 71.62 | 74.97 | 34.38

Table 3: Performance Statistics on Standard Benchmarks.The best results in
each section are Bold , the second-best results are underlined, while the
results of our best model are in Blue.

Our human evaluation results are illustrated through the heatmap in Figure 11,
which shows the model vs. model win rates, with color variations indicating
the relative performance strengths. The heatmap highlights that the _Kun-360k_
model emerges as the most dominant, followed by the mixed model, showcasing
their superior ability to handle a wide range of prompts. In contrast, the
baseline models _Firefly-52k_ , _COIG-38k_ , and _Belle-52k_ , garner lower
preference percentages. This suggests that despite their strengths, these
models may not align as closely with user expectations or prompt requirements
as the Kun models.

Further analysis is provided in Figures 9 and 9, which depict the human
preference evaluation win rates of _Kun-52k_ and the mixed model,
respectively, when compared with other models. Notably, the mixed model
demonstrates a higher win rate than _Kun-52k_ , indicating its enhanced
effectiveness in meeting human evaluators’ preferences.

Figure 11: Heatmap of Model Comparative Win Rates in Human Evaluations

####  4.3.2 Standard Benchmarks

The performance of the models on standard benchmarks, specifically C-EVAL and
CMMLU, provides an objective measure of their capabilities in understanding
and reasoning within the Chinese context. These benchmarks assess the model’s
ppl and generation quality (gen), key indicators of their language
comprehension and output proficiency. Table 3 presents the performance
statistics of various models on the C-EVAL and CMMLU benchmarks. As shown, we
evaluate numerous models, including different sizes of the Kun model, baseline
models, and mixed models. Each model’s performance is measured in terms of
perplexity and generation quality, providing a comprehensive view of its
strengths and weaknesses.

From the table, we observe that _Kun-360k_ generally exhibits lower perplexity
and higher generation quality, confirming its top-tier performance in language
understanding and generation. Interestingly, the mixed model and _Firefly_
display robust performance, with the mixed model often outperforming
_Kun-52k_. The baseline models and smaller Kun variants present mixed results,
excelling in some metrics while falling short in others. These highlight
potential areas for further improvement in model training and fine-tuning
strategies.

##  5 Conclusion

Our approach represents a breakthrough in instruction-tuning for LLMs,
utilizing a novel self-training algorithm to leverage over a million quality
Chinese instructional data points from diverse sources effectively. This
strategy, different from manual annotations, not only enhances the
instruction-following capabilities of LLMs but also ensures the high quality
and diversity of training data. Empirical evaluations using the 6B-parameter
Yi model across benchmarks like C-EVAL, CMMLU, and human evaluations, have
demonstrated its robustness and scalability, particularly of the Kun-52k
variant. Innovations within our approach, such as AP, have notably improved
data retention and clarity, offering a scalable, efficient way to augment
LLMs’ instructional capabilities. This research not only progresses the field
but also broadens LLMs’ application scope, offering a novel solution to the
challenges in developing instruction-tuning datasets.

## References

  * Azerbayev et al. (2023) Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. 2023.  Llemma: An open language model for mathematics.  _arXiv preprint arXiv:2310.10631_.
  * BAAI (2023a) BAAI. 2023a.  Coig-pc.
  * BAAI (2023b) BAAI. 2023b.  Coig-pc-lite.
  * Bai et al. (2023) Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023.  Qwen technical report.  _arXiv preprint arXiv:2309.16609_.
  * Chen et al. (2024) Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. 2024.  Self-play fine-tuning converts weak language models to strong language models.  _arXiv preprint arXiv:2401.01335_.
  * Cui and Wang (2023) Wanyun Cui and Qianle Wang. 2023.  Ada-instruct: Adapting instruction generators for complex reasoning.  _arXiv preprint arXiv:2310.04484_.
  * Deng et al. (2023) Zihao Deng, Yinghao Ma, Yudong Liu, Rongchen Guo, Ge Zhang, Wenhu Chen, Wenhao Huang, and Emmanouil Benetos. 2023.  Musilingo: Bridging music and text with pre-trained language models for music captioning and query response.  _arXiv preprint arXiv:2309.08730_.
  * Dubois et al. (2023) Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023.  Alpacafarm: A simulation framework for methods that learn from human feedback.
  * He et al. (2023) Conghui He, Zhenjiang Jin, Chao Xu, Jiantao Qiu, Bin Wang, Wei Li, Hang Yan, Jiaqi Wang, and Dahua Lin. 2023.  Wanjuan: A comprehensive multimodal dataset for advancing english and chinese large models.
  * Huang et al. (2023) Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. 2023.  C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models.  _CoRR_ , abs/2305.08322.
  * Ji et al. (2023) Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, and Xiangang Li. 2023.  Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases.
  * Jiang et al. (2023) Dongfu Jiang, Yishan Li, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, and Wenhu Chen. 2023.  Tigerscore: Towards building explainable metric for all text generation tasks.  _arXiv preprint arXiv:2310.00752_.
  * Li et al. (2023a) Dichucheng Li, Yinghao Ma, Weixing Wei, Qiuqiang Kong, Yulun Wu, Mingjin Che, Fan Xia, Emmanouil Benetos, and Wei Li. 2023a.  Mertech: Instrument playing technique detection using self-supervised pretrained model with multi-task finetuning.  _arXiv preprint arXiv:2310.09853_.
  * Li et al. (2023b) Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. 2023b.  CMMLU: measuring massive multitask language understanding in chinese.  _CoRR_ , abs/2306.09212.
  * Li et al. (2023c) Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, and Mike Lewis. 2023c.  Self-alignment with instruction backtranslation.  _arXiv preprint arXiv:2308.06259_.
  * Li et al. (2023d) Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023d.  Alpacaeval: An automatic evaluator of instruction-following models.  https://github.com/tatsu-lab/alpaca_eval.
  * Luo et al. (2023) Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023.  Wizardcoder: Empowering code large language models with evol-instruct.  _arXiv preprint arXiv:2306.08568_.
  * Mishra et al. (2022) Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022.  Cross-task generalization via natural language crowdsourcing instructions.  In _ACL_.
  * OL-CC (2023) OL-CC. 2023.  Openlabel-chinese conversations dataset (ol-cc).
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022.  Training language models to follow instructions with human feedback.  _Advances in Neural Information Processing Systems_ , 35:27730–27744.
  * Peng et al. (2023) Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023.  Instruction tuning with gpt-4.  _arXiv preprint arXiv:2304.03277_.
  * Sanh et al. (2022) Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022.  Multitask prompted training enables zero-shot task generalization.  In _International Conference on Learning Representations_.
  * Sun et al. (2023) Tianxiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Hang Yan, Xiangyang Liu, Yunfan Shao, Qiong Tang, Xingjian Zhao, Ke Chen, Yining Zheng, Zhejian Zhou, Ruixiao Li, Jun Zhan, Yunhua Zhou, Linyang Li, Xiaogui Yang, Lingling Wu, Zhangyue Yin, Xuanjing Huang, and Xipeng Qiu. 2023.  Moss: Training conversational language models from synthetic data.
  * Taori et al. (2023a) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023a.  Alpaca: A strong, replicable instruction-following model.  _Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03/13/alpaca. html_ , 3(6):7.
  * Taori et al. (2023b) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023b.  Stanford alpaca: An instruction-following llama model.  https://github.com/tatsu-lab/stanford_alpaca.
  * Wang et al. (2023a) Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. 2023a.  Huatuo: Tuning llama model with chinese medical knowledge.  _arXiv preprint arXiv:2304.06975_.
  * Wang et al. (2022a) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022a.  Self-instruct: Aligning language model with self generated instructions.
  * Wang et al. (2022b) Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022b.  Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks.  In _EMNLP_.
  * Wang et al. (2023b) Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, et al. 2023b.  Interactive natural language processing.  _arXiv preprint arXiv:2305.13246_.
  * Wei et al. (2021) Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021.  Finetuned language models are zero-shot learners.  In _International Conference on Learning Representations_.
  * Wei et al. (2023) Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Haihua Yang, Biye Li, Cheng Cheng, Weiwei Lü, Rui Hu, Chenxia Li, Liu Yang, Xilin Luo, Xuejie Wu, Lunan Liu, Wenjun Cheng, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Lei Lin, Xiaokun Wang, Yutuan Ma, Chuanhai Dong, Yanqi Sun, Yifu Chen, Yongyi Peng, Xiaojuan Liang, Shuicheng Yan, Han Fang, and Yahui Zhou. 2023.  Skywork: A more open bilingual foundation model.
  * Xu et al. (2023a) Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023a.  Wizardlm: Empowering large language models to follow complex instructions.  _arXiv preprint arXiv:2304.12244_.
  * Xu et al. (2023b) Liang Xu, Anqi Li, Lei Zhu, Hang Xue, Changtai Zhu, Kangkang Zhao, Haonan He, Xuanwei Zhang, Qiyue Kang, and Zhenzhong Lan. 2023b.  Superclue: A comprehensive chinese large language model benchmark.  _arXiv preprint arXiv:2307.15020_.
  * Xue et al. (2022) Zhao Xue, Hanyu Zhao, Sha Yuan, and Yequan Wang. 2022.  WuDaoCorpora Text.
  * Yang (2023) Jianxin Yang. 2023.  Firefly: Chinese conversational large language model.  https://github.com/yangjianxin1/Firefly.
  * Yue et al. (2023) Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2023.  Mammoth: Building math generalist models through hybrid instruction tuning.  _arXiv preprint arXiv:2309.05653_.
  * Zhang et al. (2023) Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, and Jie Fu. 2023.  Chinese open instruction generalist: A preliminary release.
  * Zheng et al. (2023) Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023.  Judging llm-as-a-judge with mt-bench and chatbot arena.  _arXiv preprint arXiv:2306.05685_.

##  Appendix A Example Appendix

This is a section in the appendix.

###  A.1 Examples in Human Evaluation of Data and Model Performance

Figure 12 illustrates specific examples from our human evaluation process, as
detailed in Sections 4.2.2 and 4.1.3. This figure includes the evaluation of
both instruction and output quality, along with a comparison of model answers.
For each example, the original question is provided, followed by the evaluated
responses.

The instruction quality is assessed for clarity, feasibility, and practicality
to evaluate the precision in formulating instructions. The output quality
assessment focuses on the extent to which each response meets the
instruction’s requirements. Additionally, the model comparison examines which
answers align most closely with human preferences, highlighting the practical
effectiveness of the models under consideration.

Figure 12: Examples in Human Evaluation of Data and Model Performance.


# Q-GroundCAM: Quantifying Grounding in Vision Language Models
via GradCAM

Navid Rajabi and Jana Košecká
George Mason University
{nrajabi<EMAIL_ADDRESS>
###### Abstract

Vision and Language Models (VLMs) continue to demonstrate remarkable zero-shot
(ZS) performance across various tasks. However, many probing studies have
revealed that even the best-performing VLMs struggle to capture aspects of
compositional scene understanding, lacking the ability to properly ground and
localize linguistic phrases in images. Recent VLM advancements include scaling
up both model and dataset sizes, additional training objectives and levels of
supervision, and variations in the model architectures. To characterize the
grounding ability of VLMs, such as phrase grounding, referring expressions
comprehension, and relationship understanding, Pointing Game has been used as
an evaluation metric for datasets with bounding box annotations. In this
paper, we introduce a novel suite of quantitative metrics that utilize GradCAM
activations to rigorously evaluate the grounding capabilities of pre-trained
VLMs like CLIP, BLIP, and ALBEF. These metrics offer an explainable and
quantifiable approach for a more detailed comparison of the zero-shot
capabilities of VLMs and enable measuring models’ grounding uncertainty. This
characterization reveals interesting tradeoffs between the size of the model,
the dataset size, and their performance.

Figure 1: Uncertainty in Pointing Game (PG) accuracy, when having multiple
top-k identical activations with inconsistent PG binary labels (Scenario 1).
As depicted in the bottom-right figure, three top high-confidence activations
exist, each with a value of 1.0, after our NMS analysis. One falls outside the
bounding box, one inside, and one at the border. In these cases, PG lacks any
additional clues or heuristics to determine which one to select. Figure 2:
Given the ”his daughter” prompt, PG returns the same accuracy of 1 for all
four model outputs in a discrete manner, and overlooks the differences in
their holistic grounding qualities (Scenario 2). On the other hand, our
$\mathbf{IO}_{ratio}$ metric can differentiate and rank them in a more
explainable and continuous manner by quantifying them each as a single
normalized value between 0 and 1.

##  1 Introduction

Foundational Vision and Language Models (VLMs) have demonstrated impressive
performance in various vision and language tasks, including visual question
answering (VQA), retrieval, image-text matching, referring expression
comprehension, or captioning. For instance, CLIP has become a widely-used
backbone in various applications, ranging from open-vocabulary object
localization & referring expression grounding [3, 27] , to in-context learning
in generative Multimodal Large Language Models (MLLMs) such as LLaVa [18], and
Embodied AI for navigation-related tasks [10]. Despite that, state-of-the-art
VLMs still struggle to capture aspects of compositional scene understanding
and lack proper grounding for noun phrases, verbs, or relations [9, 33, 28].
Earlier model architectures localize objects of interest by predicting
bounding box locations and use traditional IoU evaluation metrics with respect
to ground truth datasets [8, 11, 6]. Recent MLLMs forgo the bounding box
prediction and tackle the location prediction on the language side [18, 20].
The ability to ground and localize the concepts continues to be of central
interest [31], and GradCAM visualizations [25] or Pointing Game accuracy [34]
are often used to quantify models’ grounding ability. Pointing Game (PG)
considers grounding successful if the highest GradCAM activation falls inside
the ground truth bounding box [7]. While Pointing Game has been utilized
effectively in characterization of grounding performance [4, 1, 7, 2, 29, 31],
it offers only coarse 0/1 characterization, which is susceptible to spurious
local maxima or existence of multiple maxima, and does not capture well the
confidence in the grounded concept. The examples in Figures 1 & 2 would be
incorrectly handled by the Pointing Game approach due to the following
scenarios:

  * •

Scenario 1: Highly confident activations, some in and some outside of the
bounding box, are predicted by the model as in Figure 1. In these cases, the
Pointing Game must randomly pick the local maximum. We refer to this as
$\mathrm{PG}_{Uncertainty}$ metric and report the total number of cases where
this type of uncertainty occurs for each dataset in Table 1.

  * •

Scenario 2: Pointing Game does not consider the spurious activation
predictions outside the ground-truth bounding box when comparing the
performance of different models on the same instance. As shown in Figure 2,
while PG returns the label of 1 for all four models, our $\mathrm{IO}_{ratio}$
metric provides a finer-grained characterization of grounding quality.

In order to provide a finer-grained characterization of the model’s grounding
ability, we propose a set of metrics to (1) compute the similarity between the
$\mathrm{GradCAM}$ activation maps and ground-truth binary mask, (2) reward
the activations inside while penalizing the spurious activations outside of
the ground-truth bounding box, and (3) measure the Scenario 1 uncertainty. Our
inspiration stems from the Attention Mask Consistency loss (AMC) introduced in
[31] which is designed to maximize the activations inside and minimize them
outside the ground-truth bounding box, used in the pre-training of ALBEF [15].
Since our metrics are applicable to a wide range of VLMs, including both
patch-based ViT [5] and CNN-based vision transformer encoders [23, 3], they
can thoroughly assess the grounding ability of VLMs, even without the bounding
box prediction mechanism. The proposed metrics can distinguish nuances in the
comparison of models’ grounding abilities that PG cannot capture. We will use
these metrics to evaluate four state-of-the-art VLMs ($\mathrm{BLIP}_{base}$
[16], $\mathrm{BLIP}_{large}$ [16], $\mathrm{CLIP}$ $\mathrm{gScoreCAM}$ [23,
3], and the $\mathrm{AMC}$ [31] variation of $\mathrm{ALBEF}$ [15]) on a wide
spectrum of grounding tasks111We release the code at
Github.com/NavidRajabi/Q-GroundCAM..

Dataset | Model |  $\mathbf{IoU}$ $\mathbf{\uparrow}$ |  $\mathbf{Dice}$ $\mathbf{\uparrow}$ |  $\mathbf{WDP}$ $\mathbf{\downarrow}$ |  $\mathbf{IO}_{ratio}$ $\uparrow$ | $\mathbf{PG}$
---|---|---|---|---|---|---
Soft | Binary | Soft | Binary | Soft | Binary | LogSig | Accuracy $\uparrow$ | Uncertainty $\downarrow$
Flickr30K Entities |  BLIPbase | 0.12 | 0.09 | 0.21 | 0.15 | 0.95 | 0.79 | 0.43 | 60.08 | 317 / 14481
(test) |  BLIPlarge | 0.14 | 0.11 | 0.23 | 0.17 | 0.94 | 0.77 | 0.44 | 71.43 | 965 / 14481
|  CLIPgScoreCAM | 0.21 | 0.23 | 0.33 | 0.34 | 0.92 | 0.56 | 0.44 | 75.41 |  0 / 14481
|  ALBEFAMC | 0.19 | 0.16 | 0.31 | 0.25 | 0.86 | 0.60 | 0.62 | 87.69 |  10 / 14481
RefCOCO+ |  BLIPbase | 0.10 | 0.06 | 0.18 | 0.10 | 0.98 | 0.75 | 0.39 | 67.90 | 198 / 5726
(testA) |  BLIPlarge | 0.10 | 0.06 | 0.18 | 0.10 | 0.98 | 0.76 | 0.38 | 67.27 | 336 / 5726
|  CLIPgScoreCAM | 0.17 | 0.15 | 0.28 | 0.24 | 0.98 | 0.68 | 0.34 | 63.36 |  0 / 5726
|  ALBEFAMC | 0.14 | 0.09 | 0.24 | 0.15 | 0.96 | 0.66 | 0.54 | 78.81 |  14 / 5726
RefCOCO+ |  BLIPbase | 0.11 | 0.06 | 0.20 | 0.11 | 0.98 | 0.86 | 0.31 | 46.96 | 201 / 4889
(testB) |  BLIPlarge | 0.11 | 0.06 | 0.20 | 0.11 | 0.98 | 0.87 | 0.30 | 46.71 | 449 / 4889
|  CLIPgScoreCAM | 0.18 | 0.19 | 0.29 | 0.29 | 0.97 | 0.85 | 0.30 | 49.02 |  0 / 4889
|  ALBEFAMC | 0.16 | 0.11 | 0.27 | 0.19 | 0.96 | 0.76 | 0.45 | 64.34 |  16 / 4889
SpatialSense |  BLIPbase | 0.11 | 0.12 | 0.20 | 0.18 | 0.97 | 0.75 | 0.29 | 46.99 | 50 / 1811
(triplets) |  BLIPlarge | 0.12 | 0.12 | 0.21 | 0.19 | 0.97 | 0.73 | 0.31 | 51.68 | 100 / 1811
|  CLIPgScoreCAM | 0.16 | 0.20 | 0.26 | 0.30 | 0.95 | 0.81 | 0.28 | 50.57 |  0 / 1811
|  ALBEFAMC | 0.17 | 0.16 | 0.27 | 0.26 | 0.93 | 0.71 | 0.42 | 67.64 |  3 / 1811
SpatialSense |  BLIPbase | 0.10 | 0.08 | 0.17 | 0.14 | 0.97 | 0.76 | 0.29 | 39.81 | 42 / 1811
(subjects) |  BLIPlarge | 0.11 | 0.11 | 0.20 | 0.17 | 0.97 | 0.74 | 0.30 | 49.75 | 116 / 1811
|  CLIPgScoreCAM | 0.17 | 0.23 | 0.28 | 0.34 | 0.95 | 0.69 | 0.31 | 60.07 |  0 / 1811
|  ALBEFAMC | 0.16 | 0.15 | 0.27 | 0.24 | 0.89 | 0.70 | 0.52 | 70.01 |  4 / 1811
SpatialSense |  BLIPbase | 0.11 | 0.07 | 0.19 | 0.12 | 0.92 | 0.69 | 0.42 | 50.02 | 37 / 1811
(objects) |  BLIPlarge | 0.12 | 0.08 | 0.21 | 0.13 | 0.92 | 0.74 | 0.40 | 54.55 | 158 / 1811
|  CLIPgScoreCAM | 0.20 | 0.21 | 0.32 | 0.32 | 0.90 | 0.65 | 0.41 | 64.93 |  0 / 1811
|  ALBEFAMC | 0.16 | 0.12 | 0.27 | 0.19 | 0.81 | 0.65 | 0.63 | 78.24 |  3 / 1811
Table 1: Quantitative results comparison for all settings where numbers are
reported as $\mathrm{mean}$ $\mathrm{IoU}$ ($\mathbf{mIoU}$) across each
setting. For each setting, the top performance across all models is
highlighted as bold, and the second-best with underline.

##  2 Method

Consider $\mathrm{GradCAM}$ activation map $A_{i,j}$, obtained by passing an
image and a corresponding text prompt to the model, and $M_{i,j}$ are pixel
locations of the binary ground-truth bounding box mask. For computing
$\mathrm{IoU}_{Soft}$ and $\mathrm{Dice}_{Soft}$, we’ve followed the formulas
used for semantic segmentation, according to [19]. We first apply the
threshold of 0.5 to each pixel value in $A_{i,j}$ and pass the thresholded
binary activation maps to compute the $\mathrm{IoU}_{Binary}$ and
$\mathrm{Dice}_{Binary}$. Distance Maps. Given the bounding box coordinates as
$(y_{0},x_{0},y_{1},x_{1})$, distance maps $D_{i,j}$ is computed as for all
$(i,j)$ where $0\leq i<H,0\leq j<W$:

$D_{i,j}=\max\\{\max(y_{0}-i,i-y_{1}),\max(x_{0}-j,j-x_{1}))\\}$

And the weighted penalties matrix $P_{i,j}$ is computed as:

$P_{i,j}=[A_{i,j}\cdot(1-M_{i,j})\cdot D_{i,j}]$ (1)

Weighted Distance Penalty (WDP) is designed to penalize the spurious
activations outside the ground-truth bounding box, proportional to their
magnitudes and distances from it, computed and normalized as follows:

$\mathrm{WDP}={\tt Sigmoid}\left({\tt
log}\frac{\sum_{i=1}^{H}\sum_{j=1}^{W}P_{i,j}}{\sum_{i=1}^{H}\sum_{j=1}^{W}A_{i,j}+\epsilon}\right)$
(2)

Inside/Outside Activations Ratio ($\mathrm{IO}_{ratio}$) is computed as
follows:

$\displaystyle S_{\mathrm{inside}}$
$\displaystyle=\sum_{i=y_{0}}^{y_{1}-1}\sum_{j=x_{0}}^{x_{1}-1}A_{i,j}$ (3)
$\displaystyle S_{\mathrm{outside}}$
$\displaystyle=\sum_{i=1}^{H}\sum_{j=1}^{W}A_{i,j}-S_{\mathrm{inside}}$ (4)
$\mathrm{IO}_{ratio}={\tt Sigmoid}\left({\tt
log}\frac{S_{\mathrm{inside}}}{S_{\mathrm{outside}}}\right)$ (5)

Pointing Game Uncertainty Analysis: We extract the local maxima ($v$) of the
activation map $A_{i,j}$ higher than the activation threshold ($\tau$) of
$0.7$ as the set of $V=\left\\{v_{1},v_{2},...,v_{n}\right\\}$. $V$ is then
sorted, followed by an additional NMS step of suppressing maxima that are
within Euclidean distance $\delta$ 222$\delta=50$ in our experiments of larger
extremum. $V_{nms}$ and $C_{nms}$, represent remaining activation values and
their corresponding coordinates. For every point in $C_{nms}$, we check
whether it falls inside the ground-truth bounding box and count the number of
cases where top-$k$ equal activations in $V_{nms}$ are not all either inside
or outside of the bounding box.

Figure 3: Histogram of $\mathrm{IoU}_{Soft}$ and $\mathrm{IO}_{ratio}$
distributions for ID vs. OOD. Note that the histograms are more peaked for in-
distribution datasets, as shown in blue on the left, and for better-performing
models, they are shifted to the right. The out-of-distribution experiments for
all models have less peaked, flatter histograms, where shown in orange on the
right. Full visualizations can be found in Appendix 8.

##  3 Experiments

We’ve conducted experiments to evaluate the grounding ability of four state-
of-the-art VLMs ($\mathrm{BLIP}_{base}$, $\mathrm{BLIP}_{large}$,
$\mathrm{CLIP}$ $\mathrm{gScoreCAM}$, and the $\mathrm{AMC}$ variation of
$\mathrm{ALBEF}$) on a wide range of grounding tasks, varied by the text
prompt granularity, and with both in-distribution (ID) and out-of-distribution
(OOD) data. The quantitative results are summarized in Table 1, and sample
score distributions are shown in Figure 3. For phrase grounding and referring
expression comprehension, we used the test split of Flickr30K Entities [21]
and RefCOCO+ testA & testB [12] datasets that are considered from in-domain
distribution for the models. In order to investigate the out-of-distribution
generalizability of these models, we ran the same experiment on the
SpatialSense [30] dataset test split, which has both visual domain shift due
to the inclusion of NYU dataset [26] images with a total of 3,679 unique
objects, including small/long-tail concepts, and the language domain shift by
the annotation of 17,498 triplets including spatial relations. The dataset is
designed for spatial relationship recognition and is annotated in the triplet
{Subject-Relation-Object} format with the ground-truth Object and Subject
bounding boxes. We ran our experiments in three different settings for
Triplet, Subject, and Object grounding. An instance of the SpatialSense
dataset is demonstrated in Figure 4. Experiment details for all settings can
be found in the Appendix 6. Additional qualitative results can be found in
Appendix 7, where another case of Scenario 1 uncertainty is shown in Figure 6,
obtained by $\mathrm{BLIP}_{base}$ in the top-left sub-figure of the first
example.

(a) Triplet GT-bbox

(b) Subject GT-bbox

(c) Object GT-bbox

Figure 4: A sample from SpatialSense NYU bedroom set. We consider the ”wifi
router to the right of television” prompt as Triplet, ”wifi router” as
Subject, and ”television” as Object.

##  4 Discussion

According to Table 1, $\mathrm{ALBEF}_{AMC}$ is the winner, considering the
combination of $\mathrm{PG}_{Accuracy}$, $\mathrm{PG}_{Uncertainty}$, and
$\mathrm{IO}_{ratio}$ metrics. This highlights the importance of fine-tuning
ALBEF bounding box-level supervision, compared to scaling models’ size and
training set size using often noisy image-text pairs. Regarding the similarity
between the activations and ground-truth masks, $\mathrm{CLIP}_{gScoreCAM}$
and $\mathrm{ALBEF}_{AMC}$ are the best and second-best performing models
according to IoU and Dice, while in terms of $\mathrm{WDP}$, it is reversed in
most cases. This suggests that CLIP has more spurious GradCAM activations.
Furthermore, we believe that due to the prevalent noisiness in
$\mathrm{GradCAM}$ activations, the $\mathrm{WDP}_{Soft}$ penalization is much
more strict than $\mathrm{WDP}_{Binary}$ variation, making
$\mathrm{WDP}_{Binary}$ a more practical metric to use.
$\mathrm{PG}_{Uncertainty}$ is zero in $\mathrm{CLIP}_{gScoreCAM}$, which we
believe stems from the difference in the vision backbone and nuances in how
the $\mathrm{GradCAM}$ is being computed in [3]. $\mathrm{ALBEF}_{AMC}$ is the
second-best in $\mathrm{PG}_{Uncertainty}$, marginally. In contrast,
$\mathrm{BLIP}_{large}$ has shown the highest relative
$\mathrm{PG}_{Uncertainty}$ across all the settings. Our $\mathrm{IO}_{ratio}$
metric has a strong positive correlation with $\mathrm{PG}_{Accuracy}$ while
being more strict. This makes it a suitable standalone metric for assessing
the model’s grounding performance, as it considers both inside & outside
activations, in addition to $\mathrm{PG}_{Accuracy}$. Note that
$\mathrm{PG}_{Accuracy}$ is not positively correlated with
$\mathrm{PG}_{Uncertainty}$ in all models. This finding is insightful as it
demonstrates that not always the better performing model in terms of
$\mathrm{PG}_{Accuracy}$ has the lowest $\mathrm{PG}_{Uncertainty}$. Our OOD
experiments also demonstrate the applicability of our metrics to the
evaluation of triplet-based spatial understanding grounding [24, 13], and
shifted visual domains.

Model size vs. training data impact. Apart from the $\mathrm{ALBEF}_{AMC}$
superiority, our experiments show that $\mathrm{BLIP}_{large}$, which has
$\sim$ 446M parameters and pre-trained on 129M image-text pairs, under-
performs $\mathrm{BLIP}_{base}$, which has $\sim$ 223M parameters and pre-
trained on 14M image-text pairs, in terms of the $\mathrm{PG}_{Uncertainty}$
in all 6 dataset splits, and in 2 dataset splits in terms of
$\mathrm{PG}_{Accuracy}$. Considering $\mathrm{CLIP}$, which is pre-trained on
$\sim$ 400M image-text pairs, as the highest end, and both $\mathrm{ALBEF}$ &
$\mathrm{BLIP}_{base}$ as the lowest end of the model & data size spectrum in
our experiments, the performance of $\mathrm{ALBEF}_{AMC}$ again highlights
the effectiveness of finer-grained fine-tuning. We suggest running our
$\mathrm{IO}_{ratio}$, $\mathrm{PG}_{Uncertainty}$, $\mathrm{WDP}_{Binary}$,
and either of $\mathrm{IoU}_{Soft}$ or $\mathrm{Dice}_{Soft}$ metrics, in
addition to $\mathrm{PG}_{Accuracy}$, for a thorough grounding evaluation.

##  5 Conclusion

We first demonstrate two scenarios that the Pointing Game (PG) evaluation
metric fails to handle properly, and introduce a new set of metrics for
evaluation of models’ grounding ability that captures finer-grained
differences between models. According to our experiments,
$\mathrm{ALBEF}_{AMC}$ has shown its superiority quantitatively, compared to
the other three models, and demonstrated better performance qualitatively in
terms of the sharpness of the activations it predicts inside the ground-truth
bounding box, as characterized by $\mathrm{IO}_{ratio}$. The proposed metrics
enable a finer-grained evaluation of grounding ability for phrase grounding
and referring expression comprehension, and how it varies across in-
distribution and out-of-distribution datasets.

## References

  * Akbari et al. [2019] Hassan Akbari, Svebor Karaman, Surabhi Bhargava, Brian Chen, Carl Vondrick, and Shih-Fu Chang.  Multi-level multimodal common semantic space for image-phrase grounding.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 12476–12486, 2019.
  * Arbelle et al. [2021] Assaf Arbelle, Sivan Doveh, Amit Alfassy, Joseph Shtok, Guy Lev, Eli Schwartz, Hilde Kuehne, Hila Barak Levi, Prasanna Sattigeri, Rameswar Panda, et al.  Detector-free weakly supervised grounding by separation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 1801–1812, 2021.
  * Chen et al. [2022] Peijie Chen, Qi Li, Saad Biaz, Trung Bui, and Anh Nguyen.  gscorecam: What objects is clip looking at?  In _Proceedings of the Asian Conference on Computer Vision_ , pages 1959–1975, 2022.
  * Datta et al. [2019] Samyak Datta, Karan Sikka, Anirban Roy, Karuna Ahuja, Devi Parikh, and Ajay Divakaran.  Align2ground: Weakly supervised phrase grounding guided by image-caption alignment.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 2601–2610, 2019.
  * Dosovitskiy et al. [2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Dou et al. [2022] Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, et al.  Coarse-to-fine vision-language pre-training with fusion in the backbone.  _Advances in neural information processing systems_ , 35:32942–32956, 2022.
  * Gupta et al. [2020] Tanmay Gupta, Arash Vahdat, Gal Chechik, Xiaodong Yang, Jan Kautz, and Derek Hoiem.  Contrastive learning for weakly supervised phrase grounding.  In _European Conference on Computer Vision_ , pages 752–768. Springer, 2020.
  * Gupta et al. [2022] Tanmay Gupta, Amita Kamath, Aniruddha Kembhavi, and Derek Hoiem.  Towards general purpose vision systems: An end-to-end task-agnostic vision-language architecture.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 16399–16409, 2022.
  * Hendricks and Nematzadeh [2021] Lisa Anne Hendricks and Aida Nematzadeh.  Probing image-language transformers for verb understanding.  _arXiv preprint arXiv:2106.09141_ , 2021.
  * Huang et al. [2023] Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard.  Visual language maps for robot navigation.  In _2023 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 10608–10615. IEEE, 2023.
  * Kamath et al. [2021] Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion.  Mdetr-modulated detection for end-to-end multi-modal understanding.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 1780–1790, 2021.
  * Kazemzadeh et al. [2014] Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg.  Referitgame: Referring to objects in photographs of natural scenes.  In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , pages 787–798, 2014.
  * Krishna et al. [2017] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al.  Visual genome: Connecting language and vision using crowdsourced dense image annotations.  _International journal of computer vision_ , 123:32–73, 2017.
  * Li et al. [2023] Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Silvio Savarese, and Steven C.H. Hoi.  LAVIS: A one-stop library for language-vision intelligence.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)_ , pages 31–41, Toronto, Canada, 2023. Association for Computational Linguistics.
  * Li et al. [2021] Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi.  Align before fuse: Vision and language representation learning with momentum distillation.  _Advances in neural information processing systems_ , 34:9694–9705, 2021.
  * Li et al. [2022] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi.  Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation.  In _International Conference on Machine Learning_ , pages 12888–12900. PMLR, 2022.
  * Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_ , pages 740–755. Springer, 2014.
  * Liu et al. [2023] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.  Visual instruction tuning.  In _NeurIPS_ , 2023.
  * Monteux [2019] Angelo Monteux.  Metrics for semantic segmentation, 2019.
  * Peng et al. [2023] Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei.  Kosmos-2: Grounding multimodal large language models to the world.  _arXiv preprint arXiv:2306.14824_ , 2023.
  * Plummer et al. [2015] Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik.  Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models.  In _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , 2015.
  * Qu et al. [2022] Mengxue Qu, Yu Wu, Wu Liu, Qiqi Gong, Xiaodan Liang, Olga Russakovsky, Yao Zhao, and Yunchao Wei.  Siri: A simple selective retraining mechanism for transformer-based visual grounding.  In _European Conference on Computer Vision_ , pages 546–562. Springer, 2022.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International Conference on Machine Learning_ , pages 8748–8763. PMLR, 2021.
  * Rajabi and Kosecka [2023] Navid Rajabi and Jana Kosecka.  Towards grounded visual spatial reasoning in multi-modal vision language models.  _arXiv preprint arXiv:2308.09778_ , 2023.
  * Selvaraju et al. [2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _Proceedings of the IEEE international conference on computer vision_ , pages 618–626, 2017.
  * Silberman et al. [2012] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus.  Indoor segmentation and support inference from rgbd images.  In _Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part V 12_ , pages 746–760. Springer, 2012.
  * Subramanian et al. [2022] Sanjay Subramanian, Will Merrill, Trevor Darrell, Matt Gardner, Sameer Singh, and Anna Rohrbach.  Reclip: A strong zero-shot baseline for referring expression comprehension.  _arXiv preprint arXiv:2204.05991_ , 2022.
  * Thrush et al. [2022] Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross.  Winoground: Probing vision and language models for visio-linguistic compositionality.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 5238–5248, 2022.
  * Wang et al. [2021] Liwei Wang, Jing Huang, Yin Li, Kun Xu, Zhengyuan Yang, and Dong Yu.  Improving weakly supervised visual grounding by contrastive knowledge distillation.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 14090–14100, 2021.
  * Yang et al. [2019] Kaiyu Yang, Olga Russakovsky, and Jia Deng.  Spatialsense: An adversarially crowdsourced benchmark for spatial relation recognition.  In _International Conference on Computer Vision (ICCV)_ , 2019.
  * Yang et al. [2023] Ziyan Yang, Kushal Kafle, Franck Dernoncourt, and Vicente Ordonez.  Improving visual grounding by encouraging consistent gradient-based explanations.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 19165–19174, 2023.
  * Young et al. [2014] Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.  _Transactions of the Association for Computational Linguistics_ , 2:67–78, 2014.
  * Yuksekgonul et al. [2022] Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou.  When and why vision-language models behave like bag-of-words models, and what to do about it?  _arXiv preprint arXiv:2210.01936_ , 2022.
  * Zhang et al. [2018] Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff.  Top-down neural attention by excitation backprop.  _International Journal of Computer Vision_ , 126(10):1084–1102, 2018.

Supplementary Material

##  6 Experiments Details

For the phrase grounding experiments, we used the test split of Flickr30K
Entities [21] dataset, specifically, the merged version which has also been
used by MDETR [11] and SiRi [22]. In terms of the pre-trained checkpoints, we
used blip-image-text-matching checkpoint for BLIP (both variations of base and
large), through the LAVIS [14] codebase. For $\mathrm{ALBEF}_{AMC}$, we used
the best-flickr checkpoint released by $\mathrm{AMC}$ [31] where sample
qualitative results are shown in Figure 1 & 2. For all
$\mathrm{CLIP}_{gScoreCAM}$ experiments, we used the ResNet50$\times$16
variation of CLIP with the average pooling of top 300-channel activation maps,
as the best-performing setting reported by [3].

For the referring expressions comprehension experiments, we used the same
checkpoints as phrase grounding ones for both $\mathrm{BLIP}$ experiments, but
used the best-refcoco checkpoint released by [31], where sample qualitative
results are shown in Figure 5 & 6. Also, an additional example of PG
Uncertainty (Scenario 1) is demonstrated in the first example of Figure 6, in
the top-left $\mathrm{GradCAM}$ activations of the first obtained by running
$\mathrm{BLIP}_{base}$. Again, there are two top activations with a value of
1.0 each, while one of them falls inside and the other one outside of the
ground-truth bounding box. In these cases, PG becomes indecisive in picking
the maximum to compute the accuracy, as this part of the evaluation becomes
stochastic and directly affects the evaluation reliability.

For the Out-of-Ditribution (OOD) experiments, we ran the models on the
SpatialSense [30] dataset. We consider the Spatial Sense dataset OOD because
it has a domain shift in both visual side (due to including new images from
Flickr [32] and NYU [26], instead of widely-used pre-training/fine-tuning
datasets like MSCOCO [17] and Visual Genome [13]), and language side, by
including spatial clauses and small/long-tail objects for increasing the
detection/grounding difficulty. To be more specific, we ran this experiment on
the instances with True labels in the test set, with a total number of 1811
instances in three settings. The first setting grounds the entire triplet
{Subject-Relation-Object} and we consider the Subject ground-truth bounding
box as the correct bounding box supervision for the entire triplet, since
according to their convention, the Subject acts as the target and Object as
reference. Also, we have conducted two separate experiments for individual
object grounding for Subjects and Objects using their corresponding ground-
truth bounding boxes each. In addition to confirming the high degree of
difficulty we have seen during the Spatial Sense experiments, in which we have
also shown a sample qualitative example in Figure 7, we’ve noticed that in
some of the instances, grounding the Subjects is more complicated due to the
underlying ambiguities/distractions when grounding the subject as a standalone
phrase. This happens because of the dependency on the relation and object as
the context required for disambiguation, which is shown in Figure 8 as an
example. Therefore, the triplet and object grounding settings results seem
more reliable in general, but since we are comparing different models on
exactly the same footing, our subject grounding results should also be
considered fair among all the models.

##  7 Sample Qualitative Results

Flickr30K Entities examples are provided in Figure 1 & 2.
RefCOCO+ testA example is provided in Figure 5.
RefCOCO+ testB example is provided in Figure 6.
SpatialSense first example is provided in Figure 7.
SpatialSense second example is provided in Figure 8.

Figure 5: RefCOCO+ (testA) \- two sample qualitative results. Figure 6:
RefCOCO+ (testB) \- two sample qualitative results. Figure 7: Spatial Sense
sample qualitative results without the ambiguity for Subject grounding. Figure
8: Spatial Sense sample qualitative results with ambiguity for Subject
grounding, due to the underlying distraction.

##  8 Histograms of Score Distributions

Flickr30K Entities test is provided in Figure 9.
RefCOCO+ testA is provided in Figure 10.
RefCOCO+ testB is provided in Figure 11.
SpatialSense triplets is provided in Figure 12.
SpatialSense subjects is provided in Figure 13.
SpatialSense objects is provided in Figure 14.

Figure 9: Flickr30K Entities test. Figure 10: RefCOCO+ testA. Figure 11:
RefCOCO+ testB. Figure 12: SpatialSense triplets. Figure 13: SpatialSense
subjects. Figure 14: SpatialSense objects.