# COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning

Yuelin Bai1 Xinrun Du211footnotemark: 1 Yiming Liang311footnotemark: 1
Yonggang Jin211footnotemark: 1
Ziqiang Liu1 Junting Zhou4,2 Tianyu Zheng2 Xincheng Zhang5 Nuo Ma6
Zekun Wang2 Ruibin Yuan7,2 Haihong Wu5 Hongquan Lin5 Wenhao Huang6
Jiajun Zhang3 Wenhu Chen8,9,2 Chenghua Lin10,2 Jie Fu7,2 Min Yang1
Shiwen Ni1 Ge Zhang8,922footnotemark: 2
1Shenzhen Institute of Advanced Technology, CAS 2M-A-P 3Institute of
Automation, CAS
4Peking University 5University of Science and Technology of China 601.ai
7HKUST
8University of Waterloo 9Vector Institute 10University of Manchester
Equal contributionCorresponding author

###### Abstract

Recently, there have been significant advancements in large language models
(LLMs), particularly focused on the English language. These advancements have
enabled these LLMs to understand and execute complex instructions with
unprecedented accuracy and fluency. However, despite these advancements, there
remains a noticeable gap in the development of Chinese instruction tuning. The
unique linguistic features and cultural depth of the Chinese language pose
challenges for instruction tuning tasks. Existing datasets are either derived
from English-centric LLMs or are ill-suited for aligning with the interaction
patterns of real-world Chinese users. To bridge this gap, we introduce COIG-
CQIA, a high-quality Chinese instruction tuning dataset. Our aim is to build a
diverse, wide-ranging instruction-tuning dataset to better align model
behavior with human interactions. To this end, we collect a high-quality
human-written corpus from various sources on the Chinese Internet, including
Q&A communities, Wikis, examinations, and existing NLP datasets. This corpus
was rigorously filtered and carefully processed to form the COIG-CQIA dataset.
Furthermore, we train models of various scales on different subsets of CQIA,
following in-depth evaluation and analyses. The findings from our experiments
offer valuable insights for selecting and developing Chinese instruction-
tuning datasets. We also find that models trained on CQIA-Subset achieve
competitive results in human assessment as well as knowledge and security
benchmarks. Data are available at https://huggingface.co/datasets/m-a-p/COIG-
CQIA

##  1 Introduction

Large Language Models (LLMs), such as GPT-3 (Brown et al., 2020), LLaMA
(Touvron et al., 2023), and PaLM (Chowdhery et al., 2023), have demonstrated
remarkable capabilities as general-purpose assistants. The cornerstone of this
achievement is instruction tuning, which significantly enhances the
capabilities and controllability of LLMs through training on datasets composed
of instruction-output pairs (Zhang et al., 2023b). This technique effectively
aligns the models’ training objectives with human intentions, thereby ensuring
that the models can interpret and execute human instructions both effectively
and safely. Therefore, the availability of high-quality instruction tuning
datasets is crucial for LLMs to operate as efficient and dependable
assistants.

There exists many English instruction tuning datasets. However, the available
datasets for Chinese instruction tuning are generally either limited in size
or lacking in quality. Chinese instruction tuning datasets are categorized
into three main types: (1) datasets derived from English instruction datasets
(Peng et al., 2023) or NLP datasets (CLUEbenchmark, 2022; Yang, 2023), (2)
datasets generated by LLMs (Guo et al., 2023), and (3) self-generated
instruction tuning datasets (Ji et al., 2023; Sun et al., 2023). COIG (Zhang
et al., 2023a) integrates multiple approaches to construct a human-verified
universal high-quality Chinese instruction corpus. However, the previously
mentioned Chinese instruction tuning datasets have inherent issues such as not
aligning with natural Chinese communication patterns, lacking genuine Chinese
linguistic data, containing numerous problematic data points, and having
small-scale data. This paper focuses on constructing a Chinese instruction
tuning dataset sourced from authentic Chinese linguistic data across diverse
domains and undergone meticulous manual cleaning procedures aimed at enhancing
the proficiency of LLMs in following Chinese instructions.

In this paper, we introduce COIG-CQIA (Chinese Open Instruction Generalist -
Quality Is All You Need), a high-quality Chinese instruction tuning dataset,
which is designed to provide the Chinese NLP community with high-quality and
human interaction-aligned instruction fine-tuning data. Inspired by the work
of LIMA (Zhou et al., 2023), COIG-CQIA focuses on curating a dataset from
Chinese internet sources, comprising Q&A sessions and articles. These sources
undergo thorough cleaning, restructuring, and manual review to ensure high
quality, diversity, and relevance. Furthermore, we conduct analytical
experiments to assess the effects of data quality, provenance, and mixing
ratio.

In summary, the contributions are as follows:

  * •

We propose a high-quality Chinese instruction fine-tuning dataset,
specifically designed to align with human interaction, achieved through
rigorous filtering procedures.

  * •

We explore the influence of various data sources, including social media,
encyclopedias, and traditional NLP tasks, on model performance. Our analysis
offers essential insights for selecting training data from the Chinese
internet.

  * •

Various benchmark tests and human evaluations confirm that models fine-tuned
on our CQIA dataset exhibit superior performance, thus establishing CQIA as a
valuable resource for the Chinese NLP community.

##  2 Related Work

###  2.1 Instruction Tuning Dataset

Instruction tuning aims to train large language models to generate responses
that align with input instructions, thereby enable LLMs with conversational
and task execution capabilities. Compared with standard LLM, SFT allows the
behavior of the model to be more controllable and predictable, thereby
achieving the purpose of aligning human will. Methods to build instruction
tuning datasets include: (1) Pure manual annotation (Conover et al., 2023).
This method completely constructs instructions and answers manually, which is
very time-consuming and laborious; (2) Converted from existing datasets
(Mishra et al., 2022; Sanh et al., 2022; Chung et al., 2022). Some studies use
supervised data sets from NLP tasks to construct instruction tuning data; (3)
Automatically generated using LLM (Honovich et al., 2022; Wang et al., 2023;
Xu et al., 2023a; Ji et al., 2023; Xu et al., 2023b). Others use existing LLMs
to generate instruction tuning data. A common practice is to first manually
annotate high-quality seed datasets, and then use LLM to expand the seed
instructions and corresponding outputs, which can generate large-scale
instruction tuning data with very little human annotation. However, the
quality cannot be guaranteed and there is a certain amount of noisy data,
which can lead to hallucinations.

There exists many English instruction tuning datasets. In comparison, existing
Chinese instruction tuning datasets are either small in scale or have quality
issues. Some studies have translated the English instruction tuning datasets
into Chinese (Peng et al., 2023), but this may lead to accumulation of
translation errors. pCLUE (CLUEbenchmark, 2022) and Firefly (Yang, 2023)
transform the original NLP task dataset into instruction tuning datasets. HC3
(Guo et al., 2023) collects tens of thousands of comparison responses from
both human experts and ChatGPT. COIG (Zhang et al., 2023a) builds a human-
verified universal high-quality Chinese instruction corpus. BELLE (Ji et al.,
2023) and MOSS (Sun et al., 2023) use a method similar to self-instruct (Wang
et al., 2023) to automatically generate Chinese instruction tuning datasets.

###  2.2 Data Mixture of SFT

Currently, more and more studies has begun to pay attention to the importance
of data quality of instruction tuning. LIMA (Zhou et al., 2023) only uses
1,000 high-quality instructions and outputs for SFT, and does not even need to
perform RLHF training to achieve very strong performance. AlpaGasus (Chen et
al., 2023) uses powerful LLM to automatically identify and filter low-quality
data, resulting in high-quality instruction tuning data to improve performance
and training speed. Humpback (Li et al., 2023) filters out high-quality
samples to fine-tune a more powerful LLM.

Others (Song et al., 2023) explores the impact of the mixture strategies of
different instruction tuning datasets. Tulu series (Konchakov et al., 2023;
Ivison et al., 2023) show that increasing instruction diversity can
effectively improve the performance and different instruction tuning datasets
can discover or enhance specific skills, while no one dataset (or combination)
provides the best performance across all assessments.

##  3 CQIA CURATION

Source | Quantity | Source | Quantity
---|---|---|---
Zhihu | 8837 | Douban | 3132
Xiaohongshu | 1508 | Segment Fault | 458
Encyclopedia Article | 980 | Encyclopedia of China | 1706
WikiHow | 1876 | COIG PC | 3000
Middle school Exam | 2000 | Graduate Entrance Examination | 475
Logi QA | 422 | CValue | 906
COIG-Human-Value | 101 | Chinese Traditional | 232
Idiom Explanation | 112 | Poem Writing | 47
Classical Chinese Translation | 112 | MBA Encyclopedia | 10689
Finance NLP Task | 600 | Medical Encyclopedia | 8351
Medical Article | 186 | Law | 2645
Total |  | 48375 |

Table 1:  The amount of data from different sources of the dataset mixture

To ensure the quality and diversity of our data, we manually selected 13 data
sources from high-quality websites and data resources within the Chinese
Internet. These sources include community Q&A forums, encyclopedic sites,
content creation platforms, examinations, etc. We also incorporated high-
quality Chinese NLP datasets to enrich the diversity of tasks. Specifically,
we categorized all data sources into four types: Social Media & forums, World
Knowledge, NLP tasks, and Examinations. The data sources and their
descriptions are as follows.

###  3.1 Social Media & Forums

##### Zhihu

is a vibrant question-and-answer platform where users can ask and answer
questions on a wide range of topics, making it a comprehensive repository of
knowledge and insights. Zhihu encourages its users to provide well-thought-out
answers that are informative and reflective of expert knowledge or personal
experience. However, the absence of a review mechanism for answers on Zhihu
leads to a large volume of content that falls short of our quality standards.
To filter low quality answers, we selected answers with more than 50 upvotes,
then filtering out content containing sensitive or harmful keywords using a
rule-based method. Subsequently, we employed GPT-4 to score the responses on a
scale of 1-10, retaining those with scores above 8.

##### SegmentFault

is a question-and-answer community focused on IT technology, providing Chinese
developers with a high-quality platform for exchange, similar to Stack
Overflow. In this community, users engage in asking and answering questions
related to IT technology, where the questioner can accept the most useful
answer. Additionally, community members can also upvote or comment on answers.
Our data are collected from the contents posted before 2018, as earlier
content may become outdated due to changes in programming languages or
software versions. We then select the "accepted" answers with at least 5
upvotes. Furthermore, we manually review all the (question, answer) pairs to
remove or modify low-quality content.

##### Douban

is a social network and database that allows users to create content related
to literature and artistic works such as films, books, TV series, music, etc.
We sample data from books, movies, and TV series, extracting metadata that
includes ratings, detailed information on actors/crew, and long reviews. Then,
we design three tasks in total: synopsis generation, review generation, and
recommendations. For each task, we manually design various prompt templates
and used these templates in combination with metadata to construct
instructions. For synopsis generation and review generation, we construct
instructions using prompt templates combined with movie or TV series names,
with responses generated by Douban users. Then we remove responses with
lengths shorter than a threshold and delete personal information and
irrelevant content(e.g., "Subscribe our Official Accounts"). Additionally, we
manually adjusted some instructions to add more complex implicit intents,
aligning better with the details of the response.

##### Xiaohongshu

provides a space for users to share their lives, travel, food, and product
recommendations. Contents in this platform are renowned on the Chinese
internet for their unique and expressive style. We sample posts with lengths
ranging from 500 to 2000, excluding those that involve interactions with other
users ("@User_Name") and those referencing images or videos ("as shown in the
picture/video").

##### Ruozhiba

is a sub-forum of Baidu Tieba, an interests-based community forum. Its posts
often contain puns, polysemous terms, causal reversals, and homophones, many
of which are designed with logical traps, posing challenges even for humans.
We collected the 500 most upvoted threads. Using the titles as instructions,
we eliminate those that were either non-instructive (i.e., declarative
statements or unanswerable) or toxic. Responses were generated by either
humans or GPT-4. We conducted manual reviews for GPT4’s responses to ensure
accuracy, ultimately obtaining 240 (instruction, response) pairs.

###  3.2 World Knowledge

####  3.2.1 General Encyclopedia

General Encyclopedia provides comprehensive coverage of a wide range of topics
across various fields. We collect data from three Chinese Encyclopedia
websites: One Hundred Thousand Whys111https://10why.net/, wikiHow-
zh222https://zh.wikihow.com and Encyclopedia of China333https://www.zgbk.com/.
One Hundred Thousand Whys is an encyclopedic website aimed at popular science,
featuring thousands of high-quality articles asking "why" across topics from
natural science to humanities. We collect data from all 15 categories and
ensure uniform distribution across each category. Article titles are used as
instruction(e.g."Why don’t I get altitude sickness when I fly?"), and the
content as responses, with responses under 300 characters being filtered out.
wikiHow-zh, the Chinese version of wikiHow, is an encyclopedia-style website
covering a wide range of topics, featuring tens of thousands of "how-to"
articles with multiple revisions. We collected xxx articles from the site and
sampled 1500 entries from all 19 categories, with a sampling temperature of 3.
Since the original data are in HTML, we parse the HTML and concatenate the
article content using Markdown. Subsequently, we filtered out low-quality data
(e.g., incorrect formula conversions) and articles exceeding 3000 words in
length. We use titles as the instructions and the article contents as
responses. Encyclopedia of China is a comprehensive encyclopedia comprising
approximately 500,000 entries, authored and revised by domain experts. We
design various prompt templates for concept explanation tasks. We sample the
entries from all 74 categories, with structures comprising entry names and
several subtitles, along with their respective contents. We randomly combined
entry names or subtitles with prompt templates to construct instructions. For
instance, for the "Confucius" entry , which includes subtitles "Biography",
"Academic Theories", and "Impacts", we selected "Academic Theories" to create
instruction such as "Write the details of Confucius’s academic theories." and
then, we use this subtitle’s content as a response.

####  3.2.2 Domain Specific Knowledge

We collected data from four specific domains: medicine, economic management,
electronics, and agriculture.

Medical Domain sources from three websites: Baobaozhidao, Qianwen Health, and
Baikemingyi. Both Baobaozhidao and Qianwen Health feature question-and-answer
style articles written by medical experts, with the former primarily focusing
on a broad range of medical fields, while the latter on maternal and infant
health. We collected articles from these two sites, excluding those whose
titles are not in question form. Subsequently, we used the titles as
instructions and the article content as responses. Baikemingyi contains
Wikipedia-style structured data, featuring introductions to tens of thousands
of diseases and medications. We designed various prompt templates and combined
entry names with these templates to construct commands (e.g., "Write a
professional introduction about joint pain").

Economic Management Domain data is collected from MBA Wiki Encyclopedia, a
website that encompasses Wikipedia-style structured knowledge, authored and
revised by numerous contributors. We designed various prompt templates,
combining entry names with random templates to construct instructions, such as
"Please explain the following term in detail: Remittance Agent". Ultimately,
the content of the entries is concatenated and constructed into responses in
markdown format.

Electronics Domain data is sourced from the EETrees electronic encyclopedia,
which is also structured in form. We design various prompt templates and
combine these with entry names to construct instructions, with the
corresponding content as the response.

Agriculture Domain sources from an agricultural encyclopedia website,
containing a range of topics from plant cultivation to animal breeding. We
collected articles on all ten topics, excluding those with non-question
titles, containing images, or shorter than 300 words in length. Subsequently,
we construct (instruction, response) pairs from the titles and content of the
articles.

###  3.3 Examinations

##### The Middle School and College Entrance Examinations

primarily derives from the COIG datasetZhang et al. (2023a), a harmless,
helpful, and diverse Chinese Instruction dataset. Chinese examination is a
subset of it, with the Middle School and College Entrance Examinations being
China’s principal general competency tests. These data contain a variety of
question types and detailed answer explanations, primarily covering humanities
subjects (Chinese, English, Politics, Biology, History, and Geography). We use
temperature sampling on the data across these subjects and then filtered out
questions and answers with formatting errors. The questions were used as
instructions and, the "answer" and "analysis" fields were concatenated to form
extended responses, resulting in 1964 (instruction, response) pairs.

##### Graduate Entrance Examination

is one of the most challenging examinations in China, exceeding college
entrance exams in difficulty and requiring advanced knowledge application and
depth. We have collected a variety of exam papers from recent years across
disciplines including mathematics, computer science, chemistry, law,
psychology, medicine, etc. Using Mathpix444https://mathpix.com/ for image-to-
text conversion, we extracted questions and answers and converted them into
LaTeX format. We eliminate data without analysis and manually verified the
accuracy of the questions and answers. We eliminate data without analysis and
manually verified the accuracy of the questions and answers.

##### Logical Reasoning Test

aims to assess the ability to apply logical and analytical reasoning to solve
problems. This type of test is widely used in various competitive examinations
to evaluate critical thinking and problem-solving skills. We collect logic
reasoning questions from the internet, retaining those containing detailed
answer analyses, and then construct them into (instruction, response) pairs.

##### Chinese Culture Test

investigates the mastery of traditional Chinese culture and history. We
collected multiple-choice questions on traditional Chinese culture from the
internet, retaining those with answer analyses, and constructed them into
(instruction, response) pairs.

###  3.4 NLP Datasets

##### COIG-PC

The COIG-PC Dataset is a comprehensive collection of Chinese NLP tasks, aimed
at advancing Chinese NLP research. The goal of this dataset is to provide a
rich set of resources to researchers and developers, which they can use to
improve the capabilities of language models in handling Chinese text. It
offers a comprehensive suite of resources for researchers and developers,
facilitating advancements in language model capabilities across various
domains including text generation, information extraction, sentiment analysis,
and machine translation, etc. Initially, we selected 1,413 tasks involving
both Chinese and English languages from COIG-PC. Then, we manually select 250
tasks that meet our quality criteria, including information extraction,
classification, summary, and others, primarily sourced from traditional NLP
datasets. Through temperature sampling, we eventually sample 3,000
(instruction, response) pairs, which are further verified by human to ensure
quality.

##### COIG Human Value

is a subset of the COIG datasetZhang et al. (2023a) designed to provide
instruction fine-tuning data aligned with human values. We selected the
portion reflecting Chinese cultural values, constructed using the Self-
InstructWang et al. (2023) method from manually selected seed instructions. We
manually filtered out data with formatting errors and incorrect answers,
retaining those that include explanations of the answers to form (instruction,
response) pairs.

##### Firefly Chinese Traditional

comprises three tasks: Classical Chinese Translation, Ancient Poetry Writing,
and Idiom Interpretation, which are the subset of the Firefly datasetYang
(2023) related to traditional Chinese culture. We filter the responses shorter
than 300 characters, and sample 300 instances from each task. Then, we
manually filtered out low-quality data such as instruction-response mismatch,
response error, and unanswerable instructions.

##### 100PoisonMpts

addressing issues of anti-discrimination and empathy, spans various dimensions
including jurisprudence, psychology, child education, obscure facts, intimate
relationships, etc. It involves human-generated prompts that evoke bias and
discrimination, followed by expert-crafted responses that align with human
values. To enhance the harmlessness of the CQIA, we sample all the data from
100PoisonMpts.

Figure 1: Most common root verbs (inner circle) and top direct noun objects
(outer circle) in the CQIA dataset. Note that we only visualize when a certain
verb-noun pair has more than 30 instances, and many instructions do not
contain a verb-noun structure.  Figure 2: Overview of CQIA Task Types. Figure
3: Length distribution of instruction and responses. Note that the instruction
is the concatenation of original instructions and inputs in our dataset.

##  4 Data Analysis

###  4.1 Statistics

Table 1 describe the data statistics for all sources. We collected a total of
48,375 instances from 22 sources within the Chinese Internet and Community,
covering domains ranging from general knowledge and STEM to humanities. Figure
2 illustrates the variety of task types, encompassing information extraction,
question answering, code generation, etc. We demonstrated the distribution in
the length of the instructions and responses in Figure3.

###  4.2 Diversity

To analyze the diversity of the COIG-CQIA dataset, we follow prior workWang et
al. (2023); Lou et al. (2023) by employing the Hanlp toolHe and Choi (2021) to
parse the instructions and then extract the verb closest to the root along
with its top direct noun object. We then plot the top 20 most common root
verbs and their corresponding direct noun objects in Figure1. From this figure
we can observe that CQIA features a diverse range of instructions and
intentions.

| Source | Open QA | Brainstorming | Classification | Generation | Summarization | Rewrite | Closed QA | Extract | Math | Code | Average
---|---|---|---|---|---|---|---|---|---|---|---|---
Yi-6B | SegmentFault | 26.3 | 33.6 | 8.9 | 23.5 | 29.9 | 20.1 | 25.0 | 22.6 | 14.5 | 32.1 | 23.7
COIG PC | 31.4 | 47.7 | 28.7 | 44.8 | 43.4 | 53.3 | 45.5 | 28.4 | 35.6 | 23.2 | 38.2
Douban | 45.2 | 64.5 | 23.3 | 64.4 | 38.6 | 37.1 | 34.2 | 25.4 | 32.9 | 42.1 | 40.8
Zhihu | 48.1 | 72.3 | 19.0 | 66.9 | 24.3 | 29.5 | 28.9 | 12.3 | 8.5 | 40.0 | 35.0
Logi QA | 45.0 | 65.4 | 32.9 | 55.3 | 37.1 | 49.5 | 47.1 | 38.9 | 17.9 | 40.0 | 42.9
Ruozhiba | 64.8 | 84.6 | 50.3 | 73.1 | 45.0 | 39.9 | 57.0 | 30.3 | 27.3 | 63.6 | 53.6
Wiki | 51.0 | 67.6 | 21.5 | 60.4 | 30.8 | 31.5 | 30.2 | 21.4 | 22.7 | 34.7 | 37.2
Finance | 43.2 | 65.7 | 30.0 | 57.3 | 36.4 | 30.2 | 34.6 | 31.4 | 15.7 | 27.5 | 37.2
Exam | 51.5 | 74.3 | 42.0 | 70.9 | 54.1 | 60.5 | 56.2 | 47.7 | 41.5 | 49.9 | 54.9
Xhs | 25.2 | 47.0 | 8.4 | 45.4 | 8.6 | 21.4 | 22.5 | 7.0 | 28.5 | 27.1 | 24.1
WikiHow | 0.5 | 1.2 | 1.5 | 7.7 | 18.1 | 3.1 | 23.1 | 0.5 | 0.0 | 2.1 | 5.8
CQIA-Subset | 59.8 | 86.4 | 48.2 | 79.4 | 60.9 | 69.9 | 50.6 | 42.0 | 37.8 | 55.8 | 64.2

Table 2: The performance of Yi-6B trained on various datasets evaluated on
BELLE-EVAL using GPT4.

| Source | Open QA | Brainstorming | Classification | Generation | Summarization | Rewrite | Closed QA | Extract | Math | Code | Average
---|---|---|---|---|---|---|---|---|---|---|---|---
Yi-6B | SegmentFault | 51.3 | 70.7 | 43.8 | 66.8 | 57.1 | 75.7 | 41.4 | 47.1 | 75.5 | 65.0 | 60.7
COIG PC | 19.1 | 38.0 | 27.8 | 43.9 | 37.8 | 63.7 | 40.6 | 25.8 | 43.6 | 19.1 | 37.2
Douban | 57.4 | 81.7 | 63.6 | 76.2 | 54.7 | 60.1 | 47.5 | 50.4 | 73.8 | 50.6 | 63.2
Zhihu | 66.5 | 90.4 | 52.6 | 82.6 | 71.2 | 78.2 | 46.4 | 39.6 | 76.0 | 62.2 | 69.1
Logi QA | 51.4 | 76.4 | 64.9 | 75.6 | 60.0 | 71.4 | 61.6 | 52.7 | 47.0 | 45.3 | 62.0
Ruozhiba | 75.9 | 92.3 | 76.5 | 92.1 | 77.3 | 70.9 | 67.2 | 68.5 | 72.6 | 65.2 | 76.9
Wiki | 63.0 | 75.7 | 44.0 | 80.6 | 47.9 | 66.6 | 47.9 | 50.0 | 56.8 | 55.6 | 60.5
Finance | 46.8 | 71.1 | 17.1 | 60.1 | 27.4 | 23.6 | 17.2 | 29.4 | 28.5 | 24.8 | 36.7
Exam | 49.4 | 79.7 | 64.7 | 79.9 | 61.5 | 79.8 | 66.2 | 61.0 | 52.8 | 56.3 | 66.2
Xhs | 51.3 | 76.1 | 38.5 | 68.0 | 25.8 | 46.0 | 28.4 | 32.1 | 74.6 | 36.3 | 50.3
Wikihow | 54.7 | 75.2 | 32.1 | 68.2 | 45.3 | 55.9 | 40.9 | 55.8 | 41.0 | 44.4 | 52.7
CQIA-Subset | 56.2 | 84.5 | 48.1 | 72.9 | 60.5 | 70.9 | 54.6 | 50.8 | 52.5 | 49.5 | 61.9

Table 3: The performance of Yi-34B trained on various datasets evaluated on
BELLE-EVAL using GPT4.

##  5 EXPERIMENTAL SETUP

In this section, we describe how we use COIG-CQIA to fine-tune models and
elaborate our our evaluation methods.

###  5.1 Evaluation

##### C-Eval

is a comprehensive Chinese evaluation suite for foundation models. It consists
of 13948 multi-choice questions spanning 52 diverse disciplines and four
difficulty levels. We choosing the answer option with the highest log-
likelihood as the final prediction of the model.

##### CMMLU

is a comprehensive evaluation benchmark specifically designed to evaluate the
knowledge and reasoning abilities of LLMs within the context of Chinese
language and culture. CMMLU covers a wide range of subjects, comprising 67
topics that span from elementary to advanced professional levels. It includes
subjects that require computational expertise, such as physics and
mathematics, as well as disciplines within humanities and social sciences.

##### BELLE-EVAL

is an open-ended test set comprising 12 different instruction types across
various domains, including open question answering, brainstorming,
mathematics, coding, etc. It can be used to assess a model’s ability to follow
instructions across different types. We employ sampling generation for
generating responses to instructions and use a model-based evaluation method.

##### SafetyBench

comprises 11,435 diverse multiple choice questions spanning across 7 distinct
categories of safety concerns. We evaluate models in few-shot setting.

###  5.2 Implementation Details

We fine-tuned bilingual LLMs(Chinese and English) on COIG-CQIA, including
YiYoung et al. (2024), QwenBai et al. (2023), and InternLMTeam (2023), which
represent the forefront of Chinese models. We will provide implementation
details in the next revision.

Model | SafetyBench
---|---
COIG PC | 81.2
Chinese Tradiational | 76.6
Douban | 76.2
Exam | 77.6
Finance | 75.1
Logi QA | 79.1
Ruozhiba | 81.3
Segmentfault | 78.0
Wiki | 75.8
Wikihow | 76.4
Xhs | 76.0
Zhihu | 75.8
Human Value | 79.1
CQIA-Sub-6B | 81.7
GPT-4-0613 | 89.2
GPT-3.5-turbo-0613 | 80.4

Table 4: SafetyBench scores of Yi-6B trained on various data sources.

Model | Ceval (val 5-shot) | CMMLU (test 5-shot)
---|---|---
Qwen-1.8b | 51.34 | 47.26
Yi-6B | 73.40 | 74.85
Qwen-14b | 68.20 | 67.96
InternLM2-20b | 71.25 | 67.48
Yi-34b | 77.04 | 78.18
Qwen-72b | 78.68 | 76.79

Table 5: Performance of different base models after training on the COIG
Subset data. Figure 4: Human evaluation of pair-wise comparison between CQIA-
Subset and 5 strong baselines at similar parameter scale.

##  6 EXPERIMENT RESULTS

###  6.1 Ablating Instruction Data Sources

We finetune the Yi series modelYoung et al. (2024) and the Qwen-72BBai et al.
(2023) model on different data sources of our datasets. to analyze the impact
of data source on model capabilities across different domain knowledge. Then,
We evaluate each model performance on various type of assistant-style tasks
using model-based(i.e. GPT-4) automatic evaluation on Belle-Eval.

To understand the correlation between training data sources and the downstream
performance of different tasks, we evaluate the models on 10 tasks from BELLE-
Eval. We employ GPT-4 as evaluator for scoring model responses, with scores
ranging from 0 to 1.

Tabel 2 shows the results of different Yi-6B-based models fine-tuned on
different subsets. From the table, we can see that models trained on our data
excel in generative tasks such as brainstorming, generation, and
summarization, but perform poorly in math and coding. The Exam subset achieves
the best performance across all subsets with an average score of 54.9,
particularly excelling in Extract and Math tasks. This is expected as Exam
contains more math quizzes and exam types(e.g., reading comprehension),
potentially boosting the model’s performance in most tasks. Interestingly,
Ruozhiba ranks second on average across all subsets. We conjecture this is
because it may enhance the model’s logical reasoning ability, thereby
benefiting most of the instruct-following tasks. COIG-PC demonstrates
proficiency in evaluations of the knowledge dimension, such as C-Eval, yet
underperforms in Belle-Eval. We attribute this discrepancy to its origin in
traditional NLP datasets and the short length of responses, which can impair
reasoning tasks and are less favored by model-based evaluators. The
substantial gap between C-Eval and Belle-Eval highlights the importance of
developing assessments that can comprehensively and accurately evaluate
Chinese LLMs. Moreover, WikiHow scores only 5.8, which we believe is due to
the lack of diversity in its "how-to" instructions.

###  6.2 Human Evaluation

In addition to automatic evaluation, we also evaluate Yi-6B fine-tuned on
CQIA-subset by comparing it to state-of-the-art Chinese open-source chat
models in similar parameter scale. As we focus on questions posed by real-
world Chinese-speaking users. We sample 200 questions from OL-
CC555https://data.baai.ac.cn/details/OL-CC and Zhihu which are not present in
the training set for human evaluation. We conduct pair-wise comparison, aiming
to demonstrate how our model performs in comparison to others when facing
real-world human prompt.

For each prompt, we generate one response from each model respectively666We
generate using nucleus sampling with $p$=0.85, $k$=50 and temperature=0.9..
Annotators are then presented with the prompt and two responses: one generated
by CQIA model and one by another baseline model. Subsequently, we ask which
response the annotator prefers, allowing for a "tie" selection when a better
response is hard to judge.

Figure 4 shows human evaluation results with CQIA and other 5 baselines,
namely Yi-6B-Chat, Baichuan2-7B-Chat, ChatGLM2-6B, Qwen-7B-Chat and
InternLM-7B-Chat. The results demonstrate that, compared to strong baselines,
CQIA-Subset achieved higher human preference, with at least over 60% of
responses being better than or on par with the baseline models. This can be
attributed to CQIA not solely in generating high-quality responses to human
questions or instructions, but also in its responses being more aligned with
real-world human communication patterns, leading to higher human preference.

###  6.3 Scaling Model Size

We investigate the performance of different base model with varying parameter
sizes after fine-tuned on our CQIA-Subset. Notably, Yi-6B surpasses Qwen-14B
and InternLM-20B, which have at least twice its parameter size. Further,
Yi-34B achieved comparable results to Qwen-72B in both C-Eval and CMMLU
benchmarks. This observervation underscores the balance between model size,
architectural optimizations, and training methodologies. While the scaling law
might suggest that larger models inherently perform better due to their
increased language understanding capacity, our results indicate that this is
not always the case. Specifically, the Yi-6B model’s superior performance
against models with significantly more parameters challenges the notion that
parameter count alone is a sufficient predictor of model efficacy.

###  6.4 Safety

We explore the impact of data sources on model safety by evaluating our models
on SafetyBench. Models trained on CQIA-Subset scores the highest within CQIA
series, surpassing GPT-3.5-turbo-0613. Model trained on Social Media& Forums
such as Douban, Zhihu, and Xhs perform moderate safety scores, we conjecture
this is due to the diversity and openness of social media content, which also
highlights the risks of harmful information. Additionally, models trained on
Wiki-style data tend to perform lower safety scores, potentially reflecting
the limited diversity within professional data sources, leading to poor
performance on safety issues outside specialty domains.

##  7 Conclusion

In this paper, we introduce a a high-quality Chinese instruction fine-tuning
dataset. COIG-CQIA focuses on creating a dataset from Chinese internet sources
including Q&A and articles. These are deeply cleansed, restructured, and
manually reviewed to ensure quality, diversity, and relevance. This dataset is
designed to provide the Chinese NLP community with high-quality and human
interaction-aligned instruction fine-tuning data.

## References

  * Bai et al. (2023) Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023.  Qwen technical report.  _arXiv preprint arXiv:2309.16609_.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Chen et al. (2023) Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. 2023.  Alpagasus: Training a better alpaca with fewer data.
  * Chowdhery et al. (2023) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023.  Palm: Scaling language modeling with pathways.  _Journal of Machine Learning Research_ , 24(240):1–113.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022.  Scaling instruction-finetuned language models.
  * CLUEbenchmark (2022) CLUEbenchmark. 2022.  pclue: Large-scale prompt-based dataset for multi-task and zero-shot learning in chinese.
  * Conover et al. (2023) Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023.  Free dolly: Introducing the world’s first truly open instruction-tuned llm.
  * Guo et al. (2023) Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023.  How close is chatgpt to human experts? comparison corpus, evaluation, and detection.
  * He and Choi (2021) Han He and Jinho D Choi. 2021.  The stem cell hypothesis: Dilemma behind multi-task learning with transformer encoders.  _arXiv preprint arXiv:2109.06939_.
  * Honovich et al. (2022) Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022.  Unnatural instructions: Tuning language models with (almost) no human labor.
  * Ivison et al. (2023) Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023.  Camels in a changing climate: Enhancing lm adaptation with tulu 2.
  * Ji et al. (2023) Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, and Xiangang Li. 2023.  Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases.
  * Konchakov et al. (2023) R. A. Konchakov, A. S. Makarov, G. V. Afonin, J. C. Qiao, M. G. Vasin, N. P. Kobelev, and V. A. Khonik. 2023.  Critical behavior of the fluctuation heat capacity near the glass transition of metallic glasses.
  * Li et al. (2023) Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, and Mike Lewis. 2023.  Self-alignment with instruction backtranslation.
  * Lou et al. (2023) Renze Lou, Kai Zhang, Jian Xie, Yuxuan Sun, Janice Ahn, Hanzi Xu, Yu Su, and Wenpeng Yin. 2023.  Muffin: Curating multi-faceted instructions for improving instruction following.  In _The Twelfth International Conference on Learning Representations_.
  * Mishra et al. (2022) Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022.  Cross-task generalization via natural language crowdsourcing instructions.
  * Peng et al. (2023) Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023.  Instruction tuning with gpt-4.  _arXiv preprint arXiv:2304.03277_.
  * Sanh et al. (2022) Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Tali Bers, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M. Rush. 2022.  Multitask prompted training enables zero-shot task generalization.
  * Song et al. (2023) Chiyu Song, Zhanchao Zhou, Jianhao Yan, Yuejiao Fei, Zhenzhong Lan, and Yue Zhang. 2023.  Dynamics of instruction tuning: Each ability of large language models has its own growth pace.
  * Sun et al. (2023) Tianxiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Hang Yan, Xiangyang Liu, Yunfan Shao, Qiong Tang, Xingjian Zhao, Ke Chen, Yining Zheng, Zhejian Zhou, Ruixiao Li, Jun Zhan, Yunhua Zhou, Linyang Li, Xiaogui Yang, Lingling Wu, Zhangyue Yin, Xuanjing Huang, and Xipeng Qiu. 2023.  Moss: Training conversational language models from synthetic data.
  * Team (2023) InternLM Team. 2023.  Internlm: A multilingual language model with progressively enhanced capabilities.  https://github.com/InternLM/InternLM.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_.
  * Wang et al. (2023) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023.  Self-instruct: Aligning language models with self-generated instructions.
  * Xu et al. (2023a) Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023a.  Wizardlm: Empowering large language models to follow complex instructions.
  * Xu et al. (2023b) Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023b.  Baize: An open-source chat model with parameter-efficient tuning on self-chat data.
  * Yang (2023) Jianxin Yang. 2023.  Firefly(流萤): 中文对话式大语言模型.  https://github.com/yangjianxin1/Firefly.
  * Young et al. (2024) Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, et al. 2024.  Yi: Open foundation models by 01. ai.  _arXiv preprint arXiv:2403.04652_.
  * Zhang et al. (2023a) Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, and Jie Fu. 2023a.  Chinese open instruction generalist: A preliminary release.
  * Zhang et al. (2023b) Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. 2023b.  Instruction tuning for large language models: A survey.  _arXiv preprint arXiv:2308.10792_.
  * Zhou et al. (2023) Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023.  Lima: Less is more for alignment.