Now-or-Never commited on
Commit
36a1385
·
1 Parent(s): 0755f49

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +92 -0
README.md CHANGED
@@ -1,3 +1,95 @@
1
  ---
2
  license: apache-2.0
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - zh
5
+ pipeline_tag: text-generation
6
  ---
7
+ ---
8
+ <div style="text-align:center">
9
+ <!-- <img src="https://big-cheng.com/k2/k2.png" alt="k2-logo" width="200"/> -->
10
+ <h2>📈 CFGPT: Chinese Financial Assistant with Large Language Model (CFGPT1-sft-7b-Full)</h2>
11
+ </div>
12
+
13
+ ## Introduction
14
+
15
+ We introduce **CFGPT**, an open-source language model trained by firstly further pretraining general LLMs on collected and cleaned Chinese finance text data (CFData-pt), including financial domain-specific data (announcement, finance articles, finance exams, finance news, finance research papers) and general data (Wikipedia), and secondly fine-tuning with knowledge-intensive instruction tuning data (CFData-sft).
16
+ As for preliminary evaluation, we use CFBenchmark-Basic.
17
+ CFGPT outperforms the baselines on objective and subjective tasks compared to several baseline models with similar parameters.
18
+
19
+ In this repository, we will share the supervised finetuning Full model.
20
+
21
+ - [Supervised Finetuned Model (Full)](https://huggingface.co/TongjiFinLab/CFGPT1-sft-7B-Full): Full model trained weights based on the pretrained model.
22
+
23
+ ## How to Use
24
+
25
+ **1. Prepare the code and the environment**
26
+
27
+ Clone [CFGPT]() repository, create a Python environment, and activate it via the following command
28
+ ```bash
29
+ git clone https://github.com/TongjiFinLab/CFGPT.git
30
+ cd CFGPT
31
+ conda create -n env_name python=3.10
32
+ source activate env_name
33
+ pip install -r requirements.txt
34
+ ```
35
+
36
+ **2. Use CFGPT1-sft-7B-Full**
37
+
38
+
39
+ ```python
40
+ from transformers import AutoModel, AutoTokenizer
41
+ base_model = 'TongjiFinLab/CFGPT1-sft-7B-Full'
42
+ device_map = 'cuda:0'
43
+ tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
44
+ model = AutoModel.from_pretrained(
45
+ base_model,
46
+ trust_remote_code=True,
47
+ device_map=device_map,
48
+ torch_dtype=torch.bfloat16
49
+ )
50
+ model = model.eval()
51
+ inputs = tokenizer("""你是一名金融从业者,请对这篇新闻进行情感分析。请从(中性、积极、消极)中选取答案。新闻内容:挖贝快讯:特步国际发布2023年第二季度中国内地业务营运状况,披露截至2023年6月30日止3个月零售销售实现高双位数同比增长(包括线上线下渠道),零售折扣水平约七五折。同时,2022年7月MSCI首次予以特步ESG评级,一年后评级表现即迎来提升。明晟MSCI上调特步ESG评级,由“BB”升至“BBB”。\n回答:""", return_tensors='pt').to('cuda:4')
52
+ pred = model.generate(**inputs, max_new_tokens=64, do_sample=False, repetition_penalty=1.0)
53
+ print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True).split('回答:')[1])
54
+ ```
55
+
56
+ ## 简介
57
+
58
+ **CFGPT**是一个开源的语言模型,首先通过在收集和清理的中国金融文本数据(CFData-pt)上进行继续预训练,包括金融领域特定数据(公告、金融文章、金融考试、金融新闻、金融研究论文)和通用数据(维基百科),然后使用知识密集的指导调整数据(CFData-sft)进行微调。
59
+ 我们使用CFBenchmark-Basic进行初步评估。与几个具有相似参数的基线模型相比,CFGPT在识别,分类和生成任务上表现优越。
60
+
61
+ 在这个仓库中,我们将分享以下全参数有监督微调的模型。
62
+
63
+ - [Supervised Finetuned Model (Full)](https://huggingface.co/TongjiFinLab/CFGPT1-sft-7B-Full): 基于我们继续预训练模型的进一步全参数微调的完整模型训练权重。
64
+
65
+ ## 如何使用
66
+
67
+ **1. 准备代码和环境**
68
+
69
+ 克隆[CFGPT]()的仓库,创建一个Python环境,并通过以下命令激活它:
70
+ ```bash
71
+ git clone https://github.com/TongjiFinLab/CFGPT.git
72
+ cd CFGPT
73
+ conda create -n env_name python=3.10
74
+ source activate env_name
75
+ pip install -r requirements.txt
76
+ ```
77
+
78
+ **2. 使用 CFGPT1-sft-7B-Full**
79
+
80
+ ```python
81
+ from transformers import AutoModel, AutoTokenizer
82
+ base_model = 'TongjiFinLab/CFGPT1-sft-7B-Full'
83
+ device_map = 'cuda:0'
84
+ tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
85
+ model = AutoModel.from_pretrained(
86
+ base_model,
87
+ trust_remote_code=True,
88
+ device_map=device_map,
89
+ torch_dtype=torch.bfloat16
90
+ )
91
+ model = model.eval()
92
+ inputs = tokenizer("""你是一名金融从业者,请对这篇新闻进行情感分析。请从(中性、积极、消极)中选取答案。新闻内容:挖贝快讯:特步国际发布2023年第二季度中国内地业务营运状况,披露截至2023年6月30日止3个月零售销售实现高双位数同比增长(包括线上线下渠道),零售折扣水平约七五折。同时,2022年7月MSCI首次予以特步ESG评级,一年后评级表现即迎来提升。明晟MSCI上调特步ESG评级,由“BB”升至“BBB”。\n回答:""", return_tensors='pt').to(device_map)
93
+ pred = model.generate(**inputs, max_new_tokens=64, do_sample=False, repetition_penalty=1.0)
94
+ print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True).split('回答:')[1])
95
+ ```