shleeeee commited on
Commit
7bea71a
·
1 Parent(s): be5b661

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -1
README.md CHANGED
@@ -1 +1,43 @@
1
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/654495fa893aec5da96e9134/6z75dYa8TdTy4Y7EIl0CK.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - ko
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - finetune
7
+ ---
8
+ # Model Card for mistral-ko-7b-tech
9
+ It is a fine-tuned model using Korean and NEFT in the mistral-7b model.
10
+
11
+ ## Model Details
12
+
13
+ * **Model Developers** : shleeeee(Seunghyeon Lee)
14
+ * **Repository** : To be added
15
+ * **Model Architecture** : The mistral-ko-7b-wiki-neft is is a fine-tuned version of the Mistral-7B-v0.1.
16
+ * **Lora target modules** : q_proj, k_proj, v_proj, o_proj,gate_proj
17
+ * **train_batch** : 4
18
+ * **Max_step** : 500
19
+
20
+ ## Dataset
21
+ Korean Custom Dataset
22
+
23
+ ## Prompt template: Mistral
24
+ ```
25
+ <s>[INST]{['instruction']}[/INST]{['output']}</s>
26
+ ```
27
+
28
+ ## Usage
29
+ ```
30
+ # Load model directly
31
+ from transformers import AutoTokenizer, AutoModelForCausalLM
32
+
33
+ tokenizer = AutoTokenizer.from_pretrained("shleeeee/mistral-ko-7b-tech")
34
+ model = AutoModelForCausalLM.from_pretrained("shleeeee/mistral-ko-7b-tech")
35
+
36
+ # Use a pipeline as a high-level helper
37
+ from transformers import pipeline
38
+
39
+ pipe = pipeline("text-generation", model="shleeeee/mistral-ko-7b-tech")
40
+ ```
41
+
42
+ ## Evaluation
43
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/654495fa893aec5da96e9134/6z75dYa8TdTy4Y7EIl0CK.png)