Update README.md
Browse files
README.md
CHANGED
@@ -8,3 +8,47 @@ metrics:
|
|
8 |
- accuracy
|
9 |
pipeline_tag: text-generation
|
10 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8 |
- accuracy
|
9 |
pipeline_tag: text-generation
|
10 |
---
|
11 |
+
# How Do Humans Write Code? Large Models Do It the Same Way Too
|
12 |
+
|
13 |
+
Paper: [https://arxiv.org/pdf/2402.15729](https://arxiv.org/pdf/2402.15729)
|
14 |
+
|
15 |
+
Code: [https://github.com/seamoke/Human-Think-Language](https://github.com/seamoke/Human-Think-Language)
|
16 |
+
|
17 |
+
|
18 |
+
## Introduction
|
19 |
+
We introduce HTL, a model which utilizes the complete reasoning process of CoT to enhance PoT. This model was secondarily fine-tuned based on [MAmmoTH-Coder-7B](https://huggingface.co/TIGER-Lab/MAmmoTH-Coder-7B)
|
20 |
+
|
21 |
+
## Evaluation
|
22 |
+
The models are evaluated using open-ended and multiple-choice math problems from several datasets. Here are the results:
|
23 |
+
|
24 |
+
|
25 |
+
| **Model** | **GSM** |**GSM-Hard** | **NumGLUE** | **MATH** | **Sim** | **SVAMP** | **MAWPS** | **ASDiV** |
|
26 |
+
|---------------------------| ----------|---------------|---------------|-----------|----------|---------- |------------|---------------|
|
27 |
+
| **MAmmoTH-Coder-7B** | 59.4 |56.3 | 66.4 |33.4| 45.9 | 70.7 | 91.9 | 69.3 |
|
28 |
+
| **TORA** | **72.6** |56.0 | 46.2 |**44.6**| 48.5 | 70.4 | 91.3 | **78.7** |
|
29 |
+
| **MAmmoTH-Coder-7B** | 65.7 |**58.3** | **75.1** |34.9| **50.8** | **74.4** | **94.2** | 73.1 |
|
30 |
+
|
31 |
+
## Prompt Format
|
32 |
+
If you want to do HTL:
|
33 |
+
```
|
34 |
+
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
35 |
+
I'd like you to solve this problem in 3 steps:
|
36 |
+
1.Answer the question in plain language without writing any code.\n
|
37 |
+
2.Output one line of *\n.
|
38 |
+
3.Write program code based on the solution process in step 1 to solve the problem.\n
|
39 |
+
### Instruction:
|
40 |
+
{query}
|
41 |
+
Let's write a program.
|
42 |
+
### Response:"
|
43 |
+
```
|
44 |
+
## Citation
|
45 |
+
If you use the models, data, or code from this project, please cite the original paper:
|
46 |
+
|
47 |
+
```
|
48 |
+
@article{li2024humans,
|
49 |
+
title={How Do Humans Write Code? Large Models Do It the Same Way Too},
|
50 |
+
author={Li, Long},
|
51 |
+
journal={arXiv preprint arXiv:2402.15729},
|
52 |
+
year={2024}
|
53 |
+
}
|
54 |
+
```
|