|
--- |
|
license: apache-2.0 |
|
language: |
|
- zh |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
tags: |
|
- Doctor_consultation |
|
- Taiwan |
|
- fine-tuning |
|
- medicine |
|
--- |
|
|
|
# 🔎 Taiwan-inquiry_7B_v_2.1 |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
"The model was fine-tuned based on the **Breeze-7B-Instruct-v1_0** model using a dataset that includes 614 authentic dialogues from the National Cheng Kung University Hospital. |
|
Additionally, 336 synthetic dialogues were included in the training set, carefully crafted to encompass themes drawn from sample questions of the OSCE (臨床技能測驗) sample questions in Taiwan. |
|
These synthetic dialogues were generated using GPT-3.5, Gemini-Pro and Breexe-8x7B-Instruct-v0_1. |
|
The training process was geared towards simulating verbal exchanges between doctors and patients within a hospital environment." |
|
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/65c07d1b2357c1bded7a92fa/e7QbiYh07kcGwyKniAo0e.png" alt="image/png" style="width:80%; height:auto;"> |
|
|
|
**************************** **Updates** **************************** |
|
* 2024/04/25 🎉 Released [Taiwan-inquiry_7B_v2.1-awq](https://huggingface.co/ChenWeiLi/Taiwan-inquiry_7B_v2.1-awq) |
|
* 2024/04/29 🎉 Released [Taiwan-inquiry_7B_v2.1.gguf](https://huggingface.co/ChenWeiLi/Taiwan-inquiry_7B_v2.1.gguf) |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
- **Developed by:** [Joseph (Chen-Wei) Li](https://www.linkedin.com/in/joseph-li-3a453b231/), researcher assistant from National Taiwan University Hospital. |
|
- **Model type:** A 7B parameter GPT-like model fine-tuned on a combination of private and synthetic dialogue datasets. |
|
- **Language(s) (NLP):** Traditional Chinese (zh-tw) |
|
- **Finetuned from model :** [Breeze-7B-Instruct-v1_0 ](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v1_0) |
|
|
|
### Usage of the model |
|
- The user can take on the role of a doctor, and the model can engage in conversation with you as if it were a patient. |
|
- You can provide the model with a brief patient background in the system prompt, and the model will respond based on that prompt. **(using my patient generator: [**colab**](https://colab.research.google.com/drive/17MSob_tQ2hPtMBL0xOF2zzV6WWe4dEG6?usp=sharing))** |
|
- Directly asking the certain disease about the symptoms and the possible therapies.**（Warning: It's not medical advice!)** |
|
|
|
### Model evaluation |
|
The model got the **TMMLU+** (0 shot) performance using [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) with default settings. |
|
|
|
|Details on TMMLU+ (0 shot):<br/>Model | Base Model | STEM | Social Science | Humanities | Other | AVG | |
|
|-----------------------------------------------------|:---------------------:|:---------------:|:--------------:|:----------:|:----------:|:-------:| |
|
| Taiwan-inquiry_7B_v2.1 |Breeze-7B-Instruct-v1_0| 36.06 | 44.61 | 37.49 | 39.61 | 40.29 | |
|
| Taiwan-inquiry_7B_v2.0 |Breeze-7B-Instruct-v0_1| 36.17 | 43.59 | 35.45 | 37.63 | 38.95 | |
|
| Taiwan-inquiry_7B_v1.0 |Taiwan-LLM-7B-v2.1-chat| 26.74 | 29.47 | 26.83 | 29.61 | 28.83 | |
|
|
|
### DEMO |
|
|
|
- **User: <span style="color: orange;">doctor</span>** |
|
- **Chatbot: <span style="color: gray;">patient</span>** |
|
 |
|
 |
|
|
|
|
