Update README.md
Browse files
README.md
CHANGED
@@ -7,12 +7,28 @@ language:
|
|
7 |
- en
|
8 |
- zh
|
9 |
base_model:
|
10 |
-
-
|
11 |
pipeline_tag: text-generation
|
12 |
tags:
|
13 |
- medical
|
14 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
<div align="center">
|
17 |
<h1>
|
18 |
HuatuoGPT-o1-7B
|
|
|
7 |
- en
|
8 |
- zh
|
9 |
base_model:
|
10 |
+
- FreedomIntelligence/HuatuoGPT-o1-7B
|
11 |
pipeline_tag: text-generation
|
12 |
tags:
|
13 |
- medical
|
14 |
---
|
15 |
+
# HuatuoGPT-o1-7B-exl2
|
16 |
+
Original model: [HuatuoGPT-o1-7B](https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-7B) made by [FreedomIntelligence](https://huggingface.co/FreedomIntelligence)
|
17 |
+
Based on: [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) by [Qwen](https://huggingface.co/Qwen)
|
18 |
+
## Quants
|
19 |
+
[4bpw h6 (main)](https://huggingface.co/cgus/HuatuoGPT-o1-7B-exl2/tree/main)
|
20 |
+
[4.5bpw h6](https://huggingface.co/cgus/HuatuoGPT-o1-7B-exl2/tree/4.5bpw-h6)
|
21 |
+
[5bpw h6](https://huggingface.co/cgus/HuatuoGPT-o1-7B-exl2/tree/5bpw-h6)
|
22 |
+
[6bpw h6](https://huggingface.co/cgus/HuatuoGPT-o1-7B-exl2/tree/6bpw-h6)
|
23 |
+
[8bpw h8](https://huggingface.co/cgus/HuatuoGPT-o1-7B-exl2/tree/8bpw-h8)
|
24 |
|
25 |
+
## Quantization notes
|
26 |
+
Made with Exllamav2 0.2.7 with default dataset.
|
27 |
+
Exl2 quants require Nvidia RTX on Windows or Nvidia RTX/AMD ROCm on Linux.
|
28 |
+
Model has to fully fit GPU as RAM offloading isn't supported natively.
|
29 |
+
It can be used with apps such as TabbyAPI, Text-Generation-WebUI, LoLLMs and others.
|
30 |
+
|
31 |
+
# Original model card
|
32 |
<div align="center">
|
33 |
<h1>
|
34 |
HuatuoGPT-o1-7B
|