Files changed (1) hide show
  1. README.md +51 -39
README.md CHANGED
@@ -1,39 +1,51 @@
1
- ---
2
- license: other
3
- license_name: qwen
4
- license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE
5
- language:
6
- - en
7
- pipeline_tag: text-generation
8
- base_model: Qwen/Qwen2.5-72B-Instruct
9
- tags:
10
- - chat
11
- - mlx
12
- library_name: transformers
13
- ---
14
-
15
- # GGorman/Qwen2.5-72B-Instruct-Q8-mlx
16
-
17
- The Model [GGorman/Qwen2.5-72B-Instruct-Q8-mlx](https://huggingface.co/GGorman/Qwen2.5-72B-Instruct-Q8-mlx) was converted to MLX format from [Qwen/Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) using mlx-lm version **0.19.1**.
18
-
19
- ## Use with mlx
20
-
21
- ```bash
22
- pip install mlx-lm
23
- ```
24
-
25
- ```python
26
- from mlx_lm import load, generate
27
-
28
- model, tokenizer = load("GGorman/Qwen2.5-72B-Instruct-Q8-mlx")
29
-
30
- prompt="hello"
31
-
32
- if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
33
- messages = [{"role": "user", "content": prompt}]
34
- prompt = tokenizer.apply_chat_template(
35
- messages, tokenize=False, add_generation_prompt=True
36
- )
37
-
38
- response = generate(model, tokenizer, prompt=prompt, verbose=True)
39
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: qwen
4
+ license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE
5
+ language:
6
+ - zho
7
+ - eng
8
+ - fra
9
+ - spa
10
+ - por
11
+ - deu
12
+ - ita
13
+ - rus
14
+ - jpn
15
+ - kor
16
+ - vie
17
+ - tha
18
+ - ara
19
+ pipeline_tag: text-generation
20
+ base_model: Qwen/Qwen2.5-72B-Instruct
21
+ tags:
22
+ - chat
23
+ - mlx
24
+ library_name: transformers
25
+ ---
26
+
27
+ # GGorman/Qwen2.5-72B-Instruct-Q8-mlx
28
+
29
+ The Model [GGorman/Qwen2.5-72B-Instruct-Q8-mlx](https://huggingface.co/GGorman/Qwen2.5-72B-Instruct-Q8-mlx) was converted to MLX format from [Qwen/Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) using mlx-lm version **0.19.1**.
30
+
31
+ ## Use with mlx
32
+
33
+ ```bash
34
+ pip install mlx-lm
35
+ ```
36
+
37
+ ```python
38
+ from mlx_lm import load, generate
39
+
40
+ model, tokenizer = load("GGorman/Qwen2.5-72B-Instruct-Q8-mlx")
41
+
42
+ prompt="hello"
43
+
44
+ if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
45
+ messages = [{"role": "user", "content": prompt}]
46
+ prompt = tokenizer.apply_chat_template(
47
+ messages, tokenize=False, add_generation_prompt=True
48
+ )
49
+
50
+ response = generate(model, tokenizer, prompt=prompt, verbose=True)
51
+ ```