Update README.md
Browse files
README.md
CHANGED
@@ -1,62 +1,60 @@
|
|
1 |
-
ANEMLL
|
2 |
|
3 |
-
ANEMLL (pronounced like “animal”) is an open-source project
|
4 |
-
focused on accelerating the porting of Large Language Models (LLMs)
|
5 |
-
to tensor processors, starting with the Apple Neural Engine (ANE).
|
6 |
|
7 |
-
The goal is to provide a fully open-source pipeline
|
8 |
-
from model conversion to inference for common LLM architectures
|
9 |
-
running on ANE.
|
10 |
|
11 |
-
This enables seamless integration and on-device inference
|
12 |
-
for low-power applications on edge devices,
|
13 |
-
ensuring maximum privacy and security.
|
14 |
|
15 |
-
This is critical for autonomous applications,
|
16 |
-
where models run directly on the device
|
17 |
-
without requiring an internet connection.
|
18 |
|
19 |
-
|
20 |
-
|
21 |
-
ANEMLL is licensed under the MIT License.
|
22 |
-
https://opensource.org/license/mit
|
23 |
-
The model is based on Meta’s LLaMA 3.2 and may require a separate license.
|
24 |
|
|
|
25 |
|
|
|
|
|
26 |
|
27 |
-
This test model is exclusively for the Meta's LLaMA 3.2 1B (1024 context) model converted for CoreML,
|
28 |
-
released before the official launch of the ANEMLL repository and minimal documentation.
|
29 |
-
It is intended for early adopters only who requested an early release.
|
30 |
|
|
|
31 |
|
32 |
-
Requirements
|
33 |
-
• macOS Sequoia with Apple Neural Engine and 16GB RAM
|
34 |
-
• CoreML Tools and HuggingFace Transformers libraries
|
35 |
-
• Python 3.9
|
36 |
|
37 |
-
|
38 |
-
|
|
|
39 |
|
|
|
|
|
40 |
|
41 |
-
|
|
|
|
|
42 |
pip install coremltools transformers
|
|
|
43 |
|
44 |
-
How to RUN:
|
45 |
python chat.py
|
46 |
|
47 |
Ctr-D to exit, Ctr-C to interrupt inference.
|
48 |
|
49 |
alternative way to run:
|
|
|
50 |
python chat.py S123 -d /path/to/anemll-LLAMA32-1B-ctx1024 ctx=1024
|
|
|
51 |
|
52 |
The first time the model loads, macOS will take some time to place it on the device.
|
53 |
Subsequent loads will be instantaneous.
|
54 |
|
|
|
|
|
55 |
Please check following links for later updates:
|
56 |
-
|
57 |
-
https://
|
58 |
-
https://
|
59 |
-
https://
|
|
|
60 |
|
61 | |
62 |
|
|
|
1 |
+
# ANEMLL
|
2 |
|
3 |
+
**ANEMLL** (pronounced like “animal”) is an open-source project focused on accelerating the porting of Large Language Models (LLMs) to tensor processors, starting with the Apple Neural Engine (ANE).
|
|
|
|
|
4 |
|
5 |
+
The goal is to provide a fully open-source pipeline from model conversion to inference for common LLM architectures running on ANE.
|
|
|
|
|
6 |
|
7 |
+
This enables seamless integration and on-device inference for low-power applications on edge devices, ensuring maximum privacy and security.
|
|
|
|
|
8 |
|
9 |
+
This is critical for autonomous applications, where models run directly on the device without requiring an internet connection.
|
|
|
|
|
10 |
|
11 |
+
---
|
|
|
|
|
|
|
|
|
12 |
|
13 |
+
## License
|
14 |
|
15 |
+
ANEMLL is licensed under the [MIT License](https://opensource.org/license/mit).
|
16 |
+
The model is based on Meta’s LLaMA 3.2 and may require a separate license.
|
17 |
|
18 |
+
This test model is exclusively for the Meta's LLaMA 3.2 1B (1024 context) model converted for CoreML, released before the official launch of the ANEMLL repository and minimal documentation. It is intended for early adopters only who requested an early release.
|
|
|
|
|
19 |
|
20 |
+
---
|
21 |
|
22 |
+
## Requirements
|
|
|
|
|
|
|
23 |
|
24 |
+
- **macOS Sequoia** with Apple Neural Engine and 16GB RAM
|
25 |
+
- **CoreML Tools** and **HuggingFace Transformers** libraries
|
26 |
+
- **Python 3.9**
|
27 |
|
28 |
+
`chat.py` provides a sample inference script.
|
29 |
+
*We apologize for the current quality of `chat.py` and appreciate your patience.*
|
30 |
|
31 |
+
#Installation
|
32 |
+
Unzip all ZIP files with CoreML Models files
|
33 |
+
```bash
|
34 |
pip install coremltools transformers
|
35 |
+
```
|
36 |
|
37 |
+
#How to RUN:
|
38 |
python chat.py
|
39 |
|
40 |
Ctr-D to exit, Ctr-C to interrupt inference.
|
41 |
|
42 |
alternative way to run:
|
43 |
+
```bash
|
44 |
python chat.py S123 -d /path/to/anemll-LLAMA32-1B-ctx1024 ctx=1024
|
45 |
+
```
|
46 |
|
47 |
The first time the model loads, macOS will take some time to place it on the device.
|
48 |
Subsequent loads will be instantaneous.
|
49 |
|
50 |
+
|
51 |
+
##
|
52 |
Please check following links for later updates:
|
53 |
+
|
54 |
+
• https://huggingface.co/anemll
|
55 |
+
• https://x.com/anemll
|
56 |
+
• https://github.com/anemll
|
57 |
+
• https://anemll.com
|
58 |
|
59 | |
60 |
|