Groovy1 commited on
Commit
a203e90
·
verified ·
1 Parent(s): e52fe68

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +31 -33
README.md CHANGED
@@ -1,62 +1,60 @@
1
- ANEMLL
2
 
3
- ANEMLL (pronounced like “animal”) is an open-source project
4
- focused on accelerating the porting of Large Language Models (LLMs)
5
- to tensor processors, starting with the Apple Neural Engine (ANE).
6
 
7
- The goal is to provide a fully open-source pipeline
8
- from model conversion to inference for common LLM architectures
9
- running on ANE.
10
 
11
- This enables seamless integration and on-device inference
12
- for low-power applications on edge devices,
13
- ensuring maximum privacy and security.
14
 
15
- This is critical for autonomous applications,
16
- where models run directly on the device
17
- without requiring an internet connection.
18
 
19
- License
20
-
21
- ANEMLL is licensed under the MIT License.
22
- https://opensource.org/license/mit
23
- The model is based on Meta’s LLaMA 3.2 and may require a separate license.
24
 
 
25
 
 
 
26
 
27
- This test model is exclusively for the Meta's LLaMA 3.2 1B (1024 context) model converted for CoreML,
28
- released before the official launch of the ANEMLL repository and minimal documentation.
29
- It is intended for early adopters only who requested an early release.
30
 
 
31
 
32
- Requirements
33
- • macOS Sequoia with Apple Neural Engine and 16GB RAM
34
- • CoreML Tools and HuggingFace Transformers libraries
35
- • Python 3.9
36
 
37
- chat.py provides a sample inference script.
38
- We apologize for the current quality of chat.py and appreciate your patience.
 
39
 
 
 
40
 
41
- Prerequisites:
 
 
42
  pip install coremltools transformers
 
43
 
44
- How to RUN:
45
  python chat.py
46
 
47
  Ctr-D to exit, Ctr-C to interrupt inference.
48
 
49
  alternative way to run:
 
50
  python chat.py S123 -d /path/to/anemll-LLAMA32-1B-ctx1024 ctx=1024
 
51
 
52
  The first time the model loads, macOS will take some time to place it on the device.
53
  Subsequent loads will be instantaneous.
54
 
 
 
55
  Please check following links for later updates:
56
- https://huggingface.co/anemll
57
- https://x.com/anemll
58
- https://github.com/anemll
59
- https://anemll.com
 
60
 
61
62
 
 
1
+ # ANEMLL
2
 
3
+ **ANEMLL** (pronounced like “animal”) is an open-source project focused on accelerating the porting of Large Language Models (LLMs) to tensor processors, starting with the Apple Neural Engine (ANE).
 
 
4
 
5
+ The goal is to provide a fully open-source pipeline from model conversion to inference for common LLM architectures running on ANE.
 
 
6
 
7
+ This enables seamless integration and on-device inference for low-power applications on edge devices, ensuring maximum privacy and security.
 
 
8
 
9
+ This is critical for autonomous applications, where models run directly on the device without requiring an internet connection.
 
 
10
 
11
+ ---
 
 
 
 
12
 
13
+ ## License
14
 
15
+ ANEMLL is licensed under the [MIT License](https://opensource.org/license/mit).
16
+ The model is based on Meta’s LLaMA 3.2 and may require a separate license.
17
 
18
+ This test model is exclusively for the Meta's LLaMA 3.2 1B (1024 context) model converted for CoreML, released before the official launch of the ANEMLL repository and minimal documentation. It is intended for early adopters only who requested an early release.
 
 
19
 
20
+ ---
21
 
22
+ ## Requirements
 
 
 
23
 
24
+ - **macOS Sequoia** with Apple Neural Engine and 16GB RAM
25
+ - **CoreML Tools** and **HuggingFace Transformers** libraries
26
+ - **Python 3.9**
27
 
28
+ `chat.py` provides a sample inference script.
29
+ *We apologize for the current quality of `chat.py` and appreciate your patience.*
30
 
31
+ #Installation
32
+ Unzip all ZIP files with CoreML Models files
33
+ ```bash
34
  pip install coremltools transformers
35
+ ```
36
 
37
+ #How to RUN:
38
  python chat.py
39
 
40
  Ctr-D to exit, Ctr-C to interrupt inference.
41
 
42
  alternative way to run:
43
+ ```bash
44
  python chat.py S123 -d /path/to/anemll-LLAMA32-1B-ctx1024 ctx=1024
45
+ ```
46
 
47
  The first time the model loads, macOS will take some time to place it on the device.
48
  Subsequent loads will be instantaneous.
49
 
50
+
51
+ ##
52
  Please check following links for later updates:
53
+
54
+ https://huggingface.co/anemll
55
+ https://x.com/anemll
56
+ https://github.com/anemll
57
+ • https://anemll.com
58
 
59
60