Update README.md
Browse files
README.md
CHANGED
@@ -3,7 +3,7 @@ language: en
|
|
3 |
license: apache-2.0
|
4 |
---
|
5 |
|
6 |
-
# Shears Model Card: shears-llama-7b-50-
|
7 |
|
8 |
The super-adapter fine-tuned on sparsified LLaMA-7B with some commonsense reasoning datasets using Shears.
|
9 |
|
@@ -13,7 +13,7 @@ The release of the super-network is to facilitate users to apply their own searc
|
|
13 |
|
14 |
### Information
|
15 |
|
16 |
-
- **Model name:** shears-llama-7b-50-
|
17 |
- **Base model:** [IntelLabs/shears-llama-7b-50-base](https://huggingface.co/IntelLabs/shears-llama-7b-50-base)
|
18 |
- **Sparsity:** 50%
|
19 |
- **Domain:** Commonsense
|
@@ -65,7 +65,7 @@ def generate_prompt(instruction):
|
|
65 |
"""
|
66 |
|
67 |
base_model = AutoModelForCausalLM.from_pretrained("IntelLabs/shears-llama-7b-50-base")
|
68 |
-
model = PeftModel.from_pretrained(base_model, "IntelLabs/shears-llama-7b-50-
|
69 |
model.eval()
|
70 |
|
71 |
non_zero_params = sum([(param.data != 0).sum().item() for _, param in model.named_parameters()])
|
@@ -101,7 +101,7 @@ Results of the heuristic sub-network discoverd from the super-network:
|
|
101 |
|----------------------|-----------|---------|--------|--------|------------|--------|--------|---------|--------|----------|
|
102 |
| ChatGPT | - | 73.1 | 85.4 | 68.5 | 78.5 | 66.1 | 89.8 | 79.9 | 74.8 | 77.0 |
|
103 |
| LLaMA-7B-LoRA | - | 68.9 | 80.7 | 77.4 | 78.1 | 78.8 | 77.8 | 61.3 | 74.8 | 74.7 |
|
104 |
-
| [**LLaMA-7B-Shears**](https://huggingface.co/IntelLabs/shears-llama-7b-50-
|
105 |
|
106 |
## Model Sources
|
107 |
|
|
|
3 |
license: apache-2.0
|
4 |
---
|
5 |
|
6 |
+
# Shears Model Card: shears-llama-7b-50-cs-super-adapter
|
7 |
|
8 |
The super-adapter fine-tuned on sparsified LLaMA-7B with some commonsense reasoning datasets using Shears.
|
9 |
|
|
|
13 |
|
14 |
### Information
|
15 |
|
16 |
+
- **Model name:** shears-llama-7b-50-cs-super-adapter
|
17 |
- **Base model:** [IntelLabs/shears-llama-7b-50-base](https://huggingface.co/IntelLabs/shears-llama-7b-50-base)
|
18 |
- **Sparsity:** 50%
|
19 |
- **Domain:** Commonsense
|
|
|
65 |
"""
|
66 |
|
67 |
base_model = AutoModelForCausalLM.from_pretrained("IntelLabs/shears-llama-7b-50-base")
|
68 |
+
model = PeftModel.from_pretrained(base_model, "IntelLabs/shears-llama-7b-50-cs-super-adapter")
|
69 |
model.eval()
|
70 |
|
71 |
non_zero_params = sum([(param.data != 0).sum().item() for _, param in model.named_parameters()])
|
|
|
101 |
|----------------------|-----------|---------|--------|--------|------------|--------|--------|---------|--------|----------|
|
102 |
| ChatGPT | - | 73.1 | 85.4 | 68.5 | 78.5 | 66.1 | 89.8 | 79.9 | 74.8 | 77.0 |
|
103 |
| LLaMA-7B-LoRA | - | 68.9 | 80.7 | 77.4 | 78.1 | 78.8 | 77.8 | 61.3 | 74.8 | 74.7 |
|
104 |
+
| [**LLaMA-7B-Shears**](https://huggingface.co/IntelLabs/shears-llama-7b-50-cs-heuristic-adapter) | **50%** | 67.3 | 79.1 | 77.5 | 73.3 | 77.7 | 74.4 | 57.9 | 72.8 | 72.5 |
|
105 |
|
106 |
## Model Sources
|
107 |
|