PEFT
Safetensors
English
jinjieyuan commited on
Commit
abb3b22
·
verified ·
1 Parent(s): 724904c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -3,7 +3,7 @@ language: en
3
  license: apache-2.0
4
  ---
5
 
6
- # Shears Model Card: shears-llama-7b-50-commonsense-super-adapter
7
 
8
  The super-adapter fine-tuned on sparsified LLaMA-7B with some commonsense reasoning datasets using Shears.
9
 
@@ -13,7 +13,7 @@ The release of the super-network is to facilitate users to apply their own searc
13
 
14
  ### Information
15
 
16
- - **Model name:** shears-llama-7b-50-commonsense-super-adapter
17
  - **Base model:** [IntelLabs/shears-llama-7b-50-base](https://huggingface.co/IntelLabs/shears-llama-7b-50-base)
18
  - **Sparsity:** 50%
19
  - **Domain:** Commonsense
@@ -65,7 +65,7 @@ def generate_prompt(instruction):
65
  """
66
 
67
  base_model = AutoModelForCausalLM.from_pretrained("IntelLabs/shears-llama-7b-50-base")
68
- model = PeftModel.from_pretrained(base_model, "IntelLabs/shears-llama-7b-50-commonsense-super-adapter")
69
  model.eval()
70
 
71
  non_zero_params = sum([(param.data != 0).sum().item() for _, param in model.named_parameters()])
@@ -101,7 +101,7 @@ Results of the heuristic sub-network discoverd from the super-network:
101
  |----------------------|-----------|---------|--------|--------|------------|--------|--------|---------|--------|----------|
102
  | ChatGPT | - | 73.1 | 85.4 | 68.5 | 78.5 | 66.1 | 89.8 | 79.9 | 74.8 | 77.0 |
103
  | LLaMA-7B-LoRA | - | 68.9 | 80.7 | 77.4 | 78.1 | 78.8 | 77.8 | 61.3 | 74.8 | 74.7 |
104
- | [**LLaMA-7B-Shears**](https://huggingface.co/IntelLabs/shears-llama-7b-50-commonsense-heuristic-adapter) | **50%** | 67.3 | 79.1 | 77.5 | 73.3 | 77.7 | 74.4 | 57.9 | 72.8 | 72.5 |
105
 
106
  ## Model Sources
107
 
 
3
  license: apache-2.0
4
  ---
5
 
6
+ # Shears Model Card: shears-llama-7b-50-cs-super-adapter
7
 
8
  The super-adapter fine-tuned on sparsified LLaMA-7B with some commonsense reasoning datasets using Shears.
9
 
 
13
 
14
  ### Information
15
 
16
+ - **Model name:** shears-llama-7b-50-cs-super-adapter
17
  - **Base model:** [IntelLabs/shears-llama-7b-50-base](https://huggingface.co/IntelLabs/shears-llama-7b-50-base)
18
  - **Sparsity:** 50%
19
  - **Domain:** Commonsense
 
65
  """
66
 
67
  base_model = AutoModelForCausalLM.from_pretrained("IntelLabs/shears-llama-7b-50-base")
68
+ model = PeftModel.from_pretrained(base_model, "IntelLabs/shears-llama-7b-50-cs-super-adapter")
69
  model.eval()
70
 
71
  non_zero_params = sum([(param.data != 0).sum().item() for _, param in model.named_parameters()])
 
101
  |----------------------|-----------|---------|--------|--------|------------|--------|--------|---------|--------|----------|
102
  | ChatGPT | - | 73.1 | 85.4 | 68.5 | 78.5 | 66.1 | 89.8 | 79.9 | 74.8 | 77.0 |
103
  | LLaMA-7B-LoRA | - | 68.9 | 80.7 | 77.4 | 78.1 | 78.8 | 77.8 | 61.3 | 74.8 | 74.7 |
104
+ | [**LLaMA-7B-Shears**](https://huggingface.co/IntelLabs/shears-llama-7b-50-cs-heuristic-adapter) | **50%** | 67.3 | 79.1 | 77.5 | 73.3 | 77.7 | 74.4 | 57.9 | 72.8 | 72.5 |
105
 
106
  ## Model Sources
107