MTSAIR
/

Llama3.1-6B-ReplaceMe

Model card Files Files and versions

ammarali32 commited on Apr 29

Commit

b85eced

·

verified ·

1 Parent(s): 70badd0

Update README.md

Files changed (1) hide show

README.md +6 -6

README.md CHANGED Viewed

@@ -25,12 +25,12 @@ ReplaceMe is a novel method for transformer model compression that enables **tra
 ## 🔥 Performance Comparison of Pruning Methods (Llama 2 7B, 25% Compression)
-| Method       | num_pruned_layers | Dataset    | State         | race 🏁 | winogrande 🎲 | piqa 🧠 | boolq ❓ | openbookqa 📖 | sciq 🔬 | lambada_openai 🦙 | Avg-acc 📊 |
-|--------------|-------------------|------------|---------------|--------|--------------|--------|---------|--------------|--------|------------------|------------|
-|              |                   |            |               | acc    | acc          | acc_norm | acc     | acc_norm     | acc_norm | acc              | acc        |
-| **Llama 3.1** (baseline) | - | - | - | 0.449761 | 0.779006 | 0.809576 | 0.84159 | 0.43 | 0.961 | 0.732195 | 3.403683 | **0.711822** |
-| **UIDL*** | - | 8 | slim_orca | no training | 0.34067 | 0.719021 | 0.68988 | 0.773394 | 0.31 | 0.719 | 0.087328 | 932.0 | 0.591994 |
-| **ReplaceMe** (Ours) ✅ | Cosine | 8 | slim_orca | no training | **0.405742**🏆 | **0.74191** 🏆| **0.705658** 🏆| **0.830275** 🏆| **0.338** 🏆| **0.901** 🏆| **0.470794** 🏆| 16.759605 🏆| **0.653764** 🏆 |
 **Key:**

 ## 🔥 Performance Comparison of Pruning Methods (Llama 2 7B, 25% Compression)
+| Method                | num_pruned_layers | Dataset    | State         | race 🏁 | winogrande 🎲 | piqa 🧠 | boolq ❓ | openbookqa 📖 | sciq 🔬 | lambada_openai 🦙 | ppl       | Avg-acc 📊 |
+|-----------------------|-------------------|------------|---------------|--------|--------------|--------|---------|--------------|--------|------------------|-----------|------------|
+|                       |                   |            |               | acc    | acc          | acc_norm | acc     | acc_norm     | acc_norm | acc              |           |            |
+| **Llama 3.1** (baseline) | -               | -          | -             | 0.450  | 0.779        | 0.810   | 0.842   | 0.430        | 0.961  | 0.732            | 3.404     | **0.712**  |
+| **UIDL***            | 8                 | slim_orca  | no training   | 0.341  | 0.719        | 0.690   | 0.773   | 0.310        | 0.719  | 0.087            | 932.000   | 0.592      |
+| **ReplaceMe** (Ours) ✅ | 8                | slim_orca  | no training   | **0.406** 🏆 | **0.742** 🏆 | **0.706** 🏆 | **0.830** 🏆 | **0.338** 🏆 | **0.901** 🏆 | **0.471** 🏆 | 16.760 🏆 | **0.654** 🏆 |
 **Key:**