Upload 3 files
Browse filesThis commit adds the trained logistic regression (LR) probing model files used in our paper:
**"Sentinel: Attention Probing of Proxy Models for LLM Context Compression with an Understanding Perspective"**
These models perform sentence-level relevance prediction based on decoder attention features from a 0.5B-scale proxy model (Qwen2.5-0.5B-Instruct). The logistic regression classifier is trained on attention-derived features from SQuAD, NewsQA, and HotpotQA under weak supervision.
📂 Files:
- `.pkl`: Serialized LR classifier weights
- `.json`: Feature configuration used during probing
These files correspond to the setup described in Section 2.3 of the paper. Use them to replicate the results in our LongBench evaluations.
📄 Paper: https://arxiv.org/abs/2505.23277
🔗 Code: https://github.com/yzhangchuck/Sentinel
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"19": [
|
| 3 |
+
6,
|
| 4 |
+
2
|
| 5 |
+
],
|
| 6 |
+
"3": [
|
| 7 |
+
4
|
| 8 |
+
],
|
| 9 |
+
"15": [
|
| 10 |
+
12
|
| 11 |
+
],
|
| 12 |
+
"0": [
|
| 13 |
+
5,
|
| 14 |
+
7,
|
| 15 |
+
9
|
| 16 |
+
],
|
| 17 |
+
"8": [
|
| 18 |
+
3
|
| 19 |
+
],
|
| 20 |
+
"20": [
|
| 21 |
+
12
|
| 22 |
+
],
|
| 23 |
+
"11": [
|
| 24 |
+
7
|
| 25 |
+
],
|
| 26 |
+
"16": [
|
| 27 |
+
3,
|
| 28 |
+
7
|
| 29 |
+
],
|
| 30 |
+
"18": [
|
| 31 |
+
13
|
| 32 |
+
]
|
| 33 |
+
}
|
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6021b0aca86f909ebee4ec9675abd6677114ccd2140ed6ce451a40c519897ae5
|
| 3 |
+
size 105439
|
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:73cc99327606b25a8f9752f5c80fbd7ea5be8c323b928c388591163d7626de8a
|
| 3 |
+
size 107984
|