ReRaWo commited on
Commit
d07f14c
·
verified ·
1 Parent(s): dceac73

Upload 3 files

Browse files

This commit adds the trained logistic regression (LR) probing model files used in our paper:

**"Sentinel: Attention Probing of Proxy Models for LLM Context Compression with an Understanding Perspective"**

These models perform sentence-level relevance prediction based on decoder attention features from a 0.5B-scale proxy model (Qwen2.5-0.5B-Instruct). The logistic regression classifier is trained on attention-derived features from SQuAD, NewsQA, and HotpotQA under weak supervision.

📂 Files:
- `.pkl`: Serialized LR classifier weights
- `.json`: Feature configuration used during probing

These files correspond to the setup described in Section 2.3 of the paper. Use them to replicate the results in our LongBench evaluations.

📄 Paper: https://arxiv.org/abs/2505.23277
🔗 Code: https://github.com/yzhangchuck/Sentinel

qwen2.5-0.5b-instruct-3000_all_layer_last_token_20250515_033938_model.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "19": [
3
+ 6,
4
+ 2
5
+ ],
6
+ "3": [
7
+ 4
8
+ ],
9
+ "15": [
10
+ 12
11
+ ],
12
+ "0": [
13
+ 5,
14
+ 7,
15
+ 9
16
+ ],
17
+ "8": [
18
+ 3
19
+ ],
20
+ "20": [
21
+ 12
22
+ ],
23
+ "11": [
24
+ 7
25
+ ],
26
+ "16": [
27
+ 3,
28
+ 7
29
+ ],
30
+ "18": [
31
+ 13
32
+ ]
33
+ }
qwen2.5-0.5b-instruct-3000_all_layer_last_token_20250515_033938_model.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6021b0aca86f909ebee4ec9675abd6677114ccd2140ed6ce451a40c519897ae5
3
+ size 105439
qwen2.5-0.5b-instruct-3000_all_layer_last_token_qwen2_20250507_212739_model.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73cc99327606b25a8f9752f5c80fbd7ea5be8c323b928c388591163d7626de8a
3
+ size 107984