Improve model card with metadata and details
Browse filesThis PR improves the model card by adding the necessary metadata (pipeline tag, library name, license) and populating some sections with information from the paper and GitHub README. It also adds relevant tags to improve searchability.
README.md
CHANGED
@@ -1,199 +1,125 @@
|
|
1 |
---
|
2 |
library_name: transformers
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
4 |
---
|
5 |
|
6 |
-
# Model Card for
|
7 |
-
|
8 |
-
<!-- Provide a quick summary of what the model is/does. -->
|
9 |
|
|
|
10 |
|
11 |
|
12 |
## Model Details
|
13 |
|
14 |
### Model Description
|
15 |
|
16 |
-
|
17 |
-
|
18 |
-
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
|
19 |
|
20 |
-
- **Developed by:**
|
21 |
-
- **
|
22 |
-
- **
|
23 |
-
- **
|
24 |
-
- **
|
25 |
-
- **License:** [More Information Needed]
|
26 |
-
- **Finetuned from model [optional]:** [More Information Needed]
|
27 |
|
28 |
-
### Model Sources
|
29 |
|
30 |
-
|
|
|
31 |
|
32 |
-
- **Repository:** [More Information Needed]
|
33 |
-
- **Paper [optional]:** [More Information Needed]
|
34 |
-
- **Demo [optional]:** [More Information Needed]
|
35 |
|
36 |
## Uses
|
37 |
|
38 |
-
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
39 |
-
|
40 |
### Direct Use
|
41 |
|
42 |
-
|
43 |
-
|
44 |
-
[More Information Needed]
|
45 |
-
|
46 |
-
### Downstream Use [optional]
|
47 |
|
48 |
-
|
49 |
|
50 |
-
|
51 |
|
52 |
### Out-of-Scope Use
|
53 |
|
54 |
-
|
55 |
|
56 |
-
[More Information Needed]
|
57 |
|
58 |
## Bias, Risks, and Limitations
|
59 |
|
60 |
-
|
61 |
-
|
62 |
-
[More Information Needed]
|
63 |
|
64 |
### Recommendations
|
65 |
|
66 |
-
|
67 |
|
68 |
-
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
69 |
|
70 |
## How to Get Started with the Model
|
71 |
|
72 |
-
|
73 |
|
74 |
-
[More Information Needed]
|
75 |
|
76 |
## Training Details
|
77 |
|
78 |
### Training Data
|
79 |
|
80 |
-
|
81 |
-
|
82 |
-
[More Information Needed]
|
83 |
|
84 |
### Training Procedure
|
85 |
|
86 |
-
|
87 |
-
|
88 |
-
#### Preprocessing [optional]
|
89 |
-
|
90 |
-
[More Information Needed]
|
91 |
-
|
92 |
|
93 |
#### Training Hyperparameters
|
94 |
|
95 |
-
|
96 |
|
97 |
-
#### Speeds, Sizes, Times
|
98 |
|
99 |
-
|
100 |
|
101 |
-
[More Information Needed]
|
102 |
|
103 |
## Evaluation
|
104 |
|
105 |
-
<!-- This section describes the evaluation protocols and provides the results. -->
|
106 |
-
|
107 |
### Testing Data, Factors & Metrics
|
108 |
|
109 |
#### Testing Data
|
110 |
|
111 |
-
|
112 |
-
|
113 |
-
[More Information Needed]
|
114 |
|
115 |
#### Factors
|
116 |
|
117 |
-
|
118 |
-
|
119 |
-
[More Information Needed]
|
120 |
|
121 |
#### Metrics
|
122 |
|
123 |
-
|
124 |
-
|
125 |
-
[More Information Needed]
|
126 |
|
127 |
### Results
|
128 |
|
129 |
-
[
|
130 |
|
131 |
#### Summary
|
132 |
|
|
|
133 |
|
134 |
|
135 |
-
##
|
136 |
-
|
137 |
-
<!-- Relevant interpretability work for the model goes here -->
|
138 |
-
|
139 |
-
[More Information Needed]
|
140 |
-
|
141 |
-
## Environmental Impact
|
142 |
-
|
143 |
-
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
144 |
-
|
145 |
-
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
146 |
-
|
147 |
-
- **Hardware Type:** [More Information Needed]
|
148 |
-
- **Hours used:** [More Information Needed]
|
149 |
-
- **Cloud Provider:** [More Information Needed]
|
150 |
-
- **Compute Region:** [More Information Needed]
|
151 |
-
- **Carbon Emitted:** [More Information Needed]
|
152 |
-
|
153 |
-
## Technical Specifications [optional]
|
154 |
-
|
155 |
-
### Model Architecture and Objective
|
156 |
-
|
157 |
-
[More Information Needed]
|
158 |
-
|
159 |
-
### Compute Infrastructure
|
160 |
-
|
161 |
-
[More Information Needed]
|
162 |
-
|
163 |
-
#### Hardware
|
164 |
-
|
165 |
-
[More Information Needed]
|
166 |
-
|
167 |
-
#### Software
|
168 |
-
|
169 |
-
[More Information Needed]
|
170 |
-
|
171 |
-
## Citation [optional]
|
172 |
-
|
173 |
-
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
174 |
|
175 |
**BibTeX:**
|
176 |
|
177 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
178 |
|
179 |
**APA:**
|
180 |
|
181 |
-
[
|
182 |
-
|
183 |
-
## Glossary [optional]
|
184 |
-
|
185 |
-
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
186 |
-
|
187 |
-
[More Information Needed]
|
188 |
-
|
189 |
-
## More Information [optional]
|
190 |
-
|
191 |
-
[More Information Needed]
|
192 |
-
|
193 |
-
## Model Card Authors [optional]
|
194 |
-
|
195 |
-
[More Information Needed]
|
196 |
-
|
197 |
-
## Model Card Contact
|
198 |
-
|
199 |
-
[More Information Needed]
|
|
|
1 |
---
|
2 |
library_name: transformers
|
3 |
+
pipeline_tag: text-generation
|
4 |
+
license: apache-2.0
|
5 |
+
tags:
|
6 |
+
- self-calibration
|
7 |
+
- confidence-estimation
|
8 |
+
- test-time-scaling
|
9 |
---
|
10 |
|
11 |
+
# Model Card for Efficient Test-Time Scaling via Self-Calibration
|
|
|
|
|
12 |
|
13 |
+
This model implements an efficient test-time scaling method using model confidence for dynamic sampling adjustment. Higher confidence responses have a greater influence on the final answer, leading to improved computational efficiency. The model uses a self-calibration framework to generate more calibrated confidence scores.
|
14 |
|
15 |
|
16 |
## Model Details
|
17 |
|
18 |
### Model Description
|
19 |
|
20 |
+
This model utilizes a self-calibration framework to generate calibrated confidence scores, which are then used to improve the efficiency of test-time scaling methods. This allows for comparable performance with substantially fewer computational resources.
|
|
|
|
|
21 |
|
22 |
+
- **Developed by:** HINT-lab
|
23 |
+
- **Model type:** Large Language Model
|
24 |
+
- **Language(s) (NLP):** English
|
25 |
+
- **License:** Apache-2.0
|
26 |
+
- **Finetuned from model [optional]:** [Specify base model here, e.g., `meta-llama/Llama-3.1-8B-Instruct`]
|
|
|
|
|
27 |
|
28 |
+
### Model Sources
|
29 |
|
30 |
+
- **Repository:** https://github.com/HINT-lab/Efficient-Test-Time-Scaling
|
31 |
+
- **Paper:** [Efficient Test-Time Scaling via Self-Calibration](https://arxiv.org/abs/2503.00031)
|
32 |
|
|
|
|
|
|
|
33 |
|
34 |
## Uses
|
35 |
|
|
|
|
|
36 |
### Direct Use
|
37 |
|
38 |
+
The model can be used directly for text generation tasks, leveraging its self-calibration capabilities for improved efficiency.
|
|
|
|
|
|
|
|
|
39 |
|
40 |
+
### Downstream Use
|
41 |
|
42 |
+
The calibrated confidence scores generated by the model can be incorporated into various test-time scaling methods (e.g., Self-Consistency, Best-of-N) to enhance their performance and reduce computational costs.
|
43 |
|
44 |
### Out-of-Scope Use
|
45 |
|
46 |
+
The model is not intended for tasks requiring high accuracy in scenarios where confidence calibration is not crucial.
|
47 |
|
|
|
48 |
|
49 |
## Bias, Risks, and Limitations
|
50 |
|
51 |
+
The model's performance and calibration accuracy may vary depending on the specific dataset and task. Like other LLMs, it may exhibit biases present in its training data.
|
|
|
|
|
52 |
|
53 |
### Recommendations
|
54 |
|
55 |
+
Users should be aware of the potential biases and limitations of the model and carefully evaluate its performance on their specific tasks. Further investigation into bias mitigation techniques is recommended.
|
56 |
|
|
|
57 |
|
58 |
## How to Get Started with the Model
|
59 |
|
60 |
+
See the [GitHub README](https://github.com/HINT-lab/Efficient-Test-Time-Scaling) for detailed instructions.
|
61 |
|
|
|
62 |
|
63 |
## Training Details
|
64 |
|
65 |
### Training Data
|
66 |
|
67 |
+
[Link to Hugging Face Dataset, if available. Otherwise, provide a brief description]
|
|
|
|
|
68 |
|
69 |
### Training Procedure
|
70 |
|
71 |
+
See the training section in the [GitHub README](https://github.com/HINT-lab/Efficient-Test-Time-Scaling).
|
|
|
|
|
|
|
|
|
|
|
72 |
|
73 |
#### Training Hyperparameters
|
74 |
|
75 |
+
[Information from GitHub README regarding hyperparameters]
|
76 |
|
77 |
+
#### Speeds, Sizes, Times
|
78 |
|
79 |
+
[Information from GitHub README about training times, model sizes, etc.]
|
80 |
|
|
|
81 |
|
82 |
## Evaluation
|
83 |
|
|
|
|
|
84 |
### Testing Data, Factors & Metrics
|
85 |
|
86 |
#### Testing Data
|
87 |
|
88 |
+
[Link to Hugging Face Dataset, if available. Otherwise, provide a brief description]
|
|
|
|
|
89 |
|
90 |
#### Factors
|
91 |
|
92 |
+
[List factors from GitHub README, e.g., different datasets]
|
|
|
|
|
93 |
|
94 |
#### Metrics
|
95 |
|
96 |
+
[List metrics used from GitHub README, e.g., accuracy]
|
|
|
|
|
97 |
|
98 |
### Results
|
99 |
|
100 |
+
[Summary of results from GitHub README]
|
101 |
|
102 |
#### Summary
|
103 |
|
104 |
+
[Concise summary of overall evaluation performance]
|
105 |
|
106 |
|
107 |
+
## Citation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
108 |
|
109 |
**BibTeX:**
|
110 |
|
111 |
+
```bibtex
|
112 |
+
@misc{huang2025efficienttesttimescalingselfcalibration,
|
113 |
+
title={Efficient Test-Time Scaling via Self-Calibration},
|
114 |
+
author={Chengsong Huang and Langlin Huang and Jixuan Leng and Jiacheng Liu and Jiaxin Huang},
|
115 |
+
year={2025},
|
116 |
+
eprint={2503.00031},
|
117 |
+
archivePrefix={arXiv},
|
118 |
+
primaryClass={cs.LG},
|
119 |
+
url={https://arxiv.org/abs/2503.00031},
|
120 |
+
}
|
121 |
+
```
|
122 |
|
123 |
**APA:**
|
124 |
|
125 |
+
[APA citation for the paper - Needs to be constructed based on the paper's full details]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|