SetFit with sentence-transformers/all-mpnet-base-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
Model Sources
Model Labels
Label |
Examples |
yes |
- '40K Americans a year die in auto accidents. Countless more die from cancer, guns, suicide. Many of these deaths are preventable but you can still buy cigarettes and guns. Yesterday Ken Block died while snowmobiling -- he is most famous for his automotive exploits, either driving in rally stages or in gymkhana stunt driving videos. Jeremy Renner was hit by a plow truck and airlifted to a hospital yesterday.You can look at anything Red Bull sponsors and see people BASE jumping, wearing squirrel suits flying through mountain passes, or free climbing at deathly heights. They even sponsored a parachute drop from space.Athletes have died while playing other sports that are seemingly "safe". Kids still play with aluminum bats. Hockey players routinely check each other. Men's lacrosse has pads and allows physical contact, not quite as much as football, but not far off, either. MMA leagues are very popular, too -- and feature men and women both. Not to mention boxing, which fills arenas with title matches.\n'
- 'My one and only flight on a 74 was from NYC to Amsterdam, Amsterdam to Dubai. We stopped in Amsterdam for 90 minutes while they did a security sweep, got on the same plane to Dubai. That plane barring unforeseen tragedy will fly on at least for 40 years.\n'
- 'That is a pretty darned low body count compared to independent estimates.Near the end of December it was reported that..."Around 9,000 people in China are probably dying each day from COVID-19, UK-based health data firm Airfinity "Based on a rate of 9k per day over the same period, the Chinese Government estimate should be closer to 315,000 deaths due to COVID\n'
|
no |
- 'A cautionary story: My mother and my brother are both alcoholics. Several years apart they each suffered a traumatic brain injury while drinking. Within a few years, due to their brain injuries, they each developed vascular dementia. They now reside in the same care facility. It is endless sadness thinking of the lives they should be leading, and everything that has been lost to alcohol.\n'
- "I've concluded that if states such as Mississippi, Alabama, Kentucky, Arkansas, Louisiana, South Carolina, Nebraska, et al, want to leave the Union, they should. And, yes, that includes Texas as well. If one looks at critical performance levels such as life expectancy, infant mortality, income, educational levels, access to health care and more (even voter participation levels), invariably this group ranks at the bottom. And, if one looks at federal spending levels, these states receive more than they contribute. Essentially, their ROI is terrible. Yet, what they have in their states is what they envision for the country as a whole. No, thanks. Let them go. I think that is what they'd prefer anyway. How they provide for their common defense and promote their own general welfare is up to them.\n"
- 'Unless the US gov't has borrowed euros or rubles or renminbi that we don't know about, it has zero "debt." What we're looking at is $31.5 trillion of ACCUMULATED DEFICITS expressed in US dollars. Why would the US Treasury/Federal Reserve ever have to borrow money that it can create at will out of thin air? And "debt" implies that we're going to pay it back, which is an absurd thought. Balances have to balance. If the public sector of our economy shows a deficit of $31.5 trillion, this means by force of logic that the private sector (plus the rest of the world) must show a SURPLUS of $31.5 trillion. That's our aggregate financial wealth. Trying to pay back the "debt" would drive us all into extreme poverty.BTW, it's one thing to say that the Fed is raising interest rates to fight inflation, and it's quite another thing (and incorrect) to claim the market is demanding higher rates. That is simply nonsense. The Fed sets interest rates and if you don't like it you can just go pound sand. The Fed doesn't sell bonds to "borrow" back the money it has just spent into the economy. It does so to remove excess currency from the banking system in order to meet its target interest rate. If it didn't, all banks would be flush with currency and never have to borrow from another bank to settle its accounts. This would drive interest rates down to zero.The notion that the gov't borrows its own currency and accumulates 'debt" is a zombie idea that just refuses to die.\n'
|
Evaluation
Metrics
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
model = SetFitModel.from_pretrained("davidadamczyk/setfit-model-6")
preds = model("Jen that was a prop plane.in Buffalo....but still awfulAlso there was a Delta jet accident in 2006 on kentucky ...the plane took of on the wrong runway...49 killed
")
Training Details
Training Set Metrics
Training set |
Min |
Median |
Max |
Word count |
9 |
127.2 |
277 |
Label |
Training Sample Count |
no |
18 |
yes |
22 |
Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 120
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
Epoch |
Step |
Training Loss |
Validation Loss |
0.0017 |
1 |
0.4205 |
- |
0.0833 |
50 |
0.1936 |
- |
0.1667 |
100 |
0.0058 |
- |
0.25 |
150 |
0.0003 |
- |
0.3333 |
200 |
0.0002 |
- |
0.4167 |
250 |
0.0001 |
- |
0.5 |
300 |
0.0001 |
- |
0.5833 |
350 |
0.0001 |
- |
0.6667 |
400 |
0.0001 |
- |
0.75 |
450 |
0.0001 |
- |
0.8333 |
500 |
0.0001 |
- |
0.9167 |
550 |
0.0001 |
- |
1.0 |
600 |
0.0001 |
- |
Framework Versions
- Python: 3.10.13
- SetFit: 1.1.0
- Sentence Transformers: 3.0.1
- Transformers: 4.45.2
- PyTorch: 2.4.0+cu124
- Datasets: 2.21.0
- Tokenizers: 0.20.0
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}