--- base_model: sentence-transformers/all-mpnet-base-v2 library_name: setfit metrics: - f1 pipeline_tag: text-classification tags: - setfit - absa - sentence-transformers - text-classification - generated_from_setfit_trainer widget: - text: bargain:Monday nights are a bargain at the $28 prix fix - this includes a three course meal plus *three* glasses of wine paired with each course. - text: seated:We walked in on a Wednesday night and were seated promptly. - text: drinks:While most people can attest to spending over $50 on drinks in New York bars and hardly feeling a thing, the drinks here are plentiful and unique. - text: Lassi:I ordered a Lassi and asked 4 times for it but never got it. - text: stomach:Check it out, it won't hurt your stomach or your wallet. inference: false model-index: - name: SetFit Aspect Model with sentence-transformers/all-mpnet-base-v2 results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: f1 value: 0.923076923076923 name: F1 --- # SetFit Aspect Model with sentence-transformers/all-mpnet-base-v2 This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A [SetFitHead](huggingface.co/docs/setfit/reference/main#setfit.SetFitHead) instance is used for classification. In particular, this model is in charge of filtering aspect span candidates. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. This model was trained within the context of a larger system for ABSA, which looks like so: 1. Use a spaCy model to select possible aspect span candidates. 2. **Use this SetFit model to filter these possible aspect span candidates.** 3. Use a SetFit model to classify the filtered aspect span candidates. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) - **Classification head:** a [SetFitHead](huggingface.co/docs/setfit/reference/main#setfit.SetFitHead) instance - **spaCy Model:** en_core_web_trf - **SetFitABSA Aspect Model:** [MattiaTintori/Final_aspect_Colab](https://huggingface.co/MattiaTintori/Final_aspect_Colab) - **SetFitABSA Polarity Model:** [setfit-absa-polarity](https://huggingface.co/setfit-absa-polarity) - **Maximum Sequence Length:** 384 tokens - **Number of Classes:** 2 classes <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown --> ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:----------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | aspect | <ul><li>'price:The price is reasonable although the service is poor.'</li><li>'service:The price is reasonable although the service is poor.'</li><li>'service:The place is so cool and the service is prompt and curtious.'</li></ul> | | no aspect | <ul><li>'stomach:The food was delicious but do not come here on a empty stomach.'</li><li>'place:I grew up eating Dosa and have yet to find a place in NY to satisfy my taste buds.'</li><li>'NY:I grew up eating Dosa and have yet to find a place in NY to satisfy my taste buds.'</li></ul> | ## Evaluation ### Metrics | Label | F1 | |:--------|:-------| | **all** | 0.9231 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import AbsaModel # Download from the 🤗 Hub model = AbsaModel.from_pretrained( "MattiaTintori/Final_aspect_Colab", "setfit-absa-polarity", ) # Run inference preds = model("The food was great, but the venue is just way too busy.") ``` <!-- ### Downstream Use *List how someone could finetune this model on their own dataset.* --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* --> ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:--------|:----| | Word count | 3 | 19.4137 | 62 | | Label | Training Sample Count | |:----------|:----------------------| | no aspect | 430 | | aspect | 711 | ### Training Hyperparameters - batch_size: (64, 4) - num_epochs: (5, 32) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 10 - body_learning_rate: (8e-05, 8e-05) - head_learning_rate: 0.04 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: True - warmup_proportion: 0.1 - l2_weight: 0.01 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: True ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:----------:|:------:|:-------------:|:---------------:| | 0.0028 | 1 | 0.2878 | - | | 0.0560 | 20 | 0.2409 | 0.2515 | | 0.1120 | 40 | 0.2291 | 0.2319 | | 0.1681 | 60 | 0.1354 | 0.1835 | | **0.2241** | **80** | **0.0654** | **0.1389** | | 0.2801 | 100 | 0.0334 | 0.1818 | | 0.3361 | 120 | 0.0535 | 0.1408 | | 0.3922 | 140 | 0.014 | 0.1564 | | 0.4482 | 160 | 0.0119 | 0.1453 | | 0.5042 | 180 | 0.0158 | 0.1511 | | 0.5602 | 200 | 0.0157 | 0.1393 | | 0.6162 | 220 | 0.005 | 0.1536 | | 0.6723 | 240 | 0.0002 | 0.1546 | | 0.7283 | 260 | 0.0002 | 0.1673 | | 0.7843 | 280 | 0.0004 | 0.1655 | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.12 - SetFit: 1.0.3 - Sentence Transformers: 3.0.1 - spaCy: 3.7.6 - Transformers: 4.39.0 - PyTorch: 2.3.1+cu121 - Datasets: 2.21.0 - Tokenizers: 0.15.2 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ``` <!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->