--- library_name: setfit metrics: - accuracy pipeline_tag: text-classification tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer widget: - text: ' A otectos de probar la comisión do la infracción que indobidamento so me imputa, esta parto solicita como medios probatos ¡que se expida y omita por los Órganos correspondientes de la Administración' - text: procedo al estacionamiento por autorización del agente 12289 (Policia laca. Aa vuelta en 5 minutos, me encuentro con una multa LL5898790 por parte del agente 12312 - text: En el momento de la denuncia, mi vehículo contaba con la tarjeta de persona con movilidad reducida colocada en el cristal delantero, lugar habitual donde se posición este tipo de tarjetas. Me sorprende que no se haya tenido en cuenta esta circunstancia. - text: La presunción de veracidad de las denuncias efectuadas por los agentes de la autoridad no es absoluta, ya que es necesario que se aporten pruebas adicionales que respalden la versión de los hechos - text: La sanción carece de una descripción detallada de los hechos, lo que impide conocer la conducta real del denunciado y subsumir el hecho denunciado en el artículo correspondiente. inference: true model-index: - name: SetFit results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: accuracy value: 0.8 name: Accuracy --- # SetFit This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 512 tokens - **Number of Classes:** 20 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 2014 | | | 2001 | | | 2026 | | | 2013 | | | 1001 | | | 304 | | | 237 | | | 2038 | | | 49 | | | 357 | | | 2022 | | | 2017 | | | 78 | | | 2037 | | | 2039 | | | 353 | | | 2002 | | | 2010 | | | 994 | | | 2060 | | ## Evaluation ### Metrics | Label | Accuracy | |:--------|:---------| | **all** | 0.8 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("desarrolloasesoreslocales/bert-leg-al-setfit-grok") # Run inference preds = model("procedo al estacionamiento por autorización del agente 12289 (Policia laca. Aa vuelta en 5 minutos, me encuentro con una multa LL5898790 por parte del agente 12312") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:--------|:----| | Word count | 4 | 44.1625 | 212 | | Label | Training Sample Count | |:------|:----------------------| | 49 | 8 | | 78 | 8 | | 237 | 8 | | 304 | 8 | | 353 | 8 | | 357 | 8 | | 994 | 8 | | 1001 | 8 | | 2001 | 8 | | 2002 | 8 | | 2010 | 8 | | 2013 | 8 | | 2014 | 8 | | 2017 | 8 | | 2022 | 8 | | 2026 | 8 | | 2037 | 8 | | 2038 | 8 | | 2039 | 8 | | 2060 | 8 | ### Training Hyperparameters - batch_size: (16, 16) - num_epochs: (10, 10) - max_steps: -1 - sampling_strategy: oversampling - body_learning_rate: (1e-06, 1e-06) - head_learning_rate: 0.003 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: True - use_amp: True - warmup_proportion: 0.1 - l2_weight: 0.001 - seed: 42 - eval_max_steps: 100 - load_best_model_at_end: True ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:----------:|:-------:|:-------------:|:---------------:| | 0.0007 | 1 | 0.0031 | - | | **0.0658** | **100** | **0.0003** | **0.067** | | 0.1316 | 200 | 0.0001 | 0.0717 | | 0.1974 | 300 | 0.0001 | 0.0711 | | 0.2632 | 400 | 0.0003 | 0.0721 | | 0.3289 | 500 | 0.0021 | 0.0667 | | 0.3947 | 600 | 0.0001 | 0.0611 | | 0.4605 | 700 | 0.0002 | 0.0672 | | 0.5263 | 800 | 0.0001 | 0.0777 | | 0.5921 | 900 | 0.0001 | 0.067 | | 0.6579 | 1000 | 0.0001 | 0.0687 | | 0.7237 | 1100 | 0.0 | 0.0661 | | 0.7895 | 1200 | 0.005 | 0.0695 | | 0.8553 | 1300 | 0.0004 | 0.0661 | | 0.9211 | 1400 | 0.0019 | 0.0667 | | 0.9868 | 1500 | 0.0001 | 0.0672 | | 1.0526 | 1600 | 0.0001 | 0.0714 | | 1.1184 | 1700 | 0.0001 | 0.0687 | | 1.1842 | 1800 | 0.0001 | 0.0723 | | 1.25 | 1900 | 0.0 | 0.0722 | | 1.3158 | 2000 | 0.0001 | 0.0728 | | 1.3816 | 2100 | 0.0 | 0.0713 | | 1.4474 | 2200 | 0.0 | 0.0733 | | 1.5132 | 2300 | 0.0025 | 0.0719 | | 1.5789 | 2400 | 0.0 | 0.0708 | | 1.6447 | 2500 | 0.0 | 0.0722 | | 1.7105 | 2600 | 0.0 | 0.0723 | | 1.7763 | 2700 | 0.0 | 0.069 | | 1.8421 | 2800 | 0.0 | 0.0703 | | 1.9079 | 2900 | 0.0 | 0.0722 | | 1.9737 | 3000 | 0.0001 | 0.0701 | | 2.0395 | 3100 | 0.0 | 0.0691 | | 2.1053 | 3200 | 0.0024 | 0.0706 | | 2.1711 | 3300 | 0.0001 | 0.0716 | | 2.2368 | 3400 | 0.0001 | 0.0886 | | 2.3026 | 3500 | 0.0011 | 0.0734 | | 2.3684 | 3600 | 0.0001 | 0.0875 | | 2.4342 | 3700 | 0.0001 | 0.0809 | | 2.5 | 3800 | 0.0 | 0.0818 | | 2.5658 | 3900 | 0.0001 | 0.0829 | | 2.6316 | 4000 | 0.0 | 0.0833 | | 2.6974 | 4100 | 0.0036 | 0.0841 | | 2.7632 | 4200 | 0.0 | 0.0833 | | 2.8289 | 4300 | 0.0 | 0.0831 | | 2.8947 | 4400 | 0.0374 | 0.083 | | 2.9605 | 4500 | 0.0 | 0.083 | | 3.0263 | 4600 | 0.0001 | 0.0831 | | 3.0921 | 4700 | 0.0 | 0.0829 | | 3.1579 | 4800 | 0.0 | 0.0828 | | 3.2237 | 4900 | 0.0 | 0.0828 | | 3.2895 | 5000 | 0.0068 | 0.0829 | | 3.3553 | 5100 | 0.0 | 0.0826 | | 3.4211 | 5200 | 0.0 | 0.0827 | | 3.4868 | 5300 | 0.0 | 0.0824 | | 3.5526 | 5400 | 0.0 | 0.0823 | | 3.6184 | 5500 | 0.0 | 0.0822 | | 3.6842 | 5600 | 0.0 | 0.0821 | | 3.75 | 5700 | 0.0 | 0.0822 | | 3.8158 | 5800 | 0.0 | 0.082 | | 3.8816 | 5900 | 0.0032 | 0.0819 | | 3.9474 | 6000 | 0.0 | 0.0822 | | 4.0132 | 6100 | 0.0 | 0.0824 | | 4.0789 | 6200 | 0.0 | 0.0822 | | 4.1447 | 6300 | 0.0 | 0.0819 | | 4.2105 | 6400 | 0.0 | 0.0822 | | 4.2763 | 6500 | 0.0057 | 0.0824 | | 4.3421 | 6600 | 0.0 | 0.0824 | | 4.4079 | 6700 | 0.0 | 0.0824 | | 4.4737 | 6800 | 0.0022 | 0.0822 | | 4.5395 | 6900 | 0.0 | 0.0822 | | 4.6053 | 7000 | 0.0 | 0.0823 | | 4.6711 | 7100 | 0.0 | 0.0822 | | 4.7368 | 7200 | 0.0034 | 0.0822 | | 4.8026 | 7300 | 0.0 | 0.0822 | | 4.8684 | 7400 | 0.0 | 0.0822 | | 4.9342 | 7500 | 0.0 | 0.0822 | | 5.0 | 7600 | 0.0 | 0.0822 | | 0.0007 | 1 | 0.0018 | - | | **0.0658** | **100** | **0.0002** | **0.0612** | | 0.1316 | 200 | 0.0002 | 0.0613 | | 0.1974 | 300 | 0.0002 | 0.0615 | | 0.2632 | 400 | 0.0 | 0.0619 | | 0.3289 | 500 | 0.0021 | 0.0626 | | 0.3947 | 600 | 0.0001 | 0.0628 | | 0.4605 | 700 | 0.0001 | 0.0633 | | 0.5263 | 800 | 0.0001 | 0.064 | | 0.5921 | 900 | 0.0001 | 0.0635 | | 0.6579 | 1000 | 0.0001 | 0.0645 | | 0.7237 | 1100 | 0.0001 | 0.0659 | | 0.7895 | 1200 | 0.0055 | 0.0662 | | 0.8553 | 1300 | 0.0001 | 0.0667 | | 0.9211 | 1400 | 0.0032 | 0.0673 | | 0.9868 | 1500 | 0.0001 | 0.067 | | 1.0526 | 1600 | 0.0001 | 0.0668 | | 1.1184 | 1700 | 0.0001 | 0.0667 | | 1.1842 | 1800 | 0.0001 | 0.0664 | | 1.25 | 1900 | 0.0001 | 0.0667 | | 1.3158 | 2000 | 0.0 | 0.0674 | | 1.3816 | 2100 | 0.0001 | 0.0667 | | 1.4474 | 2200 | 0.0 | 0.0669 | | 1.5132 | 2300 | 0.0028 | 0.0669 | | 1.5789 | 2400 | 0.0001 | 0.0671 | | 1.6447 | 2500 | 0.0001 | 0.0676 | | 1.7105 | 2600 | 0.0001 | 0.0689 | | 1.7763 | 2700 | 0.0001 | 0.069 | | 1.8421 | 2800 | 0.0001 | 0.0691 | | 1.9079 | 2900 | 0.0001 | 0.0696 | | 1.9737 | 3000 | 0.0001 | 0.0688 | | 2.0395 | 3100 | 0.0 | 0.0678 | | 2.1053 | 3200 | 0.0027 | 0.0677 | | 2.1711 | 3300 | 0.0001 | 0.0675 | | 2.2368 | 3400 | 0.0 | 0.0676 | | 2.3026 | 3500 | 0.0001 | 0.068 | | 2.3684 | 3600 | 0.0001 | 0.0672 | | 2.4342 | 3700 | 0.0 | 0.0669 | | 2.5 | 3800 | 0.0 | 0.0667 | | 2.5658 | 3900 | 0.0 | 0.0673 | | 2.6316 | 4000 | 0.0 | 0.0672 | | 2.6974 | 4100 | 0.0032 | 0.0689 | | 2.7632 | 4200 | 0.0 | 0.0691 | | 2.8289 | 4300 | 0.0001 | 0.0693 | | 2.8947 | 4400 | 0.0388 | 0.0692 | | 2.9605 | 4500 | 0.0001 | 0.0691 | | 3.0263 | 4600 | 0.0 | 0.0683 | | 3.0921 | 4700 | 0.0 | 0.0685 | | 3.1579 | 4800 | 0.0001 | 0.0681 | | 3.2237 | 4900 | 0.0 | 0.0677 | | 3.2895 | 5000 | 0.0081 | 0.0684 | | 3.3553 | 5100 | 0.0 | 0.0685 | | 3.4211 | 5200 | 0.0 | 0.0681 | | 3.4868 | 5300 | 0.0001 | 0.0683 | | 3.5526 | 5400 | 0.0001 | 0.0681 | | 3.6184 | 5500 | 0.0 | 0.0675 | | 3.6842 | 5600 | 0.0 | 0.0687 | | 3.75 | 5700 | 0.0001 | 0.0692 | | 3.8158 | 5800 | 0.0 | 0.0695 | | 3.8816 | 5900 | 0.0038 | 0.069 | | 3.9474 | 6000 | 0.0001 | 0.069 | | 4.0132 | 6100 | 0.0 | 0.0684 | | 4.0789 | 6200 | 0.0001 | 0.0688 | | 4.1447 | 6300 | 0.0 | 0.0682 | | 4.2105 | 6400 | 0.0 | 0.0677 | | 4.2763 | 6500 | 0.0049 | 0.0678 | | 4.3421 | 6600 | 0.0001 | 0.068 | | 4.4079 | 6700 | 0.0 | 0.0679 | | 4.4737 | 6800 | 0.0029 | 0.0679 | | 4.5395 | 6900 | 0.0 | 0.0684 | | 4.6053 | 7000 | 0.0 | 0.0678 | | 4.6711 | 7100 | 0.0 | 0.0688 | | 4.7368 | 7200 | 0.004 | 0.0695 | | 4.8026 | 7300 | 0.0 | 0.0696 | | 4.8684 | 7400 | 0.0 | 0.0695 | | 4.9342 | 7500 | 0.0 | 0.0695 | | 5.0 | 7600 | 0.0 | 0.0691 | | 5.0658 | 7700 | 0.0033 | 0.0691 | | 5.1316 | 7800 | 0.0 | 0.0691 | | 5.1974 | 7900 | 0.0 | 0.0688 | | 5.2632 | 8000 | 0.0 | 0.0689 | | 5.3289 | 8100 | 0.0001 | 0.0689 | | 5.3947 | 8200 | 0.0 | 0.0688 | | 5.4605 | 8300 | 0.0 | 0.0685 | | 5.5263 | 8400 | 0.0 | 0.0688 | | 5.5921 | 8500 | 0.0 | 0.0683 | | 5.6579 | 8600 | 0.003 | 0.0688 | | 5.7237 | 8700 | 0.0 | 0.0698 | | 5.7895 | 8800 | 0.0037 | 0.0701 | | 5.8553 | 8900 | 0.0 | 0.0701 | | 5.9211 | 9000 | 0.0001 | 0.0695 | | 5.9868 | 9100 | 0.0001 | 0.0697 | | 6.0526 | 9200 | 0.0 | 0.0694 | | 6.1184 | 9300 | 0.0 | 0.0689 | | 6.1842 | 9400 | 0.0 | 0.0686 | | 6.25 | 9500 | 0.0025 | 0.0686 | | 6.3158 | 9600 | 0.0 | 0.069 | | 6.3816 | 9700 | 0.0 | 0.069 | | 6.4474 | 9800 | 0.0 | 0.0687 | | 6.5132 | 9900 | 0.0001 | 0.0683 | | 6.5789 | 10000 | 0.0 | 0.0684 | | 6.6447 | 10100 | 0.0 | 0.0684 | | 6.7105 | 10200 | 0.0001 | 0.069 | | 6.7763 | 10300 | 0.0 | 0.0694 | | 6.8421 | 10400 | 0.0028 | 0.0696 | | 6.9079 | 10500 | 0.0 | 0.0697 | | 6.9737 | 10600 | 0.0 | 0.0697 | | 7.0395 | 10700 | 0.0 | 0.0694 | | 7.1053 | 10800 | 0.0 | 0.0692 | | 7.1711 | 10900 | 0.0 | 0.069 | | 7.2368 | 11000 | 0.0 | 0.0691 | | 7.3026 | 11100 | 0.0 | 0.0691 | | 7.3684 | 11200 | 0.0 | 0.0691 | | 7.4342 | 11300 | 0.0025 | 0.069 | | 7.5 | 11400 | 0.0 | 0.0687 | | 7.5658 | 11500 | 0.0 | 0.0688 | | 7.6316 | 11600 | 0.0 | 0.0688 | | 7.6974 | 11700 | 0.0001 | 0.0691 | | 7.7632 | 11800 | 0.0 | 0.0692 | | 7.8289 | 11900 | 0.0001 | 0.0692 | | 7.8947 | 12000 | 0.0405 | 0.0693 | | 7.9605 | 12100 | 0.0 | 0.0695 | | 8.0263 | 12200 | 0.0029 | 0.0694 | | 8.0921 | 12300 | 0.0001 | 0.0693 | | 8.1579 | 12400 | 0.0 | 0.0692 | | 8.2237 | 12500 | 0.0001 | 0.0691 | | 8.2895 | 12600 | 0.0045 | 0.0693 | | 8.3553 | 12700 | 0.0 | 0.0693 | | 8.4211 | 12800 | 0.0 | 0.0692 | | 8.4868 | 12900 | 0.0 | 0.0691 | | 8.5526 | 13000 | 0.0 | 0.0691 | | 8.6184 | 13100 | 0.0026 | 0.069 | | 8.6842 | 13200 | 0.0 | 0.0692 | | 8.75 | 13300 | 0.0 | 0.0694 | | 8.8158 | 13400 | 0.0 | 0.0694 | | 8.8816 | 13500 | 0.0 | 0.0693 | | 8.9474 | 13600 | 0.0 | 0.0694 | | 9.0132 | 13700 | 0.0 | 0.0693 | | 9.0789 | 13800 | 0.0 | 0.0693 | | 9.1447 | 13900 | 0.0 | 0.0692 | | 9.2105 | 14000 | 0.003 | 0.0692 | | 9.2763 | 14100 | 0.0044 | 0.0692 | | 9.3421 | 14200 | 0.0 | 0.0692 | | 9.4079 | 14300 | 0.0 | 0.0692 | | 9.4737 | 14400 | 0.0 | 0.0691 | | 9.5395 | 14500 | 0.0 | 0.0691 | | 9.6053 | 14600 | 0.0 | 0.0691 | | 9.6711 | 14700 | 0.0 | 0.0691 | | 9.7368 | 14800 | 0.0043 | 0.0692 | | 9.8026 | 14900 | 0.0028 | 0.0692 | | 9.8684 | 15000 | 0.0 | 0.0692 | | 9.9342 | 15100 | 0.0 | 0.0692 | | 10.0 | 15200 | 0.0 | 0.0692 | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.12 - SetFit: 1.0.3 - Sentence Transformers: 3.0.1 - Transformers: 4.40.2 - PyTorch: 2.3.0+cu121 - Datasets: 2.20.0 - Tokenizers: 0.19.1 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```