metadata

license: apache-2.0
datasets:
  - MoritzLaurer/synthetic_zeroshot_mixtral_v0.1
  - knowledgator/gliclass-v1.0
  - fancyzhx/amazon_polarity
  - cnmoro/QuestionClassification
  - Arsive/toxicity_classification_jigsaw
  - shishir-dwi/News-Article-Categorization_IAB
  - SetFit/qnli
  - nyu-mll/multi_nli
  - SetFit/student-question-categories
  - SetFit/tweet_sentiment_extraction
  - SetFit/hate_speech18
  - saattrupdan/doc-nli
language:
  - en
  - fr
  - ge
metrics:
  - f1
pipeline_tag: zero-shot-classification
tags:
  - text classification
  - zero-shot
  - small language models
  - RAG
  - sentiment analysis

⭐ GLiClass: Generalist and Lightweight Model for Sequence Classification

This is an efficient zero-shot classifier inspired by GLiNER work. It demonstrates the same performance as a cross-encoder while being more compute-efficient because classification is done at a single forward path.

It can be used for topic classification, sentiment analysis and as a reranker in RAG pipelines.

The model was trained on synthetic and licensed data that allow commercial use and can be used in commercial applications.

This version of the model uses a layer-wise selection of features that enables a better understanding of different levels of language. The backbone model is ModernBERT-large, which effectively processes long sequences.

How to use:

First of all, you need to install GLiClass library:

pip install gliclass
pip install -U transformers>=4.48.0

Than you need to initialize a model and a pipeline:

from gliclass import GLiClassModel, ZeroShotClassificationPipeline
from transformers import AutoTokenizer
model = GLiClassModel.from_pretrained("knowledgator/gliclass-modern-large-v2.0-init")
tokenizer = AutoTokenizer.from_pretrained("knowledgator/gliclass-modern-large-v2.0-init")
pipeline = ZeroShotClassificationPipeline(model, tokenizer, classification_type='multi-label', device='cuda:0')
text = "One day I will see the world!"
labels = ["travel", "dreams", "sport", "science", "politics"]
results = pipeline(text, labels, threshold=0.5)[0] #because we have one text
for result in results:
 print(result["label"], "=>", result["score"])

Benchmarks:

Below, you can see the F1 score on several text classification datasets. All tested models were not fine-tuned on those datasets and were tested in a zero-shot setting.

Model	IMDB	AG_NEWS	Emotions
gliclass-modern-large-v2.0-init (399 M)	0.9137	0.7357	0.4140
gliclass-modern-base-v2.0-init (151 M)	0.8264	0.6637	0.2985
gliclass-large-v1.0 (438 M)	0.9404	0.7516	0.4874
gliclass-base-v1.0 (186 M)	0.8650	0.6837	0.4749
gliclass-small-v1.0 (144 M)	0.8650	0.6805	0.4664
Bart-large-mnli (407 M)	0.89	0.6887	0.3765
Deberta-base-v3 (184 M)	0.85	0.6455	0.5095
Comprehendo (184M)	0.90	0.7982	0.5660
SetFit BAAI/bge-small-en-v1.5 (33.4M)	0.86	0.5636	0.5754

Below you can find a comparison with other GLiClass models:

Dataset	gliclass-small-v1.0-lw	gliclass-base-v1.0-lw	gliclass-large-v1.0-lw	gliclass-small-v1.0	gliclass-base-v1.0	gliclass-large-v1.0	gliclass-modern-base-v2.0-init	gliclass-modern-large-v2.0-init
CR	0.8886	0.9097	0.9226	0.8824	0.8942	0.9219	0.9041	0.8980
sst2	0.8392	0.8987	0.9247	0.8518	0.8979	0.9269	0.9011	0.9434
sst5	0.2865	0.3779	0.2891	0.2424	0.2789	0.3900	0.1972	0.1123
20_news_groups	0.4572	0.3953	0.4083	0.3366	0.3576	0.3863	0.2448	0.2792
spam	0.5118	0.5126	0.3642	0.4089	0.4938	0.3661	0.5074	0.6364
rotten_tomatoes	0.8015	0.8429	0.8807	0.7987	0.8508	0.8808	0.6630	0.5928
financial_phrasebank	0.8665	0.8880	0.9044	0.8901	0.8955	0.8735	0.2537	0.2562
imdb	0.9048	0.9351	0.9429	0.8982	0.9238	0.9333	0.8255	0.9137
ag_news	0.7252	0.6985	0.7559	0.7242	0.6848	0.7503	0.6050	0.6933
dair_emotion	0.4012	0.3516	0.3951	0.3450	0.2357	0.4013	0.2474	0.3746
capsotu	0.3794	0.4643	0.4749	0.3432	0.4375	0.4644	0.2929	0.2919
Average:	0.5732	0.6183	0.6165	0.5401	0.5571	0.6078	0.5129	0.5447