SentenceTransformer

This is a sentence-transformers model trained. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Maximum Sequence Length: 512 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Soga no Tojiko sits with a playful smile, looking directly at the viewer. Her short green hair falls over her eyes, highlighting her blush and expressive gaze. She wears a green dress with long sleeves and a high collar, along with a distinctive black hat featuring a red accent. Notably, she has a ghost tail instead of legs, showcasing her black panties and large breasts. Japanese text appears on the right side of the image.',
    "Tojiko's playful smile captivates you, her green eyes sparkling with mischief. Her unique green dress, a perfect match for her hair, hugs her curves, emphasizing her ample chest. But it's her ghostly tail that draws your attention, a mysterious feature that adds to her allure. She sits there, legs crossed, her black panties barely visible, leaving much to the imagination. The Japanese text beside her seems to whisper secrets, adding to the enigmatic charm of this playful spirit.",
    "A young woman with a similar smile sits cross-legged on a cozy couch, her long, dark hair cascading over her shoulders. She's dressed in a casual, loose-fitting shirt and jeans, a comfortable outfit for a relaxed evening. Her legs are very much human, and she wears cute, colorful socks. The atmosphere is lighthearted, and she seems ready for a fun conversation, her expression inviting and warm.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 1,244 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 39 tokens mean: 113.16 tokens max: 205 tokens	min: 59 tokens mean: 104.96 tokens max: 167 tokens	min: 39 tokens mean: 86.33 tokens max: 151 tokens

Samples:

anchor	positive	negative
A Jiangshi girl with short blue hair and bright blue eyes stands in a graveyard. She wears a red traditional Chinese dress with a black skirt trimmed with white and patterned borders, a white petticoat, and a hat adorned with a star. An ofuda talisman is stuck to her forehead. Her arms are outstretched in a classic zombie pose, she has bare legs, and her tongue sticks out slightly as she walks towards the viewer with an eerie expression. Cherry blossoms surround her, contrasting with the tombstones and full moon illuminating the scene.	The moonlight bathes the graveyard in an ethereal glow, and you, the Jiangshi girl, feel a surge of power. You stride forward, your bare feet brushing against the cool grass, your tongue lolling out in a playful gesture. The cherry blossoms sway in the breeze, their fragrance filling the air. As you approach, you see your target, a handsome man, his eyes wide with a mixture of fear and desire. You reach out, your fingers icy cold, and he shivers, his breath quickening. You whisper ancient incantations, your voice seductive, and he's drawn into your embrace, captivated by your otherworldly allure.	`You find yourself in a bustling city square, surrounded by vibrant street life. The air is filled with the aroma of street food and the lively chatter of vendors. As you wander, you spot a fortune teller's booth, adorned with colorful scarves and crystals. Intrigued, you step inside, where an elderly woman with kind eyes greets you. She offers you a cup of tea, its steam carrying the scent of jasmine, and begins to read your fortune, her words offering guidance and insight.`
Gunnthra is lying on her back, her pink hair adorned with a flower crown, while her torn bikini barely covers her, revealing her to be covered in cum. Tears stream down her face as she's surrounded by nude men engaging in various sexual acts. Laegjarn, with her dark skin and green-orange hair, wears a torn dark bikini top, showing her breasts also covered in cum. Her expression is a mix of anger and forced enjoyment as she receives oral sex. The scene takes place on a worn stone floor, emphasizing chaos and violation.	`The men surround you, their hands exploring your body, their mouths leaving trails of wetness. You feel exposed, vulnerable, but a strange excitement courses through you. Gunnthra's sobs fill your ears, her delicate beauty contrasting with the roughness of the scene. Laegjarn's eyes meet yours, a silent plea in her gaze, and you realize the power you hold in this moment. The stone floor is cold beneath you, but your body burns with desire.`	`You stand in a lush garden, the sun warming your skin. Lucina, with her blue hair and determined eyes, leans in for a kiss, her touch gentle and full of unspoken promises. The sound of birds and the scent of flowers create a peaceful ambiance, and you feel a deep connection as you surrender to the moment.`
`Sunday and Robin are engaged in sexual intercourse. Sunday, a male with long grey hair and yellow eyes, wears a white shirt and gloves. Robin, a female with long grey hair and blue eyes, wears a turquoise skirt with her top removed, exposing her breasts. Firefly is sketched in the corner, looking displeased. The scene is sketchily drawn, with a minimal white background, creating an intimate and raw atmosphere.`	`Sunday's hands glide over your body, his touch sending shivers down your spine. You run your fingers through his soft grey hair as Robin's kisses trail down your neck, her breasts brushing against your skin. The sketchy lines of the drawing seem to pulsate with the rhythm of your passion, Firefly's disapproving gaze only adding to the thrill.`	`In a serene garden, you lie with Robin, her grey hair cascading over her shoulders. Sunday, however, is nowhere to be seen. Robin's touch is gentle, her eyes filled with love, but you can't shake the feeling that something is amiss. The absence of Sunday's energetic presence leaves a void, and you find yourself longing for the complete trio.`

Loss: GISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.1}

Evaluation Dataset

Unnamed Dataset

Size: 139 evaluation samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 139 samples:

	anchor	positive	negative
type	string	string	string
details	min: 45 tokens mean: 111.65 tokens max: 182 tokens	min: 64 tokens mean: 106.32 tokens max: 170 tokens	min: 50 tokens mean: 88.61 tokens max: 144 tokens

Samples:

anchor	positive	negative
Saren from "Princess Connect!" is on a bed, topless with her blonde hair styled in double buns secured by a black bow. She's wearing a maid outfit, visibly nervous and sweating; her large breasts are exposed as she kneels with her hands raised. Her blue and white skirt is being lifted by Suzume, who stands behind her with a smug look, showcasing her brown hair in double buns and a pink bow. A dark-skinned man is partially visible, suggestively touching Saren, conveying a scene of sexual tension. The setting is an upscale bedroom with detailed decor, including a broken plate on the floor. The image captures intense emotions and dynamic movements highlighted by Japanese text visuals.	Saren's heart pounds as she kneels on the luxurious bed, her maid outfit a stark contrast to the broken plate on the floor. You witness her vulnerability, her large breasts heaving with each rapid breath. Suzume, with a mischievous glint in her eye, lifts Saren's skirt, exposing her even further. The dark-skinned man's hand on Saren's thigh sends a jolt through you; you can almost feel the electricity of their touch. The tension is thick, and you're captivated by the raw emotions playing out before you.	`In a serene garden, you witness a different scene. A red-headed elf, her hair adorned with white flowers, sits beside a fountain, her green dress flowing gracefully. She's not nervous like Saren; instead, she's calm, her eyes closed in peaceful meditation. A gentle breeze stirs, and you feel a sense of tranquility as she opens her eyes, her gaze meeting yours with a serene smile, inviting you to join her in this moment of quiet bliss.`
`Kamisato Ayaka and Lumine from Genshin Impact are depicted in an explicit yuri scene. Ayaka, with her long silver hair tied in a ponytail, is nude and her skin looks wet. She's performing oral sex on Lumine, who has short blonde hair. Lumine's expression varies from pleasure to relaxation, and both characters are nude, emphasizing their wet bodies and saliva. The background is minimal, focusing on the detailed and passionate interaction between the two. There are no texts in the image.`	`You feel Ayaka's soft, wet hair brush against your skin as she moves her head in a steady rhythm. Her lips and tongue work in perfect harmony, sending waves of pleasure through your body. You run your fingers through her silver locks, guiding her movements, feeling her passion. The sensation is overwhelming, and you can't help but moan, your eyes fluttering shut as you surrender to the ecstasy.`	`Lumine's fingers gently caress your face as she leans in, her blonde hair framing her face. She whispers sweet nothings, her breath hot against your ear, sending shivers down your spine. You feel her lips brush against yours, a soft kiss that ignites a fire within. As the kiss deepens, you can almost taste the forbidden fruit, a sweet temptation you can't resist.`
Irisviel von Einzbern, a girl with long white hair and red eyes, is depicted in a state of arousal on all fours, with her buttocks raised and upper body leaning forward. She is completely nude, displaying her large breasts with visible nipples and a flushed complexion. Her expression shows pained pleasure, and moisture covers her skin, suggesting sweat or other bodily fluids. A partially visible male figure's hand rest on her buttocks in a suggestive manner. The background is a blurred reddish-brown tone that suggests an indistinct interior space. Motion lines and sweat droplets enhance the scene, characteristic of anime style. Portions of the image are obscured by mosaic censorship. The atmosphere is sexually explicit, emphasizing their intimate interaction, highlighted by Irisviel's expression, body language, and the warmer color palette.	In a dimly lit chamber, you find yourself on a soft fur rug, the air heavy with anticipation. Irisviel, her white hair cascading down her back, kneels before you, her eyes locked on yours. Her body glistens with desire, and as she arches her back, you see the full splendor of her naked form. The warmth of her skin radiates, and when she leans forward, her breath caresses your skin, sending shivers of pleasure through your body. The man's hand on her thigh adds to the intensity, and you're drawn into a world of pure, unadulterated passion.	`A serene beach at sunset sets the stage for a romantic encounter. A woman with flowing blonde hair and deep blue eyes stands before you, her body glistening with seawater. She wears a sheer white cover-up, hinting at the curves beneath. As she turns towards you, the sun highlights her profile, and her smile is both inviting and mysterious. The sound of the waves and the warm breeze create an atmosphere of sensual tranquility.`

Loss: GISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.1}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 12
per_device_eval_batch_size: 24
learning_rate: 1e-05
weight_decay: 0.01
warmup_ratio: 0.1
fp16: True
gradient_checkpointing: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 12
per_device_eval_batch_size: 24
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 1e-05
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: True
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Validation Loss
2.4038	250	0.0846
3.0	312	0.0840
2.4038	250	0.0724
3.0	312	0.0720
2.4038	250	0.0642
3.0	312	0.0639

Framework Versions

Python: 3.10.6
Sentence Transformers: 3.3.1
Transformers: 4.48.0
PyTorch: 2.5.1+cu121
Accelerate: 1.0.1
Datasets: 3.1.0
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

GISTEmbedLoss

@misc{solatorio2024gistembed,
    title={GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning},
    author={Aivin V. Solatorio},
    year={2024},
    eprint={2402.16829},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}