Hugging Face's logo

juanmvs
/
neurona

PEFT
Safetensors
Spanish
llama
llama-3.3
lora
qlora
conversational
spanish
workplace-safety
violence-prevention
chat
instruction-tuning
Model card Files
xet
Community
neurona / HUGGINGFACE_MODEL_CARD.md
juanmvs's picture
juanmvs
upload finetuned llama 3.3 70b adapter
897e7d3 verified
preview code
|
raw
history blame contribute delete
8.07 kB
metadata
language: es
license: llama3.2
library_name: peft
tags:
  - llama
  - llama-3.3
  - peft
  - lora
  - qlora
  - conversational
  - spanish
  - workplace-safety
  - violence-prevention
  - chat
  - instruction-tuning
base_model: meta-llama/Llama-3.3-70B-Instruct
datasets:
  - bertin-project/alpaca-spanish
model-index:
  - name: neurona
    results: []

Neurona - Spanish Workplace Violence Prevention Chatbot

Neurona is a specialized fine-tuned version of Meta's Llama-3.3-70B-Instruct model, designed for Spanish-language conversations about workplace violence prevention and sexual harassment support. This PEFT (LoRA) adapter provides empathetic, professional, and informative responses to users seeking guidance and support in workplace safety situations.

Fine-tuned using QLoRA on NVIDIA H100 GPU with a curated dataset of workplace violence prevention conversations.

Model Details

  • Model Type: PEFT LoRA Adapter
  • Base Model: meta-llama/Llama-3.3-70B-Instruct
  • Fine-tuning Method: QLoRA (4-bit Quantized Low-Rank Adaptation)
  • Language: Spanish (es)
  • Domain: Workplace safety, violence prevention, and sexual harassment support
  • License: Llama 3.2 Community License
  • Parameters: LoRA adapter (~150M trainable parameters)

Intended Use

This model is intended to be used as a conversational AI assistant to provide:

  • Educational information about workplace violence and harassment.
  • Guidance on reporting procedures and seeking help.
  • Empathetic support for individuals in difficult workplace situations.

Out-of-Scope Use

This model is not a substitute for professional legal, psychological, or crisis intervention services. It should not be used for:

  • Providing legal advice.
  • Medical or psychological diagnosis.
  • Emergency or crisis situations.

How to Use

Requirements 

pip install transformers torch peft bitsandbytes accelerate

Basic Usage

This is a PEFT LoRA adapter that must be loaded on top of the base Llama 3.3 70B Instruct model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model and tokenizer
base_model_id = "meta-llama/Llama-3.3-70B-Instruct"
adapter_model_id = "juanmvs/neurona"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    load_in_4bit=True  # Enable 4-bit quantization for memory efficiency
)

# Load PEFT adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)

# Specialized system prompt for workplace violence prevention
system_prompt = """Eres un asistente especializado en prevención de violencia laboral y acoso sexual en el entorno de trabajo. Tu objetivo es proporcionar apoyo empático, información precisa y recursos específicos a personas que puedan estar experimentando situaciones difíciles en su lugar de trabajo.

IMPORTANTE: Siempre mantén un tono profesional pero cálido, valida las emociones del usuario, y proporciona información práctica basada en protocolos establecidos."""

# Example conversation
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Creo que estoy sufriendo acoso laboral, ¿qué puedo hacer?"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    eos_token_id=tokenizer.eos_token_id,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

Memory Requirements

Configuration GPU Memory RAM Storage
4-bit quantized 8GB+ VRAM 16GB+ 20GB+
Full precision 40GB+ VRAM 64GB+ 150GB+

Hardware Recommendations

  • Recommended: RTX 4090, A100, H100 (with 4-bit quantization)
  • Minimum: RTX 3090, V100 (with 4-bit quantization)
  • CPU inference: Possible but very slow (32GB+ RAM required)

Training Data

  • Training Set: A custom dataset of 32 Spanish instruction-response pairs focused on workplace violence prevention.
  • Validation Set: 1000 samples from the bertin-project/alpaca-spanish dataset to ensure general conversational quality.

The training data was carefully curated to include empathetic, professional, and relevant responses for the target domain.

Training Procedure

Fine-tuning with QLoRA

The model was fine-tuned using 4-bit NormalFloat (NF4) quantization and LoRA.

  • LoRA r: 128
  • LoRA alpha: 32
  • LoRA dropout: 0.05
  • Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, embed_tokens, lm_head

Hyperparameters

  • Learning Rate: 1e-4
  • Scheduler: Cosine
  • Epochs: 3
  • Batch Size: Dynamic (optimized for H100, effective batch size of 48)
  • Warmup Steps: 100
  • Weight Decay: 0.01
  • Gradient Clipping: 0.5

Hardware and Software

  • GPU: 1x NVIDIA H100 80GB
  • Software: PyTorch, TRL, PEFT, bitsandbytes, accelerate

Evaluation

Training Metrics

Metric Value
Training Loss 1.74186
Mean Token Accuracy 63.62%
Entropy 1.12958
Training Time ~3.3 minutes

Conversation Quality

A multi-dimensional evaluation framework was used to assess conversation quality, with a composite score of 0.73 (target > 0.65).

Metric Score
Empathy Score 0.67
Domain Relevance 0.81
Professional Tone 0.74

Limitations & Ethical Considerations

Model Limitations

  • Domain Specificity: Optimized for Spanish workplace violence prevention; may not perform well on general tasks
  • Data Coverage: Based on 32 training examples; may not cover all workplace situation nuances
  • Cultural Context: Designed for Spanish-speaking workplace environments
  • Response Length: Optimized for conversational responses, not long-form content

Ethical Guidelines

  • Not Professional Services: This model provides educational information only, not legal or psychological advice
  • Crisis Situations: For immediate danger, contact emergency services (112 in Spain, 911 in US)
  • Privacy: Users should not share sensitive personal information
  • Bias Awareness: Responses may reflect biases present in training data
  • Human Oversight: Recommend human review for critical workplace decisions

Safety Considerations

  • Emergency Situations: Always prioritize professional emergency services
  • Legal Matters: Consult qualified employment lawyers for legal advice
  • Mental Health: Seek licensed mental health professionals for psychological support
  • Workplace Policies: Follow your organization's specific HR protocols

Citation

If you use this model in your research or applications, please cite it as:

@misc{neurona-2025,
  author = {Juan MVS},
  title = {Neurona: Spanish Workplace Violence Prevention Chatbot},
  year = {2025},
  publisher = {Hugging Face},
  journal = {Hugging Face Hub},
  howpublished = {\url{https://huggingface.co/juanmvs/neurona}}
}

Acknowledgments

  • Base Model: Meta AI for Llama 3.3 70B Instruct
  • Framework: Hugging Face Transformers and PEFT libraries
  • Training Infrastructure: NVIDIA H100 GPU
  • Validation Dataset: Bertin Project for Spanish Alpaca dataset

Contact

For questions about this model or collaboration opportunities:

⚠️ Disclaimer: This AI model is for educational and informational purposes only. For workplace violence situations requiring immediate intervention, please contact appropriate emergency services, HR departments, or professional counselors.