Hugging Face's logo

macadeliccc
/
Orpo-GutenLlama-3-8B-v2

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
Model card Files Community

Orpo-GutenLlama-3-8B-v2

Training Params

  • Learning Rate: 8e-6
  • Batch Size: 1
  • Eval Batch size: 1
  • Gradient accumulation steps: 4
  • Epochs: 3
  • Training Loss: 0.88

Training time: 4 hours on 1x4090. This is a small 1800 sample fine tune to get comfortable with ORPO fine tuning before scaling up.

image/png

Downloads last month
6
Safetensors
Model size
8.03B params
Tensor type
FP16
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Model tree for macadeliccc/Orpo-GutenLlama-3-8B-v2

Quantizations
2 models

Datasets used to train macadeliccc/Orpo-GutenLlama-3-8B-v2