|
--- |
|
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit |
|
library_name: peft |
|
license: llama3.1 |
|
tags: |
|
- trl |
|
- sft |
|
- unsloth |
|
- generated_from_trainer |
|
model-index: |
|
- name: outputs |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# outputs |
|
|
|
This model is a fine-tuned version of [unsloth/llama-3-8b-Instruct-bnb-4bit](https://huggingface.co/unsloth/llama-3-8b-Instruct-bnb-4bit) on the None dataset. |
|
|
|
## Model description |
|
|
|
This fine-tuning model is a large language model using the unsloth library, which focuses on memory efficiency and speed. |
|
It demonstrates data preparation, model configuration with LoRA, training with SFTTrainer, and inference with optimized settings. |
|
The unsloth models, especially the 4-bit quantized versions, enable efficient and faster training and inference, making them suitable for various AI and ML applications. |
|
|
|
## How to use |
|
|
|
1. **Install Required Libraries** |
|
|
|
```python |
|
import torch |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
from unsloth import FastLanguageModel |
|
from unsloth.chat_templates import get_chat_template |
|
from peft import PeftModel, PeftConfig |
|
``` |
|
|
|
|
|
2. **Load the Model and Tokenizer** |
|
|
|
```python |
|
# Load the tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained("m00bs/llama-3-8b-inst-CausalRelationship-finetune-tokenizer") |
|
|
|
# Load the model |
|
config = PeftConfig.from_pretrained("m00bs/outputs") |
|
base_model = AutoModelForCausalLM.from_pretrained("unsloth/llama-3-8b-Instruct-bnb-4bit") |
|
model = PeftModel.from_pretrained(base_model, "m00bs/outputs") |
|
|
|
# Move model to GPU if available |
|
device = "cuda" if torch.cuda.is_available() else "cpu" |
|
model.to(device) |
|
|
|
``` |
|
|
|
3. **Prepare Inputs** |
|
|
|
```python |
|
# Prepare the input text |
|
input_text = """As a finance expert, answer the following question about the following market event about Market Event: |
|
Given that China's full reopening announcement on December 26, 2022 caused an immediate jump in Chinese stock prices, What was the impact of China's full reopening announcement on December 26, 2022 on Chinese stock prices?""" |
|
|
|
# Tokenize the input text |
|
inputs = tokenizer(input_text, return_tensors="pt").to(device) |
|
``` |
|
|
|
4. **Run Inference** |
|
|
|
```python |
|
# Generate the response |
|
outputs = model.generate(**inputs, max_new_tokens=300, use_cache=True) |
|
|
|
# Decode the output |
|
response = tokenizer.batch_decode(outputs, skip_special_tokens=True) |
|
``` |
|
|
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0002 |
|
- train_batch_size: 1 |
|
- eval_batch_size: 8 |
|
- seed: 3407 |
|
- gradient_accumulation_steps: 4 |
|
- total_train_batch_size: 4 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 5 |
|
- training_steps: 60 |
|
- mixed_precision_training: Native AMP |
|
|
|
|
|
|
|
### Framework versions |
|
|
|
- PEFT 0.12.0 |
|
- Transformers 4.43.3 |
|
- Pytorch 2.3.1+cu121 |
|
- Datasets 2.20.0 |
|
- Tokenizers 0.19.1 |
