File size: 5,305 Bytes
3995f2f c0e83ed 3995f2f c0e83ed 3995f2f c0e83ed 3995f2f c0e83ed 3995f2f c0e83ed |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 |
---
language:
- en
library_name: transformers
base_model: meta-llama/Llama-3.2-3B-Instruct
base_model_relation: finetune
tags:
- llama
- chatgpt-prompts
- role-playing
- instruction-tuning
- conversational
- lora
- peft
license: llama3.2
datasets:
- fka/awesome-chatgpt-prompts
pipeline_tag: text-generation
model-index:
- name: Llama-3.2-3B-ChatGPT-Prompts-Instruct
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: awesome-chatgpt-prompts
type: fka/awesome-chatgpt-prompts
metrics:
- name: Training Loss
type: loss
value: 0.28
---
# Llama-3.2-3B-ChatGPT-Prompts-Instruct
## Model Description
This is a fine-tuned version of Meta's Llama-3.2-3B-Instruct model, specifically trained on the awesome-chatgpt-prompts dataset to excel at role-playing and prompt-based interactions. The model has been optimized to understand and respond to various professional and creative roles with enhanced accuracy and context awareness.
## Model Details
- **Base Model:** meta-llama/Llama-3.2-3B-Instruct
- **Model Type:** Causal Language Model
- **Fine-tuning Method:** LoRA (Low-Rank Adaptation)
- **Training Dataset:** fka/awesome-chatgpt-prompts
- **Model Size:** 3B parameters
- **Quantization:** 4-bit (BitsAndBytesConfig)
## Training Details
### Training Configuration
- **LoRA Rank:** 4
- **LoRA Alpha:** 8
- **Learning Rate:** 3e-4
- **Batch Size:** 8
- **Epochs:** 10
- **Max Sequence Length:** 64
- **Gradient Accumulation Steps:** 3
- **Optimizer:** AdamW with cosine learning rate scheduler
- **Weight Decay:** 0.01
- **Warmup Ratio:** 0.05
### Training Results
- **Final Training Loss:** 0.28
- **Training Steps:** 190
- **Training Runtime:** 399.47 seconds
- **Convergence:** Stable convergence with proper gradient norms
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_name = "sweatSmile/Llama-3.2-3B-ChatGPT-Prompts-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map="auto"
)
# Example usage
prompt = "Linux Terminal"
messages = [
{"role": "user", "content": prompt}
]
# Apply chat template
formatted_prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer(formatted_prompt, return_tensors="pt")
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.7,
do_sample=True,
pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)
```
## Intended Use
This model is designed for:
- **Role-playing scenarios:** Acting as various professionals (developers, translators, terminals, etc.)
- **Educational purposes:** Learning different professional contexts and responses
- **Creative writing assistance:** Generating contextually appropriate responses for different roles
- **Prompt engineering research:** Understanding how models respond to role-based instructions
## Capabilities
The model excels at:
- Understanding and adopting various professional roles
- Generating contextually appropriate responses
- Maintaining consistency within assigned roles
- Following complex instructions with role-specific knowledge
- Adapting communication style based on the requested persona
## Example Interactions
**Input:** "English Translator and Improver"
**Output:** The model will adopt the role of a professional translator and language improver, offering translation services and language enhancement capabilities.
**Input:** "Linux Terminal"
**Output:** The model will simulate a Linux terminal environment, responding to commands as a real terminal would.
## Limitations
- Model responses are generated based on training data and may not always reflect real-world accuracy
- Performance may vary depending on the complexity and specificity of role-based requests
- The model should not be used for generating harmful, biased, or inappropriate content
- Outputs should be verified for factual accuracy, especially in professional contexts
## Ethical Considerations
- This model should be used responsibly and ethically
- Users should be aware that this is an AI model and not substitute for real professional expertise
- The model should not be used to impersonate real individuals or for deceptive purposes
- Always disclose when content is AI-generated in professional or public contexts
## Framework Versions
- **Transformers:** 4.x
- **PyTorch:** 2.x
- **PEFT:** Latest
- **Datasets:** Latest
- **Tokenizers:** Latest
## License
This model inherits the license from the base Llama-3.2-3B-Instruct model. Please refer to Meta's license terms for usage restrictions and requirements.
## Citation
```bibtex
@model{llama32-chatgpt-prompts-instruct,
title={Llama-3.2-3B-ChatGPT-Prompts-Instruct},
author={sweatSmile},
year={2025},
base_model={meta-llama/Llama-3.2-3B-Instruct},
dataset={fka/awesome-chatgpt-prompts}
}
```
## Contact
For questions, issues, or feedback regarding this model, please create an issue in the model repository or contact the model author. |