File size: 1,838 Bytes
fa057ce f08c8ce fa057ce b609478 fa057ce a279423 fa057ce b609478 fa057ce b609478 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
---
license: llama3
language:
- en
base_model: meta-llama/Meta-Llama-3-8B
---
## Overview
**nephra v1** is primarily a model built for roleplaying sessions, trained on roleplay and instruction-style datasets.
## Model Details
- **Developed by**: [Sao10K](https://huggingface.co/Sao10K)
- **Model type**: Text-based Large Language Model
- **License**: [Meta Llama 3 Community License Agreement](https://huggingface.co/meta-llama/Meta-Llama-3-8B/blob/main/LICENSE)
- **Finetuned from model**: [Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B)
## Inference Guidelines
```python
import transformers
import torch
model_id = "yodayo-ai/nephra_v1.0"
pipeline = transformers.pipeline(
"text-generation",
model=model_id,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
messages = [
{"role": "system", "content": "You are to play the role of a cheerful assistant."},
{"role": "user", "content": "Hi there, how's your day?"},
]
prompt = pipeline.tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
outputs = pipeline(
prompt,
max_new_tokens=512,
eos_token_id=[
pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>"),
pipeline.tokenizer.eos_token_id,
],
do_sample=True,
temperature=1.12,
min_p=0.075,
)
print(outputs[0]["generated_text"][len(prompt):])
```
### Recommended Settings
To guide the model to generate high-quality responses, here are the ideal settings:
```
Prompt Format: Same Prompt Format as Llama-3-Instruct
Temperature - 1.12
min-p: 0.075
Repetition Penalty: 1.1
Custom Stopping Strings: "\n{{user}}", "<" , "```" , -> Has occasional broken generations.
```
## License
Nephra v1 falls under [META LLAMA 3 COMMUNITY LICENSE AGREEMENT](https://huggingface.co/meta-llama/Meta-Llama-3-8B/blob/main/LICENSE).
|