File size: 1,569 Bytes
dcaddb3 a4561b1 3e6a0ac 8192591 3e6a0ac 0b07703 3e6a0ac |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
---
license: llama3
language:
- ko
- en
library_name: transformers
pipeline_tag: text-generation
---
- Basemodel [MLP-KTLim/llama-3-Korean-Bllossom-8B](https://huggingface.co/MLP-KTLim/llama-3-Korean-Bllossom-8B)
- Dataset
- [AI Hub - ํ๊ตญ์ด ์ฑ๋ฅ์ด ๊ฐ์ ๋ ์ด๊ฑฐ๋AI ์ธ์ด๋ชจ๋ธ ๊ฐ๋ฐ ๋ฐ ๋ฐ์ดํฐ](https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&dataSetSn=71748)
### Python code with Pipeline
```python
import transformers
import torch
model_id = "VIRNECT/llama-3-Korean-8B-V2"
pipeline = transformers.pipeline(
"text-generation",
model=model_id,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
pipeline.model.eval()
PROMPT = '''๋น์ ์ ์ธ๊ฐ๊ณผ ๋ํํ๋ ์น์ ํ ์ฑ๋ด์
๋๋ค. ์ง๋ฌธ์ ๋ํ ์ ๋ณด๋ฅผ ์ํฉ์ ๋ง๊ฒ ์์ธํ ์ ๊ณตํฉ๋๋ค. ๋น์ ์ด ์ง๋ฌธ์ ๋ํ ๋ต์ ๋ชจ๋ฅธ๋ค๋ฉด, ์ฌ์ค์ ๋ชจ๋ฅธ๋ค๊ณ ๋งํฉ๋๋ค.'''
instruction = "ํํ๊ณตํ์ด ๋ค๋ฅธ ๊ณตํ ๋ถ์ผ์ ์ด๋ป๊ฒ ๋ค๋ฅธ๊ฐ์?"
messages = [
{"role": "system", "content": f"{PROMPT}"},
{"role": "user", "content": f"{instruction}"}
]
prompt = pipeline.tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
terminators = [
pipeline.tokenizer.eos_token_id,
pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = pipeline(
prompt,
max_new_tokens=2048,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9
)
print(outputs[0]["generated_text"][len(prompt):])
``` |