|
|
--- |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- en |
|
|
library_name: adapter-transformers |
|
|
--- |
|
|
|
|
|
I followed [this script](https://github.com/huggingface/trl/blob/main/examples/research_projects/stack_llama_2/scripts/sft_llama2.py) to train this model. |
|
|
|
|
|
instead of the official [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) model, I used this repo [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf). |
|
|
|
|
|
The model trained on [lvwerra/stack-exchange-paired](https://huggingface.co/datasets/lvwerra/stack-exchange-paired) dataset. |
|
|
|
|
|
seq_length: 1024 |
|
|
|
|
|
steps: 1600 |
|
|
|
|
|
|
