File size: 936 Bytes
39a5343 e054a5f 39a5343 e054a5f 9cc4ac4 39a5343 98a23f7 39a5343 e054a5f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
---
license: apache-2.0
base_model: mistralai/Mistral-7B-v0.3
tags:
- generated_from_trainer
metrics:
- accuracy
language:
- en
datasets:
- BEE-spoke-data/stepbasin-books
---
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/pszemraj/long-generation-tests/runs/o2w2acw6)
# Mistral-7B-v0.3-stepbasin-books-20480
This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on [this dataset](https://github.com/stepbasin/books/tree/master/books) for the purpose of testing out super-long text generation.
- fine-tuned at context length 20480, should consistently generate 8k+ tokens ([example](https://pastebin.com/raw/btzSQNVL))
---
It achieves the following results on the evaluation set:
- Loss: 2.0784
- Accuracy: 0.5396
- Num Input Tokens Seen: 16384000 |