File size: 734 Bytes
b6f898b 0ccb0d2 5446716 0ccb0d2 5446716 9d7dbaf b6f898b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
---
license: mit
---
A Llama3 based model fine-tuned for cybersecurity domain.
The model was finetuned over https://huggingface.co/datasets/unibuc-cs/CyberGuardianDataset.
While the Llama3 original perplexity on the dataset started ~21.9, we managed to fine-tune to ~7.8 on this dataset, still preserving the general language abilities.
You can load the model and see its LoRA config as below:
```
from peft import get_peft_model, AutoPeftModelForCausalLM, PeftModel, PeftConfig
from transformers import AutoModel
config = PeftConfig.from_pretrained("unibuc-cs/CyberGuardian")
model = AutoModel.from_pretrained(config.base_model_name_or_path)
```
Refer to our github page for details: https://github.com/unibuc-cs/CyberGuardian |