File size: 1,893 Bytes
62893f9 f1fbf0f 048fc98 f1fbf0f 048fc98 f1fbf0f 048fc98 f1fbf0f 048fc98 f1fbf0f 048fc98 f1fbf0f 048fc98 4169eb0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
---
license: apache-2.0
datasets:
- sjsq/PrivacyPolicy
language:
- en
---
# Fine-tuning Llama2-7B-Chat for Privacy Policy Q&A and Summarization
__By Chris Puzzo and Christain Jackson__
__For Comp741/841__
__README updated: 5/22/24__
The basis of this problem is to fine-tune Meta's Llama 2 Transformer using PEFT
and QloRA in order to be able to analyze privacy Policies.
Privacy Policies are written to be confusing and extremely technical, so a tool
that helps users answer questions and summerize privacy policies can be very
useful in knowing how personal data is being used. the HOWTO.md file contains
simple instructions on running the tool
## Setup
This tool is designed to be used on colab with the huggingface transformers
library. For more info check out the model on [github](https://github.com/ChrisPuzzo/Llama-2-7b-Privacy-Policies). The model was trained
using a training code for llama from [Maxime Labonne](https://twitter.com/maximelabonne)
avalible [here](https://colab.research.google.com/drive/1PEQyJO1-f6j0S_XJ8DV50NkpzasXkrzd?usp=sharing)
There is no requirments file in this repo because it uses dependencies pre-installed
on colab.
## Usage
This model is used with the huggingface transformers library it is run using the following:
```
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("ChrisPuzzo/llama-2-7b-privacy")
model = AutoModelForCausalLM.from_pretrained("ChrisPuzzo/llama-2-7b-privacy")
```
## Results
We did some small amouts of testing on the model however we got pretty unconclusive data. As you can see in the rouge
excel sheet, the scores weren't great. However we believe that the rouge testinging metric might not be the best way
to judge this.
## Next Steps
The next steps will be to train this data on another dataset I found summerizing text in privacy policies
|