|
--- |
|
language: |
|
- en |
|
pipeline_tag: zero-shot-classification |
|
tags: |
|
- finance |
|
- compliance |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
<!-- |
|
--> |
|
|
|
## Model Details |
|
Based of the full weight llama 2-hermes from Nous Research. |
|
|
|
### Model Description |
|
This model was fine tuned off the full weight llama-2-hermes-7B from Nous Research. This model is a preemptive V1, and a hastily put together model to assist |
|
in finance and compliance tasks, mostly tuned to the new SEC Marketing and Compliance rules established in 2021. Later iterations will have more guidelines and rulings |
|
unrelated to the SEC Marketing rule. |
|
https://www.sec.gov/files/rules/final/2020/ia-5653.pdf |
|
<!-- --> |
|
|
|
|
|
|
|
- **Developed by:** [More Information Needed] |
|
- **Shared by [optional]:** [More Information Needed] |
|
- **Model type:** [More Information Needed] |
|
- **Language(s) (NLP):** [Enlgish] |
|
- **License:** [More Information Needed] |
|
- **Finetuned from model [optional]:** [llama 2-hermes-7b] |
|
|
|
### Model Sources [optional] |
|
|
|
<!-- Provide the basic links for the model. --> |
|
|
|
- **Repository:** [More Information Needed] |
|
- **Paper [optional]:** [More Information Needed] |
|
- **Demo [optional]:** [More Information Needed] |
|
|
|
## Uses |
|
|
|
This is to help companies and individuals within compliance and marketing departments to determine and find issues within their marketing or public facing documents. |
|
Since the new marketing rule is principles based it requires logic, experience, and reasoning to determine if a statement or advertisement would be compliant within |
|
the SEC's new guidelines. This can lead to multiple viewpoints of compliant or not depending on the viewer. Thus this is a small/high quality dataset version |
|
to aid or provide an second viewpoint of a public facing statement to help determine if something is compliant per the SEC's guidelines. The dataset was crafted by |
|
reviewing the SEC Marketing rule, other scenarios, and providing reasoning within the ###n\ Response n\### to help guide the model in reasoning tasks. |
|
Further versions will be reviewed more for accuracy, bias, and more data. |
|
|
|
<!-- --> |
|
|
|
### Direct Use |
|
|
|
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. --> |
|
|
|
[More Information Needed] |
|
|
|
### Downstream Use [optional] |
|
|
|
For use by marketing and compliance finance teams to assist in determination and interpretation of SEC Marketing rule and other SEC interpretations. No outputs should be guaranteed as fact, |
|
and review of data is encouraged. This is to simply assist, and aid those in remembering certain aspects and interpretation of aspects of the long SEC Marketing guidelines |
|
amongst other SEC rulings. |
|
|
|
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app --> |
|
|
|
[More Information Needed] |
|
|
|
### Out-of-Scope Use |
|
|
|
This model should not be intended to be used as fact, as evidence/proof in a trial hearing, or be used as indication of innocence in an SEC audit/investigation. |
|
This model should be used by professionals deeply familiar with the SEC's guidelines and compliance procedures. |
|
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. --> |
|
|
|
[More Information Needed] |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
This is the first model iteration, and has not be fully reviewed by multiple professional peers for its accuracy, bias, and output variations. |
|
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. |
|
<!-- This section is meant to convey both technical and sociotechnical limitations. --> |
|
|
|
[More Information Needed] |
|
|
|
### Recommendations |
|
|
|
<!-- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. --> |
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model. |
|
|
|
[More Information Needed] |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
<!-- --> |
|
|
|
|
|
### Training Procedure |
|
|
|
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
- <!--# Compute dtype for 4-bit base models |
|
bnb_4bit_compute_dtype = "float16" |
|
|
|
bnb_4bit_quant_type = "nf4" |
|
use_nested_quant = False |
|
fp16 = False |
|
bf16 = False - this will be True for next training run. |
|
per_device_train_batch_size = 4 |
|
per_device_eval_batch_size = 4 |
|
gradient_accumulation_steps = 1 |
|
gradient_checkpointing = True |
|
max_grad_norm = 0.3 |
|
learning_rate = 2e-5 -1 e-4 for a 13B will be applied. |
|
weight_decay = 0.001 |
|
optim = "paged_adamw_32bit" |
|
lr_scheduler_type = "constant" |
|
max_steps = 13000 |
|
warmup_ratio = 0.03 |
|
group_by_length = True |
|
--> |
|
|
|
|
|
|
|
## Evaluation |
|
|
|
<!-- This section describes the evaluation protocols and provides the results. --> |
|
|
|
### Testing Data, Factors & Metrics |
|
|
|
#### Testing Data |
|
|
|
<!-- This should link to a Data Card if possible. --> |
|
|
|
|
|
|
|
[More Information Needed] |
|
|
|
#### Metrics |
|
|
|
<!-- --> |
|
|
|
[More Information Needed] |
|
|
|
### Results |
|
|
|
[More Information Needed] |
|
|
|
#### Summary |
|
|
|
|
|
|
|
## Model Examination [optional] |
|
|
|
<!-- Relevant interpretability work for the model goes here --> |
|
|
|
[More Information Needed] |
|
|
|
### Model Architecture and Objective |
|
|
|
[More Information Needed] |
|
|
|
### Compute Infrastructure |
|
|
|
[Google Colab] |
|
|
|
#### Hardware |
|
|
|
[1xA100] |
|
|
