README.md

metadata

language: en
tags:
  - financial-analysis
  - covenant-extraction
  - llama
  - lora
license: llama2
datasets:
  - custom_financial_covenants
metrics:
  - accuracy
pipeline_tag: text-generation
inference: true
library_name: transformers
widget:
  - text: >
      ### Instruction: Extract covenant details from the following credit
      agreement section and structure it into JSON format only.


      ### Input: Section 4.2:

      The Borrower shall maintain a Fixed Charge Coverage Ratio of not less than
      1.25:1.00 for any fiscal quarter ending after June 30, 2024.


      ### Response:
model-index:
  - name: covenant-extractor
    results:
      - task:
          type: text2json
          name: Financial Covenant Extraction
        metrics:
          - type: accuracy
            value: 90
            name: Test Accuracy

Covenant Extractor Model

This model is fine-tuned on Llama-3.2-3B-Instruct for extracting and structuring financial covenants from credit agreements into standardized JSON format.

Model Description

• Base Model: meta-llama/Llama-3.2-3B-Instruct
• Task: Financial Covenant Extraction
• Training Method: LoRA Fine-tuning
• Language: English
• License: Same as base model

Intended Use

This model is designed to:

• Extract covenant details from credit agreement sections
• Structure the information into standardized JSON format
• Handle various types of financial covenants (leverage ratios, coverage ratios, etc.)

Input Format

### Instruction: Extract covenant details from the following credit agreement section and structure it into JSON format only.

### Input: Section 4.2:
The Borrower shall maintain a Fixed Charge Coverage Ratio of not less than 1.25:1.00 for any fiscal quarter ending after June 30, 2024.

### Response:

Output Format

{
    "type": "financial",
    "category": "fixed_charge_coverage_ratio",
    "section": "4.2",
    "requirements": {
        "threshold": "1.25:1.00",
        "measurement_period": "quarterly",
        "timeline": ["June 30, 2024"]
    }
}

Training Details

• Training Method: LoRA (Low-Rank Adaptation)
• LoRA Config:
  • Rank: 16
  • Alpha: 32
  • Target Modules: q_proj, k_proj, v_proj, o_proj
  • Dropout: 0.1
• Training Parameters:
  • Batch Size: 4
  • Gradient Accumulation Steps: 16
  • Learning Rate: 1e-4
  • Number of Epochs: 3
  • Weight Decay: 0.01
  • Max Gradient Norm: 1.0

Limitations

• Only processes English language credit agreements
• Best suited for standard financial covenants
• May require adjustment for complex or non-standard covenant structures

Citation

If you use this model in your work, please cite:

@misc{covenant-extractor,
  author = {[Bikram Adhikari]},
  title = {Covenant Extractor: Fine-tuned LLM for Financial Covenant Analysis},
  year = {2024}
}