README.md

---
title: MT564AITraining
emoji: 🚀
colorFrom: blue
colorTo: gray
sdk: docker
pinned: false
license: apache-2.0
short_description: MT564Model training

---


# SWIFT-MT564-Assistant
  Added MT564 TinyLlama training interface
✓ Created comprehensive training UI with file upload
✓ Integrated horoscope harvesting with MT564 training
✓ Both systems running in unified application
✓ Navigation links connect both functionalities

The application now provides both data harvesting for horoscopes AND MT564 TinyLlama training with a complete UI. You can access the MT564 training interface through the navigation menu.

## Project Overview

This project creates an AI-powered documentation assistant for financial messaging standards, specifically focused on the SWIFT MT564 message type. It combines web scraping, data processing, TinyLlama fine-tuning, and a user-friendly interface to provide an intelligent assistant for financial messaging professionals.

## Key Components

### 1. Data Collection & Processing

- **Web Scraper**: Extracts structured data from [ISO20022 SWIFT MT564 documentation](https://www.iso20022.org/15022/uhb/finmt564.htm)
- **PDF Parser**: Extracts text and structural information from uploaded SWIFT documentation PDFs
- **Data Formatter**: Converts scraped and parsed data into training examples for the model

### 2. Model Training Pipeline

- **TinyLlama Integration**: Implementation of TinyLlama, a smaller and more efficient LLM
- **Fine-tuning Scripts**: Specialized scripts for training on SWIFT message documentation
- **Evaluation Tools**: Methods to test the model's understanding of SWIFT message formats

### 3. User Interface

- **Web Application**: Flask-based interface for interacting with the model
- **PDF Upload**: Functionality to upload and process SWIFT documentation PDFs
- **Question-Answering System**: Interactive chat interface for asking questions about MT564 and related formats

## Technical Architecture

```
SWIFT-MT564-Assistant/
├── scrapers/                 # Web scraping components
│   ├── iso20022_scraper.py   # Scraper for ISO20022 website
│   ├── pdf_parser.py         # PDF extraction utilities
│   └── data_processor.py     # Converts raw data to training format
│
├── model/                    # ML model components
│   ├── tinyllama_trainer.py  # Fine-tuning implementation
│   ├── data_formatter.py     # Prepares data for training
│   └── evaluator.py          # Tests model performance
│
├── webapp/                   # Web application
│   ├── app.py                # Flask application
│   ├── templates/            # HTML templates
│   │   ├── index.html        # Main page
│   │   └── result.html       # Results display
│   └── static/               # CSS, JS, and other static files
│
├── data/                     # Data storage
│   ├── raw/                  # Raw scraped data
│   ├── processed/            # Processed training data
│   └── uploaded/             # User-uploaded PDFs
│
├── train_mt564_model.py      # Script to train the model
├── requirements.txt          # Project dependencies
└── README.md                 # Project documentation
```

## How It Works

1. **Data Collection Phase**:
   - The ISO20022 scraper extracts structured data from the SWIFT MT564 documentation
   - The data is processed and converted into a training dataset of instruction-response pairs

2. **Model Training Phase**:
   - TinyLlama is fine-tuned on the specialized SWIFT message format data
   - The model learns the structure, fields, and usage of MT564 messages

3. **User Interaction Phase**:
   - Users upload SWIFT documentation PDFs through the web interface
   - The system extracts and processes the PDF content
   - Users ask questions about SWIFT messages and receive accurate, contextual responses

## Installation & Setup

### Prerequisites

- Python 3.8+
- PyTorch
- Transformers library
- Flask
- PDF processing libraries

### Installation Steps

```bash
# Clone the repository
git clone <repository-url>
cd SWIFT-MT564-Assistant

# Create a virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Download and prepare the model
python prepare_mt564_data.py

# Run the web application
python main.py
```

## Usage

### Training the Model

```bash
# Run the scraper to collect data
python scrapers/iso20022_scraper.py

# Process the data
python scrapers/data_processor.py

# Train the model
python train_mt564_model.py
```

### Using the Web Interface

1. Start the Flask application: `python main.py`
2. Open a browser and navigate to: `http://localhost:5000`
3. Upload a SWIFT MT564 documentation PDF
4. Ask questions about the SWIFT message format

## Future Enhancements

- Expand coverage to additional SWIFT message types (MT565, MT566, etc.)
- Implement multi-document reasoning across different SWIFT standards
- Add support for ISO20022 MX message formats
- Develop specialized modules for message validation and conversion