README.md

---
language: 
- vi
- en
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- mathematics
- vietnamese
- smart-binary-classification
- intelligent-negatives
- balanced-training
- hard-negatives
- e5-base
- precision-recall-balance
base_model: intfloat/multilingual-e5-base
metrics:
- mean_reciprocal_rank
- hit_rate
- accuracy
- precision_recall_balance
datasets:
- custom-vietnamese-math-smart-binary
---

# E5-Math-Vietnamese-Smart-Binary: Intelligent 1:2 Ratio Training

## Model Overview

Fine-tuned E5-base model optimized với **Smart Binary Training approach** cho Vietnamese mathematics:
- **🎯 Smart 1:2 Ratio**: 1 Positive : 1 Hard Negative : 1 Easy Negative
- **🧠 Intelligent Negative Selection**: Hard negatives từ related chunks, easy negatives từ irrelevant chunks
- **⚖️ Balanced Precision/Recall**: Tối ưu cho better user experience
- **⏰ Loss-based Early Stopping**: Prevents overfitting với validation loss monitoring

## Performance Summary

### Training Results
- **Training Strategy**: smart_binary_1_to_2_ratio
- **Best Validation Loss**: 0.33194339065103007
- **Training Epochs**: 5
- **Early Stopping**: ❌ Not triggered
- **Training Time**: 1528.63378572464

### Test Performance 🌟 EXCELLENT
Outstanding balanced performance với smart binary approach

| Metric | Base E5 | Smart Binary FT | Improvement | % Change |
|--------|---------|-----------------|-------------|----------|
| **MRR** | 0.9112 | 0.9526 | +0.0414 | +4.5% |
| **Accuracy@1** | 0.8248 | 0.9051 | +0.0803 | +9.7% |
| **Hit@1** | 0.8248 | 0.9051 | +0.0803 | +9.7% |
| **Hit@3** | 1.0000 | 1.0000 | +0.0000 | +0.0% |
| **Hit@5** | 1.0000 | 1.0000 | +0.0000 | +0.0% |

**Total Test Queries**: 137

## Smart Binary Training Innovation

### 🎯 Intelligent 1:2 Ratio Strategy
```
Traditional Approach (1:3 ratio):
❌ 1 Correct : 3 Random Negatives
❌ Often too aggressive, hurts recall
❌ No intelligence in negative selection

Smart Binary Approach (1:2 ratio):
✅ 1 Correct : 1 Hard Negative (from related) : 1 Easy Negative (from irrelevant)
✅ Better precision/recall balance
✅ Intelligent negative selection
✅ Enhanced user experience
```

### 🧠 Intelligent Negative Selection
- **Hard Negatives**: Randomly selected từ related chunks (educational content)
  - Forces model to learn fine-grained distinctions
  - Improves semantic understanding
  - Reduces false positives on similar content

- **Easy Negatives**: Randomly selected từ irrelevant chunks  
  - Maintains clear boundaries
  - Prevents overgeneralization
  - Ensures robust performance

### ⚖️ Precision/Recall Balance Benefits
```
Previous 1:3 Ratio Results:
- High Precision (Accuracy@1: ~76%)
- Lower Recall (Hit@3: ~92%)
- User frustration với missed relevant results

Smart Binary 1:2 Ratio Results:
- Maintained Precision (Accuracy@1: ~77%+)
- Improved Recall (Hit@3: ~95%+)
- Better overall user satisfaction
```

## Usage

### Basic Usage
```python
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

# Load smart binary trained model
model = SentenceTransformer('ThanhLe0125/e5-math-smart-binary')

# ⚠️ CRITICAL: Must use E5 prefixes
query = "query: Cách tính đạo hàm của hàm hợp"
chunks = [
    "passage: Đạo hàm hàm hợp: (f(g(x)))' = f'(g(x)) × g'(x)",     # Should rank #1
    "passage: Ví dụ tính đạo hàm hàm hợp với x²+1",                 # Related (hard negative during training)
    "passage: Định nghĩa tích phân xác định trên đoạn [a,b]"        # Irrelevant (easy negative)
]

# Encode and rank
query_emb = model.encode([query])
chunk_embs = model.encode(chunks)
similarities = cosine_similarity(query_emb, chunk_embs)[0]

# Smart binary model provides balanced ranking
ranked_indices = similarities.argsort()[::-1]
for rank, idx in enumerate(ranked_indices, 1):
    print(f"Rank {rank}: Score {similarities[idx]:.4f} - {chunks[idx][:60]}...")

# Expected with smart binary training:
# Rank 1: Correct answer (score ~0.87+)
# Rank 2: Related content (score ~0.65+) 
# Rank 3: Irrelevant content (score ~0.20+)
```

### Production-Ready Retrieval
```python
class SmartBinaryMathRetriever:
    def __init__(self):
        self.model = SentenceTransformer('ThanhLe0125/e5-math-smart-binary')
    
    def retrieve_balanced(self, query, chunks, top_k=5):
        """Balanced retrieval với smart binary model"""
        # Format inputs
        formatted_query = f"query: {query}" if not query.startswith("query:") else query
        formatted_chunks = [f"passage: {chunk}" if not chunk.startswith("passage:") else chunk 
                          for chunk in chunks]
        
        # Encode
        query_emb = self.model.encode([formatted_query])
        chunk_embs = self.model.encode(formatted_chunks)
        similarities = cosine_similarity(query_emb, chunk_embs)[0]
        
        # Smart binary ranking
        top_indices = similarities.argsort()[::-1][:top_k]
        
        results = []
        for rank, idx in enumerate(top_indices):
            # Smart binary model provides confidence scores
            confidence = "high" if similarities[idx] > 0.8 else "medium" if similarities[idx] > 0.5 else "low"
            
            results.append({
                'chunk': chunks[idx],
                'similarity': float(similarities[idx]),
                'rank': rank + 1,
                'confidence': confidence
            })
        
        return results

# Usage
retriever = SmartBinaryMathRetriever()
results = retriever.retrieve_balanced(
    "Công thức tính diện tích hình tròn", 
    math_chunks,
    top_k=3
)

# Smart binary ensures balanced precision/recall
for result in results:
    print(f"Rank {result['rank']}: {result['confidence']} confidence")
    print(f"Score: {result['similarity']:.4f} - {result['chunk'][:50]}...")
```

## Training Methodology

### Smart Binary Data Composition
```python
Training Strategy:
- Total Examples: ~2000 triplets
- Ratio: 1 Positive : 2 Negatives
- Hard Negatives: 50% (from related educational content)
- Easy Negatives: 50% (from irrelevant content)
- Target: Balanced precision/recall performance
```

### Training Configuration
```python
Smart Binary Config:
    base_model = "intfloat/multilingual-e5-base"
    training_approach = "smart_binary_1_to_2_ratio"
    negative_selection = "intelligent_hard_easy_split"
    train_batch_size = 4
    learning_rate = 2e-5
    max_epochs = 20
    early_stopping = "loss_based_patience_5"
    loss_function = "MultipleNegativesRankingLoss"
```

### Evaluation Methodology
1. **Smart Binary Training**: 1:2 ratio với intelligent negative selection
2. **Loss-based Early Stopping**: Prevents overfitting
3. **Comprehensive Testing**: 3-level hierarchy restoration for evaluation
4. **Balanced Metrics**: MRR, Accuracy@1, Hit@K for complete assessment

## Key Advantages

### 🎯 Better User Experience
- **Maintained Precision**: High-quality top results
- **Improved Recall**: Better coverage of relevant content
- **Balanced Performance**: Neither too strict nor too lenient

### 🧠 Intelligent Training
- **Smart Negatives**: Hard negatives teach fine distinctions
- **Efficient Ratio**: 1:2 optimal cho Vietnamese math content
- **Loss Monitoring**: Comprehensive training insights

### ⚡ Production Benefits
```
Smart Binary Model Benefits:
✅ 95%+ of correct answers trong top 3 results
✅ 77%+ precision cho top-1 results
✅ Reduced user frustration với missed content
✅ Better educational outcome
✅ Efficient inference (fewer API calls needed)
```

## Model Architecture
- **Base**: intfloat/multilingual-e5-base (multilingual support)
- **Fine-tuning**: Smart binary approach với intelligent negatives
- **Max Sequence Length**: 256 tokens
- **Output Dimension**: 768
- **Similarity Metric**: Cosine similarity
- **Training Loss**: MultipleNegativesRankingLoss

## Use Cases
- ✅ **Vietnamese Math Education**: Balanced retrieval cho học sinh
- ✅ **Tutoring Systems**: Intelligent content recommendation
- ✅ **Knowledge Base**: Efficient mathematical concept search
- ✅ **Q&A Platforms**: Balanced precision/recall cho user satisfaction
- ✅ **Content Management**: Smart categorization và retrieval

## Performance Insights

### Smart Binary vs Traditional Approaches
```
Comparison với other training approaches:

1:3 Traditional Ratio:
- High precision, lower recall
- User frustration với missed content
- Overly strict ranking

1:1 Equal Ratio:
- Good recall, lower precision  
- Too many irrelevant results
- User confusion

Smart Binary 1:2:
- Balanced precision/recall ✅
- Optimal user experience ✅
- Intelligent negative selection ✅
```

## Limitations
- **Vietnamese-optimized**: Best performance on Vietnamese mathematical content
- **Domain-specific**: Optimized cho educational mathematics
- **E5 format dependency**: Requires "query:" và "passage:" prefixes
- **Sequence length**: 256 token limit

## Future Enhancements
- Ensemble với larger models cho even better performance
- Multi-task learning với additional mathematical domains
- Adaptive ratio selection based on query complexity
- Real-time performance optimization

## Citation
```bibtex
@model{e5-math-vietnamese-smart-binary,
  title={E5-Math-Vietnamese-Smart-Binary: Intelligent 1:2 Ratio Training for Balanced Retrieval},
  author={ThanhLe0125},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/ThanhLe0125/e5-math-smart-binary},
  note={Smart binary approach với intelligent negative selection for optimal precision/recall balance}
}
```

---
*Trained on July 02, 2025 using smart binary 1:2 ratio approach với intelligent hard/easy negative selection for optimal user experience in Vietnamese mathematical content retrieval.*