File size: 1,258 Bytes
061a3c2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
---
language:
- en
tags:
- two-tower
- dual-encoder
- semantic-search
- document-retrieval
- information-retrieval
license: mit
datasets:
- ms_marco
---
# mlx7-two-tower-retrieval
This is a Two-Tower (Dual Encoder) model for document retrieval.
## Model Description
The Two-Tower model maps queries and documents to dense vector representations in the same semantic space, allowing for efficient similarity-based retrieval.
### Architecture
- **Tokenizer**: Character-level tokenization
- **Embedding**: Lookup embeddings with 64-dimensional vectors
- **Encoder**: Mean pooling with 128-dimensional hidden layer
## Intended Use
This model is designed for semantic search applications where traditional keyword matching is insufficient. It can be used to:
- Encode documents and queries into dense vector representations
- Retrieve relevant documents for a given query using vector similarity
- Build semantic search engines
## Limitations
- Limited context window (maximum sequence length of 64 tokens)
- English-language focused
- No contextual understanding beyond simple semantic similarity
## Training
- **Dataset**: MS MARCO passage retrieval dataset
- **Training Method**: Contrastive learning with triplet loss
- **Hardware**: NVIDIA GPU
|