MorrBERT
MorrBERT is a Transformer-based Language Model designed specifically for the Moroccan Dialect. Developed by Moussaoui Otman and El Younoussi Yacine.
About MorrBERT
MorrBERT, specifically tailored for the Moroccan dialect, is structured identically to BERTBASE. The training process took approximately 120 hours to complete 12 epochs using the entire training set. A massive corpus of six million Moroccan dialect sentences, totaling 71 billion tokens, was utilized to train this model.
Usage
The model weights can be loaded using transformers library by HuggingFace.
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("otmangi/MorrBERT")
model = AutoModel.from_pretrained("otmangi/MorrBERT")
Acknowledgments
This research was supported through computational resources of HPC-MARWAN (www.marwan.ma/hpc) provided by the National Center for Scientific and Technical Research (CNRST). Rabat. Morocco.
Contact
For any inquiries, feedback, or requests, please feel free to reach out to :