Malaysian MaskLM
Collection
Trained on 17B tokens, 81GB of cleaned texts, able to understand standard Malay, local Malay, local Mandarin, Manglish, and local Tamil.
•
7 items
•
Updated
Pretrain BASE 512 masking context length DebertaV2 on Malaysian text
Special thanks to https://github.com/aisyahrzk for pretraining DebertaV2 Base.
WanDB at https://wandb.ai/aisyahrazak/deberta-base?nw=nwuseraisyahrazak
- Downloads last month
- 34
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model’s pipeline type.