Refactor tokenize_text function to include better tokenization for Arabic Text a9aadc2 MohamedRashad commited on May 17, 2024
Add Arabic Tokenizers Leaderboard and Gradio Interface 203ba9b MohamedRashad commited on May 17, 2024