TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior
Paper
•
2512.20757
•
Published
•
16
AI & ML interests
Contributors who are invited to beta-test our next big feature! Contact us if you want to join this team :-)
Recent Activity
View all activity
craffel
authored
a
paper
5 days ago
lvwerra
authored
a
paper
3 months ago
osanseviero
authored
a
paper
3 months ago
nguyenvulebinh
authored
5
papers
4 months ago
KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025
Paper
•
2505.13036
•
Published
ViCocktail: Automated Multi-Modal Data Collection for Vietnamese Audio-Visual Speech Recognition
Paper
•
2506.04635
•
Published
Cocktail-Party Audio-Visual Speech Recognition
Paper
•
2506.02178
•
Published
Fast and Accurate Capitalization and Punctuation for Automatic Speech Recognition Using Transformer and Chunk Merging
Paper
•
1908.02404
•
Published
Improving Vietnamese Named Entity Recognition from Speech Using Word Capitalization and Punctuation Recovery Models
Paper
•
2010.00198
•
Published
lvwerra
authored
a
paper
6 months ago
craffel
authored
a
paper
6 months ago
craffel
authored
a
paper
7 months ago
Post
3639
Having some fun with long context benchmarks (watch the video!!)
NoLiMA: NoLiMa: Long-Context Evaluation Beyond Literal Matching (2502.05167)
Fiction LiveBench: https://fiction.live/stories/Fiction-liveBench-Mar-25-2025/oQdzQvKHw8JyXbN87
Michalenglo: https://deepmind.google/research/publications/117639/
LongGenBench: Spinning the Golden Thread: Benchmarking Long-Form Generation in Language Models (2409.02076)
NeedleBench: NeedleBench: Can LLMs Do Retrieval and Reasoning in 1 Million Context Window? (2407.11963)
RULER: RULER: What's the Real Context Size of Your Long-Context Language Models? (2404.06654)
For more: https://www.reddit.com/r/rajistics/comments/1jxwk29/long_context_llm_benchmarks_video/
let me know if you like these posts
NoLiMA: NoLiMa: Long-Context Evaluation Beyond Literal Matching (2502.05167)
Fiction LiveBench: https://fiction.live/stories/Fiction-liveBench-Mar-25-2025/oQdzQvKHw8JyXbN87
Michalenglo: https://deepmind.google/research/publications/117639/
LongGenBench: Spinning the Golden Thread: Benchmarking Long-Form Generation in Language Models (2409.02076)
NeedleBench: NeedleBench: Can LLMs Do Retrieval and Reasoning in 1 Million Context Window? (2407.11963)
RULER: RULER: What's the Real Context Size of Your Long-Context Language Models? (2404.06654)
For more: https://www.reddit.com/r/rajistics/comments/1jxwk29/long_context_llm_benchmarks_video/
let me know if you like these posts
lvwerra
authored
a
paper
9 months ago
osanseviero
authored
a
paper
9 months ago
nguyenvulebinh
authored
a
paper
10 months ago
craffel
authored
a
paper
11 months ago
lvwerra
authored
a
paper
11 months ago
lvwerra
authored
a
paper
12 months ago
lvwerra
authored
a
paper
about 1 year ago
kobiso
authored
a
paper
about 1 year ago