|
--- |
|
library_name: transformers |
|
license: apache-2.0 |
|
language: |
|
- ja |
|
pipeline_tag: fill-mask |
|
widget: |
|
- text: 彼のダンス、めっちゃ[MASK]！😂 |
|
--- |
|
|
|
# BERT for Japanese Twitter |
|
|
|
This is a [Japanese BERT](https://huggingface.co/tohoku-nlp/bert-base-japanese-v3) that has been adapted to Twitter. |
|
|
|
It began with the base Japnese BERT by Tohoku NLP and continued pretraining on a Twitter corpus. |
|
|
|
It is reccomended to use with Japanese SNS tasks. |
|
|
|
|
|
## Training Data |
|
|
|
The Twitter API was used to collect Japnaese Tweets from June 2022 to April 2023. |
|
|
|
N-gram based deduplication was used to reduce spam content and improve the diversity of the training corpus. |
|
The refined training corpus was 28 million tweets. |
|
|
|
## Tokenization |
|
|
|
The vocabulary was prepared using the [WordPieceTrainer](https://huggingface.co/docs/tokenizers/api/trainers) with the Twitter training corpus. |
|
It shares 60% of its vocabulary with Japanese BERT. |
|
|
|
The vocuabulary includes colloquialisms, neologisms, emoji and kaomoji expressions that are common on Twitter. |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. |
|
|
|
- **Developed by:** Jordan Wolfgang Klein, as Master's candiate at the University Malta. |
|
- **Model type:** BERT |
|
- **Language(s) (NLP):** Japanese |
|
- **License:** [More Information Needed] |
|
- **Finetuned from model [optional]:** |
