title: Train BERT model from scratch on a different language
body: <p>First i create tokenizer as follow</p>
  <pre><code>from tokenizers import Tokenizer
  from tokenizers.models import BPE,WordPiece
  tokenizer = Tokenizer(WordPiece(unk_token=&quot;[UNK]&quot;))
  
  from tokenizers.trainers import BpeTrainer,WordPieceTrainer
  trainer = WordPieceTrainer(vocab_size=5000,min_frequency=3,
                       special_tokens=[&quot;[UNK]&quot;, &quot;[CLS]&quot;, &quot;[SEP]&quot;, &quot;[PAD]&quot;, &quot;[MASK]&quot;])
  
  from tokenizers.pre_tokenizers import Whitespace,WhitespaceSplit
  tokenizer.pre_tokenizer = WhitespaceSplit()
  tokenizer.train(files, trainer)
  
  from tokenizers.processors import TemplateProcessing
  tokenizer.token_to_id(&quot;[SEP]&quot;),tokenizer.token_to_id(&quot;[CLS]&quot;)
  tokenizer.post_processor = TemplateProcessing(
      single=&quot;[CLS] $A [SEP]&quot;,
      pair=&quot;[CLS] $A [SEP] $B:1 [SEP]:1&quot;,
      special_tokens=[
          (&quot;[CLS]&quot;, tokenizer.token_to_id(&quot;[CLS]&quot;)),
          (&quot;[SEP]&quot;, tokenizer.token_to_id(&quot;[SEP]&quot;)),
      ],
  )
  </code></pre>
  <p>Next, I want to train BERT model on these tokens. I tried as follow</p>
  <pre><code>from transformers import DataCollatorForLanguageModeling
  data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer,mlm=True, mlm_probability=0.15)
  </code></pre>
  <p>But it gives me an error
  <code>AttributeError: 'tokenizers.Tokenizer' object has no attribute 'mask_token'</code>
  &quot;This tokenizer does not have a mask token which is necessary for masked language modeling. &quot;
  Though I have <code>attention_mask</code>. Is is different than <code>mask token</code></p>