title: huggingface-transformers: Train BERT and evaluate it using different attentions
body: <p>This is a clarification question. I am trying to train <a href="https://github.com/huggingface/transformers" rel="nofollow noreferrer">BERT provided by huggingface</a> using standard attention, and evaluate using a different attention definition.</p>
  
  <p>The operation I was thinking about was change <code>bert-base-uncased</code> to the path of my trained model(using standard attention) in the following command, and run <code>--do_eval</code> under the installation of my customized attention version.</p>
  
  <pre><code>export GLUE_DIR=/path/to/glue
  export TASK_NAME=MRPC
  
  python ./examples/run_glue.py \
      --model_name_or_path bert-base-uncased \
      --task_name $TASK_NAME \
      --do_eval \
      --data_dir $GLUE_DIR/$TASK_NAME \
      --max_seq_length 128 \
      --per_gpu_eval_batch_size=8   \
      --per_gpu_train_batch_size=8   \
      --learning_rate 2e-5 \
      --num_train_epochs 3.0 \
      --output_dir /tmp/$TASK_NAME/
  </code></pre>
  
  <p>However, I was getting unexpected results. So I want to make sure that I was using the right command. Could anyone confirm with me or correct me?</p>
  
  <p>Edited: The version was 2.8.0.</p>