title: How to get output_attentions of a pretrained Distilbert Model?
body: <p>I am using a pretrained DistilBert model:</p>
  <pre><code>from transformers import TFDistilBertModel,DistilBertConfig
  
  dbert = 'distilbert-base-uncased'
  
  config = DistilBertConfig(max_position_embeddings=256 , dropout=0.2, 
                            attention_dropout=0.2, 
                            output_hidden_states=True,
                            output_attentions=True) #or true
  
  dbert_model = TFDistilBertModel.from_pretrained(dbert, config)
  
  input_ids_in = tf.keras.layers.Input(shape=(256,), name='input_id', dtype='int32')
  input_masks_in = tf.keras.layers.Input(shape=(256,), name='attn_mask', dtype='int32') 
  
  outputs = dbert_model([input_ids_in, input_masks_in], output_attentions = 1)
  </code></pre>
  <p>I am trying to get the output_attentions. But the output is of length 1 and is given as:</p>
  <blockquote>
  <p>TFBaseModelOutput([('last_hidden_state',
  &lt;KerasTensor: shape=(None, 256, 768) dtype=float32 (created by layer 'tf_distil_bert_model_6')&gt;)])</p>
  </blockquote>
  <p>i have given &quot;output_attentions = True&quot; in config and in forward pass &quot;output_attentions = 1&quot; is specified. Can anyone let me know what i am doing wrong?
  EDIT:
  I have changed the default configuration value of <code>max_positional_embeddings </code> of 512 to 256. if i change my model instantiation to</p>
  <pre><code>dbert_model = TFDistilBertModel.from_pretrained('distilbert-base-uncased',config=config)
  </code></pre>
  <p>it gives me the following error.</p>
  <pre><code>ValueError: cannot reshape array of size 393216 into shape (256,768)
  </code></pre>
  <p>768*512 being 393216. So it might be related to config code.</p>
  <p>Any ideas?</p>