title: Load a model as DPRQuestionEncoder in HuggingFace
body: <p>I would like to load the BERT's weights (or whatever transformer) into a <a href="https://huggingface.co/transformers/model_doc/dpr.html#transformers.DPRQuestionEncoder" rel="nofollow noreferrer">DPRQuestionEncoder</a> architecture, such that I can use the HuggingFace <em>save_pretrained</em> method and plug the saved model into the <a href="https://github.com/huggingface/transformers/tree/master/examples/research_projects/rag" rel="nofollow noreferrer">RAG architecture to do end-to-end fine-tuning</a>.</p>
  <pre><code>from transformers import DPRQuestionEncoder
  model = DPRQuestionEncoder.from_pretrained('bert-base-uncased')
  </code></pre>
  <p>But I got the following error</p>
  <pre><code>You are using a model of type bert to instantiate a model of type dpr. This is not supported for all configurations of models and can yield errors.
  
  NotImplementedErrorTraceback (most recent call last)
  &lt;ipython-input-27-1f1b990b906b&gt; in &lt;module&gt;
  ----&gt; 1 model = DPRQuestionEncoder.from_pretrained(model_name)
        2 # https://github.com/huggingface/transformers/blob/41cd52a768a222a13da0c6aaae877a92fc6c783c/src/transformers/models/dpr/modeling_dpr.py#L520
  
  /opt/conda/lib/python3.8/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
     1211                     )
     1212 
  -&gt; 1213             model, missing_keys, unexpected_keys, error_msgs = cls._load_state_dict_into_model(
     1214                 model, state_dict, pretrained_model_name_or_path, _fast_init=_fast_init
     1215             )
  
  /opt/conda/lib/python3.8/site-packages/transformers/modeling_utils.py in _load_state_dict_into_model(cls, model, state_dict, pretrained_model_name_or_path, _fast_init)
     1286             )
     1287             for module in unintialized_modules:
  -&gt; 1288                 model._init_weights(module)
     1289 
     1290         # copy state_dict so _load_from_state_dict can modify it
  
  /opt/conda/lib/python3.8/site-packages/transformers/modeling_utils.py in _init_weights(self, module)
      515         Initialize the weights. This method should be overridden by derived class.
      516         &quot;&quot;&quot;
  --&gt; 517         raise NotImplementedError(f&quot;Make sure `_init_weigths` is implemented for {self.__class__}&quot;)
      518 
      519     def tie_weights(self):
  
  NotImplementedError: Make sure `_init_weigths` is implemented for &lt;class 'transformers.models.dpr.modeling_dpr.DPRQuestionEncoder'&gt;
  </code></pre>
  <p>I am using the last version of Transformers.</p>