title: How to get the size of a Hugging Face pretrained model?
body: <p>I keep getting a <code>CUDA out of memory</code> error when trying to fine-tune a Hugging Face pretrained XML Roberta model. So, the first thing I want to find out is the size of the pretrained model.</p>
  <pre><code>model = XLMRobertaForCausalLM.from_pretrained('xlm-roberta-base', config=config)
  device = torch.device(&quot;cuda&quot;) if torch.cuda.is_available() else torch.device(&quot;cpu&quot;)
  
  model.to(device)
  </code></pre>
  <p>I have tried to get the size of the model with</p>
  <pre><code>sys.getsizeof(model)
  </code></pre>
  <p>and, unsurprisingly, I get an incorrect result.  I get 56 as a result, which is the size of the python object.</p>
  <p>But then, I tried <code>model. element_size()</code>, and I get the error</p>
  <pre><code>ModuleAttributeError: 'XLMRobertaForCausalLM' object has no attribute 'element_size'
  </code></pre>
  <p>I have searched in the Hugging Face documentation, but I have not found how to do it. Does anyone here know how to do it?</p>