title: Error: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!"
body: <p>I'm running this code on Azure machine learning notebook.</p>
  <pre><code>import os
  import torch
  import gradio as gr
  from vilmedic import AutoModel
  from vilmedic.blocks.scorers import RadGraph
  import glob
  
  model, processor = AutoModel.from_pretrained(&quot;rrg/baseline-mimic&quot;)
  device = torch.device(&quot;cuda&quot;) if torch.cuda.is_available() else torch.device(&quot;cpu&quot;)
  model = model.to(device)
  radgraph = RadGraph(cuda=-1)
  </code></pre>
  <p>.....</p>
  <pre><code>def run(image, beam_size, num_return_sequences, include_words, exclude_words, do_radgraph):
      if image is None:
          return {}, '&lt;b&gt;Please select an image&lt;/b&gt;'  # , &quot;&quot;
      if num_return_sequences &gt; beam_size:
          return {}, '&lt;b&gt;&quot;Beam size&quot;&lt;/b&gt; must be greater or equal than &lt;b&gt;&quot;Number of generated reports&quot;&lt;/b&gt;'  # , &quot;&quot;
  
      try:
          include_words_ids, include_words = get_token_from_strings(include_words)
          exclude_words_ids, exclude_words = get_token_from_strings(exclude_words)
  
          if include_words_ids is not None and [3] in include_words_ids:
              return {}, '&lt;b&gt;&quot;' + include_words[
                  include_words_ids.index([3])] + '&quot;&lt;/b&gt; is not in the vocabulary&quot;&lt;/b&gt;'  # , &quot;&quot;
  
          with torch.no_grad():
              batch = processor.inference(image=[
                  [image]
              ])
              batch_size = 1
              encoder_output, encoder_attention_mask = model.encode(**batch)
              expanded_idx = torch.arange(batch_size).view(-1, 1).repeat(1, beam_size).view(-1)
              input_ids = torch.ones((len(batch[&quot;images&quot;]), 1), dtype=torch.long)
              if torch.cuda.is_available():
                  expanded_idx = expanded_idx.cuda()
                  print(&quot;89 line&quot;)
                  input_ids = input_ids.cuda()
  
              # Using huggingface generate method
              hyps = model.dec.generate(
                  input_ids=input_ids * model.dec.config.bos_token_id,
                  encoder_hidden_states=encoder_output.index_select(0, expanded_idx),
                  encoder_attention_mask=encoder_attention_mask.index_select(0, expanded_idx),
                  num_return_sequences=num_return_sequences,
                  max_length=processor.tokenizer_max_len,
                  num_beams=beam_size,
                  bad_words_ids=exclude_words_ids,
                  force_words_ids=include_words_ids,
              )
  
              # Decode
              hyps = [processor.tokenizer.decode(h, skip_special_tokens=True, clean_up_tokenization_spaces=False) for h in
                      hyps]
  
              # RadGraph
              if do_radgraph:
                  radgraph_annots = [radgraph(hyps=[h], refs=[h])[-1][0][&quot;entities&quot;] for h in hyps]
                  # Find entites : Radgraph
                  new_hyp_strs = []
                  for hyp_str, radgraph_annot in zip(hyps, radgraph_annots):
                      values = radgraph_annot.values()
                      new_hyp_str = hyp_str.split()
                      for v in values:
                          new_hyp_str[v[&quot;start_ix&quot;]] = highlight_radgraph_entities(v[&quot;tokens&quot;], v[&quot;label&quot;])
                      new_hyp_strs.append(' '.join(new_hyp_str))
              else:
                  new_hyp_strs = hyps
  
              # Find user entites
              if include_words is not None:
                  for w in include_words:
                      new_hyp_strs = [h.replace(w, highlight_word(w, &quot;user&quot;)) for h in new_hyp_strs]
  
              # Formating
              new_hyp_strs = [&quot;&lt;p&gt;&lt;b&gt;Hypothesis {}:&lt;/b&gt; &lt;br/&gt; {} &lt;/p&gt;&quot; \
                              &quot;&quot;.format(i + 1, h) for i, h in enumerate(new_hyp_strs)] + (
                                 [&quot;&lt;br/&gt;&lt;br/&gt;&lt;i&gt;Anat: anatomy&lt;br/&gt;&quot;
                                  &quot;OBS: observation&lt;br/&gt;&quot;
                                  &quot;DA: definitely absent&lt;br/&gt;&quot;
                                  &quot;DP: definitely present&lt;/i&gt;&quot;] if do_radgraph else [&quot;&quot;])
  
              # Params
              out_json = {
                  &quot;beam size&quot;: beam_size, &quot;number of generated reports&quot;: num_return_sequences,
                  &quot;included words&quot;: include_words, &quot;excluded words&quot;: exclude_words, &quot;show radgraph&quot;: do_radgraph
              }
  
              return out_json, str(''.join(new_hyp_strs))  # , str(refs[os.path.basename(image)])
  
      except Exception as e:
          print(e)
          return {}, &quot;&lt;b&gt;An error occured, try again...&quot;
  
  </code></pre>
  <p>The full code is here:</p>
  <p><a href="https://huggingface.co/spaces/StanfordAIMI/radiology_report_generation/blob/main/app.py" rel="nofollow noreferrer">https://huggingface.co/spaces/StanfordAIMI/radiology_report_generation/blob/main/app.py</a></p>
  <p>It keeps giving me this error when I upload an image and click the submit button:
  <strong>Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!</strong>.
  I'm using an azure computation with following specifications:
  Virtual machine size: <em>Standard_NV6 (6 cores, 56 GB RAM, 380 GB disk)</em>
  Processing unit: <em>GPU - 1 x NVIDIA Tesla M60</em></p>