title: Custom Loss in Pytorch where object does not have attribute backward()
body: <p>I am new to pytorch and I tried creating my own custom loss. This has been really challenging. Below is what I have for my loss.</p>
  
  <pre class="lang-py prettyprint-override"><code>class CustomLoss(nn.Module):
  
      def __init__(self,  size_average=True, reduce=True):
          """
  
          Args:
              size_average (bool, optional): By default, the losses are averaged
                 over observations for each minibatch. However, if the field
                 size_average is set to ``False``, the losses are instead summed for
                 each minibatch. Only applies when reduce is ``True``. Default: ``True``
              reduce (bool, optional): By default, the losses are averaged
                 over observations for each minibatch, or summed, depending on
                 size_average. When reduce is ``False``, returns a loss per input/target
                 element instead and ignores size_average. Default: ``True``
          """
          super(CustomLoss, self).__init__()      
  
  
      def forward(self, S, N, M, type='softmax',):
  
          return self.loss_cal(S, N, M, type)
  
  
  
      ### new loss cal
      def loss_cal(self, S, N, M, type="softmax",):
          """ calculate loss with similarity matrix(S) eq.(6) (7)
          :type: "softmax" or "contrast"
          :return: loss
          """
  
          self.A = torch.cat([S[i * M:(i + 1) * M, i:(i + 1)]
                                 for i in range(N)], dim=0)        
          self.A = torch.autograd.Variable(self.A)        
  
  
          if type == "softmax":
              self.B = torch.log(torch.sum(torch.exp(S.float()), dim=1, keepdim=True) + 1e-8)
              self.B = torch.autograd.Variable(self.B)       
              total = torch.abs(torch.sum(self.A - self.B))        
          else:
              raise AssertionError("loss type should be softmax or contrast !")
          return total
  </code></pre>
  
  <p>When I run the following:</p>
  
  <pre class="lang-py prettyprint-override"><code>loss = CustomLoss()          
  (loss.loss_cal(S=S,N=N,M=M))
  loss.backward()
  </code></pre>
  
  <p>I get the following error:</p>
  
  <pre class="lang-none prettyprint-override"><code>C:\Program Files\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
     2113             magic_arg_s = self.var_expand(line, stack_depth)
     2114             with self.builtin_trap:
  -&gt; 2115                 result = fn(magic_arg_s, cell)
     2116             return result
     2117 
  
  &lt;decorator-gen-60&gt; in time(self, line, cell, local_ns)
  
  C:\Program Files\Anaconda3\lib\site-packages\IPython\core\magic.py in &lt;lambda&gt;(f, *a, **k)
      186     # but it's overkill for just that one bit of state.
      187     def magic_deco(arg):
  --&gt; 188         call = lambda f, *a, **k: f(*a, **k)
      189 
      190         if callable(arg):
  
  C:\Program Files\Anaconda3\lib\site-packages\IPython\core\magics\execution.py in time(self, line, cell, local_ns)
     1178         else:
     1179             st = clock2()
  -&gt; 1180             exec(code, glob, local_ns)
     1181             end = clock2()
     1182             out = None
  
  &lt;timed exec&gt; in &lt;module&gt;()
  
  C:\Program Files\Anaconda3\lib\site-packages\torch\nn\modules\module.py in __getattr__(self, name)
      530                 return modules[name]
      531         raise AttributeError("'{}' object has no attribute '{}'".format(
  --&gt; 532             type(self).__name__, name))
      533 
      534     def __setattr__(self, name, value):
  
  AttributeError: 'CustomLoss' object has no attribute 'backward'
  </code></pre>
  
  <p>Why am I getting this error? I did not face this error with TF. My understanding is, it has to do with the autograd? If someone can explain why I am facing this error, I can figure the rest out. </p>