title: how can I do calibration for 8-bit quantization using KL Divergence
body: <p>I have FP32 tensor inputs which shape are [1, 4, 1024, 256]
  I need to quantize the tensor to INT8, but naive quantization has triggered a problem in my NLP model that actually get rid of EOS.</p>
  <p>So I have to do calibration rather than go with 'absolute min, max value'. And I have a problem with doing KLD calculation.</p>
  <pre><code>INPUT : A (tensor of FP32, [1, 4, 1024, 256])
  A_int8 = A.clone().to(torch.int8).cpu()
  P = norm.pdf(A)
  Q = norm.pdf(A_int8)
  P = torch.from_numpy(P)
  Q = torch.from_numpy(Q)
  
  kld = torch.nn.KLDivLoss(size_average=False)(A_int8, A)
  print(kld)    
  
  # After this, I'm going to apply self-attention operation.
  # B_int8 = A_int8.clone()
  # AB = A_int8.matmul(B_int8.transpose(-1, -2))
  </code></pre>
  <p>And I get the negative value for kld. Please help me with this.</p>