title: Why TensorBoard summary is not updating?
body: <p>I use tensorboard with pytorch1.1 to log loss values.</p>
  
  <p>I use <code>writer.add_scalar("loss", loss.item(), global_step)</code> in every for- loop body.</p>
  
  <p>However, the plotting graph does not update while the training is processing.</p>
  
  <p>Every time I want to see the latest loss, I have to restart the tensorboard server.</p>
  
  <p>The code is here</p>
  
  <pre><code>import torch
  import torch.nn as nn
  import torch.nn.functional as F
  import torchvision
  from torch.utils.tensorboard import SummaryWriter
  from torchvision import datasets, transforms
  
  # Writer will output to ./runs/ directory by default
  writer = SummaryWriter()
  
  transform = transforms.Compose(
      [transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))]
  )
  trainset = datasets.MNIST("mnist_train", train=True, download=True, transform=transform)
  trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)
  model = torchvision.models.resnet50(False)
  # Have ResNet model take in grayscale rather than RGB
  model.conv1 = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)
  model.fc = nn.Linear(2048, 10, True)
  
  criterion = nn.CrossEntropyLoss()
  
  epochs = 100
  
  opt = torch.optim.Adam(model.parameters())
  
  niter = 0
  
  for epoch in range(epochs):
      for step, (x, y) in enumerate(trainloader):
          yp = model(x)
          loss = criterion(yp, y)
          opt.zero_grad()
          loss.backward()
          opt.step()
          writer.add_scalar("loss", loss.item(), niter)
          niter += 1
          print(loss.item())
  
  grid = torchvision.utils.make_grid(images)
  writer.add_image("images", grid, 0)
  writer.add_graph(model, images)
  writer.close()
  </code></pre>
  
  <p>The training is still going on, and the global steps has already been 3594. However, the tensorboard still shows around 1900. </p>
  
  <p><a href="https://i.stack.imgur.com/BjfPR.png" rel="noreferrer"><img src="https://i.stack.imgur.com/BjfPR.png" alt="enter image description here"></a></p>