original_docstring: Run stochastic gradient descent on binary cross-entropy loss for a given
    deep network
    
    Args:
      net (nn.Module): deep network whose parameters to optimize with SGD
    
docstring: Run stochastic gradient descent on binary cross-entropy loss for a given
  deep network
docstring_tokens: Run stochastic gradient descent on binary cross - entropy loss for a given deep network
code: def train(net):
    """Run stochastic gradient descent on binary cross-entropy loss for a given
    deep network
    
    Args:
      net (nn.Module): deep network whose parameters to optimize with SGD
    """
  
    # Set SGD hyperparameters
    n_iter = 200  # number of iterations of SGD
    learning_rate = 1e-3  # learning rate for SGD
    momentum = .99  # momentum parameter for SGD
    batch_size = 100  # number of data points in each mini-batch
  
    # Initialize binary cross-entropy loss function
    loss_fn = nn.BCELoss()
  
    # Initialize SGD optimizer with momentum
    optimizer = optim.SGD(net.parameters(), lr=learning_rate, momentum=momentum)
  
    # Placeholder to save loss at each iteration
    track_loss = []
  
    # Loop over iterations
    for i in range(n_iter):
  
      # Sample minibatch of oriented grating stimuli
      stimuli, tilt = sample_stimuli(batch_size)
  
      # Evaluate loss and update network weights
      out = net(stimuli)  # predicted probability of tilt right
      loss = loss_fn(out, tilt)  # evaluate loss
      optimizer.zero_grad()  # clear gradients
      loss.backward()  # compute gradients
      optimizer.step()  # update weights
      
      # Keep track of loss at each iteration
      track_loss.append(loss.item())
  
      # Track progress
      if (i + 1) % (n_iter / 10) == 0:
        print('iteration %i | loss: %.3f | percent correct: %.2f%%' % (i + 1, loss.item(), 100 * pcorrect(out, tilt)))
    
    # Plot loss
    plt.plot(track_loss)
    plt.xlabel('iterations of SGD')
    plt.ylabel('binary cross-entropy loss')
    plt.xlim([0, None])
    plt.ylim([0, None])
    plt.show()
code_tokens: def train ( net ) : n_iter = 200 learning_rate = 1e-3 momentum = .99 batch_size = 100 loss_fn = nn . BCELoss ( ) optimizer = optim . SGD ( net . parameters ( ) , lr = learning_rate , momentum = momentum ) track_loss = [ ] for i in range ( n_iter ) : stimuli , tilt = sample_stimuli ( batch_size ) out = net ( stimuli ) loss = loss_fn ( out , tilt ) optimizer . zero_grad ( ) loss . backward ( ) optimizer . step ( ) track_loss . append ( loss . item ( ) ) if ( i + 1 ) % ( n_iter / 10 ) == 0 : print ( 'iteration %i | loss: %.3f | percent correct: %.2f%%' % ( i + 1 , loss . item ( ) , 100 * pcorrect ( out , tilt ) ) ) plt . plot ( track_loss ) plt . xlabel ( 'iterations of SGD' ) plt . ylabel ( 'binary cross-entropy loss' ) plt . xlim ( [ 0 , None ] ) plt . ylim ( [ 0 , None ] ) plt . show ( )
short_docstring: Run stochastic gradient descent on binary cross-entropy loss for a given
  deep network
short_docstring_tokens: Run stochastic gradient descent on binary cross - entropy loss for a given deep network
comment: """Run stochastic gradient descent on binary cross-entropy loss for a given
    deep network
    
    Args:
      net (nn.Module): deep network whose parameters to optimize with SGD
    """$SEP$# Set SGD hyperparameters$SEP$# number of iterations of SGD$SEP$# learning rate for SGD$SEP$# momentum parameter for SGD$SEP$# number of data points in each mini-batch$SEP$# Initialize binary cross-entropy loss function$SEP$# Initialize SGD optimizer with momentum$SEP$# Placeholder to save loss at each iteration$SEP$# Loop over iterations$SEP$# Sample minibatch of oriented grating stimuli$SEP$# Evaluate loss and update network weights$SEP$# predicted probability of tilt right$SEP$# evaluate loss$SEP$# clear gradients$SEP$# compute gradients$SEP$# update weights$SEP$# Keep track of loss at each iteration$SEP$# Track progress$SEP$# Plot loss