original_docstring: LR schedule that should yield 76% converged accuracy with batch size 256
docstring: LR schedule that should yield 76% converged accuracy with batch size 256
docstring_tokens: LR schedule that should yield 76% converged accuracy with batch size 256
code: def adjust_learning_rate(optimizer, epoch, step, len_epoch):
      """LR schedule that should yield 76% converged accuracy with batch size 256"""
      factor = epoch // 30
  
      if epoch >= 80:
          factor = factor + 1
  
      lr = args.lr*(0.1**factor)
  
      """Warmup"""
      if epoch < 5:
          lr = lr*float(1 + step + epoch*len_epoch)/(5.*len_epoch)
      
      wandb.log({"cur_lr": lr})
  
      # if(args.local_rank == 0):
      #     print("epoch = {}, step = {}, lr = {}".format(epoch, step, lr))
  
      for param_group in optimizer.param_groups:
          param_group['lr'] = lr
code_tokens: def adjust_learning_rate ( optimizer , epoch , step , len_epoch ) : factor = epoch // 30 if epoch >= 80 : factor = factor + 1 lr = args . lr * ( 0.1 ** factor ) """Warmup""" if epoch < 5 : lr = lr * float ( 1 + step + epoch * len_epoch ) / ( 5. * len_epoch ) wandb . log ( { "cur_lr" : lr } ) for param_group in optimizer . param_groups : param_group [ 'lr' ] = lr
short_docstring: LR schedule that should yield 76% converged accuracy with batch size 256
short_docstring_tokens: LR schedule that should yield 76% converged accuracy with batch size 256
comment: """LR schedule that should yield 76% converged accuracy with batch size 256"""$SEP$"""Warmup"""$SEP$# if(args.local_rank == 0):$SEP$#     print("epoch = {}, step = {}, lr = {}".format(epoch, step, lr))