Over which devices do we split each training batch.

  In old-fashioned async mode, we split the batch over all GPUs on the
  current worker.

  In sync mode, we split the batch over all the parameter server GPUs.

  This function returns an expert_utils.Parallelism object, which can be used
  to build the model.  It is configured in a way that any variables created
  by `tf.get_variable` will be assigned to the parameter servers and shared
  between datashards.

  Args:
    daisy_chain_variables: whether to copy variables in a daisy chain on GPUs.
    all_workers: whether the devices are all async workers or just this one.

  Returns:
    a expert_utils.Parallelism.

def data_parallelism_from_flags(daisy_chain_variables=True, all_workers=False):
  """Over which devices do we split each training batch.

  In old-fashioned async mode, we split the batch over all GPUs on the
  current worker.

  In sync mode, we split the batch over all the parameter server GPUs.

  This function returns an expert_utils.Parallelism object, which can be used
  to build the model.  It is configured in a way that any variables created
  by `tf.get_variable` will be assigned to the parameter servers and shared
  between datashards.

  Args:
    daisy_chain_variables: whether to copy variables in a daisy chain on GPUs.
    all_workers: whether the devices are all async workers or just this one.

  Returns:
    a expert_utils.Parallelism.
  """
  dp_arg_names = inspect.getargspec(data_parallelism).args

  blacklist = ["daisy_chain_variables", "all_workers"]

  kwargs = {}
  for arg in dp_arg_names:
    if arg in blacklist:
      continue
    kwargs[arg] = getattr(tf.flags.FLAGS, arg)

  return data_parallelism(
      daisy_chain_variables=daisy_chain_variables,
      all_workers=all_workers,
      **kwargs)