title: How to wait until a job is done or a file is updated in airflow
body: <p>I am trying to use apache-airflow, with google cloud-composer, to shedule batch processing that result in the training of a model with google ai platform. I failed to use airflow operators as I explain in this question <a href="https://stackoverflow.com/questions/60194310/unable-to-specify-master-type-in-mlenginetrainingoperator">unable to specify master_type in MLEngineTrainingOperator</a></p>
  
  <p>Using the command line I managed to launch a job successfully.
  So now my issue is to integrate this command in airflow.</p>
  
  <p>Using BashOperator I can train the model but I need to wait for the job to be completed before creating a version and setting it as the default. This DAG create a version before the job is done</p>
  
  <pre class="lang-py prettyprint-override"><code>    bash_command_train = "gcloud ai-platform jobs submit training training_job_name " \
                           "--packages=gs://path/to/the/package.tar.gz " \
                           "--python-version=3.5 --region=europe-west1 --runtime-version=1.14" \
                           " --module-name=trainer.train --scale-tier=CUSTOM --master-machine-type=n1-highmem-16"
      bash_train_operator = BashOperator(task_id='train_with_bash_command',
                                         bash_command=bash_command_train,
                                         dag=dag,)
  
  
  
      ...
      create_version_op = MLEngineVersionOperator(
          task_id='create_version',
          project_id=PROJECT,
          model_name=MODEL_NAME,
          version={
              'name': version_name,
              'deploymentUri': export_uri,
              'runtimeVersion': RUNTIME_VERSION,
              'pythonVersion': '3.5',
              'framework': 'SCIKIT_LEARN',
          },
          operation='create')
  
      set_version_default_op = MLEngineVersionOperator(
          task_id='set_version_as_default',
          project_id=PROJECT,
          model_name=MODEL_NAME,
          version={'name': version_name},
          operation='set_default')
  
      # Ordering the tasks
      bash_train_operator &gt;&gt; create_version_op &gt;&gt; set_version_default_op
  </code></pre>
  
  <p>The training result in updating of a file in Gcloud storage. So I am looking for an operator or a sensor that will wait until this file is updated, I noticed GoogleCloudStorageObjectUpdatedSensor, but I dont know how to make it retry until this file is updated. 
  An other solution would be to check for the job to be completed, but I can't find how too.</p>
  
  <p>Any help would be greatly appreciated. </p>