title: GCP Dataflow Computation Graph and Job Execution
body: <p>Hi Everyone I tried hard to understand what is happening when I create a custom template in Google cloud Dataflow but failed to understand. Thanks to GCP documentations. Below is what I am achieving.</p>
  <ol>
  <li>Read Data from Google cloud Bucket</li>
  <li>Pre-Process it</li>
  <li>Load Deeplearning models (1 GB each) and get the predictions</li>
  <li>Dump the results in BigQuery.</li>
  </ol>
  <p>I successfully created the template and I am able to execute the job. But I have following questions.</p>
  <ol>
  <li>When I  execute the job, Everytime the models (5 models and each of 1GB) gets downloaded during execution <strong>OR</strong> the models are loaded and placed in the template (Execution Graph) and during execution it uses the loaded ones</li>
  <li>If loading of the models happen only during the job execution, then does it not impact the execution time? Since it has to load GBs of Model files everytime the job is triggered?</li>
  <li>Can multiple users trigger the same template at same time? Since I want to productionize it, I am not sure how this will handle multiple requests at same time?</li>
  </ol>
  <p>Can anyone please share some information on it?</p>
  <p>Sources I referred and failed to get the answer: <a href="https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#pipeline-lifecycle-from-pipeline-code-to-dataflow-job" rel="nofollow noreferrer">https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#pipeline-lifecycle-from-pipeline-code-to-dataflow-job</a> <a href="http://alumni.media.mit.edu/%7Ewad/magiceight/isa/node3.html" rel="nofollow noreferrer">http://alumni.media.mit.edu/~wad/magiceight/isa/node3.html</a> <a href="https://cloud.google.com/dataflow/docs/guides/setting-pipeline-options#configuring-pipelineoptions-for-local-execution" rel="nofollow noreferrer">https://cloud.google.com/dataflow/docs/guides/setting-pipeline-options#configuring-pipelineoptions-for-local-execution</a> <a href="https://beam.apache.org/documentation/basics/" rel="nofollow noreferrer">https://beam.apache.org/documentation/basics/</a> <a href="https://beam.apache.org/documentation/runtime/model/" rel="nofollow noreferrer">https://beam.apache.org/documentation/runtime/model/</a> <a href="https://mehmandarov.com/apache-beam-pipeline-graph/" rel="nofollow noreferrer">https://mehmandarov.com/apache-beam-pipeline-graph/</a></p>