<p>I experiment a weird error with my Dataflow pipeline when I want to use specific library from PyPI.</p>

<p>I need <code>jsonschema</code> in a ParDo, so, in my <code>requirements.txt</code>file, I added <code>jsonschema==3.2.0</code>. 
I launch my pipeline with the command line below:</p>

<pre class="lang-sh prettyprint-override"><code>python -m gcs_to_all \
    --runner DataflowRunner \
    --project &lt;my-project-id&gt; \
    --region europe-west1 \
    --temp_location gs://&lt;my-bucket-name&gt;/temp/ \
    --input_topic "projects/&lt;my-project-id&gt;/topics/&lt;my-topic&gt;" \
    --network=&lt;my-network&gt; \
    --subnetwork=&lt;my-subnet&gt; \
    --requirements_file=requirements.txt \
    --experiments=allow_non_updatable_job \
    --streaming  
</code></pre>

<p>In the terminal, all seems to be good: </p>

<pre class="lang-sh prettyprint-override"><code>INFO:root:2020-01-03T09:18:35.569Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in europe-west1-b.
INFO:root:2020-01-03T09:18:35.806Z: JOB_MESSAGE_WARNING: The network default doesn't have rules that open TCP ports 12345-12346 for internal connection with other VMs. Only rules with a target tag 'dataflow' or empty target tags set apply. If you don't specify such a rule, any pipeline with more than one worker that shuffles data will hang. Causes: Firewall rules associated with your network don't open TCP ports 12345-12346 for Dataflow instances. If a firewall rule opens connection in these ports, ensure target tags aren't specified, or that the rule includes the tag 'dataflow'.
INFO:root:2020-01-03T09:18:48.549Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
</code></pre>

<p>Where's no error in the log tab on Dataflow webpage, but in stackdriver:</p>

<pre><code>message: "Error syncing pod 6515c378c6bed37a2c0eec1fcfea300c ("&lt;dataflow-id&gt;--01030117-c9pc-harness-5lkv_default(6515c378c6bed37a2c0eec1fcfea300c)"), skipping: [failed to "StartContainer" for "sdk0" with CrashLoopBackOff: "Back-off 10s restarting failed container=sdk0 pod=&lt;dataflow-id&gt;--01030117-c9pc-harness-5lkv_default(6515c378c6bed37a2c0eec1fcfea300c)""
message: ", failed to "StartContainer" for "sdk1" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=sdk1 pod=&lt;dataflow-id&gt;--01030117-c9pc-harness-5lkv_default(6515c378c6bed37a2c0eec1fcfea300c)"" 
message: ", failed to "StartContainer" for "sdk2" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=sdk2 pod=&lt;dataflow-id&gt;--01030117-c9pc-harness-5lkv_default(6515c378c6bed37a2c0eec1fcfea300c)"" 
message: ", failed to "StartContainer" for "sdk3" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=sdk3 pod=&lt;dataflow-id&gt;--01030117-c9pc-harness-5lkv_default(6515c378c6bed37a2c0eec1fcfea300c)"" 
</code></pre>

<p>I find this error too (in info mode):</p>

<pre><code>Collecting jsonschema (from -r /var/opt/google/staged/requirements.txt (line 1))
  Installing build dependencies: started
Looking in links: /var/opt/google/staged
  Installing build dependencies: started
Collecting jsonschema (from -r /var/opt/google/staged/requirements.txt (line 1))
  Installing build dependencies: started
Looking in links: /var/opt/google/staged
Collecting jsonschema (from -r /var/opt/google/staged/requirements.txt (line 1))
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python3 /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-mdurhav9/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links /var/opt/google/staged -- 'setuptools&gt;=40.6.0' wheel
       cwd: None
  Complete output (5 lines):
  Looking in links: /var/opt/google/staged
  Collecting setuptools&gt;=40.6.0
  Collecting wheel
    ERROR: Could not find a version that satisfies the requirement wheel (from versions: none)
  ERROR: No matching distribution found for wheel
</code></pre>

<p>But I don't know why it can get this dependency...</p>

<p>Do you have any idea how I can debug this? or why I encounter this error?</p>

<p>Thanks</p>

<p>When Dataflow workers start, they execute several steps:</p>

<ol>
<li>Install packages from <code>requirements.txt</code></li>
<li>Install packages  specified as <code>extra_packages</code></li>
<li>Install workflow tarball and execute actions provided in <code>setup.py</code>.</li>
</ol>

<p><code>Error syncing pod</code> with <code>CrashLoopBackOff</code> message can be related to dependency conflict. You need to verify that there are no conflicts with the libraries and versions used for the job. Please refer to the <a href="https://beam.apache.org/documentation/sdks/python-pipeline-dependencies" rel="noreferrer">documentation</a> for staging required dependencies of the pipeline.</p>

<p>Also, take a look for <a href="https://cloud.google.com/dataflow/docs/concepts/sdk-worker-dependencies#sdk-for-python" rel="noreferrer">preinstalled dependencies</a> and this <a href="https://stackoverflow.com/a/54073000/9398998">StackOverflow thread</a>. </p>

<p>What you can try is change the version of <code>jsonschema</code> and try run it again. If it wouldn't help, please provide <code>requirements.txt</code> file.</p>

<p>I hope it will help you.</p>
