<p>I am trying to send an API request to a Kubernetes Service using Cloud Composer. I have the following snippet:</p>
<pre><code>def fetchToken():
    ...


def start_application():
keycloak_token = fetchToken()
url = &quot;http://&lt;name&gt;.svc.cluster.local/tasks&quot;

with open(&quot;/home/airflow/gcs/dags/src/somefile.json&quot;) as f:
    d = json.load(f)

response = requests.put(
    url,
    headers={
        &quot;Authorization&quot;: f&quot;Bearer {keycloak_token}&quot;,
        &quot;Content-type&quot;: &quot;application/json&quot;,
    },
    data=json.dumps(d),
)
</code></pre>
<p>However, this gives me the following error:</p>
<pre><code>(HTTPConnectionPool(host='&lt;name&gt;.svc.cluster.local', port=80): Max retries exceeded with url: /tasks:8080 (Caused by NewConnectionError('&lt;urllib3.connection.HTTPConnection object at 0x7f03e0d97d60&gt;: Failed to establish a new connection: [Errno -2] Name or service not known')); 25941)
</code></pre>
<p>Seems like I cannot access as the URL is &quot;svc.cluster.local&quot;. Any idea how to access the endpoint?</p>
<p>Thanks!</p>

emilk
<p>Cloud composer is a google service independent from GKE, if you don't want to expose your api to public, you can create an <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balancing" rel="nofollow noreferrer">internal load balancer</a> to expose the service in the vpc, so by using the same vpc on cloud composer and GKE, your cloud composer dags could access the API in GKE.</p>
<p>There are many other options and tools to achieve this.</p>
