<p>I have deployed a demo service (running on port 8000) in our K8s environment which has Istio installed (1.5.6, default profile).
When I make a call from outside the cluster to the public address, it succeeds.
When I make a call from a pod inside the cluster to the internal cluster address, it fails with response code 503.</p>
<p>When I change my Virtual Service to use the port instead of the subset, then it succeeds in both cases (external and internal call).</p>
<p>Any ideas what I'm doing wrong?</p>
<pre><code>apiVersion: v1
kind: Namespace
metadata:
  labels:
    dgp-origin: demo-app
    istio-injection: enabled
  name: demo
---
apiVersion: v1
kind: Service
metadata:
  name: demo
  namespace: demo
  labels:
    app: demo
    version: v1
  annotations:
    networking.istio.io/exportTo: &quot;*&quot;
spec:
  ports:
  - name: http
    port: 8000
  selector:
    app: demo
    version: v1
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  namespace: demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: &quot;true&quot;
      labels:
        app: demo
        version: v1
    spec:
      containers:
      - name: echo
        image: paddycarey/go-echo
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8000
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: demo
  namespace: demo
spec:
  exportTo:
  - &quot;*&quot;
  host: demo.demo.svc.cluster.local
  subsets:
  - name: v1
    labels:
      app: demo
      version: v1
---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: demo
  namespace: demo
spec:
  selector:
    app: istio-ingressgateway
  servers:
  - hosts:
    - demo.external.com
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      mode: SIMPLE
      privateKey: /etc/istio/ingressgateway-certs/tls.key
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
  - hosts:
    - demo.demo.svc.cluster.local
    port:
      name: http
      number: 80
      protocol: HTTP
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: demo
  namespace: demo
spec:
  exportTo:
  - &quot;*&quot;
  hosts:
  - demo.external.com
  - demo.demo.svc.cluster.local
  gateways:
  - mesh
  - demo/demo
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: demo.demo.svc.cluster.local
#        port: 
#          number: 8000
        subset: v1
    timeout: 55s
</code></pre>
<p><strong>log info (from istio-proxy of another container)</strong></p>
<p>external call: OK</p>
<pre><code>{
    &quot;authority&quot;: &quot;-&quot;,
    &quot;bytes_received&quot;: &quot;511&quot;,
    &quot;bytes_sent&quot;: &quot;4744&quot;,
    &quot;downstream_local_address&quot;: &quot;172.19.2.100:443&quot;,
    &quot;downstream_remote_address&quot;: &quot;172.18.140.129:37992&quot;,
    &quot;duration&quot;: &quot;43&quot;,
    &quot;istio_policy_status&quot;: &quot;-&quot;,
    &quot;method&quot;: &quot;-&quot;,
    &quot;path&quot;: &quot;-&quot;,
    &quot;protocol&quot;: &quot;-&quot;,
    &quot;request_id&quot;: &quot;-&quot;,
    &quot;requested_server_name&quot;: &quot;-&quot;,
    &quot;response_code&quot;: &quot;0&quot;,
    &quot;response_flags&quot;: &quot;-&quot;,
    &quot;route_name&quot;: &quot;-&quot;,
    &quot;start_time&quot;: &quot;2020-08-10T10:32:25.149Z&quot;,
    &quot;upstream_cluster&quot;: &quot;PassthroughCluster&quot;,
    &quot;upstream_host&quot;: &quot;172.19.2.100:443&quot;,
    &quot;upstream_local_address&quot;: &quot;172.18.140.129:37994&quot;,
    &quot;upstream_service_time&quot;: &quot;-&quot;,
    &quot;upstream_transport_failure_reason&quot;: &quot;-&quot;,
    &quot;user_agent&quot;: &quot;-&quot;,
    &quot;x_forwarded_for&quot;: &quot;-&quot;
}
</code></pre>
<p>internal call : NOT OK</p>
<pre><code>{
    &quot;authority&quot;: &quot;demo.demo.svc.cluster.local&quot;,
    &quot;bytes_received&quot;: &quot;0&quot;,
    &quot;bytes_sent&quot;: &quot;0&quot;,
    &quot;downstream_local_address&quot;: &quot;172.18.212.107:80&quot;,
    &quot;downstream_remote_address&quot;: &quot;172.18.140.129:37802&quot;,
    &quot;duration&quot;: &quot;0&quot;,
    &quot;istio_policy_status&quot;: &quot;-&quot;,
    &quot;method&quot;: &quot;GET&quot;,
    &quot;path&quot;: &quot;/&quot;,
    &quot;protocol&quot;: &quot;HTTP/1.1&quot;,
    &quot;request_id&quot;: &quot;f875b032-f7d4-4f36-9ce1-38166aced074&quot;,
    &quot;requested_server_name&quot;: &quot;-&quot;,
    &quot;response_code&quot;: &quot;503&quot;,
    &quot;response_flags&quot;: &quot;NR&quot;,
    &quot;route_name&quot;: &quot;-&quot;,
    &quot;start_time&quot;: &quot;2020-08-10T10:33:51.262Z&quot;,
    &quot;upstream_cluster&quot;: &quot;-&quot;,
    &quot;upstream_host&quot;: &quot;-&quot;,
    &quot;upstream_local_address&quot;: &quot;-&quot;,
    &quot;upstream_service_time&quot;: &quot;-&quot;,
    &quot;upstream_transport_failure_reason&quot;: &quot;-&quot;,
    &quot;user_agent&quot;: &quot;curl/7.61.1&quot;,
    &quot;x_forwarded_for&quot;: &quot;-&quot;
}
</code></pre>
<p><strong>UPDATE : When service is on port 80 it works</strong></p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: demo
  namespace: demo
  labels:
    app: demo
    version: v1
  annotations:
    networking.istio.io/exportTo: &quot;*&quot;
spec:
  ports:
  - name: http
    port: 80
    targetPort: 8000
  selector:
    app: demo
    version: v1
</code></pre>

<p>Based on the istio <a href="https://istio.io/latest/docs/examples/bookinfo/" rel="nofollow noreferrer">bookinfo</a> app I would say the issue here are the missing <strong>labels</strong> in your deployment.</p>
<p>There is an productpage <a href="https://raw.githubusercontent.com/istio/istio/release-1.6/samples/bookinfo/platform/kube/bookinfo.yaml" rel="nofollow noreferrer">example</a></p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: details-v1
  labels:
    app: details
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: details
      version: v1
  template:
    metadata:
      labels:
        app: details
        version: v1
    spec:
      serviceAccountName: bookinfo-details
      containers:
      - name: details
        image: docker.io/istio/examples-bookinfo-details-v1:1.16.2
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
</code></pre>
<p>Could you try to use your deployment after my edit?</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  namespace: demo
  labels:
    app: demo
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo
      version: v1
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: &quot;true&quot;
      labels:
        app: demo
        version: v1
    spec:
      containers:
      - name: echo
        image: paddycarey/go-echo
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8000
</code></pre>
<p><strong>EDIT</strong></p>
<p>I have test your yamls, and additionally I have created my own example with nginx pod.</p>
<p>I have the same issue as you, mesh internall call works only if I add port 8000 to virtual service.</p>
<hr />
<p>In my example with nginx everything works just fine.</p>
<hr />
<p>So based on that I assume there is either something wrong with</p>
<ul>
<li>paddycarey/go-echo image, as far as I checked, last time it was updated 4 years ago.</li>
<li>mesh gateway require port in virtual service, if it's other port than 80.</li>
</ul>
<hr />
<p>There are my yamls to test with nginx.</p>
<pre><code>apiVersion: v1
kind: Namespace
metadata:
  labels:
    istio-injection: enabled
  name: demo-app

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
  namespace: demo-app
spec:
  selector:
    matchLabels:
      app: nginx1
      version: v1
  replicas: 1
  template:
    metadata:
      labels:
        version: v1
        app: nginx1
    spec:
      containers:
      - name: nginx1
        image: nginx
        ports:
        - containerPort: 80
        lifecycle:
          postStart:
            exec:
              command: [&quot;/bin/sh&quot;, &quot;-c&quot;, &quot;echo Hello nginx1 &gt; /usr/share/nginx/html/index.html&quot;]

---

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: demo-app
  labels:
    app: nginx1
spec:
  ports:
  - name: http-front
    port: 80
    protocol: TCP
  selector:
    app: nginx1

---

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: simpleexample
  namespace: demo-app
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP

---

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginxvirt
  namespace: demo-app
spec:
  gateways:
  - simpleexample
  - mesh
  hosts:
  - 'nginx.demo-app.svc.cluster.local'
  - 'example.com' 
  http:
  - route:
    - destination:
        host: nginx
        subset: v1

  
---

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginxdest
  namespace: demo-app
spec:
  host: nginx
  subsets:
  - name: v1
    labels:
      version: v1

---

apiVersion: v1
kind: Pod
metadata:
  name: ubu1
  namespace: demo-app
spec:
  containers:
  - name: ubu1
    image: ubuntu
    command: [&quot;/bin/sh&quot;]
    args: [&quot;-c&quot;, &quot;apt-get update &amp;&amp; apt-get install curl -y &amp;&amp; sleep 3000&quot;]
</code></pre>
<hr />
<p>External call test</p>
<pre><code>curl -v -H &quot;host: example.com&quot; xx.xx.xx.xx/
HTTP/1.1 200 OK
Hello nginx1
</code></pre>
<p>Internal call test</p>
<pre><code>root@ubu1:/# curl nginx/
Hello nginx1
</code></pre>
<hr />
<p>Let me know if that was it or do you need further help.</p>
