<p>I setup a k8s cluster on my server(all in one node) and installed <a href="https://github.com/kubernetes/ingress-nginx" rel="nofollow noreferrer">ingress-nginx</a>.
Now I want to deploy a <a href="https://github.com/fatedier/frp" rel="nofollow noreferrer">frps</a> service to the cluster. Here is my code:</p>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: v1
kind: Namespace
metadata:
  name: helloworld
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: frps-configmap
  namespace: helloworld
data:
  frps.ini: |
    [common]
    bind_port = 7000
    vhost_http_port = 8080
    custom_404_page = /etc/frp/frps.html
  frps.html: |
    &lt;!DOCTYPE html&gt;
    &lt;html lang=&quot;en&quot;&gt;
      // ...
    &lt;/html&gt;
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frps-deployment
  namespace: helloworld
  labels:
    app: frps
spec:
  replicas: 1
  selector:
    matchLabels:
      app: frps
  template:
    metadata:
      labels:
        app: frps
    spec:
      containers:
        - name: frps
          image: 'snowdreamtech/frps:0.34.3'
          ports:
            - containerPort: 8080
            - containerPort: 7000
          volumeMounts:
            - name: frps-etc
              mountPath: '/etc/frp'
              readOnly: true
      volumes:
        - name: frps-etc
          configMap:
            name: frps-configmap
            items:
              - key: frps.ini
                path: frps.ini
              - key: frps.html
                path: frps.html
---
apiVersion: v1
kind: Service
metadata:
  name: frps
  namespace: helloworld
spec:
  selector:
    app: frps
  type: ClusterIP
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      name: vhost
    - protocol: TCP
      port: 7000
      targetPort: 7000
      name: frps
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: frps-ingress
  namespace: helloworld
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: 'nginx'
  rules:
    - host: dev.helloworld.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frps
                port: 
                  name: vhost
</code></pre>
<p>And here are some cluster information after apply previous yaml:</p>
<pre><code>kubectl get deploy -A
NAMESPACE         NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
helloworld        frps-deployment            1/1     1            1           12m
ingress-nginx     ingress-nginx-controller   1/1     1            1           23d
...

kubectl get svc -A
NAMESPACE       NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
helloworld      frps                                 ClusterIP   10.97.67.98      &lt;none&gt;        8080/TCP,7000/TCP            6m18s
ingress-nginx   ingress-nginx-controller             NodePort    10.101.128.200   &lt;none&gt;        80:32038/TCP,443:31363/TCP   23d
ingress-nginx   ingress-nginx-controller-admission   ClusterIP   10.97.36.119     &lt;none&gt;        443/TCP                      23d
...

kubectl describe ingress --namespace=helloworld frps-ingress
Name:             frps-ingress
Namespace:        helloworld
Address:          192.168.0.176
Default backend:  default-http-backend:80 (&lt;error: endpoints &quot;default-http-backend&quot; not found&gt;)
Rules:
  Host                Path  Backends
  ----                ----  --------
  dev.helloworld.com
                      /   frps:vhost (192.168.103.147:8080)
Annotations:          nginx.ingress.kubernetes.io/rewrite-target: /
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    13m (x2 over 13m)  nginx-ingress-controller  Scheduled for sync
</code></pre>
<p>But when I ran <code>curl -H 'HOST: dev.helloworld.com' http://localhost:32038/</code>, it return 502:</p>
<pre class="lang-html prettyprint-override"><code>&lt;html&gt;
&lt;head&gt;&lt;title&gt;502 Bad Gateway&lt;/title&gt;&lt;/head&gt;
&lt;body&gt;
&lt;center&gt;&lt;h1&gt;502 Bad Gateway&lt;/h1&gt;&lt;/center&gt;
&lt;hr&gt;&lt;center&gt;nginx&lt;/center&gt;
&lt;/body&gt;
&lt;/html&gt;
</code></pre>
<p>I exexcuted following command in nginx pod, and got correct content:</p>
<pre><code>curl http://frps.helloworld:8080
&lt;!DOCTYPE html&gt;
&lt;html lang=&quot;en&quot;&gt;
  // ...
&lt;/html&gt;
</code></pre>
<p>So I'm sure the frps service is correctly running. I have tried not only using header but also change cloud dns record(under another domain), but got same result.
What happened, and why ingress-nginx return 502?</p>
<hr />
<p>I have see something wrong, it seems nginx found wrong upstream. The cluster ip of frps service is 10.97.67.98, while ingress-nginx use pod ip 192.168.103.147. The pod ip is unreachable while service cluster ip is reachable. I don't known why and how to fix it.</p>

Vista Chyi
<p>I didn't think that this problem may be caused by the firewall daemon of the host.
After enbale masquerade and open ports of kubernetes, everything work fine :)
Something like:</p>
<pre><code>firewall-cmd --add-masquerade --permanent
firewall-cmd --permanent --zone=public --add-port=10250-10252/tcp
firewall-cmd --permanent --zone=public --add-port=10255/tcp
firewall-cmd --permanent --zone=public --add-port=30000-32767/tcp
firewall-cmd --reload
</code></pre>
