<p>I would like to redirect incoming traffic to myserver.mydomain.com/prometheus to my prometheus pod. Here are my YAML files where I try to achieve this:</p>

<p>Here's the deployment manifest:</p>

<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-deployment
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus-server
  template:
    metadata:
      labels:
        app: prometheus-server
    spec:
      containers:
        - name: prometheus
          image: prom/prometheus:latest
          args:
            - "--config.file=/etc/prometheus/prometheus.yml"
            - "--storage.tsdb.path=/prometheus/"
            - "--web.external-url=http://myserver.mydomain.com/prometheus"
          ports:
            - containerPort: 9090
          volumeMounts:
            - name: prometheus-config-volume
              mountPath: /etc/prometheus/
            - name: prometheus-storage-volume
              mountPath: /prometheus/
      volumes:
        - name: prometheus-config-volume
          configMap:
            defaultMode: 420
            name: prometheus-server-conf

        - name: prometheus-storage-volume
          persistentVolumeClaim:
            claimName: prometheus-local-zfs-pvc

</code></pre>

<p>the service manifest...</p>

<pre><code>apiVersion: v1
kind: Service
metadata:
  name: prometheus-service
  namespace: monitoring
  annotations:
      prometheus.io/scrape: 'true'
      prometheus.io/port:   '9090'
spec:
  selector:
    app: prometheus-server
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 9090

</code></pre>

<p>...and the ingress manifest...</p>

<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: prometheus-ingress
  namespace: monitoring
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: myserver.mydomain.com
    http:
      paths:
      - path: /prometheus
        backend:
          serviceName: prometheus-service
          servicePort: 80
</code></pre>

<p>However, this returns a 404. What am I doing wrong?</p>

<p>EDIT: I have added the option as suggested by @coderanger. One thing I noticed when applying the new deployment was that there seemed to be a locking issue when the new pod is brought up before the old one is deleted, the error in the logs was <code>err="opening storage failed: lock DB directory: resource temporarily unavailable....</code></p>

<p>You need to pass <code>--web.external-url</code> on the Prometheus command line options since you are moving it to a sub-path.</p>
