<p>When deploying a Ruby based Passenger standalone application via Kubernetes we
ran into the issue of losing the capability of monitoring them via
<code>passenger-status</code>. There is a <a href="https://github.com/influxdata/telegraf/tree/master/plugins/inputs/passenger" rel="nofollow noreferrer">telegraf
plugin</a>
 or a <a href="https://github.com/stuartnelson3/passenger_exporter" rel="nofollow noreferrer">passenger exporter</a> to forward metrics, which need access to the output of the
<code>passenger-status</code> binary.</p>

<p>Following the philosophy of having one (main) process per Container using a
Sidecar container to gather the metrics would be reasonable when deploying to
Kubernetes. Accessing the output of <code>passenger-status</code> from the other container is the challenge here. Link the files into another container is <a href="https://github.com/kubernetes/kubernetes/issues/6120#issuecomment-87139953" rel="nofollow noreferrer">not supported</a>. Setting up an directory for both containers and copy executables seems to be over complex.</p>

<p>Communication between Containers within one Pod works via the loopback network. Therefore exposing metrics via HTTP is a common pattern to export those. So we are looking into different ways of exposing the <code>passenger-status</code> metrics via HTTP:</p>

<h2>via Application</h2>

<p>Running the command via <em><a href="http://ruby-doc.org/core-2.5.0/Kernel.html#method-i-60" rel="nofollow noreferrer">Kernel#`</a></em> kind of defeats the purpose of monitoring it. This will only return when there are enough passenger processes free to answer this request. Once the passenger queue gets full the monitoring will also not work anymore, which is exactly what we want to see here.</p>

<h2>CGI script</h2>

<p>As nginx only supports FastCGI it is necessary to have something like <a href="https://github.com/gnosek/fcgiwrap" rel="nofollow noreferrer">fcgiwrap</a> to execute scripts. fciwrap itself needs to have another process running though, which itself needs monitoring. Furthermore it violates the idea of having one process per container.</p>

<h2>Lua script</h2>

<p>A lua snippet like this would probably work:</p>

<pre><code>location /passenger-status {
  content_by_lua_block {
    os.execute("/opt/ruby/bin/passenger-status")
  }
}
</code></pre>

<p>However, adding Lua scripting to every production container just for this purpose seems to be cracking a walnut with a sledgehammer.</p>

<h2>Second Passenger instance</h2>

<p>Having a second tiny ruby script as passenger endpoint for the monitoring would also probably work:</p>

<pre><code>http {
    ...

    server {
        listen 80;
        server_name _;
        root /app;
        passenger_enabled on;
        ...
    }

    server {
        listen 8080;
        server_name _;
        root /monitoring;
        passenger_enabled on;
        ...
    }

    ...
}
</code></pre>

<hr>

<p>All in all I don't find any of those approaches satisfactory. What are your thoughts or solutions on this topic?</p>

<p>We went the approach "Second Passenger instance" and have a <a href="https://github.com/xing/passenger-prometheus-exporter-app" rel="nofollow noreferrer">second ruby process group within passenger</a>. As described in the question already integrating works via adding a snippet like this to your <code>nginx.conf</code>:</p>

<pre class="lang-sh prettyprint-override"><code>    server {
        server_name _;
        listen 0.0.0.0:10254;
        root '/monitor/public';
        passenger_app_root '/monitor';
        passenger_app_group_name 'Prometheus exporter';
        passenger_spawn_method direct;
        passenger_enabled on;
        passenger_min_instances 1;
        passenger_load_shell_envvars off;
    }
</code></pre>

<p>This will start another ruby process serving a prometheus endpoint on <code>http://&lt;ip-of-this-server&gt;:10254/metrics</code> that exposes the passenger metrics to be collected by your usual Kubernetes monitoring infrastructure. A response of this could look like:</p>

<pre><code># HELP passenger_capacity Capacity used
# TYPE passenger_capacity gauge
passenger_capacity{supergroup_name="/app (development)",group_name="/app (development)",hostname="my-container"} 1
# HELP passenger_wait_list_size Requests in the queue
# TYPE passenger_wait_list_size gauge
passenger_wait_list_size{supergroup_name="/app (development)",group_name="/app (development)",hostname="my-container"} 0
# HELP passenger_processes_active Active processes
# TYPE passenger_processes_active gauge
passenger_processes_active{supergroup_name="/app (development)",group_name="/app (development)",hostname="my-container"} 0
</code></pre>

<p>Find the project at <a href="https://github.com/xing/passenger-prometheus-exporter-app" rel="nofollow noreferrer">passenger-prometheus-exporter-app</a>.</p>
