<blockquote>
  <p>Updated with more information</p>
</blockquote>

<p>I am trying to set up OpenTSDB on Bigtable, following this guide:
<a href="https://cloud.google.com/solutions/opentsdb-cloud-platform" rel="nofollow noreferrer">https://cloud.google.com/solutions/opentsdb-cloud-platform</a></p>

<p>Works well, all good. </p>

<p>Now I was trying to open the <code>opentsdb-write</code> service with a LoadBalancer (type). Seems to work well, too.</p>

<p>Note: using a GCP load balancer.</p>

<p>I am then using insomnia to send a POST to the <code>./api/put</code> endpoint - and I get a <code>204</code> as expected (also, using the <code>?details</code> shows no errors, neither does the <code>?sync</code>) (see <a href="http://opentsdb.net/docs/build/html/api_http/put.html" rel="nofollow noreferrer">http://opentsdb.net/docs/build/html/api_http/put.html</a>)</p>

<p>When querying the data (GET on <code>./api/query</code>), I don't see the data (same effect in grafana). Also, I do not see any data added in the <code>tsdb</code> table in bigtable.</p>

<p>My conclusion: no data is written to Bigtable, although tsd is returning 204. </p>

<p>Interesting fact: the <strong>metric</strong> is created (I can see it in Bigtable (<code>cbt read tsdb-uid</code>) and also the autocomplete in the opentsdb-ui (and grafana) pick the metric up right away. But no data.</p>

<p>When I use the Heapster-Example as in the tutorial, it all works.</p>

<p>And the interesting part (to me):</p>

<p>NOTE: It happened a few times, with massive delay or after stoping/restarting the kubernetes cluster, that the data appeared. Suddenly. I could not reproduce as of now.</p>

<p>I must be missing something really simple. </p>

<p>Note: I don't see any errors in the logs (stackdriver) and UI (opentsdb UI), neither bigtable, nor Kubernetes, nor anything I can think of.</p>

<p>Note: the configs I am using are as linked in the tutorial.</p>

<p>The put I am using (see the 204):</p>

<p><a href="https://i.stack.imgur.com/pmaBR.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pmaBR.png" alt="enter image description here"></a></p>

<p>and if I add <code>?details</code>, it indicates success:</p>

<p><a href="https://i.stack.imgur.com/cSsJO.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/cSsJO.png" alt="enter image description here"></a></p>

<p>My guess is that this relates to the opentsdb flush frequency.  When a tsdb cluster is shutdown, there's an automatic flush.  I'm not 100% sure, but I think that the <code>tsd.storage.flush_interval</code> configuration manages that process.</p>

<p>You can reach the team that maintains the libraries via the google-cloud-bigtable-discuss group, which you can get to from the <a href="https://cloud.google.com/bigtable/docs/support/getting-support" rel="nofollow noreferrer">Cloud Bigtable support page</a> for more nuanced discussions.</p>

<p>As an FYI, we (Google) are actively updating the <a href="https://cloud.google.com/solutions/opentsdb-cloud-platform" rel="nofollow noreferrer">https://cloud.google.com/solutions/opentsdb-cloud-platform</a> to the latest versions of OpenTSDB and AsyncBigtable which should improve performance at high volumes.</p>
