<p>I am trying to estimate the resource (cpu) request and limit values, for which I want to know the max cpu usage of a pod in last one month using prometheus. </p>

<p>I checked this question but couldn't get what i want <a href="https://stackoverflow.com/questions/40717605/generating-range-vectors-from-return-values-in-prometheus-queries">Generating range vectors from return values in Prometheus queries
</a></p>

<p>I tried this but it seems max_over_time doesn't worker over rate</p>

<pre><code>max (  
  max_over_time(
    rate(
      container_cpu_usage_seconds_total[5m]
    )[30d]
  )
) by (pod_name)
</code></pre>

<p>invalid parameter 'query': parse error at char 64: range specification must be preceded by a metric selector, but follows a *promql.Call instead</p>

<p>You'd need to capture the inner expression (rate of container cpu usage) as a <a href="https://prometheus.io/docs/practices/rules/" rel="noreferrer">recording rule</a>:</p>

<pre><code>- record: container_cpu_usage_seconds_total:rate5m
  expr: rate(container_cpu_usage_seconds_total[5m])
</code></pre>

<p>then use this new timeseries to calculate max_over_time:</p>

<pre><code>max (  
  max_over_time(container_cpu_usage_seconds_total:rate5m[30d])
) by (pod_name)
</code></pre>

<p>This is only need in Prometheus versions older to 2.7 as <a href="https://prometheus.io/docs/prometheus/latest/querying/examples/#subquery" rel="noreferrer">subqueries can be calculated on the fly</a>, see <a href="https://prometheus.io/blog/2019/01/28/subquery-support/" rel="noreferrer">this blog post for more details</a>.</p>

<p><em>Bear in mind though</em>, if you're planning to use this composite query (max of  max_per_time of data collected in the last 30 days) for alerting or visualisation (rather than an one-off query), then you'd <em>still want to use the recording rule</em> to improve the query's performance. Its the classic CS computational complexity tradeoff (memory/storage space required for storing the recording rule as a separate timeseries vs. the computational resources needed to process data for 30 days!)</p>
