title: Error while creating custom metrics with 'monitoring_v3' for containers in GKE
body: <p>We are getting the following error</p>
  
  <blockquote>
    <p>"One or more TimeSeries could not be written: One or more points were
    written more frequently than the maximum sampling period configured
    for the metric.: timeSeries[0]"</p>
  </blockquote>
  
  <p>when we tried to create container level metric in GKE. We are creating custom metric to update app_queue length for every second. It works well under normal case. But when the pods scales beyond 100 - we receive this error for some of the API calls and it is very high in number.</p>
  
  <p><strong>Question:</strong>
  Is the any quota or setting we are missing? or some best practice of creating custom metrics?</p>
  
  <p><strong>Stack trace:</strong></p>
  
  <pre><code>      File "/usr/local/lib/python3.6/dist-packages/Base-1.0-py3.6.egg/base/custom_metrics.py", line 63, in create_custom_metric
          client.create_time_series(project_name, [series])
        File "/usr/local/lib/python3.6/dist-packages/google/cloud/monitoring_v3/gapic/metric_service_client.py", line 775, in create_time_series
          request, retry=retry, timeout=timeout, metadata=metadata)
        File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
          return wrapped_func(*args, **kwargs)
        File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
          on_error=on_error,
        File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 177, in retry_target
          return target()
        File "/usr/local/lib/python3.6/dist-packages/google/api_core/timeout.py", line 206, in func_with_timeout
          return func(*args, **kwargs)
        File "/usr/local/lib/python3.6/dist-packages/google/api_core/grpc_helpers.py", line 61, in error_remapped_callable
          six.raise_from(exceptions.from_grpc_error(exc), exc)
        File "&lt;string&gt;", line 3, in raise_from
      google.api_core.exceptions.InvalidArgument: 400 One or more TimeSeries could not be written: One or more points were written more frequently than the maximum sampling period configured for the metric.: timeSeries[0]
  </code></pre>