title: Google Sign-In: "Certificate for key id xxxx not found" when using google-auth Python package
body: <p>I'm maintaining a website and its mobile apps (iOS and Android). For Google Sign-In in mobile app, I'm using <a href="https://github.com/googleapis/google-auth-library-python" rel="nofollow noreferrer">google-auth Python package</a> on the server side.</p>
  
  <p>Since about one month ago, I started to receive error reports related to Google Sign-In from the server side. The error message looks like the following:</p>
  
  <blockquote>
    <p>Certificate for key id 728f4016652079b9ed99861bb09bafc5a45baa86 not found.</p>
  </blockquote>
  
  <p>The server-side Google Sign-In authentication backend follows <a href="https://developers.google.com/identity/sign-in/android/backend-auth" rel="nofollow noreferrer">this document</a>:</p>
  
  <pre class="lang-python prettyprint-override"><code>from google.oauth2 import id_token
  from google.auth.transport import requests
  
  # ...
  
  try:
      # The following line may raise ValueError with message:
      # Certificate for key id xxxx not found.
      id_info = id_token.verify_oauth2_token(google_id_token, requests.Request())
  
      if id_info['aud'] not in VALID_CLIENT_IDS:
          logger.error('Invalid aud from Google ID token: %s', id_info['aud'])
          raise ValueError('Unverified audience.')
      # ...
  except ValueError as exc:
      logger.error('Fail to verify Google ID token: %s', exc, extra={'request': request})
  </code></pre>
  
  <p>The error comes from <a href="https://github.com/googleapis/google-auth-library-python/blob/master/google/auth/jwt.py#L226" rel="nofollow noreferrer">the google.auth.jwt module</a>, when verifying the Google-issued JWT against a list of Google public certificates.</p>
  
  <p><a href="https://github.com/googleapis/google-auth-library-python/blob/master/google/oauth2/id_token.py#L139" rel="nofollow noreferrer">Diving into the google-auth code</a>, I can see that <code>verify_oauth2_token()</code> function is fetching Google public certificates from the URL <a href="https://www.googleapis.com/oauth2/v1/certs" rel="nofollow noreferrer">https://www.googleapis.com/oauth2/v1/certs</a>. It seems that sometimes, for some Google ID token sent from some Android phones, the key id cannot be found in that URL.</p>
  
  <p>Here are some other details which might be useful:</p>
  
  <ul>
  <li>It seems that iOS app does not have this kind of issue. From the <code>USER_AGENT</code> header, I can see that the error happens only in Android app (<code>USER_AGENT=okhttp/3.11.0</code>). And it happens only in some Android device, not all.</li>
  <li>I was wondering if this happens only for Android phones from China (e.g. if they are connecting via a VPN). So I also checked the user IP address. But it turned out that some of those users were from Europe.</li>
  <li>Some key id is recurring again and again in the server error logs. For example, the key id <code>aa436c3f63b281ce0d976da0b51a34860ff960eb</code> is seen dozens of times, from the beginning of November to now (end of Dec).</li>
  <li>I am constantly seeing this error, several times (10 ~ 30 times) every day.</li>
  </ul>
  
  <p>My website is running on the following environment:</p>
  
  <ul>
  <li>OS: Linux (CentOS 7) 64-bit</li>
  <li>Apache 2 with mod_wsgi 4.5.24</li>
  <li>Python 3.6.7 and Django 2.1.2</li>
  <li>google-auth version: tried both 1.3.0 and 1.6.1</li>
  </ul>
  
  <p>As I could not reproduce this issue either with my iPhone, or with my Android phone (Huawei P20, bought in France), I am completely stuck.</p>
  
  <p>But, one of my friends is having this issue now, and he bought his Android phone from Hongkong. This makes me think if it is possible that for some countries, Google Sign-In uses some different certs other than the public ones at <a href="https://www.googleapis.com/oauth2/v1/certs" rel="nofollow noreferrer">https://www.googleapis.com/oauth2/v1/certs</a>?</p>
  
  <p>I don't think this is a bug in google-auth package. I'm wondering if any of you have ever heard of this error, and could give me a hint about the possible cause of it?</p>
  
  <p>Thanks in advance!</p>