title: Python requests ODDITY
body: <p>I am trying to debug python request to scrap data from Amazon
  but it always failed with error 503. I am compare to POST command from bash, the result always 200 OK. You can see in output, the request header is exacly the same, almost identical. Can any body expalain why ?</p>
  <p>the script something like this</p>
  <pre><code>import requests
  from http.client import HTTPConnection
  HTTPConnection.debuglevel = 1
  
  client = requests.session()
  res = client.post( 'https://www.amazon.com/s/query',
        headers = {
          'User-Agent': 'Chrome',
          'Content-Type': 'application/json',
          'Accept-Encoding': 'deflate'
        },
        params = {'k': 'xbox'})
  </code></pre>
  <p>the output always with error 503</p>
  <pre><code>send: b'POST /s/query?k=xbox HTTP/1.1\r\nHost: www.amazon.com\r\nUser-Agent: Chrome\r\nAccept-Encoding: deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Type: application/json\r\nContent-Length: 0\r\n\r\n'
  reply: 'HTTP/1.1 503 Service Unavailable\r\n'
  header: Connection: keep-alive
  header: Content-Length: 2671
  header: server: Server
  header: date: Wed, 15 Jun 2022 01:46:07 GMT
  header: content-type: text/html
  header: report-to: {&quot;group&quot;: &quot;nel&quot;, &quot;max_age&quot;: 3600, &quot;endpoints&quot;: [{ &quot;url&quot;: &quot;https://primary.prod.flex.frontier.a2z.com/nel&quot;}, { &quot;url&quot;: &quot;https://secondary.prod.flex.frontier.a2z.com/nel&quot;}]}
  header: x-amz-rid: PRKZBYW6JNKFJ4221E28
  header: vary: Content-Type,Accept-Encoding,X-Amzn-CDN-Cache,X-Amzn-AX-Treatment,User-Agent
  header: last-modified: Mon, 25 Apr 2022 20:07:01 GMT
  header: etag: &quot;a6f-5dd801e82e340&quot;
  header: accept-ranges: bytes
  header: strict-transport-security: max-age=47474747; includeSubDomains; preload
  header: permissions-policy: interest-cohort=()
  header: nel: { &quot;report_to&quot;: &quot;nel&quot;, &quot;max_age&quot;: 3600, &quot;success_fraction&quot;: 0.0, &quot;failure_fraction&quot;: 1.0 }
  header: x-served-by: cache-qpg1230-QPG
  </code></pre>
  <p>when I try using POST command from bash it always succeed :</p>
  <pre><code>POST 'https://www.amazon.com/s/query?k=xbox' \
  -H 'Host: www.amazon.com' \
  -H 'User-Agent: Chrome' \
  -H 'Accept-Encoding: deflate' \
  -H 'Accept: */*' \
  -H 'Connection: keep-alive' \
  -H 'Content-Length: 0' \
  -H 'Content-Type: application/json' \
  -sSdE
  </code></pre>
  <p>the output:</p>
  <pre><code>POST https://www.amazon.com/s/query?k=xbox
  Connection: keep-alive
  Accept: */*
  Accept-Encoding: deflate
  Host: www.amazon.com
  User-Agent: Chrome
  Content-Length: 0
  Content-Type: application/json
  
  200 OK
  Connection: keep-alive
  Date: Wed, 15 Jun 2022 02:07:00 GMT
  Server: Server
  Vary: Content-Type,X-Amazon-Wtm-Tag-SP-Search-Secured-Port-Enabled,Accept-Encoding,X-Amzn-CDN-Cache,X-Amzn-AX-Treatment,User-Agent
  Content-Language: en-US
  Content-Type: application/json-amazonui-streaming;charset=UTF-8
  Accept-Ch: ect,rtt,downlink,device-memory,sec-ch-device-memory,viewport-width,sec-ch-viewport-width,dpr,sec-ch-dpr
  Accept-Ch-Lifetime: 86400
  Client-Date: Wed, 15 Jun 2022 02:07:01 GMT
  Client-Peer: 162.219.225.118:443
  Client-Response-Num: 1
  Client-SSL-Cert-Issuer: /C=US/O=DigiCert Inc/CN=DigiCert Global CA G2
  Client-SSL-Cert-Subject: /CN=www.amazon.com
  Client-SSL-Cipher: TLS_AES_256_GCM_SHA384
  Client-SSL-Socket-Class: IO::Socket::SSL
  Client-SSL-Version: TLSv1_3
  Client-Transfer-Encoding: chunked
  Nel: { &quot;report_to&quot;: &quot;nel&quot;, &quot;max_age&quot;: 3600, &quot;success_fraction&quot;: 0.0, &quot;failure_fraction&quot;: 1.0 }
  Permissions-Policy: interest-cohort=()
  Report-To: {&quot;group&quot;: &quot;nel&quot;, &quot;max_age&quot;: 3600, &quot;endpoints&quot;: [{ &quot;url&quot;: &quot;https://primary.prod.flex.frontier.a2z.com/nel&quot;}, { &quot;url&quot;: &quot;https://secondary.prod.flex.frontier.a2z.com/nel&quot;}]}
  Set-Cookie: session-id=138-2883811-7779558; Domain=.amazon.com; Expires=Thu, 15-Jun-2023 02:07:00 GMT; Path=/; Secure
  Set-Cookie: session-id-time=2082787201l; Domain=.amazon.com; Expires=Thu, 15-Jun-2023 02:07:00 GMT; Path=/; Secure
  Set-Cookie: i18n-prefs=USD; Domain=.amazon.com; Expires=Thu, 15-Jun-2023 02:07:00 GMT; Path=/
  Set-Cookie: sp-cdn=&quot;L5Z9:ID&quot;; Version=1; Domain=.amazon.com; Max-Age=31536000; Expires=Thu, 15-Jun-2023 02:07:00 GMT; Path=/; Secure; HttpOnly
  Strict-Transport-Security: max-age=47474747; includeSubDomains; preload
  X-Amz-Rid: TG7103CWM9SZ51WTR8GJ
  X-Content-Type-Options: nosniff
  X-Frame-Options: SAMEORIGIN
  X-Served-By: cache-qpg1279-QPG
  X-Xss-Protection: 1;
  </code></pre>
  <p>as you can see the headers is identical. what make the python request different ?
  thank you.</p>