title: Unable to get the content of webpage using requests
body: <p>I use <code>requests</code> to get a webpage but failed.</p>
  <p>Here is my code:</p>
  <p>import requests
  from bs4 import BeautifulSoup</p>
  <pre><code>url = 'http://db.house.qq.com/index.php?mod=search&amp;city=bj'
  headers = {}
  headers['authority'] = 'db.house.qq.com'
  headers['method'] = 'GET'
  headers['path'] = '/index.php?mod=search&amp;city=bj'
  headers['scheme'] = 'https'
  headers['accept'] = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9'
  headers['accept-encoding'] = 'gzip, deflate, br'
  headers['accept-language'] = 'en-US,en;q=0.9,zh-HK;q=0.8,zh;q=0.7,zh-CN;q=0.6,an;q=0.5'
  headers['cookie'] = 'pgv_info=ssid=s9739254340; pgv_pvid=9743767040; ts_uid=2023229671; pac_uid=0_da940c972d7c0; h_uid=h592060229584922854; Hm_lvt_73f18bb34ff30f1061b904f30f86c5cb=1602238779; ts_refer=www.google.com/; ts_uid=6802299874; pgv_pvi=196710400; pgv_si=s9373821952; Hm_lpvt_73f18bb34ff30f1061b904f30f86c5cb=1602767734; hisuid=[%22h592060229584922854%22]; hisuin=[null]; feature={%2295%22:1%2C%2298%22:1}; ts_last=db.house.qq.com/index.php; ad_play_index=86'
  headers['dnt'] = '1'
  headers['sec-ch-ua'] = '&quot;Chromium&quot;;v=&quot;86&quot;, &quot;\&quot;Not\\A;Brand&quot;;v=&quot;99&quot;, &quot;Google Chrome&quot;;v=&quot;86&quot;'
  headers['sec-ch-ua-mobile'] = '?0'
  headers['sec-fetch-dest'] = 'document'
  headers['sec-fetch-mode'] = 'navigate'
  headers['sec-fetch-site'] = 'none'
  headers['upgrade-insecure-requests'] = '1'
  headers['user-agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
  
  res = requests.get(url, headers=headers)
  res.encoding = res.apparent_encoding
  print(soup.find('em', {'id':'search_result_num'}).text)#0, should be 3767
  print('三湘印象·森林海尚城' in res.text)#False, should be True
  </code></pre>
  <p>How to solve this problem?</p>
  <p>Thanks.</p>
  <p><a href="https://i.stack.imgur.com/PsGDL.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/PsGDL.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.stack.imgur.com/sXjZy.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/sXjZy.png" alt="enter image description here" /></a></p>