title: python asyncio aiohttp timeout
body: <p>Word of notice: This is my first approach with asyncio, so I might have done something really stupid.</p>
  <p>Scenario is as follows:</p>
  <p>I need to &quot;http-ping&quot; a humongous list of urls to check if they respond 200 or any other value. I get timeouts for each and every request, though tools like gobuster report 200,403, etc.</p>
  <p>My code is sth similar to this:</p>
  <pre><code>import asyncio,aiohttp
  import datetime 
  #-------------------------------------------------------------------------------------
  async def get_data_coroutine(session,url,follow_redirects,timeout_seconds,retries):
      #print('#DEBUG '+datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')+' '+url)
      try:
          async with session.get(url,allow_redirects=False,timeout=timeout_seconds) as response:
              status  =   response.status
              #res     =   await response.text()
              if(  status==404):
                  pass
              elif(300&lt;=status and status&lt;400):
                  location = str(response).split(&quot;Location': \'&quot;)[1].split(&quot;\'&quot;)[0]
                  print('#HIT   '+datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')+' '+str(status)+' '+url+' ---&gt; '+location)
                  if(follow_redirects==True):
                      return await get_data_coroutine(session,location,follow_redirects,timeout_seconds,retries)
              else:
                  print('#HIT   '+datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')+' '+str(status)+' '+url)
              return None
      except asyncio.exceptions.TimeoutError as e:
          print('#ERROR '+datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')+' '+'   '+' '+url+' TIMEOUT '+str(e))
          return None
  #---------------------------------------------------------------------------    
  async def main(loop):
          base_url                =   'http://192.168.59.37'
          extensions              =   ['','.html','php']
          fd                      =   open('/usr/share/wordlists/dirb/common.txt','r')
          words_without_suffix    =   [x.strip() for x in fd.readlines()]#[-5:] #DEBUG!
          words_with_suffix       =   [base_url+'/'+x+y for x in words_without_suffix for y in extensions]
          follow                  =   True
          total_timeout           =   aiohttp.ClientTimeout(total=60*60*24)
          timeout_seconds         =   10
          retries                 =   1
          async with aiohttp.ClientSession(loop=loop,timeout=total_timeout) as session:
              tasks = [get_data_coroutine(session,url,follow,timeout_seconds,retries) for url in words_with_suffix]
              await asyncio.gather(*tasks)
          print('DONE')
  #---------------------------------------------------------------------------    
  if(__name__=='__main__'):
      loop    =   asyncio.get_event_loop()
      result  =   loop.run_until_complete(main(loop))
     
  </code></pre>
  <p>Did I do something really wrong?</p>
  <p>Any word of advice?</p>
  <p>Thank you SO much!</p>