title: username web scraping from forum with python
body: <p>Hope to seek help from you guys! I want to scrap the username from a forum with Python, but I couldn't figure out the method. The following is part of the code:</p>
  
  <p>Part1</p>
  
  <pre><code>&lt;td class="alt2" title="reply: 11，view: 1,097"&gt;
      &lt;div class="smallfont" style="text-align:right; white-space:nowrap"&gt;
      2017-03-28 &lt;span class="time"&gt;23:44&lt;/span&gt;&lt;br&gt;
  
      &lt;a href="member.php?find=lastposter&amp;amp;t=1907777" rel="nofollow"&gt;username&lt;/a&gt;  &lt;a href="showthread.php?p=9575713#post9575713"&gt;&lt;img class="inlineimg" src="http://s.bbkz.net/forum/images/buttons_style/tc_2/lastpost.gif" alt="last" title="last" border="0"&gt;&lt;/a&gt;
      &lt;/div&gt;
  &lt;/td&gt;
  </code></pre>
  
  <p>Part 2</p>
  
  <pre><code>&lt;div class="smallfont"&gt;
      &lt;span style="cursor:pointer" onclick="window.open('member.php?u=353562', '_self')"&gt;username&lt;/span&gt;
  &lt;/div&gt;
  </code></pre>
  
  <p>Also, the format for forum link is this: <a href="http://www.example.com/forum/forumdisplay.php?f=148&amp;order=desc&amp;page=3" rel="nofollow noreferrer">http://www.example.com/forum/forumdisplay.php?f=148&amp;order=desc&amp;page=3</a></p>
  
  <p>I want to scrap the 'username' from these codes on different pages with Python, may I have your help?</p>
  
  <p>Thanks a lot!!</p>
  
  <p>[Edit - Time sleep added]
  should it be like this?</p>
  
  <pre><code>import requests
  from bs4 import BeautifulSoup
  import time
  
  url = 'http://www.example.com/forum/forumdisplay.php?f=148&amp;order=desc&amp;page=3'
  
  html_source = requests.get(url).text
  
  soup = BeautifulSoup(html_source, 'html.parser')
  
  a_tags = soup.find_all('a')
  
  for a in a_tags:
      if 'member.php?' in a['href']:
          print(a.text)
  
  time.sleep(10)
  </code></pre>
  
  <p>These are the error messages:</p>
  
  <pre><code>Traceback (most recent call last): 
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connection.py", line 138, in _new_conn
  (self.host, self.port), self.timeout, **extra_kw)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\util\connection.py", line 98, in create_connection
  raise err
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\util\connection.py", line 88, in create_connection
  sock.connect(sa)
  TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 594, in urlopen
  chunked=chunked)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 361, in _make_request
  conn.request(method, url, **httplib_request_kw)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1106, in request
  self._send_request(method, url, body, headers)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1151, in _send_request
  self.endheaders(body)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1102, in endheaders
  self._send_output(message_body)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 934, in _send_output
  self.send(msg)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 877, in send
  self.connect()
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connection.py", line 163, in connect
  conn = self._new_conn()
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connection.py", line 147, in _new_conn
  self, "Failed to establish a new connection: %s" % e)
  requests.packages.urllib3.exceptions.NewConnectionError: &lt;requests.packages.urllib3.connection.HTTPConnection object at 0x029131F0&gt;:     Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\adapters.py", line 423, in send
  timeout=timeout
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 643, in urlopen
  _stacktrace=sys.exc_info()[2])
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\util\retry.py", line 363, in increment
  raise MaxRetryError(_pool, url, error or ResponseError(cause))
  requests.packages.urllib3.exceptions.MaxRetryError:
  HTTPConnectionPool(host='www.example.com', port=80): Max retries exceeded with url: /forum/forumdisplay.php?f=148&amp;order=desc&amp;page=3 (Caused by NewConnectionError('&lt;requests.packages.urllib3.connection.HTTPConnection object at 0x029131F0&gt;: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
  File "C:/Users/user/PycharmProjects/untitled/backpackertw_v1.py", line 6, in &lt;module&gt;
  html_source = requests.get(url).text
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\api.py", line 70, in get
  return request('get', url, params=params, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\api.py", line 56, in request
  return session.request(method=method, url=url, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\sessions.py", line 488, in request
  resp = self.send(prep, **send_kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\sessions.py", line 609, in send
  r = adapter.send(request, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\adapters.py", line 487, in send
  raise ConnectionError(e, request=request)
  requests.exceptions.ConnectionError:
  HTTPConnectionPool(host='www.example.com', port=80): Max retries exceeded with url: /forum/forumdisplay.php?f=148&amp;order=desc&amp;page=3 (Caused by NewConnectionError('&lt;requests.packages.urllib3.connection.HTTPConnection object at 0x029131F0&gt;: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))
  </code></pre>