title: Python BaseHTTPServer, how do I catch/trap "broken pipe" errors?
body: <p>I build a short url translator engine in Python, and I'm seeing a TON of "broken pipe" errors, and I'm curious how to trap it best when using the BaseHTTPServer classes. This isn't the entire code, but gives you an idea of what I'm doing so far:</p>
  
  <pre><code>    from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
      import memcache
  
      class clientThread(BaseHTTPRequestHandler):
  
              def do_GET(self):
                      content = None
                      http_code,response_txt,long_url = \
                              self.ag_trans_url(self.path,content,'GET')
                      self.http_output( http_code, response_txt, long_url )
                      return
  
              def http_output(self,http_code,response_txt,long_url):
                      self.send_response(http_code)
                      self.send_header('Content-type','text/plain')
                      if long_url:
                              self.send_header('Location', long_url)
                      self.end_headers()
                      if response_txt:
                              self.wfile.write(response_txt)
                      return
  
          def ag_trans_url(self, orig_short_url, post_action, getpost):
                  short_url = 'http://foo.co' + orig_short_url
  
                  # fetch it from memcache
                  long_url = mc.get(short_url)
  
                  # other magic happens to look it up from db if there was nothing
                  # in memcache, etc
                  return (302, None, log_url)
  
  def populate_memcache()
          # connect to db, do lots of mc.set() calls
  
  def main():
          populate_memcache()
          try:
                  port = 8001
                  if len(sys.argv) &gt; 1:
                          port = int(sys.argv[1])
                  server = HTTPServer(('',port), clientThread)
                  #server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
                  print '[',str(datetime.datetime.now()),'] short url processing has begun'
  
                  server.serve_forever()
          except KeyboardInterrupt,SystemExit:
                  print '^C received, shutting down server'
                  server.socket.close()
  </code></pre>
  
  <p>The code itself works great, but started throwing errors almost immediately when in production:</p>
  
  <pre><code>Traceback (most recent call last):
    File "/usr/lib/python2.5/SocketServer.py", line 222, in handle_request
      self.process_request(request, client_address)
    File "/usr/lib/python2.5/SocketServer.py", line 241, in process_request
      self.finish_request(request, client_address)
    File "/usr/lib/python2.5/SocketServer.py", line 254, in finish_request
      self.RequestHandlerClass(request, client_address, self)
    File "/usr/lib/python2.5/SocketServer.py", line 522, in __init__
      self.handle()
    File "/usr/lib/python2.5/BaseHTTPServer.py", line 316, in handle
      self.handle_one_request()
    File "/usr/lib/python2.5/BaseHTTPServer.py", line 310, in handle_one_request
      method()
    File "/opt/short_url_redirector/shorturl.py", line 38, in do_GET
      self.http_output( http_code, response_txt, long_url )
    File "/opt/short_url_redirector/shorturl.py", line 52, in http_output
      self.send_response(http_code)
    File "/usr/lib/python2.5/BaseHTTPServer.py", line 370, in send_response
      self.send_header('Server', self.version_string())
    File "/usr/lib/python2.5/BaseHTTPServer.py", line 376, in send_header
      self.wfile.write("%s: %s\r\n" % (keyword, value))
    File "/usr/lib/python2.5/socket.py", line 274, in write
      self.flush()
    File "/usr/lib/python2.5/socket.py", line 261, in flush
      self._sock.sendall(buffer)
  error: (32, 'Broken pipe')
  </code></pre>
  
  <p>The bulk of these errors seem to stem from having a problem calling the send_header() method where all I'm writing out is this:</p>
  
  <pre><code>self.send_header('Location', long_url)
  </code></pre>
  
  <p>So I'm curious where in my code to try to trap for this IO exception... do I write try/except calls around each of the self.send_header/self.end_headers/self.wfile.write calls? The other error I see from time to time is this one, but not sure which exception to watch to even catch this:</p>
  
  <pre><code>Traceback (most recent call last):
    File "/usr/lib/python2.5/SocketServer.py", line 222, in handle_request
      self.process_request(request, client_address)
    File "/usr/lib/python2.5/SocketServer.py", line 241, in process_request
      self.finish_request(request, client_address)
    File "/usr/lib/python2.5/SocketServer.py", line 254, in finish_request
      self.RequestHandlerClass(request, client_address, self)
    File "/usr/lib/python2.5/SocketServer.py", line 522, in __init__
      self.handle()
    File "/usr/lib/python2.5/BaseHTTPServer.py", line 316, in handle
      self.handle_one_request()
    File "/usr/lib/python2.5/BaseHTTPServer.py", line 299, in handle_one_request
      self.raw_requestline = self.rfile.readline()
    File "/usr/lib/python2.5/socket.py", line 381, in readline
      data = self._sock.recv(self._rbufsize)
  error: (104, 'Connection reset by peer')
  </code></pre>