title: Python interface to PayPal - urllib.urlencode non-ASCII characters failing
body: <p>I am trying to implement PayPal IPN functionality. The basic protocol is as such:</p>
  
  <ol>
  <li>The client is redirected from my site to PayPal's site to complete payment. He logs into his account, authorizes payment.</li>
  <li>PayPal calls a page on my server passing in details as POST. Details include a person's name, address, and payment info etc.</li>
  <li>I need to call a URL on PayPal's site internally from my processing page passing back all the params that were passed in abovem and an additional one called 'cmd' with a value of '_notify-validate'. </li>
  </ol>
  
  <p>When I try to urllib.urlencode the params which PayPal has sent to me, I get a:</p>
  
  <pre><code>While calling send_response_to_paypal. Traceback (most recent call last):
    File "&lt;snip&gt;/account/paypal/views.py", line 108, in process_paypal_ipn
      verify_result = send_response_to_paypal(params)
    File "&lt;snip&gt;/account/paypal/views.py", line 41, in send_response_to_paypal
      params = urllib.urlencode(params)
    File "/usr/local/lib/python2.6/urllib.py", line 1261, in urlencode
      v = quote_plus(str(v))
  UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 9: ordinal not in range(128)
  </code></pre>
  
  <p>I understand that urlencode does ASCII encoding, and in certain cases, a user's contact info can contain non-ASCII characters. This is understandable. My question is, how do I encode non-ASCII characters for POSTing to a URL using urllib2.urlopen(req) (or other method)</p>
  
  <p><strong>Details:</strong></p>
  
  <p>I read the params in PayPal's original request as follows (the GET is for testing):</p>
  
  <pre><code>def read_ipn_params(request):
      if request.POST:  
          params= request.POST.copy()  
          if "ipn_auth" in request.GET:
              params["ipn_auth"]=request.GET["ipn_auth"]
          return params
      else:  
          return request.GET.copy()  
  </code></pre>
  
  <p>The code I use for sending back the request to PayPal from the processing page is:</p>
  
  <pre><code>def send_response_to_paypal(params):
      params['cmd']='_notify-validate'  
      params = urllib.urlencode(params)  
      req = urllib2.Request(PAYPAL_API_WEBSITE, params)  
      req.add_header("Content-type", "application/x-www-form-urlencoded") 
      response = urllib2.urlopen(req)  
      status = response.read()  
      if not status == "VERIFIED":  
          logging.warn("PayPal cannot verify IPN responses: " + status)
          return False
  
      return True
  </code></pre>
  
  <p>Obviously, the problem only arises if someone's name or address or other field used for the PayPal payment does not fall into the ASCII range.</p>